2024-09-10T21:08:47.171 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-10T21:08:47.197 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-10T21:08:47.289 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899674 branch: wip-hemanth-testing-2024-09-10-0723 description: fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse msgr-failures/none objectstore-ec/bluestore-comp-ec-root overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down pg_health prefetch_dirfrags/no prefetch_dirfrags/yes prefetch_entire_dirfrags/no prefetch_entire_dirfrags/yes races session_timeout thrashosds-health} ranks/1 tasks/{1-thrash/mon 2-workunit/suites/ffsb}} email: hyelloji@redhat.com first_in_suite: false job_id: '7899674' kernel: flavor: default kdb: 1 sha1: 2ca44a2f35ca8f36983aaaa46d49312239ef519a ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: ubuntu os_version: '22.04' overrides: admin_socket: branch: wip-hemanth-testing-2024-09-10-0723 ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 1 session_timeout: 300 cluster-conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 2 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_dir_prefetch: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mon: mon op complaint time: 120 mon warn on pool no app: false osd: osd op complaint time: 180 conf: client: client_shutdown_timeout: 120 mds: mds oft prefetch dirfrags: true mds_sleep_rank_change: 5000000.0 mgr: client mount timeout: 30 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 filestore flush min: 0 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd blocked scrub grace period: 3600 osd failsafe full ratio: 0.95 osd max markdown count: 1000 osd objectstore: bluestore flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - fs.*is degraded - filesystem is degraded - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - filesystem is offline - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - insufficient standby MDS daemons available - MDS_UP_LESS_THAN_MAX - online, but wants - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ - \(POOL_ - \(CACHE_POOL_ - \(SMALLER_PGP_NUM\) - \(OBJECT_ - SLOW_OPS - \(REQUEST_SLOW\) - \(TOO_FEW_PGS\) - slow request - timeout on replica - late reservation from - MON_DOWN - OSDMAP_FLAGS - OSD_DOWN - PG_DEGRADED - PG_AVAILABILITY - POOL_APP_NOT_ENABLED - mons down - mon down - out of quorum - noscrub - nodeep-scrub - overall HEALTH_ - \(MON_DOWN\) - SLOW_OPS - slow request sha1: 8f219c476cb4f26fc5719c661c0323719a45cb30 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} install: ceph: flavor: default sha1: 8f219c476cb4f26fc5719c661c0323719a45cb30 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-hemanth-testing-2024-09-10-0723 sha1: 8f219c476cb4f26fc5719c661c0323719a45cb30 owner: scheduled_hyelloji@teuthology priority: 100 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - mds.e - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - - mon.b - mon.c - mgr.x - mds.b - mds.d - mds.f - osd.4 - osd.5 - osd.6 - osd.7 seed: 6190 sha1: 8f219c476cb4f26fc5719c661c0323719a45cb30 sleep_before_teardown: 0 subset: 479/1024 suite: fs suite_branch: wip-hemanth-testing-2024-09-10-0723 suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_8f219c476cb4f26fc5719c661c0323719a45cb30/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: 8f219c476cb4f26fc5719c661c0323719a45cb30 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi017.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMdVGJLkxjFoSuiC7ENO8miovADl+rrihzPwA71fmjeWR6D5PXrFr25hGSAkVomlBKEW8BMoyc6xiGzj1oYCvSs= smithi031.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLOzCyCbCP19GI36Ugek8kcKz2SzTWmzUxBzO1exGW1rJUNRtcir7zCZsp33ksPDirBshanGA5Npfx+oSf+kFes= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - ceph-fuse: null - mon_thrash: check_mds_failover: true revive_delay: 20 thrash_delay: 10 - workunit: clients: all: - suites/ffsb.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-10_16:59:47 tube: smithi user: hyelloji verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-10T21:08:47.290 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_8f219c476cb4f26fc5719c661c0323719a45cb30/qa; will attempt to use it 2024-09-10T21:08:47.291 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_8f219c476cb4f26fc5719c661c0323719a45cb30/qa/tasks 2024-09-10T21:08:47.291 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-10T21:08:47.293 INFO:teuthology.task.internal:Checking packages... 2024-09-10T21:08:47.313 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash '8f219c476cb4f26fc5719c661c0323719a45cb30' 2024-09-10T21:08:47.313 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-10T21:08:47.313 INFO:teuthology.packaging:ref: None 2024-09-10T21:08:47.313 INFO:teuthology.packaging:tag: None 2024-09-10T21:08:47.314 INFO:teuthology.packaging:branch: wip-hemanth-testing-2024-09-10-0723 2024-09-10T21:08:47.314 INFO:teuthology.packaging:sha1: 8f219c476cb4f26fc5719c661c0323719a45cb30 2024-09-10T21:08:47.314 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=wip-hemanth-testing-2024-09-10-0723 2024-09-10T21:08:47.463 INFO:teuthology.task.internal:Found packages for ceph version 19.3.0-4854-g8f219c47-1jammy 2024-09-10T21:08:47.464 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-10T21:08:47.471 INFO:teuthology.task.internal:no buildpackages task found 2024-09-10T21:08:47.471 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-10T21:08:47.492 INFO:teuthology.task.internal:Saving configuration 2024-09-10T21:08:47.508 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-10T21:08:47.516 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-10T21:08:47.545 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi017.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899674', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-10 21:03:27.824308', 'locked_by': 'scheduled_hyelloji@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIAtNHoxHUfqxmWeXu9jVIZS44PllFG5yMtkHMemx00hB'} 2024-09-10T21:08:47.570 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi031.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899674', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-10 21:03:27.821438', 'locked_by': 'scheduled_hyelloji@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIPOZyG9wShcHinADLBmkg1BrFoIrhXq5HBzxOrDrEDPI'} 2024-09-10T21:08:47.570 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-10T21:08:47.576 INFO:teuthology.task.internal:roles: ubuntu@smithi017.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'mds.e', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'] 2024-09-10T21:08:47.577 INFO:teuthology.task.internal:roles: ubuntu@smithi031.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'mds.f', 'osd.4', 'osd.5', 'osd.6', 'osd.7'] 2024-09-10T21:08:47.577 INFO:teuthology.run_tasks:Running task console_log... 2024-09-10T21:08:47.649 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f7f277a3be0>, signals=[15]) 2024-09-10T21:08:47.649 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-10T21:08:47.655 INFO:teuthology.task.internal:Opening connections... 2024-09-10T21:08:47.656 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi017.front.sepia.ceph.com 2024-09-10T21:08:47.657 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi017.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T21:08:47.749 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi031.front.sepia.ceph.com 2024-09-10T21:08:47.750 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi031.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T21:08:47.839 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-10T21:08:47.845 DEBUG:teuthology.orchestra.run.smithi017:> uname -m 2024-09-10T21:08:47.850 INFO:teuthology.orchestra.run.smithi017.stdout:x86_64 2024-09-10T21:08:47.851 DEBUG:teuthology.orchestra.run.smithi017:> cat /etc/os-release 2024-09-10T21:08:47.897 INFO:teuthology.orchestra.run.smithi017.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-09-10T21:08:47.897 INFO:teuthology.orchestra.run.smithi017.stdout:NAME="Ubuntu" 2024-09-10T21:08:47.898 INFO:teuthology.orchestra.run.smithi017.stdout:VERSION_ID="22.04" 2024-09-10T21:08:47.898 INFO:teuthology.orchestra.run.smithi017.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-09-10T21:08:47.898 INFO:teuthology.orchestra.run.smithi017.stdout:VERSION_CODENAME=jammy 2024-09-10T21:08:47.898 INFO:teuthology.orchestra.run.smithi017.stdout:ID=ubuntu 2024-09-10T21:08:47.898 INFO:teuthology.orchestra.run.smithi017.stdout:ID_LIKE=debian 2024-09-10T21:08:47.898 INFO:teuthology.orchestra.run.smithi017.stdout:HOME_URL="https://www.ubuntu.com/" 2024-09-10T21:08:47.898 INFO:teuthology.orchestra.run.smithi017.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-09-10T21:08:47.898 INFO:teuthology.orchestra.run.smithi017.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-09-10T21:08:47.898 INFO:teuthology.orchestra.run.smithi017.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-09-10T21:08:47.898 INFO:teuthology.orchestra.run.smithi017.stdout:UBUNTU_CODENAME=jammy 2024-09-10T21:08:47.899 INFO:teuthology.lock.ops:Updating smithi017.front.sepia.ceph.com on lock server 2024-09-10T21:08:47.926 DEBUG:teuthology.orchestra.run.smithi031:> uname -m 2024-09-10T21:08:47.931 INFO:teuthology.orchestra.run.smithi031.stdout:x86_64 2024-09-10T21:08:47.932 DEBUG:teuthology.orchestra.run.smithi031:> cat /etc/os-release 2024-09-10T21:08:47.978 INFO:teuthology.orchestra.run.smithi031.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-09-10T21:08:47.979 INFO:teuthology.orchestra.run.smithi031.stdout:NAME="Ubuntu" 2024-09-10T21:08:47.979 INFO:teuthology.orchestra.run.smithi031.stdout:VERSION_ID="22.04" 2024-09-10T21:08:47.979 INFO:teuthology.orchestra.run.smithi031.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-09-10T21:08:47.979 INFO:teuthology.orchestra.run.smithi031.stdout:VERSION_CODENAME=jammy 2024-09-10T21:08:47.979 INFO:teuthology.orchestra.run.smithi031.stdout:ID=ubuntu 2024-09-10T21:08:47.979 INFO:teuthology.orchestra.run.smithi031.stdout:ID_LIKE=debian 2024-09-10T21:08:47.979 INFO:teuthology.orchestra.run.smithi031.stdout:HOME_URL="https://www.ubuntu.com/" 2024-09-10T21:08:47.979 INFO:teuthology.orchestra.run.smithi031.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-09-10T21:08:47.979 INFO:teuthology.orchestra.run.smithi031.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-09-10T21:08:47.979 INFO:teuthology.orchestra.run.smithi031.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-09-10T21:08:47.979 INFO:teuthology.orchestra.run.smithi031.stdout:UBUNTU_CODENAME=jammy 2024-09-10T21:08:47.980 INFO:teuthology.lock.ops:Updating smithi031.front.sepia.ceph.com on lock server 2024-09-10T21:08:48.010 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-10T21:08:48.019 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-10T21:08:48.039 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-10T21:08:48.039 DEBUG:teuthology.orchestra.run.smithi017:> test '!' -e /home/ubuntu/cephtest 2024-09-10T21:08:48.041 DEBUG:teuthology.orchestra.run.smithi031:> test '!' -e /home/ubuntu/cephtest 2024-09-10T21:08:48.046 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-10T21:08:48.114 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-10T21:08:48.114 DEBUG:teuthology.orchestra.run.smithi017:> test -z $(ls -A /var/lib/ceph) 2024-09-10T21:08:48.117 DEBUG:teuthology.orchestra.run.smithi031:> test -z $(ls -A /var/lib/ceph) 2024-09-10T21:08:48.132 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-10T21:08:48.242 INFO:teuthology.run_tasks:Running task kernel... 2024-09-10T21:08:48.256 INFO:teuthology.task.kernel:normalize config orig: {'flavor': 'default', 'kdb': 1, 'sha1': '2ca44a2f35ca8f36983aaaa46d49312239ef519a'} 2024-09-10T21:08:48.256 INFO:teuthology.task.kernel:config {'mon.a': {'flavor': 'default', 'kdb': 1, 'sha1': '2ca44a2f35ca8f36983aaaa46d49312239ef519a'}, 'mon.b': {'flavor': 'default', 'kdb': 1, 'sha1': '2ca44a2f35ca8f36983aaaa46d49312239ef519a'}}, timeout 300 2024-09-10T21:08:48.256 DEBUG:teuthology.orchestra.run.smithi017:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-10T21:08:48.257 DEBUG:teuthology.orchestra.run.smithi031:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-10T21:08:48.261 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T21:08:48.262 DEBUG:teuthology.task.kernel:sha1 for mon.a is 2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-10T21:08:48.262 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "2ca44a2f35ca8f36983aaaa46d49312239ef519a"... 2024-09-10T21:08:48.262 DEBUG:teuthology.orchestra.run.smithi017:> uname -r 2024-09-10T21:08:48.262 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T21:08:48.263 DEBUG:teuthology.task.kernel:sha1 for mon.b is 2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-10T21:08:48.263 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "2ca44a2f35ca8f36983aaaa46d49312239ef519a"... 2024-09-10T21:08:48.263 DEBUG:teuthology.orchestra.run.smithi031:> uname -r 2024-09-10T21:08:48.309 INFO:teuthology.orchestra.run.smithi017.stdout:5.15.0-56-generic 2024-09-10T21:08:48.310 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-56-generic vs 2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-10T21:08:48.310 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-09-10T21:08:48.310 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-10T21:08:48.322 INFO:teuthology.orchestra.run.smithi031.stdout:5.15.0-56-generic 2024-09-10T21:08:48.322 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-56-generic vs 2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-10T21:08:48.322 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-09-10T21:08:48.322 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-10T21:08:48.469 INFO:teuthology.task.kernel:Installing linux-firmware on mon.b... 2024-09-10T21:08:48.470 DEBUG:teuthology.orchestra.run.smithi031:> cd /lib/firmware/updates && test -d .git && test '!' -s .git/HEAD && sudo rm -rf .git ; sudo install -d -m0755 /lib/firmware/updates && cd /lib/firmware/updates && sudo git init 2024-09-10T21:08:48.471 INFO:teuthology.task.kernel:Installing linux-firmware on mon.a... 2024-09-10T21:08:48.471 DEBUG:teuthology.orchestra.run.smithi017:> cd /lib/firmware/updates && test -d .git && test '!' -s .git/HEAD && sudo rm -rf .git ; sudo install -d -m0755 /lib/firmware/updates && cd /lib/firmware/updates && sudo git init 2024-09-10T21:08:48.483 INFO:teuthology.orchestra.run.smithi031.stderr:bash: line 1: cd: /lib/firmware/updates: No such file or directory 2024-09-10T21:08:48.485 INFO:teuthology.orchestra.run.smithi017.stderr:bash: line 1: cd: /lib/firmware/updates: No such file or directory 2024-09-10T21:08:48.603 INFO:teuthology.orchestra.run.smithi017.stderr:hint: Using 'master' as the name for the initial branch. This default branch name 2024-09-10T21:08:48.603 INFO:teuthology.orchestra.run.smithi017.stderr:hint: is subject to change. To configure the initial branch name to use in all 2024-09-10T21:08:48.603 INFO:teuthology.orchestra.run.smithi017.stderr:hint: of your new repositories, which will suppress this warning, call: 2024-09-10T21:08:48.603 INFO:teuthology.orchestra.run.smithi017.stderr:hint: 2024-09-10T21:08:48.604 INFO:teuthology.orchestra.run.smithi017.stderr:hint: git config --global init.defaultBranch 2024-09-10T21:08:48.604 INFO:teuthology.orchestra.run.smithi017.stderr:hint: 2024-09-10T21:08:48.604 INFO:teuthology.orchestra.run.smithi017.stderr:hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and 2024-09-10T21:08:48.604 INFO:teuthology.orchestra.run.smithi017.stderr:hint: 'development'. The just-created branch can be renamed via this command: 2024-09-10T21:08:48.604 INFO:teuthology.orchestra.run.smithi017.stderr:hint: 2024-09-10T21:08:48.604 INFO:teuthology.orchestra.run.smithi017.stderr:hint: git branch -m 2024-09-10T21:08:48.605 INFO:teuthology.orchestra.run.smithi017.stdout:Initialized empty Git repository in /usr/lib/firmware/updates/.git/ 2024-09-10T21:08:48.606 DEBUG:teuthology.orchestra.run.smithi017:> sudo git --git-dir=/lib/firmware/updates/.git config --get remote.origin.url >/dev/null || sudo git --git-dir=/lib/firmware/updates/.git remote add origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git 2024-09-10T21:08:48.639 INFO:teuthology.orchestra.run.smithi031.stderr:hint: Using 'master' as the name for the initial branch. This default branch name 2024-09-10T21:08:48.639 INFO:teuthology.orchestra.run.smithi031.stderr:hint: is subject to change. To configure the initial branch name to use in all 2024-09-10T21:08:48.639 INFO:teuthology.orchestra.run.smithi031.stderr:hint: of your new repositories, which will suppress this warning, call: 2024-09-10T21:08:48.639 INFO:teuthology.orchestra.run.smithi031.stderr:hint: 2024-09-10T21:08:48.639 INFO:teuthology.orchestra.run.smithi031.stderr:hint: git config --global init.defaultBranch 2024-09-10T21:08:48.640 INFO:teuthology.orchestra.run.smithi031.stderr:hint: 2024-09-10T21:08:48.640 INFO:teuthology.orchestra.run.smithi031.stderr:hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and 2024-09-10T21:08:48.640 INFO:teuthology.orchestra.run.smithi031.stderr:hint: 'development'. The just-created branch can be renamed via this command: 2024-09-10T21:08:48.640 INFO:teuthology.orchestra.run.smithi031.stderr:hint: 2024-09-10T21:08:48.640 INFO:teuthology.orchestra.run.smithi031.stderr:hint: git branch -m 2024-09-10T21:08:48.640 INFO:teuthology.orchestra.run.smithi031.stdout:Initialized empty Git repository in /usr/lib/firmware/updates/.git/ 2024-09-10T21:08:48.642 DEBUG:teuthology.orchestra.run.smithi031:> sudo git --git-dir=/lib/firmware/updates/.git config --get remote.origin.url >/dev/null || sudo git --git-dir=/lib/firmware/updates/.git remote add origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git 2024-09-10T21:08:48.682 DEBUG:teuthology.orchestra.run.smithi017:> sudo git --git-dir=/lib/firmware/updates/.git remote set-url origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git >/dev/null 2024-09-10T21:08:48.720 DEBUG:teuthology.orchestra.run.smithi031:> sudo git --git-dir=/lib/firmware/updates/.git remote set-url origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git >/dev/null 2024-09-10T21:08:48.739 DEBUG:teuthology.orchestra.run.smithi017:> cd /lib/firmware/updates && sudo git fetch origin && sudo git reset --hard origin/main 2024-09-10T21:08:48.776 DEBUG:teuthology.orchestra.run.smithi031:> cd /lib/firmware/updates && sudo git fetch origin && sudo git reset --hard origin/main 2024-09-10T21:09:32.800 INFO:teuthology.orchestra.run.smithi031.stderr:From git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware 2024-09-10T21:09:32.800 INFO:teuthology.orchestra.run.smithi031.stderr: * [new branch] main -> origin/main 2024-09-10T21:09:32.839 INFO:teuthology.orchestra.run.smithi031.stderr: * [new tag] 20190312 -> 20190312 2024-09-10T21:09:32.839 INFO:teuthology.orchestra.run.smithi031.stderr: * [new tag] 20190416 -> 20190416 2024-09-10T21:09:32.839 INFO:teuthology.orchestra.run.smithi031.stderr: * [new tag] 20190514 -> 20190514 2024-09-10T21:09:32.840 INFO:teuthology.orchestra.run.smithi031.stderr: * [new tag] 20190618 -> 20190618 2024-09-10T21:09:32.840 INFO:teuthology.orchestra.run.smithi031.stderr: * [new tag] 20190717 -> 20190717 2024-09-10T21:09:32.840 INFO:teuthology.orchestra.run.smithi031.stderr: * [new tag] 20190815 -> 20190815 2024-09-10T21:09:32.840 INFO:teuthology.orchestra.run.smithi031.stderr: * [new tag] 20190923 -> 20190923 2024-09-10T21:09:32.840 INFO:teuthology.orchestra.run.smithi031.stderr: * [new tag] 20191022 -> 20191022 2024-09-10T21:09:32.840 INFO:teuthology.orchestra.run.smithi031.stderr: * [new tag] 20191215 -> 20191215 2024-09-10T21:09:32.842 INFO:teuthology.orchestra.run.smithi031.stderr: * [new tag] 20200122 -> 20200122 2024-09-10T21:09:32.842 INFO:teuthology.orchestra.run.smithi031.stderr: * [new tag] 20200316 -> 20200316 2024-09-10T21:09:32.843 INFO:teuthology.orchestra.run.smithi031.stderr: * [new tag] 20200421 -> 20200421 2024-09-10T21:09:32.843 INFO:teuthology.orchestra.run.smithi031.stderr: * [new tag] 20200519 -> 20200519 2024-09-10T21:09:32.843 INFO:teuthology.orchestra.run.smithi031.stderr: * [new tag] 20200619 -> 20200619 2024-09-10T21:09:32.843 INFO:teuthology.orchestra.run.smithi031.stderr: * [new tag] 20200721 -> 20200721 2024-09-10T21:09:32.843 INFO:teuthology.orchestra.run.smithi031.stderr: * [new tag] 20200817 -> 20200817 2024-09-10T21:09:32.843 INFO:teuthology.orchestra.run.smithi031.stderr: * [new tag] 20200918 -> 20200918 2024-09-10T21:09:32.843 INFO:teuthology.orchestra.run.smithi031.stderr: * [new tag] 20201022 -> 20201022 2024-09-10T21:09:32.843 INFO:teuthology.orchestra.run.smithi031.stderr: * [new tag] 20201118 -> 20201118 2024-09-10T21:09:32.843 INFO:teuthology.orchestra.run.smithi031.stderr: * [new tag] 20201218 -> 20201218 2024-09-10T21:09:32.843 INFO:teuthology.orchestra.run.smithi031.stderr: * [new tag] 20210208 -> 20210208 2024-09-10T21:09:32.843 INFO:teuthology.orchestra.run.smithi031.stderr: * [new tag] 20210315 -> 20210315 2024-09-10T21:09:32.843 INFO:teuthology.orchestra.run.smithi031.stderr: * [new tag] 20210511 -> 20210511 2024-09-10T21:09:32.843 INFO:teuthology.orchestra.run.smithi031.stderr: * [new tag] 20210716 -> 20210716 2024-09-10T21:09:32.843 INFO:teuthology.orchestra.run.smithi031.stderr: * [new tag] 20210818 -> 20210818 2024-09-10T21:09:32.844 INFO:teuthology.orchestra.run.smithi031.stderr: * [new tag] 20210919 -> 20210919 2024-09-10T21:09:32.844 INFO:teuthology.orchestra.run.smithi031.stderr: * [new tag] 20211027 -> 20211027 2024-09-10T21:09:32.844 INFO:teuthology.orchestra.run.smithi031.stderr: * [new tag] 20211216 -> 20211216 2024-09-10T21:09:32.844 INFO:teuthology.orchestra.run.smithi031.stderr: * [new tag] 20220209 -> 20220209 2024-09-10T21:09:32.847 INFO:teuthology.orchestra.run.smithi031.stderr: * [new tag] 20220310 -> 20220310 2024-09-10T21:09:32.847 INFO:teuthology.orchestra.run.smithi031.stderr: * [new tag] 20220411 -> 20220411 2024-09-10T21:09:32.847 INFO:teuthology.orchestra.run.smithi031.stderr: * [new tag] 20220509 -> 20220509 2024-09-10T21:09:32.847 INFO:teuthology.orchestra.run.smithi031.stderr: * [new tag] 20220610 -> 20220610 2024-09-10T21:09:32.847 INFO:teuthology.orchestra.run.smithi031.stderr: * [new tag] 20220708 -> 20220708 2024-09-10T21:09:32.847 INFO:teuthology.orchestra.run.smithi031.stderr: * [new tag] 20220815 -> 20220815 2024-09-10T21:09:32.847 INFO:teuthology.orchestra.run.smithi031.stderr: * [new tag] 20220913 -> 20220913 2024-09-10T21:09:32.847 INFO:teuthology.orchestra.run.smithi031.stderr: * [new tag] 20221012 -> 20221012 2024-09-10T21:09:32.847 INFO:teuthology.orchestra.run.smithi031.stderr: * [new tag] 20221109 -> 20221109 2024-09-10T21:09:32.848 INFO:teuthology.orchestra.run.smithi031.stderr: * [new tag] 20221214 -> 20221214 2024-09-10T21:09:32.848 INFO:teuthology.orchestra.run.smithi031.stderr: * [new tag] 20230117 -> 20230117 2024-09-10T21:09:32.848 INFO:teuthology.orchestra.run.smithi031.stderr: * [new tag] 20230210 -> 20230210 2024-09-10T21:09:32.848 INFO:teuthology.orchestra.run.smithi031.stderr: * [new tag] 20230310 -> 20230310 2024-09-10T21:09:32.848 INFO:teuthology.orchestra.run.smithi031.stderr: * [new tag] 20230404 -> 20230404 2024-09-10T21:09:32.848 INFO:teuthology.orchestra.run.smithi031.stderr: * [new tag] 20230515 -> 20230515 2024-09-10T21:09:32.848 INFO:teuthology.orchestra.run.smithi031.stderr: * [new tag] 20230625 -> 20230625 2024-09-10T21:09:32.848 INFO:teuthology.orchestra.run.smithi031.stderr: * [new tag] 20230804 -> 20230804 2024-09-10T21:09:32.848 INFO:teuthology.orchestra.run.smithi031.stderr: * [new tag] 20230919 -> 20230919 2024-09-10T21:09:32.848 INFO:teuthology.orchestra.run.smithi031.stderr: * [new tag] 20231030 -> 20231030 2024-09-10T21:09:32.848 INFO:teuthology.orchestra.run.smithi031.stderr: * [new tag] 20231111 -> 20231111 2024-09-10T21:09:32.848 INFO:teuthology.orchestra.run.smithi031.stderr: * [new tag] 20231211 -> 20231211 2024-09-10T21:09:32.848 INFO:teuthology.orchestra.run.smithi031.stderr: * [new tag] 20240115 -> 20240115 2024-09-10T21:09:32.849 INFO:teuthology.orchestra.run.smithi031.stderr: * [new tag] 20240220 -> 20240220 2024-09-10T21:09:32.849 INFO:teuthology.orchestra.run.smithi031.stderr: * [new tag] 20240312 -> 20240312 2024-09-10T21:09:32.849 INFO:teuthology.orchestra.run.smithi031.stderr: * [new tag] 20240410 -> 20240410 2024-09-10T21:09:32.849 INFO:teuthology.orchestra.run.smithi031.stderr: * [new tag] 20240513 -> 20240513 2024-09-10T21:09:32.849 INFO:teuthology.orchestra.run.smithi031.stderr: * [new tag] 20240610 -> 20240610 2024-09-10T21:09:32.849 INFO:teuthology.orchestra.run.smithi031.stderr: * [new tag] 20240709 -> 20240709 2024-09-10T21:09:32.849 INFO:teuthology.orchestra.run.smithi031.stderr: * [new tag] 20240811 -> 20240811 2024-09-10T21:09:32.849 INFO:teuthology.orchestra.run.smithi031.stderr: * [new tag] 20240909 -> 20240909 2024-09-10T21:09:36.455 INFO:teuthology.orchestra.run.smithi017.stderr:From git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware 2024-09-10T21:09:36.455 INFO:teuthology.orchestra.run.smithi017.stderr: * [new branch] main -> origin/main 2024-09-10T21:09:36.485 INFO:teuthology.orchestra.run.smithi017.stderr: * [new tag] 20190312 -> 20190312 2024-09-10T21:09:36.485 INFO:teuthology.orchestra.run.smithi017.stderr: * [new tag] 20190416 -> 20190416 2024-09-10T21:09:36.485 INFO:teuthology.orchestra.run.smithi017.stderr: * [new tag] 20190514 -> 20190514 2024-09-10T21:09:36.485 INFO:teuthology.orchestra.run.smithi017.stderr: * [new tag] 20190618 -> 20190618 2024-09-10T21:09:36.485 INFO:teuthology.orchestra.run.smithi017.stderr: * [new tag] 20190717 -> 20190717 2024-09-10T21:09:36.485 INFO:teuthology.orchestra.run.smithi017.stderr: * [new tag] 20190815 -> 20190815 2024-09-10T21:09:36.485 INFO:teuthology.orchestra.run.smithi017.stderr: * [new tag] 20190923 -> 20190923 2024-09-10T21:09:36.485 INFO:teuthology.orchestra.run.smithi017.stderr: * [new tag] 20191022 -> 20191022 2024-09-10T21:09:36.485 INFO:teuthology.orchestra.run.smithi017.stderr: * [new tag] 20191215 -> 20191215 2024-09-10T21:09:36.493 INFO:teuthology.orchestra.run.smithi017.stderr: * [new tag] 20200122 -> 20200122 2024-09-10T21:09:36.493 INFO:teuthology.orchestra.run.smithi017.stderr: * [new tag] 20200316 -> 20200316 2024-09-10T21:09:36.493 INFO:teuthology.orchestra.run.smithi017.stderr: * [new tag] 20200421 -> 20200421 2024-09-10T21:09:36.493 INFO:teuthology.orchestra.run.smithi017.stderr: * [new tag] 20200519 -> 20200519 2024-09-10T21:09:36.493 INFO:teuthology.orchestra.run.smithi017.stderr: * [new tag] 20200619 -> 20200619 2024-09-10T21:09:36.493 INFO:teuthology.orchestra.run.smithi017.stderr: * [new tag] 20200721 -> 20200721 2024-09-10T21:09:36.493 INFO:teuthology.orchestra.run.smithi017.stderr: * [new tag] 20200817 -> 20200817 2024-09-10T21:09:36.494 INFO:teuthology.orchestra.run.smithi017.stderr: * [new tag] 20200918 -> 20200918 2024-09-10T21:09:36.494 INFO:teuthology.orchestra.run.smithi017.stderr: * [new tag] 20201022 -> 20201022 2024-09-10T21:09:36.494 INFO:teuthology.orchestra.run.smithi017.stderr: * [new tag] 20201118 -> 20201118 2024-09-10T21:09:36.494 INFO:teuthology.orchestra.run.smithi017.stderr: * [new tag] 20201218 -> 20201218 2024-09-10T21:09:36.494 INFO:teuthology.orchestra.run.smithi017.stderr: * [new tag] 20210208 -> 20210208 2024-09-10T21:09:36.494 INFO:teuthology.orchestra.run.smithi017.stderr: * [new tag] 20210315 -> 20210315 2024-09-10T21:09:36.494 INFO:teuthology.orchestra.run.smithi017.stderr: * [new tag] 20210511 -> 20210511 2024-09-10T21:09:36.494 INFO:teuthology.orchestra.run.smithi017.stderr: * [new tag] 20210716 -> 20210716 2024-09-10T21:09:36.494 INFO:teuthology.orchestra.run.smithi017.stderr: * [new tag] 20210818 -> 20210818 2024-09-10T21:09:36.494 INFO:teuthology.orchestra.run.smithi017.stderr: * [new tag] 20210919 -> 20210919 2024-09-10T21:09:36.494 INFO:teuthology.orchestra.run.smithi017.stderr: * [new tag] 20211027 -> 20211027 2024-09-10T21:09:36.494 INFO:teuthology.orchestra.run.smithi017.stderr: * [new tag] 20211216 -> 20211216 2024-09-10T21:09:36.494 INFO:teuthology.orchestra.run.smithi017.stderr: * [new tag] 20220209 -> 20220209 2024-09-10T21:09:36.494 INFO:teuthology.orchestra.run.smithi017.stderr: * [new tag] 20220310 -> 20220310 2024-09-10T21:09:36.495 INFO:teuthology.orchestra.run.smithi017.stderr: * [new tag] 20220411 -> 20220411 2024-09-10T21:09:36.495 INFO:teuthology.orchestra.run.smithi017.stderr: * [new tag] 20220509 -> 20220509 2024-09-10T21:09:36.495 INFO:teuthology.orchestra.run.smithi017.stderr: * [new tag] 20220610 -> 20220610 2024-09-10T21:09:36.495 INFO:teuthology.orchestra.run.smithi017.stderr: * [new tag] 20220708 -> 20220708 2024-09-10T21:09:36.495 INFO:teuthology.orchestra.run.smithi017.stderr: * [new tag] 20220815 -> 20220815 2024-09-10T21:09:36.495 INFO:teuthology.orchestra.run.smithi017.stderr: * [new tag] 20220913 -> 20220913 2024-09-10T21:09:36.495 INFO:teuthology.orchestra.run.smithi017.stderr: * [new tag] 20221012 -> 20221012 2024-09-10T21:09:36.495 INFO:teuthology.orchestra.run.smithi017.stderr: * [new tag] 20221109 -> 20221109 2024-09-10T21:09:36.495 INFO:teuthology.orchestra.run.smithi017.stderr: * [new tag] 20221214 -> 20221214 2024-09-10T21:09:36.495 INFO:teuthology.orchestra.run.smithi017.stderr: * [new tag] 20230117 -> 20230117 2024-09-10T21:09:36.495 INFO:teuthology.orchestra.run.smithi017.stderr: * [new tag] 20230210 -> 20230210 2024-09-10T21:09:36.495 INFO:teuthology.orchestra.run.smithi017.stderr: * [new tag] 20230310 -> 20230310 2024-09-10T21:09:36.495 INFO:teuthology.orchestra.run.smithi017.stderr: * [new tag] 20230404 -> 20230404 2024-09-10T21:09:36.495 INFO:teuthology.orchestra.run.smithi017.stderr: * [new tag] 20230515 -> 20230515 2024-09-10T21:09:36.496 INFO:teuthology.orchestra.run.smithi017.stderr: * [new tag] 20230625 -> 20230625 2024-09-10T21:09:36.496 INFO:teuthology.orchestra.run.smithi017.stderr: * [new tag] 20230804 -> 20230804 2024-09-10T21:09:36.496 INFO:teuthology.orchestra.run.smithi017.stderr: * [new tag] 20230919 -> 20230919 2024-09-10T21:09:36.496 INFO:teuthology.orchestra.run.smithi017.stderr: * [new tag] 20231030 -> 20231030 2024-09-10T21:09:36.496 INFO:teuthology.orchestra.run.smithi017.stderr: * [new tag] 20231111 -> 20231111 2024-09-10T21:09:36.496 INFO:teuthology.orchestra.run.smithi017.stderr: * [new tag] 20231211 -> 20231211 2024-09-10T21:09:36.496 INFO:teuthology.orchestra.run.smithi017.stderr: * [new tag] 20240115 -> 20240115 2024-09-10T21:09:36.496 INFO:teuthology.orchestra.run.smithi017.stderr: * [new tag] 20240220 -> 20240220 2024-09-10T21:09:36.496 INFO:teuthology.orchestra.run.smithi017.stderr: * [new tag] 20240312 -> 20240312 2024-09-10T21:09:36.496 INFO:teuthology.orchestra.run.smithi017.stderr: * [new tag] 20240410 -> 20240410 2024-09-10T21:09:36.496 INFO:teuthology.orchestra.run.smithi017.stderr: * [new tag] 20240513 -> 20240513 2024-09-10T21:09:36.496 INFO:teuthology.orchestra.run.smithi017.stderr: * [new tag] 20240610 -> 20240610 2024-09-10T21:09:36.496 INFO:teuthology.orchestra.run.smithi017.stderr: * [new tag] 20240709 -> 20240709 2024-09-10T21:09:36.497 INFO:teuthology.orchestra.run.smithi017.stderr: * [new tag] 20240811 -> 20240811 2024-09-10T21:09:36.497 INFO:teuthology.orchestra.run.smithi017.stderr: * [new tag] 20240909 -> 20240909 2024-09-10T21:09:37.598 INFO:teuthology.orchestra.run.smithi031.stderr:Updating files: 29% (1021/3501) Updating files: 30% (1051/3501) Updating files: 31% (1086/3501) Updating files: 32% (1121/3501) Updating files: 33% (1156/3501) Updating files: 34% (1191/3501) Updating files: 35% (1226/3501) Updating files: 36% (1261/3501) Updating files: 37% (1296/3501) Updating files: 38% (1331/3501) Updating files: 39% (1366/3501) Updating files: 40% (1401/3501) Updating files: 41% (1436/3501) Updating files: 42% (1471/3501) Updating files: 43% (1506/3501) Updating files: 44% (1541/3501) Updating files: 45% (1576/3501) Updating files: 46% (1611/3501) Updating files: 47% (1646/3501) Updating files: 48% (1681/3501) Updating files: 49% (1716/3501) Updating files: 50% (1751/3501) Updating files: 51% (1786/3501) Updating files: 52% (1821/3501) Updating files: 53% (1856/3501) Updating files: 54% (1891/3501) Updating files: 55% (1926/3501) Updating files: 56% (1961/3501) Updating files: 57% (1996/3501) Updating files: 58% (2031/3501) Updating files: 58% (2048/3501) Updating files: 59% (2066/3501) Updating files: 60% (2101/3501) Updating files: 61% (2136/3501) Updating files: 62% (2171/3501) Updating files: 63% (2206/3501) Updating files: 64% (2241/3501) Updating files: 65% (2276/3501) Updating files: 66% (2311/3501) Updating files: 67% (2346/3501) Updating files: 68% (2381/3501) Updating files: 69% (2416/3501) Updating files: 70% (2451/3501) Updating files: 71% (2486/3501) Updating files: 72% (2521/3501) Updating files: 73% (2556/3501) Updating files: 74% (2591/3501) Updating files: 75% (2626/3501) Updating files: 76% (2661/3501) Updating files: 77% (2696/3501) Updating files: 78% (2731/3501) Updating files: 79% (2766/3501) Updating files: 80% (2801/3501) Updating files: 81% (2836/3501) Updating files: 82% (2871/3501) Updating files: 82% (2876/3501) Updating files: 83% (2906/3501) Updating files: 84% (2941/3501) Updating files: 85% (2976/3501) Updating files: 86% (3011/3501) Updating files: 87% (3046/3501) Updating files: 88% (3081/3501) Updating files: 89% (3116/3501) Updating files: 90% (3151/3501) Updating files: 91% (3186/3501) Updating files: 92% (3221/3501) Updating files: 93% (3256/3501) Updating files: 94% (3291/3501) Updating files: 95% (3326/3501) Updating files: 96% (3361/3501) Updating files: 97% (3396/3501) Updating files: 98% (3431/3501) Updating files: 99% (3466/3501) Updating files: 100% (3501/3501) Updating files: 100% (3501/3501), done. 2024-09-10T21:09:37.603 INFO:teuthology.orchestra.run.smithi031.stdout:HEAD is now at ce299c0f Merge branch 'rt1320' into 'main' 2024-09-10T21:09:37.626 INFO:teuthology.task.kernel:Downloading kernel 2ca44a2f35ca8f36983aaaa46d49312239ef519a on mon.b... 2024-09-10T21:09:37.627 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-10T21:09:37.781 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/ubuntu/jammy/flavors/default/pool/main/l/linux-6.11.0-rc5-g2ca44a2f35ca/ 2024-09-10T21:09:37.781 DEBUG:teuthology.orchestra.run.smithi031:> rm -f /tmp/linux-image.deb && echo linux-image-6.11.0-rc5-g2ca44a2f35ca_6.11.0-rc5-g2ca44a2f35ca-1_amd64.deb | wget -nv -O /tmp/linux-image.deb --base=https://4.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/ubuntu/jammy/flavors/default/pool/main/l/linux-6.11.0-rc5-g2ca44a2f35ca/ --input-file=- 2024-09-10T21:09:37.784 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi031.front.sepia.ceph.com to complete... 2024-09-10T21:09:38.827 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10 21:09:39 URL:https://4.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/ubuntu/jammy/flavors/default/pool/main/l/linux-6.11.0-rc5-g2ca44a2f35ca/linux-image-6.11.0-rc5-g2ca44a2f35ca_6.11.0-rc5-g2ca44a2f35ca-1_amd64.deb [45349154/45349154] -> "/tmp/linux-image.deb" [1] 2024-09-10T21:09:38.827 INFO:teuthology.orchestra.run.smithi031.stderr:FINISHED --2024-09-10 21:09:39-- 2024-09-10T21:09:38.827 INFO:teuthology.orchestra.run.smithi031.stderr:Total wall clock time: 1.0s 2024-09-10T21:09:38.828 INFO:teuthology.orchestra.run.smithi031.stderr:Downloaded: 1 files, 43M in 0.8s (51.7 MB/s) 2024-09-10T21:09:38.832 INFO:teuthology.task.kernel:Installing kernel 2ca44a2f35ca8f36983aaaa46d49312239ef519a on mon.b... 2024-09-10T21:09:38.832 DEBUG:teuthology.orchestra.run.smithi031:> sudo dpkg -i /tmp/linux-image.deb 2024-09-10T21:09:39.123 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package linux-image-6.11.0-rc5-g2ca44a2f35ca. 2024-09-10T21:09:41.118 INFO:teuthology.orchestra.run.smithi031.stdout:(Reading database ... 129542 files and directories currently installed.) 2024-09-10T21:09:41.122 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack /tmp/linux-image.deb ... 2024-09-10T21:09:41.237 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking linux-image-6.11.0-rc5-g2ca44a2f35ca (6.11.0-rc5-g2ca44a2f35ca-1) ... 2024-09-10T21:09:41.245 INFO:teuthology.orchestra.run.smithi017.stderr:Updating files: 29% (1020/3501) Updating files: 30% (1051/3501) Updating files: 31% (1086/3501) Updating files: 32% (1121/3501) Updating files: 33% (1156/3501) Updating files: 34% (1191/3501) Updating files: 35% (1226/3501) Updating files: 36% (1261/3501) Updating files: 37% (1296/3501) Updating files: 38% (1331/3501) Updating files: 39% (1366/3501) Updating files: 40% (1401/3501) Updating files: 41% (1436/3501) Updating files: 42% (1471/3501) Updating files: 43% (1506/3501) Updating files: 44% (1541/3501) Updating files: 45% (1576/3501) Updating files: 46% (1611/3501) Updating files: 47% (1646/3501) Updating files: 48% (1681/3501) Updating files: 49% (1716/3501) Updating files: 50% (1751/3501) Updating files: 51% (1786/3501) Updating files: 52% (1821/3501) Updating files: 53% (1856/3501) Updating files: 54% (1891/3501) Updating files: 55% (1926/3501) Updating files: 56% (1961/3501) Updating files: 57% (1996/3501) Updating files: 58% (2031/3501) Updating files: 58% (2048/3501) Updating files: 59% (2066/3501) Updating files: 60% (2101/3501) Updating files: 61% (2136/3501) Updating files: 62% (2171/3501) Updating files: 63% (2206/3501) Updating files: 64% (2241/3501) Updating files: 65% (2276/3501) Updating files: 66% (2311/3501) Updating files: 67% (2346/3501) Updating files: 68% (2381/3501) Updating files: 69% (2416/3501) Updating files: 70% (2451/3501) Updating files: 71% (2486/3501) Updating files: 72% (2521/3501) Updating files: 73% (2556/3501) Updating files: 74% (2591/3501) Updating files: 75% (2626/3501) Updating files: 76% (2661/3501) Updating files: 77% (2696/3501) Updating files: 78% (2731/3501) Updating files: 79% (2766/3501) Updating files: 80% (2801/3501) Updating files: 81% (2836/3501) Updating files: 82% (2871/3501) Updating files: 82% (2875/3501) Updating files: 83% (2906/3501) Updating files: 84% (2941/3501) Updating files: 85% (2976/3501) Updating files: 86% (3011/3501) Updating files: 87% (3046/3501) Updating files: 88% (3081/3501) Updating files: 89% (3116/3501) Updating files: 90% (3151/3501) Updating files: 91% (3186/3501) Updating files: 92% (3221/3501) Updating files: 93% (3256/3501) Updating files: 94% (3291/3501) Updating files: 95% (3326/3501) Updating files: 96% (3361/3501) Updating files: 97% (3396/3501) Updating files: 98% (3431/3501) Updating files: 99% (3466/3501) Updating files: 100% (3501/3501) Updating files: 100% (3501/3501), done. 2024-09-10T21:09:41.251 INFO:teuthology.orchestra.run.smithi017.stdout:HEAD is now at ce299c0f Merge branch 'rt1320' into 'main' 2024-09-10T21:09:41.274 INFO:teuthology.task.kernel:Downloading kernel 2ca44a2f35ca8f36983aaaa46d49312239ef519a on mon.a... 2024-09-10T21:09:41.274 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-10T21:09:41.428 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/ubuntu/jammy/flavors/default/pool/main/l/linux-6.11.0-rc5-g2ca44a2f35ca/ 2024-09-10T21:09:41.428 DEBUG:teuthology.orchestra.run.smithi017:> rm -f /tmp/linux-image.deb && echo linux-image-6.11.0-rc5-g2ca44a2f35ca_6.11.0-rc5-g2ca44a2f35ca-1_amd64.deb | wget -nv -O /tmp/linux-image.deb --base=https://4.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/ubuntu/jammy/flavors/default/pool/main/l/linux-6.11.0-rc5-g2ca44a2f35ca/ --input-file=- 2024-09-10T21:09:41.431 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi017.front.sepia.ceph.com to complete... 2024-09-10T21:09:42.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10 21:09:42 URL:https://4.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/ubuntu/jammy/flavors/default/pool/main/l/linux-6.11.0-rc5-g2ca44a2f35ca/linux-image-6.11.0-rc5-g2ca44a2f35ca_6.11.0-rc5-g2ca44a2f35ca-1_amd64.deb [45349154/45349154] -> "/tmp/linux-image.deb" [1] 2024-09-10T21:09:42.461 INFO:teuthology.orchestra.run.smithi017.stderr:FINISHED --2024-09-10 21:09:42-- 2024-09-10T21:09:42.461 INFO:teuthology.orchestra.run.smithi017.stderr:Total wall clock time: 1.0s 2024-09-10T21:09:42.461 INFO:teuthology.orchestra.run.smithi017.stderr:Downloaded: 1 files, 43M in 0.8s (51.0 MB/s) 2024-09-10T21:09:42.466 INFO:teuthology.task.kernel:Installing kernel 2ca44a2f35ca8f36983aaaa46d49312239ef519a on mon.a... 2024-09-10T21:09:42.466 DEBUG:teuthology.orchestra.run.smithi017:> sudo dpkg -i /tmp/linux-image.deb 2024-09-10T21:09:42.772 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package linux-image-6.11.0-rc5-g2ca44a2f35ca. 2024-09-10T21:09:44.709 INFO:teuthology.orchestra.run.smithi017.stdout:(Reading database ... 129542 files and directories currently installed.) 2024-09-10T21:09:44.713 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack /tmp/linux-image.deb ... 2024-09-10T21:09:44.860 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking linux-image-6.11.0-rc5-g2ca44a2f35ca (6.11.0-rc5-g2ca44a2f35ca-1) ... 2024-09-10T21:09:45.063 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up linux-image-6.11.0-rc5-g2ca44a2f35ca (6.11.0-rc5-g2ca44a2f35ca-1) ... 2024-09-10T21:09:45.170 INFO:teuthology.orchestra.run.smithi031.stderr:update-initramfs: Generating /boot/initrd.img-6.11.0-rc5-g2ca44a2f35ca 2024-09-10T21:09:49.121 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up linux-image-6.11.0-rc5-g2ca44a2f35ca (6.11.0-rc5-g2ca44a2f35ca-1) ... 2024-09-10T21:09:49.255 INFO:teuthology.orchestra.run.smithi017.stderr:update-initramfs: Generating /boot/initrd.img-6.11.0-rc5-g2ca44a2f35ca 2024-09-10T21:10:08.735 INFO:teuthology.orchestra.run.smithi031.stderr:Sourcing file `/etc/default/grub' 2024-09-10T21:10:08.745 INFO:teuthology.orchestra.run.smithi031.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-10T21:10:08.765 INFO:teuthology.orchestra.run.smithi031.stderr:Generating grub configuration file ... 2024-09-10T21:10:09.048 INFO:teuthology.orchestra.run.smithi031.stderr:Found linux image: /boot/vmlinuz-6.11.0-rc5-g2ca44a2f35ca 2024-09-10T21:10:09.063 INFO:teuthology.orchestra.run.smithi031.stderr:Found initrd image: /boot/initrd.img-6.11.0-rc5-g2ca44a2f35ca 2024-09-10T21:10:09.518 INFO:teuthology.orchestra.run.smithi031.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-10T21:10:09.533 INFO:teuthology.orchestra.run.smithi031.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-10T21:10:09.598 INFO:teuthology.orchestra.run.smithi031.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-10T21:10:09.603 INFO:teuthology.orchestra.run.smithi031.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-10T21:10:09.751 INFO:teuthology.orchestra.run.smithi031.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-10T21:10:09.751 INFO:teuthology.orchestra.run.smithi031.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-09-10T21:10:09.751 INFO:teuthology.orchestra.run.smithi031.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-10T21:10:09.799 INFO:teuthology.orchestra.run.smithi031.stderr:done 2024-09-10T21:10:10.003 DEBUG:teuthology.orchestra.run.smithi031:> dpkg-deb -c /tmp/linux-image.deb 2024-09-10T21:10:10.011 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./ 2024-09-10T21:10:10.011 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./boot/ 2024-09-10T21:10:10.011 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 7098725 2024-09-06 14:57 ./boot/System.map-6.11.0-rc5-g2ca44a2f35ca 2024-09-10T21:10:10.026 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 174121 2024-09-06 14:57 ./boot/config-6.11.0-rc5-g2ca44a2f35ca 2024-09-10T21:10:10.026 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13292544 2024-09-06 14:57 ./boot/vmlinuz-6.11.0-rc5-g2ca44a2f35ca 2024-09-10T21:10:10.048 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./etc/ 2024-09-10T21:10:10.048 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./etc/kernel/ 2024-09-10T21:10:10.048 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./etc/kernel/postinst.d/ 2024-09-10T21:10:10.048 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./etc/kernel/postrm.d/ 2024-09-10T21:10:10.048 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./etc/kernel/preinst.d/ 2024-09-10T21:10:10.049 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./etc/kernel/prerm.d/ 2024-09-10T21:10:10.049 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/ 2024-09-10T21:10:10.049 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/ 2024-09-10T21:10:10.049 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/ 2024-09-10T21:10:10.049 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/ 2024-09-10T21:10:10.049 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/ 2024-09-10T21:10:10.049 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/ 2024-09-10T21:10:10.049 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/ 2024-09-10T21:10:10.049 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 189809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/aesni-intel.ko 2024-09-10T21:10:10.049 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 28449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-09-10T21:10:10.049 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 33993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-09-10T21:10:10.049 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 35097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-09-10T21:10:10.049 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 97321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-09-10T21:10:10.050 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 109985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-09-10T21:10:10.050 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 155337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-09-10T21:10:10.050 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 15441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-09-10T21:10:10.050 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 16177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-09-10T21:10:10.050 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 87097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-09-10T21:10:10.050 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 22945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-09-10T21:10:10.050 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 49905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-09-10T21:10:10.050 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 50457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-avx2.ko 2024-09-10T21:10:10.050 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 54593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-09-10T21:10:10.050 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 44745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-09-10T21:10:10.050 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 48073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-09-10T21:10:10.050 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 62961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-09-10T21:10:10.050 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 53657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-09-10T21:10:10.051 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 34441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-09-10T21:10:10.051 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 14585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-09-10T21:10:10.051 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/ 2024-09-10T21:10:10.051 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/ 2024-09-10T21:10:10.051 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/mce/ 2024-09-10T21:10:10.051 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 49497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-09-10T21:10:10.051 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 16073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpuid.ko 2024-09-10T21:10:10.051 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 21713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/msr.ko 2024-09-10T21:10:10.051 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/ 2024-09-10T21:10:10.051 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 505169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm-amd.ko 2024-09-10T21:10:10.051 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 1060585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm-intel.ko 2024-09-10T21:10:10.052 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 2858865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm.ko 2024-09-10T21:10:10.115 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ 2024-09-10T21:10:10.115 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 62113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/af_alg.ko 2024-09-10T21:10:10.115 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 28033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/algif_hash.ko 2024-09-10T21:10:10.115 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 25921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/algif_skcipher.ko 2024-09-10T21:10:10.115 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 20313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ansi_cprng.ko 2024-09-10T21:10:10.115 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 21153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/anubis.ko 2024-09-10T21:10:10.115 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 11017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/arc4.ko 2024-09-10T21:10:10.115 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/asymmetric_keys/ 2024-09-10T21:10:10.116 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 11545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/asymmetric_keys/pkcs7_test_key.ko 2024-09-10T21:10:10.116 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/ 2024-09-10T21:10:10.116 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 12545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_memcpy.ko 2024-09-10T21:10:10.116 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 22321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_pq.ko 2024-09-10T21:10:10.116 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 24537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_raid6_recov.ko 2024-09-10T21:10:10.116 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 15841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_tx.ko 2024-09-10T21:10:10.116 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 20145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_xor.ko 2024-09-10T21:10:10.116 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 20825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/raid6test.ko 2024-09-10T21:10:10.116 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 24985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/authenc.ko 2024-09-10T21:10:10.116 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 28049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/authencesn.ko 2024-09-10T21:10:10.116 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 23905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blake2b_generic.ko 2024-09-10T21:10:10.116 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blowfish_common.ko 2024-09-10T21:10:10.116 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 11697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blowfish_generic.ko 2024-09-10T21:10:10.117 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 40977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/camellia_generic.ko 2024-09-10T21:10:10.117 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 28817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast5_generic.ko 2024-09-10T21:10:10.117 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 20185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast6_generic.ko 2024-09-10T21:10:10.117 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 10193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast_common.ko 2024-09-10T21:10:10.117 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 43377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ccm.ko 2024-09-10T21:10:10.117 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 17593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cmac.ko 2024-09-10T21:10:10.117 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 11809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crc32_generic.ko 2024-09-10T21:10:10.117 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 57801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cryptd.ko 2024-09-10T21:10:10.117 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 44585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_engine.ko 2024-09-10T21:10:10.117 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 31169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_simd.ko 2024-09-10T21:10:10.117 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 30793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_user.ko 2024-09-10T21:10:10.117 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 21793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cts.ko 2024-09-10T21:10:10.117 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 12569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/des_generic.ko 2024-09-10T21:10:10.117 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 18177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/dh_generic.ko 2024-09-10T21:10:10.118 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 12945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/echainiv.ko 2024-09-10T21:10:10.118 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 30793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/essiv.ko 2024-09-10T21:10:10.118 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 19289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/fcrypt.ko 2024-09-10T21:10:10.118 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 28289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/khazad.ko 2024-09-10T21:10:10.118 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 19177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lrw.ko 2024-09-10T21:10:10.118 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 14201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lz4.ko 2024-09-10T21:10:10.118 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 14169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lz4hc.ko 2024-09-10T21:10:10.118 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/md4.ko 2024-09-10T21:10:10.118 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 11329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/michael_mic.ko 2024-09-10T21:10:10.118 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/pcbc.ko 2024-09-10T21:10:10.118 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 23161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/pcrypt.ko 2024-09-10T21:10:10.118 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 17745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/rmd160.ko 2024-09-10T21:10:10.118 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 30753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/seed.ko 2024-09-10T21:10:10.119 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 28417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/serpent_generic.ko 2024-09-10T21:10:10.119 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 169521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/tcrypt.ko 2024-09-10T21:10:10.119 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/tea.ko 2024-09-10T21:10:10.119 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 29417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/twofish_common.ko 2024-09-10T21:10:10.119 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 14393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/twofish_generic.ko 2024-09-10T21:10:10.119 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 20001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/vmac.ko 2024-09-10T21:10:10.119 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 38297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/wp512.ko 2024-09-10T21:10:10.119 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 16713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xcbc.ko 2024-09-10T21:10:10.119 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 30329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xor.ko 2024-09-10T21:10:10.119 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 23929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xts.ko 2024-09-10T21:10:10.119 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 11361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xxhash_generic.ko 2024-09-10T21:10:10.119 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ 2024-09-10T21:10:10.119 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/ 2024-09-10T21:10:10.120 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 21217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_extlog.ko 2024-09-10T21:10:10.120 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 33625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_ipmi.ko 2024-09-10T21:10:10.120 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 32593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_pad.ko 2024-09-10T21:10:10.120 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/apei/ 2024-09-10T21:10:10.120 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 44825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/apei/einj.ko 2024-09-10T21:10:10.120 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 14593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/ec_sys.ko 2024-09-10T21:10:10.120 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 19561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/platform_profile.ko 2024-09-10T21:10:10.120 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 29009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/sbs.ko 2024-09-10T21:10:10.120 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 22937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/sbshc.ko 2024-09-10T21:10:10.120 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 120321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/video.ko 2024-09-10T21:10:10.120 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ 2024-09-10T21:10:10.120 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 24585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/acard-ahci.ko 2024-09-10T21:10:10.120 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 100457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ahci.ko 2024-09-10T21:10:10.121 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 15945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ahci_platform.ko 2024-09-10T21:10:10.121 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 105105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/libahci.ko 2024-09-10T21:10:10.121 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 39913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/libahci_platform.ko 2024-09-10T21:10:10.121 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 19753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_acpi.ko 2024-09-10T21:10:10.121 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 35793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ali.ko 2024-09-10T21:10:10.121 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 38417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_amd.ko 2024-09-10T21:10:10.121 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 24801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_artop.ko 2024-09-10T21:10:10.121 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 23865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_atiixp.ko 2024-09-10T21:10:10.121 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 28873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_atp867x.ko 2024-09-10T21:10:10.121 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 20505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cmd640.ko 2024-09-10T21:10:10.121 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 29081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cmd64x.ko 2024-09-10T21:10:10.121 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 16721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cypress.ko 2024-09-10T21:10:10.121 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 20225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_efar.ko 2024-09-10T21:10:10.122 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 25297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt366.ko 2024-09-10T21:10:10.122 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 39065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt37x.ko 2024-09-10T21:10:10.122 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 27057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt3x2n.ko 2024-09-10T21:10:10.122 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 19257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt3x3.ko 2024-09-10T21:10:10.122 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 19289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_it8213.ko 2024-09-10T21:10:10.122 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 41385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_it821x.ko 2024-09-10T21:10:10.122 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 14185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_jmicron.ko 2024-09-10T21:10:10.122 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 60665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_legacy.ko 2024-09-10T21:10:10.122 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 17105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_marvell.ko 2024-09-10T21:10:10.122 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 18153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_mpiix.ko 2024-09-10T21:10:10.122 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 14225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_netcell.ko 2024-09-10T21:10:10.122 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 19065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ninja32.ko 2024-09-10T21:10:10.122 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 16849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ns87410.ko 2024-09-10T21:10:10.123 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 20417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ns87415.ko 2024-09-10T21:10:10.123 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 18513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_oldpiix.ko 2024-09-10T21:10:10.123 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 17569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_opti.ko 2024-09-10T21:10:10.123 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 23769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_optidma.ko 2024-09-10T21:10:10.123 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 35233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pcmcia.ko 2024-09-10T21:10:10.123 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 43097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pdc2027x.ko 2024-09-10T21:10:10.123 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 24025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-09-10T21:10:10.123 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 15697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_piccolo.ko 2024-09-10T21:10:10.123 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 17345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_radisys.ko 2024-09-10T21:10:10.123 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 22321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_rdc.ko 2024-09-10T21:10:10.123 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 17385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_rz1000.ko 2024-09-10T21:10:10.123 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 15529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sch.ko 2024-09-10T21:10:10.123 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 26953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_serverworks.ko 2024-09-10T21:10:10.124 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 27393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sil680.ko 2024-09-10T21:10:10.124 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 22985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sl82c105.ko 2024-09-10T21:10:10.124 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 18129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_triflex.ko 2024-09-10T21:10:10.124 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 35121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_via.ko 2024-09-10T21:10:10.124 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 26425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pdc_adma.ko 2024-09-10T21:10:10.124 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 34977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_inic162x.ko 2024-09-10T21:10:10.124 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 102273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_mv.ko 2024-09-10T21:10:10.124 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 85385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_nv.ko 2024-09-10T21:10:10.124 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 44401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_promise.ko 2024-09-10T21:10:10.124 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 26761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_qstor.ko 2024-09-10T21:10:10.124 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 34113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sil.ko 2024-09-10T21:10:10.124 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 45993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sil24.ko 2024-09-10T21:10:10.124 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 20337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sis.ko 2024-09-10T21:10:10.125 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 24345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_svw.ko 2024-09-10T21:10:10.125 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 53449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sx4.ko 2024-09-10T21:10:10.125 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 17433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_uli.ko 2024-09-10T21:10:10.125 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 41137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_via.ko 2024-09-10T21:10:10.125 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 23409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_vsc.ko 2024-09-10T21:10:10.125 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/auxdisplay/ 2024-09-10T21:10:10.125 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 18361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/auxdisplay/ks0108.ko 2024-09-10T21:10:10.125 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/bcma/ 2024-09-10T21:10:10.125 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 161225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/bcma/bcma.ko 2024-09-10T21:10:10.125 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/ 2024-09-10T21:10:10.125 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/aoe/ 2024-09-10T21:10:10.125 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 141025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/aoe/aoe.ko 2024-09-10T21:10:10.125 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/drbd/ 2024-09-10T21:10:10.126 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 1240825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/drbd/drbd.ko 2024-09-10T21:10:10.126 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 203657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/floppy.ko 2024-09-10T21:10:10.126 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/mtip32xx/ 2024-09-10T21:10:10.126 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 115057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-09-10T21:10:10.126 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 143337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/nbd.ko 2024-09-10T21:10:10.126 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/null_blk/ 2024-09-10T21:10:10.126 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 133345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/null_blk/null_blk.ko 2024-09-10T21:10:10.126 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 119929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/pktcdvd.ko 2024-09-10T21:10:10.126 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 312817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/rbd.ko 2024-09-10T21:10:10.126 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkback/ 2024-09-10T21:10:10.126 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 117753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-09-10T21:10:10.126 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/zram/ 2024-09-10T21:10:10.126 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 70201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/zram/zram.ko 2024-09-10T21:10:10.127 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ 2024-09-10T21:10:10.127 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 35289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/applicom.ko 2024-09-10T21:10:10.127 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hangcheck-timer.ko 2024-09-10T21:10:10.127 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/ 2024-09-10T21:10:10.127 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 16417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/amd-rng.ko 2024-09-10T21:10:10.127 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 21633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/intel-rng.ko 2024-09-10T21:10:10.127 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 15393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-09-10T21:10:10.127 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 12881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/via-rng.ko 2024-09-10T21:10:10.127 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 19345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/virtio-rng.ko 2024-09-10T21:10:10.127 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ 2024-09-10T21:10:10.127 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 37353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-09-10T21:10:10.127 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 183745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-09-10T21:10:10.127 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 32601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-09-10T21:10:10.128 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 179465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_si.ko 2024-09-10T21:10:10.128 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 85465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-09-10T21:10:10.128 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 65425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-09-10T21:10:10.128 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 43993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/lp.ko 2024-09-10T21:10:10.128 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/mwave/ 2024-09-10T21:10:10.128 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 103521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/mwave/mwave.ko 2024-09-10T21:10:10.128 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 31745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/nvram.ko 2024-09-10T21:10:10.128 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 39249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ppdev.ko 2024-09-10T21:10:10.128 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 59449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tlclk.ko 2024-09-10T21:10:10.128 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/ 2024-09-10T21:10:10.128 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 18977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_atmel.ko 2024-09-10T21:10:10.128 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 18265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_i2c_atmel.ko 2024-09-10T21:10:10.128 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 29513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_i2c_infineon.ko 2024-09-10T21:10:10.129 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 39281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_i2c_nuvoton.ko 2024-09-10T21:10:10.129 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 34257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_infineon.ko 2024-09-10T21:10:10.129 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 26041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_nsc.ko 2024-09-10T21:10:10.129 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 28601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/xen-tpmfront.ko 2024-09-10T21:10:10.129 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/xillybus/ 2024-09-10T21:10:10.129 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 18849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/xillybus/xillybus_class.ko 2024-09-10T21:10:10.129 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 62201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/xillybus/xillybus_core.ko 2024-09-10T21:10:10.129 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 14017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/xillybus/xillybus_pcie.ko 2024-09-10T21:10:10.129 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/clk/ 2024-09-10T21:10:10.129 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 14681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/clk/clk-palmas.ko 2024-09-10T21:10:10.129 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 57313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/clk/clk-si5351.ko 2024-09-10T21:10:10.129 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/clk/clk-twl6040.ko 2024-09-10T21:10:10.129 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 24209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/clk/clk-wm831x.ko 2024-09-10T21:10:10.130 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/ 2024-09-10T21:10:10.130 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 11673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-09-10T21:10:10.130 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 19265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/p4-clockmod.ko 2024-09-10T21:10:10.130 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 28241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/speedstep-lib.ko 2024-09-10T21:10:10.130 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ 2024-09-10T21:10:10.130 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ 2024-09-10T21:10:10.130 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 89361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-09-10T21:10:10.130 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 334505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ccp.ko 2024-09-10T21:10:10.130 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/ 2024-09-10T21:10:10.130 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/ 2024-09-10T21:10:10.130 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_common/ 2024-09-10T21:10:10.130 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 833113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-09-10T21:10:10.130 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_dh895xcc/ 2024-09-10T21:10:10.130 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 29657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-09-10T21:10:10.131 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 23273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/padlock-aes.ko 2024-09-10T21:10:10.131 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 26809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/padlock-sha.ko 2024-09-10T21:10:10.131 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/virtio/ 2024-09-10T21:10:10.131 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 90209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-09-10T21:10:10.131 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dca/ 2024-09-10T21:10:10.131 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 35681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dca/dca.ko 2024-09-10T21:10:10.131 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ 2024-09-10T21:10:10.131 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/dw/ 2024-09-10T21:10:10.131 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 18681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/dw/dw_dmac.ko 2024-09-10T21:10:10.131 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ioat/ 2024-09-10T21:10:10.131 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 147345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ioat/ioatdma.ko 2024-09-10T21:10:10.131 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/ 2024-09-10T21:10:10.132 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 90201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/amd64_edac.ko 2024-09-10T21:10:10.132 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 42073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/e752x_edac.ko 2024-09-10T21:10:10.132 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 50729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/edac_mce_amd.ko 2024-09-10T21:10:10.132 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 21097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i3000_edac.ko 2024-09-10T21:10:10.132 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 20817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i3200_edac.ko 2024-09-10T21:10:10.132 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 26817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i5100_edac.ko 2024-09-10T21:10:10.132 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 29689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i5400_edac.ko 2024-09-10T21:10:10.132 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 28481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i7300_edac.ko 2024-09-10T21:10:10.132 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 64265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i7core_edac.ko 2024-09-10T21:10:10.132 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 21369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i82975x_edac.ko 2024-09-10T21:10:10.132 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 22449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/ie31200_edac.ko 2024-09-10T21:10:10.132 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 68801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/sb_edac.ko 2024-09-10T21:10:10.133 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 20385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/x38_edac.ko 2024-09-10T21:10:10.133 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/ 2024-09-10T21:10:10.133 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 14681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-gpio.ko 2024-09-10T21:10:10.133 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 31457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-max14577.ko 2024-09-10T21:10:10.133 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 35681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-max77693.ko 2024-09-10T21:10:10.133 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 29361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-max8997.ko 2024-09-10T21:10:10.133 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 31049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-palmas.ko 2024-09-10T21:10:10.133 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 27289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-rt8973a.ko 2024-09-10T21:10:10.133 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 32073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-sm5502.ko 2024-09-10T21:10:10.133 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/ 2024-09-10T21:10:10.133 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 43049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/dmi-sysfs.ko 2024-09-10T21:10:10.133 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/efi/ 2024-09-10T21:10:10.133 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 20121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/efi/efi-pstore.ko 2024-09-10T21:10:10.134 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 26449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/iscsi_ibft.ko 2024-09-10T21:10:10.134 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/ 2024-09-10T21:10:10.134 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 14361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-adp5520.ko 2024-09-10T21:10:10.134 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 23441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-amd8111.ko 2024-09-10T21:10:10.134 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 17769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-arizona.ko 2024-09-10T21:10:10.134 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 16825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-da9052.ko 2024-09-10T21:10:10.134 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 14081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-da9055.ko 2024-09-10T21:10:10.134 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 27025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-dln2.ko 2024-09-10T21:10:10.134 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 81153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-f7188x.ko 2024-09-10T21:10:10.134 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 40553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-generic.ko 2024-09-10T21:10:10.134 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 22593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-ich.ko 2024-09-10T21:10:10.134 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 14113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-janz-ttl.ko 2024-09-10T21:10:10.135 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 16977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-kempld.ko 2024-09-10T21:10:10.135 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 14817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-lp3943.ko 2024-09-10T21:10:10.135 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 11241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-max7300.ko 2024-09-10T21:10:10.135 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 14529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-max730x.ko 2024-09-10T21:10:10.135 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 24825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-max732x.ko 2024-09-10T21:10:10.135 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 30033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-ml-ioh.ko 2024-09-10T21:10:10.135 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 56001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-pca953x.ko 2024-09-10T21:10:10.135 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 32081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-pcf857x.ko 2024-09-10T21:10:10.135 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 17337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-rdc321x.ko 2024-09-10T21:10:10.135 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 27025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-sch.ko 2024-09-10T21:10:10.135 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 23233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-sch311x.ko 2024-09-10T21:10:10.135 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 14353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-tps65912.ko 2024-09-10T21:10:10.135 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 25097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-twl4030.ko 2024-09-10T21:10:10.135 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-twl6040.ko 2024-09-10T21:10:10.136 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 24545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-viperboard.ko 2024-09-10T21:10:10.136 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 17985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-vx855.ko 2024-09-10T21:10:10.136 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 19585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-wm831x.ko 2024-09-10T21:10:10.136 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-wm8350.ko 2024-09-10T21:10:10.136 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 20489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-wm8994.ko 2024-09-10T21:10:10.136 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/ 2024-09-10T21:10:10.136 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-a4tech.ko 2024-09-10T21:10:10.136 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 44473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-apple.ko 2024-09-10T21:10:10.136 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 16969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-appleir.ko 2024-09-10T21:10:10.136 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 9593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-aureal.ko 2024-09-10T21:10:10.136 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 17193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-axff.ko 2024-09-10T21:10:10.136 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 12833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-belkin.ko 2024-09-10T21:10:10.136 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 11913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-cherry.ko 2024-09-10T21:10:10.137 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 19225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-chicony.ko 2024-09-10T21:10:10.137 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 74497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-cp2112.ko 2024-09-10T21:10:10.137 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-cypress.ko 2024-09-10T21:10:10.137 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 19041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-dr.ko 2024-09-10T21:10:10.137 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 12313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-elecom.ko 2024-09-10T21:10:10.137 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 20665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-elo.ko 2024-09-10T21:10:10.137 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 15345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-emsff.ko 2024-09-10T21:10:10.137 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 11865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-ezkey.ko 2024-09-10T21:10:10.137 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 16361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gaff.ko 2024-09-10T21:10:10.137 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 10769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-generic.ko 2024-09-10T21:10:10.137 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 19881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gt683r.ko 2024-09-10T21:10:10.137 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 14865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gyration.ko 2024-09-10T21:10:10.137 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 12281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtek-kbd.ko 2024-09-10T21:10:10.138 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 12121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtek-mouse.ko 2024-09-10T21:10:10.138 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 16521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtekff.ko 2024-09-10T21:10:10.138 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 22993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-hyperv.ko 2024-09-10T21:10:10.138 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 11977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-icade.ko 2024-09-10T21:10:10.138 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 10593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-kensington.ko 2024-09-10T21:10:10.138 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 9729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-keytouch.ko 2024-09-10T21:10:10.138 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 19617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-kye.ko 2024-09-10T21:10:10.138 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 12401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lcpower.ko 2024-09-10T21:10:10.138 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 25049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-led.ko 2024-09-10T21:10:10.138 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 54337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lenovo.ko 2024-09-10T21:10:10.138 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 37753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lg-g15.ko 2024-09-10T21:10:10.138 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 67841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech-dj.ko 2024-09-10T21:10:10.138 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 122145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-09-10T21:10:10.139 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 98177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech.ko 2024-09-10T21:10:10.139 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 36897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-magicmouse.ko 2024-09-10T21:10:10.139 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 25969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-microsoft.ko 2024-09-10T21:10:10.139 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 12025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-monterey.ko 2024-09-10T21:10:10.139 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 60505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-multitouch.ko 2024-09-10T21:10:10.139 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 39905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-ntrig.ko 2024-09-10T21:10:10.139 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 10185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-ortek.ko 2024-09-10T21:10:10.139 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 10265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-penmount.ko 2024-09-10T21:10:10.139 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 14481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-petalynx.ko 2024-09-10T21:10:10.139 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 91217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-picolcd.ko 2024-09-10T21:10:10.139 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 16073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-pl.ko 2024-09-10T21:10:10.139 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 16897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-plantronics.ko 2024-09-10T21:10:10.139 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 9617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-primax.ko 2024-09-10T21:10:10.140 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 44329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-rmi.ko 2024-09-10T21:10:10.140 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 25273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-arvo.ko 2024-09-10T21:10:10.140 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 15657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-common.ko 2024-09-10T21:10:10.140 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 39009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-isku.ko 2024-09-10T21:10:10.140 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 36377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-kone.ko 2024-09-10T21:10:10.140 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 38169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-09-10T21:10:10.140 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 26193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-konepure.ko 2024-09-10T21:10:10.140 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 35721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-09-10T21:10:10.140 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 15577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-lua.ko 2024-09-10T21:10:10.140 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 35497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-pyra.ko 2024-09-10T21:10:10.140 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 35657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-ryos.ko 2024-09-10T21:10:10.140 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 24089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-savu.ko 2024-09-10T21:10:10.140 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 27625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat.ko 2024-09-10T21:10:10.141 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 15921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-saitek.ko 2024-09-10T21:10:10.141 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 42505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-samsung.ko 2024-09-10T21:10:10.141 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 42609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sensor-hub.ko 2024-09-10T21:10:10.141 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 15353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sjoy.ko 2024-09-10T21:10:10.141 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 62345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sony.ko 2024-09-10T21:10:10.141 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 9553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-speedlink.ko 2024-09-10T21:10:10.141 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 30241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-steelseries.ko 2024-09-10T21:10:10.141 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 11385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sunplus.ko 2024-09-10T21:10:10.141 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 19393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-thrustmaster.ko 2024-09-10T21:10:10.141 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 12057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-tivo.ko 2024-09-10T21:10:10.141 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 16889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-tmff.ko 2024-09-10T21:10:10.141 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 16737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-topseed.ko 2024-09-10T21:10:10.141 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 19409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-twinhan.ko 2024-09-10T21:10:10.142 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 81673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-uclogic.ko 2024-09-10T21:10:10.142 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 12769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-waltop.ko 2024-09-10T21:10:10.142 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 184865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-wiimote.ko 2024-09-10T21:10:10.142 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 9385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-xinmo.ko 2024-09-10T21:10:10.142 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 15169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-zpff.ko 2024-09-10T21:10:10.142 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 19401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-zydacron.ko 2024-09-10T21:10:10.142 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 462849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid.ko 2024-09-10T21:10:10.142 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 39761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/uhid.ko 2024-09-10T21:10:10.142 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/usbhid/ 2024-09-10T21:10:10.142 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 153953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/usbhid/usbhid.ko 2024-09-10T21:10:10.142 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 22457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/usbhid/usbkbd.ko 2024-09-10T21:10:10.142 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 17553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/usbhid/usbmouse.ko 2024-09-10T21:10:10.142 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 261241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/wacom.ko 2024-09-10T21:10:10.143 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/ 2024-09-10T21:10:10.143 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 76441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_balloon.ko 2024-09-10T21:10:10.143 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 96073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_utils.ko 2024-09-10T21:10:10.143 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 378337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_vmbus.ko 2024-09-10T21:10:10.143 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ 2024-09-10T21:10:10.143 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 77217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/abituguru.ko 2024-09-10T21:10:10.143 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 67097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/abituguru3.ko 2024-09-10T21:10:10.143 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 43273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/acpi_power_meter.ko 2024-09-10T21:10:10.143 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 20377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ad7414.ko 2024-09-10T21:10:10.143 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 22105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ad7418.ko 2024-09-10T21:10:10.143 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 29121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adc128d818.ko 2024-09-10T21:10:10.143 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 38185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1025.ko 2024-09-10T21:10:10.143 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 115201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1026.ko 2024-09-10T21:10:10.143 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 23401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1029.ko 2024-09-10T21:10:10.144 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 55225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1031.ko 2024-09-10T21:10:10.144 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 31529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm9240.ko 2024-09-10T21:10:10.144 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 16249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ads7828.ko 2024-09-10T21:10:10.144 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 12337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7410.ko 2024-09-10T21:10:10.144 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 26593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7411.ko 2024-09-10T21:10:10.144 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 89113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7462.ko 2024-09-10T21:10:10.144 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 49233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7470.ko 2024-09-10T21:10:10.144 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 81665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7475.ko 2024-09-10T21:10:10.144 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 22681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7x10.ko 2024-09-10T21:10:10.144 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 34417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/amc6821.ko 2024-09-10T21:10:10.144 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 68953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/applesmc.ko 2024-09-10T21:10:10.144 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 60281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asb100.ko 2024-09-10T21:10:10.144 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 69097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asc7621.ko 2024-09-10T21:10:10.145 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 48145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asus_atk0110.ko 2024-09-10T21:10:10.145 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 20409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/atxp1.ko 2024-09-10T21:10:10.145 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 33761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/coretemp.ko 2024-09-10T21:10:10.145 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 14201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/cros_ec_hwmon.ko 2024-09-10T21:10:10.145 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 29689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/da9052-hwmon.ko 2024-09-10T21:10:10.145 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 19889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/da9055-hwmon.ko 2024-09-10T21:10:10.145 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 57681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-09-10T21:10:10.145 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 113265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/dme1737.ko 2024-09-10T21:10:10.145 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 22753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ds1621.ko 2024-09-10T21:10:10.145 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 17577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ds620.ko 2024-09-10T21:10:10.145 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 26945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc1403.ko 2024-09-10T21:10:10.145 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 40265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc2103.ko 2024-09-10T21:10:10.145 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 34729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc6w201.ko 2024-09-10T21:10:10.145 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 79009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f71805f.ko 2024-09-10T21:10:10.146 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 109049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f71882fg.ko 2024-09-10T21:10:10.146 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 50769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f75375s.ko 2024-09-10T21:10:10.146 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 25449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/fam15h_power.ko 2024-09-10T21:10:10.146 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 65849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/fschmd.ko 2024-09-10T21:10:10.146 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 17289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/g760a.ko 2024-09-10T21:10:10.146 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 30889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/g762.ko 2024-09-10T21:10:10.146 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 59849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/gl518sm.ko 2024-09-10T21:10:10.146 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 56393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/gl520sm.ko 2024-09-10T21:10:10.146 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 15561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/hih6130.ko 2024-09-10T21:10:10.146 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 11865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/hwmon-vid.ko 2024-09-10T21:10:10.146 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/i5500_temp.ko 2024-09-10T21:10:10.146 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 27385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/i5k_amb.ko 2024-09-10T21:10:10.146 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 42681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ibmaem.ko 2024-09-10T21:10:10.147 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 24697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ibmpex.ko 2024-09-10T21:10:10.147 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 28497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ina209.ko 2024-09-10T21:10:10.147 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 32729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ina2xx.ko 2024-09-10T21:10:10.147 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 150681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/it87.ko 2024-09-10T21:10:10.147 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 23281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/jc42.ko 2024-09-10T21:10:10.147 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 28657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/k10temp.ko 2024-09-10T21:10:10.147 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 15817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/k8temp.ko 2024-09-10T21:10:10.147 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 21801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lineage-pem.ko 2024-09-10T21:10:10.147 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 57769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm63.ko 2024-09-10T21:10:10.147 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 19865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm73.ko 2024-09-10T21:10:10.147 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 44185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm75.ko 2024-09-10T21:10:10.147 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 23329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm77.ko 2024-09-10T21:10:10.147 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 41537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm78.ko 2024-09-10T21:10:10.148 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 37129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm80.ko 2024-09-10T21:10:10.148 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 20609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm83.ko 2024-09-10T21:10:10.148 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 86561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm85.ko 2024-09-10T21:10:10.148 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 55721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm87.ko 2024-09-10T21:10:10.148 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 73649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm90.ko 2024-09-10T21:10:10.148 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 23473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm92.ko 2024-09-10T21:10:10.148 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 131585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm93.ko 2024-09-10T21:10:10.148 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 40369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95234.ko 2024-09-10T21:10:10.148 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 19225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95241.ko 2024-09-10T21:10:10.148 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 21777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95245.ko 2024-09-10T21:10:10.148 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 26409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc2945.ko 2024-09-10T21:10:10.148 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 15569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4151.ko 2024-09-10T21:10:10.148 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 17881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4215.ko 2024-09-10T21:10:10.149 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 16657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4222.ko 2024-09-10T21:10:10.149 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 18457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4245.ko 2024-09-10T21:10:10.149 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 14993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4260.ko 2024-09-10T21:10:10.149 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 17201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4261.ko 2024-09-10T21:10:10.149 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 36857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max16065.ko 2024-09-10T21:10:10.149 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 21673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max1619.ko 2024-09-10T21:10:10.149 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 28321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max1668.ko 2024-09-10T21:10:10.149 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 22841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max197.ko 2024-09-10T21:10:10.149 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 24097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6639.ko 2024-09-10T21:10:10.149 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 27385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6650.ko 2024-09-10T21:10:10.149 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 39777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6697.ko 2024-09-10T21:10:10.149 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 21601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/mc13783-adc.ko 2024-09-10T21:10:10.149 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 12417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/mcp3021.ko 2024-09-10T21:10:10.149 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 17961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/menf21bmc_hwmon.ko 2024-09-10T21:10:10.150 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 56825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6683.ko 2024-09-10T21:10:10.150 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 140657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6775-core.ko 2024-09-10T21:10:10.150 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 72761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6775.ko 2024-09-10T21:10:10.150 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 67865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct7802.ko 2024-09-10T21:10:10.150 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 97865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pc87360.ko 2024-09-10T21:10:10.150 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 62009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pc87427.ko 2024-09-10T21:10:10.150 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 21905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pcf8591.ko 2024-09-10T21:10:10.150 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ 2024-09-10T21:10:10.150 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 25273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-09-10T21:10:10.150 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 19425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-09-10T21:10:10.150 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 33345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-09-10T21:10:10.150 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 10881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max16064.ko 2024-09-10T21:10:10.150 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 19921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max34440.ko 2024-09-10T21:10:10.151 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 12049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max8688.ko 2024-09-10T21:10:10.151 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 17673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-09-10T21:10:10.151 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 98961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-09-10T21:10:10.151 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 9425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-09-10T21:10:10.151 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 36265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-09-10T21:10:10.151 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 16961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-09-10T21:10:10.151 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 24529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-09-10T21:10:10.151 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 15417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/powr1220.ko 2024-09-10T21:10:10.151 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 25521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch5627.ko 2024-09-10T21:10:10.151 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 33033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch5636.ko 2024-09-10T21:10:10.151 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 33665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch56xx-common.ko 2024-09-10T21:10:10.151 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 40937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sht15.ko 2024-09-10T21:10:10.151 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 16065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sht21.ko 2024-09-10T21:10:10.152 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 19337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/shtc1.ko 2024-09-10T21:10:10.152 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 48793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sis5595.ko 2024-09-10T21:10:10.152 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 22489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47b397.ko 2024-09-10T21:10:10.152 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 46561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47m1.ko 2024-09-10T21:10:10.152 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 44401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47m192.ko 2024-09-10T21:10:10.152 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 31489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/thmc50.ko 2024-09-10T21:10:10.152 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 20177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp102.ko 2024-09-10T21:10:10.152 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 16529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp103.ko 2024-09-10T21:10:10.152 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 25185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp401.ko 2024-09-10T21:10:10.152 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 23097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp421.ko 2024-09-10T21:10:10.152 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 23129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/via-cputemp.ko 2024-09-10T21:10:10.152 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 50297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/via686a.ko 2024-09-10T21:10:10.152 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 59745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/vt1211.ko 2024-09-10T21:10:10.153 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 62817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/vt8231.ko 2024-09-10T21:10:10.153 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 71193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83627ehf.ko 2024-09-10T21:10:10.153 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 96665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83627hf.ko 2024-09-10T21:10:10.153 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 85977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83781d.ko 2024-09-10T21:10:10.153 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 75217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83791d.ko 2024-09-10T21:10:10.153 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 83505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83792d.ko 2024-09-10T21:10:10.153 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 116873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83793.ko 2024-09-10T21:10:10.153 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 134513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83795.ko 2024-09-10T21:10:10.153 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 20209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83l785ts.ko 2024-09-10T21:10:10.153 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 43473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83l786ng.ko 2024-09-10T21:10:10.153 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 15569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/wm831x-hwmon.ko 2024-09-10T21:10:10.153 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 12081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/wm8350-hwmon.ko 2024-09-10T21:10:10.153 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/ 2024-09-10T21:10:10.154 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/ 2024-09-10T21:10:10.154 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 26977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-09-10T21:10:10.154 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 25041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-09-10T21:10:10.154 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/ 2024-09-10T21:10:10.154 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 33225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ali1535.ko 2024-09-10T21:10:10.154 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 35073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ali1563.ko 2024-09-10T21:10:10.154 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 32921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ali15x3.ko 2024-09-10T21:10:10.154 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 21473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-09-10T21:10:10.154 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 30217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-09-10T21:10:10.154 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 19409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-09-10T21:10:10.154 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 17825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-cbus-gpio.ko 2024-09-10T21:10:10.154 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 7697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-09-10T21:10:10.154 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 15929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-cros-ec-tunnel.ko 2024-09-10T21:10:10.155 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 63537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-09-10T21:10:10.155 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 28201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-09-10T21:10:10.155 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 24569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-09-10T21:10:10.155 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 20001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-09-10T21:10:10.155 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 14489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-dln2.ko 2024-09-10T21:10:10.155 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 20345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-gpio.ko 2024-09-10T21:10:10.155 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 78577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-i801.ko 2024-09-10T21:10:10.155 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 24793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-isch.ko 2024-09-10T21:10:10.155 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 52945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-09-10T21:10:10.155 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 20969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-kempld.ko 2024-09-10T21:10:10.155 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 21409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-09-10T21:10:10.155 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 25809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-09-10T21:10:10.155 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 33001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ocores.ko 2024-09-10T21:10:10.155 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 30145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-parport.ko 2024-09-10T21:10:10.156 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 20137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-09-10T21:10:10.156 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 57137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-09-10T21:10:10.156 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 17521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-robotfuzz-osif.ko 2024-09-10T21:10:10.156 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 21241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-09-10T21:10:10.156 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 14057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-09-10T21:10:10.156 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 24793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-sis5595.ko 2024-09-10T21:10:10.156 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 32673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-sis630.ko 2024-09-10T21:10:10.156 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 22697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-09-10T21:10:10.156 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 22921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-taos-evm.ko 2024-09-10T21:10:10.156 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 22249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-09-10T21:10:10.156 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 15913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-via.ko 2024-09-10T21:10:10.156 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 30073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-09-10T21:10:10.156 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 21353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-09-10T21:10:10.157 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 59145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-xiic.ko 2024-09-10T21:10:10.157 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 27185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-mux.ko 2024-09-10T21:10:10.157 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 22913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-smbus.ko 2024-09-10T21:10:10.157 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 29665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-stub.ko 2024-09-10T21:10:10.157 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/muxes/ 2024-09-10T21:10:10.157 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 18641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/muxes/i2c-mux-gpio.ko 2024-09-10T21:10:10.157 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 16081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/muxes/i2c-mux-pca9541.ko 2024-09-10T21:10:10.157 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 36729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/muxes/i2c-mux-pca954x.ko 2024-09-10T21:10:10.157 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ 2024-09-10T21:10:10.157 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ 2024-09-10T21:10:10.157 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 328633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_cm.ko 2024-09-10T21:10:10.157 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 1107329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_core.ko 2024-09-10T21:10:10.157 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 84409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_umad.ko 2024-09-10T21:10:10.157 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 404889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-09-10T21:10:10.158 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 120105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/iw_cm.ko 2024-09-10T21:10:10.158 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 343753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/rdma_cm.ko 2024-09-10T21:10:10.158 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 85057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-09-10T21:10:10.158 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/ 2024-09-10T21:10:10.158 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/cxgb4/ 2024-09-10T21:10:10.158 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 591177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-09-10T21:10:10.158 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx4/ 2024-09-10T21:10:10.158 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 515585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-09-10T21:10:10.158 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx5/ 2024-09-10T21:10:10.158 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 919921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-09-10T21:10:10.158 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mthca/ 2024-09-10T21:10:10.158 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 375761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-09-10T21:10:10.159 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/ocrdma/ 2024-09-10T21:10:10.159 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 203241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/ocrdma/ocrdma.ko 2024-09-10T21:10:10.159 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/usnic/ 2024-09-10T21:10:10.159 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 212353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/usnic/usnic_verbs.ko 2024-09-10T21:10:10.159 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ 2024-09-10T21:10:10.159 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ipoib/ 2024-09-10T21:10:10.159 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 334665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-09-10T21:10:10.159 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/iser/ 2024-09-10T21:10:10.159 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 120785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-09-10T21:10:10.159 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/isert/ 2024-09-10T21:10:10.159 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 132457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-09-10T21:10:10.159 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srp/ 2024-09-10T21:10:10.159 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 158257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-09-10T21:10:10.159 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srpt/ 2024-09-10T21:10:10.160 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 157209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-09-10T21:10:10.160 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/ 2024-09-10T21:10:10.160 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 12297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/evbug.ko 2024-09-10T21:10:10.160 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 27465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/ff-memless.ko 2024-09-10T21:10:10.160 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/ 2024-09-10T21:10:10.160 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-09-10T21:10:10.160 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 14081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/fm801-gp.ko 2024-09-10T21:10:10.160 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 49769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/gameport.ko 2024-09-10T21:10:10.160 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 49697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/joydev.ko 2024-09-10T21:10:10.160 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/ 2024-09-10T21:10:10.160 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 15617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/adp5520-keys.ko 2024-09-10T21:10:10.160 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 36289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/adp5588-keys.ko 2024-09-10T21:10:10.160 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 35753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/adp5589-keys.ko 2024-09-10T21:10:10.161 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 28393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/cros_ec_keyb.ko 2024-09-10T21:10:10.161 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 45537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/gpio_keys.ko 2024-09-10T21:10:10.161 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 22473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/gpio_keys_polled.ko 2024-09-10T21:10:10.161 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 25713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/lkkbd.ko 2024-09-10T21:10:10.161 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 31945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/lm8323.ko 2024-09-10T21:10:10.161 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 16849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/lm8333.ko 2024-09-10T21:10:10.161 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 25913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/matrix_keypad.ko 2024-09-10T21:10:10.161 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 21545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/max7359_keypad.ko 2024-09-10T21:10:10.161 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 19337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/mcs_touchkey.ko 2024-09-10T21:10:10.161 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 19201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/mpr121_touchkey.ko 2024-09-10T21:10:10.161 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/newtonkbd.ko 2024-09-10T21:10:10.161 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 12321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/opencores-kbd.ko 2024-09-10T21:10:10.161 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 17185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/qt1070.ko 2024-09-10T21:10:10.162 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 21649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/qt2160.ko 2024-09-10T21:10:10.162 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 28769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/samsung-keypad.ko 2024-09-10T21:10:10.162 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/stowaway.ko 2024-09-10T21:10:10.162 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 22433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/sunkbd.ko 2024-09-10T21:10:10.162 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 20737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/tca6416-keypad.ko 2024-09-10T21:10:10.162 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 19009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/tca8418_keypad.ko 2024-09-10T21:10:10.162 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 19657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/twl4030_keypad.ko 2024-09-10T21:10:10.162 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/xtkbd.ko 2024-09-10T21:10:10.162 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/matrix-keymap.ko 2024-09-10T21:10:10.162 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/ 2024-09-10T21:10:10.162 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 40713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/appletouch.ko 2024-09-10T21:10:10.162 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 39657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/bcm5974.ko 2024-09-10T21:10:10.162 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 160569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/cyapatp.ko 2024-09-10T21:10:10.163 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 114801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/elan_i2c.ko 2024-09-10T21:10:10.163 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 16065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/gpio_mouse.ko 2024-09-10T21:10:10.163 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 430033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/psmouse.ko 2024-09-10T21:10:10.163 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 17025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/sermouse.ko 2024-09-10T21:10:10.163 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 27961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-09-10T21:10:10.163 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 27281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/synaptics_usb.ko 2024-09-10T21:10:10.163 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 18897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/vsxxxaa.ko 2024-09-10T21:10:10.163 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/rmi4/ 2024-09-10T21:10:10.163 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 139497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/rmi4/rmi_core.ko 2024-09-10T21:10:10.163 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/ 2024-09-10T21:10:10.163 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/altera_ps2.ko 2024-09-10T21:10:10.163 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 18961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/arc_ps2.ko 2024-09-10T21:10:10.163 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 20481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/ct82c710.ko 2024-09-10T21:10:10.164 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 20377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-09-10T21:10:10.164 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 18481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/parkbd.ko 2024-09-10T21:10:10.164 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 15633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/pcips2.ko 2024-09-10T21:10:10.164 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 23873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/ps2mult.ko 2024-09-10T21:10:10.164 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 29001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/serio_raw.ko 2024-09-10T21:10:10.164 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 21817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/serport.ko 2024-09-10T21:10:10.164 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 15665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/sparse-keymap.ko 2024-09-10T21:10:10.164 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/ 2024-09-10T21:10:10.164 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 15817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-88pm860x.ko 2024-09-10T21:10:10.164 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-adp5520.ko 2024-09-10T21:10:10.164 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 77345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-bd2802.ko 2024-09-10T21:10:10.164 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 31177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-blinkm.ko 2024-09-10T21:10:10.164 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 11985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-da903x.ko 2024-09-10T21:10:10.164 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 14817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-da9052.ko 2024-09-10T21:10:10.165 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 20169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-gpio.ko 2024-09-10T21:10:10.165 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 18801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lm3530.ko 2024-09-10T21:10:10.165 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 36793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lm3533.ko 2024-09-10T21:10:10.165 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 22889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lm355x.ko 2024-09-10T21:10:10.165 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 23921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lm3642.ko 2024-09-10T21:10:10.165 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 18849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lp3944.ko 2024-09-10T21:10:10.165 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 12897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lp8788.ko 2024-09-10T21:10:10.165 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 18129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-max8997.ko 2024-09-10T21:10:10.165 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 14257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-mc13783.ko 2024-09-10T21:10:10.165 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 14705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-menf21bmc.ko 2024-09-10T21:10:10.165 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 26065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-pca9532.ko 2024-09-10T21:10:10.165 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 23897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-pca955x.ko 2024-09-10T21:10:10.165 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 22065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-pca963x.ko 2024-09-10T21:10:10.166 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 38249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-ss4200.ko 2024-09-10T21:10:10.166 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 28337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-tca6507.ko 2024-09-10T21:10:10.166 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 18265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-wm831x-status.ko 2024-09-10T21:10:10.166 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-wm8350.ko 2024-09-10T21:10:10.166 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ 2024-09-10T21:10:10.166 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 12745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-09-10T21:10:10.166 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 9937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-09-10T21:10:10.166 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 8721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-09-10T21:10:10.166 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 15169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-09-10T21:10:10.166 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 16697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-09-10T21:10:10.166 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 16865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-09-10T21:10:10.166 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-09-10T21:10:10.167 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 16809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-09-10T21:10:10.167 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/ 2024-09-10T21:10:10.167 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/bcache/ 2024-09-10T21:10:10.167 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 776777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/bcache/bcache.ko 2024-09-10T21:10:10.167 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 46361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-bio-prison.ko 2024-09-10T21:10:10.167 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 113977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-bufio.ko 2024-09-10T21:10:10.167 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 58393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-cache-smq.ko 2024-09-10T21:10:10.167 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 197121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-cache.ko 2024-09-10T21:10:10.167 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 137841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-crypt.ko 2024-09-10T21:10:10.167 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 28761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-delay.ko 2024-09-10T21:10:10.167 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 67153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-era.ko 2024-09-10T21:10:10.167 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 33121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-flakey.ko 2024-09-10T21:10:10.168 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 50665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-log-userspace.ko 2024-09-10T21:10:10.168 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 40713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-log.ko 2024-09-10T21:10:10.168 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 61073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-mirror.ko 2024-09-10T21:10:10.168 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 96937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-multipath.ko 2024-09-10T21:10:10.168 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 19905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-queue-length.ko 2024-09-10T21:10:10.168 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 86937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-raid.ko 2024-09-10T21:10:10.168 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 42121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-region-hash.ko 2024-09-10T21:10:10.168 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 19089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-round-robin.ko 2024-09-10T21:10:10.168 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 20193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-service-time.ko 2024-09-10T21:10:10.168 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 139257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-snapshot.ko 2024-09-10T21:10:10.168 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 23033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-switch.ko 2024-09-10T21:10:10.168 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 234353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-thin-pool.ko 2024-09-10T21:10:10.168 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 64945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-verity.ko 2024-09-10T21:10:10.169 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 10073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-zero.ko 2024-09-10T21:10:10.169 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/persistent-data/ 2024-09-10T21:10:10.169 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 257025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-09-10T21:10:10.169 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 43417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid0.ko 2024-09-10T21:10:10.169 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 124833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid1.ko 2024-09-10T21:10:10.169 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 157401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid10.ko 2024-09-10T21:10:10.169 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 443385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid456.ko 2024-09-10T21:10:10.169 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/ 2024-09-10T21:10:10.169 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/ 2024-09-10T21:10:10.169 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 211273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptbase.ko 2024-09-10T21:10:10.169 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 78081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptctl.ko 2024-09-10T21:10:10.169 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 54889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptfc.ko 2024-09-10T21:10:10.170 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 44793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptlan.ko 2024-09-10T21:10:10.170 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 150977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptsas.ko 2024-09-10T21:10:10.170 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 94401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptscsih.ko 2024-09-10T21:10:10.170 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 64033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptspi.ko 2024-09-10T21:10:10.170 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/ 2024-09-10T21:10:10.170 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 20065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/88pm800.ko 2024-09-10T21:10:10.170 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 14769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/88pm805.ko 2024-09-10T21:10:10.170 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 12465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/88pm80x.ko 2024-09-10T21:10:10.170 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/arizona-i2c.ko 2024-09-10T21:10:10.170 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 146569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/arizona.ko 2024-09-10T21:10:10.170 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/bcm590xx.ko 2024-09-10T21:10:10.170 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 20961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/cros_ec_dev.ko 2024-09-10T21:10:10.170 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 40657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/dln2.ko 2024-09-10T21:10:10.171 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 18689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/janz-cmodio.ko 2024-09-10T21:10:10.171 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 54473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/kempld-core.ko 2024-09-10T21:10:10.171 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 31409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lm3533-core.ko 2024-09-10T21:10:10.171 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 17057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lm3533-ctrlbank.ko 2024-09-10T21:10:10.171 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 14305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lp3943.ko 2024-09-10T21:10:10.171 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 60577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lpc_ich.ko 2024-09-10T21:10:10.171 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lpc_sch.ko 2024-09-10T21:10:10.171 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 21137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/max8907.ko 2024-09-10T21:10:10.171 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 33817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/mc13xxx-core.ko 2024-09-10T21:10:10.171 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/mc13xxx-i2c.ko 2024-09-10T21:10:10.171 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/menf21bmc.ko 2024-09-10T21:10:10.171 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 20049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/pcf50633-adc.ko 2024-09-10T21:10:10.171 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 10001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/pcf50633-gpio.ko 2024-09-10T21:10:10.172 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 41041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/pcf50633.ko 2024-09-10T21:10:10.172 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 11265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/rdc321x-southbridge.ko 2024-09-10T21:10:10.172 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 22633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/retu-mfd.ko 2024-09-10T21:10:10.172 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 95625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/si476x-core.ko 2024-09-10T21:10:10.172 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 57473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/sm501.ko 2024-09-10T21:10:10.172 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 59801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/tps65010.ko 2024-09-10T21:10:10.172 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 11873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/tps6507x.ko 2024-09-10T21:10:10.172 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 15049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/viperboard.ko 2024-09-10T21:10:10.172 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 11833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/vx855.ko 2024-09-10T21:10:10.172 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 21225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/wl1273-core.ko 2024-09-10T21:10:10.172 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 51201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/wm8994.ko 2024-09-10T21:10:10.172 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ 2024-09-10T21:10:10.172 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 16025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ad525x_dpot-i2c.ko 2024-09-10T21:10:10.173 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 63993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ad525x_dpot.ko 2024-09-10T21:10:10.173 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/altera-stapl/ 2024-09-10T21:10:10.173 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 57393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-09-10T21:10:10.173 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 22289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/apds9802als.ko 2024-09-10T21:10:10.173 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 47785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/apds990x.ko 2024-09-10T21:10:10.173 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 58785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/bh1770glc.ko 2024-09-10T21:10:10.173 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/c2port/ 2024-09-10T21:10:10.173 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 12905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/c2port/c2port-duramar2150.ko 2024-09-10T21:10:10.173 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 47313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/c2port/core.ko 2024-09-10T21:10:10.173 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/cb710/ 2024-09-10T21:10:10.173 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 29225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/cb710/cb710.ko 2024-09-10T21:10:10.173 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 21705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ds1682.ko 2024-09-10T21:10:10.173 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 10233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/dummy-irq.ko 2024-09-10T21:10:10.174 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/echo/ 2024-09-10T21:10:10.174 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 15193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/echo/echo.ko 2024-09-10T21:10:10.174 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/ 2024-09-10T21:10:10.174 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 44177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/at24.ko 2024-09-10T21:10:10.174 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 18145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-09-10T21:10:10.174 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 15705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/max6875.ko 2024-09-10T21:10:10.174 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 40305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/enclosure.ko 2024-09-10T21:10:10.174 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/genwqe/ 2024-09-10T21:10:10.174 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 191761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/genwqe/genwqe_card.ko 2024-09-10T21:10:10.174 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 16625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/hmc6352.ko 2024-09-10T21:10:10.174 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 37857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/hpilo.ko 2024-09-10T21:10:10.174 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ibmasm/ 2024-09-10T21:10:10.174 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 92489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ibmasm/ibmasm.ko 2024-09-10T21:10:10.174 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 24057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ics932s401.ko 2024-09-10T21:10:10.175 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 26833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/isl29003.ko 2024-09-10T21:10:10.175 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 18697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/isl29020.ko 2024-09-10T21:10:10.175 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/ 2024-09-10T21:10:10.175 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 53961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-09-10T21:10:10.175 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 17905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-09-10T21:10:10.175 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/ 2024-09-10T21:10:10.175 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 153777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei-me.ko 2024-09-10T21:10:10.175 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 69897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei-txe.ko 2024-09-10T21:10:10.175 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 381905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei.ko 2024-09-10T21:10:10.175 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 38049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/phantom.ko 2024-09-10T21:10:10.175 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ti-st/ 2024-09-10T21:10:10.175 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 90881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ti-st/st_drv.ko 2024-09-10T21:10:10.175 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 28945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tifm_7xx1.ko 2024-09-10T21:10:10.176 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 33257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tifm_core.ko 2024-09-10T21:10:10.176 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 22329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tsl2550.ko 2024-09-10T21:10:10.176 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 56809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_balloon.ko 2024-09-10T21:10:10.176 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_vmci/ 2024-09-10T21:10:10.176 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 232257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-09-10T21:10:10.176 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ 2024-09-10T21:10:10.176 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/bonding/ 2024-09-10T21:10:10.176 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 623361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/bonding/bonding.ko 2024-09-10T21:10:10.241 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/ 2024-09-10T21:10:10.244 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/b53/ 2024-09-10T21:10:10.244 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 133161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/b53/b53_common.ko 2024-09-10T21:10:10.244 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 49641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/bcm-sf2.ko 2024-09-10T21:10:10.244 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 19001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6060.ko 2024-09-10T21:10:10.244 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6xxx/ 2024-09-10T21:10:10.244 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 592833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-09-10T21:10:10.244 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 19697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dummy.ko 2024-09-10T21:10:10.244 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 21465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/eql.ko 2024-09-10T21:10:10.244 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ 2024-09-10T21:10:10.244 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/ 2024-09-10T21:10:10.244 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 53841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-09-10T21:10:10.244 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 49281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-09-10T21:10:10.245 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 158537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-09-10T21:10:10.245 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 86233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-09-10T21:10:10.245 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/ 2024-09-10T21:10:10.245 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 43097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/8390.ko 2024-09-10T21:10:10.245 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 73953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-09-10T21:10:10.245 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 33457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-09-10T21:10:10.245 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 109673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-09-10T21:10:10.245 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/adaptec/ 2024-09-10T21:10:10.245 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 64817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-09-10T21:10:10.245 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/agere/ 2024-09-10T21:10:10.245 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 84433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/agere/et131x.ko 2024-09-10T21:10:10.245 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/alteon/ 2024-09-10T21:10:10.245 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 70897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-09-10T21:10:10.245 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/altera/ 2024-09-10T21:10:10.246 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 52713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-09-10T21:10:10.246 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/ 2024-09-10T21:10:10.246 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 65945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-09-10T21:10:10.246 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 56145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-09-10T21:10:10.246 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 116105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-09-10T21:10:10.246 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/ 2024-09-10T21:10:10.246 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/alx/ 2024-09-10T21:10:10.246 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 124721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-09-10T21:10:10.246 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1c/ 2024-09-10T21:10:10.246 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 128625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-09-10T21:10:10.246 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1e/ 2024-09-10T21:10:10.246 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 110505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-09-10T21:10:10.246 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/ 2024-09-10T21:10:10.247 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 102017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-09-10T21:10:10.247 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 89121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-09-10T21:10:10.247 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/ 2024-09-10T21:10:10.247 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 106593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-09-10T21:10:10.247 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 229409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-09-10T21:10:10.247 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2x/ 2024-09-10T21:10:10.247 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 1734265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-09-10T21:10:10.247 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 166913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-09-10T21:10:10.247 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/genet/ 2024-09-10T21:10:10.247 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 164665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-09-10T21:10:10.247 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 439633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-09-10T21:10:10.247 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/ 2024-09-10T21:10:10.247 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/bna/ 2024-09-10T21:10:10.247 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 432417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-09-10T21:10:10.248 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/ 2024-09-10T21:10:10.248 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb/ 2024-09-10T21:10:10.248 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 257433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-09-10T21:10:10.248 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb3/ 2024-09-10T21:10:10.248 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 444721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-09-10T21:10:10.248 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4/ 2024-09-10T21:10:10.248 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 1309441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-09-10T21:10:10.248 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4vf/ 2024-09-10T21:10:10.248 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 207073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-09-10T21:10:10.248 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/libcxgb/ 2024-09-10T21:10:10.248 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 39233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-09-10T21:10:10.248 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/ 2024-09-10T21:10:10.248 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/enic/ 2024-09-10T21:10:10.249 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 269777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-09-10T21:10:10.249 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/ 2024-09-10T21:10:10.249 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/ 2024-09-10T21:10:10.249 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 85609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-09-10T21:10:10.249 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 77601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-09-10T21:10:10.249 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 169313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-09-10T21:10:10.249 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 66065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-09-10T21:10:10.249 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 76361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-09-10T21:10:10.249 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 38153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-09-10T21:10:10.249 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/ 2024-09-10T21:10:10.249 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 76161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-09-10T21:10:10.249 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 78929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-09-10T21:10:10.249 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 52401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dnet.ko 2024-09-10T21:10:10.250 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 30489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ec_bhf.ko 2024-09-10T21:10:10.250 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/ 2024-09-10T21:10:10.250 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/benet/ 2024-09-10T21:10:10.250 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 356153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-09-10T21:10:10.250 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 61881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ethoc.ko 2024-09-10T21:10:10.250 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 63345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/fealnx.ko 2024-09-10T21:10:10.250 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/fujitsu/ 2024-09-10T21:10:10.250 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 51657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/fujitsu/fmvj18x_cs.ko 2024-09-10T21:10:10.250 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ 2024-09-10T21:10:10.250 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 115697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e100.ko 2024-09-10T21:10:10.250 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000/ 2024-09-10T21:10:10.250 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 366465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-09-10T21:10:10.251 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000e/ 2024-09-10T21:10:10.251 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 755425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-09-10T21:10:10.251 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/fm10k/ 2024-09-10T21:10:10.251 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 324721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-09-10T21:10:10.251 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/i40e/ 2024-09-10T21:10:10.251 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 1193705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-09-10T21:10:10.251 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/iavf/ 2024-09-10T21:10:10.251 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 356609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-09-10T21:10:10.251 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igb/ 2024-09-10T21:10:10.251 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 672177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-09-10T21:10:10.251 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igbvf/ 2024-09-10T21:10:10.251 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 121609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-09-10T21:10:10.251 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbe/ 2024-09-10T21:10:10.251 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 966193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-09-10T21:10:10.252 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbevf/ 2024-09-10T21:10:10.252 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 193889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-09-10T21:10:10.252 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libeth/ 2024-09-10T21:10:10.252 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 11561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libeth/libeth.ko 2024-09-10T21:10:10.252 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libie/ 2024-09-10T21:10:10.252 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 5969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libie/libie.ko 2024-09-10T21:10:10.252 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 109353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/jme.ko 2024-09-10T21:10:10.252 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/ 2024-09-10T21:10:10.252 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 25785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-09-10T21:10:10.252 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 127057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/skge.ko 2024-09-10T21:10:10.252 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 151049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-09-10T21:10:10.252 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/ 2024-09-10T21:10:10.252 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/ 2024-09-10T21:10:10.253 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 888945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-09-10T21:10:10.253 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 346977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-09-10T21:10:10.253 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/ 2024-09-10T21:10:10.253 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/core/ 2024-09-10T21:10:10.253 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 1408481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-09-10T21:10:10.253 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ 2024-09-10T21:10:10.253 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 66097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ks8842.ko 2024-09-10T21:10:10.253 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 53497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ks8851_common.ko 2024-09-10T21:10:10.253 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 22009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ks8851_par.ko 2024-09-10T21:10:10.253 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 117585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-09-10T21:10:10.253 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/ 2024-09-10T21:10:10.253 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/myri10ge/ 2024-09-10T21:10:10.253 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 131601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-09-10T21:10:10.253 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/ 2024-09-10T21:10:10.254 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 91689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-09-10T21:10:10.254 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 56777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-09-10T21:10:10.254 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/neterion/ 2024-09-10T21:10:10.254 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 181409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-09-10T21:10:10.254 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/nvidia/ 2024-09-10T21:10:10.254 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 171649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-09-10T21:10:10.254 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/ 2024-09-10T21:10:10.254 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 58529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-09-10T21:10:10.254 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 50409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-09-10T21:10:10.254 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/ 2024-09-10T21:10:10.254 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/netxen/ 2024-09-10T21:10:10.254 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 301017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-09-10T21:10:10.254 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 105857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-09-10T21:10:10.255 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qlcnic/ 2024-09-10T21:10:10.255 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 885465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-09-10T21:10:10.255 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rdc/ 2024-09-10T21:10:10.255 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 52481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-09-10T21:10:10.255 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/ 2024-09-10T21:10:10.255 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 81785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-09-10T21:10:10.255 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 100833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-09-10T21:10:10.255 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 34217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/atp.ko 2024-09-10T21:10:10.255 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 264393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-09-10T21:10:10.255 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/samsung/ 2024-09-10T21:10:10.255 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/samsung/sxgbe/ 2024-09-10T21:10:10.255 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 142481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/samsung/sxgbe/samsung-sxgbe.ko 2024-09-10T21:10:10.255 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sfc/ 2024-09-10T21:10:10.255 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 1346017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-09-10T21:10:10.256 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/silan/ 2024-09-10T21:10:10.256 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 58097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-09-10T21:10:10.256 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/ 2024-09-10T21:10:10.256 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 66737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/sis190.ko 2024-09-10T21:10:10.256 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 80233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/sis900.ko 2024-09-10T21:10:10.256 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/ 2024-09-10T21:10:10.256 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 74529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-09-10T21:10:10.256 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 79601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-09-10T21:10:10.256 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 100937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-09-10T21:10:10.256 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 83593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-09-10T21:10:10.256 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/ 2024-09-10T21:10:10.256 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/ 2024-09-10T21:10:10.256 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 14305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-generic.ko 2024-09-10T21:10:10.256 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 47193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-09-10T21:10:10.257 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 20969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac-platform.ko 2024-09-10T21:10:10.257 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 644913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-09-10T21:10:10.257 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/ 2024-09-10T21:10:10.257 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 129089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/cassini.ko 2024-09-10T21:10:10.257 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 208177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/niu.ko 2024-09-10T21:10:10.257 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 82033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/sungem.ko 2024-09-10T21:10:10.257 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 91641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-09-10T21:10:10.257 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/tehuti/ 2024-09-10T21:10:10.257 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 60113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-09-10T21:10:10.257 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ti/ 2024-09-10T21:10:10.257 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 110881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ti/tlan.ko 2024-09-10T21:10:10.257 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/ 2024-09-10T21:10:10.257 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 98993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-09-10T21:10:10.258 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 102745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-09-10T21:10:10.258 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/ 2024-09-10T21:10:10.258 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 61121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-09-10T21:10:10.258 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 40689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-09-10T21:10:10.258 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/xircom/ 2024-09-10T21:10:10.258 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 70057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-09-10T21:10:10.258 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/fddi/ 2024-09-10T21:10:10.258 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 49537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/fddi/defxx.ko 2024-09-10T21:10:10.258 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/fddi/skfp/ 2024-09-10T21:10:10.258 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 275889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/fddi/skfp/skfp.ko 2024-09-10T21:10:10.258 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 91785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/geneve.ko 2024-09-10T21:10:10.258 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/hyperv/ 2024-09-10T21:10:10.258 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 290697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-09-10T21:10:10.259 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ieee802154/ 2024-09-10T21:10:10.259 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 21489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ieee802154/fakelb.ko 2024-09-10T21:10:10.259 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 30153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ifb.ko 2024-09-10T21:10:10.259 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ipvlan/ 2024-09-10T21:10:10.259 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 107849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ipvlan/ipvlan.ko 2024-09-10T21:10:10.259 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 77585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/macvlan.ko 2024-09-10T21:10:10.259 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 19361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/macvtap.ko 2024-09-10T21:10:10.259 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/ 2024-09-10T21:10:10.259 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 26193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-09-10T21:10:10.259 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 17593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-09-10T21:10:10.259 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 16441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-gpio.ko 2024-09-10T21:10:10.259 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 9865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-regmap.ko 2024-09-10T21:10:10.259 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 20073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio.ko 2024-09-10T21:10:10.260 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 25385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mii.ko 2024-09-10T21:10:10.260 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 62857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/netconsole.ko 2024-09-10T21:10:10.260 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/nlmon.ko 2024-09-10T21:10:10.260 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/ 2024-09-10T21:10:10.260 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 20185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/pcs-lynx.ko 2024-09-10T21:10:10.260 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 72969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-09-10T21:10:10.260 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/ 2024-09-10T21:10:10.260 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 12113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/amd.ko 2024-09-10T21:10:10.260 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 61801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-09-10T21:10:10.260 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 52945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm7xxx.ko 2024-09-10T21:10:10.260 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 14369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm87xx.ko 2024-09-10T21:10:10.260 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 58729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/broadcom.ko 2024-09-10T21:10:10.260 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 12369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/cicada.ko 2024-09-10T21:10:10.260 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 15145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/davicom.ko 2024-09-10T21:10:10.261 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 10841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/et1011c.ko 2024-09-10T21:10:10.261 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 31393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/icplus.ko 2024-09-10T21:10:10.261 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 19265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/lxt.ko 2024-09-10T21:10:10.261 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 113681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/marvell.ko 2024-09-10T21:10:10.261 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 108257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/micrel.ko 2024-09-10T21:10:10.261 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 14137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/national.ko 2024-09-10T21:10:10.261 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 118505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/phylink.ko 2024-09-10T21:10:10.261 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 11889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/qsemi.ko 2024-09-10T21:10:10.261 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 76177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/realtek.ko 2024-09-10T21:10:10.261 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 34737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/smsc.ko 2024-09-10T21:10:10.261 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 12761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/ste10Xp.ko 2024-09-10T21:10:10.261 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 28857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/vitesse.ko 2024-09-10T21:10:10.261 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/plip/ 2024-09-10T21:10:10.262 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 55377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/plip/plip.ko 2024-09-10T21:10:10.262 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ 2024-09-10T21:10:10.262 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 19457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/bsd_comp.ko 2024-09-10T21:10:10.262 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 36393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_async.ko 2024-09-10T21:10:10.262 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 21625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_deflate.ko 2024-09-10T21:10:10.262 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 23201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_mppe.ko 2024-09-10T21:10:10.262 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 35577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_synctty.ko 2024-09-10T21:10:10.262 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 53409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pppoe.ko 2024-09-10T21:10:10.262 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 15313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pppox.ko 2024-09-10T21:10:10.262 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 39945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pptp.ko 2024-09-10T21:10:10.262 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip/ 2024-09-10T21:10:10.262 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 52177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip/slip.ko 2024-09-10T21:10:10.262 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 42257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/sungem_phy.ko 2024-09-10T21:10:10.263 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 79233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/tap.ko 2024-09-10T21:10:10.263 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/ 2024-09-10T21:10:10.263 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 136729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team.ko 2024-09-10T21:10:10.263 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 16113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_activebackup.ko 2024-09-10T21:10:10.263 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 11489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_broadcast.ko 2024-09-10T21:10:10.263 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 36321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-09-10T21:10:10.263 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 10929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_random.ko 2024-09-10T21:10:10.263 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 10553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-09-10T21:10:10.263 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 108409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/veth.ko 2024-09-10T21:10:10.263 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vmxnet3/ 2024-09-10T21:10:10.263 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 179329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-09-10T21:10:10.263 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vxlan/ 2024-09-10T21:10:10.264 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 384193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vxlan/vxlan.ko 2024-09-10T21:10:10.264 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netback/ 2024-09-10T21:10:10.264 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 175857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netback/xen-netback.ko 2024-09-10T21:10:10.264 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ntb/ 2024-09-10T21:10:10.264 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 24137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ntb/ntb.ko 2024-09-10T21:10:10.264 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/ 2024-09-10T21:10:10.264 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/ 2024-09-10T21:10:10.264 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 337697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/nvme-core.ko 2024-09-10T21:10:10.264 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 128353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/nvme.ko 2024-09-10T21:10:10.264 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/ 2024-09-10T21:10:10.264 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 164153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport.ko 2024-09-10T21:10:10.264 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 16281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_cs.ko 2024-09-10T21:10:10.264 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 103657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_pc.ko 2024-09-10T21:10:10.265 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 30401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_serial.ko 2024-09-10T21:10:10.265 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/ 2024-09-10T21:10:10.265 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug/ 2024-09-10T21:10:10.265 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 28593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-09-10T21:10:10.265 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 20289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug/cpcihp_generic.ko 2024-09-10T21:10:10.265 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 23969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug/cpcihp_zt5550.ko 2024-09-10T21:10:10.265 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/pci-stub.ko 2024-09-10T21:10:10.265 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 55425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/xen-pcifront.ko 2024-09-10T21:10:10.265 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/ 2024-09-10T21:10:10.265 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 31665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/i82092.ko 2024-09-10T21:10:10.265 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 177817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/pcmcia.ko 2024-09-10T21:10:10.265 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 76785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/pcmcia_core.ko 2024-09-10T21:10:10.265 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 41777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/pcmcia_rsrc.ko 2024-09-10T21:10:10.265 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 40553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/pd6729.ko 2024-09-10T21:10:10.266 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 133969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/yenta_socket.ko 2024-09-10T21:10:10.266 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/phy/ 2024-09-10T21:10:10.266 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/phy/broadcom/ 2024-09-10T21:10:10.266 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 12513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/phy/broadcom/phy-bcm-kona-usb2.ko 2024-09-10T21:10:10.266 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/phy/samsung/ 2024-09-10T21:10:10.266 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 9561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/phy/samsung/phy-exynos-usb2.ko 2024-09-10T21:10:10.266 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/ 2024-09-10T21:10:10.266 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/intel/ 2024-09-10T21:10:10.266 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 78281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-09-10T21:10:10.266 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/ 2024-09-10T21:10:10.266 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/ 2024-09-10T21:10:10.266 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 60889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros-ec-sensorhub.ko 2024-09-10T21:10:10.266 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 28953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros_ec.ko 2024-09-10T21:10:10.267 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 25825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros_ec_chardev.ko 2024-09-10T21:10:10.267 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 31481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros_ec_debugfs.ko 2024-09-10T21:10:10.267 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 32113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros_ec_lightbar.ko 2024-09-10T21:10:10.267 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 22009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros_ec_sysfs.ko 2024-09-10T21:10:10.267 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 20121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros_usbpd_notify.ko 2024-09-10T21:10:10.267 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/ 2024-09-10T21:10:10.267 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 88017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/acer-wmi.ko 2024-09-10T21:10:10.267 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 44233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/acerhdf.ko 2024-09-10T21:10:10.267 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 48353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/apple-gmux.ko 2024-09-10T21:10:10.267 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 74033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-laptop.ko 2024-09-10T21:10:10.267 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 33401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-09-10T21:10:10.267 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 172313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-wmi.ko 2024-09-10T21:10:10.267 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 46505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/classmate-laptop.ko 2024-09-10T21:10:10.267 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 62105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-09-10T21:10:10.268 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 15953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-09-10T21:10:10.268 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 52665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-09-10T21:10:10.268 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 25929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-09-10T21:10:10.268 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 42913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/hdaps.ko 2024-09-10T21:10:10.268 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 23105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/ibm_rtl.ko 2024-09-10T21:10:10.268 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel/ 2024-09-10T21:10:10.268 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 14401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-09-10T21:10:10.268 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 10233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel/intel-smartconnect.ko 2024-09-10T21:10:10.268 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 53801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel_ips.ko 2024-09-10T21:10:10.268 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 23001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/msi-wmi.ko 2024-09-10T21:10:10.268 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 11657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/mxm-wmi.ko 2024-09-10T21:10:10.268 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 43793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-09-10T21:10:10.268 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 51857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/samsung-laptop.ko 2024-09-10T21:10:10.269 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 16033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/samsung-q10.ko 2024-09-10T21:10:10.269 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 20289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/topstar-laptop.ko 2024-09-10T21:10:10.269 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 15625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-09-10T21:10:10.269 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 20369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/toshiba_haps.ko 2024-09-10T21:10:10.269 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 11737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/wmi-bmof.ko 2024-09-10T21:10:10.269 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 59729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/wmi.ko 2024-09-10T21:10:10.269 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/ 2024-09-10T21:10:10.269 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/ 2024-09-10T21:10:10.269 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 36665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/88pm860x_battery.ko 2024-09-10T21:10:10.269 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 40449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/88pm860x_charger.ko 2024-09-10T21:10:10.269 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 68921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/bq2415x_charger.ko 2024-09-10T21:10:10.269 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 73161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/bq24190_charger.ko 2024-09-10T21:10:10.269 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 26041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/bq24735-charger.ko 2024-09-10T21:10:10.270 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 20745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/cros_charge-control.ko 2024-09-10T21:10:10.270 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 20521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/cros_peripheral_charger.ko 2024-09-10T21:10:10.270 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 23321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/da9030_battery.ko 2024-09-10T21:10:10.270 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 23377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/da9052-battery.ko 2024-09-10T21:10:10.270 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 34569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/ds2760_battery.ko 2024-09-10T21:10:10.270 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 28729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/ds2780_battery.ko 2024-09-10T21:10:10.270 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 28889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/ds2781_battery.ko 2024-09-10T21:10:10.270 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 28745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/ds2782_battery.ko 2024-09-10T21:10:10.270 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 24297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/gpio-charger.ko 2024-09-10T21:10:10.270 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 23305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/isp1704_charger.ko 2024-09-10T21:10:10.270 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 24201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/lp8727_charger.ko 2024-09-10T21:10:10.270 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 14089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/max14577_charger.ko 2024-09-10T21:10:10.270 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 37929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/max17042_battery.ko 2024-09-10T21:10:10.271 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 22105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/max8903_charger.ko 2024-09-10T21:10:10.271 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 27769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/max8925_power.ko 2024-09-10T21:10:10.271 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 25433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/pcf50633-charger.ko 2024-09-10T21:10:10.271 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 44929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/sbs-battery.ko 2024-09-10T21:10:10.271 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 35993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/test_power.ko 2024-09-10T21:10:10.271 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 21681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/tps65090-charger.ko 2024-09-10T21:10:10.271 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/wm831x_backup.ko 2024-09-10T21:10:10.271 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 29161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/wm831x_power.ko 2024-09-10T21:10:10.271 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 27529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/wm8350_power.ko 2024-09-10T21:10:10.271 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/ 2024-09-10T21:10:10.271 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 98737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/intel_rapl_common.ko 2024-09-10T21:10:10.271 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 20521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/intel_rapl_msr.ko 2024-09-10T21:10:10.271 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/ 2024-09-10T21:10:10.272 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/ 2024-09-10T21:10:10.272 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 17921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps-gpio.ko 2024-09-10T21:10:10.272 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 15617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps-ldisc.ko 2024-09-10T21:10:10.272 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 19017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps_parport.ko 2024-09-10T21:10:10.272 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 49385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/pps_core.ko 2024-09-10T21:10:10.272 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ 2024-09-10T21:10:10.272 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 90617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ptp.ko 2024-09-10T21:10:10.272 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 22033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ptp_kvm.ko 2024-09-10T21:10:10.272 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/ 2024-09-10T21:10:10.272 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/ 2024-09-10T21:10:10.272 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/atl/ 2024-09-10T21:10:10.272 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 89769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-09-10T21:10:10.272 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 47369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/fmpm.ko 2024-09-10T21:10:10.273 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/ 2024-09-10T21:10:10.273 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 27681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-88pm80x.ko 2024-09-10T21:10:10.273 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 29057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-88pm860x.ko 2024-09-10T21:10:10.273 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 18545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-bq32k.ko 2024-09-10T21:10:10.273 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 24809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-da9052.ko 2024-09-10T21:10:10.273 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 22657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-da9055.ko 2024-09-10T21:10:10.273 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 24425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-da9063.ko 2024-09-10T21:10:10.273 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 19817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1286.ko 2024-09-10T21:10:10.273 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 82585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1307.ko 2024-09-10T21:10:10.273 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 24569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1374.ko 2024-09-10T21:10:10.273 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 22073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1511.ko 2024-09-10T21:10:10.273 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 20369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1553.ko 2024-09-10T21:10:10.274 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 16137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1672.ko 2024-09-10T21:10:10.274 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 15753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1742.ko 2024-09-10T21:10:10.274 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 18113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds2404.ko 2024-09-10T21:10:10.274 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 29097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds3232.ko 2024-09-10T21:10:10.274 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-em3027.ko 2024-09-10T21:10:10.274 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 28753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-fm3130.ko 2024-09-10T21:10:10.274 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 24185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-isl12022.ko 2024-09-10T21:10:10.274 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 44369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-isl1208.ko 2024-09-10T21:10:10.274 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 17233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-lp8788.ko 2024-09-10T21:10:10.274 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 47001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m41t80.ko 2024-09-10T21:10:10.274 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 14049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t35.ko 2024-09-10T21:10:10.274 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 34057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t59.ko 2024-09-10T21:10:10.274 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 17425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t86.ko 2024-09-10T21:10:10.275 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 14073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max6900.ko 2024-09-10T21:10:10.275 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 18009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max8907.ko 2024-09-10T21:10:10.275 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 18601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max8925.ko 2024-09-10T21:10:10.275 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 31425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max8997.ko 2024-09-10T21:10:10.275 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 20409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max8998.ko 2024-09-10T21:10:10.275 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 22489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-mc13xxx.ko 2024-09-10T21:10:10.275 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-msm6242.ko 2024-09-10T21:10:10.275 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 24417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-palmas.ko 2024-09-10T21:10:10.275 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 56873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf2127.ko 2024-09-10T21:10:10.275 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 21689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf50633.ko 2024-09-10T21:10:10.275 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 32609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf85063.ko 2024-09-10T21:10:10.275 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 28313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8523.ko 2024-09-10T21:10:10.275 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 36641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8563.ko 2024-09-10T21:10:10.275 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 14993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8583.ko 2024-09-10T21:10:10.276 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 22121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rc5t583.ko 2024-09-10T21:10:10.276 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 15289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rp5c01.ko 2024-09-10T21:10:10.276 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 40169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rs5c372.ko 2024-09-10T21:10:10.276 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 34617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-09-10T21:10:10.276 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 30065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rx8025.ko 2024-09-10T21:10:10.276 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 22889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rx8581.ko 2024-09-10T21:10:10.276 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 26265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-s35390a.ko 2024-09-10T21:10:10.276 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 20513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-09-10T21:10:10.276 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 21785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-tps6586x.ko 2024-09-10T21:10:10.276 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 26193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-tps65910.ko 2024-09-10T21:10:10.276 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 24889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-wm831x.ko 2024-09-10T21:10:10.276 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 25889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-wm8350.ko 2024-09-10T21:10:10.277 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 33489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-x1205.ko 2024-09-10T21:10:10.277 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ 2024-09-10T21:10:10.277 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 90537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-9xxx.ko 2024-09-10T21:10:10.277 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 69689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-sas.ko 2024-09-10T21:10:10.277 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 66033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-xxxx.ko 2024-09-10T21:10:10.277 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 164753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/BusLogic.ko 2024-09-10T21:10:10.277 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 31705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/a100u2w.ko 2024-09-10T21:10:10.277 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aacraid/ 2024-09-10T21:10:10.277 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 341105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aacraid/aacraid.ko 2024-09-10T21:10:10.277 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 120833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/advansys.ko 2024-09-10T21:10:10.277 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/ 2024-09-10T21:10:10.277 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 468305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-09-10T21:10:10.277 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 380857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-09-10T21:10:10.278 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic94xx/ 2024-09-10T21:10:10.278 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 244049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic94xx/aic94xx.ko 2024-09-10T21:10:10.278 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 28937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/am53c974.ko 2024-09-10T21:10:10.278 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/arcmsr/ 2024-09-10T21:10:10.278 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 135801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-09-10T21:10:10.278 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 44489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/atp870u.ko 2024-09-10T21:10:10.278 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/be2iscsi/ 2024-09-10T21:10:10.278 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 305161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-09-10T21:10:10.278 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bfa/ 2024-09-10T21:10:10.278 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 1526289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bfa/bfa.ko 2024-09-10T21:10:10.278 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2fc/ 2024-09-10T21:10:10.278 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 298321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-09-10T21:10:10.278 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2i/ 2024-09-10T21:10:10.279 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 144553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-09-10T21:10:10.279 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 47673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ch.ko 2024-09-10T21:10:10.279 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/csiostor/ 2024-09-10T21:10:10.279 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 390817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/csiostor/csiostor.ko 2024-09-10T21:10:10.279 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/ 2024-09-10T21:10:10.279 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb3i/ 2024-09-10T21:10:10.279 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 89041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-09-10T21:10:10.279 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb4i/ 2024-09-10T21:10:10.279 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 117561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-09-10T21:10:10.279 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 168361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-09-10T21:10:10.279 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 91777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/dc395x.ko 2024-09-10T21:10:10.279 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 45489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/dmx3191d.ko 2024-09-10T21:10:10.279 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esas2r/ 2024-09-10T21:10:10.279 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 267737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esas2r/esas2r.ko 2024-09-10T21:10:10.280 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 78441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esp_scsi.ko 2024-09-10T21:10:10.280 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/ 2024-09-10T21:10:10.280 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 95553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/fcoe.ko 2024-09-10T21:10:10.280 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 191041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-09-10T21:10:10.280 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 22289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fdomain.ko 2024-09-10T21:10:10.280 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fnic/ 2024-09-10T21:10:10.280 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 366737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fnic/fnic.ko 2024-09-10T21:10:10.280 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 251641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hpsa.ko 2024-09-10T21:10:10.280 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 65529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hptiop.ko 2024-09-10T21:10:10.280 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 57201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hv_storvsc.ko 2024-09-10T21:10:10.280 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 44393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/imm.ko 2024-09-10T21:10:10.280 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 55241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/initio.ko 2024-09-10T21:10:10.280 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 327841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ipr.ko 2024-09-10T21:10:10.281 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 129881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ips.ko 2024-09-10T21:10:10.281 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/isci/ 2024-09-10T21:10:10.281 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 398505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/isci/isci.ko 2024-09-10T21:10:10.281 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 31201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-09-10T21:10:10.281 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 62785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/iscsi_tcp.ko 2024-09-10T21:10:10.281 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libfc/ 2024-09-10T21:10:10.281 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 380649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libfc/libfc.ko 2024-09-10T21:10:10.281 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 172833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libiscsi.ko 2024-09-10T21:10:10.281 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 63097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libiscsi_tcp.ko 2024-09-10T21:10:10.281 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libsas/ 2024-09-10T21:10:10.281 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 274913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libsas/libsas.ko 2024-09-10T21:10:10.281 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/lpfc/ 2024-09-10T21:10:10.281 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 2737441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/lpfc/lpfc.ko 2024-09-10T21:10:10.282 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/ 2024-09-10T21:10:10.282 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 88625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-09-10T21:10:10.282 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 39769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-09-10T21:10:10.282 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 388737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-09-10T21:10:10.282 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 105177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid.ko 2024-09-10T21:10:10.282 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mpt3sas/ 2024-09-10T21:10:10.282 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 833969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-09-10T21:10:10.282 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvsas/ 2024-09-10T21:10:10.282 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 158857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvsas/mvsas.ko 2024-09-10T21:10:10.282 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 96697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvumi.ko 2024-09-10T21:10:10.282 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pcmcia/ 2024-09-10T21:10:10.282 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 102113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pcmcia/aha152x_cs.ko 2024-09-10T21:10:10.282 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pcmcia/fdomain_cs.ko 2024-09-10T21:10:10.282 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 23241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pcmcia/qlogic_cs.ko 2024-09-10T21:10:10.283 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 28417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pcmcia/sym53c500_cs.ko 2024-09-10T21:10:10.283 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pm8001/ 2024-09-10T21:10:10.283 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 444273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-09-10T21:10:10.283 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 152097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pmcraid.ko 2024-09-10T21:10:10.283 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 41265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ppa.ko 2024-09-10T21:10:10.283 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 67129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla1280.ko 2024-09-10T21:10:10.283 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/ 2024-09-10T21:10:10.283 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 2323537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-09-10T21:10:10.283 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 103793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-09-10T21:10:10.283 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla4xxx/ 2024-09-10T21:10:10.283 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 697393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-09-10T21:10:10.283 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 22745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qlogicfas408.ko 2024-09-10T21:10:10.283 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 18705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/raid_class.ko 2024-09-10T21:10:10.284 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 331577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_debug.ko 2024-09-10T21:10:10.284 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 219497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_fc.ko 2024-09-10T21:10:10.284 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 349593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-09-10T21:10:10.284 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 121777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_sas.ko 2024-09-10T21:10:10.284 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 92385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_spi.ko 2024-09-10T21:10:10.284 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 54097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_srp.ko 2024-09-10T21:10:10.284 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 33113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ses.ko 2024-09-10T21:10:10.284 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 138265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/st.ko 2024-09-10T21:10:10.284 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 57489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/stex.ko 2024-09-10T21:10:10.284 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/sym53c8xx_2/ 2024-09-10T21:10:10.284 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 205137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-09-10T21:10:10.284 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 55009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/virtio_scsi.ko 2024-09-10T21:10:10.284 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 59289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/vmw_pvscsi.ko 2024-09-10T21:10:10.285 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 48865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/wd719x.ko 2024-09-10T21:10:10.285 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 59521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/xen-scsifront.ko 2024-09-10T21:10:10.285 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ssb/ 2024-09-10T21:10:10.285 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 172697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ssb/ssb.ko 2024-09-10T21:10:10.285 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/ 2024-09-10T21:10:10.285 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/iscsi/ 2024-09-10T21:10:10.285 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 915121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-09-10T21:10:10.285 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/loopback/ 2024-09-10T21:10:10.285 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 60497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/loopback/tcm_loop.ko 2024-09-10T21:10:10.285 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 48281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_file.ko 2024-09-10T21:10:10.285 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 53881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_iblock.ko 2024-09-10T21:10:10.285 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 1161521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_mod.ko 2024-09-10T21:10:10.285 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 51985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_pscsi.ko 2024-09-10T21:10:10.286 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 149801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_user.ko 2024-09-10T21:10:10.286 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/tcm_fc/ 2024-09-10T21:10:10.286 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 90217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-09-10T21:10:10.286 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/ 2024-09-10T21:10:10.286 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/ 2024-09-10T21:10:10.286 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/ 2024-09-10T21:10:10.286 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 27681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-09-10T21:10:10.286 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 34961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-09-10T21:10:10.286 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 11409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-09-10T21:10:10.286 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 11785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-09-10T21:10:10.286 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 17513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-09-10T21:10:10.286 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 18673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-09-10T21:10:10.286 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 36033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-09-10T21:10:10.287 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 28457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-09-10T21:10:10.287 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 16833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-09-10T21:10:10.287 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 12553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-09-10T21:10:10.287 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 11633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-09-10T21:10:10.287 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 15049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-09-10T21:10:10.287 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 46289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-09-10T21:10:10.287 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 17233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-09-10T21:10:10.287 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-09-10T21:10:10.287 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 36001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_powerclamp.ko 2024-09-10T21:10:10.287 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 21577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-09-10T21:10:10.287 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 12937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-09-10T21:10:10.287 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 30721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-09-10T21:10:10.287 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ 2024-09-10T21:10:10.288 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ipwireless/ 2024-09-10T21:10:10.288 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 99417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-09-10T21:10:10.288 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 69825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/moxa.ko 2024-09-10T21:10:10.288 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 61737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/mxser.ko 2024-09-10T21:10:10.288 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 134633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/n_gsm.ko 2024-09-10T21:10:10.288 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 37993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/n_hdlc.ko 2024-09-10T21:10:10.288 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 79057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/nozomi.ko 2024-09-10T21:10:10.288 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/ 2024-09-10T21:10:10.288 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250/ 2024-09-10T21:10:10.288 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 43209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250/8250_dw.ko 2024-09-10T21:10:10.288 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 65273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-09-10T21:10:10.288 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 23161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/altera_jtaguart.ko 2024-09-10T21:10:10.289 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 28137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/altera_uart.ko 2024-09-10T21:10:10.289 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 10761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/arc_uart.ko 2024-09-10T21:10:10.289 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 90697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/fsl_lpuart.ko 2024-09-10T21:10:10.289 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/jsm/ 2024-09-10T21:10:10.289 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 121249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/jsm/jsm.ko 2024-09-10T21:10:10.289 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 35073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/rp2.ko 2024-09-10T21:10:10.289 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 68009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/sc16is7xx.ko 2024-09-10T21:10:10.289 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/sc16is7xx_i2c.ko 2024-09-10T21:10:10.289 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/ 2024-09-10T21:10:10.289 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/core/ 2024-09-10T21:10:10.289 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 689289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/core/ufshcd-core.ko 2024-09-10T21:10:10.289 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host/ 2024-09-10T21:10:10.289 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 36673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-09-10T21:10:10.290 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 16217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host/ufshcd-pltfrm.ko 2024-09-10T21:10:10.290 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/ 2024-09-10T21:10:10.290 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 55041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio.ko 2024-09-10T21:10:10.290 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 14505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_aec.ko 2024-09-10T21:10:10.290 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_cif.ko 2024-09-10T21:10:10.290 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 21001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_dmem_genirq.ko 2024-09-10T21:10:10.290 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 15833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_mf624.ko 2024-09-10T21:10:10.290 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 14113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_netx.ko 2024-09-10T21:10:10.290 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 12129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_pci_generic.ko 2024-09-10T21:10:10.290 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 19273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_pdrv_genirq.ko 2024-09-10T21:10:10.290 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 16105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_sercos3.ko 2024-09-10T21:10:10.290 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/ 2024-09-10T21:10:10.290 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/c67x00/ 2024-09-10T21:10:10.290 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 90417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/c67x00/c67x00.ko 2024-09-10T21:10:10.291 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/chipidea/ 2024-09-10T21:10:10.291 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 187689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/chipidea/ci_hdrc.ko 2024-09-10T21:10:10.291 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 21729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/chipidea/ci_hdrc_msm.ko 2024-09-10T21:10:10.291 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 15137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/chipidea/ci_hdrc_npcm.ko 2024-09-10T21:10:10.291 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 14193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/chipidea/ci_hdrc_pci.ko 2024-09-10T21:10:10.291 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 14857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/chipidea/ci_hdrc_usb2.ko 2024-09-10T21:10:10.291 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/ 2024-09-10T21:10:10.291 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 101657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/cdc-acm.ko 2024-09-10T21:10:10.291 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 63545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/cdc-wdm.ko 2024-09-10T21:10:10.291 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 56841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/usblp.ko 2024-09-10T21:10:10.291 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 107633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/usbtmc.ko 2024-09-10T21:10:10.291 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/common/ 2024-09-10T21:10:10.291 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 28737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/common/ulpi.ko 2024-09-10T21:10:10.292 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/dwc2/ 2024-09-10T21:10:10.292 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/dwc2/dwc2_pci.ko 2024-09-10T21:10:10.292 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/dwc3/ 2024-09-10T21:10:10.292 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/dwc3/dwc3-haps.ko 2024-09-10T21:10:10.292 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 34537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/dwc3/dwc3-pci.ko 2024-09-10T21:10:10.292 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 576329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/dwc3/dwc3.ko 2024-09-10T21:10:10.292 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/ 2024-09-10T21:10:10.292 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/ 2024-09-10T21:10:10.292 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 70081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/u_ether.ko 2024-09-10T21:10:10.292 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 61153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/u_serial.ko 2024-09-10T21:10:10.292 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 33937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_acm.ko 2024-09-10T21:10:10.292 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 49945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_ecm.ko 2024-09-10T21:10:10.292 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 33249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_ecm_subset.ko 2024-09-10T21:10:10.293 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 43705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_eem.ko 2024-09-10T21:10:10.293 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 132897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_fs.ko 2024-09-10T21:10:10.293 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 62473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_hid.ko 2024-09-10T21:10:10.293 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 142801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_mass_storage.ko 2024-09-10T21:10:10.293 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 70785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_ncm.ko 2024-09-10T21:10:10.293 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 25913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_obex.ko 2024-09-10T21:10:10.293 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 37801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_phonet.ko 2024-09-10T21:10:10.293 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 70241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_printer.ko 2024-09-10T21:10:10.293 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 99913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_rndis.ko 2024-09-10T21:10:10.293 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 22633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_serial.ko 2024-09-10T21:10:10.293 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 71577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_ss_lb.ko 2024-09-10T21:10:10.293 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 85089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_tcm.ko 2024-09-10T21:10:10.293 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/ 2024-09-10T21:10:10.294 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 23529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_acm_ms.ko 2024-09-10T21:10:10.294 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 20369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_cdc.ko 2024-09-10T21:10:10.294 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 21393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_dbgp.ko 2024-09-10T21:10:10.294 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 24585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_ether.ko 2024-09-10T21:10:10.294 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 33129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_ffs.ko 2024-09-10T21:10:10.294 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 21697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_hid.ko 2024-09-10T21:10:10.294 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 22473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_mass_storage.ko 2024-09-10T21:10:10.294 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 19001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_ncm.ko 2024-09-10T21:10:10.294 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 35777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_nokia.ko 2024-09-10T21:10:10.294 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 18609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_printer.ko 2024-09-10T21:10:10.294 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 21785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_serial.ko 2024-09-10T21:10:10.294 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 30481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_zero.ko 2024-09-10T21:10:10.294 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 78185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/gadgetfs.ko 2024-09-10T21:10:10.295 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 17593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/tcm_usb_gadget.ko 2024-09-10T21:10:10.295 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 200137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/libcomposite.ko 2024-09-10T21:10:10.295 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/ 2024-09-10T21:10:10.295 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 19665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/amd5536udc_pci.ko 2024-09-10T21:10:10.295 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/bdc/ 2024-09-10T21:10:10.295 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 182665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/bdc/bdc.ko 2024-09-10T21:10:10.295 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 45633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/goku_udc.ko 2024-09-10T21:10:10.295 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 66545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/gr_udc.ko 2024-09-10T21:10:10.295 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 85393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/mv_u3d_core.ko 2024-09-10T21:10:10.295 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 69409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/mv_udc.ko 2024-09-10T21:10:10.295 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 90409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/net2272.ko 2024-09-10T21:10:10.295 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 90457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/net2280.ko 2024-09-10T21:10:10.295 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 94929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/pch_udc.ko 2024-09-10T21:10:10.296 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 80617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/pxa27x_udc.ko 2024-09-10T21:10:10.296 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 87561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/r8a66597-udc.ko 2024-09-10T21:10:10.296 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 94385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/snps_udc_core.ko 2024-09-10T21:10:10.296 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 191281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/udc-core.ko 2024-09-10T21:10:10.296 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/ 2024-09-10T21:10:10.296 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 14033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/bcma-hcd.ko 2024-09-10T21:10:10.296 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 108417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/isp116x-hcd.ko 2024-09-10T21:10:10.296 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 104817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/oxu210hp-hcd.ko 2024-09-10T21:10:10.296 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 102361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/r8a66597-hcd.ko 2024-09-10T21:10:10.296 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 59905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/sl811-hcd.ko 2024-09-10T21:10:10.296 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 17593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/sl811_cs.ko 2024-09-10T21:10:10.296 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 16465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/ssb-hcd.ko 2024-09-10T21:10:10.296 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 32161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/xhci-plat-hcd.ko 2024-09-10T21:10:10.296 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/ 2024-09-10T21:10:10.297 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 40633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/mdc800.ko 2024-09-10T21:10:10.297 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 24761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/microtek.ko 2024-09-10T21:10:10.297 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ 2024-09-10T21:10:10.297 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 47593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/adutux.ko 2024-09-10T21:10:10.297 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 23985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/appledisplay.ko 2024-09-10T21:10:10.297 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 20345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/cypress_cy7c63.ko 2024-09-10T21:10:10.297 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 26769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/cytherm.ko 2024-09-10T21:10:10.297 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 15641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ehset.ko 2024-09-10T21:10:10.297 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 17977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/emi26.ko 2024-09-10T21:10:10.297 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 19041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/emi62.ko 2024-09-10T21:10:10.297 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 12857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ezusb.ko 2024-09-10T21:10:10.297 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 22433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/idmouse.ko 2024-09-10T21:10:10.297 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 42889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/iowarrior.ko 2024-09-10T21:10:10.297 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/isight_firmware.ko 2024-09-10T21:10:10.298 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 42345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ldusb.ko 2024-09-10T21:10:10.298 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 42337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/legousbtower.ko 2024-09-10T21:10:10.298 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 33737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/lvstest.ko 2024-09-10T21:10:10.298 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/sisusbvga/ 2024-09-10T21:10:10.298 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 85417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-09-10T21:10:10.298 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 14689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/trancevibrator.ko 2024-09-10T21:10:10.298 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 24609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usb3503.ko 2024-09-10T21:10:10.298 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 29729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usblcd.ko 2024-09-10T21:10:10.298 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 27897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usbsevseg.ko 2024-09-10T21:10:10.298 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 84369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usbtest.ko 2024-09-10T21:10:10.298 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 52377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/uss720.ko 2024-09-10T21:10:10.298 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 32561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/yurex.ko 2024-09-10T21:10:10.298 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/mon/ 2024-09-10T21:10:10.299 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 91561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/mon/usbmon.ko 2024-09-10T21:10:10.299 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/musb/ 2024-09-10T21:10:10.299 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 391209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/musb/musb_hdrc.ko 2024-09-10T21:10:10.299 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/ 2024-09-10T21:10:10.299 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 24993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/phy-generic.ko 2024-09-10T21:10:10.299 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 23713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/phy-gpio-vbus-usb.ko 2024-09-10T21:10:10.299 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 14273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/phy-isp1301.ko 2024-09-10T21:10:10.299 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 26449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/phy-tahvo.ko 2024-09-10T21:10:10.299 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ 2024-09-10T21:10:10.299 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 14841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/aircable.ko 2024-09-10T21:10:10.299 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 38305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ark3116.ko 2024-09-10T21:10:10.299 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 30769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/belkin_sa.ko 2024-09-10T21:10:10.299 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 44537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ch341.ko 2024-09-10T21:10:10.300 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 79769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cp210x.ko 2024-09-10T21:10:10.300 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 35233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cyberjack.ko 2024-09-10T21:10:10.300 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 66961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cypress_m8.ko 2024-09-10T21:10:10.300 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 58425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/digi_acceleport.ko 2024-09-10T21:10:10.300 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 10705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/empeg.ko 2024-09-10T21:10:10.300 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 53161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/f81232.ko 2024-09-10T21:10:10.300 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 188729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ftdi_sio.ko 2024-09-10T21:10:10.300 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 61601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/garmin_gps.ko 2024-09-10T21:10:10.300 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 152409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/io_edgeport.ko 2024-09-10T21:10:10.300 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 142937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/io_ti.ko 2024-09-10T21:10:10.300 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 69297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ipaq.ko 2024-09-10T21:10:10.300 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 21529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ipw.ko 2024-09-10T21:10:10.301 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 29857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ir-usb.ko 2024-09-10T21:10:10.301 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 66449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-09-10T21:10:10.301 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 122425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/keyspan.ko 2024-09-10T21:10:10.301 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 41393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/keyspan_pda.ko 2024-09-10T21:10:10.301 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 29561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/kl5kusb105.ko 2024-09-10T21:10:10.301 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 38145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/kobil_sct.ko 2024-09-10T21:10:10.301 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 38137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mct_u232.ko 2024-09-10T21:10:10.301 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 25657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/metro-usb.ko 2024-09-10T21:10:10.301 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 79801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mos7720.ko 2024-09-10T21:10:10.301 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 91625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mos7840.ko 2024-09-10T21:10:10.301 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 65281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mxuport.ko 2024-09-10T21:10:10.301 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 16057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/navman.ko 2024-09-10T21:10:10.301 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/omninet.ko 2024-09-10T21:10:10.302 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 28985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/opticon.ko 2024-09-10T21:10:10.302 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 201257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/option.ko 2024-09-10T21:10:10.302 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 45433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/oti6858.ko 2024-09-10T21:10:10.302 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 61857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/pl2303.ko 2024-09-10T21:10:10.302 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 10521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/qcaux.ko 2024-09-10T21:10:10.302 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 40081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/qcserial.ko 2024-09-10T21:10:10.302 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 40625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/quatech2.ko 2024-09-10T21:10:10.302 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 17073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/safe_serial.ko 2024-09-10T21:10:10.302 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 59449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/sierra.ko 2024-09-10T21:10:10.302 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 25321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/spcp8x5.ko 2024-09-10T21:10:10.302 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 34161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ssu100.ko 2024-09-10T21:10:10.302 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 19809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/symbolserial.ko 2024-09-10T21:10:10.302 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 70601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-09-10T21:10:10.303 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 24433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-09-10T21:10:10.303 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 12049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb_debug.ko 2024-09-10T21:10:10.303 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 38169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb_wwan.ko 2024-09-10T21:10:10.303 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 148729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usbserial.ko 2024-09-10T21:10:10.303 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 32601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/visor.ko 2024-09-10T21:10:10.303 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 43673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/whiteheat.ko 2024-09-10T21:10:10.303 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 12025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/wishbone-serial.ko 2024-09-10T21:10:10.303 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 10345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/xsens_mt.ko 2024-09-10T21:10:10.303 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ 2024-09-10T21:10:10.303 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 63361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/uas.ko 2024-09-10T21:10:10.303 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 30489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-alauda.ko 2024-09-10T21:10:10.303 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 14521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-cypress.ko 2024-09-10T21:10:10.303 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 25761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-datafab.ko 2024-09-10T21:10:10.304 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 41177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-09-10T21:10:10.304 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 15841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-freecom.ko 2024-09-10T21:10:10.304 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 23113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-isd200.ko 2024-09-10T21:10:10.304 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 22929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-09-10T21:10:10.304 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 15713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-karma.ko 2024-09-10T21:10:10.304 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 20817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-onetouch.ko 2024-09-10T21:10:10.304 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 34041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-realtek.ko 2024-09-10T21:10:10.304 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 34009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-sddr09.ko 2024-09-10T21:10:10.304 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 22681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-sddr55.ko 2024-09-10T21:10:10.304 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 41953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-usbat.ko 2024-09-10T21:10:10.304 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 201057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/usb-storage.ko 2024-09-10T21:10:10.304 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/ 2024-09-10T21:10:10.304 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 74449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/usbip-core.ko 2024-09-10T21:10:10.305 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 85513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/usbip-host.ko 2024-09-10T21:10:10.305 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 122537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-09-10T21:10:10.305 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/ 2024-09-10T21:10:10.305 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/ 2024-09-10T21:10:10.305 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 196977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-09-10T21:10:10.305 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 33737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/vfio-pci.ko 2024-09-10T21:10:10.305 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 134401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/vfio.ko 2024-09-10T21:10:10.305 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 93481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-09-10T21:10:10.305 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/ 2024-09-10T21:10:10.305 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 135241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost.ko 2024-09-10T21:10:10.305 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 19073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_iotlb.ko 2024-09-10T21:10:10.305 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 71553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_net.ko 2024-09-10T21:10:10.306 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 110185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_scsi.ko 2024-09-10T21:10:10.306 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/ 2024-09-10T21:10:10.306 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/ 2024-09-10T21:10:10.306 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 18081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/88pm860x_bl.ko 2024-09-10T21:10:10.306 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 14169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/aat2870_bl.ko 2024-09-10T21:10:10.306 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 29241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/adp5520_bl.ko 2024-09-10T21:10:10.306 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 41257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/adp8860_bl.ko 2024-09-10T21:10:10.306 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 51825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/adp8870_bl.ko 2024-09-10T21:10:10.306 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 20417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/apple_bl.ko 2024-09-10T21:10:10.306 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 19913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/as3711_bl.ko 2024-09-10T21:10:10.306 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 14761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/bd6107.ko 2024-09-10T21:10:10.306 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/da903x_bl.ko 2024-09-10T21:10:10.306 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/da9052_bl.ko 2024-09-10T21:10:10.306 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/gpio_backlight.ko 2024-09-10T21:10:10.307 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 17577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/kb3886_bl.ko 2024-09-10T21:10:10.307 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 26185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/lcd.ko 2024-09-10T21:10:10.307 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 28321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/lm3533_bl.ko 2024-09-10T21:10:10.307 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 23481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/lm3639_bl.ko 2024-09-10T21:10:10.307 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/lv5207lp.ko 2024-09-10T21:10:10.307 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 16689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/max8925_bl.ko 2024-09-10T21:10:10.307 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/pandora_bl.ko 2024-09-10T21:10:10.307 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/pcf50633-backlight.ko 2024-09-10T21:10:10.307 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/platform_lcd.ko 2024-09-10T21:10:10.307 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 15201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/wm831x_bl.ko 2024-09-10T21:10:10.307 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/ 2024-09-10T21:10:10.307 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ 2024-09-10T21:10:10.307 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 23129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ds2482.ko 2024-09-10T21:10:10.307 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 46913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ds2490.ko 2024-09-10T21:10:10.308 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/matrox_w1.ko 2024-09-10T21:10:10.308 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 14209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/w1-gpio.ko 2024-09-10T21:10:10.308 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/ 2024-09-10T21:10:10.308 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 14089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-09-10T21:10:10.308 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 27337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-09-10T21:10:10.308 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 16945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-09-10T21:10:10.308 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 12209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-09-10T21:10:10.308 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 16537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-09-10T21:10:10.308 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 16297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-09-10T21:10:10.308 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 16177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-09-10T21:10:10.309 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 16177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-09-10T21:10:10.309 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 21609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-09-10T21:10:10.309 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 8393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_smem.ko 2024-09-10T21:10:10.309 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 70473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_therm.ko 2024-09-10T21:10:10.309 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 127601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/wire.ko 2024-09-10T21:10:10.309 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ 2024-09-10T21:10:10.309 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 22425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/acquirewdt.ko 2024-09-10T21:10:10.309 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 23913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/advantechwdt.ko 2024-09-10T21:10:10.309 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 22289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/alim1535_wdt.ko 2024-09-10T21:10:10.309 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 27481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/alim7101_wdt.ko 2024-09-10T21:10:10.309 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 22081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/cpu5wdt.ko 2024-09-10T21:10:10.309 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 16545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/da9052_wdt.ko 2024-09-10T21:10:10.309 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 14713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/da9055_wdt.ko 2024-09-10T21:10:10.310 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 20993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/da9063_wdt.ko 2024-09-10T21:10:10.310 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 24545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/dw_wdt.ko 2024-09-10T21:10:10.310 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 23609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/eurotechwdt.ko 2024-09-10T21:10:10.310 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 26841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/f71808e_wdt.ko 2024-09-10T21:10:10.310 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 33009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/hpwdt.ko 2024-09-10T21:10:10.310 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 18129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/i6300esb.ko 2024-09-10T21:10:10.310 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 12201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-09-10T21:10:10.310 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 26817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/iTCO_wdt.ko 2024-09-10T21:10:10.310 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 23377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ib700wdt.ko 2024-09-10T21:10:10.310 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 21809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ibmasr.ko 2024-09-10T21:10:10.310 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 22793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-09-10T21:10:10.310 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 24937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/it8712f_wdt.ko 2024-09-10T21:10:10.310 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 16009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/it87_wdt.ko 2024-09-10T21:10:10.311 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 26465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/kempld_wdt.ko 2024-09-10T21:10:10.311 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 21649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/machzwd.ko 2024-09-10T21:10:10.311 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 17185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/mena21_wdt.ko 2024-09-10T21:10:10.311 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 15145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/menf21bmc_wdt.ko 2024-09-10T21:10:10.311 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 29841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/nv_tco.ko 2024-09-10T21:10:10.311 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 20657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/of_xilinx_wdt.ko 2024-09-10T21:10:10.311 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 27809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pc87413_wdt.ko 2024-09-10T21:10:10.311 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 47033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pcwd_pci.ko 2024-09-10T21:10:10.311 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 37289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pcwd_usb.ko 2024-09-10T21:10:10.311 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 15257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/retu_wdt.ko 2024-09-10T21:10:10.311 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 24521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sbc60xxwdt.ko 2024-09-10T21:10:10.311 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 17065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sbc_epx_c3.ko 2024-09-10T21:10:10.311 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 20433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-09-10T21:10:10.312 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 29257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sc1200wdt.ko 2024-09-10T21:10:10.312 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 27569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sch311x_wdt.ko 2024-09-10T21:10:10.312 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 25705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/smsc37b787_wdt.ko 2024-09-10T21:10:10.312 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 19801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/softdog.ko 2024-09-10T21:10:10.312 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 26089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sp5100_tco.ko 2024-09-10T21:10:10.312 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 15001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/twl4030_wdt.ko 2024-09-10T21:10:10.312 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 20257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/via_wdt.ko 2024-09-10T21:10:10.312 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 23481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-09-10T21:10:10.312 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 24985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83877f_wdt.ko 2024-09-10T21:10:10.312 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 32569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83977f_wdt.ko 2024-09-10T21:10:10.312 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 22657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/wafer5823wdt.ko 2024-09-10T21:10:10.312 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 33545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/wdt_pci.ko 2024-09-10T21:10:10.312 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 18985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/wm831x_wdt.ko 2024-09-10T21:10:10.312 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 16161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/wm8350_wdt.ko 2024-09-10T21:10:10.313 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 18537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/xen_wdt.ko 2024-09-10T21:10:10.313 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/ 2024-09-10T21:10:10.313 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 30225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-evtchn.ko 2024-09-10T21:10:10.313 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 30569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-gntalloc.ko 2024-09-10T21:10:10.313 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 55425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-gntdev.ko 2024-09-10T21:10:10.313 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-pciback/ 2024-09-10T21:10:10.313 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 214065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-09-10T21:10:10.313 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 46705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-privcmd.ko 2024-09-10T21:10:10.313 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 85905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-scsiback.ko 2024-09-10T21:10:10.313 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xenfs/ 2024-09-10T21:10:10.313 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 23537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xenfs/xenfs.ko 2024-09-10T21:10:10.313 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ 2024-09-10T21:10:10.314 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/9p/ 2024-09-10T21:10:10.314 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 195161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/9p/9p.ko 2024-09-10T21:10:10.314 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/adfs/ 2024-09-10T21:10:10.314 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 78369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/adfs/adfs.ko 2024-09-10T21:10:10.314 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/affs/ 2024-09-10T21:10:10.314 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 197849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/affs/affs.ko 2024-09-10T21:10:10.314 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/afs/ 2024-09-10T21:10:10.314 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 1259073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/afs/kafs.ko 2024-09-10T21:10:10.314 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/autofs/ 2024-09-10T21:10:10.314 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 127665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/autofs/autofs4.ko 2024-09-10T21:10:10.314 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/befs/ 2024-09-10T21:10:10.314 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 80561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/befs/befs.ko 2024-09-10T21:10:10.314 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/bfs/ 2024-09-10T21:10:10.314 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 50577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/bfs/bfs.ko 2024-09-10T21:10:10.315 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 45705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/binfmt_misc.ko 2024-09-10T21:10:10.315 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/btrfs/ 2024-09-10T21:10:10.315 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 4575041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/btrfs/btrfs.ko 2024-09-10T21:10:10.315 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cachefiles/ 2024-09-10T21:10:10.315 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 384329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cachefiles/cachefiles.ko 2024-09-10T21:10:10.357 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ceph/ 2024-09-10T21:10:10.357 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 1591681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ceph/ceph.ko 2024-09-10T21:10:10.357 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/coda/ 2024-09-10T21:10:10.357 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 130593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/coda/coda.ko 2024-09-10T21:10:10.357 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/configfs/ 2024-09-10T21:10:10.357 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 137313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/configfs/configfs.ko 2024-09-10T21:10:10.357 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cramfs/ 2024-09-10T21:10:10.357 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 35369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cramfs/cramfs.ko 2024-09-10T21:10:10.357 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/dlm/ 2024-09-10T21:10:10.357 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 841817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/dlm/dlm.ko 2024-09-10T21:10:10.357 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/efs/ 2024-09-10T21:10:10.358 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 45713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/efs/efs.ko 2024-09-10T21:10:10.358 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/f2fs/ 2024-09-10T21:10:10.358 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 1901633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/f2fs/f2fs.ko 2024-09-10T21:10:10.358 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/ 2024-09-10T21:10:10.358 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 36809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/msdos.ko 2024-09-10T21:10:10.358 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/freevxfs/ 2024-09-10T21:10:10.358 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 53401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/freevxfs/freevxfs.ko 2024-09-10T21:10:10.358 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse/ 2024-09-10T21:10:10.358 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 29281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse/cuse.ko 2024-09-10T21:10:10.358 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/gfs2/ 2024-09-10T21:10:10.358 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 1488329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/gfs2/gfs2.ko 2024-09-10T21:10:10.358 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfs/ 2024-09-10T21:10:10.358 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 183553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfs/hfs.ko 2024-09-10T21:10:10.359 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfsplus/ 2024-09-10T21:10:10.359 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 282177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfsplus/hfsplus.ko 2024-09-10T21:10:10.359 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hpfs/ 2024-09-10T21:10:10.359 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 248625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hpfs/hpfs.ko 2024-09-10T21:10:10.359 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/isofs/ 2024-09-10T21:10:10.359 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 93513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/isofs/isofs.ko 2024-09-10T21:10:10.359 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/jfs/ 2024-09-10T21:10:10.359 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 529833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/jfs/jfs.ko 2024-09-10T21:10:10.359 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/lockd/ 2024-09-10T21:10:10.359 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 359969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/lockd/lockd.ko 2024-09-10T21:10:10.359 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/minix/ 2024-09-10T21:10:10.359 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 113265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/minix/minix.ko 2024-09-10T21:10:10.359 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/netfs/ 2024-09-10T21:10:10.359 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 646793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/netfs/netfs.ko 2024-09-10T21:10:10.360 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/ 2024-09-10T21:10:10.360 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/blocklayout/ 2024-09-10T21:10:10.360 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 111585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-09-10T21:10:10.360 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/filelayout/ 2024-09-10T21:10:10.360 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 74345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-09-10T21:10:10.360 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/flexfilelayout/ 2024-09-10T21:10:10.360 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 147553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-09-10T21:10:10.360 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 1291361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfs.ko 2024-09-10T21:10:10.360 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 83001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv2.ko 2024-09-10T21:10:10.360 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 152537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv3.ko 2024-09-10T21:10:10.360 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 2526953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv4.ko 2024-09-10T21:10:10.360 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/ 2024-09-10T21:10:10.360 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 19457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/grace.ko 2024-09-10T21:10:10.361 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 19009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/nfs_acl.ko 2024-09-10T21:10:10.361 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfsd/ 2024-09-10T21:10:10.361 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 2119673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfsd/nfsd.ko 2024-09-10T21:10:10.361 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nilfs2/ 2024-09-10T21:10:10.361 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 734385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nilfs2/nilfs2.ko 2024-09-10T21:10:10.361 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/ 2024-09-10T21:10:10.361 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 15177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-celtic.ko 2024-09-10T21:10:10.361 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 14553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-centeuro.ko 2024-09-10T21:10:10.361 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 15177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-croatian.ko 2024-09-10T21:10:10.361 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 14249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-cyrillic.ko 2024-09-10T21:10:10.361 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 14865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-gaelic.ko 2024-09-10T21:10:10.361 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 14177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-greek.ko 2024-09-10T21:10:10.361 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 15177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-iceland.ko 2024-09-10T21:10:10.362 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 14489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-inuit.ko 2024-09-10T21:10:10.362 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 15417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-roman.ko 2024-09-10T21:10:10.362 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 15177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-romanian.ko 2024-09-10T21:10:10.362 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 15177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-turkish.ko 2024-09-10T21:10:10.362 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 12617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_ascii.ko 2024-09-10T21:10:10.362 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1250.ko 2024-09-10T21:10:10.362 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1251.ko 2024-09-10T21:10:10.362 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 14305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1255.ko 2024-09-10T21:10:10.362 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp737.ko 2024-09-10T21:10:10.362 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp775.ko 2024-09-10T21:10:10.362 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp850.ko 2024-09-10T21:10:10.362 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp852.ko 2024-09-10T21:10:10.362 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp855.ko 2024-09-10T21:10:10.363 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp857.ko 2024-09-10T21:10:10.363 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 14233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp860.ko 2024-09-10T21:10:10.363 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 14545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp861.ko 2024-09-10T21:10:10.363 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 14793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp862.ko 2024-09-10T21:10:10.363 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 14545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp863.ko 2024-09-10T21:10:10.363 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 14169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp864.ko 2024-09-10T21:10:10.363 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 14545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp865.ko 2024-09-10T21:10:10.363 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp866.ko 2024-09-10T21:10:10.363 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp869.ko 2024-09-10T21:10:10.363 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp874.ko 2024-09-10T21:10:10.363 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 95761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp932.ko 2024-09-10T21:10:10.363 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 144409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp936.ko 2024-09-10T21:10:10.363 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 167273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp949.ko 2024-09-10T21:10:10.363 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 119649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp950.ko 2024-09-10T21:10:10.364 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 14249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_euc-jp.ko 2024-09-10T21:10:10.364 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 12689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-1.ko 2024-09-10T21:10:10.364 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-13.ko 2024-09-10T21:10:10.364 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-14.ko 2024-09-10T21:10:10.364 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-15.ko 2024-09-10T21:10:10.364 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-2.ko 2024-09-10T21:10:10.364 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-3.ko 2024-09-10T21:10:10.364 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-4.ko 2024-09-10T21:10:10.364 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-5.ko 2024-09-10T21:10:10.364 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-6.ko 2024-09-10T21:10:10.364 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-7.ko 2024-09-10T21:10:10.364 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-9.ko 2024-09-10T21:10:10.364 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-r.ko 2024-09-10T21:10:10.365 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 9969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-ru.ko 2024-09-10T21:10:10.365 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-u.ko 2024-09-10T21:10:10.365 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 7353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_ucs2_utils.ko 2024-09-10T21:10:10.365 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 9801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_utf8.ko 2024-09-10T21:10:10.365 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ntfs3/ 2024-09-10T21:10:10.365 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 632217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ntfs3/ntfs3.ko 2024-09-10T21:10:10.365 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ 2024-09-10T21:10:10.365 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/cluster/ 2024-09-10T21:10:10.365 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 299761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-09-10T21:10:10.365 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlm/ 2024-09-10T21:10:10.365 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 547233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-09-10T21:10:10.365 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlmfs/ 2024-09-10T21:10:10.365 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 66465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-09-10T21:10:10.366 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 3255305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2.ko 2024-09-10T21:10:10.366 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 22273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-09-10T21:10:10.366 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 39409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-09-10T21:10:10.366 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 41297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-09-10T21:10:10.366 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/omfs/ 2024-09-10T21:10:10.366 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 67633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/omfs/omfs.ko 2024-09-10T21:10:10.366 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/overlayfs/ 2024-09-10T21:10:10.366 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 468161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/overlayfs/overlay.ko 2024-09-10T21:10:10.366 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/pstore/ 2024-09-10T21:10:10.366 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 69433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/pstore/ramoops.ko 2024-09-10T21:10:10.366 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/qnx4/ 2024-09-10T21:10:10.366 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 33433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/qnx4/qnx4.ko 2024-09-10T21:10:10.366 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/qnx6/ 2024-09-10T21:10:10.367 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 58177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/qnx6/qnx6.ko 2024-09-10T21:10:10.367 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/quota/ 2024-09-10T21:10:10.367 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 39689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/quota/quota_tree.ko 2024-09-10T21:10:10.367 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 17721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/quota/quota_v1.ko 2024-09-10T21:10:10.367 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 28953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/quota/quota_v2.ko 2024-09-10T21:10:10.367 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/reiserfs/ 2024-09-10T21:10:10.367 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 684489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/reiserfs/reiserfs.ko 2024-09-10T21:10:10.367 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/romfs/ 2024-09-10T21:10:10.367 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 33433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/romfs/romfs.ko 2024-09-10T21:10:10.367 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/ 2024-09-10T21:10:10.367 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/client/ 2024-09-10T21:10:10.367 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 3333977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/client/cifs.ko 2024-09-10T21:10:10.367 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/ 2024-09-10T21:10:10.367 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 7905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/cifs_arc4.ko 2024-09-10T21:10:10.368 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 12497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/cifs_md4.ko 2024-09-10T21:10:10.368 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/squashfs/ 2024-09-10T21:10:10.368 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 163081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/squashfs/squashfs.ko 2024-09-10T21:10:10.368 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/sysv/ 2024-09-10T21:10:10.368 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 116065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/sysv/sysv.ko 2024-09-10T21:10:10.368 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/udf/ 2024-09-10T21:10:10.368 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 309513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/udf/udf.ko 2024-09-10T21:10:10.368 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ufs/ 2024-09-10T21:10:10.368 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 198129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ufs/ufs.ko 2024-09-10T21:10:10.368 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/xfs/ 2024-09-10T21:10:10.368 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 5454809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/xfs/xfs.ko 2024-09-10T21:10:10.368 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/ 2024-09-10T21:10:10.369 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/locking/ 2024-09-10T21:10:10.369 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 97945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/locking/locktorture.ko 2024-09-10T21:10:10.369 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/time/ 2024-09-10T21:10:10.369 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 16025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/time/test_udelay.ko 2024-09-10T21:10:10.369 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 70441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/torture.ko 2024-09-10T21:10:10.369 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ 2024-09-10T21:10:10.369 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 7097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc-itu-t.ko 2024-09-10T21:10:10.369 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 6793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc7.ko 2024-09-10T21:10:10.369 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 7585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc8.ko 2024-09-10T21:10:10.369 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/ 2024-09-10T21:10:10.369 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 7849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libarc4.ko 2024-09-10T21:10:10.369 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 38433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libdes.ko 2024-09-10T21:10:10.369 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 14777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/interval_tree_test.ko 2024-09-10T21:10:10.370 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 9265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/libcrc32c.ko 2024-09-10T21:10:10.370 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 27673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lru_cache.ko 2024-09-10T21:10:10.370 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/ 2024-09-10T21:10:10.370 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 35473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/lz4_compress.ko 2024-09-10T21:10:10.370 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 24265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/lz4hc_compress.ko 2024-09-10T21:10:10.370 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/math/ 2024-09-10T21:10:10.370 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 7145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/math/cordic.ko 2024-09-10T21:10:10.370 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 9257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/memory-notifier-error-inject.ko 2024-09-10T21:10:10.370 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 12241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/notifier-error-inject.ko 2024-09-10T21:10:10.370 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 16145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/percpu_test.ko 2024-09-10T21:10:10.370 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 9337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/pm-notifier-error-inject.ko 2024-09-10T21:10:10.370 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/raid6/ 2024-09-10T21:10:10.370 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 151073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/raid6/raid6_pq.ko 2024-09-10T21:10:10.370 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 24569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/rbtree_test.ko 2024-09-10T21:10:10.371 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/reed_solomon/ 2024-09-10T21:10:10.371 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 20897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/reed_solomon/reed_solomon.ko 2024-09-10T21:10:10.371 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 41169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/test-kstrtox.ko 2024-09-10T21:10:10.371 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 4912673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/test_bpf.ko 2024-09-10T21:10:10.371 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 61249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/test_firmware.ko 2024-09-10T21:10:10.371 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 7321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/test_module.ko 2024-09-10T21:10:10.371 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 11537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_bm.ko 2024-09-10T21:10:10.371 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_fsm.ko 2024-09-10T21:10:10.371 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 11577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_kmp.ko 2024-09-10T21:10:10.371 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/xz/ 2024-09-10T21:10:10.371 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 15945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/xz/xz_dec_test.ko 2024-09-10T21:10:10.371 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/mm/ 2024-09-10T21:10:10.371 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 20385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/mm/hwpoison-inject.ko 2024-09-10T21:10:10.372 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ 2024-09-10T21:10:10.372 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/ 2024-09-10T21:10:10.372 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 66073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/6lowpan.ko 2024-09-10T21:10:10.372 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 7665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_dest.ko 2024-09-10T21:10:10.372 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 7609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_fragment.ko 2024-09-10T21:10:10.372 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 7657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_hop.ko 2024-09-10T21:10:10.372 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 7601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_ipv6.ko 2024-09-10T21:10:10.372 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 7609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_mobility.ko 2024-09-10T21:10:10.372 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 7609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_routing.ko 2024-09-10T21:10:10.372 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 17305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_udp.ko 2024-09-10T21:10:10.372 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/ 2024-09-10T21:10:10.372 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 36089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/garp.ko 2024-09-10T21:10:10.372 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 41673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/mrp.ko 2024-09-10T21:10:10.373 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 9633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/p8022.ko 2024-09-10T21:10:10.373 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 19113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/psnap.ko 2024-09-10T21:10:10.373 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/stp.ko 2024-09-10T21:10:10.373 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/8021q/ 2024-09-10T21:10:10.373 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 106841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/8021q/8021q.ko 2024-09-10T21:10:10.373 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/ 2024-09-10T21:10:10.373 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 234745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet.ko 2024-09-10T21:10:10.373 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 45337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_fd.ko 2024-09-10T21:10:10.373 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 32305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_rdma.ko 2024-09-10T21:10:10.373 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 39593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_virtio.ko 2024-09-10T21:10:10.373 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/batman-adv/ 2024-09-10T21:10:10.373 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 875441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/batman-adv/batman-adv.ko 2024-09-10T21:10:10.373 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/ 2024-09-10T21:10:10.374 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 88073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/br_netfilter.ko 2024-09-10T21:10:10.374 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 1026825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/bridge.ko 2024-09-10T21:10:10.374 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ 2024-09-10T21:10:10.374 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 8993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-09-10T21:10:10.374 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 14649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_among.ko 2024-09-10T21:10:10.374 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 10537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_arp.ko 2024-09-10T21:10:10.374 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 10505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-09-10T21:10:10.374 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 11041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-09-10T21:10:10.374 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 10089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_ip.ko 2024-09-10T21:10:10.374 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 10417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-09-10T21:10:10.374 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 11233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_limit.ko 2024-09-10T21:10:10.374 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 15465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_log.ko 2024-09-10T21:10:10.374 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 8921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_mark.ko 2024-09-10T21:10:10.374 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 8865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-09-10T21:10:10.375 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 9657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-09-10T21:10:10.375 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 8713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-09-10T21:10:10.375 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 10993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-09-10T21:10:10.375 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 10041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_snat.ko 2024-09-10T21:10:10.375 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 10185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_stp.ko 2024-09-10T21:10:10.375 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-09-10T21:10:10.375 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-09-10T21:10:10.375 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 11297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-09-10T21:10:10.375 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 11273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-09-10T21:10:10.375 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 69849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtables.ko 2024-09-10T21:10:10.375 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 16201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-09-10T21:10:10.375 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 16297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-09-10T21:10:10.375 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ceph/ 2024-09-10T21:10:10.375 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 1229873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ceph/libceph.ko 2024-09-10T21:10:10.376 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core/ 2024-09-10T21:10:10.376 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 154121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core/pktgen.ko 2024-09-10T21:10:10.376 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/ 2024-09-10T21:10:10.376 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 239393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp.ko 2024-09-10T21:10:10.376 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 10353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_diag.ko 2024-09-10T21:10:10.376 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 69873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_ipv4.ko 2024-09-10T21:10:10.376 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 74433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_ipv6.ko 2024-09-10T21:10:10.376 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/ 2024-09-10T21:10:10.376 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 498249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/dsa_core.ko 2024-09-10T21:10:10.376 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 15929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_brcm.ko 2024-09-10T21:10:10.376 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 15009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_dsa.ko 2024-09-10T21:10:10.376 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 8961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_none.ko 2024-09-10T21:10:10.376 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 10545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_trailer.ko 2024-09-10T21:10:10.377 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/hsr/ 2024-09-10T21:10:10.377 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 137945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/hsr/hsr.ko 2024-09-10T21:10:10.377 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ 2024-09-10T21:10:10.377 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/6lowpan/ 2024-09-10T21:10:10.377 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 55409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-09-10T21:10:10.377 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 356385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ieee802154.ko 2024-09-10T21:10:10.377 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 59457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ieee802154_socket.ko 2024-09-10T21:10:10.377 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ 2024-09-10T21:10:10.377 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 29169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ah4.ko 2024-09-10T21:10:10.377 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 46153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/esp4.ko 2024-09-10T21:10:10.377 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 59793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/fou.ko 2024-09-10T21:10:10.377 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 21705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/gre.ko 2024-09-10T21:10:10.377 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 64777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/inet_diag.ko 2024-09-10T21:10:10.377 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 77777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_gre.ko 2024-09-10T21:10:10.378 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 78481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_tunnel.ko 2024-09-10T21:10:10.378 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 43153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_vti.ko 2024-09-10T21:10:10.378 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 16625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ipcomp.ko 2024-09-10T21:10:10.378 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 34249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ipip.ko 2024-09-10T21:10:10.378 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ 2024-09-10T21:10:10.378 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 44921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arp_tables.ko 2024-09-10T21:10:10.378 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 10225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-09-10T21:10:10.378 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 11985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-09-10T21:10:10.378 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 49609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ip_tables.ko 2024-09-10T21:10:10.378 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 11513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-09-10T21:10:10.378 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 11537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_REJECT.ko 2024-09-10T21:10:10.378 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 16441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-09-10T21:10:10.378 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 12601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-09-10T21:10:10.379 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 17377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-09-10T21:10:10.379 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_filter.ko 2024-09-10T21:10:10.379 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-09-10T21:10:10.379 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 17529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-09-10T21:10:10.379 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-09-10T21:10:10.379 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 12057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_security.ko 2024-09-10T21:10:10.379 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 14265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-09-10T21:10:10.379 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 9577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-09-10T21:10:10.379 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 32561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-09-10T21:10:10.379 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 20825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-09-10T21:10:10.379 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 15105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-09-10T21:10:10.379 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 24961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_reject_ipv4.ko 2024-09-10T21:10:10.379 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 10105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-09-10T21:10:10.379 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 16769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-09-10T21:10:10.380 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 11545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-09-10T21:10:10.380 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 19289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-09-10T21:10:10.380 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 10153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-09-10T21:10:10.380 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 14769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_bic.ko 2024-09-10T21:10:10.380 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 16817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_dctcp.ko 2024-09-10T21:10:10.380 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 19489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_diag.ko 2024-09-10T21:10:10.380 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 11105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_highspeed.ko 2024-09-10T21:10:10.380 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 14473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_htcp.ko 2024-09-10T21:10:10.380 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 11753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_hybla.ko 2024-09-10T21:10:10.380 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_illinois.ko 2024-09-10T21:10:10.380 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 11825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_lp.ko 2024-09-10T21:10:10.380 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 9689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_scalable.ko 2024-09-10T21:10:10.380 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 16113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_vegas.ko 2024-09-10T21:10:10.381 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 12193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_veno.ko 2024-09-10T21:10:10.381 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_westwood.ko 2024-09-10T21:10:10.381 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 11833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_yeah.ko 2024-09-10T21:10:10.381 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 28345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tunnel4.ko 2024-09-10T21:10:10.381 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 21201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/udp_diag.ko 2024-09-10T21:10:10.381 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 53401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/udp_tunnel.ko 2024-09-10T21:10:10.381 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/xfrm4_tunnel.ko 2024-09-10T21:10:10.381 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ 2024-09-10T21:10:10.381 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 31201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ah6.ko 2024-09-10T21:10:10.381 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 47193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/esp6.ko 2024-09-10T21:10:10.381 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 17201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/fou6.ko 2024-09-10T21:10:10.381 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 102889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_gre.ko 2024-09-10T21:10:10.381 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 117649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_tunnel.ko 2024-09-10T21:10:10.381 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 15953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-09-10T21:10:10.382 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 69881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_vti.ko 2024-09-10T21:10:10.382 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 16361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ipcomp6.ko 2024-09-10T21:10:10.382 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 23097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/mip6.ko 2024-09-10T21:10:10.382 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ 2024-09-10T21:10:10.382 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 50137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-09-10T21:10:10.382 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_NPT.ko 2024-09-10T21:10:10.382 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 11673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-09-10T21:10:10.382 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 16449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-09-10T21:10:10.382 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 16009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-09-10T21:10:10.382 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 8809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-09-10T21:10:10.382 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 18041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-09-10T21:10:10.382 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 17713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-09-10T21:10:10.382 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 10129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-09-10T21:10:10.383 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 11225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-09-10T21:10:10.383 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 12561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-09-10T21:10:10.383 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 12273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-09-10T21:10:10.383 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-09-10T21:10:10.383 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 12937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-09-10T21:10:10.383 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 17537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-09-10T21:10:10.383 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-09-10T21:10:10.383 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 12065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-09-10T21:10:10.383 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 46033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-09-10T21:10:10.383 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 9225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-09-10T21:10:10.383 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 29017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-09-10T21:10:10.383 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 10617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-09-10T21:10:10.383 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 16873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-09-10T21:10:10.383 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 11185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-09-10T21:10:10.384 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 18305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-09-10T21:10:10.384 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 10153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-09-10T21:10:10.384 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 94433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/sit.ko 2024-09-10T21:10:10.384 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 30065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/tunnel6.ko 2024-09-10T21:10:10.384 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 28137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/xfrm6_tunnel.ko 2024-09-10T21:10:10.384 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/key/ 2024-09-10T21:10:10.384 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 106217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/key/af_key.ko 2024-09-10T21:10:10.384 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/ 2024-09-10T21:10:10.384 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 155593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_core.ko 2024-09-10T21:10:10.384 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 24489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_debugfs.ko 2024-09-10T21:10:10.384 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 32137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_eth.ko 2024-09-10T21:10:10.384 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 43553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ip.ko 2024-09-10T21:10:10.384 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 45057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ip6.ko 2024-09-10T21:10:10.385 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 42041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_netlink.ko 2024-09-10T21:10:10.385 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 66137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ppp.ko 2024-09-10T21:10:10.385 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/lapb/ 2024-09-10T21:10:10.385 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 61609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/lapb/lapb.ko 2024-09-10T21:10:10.385 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc/ 2024-09-10T21:10:10.385 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 26481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc/llc.ko 2024-09-10T21:10:10.385 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 333641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc/llc2.ko 2024-09-10T21:10:10.385 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mac802154/ 2024-09-10T21:10:10.385 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 388577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mac802154/mac802154.ko 2024-09-10T21:10:10.385 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mpls/ 2024-09-10T21:10:10.385 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 10665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mpls/mpls_gso.ko 2024-09-10T21:10:10.385 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ 2024-09-10T21:10:10.385 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ 2024-09-10T21:10:10.386 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 144881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set.ko 2024-09-10T21:10:10.386 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 34561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-09-10T21:10:10.386 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 33153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-09-10T21:10:10.386 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 32633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-09-10T21:10:10.386 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 124137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-09-10T21:10:10.386 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 124449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-09-10T21:10:10.386 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 126777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-09-10T21:10:10.386 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 126577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-09-10T21:10:10.386 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 137937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-09-10T21:10:10.386 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 72457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-09-10T21:10:10.386 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 133785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-09-10T21:10:10.386 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 139177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-09-10T21:10:10.386 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 138369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-09-10T21:10:10.386 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 136281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-09-10T21:10:10.387 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 140001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-09-10T21:10:10.387 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 38849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-09-10T21:10:10.387 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ 2024-09-10T21:10:10.387 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 532393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs.ko 2024-09-10T21:10:10.387 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 14297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-09-10T21:10:10.387 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 8889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-09-10T21:10:10.387 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 20649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-09-10T21:10:10.387 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 20937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-09-10T21:10:10.387 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 24625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-09-10T21:10:10.387 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 8889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-09-10T21:10:10.387 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 8953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-09-10T21:10:10.387 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 12745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-09-10T21:10:10.387 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 10961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-09-10T21:10:10.388 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 9001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-09-10T21:10:10.388 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 16065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-09-10T21:10:10.388 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 9001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-09-10T21:10:10.388 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 12905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-09-10T21:10:10.388 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 54465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conncount.ko 2024-09-10T21:10:10.388 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 485769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack.ko 2024-09-10T21:10:10.388 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 17953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-09-10T21:10:10.388 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 11633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-09-10T21:10:10.388 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 37729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-09-10T21:10:10.388 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 134177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_h323.ko 2024-09-10T21:10:10.388 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 24385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_irc.ko 2024-09-10T21:10:10.388 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 9545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-09-10T21:10:10.388 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 140921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-09-10T21:10:10.388 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 38089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-09-10T21:10:10.389 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 19025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_sane.ko 2024-09-10T21:10:10.389 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 79081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_sip.ko 2024-09-10T21:10:10.389 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 12417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-09-10T21:10:10.389 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 20945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-09-10T21:10:10.389 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 10993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_dup_netdev.ko 2024-09-10T21:10:10.389 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 119745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_flow_table.ko 2024-09-10T21:10:10.389 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 11929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_flow_table_inet.ko 2024-09-10T21:10:10.389 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 46193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_log_syslog.ko 2024-09-10T21:10:10.389 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 115817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat.ko 2024-09-10T21:10:10.389 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 11457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_amanda.ko 2024-09-10T21:10:10.389 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 14273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_ftp.ko 2024-09-10T21:10:10.389 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_irc.ko 2024-09-10T21:10:10.389 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 34697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_sip.ko 2024-09-10T21:10:10.390 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 9977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_tftp.ko 2024-09-10T21:10:10.390 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 51953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_synproxy_core.ko 2024-09-10T21:10:10.390 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 813705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_tables.ko 2024-09-10T21:10:10.390 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 52913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink.ko 2024-09-10T21:10:10.390 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 36049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_acct.ko 2024-09-10T21:10:10.390 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 37241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_cttimeout.ko 2024-09-10T21:10:10.390 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 49457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_log.ko 2024-09-10T21:10:10.390 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 26129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_osf.ko 2024-09-10T21:10:10.390 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 66593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_queue.ko 2024-09-10T21:10:10.390 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 12081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_chain_nat.ko 2024-09-10T21:10:10.390 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 41849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_compat.ko 2024-09-10T21:10:10.390 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 50641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_ct.ko 2024-09-10T21:10:10.390 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 11161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_dup_netdev.ko 2024-09-10T21:10:10.390 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_fib.ko 2024-09-10T21:10:10.391 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 16833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_fwd_netdev.ko 2024-09-10T21:10:10.391 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 17329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_hash.ko 2024-09-10T21:10:10.391 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 25385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_limit.ko 2024-09-10T21:10:10.391 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 18017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_log.ko 2024-09-10T21:10:10.391 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 17529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_masq.ko 2024-09-10T21:10:10.391 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 16953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_nat.ko 2024-09-10T21:10:10.391 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 16569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_numgen.ko 2024-09-10T21:10:10.391 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 14097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_osf.ko 2024-09-10T21:10:10.391 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 16921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_queue.ko 2024-09-10T21:10:10.391 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 16745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_redir.ko 2024-09-10T21:10:10.391 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 11593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_reject.ko 2024-09-10T21:10:10.391 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 11465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_reject_inet.ko 2024-09-10T21:10:10.391 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 15145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_socket.ko 2024-09-10T21:10:10.391 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 25081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_synproxy.ko 2024-09-10T21:10:10.392 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 17721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_tproxy.ko 2024-09-10T21:10:10.392 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 104257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/x_tables.ko 2024-09-10T21:10:10.392 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 14281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_AUDIT.ko 2024-09-10T21:10:10.392 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 10953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CHECKSUM.ko 2024-09-10T21:10:10.392 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 8713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CLASSIFY.ko 2024-09-10T21:10:10.392 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-09-10T21:10:10.392 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 21953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CT.ko 2024-09-10T21:10:10.392 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 12377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_DSCP.ko 2024-09-10T21:10:10.392 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 11153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_HL.ko 2024-09-10T21:10:10.392 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 15833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_HMARK.ko 2024-09-10T21:10:10.392 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 45305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_IDLETIMER.ko 2024-09-10T21:10:10.392 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 16033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_LED.ko 2024-09-10T21:10:10.392 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_LOG.ko 2024-09-10T21:10:10.392 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_MASQUERADE.ko 2024-09-10T21:10:10.393 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NETMAP.ko 2024-09-10T21:10:10.393 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 10937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NFLOG.ko 2024-09-10T21:10:10.393 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 15449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NFQUEUE.ko 2024-09-10T21:10:10.393 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 22809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_RATEEST.ko 2024-09-10T21:10:10.393 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 12633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_REDIRECT.ko 2024-09-10T21:10:10.393 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 16777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_SECMARK.ko 2024-09-10T21:10:10.393 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 19065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TCPMSS.ko 2024-09-10T21:10:10.393 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 11849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-09-10T21:10:10.393 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 20793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TEE.ko 2024-09-10T21:10:10.393 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 19713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TPROXY.ko 2024-09-10T21:10:10.393 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 9529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TRACE.ko 2024-09-10T21:10:10.393 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 15377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_addrtype.ko 2024-09-10T21:10:10.393 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 16665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_bpf.ko 2024-09-10T21:10:10.394 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 20641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cgroup.ko 2024-09-10T21:10:10.394 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 12953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cluster.ko 2024-09-10T21:10:10.394 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 8545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_comment.ko 2024-09-10T21:10:10.394 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 12041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connbytes.ko 2024-09-10T21:10:10.394 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connlabel.ko 2024-09-10T21:10:10.394 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 10705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connlimit.ko 2024-09-10T21:10:10.394 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 15921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connmark.ko 2024-09-10T21:10:10.394 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 14297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_conntrack.ko 2024-09-10T21:10:10.394 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 9001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cpu.ko 2024-09-10T21:10:10.394 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 12585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_dccp.ko 2024-09-10T21:10:10.394 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 9089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_devgroup.ko 2024-09-10T21:10:10.394 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 10345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_dscp.ko 2024-09-10T21:10:10.394 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 12745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ecn.ko 2024-09-10T21:10:10.394 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 12961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_esp.ko 2024-09-10T21:10:10.395 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 52929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_hashlimit.ko 2024-09-10T21:10:10.395 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 12921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_helper.ko 2024-09-10T21:10:10.395 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 9601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_hl.ko 2024-09-10T21:10:10.395 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ipcomp.ko 2024-09-10T21:10:10.395 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 12945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_iprange.ko 2024-09-10T21:10:10.395 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ipvs.ko 2024-09-10T21:10:10.395 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 19273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_l2tp.ko 2024-09-10T21:10:10.395 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 9225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_length.ko 2024-09-10T21:10:10.395 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 11609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_limit.ko 2024-09-10T21:10:10.395 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 8529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_mac.ko 2024-09-10T21:10:10.395 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 9969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_mark.ko 2024-09-10T21:10:10.395 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_multiport.ko 2024-09-10T21:10:10.395 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 16249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_nat.ko 2024-09-10T21:10:10.395 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 11521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_nfacct.ko 2024-09-10T21:10:10.396 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 9521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_osf.ko 2024-09-10T21:10:10.396 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 11649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_owner.ko 2024-09-10T21:10:10.396 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 11545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_physdev.ko 2024-09-10T21:10:10.396 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 8769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_pkttype.ko 2024-09-10T21:10:10.396 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_policy.ko 2024-09-10T21:10:10.396 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 11001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_quota.ko 2024-09-10T21:10:10.396 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 11969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_rateest.ko 2024-09-10T21:10:10.396 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 9361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_realm.ko 2024-09-10T21:10:10.396 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 43569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_recent.ko 2024-09-10T21:10:10.396 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 14985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_sctp.ko 2024-09-10T21:10:10.396 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 36577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_set.ko 2024-09-10T21:10:10.396 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 19865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_socket.ko 2024-09-10T21:10:10.396 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 10833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_state.ko 2024-09-10T21:10:10.396 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 10945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_statistic.ko 2024-09-10T21:10:10.397 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 10105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_string.ko 2024-09-10T21:10:10.397 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 9673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_tcpmss.ko 2024-09-10T21:10:10.397 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 18529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_tcpudp.ko 2024-09-10T21:10:10.397 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 12529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_time.ko 2024-09-10T21:10:10.397 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 10697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_u32.ko 2024-09-10T21:10:10.397 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netlink/ 2024-09-10T21:10:10.397 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 17321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netlink/netlink_diag.ko 2024-09-10T21:10:10.397 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/nsh/ 2024-09-10T21:10:10.397 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/nsh/nsh.ko 2024-09-10T21:10:10.397 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/ 2024-09-10T21:10:10.397 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 485961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/openvswitch.ko 2024-09-10T21:10:10.397 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 12097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-geneve.ko 2024-09-10T21:10:10.397 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 10961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-gre.ko 2024-09-10T21:10:10.397 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-vxlan.ko 2024-09-10T21:10:10.398 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/packet/ 2024-09-10T21:10:10.398 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 16009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/packet/af_packet_diag.ko 2024-09-10T21:10:10.398 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/phonet/ 2024-09-10T21:10:10.398 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 141545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/phonet/phonet.ko 2024-09-10T21:10:10.398 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 69001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/phonet/pn_pep.ko 2024-09-10T21:10:10.398 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/ 2024-09-10T21:10:10.398 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 314473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds.ko 2024-09-10T21:10:10.398 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 202161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds_rdma.ko 2024-09-10T21:10:10.398 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 85265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds_tcp.ko 2024-09-10T21:10:10.398 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rxrpc/ 2024-09-10T21:10:10.398 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 1024009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rxrpc/rxrpc.ko 2024-09-10T21:10:10.398 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/ 2024-09-10T21:10:10.398 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 39097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_csum.ko 2024-09-10T21:10:10.399 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 28817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_gact.ko 2024-09-10T21:10:10.399 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 45505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_mirred.ko 2024-09-10T21:10:10.399 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 27689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_nat.ko 2024-09-10T21:10:10.399 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 53577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_pedit.ko 2024-09-10T21:10:10.399 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 38801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_police.ko 2024-09-10T21:10:10.399 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 24057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_simple.ko 2024-09-10T21:10:10.399 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 43513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_skbedit.ko 2024-09-10T21:10:10.399 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 45769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_vlan.ko 2024-09-10T21:10:10.399 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 32289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_basic.ko 2024-09-10T21:10:10.399 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 41561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_bpf.ko 2024-09-10T21:10:10.399 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 23417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_cgroup.ko 2024-09-10T21:10:10.399 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 37409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_flow.ko 2024-09-10T21:10:10.399 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 39697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_fw.ko 2024-09-10T21:10:10.399 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 49417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_route.ko 2024-09-10T21:10:10.400 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 79257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_u32.ko 2024-09-10T21:10:10.400 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 8793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_cmp.ko 2024-09-10T21:10:10.400 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 15481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_ipset.ko 2024-09-10T21:10:10.400 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 48889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_meta.ko 2024-09-10T21:10:10.400 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 9337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_nbyte.ko 2024-09-10T21:10:10.400 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_text.ko 2024-09-10T21:10:10.400 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 8457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_u32.ko 2024-09-10T21:10:10.400 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 25921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_choke.ko 2024-09-10T21:10:10.400 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 25209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_codel.ko 2024-09-10T21:10:10.400 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 43577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_drr.ko 2024-09-10T21:10:10.400 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 43865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_fq.ko 2024-09-10T21:10:10.400 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 39273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_fq_codel.ko 2024-09-10T21:10:10.400 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 42985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_gred.ko 2024-09-10T21:10:10.401 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 60913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_hfsc.ko 2024-09-10T21:10:10.401 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 26689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_hhf.ko 2024-09-10T21:10:10.401 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 81065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_htb.ko 2024-09-10T21:10:10.401 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 33745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_ingress.ko 2024-09-10T21:10:10.401 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 36713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_mqprio.ko 2024-09-10T21:10:10.401 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 11905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_mqprio_lib.ko 2024-09-10T21:10:10.401 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 33153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_multiq.ko 2024-09-10T21:10:10.401 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 44857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_netem.ko 2024-09-10T21:10:10.401 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 32425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_pie.ko 2024-09-10T21:10:10.401 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 13593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_plug.ko 2024-09-10T21:10:10.401 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 31889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_prio.ko 2024-09-10T21:10:10.401 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 60337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_qfq.ko 2024-09-10T21:10:10.401 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 40977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_red.ko 2024-09-10T21:10:10.401 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 33521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_sfb.ko 2024-09-10T21:10:10.402 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 38457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_sfq.ko 2024-09-10T21:10:10.402 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 35857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_tbf.ko 2024-09-10T21:10:10.402 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 34241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_teql.ko 2024-09-10T21:10:10.402 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/ 2024-09-10T21:10:10.402 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 1184689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/sctp.ko 2024-09-10T21:10:10.402 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 24001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/sctp_diag.ko 2024-09-10T21:10:10.402 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/ 2024-09-10T21:10:10.402 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/ 2024-09-10T21:10:10.402 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 426969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-09-10T21:10:10.402 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 70273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-09-10T21:10:10.402 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 1895553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/sunrpc.ko 2024-09-10T21:10:10.402 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/xprtrdma/ 2024-09-10T21:10:10.402 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 925089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-09-10T21:10:10.403 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/ 2024-09-10T21:10:10.403 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 1122601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/tipc.ko 2024-09-10T21:10:10.403 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 12241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/tipc_diag.ko 2024-09-10T21:10:10.403 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/unix/ 2024-09-10T21:10:10.403 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 18273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/unix/unix_diag.ko 2024-09-10T21:10:10.403 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/ 2024-09-10T21:10:10.403 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 101161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-09-10T21:10:10.403 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 90369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-09-10T21:10:10.403 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 120625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock.ko 2024-09-10T21:10:10.403 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 12985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock_diag.ko 2024-09-10T21:10:10.403 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 20489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock_loopback.ko 2024-09-10T21:10:10.403 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/x25/ 2024-09-10T21:10:10.403 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 165529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/x25/x25.ko 2024-09-10T21:10:10.404 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm/ 2024-09-10T21:10:10.404 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 25449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm/xfrm_algo.ko 2024-09-10T21:10:10.404 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 22929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm/xfrm_ipcomp.ko 2024-09-10T21:10:10.404 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 135873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm/xfrm_user.ko 2024-09-10T21:10:10.404 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 649700 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.alias 2024-09-10T21:10:10.404 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 633853 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.alias.bin 2024-09-10T21:10:10.404 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 7850 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin 2024-09-10T21:10:10.404 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 23959 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.alias.bin 2024-09-10T21:10:10.404 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 10056 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.bin 2024-09-10T21:10:10.404 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 74053 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.modinfo 2024-09-10T21:10:10.404 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 147543 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep 2024-09-10T21:10:10.404 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 233297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep.bin 2024-09-10T21:10:10.404 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 210 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.devname 2024-09-10T21:10:10.404 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 72756 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.order 2024-09-10T21:10:10.405 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 879 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.softdep 2024-09-10T21:10:10.405 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 261535 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.symbols 2024-09-10T21:10:10.405 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 316586 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.symbols.bin 2024-09-10T21:10:10.405 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./usr/ 2024-09-10T21:10:10.405 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./usr/share/ 2024-09-10T21:10:10.405 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./usr/share/doc/ 2024-09-10T21:10:10.405 INFO:teuthology.orchestra.run.smithi031.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./usr/share/doc/linux-image-6.11.0-rc5-g2ca44a2f35ca/ 2024-09-10T21:10:10.405 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 176 2024-09-06 14:57 ./usr/share/doc/linux-image-6.11.0-rc5-g2ca44a2f35ca/changelog.Debian.gz 2024-09-10T21:10:10.405 INFO:teuthology.orchestra.run.smithi031.stdout:-rw-r--r-- root/root 691 2024-09-06 14:57 ./usr/share/doc/linux-image-6.11.0-rc5-g2ca44a2f35ca/copyright 2024-09-10T21:10:10.410 DEBUG:teuthology.task.kernel:get_image_version: 6.11.0-rc5-g2ca44a2f35ca 2024-09-10T21:10:10.411 INFO:teuthology.task.kernel:searching for kernel 6.11.0-rc5-g2ca44a2f35ca 2024-09-10T21:10:10.411 DEBUG:teuthology.orchestra.run.smithi031:> egrep '(submenu|menuentry.*6.11.0-rc5-g2ca44a2f35ca).*{' /boot/grub/grub.cfg 2024-09-10T21:10:10.417 INFO:teuthology.orchestra.run.smithi031.stdout:submenu 'Advanced options for Ubuntu' $menuentry_id_option 'gnulinux-advanced-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2024-09-10T21:10:10.417 INFO:teuthology.orchestra.run.smithi031.stdout: menuentry 'Ubuntu, with Linux 6.11.0-rc5-g2ca44a2f35ca' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.11.0-rc5-g2ca44a2f35ca-advanced-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2024-09-10T21:10:10.417 INFO:teuthology.orchestra.run.smithi031.stdout: menuentry 'Ubuntu, with Linux 6.11.0-rc5-g2ca44a2f35ca (recovery mode)' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.11.0-rc5-g2ca44a2f35ca-recovery-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2024-09-10T21:10:10.418 INFO:teuthology.task.kernel:submenu_title:Advanced options for Ubuntu> 2024-09-10T21:10:10.418 INFO:teuthology.task.kernel:default_title:Ubuntu, with Linux 6.11.0-rc5-g2ca44a2f35ca 2024-09-10T21:10:10.418 DEBUG:teuthology.orchestra.run.smithi031:> /bin/echo -e 'cat </dev/null && sudo chmod a+x -- '/etc/grub.d/01_ceph_kernel.tmp~' && sudo mv -- '/etc/grub.d/01_ceph_kernel.tmp~' /etc/grub.d/01_ceph_kernel && sudo update-grub && rm /tmp/linux-image.deb && ( sleep 1 && sudo shutdown -r now & ) 2024-09-10T21:10:10.463 DEBUG:teuthology.task.kernel:Waiting for install on ubuntu@smithi031.front.sepia.ceph.com to complete... 2024-09-10T21:10:10.817 INFO:teuthology.orchestra.run.smithi031.stderr:Sourcing file `/etc/default/grub' 2024-09-10T21:10:10.818 INFO:teuthology.orchestra.run.smithi031.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-10T21:10:10.827 INFO:teuthology.orchestra.run.smithi031.stderr:Generating grub configuration file ... 2024-09-10T21:10:10.999 INFO:teuthology.orchestra.run.smithi031.stderr:Found linux image: /boot/vmlinuz-6.11.0-rc5-g2ca44a2f35ca 2024-09-10T21:10:11.005 INFO:teuthology.orchestra.run.smithi031.stderr:Found initrd image: /boot/initrd.img-6.11.0-rc5-g2ca44a2f35ca 2024-09-10T21:10:11.253 INFO:teuthology.orchestra.run.smithi031.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-10T21:10:11.257 INFO:teuthology.orchestra.run.smithi031.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-10T21:10:11.297 INFO:teuthology.orchestra.run.smithi031.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-10T21:10:11.302 INFO:teuthology.orchestra.run.smithi031.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-10T21:10:11.445 INFO:teuthology.orchestra.run.smithi031.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-10T21:10:11.445 INFO:teuthology.orchestra.run.smithi031.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-09-10T21:10:11.445 INFO:teuthology.orchestra.run.smithi031.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-10T21:10:11.453 INFO:teuthology.orchestra.run.smithi031.stderr:done 2024-09-10T21:10:12.075 INFO:teuthology.orchestra.run.smithi017.stderr:Sourcing file `/etc/default/grub' 2024-09-10T21:10:12.089 INFO:teuthology.orchestra.run.smithi017.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-10T21:10:12.105 INFO:teuthology.orchestra.run.smithi017.stderr:Generating grub configuration file ... 2024-09-10T21:10:12.433 INFO:teuthology.orchestra.run.smithi017.stderr:Found linux image: /boot/vmlinuz-6.11.0-rc5-g2ca44a2f35ca 2024-09-10T21:10:12.448 INFO:teuthology.orchestra.run.smithi017.stderr:Found initrd image: /boot/initrd.img-6.11.0-rc5-g2ca44a2f35ca 2024-09-10T21:10:12.731 INFO:teuthology.orchestra.run.smithi017.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-10T21:10:12.740 INFO:teuthology.orchestra.run.smithi017.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-10T21:10:12.806 INFO:teuthology.orchestra.run.smithi017.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-10T21:10:12.811 INFO:teuthology.orchestra.run.smithi017.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-10T21:10:12.957 INFO:teuthology.orchestra.run.smithi017.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-10T21:10:12.958 INFO:teuthology.orchestra.run.smithi017.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-09-10T21:10:12.958 INFO:teuthology.orchestra.run.smithi017.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-10T21:10:12.994 INFO:teuthology.orchestra.run.smithi017.stderr:done 2024-09-10T21:10:13.152 DEBUG:teuthology.orchestra.run.smithi017:> dpkg-deb -c /tmp/linux-image.deb 2024-09-10T21:10:13.160 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./ 2024-09-10T21:10:13.161 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./boot/ 2024-09-10T21:10:13.161 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 7098725 2024-09-06 14:57 ./boot/System.map-6.11.0-rc5-g2ca44a2f35ca 2024-09-10T21:10:13.180 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 174121 2024-09-06 14:57 ./boot/config-6.11.0-rc5-g2ca44a2f35ca 2024-09-10T21:10:13.180 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 13292544 2024-09-06 14:57 ./boot/vmlinuz-6.11.0-rc5-g2ca44a2f35ca 2024-09-10T21:10:13.203 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./etc/ 2024-09-10T21:10:13.203 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./etc/kernel/ 2024-09-10T21:10:13.203 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./etc/kernel/postinst.d/ 2024-09-10T21:10:13.203 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./etc/kernel/postrm.d/ 2024-09-10T21:10:13.203 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./etc/kernel/preinst.d/ 2024-09-10T21:10:13.204 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./etc/kernel/prerm.d/ 2024-09-10T21:10:13.204 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/ 2024-09-10T21:10:13.204 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/ 2024-09-10T21:10:13.204 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/ 2024-09-10T21:10:13.204 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/ 2024-09-10T21:10:13.204 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/ 2024-09-10T21:10:13.204 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/ 2024-09-10T21:10:13.204 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/ 2024-09-10T21:10:13.204 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 189809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/aesni-intel.ko 2024-09-10T21:10:13.204 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 28449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-09-10T21:10:13.204 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 33993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-09-10T21:10:13.205 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 35097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-09-10T21:10:13.205 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 97321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-09-10T21:10:13.205 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 109985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-09-10T21:10:13.205 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 155337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-09-10T21:10:13.206 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 15441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-09-10T21:10:13.207 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 16177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-09-10T21:10:13.207 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 87097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-09-10T21:10:13.207 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 22945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-09-10T21:10:13.207 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 49905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-09-10T21:10:13.207 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 50457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-avx2.ko 2024-09-10T21:10:13.207 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 54593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-09-10T21:10:13.207 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 44745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-09-10T21:10:13.207 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 48073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-09-10T21:10:13.207 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 62961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-09-10T21:10:13.207 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 53657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-09-10T21:10:13.207 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 34441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-09-10T21:10:13.207 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 14585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-09-10T21:10:13.208 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/ 2024-09-10T21:10:13.208 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/ 2024-09-10T21:10:13.208 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/mce/ 2024-09-10T21:10:13.208 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 49497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-09-10T21:10:13.208 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 16073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpuid.ko 2024-09-10T21:10:13.208 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 21713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/msr.ko 2024-09-10T21:10:13.208 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/ 2024-09-10T21:10:13.208 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 505169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm-amd.ko 2024-09-10T21:10:13.208 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 1060585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm-intel.ko 2024-09-10T21:10:13.209 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 2858865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm.ko 2024-09-10T21:10:13.227 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ 2024-09-10T21:10:13.227 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 62113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/af_alg.ko 2024-09-10T21:10:13.227 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 28033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/algif_hash.ko 2024-09-10T21:10:13.228 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 25921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/algif_skcipher.ko 2024-09-10T21:10:13.228 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 20313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ansi_cprng.ko 2024-09-10T21:10:13.228 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 21153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/anubis.ko 2024-09-10T21:10:13.228 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 11017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/arc4.ko 2024-09-10T21:10:13.228 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/asymmetric_keys/ 2024-09-10T21:10:13.228 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 11545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/asymmetric_keys/pkcs7_test_key.ko 2024-09-10T21:10:13.228 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/ 2024-09-10T21:10:13.228 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 12545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_memcpy.ko 2024-09-10T21:10:13.228 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 22321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_pq.ko 2024-09-10T21:10:13.228 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 24537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_raid6_recov.ko 2024-09-10T21:10:13.228 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 15841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_tx.ko 2024-09-10T21:10:13.228 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 20145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_xor.ko 2024-09-10T21:10:13.229 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 20825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/raid6test.ko 2024-09-10T21:10:13.229 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 24985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/authenc.ko 2024-09-10T21:10:13.229 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 28049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/authencesn.ko 2024-09-10T21:10:13.229 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 23905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blake2b_generic.ko 2024-09-10T21:10:13.229 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 13577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blowfish_common.ko 2024-09-10T21:10:13.229 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 11697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blowfish_generic.ko 2024-09-10T21:10:13.229 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 40977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/camellia_generic.ko 2024-09-10T21:10:13.229 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 28817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast5_generic.ko 2024-09-10T21:10:13.229 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 20185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast6_generic.ko 2024-09-10T21:10:13.229 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 10193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast_common.ko 2024-09-10T21:10:13.229 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 43377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ccm.ko 2024-09-10T21:10:13.229 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 17593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cmac.ko 2024-09-10T21:10:13.230 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 11809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crc32_generic.ko 2024-09-10T21:10:13.230 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 57801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cryptd.ko 2024-09-10T21:10:13.230 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 44585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_engine.ko 2024-09-10T21:10:13.230 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 31169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_simd.ko 2024-09-10T21:10:13.230 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 30793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_user.ko 2024-09-10T21:10:13.230 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 21793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cts.ko 2024-09-10T21:10:13.230 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 12569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/des_generic.ko 2024-09-10T21:10:13.230 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 18177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/dh_generic.ko 2024-09-10T21:10:13.230 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 12945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/echainiv.ko 2024-09-10T21:10:13.230 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 30793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/essiv.ko 2024-09-10T21:10:13.230 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 19289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/fcrypt.ko 2024-09-10T21:10:13.230 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 28289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/khazad.ko 2024-09-10T21:10:13.230 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 19177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lrw.ko 2024-09-10T21:10:13.231 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 14201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lz4.ko 2024-09-10T21:10:13.231 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 14169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lz4hc.ko 2024-09-10T21:10:13.231 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 13481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/md4.ko 2024-09-10T21:10:13.231 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 11329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/michael_mic.ko 2024-09-10T21:10:13.231 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 13081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/pcbc.ko 2024-09-10T21:10:13.231 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 23161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/pcrypt.ko 2024-09-10T21:10:13.231 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 17745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/rmd160.ko 2024-09-10T21:10:13.231 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 30753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/seed.ko 2024-09-10T21:10:13.231 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 28417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/serpent_generic.ko 2024-09-10T21:10:13.231 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 169521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/tcrypt.ko 2024-09-10T21:10:13.231 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 13033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/tea.ko 2024-09-10T21:10:13.232 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 29417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/twofish_common.ko 2024-09-10T21:10:13.232 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 14393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/twofish_generic.ko 2024-09-10T21:10:13.232 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 20001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/vmac.ko 2024-09-10T21:10:13.232 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 38297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/wp512.ko 2024-09-10T21:10:13.232 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 16713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xcbc.ko 2024-09-10T21:10:13.232 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 30329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xor.ko 2024-09-10T21:10:13.232 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 23929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xts.ko 2024-09-10T21:10:13.232 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 11361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xxhash_generic.ko 2024-09-10T21:10:13.232 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ 2024-09-10T21:10:13.232 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/ 2024-09-10T21:10:13.232 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 21217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_extlog.ko 2024-09-10T21:10:13.232 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 33625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_ipmi.ko 2024-09-10T21:10:13.233 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 32593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_pad.ko 2024-09-10T21:10:13.233 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/apei/ 2024-09-10T21:10:13.233 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 44825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/apei/einj.ko 2024-09-10T21:10:13.233 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 14593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/ec_sys.ko 2024-09-10T21:10:13.233 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 19561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/platform_profile.ko 2024-09-10T21:10:13.233 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 29009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/sbs.ko 2024-09-10T21:10:13.233 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 22937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/sbshc.ko 2024-09-10T21:10:13.233 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 120321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/video.ko 2024-09-10T21:10:13.233 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ 2024-09-10T21:10:13.233 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 24585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/acard-ahci.ko 2024-09-10T21:10:13.233 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 100457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ahci.ko 2024-09-10T21:10:13.233 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 15945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ahci_platform.ko 2024-09-10T21:10:13.234 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 105105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/libahci.ko 2024-09-10T21:10:13.234 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 39913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/libahci_platform.ko 2024-09-10T21:10:13.234 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 19753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_acpi.ko 2024-09-10T21:10:13.234 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 35793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ali.ko 2024-09-10T21:10:13.234 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 38417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_amd.ko 2024-09-10T21:10:13.234 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 24801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_artop.ko 2024-09-10T21:10:13.234 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 23865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_atiixp.ko 2024-09-10T21:10:13.234 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 28873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_atp867x.ko 2024-09-10T21:10:13.234 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 20505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cmd640.ko 2024-09-10T21:10:13.234 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 29081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cmd64x.ko 2024-09-10T21:10:13.234 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 16721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cypress.ko 2024-09-10T21:10:13.234 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 20225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_efar.ko 2024-09-10T21:10:13.234 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 25297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt366.ko 2024-09-10T21:10:13.235 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 39065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt37x.ko 2024-09-10T21:10:13.235 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 27057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt3x2n.ko 2024-09-10T21:10:13.235 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 19257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt3x3.ko 2024-09-10T21:10:13.235 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 19289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_it8213.ko 2024-09-10T21:10:13.235 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 41385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_it821x.ko 2024-09-10T21:10:13.235 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 14185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_jmicron.ko 2024-09-10T21:10:13.235 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 60665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_legacy.ko 2024-09-10T21:10:13.235 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 17105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_marvell.ko 2024-09-10T21:10:13.235 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 18153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_mpiix.ko 2024-09-10T21:10:13.235 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 14225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_netcell.ko 2024-09-10T21:10:13.235 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 19065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ninja32.ko 2024-09-10T21:10:13.235 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 16849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ns87410.ko 2024-09-10T21:10:13.236 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 20417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ns87415.ko 2024-09-10T21:10:13.236 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 18513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_oldpiix.ko 2024-09-10T21:10:13.236 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 17569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_opti.ko 2024-09-10T21:10:13.236 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 23769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_optidma.ko 2024-09-10T21:10:13.236 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 35233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pcmcia.ko 2024-09-10T21:10:13.236 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 43097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pdc2027x.ko 2024-09-10T21:10:13.236 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 24025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-09-10T21:10:13.236 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 15697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_piccolo.ko 2024-09-10T21:10:13.236 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 17345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_radisys.ko 2024-09-10T21:10:13.236 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 22321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_rdc.ko 2024-09-10T21:10:13.236 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 17385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_rz1000.ko 2024-09-10T21:10:13.236 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 15529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sch.ko 2024-09-10T21:10:13.236 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 26953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_serverworks.ko 2024-09-10T21:10:13.237 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 27393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sil680.ko 2024-09-10T21:10:13.237 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 22985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sl82c105.ko 2024-09-10T21:10:13.237 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 18129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_triflex.ko 2024-09-10T21:10:13.237 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 35121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_via.ko 2024-09-10T21:10:13.237 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 26425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pdc_adma.ko 2024-09-10T21:10:13.237 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 34977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_inic162x.ko 2024-09-10T21:10:13.237 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 102273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_mv.ko 2024-09-10T21:10:13.237 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 85385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_nv.ko 2024-09-10T21:10:13.237 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 44401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_promise.ko 2024-09-10T21:10:13.237 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 26761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_qstor.ko 2024-09-10T21:10:13.237 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 34113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sil.ko 2024-09-10T21:10:13.237 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 45993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sil24.ko 2024-09-10T21:10:13.238 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 20337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sis.ko 2024-09-10T21:10:13.238 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 24345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_svw.ko 2024-09-10T21:10:13.238 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 53449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sx4.ko 2024-09-10T21:10:13.238 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 17433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_uli.ko 2024-09-10T21:10:13.238 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 41137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_via.ko 2024-09-10T21:10:13.238 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 23409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_vsc.ko 2024-09-10T21:10:13.238 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/auxdisplay/ 2024-09-10T21:10:13.238 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 18361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/auxdisplay/ks0108.ko 2024-09-10T21:10:13.238 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/bcma/ 2024-09-10T21:10:13.238 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 161225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/bcma/bcma.ko 2024-09-10T21:10:13.239 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/ 2024-09-10T21:10:13.239 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/aoe/ 2024-09-10T21:10:13.239 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 141025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/aoe/aoe.ko 2024-09-10T21:10:13.239 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/drbd/ 2024-09-10T21:10:13.239 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 1240825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/drbd/drbd.ko 2024-09-10T21:10:13.239 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 203657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/floppy.ko 2024-09-10T21:10:13.239 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/mtip32xx/ 2024-09-10T21:10:13.239 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 115057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-09-10T21:10:13.239 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 143337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/nbd.ko 2024-09-10T21:10:13.239 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/null_blk/ 2024-09-10T21:10:13.239 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 133345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/null_blk/null_blk.ko 2024-09-10T21:10:13.273 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 119929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/pktcdvd.ko 2024-09-10T21:10:13.273 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 312817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/rbd.ko 2024-09-10T21:10:13.273 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkback/ 2024-09-10T21:10:13.273 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 117753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-09-10T21:10:13.273 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/zram/ 2024-09-10T21:10:13.273 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 70201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/zram/zram.ko 2024-09-10T21:10:13.273 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ 2024-09-10T21:10:13.273 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 35289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/applicom.ko 2024-09-10T21:10:13.273 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 13457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hangcheck-timer.ko 2024-09-10T21:10:13.273 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/ 2024-09-10T21:10:13.274 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 16417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/amd-rng.ko 2024-09-10T21:10:13.274 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 21633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/intel-rng.ko 2024-09-10T21:10:13.274 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 15393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-09-10T21:10:13.274 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 12881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/via-rng.ko 2024-09-10T21:10:13.274 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 19345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/virtio-rng.ko 2024-09-10T21:10:13.274 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ 2024-09-10T21:10:13.274 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 37353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-09-10T21:10:13.274 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 183745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-09-10T21:10:13.274 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 32601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-09-10T21:10:13.274 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 179465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_si.ko 2024-09-10T21:10:13.274 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 85465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-09-10T21:10:13.274 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 65425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-09-10T21:10:13.275 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 43993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/lp.ko 2024-09-10T21:10:13.275 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/mwave/ 2024-09-10T21:10:13.275 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 103521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/mwave/mwave.ko 2024-09-10T21:10:13.275 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 31745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/nvram.ko 2024-09-10T21:10:13.275 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 39249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ppdev.ko 2024-09-10T21:10:13.275 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 59449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tlclk.ko 2024-09-10T21:10:13.275 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/ 2024-09-10T21:10:13.275 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 18977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_atmel.ko 2024-09-10T21:10:13.275 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 18265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_i2c_atmel.ko 2024-09-10T21:10:13.275 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 29513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_i2c_infineon.ko 2024-09-10T21:10:13.275 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 39281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_i2c_nuvoton.ko 2024-09-10T21:10:13.275 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 34257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_infineon.ko 2024-09-10T21:10:13.275 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 26041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_nsc.ko 2024-09-10T21:10:13.275 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 28601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/xen-tpmfront.ko 2024-09-10T21:10:13.276 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/xillybus/ 2024-09-10T21:10:13.276 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 18849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/xillybus/xillybus_class.ko 2024-09-10T21:10:13.276 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 62201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/xillybus/xillybus_core.ko 2024-09-10T21:10:13.276 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 14017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/xillybus/xillybus_pcie.ko 2024-09-10T21:10:13.276 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/clk/ 2024-09-10T21:10:13.276 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 14681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/clk/clk-palmas.ko 2024-09-10T21:10:13.276 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 57313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/clk/clk-si5351.ko 2024-09-10T21:10:13.276 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 13473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/clk/clk-twl6040.ko 2024-09-10T21:10:13.276 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 24209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/clk/clk-wm831x.ko 2024-09-10T21:10:13.276 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/ 2024-09-10T21:10:13.276 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 11673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-09-10T21:10:13.276 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 19265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/p4-clockmod.ko 2024-09-10T21:10:13.277 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 28241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/speedstep-lib.ko 2024-09-10T21:10:13.277 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ 2024-09-10T21:10:13.277 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ 2024-09-10T21:10:13.277 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 89361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-09-10T21:10:13.277 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 334505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ccp.ko 2024-09-10T21:10:13.277 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/ 2024-09-10T21:10:13.277 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/ 2024-09-10T21:10:13.277 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_common/ 2024-09-10T21:10:13.277 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 833113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-09-10T21:10:13.277 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_dh895xcc/ 2024-09-10T21:10:13.277 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 29657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-09-10T21:10:13.277 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 23273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/padlock-aes.ko 2024-09-10T21:10:13.277 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 26809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/padlock-sha.ko 2024-09-10T21:10:13.278 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/virtio/ 2024-09-10T21:10:13.278 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 90209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-09-10T21:10:13.278 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dca/ 2024-09-10T21:10:13.278 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 35681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dca/dca.ko 2024-09-10T21:10:13.278 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ 2024-09-10T21:10:13.278 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/dw/ 2024-09-10T21:10:13.278 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 18681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/dw/dw_dmac.ko 2024-09-10T21:10:13.278 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ioat/ 2024-09-10T21:10:13.278 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 147345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ioat/ioatdma.ko 2024-09-10T21:10:13.278 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/ 2024-09-10T21:10:13.278 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 90201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/amd64_edac.ko 2024-09-10T21:10:13.278 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 42073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/e752x_edac.ko 2024-09-10T21:10:13.278 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 50729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/edac_mce_amd.ko 2024-09-10T21:10:13.279 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 21097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i3000_edac.ko 2024-09-10T21:10:13.279 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 20817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i3200_edac.ko 2024-09-10T21:10:13.279 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 26817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i5100_edac.ko 2024-09-10T21:10:13.279 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 29689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i5400_edac.ko 2024-09-10T21:10:13.279 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 28481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i7300_edac.ko 2024-09-10T21:10:13.279 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 64265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i7core_edac.ko 2024-09-10T21:10:13.279 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 21369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i82975x_edac.ko 2024-09-10T21:10:13.279 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 22449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/ie31200_edac.ko 2024-09-10T21:10:13.279 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 68801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/sb_edac.ko 2024-09-10T21:10:13.279 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 20385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/x38_edac.ko 2024-09-10T21:10:13.279 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/ 2024-09-10T21:10:13.279 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 14681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-gpio.ko 2024-09-10T21:10:13.279 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 31457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-max14577.ko 2024-09-10T21:10:13.279 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 35681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-max77693.ko 2024-09-10T21:10:13.280 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 29361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-max8997.ko 2024-09-10T21:10:13.280 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 31049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-palmas.ko 2024-09-10T21:10:13.280 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 27289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-rt8973a.ko 2024-09-10T21:10:13.280 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 32073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/extcon/extcon-sm5502.ko 2024-09-10T21:10:13.280 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/ 2024-09-10T21:10:13.280 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 43049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/dmi-sysfs.ko 2024-09-10T21:10:13.280 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/efi/ 2024-09-10T21:10:13.280 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 20121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/efi/efi-pstore.ko 2024-09-10T21:10:13.280 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 26449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/iscsi_ibft.ko 2024-09-10T21:10:13.280 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/ 2024-09-10T21:10:13.280 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 14361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-adp5520.ko 2024-09-10T21:10:13.280 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 23441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-amd8111.ko 2024-09-10T21:10:13.280 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 17769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-arizona.ko 2024-09-10T21:10:13.281 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 16825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-da9052.ko 2024-09-10T21:10:13.281 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 14081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-da9055.ko 2024-09-10T21:10:13.281 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 27025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-dln2.ko 2024-09-10T21:10:13.281 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 81153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-f7188x.ko 2024-09-10T21:10:13.281 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 40553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-generic.ko 2024-09-10T21:10:13.281 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 22593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-ich.ko 2024-09-10T21:10:13.281 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 14113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-janz-ttl.ko 2024-09-10T21:10:13.281 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 16977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-kempld.ko 2024-09-10T21:10:13.281 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 14817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-lp3943.ko 2024-09-10T21:10:13.281 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 11241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-max7300.ko 2024-09-10T21:10:13.281 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 14529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-max730x.ko 2024-09-10T21:10:13.281 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 24825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-max732x.ko 2024-09-10T21:10:13.281 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 30033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-ml-ioh.ko 2024-09-10T21:10:13.282 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 56001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-pca953x.ko 2024-09-10T21:10:13.282 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 32081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-pcf857x.ko 2024-09-10T21:10:13.282 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 17337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-rdc321x.ko 2024-09-10T21:10:13.282 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 27025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-sch.ko 2024-09-10T21:10:13.282 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 23233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-sch311x.ko 2024-09-10T21:10:13.282 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 14353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-tps65912.ko 2024-09-10T21:10:13.282 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 25097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-twl4030.ko 2024-09-10T21:10:13.282 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 13201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-twl6040.ko 2024-09-10T21:10:13.282 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 24545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-viperboard.ko 2024-09-10T21:10:13.282 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 17985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-vx855.ko 2024-09-10T21:10:13.282 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 19585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-wm831x.ko 2024-09-10T21:10:13.282 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 13745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-wm8350.ko 2024-09-10T21:10:13.282 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 20489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-wm8994.ko 2024-09-10T21:10:13.282 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/ 2024-09-10T21:10:13.283 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 13721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-a4tech.ko 2024-09-10T21:10:13.283 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 44473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-apple.ko 2024-09-10T21:10:13.283 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 16969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-appleir.ko 2024-09-10T21:10:13.283 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 9593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-aureal.ko 2024-09-10T21:10:13.283 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 17193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-axff.ko 2024-09-10T21:10:13.283 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 12833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-belkin.ko 2024-09-10T21:10:13.283 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 11913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-cherry.ko 2024-09-10T21:10:13.283 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 19225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-chicony.ko 2024-09-10T21:10:13.283 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 74497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-cp2112.ko 2024-09-10T21:10:13.283 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 13017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-cypress.ko 2024-09-10T21:10:13.283 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 19041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-dr.ko 2024-09-10T21:10:13.283 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 12313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-elecom.ko 2024-09-10T21:10:13.283 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 20665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-elo.ko 2024-09-10T21:10:13.284 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 15345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-emsff.ko 2024-09-10T21:10:13.284 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 11865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-ezkey.ko 2024-09-10T21:10:13.284 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 16361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gaff.ko 2024-09-10T21:10:13.284 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 10769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-generic.ko 2024-09-10T21:10:13.284 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 19881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gt683r.ko 2024-09-10T21:10:13.284 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 14865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gyration.ko 2024-09-10T21:10:13.284 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 12281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtek-kbd.ko 2024-09-10T21:10:13.284 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 12121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtek-mouse.ko 2024-09-10T21:10:13.284 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 16521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtekff.ko 2024-09-10T21:10:13.284 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 22993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-hyperv.ko 2024-09-10T21:10:13.284 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 11977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-icade.ko 2024-09-10T21:10:13.284 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 10593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-kensington.ko 2024-09-10T21:10:13.284 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 9729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-keytouch.ko 2024-09-10T21:10:13.285 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 19617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-kye.ko 2024-09-10T21:10:13.285 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 12401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lcpower.ko 2024-09-10T21:10:13.285 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 25049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-led.ko 2024-09-10T21:10:13.285 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 54337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lenovo.ko 2024-09-10T21:10:13.285 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 37753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lg-g15.ko 2024-09-10T21:10:13.285 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 67841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech-dj.ko 2024-09-10T21:10:13.285 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 122145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-09-10T21:10:13.285 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 98177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech.ko 2024-09-10T21:10:13.285 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 36897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-magicmouse.ko 2024-09-10T21:10:13.285 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 25969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-microsoft.ko 2024-09-10T21:10:13.285 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 12025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-monterey.ko 2024-09-10T21:10:13.285 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 60505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-multitouch.ko 2024-09-10T21:10:13.285 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 39905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-ntrig.ko 2024-09-10T21:10:13.286 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 10185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-ortek.ko 2024-09-10T21:10:13.286 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 10265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-penmount.ko 2024-09-10T21:10:13.286 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 14481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-petalynx.ko 2024-09-10T21:10:13.286 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 91217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-picolcd.ko 2024-09-10T21:10:13.286 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 16073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-pl.ko 2024-09-10T21:10:13.286 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 16897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-plantronics.ko 2024-09-10T21:10:13.286 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 9617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-primax.ko 2024-09-10T21:10:13.286 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 44329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-rmi.ko 2024-09-10T21:10:13.286 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 25273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-arvo.ko 2024-09-10T21:10:13.286 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 15657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-common.ko 2024-09-10T21:10:13.286 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 39009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-isku.ko 2024-09-10T21:10:13.286 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 36377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-kone.ko 2024-09-10T21:10:13.286 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 38169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-09-10T21:10:13.287 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 26193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-konepure.ko 2024-09-10T21:10:13.287 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 35721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-09-10T21:10:13.287 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 15577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-lua.ko 2024-09-10T21:10:13.287 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 35497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-pyra.ko 2024-09-10T21:10:13.287 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 35657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-ryos.ko 2024-09-10T21:10:13.287 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 24089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-savu.ko 2024-09-10T21:10:13.287 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 27625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat.ko 2024-09-10T21:10:13.287 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 15921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-saitek.ko 2024-09-10T21:10:13.287 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 42505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-samsung.ko 2024-09-10T21:10:13.287 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 42609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sensor-hub.ko 2024-09-10T21:10:13.287 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 15353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sjoy.ko 2024-09-10T21:10:13.287 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 62345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sony.ko 2024-09-10T21:10:13.287 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 9553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-speedlink.ko 2024-09-10T21:10:13.287 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 30241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-steelseries.ko 2024-09-10T21:10:13.288 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 11385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sunplus.ko 2024-09-10T21:10:13.288 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 19393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-thrustmaster.ko 2024-09-10T21:10:13.288 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 12057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-tivo.ko 2024-09-10T21:10:13.288 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 16889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-tmff.ko 2024-09-10T21:10:13.288 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 16737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-topseed.ko 2024-09-10T21:10:13.288 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 19409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-twinhan.ko 2024-09-10T21:10:13.288 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 81673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-uclogic.ko 2024-09-10T21:10:13.288 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 12769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-waltop.ko 2024-09-10T21:10:13.288 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 184865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-wiimote.ko 2024-09-10T21:10:13.288 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 9385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-xinmo.ko 2024-09-10T21:10:13.288 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 15169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-zpff.ko 2024-09-10T21:10:13.288 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 19401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-zydacron.ko 2024-09-10T21:10:13.288 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 462849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid.ko 2024-09-10T21:10:13.288 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 39761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/uhid.ko 2024-09-10T21:10:13.289 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/usbhid/ 2024-09-10T21:10:13.289 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 153953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/usbhid/usbhid.ko 2024-09-10T21:10:13.289 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 22457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/usbhid/usbkbd.ko 2024-09-10T21:10:13.289 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 17553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/usbhid/usbmouse.ko 2024-09-10T21:10:13.289 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 261241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/wacom.ko 2024-09-10T21:10:13.289 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/ 2024-09-10T21:10:13.289 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 76441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_balloon.ko 2024-09-10T21:10:13.289 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 96073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_utils.ko 2024-09-10T21:10:13.289 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 378337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_vmbus.ko 2024-09-10T21:10:13.289 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ 2024-09-10T21:10:13.289 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 77217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/abituguru.ko 2024-09-10T21:10:13.289 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 67097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/abituguru3.ko 2024-09-10T21:10:13.289 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 43273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/acpi_power_meter.ko 2024-09-10T21:10:13.290 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 20377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ad7414.ko 2024-09-10T21:10:13.290 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 22105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ad7418.ko 2024-09-10T21:10:13.290 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 29121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adc128d818.ko 2024-09-10T21:10:13.290 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 38185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1025.ko 2024-09-10T21:10:13.290 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 115201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1026.ko 2024-09-10T21:10:13.290 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 23401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1029.ko 2024-09-10T21:10:13.290 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 55225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1031.ko 2024-09-10T21:10:13.290 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 31529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm9240.ko 2024-09-10T21:10:13.290 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 16249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ads7828.ko 2024-09-10T21:10:13.290 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 12337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7410.ko 2024-09-10T21:10:13.290 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 26593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7411.ko 2024-09-10T21:10:13.290 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 89113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7462.ko 2024-09-10T21:10:13.290 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 49233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7470.ko 2024-09-10T21:10:13.291 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 81665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7475.ko 2024-09-10T21:10:13.291 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 22681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7x10.ko 2024-09-10T21:10:13.291 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 34417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/amc6821.ko 2024-09-10T21:10:13.291 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 68953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/applesmc.ko 2024-09-10T21:10:13.291 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 60281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asb100.ko 2024-09-10T21:10:13.291 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 69097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asc7621.ko 2024-09-10T21:10:13.291 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 48145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asus_atk0110.ko 2024-09-10T21:10:13.291 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 20409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/atxp1.ko 2024-09-10T21:10:13.291 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 33761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/coretemp.ko 2024-09-10T21:10:13.291 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 14201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/cros_ec_hwmon.ko 2024-09-10T21:10:13.291 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 29689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/da9052-hwmon.ko 2024-09-10T21:10:13.291 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 19889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/da9055-hwmon.ko 2024-09-10T21:10:13.291 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 57681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-09-10T21:10:13.292 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 113265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/dme1737.ko 2024-09-10T21:10:13.292 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 22753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ds1621.ko 2024-09-10T21:10:13.292 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 17577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ds620.ko 2024-09-10T21:10:13.292 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 26945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc1403.ko 2024-09-10T21:10:13.292 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 40265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc2103.ko 2024-09-10T21:10:13.292 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 34729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc6w201.ko 2024-09-10T21:10:13.292 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 79009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f71805f.ko 2024-09-10T21:10:13.292 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 109049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f71882fg.ko 2024-09-10T21:10:13.292 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 50769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f75375s.ko 2024-09-10T21:10:13.292 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 25449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/fam15h_power.ko 2024-09-10T21:10:13.292 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 65849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/fschmd.ko 2024-09-10T21:10:13.292 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 17289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/g760a.ko 2024-09-10T21:10:13.292 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 30889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/g762.ko 2024-09-10T21:10:13.293 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 59849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/gl518sm.ko 2024-09-10T21:10:13.293 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 56393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/gl520sm.ko 2024-09-10T21:10:13.293 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 15561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/hih6130.ko 2024-09-10T21:10:13.293 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 11865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/hwmon-vid.ko 2024-09-10T21:10:13.293 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 13225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/i5500_temp.ko 2024-09-10T21:10:13.293 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 27385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/i5k_amb.ko 2024-09-10T21:10:13.293 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 42681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ibmaem.ko 2024-09-10T21:10:13.293 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 24697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ibmpex.ko 2024-09-10T21:10:13.293 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 28497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ina209.ko 2024-09-10T21:10:13.293 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 32729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ina2xx.ko 2024-09-10T21:10:13.293 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 150681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/it87.ko 2024-09-10T21:10:13.293 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 23281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/jc42.ko 2024-09-10T21:10:13.294 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 28657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/k10temp.ko 2024-09-10T21:10:13.294 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 15817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/k8temp.ko 2024-09-10T21:10:13.294 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 21801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lineage-pem.ko 2024-09-10T21:10:13.294 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 57769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm63.ko 2024-09-10T21:10:13.294 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 19865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm73.ko 2024-09-10T21:10:13.294 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 44185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm75.ko 2024-09-10T21:10:13.294 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 23329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm77.ko 2024-09-10T21:10:13.294 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 41537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm78.ko 2024-09-10T21:10:13.294 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 37129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm80.ko 2024-09-10T21:10:13.294 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 20609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm83.ko 2024-09-10T21:10:13.294 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 86561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm85.ko 2024-09-10T21:10:13.294 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 55721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm87.ko 2024-09-10T21:10:13.294 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 73649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm90.ko 2024-09-10T21:10:13.295 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 23473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm92.ko 2024-09-10T21:10:13.295 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 131585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm93.ko 2024-09-10T21:10:13.295 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 40369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95234.ko 2024-09-10T21:10:13.295 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 19225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95241.ko 2024-09-10T21:10:13.295 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 21777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95245.ko 2024-09-10T21:10:13.295 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 26409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc2945.ko 2024-09-10T21:10:13.295 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 15569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4151.ko 2024-09-10T21:10:13.295 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 17881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4215.ko 2024-09-10T21:10:13.295 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 16657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4222.ko 2024-09-10T21:10:13.295 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 18457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4245.ko 2024-09-10T21:10:13.295 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 14993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4260.ko 2024-09-10T21:10:13.295 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 17201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4261.ko 2024-09-10T21:10:13.295 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 36857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max16065.ko 2024-09-10T21:10:13.296 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 21673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max1619.ko 2024-09-10T21:10:13.296 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 28321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max1668.ko 2024-09-10T21:10:13.296 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 22841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max197.ko 2024-09-10T21:10:13.296 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 24097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6639.ko 2024-09-10T21:10:13.296 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 27385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6650.ko 2024-09-10T21:10:13.296 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 39777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6697.ko 2024-09-10T21:10:13.296 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 21601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/mc13783-adc.ko 2024-09-10T21:10:13.296 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 12417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/mcp3021.ko 2024-09-10T21:10:13.296 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 17961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/menf21bmc_hwmon.ko 2024-09-10T21:10:13.296 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 56825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6683.ko 2024-09-10T21:10:13.296 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 140657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6775-core.ko 2024-09-10T21:10:13.296 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 72761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6775.ko 2024-09-10T21:10:13.296 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 67865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct7802.ko 2024-09-10T21:10:13.297 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 97865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pc87360.ko 2024-09-10T21:10:13.297 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 62009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pc87427.ko 2024-09-10T21:10:13.297 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 21905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pcf8591.ko 2024-09-10T21:10:13.297 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ 2024-09-10T21:10:13.297 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 25273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-09-10T21:10:13.297 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 19425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-09-10T21:10:13.297 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 33345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-09-10T21:10:13.297 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 10881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max16064.ko 2024-09-10T21:10:13.297 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 19921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max34440.ko 2024-09-10T21:10:13.297 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 12049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max8688.ko 2024-09-10T21:10:13.297 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 17673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-09-10T21:10:13.297 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 98961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-09-10T21:10:13.297 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 9425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-09-10T21:10:13.298 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 36265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-09-10T21:10:13.298 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 16961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-09-10T21:10:13.298 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 24529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-09-10T21:10:13.298 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 15417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/powr1220.ko 2024-09-10T21:10:13.298 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 25521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch5627.ko 2024-09-10T21:10:13.298 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 33033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch5636.ko 2024-09-10T21:10:13.298 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 33665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch56xx-common.ko 2024-09-10T21:10:13.298 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 40937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sht15.ko 2024-09-10T21:10:13.298 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 16065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sht21.ko 2024-09-10T21:10:13.298 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 19337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/shtc1.ko 2024-09-10T21:10:13.298 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 48793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sis5595.ko 2024-09-10T21:10:13.298 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 22489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47b397.ko 2024-09-10T21:10:13.298 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 46561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47m1.ko 2024-09-10T21:10:13.298 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 44401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47m192.ko 2024-09-10T21:10:13.299 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 31489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/thmc50.ko 2024-09-10T21:10:13.299 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 20177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp102.ko 2024-09-10T21:10:13.299 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 16529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp103.ko 2024-09-10T21:10:13.299 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 25185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp401.ko 2024-09-10T21:10:13.299 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 23097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp421.ko 2024-09-10T21:10:13.299 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 23129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/via-cputemp.ko 2024-09-10T21:10:13.299 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 50297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/via686a.ko 2024-09-10T21:10:13.299 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 59745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/vt1211.ko 2024-09-10T21:10:13.299 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 62817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/vt8231.ko 2024-09-10T21:10:13.299 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 71193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83627ehf.ko 2024-09-10T21:10:13.299 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 96665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83627hf.ko 2024-09-10T21:10:13.299 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 85977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83781d.ko 2024-09-10T21:10:13.300 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 75217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83791d.ko 2024-09-10T21:10:13.300 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 83505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83792d.ko 2024-09-10T21:10:13.300 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 116873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83793.ko 2024-09-10T21:10:13.300 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 134513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83795.ko 2024-09-10T21:10:13.300 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 20209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83l785ts.ko 2024-09-10T21:10:13.300 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 43473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83l786ng.ko 2024-09-10T21:10:13.300 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 15569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/wm831x-hwmon.ko 2024-09-10T21:10:13.300 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 12081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/wm8350-hwmon.ko 2024-09-10T21:10:13.300 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/ 2024-09-10T21:10:13.300 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/ 2024-09-10T21:10:13.300 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 26977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-09-10T21:10:13.300 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 25041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-09-10T21:10:13.300 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/ 2024-09-10T21:10:13.301 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 33225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ali1535.ko 2024-09-10T21:10:13.301 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 35073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ali1563.ko 2024-09-10T21:10:13.301 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 32921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ali15x3.ko 2024-09-10T21:10:13.301 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 21473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-09-10T21:10:13.301 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 30217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-09-10T21:10:13.301 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 19409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-09-10T21:10:13.301 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 17825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-cbus-gpio.ko 2024-09-10T21:10:13.301 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 7697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-09-10T21:10:13.301 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 15929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-cros-ec-tunnel.ko 2024-09-10T21:10:13.301 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 63537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-09-10T21:10:13.301 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 28201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-09-10T21:10:13.301 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 24569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-09-10T21:10:13.301 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 20001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-09-10T21:10:13.302 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 14489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-dln2.ko 2024-09-10T21:10:13.302 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 20345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-gpio.ko 2024-09-10T21:10:13.302 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 78577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-i801.ko 2024-09-10T21:10:13.302 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 24793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-isch.ko 2024-09-10T21:10:13.302 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 52945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-09-10T21:10:13.302 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 20969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-kempld.ko 2024-09-10T21:10:13.302 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 21409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-09-10T21:10:13.302 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 25809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-09-10T21:10:13.302 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 33001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ocores.ko 2024-09-10T21:10:13.302 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 30145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-parport.ko 2024-09-10T21:10:13.302 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 20137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-09-10T21:10:13.302 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 57137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-09-10T21:10:13.303 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 17521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-robotfuzz-osif.ko 2024-09-10T21:10:13.303 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 21241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-09-10T21:10:13.303 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 14057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-09-10T21:10:13.303 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 24793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-sis5595.ko 2024-09-10T21:10:13.303 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 32673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-sis630.ko 2024-09-10T21:10:13.303 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 22697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-09-10T21:10:13.303 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 22921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-taos-evm.ko 2024-09-10T21:10:13.303 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 22249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-09-10T21:10:13.303 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 15913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-via.ko 2024-09-10T21:10:13.303 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 30073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-09-10T21:10:13.303 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 21353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-09-10T21:10:13.303 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 59145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-xiic.ko 2024-09-10T21:10:13.303 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 27185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-mux.ko 2024-09-10T21:10:13.303 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 22913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-smbus.ko 2024-09-10T21:10:13.304 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 29665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-stub.ko 2024-09-10T21:10:13.304 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/muxes/ 2024-09-10T21:10:13.304 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 18641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/muxes/i2c-mux-gpio.ko 2024-09-10T21:10:13.304 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 16081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/muxes/i2c-mux-pca9541.ko 2024-09-10T21:10:13.304 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 36729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/muxes/i2c-mux-pca954x.ko 2024-09-10T21:10:13.304 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ 2024-09-10T21:10:13.304 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ 2024-09-10T21:10:13.304 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 328633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_cm.ko 2024-09-10T21:10:13.304 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 1107329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_core.ko 2024-09-10T21:10:13.304 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 84409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_umad.ko 2024-09-10T21:10:13.304 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 404889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-09-10T21:10:13.304 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 120105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/iw_cm.ko 2024-09-10T21:10:13.304 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 343753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/rdma_cm.ko 2024-09-10T21:10:13.305 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 85057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-09-10T21:10:13.305 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/ 2024-09-10T21:10:13.305 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/cxgb4/ 2024-09-10T21:10:13.305 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 591177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-09-10T21:10:13.305 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx4/ 2024-09-10T21:10:13.305 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 515585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-09-10T21:10:13.305 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx5/ 2024-09-10T21:10:13.305 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 919921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-09-10T21:10:13.305 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mthca/ 2024-09-10T21:10:13.305 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 375761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-09-10T21:10:13.327 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/ocrdma/ 2024-09-10T21:10:13.327 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 203241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/ocrdma/ocrdma.ko 2024-09-10T21:10:13.328 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/usnic/ 2024-09-10T21:10:13.328 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 212353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/usnic/usnic_verbs.ko 2024-09-10T21:10:13.328 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ 2024-09-10T21:10:13.328 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ipoib/ 2024-09-10T21:10:13.328 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 334665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-09-10T21:10:13.328 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/iser/ 2024-09-10T21:10:13.328 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 120785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-09-10T21:10:13.328 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/isert/ 2024-09-10T21:10:13.328 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 132457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-09-10T21:10:13.328 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srp/ 2024-09-10T21:10:13.328 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 158257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-09-10T21:10:13.328 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srpt/ 2024-09-10T21:10:13.329 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 157209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-09-10T21:10:13.329 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/ 2024-09-10T21:10:13.329 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 12297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/evbug.ko 2024-09-10T21:10:13.329 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 27465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/ff-memless.ko 2024-09-10T21:10:13.329 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/ 2024-09-10T21:10:13.329 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 13497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-09-10T21:10:13.329 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 14081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/fm801-gp.ko 2024-09-10T21:10:13.329 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 49769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/gameport.ko 2024-09-10T21:10:13.329 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 49697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/joydev.ko 2024-09-10T21:10:13.329 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/ 2024-09-10T21:10:13.329 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 15617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/adp5520-keys.ko 2024-09-10T21:10:13.329 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 36289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/adp5588-keys.ko 2024-09-10T21:10:13.330 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 35753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/adp5589-keys.ko 2024-09-10T21:10:13.330 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 28393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/cros_ec_keyb.ko 2024-09-10T21:10:13.330 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 45537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/gpio_keys.ko 2024-09-10T21:10:13.330 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 22473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/gpio_keys_polled.ko 2024-09-10T21:10:13.330 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 25713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/lkkbd.ko 2024-09-10T21:10:13.330 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 31945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/lm8323.ko 2024-09-10T21:10:13.330 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 16849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/lm8333.ko 2024-09-10T21:10:13.330 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 25913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/matrix_keypad.ko 2024-09-10T21:10:13.330 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 21545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/max7359_keypad.ko 2024-09-10T21:10:13.330 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 19337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/mcs_touchkey.ko 2024-09-10T21:10:13.330 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 19201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/mpr121_touchkey.ko 2024-09-10T21:10:13.330 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 13377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/newtonkbd.ko 2024-09-10T21:10:13.331 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 12321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/opencores-kbd.ko 2024-09-10T21:10:13.331 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 17185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/qt1070.ko 2024-09-10T21:10:13.331 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 21649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/qt2160.ko 2024-09-10T21:10:13.331 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 28769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/samsung-keypad.ko 2024-09-10T21:10:13.331 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 13217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/stowaway.ko 2024-09-10T21:10:13.331 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 22433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/sunkbd.ko 2024-09-10T21:10:13.331 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 20737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/tca6416-keypad.ko 2024-09-10T21:10:13.331 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 19009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/tca8418_keypad.ko 2024-09-10T21:10:13.331 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 19657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/twl4030_keypad.ko 2024-09-10T21:10:13.331 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 13553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/xtkbd.ko 2024-09-10T21:10:13.331 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 13017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/matrix-keymap.ko 2024-09-10T21:10:13.331 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/ 2024-09-10T21:10:13.331 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 40713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/appletouch.ko 2024-09-10T21:10:13.332 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 39657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/bcm5974.ko 2024-09-10T21:10:13.332 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 160569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/cyapatp.ko 2024-09-10T21:10:13.332 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 114801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/elan_i2c.ko 2024-09-10T21:10:13.332 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 16065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/gpio_mouse.ko 2024-09-10T21:10:13.332 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 430033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/psmouse.ko 2024-09-10T21:10:13.332 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 17025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/sermouse.ko 2024-09-10T21:10:13.332 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 27961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-09-10T21:10:13.332 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 27281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/synaptics_usb.ko 2024-09-10T21:10:13.332 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 18897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/vsxxxaa.ko 2024-09-10T21:10:13.332 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/rmi4/ 2024-09-10T21:10:13.332 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 139497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/rmi4/rmi_core.ko 2024-09-10T21:10:13.332 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/ 2024-09-10T21:10:13.332 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 13809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/altera_ps2.ko 2024-09-10T21:10:13.333 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 18961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/arc_ps2.ko 2024-09-10T21:10:13.333 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 20481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/ct82c710.ko 2024-09-10T21:10:13.333 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 20377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-09-10T21:10:13.333 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 18481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/parkbd.ko 2024-09-10T21:10:13.333 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 15633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/pcips2.ko 2024-09-10T21:10:13.333 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 23873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/ps2mult.ko 2024-09-10T21:10:13.333 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 29001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/serio_raw.ko 2024-09-10T21:10:13.333 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 21817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/serport.ko 2024-09-10T21:10:13.333 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 15665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/sparse-keymap.ko 2024-09-10T21:10:13.333 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/ 2024-09-10T21:10:13.333 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 15817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-88pm860x.ko 2024-09-10T21:10:13.333 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 13249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-adp5520.ko 2024-09-10T21:10:13.334 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 77345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-bd2802.ko 2024-09-10T21:10:13.334 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 31177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-blinkm.ko 2024-09-10T21:10:13.334 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 11985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-da903x.ko 2024-09-10T21:10:13.334 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 14817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-da9052.ko 2024-09-10T21:10:13.334 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 20169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-gpio.ko 2024-09-10T21:10:13.334 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 18801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lm3530.ko 2024-09-10T21:10:13.334 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 36793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lm3533.ko 2024-09-10T21:10:13.334 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 22889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lm355x.ko 2024-09-10T21:10:13.334 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 23921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lm3642.ko 2024-09-10T21:10:13.334 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 18849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lp3944.ko 2024-09-10T21:10:13.334 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 12897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lp8788.ko 2024-09-10T21:10:13.334 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 18129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-max8997.ko 2024-09-10T21:10:13.334 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 14257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-mc13783.ko 2024-09-10T21:10:13.335 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 14705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-menf21bmc.ko 2024-09-10T21:10:13.335 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 26065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-pca9532.ko 2024-09-10T21:10:13.335 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 23897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-pca955x.ko 2024-09-10T21:10:13.335 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 22065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-pca963x.ko 2024-09-10T21:10:13.335 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 38249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-ss4200.ko 2024-09-10T21:10:13.335 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 28337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-tca6507.ko 2024-09-10T21:10:13.335 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 18265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-wm831x-status.ko 2024-09-10T21:10:13.335 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 13721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-wm8350.ko 2024-09-10T21:10:13.335 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ 2024-09-10T21:10:13.335 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 12745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-09-10T21:10:13.335 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 9937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-09-10T21:10:13.335 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 8721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-09-10T21:10:13.335 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 15169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-09-10T21:10:13.336 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 16697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-09-10T21:10:13.336 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 16865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-09-10T21:10:13.336 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 13897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-09-10T21:10:13.336 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 16809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-09-10T21:10:13.336 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/ 2024-09-10T21:10:13.336 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/bcache/ 2024-09-10T21:10:13.336 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 776777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/bcache/bcache.ko 2024-09-10T21:10:13.336 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 46361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-bio-prison.ko 2024-09-10T21:10:13.336 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 113977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-bufio.ko 2024-09-10T21:10:13.336 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 58393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-cache-smq.ko 2024-09-10T21:10:13.336 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 197121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-cache.ko 2024-09-10T21:10:13.336 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 137841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-crypt.ko 2024-09-10T21:10:13.337 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 28761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-delay.ko 2024-09-10T21:10:13.337 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 67153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-era.ko 2024-09-10T21:10:13.337 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 33121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-flakey.ko 2024-09-10T21:10:13.337 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 50665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-log-userspace.ko 2024-09-10T21:10:13.337 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 40713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-log.ko 2024-09-10T21:10:13.337 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 61073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-mirror.ko 2024-09-10T21:10:13.337 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 96937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-multipath.ko 2024-09-10T21:10:13.337 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 19905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-queue-length.ko 2024-09-10T21:10:13.337 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 86937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-raid.ko 2024-09-10T21:10:13.337 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 42121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-region-hash.ko 2024-09-10T21:10:13.337 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 19089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-round-robin.ko 2024-09-10T21:10:13.337 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 20193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-service-time.ko 2024-09-10T21:10:13.337 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 139257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-snapshot.ko 2024-09-10T21:10:13.338 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 23033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-switch.ko 2024-09-10T21:10:13.338 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 234353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-thin-pool.ko 2024-09-10T21:10:13.338 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 64945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-verity.ko 2024-09-10T21:10:13.338 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 10073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-zero.ko 2024-09-10T21:10:13.338 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/persistent-data/ 2024-09-10T21:10:13.338 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 257025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-09-10T21:10:13.338 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 43417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid0.ko 2024-09-10T21:10:13.338 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 124833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid1.ko 2024-09-10T21:10:13.338 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 157401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid10.ko 2024-09-10T21:10:13.338 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 443385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid456.ko 2024-09-10T21:10:13.338 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/ 2024-09-10T21:10:13.338 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/ 2024-09-10T21:10:13.339 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 211273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptbase.ko 2024-09-10T21:10:13.339 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 78081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptctl.ko 2024-09-10T21:10:13.339 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 54889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptfc.ko 2024-09-10T21:10:13.339 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 44793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptlan.ko 2024-09-10T21:10:13.339 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 150977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptsas.ko 2024-09-10T21:10:13.339 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 94401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptscsih.ko 2024-09-10T21:10:13.339 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 64033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptspi.ko 2024-09-10T21:10:13.339 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/ 2024-09-10T21:10:13.339 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 20065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/88pm800.ko 2024-09-10T21:10:13.339 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 14769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/88pm805.ko 2024-09-10T21:10:13.339 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 12465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/88pm80x.ko 2024-09-10T21:10:13.339 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 13049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/arizona-i2c.ko 2024-09-10T21:10:13.339 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 146569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/arizona.ko 2024-09-10T21:10:13.340 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 13305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/bcm590xx.ko 2024-09-10T21:10:13.340 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 20961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/cros_ec_dev.ko 2024-09-10T21:10:13.340 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 40657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/dln2.ko 2024-09-10T21:10:13.340 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 18689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/janz-cmodio.ko 2024-09-10T21:10:13.340 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 54473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/kempld-core.ko 2024-09-10T21:10:13.340 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 31409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lm3533-core.ko 2024-09-10T21:10:13.340 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 17057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lm3533-ctrlbank.ko 2024-09-10T21:10:13.340 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 14305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lp3943.ko 2024-09-10T21:10:13.340 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 60577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lpc_ich.ko 2024-09-10T21:10:13.340 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 13697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lpc_sch.ko 2024-09-10T21:10:13.340 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 21137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/max8907.ko 2024-09-10T21:10:13.340 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 33817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/mc13xxx-core.ko 2024-09-10T21:10:13.340 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 13025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/mc13xxx-i2c.ko 2024-09-10T21:10:13.341 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 13521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/menf21bmc.ko 2024-09-10T21:10:13.341 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 20049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/pcf50633-adc.ko 2024-09-10T21:10:13.341 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 10001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/pcf50633-gpio.ko 2024-09-10T21:10:13.341 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 41041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/pcf50633.ko 2024-09-10T21:10:13.341 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 11265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/rdc321x-southbridge.ko 2024-09-10T21:10:13.341 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 22633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/retu-mfd.ko 2024-09-10T21:10:13.341 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 95625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/si476x-core.ko 2024-09-10T21:10:13.341 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 57473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/sm501.ko 2024-09-10T21:10:13.341 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 59801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/tps65010.ko 2024-09-10T21:10:13.341 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 11873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/tps6507x.ko 2024-09-10T21:10:13.341 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 15049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/viperboard.ko 2024-09-10T21:10:13.341 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 11833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/vx855.ko 2024-09-10T21:10:13.342 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 21225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/wl1273-core.ko 2024-09-10T21:10:13.342 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 51201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/wm8994.ko 2024-09-10T21:10:13.342 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ 2024-09-10T21:10:13.342 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 16025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ad525x_dpot-i2c.ko 2024-09-10T21:10:13.342 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 63993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ad525x_dpot.ko 2024-09-10T21:10:13.342 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/altera-stapl/ 2024-09-10T21:10:13.342 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 57393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-09-10T21:10:13.342 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 22289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/apds9802als.ko 2024-09-10T21:10:13.342 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 47785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/apds990x.ko 2024-09-10T21:10:13.342 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 58785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/bh1770glc.ko 2024-09-10T21:10:13.342 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/c2port/ 2024-09-10T21:10:13.342 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 12905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/c2port/c2port-duramar2150.ko 2024-09-10T21:10:13.343 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 47313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/c2port/core.ko 2024-09-10T21:10:13.343 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/cb710/ 2024-09-10T21:10:13.343 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 29225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/cb710/cb710.ko 2024-09-10T21:10:13.343 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 21705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ds1682.ko 2024-09-10T21:10:13.343 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 10233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/dummy-irq.ko 2024-09-10T21:10:13.343 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/echo/ 2024-09-10T21:10:13.343 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 15193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/echo/echo.ko 2024-09-10T21:10:13.343 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/ 2024-09-10T21:10:13.343 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 44177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/at24.ko 2024-09-10T21:10:13.343 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 18145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-09-10T21:10:13.343 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 15705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/max6875.ko 2024-09-10T21:10:13.343 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 40305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/enclosure.ko 2024-09-10T21:10:13.343 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/genwqe/ 2024-09-10T21:10:13.344 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 191761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/genwqe/genwqe_card.ko 2024-09-10T21:10:13.344 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 16625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/hmc6352.ko 2024-09-10T21:10:13.344 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 37857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/hpilo.ko 2024-09-10T21:10:13.344 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ibmasm/ 2024-09-10T21:10:13.344 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 92489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ibmasm/ibmasm.ko 2024-09-10T21:10:13.344 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 24057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ics932s401.ko 2024-09-10T21:10:13.344 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 26833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/isl29003.ko 2024-09-10T21:10:13.344 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 18697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/isl29020.ko 2024-09-10T21:10:13.344 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/ 2024-09-10T21:10:13.344 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 53961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-09-10T21:10:13.344 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 17905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-09-10T21:10:13.345 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/ 2024-09-10T21:10:13.345 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 153777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei-me.ko 2024-09-10T21:10:13.345 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 69897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei-txe.ko 2024-09-10T21:10:13.345 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 381905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei.ko 2024-09-10T21:10:13.345 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 38049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/phantom.ko 2024-09-10T21:10:13.345 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ti-st/ 2024-09-10T21:10:13.345 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 90881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/ti-st/st_drv.ko 2024-09-10T21:10:13.345 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 28945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tifm_7xx1.ko 2024-09-10T21:10:13.345 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 33257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tifm_core.ko 2024-09-10T21:10:13.345 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 22329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tsl2550.ko 2024-09-10T21:10:13.345 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 56809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_balloon.ko 2024-09-10T21:10:13.345 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_vmci/ 2024-09-10T21:10:13.345 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 232257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-09-10T21:10:13.346 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ 2024-09-10T21:10:13.346 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/bonding/ 2024-09-10T21:10:13.346 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 623361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/bonding/bonding.ko 2024-09-10T21:10:13.346 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/ 2024-09-10T21:10:13.346 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/b53/ 2024-09-10T21:10:13.346 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 133161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/b53/b53_common.ko 2024-09-10T21:10:13.346 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 49641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/bcm-sf2.ko 2024-09-10T21:10:13.346 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 19001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6060.ko 2024-09-10T21:10:13.346 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6xxx/ 2024-09-10T21:10:13.346 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 592833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-09-10T21:10:13.346 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 19697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dummy.ko 2024-09-10T21:10:13.346 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 21465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/eql.ko 2024-09-10T21:10:13.346 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ 2024-09-10T21:10:13.346 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/ 2024-09-10T21:10:13.347 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 53841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-09-10T21:10:13.347 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 49281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-09-10T21:10:13.347 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 158537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-09-10T21:10:13.347 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 86233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-09-10T21:10:13.347 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/ 2024-09-10T21:10:13.347 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 43097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/8390.ko 2024-09-10T21:10:13.347 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 73953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-09-10T21:10:13.347 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 33457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-09-10T21:10:13.347 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 109673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-09-10T21:10:13.347 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/adaptec/ 2024-09-10T21:10:13.347 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 64817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-09-10T21:10:13.347 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/agere/ 2024-09-10T21:10:13.347 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 84433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/agere/et131x.ko 2024-09-10T21:10:13.347 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/alteon/ 2024-09-10T21:10:13.347 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 70897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-09-10T21:10:13.348 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/altera/ 2024-09-10T21:10:13.348 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 52713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-09-10T21:10:13.348 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/ 2024-09-10T21:10:13.348 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 65945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-09-10T21:10:13.348 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 56145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-09-10T21:10:13.348 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 116105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-09-10T21:10:13.348 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/ 2024-09-10T21:10:13.348 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/alx/ 2024-09-10T21:10:13.348 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 124721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-09-10T21:10:13.348 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1c/ 2024-09-10T21:10:13.348 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 128625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-09-10T21:10:13.348 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1e/ 2024-09-10T21:10:13.348 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 110505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-09-10T21:10:13.348 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/ 2024-09-10T21:10:13.349 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 102017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-09-10T21:10:13.349 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 89121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-09-10T21:10:13.349 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/ 2024-09-10T21:10:13.349 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 106593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-09-10T21:10:13.349 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 229409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-09-10T21:10:13.349 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2x/ 2024-09-10T21:10:13.349 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 1734265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-09-10T21:10:13.349 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 166913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-09-10T21:10:13.349 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/genet/ 2024-09-10T21:10:13.349 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 164665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-09-10T21:10:13.349 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 439633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-09-10T21:10:13.349 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/ 2024-09-10T21:10:13.349 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/bna/ 2024-09-10T21:10:13.349 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 432417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-09-10T21:10:13.350 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/ 2024-09-10T21:10:13.350 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb/ 2024-09-10T21:10:13.350 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 257433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-09-10T21:10:13.350 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb3/ 2024-09-10T21:10:13.350 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 444721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-09-10T21:10:13.350 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4/ 2024-09-10T21:10:13.350 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 1309441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-09-10T21:10:13.350 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4vf/ 2024-09-10T21:10:13.350 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 207073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-09-10T21:10:13.350 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/libcxgb/ 2024-09-10T21:10:13.350 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 39233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-09-10T21:10:13.350 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/ 2024-09-10T21:10:13.350 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/enic/ 2024-09-10T21:10:13.351 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 269777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-09-10T21:10:13.351 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/ 2024-09-10T21:10:13.351 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/ 2024-09-10T21:10:13.351 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 85609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-09-10T21:10:13.351 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 77601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-09-10T21:10:13.351 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 169313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-09-10T21:10:13.351 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 66065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-09-10T21:10:13.351 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 76361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-09-10T21:10:13.351 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 38153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-09-10T21:10:13.351 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/ 2024-09-10T21:10:13.351 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 76161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-09-10T21:10:13.351 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 78929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-09-10T21:10:13.351 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 52401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dnet.ko 2024-09-10T21:10:13.352 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 30489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ec_bhf.ko 2024-09-10T21:10:13.352 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/ 2024-09-10T21:10:13.352 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/benet/ 2024-09-10T21:10:13.352 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 356153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-09-10T21:10:13.352 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 61881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ethoc.ko 2024-09-10T21:10:13.352 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 63345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/fealnx.ko 2024-09-10T21:10:13.352 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/fujitsu/ 2024-09-10T21:10:13.352 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 51657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/fujitsu/fmvj18x_cs.ko 2024-09-10T21:10:13.352 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ 2024-09-10T21:10:13.352 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 115697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e100.ko 2024-09-10T21:10:13.352 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000/ 2024-09-10T21:10:13.352 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 366465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-09-10T21:10:13.352 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000e/ 2024-09-10T21:10:13.352 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 755425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-09-10T21:10:13.353 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/fm10k/ 2024-09-10T21:10:13.353 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 324721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-09-10T21:10:13.353 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/i40e/ 2024-09-10T21:10:13.353 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 1193705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-09-10T21:10:13.353 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/iavf/ 2024-09-10T21:10:13.353 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 356609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-09-10T21:10:13.353 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igb/ 2024-09-10T21:10:13.353 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 672177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-09-10T21:10:13.377 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igbvf/ 2024-09-10T21:10:13.377 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 121609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-09-10T21:10:13.377 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbe/ 2024-09-10T21:10:13.377 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 966193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-09-10T21:10:13.377 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbevf/ 2024-09-10T21:10:13.377 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 193889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-09-10T21:10:13.378 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libeth/ 2024-09-10T21:10:13.378 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 11561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libeth/libeth.ko 2024-09-10T21:10:13.378 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libie/ 2024-09-10T21:10:13.378 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 5969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libie/libie.ko 2024-09-10T21:10:13.378 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 109353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/jme.ko 2024-09-10T21:10:13.378 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/ 2024-09-10T21:10:13.378 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 25785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-09-10T21:10:13.378 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 127057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/skge.ko 2024-09-10T21:10:13.378 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 151049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-09-10T21:10:13.378 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/ 2024-09-10T21:10:13.378 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/ 2024-09-10T21:10:13.378 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 888945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-09-10T21:10:13.378 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 346977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-09-10T21:10:13.378 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/ 2024-09-10T21:10:13.379 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/core/ 2024-09-10T21:10:13.379 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 1408481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-09-10T21:10:13.379 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ 2024-09-10T21:10:13.379 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 66097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ks8842.ko 2024-09-10T21:10:13.379 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 53497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ks8851_common.ko 2024-09-10T21:10:13.379 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 22009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ks8851_par.ko 2024-09-10T21:10:13.379 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 117585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-09-10T21:10:13.379 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/ 2024-09-10T21:10:13.379 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/myri10ge/ 2024-09-10T21:10:13.379 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 131601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-09-10T21:10:13.379 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/ 2024-09-10T21:10:13.379 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 91689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-09-10T21:10:13.379 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 56777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-09-10T21:10:13.380 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/neterion/ 2024-09-10T21:10:13.380 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 181409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-09-10T21:10:13.380 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/nvidia/ 2024-09-10T21:10:13.380 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 171649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-09-10T21:10:13.380 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/ 2024-09-10T21:10:13.380 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 58529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-09-10T21:10:13.380 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 50409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-09-10T21:10:13.380 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/ 2024-09-10T21:10:13.380 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/netxen/ 2024-09-10T21:10:13.380 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 301017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-09-10T21:10:13.380 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 105857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-09-10T21:10:13.380 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qlcnic/ 2024-09-10T21:10:13.380 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 885465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-09-10T21:10:13.381 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rdc/ 2024-09-10T21:10:13.381 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 52481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-09-10T21:10:13.381 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/ 2024-09-10T21:10:13.381 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 81785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-09-10T21:10:13.381 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 100833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-09-10T21:10:13.381 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 34217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/atp.ko 2024-09-10T21:10:13.381 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 264393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-09-10T21:10:13.381 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/samsung/ 2024-09-10T21:10:13.381 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/samsung/sxgbe/ 2024-09-10T21:10:13.381 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 142481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/samsung/sxgbe/samsung-sxgbe.ko 2024-09-10T21:10:13.381 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sfc/ 2024-09-10T21:10:13.381 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 1346017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-09-10T21:10:13.381 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/silan/ 2024-09-10T21:10:13.382 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 58097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-09-10T21:10:13.382 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/ 2024-09-10T21:10:13.382 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 66737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/sis190.ko 2024-09-10T21:10:13.382 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 80233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/sis900.ko 2024-09-10T21:10:13.382 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/ 2024-09-10T21:10:13.382 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 74529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-09-10T21:10:13.382 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 79601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-09-10T21:10:13.382 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 100937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-09-10T21:10:13.382 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 83593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-09-10T21:10:13.382 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/ 2024-09-10T21:10:13.382 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/ 2024-09-10T21:10:13.382 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 14305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-generic.ko 2024-09-10T21:10:13.382 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 47193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-09-10T21:10:13.383 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 20969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac-platform.ko 2024-09-10T21:10:13.383 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 644913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-09-10T21:10:13.383 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/ 2024-09-10T21:10:13.383 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 129089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/cassini.ko 2024-09-10T21:10:13.383 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 208177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/niu.ko 2024-09-10T21:10:13.383 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 82033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/sungem.ko 2024-09-10T21:10:13.383 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 91641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-09-10T21:10:13.383 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/tehuti/ 2024-09-10T21:10:13.383 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 60113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-09-10T21:10:13.383 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ti/ 2024-09-10T21:10:13.383 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 110881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ti/tlan.ko 2024-09-10T21:10:13.383 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/ 2024-09-10T21:10:13.384 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 98993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-09-10T21:10:13.384 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 102745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-09-10T21:10:13.384 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/ 2024-09-10T21:10:13.384 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 61121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-09-10T21:10:13.384 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 40689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-09-10T21:10:13.384 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/xircom/ 2024-09-10T21:10:13.384 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 70057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-09-10T21:10:13.384 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/fddi/ 2024-09-10T21:10:13.384 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 49537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/fddi/defxx.ko 2024-09-10T21:10:13.384 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/fddi/skfp/ 2024-09-10T21:10:13.384 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 275889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/fddi/skfp/skfp.ko 2024-09-10T21:10:13.384 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 91785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/geneve.ko 2024-09-10T21:10:13.384 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/hyperv/ 2024-09-10T21:10:13.385 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 290697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-09-10T21:10:13.385 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ieee802154/ 2024-09-10T21:10:13.385 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 21489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ieee802154/fakelb.ko 2024-09-10T21:10:13.385 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 30153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ifb.ko 2024-09-10T21:10:13.385 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ipvlan/ 2024-09-10T21:10:13.385 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 107849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ipvlan/ipvlan.ko 2024-09-10T21:10:13.385 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 77585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/macvlan.ko 2024-09-10T21:10:13.385 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 19361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/macvtap.ko 2024-09-10T21:10:13.385 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/ 2024-09-10T21:10:13.385 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 26193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-09-10T21:10:13.385 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 17593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-09-10T21:10:13.385 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 16441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-gpio.ko 2024-09-10T21:10:13.385 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 9865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-regmap.ko 2024-09-10T21:10:13.386 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 20073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio.ko 2024-09-10T21:10:13.386 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 25385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mii.ko 2024-09-10T21:10:13.386 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 62857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/netconsole.ko 2024-09-10T21:10:13.386 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 13297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/nlmon.ko 2024-09-10T21:10:13.386 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/ 2024-09-10T21:10:13.386 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 20185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/pcs-lynx.ko 2024-09-10T21:10:13.386 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 72969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-09-10T21:10:13.386 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/ 2024-09-10T21:10:13.386 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 12113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/amd.ko 2024-09-10T21:10:13.386 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 61801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-09-10T21:10:13.386 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 52945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm7xxx.ko 2024-09-10T21:10:13.386 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 14369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm87xx.ko 2024-09-10T21:10:13.386 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 58729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/broadcom.ko 2024-09-10T21:10:13.387 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 12369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/cicada.ko 2024-09-10T21:10:13.387 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 15145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/davicom.ko 2024-09-10T21:10:13.387 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 10841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/et1011c.ko 2024-09-10T21:10:13.387 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 31393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/icplus.ko 2024-09-10T21:10:13.387 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 19265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/lxt.ko 2024-09-10T21:10:13.387 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 113681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/marvell.ko 2024-09-10T21:10:13.387 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 108257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/micrel.ko 2024-09-10T21:10:13.387 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 14137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/national.ko 2024-09-10T21:10:13.387 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 118505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/phylink.ko 2024-09-10T21:10:13.387 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 11889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/qsemi.ko 2024-09-10T21:10:13.387 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 76177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/realtek.ko 2024-09-10T21:10:13.387 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 34737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/smsc.ko 2024-09-10T21:10:13.388 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 12761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/ste10Xp.ko 2024-09-10T21:10:13.388 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 28857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/vitesse.ko 2024-09-10T21:10:13.388 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/plip/ 2024-09-10T21:10:13.388 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 55377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/plip/plip.ko 2024-09-10T21:10:13.388 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ 2024-09-10T21:10:13.388 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 19457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/bsd_comp.ko 2024-09-10T21:10:13.388 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 36393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_async.ko 2024-09-10T21:10:13.388 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 21625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_deflate.ko 2024-09-10T21:10:13.388 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 23201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_mppe.ko 2024-09-10T21:10:13.388 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 35577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_synctty.ko 2024-09-10T21:10:13.388 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 53409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pppoe.ko 2024-09-10T21:10:13.388 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 15313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pppox.ko 2024-09-10T21:10:13.388 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 39945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pptp.ko 2024-09-10T21:10:13.388 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip/ 2024-09-10T21:10:13.389 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 52177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip/slip.ko 2024-09-10T21:10:13.389 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 42257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/sungem_phy.ko 2024-09-10T21:10:13.389 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 79233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/tap.ko 2024-09-10T21:10:13.389 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/ 2024-09-10T21:10:13.389 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 136729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team.ko 2024-09-10T21:10:13.389 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 16113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_activebackup.ko 2024-09-10T21:10:13.389 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 11489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_broadcast.ko 2024-09-10T21:10:13.389 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 36321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-09-10T21:10:13.389 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 10929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_random.ko 2024-09-10T21:10:13.389 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 10553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-09-10T21:10:13.389 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 108409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/veth.ko 2024-09-10T21:10:13.389 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vmxnet3/ 2024-09-10T21:10:13.389 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 179329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-09-10T21:10:13.390 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vxlan/ 2024-09-10T21:10:13.390 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 384193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vxlan/vxlan.ko 2024-09-10T21:10:13.390 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netback/ 2024-09-10T21:10:13.390 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 175857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netback/xen-netback.ko 2024-09-10T21:10:13.390 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ntb/ 2024-09-10T21:10:13.390 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 24137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ntb/ntb.ko 2024-09-10T21:10:13.390 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/ 2024-09-10T21:10:13.390 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/ 2024-09-10T21:10:13.390 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 337697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/nvme-core.ko 2024-09-10T21:10:13.390 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 128353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/nvme.ko 2024-09-10T21:10:13.390 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/ 2024-09-10T21:10:13.390 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 164153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport.ko 2024-09-10T21:10:13.390 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 16281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_cs.ko 2024-09-10T21:10:13.391 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 103657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_pc.ko 2024-09-10T21:10:13.391 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 30401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_serial.ko 2024-09-10T21:10:13.391 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/ 2024-09-10T21:10:13.391 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug/ 2024-09-10T21:10:13.391 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 28593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-09-10T21:10:13.391 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 20289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug/cpcihp_generic.ko 2024-09-10T21:10:13.391 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 23969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug/cpcihp_zt5550.ko 2024-09-10T21:10:13.391 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 13289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/pci-stub.ko 2024-09-10T21:10:13.391 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 55425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/xen-pcifront.ko 2024-09-10T21:10:13.391 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/ 2024-09-10T21:10:13.391 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 31665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/i82092.ko 2024-09-10T21:10:13.391 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 177817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/pcmcia.ko 2024-09-10T21:10:13.391 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 76785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/pcmcia_core.ko 2024-09-10T21:10:13.391 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 41777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/pcmcia_rsrc.ko 2024-09-10T21:10:13.392 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 40553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/pd6729.ko 2024-09-10T21:10:13.392 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 133969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/yenta_socket.ko 2024-09-10T21:10:13.392 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/phy/ 2024-09-10T21:10:13.392 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/phy/broadcom/ 2024-09-10T21:10:13.392 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 12513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/phy/broadcom/phy-bcm-kona-usb2.ko 2024-09-10T21:10:13.392 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/phy/samsung/ 2024-09-10T21:10:13.392 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 9561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/phy/samsung/phy-exynos-usb2.ko 2024-09-10T21:10:13.392 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/ 2024-09-10T21:10:13.392 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/intel/ 2024-09-10T21:10:13.392 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 78281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-09-10T21:10:13.392 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/ 2024-09-10T21:10:13.392 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/ 2024-09-10T21:10:13.392 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 60889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros-ec-sensorhub.ko 2024-09-10T21:10:13.393 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 28953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros_ec.ko 2024-09-10T21:10:13.393 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 25825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros_ec_chardev.ko 2024-09-10T21:10:13.393 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 31481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros_ec_debugfs.ko 2024-09-10T21:10:13.393 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 32113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros_ec_lightbar.ko 2024-09-10T21:10:13.393 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 22009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros_ec_sysfs.ko 2024-09-10T21:10:13.393 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 20121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/chrome/cros_usbpd_notify.ko 2024-09-10T21:10:13.393 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/ 2024-09-10T21:10:13.393 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 88017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/acer-wmi.ko 2024-09-10T21:10:13.393 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 44233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/acerhdf.ko 2024-09-10T21:10:13.393 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 48353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/apple-gmux.ko 2024-09-10T21:10:13.393 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 74033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-laptop.ko 2024-09-10T21:10:13.393 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 33401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-09-10T21:10:13.393 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 172313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-wmi.ko 2024-09-10T21:10:13.394 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 46505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/classmate-laptop.ko 2024-09-10T21:10:13.394 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 62105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-09-10T21:10:13.394 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 15953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-09-10T21:10:13.394 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 52665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-09-10T21:10:13.394 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 25929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-09-10T21:10:13.394 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 42913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/hdaps.ko 2024-09-10T21:10:13.394 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 23105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/ibm_rtl.ko 2024-09-10T21:10:13.394 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel/ 2024-09-10T21:10:13.394 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 14401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-09-10T21:10:13.394 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 10233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel/intel-smartconnect.ko 2024-09-10T21:10:13.394 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 53801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel_ips.ko 2024-09-10T21:10:13.394 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 23001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/msi-wmi.ko 2024-09-10T21:10:13.394 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 11657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/mxm-wmi.ko 2024-09-10T21:10:13.395 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 43793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-09-10T21:10:13.395 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 51857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/samsung-laptop.ko 2024-09-10T21:10:13.395 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 16033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/samsung-q10.ko 2024-09-10T21:10:13.395 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 20289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/topstar-laptop.ko 2024-09-10T21:10:13.395 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 15625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-09-10T21:10:13.395 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 20369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/toshiba_haps.ko 2024-09-10T21:10:13.395 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 11737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/wmi-bmof.ko 2024-09-10T21:10:13.395 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 59729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/wmi.ko 2024-09-10T21:10:13.395 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/ 2024-09-10T21:10:13.395 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/ 2024-09-10T21:10:13.395 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 36665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/88pm860x_battery.ko 2024-09-10T21:10:13.395 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 40449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/88pm860x_charger.ko 2024-09-10T21:10:13.395 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 68921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/bq2415x_charger.ko 2024-09-10T21:10:13.396 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 73161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/bq24190_charger.ko 2024-09-10T21:10:13.396 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 26041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/bq24735-charger.ko 2024-09-10T21:10:13.396 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 20745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/cros_charge-control.ko 2024-09-10T21:10:13.396 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 20521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/cros_peripheral_charger.ko 2024-09-10T21:10:13.396 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 23321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/da9030_battery.ko 2024-09-10T21:10:13.396 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 23377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/da9052-battery.ko 2024-09-10T21:10:13.396 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 34569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/ds2760_battery.ko 2024-09-10T21:10:13.396 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 28729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/ds2780_battery.ko 2024-09-10T21:10:13.396 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 28889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/ds2781_battery.ko 2024-09-10T21:10:13.396 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 28745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/ds2782_battery.ko 2024-09-10T21:10:13.396 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 24297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/gpio-charger.ko 2024-09-10T21:10:13.396 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 23305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/isp1704_charger.ko 2024-09-10T21:10:13.396 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 24201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/lp8727_charger.ko 2024-09-10T21:10:13.396 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 14089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/max14577_charger.ko 2024-09-10T21:10:13.397 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 37929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/max17042_battery.ko 2024-09-10T21:10:13.397 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 22105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/max8903_charger.ko 2024-09-10T21:10:13.397 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 27769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/max8925_power.ko 2024-09-10T21:10:13.397 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 25433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/pcf50633-charger.ko 2024-09-10T21:10:13.397 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 44929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/sbs-battery.ko 2024-09-10T21:10:13.397 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 35993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/test_power.ko 2024-09-10T21:10:13.397 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 21681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/tps65090-charger.ko 2024-09-10T21:10:13.397 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 13441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/wm831x_backup.ko 2024-09-10T21:10:13.397 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 29161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/wm831x_power.ko 2024-09-10T21:10:13.397 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 27529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/power/supply/wm8350_power.ko 2024-09-10T21:10:13.397 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/ 2024-09-10T21:10:13.397 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 98737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/intel_rapl_common.ko 2024-09-10T21:10:13.397 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 20521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/intel_rapl_msr.ko 2024-09-10T21:10:13.398 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/ 2024-09-10T21:10:13.398 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/ 2024-09-10T21:10:13.398 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 17921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps-gpio.ko 2024-09-10T21:10:13.398 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 15617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps-ldisc.ko 2024-09-10T21:10:13.398 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 19017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps_parport.ko 2024-09-10T21:10:13.398 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 49385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/pps_core.ko 2024-09-10T21:10:13.398 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ 2024-09-10T21:10:13.398 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 90617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ptp.ko 2024-09-10T21:10:13.398 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 22033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ptp_kvm.ko 2024-09-10T21:10:13.398 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/ 2024-09-10T21:10:13.398 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/ 2024-09-10T21:10:13.398 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/atl/ 2024-09-10T21:10:13.398 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 89769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-09-10T21:10:13.398 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 47369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/fmpm.ko 2024-09-10T21:10:13.399 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/ 2024-09-10T21:10:13.399 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 27681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-88pm80x.ko 2024-09-10T21:10:13.399 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 29057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-88pm860x.ko 2024-09-10T21:10:13.399 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 18545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-bq32k.ko 2024-09-10T21:10:13.399 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 24809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-da9052.ko 2024-09-10T21:10:13.399 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 22657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-da9055.ko 2024-09-10T21:10:13.399 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 24425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-da9063.ko 2024-09-10T21:10:13.399 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 19817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1286.ko 2024-09-10T21:10:13.399 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 82585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1307.ko 2024-09-10T21:10:13.399 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 24569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1374.ko 2024-09-10T21:10:13.399 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 22073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1511.ko 2024-09-10T21:10:13.399 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 20369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1553.ko 2024-09-10T21:10:13.399 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 16137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1672.ko 2024-09-10T21:10:13.400 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 15753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1742.ko 2024-09-10T21:10:13.400 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 18113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds2404.ko 2024-09-10T21:10:13.400 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 29097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds3232.ko 2024-09-10T21:10:13.400 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 13377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-em3027.ko 2024-09-10T21:10:13.400 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 28753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-fm3130.ko 2024-09-10T21:10:13.400 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 24185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-isl12022.ko 2024-09-10T21:10:13.400 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 44369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-isl1208.ko 2024-09-10T21:10:13.400 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 17233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-lp8788.ko 2024-09-10T21:10:13.400 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 47001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m41t80.ko 2024-09-10T21:10:13.400 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 14049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t35.ko 2024-09-10T21:10:13.400 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 34057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t59.ko 2024-09-10T21:10:13.400 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 17425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t86.ko 2024-09-10T21:10:13.400 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 14073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max6900.ko 2024-09-10T21:10:13.400 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 18009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max8907.ko 2024-09-10T21:10:13.401 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 18601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max8925.ko 2024-09-10T21:10:13.401 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 31425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max8997.ko 2024-09-10T21:10:13.401 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 20409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max8998.ko 2024-09-10T21:10:13.401 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 22489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-mc13xxx.ko 2024-09-10T21:10:13.401 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 13241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-msm6242.ko 2024-09-10T21:10:13.401 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 24417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-palmas.ko 2024-09-10T21:10:13.401 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 56873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf2127.ko 2024-09-10T21:10:13.401 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 21689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf50633.ko 2024-09-10T21:10:13.401 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 32609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf85063.ko 2024-09-10T21:10:13.401 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 28313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8523.ko 2024-09-10T21:10:13.401 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 36641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8563.ko 2024-09-10T21:10:13.401 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 14993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8583.ko 2024-09-10T21:10:13.401 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 22121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rc5t583.ko 2024-09-10T21:10:13.402 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 15289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rp5c01.ko 2024-09-10T21:10:13.402 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 40169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rs5c372.ko 2024-09-10T21:10:13.402 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 34617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-09-10T21:10:13.402 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 30065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rx8025.ko 2024-09-10T21:10:13.402 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 22889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rx8581.ko 2024-09-10T21:10:13.402 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 26265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-s35390a.ko 2024-09-10T21:10:13.402 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 20513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-09-10T21:10:13.402 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 21785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-tps6586x.ko 2024-09-10T21:10:13.402 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 26193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-tps65910.ko 2024-09-10T21:10:13.402 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 24889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-wm831x.ko 2024-09-10T21:10:13.402 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 25889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-wm8350.ko 2024-09-10T21:10:13.402 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 33489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-x1205.ko 2024-09-10T21:10:13.402 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ 2024-09-10T21:10:13.402 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 90537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-9xxx.ko 2024-09-10T21:10:13.403 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 69689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-sas.ko 2024-09-10T21:10:13.403 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 66033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-xxxx.ko 2024-09-10T21:10:13.403 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 164753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/BusLogic.ko 2024-09-10T21:10:13.403 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 31705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/a100u2w.ko 2024-09-10T21:10:13.403 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aacraid/ 2024-09-10T21:10:13.403 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 341105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aacraid/aacraid.ko 2024-09-10T21:10:13.403 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 120833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/advansys.ko 2024-09-10T21:10:13.403 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/ 2024-09-10T21:10:13.403 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 468305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-09-10T21:10:13.403 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 380857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-09-10T21:10:13.403 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic94xx/ 2024-09-10T21:10:13.403 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 244049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic94xx/aic94xx.ko 2024-09-10T21:10:13.403 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 28937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/am53c974.ko 2024-09-10T21:10:13.404 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/arcmsr/ 2024-09-10T21:10:13.404 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 135801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-09-10T21:10:13.404 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 44489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/atp870u.ko 2024-09-10T21:10:13.404 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/be2iscsi/ 2024-09-10T21:10:13.404 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 305161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-09-10T21:10:13.404 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bfa/ 2024-09-10T21:10:13.404 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 1526289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bfa/bfa.ko 2024-09-10T21:10:13.404 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2fc/ 2024-09-10T21:10:13.404 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 298321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-09-10T21:10:13.404 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2i/ 2024-09-10T21:10:13.404 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 144553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-09-10T21:10:13.404 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 47673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ch.ko 2024-09-10T21:10:13.404 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/csiostor/ 2024-09-10T21:10:13.405 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 390817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/csiostor/csiostor.ko 2024-09-10T21:10:13.405 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/ 2024-09-10T21:10:13.405 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb3i/ 2024-09-10T21:10:13.405 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 89041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-09-10T21:10:13.405 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb4i/ 2024-09-10T21:10:13.405 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 117561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-09-10T21:10:13.419 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 168361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-09-10T21:10:13.419 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 91777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/dc395x.ko 2024-09-10T21:10:13.419 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 45489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/dmx3191d.ko 2024-09-10T21:10:13.419 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esas2r/ 2024-09-10T21:10:13.419 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 267737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esas2r/esas2r.ko 2024-09-10T21:10:13.420 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 78441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esp_scsi.ko 2024-09-10T21:10:13.420 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/ 2024-09-10T21:10:13.420 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 95553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/fcoe.ko 2024-09-10T21:10:13.420 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 191041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-09-10T21:10:13.420 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 22289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fdomain.ko 2024-09-10T21:10:13.420 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fnic/ 2024-09-10T21:10:13.420 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 366737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fnic/fnic.ko 2024-09-10T21:10:13.420 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 251641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hpsa.ko 2024-09-10T21:10:13.420 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 65529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hptiop.ko 2024-09-10T21:10:13.420 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 57201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hv_storvsc.ko 2024-09-10T21:10:13.420 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 44393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/imm.ko 2024-09-10T21:10:13.420 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 55241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/initio.ko 2024-09-10T21:10:13.420 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 327841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ipr.ko 2024-09-10T21:10:13.421 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 129881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ips.ko 2024-09-10T21:10:13.421 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/isci/ 2024-09-10T21:10:13.421 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 398505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/isci/isci.ko 2024-09-10T21:10:13.421 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 31201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-09-10T21:10:13.421 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 62785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/iscsi_tcp.ko 2024-09-10T21:10:13.421 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libfc/ 2024-09-10T21:10:13.421 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 380649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libfc/libfc.ko 2024-09-10T21:10:13.421 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 172833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libiscsi.ko 2024-09-10T21:10:13.421 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 63097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libiscsi_tcp.ko 2024-09-10T21:10:13.421 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libsas/ 2024-09-10T21:10:13.421 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 274913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libsas/libsas.ko 2024-09-10T21:10:13.421 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/lpfc/ 2024-09-10T21:10:13.422 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 2737441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/lpfc/lpfc.ko 2024-09-10T21:10:13.422 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/ 2024-09-10T21:10:13.422 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 88625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-09-10T21:10:13.422 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 39769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-09-10T21:10:13.422 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 388737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-09-10T21:10:13.422 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 105177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid.ko 2024-09-10T21:10:13.422 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mpt3sas/ 2024-09-10T21:10:13.422 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 833969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-09-10T21:10:13.422 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvsas/ 2024-09-10T21:10:13.423 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 158857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvsas/mvsas.ko 2024-09-10T21:10:13.423 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 96697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvumi.ko 2024-09-10T21:10:13.423 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pcmcia/ 2024-09-10T21:10:13.423 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 102113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pcmcia/aha152x_cs.ko 2024-09-10T21:10:13.423 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 13353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pcmcia/fdomain_cs.ko 2024-09-10T21:10:13.423 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 23241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pcmcia/qlogic_cs.ko 2024-09-10T21:10:13.423 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 28417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pcmcia/sym53c500_cs.ko 2024-09-10T21:10:13.423 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pm8001/ 2024-09-10T21:10:13.423 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 444273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-09-10T21:10:13.423 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 152097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pmcraid.ko 2024-09-10T21:10:13.423 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 41265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ppa.ko 2024-09-10T21:10:13.423 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 67129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla1280.ko 2024-09-10T21:10:13.423 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/ 2024-09-10T21:10:13.424 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 2323537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-09-10T21:10:13.424 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 103793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-09-10T21:10:13.424 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla4xxx/ 2024-09-10T21:10:13.424 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 697393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-09-10T21:10:13.424 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 22745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qlogicfas408.ko 2024-09-10T21:10:13.424 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 18705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/raid_class.ko 2024-09-10T21:10:13.424 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 331577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_debug.ko 2024-09-10T21:10:13.424 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 219497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_fc.ko 2024-09-10T21:10:13.424 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 349593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-09-10T21:10:13.424 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 121777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_sas.ko 2024-09-10T21:10:13.424 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 92385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_spi.ko 2024-09-10T21:10:13.424 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 54097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_srp.ko 2024-09-10T21:10:13.425 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 33113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ses.ko 2024-09-10T21:10:13.425 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 138265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/st.ko 2024-09-10T21:10:13.425 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 57489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/stex.ko 2024-09-10T21:10:13.425 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/sym53c8xx_2/ 2024-09-10T21:10:13.425 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 205137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-09-10T21:10:13.425 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 55009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/virtio_scsi.ko 2024-09-10T21:10:13.425 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 59289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/vmw_pvscsi.ko 2024-09-10T21:10:13.425 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 48865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/wd719x.ko 2024-09-10T21:10:13.425 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 59521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/xen-scsifront.ko 2024-09-10T21:10:13.425 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ssb/ 2024-09-10T21:10:13.425 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 172697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ssb/ssb.ko 2024-09-10T21:10:13.425 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/ 2024-09-10T21:10:13.425 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/iscsi/ 2024-09-10T21:10:13.426 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 915121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-09-10T21:10:13.426 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/loopback/ 2024-09-10T21:10:13.426 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 60497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/loopback/tcm_loop.ko 2024-09-10T21:10:13.426 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 48281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_file.ko 2024-09-10T21:10:13.426 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 53881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_iblock.ko 2024-09-10T21:10:13.426 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 1161521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_mod.ko 2024-09-10T21:10:13.426 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 51985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_pscsi.ko 2024-09-10T21:10:13.426 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 149801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_user.ko 2024-09-10T21:10:13.426 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/tcm_fc/ 2024-09-10T21:10:13.426 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 90217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-09-10T21:10:13.426 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/ 2024-09-10T21:10:13.426 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/ 2024-09-10T21:10:13.427 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/ 2024-09-10T21:10:13.427 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 27681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-09-10T21:10:13.427 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 34961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-09-10T21:10:13.427 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 11409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-09-10T21:10:13.427 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 11785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-09-10T21:10:13.427 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 17513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-09-10T21:10:13.427 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 18673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-09-10T21:10:13.427 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 36033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-09-10T21:10:13.427 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 28457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-09-10T21:10:13.427 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 16833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-09-10T21:10:13.427 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 12553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-09-10T21:10:13.427 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 11633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-09-10T21:10:13.427 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 15049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-09-10T21:10:13.428 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 46289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-09-10T21:10:13.428 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 17233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-09-10T21:10:13.428 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 13161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-09-10T21:10:13.428 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 36001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_powerclamp.ko 2024-09-10T21:10:13.428 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 21577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-09-10T21:10:13.428 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 12937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-09-10T21:10:13.428 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 30721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-09-10T21:10:13.428 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ 2024-09-10T21:10:13.428 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ipwireless/ 2024-09-10T21:10:13.428 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 99417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-09-10T21:10:13.428 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 69825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/moxa.ko 2024-09-10T21:10:13.428 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 61737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/mxser.ko 2024-09-10T21:10:13.428 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 134633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/n_gsm.ko 2024-09-10T21:10:13.429 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 37993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/n_hdlc.ko 2024-09-10T21:10:13.429 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 79057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/nozomi.ko 2024-09-10T21:10:13.429 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/ 2024-09-10T21:10:13.429 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250/ 2024-09-10T21:10:13.429 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 43209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250/8250_dw.ko 2024-09-10T21:10:13.429 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 65273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-09-10T21:10:13.429 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 23161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/altera_jtaguart.ko 2024-09-10T21:10:13.429 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 28137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/altera_uart.ko 2024-09-10T21:10:13.429 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 10761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/arc_uart.ko 2024-09-10T21:10:13.429 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 90697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/fsl_lpuart.ko 2024-09-10T21:10:13.429 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/jsm/ 2024-09-10T21:10:13.429 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 121249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/jsm/jsm.ko 2024-09-10T21:10:13.429 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 35073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/rp2.ko 2024-09-10T21:10:13.430 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 68009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/sc16is7xx.ko 2024-09-10T21:10:13.430 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 13545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/sc16is7xx_i2c.ko 2024-09-10T21:10:13.430 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/ 2024-09-10T21:10:13.430 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/core/ 2024-09-10T21:10:13.430 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 689289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/core/ufshcd-core.ko 2024-09-10T21:10:13.430 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host/ 2024-09-10T21:10:13.430 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 36673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-09-10T21:10:13.430 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 16217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host/ufshcd-pltfrm.ko 2024-09-10T21:10:13.430 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/ 2024-09-10T21:10:13.430 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 55041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio.ko 2024-09-10T21:10:13.430 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 14505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_aec.ko 2024-09-10T21:10:13.430 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 13097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_cif.ko 2024-09-10T21:10:13.430 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 21001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_dmem_genirq.ko 2024-09-10T21:10:13.431 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 15833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_mf624.ko 2024-09-10T21:10:13.431 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 14113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_netx.ko 2024-09-10T21:10:13.431 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 12129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_pci_generic.ko 2024-09-10T21:10:13.431 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 19273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_pdrv_genirq.ko 2024-09-10T21:10:13.431 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 16105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_sercos3.ko 2024-09-10T21:10:13.431 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/ 2024-09-10T21:10:13.431 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/c67x00/ 2024-09-10T21:10:13.431 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 90417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/c67x00/c67x00.ko 2024-09-10T21:10:13.431 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/chipidea/ 2024-09-10T21:10:13.431 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 187689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/chipidea/ci_hdrc.ko 2024-09-10T21:10:13.431 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 21729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/chipidea/ci_hdrc_msm.ko 2024-09-10T21:10:13.431 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 15137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/chipidea/ci_hdrc_npcm.ko 2024-09-10T21:10:13.432 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 14193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/chipidea/ci_hdrc_pci.ko 2024-09-10T21:10:13.432 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 14857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/chipidea/ci_hdrc_usb2.ko 2024-09-10T21:10:13.432 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/ 2024-09-10T21:10:13.432 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 101657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/cdc-acm.ko 2024-09-10T21:10:13.432 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 63545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/cdc-wdm.ko 2024-09-10T21:10:13.432 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 56841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/usblp.ko 2024-09-10T21:10:13.432 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 107633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/usbtmc.ko 2024-09-10T21:10:13.432 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/common/ 2024-09-10T21:10:13.432 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 28737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/common/ulpi.ko 2024-09-10T21:10:13.432 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/dwc2/ 2024-09-10T21:10:13.432 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 13185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/dwc2/dwc2_pci.ko 2024-09-10T21:10:13.432 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/dwc3/ 2024-09-10T21:10:13.432 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 13617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/dwc3/dwc3-haps.ko 2024-09-10T21:10:13.433 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 34537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/dwc3/dwc3-pci.ko 2024-09-10T21:10:13.433 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 576329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/dwc3/dwc3.ko 2024-09-10T21:10:13.433 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/ 2024-09-10T21:10:13.433 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/ 2024-09-10T21:10:13.433 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 70081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/u_ether.ko 2024-09-10T21:10:13.433 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 61153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/u_serial.ko 2024-09-10T21:10:13.433 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 33937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_acm.ko 2024-09-10T21:10:13.433 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 49945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_ecm.ko 2024-09-10T21:10:13.433 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 33249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_ecm_subset.ko 2024-09-10T21:10:13.433 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 43705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_eem.ko 2024-09-10T21:10:13.433 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 132897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_fs.ko 2024-09-10T21:10:13.433 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 62473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_hid.ko 2024-09-10T21:10:13.433 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 142801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_mass_storage.ko 2024-09-10T21:10:13.434 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 70785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_ncm.ko 2024-09-10T21:10:13.434 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 25913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_obex.ko 2024-09-10T21:10:13.434 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 37801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_phonet.ko 2024-09-10T21:10:13.434 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 70241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_printer.ko 2024-09-10T21:10:13.434 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 99913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_rndis.ko 2024-09-10T21:10:13.434 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 22633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_serial.ko 2024-09-10T21:10:13.434 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 71577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_ss_lb.ko 2024-09-10T21:10:13.434 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 85089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/function/usb_f_tcm.ko 2024-09-10T21:10:13.434 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/ 2024-09-10T21:10:13.434 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 23529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_acm_ms.ko 2024-09-10T21:10:13.434 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 20369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_cdc.ko 2024-09-10T21:10:13.434 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 21393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_dbgp.ko 2024-09-10T21:10:13.434 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 24585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_ether.ko 2024-09-10T21:10:13.435 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 33129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_ffs.ko 2024-09-10T21:10:13.435 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 21697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_hid.ko 2024-09-10T21:10:13.435 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 22473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_mass_storage.ko 2024-09-10T21:10:13.435 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 19001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_ncm.ko 2024-09-10T21:10:13.435 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 35777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_nokia.ko 2024-09-10T21:10:13.435 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 18609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_printer.ko 2024-09-10T21:10:13.435 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 21785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_serial.ko 2024-09-10T21:10:13.435 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 30481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/g_zero.ko 2024-09-10T21:10:13.435 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 78185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/gadgetfs.ko 2024-09-10T21:10:13.435 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 17593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/legacy/tcm_usb_gadget.ko 2024-09-10T21:10:13.435 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 200137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/libcomposite.ko 2024-09-10T21:10:13.435 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/ 2024-09-10T21:10:13.435 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 19665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/amd5536udc_pci.ko 2024-09-10T21:10:13.435 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/bdc/ 2024-09-10T21:10:13.435 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 182665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/bdc/bdc.ko 2024-09-10T21:10:13.453 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 45633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/goku_udc.ko 2024-09-10T21:10:13.453 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 66545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/gr_udc.ko 2024-09-10T21:10:13.454 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 85393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/mv_u3d_core.ko 2024-09-10T21:10:13.454 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 69409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/mv_udc.ko 2024-09-10T21:10:13.454 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 90409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/net2272.ko 2024-09-10T21:10:13.454 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 90457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/net2280.ko 2024-09-10T21:10:13.454 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 94929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/pch_udc.ko 2024-09-10T21:10:13.454 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 80617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/pxa27x_udc.ko 2024-09-10T21:10:13.454 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 87561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/r8a66597-udc.ko 2024-09-10T21:10:13.454 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 94385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/snps_udc_core.ko 2024-09-10T21:10:13.454 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 191281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/gadget/udc/udc-core.ko 2024-09-10T21:10:13.454 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/ 2024-09-10T21:10:13.454 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 14033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/bcma-hcd.ko 2024-09-10T21:10:13.454 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 108417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/isp116x-hcd.ko 2024-09-10T21:10:13.454 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 104817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/oxu210hp-hcd.ko 2024-09-10T21:10:13.455 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 102361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/r8a66597-hcd.ko 2024-09-10T21:10:13.455 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 59905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/sl811-hcd.ko 2024-09-10T21:10:13.455 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 17593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/sl811_cs.ko 2024-09-10T21:10:13.455 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 16465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/ssb-hcd.ko 2024-09-10T21:10:13.455 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 32161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/xhci-plat-hcd.ko 2024-09-10T21:10:13.455 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/ 2024-09-10T21:10:13.455 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 40633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/mdc800.ko 2024-09-10T21:10:13.455 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 24761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/microtek.ko 2024-09-10T21:10:13.455 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ 2024-09-10T21:10:13.455 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 47593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/adutux.ko 2024-09-10T21:10:13.455 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 23985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/appledisplay.ko 2024-09-10T21:10:13.455 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 20345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/cypress_cy7c63.ko 2024-09-10T21:10:13.455 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 26769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/cytherm.ko 2024-09-10T21:10:13.456 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 15641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ehset.ko 2024-09-10T21:10:13.456 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 17977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/emi26.ko 2024-09-10T21:10:13.456 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 19041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/emi62.ko 2024-09-10T21:10:13.456 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 12857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ezusb.ko 2024-09-10T21:10:13.456 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 22433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/idmouse.ko 2024-09-10T21:10:13.456 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 42889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/iowarrior.ko 2024-09-10T21:10:13.456 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 13201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/isight_firmware.ko 2024-09-10T21:10:13.456 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 42345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ldusb.ko 2024-09-10T21:10:13.456 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 42337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/legousbtower.ko 2024-09-10T21:10:13.456 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 33737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/lvstest.ko 2024-09-10T21:10:13.456 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/sisusbvga/ 2024-09-10T21:10:13.456 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 85417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-09-10T21:10:13.456 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 14689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/trancevibrator.ko 2024-09-10T21:10:13.457 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 24609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usb3503.ko 2024-09-10T21:10:13.457 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 29729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usblcd.ko 2024-09-10T21:10:13.457 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 27897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usbsevseg.ko 2024-09-10T21:10:13.457 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 84369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usbtest.ko 2024-09-10T21:10:13.457 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 52377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/uss720.ko 2024-09-10T21:10:13.457 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 32561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/yurex.ko 2024-09-10T21:10:13.457 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/mon/ 2024-09-10T21:10:13.457 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 91561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/mon/usbmon.ko 2024-09-10T21:10:13.457 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/musb/ 2024-09-10T21:10:13.457 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 391209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/musb/musb_hdrc.ko 2024-09-10T21:10:13.457 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/ 2024-09-10T21:10:13.457 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 24993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/phy-generic.ko 2024-09-10T21:10:13.457 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 23713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/phy-gpio-vbus-usb.ko 2024-09-10T21:10:13.457 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 14273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/phy-isp1301.ko 2024-09-10T21:10:13.458 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 26449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/phy-tahvo.ko 2024-09-10T21:10:13.458 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ 2024-09-10T21:10:13.458 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 14841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/aircable.ko 2024-09-10T21:10:13.458 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 38305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ark3116.ko 2024-09-10T21:10:13.458 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 30769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/belkin_sa.ko 2024-09-10T21:10:13.458 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 44537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ch341.ko 2024-09-10T21:10:13.458 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 79769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cp210x.ko 2024-09-10T21:10:13.458 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 35233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cyberjack.ko 2024-09-10T21:10:13.458 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 66961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cypress_m8.ko 2024-09-10T21:10:13.458 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 58425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/digi_acceleport.ko 2024-09-10T21:10:13.458 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 10705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/empeg.ko 2024-09-10T21:10:13.458 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 53161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/f81232.ko 2024-09-10T21:10:13.458 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 188729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ftdi_sio.ko 2024-09-10T21:10:13.459 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 61601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/garmin_gps.ko 2024-09-10T21:10:13.459 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 152409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/io_edgeport.ko 2024-09-10T21:10:13.459 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 142937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/io_ti.ko 2024-09-10T21:10:13.459 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 69297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ipaq.ko 2024-09-10T21:10:13.459 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 21529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ipw.ko 2024-09-10T21:10:13.459 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 29857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ir-usb.ko 2024-09-10T21:10:13.459 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 66449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-09-10T21:10:13.459 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 122425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/keyspan.ko 2024-09-10T21:10:13.459 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 41393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/keyspan_pda.ko 2024-09-10T21:10:13.459 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 29561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/kl5kusb105.ko 2024-09-10T21:10:13.459 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 38145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/kobil_sct.ko 2024-09-10T21:10:13.459 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 38137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mct_u232.ko 2024-09-10T21:10:13.460 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 25657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/metro-usb.ko 2024-09-10T21:10:13.460 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 79801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mos7720.ko 2024-09-10T21:10:13.460 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 91625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mos7840.ko 2024-09-10T21:10:13.460 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 65281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mxuport.ko 2024-09-10T21:10:13.460 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 16057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/navman.ko 2024-09-10T21:10:13.460 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 13849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/omninet.ko 2024-09-10T21:10:13.460 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 28985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/opticon.ko 2024-09-10T21:10:13.460 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 201257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/option.ko 2024-09-10T21:10:13.460 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 45433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/oti6858.ko 2024-09-10T21:10:13.460 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 61857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/pl2303.ko 2024-09-10T21:10:13.460 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 10521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/qcaux.ko 2024-09-10T21:10:13.460 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 40081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/qcserial.ko 2024-09-10T21:10:13.460 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 40625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/quatech2.ko 2024-09-10T21:10:13.461 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 17073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/safe_serial.ko 2024-09-10T21:10:13.461 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 59449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/sierra.ko 2024-09-10T21:10:13.461 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 25321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/spcp8x5.ko 2024-09-10T21:10:13.461 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 34161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ssu100.ko 2024-09-10T21:10:13.461 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 19809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/symbolserial.ko 2024-09-10T21:10:13.461 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 70601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-09-10T21:10:13.461 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 24433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-09-10T21:10:13.461 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 12049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb_debug.ko 2024-09-10T21:10:13.461 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 38169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb_wwan.ko 2024-09-10T21:10:13.461 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 148729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usbserial.ko 2024-09-10T21:10:13.461 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 32601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/visor.ko 2024-09-10T21:10:13.461 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 43673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/whiteheat.ko 2024-09-10T21:10:13.461 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 12025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/wishbone-serial.ko 2024-09-10T21:10:13.462 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 10345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/xsens_mt.ko 2024-09-10T21:10:13.462 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ 2024-09-10T21:10:13.462 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 63361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/uas.ko 2024-09-10T21:10:13.462 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 30489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-alauda.ko 2024-09-10T21:10:13.462 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 14521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-cypress.ko 2024-09-10T21:10:13.462 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 25761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-datafab.ko 2024-09-10T21:10:13.462 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 41177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-09-10T21:10:13.462 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 15841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-freecom.ko 2024-09-10T21:10:13.462 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 23113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-isd200.ko 2024-09-10T21:10:13.462 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 22929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-09-10T21:10:13.462 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 15713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-karma.ko 2024-09-10T21:10:13.462 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 20817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-onetouch.ko 2024-09-10T21:10:13.462 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 34041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-realtek.ko 2024-09-10T21:10:13.463 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 34009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-sddr09.ko 2024-09-10T21:10:13.463 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 22681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-sddr55.ko 2024-09-10T21:10:13.463 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 41953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-usbat.ko 2024-09-10T21:10:13.463 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 201057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/usb-storage.ko 2024-09-10T21:10:13.463 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/ 2024-09-10T21:10:13.463 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 74449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/usbip-core.ko 2024-09-10T21:10:13.463 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 85513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/usbip-host.ko 2024-09-10T21:10:13.463 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 122537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-09-10T21:10:13.463 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/ 2024-09-10T21:10:13.463 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/ 2024-09-10T21:10:13.463 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 196977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-09-10T21:10:13.463 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 33737 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/vfio-pci.ko 2024-09-10T21:10:13.464 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 134401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/vfio.ko 2024-09-10T21:10:13.464 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 93481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-09-10T21:10:13.464 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/ 2024-09-10T21:10:13.464 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 135241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost.ko 2024-09-10T21:10:13.464 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 19073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_iotlb.ko 2024-09-10T21:10:13.464 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 71553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_net.ko 2024-09-10T21:10:13.464 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 110185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_scsi.ko 2024-09-10T21:10:13.464 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/ 2024-09-10T21:10:13.464 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/ 2024-09-10T21:10:13.464 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 18081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/88pm860x_bl.ko 2024-09-10T21:10:13.464 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 14169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/aat2870_bl.ko 2024-09-10T21:10:13.464 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 29241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/adp5520_bl.ko 2024-09-10T21:10:13.465 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 41257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/adp8860_bl.ko 2024-09-10T21:10:13.465 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 51825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/adp8870_bl.ko 2024-09-10T21:10:13.465 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 20417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/apple_bl.ko 2024-09-10T21:10:13.465 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 19913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/as3711_bl.ko 2024-09-10T21:10:13.465 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 14761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/bd6107.ko 2024-09-10T21:10:13.465 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 13425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/da903x_bl.ko 2024-09-10T21:10:13.465 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 13729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/da9052_bl.ko 2024-09-10T21:10:13.465 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 13929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/gpio_backlight.ko 2024-09-10T21:10:13.465 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 17577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/kb3886_bl.ko 2024-09-10T21:10:13.465 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 26185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/lcd.ko 2024-09-10T21:10:13.465 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 28321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/lm3533_bl.ko 2024-09-10T21:10:13.465 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 23481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/lm3639_bl.ko 2024-09-10T21:10:13.465 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 13713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/lv5207lp.ko 2024-09-10T21:10:13.466 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 16689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/max8925_bl.ko 2024-09-10T21:10:13.466 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 13105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/pandora_bl.ko 2024-09-10T21:10:13.466 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 13137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/pcf50633-backlight.ko 2024-09-10T21:10:13.466 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 13529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/platform_lcd.ko 2024-09-10T21:10:13.466 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 15201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/wm831x_bl.ko 2024-09-10T21:10:13.466 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/ 2024-09-10T21:10:13.466 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ 2024-09-10T21:10:13.466 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 23129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ds2482.ko 2024-09-10T21:10:13.466 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 46913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ds2490.ko 2024-09-10T21:10:13.466 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 13041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/matrox_w1.ko 2024-09-10T21:10:13.466 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 14209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/w1-gpio.ko 2024-09-10T21:10:13.466 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/ 2024-09-10T21:10:13.466 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 14089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-09-10T21:10:13.467 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 27337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-09-10T21:10:13.467 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 16945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-09-10T21:10:13.467 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 12209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-09-10T21:10:13.467 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 16537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-09-10T21:10:13.467 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 16297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-09-10T21:10:13.467 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 16177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-09-10T21:10:13.467 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 16177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-09-10T21:10:13.467 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 21609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-09-10T21:10:13.467 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 8393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_smem.ko 2024-09-10T21:10:13.467 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 70473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_therm.ko 2024-09-10T21:10:13.467 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 127601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/wire.ko 2024-09-10T21:10:13.467 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ 2024-09-10T21:10:13.467 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 22425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/acquirewdt.ko 2024-09-10T21:10:13.468 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 23913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/advantechwdt.ko 2024-09-10T21:10:13.468 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 22289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/alim1535_wdt.ko 2024-09-10T21:10:13.468 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 27481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/alim7101_wdt.ko 2024-09-10T21:10:13.468 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 22081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/cpu5wdt.ko 2024-09-10T21:10:13.468 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 16545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/da9052_wdt.ko 2024-09-10T21:10:13.468 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 14713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/da9055_wdt.ko 2024-09-10T21:10:13.468 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 20993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/da9063_wdt.ko 2024-09-10T21:10:13.468 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 24545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/dw_wdt.ko 2024-09-10T21:10:13.468 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 23609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/eurotechwdt.ko 2024-09-10T21:10:13.468 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 26841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/f71808e_wdt.ko 2024-09-10T21:10:13.468 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 33009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/hpwdt.ko 2024-09-10T21:10:13.468 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 18129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/i6300esb.ko 2024-09-10T21:10:13.468 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 12201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-09-10T21:10:13.469 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 26817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/iTCO_wdt.ko 2024-09-10T21:10:13.469 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 23377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ib700wdt.ko 2024-09-10T21:10:13.469 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 21809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ibmasr.ko 2024-09-10T21:10:13.469 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 22793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-09-10T21:10:13.469 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 24937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/it8712f_wdt.ko 2024-09-10T21:10:13.469 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 16009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/it87_wdt.ko 2024-09-10T21:10:13.469 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 26465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/kempld_wdt.ko 2024-09-10T21:10:13.469 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 21649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/machzwd.ko 2024-09-10T21:10:13.469 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 17185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/mena21_wdt.ko 2024-09-10T21:10:13.469 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 15145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/menf21bmc_wdt.ko 2024-09-10T21:10:13.469 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 29841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/nv_tco.ko 2024-09-10T21:10:13.469 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 20657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/of_xilinx_wdt.ko 2024-09-10T21:10:13.469 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 27809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pc87413_wdt.ko 2024-09-10T21:10:13.470 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 47033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pcwd_pci.ko 2024-09-10T21:10:13.470 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 37289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pcwd_usb.ko 2024-09-10T21:10:13.470 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 15257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/retu_wdt.ko 2024-09-10T21:10:13.470 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 24521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sbc60xxwdt.ko 2024-09-10T21:10:13.470 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 17065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sbc_epx_c3.ko 2024-09-10T21:10:13.470 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 20433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-09-10T21:10:13.470 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 29257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sc1200wdt.ko 2024-09-10T21:10:13.470 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 27569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sch311x_wdt.ko 2024-09-10T21:10:13.470 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 25705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/smsc37b787_wdt.ko 2024-09-10T21:10:13.470 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 19801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/softdog.ko 2024-09-10T21:10:13.470 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 26089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sp5100_tco.ko 2024-09-10T21:10:13.470 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 15001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/twl4030_wdt.ko 2024-09-10T21:10:13.470 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 20257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/via_wdt.ko 2024-09-10T21:10:13.470 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 23481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-09-10T21:10:13.471 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 24985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83877f_wdt.ko 2024-09-10T21:10:13.471 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 32569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83977f_wdt.ko 2024-09-10T21:10:13.471 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 22657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/wafer5823wdt.ko 2024-09-10T21:10:13.471 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 33545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/wdt_pci.ko 2024-09-10T21:10:13.471 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 18985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/wm831x_wdt.ko 2024-09-10T21:10:13.471 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 16161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/wm8350_wdt.ko 2024-09-10T21:10:13.471 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 18537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/xen_wdt.ko 2024-09-10T21:10:13.471 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/ 2024-09-10T21:10:13.471 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 30225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-evtchn.ko 2024-09-10T21:10:13.471 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 30569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-gntalloc.ko 2024-09-10T21:10:13.471 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 55425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-gntdev.ko 2024-09-10T21:10:13.471 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-pciback/ 2024-09-10T21:10:13.471 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 214065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-09-10T21:10:13.472 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 46705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-privcmd.ko 2024-09-10T21:10:13.472 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 85905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-scsiback.ko 2024-09-10T21:10:13.472 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xenfs/ 2024-09-10T21:10:13.472 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 23537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xenfs/xenfs.ko 2024-09-10T21:10:13.472 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ 2024-09-10T21:10:13.472 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/9p/ 2024-09-10T21:10:13.472 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 195161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/9p/9p.ko 2024-09-10T21:10:13.472 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/adfs/ 2024-09-10T21:10:13.472 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 78369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/adfs/adfs.ko 2024-09-10T21:10:13.472 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/affs/ 2024-09-10T21:10:13.472 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 197849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/affs/affs.ko 2024-09-10T21:10:13.472 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/afs/ 2024-09-10T21:10:13.472 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 1259073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/afs/kafs.ko 2024-09-10T21:10:13.473 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/autofs/ 2024-09-10T21:10:13.473 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 127665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/autofs/autofs4.ko 2024-09-10T21:10:13.473 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/befs/ 2024-09-10T21:10:13.473 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 80561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/befs/befs.ko 2024-09-10T21:10:13.473 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/bfs/ 2024-09-10T21:10:13.473 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 50577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/bfs/bfs.ko 2024-09-10T21:10:13.473 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 45705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/binfmt_misc.ko 2024-09-10T21:10:13.473 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/btrfs/ 2024-09-10T21:10:13.473 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 4575041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/btrfs/btrfs.ko 2024-09-10T21:10:13.473 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cachefiles/ 2024-09-10T21:10:13.473 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 384329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cachefiles/cachefiles.ko 2024-09-10T21:10:13.473 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ceph/ 2024-09-10T21:10:13.473 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 1591681 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ceph/ceph.ko 2024-09-10T21:10:13.473 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/coda/ 2024-09-10T21:10:13.474 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 130593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/coda/coda.ko 2024-09-10T21:10:13.474 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/configfs/ 2024-09-10T21:10:13.474 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 137313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/configfs/configfs.ko 2024-09-10T21:10:13.474 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cramfs/ 2024-09-10T21:10:13.474 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 35369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cramfs/cramfs.ko 2024-09-10T21:10:13.474 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/dlm/ 2024-09-10T21:10:13.474 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 841817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/dlm/dlm.ko 2024-09-10T21:10:13.480 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/efs/ 2024-09-10T21:10:13.480 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 45713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/efs/efs.ko 2024-09-10T21:10:13.480 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/f2fs/ 2024-09-10T21:10:13.480 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 1901633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/f2fs/f2fs.ko 2024-09-10T21:10:13.480 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/ 2024-09-10T21:10:13.480 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 36809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/msdos.ko 2024-09-10T21:10:13.480 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/freevxfs/ 2024-09-10T21:10:13.480 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 53401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/freevxfs/freevxfs.ko 2024-09-10T21:10:13.480 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse/ 2024-09-10T21:10:13.480 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 29281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse/cuse.ko 2024-09-10T21:10:13.481 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/gfs2/ 2024-09-10T21:10:13.481 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 1488329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/gfs2/gfs2.ko 2024-09-10T21:10:13.481 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfs/ 2024-09-10T21:10:13.481 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 183553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfs/hfs.ko 2024-09-10T21:10:13.481 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfsplus/ 2024-09-10T21:10:13.481 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 282177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfsplus/hfsplus.ko 2024-09-10T21:10:13.481 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hpfs/ 2024-09-10T21:10:13.481 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 248625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hpfs/hpfs.ko 2024-09-10T21:10:13.481 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/isofs/ 2024-09-10T21:10:13.481 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 93513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/isofs/isofs.ko 2024-09-10T21:10:13.481 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/jfs/ 2024-09-10T21:10:13.481 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 529833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/jfs/jfs.ko 2024-09-10T21:10:13.481 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/lockd/ 2024-09-10T21:10:13.482 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 359969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/lockd/lockd.ko 2024-09-10T21:10:13.482 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/minix/ 2024-09-10T21:10:13.482 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 113265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/minix/minix.ko 2024-09-10T21:10:13.482 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/netfs/ 2024-09-10T21:10:13.482 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 646793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/netfs/netfs.ko 2024-09-10T21:10:13.482 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/ 2024-09-10T21:10:13.482 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/blocklayout/ 2024-09-10T21:10:13.482 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 111585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-09-10T21:10:13.482 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/filelayout/ 2024-09-10T21:10:13.482 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 74345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-09-10T21:10:13.482 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/flexfilelayout/ 2024-09-10T21:10:13.482 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 147553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-09-10T21:10:13.482 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 1291361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfs.ko 2024-09-10T21:10:13.483 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 83001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv2.ko 2024-09-10T21:10:13.483 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 152537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv3.ko 2024-09-10T21:10:13.483 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 2526953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv4.ko 2024-09-10T21:10:13.483 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/ 2024-09-10T21:10:13.483 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 19457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/grace.ko 2024-09-10T21:10:13.483 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 19009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/nfs_acl.ko 2024-09-10T21:10:13.483 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfsd/ 2024-09-10T21:10:13.483 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 2119673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfsd/nfsd.ko 2024-09-10T21:10:13.483 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nilfs2/ 2024-09-10T21:10:13.483 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 734385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nilfs2/nilfs2.ko 2024-09-10T21:10:13.483 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/ 2024-09-10T21:10:13.483 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 15177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-celtic.ko 2024-09-10T21:10:13.483 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 14553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-centeuro.ko 2024-09-10T21:10:13.484 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 15177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-croatian.ko 2024-09-10T21:10:13.484 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 14249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-cyrillic.ko 2024-09-10T21:10:13.484 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 14865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-gaelic.ko 2024-09-10T21:10:13.484 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 14177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-greek.ko 2024-09-10T21:10:13.484 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 15177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-iceland.ko 2024-09-10T21:10:13.484 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 14489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-inuit.ko 2024-09-10T21:10:13.484 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 15417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-roman.ko 2024-09-10T21:10:13.484 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 15177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-romanian.ko 2024-09-10T21:10:13.484 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 15177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-turkish.ko 2024-09-10T21:10:13.484 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 12617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_ascii.ko 2024-09-10T21:10:13.484 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 13921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1250.ko 2024-09-10T21:10:13.484 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 13617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1251.ko 2024-09-10T21:10:13.485 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 14305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1255.ko 2024-09-10T21:10:13.485 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 13857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp737.ko 2024-09-10T21:10:13.485 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 13921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp775.ko 2024-09-10T21:10:13.485 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 13545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp850.ko 2024-09-10T21:10:13.485 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 13609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp852.ko 2024-09-10T21:10:13.485 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 13609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp855.ko 2024-09-10T21:10:13.485 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 13233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp857.ko 2024-09-10T21:10:13.485 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 14233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp860.ko 2024-09-10T21:10:13.485 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 14545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp861.ko 2024-09-10T21:10:13.485 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 14793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp862.ko 2024-09-10T21:10:13.485 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 14545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp863.ko 2024-09-10T21:10:13.485 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 14169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp864.ko 2024-09-10T21:10:13.485 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 14545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp865.ko 2024-09-10T21:10:13.486 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 13921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp866.ko 2024-09-10T21:10:13.486 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 13545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp869.ko 2024-09-10T21:10:13.486 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 13369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp874.ko 2024-09-10T21:10:13.486 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 95761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp932.ko 2024-09-10T21:10:13.486 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 144409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp936.ko 2024-09-10T21:10:13.486 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 167273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp949.ko 2024-09-10T21:10:13.486 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 119649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp950.ko 2024-09-10T21:10:13.486 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 14249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_euc-jp.ko 2024-09-10T21:10:13.486 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 12689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-1.ko 2024-09-10T21:10:13.486 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 13313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-13.ko 2024-09-10T21:10:13.486 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 13313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-14.ko 2024-09-10T21:10:13.486 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 13313 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-15.ko 2024-09-10T21:10:13.486 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 13305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-2.ko 2024-09-10T21:10:13.487 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 13305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-3.ko 2024-09-10T21:10:13.487 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 13305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-4.ko 2024-09-10T21:10:13.487 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 13305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-5.ko 2024-09-10T21:10:13.487 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 13001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-6.ko 2024-09-10T21:10:13.487 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 13617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-7.ko 2024-09-10T21:10:13.487 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 13001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-9.ko 2024-09-10T21:10:13.487 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 13857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-r.ko 2024-09-10T21:10:13.487 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 9969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-ru.ko 2024-09-10T21:10:13.487 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 13857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-u.ko 2024-09-10T21:10:13.487 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 7353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_ucs2_utils.ko 2024-09-10T21:10:13.487 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 9801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_utf8.ko 2024-09-10T21:10:13.487 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ntfs3/ 2024-09-10T21:10:13.487 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 632217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ntfs3/ntfs3.ko 2024-09-10T21:10:13.487 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ 2024-09-10T21:10:13.488 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/cluster/ 2024-09-10T21:10:13.488 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 299761 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-09-10T21:10:13.488 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlm/ 2024-09-10T21:10:13.488 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 547233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-09-10T21:10:13.490 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlmfs/ 2024-09-10T21:10:13.490 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 66465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-09-10T21:10:13.490 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 3255305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2.ko 2024-09-10T21:10:13.490 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 22273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-09-10T21:10:13.490 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 39409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-09-10T21:10:13.490 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 41297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-09-10T21:10:13.490 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/omfs/ 2024-09-10T21:10:13.490 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 67633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/omfs/omfs.ko 2024-09-10T21:10:13.490 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/overlayfs/ 2024-09-10T21:10:13.490 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 468161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/overlayfs/overlay.ko 2024-09-10T21:10:13.491 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/pstore/ 2024-09-10T21:10:13.491 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 69433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/pstore/ramoops.ko 2024-09-10T21:10:13.491 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/qnx4/ 2024-09-10T21:10:13.491 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 33433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/qnx4/qnx4.ko 2024-09-10T21:10:13.491 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/qnx6/ 2024-09-10T21:10:13.491 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 58177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/qnx6/qnx6.ko 2024-09-10T21:10:13.491 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/quota/ 2024-09-10T21:10:13.491 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 39689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/quota/quota_tree.ko 2024-09-10T21:10:13.491 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 17721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/quota/quota_v1.ko 2024-09-10T21:10:13.491 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 28953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/quota/quota_v2.ko 2024-09-10T21:10:13.491 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/reiserfs/ 2024-09-10T21:10:13.491 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 684489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/reiserfs/reiserfs.ko 2024-09-10T21:10:13.491 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/romfs/ 2024-09-10T21:10:13.491 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 33433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/romfs/romfs.ko 2024-09-10T21:10:13.492 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/ 2024-09-10T21:10:13.492 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/client/ 2024-09-10T21:10:13.492 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 3333977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/client/cifs.ko 2024-09-10T21:10:13.494 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/ 2024-09-10T21:10:13.494 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 7905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/cifs_arc4.ko 2024-09-10T21:10:13.494 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 12497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/cifs_md4.ko 2024-09-10T21:10:13.494 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/squashfs/ 2024-09-10T21:10:13.494 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 163081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/squashfs/squashfs.ko 2024-09-10T21:10:13.494 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/sysv/ 2024-09-10T21:10:13.495 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 116065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/sysv/sysv.ko 2024-09-10T21:10:13.495 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/udf/ 2024-09-10T21:10:13.495 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 309513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/udf/udf.ko 2024-09-10T21:10:13.495 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ufs/ 2024-09-10T21:10:13.495 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 198129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ufs/ufs.ko 2024-09-10T21:10:13.496 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/xfs/ 2024-09-10T21:10:13.496 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 5454809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/xfs/xfs.ko 2024-09-10T21:10:13.506 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/ 2024-09-10T21:10:13.506 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/locking/ 2024-09-10T21:10:13.506 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 97945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/locking/locktorture.ko 2024-09-10T21:10:13.506 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/time/ 2024-09-10T21:10:13.506 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 16025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/time/test_udelay.ko 2024-09-10T21:10:13.506 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 70441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/torture.ko 2024-09-10T21:10:13.506 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ 2024-09-10T21:10:13.506 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 7097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc-itu-t.ko 2024-09-10T21:10:13.506 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 6793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc7.ko 2024-09-10T21:10:13.506 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 7585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc8.ko 2024-09-10T21:10:13.506 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/ 2024-09-10T21:10:13.506 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 7849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libarc4.ko 2024-09-10T21:10:13.507 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 38433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libdes.ko 2024-09-10T21:10:13.507 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 14777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/interval_tree_test.ko 2024-09-10T21:10:13.507 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 9265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/libcrc32c.ko 2024-09-10T21:10:13.507 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 27673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lru_cache.ko 2024-09-10T21:10:13.507 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/ 2024-09-10T21:10:13.507 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 35473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/lz4_compress.ko 2024-09-10T21:10:13.507 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 24265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/lz4hc_compress.ko 2024-09-10T21:10:13.507 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/math/ 2024-09-10T21:10:13.507 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 7145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/math/cordic.ko 2024-09-10T21:10:13.507 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 9257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/memory-notifier-error-inject.ko 2024-09-10T21:10:13.507 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 12241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/notifier-error-inject.ko 2024-09-10T21:10:13.507 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 16145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/percpu_test.ko 2024-09-10T21:10:13.507 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 9337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/pm-notifier-error-inject.ko 2024-09-10T21:10:13.508 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/raid6/ 2024-09-10T21:10:13.508 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 151073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/raid6/raid6_pq.ko 2024-09-10T21:10:13.508 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 24569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/rbtree_test.ko 2024-09-10T21:10:13.508 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/reed_solomon/ 2024-09-10T21:10:13.508 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 20897 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/reed_solomon/reed_solomon.ko 2024-09-10T21:10:13.508 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 41169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/test-kstrtox.ko 2024-09-10T21:10:13.508 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 4912673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/test_bpf.ko 2024-09-10T21:10:13.510 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 61249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/test_firmware.ko 2024-09-10T21:10:13.510 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 7321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/test_module.ko 2024-09-10T21:10:13.510 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 11537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_bm.ko 2024-09-10T21:10:13.511 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 13497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_fsm.ko 2024-09-10T21:10:13.511 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 11577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_kmp.ko 2024-09-10T21:10:13.511 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/xz/ 2024-09-10T21:10:13.511 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 15945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/xz/xz_dec_test.ko 2024-09-10T21:10:13.511 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/mm/ 2024-09-10T21:10:13.511 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 20385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/mm/hwpoison-inject.ko 2024-09-10T21:10:13.511 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ 2024-09-10T21:10:13.511 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/ 2024-09-10T21:10:13.511 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 66073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/6lowpan.ko 2024-09-10T21:10:13.511 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 7665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_dest.ko 2024-09-10T21:10:13.511 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 7609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_fragment.ko 2024-09-10T21:10:13.511 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 7657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_hop.ko 2024-09-10T21:10:13.512 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 7601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_ipv6.ko 2024-09-10T21:10:13.512 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 7609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_mobility.ko 2024-09-10T21:10:13.512 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 7609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_routing.ko 2024-09-10T21:10:13.512 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 17305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_udp.ko 2024-09-10T21:10:13.512 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/ 2024-09-10T21:10:13.512 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 36089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/garp.ko 2024-09-10T21:10:13.512 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 41673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/mrp.ko 2024-09-10T21:10:13.512 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 9633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/p8022.ko 2024-09-10T21:10:13.512 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 19113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/psnap.ko 2024-09-10T21:10:13.512 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 13249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/stp.ko 2024-09-10T21:10:13.512 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/8021q/ 2024-09-10T21:10:13.512 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 106841 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/8021q/8021q.ko 2024-09-10T21:10:13.512 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/ 2024-09-10T21:10:13.513 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 234745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet.ko 2024-09-10T21:10:13.513 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 45337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_fd.ko 2024-09-10T21:10:13.513 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 32305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_rdma.ko 2024-09-10T21:10:13.513 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 39593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_virtio.ko 2024-09-10T21:10:13.513 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/batman-adv/ 2024-09-10T21:10:13.513 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 875441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/batman-adv/batman-adv.ko 2024-09-10T21:10:13.517 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/ 2024-09-10T21:10:13.517 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 88073 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/br_netfilter.ko 2024-09-10T21:10:13.517 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 1026825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/bridge.ko 2024-09-10T21:10:13.517 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ 2024-09-10T21:10:13.517 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 8993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-09-10T21:10:13.518 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 14649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_among.ko 2024-09-10T21:10:13.518 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 10537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_arp.ko 2024-09-10T21:10:13.518 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 10505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-09-10T21:10:13.518 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 11041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-09-10T21:10:13.518 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 10089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_ip.ko 2024-09-10T21:10:13.518 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 10417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-09-10T21:10:13.518 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 11233 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_limit.ko 2024-09-10T21:10:13.518 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 15465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_log.ko 2024-09-10T21:10:13.518 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 8921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_mark.ko 2024-09-10T21:10:13.518 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 8865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-09-10T21:10:13.518 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 9657 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-09-10T21:10:13.518 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 8713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-09-10T21:10:13.518 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 10993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-09-10T21:10:13.519 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 10041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_snat.ko 2024-09-10T21:10:13.519 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 10185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_stp.ko 2024-09-10T21:10:13.519 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 13793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-09-10T21:10:13.519 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 13337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-09-10T21:10:13.519 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 11297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-09-10T21:10:13.519 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 11273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-09-10T21:10:13.519 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 69849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtables.ko 2024-09-10T21:10:13.519 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 16201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-09-10T21:10:13.519 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 16297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-09-10T21:10:13.519 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ceph/ 2024-09-10T21:10:13.519 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 1229873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ceph/libceph.ko 2024-09-10T21:10:13.519 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core/ 2024-09-10T21:10:13.520 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 154121 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core/pktgen.ko 2024-09-10T21:10:13.520 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/ 2024-09-10T21:10:13.520 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 239393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp.ko 2024-09-10T21:10:13.520 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 10353 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_diag.ko 2024-09-10T21:10:13.520 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 69873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_ipv4.ko 2024-09-10T21:10:13.520 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 74433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_ipv6.ko 2024-09-10T21:10:13.520 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/ 2024-09-10T21:10:13.520 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 498249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/dsa_core.ko 2024-09-10T21:10:13.539 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 15929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_brcm.ko 2024-09-10T21:10:13.539 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 15009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_dsa.ko 2024-09-10T21:10:13.539 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 8961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_none.ko 2024-09-10T21:10:13.539 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 10545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_trailer.ko 2024-09-10T21:10:13.539 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/hsr/ 2024-09-10T21:10:13.539 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 137945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/hsr/hsr.ko 2024-09-10T21:10:13.540 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ 2024-09-10T21:10:13.540 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/6lowpan/ 2024-09-10T21:10:13.540 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 55409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-09-10T21:10:13.540 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 356385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ieee802154.ko 2024-09-10T21:10:13.540 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 59457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ieee802154_socket.ko 2024-09-10T21:10:13.540 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ 2024-09-10T21:10:13.540 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 29169 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ah4.ko 2024-09-10T21:10:13.540 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 46153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/esp4.ko 2024-09-10T21:10:13.540 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 59793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/fou.ko 2024-09-10T21:10:13.540 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 21705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/gre.ko 2024-09-10T21:10:13.540 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 64777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/inet_diag.ko 2024-09-10T21:10:13.540 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 77777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_gre.ko 2024-09-10T21:10:13.541 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 78481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_tunnel.ko 2024-09-10T21:10:13.541 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 43153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_vti.ko 2024-09-10T21:10:13.541 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 16625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ipcomp.ko 2024-09-10T21:10:13.541 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 34249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ipip.ko 2024-09-10T21:10:13.541 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ 2024-09-10T21:10:13.541 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 44921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arp_tables.ko 2024-09-10T21:10:13.541 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 10225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-09-10T21:10:13.541 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 11985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-09-10T21:10:13.541 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 49609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ip_tables.ko 2024-09-10T21:10:13.541 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 11513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-09-10T21:10:13.541 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 11537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_REJECT.ko 2024-09-10T21:10:13.541 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 16441 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-09-10T21:10:13.541 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 12601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-09-10T21:10:13.542 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 17377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-09-10T21:10:13.542 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 13081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_filter.ko 2024-09-10T21:10:13.542 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 13465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-09-10T21:10:13.542 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 17529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-09-10T21:10:13.542 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 13329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-09-10T21:10:13.542 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 12057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_security.ko 2024-09-10T21:10:13.542 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 14265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-09-10T21:10:13.542 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 9577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-09-10T21:10:13.542 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 32561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-09-10T21:10:13.542 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 20825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-09-10T21:10:13.542 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 15105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-09-10T21:10:13.542 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 24961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_reject_ipv4.ko 2024-09-10T21:10:13.543 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 10105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-09-10T21:10:13.543 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 16769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-09-10T21:10:13.543 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 11545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-09-10T21:10:13.543 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 19289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-09-10T21:10:13.543 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 10153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-09-10T21:10:13.543 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 14769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_bic.ko 2024-09-10T21:10:13.543 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 16817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_dctcp.ko 2024-09-10T21:10:13.543 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 19489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_diag.ko 2024-09-10T21:10:13.543 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 11105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_highspeed.ko 2024-09-10T21:10:13.543 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 14473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_htcp.ko 2024-09-10T21:10:13.543 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 11753 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_hybla.ko 2024-09-10T21:10:13.543 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 13537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_illinois.ko 2024-09-10T21:10:13.543 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 11825 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_lp.ko 2024-09-10T21:10:13.543 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 9689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_scalable.ko 2024-09-10T21:10:13.544 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 16113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_vegas.ko 2024-09-10T21:10:13.544 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 12193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_veno.ko 2024-09-10T21:10:13.544 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 13345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_westwood.ko 2024-09-10T21:10:13.544 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 11833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_yeah.ko 2024-09-10T21:10:13.544 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 28345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tunnel4.ko 2024-09-10T21:10:13.544 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 21201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/udp_diag.ko 2024-09-10T21:10:13.544 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 53401 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/udp_tunnel.ko 2024-09-10T21:10:13.544 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 13937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/xfrm4_tunnel.ko 2024-09-10T21:10:13.544 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ 2024-09-10T21:10:13.544 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 31201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ah6.ko 2024-09-10T21:10:13.544 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 47193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/esp6.ko 2024-09-10T21:10:13.544 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 17201 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/fou6.ko 2024-09-10T21:10:13.544 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 102889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_gre.ko 2024-09-10T21:10:13.544 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 117649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_tunnel.ko 2024-09-10T21:10:13.545 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 15953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-09-10T21:10:13.545 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 69881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_vti.ko 2024-09-10T21:10:13.545 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 16361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ipcomp6.ko 2024-09-10T21:10:13.545 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 23097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/mip6.ko 2024-09-10T21:10:13.545 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ 2024-09-10T21:10:13.545 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 50137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-09-10T21:10:13.545 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 13913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_NPT.ko 2024-09-10T21:10:13.545 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 11673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-09-10T21:10:13.545 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 16449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-09-10T21:10:13.545 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 16009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-09-10T21:10:13.545 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 8809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-09-10T21:10:13.545 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 18041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-09-10T21:10:13.545 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 17713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-09-10T21:10:13.545 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 10129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-09-10T21:10:13.546 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 11225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-09-10T21:10:13.546 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 12561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-09-10T21:10:13.546 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 12273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-09-10T21:10:13.546 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 13089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-09-10T21:10:13.546 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 12937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-09-10T21:10:13.546 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 17537 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-09-10T21:10:13.546 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 13337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-09-10T21:10:13.546 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 12065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-09-10T21:10:13.546 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 46033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-09-10T21:10:13.546 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 9225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-09-10T21:10:13.546 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 29017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-09-10T21:10:13.546 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 10617 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-09-10T21:10:13.546 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 16873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-09-10T21:10:13.546 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 11185 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-09-10T21:10:13.547 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 18305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-09-10T21:10:13.547 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 10153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-09-10T21:10:13.547 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 94433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/sit.ko 2024-09-10T21:10:13.547 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 30065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/tunnel6.ko 2024-09-10T21:10:13.547 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 28137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/xfrm6_tunnel.ko 2024-09-10T21:10:13.547 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/key/ 2024-09-10T21:10:13.547 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 106217 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/key/af_key.ko 2024-09-10T21:10:13.547 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/ 2024-09-10T21:10:13.547 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 155593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_core.ko 2024-09-10T21:10:13.547 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 24489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_debugfs.ko 2024-09-10T21:10:13.547 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 32137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_eth.ko 2024-09-10T21:10:13.547 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 43553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ip.ko 2024-09-10T21:10:13.547 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 45057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ip6.ko 2024-09-10T21:10:13.547 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 42041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_netlink.ko 2024-09-10T21:10:13.548 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 66137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ppp.ko 2024-09-10T21:10:13.548 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/lapb/ 2024-09-10T21:10:13.548 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 61609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/lapb/lapb.ko 2024-09-10T21:10:13.548 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc/ 2024-09-10T21:10:13.548 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 26481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc/llc.ko 2024-09-10T21:10:13.548 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 333641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc/llc2.ko 2024-09-10T21:10:13.548 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mac802154/ 2024-09-10T21:10:13.548 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 388577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mac802154/mac802154.ko 2024-09-10T21:10:13.548 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mpls/ 2024-09-10T21:10:13.548 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 10665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mpls/mpls_gso.ko 2024-09-10T21:10:13.548 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ 2024-09-10T21:10:13.548 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ 2024-09-10T21:10:13.548 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 144881 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set.ko 2024-09-10T21:10:13.548 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 34561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-09-10T21:10:13.549 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 33153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-09-10T21:10:13.549 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 32633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-09-10T21:10:13.549 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 124137 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-09-10T21:10:13.549 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 124449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-09-10T21:10:13.549 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 126777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-09-10T21:10:13.549 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 126577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-09-10T21:10:13.549 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 137937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-09-10T21:10:13.549 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 72457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-09-10T21:10:13.549 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 133785 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-09-10T21:10:13.549 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 139177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-09-10T21:10:13.549 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 138369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-09-10T21:10:13.549 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 136281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-09-10T21:10:13.549 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 140001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-09-10T21:10:13.550 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 38849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-09-10T21:10:13.550 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ 2024-09-10T21:10:13.550 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 532393 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs.ko 2024-09-10T21:10:13.550 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 14297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-09-10T21:10:13.550 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 8889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-09-10T21:10:13.550 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 20649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-09-10T21:10:13.550 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 20937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-09-10T21:10:13.550 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 24625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-09-10T21:10:13.550 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 8889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-09-10T21:10:13.550 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 8953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-09-10T21:10:13.550 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 12745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-09-10T21:10:13.550 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 10961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-09-10T21:10:13.550 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 9001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-09-10T21:10:13.550 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 16065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-09-10T21:10:13.550 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 9001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-09-10T21:10:13.550 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 12905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-09-10T21:10:13.551 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 54465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conncount.ko 2024-09-10T21:10:13.551 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 485769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack.ko 2024-09-10T21:10:13.551 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 17953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-09-10T21:10:13.551 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 11633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-09-10T21:10:13.551 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 37729 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-09-10T21:10:13.551 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 134177 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_h323.ko 2024-09-10T21:10:13.551 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 24385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_irc.ko 2024-09-10T21:10:13.551 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 9545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-09-10T21:10:13.551 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 140921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-09-10T21:10:13.551 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 38089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-09-10T21:10:13.551 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 19025 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_sane.ko 2024-09-10T21:10:13.551 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 79081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_sip.ko 2024-09-10T21:10:13.551 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 12417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-09-10T21:10:13.551 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 20945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-09-10T21:10:13.552 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 10993 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_dup_netdev.ko 2024-09-10T21:10:13.552 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 119745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_flow_table.ko 2024-09-10T21:10:13.552 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 11929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_flow_table_inet.ko 2024-09-10T21:10:13.552 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 46193 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_log_syslog.ko 2024-09-10T21:10:13.552 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 115817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat.ko 2024-09-10T21:10:13.552 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 11457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_amanda.ko 2024-09-10T21:10:13.552 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 14273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_ftp.ko 2024-09-10T21:10:13.552 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 13433 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_irc.ko 2024-09-10T21:10:13.552 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 34697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_sip.ko 2024-09-10T21:10:13.552 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 9977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_tftp.ko 2024-09-10T21:10:13.552 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 51953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_synproxy_core.ko 2024-09-10T21:10:13.552 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 813705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_tables.ko 2024-09-10T21:10:13.552 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 52913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink.ko 2024-09-10T21:10:13.552 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 36049 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_acct.ko 2024-09-10T21:10:13.553 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 37241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_cttimeout.ko 2024-09-10T21:10:13.553 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 49457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_log.ko 2024-09-10T21:10:13.553 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 26129 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_osf.ko 2024-09-10T21:10:13.553 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 66593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_queue.ko 2024-09-10T21:10:13.553 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 12081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_chain_nat.ko 2024-09-10T21:10:13.553 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 41849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_compat.ko 2024-09-10T21:10:13.553 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 50641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_ct.ko 2024-09-10T21:10:13.553 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 11161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_dup_netdev.ko 2024-09-10T21:10:13.553 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 13081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_fib.ko 2024-09-10T21:10:13.553 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 16833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_fwd_netdev.ko 2024-09-10T21:10:13.553 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 17329 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_hash.ko 2024-09-10T21:10:13.553 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 25385 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_limit.ko 2024-09-10T21:10:13.553 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 18017 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_log.ko 2024-09-10T21:10:13.553 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 17529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_masq.ko 2024-09-10T21:10:13.554 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 16953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_nat.ko 2024-09-10T21:10:13.554 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 16569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_numgen.ko 2024-09-10T21:10:13.554 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 14097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_osf.ko 2024-09-10T21:10:13.554 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 16921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_queue.ko 2024-09-10T21:10:13.554 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 16745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_redir.ko 2024-09-10T21:10:13.554 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 11593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_reject.ko 2024-09-10T21:10:13.554 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 11465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_reject_inet.ko 2024-09-10T21:10:13.554 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 15145 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_socket.ko 2024-09-10T21:10:13.554 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 25081 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_synproxy.ko 2024-09-10T21:10:13.554 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 17721 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_tproxy.ko 2024-09-10T21:10:13.554 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 104257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/x_tables.ko 2024-09-10T21:10:13.554 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 14281 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_AUDIT.ko 2024-09-10T21:10:13.554 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 10953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CHECKSUM.ko 2024-09-10T21:10:13.554 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 8713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CLASSIFY.ko 2024-09-10T21:10:13.555 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 13497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-09-10T21:10:13.555 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 21953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CT.ko 2024-09-10T21:10:13.555 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 12377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_DSCP.ko 2024-09-10T21:10:13.555 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 11153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_HL.ko 2024-09-10T21:10:13.555 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 15833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_HMARK.ko 2024-09-10T21:10:13.555 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 45305 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_IDLETIMER.ko 2024-09-10T21:10:13.555 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 16033 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_LED.ko 2024-09-10T21:10:13.555 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 13105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_LOG.ko 2024-09-10T21:10:13.555 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 13457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_MASQUERADE.ko 2024-09-10T21:10:13.555 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 13409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NETMAP.ko 2024-09-10T21:10:13.555 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 10937 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NFLOG.ko 2024-09-10T21:10:13.555 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 15449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NFQUEUE.ko 2024-09-10T21:10:13.555 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 22809 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_RATEEST.ko 2024-09-10T21:10:13.555 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 12633 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_REDIRECT.ko 2024-09-10T21:10:13.556 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 16777 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_SECMARK.ko 2024-09-10T21:10:13.556 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 19065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TCPMSS.ko 2024-09-10T21:10:13.556 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 11849 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-09-10T21:10:13.556 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 20793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TEE.ko 2024-09-10T21:10:13.556 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 19713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TPROXY.ko 2024-09-10T21:10:13.556 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 9529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TRACE.ko 2024-09-10T21:10:13.556 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 15377 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_addrtype.ko 2024-09-10T21:10:13.556 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 16665 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_bpf.ko 2024-09-10T21:10:13.556 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 20641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cgroup.ko 2024-09-10T21:10:13.556 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 12953 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cluster.ko 2024-09-10T21:10:13.556 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 8545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_comment.ko 2024-09-10T21:10:13.556 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 12041 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connbytes.ko 2024-09-10T21:10:13.556 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 13113 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connlabel.ko 2024-09-10T21:10:13.557 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 10705 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connlimit.ko 2024-09-10T21:10:13.557 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 15921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connmark.ko 2024-09-10T21:10:13.557 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 14297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_conntrack.ko 2024-09-10T21:10:13.557 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 9001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cpu.ko 2024-09-10T21:10:13.557 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 12585 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_dccp.ko 2024-09-10T21:10:13.557 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 9089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_devgroup.ko 2024-09-10T21:10:13.557 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 10345 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_dscp.ko 2024-09-10T21:10:13.557 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 12745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ecn.ko 2024-09-10T21:10:13.557 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 12961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_esp.ko 2024-09-10T21:10:13.557 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 52929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_hashlimit.ko 2024-09-10T21:10:13.557 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 12921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_helper.ko 2024-09-10T21:10:13.557 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 9601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_hl.ko 2024-09-10T21:10:13.557 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 13321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ipcomp.ko 2024-09-10T21:10:13.558 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 12945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_iprange.ko 2024-09-10T21:10:13.558 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 13497 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ipvs.ko 2024-09-10T21:10:13.558 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 19273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_l2tp.ko 2024-09-10T21:10:13.558 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 9225 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_length.ko 2024-09-10T21:10:13.558 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 11609 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_limit.ko 2024-09-10T21:10:13.558 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 8529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_mac.ko 2024-09-10T21:10:13.558 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 9969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_mark.ko 2024-09-10T21:10:13.558 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 13505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_multiport.ko 2024-09-10T21:10:13.558 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 16249 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_nat.ko 2024-09-10T21:10:13.558 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 11521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_nfacct.ko 2024-09-10T21:10:13.558 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 9521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_osf.ko 2024-09-10T21:10:13.558 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 11649 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_owner.ko 2024-09-10T21:10:13.558 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 11545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_physdev.ko 2024-09-10T21:10:13.558 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 8769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_pkttype.ko 2024-09-10T21:10:13.559 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 13641 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_policy.ko 2024-09-10T21:10:13.559 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 11001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_quota.ko 2024-09-10T21:10:13.559 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 11969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_rateest.ko 2024-09-10T21:10:13.559 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 9361 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_realm.ko 2024-09-10T21:10:13.559 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 43569 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_recent.ko 2024-09-10T21:10:13.559 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 14985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_sctp.ko 2024-09-10T21:10:13.559 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 36577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_set.ko 2024-09-10T21:10:13.559 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 19865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_socket.ko 2024-09-10T21:10:13.559 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 10833 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_state.ko 2024-09-10T21:10:13.559 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 10945 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_statistic.ko 2024-09-10T21:10:13.559 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 10105 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_string.ko 2024-09-10T21:10:13.559 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 9673 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_tcpmss.ko 2024-09-10T21:10:13.559 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 18529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_tcpudp.ko 2024-09-10T21:10:13.560 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 12529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_time.ko 2024-09-10T21:10:13.560 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 10697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_u32.ko 2024-09-10T21:10:13.560 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netlink/ 2024-09-10T21:10:13.560 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 17321 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netlink/netlink_diag.ko 2024-09-10T21:10:13.560 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/nsh/ 2024-09-10T21:10:13.560 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 13969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/nsh/nsh.ko 2024-09-10T21:10:13.560 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/ 2024-09-10T21:10:13.560 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 485961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/openvswitch.ko 2024-09-10T21:10:13.560 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 12097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-geneve.ko 2024-09-10T21:10:13.560 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 10961 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-gre.ko 2024-09-10T21:10:13.560 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 13985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-vxlan.ko 2024-09-10T21:10:13.560 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/packet/ 2024-09-10T21:10:13.560 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 16009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/packet/af_packet_diag.ko 2024-09-10T21:10:13.560 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/phonet/ 2024-09-10T21:10:13.561 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 141545 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/phonet/phonet.ko 2024-09-10T21:10:13.561 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 69001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/phonet/pn_pep.ko 2024-09-10T21:10:13.561 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/ 2024-09-10T21:10:13.561 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 314473 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds.ko 2024-09-10T21:10:13.561 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 202161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds_rdma.ko 2024-09-10T21:10:13.561 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 85265 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds_tcp.ko 2024-09-10T21:10:13.561 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rxrpc/ 2024-09-10T21:10:13.561 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 1024009 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rxrpc/rxrpc.ko 2024-09-10T21:10:13.568 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/ 2024-09-10T21:10:13.568 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 39097 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_csum.ko 2024-09-10T21:10:13.568 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 28817 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_gact.ko 2024-09-10T21:10:13.568 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 45505 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_mirred.ko 2024-09-10T21:10:13.568 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 27689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_nat.ko 2024-09-10T21:10:13.568 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 53577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_pedit.ko 2024-09-10T21:10:13.568 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 38801 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_police.ko 2024-09-10T21:10:13.568 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 24057 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_simple.ko 2024-09-10T21:10:13.568 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 43513 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_skbedit.ko 2024-09-10T21:10:13.568 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 45769 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_vlan.ko 2024-09-10T21:10:13.568 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 32289 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_basic.ko 2024-09-10T21:10:13.568 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 41561 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_bpf.ko 2024-09-10T21:10:13.569 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 23417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_cgroup.ko 2024-09-10T21:10:13.569 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 37409 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_flow.ko 2024-09-10T21:10:13.569 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 39697 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_fw.ko 2024-09-10T21:10:13.569 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 49417 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_route.ko 2024-09-10T21:10:13.569 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 79257 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_u32.ko 2024-09-10T21:10:13.569 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 8793 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_cmp.ko 2024-09-10T21:10:13.569 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 15481 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_ipset.ko 2024-09-10T21:10:13.569 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 48889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_meta.ko 2024-09-10T21:10:13.569 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 9337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_nbyte.ko 2024-09-10T21:10:13.569 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 13465 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_text.ko 2024-09-10T21:10:13.569 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 8457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_u32.ko 2024-09-10T21:10:13.569 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 25921 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_choke.ko 2024-09-10T21:10:13.570 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 25209 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_codel.ko 2024-09-10T21:10:13.570 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 43577 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_drr.ko 2024-09-10T21:10:13.570 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 43865 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_fq.ko 2024-09-10T21:10:13.570 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 39273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_fq_codel.ko 2024-09-10T21:10:13.570 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 42985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_gred.ko 2024-09-10T21:10:13.570 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 60913 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_hfsc.ko 2024-09-10T21:10:13.570 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 26689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_hhf.ko 2024-09-10T21:10:13.570 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 81065 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_htb.ko 2024-09-10T21:10:13.570 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 33745 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_ingress.ko 2024-09-10T21:10:13.570 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 36713 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_mqprio.ko 2024-09-10T21:10:13.570 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 11905 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_mqprio_lib.ko 2024-09-10T21:10:13.570 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 33153 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_multiq.ko 2024-09-10T21:10:13.570 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 44857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_netem.ko 2024-09-10T21:10:13.571 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 32425 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_pie.ko 2024-09-10T21:10:13.571 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 13593 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_plug.ko 2024-09-10T21:10:13.571 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 31889 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_prio.ko 2024-09-10T21:10:13.571 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 60337 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_qfq.ko 2024-09-10T21:10:13.571 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 40977 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_red.ko 2024-09-10T21:10:13.571 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 33521 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_sfb.ko 2024-09-10T21:10:13.571 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 38457 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_sfq.ko 2024-09-10T21:10:13.571 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 35857 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_tbf.ko 2024-09-10T21:10:13.571 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 34241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_teql.ko 2024-09-10T21:10:13.571 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/ 2024-09-10T21:10:13.571 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 1184689 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/sctp.ko 2024-09-10T21:10:13.571 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 24001 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/sctp_diag.ko 2024-09-10T21:10:13.571 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/ 2024-09-10T21:10:13.571 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/ 2024-09-10T21:10:13.572 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 426969 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-09-10T21:10:13.572 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 70273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-09-10T21:10:13.572 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 1895553 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/sunrpc.ko 2024-09-10T21:10:13.572 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/xprtrdma/ 2024-09-10T21:10:13.572 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 925089 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-09-10T21:10:13.572 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/ 2024-09-10T21:10:13.572 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 1122601 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/tipc.ko 2024-09-10T21:10:13.572 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 12241 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/tipc_diag.ko 2024-09-10T21:10:13.572 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/unix/ 2024-09-10T21:10:13.572 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 18273 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/unix/unix_diag.ko 2024-09-10T21:10:13.572 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/ 2024-09-10T21:10:13.572 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 101161 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-09-10T21:10:13.572 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 90369 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-09-10T21:10:13.573 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 120625 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock.ko 2024-09-10T21:10:13.573 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 12985 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock_diag.ko 2024-09-10T21:10:13.573 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 20489 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock_loopback.ko 2024-09-10T21:10:13.573 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/x25/ 2024-09-10T21:10:13.573 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 165529 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/x25/x25.ko 2024-09-10T21:10:13.573 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm/ 2024-09-10T21:10:13.573 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 25449 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm/xfrm_algo.ko 2024-09-10T21:10:13.573 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 22929 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm/xfrm_ipcomp.ko 2024-09-10T21:10:13.573 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 135873 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm/xfrm_user.ko 2024-09-10T21:10:13.573 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 649700 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.alias 2024-09-10T21:10:13.573 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 633853 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.alias.bin 2024-09-10T21:10:13.573 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 7850 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin 2024-09-10T21:10:13.573 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 23959 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.alias.bin 2024-09-10T21:10:13.574 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 10056 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.bin 2024-09-10T21:10:13.574 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 74053 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.modinfo 2024-09-10T21:10:13.574 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 147543 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep 2024-09-10T21:10:13.574 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 233297 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep.bin 2024-09-10T21:10:13.574 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 210 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.devname 2024-09-10T21:10:13.574 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 72756 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.order 2024-09-10T21:10:13.574 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 879 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.softdep 2024-09-10T21:10:13.574 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 261535 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.symbols 2024-09-10T21:10:13.574 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 316586 2024-09-06 14:57 ./lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.symbols.bin 2024-09-10T21:10:13.574 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./usr/ 2024-09-10T21:10:13.574 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./usr/share/ 2024-09-10T21:10:13.574 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./usr/share/doc/ 2024-09-10T21:10:13.574 INFO:teuthology.orchestra.run.smithi017.stdout:drwxr-xr-x root/root 0 2024-09-06 14:57 ./usr/share/doc/linux-image-6.11.0-rc5-g2ca44a2f35ca/ 2024-09-10T21:10:13.574 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 176 2024-09-06 14:57 ./usr/share/doc/linux-image-6.11.0-rc5-g2ca44a2f35ca/changelog.Debian.gz 2024-09-10T21:10:13.575 INFO:teuthology.orchestra.run.smithi017.stdout:-rw-r--r-- root/root 691 2024-09-06 14:57 ./usr/share/doc/linux-image-6.11.0-rc5-g2ca44a2f35ca/copyright 2024-09-10T21:10:13.579 DEBUG:teuthology.task.kernel:get_image_version: 6.11.0-rc5-g2ca44a2f35ca 2024-09-10T21:10:13.579 INFO:teuthology.task.kernel:searching for kernel 6.11.0-rc5-g2ca44a2f35ca 2024-09-10T21:10:13.579 DEBUG:teuthology.orchestra.run.smithi017:> egrep '(submenu|menuentry.*6.11.0-rc5-g2ca44a2f35ca).*{' /boot/grub/grub.cfg 2024-09-10T21:10:13.615 INFO:teuthology.orchestra.run.smithi017.stdout:submenu 'Advanced options for Ubuntu' $menuentry_id_option 'gnulinux-advanced-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2024-09-10T21:10:13.615 INFO:teuthology.orchestra.run.smithi017.stdout: menuentry 'Ubuntu, with Linux 6.11.0-rc5-g2ca44a2f35ca' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.11.0-rc5-g2ca44a2f35ca-advanced-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2024-09-10T21:10:13.615 INFO:teuthology.orchestra.run.smithi017.stdout: menuentry 'Ubuntu, with Linux 6.11.0-rc5-g2ca44a2f35ca (recovery mode)' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.11.0-rc5-g2ca44a2f35ca-recovery-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2024-09-10T21:10:13.616 INFO:teuthology.task.kernel:submenu_title:Advanced options for Ubuntu> 2024-09-10T21:10:13.616 INFO:teuthology.task.kernel:default_title:Ubuntu, with Linux 6.11.0-rc5-g2ca44a2f35ca 2024-09-10T21:10:13.616 DEBUG:teuthology.orchestra.run.smithi017:> /bin/echo -e 'cat </dev/null && sudo chmod a+x -- '/etc/grub.d/01_ceph_kernel.tmp~' && sudo mv -- '/etc/grub.d/01_ceph_kernel.tmp~' /etc/grub.d/01_ceph_kernel && sudo update-grub && rm /tmp/linux-image.deb && ( sleep 1 && sudo shutdown -r now & ) 2024-09-10T21:10:13.659 DEBUG:teuthology.task.kernel:Waiting for install on ubuntu@smithi017.front.sepia.ceph.com to complete... 2024-09-10T21:10:14.033 INFO:teuthology.orchestra.run.smithi017.stderr:Sourcing file `/etc/default/grub' 2024-09-10T21:10:14.034 INFO:teuthology.orchestra.run.smithi017.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-10T21:10:14.046 INFO:teuthology.orchestra.run.smithi017.stderr:Generating grub configuration file ... 2024-09-10T21:10:14.306 INFO:teuthology.orchestra.run.smithi017.stderr:Found linux image: /boot/vmlinuz-6.11.0-rc5-g2ca44a2f35ca 2024-09-10T21:10:14.313 INFO:teuthology.orchestra.run.smithi017.stderr:Found initrd image: /boot/initrd.img-6.11.0-rc5-g2ca44a2f35ca 2024-09-10T21:10:14.591 INFO:teuthology.orchestra.run.smithi017.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-10T21:10:14.596 INFO:teuthology.orchestra.run.smithi017.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-10T21:10:14.639 INFO:teuthology.orchestra.run.smithi017.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-10T21:10:14.645 INFO:teuthology.orchestra.run.smithi017.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-10T21:10:14.781 INFO:teuthology.orchestra.run.smithi017.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-10T21:10:14.782 INFO:teuthology.orchestra.run.smithi017.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-09-10T21:10:14.782 INFO:teuthology.orchestra.run.smithi017.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-10T21:10:14.794 INFO:teuthology.orchestra.run.smithi017.stderr:done 2024-09-10T21:10:42.573 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-09-10T21:10:42.574 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi031.front.sepia.ceph.com' 2024-09-10T21:10:42.575 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi031.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T21:10:45.909 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-09-10T21:10:45.909 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi017.front.sepia.ceph.com' 2024-09-10T21:10:45.909 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi017.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T21:11:04.253 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.17 2024-09-10T21:11:13.261 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi017.front.sepia.ceph.com' 2024-09-10T21:11:13.262 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi017.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T21:11:42.610 DEBUG:teuthology.orchestra.remote:timed out 2024-09-10T21:11:51.617 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi031.front.sepia.ceph.com' 2024-09-10T21:11:51.618 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi031.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T21:11:52.368 DEBUG:teuthology.orchestra.run.smithi031:> true 2024-09-10T21:11:53.414 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi031.front.sepia.ceph.com' 2024-09-10T21:11:53.414 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "6.11.0-rc5-g2ca44a2f35ca"... 2024-09-10T21:11:53.415 DEBUG:teuthology.orchestra.run.smithi031:> uname -r 2024-09-10T21:11:53.469 INFO:teuthology.orchestra.run.smithi031.stdout:6.11.0-rc5-g2ca44a2f35ca 2024-09-10T21:11:53.469 DEBUG:teuthology.task.kernel:current kernel version is 6.11.0-rc5-g2ca44a2f35ca vs 6.11.0-rc5-g2ca44a2f35ca 2024-09-10T21:11:53.469 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-10T21:11:53.469 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-09-10T21:11:54.470 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-09-10T21:11:54.470 DEBUG:teuthology.orchestra.run.smithi031:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-10T21:11:54.588 INFO:teuthology.orchestra.run.smithi031.stdout:ttyS1 2024-09-10T21:11:54.631 DEBUG:teuthology.parallel:result is None 2024-09-10T21:12:13.272 DEBUG:teuthology.orchestra.remote:timed out 2024-09-10T21:12:25.273 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi017.front.sepia.ceph.com' 2024-09-10T21:12:25.274 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi017.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T21:12:25.703 DEBUG:teuthology.orchestra.run.smithi017:> true 2024-09-10T21:12:26.319 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi017.front.sepia.ceph.com' 2024-09-10T21:12:26.319 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "6.11.0-rc5-g2ca44a2f35ca"... 2024-09-10T21:12:26.319 DEBUG:teuthology.orchestra.run.smithi017:> uname -r 2024-09-10T21:12:26.369 INFO:teuthology.orchestra.run.smithi017.stdout:6.11.0-rc5-g2ca44a2f35ca 2024-09-10T21:12:26.369 DEBUG:teuthology.task.kernel:current kernel version is 6.11.0-rc5-g2ca44a2f35ca vs 6.11.0-rc5-g2ca44a2f35ca 2024-09-10T21:12:26.369 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-10T21:12:26.369 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-09-10T21:12:27.370 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-09-10T21:12:27.370 DEBUG:teuthology.orchestra.run.smithi017:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-10T21:12:27.453 INFO:teuthology.orchestra.run.smithi017.stdout:ttyS1 2024-09-10T21:12:27.533 DEBUG:teuthology.parallel:result is None 2024-09-10T21:12:27.533 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-10T21:12:27.542 INFO:teuthology.task.internal:Creating test directory... 2024-09-10T21:12:27.542 DEBUG:teuthology.orchestra.run.smithi017:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-10T21:12:27.546 DEBUG:teuthology.orchestra.run.smithi031:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-10T21:12:27.556 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-10T21:12:27.596 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-10T21:12:27.660 INFO:teuthology.task.internal:Creating archive directory... 2024-09-10T21:12:27.660 DEBUG:teuthology.orchestra.run.smithi017:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-10T21:12:27.663 DEBUG:teuthology.orchestra.run.smithi031:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-10T21:12:27.690 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-10T21:12:27.696 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-10T21:12:27.697 DEBUG:teuthology.orchestra.run.smithi017:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-10T21:12:27.724 DEBUG:teuthology.orchestra.run.smithi031:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-10T21:12:27.751 INFO:teuthology.orchestra.run.smithi017.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T21:12:27.753 INFO:teuthology.orchestra.run.smithi031.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T21:12:27.766 INFO:teuthology.orchestra.run.smithi017.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T21:12:27.767 INFO:teuthology.orchestra.run.smithi031.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T21:12:27.769 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-10T21:12:27.776 INFO:teuthology.task.internal:Configuring sudo... 2024-09-10T21:12:27.776 DEBUG:teuthology.orchestra.run.smithi017:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-10T21:12:27.812 DEBUG:teuthology.orchestra.run.smithi031:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-10T21:12:27.837 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-10T21:12:27.846 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-10T21:12:27.847 DEBUG:teuthology.orchestra.run.smithi017:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-10T21:12:27.880 DEBUG:teuthology.orchestra.run.smithi031:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-10T21:12:27.893 DEBUG:teuthology.orchestra.run.smithi017:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T21:12:27.942 DEBUG:teuthology.orchestra.run.smithi017:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T21:12:27.994 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-09-10T21:12:27.994 DEBUG:teuthology.orchestra.run.smithi017:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-10T21:12:28.064 DEBUG:teuthology.orchestra.run.smithi031:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T21:12:28.073 DEBUG:teuthology.orchestra.run.smithi031:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T21:12:28.121 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-10T21:12:28.121 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-10T21:12:28.192 DEBUG:teuthology.orchestra.run.smithi017:> sudo service rsyslog restart 2024-09-10T21:12:28.195 DEBUG:teuthology.orchestra.run.smithi031:> sudo service rsyslog restart 2024-09-10T21:12:28.314 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-10T21:12:28.325 INFO:teuthology.task.internal:Starting timer... 2024-09-10T21:12:28.325 INFO:teuthology.run_tasks:Running task pcp... 2024-09-10T21:12:28.335 INFO:teuthology.run_tasks:Running task selinux... 2024-09-10T21:12:28.345 DEBUG:teuthology.task.selinux:Excluding smithi017: OS 'ubuntu' does not support SELinux 2024-09-10T21:12:28.345 DEBUG:teuthology.task.selinux:Excluding smithi031: OS 'ubuntu' does not support SELinux 2024-09-10T21:12:28.345 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-10T21:12:28.345 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2024-09-10T21:12:28.346 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-10T21:12:28.346 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-10T21:12:28.355 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-10T21:12:28.366 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-10T21:12:28.367 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi017.front.sepia.ceph.com,smithi031.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-10T21:20:50.291 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi017.front.sepia.ceph.com'), Remote(name='ubuntu@smithi031.front.sepia.ceph.com')] 2024-09-10T21:20:50.293 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi017.front.sepia.ceph.com' 2024-09-10T21:20:50.294 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi017.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T21:20:50.404 DEBUG:teuthology.orchestra.run.smithi017:> true 2024-09-10T21:20:50.544 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi017.front.sepia.ceph.com' 2024-09-10T21:20:50.545 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi031.front.sepia.ceph.com' 2024-09-10T21:20:50.545 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi031.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T21:20:50.634 DEBUG:teuthology.orchestra.run.smithi031:> true 2024-09-10T21:20:50.763 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi031.front.sepia.ceph.com' 2024-09-10T21:20:50.764 INFO:teuthology.run_tasks:Running task clock... 2024-09-10T21:20:50.776 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-10T21:20:50.777 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-10T21:20:50.777 DEBUG:teuthology.orchestra.run.smithi017:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T21:20:50.781 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-10T21:20:50.781 DEBUG:teuthology.orchestra.run.smithi031:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T21:20:50.820 INFO:teuthology.orchestra.run.smithi017.stdout:10 Sep 21:20:50 ntpd[18332]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-09-10T21:20:50.820 INFO:teuthology.orchestra.run.smithi017.stdout:10 Sep 21:20:50 ntpd[18332]: Command line: ntpd -gq 2024-09-10T21:20:50.820 INFO:teuthology.orchestra.run.smithi017.stdout:10 Sep 21:20:50 ntpd[18332]: ---------------------------------------------------- 2024-09-10T21:20:50.820 INFO:teuthology.orchestra.run.smithi017.stdout:10 Sep 21:20:50 ntpd[18332]: ntp-4 is maintained by Network Time Foundation, 2024-09-10T21:20:50.820 INFO:teuthology.orchestra.run.smithi017.stdout:10 Sep 21:20:50 ntpd[18332]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-09-10T21:20:50.820 INFO:teuthology.orchestra.run.smithi017.stdout:10 Sep 21:20:50 ntpd[18332]: corporation. Support and training for ntp-4 are 2024-09-10T21:20:50.820 INFO:teuthology.orchestra.run.smithi017.stdout:10 Sep 21:20:50 ntpd[18332]: available at https://www.nwtime.org/support 2024-09-10T21:20:50.820 INFO:teuthology.orchestra.run.smithi017.stdout:10 Sep 21:20:50 ntpd[18332]: ---------------------------------------------------- 2024-09-10T21:20:50.821 INFO:teuthology.orchestra.run.smithi017.stdout:10 Sep 21:20:50 ntpd[18332]: proto: precision = 0.043 usec (-24) 2024-09-10T21:20:50.821 INFO:teuthology.orchestra.run.smithi017.stdout:10 Sep 21:20:50 ntpd[18332]: basedate set to 2022-02-04 2024-09-10T21:20:50.821 INFO:teuthology.orchestra.run.smithi017.stdout:10 Sep 21:20:50 ntpd[18332]: gps base set to 2022-02-06 (week 2196) 2024-09-10T21:20:50.822 INFO:teuthology.orchestra.run.smithi017.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-10T21:20:50.822 INFO:teuthology.orchestra.run.smithi017.stdout:10 Sep 21:20:50 ntpd[18332]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-10T21:20:50.822 INFO:teuthology.orchestra.run.smithi017.stdout:10 Sep 21:20:50 ntpd[18332]: restrict ::: KOD does nothing without LIMITED. 2024-09-10T21:20:50.822 INFO:teuthology.orchestra.run.smithi017.stderr:restrict ::: KOD does nothing without LIMITED. 2024-09-10T21:20:50.823 INFO:teuthology.orchestra.run.smithi017.stdout:10 Sep 21:20:50 ntpd[18332]: Listen and drop on 0 v6wildcard [::]:123 2024-09-10T21:20:50.823 INFO:teuthology.orchestra.run.smithi017.stdout:10 Sep 21:20:50 ntpd[18332]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-09-10T21:20:50.824 INFO:teuthology.orchestra.run.smithi017.stdout:10 Sep 21:20:50 ntpd[18332]: Listen normally on 2 lo 127.0.0.1:123 2024-09-10T21:20:50.824 INFO:teuthology.orchestra.run.smithi017.stdout:10 Sep 21:20:50 ntpd[18332]: Listen normally on 3 ens1f0 172.21.15.17:123 2024-09-10T21:20:50.824 INFO:teuthology.orchestra.run.smithi017.stdout:10 Sep 21:20:50 ntpd[18332]: Listen normally on 4 lo [::1]:123 2024-09-10T21:20:50.824 INFO:teuthology.orchestra.run.smithi017.stdout:10 Sep 21:20:50 ntpd[18332]: Listen normally on 5 ens1f0 [fe80::ec4:7aff:fe59:d992%4]:123 2024-09-10T21:20:50.824 INFO:teuthology.orchestra.run.smithi017.stdout:10 Sep 21:20:50 ntpd[18332]: Listening on routing socket on fd #22 for interface updates 2024-09-10T21:20:50.846 INFO:teuthology.orchestra.run.smithi031.stdout:10 Sep 21:20:50 ntpd[18393]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-09-10T21:20:50.846 INFO:teuthology.orchestra.run.smithi031.stdout:10 Sep 21:20:50 ntpd[18393]: Command line: ntpd -gq 2024-09-10T21:20:50.846 INFO:teuthology.orchestra.run.smithi031.stdout:10 Sep 21:20:50 ntpd[18393]: ---------------------------------------------------- 2024-09-10T21:20:50.846 INFO:teuthology.orchestra.run.smithi031.stdout:10 Sep 21:20:50 ntpd[18393]: ntp-4 is maintained by Network Time Foundation, 2024-09-10T21:20:50.847 INFO:teuthology.orchestra.run.smithi031.stdout:10 Sep 21:20:50 ntpd[18393]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-09-10T21:20:50.847 INFO:teuthology.orchestra.run.smithi031.stdout:10 Sep 21:20:50 ntpd[18393]: corporation. Support and training for ntp-4 are 2024-09-10T21:20:50.847 INFO:teuthology.orchestra.run.smithi031.stdout:10 Sep 21:20:50 ntpd[18393]: available at https://www.nwtime.org/support 2024-09-10T21:20:50.847 INFO:teuthology.orchestra.run.smithi031.stdout:10 Sep 21:20:50 ntpd[18393]: ---------------------------------------------------- 2024-09-10T21:20:50.847 INFO:teuthology.orchestra.run.smithi031.stdout:10 Sep 21:20:50 ntpd[18393]: proto: precision = 0.045 usec (-24) 2024-09-10T21:20:50.848 INFO:teuthology.orchestra.run.smithi031.stdout:10 Sep 21:20:50 ntpd[18393]: basedate set to 2022-02-04 2024-09-10T21:20:50.848 INFO:teuthology.orchestra.run.smithi031.stdout:10 Sep 21:20:50 ntpd[18393]: gps base set to 2022-02-06 (week 2196) 2024-09-10T21:20:50.848 INFO:teuthology.orchestra.run.smithi031.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-10T21:20:50.848 INFO:teuthology.orchestra.run.smithi031.stdout:10 Sep 21:20:50 ntpd[18393]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-10T21:20:50.848 INFO:teuthology.orchestra.run.smithi031.stdout:10 Sep 21:20:50 ntpd[18393]: restrict ::: KOD does nothing without LIMITED. 2024-09-10T21:20:50.849 INFO:teuthology.orchestra.run.smithi031.stderr:restrict ::: KOD does nothing without LIMITED. 2024-09-10T21:20:50.850 INFO:teuthology.orchestra.run.smithi031.stdout:10 Sep 21:20:50 ntpd[18393]: Listen and drop on 0 v6wildcard [::]:123 2024-09-10T21:20:50.850 INFO:teuthology.orchestra.run.smithi031.stdout:10 Sep 21:20:50 ntpd[18393]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-09-10T21:20:50.850 INFO:teuthology.orchestra.run.smithi031.stdout:10 Sep 21:20:50 ntpd[18393]: Listen normally on 2 lo 127.0.0.1:123 2024-09-10T21:20:50.850 INFO:teuthology.orchestra.run.smithi031.stdout:10 Sep 21:20:50 ntpd[18393]: Listen normally on 3 ens1f0 172.21.15.31:123 2024-09-10T21:20:50.850 INFO:teuthology.orchestra.run.smithi031.stdout:10 Sep 21:20:50 ntpd[18393]: Listen normally on 4 lo [::1]:123 2024-09-10T21:20:50.850 INFO:teuthology.orchestra.run.smithi031.stdout:10 Sep 21:20:50 ntpd[18393]: Listen normally on 5 ens1f0 [fe80::ae1f:6bff:fea5:caba%3]:123 2024-09-10T21:20:50.850 INFO:teuthology.orchestra.run.smithi031.stdout:10 Sep 21:20:50 ntpd[18393]: Listening on routing socket on fd #22 for interface updates 2024-09-10T21:20:51.821 INFO:teuthology.orchestra.run.smithi017.stderr:10 Sep 21:20:51 ntpd[18332]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T21:20:51.821 INFO:teuthology.orchestra.run.smithi017.stderr:10 Sep 21:20:51 ntpd[18332]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T21:20:51.821 INFO:teuthology.orchestra.run.smithi017.stderr:10 Sep 21:20:51 ntpd[18332]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T21:20:51.821 INFO:teuthology.orchestra.run.smithi017.stderr:10 Sep 21:20:51 ntpd[18332]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T21:20:51.821 INFO:teuthology.orchestra.run.smithi017.stderr:10 Sep 21:20:51 ntpd[18332]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-10T21:20:51.821 INFO:teuthology.orchestra.run.smithi017.stderr:10 Sep 21:20:51 ntpd[18332]: can't open /var/log/ntpstats/peerstats.20240910: Permission denied 2024-09-10T21:20:51.847 INFO:teuthology.orchestra.run.smithi031.stderr:10 Sep 21:20:51 ntpd[18393]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T21:20:51.847 INFO:teuthology.orchestra.run.smithi031.stderr:10 Sep 21:20:51 ntpd[18393]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T21:20:51.847 INFO:teuthology.orchestra.run.smithi031.stderr:10 Sep 21:20:51 ntpd[18393]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T21:20:51.847 INFO:teuthology.orchestra.run.smithi031.stderr:10 Sep 21:20:51 ntpd[18393]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T21:20:51.847 INFO:teuthology.orchestra.run.smithi031.stderr:10 Sep 21:20:51 ntpd[18393]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-10T21:20:51.847 INFO:teuthology.orchestra.run.smithi031.stderr:10 Sep 21:20:51 ntpd[18393]: can't open /var/log/ntpstats/peerstats.20240910: Permission denied 2024-09-10T21:20:52.821 INFO:teuthology.orchestra.run.smithi017.stderr:10 Sep 21:20:52 ntpd[18332]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T21:20:52.821 INFO:teuthology.orchestra.run.smithi017.stderr:10 Sep 21:20:52 ntpd[18332]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T21:20:52.821 INFO:teuthology.orchestra.run.smithi017.stderr:10 Sep 21:20:52 ntpd[18332]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T21:20:52.821 INFO:teuthology.orchestra.run.smithi017.stderr:10 Sep 21:20:52 ntpd[18332]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T21:20:52.821 INFO:teuthology.orchestra.run.smithi017.stderr:10 Sep 21:20:52 ntpd[18332]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-10T21:20:52.821 INFO:teuthology.orchestra.run.smithi017.stderr:10 Sep 21:20:52 ntpd[18332]: can't open /var/log/ntpstats/peerstats.20240910: Permission denied 2024-09-10T21:20:53.821 INFO:teuthology.orchestra.run.smithi017.stderr:10 Sep 21:20:53 ntpd[18332]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T21:20:53.821 INFO:teuthology.orchestra.run.smithi017.stderr:10 Sep 21:20:53 ntpd[18332]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T21:20:53.821 INFO:teuthology.orchestra.run.smithi017.stderr:10 Sep 21:20:53 ntpd[18332]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T21:20:53.822 INFO:teuthology.orchestra.run.smithi017.stderr:10 Sep 21:20:53 ntpd[18332]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T21:20:53.822 INFO:teuthology.orchestra.run.smithi017.stderr:10 Sep 21:20:53 ntpd[18332]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-10T21:20:53.822 INFO:teuthology.orchestra.run.smithi017.stderr:10 Sep 21:20:53 ntpd[18332]: can't open /var/log/ntpstats/peerstats.20240910: Permission denied 2024-09-10T21:20:53.848 INFO:teuthology.orchestra.run.smithi031.stderr:10 Sep 21:20:53 ntpd[18393]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T21:20:53.848 INFO:teuthology.orchestra.run.smithi031.stderr:10 Sep 21:20:53 ntpd[18393]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T21:20:53.848 INFO:teuthology.orchestra.run.smithi031.stderr:10 Sep 21:20:53 ntpd[18393]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T21:20:53.848 INFO:teuthology.orchestra.run.smithi031.stderr:10 Sep 21:20:53 ntpd[18393]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T21:20:53.848 INFO:teuthology.orchestra.run.smithi031.stderr:10 Sep 21:20:53 ntpd[18393]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T21:20:53.849 INFO:teuthology.orchestra.run.smithi031.stderr:10 Sep 21:20:53 ntpd[18393]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T21:20:53.849 INFO:teuthology.orchestra.run.smithi031.stderr:10 Sep 21:20:53 ntpd[18393]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T21:20:53.849 INFO:teuthology.orchestra.run.smithi031.stderr:10 Sep 21:20:53 ntpd[18393]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T21:20:53.849 INFO:teuthology.orchestra.run.smithi031.stderr:10 Sep 21:20:53 ntpd[18393]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-10T21:20:53.849 INFO:teuthology.orchestra.run.smithi031.stderr:10 Sep 21:20:53 ntpd[18393]: can't open /var/log/ntpstats/peerstats.20240910: Permission denied 2024-09-10T21:20:53.849 INFO:teuthology.orchestra.run.smithi031.stderr:10 Sep 21:20:53 ntpd[18393]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T21:20:53.849 INFO:teuthology.orchestra.run.smithi031.stderr:10 Sep 21:20:53 ntpd[18393]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T21:20:53.849 INFO:teuthology.orchestra.run.smithi031.stderr:10 Sep 21:20:53 ntpd[18393]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-10T21:20:53.849 INFO:teuthology.orchestra.run.smithi031.stderr:10 Sep 21:20:53 ntpd[18393]: can't open /var/log/ntpstats/peerstats.20240910: Permission denied 2024-09-10T21:20:53.849 INFO:teuthology.orchestra.run.smithi031.stderr:10 Sep 21:20:53 ntpd[18393]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T21:20:53.849 INFO:teuthology.orchestra.run.smithi031.stderr:10 Sep 21:20:53 ntpd[18393]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T21:20:53.849 INFO:teuthology.orchestra.run.smithi031.stderr:10 Sep 21:20:53 ntpd[18393]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-10T21:20:53.849 INFO:teuthology.orchestra.run.smithi031.stderr:10 Sep 21:20:53 ntpd[18393]: can't open /var/log/ntpstats/peerstats.20240910: Permission denied 2024-09-10T21:20:54.821 INFO:teuthology.orchestra.run.smithi017.stderr:10 Sep 21:20:54 ntpd[18332]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T21:20:54.821 INFO:teuthology.orchestra.run.smithi017.stderr:10 Sep 21:20:54 ntpd[18332]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T21:20:54.821 INFO:teuthology.orchestra.run.smithi017.stderr:10 Sep 21:20:54 ntpd[18332]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T21:20:54.821 INFO:teuthology.orchestra.run.smithi017.stderr:10 Sep 21:20:54 ntpd[18332]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T21:20:54.821 INFO:teuthology.orchestra.run.smithi017.stderr:10 Sep 21:20:54 ntpd[18332]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-10T21:20:54.821 INFO:teuthology.orchestra.run.smithi017.stderr:10 Sep 21:20:54 ntpd[18332]: can't open /var/log/ntpstats/peerstats.20240910: Permission denied 2024-09-10T21:20:55.825 INFO:teuthology.orchestra.run.smithi017.stderr:10 Sep 21:20:55 ntpd[18332]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T21:20:55.825 INFO:teuthology.orchestra.run.smithi017.stderr:10 Sep 21:20:55 ntpd[18332]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T21:20:55.825 INFO:teuthology.orchestra.run.smithi017.stderr:10 Sep 21:20:55 ntpd[18332]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T21:20:55.825 INFO:teuthology.orchestra.run.smithi017.stderr:10 Sep 21:20:55 ntpd[18332]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T21:20:55.826 INFO:teuthology.orchestra.run.smithi017.stderr:10 Sep 21:20:55 ntpd[18332]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T21:20:55.826 INFO:teuthology.orchestra.run.smithi017.stderr:10 Sep 21:20:55 ntpd[18332]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T21:20:55.826 INFO:teuthology.orchestra.run.smithi017.stderr:10 Sep 21:20:55 ntpd[18332]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T21:20:55.826 INFO:teuthology.orchestra.run.smithi017.stderr:10 Sep 21:20:55 ntpd[18332]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T21:20:55.826 INFO:teuthology.orchestra.run.smithi017.stderr:10 Sep 21:20:55 ntpd[18332]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-10T21:20:55.826 INFO:teuthology.orchestra.run.smithi017.stderr:10 Sep 21:20:55 ntpd[18332]: can't open /var/log/ntpstats/peerstats.20240910: Permission denied 2024-09-10T21:20:55.826 INFO:teuthology.orchestra.run.smithi017.stderr:10 Sep 21:20:55 ntpd[18332]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T21:20:55.826 INFO:teuthology.orchestra.run.smithi017.stderr:10 Sep 21:20:55 ntpd[18332]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T21:20:55.826 INFO:teuthology.orchestra.run.smithi017.stderr:10 Sep 21:20:55 ntpd[18332]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-10T21:20:55.826 INFO:teuthology.orchestra.run.smithi017.stderr:10 Sep 21:20:55 ntpd[18332]: can't open /var/log/ntpstats/peerstats.20240910: Permission denied 2024-09-10T21:20:55.848 INFO:teuthology.orchestra.run.smithi031.stderr:10 Sep 21:20:55 ntpd[18393]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T21:20:55.848 INFO:teuthology.orchestra.run.smithi031.stderr:10 Sep 21:20:55 ntpd[18393]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T21:20:55.848 INFO:teuthology.orchestra.run.smithi031.stderr:10 Sep 21:20:55 ntpd[18393]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T21:20:55.849 INFO:teuthology.orchestra.run.smithi031.stderr:10 Sep 21:20:55 ntpd[18393]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T21:20:55.849 INFO:teuthology.orchestra.run.smithi031.stderr:10 Sep 21:20:55 ntpd[18393]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T21:20:55.849 INFO:teuthology.orchestra.run.smithi031.stderr:10 Sep 21:20:55 ntpd[18393]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T21:20:55.849 INFO:teuthology.orchestra.run.smithi031.stderr:10 Sep 21:20:55 ntpd[18393]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T21:20:55.849 INFO:teuthology.orchestra.run.smithi031.stderr:10 Sep 21:20:55 ntpd[18393]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T21:20:55.849 INFO:teuthology.orchestra.run.smithi031.stderr:10 Sep 21:20:55 ntpd[18393]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T21:20:55.849 INFO:teuthology.orchestra.run.smithi031.stderr:10 Sep 21:20:55 ntpd[18393]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T21:20:55.849 INFO:teuthology.orchestra.run.smithi031.stderr:10 Sep 21:20:55 ntpd[18393]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-10T21:20:55.849 INFO:teuthology.orchestra.run.smithi031.stderr:10 Sep 21:20:55 ntpd[18393]: can't open /var/log/ntpstats/peerstats.20240910: Permission denied 2024-09-10T21:20:55.849 INFO:teuthology.orchestra.run.smithi031.stderr:10 Sep 21:20:55 ntpd[18393]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T21:20:55.849 INFO:teuthology.orchestra.run.smithi031.stderr:10 Sep 21:20:55 ntpd[18393]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T21:20:55.849 INFO:teuthology.orchestra.run.smithi031.stderr:10 Sep 21:20:55 ntpd[18393]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-10T21:20:55.850 INFO:teuthology.orchestra.run.smithi031.stderr:10 Sep 21:20:55 ntpd[18393]: can't open /var/log/ntpstats/peerstats.20240910: Permission denied 2024-09-10T21:20:55.850 INFO:teuthology.orchestra.run.smithi031.stderr:10 Sep 21:20:55 ntpd[18393]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T21:20:55.850 INFO:teuthology.orchestra.run.smithi031.stderr:10 Sep 21:20:55 ntpd[18393]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T21:20:55.850 INFO:teuthology.orchestra.run.smithi031.stderr:10 Sep 21:20:55 ntpd[18393]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-10T21:20:55.850 INFO:teuthology.orchestra.run.smithi031.stderr:10 Sep 21:20:55 ntpd[18393]: can't open /var/log/ntpstats/peerstats.20240910: Permission denied 2024-09-10T21:20:56.821 INFO:teuthology.orchestra.run.smithi017.stderr:10 Sep 21:20:56 ntpd[18332]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T21:20:56.821 INFO:teuthology.orchestra.run.smithi017.stderr:10 Sep 21:20:56 ntpd[18332]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T21:20:56.821 INFO:teuthology.orchestra.run.smithi017.stderr:10 Sep 21:20:56 ntpd[18332]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T21:20:56.821 INFO:teuthology.orchestra.run.smithi017.stderr:10 Sep 21:20:56 ntpd[18332]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T21:20:56.821 INFO:teuthology.orchestra.run.smithi017.stderr:10 Sep 21:20:56 ntpd[18332]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-10T21:20:56.821 INFO:teuthology.orchestra.run.smithi017.stderr:10 Sep 21:20:56 ntpd[18332]: can't open /var/log/ntpstats/peerstats.20240910: Permission denied 2024-09-10T21:20:57.821 INFO:teuthology.orchestra.run.smithi017.stderr:10 Sep 21:20:57 ntpd[18332]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T21:20:57.821 INFO:teuthology.orchestra.run.smithi017.stderr:10 Sep 21:20:57 ntpd[18332]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T21:20:57.822 INFO:teuthology.orchestra.run.smithi017.stderr:10 Sep 21:20:57 ntpd[18332]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T21:20:57.822 INFO:teuthology.orchestra.run.smithi017.stderr:10 Sep 21:20:57 ntpd[18332]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T21:20:57.822 INFO:teuthology.orchestra.run.smithi017.stderr:10 Sep 21:20:57 ntpd[18332]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T21:20:57.822 INFO:teuthology.orchestra.run.smithi017.stderr:10 Sep 21:20:57 ntpd[18332]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T21:20:57.822 INFO:teuthology.orchestra.run.smithi017.stderr:10 Sep 21:20:57 ntpd[18332]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-10T21:20:57.822 INFO:teuthology.orchestra.run.smithi017.stderr:10 Sep 21:20:57 ntpd[18332]: can't open /var/log/ntpstats/peerstats.20240910: Permission denied 2024-09-10T21:20:57.822 INFO:teuthology.orchestra.run.smithi017.stderr:10 Sep 21:20:57 ntpd[18332]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T21:20:57.822 INFO:teuthology.orchestra.run.smithi017.stderr:10 Sep 21:20:57 ntpd[18332]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T21:20:57.822 INFO:teuthology.orchestra.run.smithi017.stderr:10 Sep 21:20:57 ntpd[18332]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-10T21:20:57.822 INFO:teuthology.orchestra.run.smithi017.stderr:10 Sep 21:20:57 ntpd[18332]: can't open /var/log/ntpstats/peerstats.20240910: Permission denied 2024-09-10T21:20:57.823 INFO:teuthology.orchestra.run.smithi017.stdout:10 Sep 21:20:57 ntpd[18332]: ntpd: time slew +0.001186 s 2024-09-10T21:20:57.823 INFO:teuthology.orchestra.run.smithi017.stdout:ntpd: time slew +0.001186s 2024-09-10T21:20:57.823 INFO:teuthology.orchestra.run.smithi017.stderr:10 Sep 21:20:57 ntpd[18332]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-09-10T21:20:57.823 INFO:teuthology.orchestra.run.smithi017.stderr:10 Sep 21:20:57 ntpd[18332]: can't open /var/log/ntpstats/loopstats.20240910: Permission denied 2024-09-10T21:20:57.848 INFO:teuthology.orchestra.run.smithi031.stderr:10 Sep 21:20:57 ntpd[18393]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T21:20:57.848 INFO:teuthology.orchestra.run.smithi031.stderr:10 Sep 21:20:57 ntpd[18393]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T21:20:57.848 INFO:teuthology.orchestra.run.smithi031.stderr:10 Sep 21:20:57 ntpd[18393]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T21:20:57.848 INFO:teuthology.orchestra.run.smithi031.stderr:10 Sep 21:20:57 ntpd[18393]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T21:20:57.848 INFO:teuthology.orchestra.run.smithi031.stderr:10 Sep 21:20:57 ntpd[18393]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T21:20:57.848 INFO:teuthology.orchestra.run.smithi031.stderr:10 Sep 21:20:57 ntpd[18393]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T21:20:57.848 INFO:teuthology.orchestra.run.smithi031.stderr:10 Sep 21:20:57 ntpd[18393]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T21:20:57.849 INFO:teuthology.orchestra.run.smithi031.stderr:10 Sep 21:20:57 ntpd[18393]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T21:20:57.849 INFO:teuthology.orchestra.run.smithi031.stderr:10 Sep 21:20:57 ntpd[18393]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-10T21:20:57.849 INFO:teuthology.orchestra.run.smithi031.stderr:10 Sep 21:20:57 ntpd[18393]: can't open /var/log/ntpstats/peerstats.20240910: Permission denied 2024-09-10T21:20:57.849 INFO:teuthology.orchestra.run.smithi031.stderr:10 Sep 21:20:57 ntpd[18393]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T21:20:57.849 INFO:teuthology.orchestra.run.smithi031.stderr:10 Sep 21:20:57 ntpd[18393]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T21:20:57.849 INFO:teuthology.orchestra.run.smithi031.stderr:10 Sep 21:20:57 ntpd[18393]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-10T21:20:57.849 INFO:teuthology.orchestra.run.smithi031.stderr:10 Sep 21:20:57 ntpd[18393]: can't open /var/log/ntpstats/peerstats.20240910: Permission denied 2024-09-10T21:20:57.849 INFO:teuthology.orchestra.run.smithi031.stderr:10 Sep 21:20:57 ntpd[18393]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-10T21:20:57.849 INFO:teuthology.orchestra.run.smithi031.stderr:10 Sep 21:20:57 ntpd[18393]: can't open /var/log/ntpstats/rawstats.20240910: Permission denied 2024-09-10T21:20:57.849 INFO:teuthology.orchestra.run.smithi031.stderr:10 Sep 21:20:57 ntpd[18393]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-10T21:20:57.849 INFO:teuthology.orchestra.run.smithi031.stderr:10 Sep 21:20:57 ntpd[18393]: can't open /var/log/ntpstats/peerstats.20240910: Permission denied 2024-09-10T21:20:57.850 INFO:teuthology.orchestra.run.smithi031.stdout:10 Sep 21:20:57 ntpd[18393]: ntpd: time slew -0.000428 s 2024-09-10T21:20:57.850 INFO:teuthology.orchestra.run.smithi031.stdout:ntpd: time slew -0.000428s 2024-09-10T21:20:57.850 INFO:teuthology.orchestra.run.smithi031.stderr:10 Sep 21:20:57 ntpd[18393]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-09-10T21:20:57.850 INFO:teuthology.orchestra.run.smithi031.stderr:10 Sep 21:20:57 ntpd[18393]: can't open /var/log/ntpstats/loopstats.20240910: Permission denied 2024-09-10T21:20:57.920 INFO:teuthology.orchestra.run.smithi017.stdout: remote refid st t when poll reach delay offset jitter 2024-09-10T21:20:57.920 INFO:teuthology.orchestra.run.smithi017.stdout:============================================================================== 2024-09-10T21:20:57.920 INFO:teuthology.orchestra.run.smithi017.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-10T21:20:57.921 INFO:teuthology.orchestra.run.smithi017.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-10T21:20:57.921 INFO:teuthology.orchestra.run.smithi017.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-10T21:20:57.921 INFO:teuthology.orchestra.run.smithi017.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-10T21:20:57.947 INFO:teuthology.orchestra.run.smithi031.stdout: remote refid st t when poll reach delay offset jitter 2024-09-10T21:20:57.947 INFO:teuthology.orchestra.run.smithi031.stdout:============================================================================== 2024-09-10T21:20:57.947 INFO:teuthology.orchestra.run.smithi031.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-10T21:20:57.947 INFO:teuthology.orchestra.run.smithi031.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-10T21:20:57.947 INFO:teuthology.orchestra.run.smithi031.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-10T21:20:57.947 INFO:teuthology.orchestra.run.smithi031.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-10T21:20:57.948 INFO:teuthology.run_tasks:Running task install... 2024-09-10T21:20:57.956 DEBUG:teuthology.task.install:project ceph 2024-09-10T21:20:57.956 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '8f219c476cb4f26fc5719c661c0323719a45cb30'}} 2024-09-10T21:20:57.956 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '8f219c476cb4f26fc5719c661c0323719a45cb30'} 2024-09-10T21:20:57.957 INFO:teuthology.task.install:Using flavor: default 2024-09-10T21:20:57.962 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-10T21:20:57.962 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-09-10T21:20:57.963 DEBUG:teuthology.orchestra.run.smithi017:> sudo apt-key list | grep Ceph 2024-09-10T21:20:57.963 DEBUG:teuthology.orchestra.run.smithi031:> sudo apt-key list | grep Ceph 2024-09-10T21:20:58.073 INFO:teuthology.orchestra.run.smithi017.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-09-10T21:20:58.100 INFO:teuthology.orchestra.run.smithi031.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-09-10T21:20:58.152 INFO:teuthology.orchestra.run.smithi031.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-09-10T21:20:58.152 INFO:teuthology.orchestra.run.smithi031.stdout:uid [ unknown] Ceph.com (release key) 2024-09-10T21:20:58.153 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2024-09-10T21:20:58.153 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2024-09-10T21:20:58.154 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=8f219c476cb4f26fc5719c661c0323719a45cb30 2024-09-10T21:20:58.310 INFO:teuthology.task.install.deb:Pulling from https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default/ 2024-09-10T21:20:58.310 INFO:teuthology.task.install.deb:Package version is 19.3.0-4854-g8f219c47-1jammy 2024-09-10T21:20:58.429 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-10T21:20:58.429 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-09-10T21:20:58.449 DEBUG:teuthology.orchestra.run.smithi031:> sudo apt-get update 2024-09-10T21:20:58.493 INFO:teuthology.orchestra.run.smithi017.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-09-10T21:20:58.494 INFO:teuthology.orchestra.run.smithi017.stdout:uid [ unknown] Ceph.com (release key) 2024-09-10T21:20:58.495 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2024-09-10T21:20:58.495 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2024-09-10T21:20:58.495 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=8f219c476cb4f26fc5719c661c0323719a45cb30 2024-09-10T21:20:58.636 INFO:teuthology.orchestra.run.smithi031.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-10T21:20:58.639 INFO:teuthology.task.install.deb:Pulling from https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default/ 2024-09-10T21:20:58.640 INFO:teuthology.task.install.deb:Package version is 19.3.0-4854-g8f219c47-1jammy 2024-09-10T21:20:58.651 INFO:teuthology.orchestra.run.smithi031.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-10T21:20:58.659 INFO:teuthology.orchestra.run.smithi031.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-10T21:20:58.686 INFO:teuthology.orchestra.run.smithi031.stdout:Ign:4 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy InRelease 2024-09-10T21:20:58.725 INFO:teuthology.orchestra.run.smithi031.stdout:Get:5 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy Release [7,689 B] 2024-09-10T21:20:58.747 INFO:teuthology.orchestra.run.smithi031.stdout:Ign:6 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy Release.gpg 2024-09-10T21:20:58.755 INFO:teuthology.orchestra.run.smithi031.stdout:Hit:7 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-10T21:20:58.769 INFO:teuthology.orchestra.run.smithi031.stdout:Get:8 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 Packages [18.2 kB] 2024-09-10T21:20:58.781 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-09-10T21:20:58.781 DEBUG:teuthology.orchestra.run.smithi017:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-09-10T21:20:58.801 INFO:teuthology.orchestra.run.smithi031.stdout:Get:9 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,412 B] 2024-09-10T21:20:58.802 DEBUG:teuthology.orchestra.run.smithi017:> sudo apt-get update 2024-09-10T21:20:58.997 INFO:teuthology.orchestra.run.smithi017.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-10T21:20:58.997 INFO:teuthology.orchestra.run.smithi017.stdout:Hit:2 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-10T21:20:59.003 INFO:teuthology.orchestra.run.smithi017.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-10T21:20:59.020 INFO:teuthology.orchestra.run.smithi017.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-10T21:20:59.046 INFO:teuthology.orchestra.run.smithi017.stdout:Ign:5 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy InRelease 2024-09-10T21:20:59.067 INFO:teuthology.orchestra.run.smithi017.stdout:Get:6 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy Release [7,689 B] 2024-09-10T21:20:59.089 INFO:teuthology.orchestra.run.smithi017.stdout:Ign:7 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy Release.gpg 2024-09-10T21:20:59.111 INFO:teuthology.orchestra.run.smithi017.stdout:Get:8 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 Packages [18.2 kB] 2024-09-10T21:20:59.144 INFO:teuthology.orchestra.run.smithi017.stdout:Get:9 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,412 B] 2024-09-10T21:20:59.254 INFO:teuthology.orchestra.run.smithi031.stdout:Fetched 29.3 kB in 1s (51.3 kB/s) 2024-09-10T21:20:59.615 INFO:teuthology.orchestra.run.smithi017.stdout:Fetched 29.3 kB in 1s (51.0 kB/s) 2024-09-10T21:21:00.320 INFO:teuthology.orchestra.run.smithi031.stdout:Reading package lists... 2024-09-10T21:21:00.334 DEBUG:teuthology.orchestra.run.smithi031:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.3.0-4854-g8f219c47-1jammy cephadm=19.3.0-4854-g8f219c47-1jammy ceph-mds=19.3.0-4854-g8f219c47-1jammy ceph-mgr=19.3.0-4854-g8f219c47-1jammy ceph-common=19.3.0-4854-g8f219c47-1jammy ceph-fuse=19.3.0-4854-g8f219c47-1jammy ceph-test=19.3.0-4854-g8f219c47-1jammy ceph-volume=19.3.0-4854-g8f219c47-1jammy radosgw=19.3.0-4854-g8f219c47-1jammy python3-rados=19.3.0-4854-g8f219c47-1jammy python3-rgw=19.3.0-4854-g8f219c47-1jammy python3-cephfs=19.3.0-4854-g8f219c47-1jammy python3-rbd=19.3.0-4854-g8f219c47-1jammy libcephfs2=19.3.0-4854-g8f219c47-1jammy libcephfs-dev=19.3.0-4854-g8f219c47-1jammy librados2=19.3.0-4854-g8f219c47-1jammy librbd1=19.3.0-4854-g8f219c47-1jammy rbd-fuse=19.3.0-4854-g8f219c47-1jammy python3-cephfs=19.3.0-4854-g8f219c47-1jammy cephfs-shell=19.3.0-4854-g8f219c47-1jammy cephfs-top=19.3.0-4854-g8f219c47-1jammy cephfs-mirror=19.3.0-4854-g8f219c47-1jammy 2024-09-10T21:21:00.413 INFO:teuthology.orchestra.run.smithi031.stdout:Reading package lists... 2024-09-10T21:21:00.601 INFO:teuthology.orchestra.run.smithi031.stdout:Building dependency tree... 2024-09-10T21:21:00.601 INFO:teuthology.orchestra.run.smithi031.stdout:Reading state information... 2024-09-10T21:21:00.679 INFO:teuthology.orchestra.run.smithi017.stdout:Reading package lists... 2024-09-10T21:21:00.695 DEBUG:teuthology.orchestra.run.smithi017:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.3.0-4854-g8f219c47-1jammy cephadm=19.3.0-4854-g8f219c47-1jammy ceph-mds=19.3.0-4854-g8f219c47-1jammy ceph-mgr=19.3.0-4854-g8f219c47-1jammy ceph-common=19.3.0-4854-g8f219c47-1jammy ceph-fuse=19.3.0-4854-g8f219c47-1jammy ceph-test=19.3.0-4854-g8f219c47-1jammy ceph-volume=19.3.0-4854-g8f219c47-1jammy radosgw=19.3.0-4854-g8f219c47-1jammy python3-rados=19.3.0-4854-g8f219c47-1jammy python3-rgw=19.3.0-4854-g8f219c47-1jammy python3-cephfs=19.3.0-4854-g8f219c47-1jammy python3-rbd=19.3.0-4854-g8f219c47-1jammy libcephfs2=19.3.0-4854-g8f219c47-1jammy libcephfs-dev=19.3.0-4854-g8f219c47-1jammy librados2=19.3.0-4854-g8f219c47-1jammy librbd1=19.3.0-4854-g8f219c47-1jammy rbd-fuse=19.3.0-4854-g8f219c47-1jammy python3-cephfs=19.3.0-4854-g8f219c47-1jammy cephfs-shell=19.3.0-4854-g8f219c47-1jammy cephfs-top=19.3.0-4854-g8f219c47-1jammy cephfs-mirror=19.3.0-4854-g8f219c47-1jammy 2024-09-10T21:21:00.773 INFO:teuthology.orchestra.run.smithi017.stdout:Reading package lists... 2024-09-10T21:21:00.794 INFO:teuthology.orchestra.run.smithi031.stdout:The following packages were automatically installed and are no longer required: 2024-09-10T21:21:00.794 INFO:teuthology.orchestra.run.smithi031.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-10T21:21:00.794 INFO:teuthology.orchestra.run.smithi031.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-10T21:21:00.795 INFO:teuthology.orchestra.run.smithi031.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-10T21:21:00.795 INFO:teuthology.orchestra.run.smithi031.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-10T21:21:00.795 INFO:teuthology.orchestra.run.smithi031.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-10T21:21:00.795 INFO:teuthology.orchestra.run.smithi031.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-10T21:21:00.796 INFO:teuthology.orchestra.run.smithi031.stdout:The following additional packages will be installed: 2024-09-10T21:21:00.796 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-09-10T21:21:00.796 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-09-10T21:21:00.797 INFO:teuthology.orchestra.run.smithi031.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev libnbd0 liboath0 2024-09-10T21:21:00.797 INFO:teuthology.orchestra.run.smithi031.stdout: libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 2024-09-10T21:21:00.797 INFO:teuthology.orchestra.run.smithi031.stdout: libradosstriper1 librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph 2024-09-10T21:21:00.798 INFO:teuthology.orchestra.run.smithi031.stdout: libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli 2024-09-10T21:21:00.798 INFO:teuthology.orchestra.run.smithi031.stdout: pkg-config python-asyncssh-doc python-babel-localedata 2024-09-10T21:21:00.798 INFO:teuthology.orchestra.run.smithi031.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-09-10T21:21:00.798 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-certifi 2024-09-10T21:21:00.798 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cheroot python3-cherrypy3 python3-cmd2 python3-colorama 2024-09-10T21:21:00.798 INFO:teuthology.orchestra.run.smithi031.stdout: python3-google-auth python3-idna python3-iniconfig python3-jaraco.classes 2024-09-10T21:21:00.798 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2024-09-10T21:21:00.798 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jinja2 python3-joblib python3-kubernetes python3-logutils 2024-09-10T21:21:00.798 INFO:teuthology.orchestra.run.smithi031.stdout: python3-mako python3-markupsafe python3-natsort python3-openssl 2024-09-10T21:21:00.798 INFO:teuthology.orchestra.run.smithi031.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2024-09-10T21:21:00.799 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2024-09-10T21:21:00.799 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pyasn1 python3-pyasn1-modules python3-pygments python3-pyinotify 2024-09-10T21:21:00.799 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pyperclip python3-pytest python3-repoze.lru python3-requests 2024-09-10T21:21:00.799 INFO:teuthology.orchestra.run.smithi031.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-simplegeneric 2024-09-10T21:21:00.799 INFO:teuthology.orchestra.run.smithi031.stdout: python3-simplejson python3-singledispatch python3-sklearn 2024-09-10T21:21:00.799 INFO:teuthology.orchestra.run.smithi031.stdout: python3-sklearn-lib python3-tempita python3-tempora python3-threadpoolctl 2024-09-10T21:21:00.799 INFO:teuthology.orchestra.run.smithi031.stdout: python3-toml python3-urllib3 python3-waitress python3-wcwidth python3-webob 2024-09-10T21:21:00.799 INFO:teuthology.orchestra.run.smithi031.stdout: python3-websocket python3-webtest python3-werkzeug python3-zc.lockfile 2024-09-10T21:21:00.799 INFO:teuthology.orchestra.run.smithi031.stdout: qttranslations5-l10n socat unzip xmlstarlet zip 2024-09-10T21:21:00.800 INFO:teuthology.orchestra.run.smithi031.stdout:Suggested packages: 2024-09-10T21:21:00.800 INFO:teuthology.orchestra.run.smithi031.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-09-10T21:21:00.800 INFO:teuthology.orchestra.run.smithi031.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-09-10T21:21:00.800 INFO:teuthology.orchestra.run.smithi031.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-09-10T21:21:00.800 INFO:teuthology.orchestra.run.smithi031.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-09-10T21:21:00.800 INFO:teuthology.orchestra.run.smithi031.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-09-10T21:21:00.800 INFO:teuthology.orchestra.run.smithi031.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-09-10T21:21:00.800 INFO:teuthology.orchestra.run.smithi031.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-09-10T21:21:00.800 INFO:teuthology.orchestra.run.smithi031.stdout:Recommended packages: 2024-09-10T21:21:00.800 INFO:teuthology.orchestra.run.smithi031.stdout: btrfs-tools 2024-09-10T21:21:00.960 INFO:teuthology.orchestra.run.smithi031.stdout:The following NEW packages will be installed: 2024-09-10T21:21:00.961 INFO:teuthology.orchestra.run.smithi031.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-09-10T21:21:00.961 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-09-10T21:21:00.961 INFO:teuthology.orchestra.run.smithi031.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2024-09-10T21:21:00.961 INFO:teuthology.orchestra.run.smithi031.stdout: cephfs-mirror cephfs-shell cephfs-top jq libcephfs-dev libcephfs2 2024-09-10T21:21:00.961 INFO:teuthology.orchestra.run.smithi031.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev libnbd0 liboath0 2024-09-10T21:21:00.961 INFO:teuthology.orchestra.run.smithi031.stdout: libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librados2 2024-09-10T21:21:00.961 INFO:teuthology.orchestra.run.smithi031.stdout: libradosstriper1 librbd1 librdkafka1 libreadline-dev librgw2 2024-09-10T21:21:00.962 INFO:teuthology.orchestra.run.smithi017.stdout:Building dependency tree... 2024-09-10T21:21:00.962 INFO:teuthology.orchestra.run.smithi017.stdout:Reading state information... 2024-09-10T21:21:00.963 INFO:teuthology.orchestra.run.smithi031.stdout: libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 2024-09-10T21:21:00.963 INFO:teuthology.orchestra.run.smithi031.stdout: luarocks nvme-cli pkg-config python-asyncssh-doc python-babel-localedata 2024-09-10T21:21:00.963 INFO:teuthology.orchestra.run.smithi031.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-09-10T21:21:00.964 INFO:teuthology.orchestra.run.smithi031.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2024-09-10T21:21:00.964 INFO:teuthology.orchestra.run.smithi031.stdout: python3-certifi python3-cheroot python3-cherrypy3 python3-cmd2 2024-09-10T21:21:00.964 INFO:teuthology.orchestra.run.smithi031.stdout: python3-colorama python3-google-auth python3-idna python3-iniconfig 2024-09-10T21:21:00.964 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-09-10T21:21:00.964 INFO:teuthology.orchestra.run.smithi031.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-09-10T21:21:00.964 INFO:teuthology.orchestra.run.smithi031.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-09-10T21:21:00.964 INFO:teuthology.orchestra.run.smithi031.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-09-10T21:21:00.964 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-09-10T21:21:00.964 INFO:teuthology.orchestra.run.smithi031.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-09-10T21:21:00.964 INFO:teuthology.orchestra.run.smithi031.stdout: python3-pygments python3-pyinotify python3-pyperclip python3-pytest 2024-09-10T21:21:00.964 INFO:teuthology.orchestra.run.smithi031.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests 2024-09-10T21:21:00.964 INFO:teuthology.orchestra.run.smithi031.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2024-09-10T21:21:00.964 INFO:teuthology.orchestra.run.smithi031.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-09-10T21:21:00.965 INFO:teuthology.orchestra.run.smithi031.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-09-10T21:21:00.965 INFO:teuthology.orchestra.run.smithi031.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-09-10T21:21:00.965 INFO:teuthology.orchestra.run.smithi031.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-09-10T21:21:00.965 INFO:teuthology.orchestra.run.smithi031.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2024-09-10T21:21:00.965 INFO:teuthology.orchestra.run.smithi031.stdout: socat unzip xmlstarlet zip 2024-09-10T21:21:01.031 INFO:teuthology.orchestra.run.smithi031.stdout:0 upgraded, 125 newly installed, 0 to remove and 334 not upgraded. 2024-09-10T21:21:01.031 INFO:teuthology.orchestra.run.smithi031.stdout:Need to get 251 MB of archives. 2024-09-10T21:21:01.031 INFO:teuthology.orchestra.run.smithi031.stdout:After this operation, 936 MB of additional disk space will be used. 2024-09-10T21:21:01.031 INFO:teuthology.orchestra.run.smithi031.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyperclip all 1.8.2-2 [11.5 kB] 2024-09-10T21:21:01.049 INFO:teuthology.orchestra.run.smithi031.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-09-10T21:21:01.059 INFO:teuthology.orchestra.run.smithi031.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cmd2 all 2.3.3-1 [124 kB] 2024-09-10T21:21:01.103 INFO:teuthology.orchestra.run.smithi031.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-09-10T21:21:01.112 INFO:teuthology.orchestra.run.smithi031.stdout:Get:5 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.3.0-4854-g8f219c47-1jammy [3,633 kB] 2024-09-10T21:21:01.126 INFO:teuthology.orchestra.run.smithi031.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-09-10T21:21:01.129 INFO:teuthology.orchestra.run.smithi031.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-09-10T21:21:01.141 INFO:teuthology.orchestra.run.smithi031.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-09-10T21:21:01.155 INFO:teuthology.orchestra.run.smithi017.stdout:The following packages were automatically installed and are no longer required: 2024-09-10T21:21:01.156 INFO:teuthology.orchestra.run.smithi017.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-10T21:21:01.156 INFO:teuthology.orchestra.run.smithi017.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-10T21:21:01.156 INFO:teuthology.orchestra.run.smithi017.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-10T21:21:01.156 INFO:teuthology.orchestra.run.smithi017.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-10T21:21:01.157 INFO:teuthology.orchestra.run.smithi017.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-10T21:21:01.157 INFO:teuthology.orchestra.run.smithi017.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-10T21:21:01.157 INFO:teuthology.orchestra.run.smithi017.stdout:The following additional packages will be installed: 2024-09-10T21:21:01.157 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-09-10T21:21:01.158 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-09-10T21:21:01.158 INFO:teuthology.orchestra.run.smithi017.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev libnbd0 liboath0 2024-09-10T21:21:01.158 INFO:teuthology.orchestra.run.smithi017.stdout: libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 2024-09-10T21:21:01.158 INFO:teuthology.orchestra.run.smithi017.stdout: libradosstriper1 librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph 2024-09-10T21:21:01.159 INFO:teuthology.orchestra.run.smithi017.stdout: libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli 2024-09-10T21:21:01.159 INFO:teuthology.orchestra.run.smithi017.stdout: pkg-config python-asyncssh-doc python-babel-localedata 2024-09-10T21:21:01.159 INFO:teuthology.orchestra.run.smithi017.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-09-10T21:21:01.159 INFO:teuthology.orchestra.run.smithi017.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-certifi 2024-09-10T21:21:01.159 INFO:teuthology.orchestra.run.smithi017.stdout: python3-cheroot python3-cherrypy3 python3-cmd2 python3-colorama 2024-09-10T21:21:01.159 INFO:teuthology.orchestra.run.smithi017.stdout: python3-google-auth python3-idna python3-iniconfig python3-jaraco.classes 2024-09-10T21:21:01.159 INFO:teuthology.orchestra.run.smithi017.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2024-09-10T21:21:01.160 INFO:teuthology.orchestra.run.smithi017.stdout: python3-jinja2 python3-joblib python3-kubernetes python3-logutils 2024-09-10T21:21:01.160 INFO:teuthology.orchestra.run.smithi017.stdout: python3-mako python3-markupsafe python3-natsort python3-openssl 2024-09-10T21:21:01.160 INFO:teuthology.orchestra.run.smithi017.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2024-09-10T21:21:01.160 INFO:teuthology.orchestra.run.smithi017.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2024-09-10T21:21:01.160 INFO:teuthology.orchestra.run.smithi017.stdout: python3-pyasn1 python3-pyasn1-modules python3-pygments python3-pyinotify 2024-09-10T21:21:01.160 INFO:teuthology.orchestra.run.smithi017.stdout: python3-pyperclip python3-pytest python3-repoze.lru python3-requests 2024-09-10T21:21:01.160 INFO:teuthology.orchestra.run.smithi017.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-simplegeneric 2024-09-10T21:21:01.160 INFO:teuthology.orchestra.run.smithi017.stdout: python3-simplejson python3-singledispatch python3-sklearn 2024-09-10T21:21:01.160 INFO:teuthology.orchestra.run.smithi017.stdout: python3-sklearn-lib python3-tempita python3-tempora python3-threadpoolctl 2024-09-10T21:21:01.160 INFO:teuthology.orchestra.run.smithi017.stdout: python3-toml python3-urllib3 python3-waitress python3-wcwidth python3-webob 2024-09-10T21:21:01.160 INFO:teuthology.orchestra.run.smithi017.stdout: python3-websocket python3-webtest python3-werkzeug python3-zc.lockfile 2024-09-10T21:21:01.160 INFO:teuthology.orchestra.run.smithi017.stdout: qttranslations5-l10n socat unzip xmlstarlet zip 2024-09-10T21:21:01.161 INFO:teuthology.orchestra.run.smithi017.stdout:Suggested packages: 2024-09-10T21:21:01.161 INFO:teuthology.orchestra.run.smithi017.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-09-10T21:21:01.161 INFO:teuthology.orchestra.run.smithi017.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-09-10T21:21:01.161 INFO:teuthology.orchestra.run.smithi017.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-09-10T21:21:01.161 INFO:teuthology.orchestra.run.smithi017.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-09-10T21:21:01.161 INFO:teuthology.orchestra.run.smithi017.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-09-10T21:21:01.161 INFO:teuthology.orchestra.run.smithi017.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-09-10T21:21:01.161 INFO:teuthology.orchestra.run.smithi017.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-09-10T21:21:01.161 INFO:teuthology.orchestra.run.smithi017.stdout:Recommended packages: 2024-09-10T21:21:01.161 INFO:teuthology.orchestra.run.smithi017.stdout: btrfs-tools 2024-09-10T21:21:01.180 INFO:teuthology.orchestra.run.smithi031.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-09-10T21:21:01.183 INFO:teuthology.orchestra.run.smithi031.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-09-10T21:21:01.190 INFO:teuthology.orchestra.run.smithi031.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-09-10T21:21:01.194 INFO:teuthology.orchestra.run.smithi031.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libnbd0 amd64 1.10.5-1 [71.3 kB] 2024-09-10T21:21:01.195 INFO:teuthology.orchestra.run.smithi031.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-09-10T21:21:01.196 INFO:teuthology.orchestra.run.smithi031.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-09-10T21:21:01.201 INFO:teuthology.orchestra.run.smithi031.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-09-10T21:21:01.203 INFO:teuthology.orchestra.run.smithi031.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-09-10T21:21:01.205 INFO:teuthology.orchestra.run.smithi031.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-09-10T21:21:01.206 INFO:teuthology.orchestra.run.smithi031.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-09-10T21:21:01.216 INFO:teuthology.orchestra.run.smithi031.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-09-10T21:21:01.218 INFO:teuthology.orchestra.run.smithi031.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-09-10T21:21:01.220 INFO:teuthology.orchestra.run.smithi031.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-09-10T21:21:01.222 INFO:teuthology.orchestra.run.smithi031.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-09-10T21:21:01.222 INFO:teuthology.orchestra.run.smithi031.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-09-10T21:21:01.234 INFO:teuthology.orchestra.run.smithi031.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-09-10T21:21:01.235 INFO:teuthology.orchestra.run.smithi031.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-09-10T21:21:01.235 INFO:teuthology.orchestra.run.smithi031.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-09-10T21:21:01.235 INFO:teuthology.orchestra.run.smithi031.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-09-10T21:21:01.235 INFO:teuthology.orchestra.run.smithi031.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-09-10T21:21:01.251 INFO:teuthology.orchestra.run.smithi031.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-09-10T21:21:01.251 INFO:teuthology.orchestra.run.smithi031.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-09-10T21:21:01.251 INFO:teuthology.orchestra.run.smithi031.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-09-10T21:21:01.254 INFO:teuthology.orchestra.run.smithi031.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-09-10T21:21:01.254 INFO:teuthology.orchestra.run.smithi031.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-09-10T21:21:01.268 INFO:teuthology.orchestra.run.smithi031.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-09-10T21:21:01.268 INFO:teuthology.orchestra.run.smithi031.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-09-10T21:21:01.269 INFO:teuthology.orchestra.run.smithi031.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-09-10T21:21:01.269 INFO:teuthology.orchestra.run.smithi031.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-09-10T21:21:01.270 INFO:teuthology.orchestra.run.smithi031.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-webob all 1:1.8.6-1.1ubuntu0.1 [86.7 kB] 2024-09-10T21:21:01.285 INFO:teuthology.orchestra.run.smithi031.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-09-10T21:21:01.286 INFO:teuthology.orchestra.run.smithi031.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-09-10T21:21:01.286 INFO:teuthology.orchestra.run.smithi031.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-09-10T21:21:01.291 INFO:teuthology.orchestra.run.smithi031.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-09-10T21:21:01.291 INFO:teuthology.orchestra.run.smithi031.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-09-10T21:21:01.302 INFO:teuthology.orchestra.run.smithi031.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-09-10T21:21:01.302 INFO:teuthology.orchestra.run.smithi031.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-09-10T21:21:01.303 INFO:teuthology.orchestra.run.smithi031.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-09-10T21:21:01.305 INFO:teuthology.orchestra.run.smithi031.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-idna all 3.3-1ubuntu0.1 [52.1 kB] 2024-09-10T21:21:01.306 INFO:teuthology.orchestra.run.smithi031.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-09-10T21:21:01.319 INFO:teuthology.orchestra.run.smithi031.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-09-10T21:21:01.320 INFO:teuthology.orchestra.run.smithi031.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.2 [181 kB] 2024-09-10T21:21:01.321 INFO:teuthology.orchestra.run.smithi017.stdout:The following NEW packages will be installed: 2024-09-10T21:21:01.321 INFO:teuthology.orchestra.run.smithi017.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-09-10T21:21:01.321 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-09-10T21:21:01.321 INFO:teuthology.orchestra.run.smithi017.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2024-09-10T21:21:01.321 INFO:teuthology.orchestra.run.smithi017.stdout: cephfs-mirror cephfs-shell cephfs-top jq libcephfs-dev libcephfs2 2024-09-10T21:21:01.323 INFO:teuthology.orchestra.run.smithi017.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev libnbd0 liboath0 2024-09-10T21:21:01.323 INFO:teuthology.orchestra.run.smithi017.stdout: libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librados2 2024-09-10T21:21:01.323 INFO:teuthology.orchestra.run.smithi017.stdout: libradosstriper1 librbd1 librdkafka1 libreadline-dev librgw2 2024-09-10T21:21:01.323 INFO:teuthology.orchestra.run.smithi017.stdout: libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 2024-09-10T21:21:01.323 INFO:teuthology.orchestra.run.smithi017.stdout: luarocks nvme-cli pkg-config python-asyncssh-doc python-babel-localedata 2024-09-10T21:21:01.323 INFO:teuthology.orchestra.run.smithi017.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-09-10T21:21:01.323 INFO:teuthology.orchestra.run.smithi017.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2024-09-10T21:21:01.323 INFO:teuthology.orchestra.run.smithi017.stdout: python3-certifi python3-cheroot python3-cherrypy3 python3-cmd2 2024-09-10T21:21:01.323 INFO:teuthology.orchestra.run.smithi017.stdout: python3-colorama python3-google-auth python3-idna python3-iniconfig 2024-09-10T21:21:01.323 INFO:teuthology.orchestra.run.smithi017.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-09-10T21:21:01.324 INFO:teuthology.orchestra.run.smithi017.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-09-10T21:21:01.324 INFO:teuthology.orchestra.run.smithi017.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-09-10T21:21:01.324 INFO:teuthology.orchestra.run.smithi017.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-09-10T21:21:01.324 INFO:teuthology.orchestra.run.smithi017.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-09-10T21:21:01.324 INFO:teuthology.orchestra.run.smithi017.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-09-10T21:21:01.324 INFO:teuthology.orchestra.run.smithi017.stdout: python3-pygments python3-pyinotify python3-pyperclip python3-pytest 2024-09-10T21:21:01.324 INFO:teuthology.orchestra.run.smithi017.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests 2024-09-10T21:21:01.324 INFO:teuthology.orchestra.run.smithi017.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2024-09-10T21:21:01.324 INFO:teuthology.orchestra.run.smithi017.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-09-10T21:21:01.324 INFO:teuthology.orchestra.run.smithi017.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-09-10T21:21:01.324 INFO:teuthology.orchestra.run.smithi017.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-09-10T21:21:01.324 INFO:teuthology.orchestra.run.smithi017.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-09-10T21:21:01.324 INFO:teuthology.orchestra.run.smithi017.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2024-09-10T21:21:01.324 INFO:teuthology.orchestra.run.smithi017.stdout: socat unzip xmlstarlet zip 2024-09-10T21:21:01.325 INFO:teuthology.orchestra.run.smithi031.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-09-10T21:21:01.325 INFO:teuthology.orchestra.run.smithi031.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-09-10T21:21:01.325 INFO:teuthology.orchestra.run.smithi031.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-09-10T21:21:01.336 INFO:teuthology.orchestra.run.smithi031.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-09-10T21:21:01.386 INFO:teuthology.orchestra.run.smithi031.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-09-10T21:21:01.387 INFO:teuthology.orchestra.run.smithi031.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.2 [108 kB] 2024-09-10T21:21:01.389 INFO:teuthology.orchestra.run.smithi031.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-09-10T21:21:01.389 INFO:teuthology.orchestra.run.smithi031.stdout:Get:58 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.3.0-4854-g8f219c47-1jammy [3,330 kB] 2024-09-10T21:21:01.389 INFO:teuthology.orchestra.run.smithi031.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-09-10T21:21:01.391 INFO:teuthology.orchestra.run.smithi031.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-09-10T21:21:01.394 INFO:teuthology.orchestra.run.smithi017.stdout:0 upgraded, 125 newly installed, 0 to remove and 334 not upgraded. 2024-09-10T21:21:01.394 INFO:teuthology.orchestra.run.smithi017.stdout:Need to get 251 MB of archives. 2024-09-10T21:21:01.394 INFO:teuthology.orchestra.run.smithi017.stdout:After this operation, 936 MB of additional disk space will be used. 2024-09-10T21:21:01.395 INFO:teuthology.orchestra.run.smithi017.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyperclip all 1.8.2-2 [11.5 kB] 2024-09-10T21:21:01.411 INFO:teuthology.orchestra.run.smithi031.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-09-10T21:21:01.413 INFO:teuthology.orchestra.run.smithi031.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-09-10T21:21:01.413 INFO:teuthology.orchestra.run.smithi031.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-09-10T21:21:01.414 INFO:teuthology.orchestra.run.smithi017.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-09-10T21:21:01.423 INFO:teuthology.orchestra.run.smithi017.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cmd2 all 2.3.3-1 [124 kB] 2024-09-10T21:21:01.430 INFO:teuthology.orchestra.run.smithi031.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-09-10T21:21:01.430 INFO:teuthology.orchestra.run.smithi031.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-09-10T21:21:01.431 INFO:teuthology.orchestra.run.smithi031.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-09-10T21:21:01.431 INFO:teuthology.orchestra.run.smithi031.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-09-10T21:21:01.432 INFO:teuthology.orchestra.run.smithi031.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-09-10T21:21:01.433 INFO:teuthology.orchestra.run.smithi031.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-09-10T21:21:01.442 INFO:teuthology.orchestra.run.smithi031.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-09-10T21:21:01.442 INFO:teuthology.orchestra.run.smithi031.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-09-10T21:21:01.446 INFO:teuthology.orchestra.run.smithi031.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-09-10T21:21:01.458 INFO:teuthology.orchestra.run.smithi031.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-09-10T21:21:01.460 INFO:teuthology.orchestra.run.smithi031.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-09-10T21:21:01.461 INFO:teuthology.orchestra.run.smithi031.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-09-10T21:21:01.465 INFO:teuthology.orchestra.run.smithi031.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-09-10T21:21:01.467 INFO:teuthology.orchestra.run.smithi017.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-09-10T21:21:01.468 INFO:teuthology.orchestra.run.smithi031.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-colorama all 0.4.4-1 [24.5 kB] 2024-09-10T21:21:01.468 INFO:teuthology.orchestra.run.smithi031.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-09-10T21:21:01.469 INFO:teuthology.orchestra.run.smithi031.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-09-10T21:21:01.474 INFO:teuthology.orchestra.run.smithi017.stdout:Get:5 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.3.0-4854-g8f219c47-1jammy [3,633 kB] 2024-09-10T21:21:01.476 INFO:teuthology.orchestra.run.smithi031.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.2 [473 kB] 2024-09-10T21:21:01.481 INFO:teuthology.orchestra.run.smithi031.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-09-10T21:21:01.481 INFO:teuthology.orchestra.run.smithi031.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-09-10T21:21:01.489 INFO:teuthology.orchestra.run.smithi017.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-09-10T21:21:01.491 INFO:teuthology.orchestra.run.smithi017.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-09-10T21:21:01.493 INFO:teuthology.orchestra.run.smithi031.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-09-10T21:21:01.494 INFO:teuthology.orchestra.run.smithi031.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-09-10T21:21:01.495 INFO:teuthology.orchestra.run.smithi031.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-09-10T21:21:01.495 INFO:teuthology.orchestra.run.smithi031.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-09-10T21:21:01.497 INFO:teuthology.orchestra.run.smithi031.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-09-10T21:21:01.498 INFO:teuthology.orchestra.run.smithi031.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-09-10T21:21:01.504 INFO:teuthology.orchestra.run.smithi017.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-09-10T21:21:01.505 INFO:teuthology.orchestra.run.smithi031.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-09-10T21:21:01.511 INFO:teuthology.orchestra.run.smithi031.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-09-10T21:21:01.511 INFO:teuthology.orchestra.run.smithi031.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-09-10T21:21:01.513 INFO:teuthology.orchestra.run.smithi031.stdout:Get:92 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-09-10T21:21:01.520 INFO:teuthology.orchestra.run.smithi031.stdout:Get:93 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.3.0-4854-g8f219c47-1jammy [721 kB] 2024-09-10T21:21:01.528 INFO:teuthology.orchestra.run.smithi031.stdout:Get:94 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-09-10T21:21:01.543 INFO:teuthology.orchestra.run.smithi031.stdout:Get:95 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.3.0-4854-g8f219c47-1jammy [356 kB] 2024-09-10T21:21:01.546 INFO:teuthology.orchestra.run.smithi031.stdout:Get:96 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.3.0-4854-g8f219c47-1jammy [32.8 kB] 2024-09-10T21:21:01.546 INFO:teuthology.orchestra.run.smithi031.stdout:Get:97 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.3.0-4854-g8f219c47-1jammy [180 kB] 2024-09-10T21:21:01.547 INFO:teuthology.orchestra.run.smithi017.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-09-10T21:21:01.549 INFO:teuthology.orchestra.run.smithi017.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-09-10T21:21:01.556 INFO:teuthology.orchestra.run.smithi017.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-09-10T21:21:01.559 INFO:teuthology.orchestra.run.smithi017.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libnbd0 amd64 1.10.5-1 [71.3 kB] 2024-09-10T21:21:01.560 INFO:teuthology.orchestra.run.smithi017.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-09-10T21:21:01.560 INFO:teuthology.orchestra.run.smithi017.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-09-10T21:21:01.562 INFO:teuthology.orchestra.run.smithi031.stdout:Get:98 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.3.0-4854-g8f219c47-1jammy [71.2 kB] 2024-09-10T21:21:01.565 INFO:teuthology.orchestra.run.smithi031.stdout:Get:99 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.3.0-4854-g8f219c47-1jammy [338 kB] 2024-09-10T21:21:01.566 INFO:teuthology.orchestra.run.smithi017.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-09-10T21:21:01.568 INFO:teuthology.orchestra.run.smithi031.stdout:Get:100 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.3.0-4854-g8f219c47-1jammy [7,988 kB] 2024-09-10T21:21:01.568 INFO:teuthology.orchestra.run.smithi017.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-09-10T21:21:01.570 INFO:teuthology.orchestra.run.smithi017.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-09-10T21:21:01.571 INFO:teuthology.orchestra.run.smithi017.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-09-10T21:21:01.585 INFO:teuthology.orchestra.run.smithi017.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-09-10T21:21:01.586 INFO:teuthology.orchestra.run.smithi017.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-09-10T21:21:01.589 INFO:teuthology.orchestra.run.smithi017.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-09-10T21:21:01.590 INFO:teuthology.orchestra.run.smithi017.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-09-10T21:21:01.590 INFO:teuthology.orchestra.run.smithi017.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-09-10T21:21:01.602 INFO:teuthology.orchestra.run.smithi017.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-09-10T21:21:01.603 INFO:teuthology.orchestra.run.smithi017.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-09-10T21:21:01.604 INFO:teuthology.orchestra.run.smithi017.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-09-10T21:21:01.604 INFO:teuthology.orchestra.run.smithi017.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-09-10T21:21:01.605 INFO:teuthology.orchestra.run.smithi017.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-09-10T21:21:01.620 INFO:teuthology.orchestra.run.smithi017.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-09-10T21:21:01.621 INFO:teuthology.orchestra.run.smithi017.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-09-10T21:21:01.621 INFO:teuthology.orchestra.run.smithi017.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-09-10T21:21:01.623 INFO:teuthology.orchestra.run.smithi017.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-09-10T21:21:01.623 INFO:teuthology.orchestra.run.smithi017.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-09-10T21:21:01.638 INFO:teuthology.orchestra.run.smithi017.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-09-10T21:21:01.638 INFO:teuthology.orchestra.run.smithi017.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-09-10T21:21:01.639 INFO:teuthology.orchestra.run.smithi017.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-09-10T21:21:01.639 INFO:teuthology.orchestra.run.smithi017.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-09-10T21:21:01.640 INFO:teuthology.orchestra.run.smithi017.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-webob all 1:1.8.6-1.1ubuntu0.1 [86.7 kB] 2024-09-10T21:21:01.655 INFO:teuthology.orchestra.run.smithi017.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-09-10T21:21:01.655 INFO:teuthology.orchestra.run.smithi017.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-09-10T21:21:01.656 INFO:teuthology.orchestra.run.smithi017.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-09-10T21:21:01.660 INFO:teuthology.orchestra.run.smithi017.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-09-10T21:21:01.661 INFO:teuthology.orchestra.run.smithi017.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-09-10T21:21:01.672 INFO:teuthology.orchestra.run.smithi017.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-09-10T21:21:01.673 INFO:teuthology.orchestra.run.smithi017.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-09-10T21:21:01.674 INFO:teuthology.orchestra.run.smithi017.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-09-10T21:21:01.675 INFO:teuthology.orchestra.run.smithi017.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-idna all 3.3-1ubuntu0.1 [52.1 kB] 2024-09-10T21:21:01.676 INFO:teuthology.orchestra.run.smithi017.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-09-10T21:21:01.701 INFO:teuthology.orchestra.run.smithi017.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-09-10T21:21:01.701 INFO:teuthology.orchestra.run.smithi017.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.2 [181 kB] 2024-09-10T21:21:01.703 INFO:teuthology.orchestra.run.smithi017.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-09-10T21:21:01.703 INFO:teuthology.orchestra.run.smithi017.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-09-10T21:21:01.704 INFO:teuthology.orchestra.run.smithi017.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-09-10T21:21:01.717 INFO:teuthology.orchestra.run.smithi017.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-09-10T21:21:01.758 INFO:teuthology.orchestra.run.smithi017.stdout:Get:55 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.3.0-4854-g8f219c47-1jammy [3,330 kB] 2024-09-10T21:21:01.762 INFO:teuthology.orchestra.run.smithi017.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-09-10T21:21:01.763 INFO:teuthology.orchestra.run.smithi017.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.2 [108 kB] 2024-09-10T21:21:01.765 INFO:teuthology.orchestra.run.smithi017.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-09-10T21:21:01.768 INFO:teuthology.orchestra.run.smithi017.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-09-10T21:21:01.769 INFO:teuthology.orchestra.run.smithi017.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-09-10T21:21:01.769 INFO:teuthology.orchestra.run.smithi031.stdout:Get:101 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.3.0-4854-g8f219c47-1jammy [112 kB] 2024-09-10T21:21:01.771 INFO:teuthology.orchestra.run.smithi031.stdout:Get:102 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.3.0-4854-g8f219c47-1jammy [473 kB] 2024-09-10T21:21:01.777 INFO:teuthology.orchestra.run.smithi031.stdout:Get:103 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.3.0-4854-g8f219c47-1jammy [28.3 MB] 2024-09-10T21:21:01.785 INFO:teuthology.orchestra.run.smithi017.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-09-10T21:21:01.787 INFO:teuthology.orchestra.run.smithi017.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-09-10T21:21:01.787 INFO:teuthology.orchestra.run.smithi017.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-09-10T21:21:01.804 INFO:teuthology.orchestra.run.smithi017.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-09-10T21:21:01.804 INFO:teuthology.orchestra.run.smithi017.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-09-10T21:21:01.805 INFO:teuthology.orchestra.run.smithi017.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-09-10T21:21:01.805 INFO:teuthology.orchestra.run.smithi017.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-09-10T21:21:01.806 INFO:teuthology.orchestra.run.smithi017.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-09-10T21:21:01.806 INFO:teuthology.orchestra.run.smithi017.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-09-10T21:21:01.816 INFO:teuthology.orchestra.run.smithi017.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-09-10T21:21:01.817 INFO:teuthology.orchestra.run.smithi017.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-09-10T21:21:01.820 INFO:teuthology.orchestra.run.smithi017.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-09-10T21:21:01.833 INFO:teuthology.orchestra.run.smithi017.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-09-10T21:21:01.835 INFO:teuthology.orchestra.run.smithi017.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-09-10T21:21:01.835 INFO:teuthology.orchestra.run.smithi017.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-09-10T21:21:01.839 INFO:teuthology.orchestra.run.smithi017.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-09-10T21:21:01.842 INFO:teuthology.orchestra.run.smithi017.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-colorama all 0.4.4-1 [24.5 kB] 2024-09-10T21:21:01.842 INFO:teuthology.orchestra.run.smithi017.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-09-10T21:21:01.843 INFO:teuthology.orchestra.run.smithi017.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-09-10T21:21:01.851 INFO:teuthology.orchestra.run.smithi017.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.2 [473 kB] 2024-09-10T21:21:01.855 INFO:teuthology.orchestra.run.smithi017.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-09-10T21:21:01.856 INFO:teuthology.orchestra.run.smithi017.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-09-10T21:21:01.868 INFO:teuthology.orchestra.run.smithi017.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-09-10T21:21:01.868 INFO:teuthology.orchestra.run.smithi017.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-09-10T21:21:01.869 INFO:teuthology.orchestra.run.smithi017.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-09-10T21:21:01.869 INFO:teuthology.orchestra.run.smithi017.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-09-10T21:21:01.871 INFO:teuthology.orchestra.run.smithi017.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-09-10T21:21:01.872 INFO:teuthology.orchestra.run.smithi017.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-09-10T21:21:01.878 INFO:teuthology.orchestra.run.smithi017.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-09-10T21:21:01.885 INFO:teuthology.orchestra.run.smithi017.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-09-10T21:21:01.885 INFO:teuthology.orchestra.run.smithi017.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-09-10T21:21:01.888 INFO:teuthology.orchestra.run.smithi017.stdout:Get:92 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-09-10T21:21:01.902 INFO:teuthology.orchestra.run.smithi017.stdout:Get:93 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-09-10T21:21:01.934 INFO:teuthology.orchestra.run.smithi017.stdout:Get:94 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.3.0-4854-g8f219c47-1jammy [721 kB] 2024-09-10T21:21:01.969 INFO:teuthology.orchestra.run.smithi017.stdout:Get:95 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.3.0-4854-g8f219c47-1jammy [356 kB] 2024-09-10T21:21:01.976 INFO:teuthology.orchestra.run.smithi017.stdout:Get:96 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.3.0-4854-g8f219c47-1jammy [32.8 kB] 2024-09-10T21:21:01.977 INFO:teuthology.orchestra.run.smithi017.stdout:Get:97 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.3.0-4854-g8f219c47-1jammy [180 kB] 2024-09-10T21:21:02.001 INFO:teuthology.orchestra.run.smithi017.stdout:Get:98 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.3.0-4854-g8f219c47-1jammy [71.2 kB] 2024-09-10T21:21:02.005 INFO:teuthology.orchestra.run.smithi017.stdout:Get:99 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.3.0-4854-g8f219c47-1jammy [338 kB] 2024-09-10T21:21:02.010 INFO:teuthology.orchestra.run.smithi017.stdout:Get:100 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.3.0-4854-g8f219c47-1jammy [7,988 kB] 2024-09-10T21:21:02.441 INFO:teuthology.orchestra.run.smithi017.stdout:Get:101 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.3.0-4854-g8f219c47-1jammy [112 kB] 2024-09-10T21:21:02.442 INFO:teuthology.orchestra.run.smithi017.stdout:Get:102 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.3.0-4854-g8f219c47-1jammy [473 kB] 2024-09-10T21:21:02.463 INFO:teuthology.orchestra.run.smithi017.stdout:Get:103 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.3.0-4854-g8f219c47-1jammy [28.3 MB] 2024-09-10T21:21:02.464 INFO:teuthology.orchestra.run.smithi031.stdout:Get:104 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.3.0-4854-g8f219c47-1jammy [5,206 kB] 2024-09-10T21:21:02.581 INFO:teuthology.orchestra.run.smithi031.stdout:Get:105 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.3.0-4854-g8f219c47-1jammy [251 kB] 2024-09-10T21:21:02.589 INFO:teuthology.orchestra.run.smithi031.stdout:Get:106 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.3.0-4854-g8f219c47-1jammy [124 kB] 2024-09-10T21:21:02.589 INFO:teuthology.orchestra.run.smithi031.stdout:Get:107 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.3.0-4854-g8f219c47-1jammy [1,562 kB] 2024-09-10T21:21:02.618 INFO:teuthology.orchestra.run.smithi031.stdout:Get:108 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.3.0-4854-g8f219c47-1jammy [6,308 kB] 2024-09-10T21:21:02.782 INFO:teuthology.orchestra.run.smithi031.stdout:Get:109 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.3.0-4854-g8f219c47-1jammy [23.1 MB] 2024-09-10T21:21:03.447 INFO:teuthology.orchestra.run.smithi031.stdout:Get:110 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.3.0-4854-g8f219c47-1jammy [14.0 kB] 2024-09-10T21:21:03.448 INFO:teuthology.orchestra.run.smithi031.stdout:Get:111 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.3.0-4854-g8f219c47-1jammy [880 kB] 2024-09-10T21:21:03.455 INFO:teuthology.orchestra.run.smithi031.stdout:Get:112 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.3.0-4854-g8f219c47-1jammy [2,454 kB] 2024-09-10T21:21:03.497 INFO:teuthology.orchestra.run.smithi031.stdout:Get:113 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.3.0-4854-g8f219c47-1jammy [813 kB] 2024-09-10T21:21:03.511 INFO:teuthology.orchestra.run.smithi031.stdout:Get:114 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.3.0-4854-g8f219c47-1jammy [160 kB] 2024-09-10T21:21:03.527 INFO:teuthology.orchestra.run.smithi031.stdout:Get:115 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.3.0-4854-g8f219c47-1jammy [61.4 MB] 2024-09-10T21:21:03.719 INFO:teuthology.orchestra.run.smithi017.stdout:Get:104 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.3.0-4854-g8f219c47-1jammy [5,206 kB] 2024-09-10T21:21:03.924 INFO:teuthology.orchestra.run.smithi017.stdout:Get:105 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.3.0-4854-g8f219c47-1jammy [251 kB] 2024-09-10T21:21:03.952 INFO:teuthology.orchestra.run.smithi017.stdout:Get:106 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.3.0-4854-g8f219c47-1jammy [124 kB] 2024-09-10T21:21:03.954 INFO:teuthology.orchestra.run.smithi017.stdout:Get:107 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.3.0-4854-g8f219c47-1jammy [1,562 kB] 2024-09-10T21:21:04.007 INFO:teuthology.orchestra.run.smithi017.stdout:Get:108 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.3.0-4854-g8f219c47-1jammy [6,308 kB] 2024-09-10T21:21:04.301 INFO:teuthology.orchestra.run.smithi017.stdout:Get:109 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.3.0-4854-g8f219c47-1jammy [23.1 MB] 2024-09-10T21:21:05.177 INFO:teuthology.orchestra.run.smithi031.stdout:Get:116 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.3.0-4854-g8f219c47-1jammy [8,625 kB] 2024-09-10T21:21:05.253 INFO:teuthology.orchestra.run.smithi017.stdout:Get:110 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.3.0-4854-g8f219c47-1jammy [14.0 kB] 2024-09-10T21:21:05.253 INFO:teuthology.orchestra.run.smithi017.stdout:Get:111 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.3.0-4854-g8f219c47-1jammy [880 kB] 2024-09-10T21:21:05.292 INFO:teuthology.orchestra.run.smithi017.stdout:Get:112 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.3.0-4854-g8f219c47-1jammy [2,454 kB] 2024-09-10T21:21:05.389 INFO:teuthology.orchestra.run.smithi031.stdout:Get:117 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.3.0-4854-g8f219c47-1jammy [14.2 kB] 2024-09-10T21:21:05.389 INFO:teuthology.orchestra.run.smithi031.stdout:Get:118 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.3.0-4854-g8f219c47-1jammy [55.8 MB] 2024-09-10T21:21:05.398 INFO:teuthology.orchestra.run.smithi017.stdout:Get:113 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.3.0-4854-g8f219c47-1jammy [813 kB] 2024-09-10T21:21:05.425 INFO:teuthology.orchestra.run.smithi017.stdout:Get:114 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.3.0-4854-g8f219c47-1jammy [160 kB] 2024-09-10T21:21:05.445 INFO:teuthology.orchestra.run.smithi017.stdout:Get:115 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.3.0-4854-g8f219c47-1jammy [61.4 MB] 2024-09-10T21:21:06.939 INFO:teuthology.orchestra.run.smithi031.stdout:Get:119 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.3.0-4854-g8f219c47-1jammy [130 kB] 2024-09-10T21:21:06.940 INFO:teuthology.orchestra.run.smithi031.stdout:Get:120 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-mirror amd64 19.3.0-4854-g8f219c47-1jammy [233 kB] 2024-09-10T21:21:06.942 INFO:teuthology.orchestra.run.smithi031.stdout:Get:121 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-shell all 19.3.0-4854-g8f219c47-1jammy [33.9 kB] 2024-09-10T21:21:06.942 INFO:teuthology.orchestra.run.smithi031.stdout:Get:122 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-top all 19.3.0-4854-g8f219c47-1jammy [23.4 kB] 2024-09-10T21:21:06.943 INFO:teuthology.orchestra.run.smithi031.stdout:Get:123 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.3.0-4854-g8f219c47-1jammy [39.0 kB] 2024-09-10T21:21:06.943 INFO:teuthology.orchestra.run.smithi031.stdout:Get:124 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.3.0-4854-g8f219c47-1jammy [15.9 MB] 2024-09-10T21:21:07.382 INFO:teuthology.orchestra.run.smithi031.stdout:Get:125 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.3.0-4854-g8f219c47-1jammy [91.2 kB] 2024-09-10T21:21:07.621 INFO:teuthology.orchestra.run.smithi017.stdout:Get:116 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.3.0-4854-g8f219c47-1jammy [8,625 kB] 2024-09-10T21:21:07.749 INFO:teuthology.orchestra.run.smithi017.stdout:Get:117 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.3.0-4854-g8f219c47-1jammy [14.2 kB] 2024-09-10T21:21:07.749 INFO:teuthology.orchestra.run.smithi017.stdout:Get:118 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.3.0-4854-g8f219c47-1jammy [55.8 MB] 2024-09-10T21:21:08.060 INFO:teuthology.orchestra.run.smithi031.stdout:Fetched 251 MB in 6s (39.3 MB/s) 2024-09-10T21:21:08.406 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-pyperclip. 2024-09-10T21:21:08.473 INFO:teuthology.orchestra.run.smithi031.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 132928 files and directories currently installed.) 2024-09-10T21:21:08.476 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../000-python3-pyperclip_1.8.2-2_all.deb ... 2024-09-10T21:21:08.513 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-pyperclip (1.8.2-2) ... 2024-09-10T21:21:08.678 INFO:teuthology.orchestra.run.smithi017.stdout:Get:119 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.3.0-4854-g8f219c47-1jammy [130 kB] 2024-09-10T21:21:08.679 INFO:teuthology.orchestra.run.smithi017.stdout:Get:120 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-mirror amd64 19.3.0-4854-g8f219c47-1jammy [233 kB] 2024-09-10T21:21:08.681 INFO:teuthology.orchestra.run.smithi017.stdout:Get:121 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-shell all 19.3.0-4854-g8f219c47-1jammy [33.9 kB] 2024-09-10T21:21:08.682 INFO:teuthology.orchestra.run.smithi017.stdout:Get:122 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-top all 19.3.0-4854-g8f219c47-1jammy [23.4 kB] 2024-09-10T21:21:08.682 INFO:teuthology.orchestra.run.smithi017.stdout:Get:123 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.3.0-4854-g8f219c47-1jammy [39.0 kB] 2024-09-10T21:21:08.683 INFO:teuthology.orchestra.run.smithi017.stdout:Get:124 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.3.0-4854-g8f219c47-1jammy [15.9 MB] 2024-09-10T21:21:08.840 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-wcwidth. 2024-09-10T21:21:08.854 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../001-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-09-10T21:21:08.898 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-10T21:21:08.934 INFO:teuthology.orchestra.run.smithi017.stdout:Get:125 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.3.0-4854-g8f219c47-1jammy [91.2 kB] 2024-09-10T21:21:09.235 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-cmd2. 2024-09-10T21:21:09.249 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../002-python3-cmd2_2.3.3-1_all.deb ... 2024-09-10T21:21:09.293 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-cmd2 (2.3.3-1) ... 2024-09-10T21:21:09.610 INFO:teuthology.orchestra.run.smithi017.stdout:Fetched 251 MB in 8s (33.1 MB/s) 2024-09-10T21:21:09.629 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-09-10T21:21:09.643 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../003-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-09-10T21:21:09.729 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-10T21:21:09.774 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-pyperclip. 2024-09-10T21:21:09.839 INFO:teuthology.orchestra.run.smithi017.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 132928 files and directories currently installed.) 2024-09-10T21:21:09.843 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../000-python3-pyperclip_1.8.2-2_all.deb ... 2024-09-10T21:21:09.889 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-pyperclip (1.8.2-2) ... 2024-09-10T21:21:10.083 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-09-10T21:21:10.098 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../004-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-09-10T21:21:10.140 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-10T21:21:10.166 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-wcwidth. 2024-09-10T21:21:10.181 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../001-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-09-10T21:21:10.250 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-10T21:21:10.444 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-09-10T21:21:10.459 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../005-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-09-10T21:21:10.494 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-10T21:21:10.652 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-cmd2. 2024-09-10T21:21:10.667 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../002-python3-cmd2_2.3.3-1_all.deb ... 2024-09-10T21:21:10.711 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-cmd2 (2.3.3-1) ... 2024-09-10T21:21:10.924 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-09-10T21:21:10.939 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../006-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-10T21:21:10.997 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-10T21:21:11.072 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-09-10T21:21:11.087 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../003-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-09-10T21:21:11.173 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-10T21:21:11.383 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-09-10T21:21:11.398 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../007-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-10T21:21:11.441 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-10T21:21:11.509 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-09-10T21:21:11.524 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../004-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-09-10T21:21:11.567 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-10T21:21:11.753 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-09-10T21:21:11.768 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../008-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-10T21:21:11.811 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-10T21:21:11.903 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-09-10T21:21:11.918 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../005-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-09-10T21:21:11.995 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-10T21:21:12.172 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-09-10T21:21:12.188 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../009-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-09-10T21:21:12.238 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-10T21:21:12.357 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-09-10T21:21:12.372 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../006-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-10T21:21:12.422 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-10T21:21:12.558 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package librados2. 2024-09-10T21:21:12.572 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../010-librados2_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:21:12.616 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking librados2 (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:21:12.859 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-09-10T21:21:12.874 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../007-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-10T21:21:12.918 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-10T21:21:13.060 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package libnbd0. 2024-09-10T21:21:13.074 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../011-libnbd0_1.10.5-1_amd64.deb ... 2024-09-10T21:21:13.110 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking libnbd0 (1.10.5-1) ... 2024-09-10T21:21:13.262 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-09-10T21:21:13.275 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../008-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-10T21:21:13.312 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-10T21:21:13.388 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package librbd1. 2024-09-10T21:21:13.401 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../012-librbd1_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:21:13.438 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking librbd1 (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:21:13.857 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-09-10T21:21:13.857 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package libcephfs2. 2024-09-10T21:21:13.868 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../009-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-09-10T21:21:13.870 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../013-libcephfs2_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:21:13.898 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-10T21:21:13.915 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking libcephfs2 (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:21:14.193 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-rados. 2024-09-10T21:21:14.206 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../014-python3-rados_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:21:14.218 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package librados2. 2024-09-10T21:21:14.229 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../010-librados2_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:21:14.243 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-rados (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:21:14.268 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking librados2 (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:21:14.587 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-09-10T21:21:14.601 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../015-python3-ceph-argparse_19.3.0-4854-g8f219c47-1jammy_all.deb ... 2024-09-10T21:21:14.645 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-ceph-argparse (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:21:14.729 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package libnbd0. 2024-09-10T21:21:14.740 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../011-libnbd0_1.10.5-1_amd64.deb ... 2024-09-10T21:21:14.778 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking libnbd0 (1.10.5-1) ... 2024-09-10T21:21:14.884 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-cephfs. 2024-09-10T21:21:14.894 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../016-python3-cephfs_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:21:14.931 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-cephfs (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:21:15.058 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package librbd1. 2024-09-10T21:21:15.070 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../012-librbd1_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:21:15.108 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking librbd1 (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:21:15.234 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-ceph-common. 2024-09-10T21:21:15.247 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../017-python3-ceph-common_19.3.0-4854-g8f219c47-1jammy_all.deb ... 2024-09-10T21:21:15.283 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-ceph-common (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:21:15.553 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package libcephfs2. 2024-09-10T21:21:15.564 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../013-libcephfs2_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:21:15.595 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-prettytable. 2024-09-10T21:21:15.602 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking libcephfs2 (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:21:15.608 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../018-python3-prettytable_2.5.0-2_all.deb ... 2024-09-10T21:21:15.653 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-09-10T21:21:15.871 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-rados. 2024-09-10T21:21:15.883 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../014-python3-rados_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:21:15.921 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-rados (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:21:15.922 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-rbd. 2024-09-10T21:21:15.935 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../019-python3-rbd_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:21:15.972 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-rbd (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:21:16.241 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-09-10T21:21:16.252 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../015-python3-ceph-argparse_19.3.0-4854-g8f219c47-1jammy_all.deb ... 2024-09-10T21:21:16.291 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-ceph-argparse (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:21:16.341 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-09-10T21:21:16.354 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../020-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-09-10T21:21:16.399 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-10T21:21:16.510 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-cephfs. 2024-09-10T21:21:16.521 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../016-python3-cephfs_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:21:16.552 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-cephfs (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:21:16.644 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-09-10T21:21:16.657 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../021-libreadline-dev_8.1.2-1_amd64.deb ... 2024-09-10T21:21:16.718 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-09-10T21:21:16.879 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-ceph-common. 2024-09-10T21:21:16.890 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../017-python3-ceph-common_19.3.0-4854-g8f219c47-1jammy_all.deb ... 2024-09-10T21:21:16.929 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-ceph-common (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:21:16.980 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-09-10T21:21:16.993 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../022-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-09-10T21:21:17.038 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-10T21:21:17.257 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-prettytable. 2024-09-10T21:21:17.268 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../018-python3-prettytable_2.5.0-2_all.deb ... 2024-09-10T21:21:17.315 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-09-10T21:21:17.391 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package lua5.1. 2024-09-10T21:21:17.405 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../023-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-09-10T21:21:17.449 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-09-10T21:21:17.576 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-rbd. 2024-09-10T21:21:17.587 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../019-python3-rbd_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:21:17.626 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-rbd (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:21:17.962 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-09-10T21:21:17.973 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../020-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-09-10T21:21:18.012 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-10T21:21:18.027 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package lua-any. 2024-09-10T21:21:18.041 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../024-lua-any_27ubuntu1_all.deb ... 2024-09-10T21:21:18.085 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking lua-any (27ubuntu1) ... 2024-09-10T21:21:18.256 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-09-10T21:21:18.263 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package zip. 2024-09-10T21:21:18.267 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../021-libreadline-dev_8.1.2-1_amd64.deb ... 2024-09-10T21:21:18.276 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../025-zip_3.0-12build2_amd64.deb ... 2024-09-10T21:21:18.305 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-09-10T21:21:18.312 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking zip (3.0-12build2) ... 2024-09-10T21:21:18.600 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-09-10T21:21:18.611 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../022-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-09-10T21:21:18.649 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package unzip. 2024-09-10T21:21:18.650 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-10T21:21:18.662 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../026-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-09-10T21:21:18.707 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-09-10T21:21:18.987 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package lua5.1. 2024-09-10T21:21:18.998 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../023-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-09-10T21:21:19.037 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-09-10T21:21:19.111 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package luarocks. 2024-09-10T21:21:19.125 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../027-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-09-10T21:21:19.169 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-09-10T21:21:19.339 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package lua-any. 2024-09-10T21:21:19.351 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../024-lua-any_27ubuntu1_all.deb ... 2024-09-10T21:21:19.389 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking lua-any (27ubuntu1) ... 2024-09-10T21:21:19.601 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package librgw2. 2024-09-10T21:21:19.614 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../028-librgw2_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:21:19.617 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package zip. 2024-09-10T21:21:19.628 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../025-zip_3.0-12build2_amd64.deb ... 2024-09-10T21:21:19.659 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking zip (3.0-12build2) ... 2024-09-10T21:21:19.659 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking librgw2 (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:21:19.953 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package unzip. 2024-09-10T21:21:19.965 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../026-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-09-10T21:21:20.003 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-09-10T21:21:20.287 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-rgw. 2024-09-10T21:21:20.301 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../029-python3-rgw_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:21:20.345 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-rgw (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:21:20.407 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package luarocks. 2024-09-10T21:21:20.418 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../027-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-09-10T21:21:20.457 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-09-10T21:21:20.606 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package liboath0:amd64. 2024-09-10T21:21:20.620 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../030-liboath0_2.6.7-3build1_amd64.deb ... 2024-09-10T21:21:20.664 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-09-10T21:21:20.872 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package librgw2. 2024-09-10T21:21:20.886 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../028-librgw2_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:21:20.931 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking librgw2 (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:21:21.000 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package libradosstriper1. 2024-09-10T21:21:21.017 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../031-libradosstriper1_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:21:21.058 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking libradosstriper1 (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:21:21.437 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package ceph-common. 2024-09-10T21:21:21.451 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../032-ceph-common_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:21:21.466 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-rgw. 2024-09-10T21:21:21.478 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../029-python3-rgw_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:21:21.486 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking ceph-common (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:21:21.508 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-rgw (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:21:21.853 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package liboath0:amd64. 2024-09-10T21:21:21.865 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../030-liboath0_2.6.7-3build1_amd64.deb ... 2024-09-10T21:21:21.902 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-09-10T21:21:22.213 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package libradosstriper1. 2024-09-10T21:21:22.226 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../031-libradosstriper1_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:21:22.263 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking libradosstriper1 (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:21:22.683 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package ceph-base. 2024-09-10T21:21:22.696 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../033-ceph-base_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:21:22.700 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package ceph-common. 2024-09-10T21:21:22.713 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../032-ceph-common_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:21:22.742 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking ceph-base (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:21:22.749 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking ceph-common (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:21:23.312 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-09-10T21:21:23.325 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../034-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-09-10T21:21:23.362 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-09-10T21:21:23.664 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-cheroot. 2024-09-10T21:21:23.676 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../035-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-09-10T21:21:23.714 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-10T21:21:24.017 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-09-10T21:21:24.029 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../036-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-09-10T21:21:24.059 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-09-10T21:21:24.255 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package ceph-base. 2024-09-10T21:21:24.265 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../033-ceph-base_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:21:24.320 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-jaraco.text. 2024-09-10T21:21:24.322 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking ceph-base (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:21:24.332 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../037-python3-jaraco.text_3.6.0-2_all.deb ... 2024-09-10T21:21:24.361 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-09-10T21:21:24.647 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-09-10T21:21:24.660 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../038-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-09-10T21:21:24.697 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-09-10T21:21:24.943 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-09-10T21:21:24.950 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../034-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-09-10T21:21:24.992 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-09-10T21:21:25.033 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-tempora. 2024-09-10T21:21:25.045 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../039-python3-tempora_4.1.2-1_all.deb ... 2024-09-10T21:21:25.083 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-09-10T21:21:25.345 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-cheroot. 2024-09-10T21:21:25.356 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../035-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-09-10T21:21:25.380 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-portend. 2024-09-10T21:21:25.392 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../040-python3-portend_3.0.0-1_all.deb ... 2024-09-10T21:21:25.403 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-10T21:21:25.429 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-09-10T21:21:25.732 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-09-10T21:21:25.745 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../041-python3-zc.lockfile_2.0-1_all.deb ... 2024-09-10T21:21:25.764 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-09-10T21:21:25.773 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../036-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-09-10T21:21:25.782 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-09-10T21:21:25.854 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-09-10T21:21:26.093 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-cherrypy3. 2024-09-10T21:21:26.106 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../042-python3-cherrypy3_18.6.1-4_all.deb ... 2024-09-10T21:21:26.143 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-09-10T21:21:26.209 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-jaraco.text. 2024-09-10T21:21:26.218 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../037-python3-jaraco.text_3.6.0-2_all.deb ... 2024-09-10T21:21:26.267 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-09-10T21:21:26.522 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-natsort. 2024-09-10T21:21:26.535 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../043-python3-natsort_8.0.2-1_all.deb ... 2024-09-10T21:21:26.572 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-09-10T21:21:26.678 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-09-10T21:21:26.687 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../038-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-09-10T21:21:26.728 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-09-10T21:21:26.867 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-logutils. 2024-09-10T21:21:26.879 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../044-python3-logutils_0.3.3-8_all.deb ... 2024-09-10T21:21:26.916 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-09-10T21:21:27.064 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-tempora. 2024-09-10T21:21:27.073 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../039-python3-tempora_4.1.2-1_all.deb ... 2024-09-10T21:21:27.122 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-09-10T21:21:27.244 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-markupsafe. 2024-09-10T21:21:27.257 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../045-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-09-10T21:21:27.294 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-09-10T21:21:27.500 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-portend. 2024-09-10T21:21:27.509 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../040-python3-portend_3.0.0-1_all.deb ... 2024-09-10T21:21:27.547 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-mako. 2024-09-10T21:21:27.560 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../046-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-09-10T21:21:27.584 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-09-10T21:21:27.597 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-10T21:21:27.925 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-simplegeneric. 2024-09-10T21:21:27.938 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../047-python3-simplegeneric_0.8.1-3_all.deb ... 2024-09-10T21:21:27.953 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-09-10T21:21:27.962 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../041-python3-zc.lockfile_2.0-1_all.deb ... 2024-09-10T21:21:27.975 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-09-10T21:21:28.011 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-09-10T21:21:28.339 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-cherrypy3. 2024-09-10T21:21:28.348 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../042-python3-cherrypy3_18.6.1-4_all.deb ... 2024-09-10T21:21:28.389 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-09-10T21:21:28.453 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-singledispatch. 2024-09-10T21:21:28.466 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../048-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-09-10T21:21:28.503 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-09-10T21:21:28.759 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-natsort. 2024-09-10T21:21:28.768 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../043-python3-natsort_8.0.2-1_all.deb ... 2024-09-10T21:21:28.809 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-09-10T21:21:28.865 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-webob. 2024-09-10T21:21:28.881 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../049-python3-webob_1%3a1.8.6-1.1ubuntu0.1_all.deb ... 2024-09-10T21:21:28.923 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-10T21:21:29.079 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-logutils. 2024-09-10T21:21:29.087 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../044-python3-logutils_0.3.3-8_all.deb ... 2024-09-10T21:21:29.128 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-09-10T21:21:29.201 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-waitress. 2024-09-10T21:21:29.216 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../050-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-09-10T21:21:29.254 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-10T21:21:29.498 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-markupsafe. 2024-09-10T21:21:29.507 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../045-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-09-10T21:21:29.546 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-tempita. 2024-09-10T21:21:29.547 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-09-10T21:21:29.559 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../051-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-09-10T21:21:29.595 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-09-10T21:21:29.881 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-paste. 2024-09-10T21:21:29.895 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-mako. 2024-09-10T21:21:29.895 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../052-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-09-10T21:21:29.904 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../046-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-09-10T21:21:29.950 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-10T21:21:29.973 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-09-10T21:21:30.276 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-09-10T21:21:30.287 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-simplegeneric. 2024-09-10T21:21:30.290 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../053-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-09-10T21:21:30.296 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../047-python3-simplegeneric_0.8.1-3_all.deb ... 2024-09-10T21:21:30.337 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-09-10T21:21:30.342 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-09-10T21:21:30.606 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-singledispatch. 2024-09-10T21:21:30.615 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../048-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-09-10T21:21:30.637 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-pastedeploy. 2024-09-10T21:21:30.648 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-09-10T21:21:30.651 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../054-python3-pastedeploy_2.1.1-1_all.deb ... 2024-09-10T21:21:30.695 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-09-10T21:21:30.926 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-webob. 2024-09-10T21:21:30.935 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../049-python3-webob_1%3a1.8.6-1.1ubuntu0.1_all.deb ... 2024-09-10T21:21:30.975 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-10T21:21:30.982 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-webtest. 2024-09-10T21:21:30.995 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../055-python3-webtest_2.0.35-1_all.deb ... 2024-09-10T21:21:31.040 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-09-10T21:21:31.362 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-waitress. 2024-09-10T21:21:31.370 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../050-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-09-10T21:21:31.384 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-pecan. 2024-09-10T21:21:31.399 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../056-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-09-10T21:21:31.414 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-10T21:21:31.442 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-09-10T21:21:31.748 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-tempita. 2024-09-10T21:21:31.759 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../051-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-09-10T21:21:31.798 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-09-10T21:21:31.831 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-certifi. 2024-09-10T21:21:31.845 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../057-python3-certifi_2020.6.20-1_all.deb ... 2024-09-10T21:21:31.889 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-09-10T21:21:32.093 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-paste. 2024-09-10T21:21:32.102 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../052-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-09-10T21:21:32.134 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-09-10T21:21:32.209 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-idna. 2024-09-10T21:21:32.222 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../058-python3-idna_3.3-1ubuntu0.1_all.deb ... 2024-09-10T21:21:32.284 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-idna (3.3-1ubuntu0.1) ... 2024-09-10T21:21:32.512 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-09-10T21:21:32.522 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../053-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-09-10T21:21:32.561 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-09-10T21:21:32.595 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-urllib3. 2024-09-10T21:21:32.610 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../059-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-09-10T21:21:32.653 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-10T21:21:32.881 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-pastedeploy. 2024-09-10T21:21:32.891 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../054-python3-pastedeploy_2.1.1-1_all.deb ... 2024-09-10T21:21:32.931 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-09-10T21:21:32.981 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-requests. 2024-09-10T21:21:32.995 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../060-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-09-10T21:21:33.040 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-10T21:21:33.201 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-webtest. 2024-09-10T21:21:33.210 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../055-python3-webtest_2.0.35-1_all.deb ... 2024-09-10T21:21:33.242 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-09-10T21:21:33.351 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-werkzeug. 2024-09-10T21:21:33.368 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../061-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.2_all.deb ... 2024-09-10T21:21:33.426 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-10T21:21:33.537 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-pecan. 2024-09-10T21:21:33.546 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../056-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-09-10T21:21:33.586 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-09-10T21:21:34.384 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-09-10T21:21:34.398 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../062-ceph-mgr-modules-core_19.3.0-4854-g8f219c47-1jammy_all.deb ... 2024-09-10T21:21:34.563 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking ceph-mgr-modules-core (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:21:34.748 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-certifi. 2024-09-10T21:21:34.757 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../057-python3-certifi_2020.6.20-1_all.deb ... 2024-09-10T21:21:34.800 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-09-10T21:21:34.976 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-09-10T21:21:34.991 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../063-libsqlite3-mod-ceph_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:21:35.034 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking libsqlite3-mod-ceph (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:21:35.119 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-idna. 2024-09-10T21:21:35.129 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../058-python3-idna_3.3-1ubuntu0.1_all.deb ... 2024-09-10T21:21:35.169 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-idna (3.3-1ubuntu0.1) ... 2024-09-10T21:21:35.354 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-bcrypt. 2024-09-10T21:21:35.368 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../064-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-09-10T21:21:35.412 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-09-10T21:21:35.489 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-urllib3. 2024-09-10T21:21:35.498 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../059-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-09-10T21:21:35.538 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-10T21:21:35.673 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-openssl. 2024-09-10T21:21:35.688 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../065-python3-openssl_21.0.0-1_all.deb ... 2024-09-10T21:21:35.723 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-09-10T21:21:35.891 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-requests. 2024-09-10T21:21:35.901 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../060-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-09-10T21:21:35.950 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-10T21:21:36.076 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package ceph-mgr. 2024-09-10T21:21:36.091 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../066-ceph-mgr_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:21:36.134 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking ceph-mgr (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:21:36.194 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-werkzeug. 2024-09-10T21:21:36.204 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../061-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.2_all.deb ... 2024-09-10T21:21:36.236 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-10T21:21:36.530 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-09-10T21:21:36.540 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../062-ceph-mgr-modules-core_19.3.0-4854-g8f219c47-1jammy_all.deb ... 2024-09-10T21:21:36.546 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package ceph-mon. 2024-09-10T21:21:36.560 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../067-ceph-mon_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:21:36.580 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking ceph-mgr-modules-core (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:21:36.604 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking ceph-mon (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:21:36.960 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-09-10T21:21:36.970 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../063-libsqlite3-mod-ceph_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:21:37.035 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking libsqlite3-mod-ceph (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:21:37.132 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package ceph-osd. 2024-09-10T21:21:37.147 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../068-ceph-osd_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:21:37.182 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking ceph-osd (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:21:37.346 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-bcrypt. 2024-09-10T21:21:37.355 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../064-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-09-10T21:21:37.396 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-09-10T21:21:37.673 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-openssl. 2024-09-10T21:21:37.683 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../065-python3-openssl_21.0.0-1_all.deb ... 2024-09-10T21:21:37.715 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-09-10T21:21:37.935 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package ceph. 2024-09-10T21:21:37.949 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../069-ceph_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:21:37.994 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking ceph (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:21:38.051 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package ceph-mgr. 2024-09-10T21:21:38.061 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../066-ceph-mgr_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:21:38.101 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking ceph-mgr (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:21:38.347 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package ceph-fuse. 2024-09-10T21:21:38.361 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../070-ceph-fuse_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:21:38.404 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking ceph-fuse (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:21:38.504 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package ceph-mon. 2024-09-10T21:21:38.514 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../067-ceph-mon_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:21:38.554 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking ceph-mon (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:21:38.799 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package ceph-mds. 2024-09-10T21:21:38.814 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../071-ceph-mds_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:21:38.858 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking ceph-mds (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:21:39.065 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package ceph-osd. 2024-09-10T21:21:39.075 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../068-ceph-osd_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:21:39.115 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking ceph-osd (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:21:39.211 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package cephadm. 2024-09-10T21:21:39.225 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../072-cephadm_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:21:39.299 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking cephadm (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:21:39.638 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-asyncssh. 2024-09-10T21:21:39.653 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../073-python3-asyncssh_2.5.0-1_all.deb ... 2024-09-10T21:21:39.697 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-09-10T21:21:40.007 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python-babel-localedata. 2024-09-10T21:21:40.022 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../074-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-09-10T21:21:40.057 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-10T21:21:40.110 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package ceph. 2024-09-10T21:21:40.120 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../069-ceph_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:21:40.160 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking ceph (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:21:40.479 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package ceph-fuse. 2024-09-10T21:21:40.492 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../070-ceph-fuse_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:21:40.562 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking ceph-fuse (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:21:40.890 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package ceph-mds. 2024-09-10T21:21:40.900 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../071-ceph-mds_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:21:40.940 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking ceph-mds (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:21:41.054 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-babel. 2024-09-10T21:21:41.068 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../075-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-09-10T21:21:41.112 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-09-10T21:21:41.285 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package cephadm. 2024-09-10T21:21:41.295 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../072-cephadm_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:21:41.327 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking cephadm (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:21:41.424 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-jinja2. 2024-09-10T21:21:41.437 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../076-python3-jinja2_3.0.3-1ubuntu0.2_all.deb ... 2024-09-10T21:21:41.482 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-10T21:21:41.629 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-asyncssh. 2024-09-10T21:21:41.639 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../073-python3-asyncssh_2.5.0-1_all.deb ... 2024-09-10T21:21:41.712 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-09-10T21:21:41.851 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-09-10T21:21:41.865 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../077-ceph-mgr-cephadm_19.3.0-4854-g8f219c47-1jammy_all.deb ... 2024-09-10T21:21:41.910 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking ceph-mgr-cephadm (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:21:42.007 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python-babel-localedata. 2024-09-10T21:21:42.016 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../074-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-09-10T21:21:42.056 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-10T21:21:42.271 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-repoze.lru. 2024-09-10T21:21:42.285 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../078-python3-repoze.lru_0.7-2_all.deb ... 2024-09-10T21:21:42.329 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-09-10T21:21:42.633 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-routes. 2024-09-10T21:21:42.647 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../079-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-09-10T21:21:42.691 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-09-10T21:21:43.027 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-09-10T21:21:43.042 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../080-ceph-mgr-dashboard_19.3.0-4854-g8f219c47-1jammy_all.deb ... 2024-09-10T21:21:43.077 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking ceph-mgr-dashboard (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:21:43.094 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-babel. 2024-09-10T21:21:43.103 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../075-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-09-10T21:21:43.144 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-09-10T21:21:43.489 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-jinja2. 2024-09-10T21:21:43.498 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../076-python3-jinja2_3.0.3-1ubuntu0.2_all.deb ... 2024-09-10T21:21:43.538 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-10T21:21:43.883 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-09-10T21:21:43.893 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../077-ceph-mgr-cephadm_19.3.0-4854-g8f219c47-1jammy_all.deb ... 2024-09-10T21:21:43.933 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking ceph-mgr-cephadm (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:21:44.303 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-repoze.lru. 2024-09-10T21:21:44.313 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../078-python3-repoze.lru_0.7-2_all.deb ... 2024-09-10T21:21:44.353 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-09-10T21:21:44.647 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-routes. 2024-09-10T21:21:44.658 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../079-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-09-10T21:21:44.689 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-09-10T21:21:45.000 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-09-10T21:21:45.010 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../080-ceph-mgr-dashboard_19.3.0-4854-g8f219c47-1jammy_all.deb ... 2024-09-10T21:21:45.050 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking ceph-mgr-dashboard (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:21:45.424 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-09-10T21:21:45.439 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../081-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-09-10T21:21:45.482 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-10T21:21:45.986 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-joblib. 2024-09-10T21:21:45.999 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../082-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-09-10T21:21:46.121 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-09-10T21:21:46.500 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-09-10T21:21:46.514 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../083-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-09-10T21:21:46.558 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-09-10T21:21:46.861 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-sklearn. 2024-09-10T21:21:46.876 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../084-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-09-10T21:21:46.919 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-10T21:21:47.389 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-09-10T21:21:47.399 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../081-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-09-10T21:21:47.438 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-10T21:21:47.595 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-09-10T21:21:47.609 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../085-ceph-mgr-diskprediction-local_19.3.0-4854-g8f219c47-1jammy_all.deb ... 2024-09-10T21:21:47.653 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking ceph-mgr-diskprediction-local (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:21:47.933 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-joblib. 2024-09-10T21:21:47.943 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../082-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-09-10T21:21:48.077 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-09-10T21:21:48.439 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-09-10T21:21:48.449 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../083-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-09-10T21:21:48.480 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-09-10T21:21:48.742 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-sklearn. 2024-09-10T21:21:48.752 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../084-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-09-10T21:21:48.792 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-10T21:21:48.840 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-cachetools. 2024-09-10T21:21:48.855 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../086-python3-cachetools_5.0.0-1_all.deb ... 2024-09-10T21:21:48.932 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-09-10T21:21:49.234 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-pyasn1. 2024-09-10T21:21:49.253 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../087-python3-pyasn1_0.4.8-1_all.deb ... 2024-09-10T21:21:49.293 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-09-10T21:21:49.417 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-09-10T21:21:49.426 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../085-ceph-mgr-diskprediction-local_19.3.0-4854-g8f219c47-1jammy_all.deb ... 2024-09-10T21:21:49.492 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking ceph-mgr-diskprediction-local (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:21:49.621 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-09-10T21:21:49.637 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../088-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-09-10T21:21:49.679 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-09-10T21:21:50.064 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-rsa. 2024-09-10T21:21:50.080 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../089-python3-rsa_4.8-1_all.deb ... 2024-09-10T21:21:50.124 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-rsa (4.8-1) ... 2024-09-10T21:21:50.444 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-google-auth. 2024-09-10T21:21:50.461 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../090-python3-google-auth_1.5.1-3_all.deb ... 2024-09-10T21:21:50.503 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-09-10T21:21:50.687 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-cachetools. 2024-09-10T21:21:50.698 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../086-python3-cachetools_5.0.0-1_all.deb ... 2024-09-10T21:21:50.770 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-09-10T21:21:50.823 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-09-10T21:21:50.840 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../091-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-09-10T21:21:50.872 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-10T21:21:51.073 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-pyasn1. 2024-09-10T21:21:51.083 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../087-python3-pyasn1_0.4.8-1_all.deb ... 2024-09-10T21:21:51.122 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-09-10T21:21:51.159 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-websocket. 2024-09-10T21:21:51.175 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../092-python3-websocket_1.2.3-1_all.deb ... 2024-09-10T21:21:51.242 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-09-10T21:21:51.467 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-09-10T21:21:51.478 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../088-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-09-10T21:21:51.517 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-09-10T21:21:51.562 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-kubernetes. 2024-09-10T21:21:51.579 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../093-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-09-10T21:21:51.620 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-10T21:21:51.797 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-rsa. 2024-09-10T21:21:51.808 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../089-python3-rsa_4.8-1_all.deb ... 2024-09-10T21:21:51.838 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-rsa (4.8-1) ... 2024-09-10T21:21:52.175 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-google-auth. 2024-09-10T21:21:52.186 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../090-python3-google-auth_1.5.1-3_all.deb ... 2024-09-10T21:21:52.225 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-09-10T21:21:52.251 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-09-10T21:21:52.266 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../094-ceph-mgr-k8sevents_19.3.0-4854-g8f219c47-1jammy_all.deb ... 2024-09-10T21:21:52.301 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking ceph-mgr-k8sevents (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:21:52.579 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-09-10T21:21:52.589 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../091-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-09-10T21:21:52.628 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-10T21:21:52.629 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package libonig5:amd64. 2024-09-10T21:21:52.643 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../095-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-09-10T21:21:52.695 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-10T21:21:52.940 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-websocket. 2024-09-10T21:21:52.950 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../092-python3-websocket_1.2.3-1_all.deb ... 2024-09-10T21:21:52.989 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-09-10T21:21:53.040 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package libjq1:amd64. 2024-09-10T21:21:53.056 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../096-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-09-10T21:21:53.098 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-10T21:21:53.301 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-kubernetes. 2024-09-10T21:21:53.311 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../093-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-09-10T21:21:53.342 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-10T21:21:53.350 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package jq. 2024-09-10T21:21:53.366 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../097-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-09-10T21:21:53.400 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-09-10T21:21:53.703 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package socat. 2024-09-10T21:21:53.719 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../098-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-09-10T21:21:53.770 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-09-10T21:21:54.081 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-09-10T21:21:54.090 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../094-ceph-mgr-k8sevents_19.3.0-4854-g8f219c47-1jammy_all.deb ... 2024-09-10T21:21:54.131 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking ceph-mgr-k8sevents (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:21:54.140 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package xmlstarlet. 2024-09-10T21:21:54.156 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../099-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-09-10T21:21:54.207 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-09-10T21:21:54.433 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package libonig5:amd64. 2024-09-10T21:21:54.443 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../095-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-09-10T21:21:54.484 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-10T21:21:54.535 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package ceph-test. 2024-09-10T21:21:54.551 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../100-ceph-test_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:21:54.594 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking ceph-test (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:21:54.828 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package libjq1:amd64. 2024-09-10T21:21:54.838 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../096-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-09-10T21:21:54.920 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-10T21:21:55.181 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package jq. 2024-09-10T21:21:55.191 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../097-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-09-10T21:21:55.230 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-09-10T21:21:55.500 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package socat. 2024-09-10T21:21:55.510 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../098-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-09-10T21:21:55.541 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-09-10T21:21:55.803 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package xmlstarlet. 2024-09-10T21:21:55.813 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../099-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-09-10T21:21:55.852 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-09-10T21:21:56.215 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package ceph-test. 2024-09-10T21:21:56.225 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../100-ceph-test_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:21:56.265 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking ceph-test (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:21:56.857 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package ceph-volume. 2024-09-10T21:21:56.874 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../101-ceph-volume_19.3.0-4854-g8f219c47-1jammy_all.deb ... 2024-09-10T21:21:56.923 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking ceph-volume (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:21:57.336 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package cephfs-mirror. 2024-09-10T21:21:57.352 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../102-cephfs-mirror_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:21:57.393 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking cephfs-mirror (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:21:57.705 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-colorama. 2024-09-10T21:21:57.722 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../103-python3-colorama_0.4.4-1_all.deb ... 2024-09-10T21:21:57.755 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-colorama (0.4.4-1) ... 2024-09-10T21:21:57.974 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package cephfs-shell. 2024-09-10T21:21:57.991 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../104-cephfs-shell_19.3.0-4854-g8f219c47-1jammy_all.deb ... 2024-09-10T21:21:58.032 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking cephfs-shell (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:21:58.285 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package ceph-volume. 2024-09-10T21:21:58.296 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../101-ceph-volume_19.3.0-4854-g8f219c47-1jammy_all.deb ... 2024-09-10T21:21:58.318 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package cephfs-top. 2024-09-10T21:21:58.335 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking ceph-volume (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:21:58.336 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../105-cephfs-top_19.3.0-4854-g8f219c47-1jammy_all.deb ... 2024-09-10T21:21:58.376 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking cephfs-top (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:21:58.629 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package libcephfs-dev. 2024-09-10T21:21:58.646 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../106-libcephfs-dev_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:21:58.687 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking libcephfs-dev (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:21:58.756 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package cephfs-mirror. 2024-09-10T21:21:58.767 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../102-cephfs-mirror_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:21:58.806 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking cephfs-mirror (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:21:59.092 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-colorama. 2024-09-10T21:21:59.103 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../103-python3-colorama_0.4.4-1_all.deb ... 2024-09-10T21:21:59.167 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-colorama (0.4.4-1) ... 2024-09-10T21:21:59.215 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package lua-socket:amd64. 2024-09-10T21:21:59.232 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../107-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-09-10T21:21:59.273 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-10T21:21:59.353 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package cephfs-shell. 2024-09-10T21:21:59.363 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../104-cephfs-shell_19.3.0-4854-g8f219c47-1jammy_all.deb ... 2024-09-10T21:21:59.394 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking cephfs-shell (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:21:59.594 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package lua-sec:amd64. 2024-09-10T21:21:59.611 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../108-lua-sec_1.0.2-1_amd64.deb ... 2024-09-10T21:21:59.647 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package cephfs-top. 2024-09-10T21:21:59.652 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-09-10T21:21:59.657 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../105-cephfs-top_19.3.0-4854-g8f219c47-1jammy_all.deb ... 2024-09-10T21:21:59.697 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking cephfs-top (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:21:59.924 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package libcephfs-dev. 2024-09-10T21:21:59.935 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../106-libcephfs-dev_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:21:59.974 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking libcephfs-dev (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:21:59.998 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package nvme-cli. 2024-09-10T21:22:00.017 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../109-nvme-cli_1.16-3ubuntu0.2_amd64.deb ... 2024-09-10T21:22:00.073 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking nvme-cli (1.16-3ubuntu0.2) ... 2024-09-10T21:22:00.285 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package lua-socket:amd64. 2024-09-10T21:22:00.295 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../107-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-09-10T21:22:00.360 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-10T21:22:00.478 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package pkg-config. 2024-09-10T21:22:00.494 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../110-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-09-10T21:22:00.528 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-09-10T21:22:00.698 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package lua-sec:amd64. 2024-09-10T21:22:00.709 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../108-lua-sec_1.0.2-1_amd64.deb ... 2024-09-10T21:22:00.747 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-09-10T21:22:00.800 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-09-10T21:22:00.814 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../111-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-09-10T21:22:00.856 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-09-10T21:22:01.118 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package nvme-cli. 2024-09-10T21:22:01.129 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../109-nvme-cli_1.16-3ubuntu0.2_amd64.deb ... 2024-09-10T21:22:01.159 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking nvme-cli (1.16-3ubuntu0.2) ... 2024-09-10T21:22:01.301 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-iniconfig. 2024-09-10T21:22:01.317 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../112-python3-iniconfig_1.1.1-2_all.deb ... 2024-09-10T21:22:01.359 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-09-10T21:22:01.556 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package pkg-config. 2024-09-10T21:22:01.566 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../110-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-09-10T21:22:01.597 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-09-10T21:22:01.662 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-pastescript. 2024-09-10T21:22:01.678 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../113-python3-pastescript_2.0.2-4_all.deb ... 2024-09-10T21:22:01.720 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-09-10T21:22:01.875 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-09-10T21:22:01.886 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../111-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-09-10T21:22:01.925 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-09-10T21:22:02.032 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-pluggy. 2024-09-10T21:22:02.049 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../114-python3-pluggy_0.13.0-7.1_all.deb ... 2024-09-10T21:22:02.090 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-09-10T21:22:02.328 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-iniconfig. 2024-09-10T21:22:02.338 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../112-python3-iniconfig_1.1.1-2_all.deb ... 2024-09-10T21:22:02.369 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-09-10T21:22:02.401 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-psutil. 2024-09-10T21:22:02.418 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../115-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-09-10T21:22:02.459 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-09-10T21:22:02.681 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-pastescript. 2024-09-10T21:22:02.691 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../113-python3-pastescript_2.0.2-4_all.deb ... 2024-09-10T21:22:02.730 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-09-10T21:22:02.762 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-py. 2024-09-10T21:22:02.779 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../116-python3-py_1.10.0-1_all.deb ... 2024-09-10T21:22:02.812 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-py (1.10.0-1) ... 2024-09-10T21:22:03.050 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-pluggy. 2024-09-10T21:22:03.061 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../114-python3-pluggy_0.13.0-7.1_all.deb ... 2024-09-10T21:22:03.100 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-09-10T21:22:03.140 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-pygments. 2024-09-10T21:22:03.157 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../117-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-09-10T21:22:03.198 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-09-10T21:22:03.436 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-psutil. 2024-09-10T21:22:03.447 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../115-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-09-10T21:22:03.486 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-09-10T21:22:03.661 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-pyinotify. 2024-09-10T21:22:03.676 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../118-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-09-10T21:22:03.719 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-09-10T21:22:03.847 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-py. 2024-09-10T21:22:03.858 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../116-python3-py_1.10.0-1_all.deb ... 2024-09-10T21:22:03.897 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-py (1.10.0-1) ... 2024-09-10T21:22:04.039 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-toml. 2024-09-10T21:22:04.055 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../119-python3-toml_0.10.2-1_all.deb ... 2024-09-10T21:22:04.122 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-09-10T21:22:04.226 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-pygments. 2024-09-10T21:22:04.237 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../117-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-09-10T21:22:04.275 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-09-10T21:22:04.375 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-pytest. 2024-09-10T21:22:04.391 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../120-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-09-10T21:22:04.433 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-09-10T21:22:04.754 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-pyinotify. 2024-09-10T21:22:04.765 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../118-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-09-10T21:22:04.803 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package python3-simplejson. 2024-09-10T21:22:04.804 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-09-10T21:22:04.819 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../121-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-09-10T21:22:04.861 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-09-10T21:22:05.082 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-toml. 2024-09-10T21:22:05.093 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../119-python3-toml_0.10.2-1_all.deb ... 2024-09-10T21:22:05.133 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-09-10T21:22:05.140 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-09-10T21:22:05.149 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../122-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-09-10T21:22:05.191 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-09-10T21:22:05.393 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-pytest. 2024-09-10T21:22:05.404 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../120-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-09-10T21:22:05.442 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-09-10T21:22:05.911 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package radosgw. 2024-09-10T21:22:05.927 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../123-radosgw_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:22:05.961 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking radosgw (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:22:06.004 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package python3-simplejson. 2024-09-10T21:22:06.015 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../121-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-09-10T21:22:06.054 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-09-10T21:22:06.324 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-09-10T21:22:06.334 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../122-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-09-10T21:22:06.373 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-09-10T21:22:06.647 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package rbd-fuse. 2024-09-10T21:22:06.663 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../124-rbd-fuse_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:22:06.705 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking rbd-fuse (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:22:07.032 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-09-10T21:22:07.060 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package radosgw. 2024-09-10T21:22:07.070 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../123-radosgw_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:22:07.110 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking radosgw (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:22:07.310 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-10T21:22:07.436 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up nvme-cli (1.16-3ubuntu0.2) ... 2024-09-10T21:22:07.732 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-09-10T21:22:07.847 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package rbd-fuse. 2024-09-10T21:22:07.858 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../124-rbd-fuse_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-10T21:22:07.888 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking rbd-fuse (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:22:08.130 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-09-10T21:22:08.188 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-09-10T21:22:08.510 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-10T21:22:08.636 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up nvme-cli (1.16-3ubuntu0.2) ... 2024-09-10T21:22:08.791 INFO:teuthology.orchestra.run.smithi031.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-09-10T21:22:08.834 INFO:teuthology.orchestra.run.smithi031.stdout:Could not execute systemctl: at /usr/bin/deb-systemd-invoke line 142. 2024-09-10T21:22:08.871 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up cephadm (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:22:08.905 INFO:teuthology.orchestra.run.smithi017.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-09-10T21:22:09.327 INFO:teuthology.orchestra.run.smithi017.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-09-10T21:22:09.715 INFO:teuthology.orchestra.run.smithi031.stdout:Adding system user cephadm....done 2024-09-10T21:22:09.886 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-10T21:22:10.007 INFO:teuthology.orchestra.run.smithi017.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-09-10T21:22:10.022 INFO:teuthology.orchestra.run.smithi017.stdout:Could not execute systemctl: at /usr/bin/deb-systemd-invoke line 142. 2024-09-10T21:22:10.072 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up cephadm (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:22:10.179 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-09-10T21:22:10.473 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-09-10T21:22:10.565 INFO:teuthology.orchestra.run.smithi017.stdout:Adding system user cephadm....done 2024-09-10T21:22:10.581 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-09-10T21:22:10.832 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-09-10T21:22:10.987 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-10T21:22:11.108 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-09-10T21:22:11.234 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-py (1.10.0-1) ... 2024-09-10T21:22:11.341 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-09-10T21:22:11.552 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-09-10T21:22:11.645 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-09-10T21:22:11.866 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-09-10T21:22:11.946 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-09-10T21:22:12.208 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-09-10T21:22:12.213 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-colorama (0.4.4-1) ... 2024-09-10T21:22:12.475 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-09-10T21:22:12.481 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-09-10T21:22:12.602 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-py (1.10.0-1) ... 2024-09-10T21:22:12.783 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-09-10T21:22:12.945 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-09-10T21:22:13.134 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-09-10T21:22:13.347 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-09-10T21:22:13.401 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-09-10T21:22:13.623 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-colorama (0.4.4-1) ... 2024-09-10T21:22:13.653 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-09-10T21:22:13.882 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-09-10T21:22:13.929 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-ceph-argparse (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:22:14.188 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-10T21:22:14.191 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-09-10T21:22:14.322 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up libnbd0 (1.10.5-1) ... 2024-09-10T21:22:14.443 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-09-10T21:22:14.499 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-10T21:22:14.625 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-pyperclip (1.8.2-2) ... 2024-09-10T21:22:14.711 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-09-10T21:22:14.909 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-09-10T21:22:15.020 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-09-10T21:22:15.066 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-09-10T21:22:15.280 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-ceph-argparse (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:22:15.336 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-09-10T21:22:15.589 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-10T21:22:15.707 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up libnbd0 (1.10.5-1) ... 2024-09-10T21:22:15.724 INFO:teuthology.orchestra.run.smithi031.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-09-10T21:22:15.775 INFO:teuthology.orchestra.run.smithi031.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-09-10T21:22:15.807 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-10T21:22:15.839 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-10T21:22:15.933 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-pyperclip (1.8.2-2) ... 2024-09-10T21:22:15.940 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-09-10T21:22:16.201 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-09-10T21:22:16.325 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-10T21:22:16.327 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-09-10T21:22:16.451 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-09-10T21:22:16.586 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-09-10T21:22:16.710 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-09-10T21:22:17.003 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-09-10T21:22:17.146 INFO:teuthology.orchestra.run.smithi017.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-09-10T21:22:17.201 INFO:teuthology.orchestra.run.smithi017.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-09-10T21:22:17.257 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-10T21:22:17.304 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up zip (3.0-12build2) ... 2024-09-10T21:22:17.358 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-09-10T21:22:17.455 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-09-10T21:22:17.734 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-10T21:22:17.859 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-09-10T21:22:18.144 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-09-10T21:22:18.157 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-09-10T21:22:18.433 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-09-10T21:22:18.445 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-09-10T21:22:18.576 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-09-10T21:22:18.677 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-10T21:22:18.747 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up zip (3.0-12build2) ... 2024-09-10T21:22:18.897 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-09-10T21:22:18.986 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-09-10T21:22:19.429 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-09-10T21:22:19.624 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-09-10T21:22:19.705 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-09-10T21:22:19.884 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-09-10T21:22:20.035 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-09-10T21:22:20.090 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-10T21:22:20.160 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-10T21:22:20.400 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-cmd2 (2.3.3-1) ... 2024-09-10T21:22:20.520 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-09-10T21:22:20.735 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-10T21:22:20.929 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-09-10T21:22:21.111 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-09-10T21:22:21.197 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-09-10T21:22:21.403 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-09-10T21:22:21.529 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-ceph-common (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:22:21.590 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-10T21:22:21.848 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-idna (3.3-1ubuntu0.1) ... 2024-09-10T21:22:21.916 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-cmd2 (2.3.3-1) ... 2024-09-10T21:22:22.182 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-10T21:22:22.268 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-10T21:22:22.628 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-09-10T21:22:22.887 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-09-10T21:22:22.988 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-ceph-common (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:22:23.297 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-idna (3.3-1ubuntu0.1) ... 2024-09-10T21:22:23.471 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-09-10T21:22:23.682 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-10T21:22:23.739 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-10T21:22:23.874 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-toml (0.10.2-1) ... 2024-09-10T21:22:24.158 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-10T21:22:24.301 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-09-10T21:22:24.402 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-10T21:22:24.687 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-09-10T21:22:24.945 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-09-10T21:22:24.954 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-09-10T21:22:25.189 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-10T21:22:25.205 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-10T21:22:25.306 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-toml (0.10.2-1) ... 2024-09-10T21:22:25.331 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-09-10T21:22:25.557 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-10T21:22:25.658 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-09-10T21:22:25.675 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-09-10T21:22:25.818 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-10T21:22:25.942 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-09-10T21:22:26.144 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-09-10T21:22:26.235 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-09-10T21:22:26.420 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-09-10T21:22:26.503 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-09-10T21:22:26.680 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-10T21:22:26.787 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-09-10T21:22:26.806 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-09-10T21:22:27.072 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-10T21:22:27.132 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-09-10T21:22:27.206 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-09-10T21:22:27.430 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-09-10T21:22:27.507 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-10T21:22:27.633 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-10T21:22:27.701 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-09-10T21:22:27.968 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-10T21:22:27.999 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-09-10T21:22:28.253 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-09-10T21:22:28.277 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-09-10T21:22:28.529 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-10T21:22:28.553 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-10T21:22:28.654 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-09-10T21:22:28.696 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-09-10T21:22:28.764 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-09-10T21:22:28.836 INFO:teuthology.orchestra.run.smithi031.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-09-10T21:22:28.998 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-10T21:22:29.115 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-10T21:22:29.146 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-10T21:22:29.241 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-09-10T21:22:29.474 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-10T21:22:29.792 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-09-10T21:22:29.851 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-09-10T21:22:30.068 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-10T21:22:30.119 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up lua-any (27ubuntu1) ... 2024-09-10T21:22:30.186 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-09-10T21:22:30.228 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-portend (3.0.0-1) ... 2024-09-10T21:22:30.320 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-09-10T21:22:30.392 INFO:teuthology.orchestra.run.smithi017.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-09-10T21:22:30.496 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-10T21:22:30.639 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-10T21:22:30.647 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-10T21:22:30.765 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-09-10T21:22:30.973 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-09-10T21:22:31.216 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-09-10T21:22:31.291 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-10T21:22:31.500 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up lua-any (27ubuntu1) ... 2024-09-10T21:22:31.601 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-portend (3.0.0-1) ... 2024-09-10T21:22:31.661 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-09-10T21:22:31.795 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-rsa (4.8-1) ... 2024-09-10T21:22:31.861 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-10T21:22:31.962 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-10T21:22:32.063 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-09-10T21:22:32.296 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-09-10T21:22:32.347 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-10T21:22:32.615 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-09-10T21:22:32.639 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-10T21:22:33.017 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-09-10T21:22:33.017 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-09-10T21:22:33.118 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-rsa (4.8-1) ... 2024-09-10T21:22:33.318 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-09-10T21:22:33.411 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-09-10T21:22:33.662 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-10T21:22:33.671 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-10T21:22:33.771 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up librados2 (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:22:33.881 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up libsqlite3-mod-ceph (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:22:33.938 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-09-10T21:22:34.007 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-09-10T21:22:34.365 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-09-10T21:22:34.368 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up libcephfs2 (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:22:34.494 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up libradosstriper1 (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:22:34.651 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-09-10T21:22:34.666 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-09-10T21:22:34.946 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up librbd1 (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:22:35.010 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-10T21:22:35.072 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up ceph-mgr-modules-core (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:22:35.128 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up librados2 (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:22:35.230 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up ceph-fuse (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:22:35.254 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up libsqlite3-mod-ceph (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:22:35.380 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-09-10T21:22:35.452 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-10T21:22:35.452 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-10T21:22:35.758 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up libcephfs2 (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:22:35.876 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up libradosstriper1 (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:22:35.977 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-09-10T21:22:36.099 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up libcephfs-dev (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:22:36.217 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-rados (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:22:36.295 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up librbd1 (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:22:36.318 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up librgw2 (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:22:36.438 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up ceph-mgr-modules-core (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:22:36.444 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up cephfs-top (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:22:36.545 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-10T21:22:36.566 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up ceph-fuse (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:22:36.821 INFO:teuthology.orchestra.run.smithi017.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-10T21:22:36.821 INFO:teuthology.orchestra.run.smithi017.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-10T21:22:37.475 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up libcephfs-dev (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:22:37.584 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-rados (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:22:37.693 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up librgw2 (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:22:37.819 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up cephfs-top (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:22:37.882 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-rbd (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:22:37.945 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-10T21:22:38.008 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up rbd-fuse (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:22:38.159 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-rgw (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:22:38.285 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up python3-cephfs (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:22:38.411 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up cephfs-shell (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:22:38.512 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up ceph-common (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:22:39.032 INFO:teuthology.orchestra.run.smithi031.stdout:Adding group ceph....done 2024-09-10T21:22:39.291 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-rbd (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:22:39.408 INFO:teuthology.orchestra.run.smithi031.stdout:Adding system user ceph....done 2024-09-10T21:22:39.416 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up rbd-fuse (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:22:39.552 INFO:teuthology.orchestra.run.smithi031.stdout:Setting system user ceph properties....done 2024-09-10T21:22:39.567 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-rgw (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:22:39.570 INFO:teuthology.orchestra.run.smithi031.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-09-10T21:22:39.693 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up python3-cephfs (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:22:39.723 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-09-10T21:22:39.802 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up cephfs-shell (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:22:39.912 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up ceph-common (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:22:40.129 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-09-10T21:22:40.465 INFO:teuthology.orchestra.run.smithi017.stdout:Adding group ceph....done 2024-09-10T21:22:40.813 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up cephfs-mirror (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:22:40.882 INFO:teuthology.orchestra.run.smithi017.stdout:Adding system user ceph....done 2024-09-10T21:22:41.028 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-09-10T21:22:41.028 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-09-10T21:22:41.052 INFO:teuthology.orchestra.run.smithi017.stdout:Setting system user ceph properties....done 2024-09-10T21:22:41.071 INFO:teuthology.orchestra.run.smithi017.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-09-10T21:22:41.224 INFO:teuthology.orchestra.run.smithi017.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-09-10T21:22:41.648 INFO:teuthology.orchestra.run.smithi017.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-09-10T21:22:41.681 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up ceph-test (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:22:41.807 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up radosgw (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:22:42.338 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-10T21:22:42.338 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-10T21:22:42.639 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up cephfs-mirror (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:22:42.861 INFO:teuthology.orchestra.run.smithi017.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-09-10T21:22:42.861 INFO:teuthology.orchestra.run.smithi017.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2024-09-10T21:22:42.975 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up ceph-base (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:22:43.344 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-09-10T21:22:43.523 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up ceph-test (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:22:43.674 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up radosgw (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:22:44.044 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up ceph-mds (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:22:44.224 INFO:teuthology.orchestra.run.smithi017.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-10T21:22:44.224 INFO:teuthology.orchestra.run.smithi017.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-10T21:22:44.256 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-10T21:22:44.256 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-10T21:22:44.848 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up ceph-mgr (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:22:44.909 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up ceph-base (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:22:45.083 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-10T21:22:45.083 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-10T21:22:45.281 INFO:teuthology.orchestra.run.smithi017.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-09-10T21:22:45.756 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up ceph-osd (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:22:46.002 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up ceph-mds (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:22:46.134 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-10T21:22:46.135 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-10T21:22:46.241 INFO:teuthology.orchestra.run.smithi017.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-10T21:22:46.241 INFO:teuthology.orchestra.run.smithi017.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-10T21:22:46.783 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up ceph-mgr-k8sevents (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:22:46.883 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up ceph-mgr-diskprediction-local (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:22:46.885 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up ceph-mgr (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:22:47.018 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up ceph-mon (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:22:47.101 INFO:teuthology.orchestra.run.smithi017.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-10T21:22:47.101 INFO:teuthology.orchestra.run.smithi017.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-10T21:22:47.217 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-10T21:22:47.217 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-10T21:22:47.789 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up ceph-osd (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:22:47.852 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up ceph-mgr-cephadm (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:22:48.020 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up ceph (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:22:48.117 INFO:teuthology.orchestra.run.smithi017.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-10T21:22:48.117 INFO:teuthology.orchestra.run.smithi017.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-10T21:22:48.155 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up ceph-mgr-dashboard (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:22:48.306 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up ceph-volume (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:22:48.683 INFO:teuthology.orchestra.run.smithi031.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-09-10T21:22:48.790 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up ceph-mgr-k8sevents (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:22:48.792 INFO:teuthology.orchestra.run.smithi031.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-09-10T21:22:48.941 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up ceph-mgr-diskprediction-local (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:22:48.944 INFO:teuthology.orchestra.run.smithi031.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-09-10T21:22:49.059 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up ceph-mon (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:22:49.261 INFO:teuthology.orchestra.run.smithi017.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-10T21:22:49.262 INFO:teuthology.orchestra.run.smithi017.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-10T21:22:49.893 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up ceph-mgr-cephadm (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:22:50.045 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up ceph (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:22:50.120 INFO:teuthology.orchestra.run.smithi031.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-09-10T21:22:50.145 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up ceph-mgr-dashboard (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:22:50.279 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up ceph-volume (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-10T21:22:50.658 INFO:teuthology.orchestra.run.smithi017.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-09-10T21:22:50.758 INFO:teuthology.orchestra.run.smithi017.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-09-10T21:22:50.850 INFO:teuthology.orchestra.run.smithi031.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-10T21:22:50.855 DEBUG:teuthology.orchestra.run.smithi031:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2024-09-10T21:22:50.910 INFO:teuthology.orchestra.run.smithi017.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-09-10T21:22:50.934 INFO:teuthology.orchestra.run.smithi031.stdout:Reading package lists... 2024-09-10T21:22:51.121 INFO:teuthology.orchestra.run.smithi031.stdout:Building dependency tree... 2024-09-10T21:22:51.122 INFO:teuthology.orchestra.run.smithi031.stdout:Reading state information... 2024-09-10T21:22:51.315 INFO:teuthology.orchestra.run.smithi031.stdout:iproute2 is already the newest version (5.15.0-1ubuntu2). 2024-09-10T21:22:51.315 INFO:teuthology.orchestra.run.smithi031.stdout:libtool-bin is already the newest version (2.4.6-15build2). 2024-09-10T21:22:51.315 INFO:teuthology.orchestra.run.smithi031.stdout:xfslibs-dev is already the newest version (5.13.0-1ubuntu2). 2024-09-10T21:22:51.315 INFO:teuthology.orchestra.run.smithi031.stdout:The following packages were automatically installed and are no longer required: 2024-09-10T21:22:51.315 INFO:teuthology.orchestra.run.smithi031.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-10T21:22:51.315 INFO:teuthology.orchestra.run.smithi031.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-10T21:22:51.315 INFO:teuthology.orchestra.run.smithi031.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-10T21:22:51.316 INFO:teuthology.orchestra.run.smithi031.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-10T21:22:51.316 INFO:teuthology.orchestra.run.smithi031.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-10T21:22:51.316 INFO:teuthology.orchestra.run.smithi031.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-10T21:22:51.317 INFO:teuthology.orchestra.run.smithi031.stdout:The following additional packages will be installed: 2024-09-10T21:22:51.317 INFO:teuthology.orchestra.run.smithi031.stdout: libbluetooth3 libfl-dev libfl2 libjson-perl libllvm14 liblzo2-2 2024-09-10T21:22:51.317 INFO:teuthology.orchestra.run.smithi031.stdout: libmbim-glib4 libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 2024-09-10T21:22:51.317 INFO:teuthology.orchestra.run.smithi031.stdout: libqmi-proxy libssl3 libteamdctl0 libuuid1 modemmanager network-manager-pptp 2024-09-10T21:22:51.317 INFO:teuthology.orchestra.run.smithi031.stdout: postgresql-14 postgresql-client-14 postgresql-client-common ppp pptp-linux 2024-09-10T21:22:51.318 INFO:teuthology.orchestra.run.smithi031.stdout: tcl tcl8.6 usb-modeswitch usb-modeswitch-data wpasupplicant 2024-09-10T21:22:51.319 INFO:teuthology.orchestra.run.smithi031.stdout:Suggested packages: 2024-09-10T21:22:51.319 INFO:teuthology.orchestra.run.smithi031.stdout: bison-doc flex-doc indent-doc libssl-doc avahi-autoipd libteam-utils 2024-09-10T21:22:51.319 INFO:teuthology.orchestra.run.smithi031.stdout: postgresql-doc postgresql-doc-14 tcl-tclreadline comgt wvdial 2024-09-10T21:22:51.319 INFO:teuthology.orchestra.run.smithi031.stdout: util-linux-locales wpagui libengine-pkcs11-openssl 2024-09-10T21:22:51.377 INFO:teuthology.orchestra.run.smithi031.stdout:The following NEW packages will be installed: 2024-09-10T21:22:51.377 INFO:teuthology.orchestra.run.smithi031.stdout: bison dump flex indent libaio-dev libbluetooth3 libelf-dev libfl-dev libfl2 2024-09-10T21:22:51.377 INFO:teuthology.orchestra.run.smithi031.stdout: libjson-perl libllvm14 liblzo2-2 libmbim-glib4 libmbim-proxy libmm-glib0 2024-09-10T21:22:51.378 INFO:teuthology.orchestra.run.smithi031.stdout: libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl-dev libteamdctl0 2024-09-10T21:22:51.378 INFO:teuthology.orchestra.run.smithi031.stdout: modemmanager network-manager network-manager-pptp postgresql postgresql-14 2024-09-10T21:22:51.378 INFO:teuthology.orchestra.run.smithi031.stdout: postgresql-client postgresql-client-14 postgresql-client-common 2024-09-10T21:22:51.378 INFO:teuthology.orchestra.run.smithi031.stdout: postgresql-common postgresql-contrib ppp pptp-linux tcl tcl8.6 2024-09-10T21:22:51.378 INFO:teuthology.orchestra.run.smithi031.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2024-09-10T21:22:51.379 INFO:teuthology.orchestra.run.smithi031.stdout:The following packages will be upgraded: 2024-09-10T21:22:51.380 INFO:teuthology.orchestra.run.smithi031.stdout: libssl3 libuuid1 util-linux uuid-dev 2024-09-10T21:22:51.436 INFO:teuthology.orchestra.run.smithi031.stdout:4 upgraded, 39 newly installed, 0 to remove and 330 not upgraded. 2024-09-10T21:22:51.436 INFO:teuthology.orchestra.run.smithi031.stdout:Need to get 55.8 MB of archives. 2024-09-10T21:22:51.436 INFO:teuthology.orchestra.run.smithi031.stdout:After this operation, 203 MB of additional disk space will be used. 2024-09-10T21:22:51.437 INFO:teuthology.orchestra.run.smithi031.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 util-linux amd64 2.37.2-4ubuntu3.4 [1,063 kB] 2024-09-10T21:22:51.569 INFO:teuthology.orchestra.run.smithi031.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 flex amd64 2.6.4-8build2 [307 kB] 2024-09-10T21:22:51.573 INFO:teuthology.orchestra.run.smithi031.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl3 amd64 3.0.2-0ubuntu1.18 [1,905 kB] 2024-09-10T21:22:51.600 INFO:teuthology.orchestra.run.smithi031.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 uuid-dev amd64 2.37.2-4ubuntu3.4 [33.1 kB] 2024-09-10T21:22:51.600 INFO:teuthology.orchestra.run.smithi031.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libuuid1 amd64 2.37.2-4ubuntu3.4 [23.8 kB] 2024-09-10T21:22:51.601 INFO:teuthology.orchestra.run.smithi031.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/main amd64 bison amd64 2:3.8.2+dfsg-1build1 [748 kB] 2024-09-10T21:22:51.608 INFO:teuthology.orchestra.run.smithi031.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblzo2-2 amd64 2.10-2build3 [53.7 kB] 2024-09-10T21:22:51.609 INFO:teuthology.orchestra.run.smithi031.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 dump amd64 0.4b47-3 [133 kB] 2024-09-10T21:22:51.610 INFO:teuthology.orchestra.run.smithi031.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 indent amd64 2.2.12-1ubuntu0.22.04.1 [90.8 kB] 2024-09-10T21:22:51.611 INFO:teuthology.orchestra.run.smithi031.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 libaio-dev amd64 0.3.112-13build1 [21.2 kB] 2024-09-10T21:22:51.612 INFO:teuthology.orchestra.run.smithi031.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libbluetooth3 amd64 5.64-0ubuntu1.3 [86.9 kB] 2024-09-10T21:22:51.613 INFO:teuthology.orchestra.run.smithi031.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libelf-dev amd64 0.186-1build1 [64.4 kB] 2024-09-10T21:22:51.622 INFO:teuthology.orchestra.run.smithi031.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl2 amd64 2.6.4-8build2 [10.7 kB] 2024-09-10T21:22:51.623 INFO:teuthology.orchestra.run.smithi031.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl-dev amd64 2.6.4-8build2 [6,236 B] 2024-09-10T21:22:51.623 INFO:teuthology.orchestra.run.smithi031.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjson-perl all 4.04000-1 [81.8 kB] 2024-09-10T21:22:51.639 INFO:teuthology.orchestra.run.smithi031.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libllvm14 amd64 1:14.0.0-1ubuntu1.1 [24.0 MB] 2024-09-10T21:22:51.853 INFO:teuthology.orchestra.run.smithi031.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-glib4 amd64 1.28.0-1~ubuntu20.04.1 [191 kB] 2024-09-10T21:22:51.855 INFO:teuthology.orchestra.run.smithi031.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-proxy amd64 1.28.0-1~ubuntu20.04.1 [6,130 B] 2024-09-10T21:22:51.855 INFO:teuthology.orchestra.run.smithi031.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmm-glib0 amd64 1.20.0-1~ubuntu22.04.3 [263 kB] 2024-09-10T21:22:51.858 INFO:teuthology.orchestra.run.smithi031.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libndp0 amd64 1.8-0ubuntu3.1 [12.4 kB] 2024-09-10T21:22:51.858 INFO:teuthology.orchestra.run.smithi031.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libnm0 amd64 1.36.6-0ubuntu2 [456 kB] 2024-09-10T21:22:51.862 INFO:teuthology.orchestra.run.smithi031.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpq5 amd64 14.13-0ubuntu0.22.04.1 [149 kB] 2024-09-10T21:22:51.864 INFO:teuthology.orchestra.run.smithi031.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-glib5 amd64 1.32.0-1ubuntu0.22.04.1 [772 kB] 2024-09-10T21:22:51.871 INFO:teuthology.orchestra.run.smithi031.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-proxy amd64 1.32.0-1ubuntu0.22.04.1 [6,072 B] 2024-09-10T21:22:51.872 INFO:teuthology.orchestra.run.smithi031.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl-dev amd64 3.0.2-0ubuntu1.18 [2,375 kB] 2024-09-10T21:22:51.893 INFO:teuthology.orchestra.run.smithi031.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 libteamdctl0 amd64 1.31-1build2 [12.6 kB] 2024-09-10T21:22:51.894 INFO:teuthology.orchestra.run.smithi031.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 modemmanager amd64 1.20.0-1~ubuntu22.04.3 [1,094 kB] 2024-09-10T21:22:51.903 INFO:teuthology.orchestra.run.smithi031.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 network-manager amd64 1.36.6-0ubuntu2 [2,203 kB] 2024-09-10T21:22:51.924 INFO:teuthology.orchestra.run.smithi031.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 ppp amd64 2.4.9-1+1ubuntu3 [363 kB] 2024-09-10T21:22:51.927 INFO:teuthology.orchestra.run.smithi031.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 pptp-linux amd64 1.10.0-1build3 [42.5 kB] 2024-09-10T21:22:51.927 INFO:teuthology.orchestra.run.smithi031.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 network-manager-pptp amd64 1.2.10-1 [30.9 kB] 2024-09-10T21:22:51.928 INFO:teuthology.orchestra.run.smithi031.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client-common all 238 [29.6 kB] 2024-09-10T21:22:51.928 INFO:teuthology.orchestra.run.smithi031.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-client-14 amd64 14.13-0ubuntu0.22.04.1 [1,225 kB] 2024-09-10T21:22:51.940 INFO:teuthology.orchestra.run.smithi031.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-common all 238 [169 kB] 2024-09-10T21:22:51.941 INFO:teuthology.orchestra.run.smithi031.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-14 amd64 14.13-0ubuntu0.22.04.1 [16.2 MB] 2024-09-10T21:22:51.952 INFO:teuthology.orchestra.run.smithi017.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-09-10T21:22:52.086 INFO:teuthology.orchestra.run.smithi031.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql all 14+238 [3,288 B] 2024-09-10T21:22:52.086 INFO:teuthology.orchestra.run.smithi031.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client all 14+238 [3,292 B] 2024-09-10T21:22:52.086 INFO:teuthology.orchestra.run.smithi031.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-contrib all 14+238 [3,292 B] 2024-09-10T21:22:52.086 INFO:teuthology.orchestra.run.smithi031.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl8.6 amd64 8.6.12+dfsg-1build1 [15.0 kB] 2024-09-10T21:22:52.086 INFO:teuthology.orchestra.run.smithi031.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl amd64 8.6.11+1build2 [4,678 B] 2024-09-10T21:22:52.087 INFO:teuthology.orchestra.run.smithi031.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 wpasupplicant amd64 2:2.10-6ubuntu2.1 [1,482 kB] 2024-09-10T21:22:52.100 INFO:teuthology.orchestra.run.smithi031.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch-data all 20191128-4 [33.2 kB] 2024-09-10T21:22:52.100 INFO:teuthology.orchestra.run.smithi031.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch amd64 2.6.1-3ubuntu2 [46.0 kB] 2024-09-10T21:22:52.509 INFO:teuthology.orchestra.run.smithi031.stdout:Preconfiguring packages ... 2024-09-10T21:22:52.687 INFO:teuthology.orchestra.run.smithi031.stdout:Fetched 55.8 MB in 1s (78.3 MB/s) 2024-09-10T21:22:52.776 INFO:teuthology.orchestra.run.smithi017.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-10T21:22:52.781 DEBUG:teuthology.orchestra.run.smithi017:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2024-09-10T21:22:52.859 INFO:teuthology.orchestra.run.smithi017.stdout:Reading package lists... 2024-09-10T21:22:52.925 INFO:teuthology.orchestra.run.smithi031.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 143286 files and directories currently installed.) 2024-09-10T21:22:52.928 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../util-linux_2.37.2-4ubuntu3.4_amd64.deb ... 2024-09-10T21:22:53.046 INFO:teuthology.orchestra.run.smithi017.stdout:Building dependency tree... 2024-09-10T21:22:53.047 INFO:teuthology.orchestra.run.smithi017.stdout:Reading state information... 2024-09-10T21:22:53.051 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking util-linux (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-09-10T21:22:53.238 INFO:teuthology.orchestra.run.smithi017.stdout:iproute2 is already the newest version (5.15.0-1ubuntu2). 2024-09-10T21:22:53.238 INFO:teuthology.orchestra.run.smithi017.stdout:libtool-bin is already the newest version (2.4.6-15build2). 2024-09-10T21:22:53.238 INFO:teuthology.orchestra.run.smithi017.stdout:xfslibs-dev is already the newest version (5.13.0-1ubuntu2). 2024-09-10T21:22:53.238 INFO:teuthology.orchestra.run.smithi017.stdout:The following packages were automatically installed and are no longer required: 2024-09-10T21:22:53.238 INFO:teuthology.orchestra.run.smithi017.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-10T21:22:53.238 INFO:teuthology.orchestra.run.smithi017.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-10T21:22:53.238 INFO:teuthology.orchestra.run.smithi017.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-10T21:22:53.239 INFO:teuthology.orchestra.run.smithi017.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-10T21:22:53.239 INFO:teuthology.orchestra.run.smithi017.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-10T21:22:53.239 INFO:teuthology.orchestra.run.smithi017.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-10T21:22:53.240 INFO:teuthology.orchestra.run.smithi017.stdout:The following additional packages will be installed: 2024-09-10T21:22:53.240 INFO:teuthology.orchestra.run.smithi017.stdout: libbluetooth3 libfl-dev libfl2 libjson-perl libllvm14 liblzo2-2 2024-09-10T21:22:53.240 INFO:teuthology.orchestra.run.smithi017.stdout: libmbim-glib4 libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 2024-09-10T21:22:53.240 INFO:teuthology.orchestra.run.smithi017.stdout: libqmi-proxy libssl3 libteamdctl0 libuuid1 modemmanager network-manager-pptp 2024-09-10T21:22:53.241 INFO:teuthology.orchestra.run.smithi017.stdout: postgresql-14 postgresql-client-14 postgresql-client-common ppp pptp-linux 2024-09-10T21:22:53.241 INFO:teuthology.orchestra.run.smithi017.stdout: tcl tcl8.6 usb-modeswitch usb-modeswitch-data wpasupplicant 2024-09-10T21:22:53.242 INFO:teuthology.orchestra.run.smithi017.stdout:Suggested packages: 2024-09-10T21:22:53.242 INFO:teuthology.orchestra.run.smithi017.stdout: bison-doc flex-doc indent-doc libssl-doc avahi-autoipd libteam-utils 2024-09-10T21:22:53.242 INFO:teuthology.orchestra.run.smithi017.stdout: postgresql-doc postgresql-doc-14 tcl-tclreadline comgt wvdial 2024-09-10T21:22:53.242 INFO:teuthology.orchestra.run.smithi017.stdout: util-linux-locales wpagui libengine-pkcs11-openssl 2024-09-10T21:22:53.301 INFO:teuthology.orchestra.run.smithi017.stdout:The following NEW packages will be installed: 2024-09-10T21:22:53.301 INFO:teuthology.orchestra.run.smithi017.stdout: bison dump flex indent libaio-dev libbluetooth3 libelf-dev libfl-dev libfl2 2024-09-10T21:22:53.301 INFO:teuthology.orchestra.run.smithi017.stdout: libjson-perl libllvm14 liblzo2-2 libmbim-glib4 libmbim-proxy libmm-glib0 2024-09-10T21:22:53.302 INFO:teuthology.orchestra.run.smithi017.stdout: libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl-dev libteamdctl0 2024-09-10T21:22:53.302 INFO:teuthology.orchestra.run.smithi017.stdout: modemmanager network-manager network-manager-pptp postgresql postgresql-14 2024-09-10T21:22:53.302 INFO:teuthology.orchestra.run.smithi017.stdout: postgresql-client postgresql-client-14 postgresql-client-common 2024-09-10T21:22:53.302 INFO:teuthology.orchestra.run.smithi017.stdout: postgresql-common postgresql-contrib ppp pptp-linux tcl tcl8.6 2024-09-10T21:22:53.302 INFO:teuthology.orchestra.run.smithi017.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2024-09-10T21:22:53.304 INFO:teuthology.orchestra.run.smithi017.stdout:The following packages will be upgraded: 2024-09-10T21:22:53.304 INFO:teuthology.orchestra.run.smithi017.stdout: libssl3 libuuid1 util-linux uuid-dev 2024-09-10T21:22:53.490 INFO:teuthology.orchestra.run.smithi017.stdout:4 upgraded, 39 newly installed, 0 to remove and 330 not upgraded. 2024-09-10T21:22:53.490 INFO:teuthology.orchestra.run.smithi017.stdout:Need to get 55.8 MB of archives. 2024-09-10T21:22:53.490 INFO:teuthology.orchestra.run.smithi017.stdout:After this operation, 203 MB of additional disk space will be used. 2024-09-10T21:22:53.491 INFO:teuthology.orchestra.run.smithi017.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 util-linux amd64 2.37.2-4ubuntu3.4 [1,063 kB] 2024-09-10T21:22:53.719 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up util-linux (2.37.2-4ubuntu3.4) ... 2024-09-10T21:22:54.109 INFO:teuthology.orchestra.run.smithi017.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 flex amd64 2.6.4-8build2 [307 kB] 2024-09-10T21:22:54.125 INFO:teuthology.orchestra.run.smithi017.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl3 amd64 3.0.2-0ubuntu1.18 [1,905 kB] 2024-09-10T21:22:54.249 INFO:teuthology.orchestra.run.smithi017.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 uuid-dev amd64 2.37.2-4ubuntu3.4 [33.1 kB] 2024-09-10T21:22:54.250 INFO:teuthology.orchestra.run.smithi017.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libuuid1 amd64 2.37.2-4ubuntu3.4 [23.8 kB] 2024-09-10T21:22:54.251 INFO:teuthology.orchestra.run.smithi017.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/main amd64 bison amd64 2:3.8.2+dfsg-1build1 [748 kB] 2024-09-10T21:22:54.279 INFO:teuthology.orchestra.run.smithi017.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblzo2-2 amd64 2.10-2build3 [53.7 kB] 2024-09-10T21:22:54.281 INFO:teuthology.orchestra.run.smithi017.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 dump amd64 0.4b47-3 [133 kB] 2024-09-10T21:22:54.284 INFO:teuthology.orchestra.run.smithi017.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 indent amd64 2.2.12-1ubuntu0.22.04.1 [90.8 kB] 2024-09-10T21:22:54.286 INFO:teuthology.orchestra.run.smithi017.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 libaio-dev amd64 0.3.112-13build1 [21.2 kB] 2024-09-10T21:22:54.287 INFO:teuthology.orchestra.run.smithi017.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libbluetooth3 amd64 5.64-0ubuntu1.3 [86.9 kB] 2024-09-10T21:22:54.289 INFO:teuthology.orchestra.run.smithi017.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libelf-dev amd64 0.186-1build1 [64.4 kB] 2024-09-10T21:22:54.353 INFO:teuthology.orchestra.run.smithi017.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl2 amd64 2.6.4-8build2 [10.7 kB] 2024-09-10T21:22:54.353 INFO:teuthology.orchestra.run.smithi017.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl-dev amd64 2.6.4-8build2 [6,236 B] 2024-09-10T21:22:54.353 INFO:teuthology.orchestra.run.smithi017.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjson-perl all 4.04000-1 [81.8 kB] 2024-09-10T21:22:54.434 INFO:teuthology.orchestra.run.smithi017.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libllvm14 amd64 1:14.0.0-1ubuntu1.1 [24.0 MB] 2024-09-10T21:22:54.819 INFO:teuthology.orchestra.run.smithi031.stdout:fstrim.service is a disabled or a static unit not running, not starting it. 2024-09-10T21:22:55.097 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package flex. 2024-09-10T21:22:55.117 INFO:teuthology.orchestra.run.smithi017.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-glib4 amd64 1.28.0-1~ubuntu20.04.1 [191 kB] 2024-09-10T21:22:55.119 INFO:teuthology.orchestra.run.smithi017.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-proxy amd64 1.28.0-1~ubuntu20.04.1 [6,130 B] 2024-09-10T21:22:55.119 INFO:teuthology.orchestra.run.smithi017.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmm-glib0 amd64 1.20.0-1~ubuntu22.04.3 [263 kB] 2024-09-10T21:22:55.122 INFO:teuthology.orchestra.run.smithi017.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libndp0 amd64 1.8-0ubuntu3.1 [12.4 kB] 2024-09-10T21:22:55.122 INFO:teuthology.orchestra.run.smithi017.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libnm0 amd64 1.36.6-0ubuntu2 [456 kB] 2024-09-10T21:22:55.127 INFO:teuthology.orchestra.run.smithi017.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpq5 amd64 14.13-0ubuntu0.22.04.1 [149 kB] 2024-09-10T21:22:55.129 INFO:teuthology.orchestra.run.smithi017.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-glib5 amd64 1.32.0-1ubuntu0.22.04.1 [772 kB] 2024-09-10T21:22:55.170 INFO:teuthology.orchestra.run.smithi031.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 143286 files and directories currently installed.) 2024-09-10T21:22:55.174 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../flex_2.6.4-8build2_amd64.deb ... 2024-09-10T21:22:55.181 INFO:teuthology.orchestra.run.smithi017.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-proxy amd64 1.32.0-1ubuntu0.22.04.1 [6,072 B] 2024-09-10T21:22:55.181 INFO:teuthology.orchestra.run.smithi017.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl-dev amd64 3.0.2-0ubuntu1.18 [2,375 kB] 2024-09-10T21:22:55.212 INFO:teuthology.orchestra.run.smithi017.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 libteamdctl0 amd64 1.31-1build2 [12.6 kB] 2024-09-10T21:22:55.238 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking flex (2.6.4-8build2) ... 2024-09-10T21:22:55.265 INFO:teuthology.orchestra.run.smithi017.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 modemmanager amd64 1.20.0-1~ubuntu22.04.3 [1,094 kB] 2024-09-10T21:22:55.275 INFO:teuthology.orchestra.run.smithi017.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 network-manager amd64 1.36.6-0ubuntu2 [2,203 kB] 2024-09-10T21:22:55.352 INFO:teuthology.orchestra.run.smithi017.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 ppp amd64 2.4.9-1+1ubuntu3 [363 kB] 2024-09-10T21:22:55.355 INFO:teuthology.orchestra.run.smithi017.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 pptp-linux amd64 1.10.0-1build3 [42.5 kB] 2024-09-10T21:22:55.356 INFO:teuthology.orchestra.run.smithi017.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 network-manager-pptp amd64 1.2.10-1 [30.9 kB] 2024-09-10T21:22:55.357 INFO:teuthology.orchestra.run.smithi017.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client-common all 238 [29.6 kB] 2024-09-10T21:22:55.357 INFO:teuthology.orchestra.run.smithi017.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-client-14 amd64 14.13-0ubuntu0.22.04.1 [1,225 kB] 2024-09-10T21:22:55.369 INFO:teuthology.orchestra.run.smithi017.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-common all 238 [169 kB] 2024-09-10T21:22:55.371 INFO:teuthology.orchestra.run.smithi017.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-14 amd64 14.13-0ubuntu0.22.04.1 [16.2 MB] 2024-09-10T21:22:55.767 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../libssl3_3.0.2-0ubuntu1.18_amd64.deb ... 2024-09-10T21:22:55.864 INFO:teuthology.orchestra.run.smithi017.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql all 14+238 [3,288 B] 2024-09-10T21:22:55.864 INFO:teuthology.orchestra.run.smithi017.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client all 14+238 [3,292 B] 2024-09-10T21:22:55.864 INFO:teuthology.orchestra.run.smithi017.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-contrib all 14+238 [3,292 B] 2024-09-10T21:22:55.864 INFO:teuthology.orchestra.run.smithi017.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl8.6 amd64 8.6.12+dfsg-1build1 [15.0 kB] 2024-09-10T21:22:55.865 INFO:teuthology.orchestra.run.smithi017.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl amd64 8.6.11+1build2 [4,678 B] 2024-09-10T21:22:55.865 INFO:teuthology.orchestra.run.smithi017.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 wpasupplicant amd64 2:2.10-6ubuntu2.1 [1,482 kB] 2024-09-10T21:22:55.911 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking libssl3:amd64 (3.0.2-0ubuntu1.18) over (3.0.2-0ubuntu1.7) ... 2024-09-10T21:22:55.922 INFO:teuthology.orchestra.run.smithi017.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch-data all 20191128-4 [33.2 kB] 2024-09-10T21:22:55.923 INFO:teuthology.orchestra.run.smithi017.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch amd64 2.6.1-3ubuntu2 [46.0 kB] 2024-09-10T21:22:56.329 INFO:teuthology.orchestra.run.smithi017.stdout:Preconfiguring packages ... 2024-09-10T21:22:56.344 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up libssl3:amd64 (3.0.2-0ubuntu1.18) ... 2024-09-10T21:22:56.503 INFO:teuthology.orchestra.run.smithi017.stdout:Fetched 55.8 MB in 3s (21.4 MB/s) 2024-09-10T21:22:56.798 INFO:teuthology.orchestra.run.smithi017.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 143286 files and directories currently installed.) 2024-09-10T21:22:56.802 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../util-linux_2.37.2-4ubuntu3.4_amd64.deb ... 2024-09-10T21:22:56.908 INFO:teuthology.orchestra.run.smithi031.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 143359 files and directories currently installed.) 2024-09-10T21:22:56.910 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../uuid-dev_2.37.2-4ubuntu3.4_amd64.deb ... 2024-09-10T21:22:56.933 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking util-linux (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-09-10T21:22:57.035 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking uuid-dev:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-09-10T21:22:57.457 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../libuuid1_2.37.2-4ubuntu3.4_amd64.deb ... 2024-09-10T21:22:57.591 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking libuuid1:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-09-10T21:22:57.737 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up util-linux (2.37.2-4ubuntu3.4) ... 2024-09-10T21:22:57.934 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up libuuid1:amd64 (2.37.2-4ubuntu3.4) ... 2024-09-10T21:22:58.319 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package bison. 2024-09-10T21:22:58.398 INFO:teuthology.orchestra.run.smithi031.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 143359 files and directories currently installed.) 2024-09-10T21:22:58.402 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../00-bison_2%3a3.8.2+dfsg-1build1_amd64.deb ... 2024-09-10T21:22:58.446 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking bison (2:3.8.2+dfsg-1build1) ... 2024-09-10T21:22:58.865 INFO:teuthology.orchestra.run.smithi017.stdout:fstrim.service is a disabled or a static unit not running, not starting it. 2024-09-10T21:22:58.938 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package liblzo2-2:amd64. 2024-09-10T21:22:58.954 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../01-liblzo2-2_2.10-2build3_amd64.deb ... 2024-09-10T21:22:59.004 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking liblzo2-2:amd64 (2.10-2build3) ... 2024-09-10T21:22:59.090 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package flex. 2024-09-10T21:22:59.163 INFO:teuthology.orchestra.run.smithi017.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 143286 files and directories currently installed.) 2024-09-10T21:22:59.167 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../flex_2.6.4-8build2_amd64.deb ... 2024-09-10T21:22:59.205 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking flex (2.6.4-8build2) ... 2024-09-10T21:22:59.341 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package dump. 2024-09-10T21:22:59.357 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../02-dump_0.4b47-3_amd64.deb ... 2024-09-10T21:22:59.399 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking dump (0.4b47-3) ... 2024-09-10T21:22:59.742 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../libssl3_3.0.2-0ubuntu1.18_amd64.deb ... 2024-09-10T21:22:59.744 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package indent. 2024-09-10T21:22:59.761 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../03-indent_2.2.12-1ubuntu0.22.04.1_amd64.deb ... 2024-09-10T21:22:59.811 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking indent (2.2.12-1ubuntu0.22.04.1) ... 2024-09-10T21:22:59.944 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking libssl3:amd64 (3.0.2-0ubuntu1.18) over (3.0.2-0ubuntu1.7) ... 2024-09-10T21:23:00.172 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package libaio-dev:amd64. 2024-09-10T21:23:00.188 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../04-libaio-dev_0.3.112-13build1_amd64.deb ... 2024-09-10T21:23:00.344 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up libssl3:amd64 (3.0.2-0ubuntu1.18) ... 2024-09-10T21:23:00.813 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking libaio-dev:amd64 (0.3.112-13build1) ... 2024-09-10T21:23:00.883 INFO:teuthology.orchestra.run.smithi017.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 143359 files and directories currently installed.) 2024-09-10T21:23:00.887 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../uuid-dev_2.37.2-4ubuntu3.4_amd64.deb ... 2024-09-10T21:23:01.001 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking uuid-dev:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-09-10T21:23:01.234 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package libbluetooth3:amd64. 2024-09-10T21:23:01.251 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../05-libbluetooth3_5.64-0ubuntu1.3_amd64.deb ... 2024-09-10T21:23:01.292 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking libbluetooth3:amd64 (5.64-0ubuntu1.3) ... 2024-09-10T21:23:01.405 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../libuuid1_2.37.2-4ubuntu3.4_amd64.deb ... 2024-09-10T21:23:01.516 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking libuuid1:amd64 (2.37.2-4ubuntu3.4) over (2.37.2-4ubuntu3) ... 2024-09-10T21:23:01.612 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package libelf-dev:amd64. 2024-09-10T21:23:01.628 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../06-libelf-dev_0.186-1build1_amd64.deb ... 2024-09-10T21:23:01.678 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking libelf-dev:amd64 (0.186-1build1) ... 2024-09-10T21:23:01.866 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up libuuid1:amd64 (2.37.2-4ubuntu3.4) ... 2024-09-10T21:23:02.115 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package libfl2:amd64. 2024-09-10T21:23:02.132 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../07-libfl2_2.6.4-8build2_amd64.deb ... 2024-09-10T21:23:02.182 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking libfl2:amd64 (2.6.4-8build2) ... 2024-09-10T21:23:02.266 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package bison. 2024-09-10T21:23:02.344 INFO:teuthology.orchestra.run.smithi017.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 143359 files and directories currently installed.) 2024-09-10T21:23:02.348 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../00-bison_2%3a3.8.2+dfsg-1build1_amd64.deb ... 2024-09-10T21:23:02.393 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking bison (2:3.8.2+dfsg-1build1) ... 2024-09-10T21:23:02.485 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package libfl-dev:amd64. 2024-09-10T21:23:02.501 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../08-libfl-dev_2.6.4-8build2_amd64.deb ... 2024-09-10T21:23:02.543 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking libfl-dev:amd64 (2.6.4-8build2) ... 2024-09-10T21:23:02.845 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package libjson-perl. 2024-09-10T21:23:02.861 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package liblzo2-2:amd64. 2024-09-10T21:23:02.862 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../09-libjson-perl_4.04000-1_all.deb ... 2024-09-10T21:23:02.877 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../01-liblzo2-2_2.10-2build3_amd64.deb ... 2024-09-10T21:23:02.912 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking libjson-perl (4.04000-1) ... 2024-09-10T21:23:02.919 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking liblzo2-2:amd64 (2.10-2build3) ... 2024-09-10T21:23:03.257 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package libllvm14:amd64. 2024-09-10T21:23:03.273 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../10-libllvm14_1%3a14.0.0-1ubuntu1.1_amd64.deb ... 2024-09-10T21:23:03.323 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-09-10T21:23:03.430 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package dump. 2024-09-10T21:23:03.447 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../02-dump_0.4b47-3_amd64.deb ... 2024-09-10T21:23:03.488 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking dump (0.4b47-3) ... 2024-09-10T21:23:03.751 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package indent. 2024-09-10T21:23:03.767 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../03-indent_2.2.12-1ubuntu0.22.04.1_amd64.deb ... 2024-09-10T21:23:03.801 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking indent (2.2.12-1ubuntu0.22.04.1) ... 2024-09-10T21:23:04.061 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package libaio-dev:amd64. 2024-09-10T21:23:04.078 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../04-libaio-dev_0.3.112-13build1_amd64.deb ... 2024-09-10T21:23:04.120 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking libaio-dev:amd64 (0.3.112-13build1) ... 2024-09-10T21:23:04.548 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package libbluetooth3:amd64. 2024-09-10T21:23:04.565 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../05-libbluetooth3_5.64-0ubuntu1.3_amd64.deb ... 2024-09-10T21:23:04.607 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking libbluetooth3:amd64 (5.64-0ubuntu1.3) ... 2024-09-10T21:23:04.726 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2024-09-10T21:23:04.743 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../11-libmbim-glib4_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-09-10T21:23:04.793 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-09-10T21:23:04.876 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package libelf-dev:amd64. 2024-09-10T21:23:04.893 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../06-libelf-dev_0.186-1build1_amd64.deb ... 2024-09-10T21:23:04.943 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking libelf-dev:amd64 (0.186-1build1) ... 2024-09-10T21:23:05.095 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package libmbim-proxy. 2024-09-10T21:23:05.112 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../12-libmbim-proxy_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-09-10T21:23:05.153 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-09-10T21:23:05.388 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package libfl2:amd64. 2024-09-10T21:23:05.404 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../07-libfl2_2.6.4-8build2_amd64.deb ... 2024-09-10T21:23:05.454 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking libfl2:amd64 (2.6.4-8build2) ... 2024-09-10T21:23:05.556 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package libmm-glib0:amd64. 2024-09-10T21:23:05.573 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../13-libmm-glib0_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-09-10T21:23:05.623 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-09-10T21:23:05.790 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package libfl-dev:amd64. 2024-09-10T21:23:05.806 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../08-libfl-dev_2.6.4-8build2_amd64.deb ... 2024-09-10T21:23:05.865 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking libfl-dev:amd64 (2.6.4-8build2) ... 2024-09-10T21:23:06.060 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package libndp0:amd64. 2024-09-10T21:23:06.077 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../14-libndp0_1.8-0ubuntu3.1_amd64.deb ... 2024-09-10T21:23:06.118 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking libndp0:amd64 (1.8-0ubuntu3.1) ... 2024-09-10T21:23:06.126 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package libjson-perl. 2024-09-10T21:23:06.143 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../09-libjson-perl_4.04000-1_all.deb ... 2024-09-10T21:23:06.184 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking libjson-perl (4.04000-1) ... 2024-09-10T21:23:06.446 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package libnm0:amd64. 2024-09-10T21:23:06.463 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../15-libnm0_1.36.6-0ubuntu2_amd64.deb ... 2024-09-10T21:23:06.504 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-09-10T21:23:06.571 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package libllvm14:amd64. 2024-09-10T21:23:06.587 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../10-libllvm14_1%3a14.0.0-1ubuntu1.1_amd64.deb ... 2024-09-10T21:23:06.637 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-09-10T21:23:06.857 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package libpq5:amd64. 2024-09-10T21:23:06.874 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../16-libpq5_14.13-0ubuntu0.22.04.1_amd64.deb ... 2024-09-10T21:23:06.907 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking libpq5:amd64 (14.13-0ubuntu0.22.04.1) ... 2024-09-10T21:23:07.278 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2024-09-10T21:23:07.294 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../17-libqmi-glib5_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-09-10T21:23:07.335 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-09-10T21:23:07.621 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package libqmi-proxy. 2024-09-10T21:23:07.637 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../18-libqmi-proxy_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-09-10T21:23:07.679 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-09-10T21:23:07.957 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package libssl-dev:amd64. 2024-09-10T21:23:07.973 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../19-libssl-dev_3.0.2-0ubuntu1.18_amd64.deb ... 2024-09-10T21:23:08.017 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2024-09-10T21:23:08.023 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking libssl-dev:amd64 (3.0.2-0ubuntu1.18) ... 2024-09-10T21:23:08.034 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../11-libmbim-glib4_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-09-10T21:23:08.075 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-09-10T21:23:08.369 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package libmbim-proxy. 2024-09-10T21:23:08.386 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../12-libmbim-proxy_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2024-09-10T21:23:08.428 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-09-10T21:23:08.535 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package libteamdctl0:amd64. 2024-09-10T21:23:08.551 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../20-libteamdctl0_1.31-1build2_amd64.deb ... 2024-09-10T21:23:08.593 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking libteamdctl0:amd64 (1.31-1build2) ... 2024-09-10T21:23:08.730 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package libmm-glib0:amd64. 2024-09-10T21:23:08.747 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../13-libmm-glib0_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-09-10T21:23:08.814 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-09-10T21:23:08.963 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package modemmanager. 2024-09-10T21:23:08.980 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../21-modemmanager_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-09-10T21:23:09.021 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-09-10T21:23:09.125 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package libndp0:amd64. 2024-09-10T21:23:09.142 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../14-libndp0_1.8-0ubuntu3.1_amd64.deb ... 2024-09-10T21:23:09.183 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking libndp0:amd64 (1.8-0ubuntu3.1) ... 2024-09-10T21:23:09.536 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package libnm0:amd64. 2024-09-10T21:23:09.553 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../15-libnm0_1.36.6-0ubuntu2_amd64.deb ... 2024-09-10T21:23:09.594 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-09-10T21:23:09.678 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package network-manager. 2024-09-10T21:23:09.695 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../22-network-manager_1.36.6-0ubuntu2_amd64.deb ... 2024-09-10T21:23:09.745 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking network-manager (1.36.6-0ubuntu2) ... 2024-09-10T21:23:09.955 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package libpq5:amd64. 2024-09-10T21:23:09.972 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../16-libpq5_14.13-0ubuntu0.22.04.1_amd64.deb ... 2024-09-10T21:23:10.014 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking libpq5:amd64 (14.13-0ubuntu0.22.04.1) ... 2024-09-10T21:23:10.318 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package ppp. 2024-09-10T21:23:10.333 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2024-09-10T21:23:10.335 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../23-ppp_2.4.9-1+1ubuntu3_amd64.deb ... 2024-09-10T21:23:10.350 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../17-libqmi-glib5_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-09-10T21:23:10.383 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-09-10T21:23:10.686 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package libqmi-proxy. 2024-09-10T21:23:10.690 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking ppp (2.4.9-1+1ubuntu3) ... 2024-09-10T21:23:10.702 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../18-libqmi-proxy_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2024-09-10T21:23:10.744 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-09-10T21:23:10.980 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package libssl-dev:amd64. 2024-09-10T21:23:10.996 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../19-libssl-dev_3.0.2-0ubuntu1.18_amd64.deb ... 2024-09-10T21:23:11.038 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking libssl-dev:amd64 (3.0.2-0ubuntu1.18) ... 2024-09-10T21:23:11.149 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package pptp-linux. 2024-09-10T21:23:11.164 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../24-pptp-linux_1.10.0-1build3_amd64.deb ... 2024-09-10T21:23:11.249 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking pptp-linux (1.10.0-1build3) ... 2024-09-10T21:23:11.502 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package network-manager-pptp. 2024-09-10T21:23:11.517 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../25-network-manager-pptp_1.2.10-1_amd64.deb ... 2024-09-10T21:23:11.525 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package libteamdctl0:amd64. 2024-09-10T21:23:11.541 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../20-libteamdctl0_1.31-1build2_amd64.deb ... 2024-09-10T21:23:11.560 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking network-manager-pptp (1.2.10-1) ... 2024-09-10T21:23:11.575 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking libteamdctl0:amd64 (1.31-1build2) ... 2024-09-10T21:23:11.896 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package postgresql-client-common. 2024-09-10T21:23:11.912 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../26-postgresql-client-common_238_all.deb ... 2024-09-10T21:23:11.927 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package modemmanager. 2024-09-10T21:23:11.944 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../21-modemmanager_1.20.0-1~ubuntu22.04.3_amd64.deb ... 2024-09-10T21:23:11.954 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking postgresql-client-common (238) ... 2024-09-10T21:23:11.986 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-09-10T21:23:12.233 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package postgresql-client-14. 2024-09-10T21:23:12.249 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../27-postgresql-client-14_14.13-0ubuntu0.22.04.1_amd64.deb ... 2024-09-10T21:23:12.291 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking postgresql-client-14 (14.13-0ubuntu0.22.04.1) ... 2024-09-10T21:23:12.609 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package network-manager. 2024-09-10T21:23:12.626 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../22-network-manager_1.36.6-0ubuntu2_amd64.deb ... 2024-09-10T21:23:12.667 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking network-manager (1.36.6-0ubuntu2) ... 2024-09-10T21:23:12.871 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package postgresql-common. 2024-09-10T21:23:12.887 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../28-postgresql-common_238_all.deb ... 2024-09-10T21:23:12.982 INFO:teuthology.orchestra.run.smithi031.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2024-09-10T21:23:13.011 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking postgresql-common (238) ... 2024-09-10T21:23:13.469 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package postgresql-14. 2024-09-10T21:23:13.485 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../29-postgresql-14_14.13-0ubuntu0.22.04.1_amd64.deb ... 2024-09-10T21:23:13.564 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking postgresql-14 (14.13-0ubuntu0.22.04.1) ... 2024-09-10T21:23:13.800 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package ppp. 2024-09-10T21:23:13.817 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../23-ppp_2.4.9-1+1ubuntu3_amd64.deb ... 2024-09-10T21:23:14.194 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking ppp (2.4.9-1+1ubuntu3) ... 2024-09-10T21:23:14.513 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package pptp-linux. 2024-09-10T21:23:14.528 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../24-pptp-linux_1.10.0-1build3_amd64.deb ... 2024-09-10T21:23:14.571 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking pptp-linux (1.10.0-1build3) ... 2024-09-10T21:23:14.630 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package postgresql. 2024-09-10T21:23:14.646 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../30-postgresql_14+238_all.deb ... 2024-09-10T21:23:14.688 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking postgresql (14+238) ... 2024-09-10T21:23:14.849 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package network-manager-pptp. 2024-09-10T21:23:14.865 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../25-network-manager-pptp_1.2.10-1_amd64.deb ... 2024-09-10T21:23:14.907 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking network-manager-pptp (1.2.10-1) ... 2024-09-10T21:23:14.966 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package postgresql-client. 2024-09-10T21:23:14.981 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../31-postgresql-client_14+238_all.deb ... 2024-09-10T21:23:15.024 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking postgresql-client (14+238) ... 2024-09-10T21:23:15.219 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package postgresql-client-common. 2024-09-10T21:23:15.234 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../26-postgresql-client-common_238_all.deb ... 2024-09-10T21:23:15.243 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package postgresql-contrib. 2024-09-10T21:23:15.258 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../32-postgresql-contrib_14+238_all.deb ... 2024-09-10T21:23:15.277 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking postgresql-client-common (238) ... 2024-09-10T21:23:15.326 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking postgresql-contrib (14+238) ... 2024-09-10T21:23:15.580 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package postgresql-client-14. 2024-09-10T21:23:15.597 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../27-postgresql-client-14_14.13-0ubuntu0.22.04.1_amd64.deb ... 2024-09-10T21:23:15.620 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package tcl8.6. 2024-09-10T21:23:15.636 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../33-tcl8.6_8.6.12+dfsg-1build1_amd64.deb ... 2024-09-10T21:23:15.638 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking postgresql-client-14 (14.13-0ubuntu0.22.04.1) ... 2024-09-10T21:23:15.678 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking tcl8.6 (8.6.12+dfsg-1build1) ... 2024-09-10T21:23:15.998 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package tcl. 2024-09-10T21:23:16.014 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../34-tcl_8.6.11+1build2_amd64.deb ... 2024-09-10T21:23:16.061 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking tcl (8.6.11+1build2) ... 2024-09-10T21:23:16.252 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package postgresql-common. 2024-09-10T21:23:16.268 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../28-postgresql-common_238_all.deb ... 2024-09-10T21:23:16.330 INFO:teuthology.orchestra.run.smithi017.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2024-09-10T21:23:16.367 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking postgresql-common (238) ... 2024-09-10T21:23:16.426 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package wpasupplicant. 2024-09-10T21:23:16.443 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../35-wpasupplicant_2%3a2.10-6ubuntu2.1_amd64.deb ... 2024-09-10T21:23:16.509 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking wpasupplicant (2:2.10-6ubuntu2.1) ... 2024-09-10T21:23:16.821 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package usb-modeswitch-data. 2024-09-10T21:23:16.837 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../36-usb-modeswitch-data_20191128-4_all.deb ... 2024-09-10T21:23:16.871 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking usb-modeswitch-data (20191128-4) ... 2024-09-10T21:23:16.892 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package postgresql-14. 2024-09-10T21:23:16.909 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../29-postgresql-14_14.13-0ubuntu0.22.04.1_amd64.deb ... 2024-09-10T21:23:16.953 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking postgresql-14 (14.13-0ubuntu0.22.04.1) ... 2024-09-10T21:23:17.266 INFO:teuthology.orchestra.run.smithi031.stdout:Selecting previously unselected package usb-modeswitch. 2024-09-10T21:23:17.283 INFO:teuthology.orchestra.run.smithi031.stdout:Preparing to unpack .../37-usb-modeswitch_2.6.1-3ubuntu2_amd64.deb ... 2024-09-10T21:23:17.333 INFO:teuthology.orchestra.run.smithi031.stdout:Unpacking usb-modeswitch (2.6.1-3ubuntu2) ... 2024-09-10T21:23:17.661 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up postgresql-client-common (238) ... 2024-09-10T21:23:17.846 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up flex (2.6.4-8build2) ... 2024-09-10T21:23:18.014 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up tcl8.6 (8.6.12+dfsg-1build1) ... 2024-09-10T21:23:18.140 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up libpq5:amd64 (14.13-0ubuntu0.22.04.1) ... 2024-09-10T21:23:18.161 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package postgresql. 2024-09-10T21:23:18.177 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../30-postgresql_14+238_all.deb ... 2024-09-10T21:23:18.219 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking postgresql (14+238) ... 2024-09-10T21:23:18.299 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up liblzo2-2:amd64 (2.10-2build3) ... 2024-09-10T21:23:18.425 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up libaio-dev:amd64 (0.3.112-13build1) ... 2024-09-10T21:23:18.496 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package postgresql-client. 2024-09-10T21:23:18.512 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../31-postgresql-client_14+238_all.deb ... 2024-09-10T21:23:18.551 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up postgresql-client-14 (14.13-0ubuntu0.22.04.1) ... 2024-09-10T21:23:18.554 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking postgresql-client (14+238) ... 2024-09-10T21:23:18.798 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package postgresql-contrib. 2024-09-10T21:23:18.815 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../32-postgresql-contrib_14+238_all.deb ... 2024-09-10T21:23:18.890 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking postgresql-contrib (14+238) ... 2024-09-10T21:23:19.184 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package tcl8.6. 2024-09-10T21:23:19.201 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../33-tcl8.6_8.6.12+dfsg-1build1_amd64.deb ... 2024-09-10T21:23:19.234 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking tcl8.6 (8.6.12+dfsg-1build1) ... 2024-09-10T21:23:19.518 INFO:teuthology.orchestra.run.smithi031.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2024-09-10T21:23:19.528 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package tcl. 2024-09-10T21:23:19.545 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../34-tcl_8.6.11+1build2_amd64.deb ... 2024-09-10T21:23:19.591 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking tcl (8.6.11+1build2) ... 2024-09-10T21:23:19.606 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up wpasupplicant (2:2.10-6ubuntu2.1) ... 2024-09-10T21:23:19.939 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package wpasupplicant. 2024-09-10T21:23:19.956 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../35-wpasupplicant_2%3a2.10-6ubuntu2.1_amd64.deb ... 2024-09-10T21:23:19.960 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2024-09-10T21:23:19.960 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2024-09-10T21:23:20.018 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking wpasupplicant (2:2.10-6ubuntu2.1) ... 2024-09-10T21:23:20.327 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package usb-modeswitch-data. 2024-09-10T21:23:20.344 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../36-usb-modeswitch-data_20191128-4_all.deb ... 2024-09-10T21:23:20.376 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking usb-modeswitch-data (20191128-4) ... 2024-09-10T21:23:20.693 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up uuid-dev:amd64 (2.37.2-4ubuntu3.4) ... 2024-09-10T21:23:20.796 INFO:teuthology.orchestra.run.smithi017.stdout:Selecting previously unselected package usb-modeswitch. 2024-09-10T21:23:20.802 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up libteamdctl0:amd64 (1.31-1build2) ... 2024-09-10T21:23:20.813 INFO:teuthology.orchestra.run.smithi017.stdout:Preparing to unpack .../37-usb-modeswitch_2.6.1-3ubuntu2_amd64.deb ... 2024-09-10T21:23:20.863 INFO:teuthology.orchestra.run.smithi017.stdout:Unpacking usb-modeswitch (2.6.1-3ubuntu2) ... 2024-09-10T21:23:20.928 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up libssl-dev:amd64 (3.0.2-0ubuntu1.18) ... 2024-09-10T21:23:21.088 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up usb-modeswitch-data (20191128-4) ... 2024-09-10T21:23:21.149 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up postgresql-client-common (238) ... 2024-09-10T21:23:21.292 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up libelf-dev:amd64 (0.186-1build1) ... 2024-09-10T21:23:21.377 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up flex (2.6.4-8build2) ... 2024-09-10T21:23:21.494 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up tcl8.6 (8.6.12+dfsg-1build1) ... 2024-09-10T21:23:21.603 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up libpq5:amd64 (14.13-0ubuntu0.22.04.1) ... 2024-09-10T21:23:21.753 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up liblzo2-2:amd64 (2.10-2build3) ... 2024-09-10T21:23:21.766 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up libfl2:amd64 (2.6.4-8build2) ... 2024-09-10T21:23:21.881 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up libaio-dev:amd64 (0.3.112-13build1) ... 2024-09-10T21:23:21.899 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-09-10T21:23:22.006 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up postgresql-client-14 (14.13-0ubuntu0.22.04.1) ... 2024-09-10T21:23:22.025 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up ppp (2.4.9-1+1ubuntu3) ... 2024-09-10T21:23:22.701 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-09-10T21:23:22.848 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up dump (0.4b47-3) ... 2024-09-10T21:23:22.943 INFO:teuthology.orchestra.run.smithi031.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2024-09-10T21:23:22.977 INFO:teuthology.orchestra.run.smithi017.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2024-09-10T21:23:23.008 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-09-10T21:23:23.079 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up wpasupplicant (2:2.10-6ubuntu2.1) ... 2024-09-10T21:23:23.133 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up libbluetooth3:amd64 (5.64-0ubuntu1.3) ... 2024-09-10T21:23:23.260 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up tcl (8.6.11+1build2) ... 2024-09-10T21:23:23.411 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up libjson-perl (4.04000-1) ... 2024-09-10T21:23:23.520 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up bison (2:3.8.2+dfsg-1build1) ... 2024-09-10T21:23:23.593 INFO:teuthology.orchestra.run.smithi031.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2024-09-10T21:23:23.663 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up indent (2.2.12-1ubuntu0.22.04.1) ... 2024-09-10T21:23:23.788 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up usb-modeswitch (2.6.1-3ubuntu2) ... 2024-09-10T21:23:23.799 INFO:teuthology.orchestra.run.smithi017.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2024-09-10T21:23:23.800 INFO:teuthology.orchestra.run.smithi017.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2024-09-10T21:23:23.982 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up libndp0:amd64 (1.8-0ubuntu3.1) ... 2024-09-10T21:23:24.124 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-09-10T21:23:24.250 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up pptp-linux (1.10.0-1build3) ... 2024-09-10T21:23:24.418 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up postgresql-client (14+238) ... 2024-09-10T21:23:24.523 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up uuid-dev:amd64 (2.37.2-4ubuntu3.4) ... 2024-09-10T21:23:24.561 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up network-manager (1.36.6-0ubuntu2) ... 2024-09-10T21:23:24.674 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up libteamdctl0:amd64 (1.31-1build2) ... 2024-09-10T21:23:24.802 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up libssl-dev:amd64 (3.0.2-0ubuntu1.18) ... 2024-09-10T21:23:24.903 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up usb-modeswitch-data (20191128-4) ... 2024-09-10T21:23:25.021 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up libelf-dev:amd64 (0.186-1build1) ... 2024-09-10T21:23:25.159 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-10T21:23:25.159 INFO:teuthology.orchestra.run.smithi031.stdout:The following network interfaces were found in /etc/network/interfaces 2024-09-10T21:23:25.159 INFO:teuthology.orchestra.run.smithi031.stdout:which means they are currently configured by ifupdown: 2024-09-10T21:23:25.159 INFO:teuthology.orchestra.run.smithi031.stdout:- ens1f0 2024-09-10T21:23:25.159 INFO:teuthology.orchestra.run.smithi031.stdout:If you want to manage those interfaces with NetworkManager instead 2024-09-10T21:23:25.159 INFO:teuthology.orchestra.run.smithi031.stdout:remove their configuration from /etc/network/interfaces. 2024-09-10T21:23:25.159 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-10T21:23:25.163 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up libfl2:amd64 (2.6.4-8build2) ... 2024-09-10T21:23:25.300 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2024-09-10T21:23:25.414 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up libnm0:amd64 (1.36.6-0ubuntu2) ... 2024-09-10T21:23:25.540 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up ppp (2.4.9-1+1ubuntu3) ... 2024-09-10T21:23:25.725 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2024-09-10T21:23:26.153 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2024-09-10T21:23:26.238 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2024-09-10T21:23:26.359 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up dump (0.4b47-3) ... 2024-09-10T21:23:26.442 INFO:teuthology.orchestra.run.smithi017.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2024-09-10T21:23:26.507 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.3) ... 2024-09-10T21:23:26.633 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up libbluetooth3:amd64 (5.64-0ubuntu1.3) ... 2024-09-10T21:23:26.801 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up tcl (8.6.11+1build2) ... 2024-09-10T21:23:26.888 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-09-10T21:23:26.952 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up libjson-perl (4.04000-1) ... 2024-09-10T21:23:27.040 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up libfl-dev:amd64 (2.6.4-8build2) ... 2024-09-10T21:23:27.078 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up bison (2:3.8.2+dfsg-1build1) ... 2024-09-10T21:23:27.166 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up network-manager-pptp (1.2.10-1) ... 2024-09-10T21:23:27.167 INFO:teuthology.orchestra.run.smithi017.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2024-09-10T21:23:27.238 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up indent (2.2.12-1ubuntu0.22.04.1) ... 2024-09-10T21:23:27.292 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up postgresql-common (238) ... 2024-09-10T21:23:27.363 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up usb-modeswitch (2.6.1-3ubuntu2) ... 2024-09-10T21:23:27.582 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up libndp0:amd64 (1.8-0ubuntu3.1) ... 2024-09-10T21:23:27.707 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2024-09-10T21:23:27.825 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up pptp-linux (1.10.0-1build3) ... 2024-09-10T21:23:27.960 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up postgresql-client (14+238) ... 2024-09-10T21:23:28.119 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up network-manager (1.36.6-0ubuntu2) ... 2024-09-10T21:23:28.262 INFO:teuthology.orchestra.run.smithi031.stdout:Adding user postgres to group ssl-cert 2024-09-10T21:23:28.515 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-10T21:23:28.515 INFO:teuthology.orchestra.run.smithi031.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2024-09-10T21:23:28.759 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-10T21:23:28.759 INFO:teuthology.orchestra.run.smithi017.stdout:The following network interfaces were found in /etc/network/interfaces 2024-09-10T21:23:28.759 INFO:teuthology.orchestra.run.smithi017.stdout:which means they are currently configured by ifupdown: 2024-09-10T21:23:28.760 INFO:teuthology.orchestra.run.smithi017.stdout:- ens1f0 2024-09-10T21:23:28.760 INFO:teuthology.orchestra.run.smithi017.stdout:If you want to manage those interfaces with NetworkManager instead 2024-09-10T21:23:28.760 INFO:teuthology.orchestra.run.smithi017.stdout:remove their configuration from /etc/network/interfaces. 2024-09-10T21:23:28.760 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-10T21:23:28.760 INFO:teuthology.orchestra.run.smithi031.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2024-09-10T21:23:28.760 INFO:teuthology.orchestra.run.smithi031.stdout:Removing obsolete dictionary files: 2024-09-10T21:23:28.899 INFO:teuthology.orchestra.run.smithi017.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2024-09-10T21:23:29.321 INFO:teuthology.orchestra.run.smithi017.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2024-09-10T21:23:29.504 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2024-09-10T21:23:29.756 INFO:teuthology.orchestra.run.smithi017.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2024-09-10T21:23:30.051 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up postgresql-14 (14.13-0ubuntu0.22.04.1) ... 2024-09-10T21:23:30.400 INFO:teuthology.orchestra.run.smithi031.stdout:Creating new PostgreSQL cluster 14/main ... 2024-09-10T21:23:30.416 INFO:teuthology.orchestra.run.smithi031.stdout:/usr/lib/postgresql/14/bin/initdb -D /var/lib/postgresql/14/main --auth-local peer --auth-host scram-sha-256 --no-instructions 2024-09-10T21:23:30.470 INFO:teuthology.orchestra.run.smithi031.stdout:The files belonging to this database system will be owned by user "postgres". 2024-09-10T21:23:30.470 INFO:teuthology.orchestra.run.smithi031.stdout:This user must also own the server process. 2024-09-10T21:23:30.470 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-10T21:23:30.470 INFO:teuthology.orchestra.run.smithi031.stdout:The database cluster will be initialized with locale "en_US". 2024-09-10T21:23:30.470 INFO:teuthology.orchestra.run.smithi031.stdout:The default database encoding has accordingly been set to "LATIN1". 2024-09-10T21:23:30.470 INFO:teuthology.orchestra.run.smithi031.stdout:The default text search configuration will be set to "english". 2024-09-10T21:23:30.470 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-10T21:23:30.470 INFO:teuthology.orchestra.run.smithi031.stdout:Data page checksums are disabled. 2024-09-10T21:23:30.470 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-10T21:23:30.471 INFO:teuthology.orchestra.run.smithi031.stdout:fixing permissions on existing directory /var/lib/postgresql/14/main ... ok 2024-09-10T21:23:30.471 INFO:teuthology.orchestra.run.smithi031.stdout:creating subdirectories ... ok 2024-09-10T21:23:30.471 INFO:teuthology.orchestra.run.smithi031.stdout:selecting dynamic shared memory implementation ... posix 2024-09-10T21:23:30.489 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2024-09-10T21:23:30.495 INFO:teuthology.orchestra.run.smithi031.stdout:selecting default max_connections ... 100 2024-09-10T21:23:30.531 INFO:teuthology.orchestra.run.smithi031.stdout:selecting default shared_buffers ... 128MB 2024-09-10T21:23:30.545 INFO:teuthology.orchestra.run.smithi031.stdout:selecting default time zone ... Etc/UTC 2024-09-10T21:23:30.545 INFO:teuthology.orchestra.run.smithi031.stdout:creating configuration files ... ok 2024-09-10T21:23:30.640 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up libfl-dev:amd64 (2.6.4-8build2) ... 2024-09-10T21:23:30.716 INFO:teuthology.orchestra.run.smithi031.stdout:running bootstrap script ... ok 2024-09-10T21:23:30.766 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up network-manager-pptp (1.2.10-1) ... 2024-09-10T21:23:30.892 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up postgresql-common (238) ... 2024-09-10T21:23:31.329 INFO:teuthology.orchestra.run.smithi031.stdout:performing post-bootstrap initialization ... ok 2024-09-10T21:23:31.841 INFO:teuthology.orchestra.run.smithi031.stdout:syncing data to disk ... ok 2024-09-10T21:23:31.892 INFO:teuthology.orchestra.run.smithi017.stdout:Adding user postgres to group ssl-cert 2024-09-10T21:23:32.153 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-10T21:23:32.153 INFO:teuthology.orchestra.run.smithi017.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2024-09-10T21:23:32.369 INFO:teuthology.orchestra.run.smithi031.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2024-09-10T21:23:32.434 INFO:teuthology.orchestra.run.smithi017.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2024-09-10T21:23:32.434 INFO:teuthology.orchestra.run.smithi017.stdout:Removing obsolete dictionary files: 2024-09-10T21:23:33.150 INFO:teuthology.orchestra.run.smithi017.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2024-09-10T21:23:33.693 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up postgresql-14 (14.13-0ubuntu0.22.04.1) ... 2024-09-10T21:23:34.030 INFO:teuthology.orchestra.run.smithi017.stdout:Creating new PostgreSQL cluster 14/main ... 2024-09-10T21:23:34.059 INFO:teuthology.orchestra.run.smithi017.stdout:/usr/lib/postgresql/14/bin/initdb -D /var/lib/postgresql/14/main --auth-local peer --auth-host scram-sha-256 --no-instructions 2024-09-10T21:23:34.116 INFO:teuthology.orchestra.run.smithi017.stdout:The files belonging to this database system will be owned by user "postgres". 2024-09-10T21:23:34.116 INFO:teuthology.orchestra.run.smithi017.stdout:This user must also own the server process. 2024-09-10T21:23:34.116 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-10T21:23:34.116 INFO:teuthology.orchestra.run.smithi017.stdout:The database cluster will be initialized with locale "en_US". 2024-09-10T21:23:34.116 INFO:teuthology.orchestra.run.smithi017.stdout:The default database encoding has accordingly been set to "LATIN1". 2024-09-10T21:23:34.116 INFO:teuthology.orchestra.run.smithi017.stdout:The default text search configuration will be set to "english". 2024-09-10T21:23:34.116 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-10T21:23:34.116 INFO:teuthology.orchestra.run.smithi017.stdout:Data page checksums are disabled. 2024-09-10T21:23:34.117 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-10T21:23:34.117 INFO:teuthology.orchestra.run.smithi017.stdout:fixing permissions on existing directory /var/lib/postgresql/14/main ... ok 2024-09-10T21:23:34.117 INFO:teuthology.orchestra.run.smithi017.stdout:creating subdirectories ... ok 2024-09-10T21:23:34.117 INFO:teuthology.orchestra.run.smithi017.stdout:selecting dynamic shared memory implementation ... posix 2024-09-10T21:23:34.142 INFO:teuthology.orchestra.run.smithi017.stdout:selecting default max_connections ... 100 2024-09-10T21:23:34.177 INFO:teuthology.orchestra.run.smithi017.stdout:selecting default shared_buffers ... 128MB 2024-09-10T21:23:34.191 INFO:teuthology.orchestra.run.smithi017.stdout:selecting default time zone ... Etc/UTC 2024-09-10T21:23:34.192 INFO:teuthology.orchestra.run.smithi017.stdout:creating configuration files ... ok 2024-09-10T21:23:34.361 INFO:teuthology.orchestra.run.smithi017.stdout:running bootstrap script ... ok 2024-09-10T21:23:34.973 INFO:teuthology.orchestra.run.smithi017.stdout:performing post-bootstrap initialization ... ok 2024-09-10T21:23:35.533 INFO:teuthology.orchestra.run.smithi017.stdout:syncing data to disk ... ok 2024-09-10T21:23:35.733 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-09-10T21:23:35.859 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up postgresql-contrib (14+238) ... 2024-09-10T21:23:35.985 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up postgresql (14+238) ... 2024-09-10T21:23:36.082 INFO:teuthology.orchestra.run.smithi017.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2024-09-10T21:23:36.110 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-09-10T21:23:36.270 INFO:teuthology.orchestra.run.smithi031.stdout:Setting up modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-09-10T21:23:36.546 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2024-09-10T21:23:36.546 INFO:teuthology.orchestra.run.smithi031.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2024-09-10T21:23:37.357 INFO:teuthology.orchestra.run.smithi031.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-09-10T21:23:37.456 INFO:teuthology.orchestra.run.smithi031.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2024-09-10T21:23:37.758 INFO:teuthology.orchestra.run.smithi031.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-09-10T21:23:37.910 INFO:teuthology.orchestra.run.smithi031.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-09-10T21:23:39.176 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2024-09-10T21:23:39.277 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up postgresql-contrib (14+238) ... 2024-09-10T21:23:39.346 INFO:teuthology.orchestra.run.smithi031.stdout:Processing triggers for dbus (1.12.20-2ubuntu4.1) ... 2024-09-10T21:23:39.403 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up postgresql (14+238) ... 2024-09-10T21:23:39.446 INFO:teuthology.orchestra.run.smithi031.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-09-10T21:23:39.530 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2024-09-10T21:23:39.689 INFO:teuthology.orchestra.run.smithi017.stdout:Setting up modemmanager (1.20.0-1~ubuntu22.04.3) ... 2024-09-10T21:23:39.958 INFO:teuthology.orchestra.run.smithi017.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2024-09-10T21:23:39.958 INFO:teuthology.orchestra.run.smithi017.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2024-09-10T21:23:40.017 INFO:teuthology.orchestra.run.smithi031.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-10T21:23:40.025 DEBUG:teuthology.parallel:result is None 2024-09-10T21:23:40.733 INFO:teuthology.orchestra.run.smithi017.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-09-10T21:23:40.842 INFO:teuthology.orchestra.run.smithi017.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2024-09-10T21:23:41.136 INFO:teuthology.orchestra.run.smithi017.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-09-10T21:23:41.279 INFO:teuthology.orchestra.run.smithi017.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-09-10T21:23:43.008 INFO:teuthology.orchestra.run.smithi017.stdout:Processing triggers for dbus (1.12.20-2ubuntu4.1) ... 2024-09-10T21:23:43.150 INFO:teuthology.orchestra.run.smithi017.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-09-10T21:23:43.694 INFO:teuthology.orchestra.run.smithi017.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-10T21:23:43.700 DEBUG:teuthology.parallel:result is None 2024-09-10T21:23:43.701 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=8f219c476cb4f26fc5719c661c0323719a45cb30 2024-09-10T21:23:43.850 DEBUG:teuthology.orchestra.run.smithi017:> dpkg-query -W -f '${Version}' ceph 2024-09-10T21:23:43.873 INFO:teuthology.orchestra.run.smithi017.stdout:19.3.0-4854-g8f219c47-1jammy 2024-09-10T21:23:43.873 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-4854-g8f219c47-1jammy 2024-09-10T21:23:43.873 INFO:teuthology.task.install:The correct ceph version 19.3.0-4854-g8f219c47-1jammy is installed. 2024-09-10T21:23:43.875 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=8f219c476cb4f26fc5719c661c0323719a45cb30 2024-09-10T21:23:44.022 DEBUG:teuthology.orchestra.run.smithi031:> dpkg-query -W -f '${Version}' ceph 2024-09-10T21:23:44.047 INFO:teuthology.orchestra.run.smithi031.stdout:19.3.0-4854-g8f219c47-1jammy 2024-09-10T21:23:44.047 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-4854-g8f219c47-1jammy 2024-09-10T21:23:44.047 INFO:teuthology.task.install:The correct ceph version 19.3.0-4854-g8f219c47-1jammy is installed. 2024-09-10T21:23:44.049 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-10T21:23:44.049 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-09-10T21:23:44.049 DEBUG:teuthology.orchestra.run.smithi017:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-10T21:23:44.070 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-10T21:23:44.070 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-10T21:23:44.112 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-10T21:23:44.113 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-09-10T21:23:44.113 DEBUG:teuthology.orchestra.run.smithi017:> sudo dd of=/usr/bin/daemon-helper 2024-09-10T21:23:44.132 DEBUG:teuthology.orchestra.run.smithi017:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-10T21:23:44.195 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-10T21:23:44.195 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/usr/bin/daemon-helper 2024-09-10T21:23:44.217 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-10T21:23:44.278 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-10T21:23:44.279 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-09-10T21:23:44.279 DEBUG:teuthology.orchestra.run.smithi017:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-10T21:23:44.298 DEBUG:teuthology.orchestra.run.smithi017:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-10T21:23:44.359 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-10T21:23:44.360 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-10T21:23:44.381 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-10T21:23:44.443 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-10T21:23:44.443 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-09-10T21:23:44.443 DEBUG:teuthology.orchestra.run.smithi017:> sudo dd of=/usr/bin/stdin-killer 2024-09-10T21:23:44.463 DEBUG:teuthology.orchestra.run.smithi017:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-10T21:23:44.523 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-10T21:23:44.523 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/usr/bin/stdin-killer 2024-09-10T21:23:44.544 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-10T21:23:44.606 INFO:teuthology.run_tasks:Running task ceph... 2024-09-10T21:23:44.685 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-09-10T21:23:44.686 DEBUG:teuthology.orchestra.run.smithi017:> sudo chmod 777 /var/log/ceph 2024-09-10T21:23:44.689 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod 777 /var/log/ceph 2024-09-10T21:23:44.710 INFO:tasks.ceph:Disabling ceph logrotate... 2024-09-10T21:23:44.710 DEBUG:teuthology.orchestra.run.smithi017:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-10T21:23:44.751 DEBUG:teuthology.orchestra.run.smithi031:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-10T21:23:44.774 INFO:tasks.ceph:Creating extra log directories... 2024-09-10T21:23:44.774 DEBUG:teuthology.orchestra.run.smithi017:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-10T21:23:44.811 DEBUG:teuthology.orchestra.run.smithi031:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-10T21:23:44.849 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-09-10T21:23:44.849 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-09-10T21:23:44.849 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-09-10T21:23:44.849 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-09-10T21:23:44.849 DEBUG:teuthology.orchestra.run.smithi017:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-09-10T21:23:44.876 DEBUG:teuthology.orchestra.run.smithi017:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-09-10T21:23:44.963 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-10T21:23:44.963 DEBUG:teuthology.orchestra.run.smithi031:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-09-10T21:23:44.972 DEBUG:teuthology.orchestra.run.smithi031:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-09-10T21:23:45.069 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-09-10T21:23:45.069 INFO:tasks.ceph:config {'conf': {'client': {'client_shutdown_timeout': 120}, 'mds': {'mds oft prefetch dirfrags': True, 'mds_sleep_rank_change': 5000000.0}, 'mgr': {'client mount timeout': 30, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'filestore flush min': 0, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'fs.*is degraded', 'filesystem is degraded', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'filesystem is offline', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'insufficient standby MDS daemons available', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', 'SLOW_OPS', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'noscrub', 'nodeep-scrub', 'overall HEALTH_', '\\(MON_DOWN\\)', 'SLOW_OPS', 'slow request'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-09-10T21:23:45.069 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899674', 'branch': 'wip-hemanth-testing-2024-09-10-0723', 'description': 'fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse msgr-failures/none objectstore-ec/bluestore-comp-ec-root overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down pg_health prefetch_dirfrags/no prefetch_dirfrags/yes prefetch_entire_dirfrags/no prefetch_entire_dirfrags/yes races session_timeout thrashosds-health} ranks/1 tasks/{1-thrash/mon 2-workunit/suites/ffsb}}', 'email': 'hyelloji@redhat.com', 'first_in_suite': False, 'job_id': '7899674', 'kernel': {'flavor': 'default', 'kdb': 1, 'sha1': '2ca44a2f35ca8f36983aaaa46d49312239ef519a'}, 'ktype': 'distro', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi', 'no_nested_subset': False, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'ubuntu', 'os_version': '22.04', 'overrides': {'admin_socket': {'branch': 'wip-hemanth-testing-2024-09-10-0723'}, 'ceph': {'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 1, 'session_timeout': 300}, 'cluster-conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 2, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_dir_prefetch': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon': {'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'osd op complaint time': 180}}, 'conf': {'client': {'client_shutdown_timeout': 120}, 'mds': {'mds oft prefetch dirfrags': True, 'mds_sleep_rank_change': 5000000.0}, 'mgr': {'client mount timeout': 30, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'filestore flush min': 0, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore'}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'fs.*is degraded', 'filesystem is degraded', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'filesystem is offline', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'insufficient standby MDS daemons available', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', 'SLOW_OPS', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'noscrub', 'nodeep-scrub', 'overall HEALTH_', '\\(MON_DOWN\\)', 'SLOW_OPS', 'slow request'], 'sha1': '8f219c476cb4f26fc5719c661c0323719a45cb30'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'install': {'ceph': {'flavor': 'default', 'sha1': '8f219c476cb4f26fc5719c661c0323719a45cb30'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-hemanth-testing-2024-09-10-0723', 'sha1': '8f219c476cb4f26fc5719c661c0323719a45cb30'}}, 'owner': 'scheduled_hyelloji@teuthology', 'priority': 100, 'repo': 'https://github.com/ceph/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'mds.e', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'mds.f', 'osd.4', 'osd.5', 'osd.6', 'osd.7']], 'seed': 6190, 'sha1': '8f219c476cb4f26fc5719c661c0323719a45cb30', 'sleep_before_teardown': 0, 'subset': '479/1024', 'suite': 'fs', 'suite_branch': 'wip-hemanth-testing-2024-09-10-0723', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph-c_8f219c476cb4f26fc5719c661c0323719a45cb30/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph-ci.git', 'suite_sha1': '8f219c476cb4f26fc5719c661c0323719a45cb30', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi017.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMdVGJLkxjFoSuiC7ENO8miovADl+rrihzPwA71fmjeWR6D5PXrFr25hGSAkVomlBKEW8BMoyc6xiGzj1oYCvSs=', 'smithi031.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLOzCyCbCP19GI36Ugek8kcKz2SzTWmzUxBzO1exGW1rJUNRtcir7zCZsp33ksPDirBshanGA5Npfx+oSf+kFes='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'flavor': 'default', 'kdb': 1, 'sha1': '2ca44a2f35ca8f36983aaaa46d49312239ef519a'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '8f219c476cb4f26fc5719c661c0323719a45cb30'}}, {'ceph': {'create_rbd_pool': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 1, 'session_timeout': 300}, 'cluster-conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 2, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_dir_prefetch': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon': {'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'osd op complaint time': 180}}, 'conf': {'client': {'client_shutdown_timeout': 120}, 'mds': {'mds oft prefetch dirfrags': True, 'mds_sleep_rank_change': 5000000.0}, 'mgr': {'client mount timeout': 30, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'filestore flush min': 0, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore'}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'fs.*is degraded', 'filesystem is degraded', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'filesystem is offline', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'insufficient standby MDS daemons available', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', 'SLOW_OPS', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', 'mons down', 'mon down', 'out of quorum', 'noscrub', 'nodeep-scrub', 'overall HEALTH_', '\\(MON_DOWN\\)', 'SLOW_OPS', 'slow request'], 'sha1': '8f219c476cb4f26fc5719c661c0323719a45cb30', 'cluster': 'ceph'}}, {'ceph-fuse': None}, {'mon_thrash': {'check_mds_failover': True, 'revive_delay': 20, 'thrash_delay': 10}}, {'workunit': {'clients': {'all': ['suites/ffsb.sh']}}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ['local function is_kupstream() return false end', 'local function is_kdistro() return false end', 'local function is_fuse() return true end', "local function syntax_version() return '' end"]}, 'teuthology_branch': 'main', 'teuthology_sha1': '3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a', 'timestamp': '2024-09-10_16:59:47', 'tube': 'smithi', 'user': 'hyelloji', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856'} 2024-09-10T21:23:45.070 DEBUG:teuthology.orchestra.run.smithi017:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-10T21:23:45.073 DEBUG:teuthology.orchestra.run.smithi031:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-10T21:23:45.082 DEBUG:teuthology.orchestra.run.smithi017:> sudo install -d -m0777 -- /var/run/ceph 2024-09-10T21:23:45.123 DEBUG:teuthology.orchestra.run.smithi031:> sudo install -d -m0777 -- /var/run/ceph 2024-09-10T21:23:45.146 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-09-10T21:23:45.146 DEBUG:teuthology.orchestra.run.smithi017:> dd if=/scratch_devs of=/dev/stdout 2024-09-10T21:23:45.188 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-10T21:23:45.188 DEBUG:teuthology.orchestra.run.smithi017:> stat /dev/vg_nvme/lv_1 2024-09-10T21:23:45.236 INFO:teuthology.orchestra.run.smithi017.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-10T21:23:45.236 INFO:teuthology.orchestra.run.smithi017.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T21:23:45.236 INFO:teuthology.orchestra.run.smithi017.stdout:Device: 6h/6d Inode: 903 Links: 1 2024-09-10T21:23:45.237 INFO:teuthology.orchestra.run.smithi017.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T21:23:45.237 INFO:teuthology.orchestra.run.smithi017.stdout:Access: 2024-09-10 21:16:58.920152725 +0000 2024-09-10T21:23:45.237 INFO:teuthology.orchestra.run.smithi017.stdout:Modify: 2024-09-10 21:16:58.652155116 +0000 2024-09-10T21:23:45.237 INFO:teuthology.orchestra.run.smithi017.stdout:Change: 2024-09-10 21:16:58.652155116 +0000 2024-09-10T21:23:45.237 INFO:teuthology.orchestra.run.smithi017.stdout: Birth: 2024-09-10 21:16:58.652155116 +0000 2024-09-10T21:23:45.237 DEBUG:teuthology.orchestra.run.smithi017:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-10T21:23:45.297 INFO:teuthology.orchestra.run.smithi017.stderr:1+0 records in 2024-09-10T21:23:45.297 INFO:teuthology.orchestra.run.smithi017.stderr:1+0 records out 2024-09-10T21:23:45.297 INFO:teuthology.orchestra.run.smithi017.stderr:512 bytes copied, 0.000246802 s, 2.1 MB/s 2024-09-10T21:23:45.299 DEBUG:teuthology.orchestra.run.smithi017:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-10T21:23:45.354 DEBUG:teuthology.orchestra.run.smithi017:> stat /dev/vg_nvme/lv_2 2024-09-10T21:23:45.404 INFO:teuthology.orchestra.run.smithi017.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-10T21:23:45.405 INFO:teuthology.orchestra.run.smithi017.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T21:23:45.405 INFO:teuthology.orchestra.run.smithi017.stdout:Device: 6h/6d Inode: 920 Links: 1 2024-09-10T21:23:45.405 INFO:teuthology.orchestra.run.smithi017.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T21:23:45.405 INFO:teuthology.orchestra.run.smithi017.stdout:Access: 2024-09-10 21:16:59.520147371 +0000 2024-09-10T21:23:45.405 INFO:teuthology.orchestra.run.smithi017.stdout:Modify: 2024-09-10 21:16:59.260149692 +0000 2024-09-10T21:23:45.405 INFO:teuthology.orchestra.run.smithi017.stdout:Change: 2024-09-10 21:16:59.260149692 +0000 2024-09-10T21:23:45.405 INFO:teuthology.orchestra.run.smithi017.stdout: Birth: 2024-09-10 21:16:59.260149692 +0000 2024-09-10T21:23:45.405 DEBUG:teuthology.orchestra.run.smithi017:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-10T21:23:45.465 INFO:teuthology.orchestra.run.smithi017.stderr:1+0 records in 2024-09-10T21:23:45.466 INFO:teuthology.orchestra.run.smithi017.stderr:1+0 records out 2024-09-10T21:23:45.466 INFO:teuthology.orchestra.run.smithi017.stderr:512 bytes copied, 0.000239922 s, 2.1 MB/s 2024-09-10T21:23:45.467 DEBUG:teuthology.orchestra.run.smithi017:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-10T21:23:45.522 DEBUG:teuthology.orchestra.run.smithi017:> stat /dev/vg_nvme/lv_3 2024-09-10T21:23:45.576 INFO:teuthology.orchestra.run.smithi017.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-10T21:23:45.577 INFO:teuthology.orchestra.run.smithi017.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T21:23:45.577 INFO:teuthology.orchestra.run.smithi017.stdout:Device: 6h/6d Inode: 936 Links: 1 2024-09-10T21:23:45.577 INFO:teuthology.orchestra.run.smithi017.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T21:23:45.577 INFO:teuthology.orchestra.run.smithi017.stdout:Access: 2024-09-10 21:17:00.076142412 +0000 2024-09-10T21:23:45.577 INFO:teuthology.orchestra.run.smithi017.stdout:Modify: 2024-09-10 21:16:59.804144838 +0000 2024-09-10T21:23:45.577 INFO:teuthology.orchestra.run.smithi017.stdout:Change: 2024-09-10 21:16:59.804144838 +0000 2024-09-10T21:23:45.577 INFO:teuthology.orchestra.run.smithi017.stdout: Birth: 2024-09-10 21:16:59.804144838 +0000 2024-09-10T21:23:45.577 DEBUG:teuthology.orchestra.run.smithi017:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-10T21:23:45.642 INFO:teuthology.orchestra.run.smithi017.stderr:1+0 records in 2024-09-10T21:23:45.642 INFO:teuthology.orchestra.run.smithi017.stderr:1+0 records out 2024-09-10T21:23:45.642 INFO:teuthology.orchestra.run.smithi017.stderr:512 bytes copied, 0.000251138 s, 2.0 MB/s 2024-09-10T21:23:45.644 DEBUG:teuthology.orchestra.run.smithi017:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-10T21:23:45.702 DEBUG:teuthology.orchestra.run.smithi017:> stat /dev/vg_nvme/lv_4 2024-09-10T21:23:45.756 INFO:teuthology.orchestra.run.smithi017.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-10T21:23:45.756 INFO:teuthology.orchestra.run.smithi017.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T21:23:45.756 INFO:teuthology.orchestra.run.smithi017.stdout:Device: 6h/6d Inode: 953 Links: 1 2024-09-10T21:23:45.757 INFO:teuthology.orchestra.run.smithi017.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T21:23:45.757 INFO:teuthology.orchestra.run.smithi017.stdout:Access: 2024-09-10 21:17:00.716136702 +0000 2024-09-10T21:23:45.757 INFO:teuthology.orchestra.run.smithi017.stdout:Modify: 2024-09-10 21:17:00.412139414 +0000 2024-09-10T21:23:45.757 INFO:teuthology.orchestra.run.smithi017.stdout:Change: 2024-09-10 21:17:00.412139414 +0000 2024-09-10T21:23:45.757 INFO:teuthology.orchestra.run.smithi017.stdout: Birth: 2024-09-10 21:17:00.412139414 +0000 2024-09-10T21:23:45.757 DEBUG:teuthology.orchestra.run.smithi017:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-10T21:23:45.819 INFO:teuthology.orchestra.run.smithi017.stderr:1+0 records in 2024-09-10T21:23:45.819 INFO:teuthology.orchestra.run.smithi017.stderr:1+0 records out 2024-09-10T21:23:45.819 INFO:teuthology.orchestra.run.smithi017.stderr:512 bytes copied, 0.000263824 s, 1.9 MB/s 2024-09-10T21:23:45.821 DEBUG:teuthology.orchestra.run.smithi017:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-10T21:23:45.875 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-10T21:23:45.875 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-10T21:23:45.875 DEBUG:teuthology.orchestra.run.smithi031:> dd if=/scratch_devs of=/dev/stdout 2024-09-10T21:23:45.884 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-10T21:23:45.884 DEBUG:teuthology.orchestra.run.smithi031:> stat /dev/vg_nvme/lv_1 2024-09-10T21:23:45.935 INFO:teuthology.orchestra.run.smithi031.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-10T21:23:45.935 INFO:teuthology.orchestra.run.smithi031.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T21:23:45.935 INFO:teuthology.orchestra.run.smithi031.stdout:Device: 6h/6d Inode: 902 Links: 1 2024-09-10T21:23:45.935 INFO:teuthology.orchestra.run.smithi031.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T21:23:45.935 INFO:teuthology.orchestra.run.smithi031.stdout:Access: 2024-09-10 21:16:58.924549624 +0000 2024-09-10T21:23:45.935 INFO:teuthology.orchestra.run.smithi031.stdout:Modify: 2024-09-10 21:16:58.652552053 +0000 2024-09-10T21:23:45.935 INFO:teuthology.orchestra.run.smithi031.stdout:Change: 2024-09-10 21:16:58.652552053 +0000 2024-09-10T21:23:45.935 INFO:teuthology.orchestra.run.smithi031.stdout: Birth: 2024-09-10 21:16:58.652552053 +0000 2024-09-10T21:23:45.935 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-10T21:23:45.997 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records in 2024-09-10T21:23:45.997 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records out 2024-09-10T21:23:45.997 INFO:teuthology.orchestra.run.smithi031.stderr:512 bytes copied, 0.000401174 s, 1.3 MB/s 2024-09-10T21:23:45.999 DEBUG:teuthology.orchestra.run.smithi031:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-10T21:23:46.056 DEBUG:teuthology.orchestra.run.smithi031:> stat /dev/vg_nvme/lv_2 2024-09-10T21:23:46.111 INFO:teuthology.orchestra.run.smithi031.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-10T21:23:46.111 INFO:teuthology.orchestra.run.smithi031.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T21:23:46.111 INFO:teuthology.orchestra.run.smithi031.stdout:Device: 6h/6d Inode: 921 Links: 1 2024-09-10T21:23:46.111 INFO:teuthology.orchestra.run.smithi031.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T21:23:46.111 INFO:teuthology.orchestra.run.smithi031.stdout:Access: 2024-09-10 21:16:59.228546909 +0000 2024-09-10T21:23:46.111 INFO:teuthology.orchestra.run.smithi031.stdout:Modify: 2024-09-10 21:16:59.224546945 +0000 2024-09-10T21:23:46.111 INFO:teuthology.orchestra.run.smithi031.stdout:Change: 2024-09-10 21:16:59.224546945 +0000 2024-09-10T21:23:46.111 INFO:teuthology.orchestra.run.smithi031.stdout: Birth: 2024-09-10 21:16:59.224546945 +0000 2024-09-10T21:23:46.111 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-10T21:23:46.172 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records in 2024-09-10T21:23:46.173 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records out 2024-09-10T21:23:46.173 INFO:teuthology.orchestra.run.smithi031.stderr:512 bytes copied, 0.000378624 s, 1.4 MB/s 2024-09-10T21:23:46.175 DEBUG:teuthology.orchestra.run.smithi031:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-10T21:23:46.228 DEBUG:teuthology.orchestra.run.smithi031:> stat /dev/vg_nvme/lv_3 2024-09-10T21:23:46.279 INFO:teuthology.orchestra.run.smithi031.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-10T21:23:46.279 INFO:teuthology.orchestra.run.smithi031.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T21:23:46.279 INFO:teuthology.orchestra.run.smithi031.stdout:Device: 6h/6d Inode: 937 Links: 1 2024-09-10T21:23:46.279 INFO:teuthology.orchestra.run.smithi031.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T21:23:46.279 INFO:teuthology.orchestra.run.smithi031.stdout:Access: 2024-09-10 21:17:00.092539192 +0000 2024-09-10T21:23:46.279 INFO:teuthology.orchestra.run.smithi031.stdout:Modify: 2024-09-10 21:16:59.816541657 +0000 2024-09-10T21:23:46.279 INFO:teuthology.orchestra.run.smithi031.stdout:Change: 2024-09-10 21:16:59.816541657 +0000 2024-09-10T21:23:46.279 INFO:teuthology.orchestra.run.smithi031.stdout: Birth: 2024-09-10 21:16:59.816541657 +0000 2024-09-10T21:23:46.280 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-10T21:23:46.345 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records in 2024-09-10T21:23:46.345 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records out 2024-09-10T21:23:46.345 INFO:teuthology.orchestra.run.smithi031.stderr:512 bytes copied, 0.000372199 s, 1.4 MB/s 2024-09-10T21:23:46.347 DEBUG:teuthology.orchestra.run.smithi031:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-10T21:23:46.404 DEBUG:teuthology.orchestra.run.smithi031:> stat /dev/vg_nvme/lv_4 2024-09-10T21:23:46.458 INFO:teuthology.orchestra.run.smithi031.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-10T21:23:46.458 INFO:teuthology.orchestra.run.smithi031.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T21:23:46.459 INFO:teuthology.orchestra.run.smithi031.stdout:Device: 6h/6d Inode: 952 Links: 1 2024-09-10T21:23:46.459 INFO:teuthology.orchestra.run.smithi031.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T21:23:46.459 INFO:teuthology.orchestra.run.smithi031.stdout:Access: 2024-09-10 21:17:00.696533796 +0000 2024-09-10T21:23:46.459 INFO:teuthology.orchestra.run.smithi031.stdout:Modify: 2024-09-10 21:17:00.412536333 +0000 2024-09-10T21:23:46.459 INFO:teuthology.orchestra.run.smithi031.stdout:Change: 2024-09-10 21:17:00.412536333 +0000 2024-09-10T21:23:46.459 INFO:teuthology.orchestra.run.smithi031.stdout: Birth: 2024-09-10 21:17:00.412536333 +0000 2024-09-10T21:23:46.459 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-10T21:23:46.521 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records in 2024-09-10T21:23:46.521 INFO:teuthology.orchestra.run.smithi031.stderr:1+0 records out 2024-09-10T21:23:46.521 INFO:teuthology.orchestra.run.smithi031.stderr:512 bytes copied, 0.000383176 s, 1.3 MB/s 2024-09-10T21:23:46.523 DEBUG:teuthology.orchestra.run.smithi031:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-10T21:23:46.576 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-10T21:23:46.576 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi017.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi031.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-10T21:23:46.577 INFO:tasks.ceph:Generating config... 2024-09-10T21:23:46.578 INFO:tasks.ceph:[client] client_shutdown_timeout = 120 2024-09-10T21:23:46.578 INFO:tasks.ceph:[mds] mds oft prefetch dirfrags = True 2024-09-10T21:23:46.578 INFO:tasks.ceph:[mds] mds_sleep_rank_change = 5000000.0 2024-09-10T21:23:46.578 INFO:tasks.ceph:[mgr] client mount timeout = 30 2024-09-10T21:23:46.578 INFO:tasks.ceph:[mgr] debug client = 20 2024-09-10T21:23:46.578 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-09-10T21:23:46.578 INFO:tasks.ceph:[mgr] debug ms = 1 2024-09-10T21:23:46.578 INFO:tasks.ceph:[mon] debug mon = 20 2024-09-10T21:23:46.578 INFO:tasks.ceph:[mon] debug ms = 1 2024-09-10T21:23:46.579 INFO:tasks.ceph:[mon] debug paxos = 20 2024-09-10T21:23:46.579 INFO:tasks.ceph:[mon] mon down mkfs grace = 300 2024-09-10T21:23:46.579 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-09-10T21:23:46.579 INFO:tasks.ceph:[osd] bluestore compression mode = aggressive 2024-09-10T21:23:46.579 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-09-10T21:23:46.579 INFO:tasks.ceph:[osd] debug bluefs = 20 2024-09-10T21:23:46.579 INFO:tasks.ceph:[osd] debug bluestore = 20 2024-09-10T21:23:46.579 INFO:tasks.ceph:[osd] debug ms = 1 2024-09-10T21:23:46.579 INFO:tasks.ceph:[osd] debug osd = 20 2024-09-10T21:23:46.579 INFO:tasks.ceph:[osd] debug rocksdb = 10 2024-09-10T21:23:46.579 INFO:tasks.ceph:[osd] filestore flush min = 0 2024-09-10T21:23:46.579 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-09-10T21:23:46.579 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-09-10T21:23:46.579 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-09-10T21:23:46.580 INFO:tasks.ceph:[osd] osd blocked scrub grace period = 3600 2024-09-10T21:23:46.580 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-09-10T21:23:46.580 INFO:tasks.ceph:[osd] osd max markdown count = 1000 2024-09-10T21:23:46.580 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-09-10T21:23:46.580 INFO:tasks.ceph:Setting up mon.a... 2024-09-10T21:23:46.580 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-09-10T21:23:46.627 INFO:teuthology.orchestra.run.smithi017.stdout:creating /etc/ceph/ceph.keyring 2024-09-10T21:23:46.638 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-09-10T21:23:46.730 DEBUG:teuthology.orchestra.run.smithi017:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-10T21:23:46.750 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.17'), ('mon.b', '172.21.15.31'), ('mon.c', '[v2:172.21.15.31:3301,v1:172.21.15.31:6790]')] 2024-09-10T21:23:46.751 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.17,172.21.15.31,[v2:172.21.15.31:3301,v1:172.21.15.31:6790]'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'filestore flush min': 0, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore'}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'client mount timeout': 30, 'debug client': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon down mkfs grace': 300}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client_shutdown_timeout': 120}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.e': {}, 'mds.b': {}, 'mds.d': {}, 'mds.f': {}, 'mds': {'mds oft prefetch dirfrags': True, 'mds_sleep_rank_change': 5000000.0}} 2024-09-10T21:23:46.751 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-09-10T21:23:46.752 DEBUG:teuthology.orchestra.run.smithi017:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-09-10T21:23:46.800 DEBUG:teuthology.orchestra.run.smithi017:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.17 --add b 172.21.15.31 --addv c '[v2:172.21.15.31:3301,v1:172.21.15.31:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-09-10T21:23:46.893 INFO:teuthology.orchestra.run.smithi017.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-09-10T21:23:46.893 INFO:teuthology.orchestra.run.smithi017.stdout:monmaptool: generated fsid 5e2f023f-d6a6-49f4-a256-f4807dce3739 2024-09-10T21:23:46.893 INFO:teuthology.orchestra.run.smithi017.stdout:setting min_mon_release = quincy 2024-09-10T21:23:46.893 INFO:teuthology.orchestra.run.smithi017.stdout:epoch 0 2024-09-10T21:23:46.893 INFO:teuthology.orchestra.run.smithi017.stdout:fsid 5e2f023f-d6a6-49f4-a256-f4807dce3739 2024-09-10T21:23:46.893 INFO:teuthology.orchestra.run.smithi017.stdout:last_changed 2024-09-10T21:23:46.892130+0000 2024-09-10T21:23:46.893 INFO:teuthology.orchestra.run.smithi017.stdout:created 2024-09-10T21:23:46.892130+0000 2024-09-10T21:23:46.894 INFO:teuthology.orchestra.run.smithi017.stdout:min_mon_release 17 (quincy) 2024-09-10T21:23:46.894 INFO:teuthology.orchestra.run.smithi017.stdout:election_strategy: 1 2024-09-10T21:23:46.894 INFO:teuthology.orchestra.run.smithi017.stdout:0: [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] mon.a 2024-09-10T21:23:46.894 INFO:teuthology.orchestra.run.smithi017.stdout:1: [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] mon.b 2024-09-10T21:23:46.894 INFO:teuthology.orchestra.run.smithi017.stdout:2: [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] mon.c 2024-09-10T21:23:46.894 INFO:teuthology.orchestra.run.smithi017.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-09-10T21:23:46.896 DEBUG:teuthology.orchestra.run.smithi017:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-09-10T21:23:46.905 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 5e2f023f-d6a6-49f4-a256-f4807dce3739... 2024-09-10T21:23:46.905 DEBUG:teuthology.orchestra.run.smithi017:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-10T21:23:46.955 DEBUG:teuthology.orchestra.run.smithi031:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-10T21:23:46.995 INFO:teuthology.orchestra.run.smithi017.stdout:[global] 2024-09-10T21:23:46.995 INFO:teuthology.orchestra.run.smithi017.stdout: chdir = "" 2024-09-10T21:23:46.995 INFO:teuthology.orchestra.run.smithi017.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-10T21:23:46.996 INFO:teuthology.orchestra.run.smithi017.stdout: auth supported = cephx 2024-09-10T21:23:46.996 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-10T21:23:46.996 INFO:teuthology.orchestra.run.smithi017.stdout: filestore xattr use omap = true 2024-09-10T21:23:46.996 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-10T21:23:46.996 INFO:teuthology.orchestra.run.smithi017.stdout: mon clock drift allowed = 1.000 2024-09-10T21:23:46.996 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-10T21:23:46.996 INFO:teuthology.orchestra.run.smithi017.stdout: osd crush chooseleaf type = 0 2024-09-10T21:23:46.996 INFO:teuthology.orchestra.run.smithi017.stdout: auth debug = true 2024-09-10T21:23:46.996 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-10T21:23:46.996 INFO:teuthology.orchestra.run.smithi017.stdout: ms die on old message = true 2024-09-10T21:23:46.996 INFO:teuthology.orchestra.run.smithi017.stdout: ms die on bug = true 2024-09-10T21:23:46.996 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-10T21:23:46.996 INFO:teuthology.orchestra.run.smithi017.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-10T21:23:46.997 INFO:teuthology.orchestra.run.smithi017.stdout: mon pg warn max object skew = 0 2024-09-10T21:23:46.997 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-10T21:23:46.997 INFO:teuthology.orchestra.run.smithi017.stdout: # disable pg_autoscaler by default for new pools 2024-09-10T21:23:46.997 INFO:teuthology.orchestra.run.smithi017.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-10T21:23:46.997 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-10T21:23:46.997 INFO:teuthology.orchestra.run.smithi017.stdout: osd pool default size = 2 2024-09-10T21:23:46.997 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-10T21:23:46.997 INFO:teuthology.orchestra.run.smithi017.stdout: mon osd allow primary affinity = true 2024-09-10T21:23:46.997 INFO:teuthology.orchestra.run.smithi017.stdout: mon osd allow pg remap = true 2024-09-10T21:23:46.997 INFO:teuthology.orchestra.run.smithi017.stdout: mon warn on legacy crush tunables = false 2024-09-10T21:23:46.997 INFO:teuthology.orchestra.run.smithi017.stdout: mon warn on crush straw calc version zero = false 2024-09-10T21:23:46.997 INFO:teuthology.orchestra.run.smithi017.stdout: mon warn on no sortbitwise = false 2024-09-10T21:23:46.997 INFO:teuthology.orchestra.run.smithi017.stdout: mon warn on osd down out interval zero = false 2024-09-10T21:23:46.997 INFO:teuthology.orchestra.run.smithi017.stdout: mon warn on too few osds = false 2024-09-10T21:23:46.997 INFO:teuthology.orchestra.run.smithi017.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-10T21:23:46.998 INFO:teuthology.orchestra.run.smithi017.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-10T21:23:46.998 INFO:teuthology.orchestra.run.smithi017.stdout: mon_allow_pool_size_one = true 2024-09-10T21:23:46.998 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-10T21:23:46.998 INFO:teuthology.orchestra.run.smithi017.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-10T21:23:46.998 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-10T21:23:46.998 INFO:teuthology.orchestra.run.smithi017.stdout: osd default data pool replay window = 5 2024-09-10T21:23:46.998 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-10T21:23:46.998 INFO:teuthology.orchestra.run.smithi017.stdout: mon allow pool delete = true 2024-09-10T21:23:46.998 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-10T21:23:46.998 INFO:teuthology.orchestra.run.smithi017.stdout: mon cluster log file level = debug 2024-09-10T21:23:46.998 INFO:teuthology.orchestra.run.smithi017.stdout: debug asserts on shutdown = true 2024-09-10T21:23:46.998 INFO:teuthology.orchestra.run.smithi017.stdout: mon health detail to clog = false 2024-09-10T21:23:46.998 INFO:teuthology.orchestra.run.smithi017.stdout: mon host = "172.21.15.17,172.21.15.31,[v2:172.21.15.31:3301,v1:172.21.15.31:6790]" 2024-09-10T21:23:46.999 INFO:teuthology.orchestra.run.smithi017.stdout: fsid = 5e2f023f-d6a6-49f4-a256-f4807dce3739 2024-09-10T21:23:46.999 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-10T21:23:46.999 INFO:teuthology.orchestra.run.smithi017.stdout:[osd] 2024-09-10T21:23:46.999 INFO:teuthology.orchestra.run.smithi017.stdout: osd journal size = 100 2024-09-10T21:23:46.999 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-10T21:23:46.999 INFO:teuthology.orchestra.run.smithi017.stdout: osd scrub load threshold = 5.0 2024-09-10T21:23:46.999 INFO:teuthology.orchestra.run.smithi017.stdout: osd scrub max interval = 600 2024-09-10T21:23:46.999 INFO:teuthology.orchestra.run.smithi017.stdout: osd mclock profile = high_recovery_ops 2024-09-10T21:23:46.999 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-10T21:23:46.999 INFO:teuthology.orchestra.run.smithi017.stdout: osd recover clone overlap = true 2024-09-10T21:23:46.999 INFO:teuthology.orchestra.run.smithi017.stdout: osd recovery max chunk = 1048576 2024-09-10T21:23:46.999 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-10T21:23:46.999 INFO:teuthology.orchestra.run.smithi017.stdout: osd debug shutdown = true 2024-09-10T21:23:46.999 INFO:teuthology.orchestra.run.smithi017.stdout: osd debug op order = true 2024-09-10T21:23:47.000 INFO:teuthology.orchestra.run.smithi017.stdout: osd debug verify stray on activate = true 2024-09-10T21:23:47.000 INFO:teuthology.orchestra.run.smithi017.stdout: osd debug trim objects = true 2024-09-10T21:23:47.000 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-10T21:23:47.000 INFO:teuthology.orchestra.run.smithi017.stdout: osd open classes on start = true 2024-09-10T21:23:47.000 INFO:teuthology.orchestra.run.smithi017.stdout: osd debug pg log writeout = true 2024-09-10T21:23:47.000 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-10T21:23:47.000 INFO:teuthology.orchestra.run.smithi017.stdout: osd deep scrub update digest min age = 30 2024-09-10T21:23:47.000 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-10T21:23:47.000 INFO:teuthology.orchestra.run.smithi017.stdout: osd map max advance = 10 2024-09-10T21:23:47.000 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-10T21:23:47.000 INFO:teuthology.orchestra.run.smithi017.stdout: journal zero on create = true 2024-09-10T21:23:47.000 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-10T21:23:47.000 INFO:teuthology.orchestra.run.smithi017.stdout: filestore ondisk finisher threads = 3 2024-09-10T21:23:47.000 INFO:teuthology.orchestra.run.smithi017.stdout: filestore apply finisher threads = 3 2024-09-10T21:23:47.000 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-10T21:23:47.001 INFO:teuthology.orchestra.run.smithi017.stdout: bdev debug aio = true 2024-09-10T21:23:47.001 INFO:teuthology.orchestra.run.smithi017.stdout: osd debug misdirected ops = true 2024-09-10T21:23:47.001 INFO:teuthology.orchestra.run.smithi017.stdout: bluestore block size = 96636764160 2024-09-10T21:23:47.001 INFO:teuthology.orchestra.run.smithi017.stdout: bluestore compression mode = aggressive 2024-09-10T21:23:47.001 INFO:teuthology.orchestra.run.smithi017.stdout: bluestore fsck on mount = True 2024-09-10T21:23:47.001 INFO:teuthology.orchestra.run.smithi017.stdout: debug bluefs = 20 2024-09-10T21:23:47.001 INFO:teuthology.orchestra.run.smithi017.stdout: debug bluestore = 20 2024-09-10T21:23:47.001 INFO:teuthology.orchestra.run.smithi017.stdout: debug ms = 1 2024-09-10T21:23:47.001 INFO:teuthology.orchestra.run.smithi017.stdout: debug osd = 20 2024-09-10T21:23:47.001 INFO:teuthology.orchestra.run.smithi017.stdout: debug rocksdb = 10 2024-09-10T21:23:47.001 INFO:teuthology.orchestra.run.smithi017.stdout: filestore flush min = 0 2024-09-10T21:23:47.001 INFO:teuthology.orchestra.run.smithi017.stdout: mon osd backfillfull_ratio = 0.85 2024-09-10T21:23:47.001 INFO:teuthology.orchestra.run.smithi017.stdout: mon osd full ratio = 0.9 2024-09-10T21:23:47.001 INFO:teuthology.orchestra.run.smithi017.stdout: mon osd nearfull ratio = 0.8 2024-09-10T21:23:47.002 INFO:teuthology.orchestra.run.smithi017.stdout: osd blocked scrub grace period = 3600 2024-09-10T21:23:47.002 INFO:teuthology.orchestra.run.smithi017.stdout: osd failsafe full ratio = 0.95 2024-09-10T21:23:47.002 INFO:teuthology.orchestra.run.smithi017.stdout: osd max markdown count = 1000 2024-09-10T21:23:47.002 INFO:teuthology.orchestra.run.smithi017.stdout: osd objectstore = bluestore 2024-09-10T21:23:47.002 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-10T21:23:47.002 INFO:teuthology.orchestra.run.smithi017.stdout:[mgr] 2024-09-10T21:23:47.002 INFO:teuthology.orchestra.run.smithi017.stdout: debug ms = 1 2024-09-10T21:23:47.002 INFO:teuthology.orchestra.run.smithi017.stdout: debug mgr = 20 2024-09-10T21:23:47.002 INFO:teuthology.orchestra.run.smithi017.stdout: debug mon = 20 2024-09-10T21:23:47.002 INFO:teuthology.orchestra.run.smithi017.stdout: debug auth = 20 2024-09-10T21:23:47.002 INFO:teuthology.orchestra.run.smithi017.stdout: mon reweight min pgs per osd = 4 2024-09-10T21:23:47.002 INFO:teuthology.orchestra.run.smithi017.stdout: mon reweight min bytes per osd = 10 2024-09-10T21:23:47.002 INFO:teuthology.orchestra.run.smithi017.stdout: mgr/telemetry/nag = false 2024-09-10T21:23:47.002 INFO:teuthology.orchestra.run.smithi017.stdout: client mount timeout = 30 2024-09-10T21:23:47.002 INFO:teuthology.orchestra.run.smithi017.stdout: debug client = 20 2024-09-10T21:23:47.003 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-10T21:23:47.003 INFO:teuthology.orchestra.run.smithi017.stdout:[mon] 2024-09-10T21:23:47.003 INFO:teuthology.orchestra.run.smithi017.stdout: debug ms = 1 2024-09-10T21:23:47.003 INFO:teuthology.orchestra.run.smithi017.stdout: debug mon = 20 2024-09-10T21:23:47.003 INFO:teuthology.orchestra.run.smithi017.stdout: debug paxos = 20 2024-09-10T21:23:47.003 INFO:teuthology.orchestra.run.smithi017.stdout: debug auth = 20 2024-09-10T21:23:47.003 INFO:teuthology.orchestra.run.smithi017.stdout: mon data avail warn = 5 2024-09-10T21:23:47.003 INFO:teuthology.orchestra.run.smithi017.stdout: mon mgr mkfs grace = 240 2024-09-10T21:23:47.003 INFO:teuthology.orchestra.run.smithi017.stdout: mon reweight min pgs per osd = 4 2024-09-10T21:23:47.003 INFO:teuthology.orchestra.run.smithi017.stdout: mon osd reporter subtree level = osd 2024-09-10T21:23:47.003 INFO:teuthology.orchestra.run.smithi017.stdout: mon osd prime pg temp = true 2024-09-10T21:23:47.003 INFO:teuthology.orchestra.run.smithi017.stdout: mon reweight min bytes per osd = 10 2024-09-10T21:23:47.003 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-10T21:23:47.003 INFO:teuthology.orchestra.run.smithi017.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-10T21:23:47.004 INFO:teuthology.orchestra.run.smithi017.stdout: auth mon ticket ttl = 660 # 11m 2024-09-10T21:23:47.004 INFO:teuthology.orchestra.run.smithi017.stdout: auth service ticket ttl = 240 # 4m 2024-09-10T21:23:47.004 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-10T21:23:47.004 INFO:teuthology.orchestra.run.smithi017.stdout: # don't complain about insecure global_id in the test suite 2024-09-10T21:23:47.004 INFO:teuthology.orchestra.run.smithi017.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-10T21:23:47.004 INFO:teuthology.orchestra.run.smithi017.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-10T21:23:47.004 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-10T21:23:47.004 INFO:teuthology.orchestra.run.smithi017.stdout: # 1m isn't quite enough 2024-09-10T21:23:47.004 INFO:teuthology.orchestra.run.smithi017.stdout: mon_down_mkfs_grace = 2m 2024-09-10T21:23:47.004 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-10T21:23:47.004 INFO:teuthology.orchestra.run.smithi017.stdout: mon_warn_on_filestore_osds = false 2024-09-10T21:23:47.004 INFO:teuthology.orchestra.run.smithi017.stdout: mon down mkfs grace = 300 2024-09-10T21:23:47.004 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-10T21:23:47.004 INFO:teuthology.orchestra.run.smithi017.stdout:[client] 2024-09-10T21:23:47.005 INFO:teuthology.orchestra.run.smithi017.stdout: rgw cache enabled = true 2024-09-10T21:23:47.005 INFO:teuthology.orchestra.run.smithi017.stdout: rgw enable ops log = true 2024-09-10T21:23:47.005 INFO:teuthology.orchestra.run.smithi017.stdout: rgw enable usage log = true 2024-09-10T21:23:47.005 INFO:teuthology.orchestra.run.smithi017.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-10T21:23:47.005 INFO:teuthology.orchestra.run.smithi017.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-10T21:23:47.005 INFO:teuthology.orchestra.run.smithi017.stdout: client_shutdown_timeout = 120 2024-09-10T21:23:47.005 INFO:teuthology.orchestra.run.smithi017.stdout:[mon.a] 2024-09-10T21:23:47.005 INFO:teuthology.orchestra.run.smithi017.stdout:[mon.b] 2024-09-10T21:23:47.005 INFO:teuthology.orchestra.run.smithi017.stdout:[mon.c] 2024-09-10T21:23:47.005 INFO:teuthology.orchestra.run.smithi017.stdout:[mds.a] 2024-09-10T21:23:47.005 INFO:teuthology.orchestra.run.smithi017.stdout:[mds.c] 2024-09-10T21:23:47.005 INFO:teuthology.orchestra.run.smithi017.stdout:[mds.e] 2024-09-10T21:23:47.005 INFO:teuthology.orchestra.run.smithi017.stdout:[mds.b] 2024-09-10T21:23:47.006 INFO:teuthology.orchestra.run.smithi017.stdout:[mds.d] 2024-09-10T21:23:47.006 INFO:teuthology.orchestra.run.smithi017.stdout:[mds.f] 2024-09-10T21:23:47.006 INFO:teuthology.orchestra.run.smithi017.stdout:[mds] 2024-09-10T21:23:47.006 INFO:teuthology.orchestra.run.smithi017.stdout: mds oft prefetch dirfrags = True 2024-09-10T21:23:47.006 INFO:teuthology.orchestra.run.smithi017.stdout: mds_sleep_rank_change = 5000000.0 2024-09-10T21:23:47.006 INFO:teuthology.orchestra.run.smithi031.stdout:[global] 2024-09-10T21:23:47.006 INFO:teuthology.orchestra.run.smithi031.stdout: chdir = "" 2024-09-10T21:23:47.006 INFO:teuthology.orchestra.run.smithi031.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-10T21:23:47.006 INFO:teuthology.orchestra.run.smithi031.stdout: auth supported = cephx 2024-09-10T21:23:47.006 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-10T21:23:47.007 INFO:teuthology.orchestra.run.smithi031.stdout: filestore xattr use omap = true 2024-09-10T21:23:47.007 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-10T21:23:47.007 INFO:teuthology.orchestra.run.smithi031.stdout: mon clock drift allowed = 1.000 2024-09-10T21:23:47.007 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-10T21:23:47.007 INFO:teuthology.orchestra.run.smithi031.stdout: osd crush chooseleaf type = 0 2024-09-10T21:23:47.007 INFO:teuthology.orchestra.run.smithi031.stdout: auth debug = true 2024-09-10T21:23:47.007 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-10T21:23:47.007 INFO:teuthology.orchestra.run.smithi031.stdout: ms die on old message = true 2024-09-10T21:23:47.007 INFO:teuthology.orchestra.run.smithi031.stdout: ms die on bug = true 2024-09-10T21:23:47.007 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-10T21:23:47.007 INFO:teuthology.orchestra.run.smithi031.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-10T21:23:47.007 INFO:teuthology.orchestra.run.smithi031.stdout: mon pg warn max object skew = 0 2024-09-10T21:23:47.007 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-10T21:23:47.007 INFO:teuthology.orchestra.run.smithi031.stdout: # disable pg_autoscaler by default for new pools 2024-09-10T21:23:47.008 INFO:teuthology.orchestra.run.smithi031.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-10T21:23:47.008 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-10T21:23:47.008 INFO:teuthology.orchestra.run.smithi031.stdout: osd pool default size = 2 2024-09-10T21:23:47.008 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-10T21:23:47.008 INFO:teuthology.orchestra.run.smithi031.stdout: mon osd allow primary affinity = true 2024-09-10T21:23:47.008 INFO:teuthology.orchestra.run.smithi031.stdout: mon osd allow pg remap = true 2024-09-10T21:23:47.008 INFO:teuthology.orchestra.run.smithi031.stdout: mon warn on legacy crush tunables = false 2024-09-10T21:23:47.008 INFO:teuthology.orchestra.run.smithi031.stdout: mon warn on crush straw calc version zero = false 2024-09-10T21:23:47.008 INFO:teuthology.orchestra.run.smithi031.stdout: mon warn on no sortbitwise = false 2024-09-10T21:23:47.008 INFO:teuthology.orchestra.run.smithi031.stdout: mon warn on osd down out interval zero = false 2024-09-10T21:23:47.008 INFO:teuthology.orchestra.run.smithi031.stdout: mon warn on too few osds = false 2024-09-10T21:23:47.008 INFO:teuthology.orchestra.run.smithi031.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-10T21:23:47.008 INFO:teuthology.orchestra.run.smithi031.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-10T21:23:47.008 INFO:teuthology.orchestra.run.smithi031.stdout: mon_allow_pool_size_one = true 2024-09-10T21:23:47.009 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-10T21:23:47.009 INFO:teuthology.orchestra.run.smithi031.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-10T21:23:47.009 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-10T21:23:47.009 INFO:teuthology.orchestra.run.smithi031.stdout: osd default data pool replay window = 5 2024-09-10T21:23:47.009 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-10T21:23:47.009 INFO:teuthology.orchestra.run.smithi031.stdout: mon allow pool delete = true 2024-09-10T21:23:47.009 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-10T21:23:47.009 INFO:teuthology.orchestra.run.smithi031.stdout: mon cluster log file level = debug 2024-09-10T21:23:47.009 INFO:teuthology.orchestra.run.smithi031.stdout: debug asserts on shutdown = true 2024-09-10T21:23:47.009 INFO:teuthology.orchestra.run.smithi031.stdout: mon health detail to clog = false 2024-09-10T21:23:47.009 INFO:teuthology.orchestra.run.smithi031.stdout: mon host = "172.21.15.17,172.21.15.31,[v2:172.21.15.31:3301,v1:172.21.15.31:6790]" 2024-09-10T21:23:47.009 INFO:teuthology.orchestra.run.smithi031.stdout: fsid = 5e2f023f-d6a6-49f4-a256-f4807dce3739 2024-09-10T21:23:47.009 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-10T21:23:47.009 INFO:teuthology.orchestra.run.smithi031.stdout:[osd] 2024-09-10T21:23:47.010 INFO:teuthology.orchestra.run.smithi031.stdout: osd journal size = 100 2024-09-10T21:23:47.010 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-10T21:23:47.010 INFO:teuthology.orchestra.run.smithi031.stdout: osd scrub load threshold = 5.0 2024-09-10T21:23:47.010 INFO:teuthology.orchestra.run.smithi031.stdout: osd scrub max interval = 600 2024-09-10T21:23:47.010 INFO:teuthology.orchestra.run.smithi031.stdout: osd mclock profile = high_recovery_ops 2024-09-10T21:23:47.010 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-10T21:23:47.010 INFO:teuthology.orchestra.run.smithi031.stdout: osd recover clone overlap = true 2024-09-10T21:23:47.010 INFO:teuthology.orchestra.run.smithi031.stdout: osd recovery max chunk = 1048576 2024-09-10T21:23:47.010 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-10T21:23:47.010 INFO:teuthology.orchestra.run.smithi031.stdout: osd debug shutdown = true 2024-09-10T21:23:47.010 INFO:teuthology.orchestra.run.smithi031.stdout: osd debug op order = true 2024-09-10T21:23:47.010 INFO:teuthology.orchestra.run.smithi031.stdout: osd debug verify stray on activate = true 2024-09-10T21:23:47.010 INFO:teuthology.orchestra.run.smithi031.stdout: osd debug trim objects = true 2024-09-10T21:23:47.010 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-10T21:23:47.011 INFO:teuthology.orchestra.run.smithi031.stdout: osd open classes on start = true 2024-09-10T21:23:47.011 INFO:teuthology.orchestra.run.smithi031.stdout: osd debug pg log writeout = true 2024-09-10T21:23:47.011 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-10T21:23:47.011 INFO:teuthology.orchestra.run.smithi031.stdout: osd deep scrub update digest min age = 30 2024-09-10T21:23:47.011 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-10T21:23:47.011 INFO:teuthology.orchestra.run.smithi031.stdout: osd map max advance = 10 2024-09-10T21:23:47.011 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-10T21:23:47.011 INFO:teuthology.orchestra.run.smithi031.stdout: journal zero on create = true 2024-09-10T21:23:47.011 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-10T21:23:47.011 INFO:teuthology.orchestra.run.smithi031.stdout: filestore ondisk finisher threads = 3 2024-09-10T21:23:47.011 INFO:teuthology.orchestra.run.smithi031.stdout: filestore apply finisher threads = 3 2024-09-10T21:23:47.011 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-10T21:23:47.011 INFO:teuthology.orchestra.run.smithi031.stdout: bdev debug aio = true 2024-09-10T21:23:47.011 INFO:teuthology.orchestra.run.smithi031.stdout: osd debug misdirected ops = true 2024-09-10T21:23:47.012 INFO:teuthology.orchestra.run.smithi031.stdout: bluestore block size = 96636764160 2024-09-10T21:23:47.012 INFO:teuthology.orchestra.run.smithi031.stdout: bluestore compression mode = aggressive 2024-09-10T21:23:47.012 INFO:teuthology.orchestra.run.smithi031.stdout: bluestore fsck on mount = True 2024-09-10T21:23:47.012 INFO:teuthology.orchestra.run.smithi031.stdout: debug bluefs = 20 2024-09-10T21:23:47.012 INFO:teuthology.orchestra.run.smithi031.stdout: debug bluestore = 20 2024-09-10T21:23:47.012 INFO:teuthology.orchestra.run.smithi031.stdout: debug ms = 1 2024-09-10T21:23:47.012 INFO:teuthology.orchestra.run.smithi031.stdout: debug osd = 20 2024-09-10T21:23:47.012 INFO:teuthology.orchestra.run.smithi031.stdout: debug rocksdb = 10 2024-09-10T21:23:47.012 INFO:teuthology.orchestra.run.smithi031.stdout: filestore flush min = 0 2024-09-10T21:23:47.012 INFO:teuthology.orchestra.run.smithi031.stdout: mon osd backfillfull_ratio = 0.85 2024-09-10T21:23:47.012 INFO:teuthology.orchestra.run.smithi031.stdout: mon osd full ratio = 0.9 2024-09-10T21:23:47.012 INFO:teuthology.orchestra.run.smithi031.stdout: mon osd nearfull ratio = 0.8 2024-09-10T21:23:47.012 INFO:teuthology.orchestra.run.smithi031.stdout: osd blocked scrub grace period = 3600 2024-09-10T21:23:47.012 INFO:teuthology.orchestra.run.smithi031.stdout: osd failsafe full ratio = 0.95 2024-09-10T21:23:47.013 INFO:teuthology.orchestra.run.smithi031.stdout: osd max markdown count = 1000 2024-09-10T21:23:47.013 INFO:teuthology.orchestra.run.smithi031.stdout: osd objectstore = bluestore 2024-09-10T21:23:47.013 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-10T21:23:47.013 INFO:teuthology.orchestra.run.smithi031.stdout:[mgr] 2024-09-10T21:23:47.013 INFO:teuthology.orchestra.run.smithi031.stdout: debug ms = 1 2024-09-10T21:23:47.013 INFO:teuthology.orchestra.run.smithi031.stdout: debug mgr = 20 2024-09-10T21:23:47.013 INFO:teuthology.orchestra.run.smithi031.stdout: debug mon = 20 2024-09-10T21:23:47.013 INFO:teuthology.orchestra.run.smithi031.stdout: debug auth = 20 2024-09-10T21:23:47.013 INFO:teuthology.orchestra.run.smithi031.stdout: mon reweight min pgs per osd = 4 2024-09-10T21:23:47.013 INFO:teuthology.orchestra.run.smithi031.stdout: mon reweight min bytes per osd = 10 2024-09-10T21:23:47.013 INFO:teuthology.orchestra.run.smithi031.stdout: mgr/telemetry/nag = false 2024-09-10T21:23:47.013 INFO:teuthology.orchestra.run.smithi031.stdout: client mount timeout = 30 2024-09-10T21:23:47.013 INFO:teuthology.orchestra.run.smithi031.stdout: debug client = 20 2024-09-10T21:23:47.014 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-10T21:23:47.014 INFO:teuthology.orchestra.run.smithi031.stdout:[mon] 2024-09-10T21:23:47.014 INFO:teuthology.orchestra.run.smithi031.stdout: debug ms = 1 2024-09-10T21:23:47.014 INFO:teuthology.orchestra.run.smithi031.stdout: debug mon = 20 2024-09-10T21:23:47.014 INFO:teuthology.orchestra.run.smithi031.stdout: debug paxos = 20 2024-09-10T21:23:47.014 INFO:teuthology.orchestra.run.smithi031.stdout: debug auth = 20 2024-09-10T21:23:47.014 INFO:teuthology.orchestra.run.smithi031.stdout: mon data avail warn = 5 2024-09-10T21:23:47.014 INFO:teuthology.orchestra.run.smithi031.stdout: mon mgr mkfs grace = 240 2024-09-10T21:23:47.014 INFO:teuthology.orchestra.run.smithi031.stdout: mon reweight min pgs per osd = 4 2024-09-10T21:23:47.014 INFO:teuthology.orchestra.run.smithi031.stdout: mon osd reporter subtree level = osd 2024-09-10T21:23:47.014 INFO:teuthology.orchestra.run.smithi031.stdout: mon osd prime pg temp = true 2024-09-10T21:23:47.014 INFO:teuthology.orchestra.run.smithi031.stdout: mon reweight min bytes per osd = 10 2024-09-10T21:23:47.014 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-10T21:23:47.014 INFO:teuthology.orchestra.run.smithi031.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-10T21:23:47.014 INFO:teuthology.orchestra.run.smithi031.stdout: auth mon ticket ttl = 660 # 11m 2024-09-10T21:23:47.014 INFO:teuthology.orchestra.run.smithi031.stdout: auth service ticket ttl = 240 # 4m 2024-09-10T21:23:47.015 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-10T21:23:47.015 INFO:teuthology.orchestra.run.smithi031.stdout: # don't complain about insecure global_id in the test suite 2024-09-10T21:23:47.015 INFO:teuthology.orchestra.run.smithi031.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-10T21:23:47.015 INFO:teuthology.orchestra.run.smithi031.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-10T21:23:47.015 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-10T21:23:47.015 INFO:teuthology.orchestra.run.smithi031.stdout: # 1m isn't quite enough 2024-09-10T21:23:47.015 INFO:teuthology.orchestra.run.smithi031.stdout: mon_down_mkfs_grace = 2m 2024-09-10T21:23:47.015 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-10T21:23:47.015 INFO:teuthology.orchestra.run.smithi031.stdout: mon_warn_on_filestore_osds = false 2024-09-10T21:23:47.015 INFO:teuthology.orchestra.run.smithi031.stdout: mon down mkfs grace = 300 2024-09-10T21:23:47.015 INFO:teuthology.orchestra.run.smithi031.stdout: 2024-09-10T21:23:47.015 INFO:teuthology.orchestra.run.smithi031.stdout:[client] 2024-09-10T21:23:47.015 INFO:teuthology.orchestra.run.smithi031.stdout: rgw cache enabled = true 2024-09-10T21:23:47.015 INFO:teuthology.orchestra.run.smithi031.stdout: rgw enable ops log = true 2024-09-10T21:23:47.016 INFO:teuthology.orchestra.run.smithi031.stdout: rgw enable usage log = true 2024-09-10T21:23:47.016 INFO:teuthology.orchestra.run.smithi031.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-10T21:23:47.016 INFO:teuthology.orchestra.run.smithi031.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-10T21:23:47.016 INFO:teuthology.orchestra.run.smithi031.stdout: client_shutdown_timeout = 120 2024-09-10T21:23:47.016 INFO:teuthology.orchestra.run.smithi031.stdout:[mon.a] 2024-09-10T21:23:47.016 INFO:teuthology.orchestra.run.smithi031.stdout:[mon.b] 2024-09-10T21:23:47.016 INFO:teuthology.orchestra.run.smithi031.stdout:[mon.c] 2024-09-10T21:23:47.016 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.a] 2024-09-10T21:23:47.016 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.c] 2024-09-10T21:23:47.016 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.e] 2024-09-10T21:23:47.016 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.b] 2024-09-10T21:23:47.016 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.d] 2024-09-10T21:23:47.016 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.f] 2024-09-10T21:23:47.017 INFO:teuthology.orchestra.run.smithi031.stdout:[mds] 2024-09-10T21:23:47.017 INFO:teuthology.orchestra.run.smithi031.stdout: mds oft prefetch dirfrags = True 2024-09-10T21:23:47.017 INFO:teuthology.orchestra.run.smithi031.stdout: mds_sleep_rank_change = 5000000.0 2024-09-10T21:23:47.018 INFO:tasks.ceph:Creating admin key on mon.a... 2024-09-10T21:23:47.018 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-09-10T21:23:47.118 INFO:tasks.ceph:Copying monmap to all nodes... 2024-09-10T21:23:47.118 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-09-10T21:23:47.118 DEBUG:teuthology.orchestra.run.smithi017:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-09-10T21:23:47.127 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-09-10T21:23:47.127 DEBUG:teuthology.orchestra.run.smithi017:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-09-10T21:23:47.176 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi017.front.sepia.ceph.com 2024-09-10T21:23:47.176 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-09-10T21:23:47.176 DEBUG:teuthology.orchestra.run.smithi017:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-10T21:23:47.176 DEBUG:teuthology.orchestra.run.smithi017:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-10T21:23:47.247 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-09-10T21:23:47.247 DEBUG:teuthology.orchestra.run.smithi017:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-10T21:23:47.295 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi031.front.sepia.ceph.com 2024-09-10T21:23:47.296 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-10T21:23:47.296 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-10T21:23:47.296 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-10T21:23:47.331 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-10T21:23:47.331 DEBUG:teuthology.orchestra.run.smithi031:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-10T21:23:47.382 INFO:tasks.ceph:Setting up mon nodes... 2024-09-10T21:23:47.382 INFO:tasks.ceph:Setting up mgr nodes... 2024-09-10T21:23:47.383 DEBUG:teuthology.orchestra.run.smithi017:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-09-10T21:23:47.439 INFO:teuthology.orchestra.run.smithi017.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-09-10T21:23:47.444 DEBUG:teuthology.orchestra.run.smithi031:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-09-10T21:23:47.505 INFO:teuthology.orchestra.run.smithi031.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-09-10T21:23:47.523 INFO:tasks.ceph:Setting up mds nodes... 2024-09-10T21:23:47.523 DEBUG:teuthology.orchestra.run.smithi017:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-09-10T21:23:47.579 INFO:teuthology.orchestra.run.smithi017.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-09-10T21:23:47.584 DEBUG:teuthology.orchestra.run.smithi017:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-09-10T21:23:47.603 DEBUG:teuthology.orchestra.run.smithi017:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-09-10T21:23:47.702 INFO:teuthology.orchestra.run.smithi017.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-09-10T21:23:47.707 DEBUG:teuthology.orchestra.run.smithi017:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-09-10T21:23:47.726 DEBUG:teuthology.orchestra.run.smithi017:> sudo mkdir -p /var/lib/ceph/mds/ceph-e && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.e /var/lib/ceph/mds/ceph-e/keyring 2024-09-10T21:23:47.827 INFO:teuthology.orchestra.run.smithi017.stdout:creating /var/lib/ceph/mds/ceph-e/keyring 2024-09-10T21:23:47.832 DEBUG:teuthology.orchestra.run.smithi017:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-e 2024-09-10T21:23:47.851 DEBUG:teuthology.orchestra.run.smithi031:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-09-10T21:23:47.913 INFO:teuthology.orchestra.run.smithi031.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-09-10T21:23:47.917 DEBUG:teuthology.orchestra.run.smithi031:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-09-10T21:23:47.938 DEBUG:teuthology.orchestra.run.smithi031:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-09-10T21:23:48.038 INFO:teuthology.orchestra.run.smithi031.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-09-10T21:23:48.043 DEBUG:teuthology.orchestra.run.smithi031:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-09-10T21:23:48.063 DEBUG:teuthology.orchestra.run.smithi031:> sudo mkdir -p /var/lib/ceph/mds/ceph-f && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.f /var/lib/ceph/mds/ceph-f/keyring 2024-09-10T21:23:48.162 INFO:teuthology.orchestra.run.smithi031.stdout:creating /var/lib/ceph/mds/ceph-f/keyring 2024-09-10T21:23:48.166 DEBUG:teuthology.orchestra.run.smithi031:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-f 2024-09-10T21:23:48.186 INFO:tasks.ceph_client:Setting up client nodes... 2024-09-10T21:23:48.186 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-10T21:23:48.230 INFO:teuthology.orchestra.run.smithi017.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-09-10T21:23:48.247 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-09-10T21:23:48.247 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi017.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi031.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-10T21:23:48.247 DEBUG:teuthology.orchestra.run.smithi017:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-09-10T21:23:48.311 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-10T21:23:48.312 INFO:tasks.ceph:role: osd.0 2024-09-10T21:23:48.312 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi017.front.sepia.ceph.com 2024-09-10T21:23:48.312 DEBUG:teuthology.orchestra.run.smithi017:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-10T21:23:48.375 INFO:teuthology.orchestra.run.smithi017.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-10T21:23:48.375 INFO:teuthology.orchestra.run.smithi017.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-10T21:23:48.375 INFO:teuthology.orchestra.run.smithi017.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-10T21:23:48.375 INFO:teuthology.orchestra.run.smithi017.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-10T21:23:48.375 INFO:teuthology.orchestra.run.smithi017.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-10T21:23:48.375 INFO:teuthology.orchestra.run.smithi017.stdout: = sunit=0 swidth=0 blks 2024-09-10T21:23:48.376 INFO:teuthology.orchestra.run.smithi017.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-10T21:23:48.376 INFO:teuthology.orchestra.run.smithi017.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-10T21:23:48.376 INFO:teuthology.orchestra.run.smithi017.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-10T21:23:48.376 INFO:teuthology.orchestra.run.smithi017.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-10T21:23:48.400 INFO:teuthology.orchestra.run.smithi017.stdout:Discarding blocks...Done. 2024-09-10T21:23:48.402 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi017.front.sepia.ceph.com -o noatime 2024-09-10T21:23:48.403 DEBUG:teuthology.orchestra.run.smithi017:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-09-10T21:23:48.472 DEBUG:teuthology.orchestra.run.smithi017:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-09-10T21:23:48.536 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T21:23:48.537 INFO:teuthology.orchestra.run.smithi017.stderr:sudo: /sbin/restorecon: command not found 2024-09-10T21:23:48.537 DEBUG:teuthology.orchestra.run.smithi017:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-09-10T21:23:48.603 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-10T21:23:48.603 INFO:tasks.ceph:role: osd.1 2024-09-10T21:23:48.603 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi017.front.sepia.ceph.com 2024-09-10T21:23:48.603 DEBUG:teuthology.orchestra.run.smithi017:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-10T21:23:48.666 INFO:teuthology.orchestra.run.smithi017.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-10T21:23:48.666 INFO:teuthology.orchestra.run.smithi017.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-10T21:23:48.667 INFO:teuthology.orchestra.run.smithi017.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-10T21:23:48.667 INFO:teuthology.orchestra.run.smithi017.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-10T21:23:48.667 INFO:teuthology.orchestra.run.smithi017.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-10T21:23:48.667 INFO:teuthology.orchestra.run.smithi017.stdout: = sunit=0 swidth=0 blks 2024-09-10T21:23:48.667 INFO:teuthology.orchestra.run.smithi017.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-10T21:23:48.667 INFO:teuthology.orchestra.run.smithi017.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-10T21:23:48.667 INFO:teuthology.orchestra.run.smithi017.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-10T21:23:48.667 INFO:teuthology.orchestra.run.smithi017.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-10T21:23:48.691 INFO:teuthology.orchestra.run.smithi017.stdout:Discarding blocks...Done. 2024-09-10T21:23:48.693 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi017.front.sepia.ceph.com -o noatime 2024-09-10T21:23:48.694 DEBUG:teuthology.orchestra.run.smithi017:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-09-10T21:23:48.764 DEBUG:teuthology.orchestra.run.smithi017:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-09-10T21:23:48.824 INFO:teuthology.orchestra.run.smithi017.stderr:sudo: /sbin/restorecon: command not found 2024-09-10T21:23:48.824 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T21:23:48.825 DEBUG:teuthology.orchestra.run.smithi017:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-09-10T21:23:48.887 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-10T21:23:48.887 INFO:tasks.ceph:role: osd.2 2024-09-10T21:23:48.887 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi017.front.sepia.ceph.com 2024-09-10T21:23:48.888 DEBUG:teuthology.orchestra.run.smithi017:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-10T21:23:48.947 INFO:teuthology.orchestra.run.smithi017.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-10T21:23:48.947 INFO:teuthology.orchestra.run.smithi017.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-10T21:23:48.947 INFO:teuthology.orchestra.run.smithi017.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-10T21:23:48.947 INFO:teuthology.orchestra.run.smithi017.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-10T21:23:48.947 INFO:teuthology.orchestra.run.smithi017.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-10T21:23:48.947 INFO:teuthology.orchestra.run.smithi017.stdout: = sunit=0 swidth=0 blks 2024-09-10T21:23:48.947 INFO:teuthology.orchestra.run.smithi017.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-10T21:23:48.947 INFO:teuthology.orchestra.run.smithi017.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-10T21:23:48.948 INFO:teuthology.orchestra.run.smithi017.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-10T21:23:48.948 INFO:teuthology.orchestra.run.smithi017.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-10T21:23:48.971 INFO:teuthology.orchestra.run.smithi017.stdout:Discarding blocks...Done. 2024-09-10T21:23:48.974 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi017.front.sepia.ceph.com -o noatime 2024-09-10T21:23:48.974 DEBUG:teuthology.orchestra.run.smithi017:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-09-10T21:23:49.044 DEBUG:teuthology.orchestra.run.smithi017:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-09-10T21:23:49.103 INFO:teuthology.orchestra.run.smithi017.stderr:sudo: /sbin/restorecon: command not found 2024-09-10T21:23:49.104 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T21:23:49.104 DEBUG:teuthology.orchestra.run.smithi017:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-09-10T21:23:49.167 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-10T21:23:49.167 INFO:tasks.ceph:role: osd.3 2024-09-10T21:23:49.167 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi017.front.sepia.ceph.com 2024-09-10T21:23:49.167 DEBUG:teuthology.orchestra.run.smithi017:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-10T21:23:49.231 INFO:teuthology.orchestra.run.smithi017.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-10T21:23:49.231 INFO:teuthology.orchestra.run.smithi017.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-10T21:23:49.231 INFO:teuthology.orchestra.run.smithi017.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-10T21:23:49.231 INFO:teuthology.orchestra.run.smithi017.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-10T21:23:49.231 INFO:teuthology.orchestra.run.smithi017.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-10T21:23:49.231 INFO:teuthology.orchestra.run.smithi017.stdout: = sunit=0 swidth=0 blks 2024-09-10T21:23:49.231 INFO:teuthology.orchestra.run.smithi017.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-10T21:23:49.231 INFO:teuthology.orchestra.run.smithi017.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-10T21:23:49.231 INFO:teuthology.orchestra.run.smithi017.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-10T21:23:49.232 INFO:teuthology.orchestra.run.smithi017.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-10T21:23:49.256 INFO:teuthology.orchestra.run.smithi017.stdout:Discarding blocks...Done. 2024-09-10T21:23:49.259 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi017.front.sepia.ceph.com -o noatime 2024-09-10T21:23:49.259 DEBUG:teuthology.orchestra.run.smithi017:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-09-10T21:23:49.329 DEBUG:teuthology.orchestra.run.smithi017:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-09-10T21:23:49.387 INFO:teuthology.orchestra.run.smithi017.stderr:sudo: /sbin/restorecon: command not found 2024-09-10T21:23:49.388 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T21:23:49.389 DEBUG:teuthology.orchestra.run.smithi017:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-10T21:23:49.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:23:49.511+0000 7f39e69fd7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-09-10T21:23:49.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:23:49.515+0000 7f39e69fd7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-09-10T21:23:49.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:23:49.515+0000 7f39e69fd7c0 -1 bdev(0x55ff9469f800 /var/lib/ceph/osd/ceph-0/block) open stat got: (1) Operation not permitted 2024-09-10T21:23:49.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:23:49.515+0000 7f39e69fd7c0 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-09-10T21:23:52.728 DEBUG:teuthology.orchestra.run.smithi017:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-10T21:23:52.751 DEBUG:teuthology.orchestra.run.smithi017:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-10T21:23:52.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:23:52.843+0000 7fd4a39a47c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-09-10T21:23:52.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:23:52.843+0000 7fd4a39a47c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-09-10T21:23:52.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:23:52.843+0000 7fd4a39a47c0 -1 bdev(0x55e91baf3800 /var/lib/ceph/osd/ceph-1/block) open stat got: (1) Operation not permitted 2024-09-10T21:23:52.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:23:52.847+0000 7fd4a39a47c0 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-09-10T21:23:56.096 DEBUG:teuthology.orchestra.run.smithi017:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-10T21:23:56.118 DEBUG:teuthology.orchestra.run.smithi017:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-10T21:23:56.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:23:56.215+0000 7f00b7b207c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-09-10T21:23:56.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:23:56.219+0000 7f00b7b207c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-09-10T21:23:56.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:23:56.219+0000 7f00b7b207c0 -1 bdev(0x55c17905b800 /var/lib/ceph/osd/ceph-2/block) open stat got: (1) Operation not permitted 2024-09-10T21:23:56.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:23:56.219+0000 7f00b7b207c0 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-09-10T21:23:59.467 DEBUG:teuthology.orchestra.run.smithi017:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-10T21:23:59.489 DEBUG:teuthology.orchestra.run.smithi017:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-10T21:23:59.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:23:59.583+0000 7efe385c27c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-09-10T21:23:59.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:23:59.587+0000 7efe385c27c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-09-10T21:23:59.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:23:59.587+0000 7efe385c27c0 -1 bdev(0x562fd5ab7800 /var/lib/ceph/osd/ceph-3/block) open stat got: (1) Operation not permitted 2024-09-10T21:23:59.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:23:59.587+0000 7efe385c27c0 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-09-10T21:24:02.815 DEBUG:teuthology.orchestra.run.smithi017:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-09-10T21:24:02.837 DEBUG:teuthology.orchestra.run.smithi031:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-09-10T21:24:02.860 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-10T21:24:02.860 INFO:tasks.ceph:role: osd.4 2024-09-10T21:24:02.860 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi031.front.sepia.ceph.com 2024-09-10T21:24:02.860 DEBUG:teuthology.orchestra.run.smithi031:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-10T21:24:02.930 INFO:teuthology.orchestra.run.smithi031.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-10T21:24:02.931 INFO:teuthology.orchestra.run.smithi031.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-10T21:24:02.931 INFO:teuthology.orchestra.run.smithi031.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-10T21:24:02.931 INFO:teuthology.orchestra.run.smithi031.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-10T21:24:02.931 INFO:teuthology.orchestra.run.smithi031.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-10T21:24:02.931 INFO:teuthology.orchestra.run.smithi031.stdout: = sunit=0 swidth=0 blks 2024-09-10T21:24:02.931 INFO:teuthology.orchestra.run.smithi031.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-10T21:24:02.931 INFO:teuthology.orchestra.run.smithi031.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-10T21:24:02.931 INFO:teuthology.orchestra.run.smithi031.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-10T21:24:02.931 INFO:teuthology.orchestra.run.smithi031.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-10T21:24:03.328 INFO:teuthology.orchestra.run.smithi031.stdout:Discarding blocks...Done. 2024-09-10T21:24:03.331 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi031.front.sepia.ceph.com -o noatime 2024-09-10T21:24:03.331 DEBUG:teuthology.orchestra.run.smithi031:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-09-10T21:24:03.422 DEBUG:teuthology.orchestra.run.smithi031:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-09-10T21:24:03.487 INFO:teuthology.orchestra.run.smithi031.stderr:sudo: /sbin/restorecon: command not found 2024-09-10T21:24:03.487 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T21:24:03.488 DEBUG:teuthology.orchestra.run.smithi031:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-09-10T21:24:03.554 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-10T21:24:03.554 INFO:tasks.ceph:role: osd.5 2024-09-10T21:24:03.554 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi031.front.sepia.ceph.com 2024-09-10T21:24:03.555 DEBUG:teuthology.orchestra.run.smithi031:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-10T21:24:03.627 INFO:teuthology.orchestra.run.smithi031.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-10T21:24:03.627 INFO:teuthology.orchestra.run.smithi031.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-10T21:24:03.627 INFO:teuthology.orchestra.run.smithi031.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-10T21:24:03.627 INFO:teuthology.orchestra.run.smithi031.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-10T21:24:03.627 INFO:teuthology.orchestra.run.smithi031.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-10T21:24:03.627 INFO:teuthology.orchestra.run.smithi031.stdout: = sunit=0 swidth=0 blks 2024-09-10T21:24:03.627 INFO:teuthology.orchestra.run.smithi031.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-10T21:24:03.627 INFO:teuthology.orchestra.run.smithi031.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-10T21:24:03.627 INFO:teuthology.orchestra.run.smithi031.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-10T21:24:03.627 INFO:teuthology.orchestra.run.smithi031.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-10T21:24:04.024 INFO:teuthology.orchestra.run.smithi031.stdout:Discarding blocks...Done. 2024-09-10T21:24:04.026 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi031.front.sepia.ceph.com -o noatime 2024-09-10T21:24:04.027 DEBUG:teuthology.orchestra.run.smithi031:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-09-10T21:24:04.140 DEBUG:teuthology.orchestra.run.smithi031:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-09-10T21:24:04.157 INFO:teuthology.orchestra.run.smithi031.stderr:sudo: /sbin/restorecon: command not found 2024-09-10T21:24:04.158 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T21:24:04.158 DEBUG:teuthology.orchestra.run.smithi031:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-09-10T21:24:04.222 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-10T21:24:04.222 INFO:tasks.ceph:role: osd.6 2024-09-10T21:24:04.222 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi031.front.sepia.ceph.com 2024-09-10T21:24:04.222 DEBUG:teuthology.orchestra.run.smithi031:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-10T21:24:04.293 INFO:teuthology.orchestra.run.smithi031.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-10T21:24:04.294 INFO:teuthology.orchestra.run.smithi031.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-10T21:24:04.294 INFO:teuthology.orchestra.run.smithi031.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-10T21:24:04.294 INFO:teuthology.orchestra.run.smithi031.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-10T21:24:04.294 INFO:teuthology.orchestra.run.smithi031.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-10T21:24:04.294 INFO:teuthology.orchestra.run.smithi031.stdout: = sunit=0 swidth=0 blks 2024-09-10T21:24:04.294 INFO:teuthology.orchestra.run.smithi031.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-10T21:24:04.294 INFO:teuthology.orchestra.run.smithi031.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-10T21:24:04.294 INFO:teuthology.orchestra.run.smithi031.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-10T21:24:04.294 INFO:teuthology.orchestra.run.smithi031.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-10T21:24:04.707 INFO:teuthology.orchestra.run.smithi031.stdout:Discarding blocks...Done. 2024-09-10T21:24:04.710 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi031.front.sepia.ceph.com -o noatime 2024-09-10T21:24:04.710 DEBUG:teuthology.orchestra.run.smithi031:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-09-10T21:24:04.838 DEBUG:teuthology.orchestra.run.smithi031:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-09-10T21:24:04.855 INFO:teuthology.orchestra.run.smithi031.stderr:sudo: /sbin/restorecon: command not found 2024-09-10T21:24:04.856 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T21:24:04.856 DEBUG:teuthology.orchestra.run.smithi031:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-09-10T21:24:04.922 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-10T21:24:04.922 INFO:tasks.ceph:role: osd.7 2024-09-10T21:24:04.922 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi031.front.sepia.ceph.com 2024-09-10T21:24:04.923 DEBUG:teuthology.orchestra.run.smithi031:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-10T21:24:04.994 INFO:teuthology.orchestra.run.smithi031.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-10T21:24:04.994 INFO:teuthology.orchestra.run.smithi031.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-10T21:24:04.994 INFO:teuthology.orchestra.run.smithi031.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-10T21:24:04.994 INFO:teuthology.orchestra.run.smithi031.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-10T21:24:04.994 INFO:teuthology.orchestra.run.smithi031.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-10T21:24:04.994 INFO:teuthology.orchestra.run.smithi031.stdout: = sunit=0 swidth=0 blks 2024-09-10T21:24:04.994 INFO:teuthology.orchestra.run.smithi031.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-10T21:24:04.994 INFO:teuthology.orchestra.run.smithi031.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-10T21:24:04.994 INFO:teuthology.orchestra.run.smithi031.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-10T21:24:04.995 INFO:teuthology.orchestra.run.smithi031.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-10T21:24:05.350 INFO:teuthology.orchestra.run.smithi031.stdout:Discarding blocks...Done. 2024-09-10T21:24:05.353 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi031.front.sepia.ceph.com -o noatime 2024-09-10T21:24:05.353 DEBUG:teuthology.orchestra.run.smithi031:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-09-10T21:24:05.448 DEBUG:teuthology.orchestra.run.smithi031:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-09-10T21:24:05.465 INFO:teuthology.orchestra.run.smithi031.stderr:sudo: /sbin/restorecon: command not found 2024-09-10T21:24:05.466 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T21:24:05.467 DEBUG:teuthology.orchestra.run.smithi031:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-10T21:24:05.596 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:05.593+0000 7f3fe786b7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-09-10T21:24:05.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:05.597+0000 7f3fe786b7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-09-10T21:24:05.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:05.597+0000 7f3fe786b7c0 -1 bdev(0x559d507a5800 /var/lib/ceph/osd/ceph-4/block) open stat got: (1) Operation not permitted 2024-09-10T21:24:05.599 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:05.597+0000 7f3fe786b7c0 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-09-10T21:24:08.786 DEBUG:teuthology.orchestra.run.smithi031:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-09-10T21:24:08.808 DEBUG:teuthology.orchestra.run.smithi031:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-10T21:24:08.904 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:08.901+0000 7f2a1da2b7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-09-10T21:24:08.906 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:08.901+0000 7f2a1da2b7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-09-10T21:24:08.906 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:08.901+0000 7f2a1da2b7c0 -1 bdev(0x562f71751800 /var/lib/ceph/osd/ceph-5/block) open stat got: (1) Operation not permitted 2024-09-10T21:24:08.906 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:08.901+0000 7f2a1da2b7c0 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-09-10T21:24:12.078 DEBUG:teuthology.orchestra.run.smithi031:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-09-10T21:24:12.100 DEBUG:teuthology.orchestra.run.smithi031:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-10T21:24:12.196 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:12.193+0000 7f30ee9207c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-09-10T21:24:12.197 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:12.193+0000 7f30ee9207c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-09-10T21:24:12.197 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:12.193+0000 7f30ee9207c0 -1 bdev(0x55bba5b95800 /var/lib/ceph/osd/ceph-6/block) open stat got: (1) Operation not permitted 2024-09-10T21:24:12.197 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:12.193+0000 7f30ee9207c0 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-09-10T21:24:15.072 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:24:15.077 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:24:15.440 DEBUG:teuthology.orchestra.run.smithi031:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-09-10T21:24:15.462 DEBUG:teuthology.orchestra.run.smithi031:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-10T21:24:15.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:15.565+0000 7f093b56e7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-09-10T21:24:15.567 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:15.565+0000 7f093b56e7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-09-10T21:24:15.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:15.565+0000 7f093b56e7c0 -1 bdev(0x55660ccf9800 /var/lib/ceph/osd/ceph-7/block) open stat got: (1) Operation not permitted 2024-09-10T21:24:15.568 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:15.565+0000 7f093b56e7c0 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-09-10T21:24:18.748 DEBUG:teuthology.orchestra.run.smithi031:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-09-10T21:24:18.770 INFO:tasks.ceph:Reading keys from all nodes... 2024-09-10T21:24:18.770 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-09-10T21:24:18.771 DEBUG:teuthology.orchestra.run.smithi017:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-09-10T21:24:18.793 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-09-10T21:24:18.793 DEBUG:teuthology.orchestra.run.smithi017:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-09-10T21:24:18.857 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-09-10T21:24:18.857 DEBUG:teuthology.orchestra.run.smithi017:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-09-10T21:24:18.921 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-09-10T21:24:18.921 DEBUG:teuthology.orchestra.run.smithi017:> sudo dd if=/var/lib/ceph/mds/ceph-e/keyring of=/dev/stdout 2024-09-10T21:24:18.985 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-09-10T21:24:18.985 DEBUG:teuthology.orchestra.run.smithi017:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-09-10T21:24:19.053 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-09-10T21:24:19.053 DEBUG:teuthology.orchestra.run.smithi017:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-09-10T21:24:19.117 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-09-10T21:24:19.117 DEBUG:teuthology.orchestra.run.smithi017:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-09-10T21:24:19.185 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-09-10T21:24:19.185 DEBUG:teuthology.orchestra.run.smithi017:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-09-10T21:24:19.249 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-10T21:24:19.249 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-09-10T21:24:19.271 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-10T21:24:19.271 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-09-10T21:24:19.335 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-10T21:24:19.335 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-09-10T21:24:19.403 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-10T21:24:19.403 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/var/lib/ceph/mds/ceph-f/keyring of=/dev/stdout 2024-09-10T21:24:19.467 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-10T21:24:19.467 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-09-10T21:24:19.535 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-10T21:24:19.535 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-09-10T21:24:19.603 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-10T21:24:19.603 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-09-10T21:24:19.666 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-10T21:24:19.666 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-09-10T21:24:19.731 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-09-10T21:24:19.731 DEBUG:teuthology.orchestra.run.smithi017:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-09-10T21:24:19.740 INFO:tasks.ceph:Adding keys to all mons... 2024-09-10T21:24:19.740 DEBUG:teuthology.orchestra.run.smithi017:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-10T21:24:19.787 DEBUG:teuthology.orchestra.run.smithi031:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-10T21:24:19.803 INFO:teuthology.orchestra.run.smithi017.stdout:[mgr.y] 2024-09-10T21:24:19.803 INFO:teuthology.orchestra.run.smithi017.stdout: key = AQDjuOBmvIIgGhAADVzn0Xv9W2YsMj3J3AFAIA== 2024-09-10T21:24:19.803 INFO:teuthology.orchestra.run.smithi017.stdout:[mds.a] 2024-09-10T21:24:19.803 INFO:teuthology.orchestra.run.smithi017.stdout: key = AQDjuOBmHQiCIhAAcgCySKS+7q+8VmWRBJ3Rzg== 2024-09-10T21:24:19.803 INFO:teuthology.orchestra.run.smithi017.stdout:[mds.c] 2024-09-10T21:24:19.803 INFO:teuthology.orchestra.run.smithi017.stdout: key = AQDjuOBm8rnUKRAAi5fIza+WfAcKtQ2ll2rjTg== 2024-09-10T21:24:19.803 INFO:teuthology.orchestra.run.smithi017.stdout:[mds.e] 2024-09-10T21:24:19.804 INFO:teuthology.orchestra.run.smithi017.stdout: key = AQDjuOBmmBBHMRAAFfYcClz+KRkwC4xxvQ1Syw== 2024-09-10T21:24:19.804 INFO:teuthology.orchestra.run.smithi017.stdout:[osd.0] 2024-09-10T21:24:19.804 INFO:teuthology.orchestra.run.smithi017.stdout: key = AQDluOBmDey3HhAASdMcX23b5VhNXkivv9qW2Q== 2024-09-10T21:24:19.804 INFO:teuthology.orchestra.run.smithi017.stdout:[osd.1] 2024-09-10T21:24:19.804 INFO:teuthology.orchestra.run.smithi017.stdout: key = AQDouOBmB193MhAA2FJWO1U4m9V17eOw/eBX5Q== 2024-09-10T21:24:19.804 INFO:teuthology.orchestra.run.smithi017.stdout:[osd.2] 2024-09-10T21:24:19.804 INFO:teuthology.orchestra.run.smithi017.stdout: key = AQDsuOBmS5cWDRAAaHizhERUAJWPPJELez3iIg== 2024-09-10T21:24:19.804 INFO:teuthology.orchestra.run.smithi017.stdout:[osd.3] 2024-09-10T21:24:19.804 INFO:teuthology.orchestra.run.smithi017.stdout: key = AQDvuOBmHJAHIxAAXhZr9CSfAjkIAuU4v7QuGg== 2024-09-10T21:24:19.804 INFO:teuthology.orchestra.run.smithi017.stdout:[mgr.x] 2024-09-10T21:24:19.804 INFO:teuthology.orchestra.run.smithi017.stdout: key = AQDjuOBmOqUgHhAAFTw2FO8xJBqNzMUGrSJVFg== 2024-09-10T21:24:19.804 INFO:teuthology.orchestra.run.smithi017.stdout:[mds.b] 2024-09-10T21:24:19.805 INFO:teuthology.orchestra.run.smithi017.stdout: key = AQDjuOBmcJltNhAAqqilXWGTfhnyzbF49V4Xww== 2024-09-10T21:24:19.805 INFO:teuthology.orchestra.run.smithi017.stdout:[mds.d] 2024-09-10T21:24:19.805 INFO:teuthology.orchestra.run.smithi017.stdout: key = AQDkuOBmxkZSAhAAA+QBUJOqpgukdAtB01KVmQ== 2024-09-10T21:24:19.805 INFO:teuthology.orchestra.run.smithi017.stdout:[mds.f] 2024-09-10T21:24:19.805 INFO:teuthology.orchestra.run.smithi017.stdout: key = AQDkuOBm25SwCRAAUf5smszFIL+OCT/4t+lG+w== 2024-09-10T21:24:19.805 INFO:teuthology.orchestra.run.smithi017.stdout:[osd.4] 2024-09-10T21:24:19.805 INFO:teuthology.orchestra.run.smithi017.stdout: key = AQD1uOBm6ZiUIxAAUVbfnZ5AcQCSkMGJIdWeWA== 2024-09-10T21:24:19.805 INFO:teuthology.orchestra.run.smithi017.stdout:[osd.5] 2024-09-10T21:24:19.805 INFO:teuthology.orchestra.run.smithi017.stdout: key = AQD4uOBm1FXuNRAAAOz2yEN4MzI12BIY/i0WlA== 2024-09-10T21:24:19.805 INFO:teuthology.orchestra.run.smithi017.stdout:[osd.6] 2024-09-10T21:24:19.805 INFO:teuthology.orchestra.run.smithi017.stdout: key = AQD8uOBmp4W4CxAARgusVlxKwwggLYdkBCW6iA== 2024-09-10T21:24:19.805 INFO:teuthology.orchestra.run.smithi017.stdout:[osd.7] 2024-09-10T21:24:19.805 INFO:teuthology.orchestra.run.smithi017.stdout: key = AQD/uOBm8OfTIRAAuzRclqceTLRxyeGucuevkg== 2024-09-10T21:24:19.805 INFO:teuthology.orchestra.run.smithi017.stdout:[client.0] 2024-09-10T21:24:19.805 INFO:teuthology.orchestra.run.smithi017.stdout: key = AQDkuOBmQe6vDRAAh+/VUtxI1f1rfvp3JkHAhw== 2024-09-10T21:24:19.806 INFO:teuthology.orchestra.run.smithi031.stdout:[mgr.y] 2024-09-10T21:24:19.806 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQDjuOBmvIIgGhAADVzn0Xv9W2YsMj3J3AFAIA== 2024-09-10T21:24:19.806 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.a] 2024-09-10T21:24:19.806 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQDjuOBmHQiCIhAAcgCySKS+7q+8VmWRBJ3Rzg== 2024-09-10T21:24:19.806 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.c] 2024-09-10T21:24:19.806 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQDjuOBm8rnUKRAAi5fIza+WfAcKtQ2ll2rjTg== 2024-09-10T21:24:19.806 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.e] 2024-09-10T21:24:19.806 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQDjuOBmmBBHMRAAFfYcClz+KRkwC4xxvQ1Syw== 2024-09-10T21:24:19.807 INFO:teuthology.orchestra.run.smithi031.stdout:[osd.0] 2024-09-10T21:24:19.807 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQDluOBmDey3HhAASdMcX23b5VhNXkivv9qW2Q== 2024-09-10T21:24:19.807 INFO:teuthology.orchestra.run.smithi031.stdout:[osd.1] 2024-09-10T21:24:19.807 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQDouOBmB193MhAA2FJWO1U4m9V17eOw/eBX5Q== 2024-09-10T21:24:19.807 INFO:teuthology.orchestra.run.smithi031.stdout:[osd.2] 2024-09-10T21:24:19.807 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQDsuOBmS5cWDRAAaHizhERUAJWPPJELez3iIg== 2024-09-10T21:24:19.807 INFO:teuthology.orchestra.run.smithi031.stdout:[osd.3] 2024-09-10T21:24:19.807 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQDvuOBmHJAHIxAAXhZr9CSfAjkIAuU4v7QuGg== 2024-09-10T21:24:19.807 INFO:teuthology.orchestra.run.smithi031.stdout:[mgr.x] 2024-09-10T21:24:19.807 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQDjuOBmOqUgHhAAFTw2FO8xJBqNzMUGrSJVFg== 2024-09-10T21:24:19.807 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.b] 2024-09-10T21:24:19.807 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQDjuOBmcJltNhAAqqilXWGTfhnyzbF49V4Xww== 2024-09-10T21:24:19.807 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.d] 2024-09-10T21:24:19.807 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQDkuOBmxkZSAhAAA+QBUJOqpgukdAtB01KVmQ== 2024-09-10T21:24:19.808 INFO:teuthology.orchestra.run.smithi031.stdout:[mds.f] 2024-09-10T21:24:19.808 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQDkuOBm25SwCRAAUf5smszFIL+OCT/4t+lG+w== 2024-09-10T21:24:19.808 INFO:teuthology.orchestra.run.smithi031.stdout:[osd.4] 2024-09-10T21:24:19.808 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQD1uOBm6ZiUIxAAUVbfnZ5AcQCSkMGJIdWeWA== 2024-09-10T21:24:19.808 INFO:teuthology.orchestra.run.smithi031.stdout:[osd.5] 2024-09-10T21:24:19.808 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQD4uOBm1FXuNRAAAOz2yEN4MzI12BIY/i0WlA== 2024-09-10T21:24:19.808 INFO:teuthology.orchestra.run.smithi031.stdout:[osd.6] 2024-09-10T21:24:19.808 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQD8uOBmp4W4CxAARgusVlxKwwggLYdkBCW6iA== 2024-09-10T21:24:19.808 INFO:teuthology.orchestra.run.smithi031.stdout:[osd.7] 2024-09-10T21:24:19.808 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQD/uOBm8OfTIRAAuzRclqceTLRxyeGucuevkg== 2024-09-10T21:24:19.808 INFO:teuthology.orchestra.run.smithi031.stdout:[client.0] 2024-09-10T21:24:19.809 INFO:teuthology.orchestra.run.smithi031.stdout: key = AQDkuOBmQe6vDRAAh+/VUtxI1f1rfvp3JkHAhw== 2024-09-10T21:24:19.810 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-10T21:24:19.859 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-10T21:24:19.918 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-10T21:24:19.922 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-10T21:24:19.983 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-10T21:24:19.986 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-10T21:24:20.045 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.e --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-10T21:24:20.048 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.e --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-10T21:24:20.106 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T21:24:20.110 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T21:24:20.165 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T21:24:20.168 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T21:24:20.222 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T21:24:20.226 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T21:24:20.281 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T21:24:20.284 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T21:24:20.338 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-10T21:24:20.341 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-10T21:24:20.395 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-10T21:24:20.399 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-10T21:24:20.456 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-10T21:24:20.460 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-10T21:24:20.517 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.f --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-10T21:24:20.521 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.f --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-10T21:24:20.579 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T21:24:20.582 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T21:24:20.637 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T21:24:20.640 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T21:24:20.695 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T21:24:20.698 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T21:24:20.753 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T21:24:20.757 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-10T21:24:20.811 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-10T21:24:20.814 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-10T21:24:20.873 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-09-10T21:24:20.873 DEBUG:teuthology.orchestra.run.smithi017:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-09-10T21:24:20.893 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-10T21:24:21.009 DEBUG:teuthology.orchestra.run.smithi017:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-09-10T21:24:21.029 DEBUG:teuthology.orchestra.run.smithi031:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-09-10T21:24:21.049 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-10T21:24:21.159 DEBUG:teuthology.orchestra.run.smithi031:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-09-10T21:24:21.178 DEBUG:teuthology.orchestra.run.smithi031:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-09-10T21:24:21.241 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-10T21:24:21.350 DEBUG:teuthology.orchestra.run.smithi031:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-09-10T21:24:21.370 DEBUG:teuthology.orchestra.run.smithi017:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-10T21:24:21.373 DEBUG:teuthology.orchestra.run.smithi031:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-10T21:24:21.421 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-09-10T21:24:21.422 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-10T21:24:21.422 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-10T21:24:21.425 INFO:tasks.ceph.mon.a:Started 2024-09-10T21:24:21.425 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-10T21:24:21.425 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-10T21:24:21.464 INFO:tasks.ceph.mon.b:Started 2024-09-10T21:24:21.464 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-10T21:24:21.464 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-10T21:24:21.467 INFO:tasks.ceph.mon.c:Started 2024-09-10T21:24:21.467 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-09-10T21:24:21.467 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-09-10T21:24:21.467 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-09-10T21:24:21.470 INFO:tasks.ceph.mgr.y:Started 2024-09-10T21:24:21.471 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-09-10T21:24:21.471 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-09-10T21:24:21.473 INFO:tasks.ceph.mgr.x:Started 2024-09-10T21:24:21.474 INFO:tasks.ceph:setting config [client] client_mount_timeout = 600 2024-09-10T21:24:21.474 DEBUG:teuthology.orchestra.run.smithi017:> sudo ceph --cluster ceph config set client client_mount_timeout 600 2024-09-10T21:24:21.477 INFO:tasks.ceph:setting config [client] debug_client = 20 2024-09-10T21:24:21.477 DEBUG:teuthology.orchestra.run.smithi017:> sudo ceph --cluster ceph config set client debug_client 20 2024-09-10T21:24:21.523 INFO:tasks.ceph:setting config [client] debug_ms = 1 2024-09-10T21:24:21.523 DEBUG:teuthology.orchestra.run.smithi017:> sudo ceph --cluster ceph config set client debug_ms 1 2024-09-10T21:24:21.571 INFO:tasks.ceph:setting config [client] rados_mon_op_timeout = 900 2024-09-10T21:24:21.571 DEBUG:teuthology.orchestra.run.smithi017:> sudo ceph --cluster ceph config set client rados_mon_op_timeout 900 2024-09-10T21:24:21.619 INFO:tasks.ceph:setting config [client] rados_osd_op_timeout = 900 2024-09-10T21:24:21.619 DEBUG:teuthology.orchestra.run.smithi017:> sudo ceph --cluster ceph config set client rados_osd_op_timeout 900 2024-09-10T21:24:21.666 INFO:tasks.ceph:setting config [mds] debug_mds = 20 2024-09-10T21:24:21.666 DEBUG:teuthology.orchestra.run.smithi017:> sudo ceph --cluster ceph config set mds debug_mds 20 2024-09-10T21:24:21.715 INFO:tasks.ceph:setting config [mds] debug_mds_balancer = 20 2024-09-10T21:24:21.715 DEBUG:teuthology.orchestra.run.smithi017:> sudo ceph --cluster ceph config set mds debug_mds_balancer 20 2024-09-10T21:24:21.763 INFO:tasks.ceph:setting config [mds] debug_ms = 2 2024-09-10T21:24:21.763 DEBUG:teuthology.orchestra.run.smithi017:> sudo ceph --cluster ceph config set mds debug_ms 2 2024-09-10T21:24:21.810 INFO:tasks.ceph:setting config [mds] mds_bal_fragment_size_max = 10000 2024-09-10T21:24:21.810 DEBUG:teuthology.orchestra.run.smithi017:> sudo ceph --cluster ceph config set mds mds_bal_fragment_size_max 10000 2024-09-10T21:24:21.859 INFO:tasks.ceph:setting config [mds] mds_bal_merge_size = 5 2024-09-10T21:24:21.859 DEBUG:teuthology.orchestra.run.smithi017:> sudo ceph --cluster ceph config set mds mds_bal_merge_size 5 2024-09-10T21:24:21.911 INFO:tasks.ceph:setting config [mds] mds_bal_split_bits = 3 2024-09-10T21:24:21.911 DEBUG:teuthology.orchestra.run.smithi017:> sudo ceph --cluster ceph config set mds mds_bal_split_bits 3 2024-09-10T21:24:21.963 INFO:tasks.ceph:setting config [mds] mds_bal_split_size = 100 2024-09-10T21:24:21.963 DEBUG:teuthology.orchestra.run.smithi017:> sudo ceph --cluster ceph config set mds mds_bal_split_size 100 2024-09-10T21:24:21.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:21.971+0000 7f8480c00640 1 -- 172.21.15.17:0/771397853 <== mon.2 v2:172.21.15.31:3301/0 10 ==== mon_command_ack([{prefix=config set, name=debug_ms}]=0 v4)=0 v4) ==== 109+0+0 (secure 0 0 0) 0x7f8478010d50 con 0x7f846c004810 2024-09-10T21:24:21.982 INFO:tasks.ceph:setting config [mds] mds_debug_frag = True 2024-09-10T21:24:21.982 DEBUG:teuthology.orchestra.run.smithi017:> sudo ceph --cluster ceph config set mds mds_debug_frag True 2024-09-10T21:24:21.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:21.983+0000 7f8489e00640 1 -- 172.21.15.17:0/771397853 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f846c004810 msgr2=0x7f84841534f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:21.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:21.983+0000 7f8489e00640 1 --2- 172.21.15.17:0/771397853 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f846c004810 0x7f84841534f0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f8478006660 tx=0x7f84780039d0 comp rx=0 tx=0).stop 2024-09-10T21:24:21.991 INFO:tasks.ceph:setting config [mds] mds_debug_scatterstat = True 2024-09-10T21:24:21.991 DEBUG:teuthology.orchestra.run.smithi017:> sudo ceph --cluster ceph config set mds mds_debug_scatterstat True 2024-09-10T21:24:21.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:21.983+0000 7fe125600640 1 -- 172.21.15.17:0/596496329 <== mon.2 v2:172.21.15.31:3301/0 10 ==== config(4 keys) ==== 110+0+0 (secure 0 0 0) 0x7fe114004d80 con 0x7fe12806f580 2024-09-10T21:24:21.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:21.983+0000 7fe125600640 1 -- 172.21.15.17:0/596496329 <== mon.2 v2:172.21.15.31:3301/0 11 ==== mon_command_ack([{prefix=config set, name=debug_client}]=0 v5)=0 v5) ==== 114+0+0 (secure 0 0 0) 0x7fe114016030 con 0x7fe12806f580 2024-09-10T21:24:21.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:21.983+0000 7f8489e00640 1 -- 172.21.15.17:0/771397853 shutdown_connections 2024-09-10T21:24:21.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:21.983+0000 7f8489e00640 1 --2- 172.21.15.17:0/771397853 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f84841f4e30 0x7f84840c55f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:21.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:21.983+0000 7f8489e00640 1 --2- 172.21.15.17:0/771397853 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f848415e520 0x7f84841f48f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:21.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:21.983+0000 7f8489e00640 1 --2- 172.21.15.17:0/771397853 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f846c004810 0x7f84841534f0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:21.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:21.983+0000 7f8489e00640 1 -- 172.21.15.17:0/771397853 >> 172.21.15.17:0/771397853 conn(0x7f848406cdf0 msgr2=0x7f8484159ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:21.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:21.983+0000 7f8489e00640 1 -- 172.21.15.17:0/771397853 shutdown_connections 2024-09-10T21:24:21.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:21.983+0000 7f8489e00640 1 -- 172.21.15.17:0/771397853 wait complete. 2024-09-10T21:24:21.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:21.983+0000 7fbecbe00640 1 -- 172.21.15.17:0/1497349577 <== mon.2 v2:172.21.15.31:3301/0 10 ==== config(4 keys) ==== 110+0+0 (secure 0 0 0) 0x7fbec0010c40 con 0x7fbed4157230 2024-09-10T21:24:21.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:21.987+0000 7fe11aa00640 1 -- 172.21.15.17:0/596496329 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe12806f580 msgr2=0x7fe12814c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:21.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:21.987+0000 7fe11aa00640 1 --2- 172.21.15.17:0/596496329 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe12806f580 0x7fe12814c920 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fe11400e6c0 tx=0x7fe11400eb80 comp rx=0 tx=0).stop 2024-09-10T21:24:21.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:21.987+0000 7fe11aa00640 1 -- 172.21.15.17:0/596496329 shutdown_connections 2024-09-10T21:24:21.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:21.987+0000 7fe11aa00640 1 --2- 172.21.15.17:0/596496329 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe12814ce60 0x7fe12814a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:21.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:21.987+0000 7fe11aa00640 1 --2- 172.21.15.17:0/596496329 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe12806f580 0x7fe12814c920 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:21.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:21.987+0000 7fe11aa00640 1 --2- 172.21.15.17:0/596496329 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe1100051d0 0x7fe1281561d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:21.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:21.987+0000 7fe11aa00640 1 -- 172.21.15.17:0/596496329 >> 172.21.15.17:0/596496329 conn(0x7fe12806d220 msgr2=0x7fe1281521e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:21.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:21.991+0000 7fe11aa00640 1 -- 172.21.15.17:0/596496329 shutdown_connections 2024-09-10T21:24:21.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:21.991+0000 7fe11aa00640 1 -- 172.21.15.17:0/596496329 wait complete. 2024-09-10T21:24:22.022 INFO:tasks.ceph:setting config [mds] mds_op_complaint_time = 180 2024-09-10T21:24:22.022 DEBUG:teuthology.orchestra.run.smithi017:> sudo ceph --cluster ceph config set mds mds_op_complaint_time 180 2024-09-10T21:24:22.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.031+0000 7f643d800640 1 -- 172.21.15.17:0/3404256608 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f643806fdf0 msgr2=0x7f643815aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:22.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.031+0000 7f643d800640 1 --2- 172.21.15.17:0/3404256608 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f643806fdf0 0x7f643815aec0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f643815e620 tx=0x7f643001f3b0 comp rx=0 tx=0).stop 2024-09-10T21:24:22.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.035+0000 7f643d800640 1 -- 172.21.15.17:0/3404256608 shutdown_connections 2024-09-10T21:24:22.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.035+0000 7f643d800640 1 --2- 172.21.15.17:0/3404256608 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f64380b1020 0x7f643814fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:22.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.035+0000 7f643d800640 1 --2- 172.21.15.17:0/3404256608 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f643806fdf0 0x7f643815aec0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:22.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.035+0000 7f643d800640 1 -- 172.21.15.17:0/3404256608 >> 172.21.15.17:0/3404256608 conn(0x7f643806e410 msgr2=0x7f64380cbdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:22.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.035+0000 7f643d800640 1 -- 172.21.15.17:0/3404256608 shutdown_connections 2024-09-10T21:24:22.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.035+0000 7f643d800640 1 -- 172.21.15.17:0/3404256608 wait complete. 2024-09-10T21:24:22.041 INFO:tasks.ceph:setting config [mds] mds_verify_scatter = True 2024-09-10T21:24:22.041 DEBUG:teuthology.orchestra.run.smithi017:> sudo ceph --cluster ceph config set mds mds_verify_scatter True 2024-09-10T21:24:22.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.035+0000 7f88afe00640 1 -- 172.21.15.17:0/3006378200 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f88a80c2230 msgr2=0x7f88a80c2610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:22.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.035+0000 7f88afe00640 1 --2- 172.21.15.17:0/3006378200 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f88a80c2230 0x7f88a80c2610 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f889c009840 tx=0x7f889c01c860 comp rx=0 tx=0).stop 2024-09-10T21:24:22.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.039+0000 7f88afe00640 1 -- 172.21.15.17:0/3006378200 shutdown_connections 2024-09-10T21:24:22.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.039+0000 7f88afe00640 1 --2- 172.21.15.17:0/3006378200 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f88a80c6ab0 0x7f88a8157ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:22.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.039+0000 7f88afe00640 1 --2- 172.21.15.17:0/3006378200 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f88a80c2230 0x7f88a80c2610 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:22.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.039+0000 7f88afe00640 1 -- 172.21.15.17:0/3006378200 >> 172.21.15.17:0/3006378200 conn(0x7f88a8070110 msgr2=0x7f88a8070520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:22.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.039+0000 7f643d800640 1 Processor -- start 2024-09-10T21:24:22.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.039+0000 7f643d800640 1 -- start start 2024-09-10T21:24:22.046 INFO:tasks.ceph:setting config [mds] mds_dir_prefetch = True 2024-09-10T21:24:22.046 DEBUG:teuthology.orchestra.run.smithi017:> sudo ceph --cluster ceph config set mds mds_dir_prefetch True 2024-09-10T21:24:22.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.047+0000 7f643d800640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f643806fdf0 0x7f64380ae860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:22.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.047+0000 7f643d800640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f64380701d0 0x7f64380aaea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:22.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.047+0000 7f643d800640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f64380b1020 0x7f64380ab3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:22.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.047+0000 7f643d800640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f64380ccc60 con 0x7f643806fdf0 2024-09-10T21:24:22.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.047+0000 7f643d800640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f643815e130 con 0x7f64380701d0 2024-09-10T21:24:22.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.047+0000 7f643d800640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f64380c6890 con 0x7f64380b1020 2024-09-10T21:24:22.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.047+0000 7f6436000640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f64380701d0 0x7f64380aaea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:22.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.047+0000 7f6437400640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f64380b1020 0x7f64380ab3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:22.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.047+0000 7f6437400640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f64380b1020 0x7f64380ab3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3301/0 says I am v2:172.21.15.17:51726/0 (socket says 172.21.15.17:51726) 2024-09-10T21:24:22.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.047+0000 7f6436000640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f64380701d0 0x7f64380aaea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.17:47700/0 (socket says 172.21.15.17:47700) 2024-09-10T21:24:22.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.047+0000 7f6437400640 1 -- 172.21.15.17:0/340611349 learned_addr learned my addr 172.21.15.17:0/340611349 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:24:22.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.047+0000 7f6436a00640 1 --2- 172.21.15.17:0/340611349 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f643806fdf0 0x7f64380ae860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:22.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.047+0000 7f6437400640 1 -- 172.21.15.17:0/340611349 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f64380701d0 msgr2=0x7f64380aaea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:22.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.047+0000 7f6437400640 1 --2- 172.21.15.17:0/340611349 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f64380701d0 0x7f64380aaea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:22.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.047+0000 7f6437400640 1 -- 172.21.15.17:0/340611349 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f643806fdf0 msgr2=0x7f64380ae860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:22.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.047+0000 7f6437400640 1 --2- 172.21.15.17:0/340611349 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f643806fdf0 0x7f64380ae860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:22.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.047+0000 7f6437400640 1 -- 172.21.15.17:0/340611349 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64380ac870 con 0x7f64380b1020 2024-09-10T21:24:22.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.047+0000 7f6437400640 1 --2- 172.21.15.17:0/340611349 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f64380b1020 0x7f64380ab3e0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f642000e970 tx=0x7f642000ee30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:22.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.051+0000 7f642ea00640 1 -- 172.21.15.17:0/340611349 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f642000cd20 con 0x7f64380b1020 2024-09-10T21:24:22.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.051+0000 7f642ea00640 1 -- 172.21.15.17:0/340611349 <== mon.2 v2:172.21.15.31:3301/0 2 ==== config(4 keys) ==== 110+0+0 (secure 0 0 0) 0x7f6420004020 con 0x7f64380b1020 2024-09-10T21:24:22.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.051+0000 7f642ea00640 1 -- 172.21.15.17:0/340611349 <== mon.2 v2:172.21.15.31:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6420004430 con 0x7f64380b1020 2024-09-10T21:24:22.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.051+0000 7f643d800640 1 -- 172.21.15.17:0/340611349 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64380acb50 con 0x7f64380b1020 2024-09-10T21:24:22.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.055+0000 7f643d800640 1 -- 172.21.15.17:0/340611349 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f64380aa6a0 con 0x7f64380b1020 2024-09-10T21:24:22.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.055+0000 7f642ea00640 1 -- 172.21.15.17:0/340611349 <== mon.2 v2:172.21.15.31:3301/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7f6420003990 con 0x7f64380b1020 2024-09-10T21:24:22.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.055+0000 7f642ea00640 1 -- 172.21.15.17:0/340611349 <== mon.2 v2:172.21.15.31:3301/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f6420018770 con 0x7f64380b1020 2024-09-10T21:24:22.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.055+0000 7f643d800640 1 -- 172.21.15.17:0/340611349 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64380aa490 con 0x7f64380b1020 2024-09-10T21:24:22.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.039+0000 7f88afe00640 1 -- 172.21.15.17:0/3006378200 shutdown_connections 2024-09-10T21:24:22.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.047+0000 7f88afe00640 1 -- 172.21.15.17:0/3006378200 wait complete. 2024-09-10T21:24:22.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.051+0000 7f88afe00640 1 Processor -- start 2024-09-10T21:24:22.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.059+0000 7f88afe00640 1 -- start start 2024-09-10T21:24:22.065 INFO:tasks.ceph:setting config [mds] osd_op_complaint_time = 180 2024-09-10T21:24:22.065 DEBUG:teuthology.orchestra.run.smithi017:> sudo ceph --cluster ceph config set mds osd_op_complaint_time 180 2024-09-10T21:24:22.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.059+0000 7f642ea00640 1 -- 172.21.15.17:0/340611349 <== mon.2 v2:172.21.15.31:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+76091 (secure 0 0 0) 0x7f6420003e40 con 0x7f64380b1020 2024-09-10T21:24:22.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.063+0000 7f88afe00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f88a8074530 0x7f88a8144870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:22.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.063+0000 7f88afe00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f88a80c6ab0 0x7f88a8144110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:22.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.063+0000 7f88afe00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f88a81615c0 0x7f88a814eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:22.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.063+0000 7f88ad400640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f88a8074530 0x7f88a8144870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:22.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.063+0000 7f88ad400640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f88a8074530 0x7f88a8144870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3301/0 says I am v2:172.21.15.17:51740/0 (socket says 172.21.15.17:51740) 2024-09-10T21:24:22.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.067+0000 7f88ade00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f88a81615c0 0x7f88a814eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:22.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.067+0000 7f88ade00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f88a81615c0 0x7f88a814eaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35634/0 (socket says 172.21.15.17:35634) 2024-09-10T21:24:22.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.067+0000 7f88aca00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f88a80c6ab0 0x7f88a8144110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:22.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.067+0000 7f88afe00640 1 -- 172.21.15.17:0/2543282229 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f88a81577d0 con 0x7f88a81615c0 2024-09-10T21:24:22.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.067+0000 7f88ad400640 1 -- 172.21.15.17:0/2543282229 learned_addr learned my addr 172.21.15.17:0/2543282229 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:24:22.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.067+0000 7f88afe00640 1 -- 172.21.15.17:0/2543282229 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f88a80c1430 con 0x7f88a80c6ab0 2024-09-10T21:24:22.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.067+0000 7f88afe00640 1 -- 172.21.15.17:0/2543282229 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f88a8161bd0 con 0x7f88a8074530 2024-09-10T21:24:22.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.067+0000 7f88ade00640 1 -- 172.21.15.17:0/2543282229 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f88a8074530 msgr2=0x7f88a8144870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:22.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.067+0000 7f88ade00640 1 --2- 172.21.15.17:0/2543282229 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f88a8074530 0x7f88a8144870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:22.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.067+0000 7f88ade00640 1 -- 172.21.15.17:0/2543282229 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f88a80c6ab0 msgr2=0x7f88a8144110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:22.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.067+0000 7f88ade00640 1 --2- 172.21.15.17:0/2543282229 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f88a80c6ab0 0x7f88a8144110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:22.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.067+0000 7f88ade00640 1 -- 172.21.15.17:0/2543282229 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88a8148ea0 con 0x7f88a81615c0 2024-09-10T21:24:22.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.067+0000 7f88ade00640 1 --2- 172.21.15.17:0/2543282229 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f88a81615c0 0x7f88a814eaa0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f88a000ca70 tx=0x7f88a000cb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:22.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.067+0000 7f88a5600640 1 -- 172.21.15.17:0/2543282229 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88a0016020 con 0x7f88a81615c0 2024-09-10T21:24:22.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.067+0000 7f88a5600640 1 -- 172.21.15.17:0/2543282229 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(4 keys) ==== 110+0+0 (secure 0 0 0) 0x7f88a000a8b0 con 0x7f88a81615c0 2024-09-10T21:24:22.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.067+0000 7f88a5600640 1 -- 172.21.15.17:0/2543282229 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88a000ace0 con 0x7f88a81615c0 2024-09-10T21:24:22.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.071+0000 7f88afe00640 1 -- 172.21.15.17:0/2543282229 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88a8144650 con 0x7f88a81615c0 2024-09-10T21:24:22.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.071+0000 7f88afe00640 1 -- 172.21.15.17:0/2543282229 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88a8149140 con 0x7f88a81615c0 2024-09-10T21:24:22.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.075+0000 7f88a5600640 1 -- 172.21.15.17:0/2543282229 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7f88a00127b0 con 0x7f88a81615c0 2024-09-10T21:24:22.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.075+0000 7f88a5600640 1 -- 172.21.15.17:0/2543282229 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f88a001bbb0 con 0x7f88a81615c0 2024-09-10T21:24:22.083 INFO:tasks.ceph:setting config [mds] rados_mon_op_timeout = 900 2024-09-10T21:24:22.083 DEBUG:teuthology.orchestra.run.smithi017:> sudo ceph --cluster ceph config set mds rados_mon_op_timeout 900 2024-09-10T21:24:22.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.075+0000 7f88afe00640 1 -- 172.21.15.17:0/2543282229 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88a80c2610 con 0x7f88a81615c0 2024-09-10T21:24:22.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.083+0000 7f88a5600640 1 -- 172.21.15.17:0/2543282229 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+76091 (secure 0 0 0) 0x7f88a0020020 con 0x7f88a81615c0 2024-09-10T21:24:22.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.095+0000 7fbed9400640 1 -- 172.21.15.17:0/1497349577 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_command([{prefix=config set, name=rados_osd_op_timeout}] v 0) -- 0x7fbea8005320 con 0x7fbed4157230 2024-09-10T21:24:22.100 INFO:tasks.ceph:setting config [mds] rados_osd_op_timeout = 900 2024-09-10T21:24:22.100 DEBUG:teuthology.orchestra.run.smithi017:> sudo ceph --cluster ceph config set mds rados_osd_op_timeout 900 2024-09-10T21:24:22.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.123+0000 7fbecbe00640 1 -- 172.21.15.17:0/1497349577 <== mon.2 v2:172.21.15.31:3301/0 11 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbec0024430 con 0x7fbed4157230 2024-09-10T21:24:22.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.123+0000 7fbecbe00640 1 -- 172.21.15.17:0/1497349577 <== mon.2 v2:172.21.15.31:3301/0 12 ==== mon_command_ack([{prefix=config set, name=rados_osd_op_timeout}]=0 v6)=0 v6) ==== 123+0+0 (secure 0 0 0) 0x7fbec00248b0 con 0x7fbed4157230 2024-09-10T21:24:22.131 INFO:tasks.ceph:setting config [mon] mon_op_complaint_time = 120 2024-09-10T21:24:22.131 DEBUG:teuthology.orchestra.run.smithi017:> sudo ceph --cluster ceph config set mon mon_op_complaint_time 120 2024-09-10T21:24:22.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.123+0000 7f642ea00640 1 -- 172.21.15.17:0/340611349 <== mon.2 v2:172.21.15.31:3301/0 7 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f64200094a0 con 0x7f64380b1020 2024-09-10T21:24:22.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.123+0000 7f88a5600640 1 -- 172.21.15.17:0/2543282229 <== mon.0 v2:172.21.15.17:3300/0 7 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f88a0012d00 con 0x7f88a81615c0 2024-09-10T21:24:22.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.143+0000 7fbed9400640 1 -- 172.21.15.17:0/1497349577 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fbed4157230 msgr2=0x7fbed406d130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:22.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.143+0000 7fbed9400640 1 --2- 172.21.15.17:0/1497349577 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fbed4157230 0x7fbed406d130 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fbec00038f0 tx=0x7fbec0003db0 comp rx=0 tx=0).stop 2024-09-10T21:24:22.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.147+0000 7fb1b1400640 1 -- 172.21.15.17:0/15185891 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fb1a4050730 msgr2=0x7fb1a4034e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:22.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.147+0000 7fb1b1400640 1 --2- 172.21.15.17:0/15185891 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fb1a4050730 0x7fb1a4034e40 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fb1a40dc3b0 tx=0x7fb1a00207d0 comp rx=0 tx=0).stop 2024-09-10T21:24:22.152 INFO:tasks.ceph:setting config [mon] mon_warn_on_pool_no_app = False 2024-09-10T21:24:22.153 DEBUG:teuthology.orchestra.run.smithi017:> sudo ceph --cluster ceph config set mon mon_warn_on_pool_no_app False 2024-09-10T21:24:22.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.151+0000 7fbed9400640 1 -- 172.21.15.17:0/1497349577 shutdown_connections 2024-09-10T21:24:22.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.151+0000 7fbed9400640 1 --2- 172.21.15.17:0/1497349577 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbed414d3e0 0x7fbed406d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:22.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.151+0000 7fbed9400640 1 --2- 172.21.15.17:0/1497349577 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fbed4157230 0x7fbed406d130 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:22.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.151+0000 7fbed9400640 1 --2- 172.21.15.17:0/1497349577 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbed40748e0 0x7fbed4153870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:22.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.151+0000 7fbed9400640 1 -- 172.21.15.17:0/1497349577 >> 172.21.15.17:0/1497349577 conn(0x7fbed40ccee0 msgr2=0x7fbed40cd2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:22.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.163+0000 7fbed9400640 1 -- 172.21.15.17:0/1497349577 shutdown_connections 2024-09-10T21:24:22.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.163+0000 7fbed9400640 1 -- 172.21.15.17:0/1497349577 wait complete. 2024-09-10T21:24:22.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.159+0000 7fb1b1400640 1 -- 172.21.15.17:0/15185891 shutdown_connections 2024-09-10T21:24:22.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.159+0000 7fb1b1400640 1 --2- 172.21.15.17:0/15185891 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fb1a4050730 0x7fb1a4034e40 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:22.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.159+0000 7fb1b1400640 1 --2- 172.21.15.17:0/15185891 >> v2:172.21.15.31:3300/0 conn(0x7fb1a4045cd0 0x7fb1a4034900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:22.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.159+0000 7fb1b1400640 1 -- 172.21.15.17:0/15185891 >> 172.21.15.17:0/15185891 conn(0x7fb1a404fc10 msgr2=0x7fb1a4050020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:22.171 INFO:tasks.ceph:setting config [osd] osd_op_complaint_time = 180 2024-09-10T21:24:22.171 DEBUG:teuthology.orchestra.run.smithi017:> sudo ceph --cluster ceph config set osd osd_op_complaint_time 180 2024-09-10T21:24:22.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.167+0000 7fb1b1400640 1 -- 172.21.15.17:0/15185891 shutdown_connections 2024-09-10T21:24:22.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.167+0000 7fb1b1400640 1 -- 172.21.15.17:0/15185891 wait complete. 2024-09-10T21:24:22.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.167+0000 7fb1b1400640 1 Processor -- start 2024-09-10T21:24:22.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.167+0000 7fb1b1400640 1 -- start start 2024-09-10T21:24:22.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.171+0000 7fb1b1400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1a4045cd0 0x7fb1a4047c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:22.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.171+0000 7fb1b1400640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fb1a4050730 0x7fb1a40ceab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:22.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.171+0000 7fb1b1400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb1a40d52e0 0x7fb1a40d3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:22.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.171+0000 7fb1b1400640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fb1a40db280 con 0x7fb1a40d52e0 2024-09-10T21:24:22.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.171+0000 7fb1b1400640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fb1a4033130 con 0x7fb1a4045cd0 2024-09-10T21:24:22.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.171+0000 7fb1b1400640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7fb1a40db400 con 0x7fb1a4050730 2024-09-10T21:24:22.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.171+0000 7fb1abe00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1a4045cd0 0x7fb1a4047c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:22.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.171+0000 7fb1ab400640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fb1a4050730 0x7fb1a40ceab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:22.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.171+0000 7fb1abe00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1a4045cd0 0x7fb1a4047c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.17:47718/0 (socket says 172.21.15.17:47718) 2024-09-10T21:24:22.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.171+0000 7fb1abe00640 1 -- 172.21.15.17:0/2757040539 learned_addr learned my addr 172.21.15.17:0/2757040539 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:24:22.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.171+0000 7fb1abe00640 1 -- 172.21.15.17:0/2757040539 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fb1a4050730 msgr2=0x7fb1a40ceab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:22.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.171+0000 7fb1abe00640 1 --2- 172.21.15.17:0/2757040539 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fb1a4050730 0x7fb1a40ceab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:22.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.171+0000 7fb1abe00640 1 -- 172.21.15.17:0/2757040539 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb1a40d52e0 msgr2=0x7fb1a40d3740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:22.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.171+0000 7fb1abe00640 1 --2- 172.21.15.17:0/2757040539 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb1a40d52e0 0x7fb1a40d3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:22.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.171+0000 7fb1abe00640 1 -- 172.21.15.17:0/2757040539 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1a40d3c80 con 0x7fb1a4045cd0 2024-09-10T21:24:22.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.171+0000 7fb1abe00640 1 --2- 172.21.15.17:0/2757040539 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1a4045cd0 0x7fb1a4047c40 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fb19800c4b0 tx=0x7fb19800c8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:22.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.175+0000 7fb19fe00640 1 -- 172.21.15.17:0/2757040539 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1980068a0 con 0x7fb1a4045cd0 2024-09-10T21:24:22.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.175+0000 7fb19fe00640 1 -- 172.21.15.17:0/2757040539 <== mon.1 v2:172.21.15.31:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb198006a60 con 0x7fb1a4045cd0 2024-09-10T21:24:22.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.175+0000 7fb19fe00640 1 -- 172.21.15.17:0/2757040539 <== mon.1 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb198007020 con 0x7fb1a4045cd0 2024-09-10T21:24:22.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.179+0000 7fb1b1400640 1 -- 172.21.15.17:0/2757040539 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1a4184950 con 0x7fb1a4045cd0 2024-09-10T21:24:22.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.179+0000 7fb1b1400640 1 -- 172.21.15.17:0/2757040539 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1a4047440 con 0x7fb1a4045cd0 2024-09-10T21:24:22.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.179+0000 7fb1b1400640 1 -- 172.21.15.17:0/2757040539 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1a4006da0 con 0x7fb1a4045cd0 2024-09-10T21:24:22.187 DEBUG:tasks.ceph:set 23 configs 2024-09-10T21:24:22.187 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'client_mount_timeout', '600'], check_status=True, hostname='smithi017') 2024-09-10T21:24:22.188 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'debug_client', '20'], check_status=True, hostname='smithi017') 2024-09-10T21:24:22.188 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'debug_ms', '1'], check_status=True, hostname='smithi017') 2024-09-10T21:24:22.188 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'rados_mon_op_timeout', '900'], check_status=True, hostname='smithi017') 2024-09-10T21:24:22.188 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'rados_osd_op_timeout', '900'], check_status=True, hostname='smithi017') 2024-09-10T21:24:22.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.183+0000 7fb19fe00640 1 -- 172.21.15.17:0/2757040539 <== mon.1 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7fb198007550 con 0x7fb1a4045cd0 2024-09-10T21:24:22.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.183+0000 7fb19fe00640 1 -- 172.21.15.17:0/2757040539 <== mon.1 v2:172.21.15.31:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fb1980125a0 con 0x7fb1a4045cd0 2024-09-10T21:24:22.200 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.187+0000 7fb19fe00640 1 -- 172.21.15.17:0/2757040539 <== mon.1 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+76091 (secure 0 0 0) 0x7fb198012950 con 0x7fb1a4045cd0 2024-09-10T21:24:22.289 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'debug_mds', '20'], check_status=True, hostname='smithi017') 2024-09-10T21:24:22.392 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.391+0000 7f88afe00640 1 -- 172.21.15.17:0/2543282229 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command([{prefix=config set, name=debug_mds_balancer}] v 0) -- 0x7f88a80c6790 con 0x7f88a81615c0 2024-09-10T21:24:22.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.407+0000 7f88a5600640 1 -- 172.21.15.17:0/2543282229 <== mon.0 v2:172.21.15.17:3300/0 8 ==== mon_command_ack([{prefix=config set, name=debug_mds_balancer}]=0 v7)=0 v7) ==== 117+0+0 (secure 0 0 0) 0x7f88a001b400 con 0x7f88a81615c0 2024-09-10T21:24:22.411 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.407+0000 7f643d800640 1 -- 172.21.15.17:0/340611349 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_command([{prefix=config set, name=debug_mds}] v 0) -- 0x7f6438069e60 con 0x7f64380b1020 2024-09-10T21:24:22.415 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.411+0000 7ff675400640 1 -- 172.21.15.17:0/3481493525 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff6700c6ab0 msgr2=0x7ff67015e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:22.415 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.411+0000 7ff66d400640 1 -- 172.21.15.17:0/3481493525 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6680ad440 con 0x7ff6700c6ab0 2024-09-10T21:24:22.415 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.411+0000 7ff675400640 1 --2- 172.21.15.17:0/3481493525 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff6700c6ab0 0x7ff67015e720 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7ff668009bc0 tx=0x7ff66801cba0 comp rx=0 tx=0).stop 2024-09-10T21:24:22.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.411+0000 7ff675400640 1 -- 172.21.15.17:0/3481493525 shutdown_connections 2024-09-10T21:24:22.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.411+0000 7ff675400640 1 --2- 172.21.15.17:0/3481493525 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff6700c6ab0 0x7ff67015e720 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:22.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.411+0000 7ff675400640 1 --2- 172.21.15.17:0/3481493525 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff6700c72b0 0x7ff67015e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:22.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.411+0000 7ff675400640 1 -- 172.21.15.17:0/3481493525 >> 172.21.15.17:0/3481493525 conn(0x7ff670070110 msgr2=0x7ff670070520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:22.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.419+0000 7ff675400640 1 -- 172.21.15.17:0/3481493525 shutdown_connections 2024-09-10T21:24:22.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.423+0000 7f2b12800640 1 -- 172.21.15.17:0/2477906233 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f2b0c070780 msgr2=0x7f2b0c0c44a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:22.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.423+0000 7f2b12800640 1 --2- 172.21.15.17:0/2477906233 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f2b0c070780 0x7f2b0c0c44a0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f2b00009930 tx=0x7f2b0001c920 comp rx=0 tx=0).stop 2024-09-10T21:24:22.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.419+0000 7ff675400640 1 -- 172.21.15.17:0/3481493525 wait complete. 2024-09-10T21:24:22.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.423+0000 7ff675400640 1 Processor -- start 2024-09-10T21:24:22.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.423+0000 7f2b12800640 1 -- 172.21.15.17:0/2477906233 shutdown_connections 2024-09-10T21:24:22.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.423+0000 7f2b12800640 1 --2- 172.21.15.17:0/2477906233 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f2b0c070780 0x7f2b0c0c44a0 secure :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f2b00009930 tx=0x7f2b0001c920 comp rx=0 tx=0).stop 2024-09-10T21:24:22.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.423+0000 7f2b12800640 1 -- 172.21.15.17:0/2477906233 >> 172.21.15.17:0/2477906233 conn(0x7f2b0c06fc60 msgr2=0x7f2b0c070070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:22.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.427+0000 7ff675400640 1 -- start start 2024-09-10T21:24:22.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.427+0000 7f88afe00640 1 -- 172.21.15.17:0/2543282229 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f88a81615c0 msgr2=0x7f88a814eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:22.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.427+0000 7f88afe00640 1 --2- 172.21.15.17:0/2543282229 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f88a81615c0 0x7f88a814eaa0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f88a000ca70 tx=0x7f88a000cb60 comp rx=0 tx=0).stop 2024-09-10T21:24:22.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.427+0000 7f2b12800640 1 -- 172.21.15.17:0/2477906233 shutdown_connections 2024-09-10T21:24:22.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.427+0000 7ff675400640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff6700c6ab0 0x7ff6700c3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:22.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.427+0000 7ff675400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff6700c72b0 0x7ff6700c5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:22.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.427+0000 7ff66e800640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff6700c6ab0 0x7ff6700c3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:22.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.427+0000 7ff66e800640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff6700c6ab0 0x7ff6700c3710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3301/0 says I am v2:172.21.15.17:51782/0 (socket says 172.21.15.17:51782) 2024-09-10T21:24:22.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.431+0000 7ff66e800640 1 -- 172.21.15.17:0/785739415 learned_addr learned my addr 172.21.15.17:0/785739415 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:24:22.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.431+0000 7ff675400640 1 --2- 172.21.15.17:0/785739415 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff67015de00 0x7ff6700c45c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:22.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.431+0000 7ff66de00640 1 --2- 172.21.15.17:0/785739415 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff6700c72b0 0x7ff6700c5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:22.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.431+0000 7ff675400640 1 -- 172.21.15.17:0/785739415 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7ff67015af80 con 0x7ff6700c72b0 2024-09-10T21:24:22.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.431+0000 7ff675400640 1 -- 172.21.15.17:0/785739415 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7ff67015ae00 con 0x7ff67015de00 2024-09-10T21:24:22.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.431+0000 7ff675400640 1 -- 172.21.15.17:0/785739415 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7ff67015b100 con 0x7ff6700c6ab0 2024-09-10T21:24:22.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.431+0000 7ff66f200640 1 --2- 172.21.15.17:0/785739415 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff67015de00 0x7ff6700c45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:22.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.431+0000 7ff66f200640 1 -- 172.21.15.17:0/785739415 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff6700c6ab0 msgr2=0x7ff6700c3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:22.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.431+0000 7ff66f200640 1 --2- 172.21.15.17:0/785739415 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff6700c6ab0 0x7ff6700c3710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:22.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.431+0000 7ff66f200640 1 -- 172.21.15.17:0/785739415 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff6700c72b0 msgr2=0x7ff6700c5480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:22.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.431+0000 7ff66f200640 1 --2- 172.21.15.17:0/785739415 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff6700c72b0 0x7ff6700c5480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:22.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.431+0000 7ff66f200640 1 -- 172.21.15.17:0/785739415 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6700ca6e0 con 0x7ff67015de00 2024-09-10T21:24:22.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.431+0000 7ff66f200640 1 --2- 172.21.15.17:0/785739415 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff67015de00 0x7ff6700c45c0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7ff6680ac000 tx=0x7ff668005ae0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:22.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.431+0000 7f2b12800640 1 -- 172.21.15.17:0/2477906233 wait complete. 2024-09-10T21:24:22.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.431+0000 7f2b12800640 1 Processor -- start 2024-09-10T21:24:22.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.431+0000 7ff666a00640 1 -- 172.21.15.17:0/785739415 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6680b2040 con 0x7ff67015de00 2024-09-10T21:24:22.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.431+0000 7ff666a00640 1 -- 172.21.15.17:0/785739415 <== mon.1 v2:172.21.15.31:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff668004040 con 0x7ff67015de00 2024-09-10T21:24:22.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.431+0000 7ff666a00640 1 -- 172.21.15.17:0/785739415 <== mon.1 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff668004490 con 0x7ff67015de00 2024-09-10T21:24:22.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.431+0000 7ff675400640 1 -- 172.21.15.17:0/785739415 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6700ccd10 con 0x7ff67015de00 2024-09-10T21:24:22.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.435+0000 7ff675400640 1 -- 172.21.15.17:0/785739415 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6700caaa0 con 0x7ff67015de00 2024-09-10T21:24:22.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.435+0000 7ff66de00640 1 --2- 172.21.15.17:0/785739415 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff6700c72b0 0x7ff6700c5480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:24:22.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.435+0000 7ff66e800640 1 --2- 172.21.15.17:0/785739415 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff6700c6ab0 0x7ff6700c3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:24:22.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.435+0000 7ff666a00640 1 -- 172.21.15.17:0/785739415 <== mon.1 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7ff6680b5070 con 0x7ff67015de00 2024-09-10T21:24:22.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.435+0000 7ff666a00640 1 -- 172.21.15.17:0/785739415 <== mon.1 v2:172.21.15.31:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7ff6680b6430 con 0x7ff67015de00 2024-09-10T21:24:22.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.435+0000 7ff675400640 1 -- 172.21.15.17:0/785739415 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6700cba30 con 0x7ff67015de00 2024-09-10T21:24:22.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.439+0000 7f88afe00640 1 -- 172.21.15.17:0/2543282229 shutdown_connections 2024-09-10T21:24:22.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.439+0000 7f88afe00640 1 --2- 172.21.15.17:0/2543282229 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f88a81615c0 0x7f88a814eaa0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:22.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.439+0000 7f88afe00640 1 --2- 172.21.15.17:0/2543282229 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f88a80c6ab0 0x7f88a8144110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:22.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.439+0000 7f88afe00640 1 --2- 172.21.15.17:0/2543282229 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f88a8074530 0x7f88a8144870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:22.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.439+0000 7f88afe00640 1 -- 172.21.15.17:0/2543282229 >> 172.21.15.17:0/2543282229 conn(0x7f88a8070110 msgr2=0x7f88a815af10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:22.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.439+0000 7f2b12800640 1 -- start start 2024-09-10T21:24:22.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.439+0000 7f2b12800640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f2b0c06efb0 0x7f2b0c152520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:22.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.439+0000 7f2b12800640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2b0c157720 0x7f2b0c1514c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:22.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.443+0000 7f642ea00640 1 -- 172.21.15.17:0/340611349 <== mon.2 v2:172.21.15.31:3301/0 8 ==== mon_command_ack([{prefix=config set, name=debug_mds}]=0 v8)=0 v8) ==== 108+0+0 (secure 0 0 0) 0x7f6420021d70 con 0x7f64380b1020 2024-09-10T21:24:22.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.439+0000 7f2b12800640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2b0c153250 0x7f2b0c0bed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:22.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.439+0000 7f2b11400640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f2b0c06efb0 0x7f2b0c152520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:22.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.439+0000 7f2b10a00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2b0c157720 0x7f2b0c1514c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:22.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.439+0000 7f2b10a00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2b0c157720 0x7f2b0c1514c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.17:47738/0 (socket says 172.21.15.17:47738) 2024-09-10T21:24:22.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.439+0000 7f2b11400640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f2b0c06efb0 0x7f2b0c152520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3301/0 says I am v2:172.21.15.17:51816/0 (socket says 172.21.15.17:51816) 2024-09-10T21:24:22.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.439+0000 7f2b12800640 1 -- 172.21.15.17:0/618969446 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f2b0c0c70e0 con 0x7f2b0c153250 2024-09-10T21:24:22.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.439+0000 7f2b10a00640 1 -- 172.21.15.17:0/618969446 learned_addr learned my addr 172.21.15.17:0/618969446 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:24:22.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.439+0000 7f2b12800640 1 -- 172.21.15.17:0/618969446 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f2b0c0c7d10 con 0x7f2b0c157720 2024-09-10T21:24:22.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.439+0000 7f2b12800640 1 -- 172.21.15.17:0/618969446 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f2b0c152f90 con 0x7f2b0c06efb0 2024-09-10T21:24:22.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.439+0000 7f2b10a00640 1 -- 172.21.15.17:0/618969446 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f2b0c06efb0 msgr2=0x7f2b0c152520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:22.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.443+0000 7f2b10a00640 1 --2- 172.21.15.17:0/618969446 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f2b0c06efb0 0x7f2b0c152520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:22.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.443+0000 7f2b10a00640 1 -- 172.21.15.17:0/618969446 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2b0c153250 msgr2=0x7f2b0c0bed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:22.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.443+0000 7f2b10a00640 1 --2- 172.21.15.17:0/618969446 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2b0c153250 0x7f2b0c0bed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:22.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.443+0000 7f2b10a00640 1 -- 172.21.15.17:0/618969446 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b0c0cb6c0 con 0x7f2b0c157720 2024-09-10T21:24:22.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.443+0000 7f2b11400640 1 --2- 172.21.15.17:0/618969446 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f2b0c06efb0 0x7f2b0c152520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:24:22.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.443+0000 7f88afe00640 1 -- 172.21.15.17:0/2543282229 shutdown_connections 2024-09-10T21:24:22.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.443+0000 7f88afe00640 1 -- 172.21.15.17:0/2543282229 wait complete. 2024-09-10T21:24:22.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.443+0000 7f643d800640 1 -- 172.21.15.17:0/340611349 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f64380b1020 msgr2=0x7f64380ab3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:22.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.443+0000 7f643d800640 1 --2- 172.21.15.17:0/340611349 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f64380b1020 0x7f64380ab3e0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f642000e970 tx=0x7f642000ee30 comp rx=0 tx=0).stop 2024-09-10T21:24:22.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.439+0000 7ff666a00640 1 -- 172.21.15.17:0/785739415 <== mon.1 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+76091 (secure 0 0 0) 0x7ff6680c4060 con 0x7ff67015de00 2024-09-10T21:24:22.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.447+0000 7f2b10a00640 1 --2- 172.21.15.17:0/618969446 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2b0c157720 0x7f2b0c1514c0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f2afc013f70 tx=0x7f2afc007410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:22.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.447+0000 7f2b09600640 1 -- 172.21.15.17:0/618969446 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2afc00d070 con 0x7f2b0c157720 2024-09-10T21:24:22.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.451+0000 7f643d800640 1 -- 172.21.15.17:0/340611349 shutdown_connections 2024-09-10T21:24:22.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.451+0000 7f643d800640 1 --2- 172.21.15.17:0/340611349 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f64380b1020 0x7f64380ab3e0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:22.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.451+0000 7f643d800640 1 --2- 172.21.15.17:0/340611349 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f64380701d0 0x7f64380aaea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:22.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.451+0000 7f643d800640 1 --2- 172.21.15.17:0/340611349 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f643806fdf0 0x7f64380ae860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:22.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.451+0000 7f643d800640 1 -- 172.21.15.17:0/340611349 >> 172.21.15.17:0/340611349 conn(0x7f643806e410 msgr2=0x7f64380cadd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:22.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.455+0000 7f2b09600640 1 -- 172.21.15.17:0/618969446 <== mon.1 v2:172.21.15.31:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2afc007cd0 con 0x7f2b0c157720 2024-09-10T21:24:22.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.455+0000 7f2b09600640 1 -- 172.21.15.17:0/618969446 <== mon.1 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2afc011660 con 0x7f2b0c157720 2024-09-10T21:24:22.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.455+0000 7f2b12800640 1 -- 172.21.15.17:0/618969446 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b0c0cb9a0 con 0x7f2b0c157720 2024-09-10T21:24:22.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.455+0000 7f2b12800640 1 -- 172.21.15.17:0/618969446 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b0c1487f0 con 0x7f2b0c157720 2024-09-10T21:24:22.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.455+0000 7f2b09600640 1 -- 172.21.15.17:0/618969446 <== mon.1 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7f2afc0043b0 con 0x7f2b0c157720 2024-09-10T21:24:22.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.459+0000 7f643d800640 1 -- 172.21.15.17:0/340611349 shutdown_connections 2024-09-10T21:24:22.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.463+0000 7f2b09600640 1 -- 172.21.15.17:0/618969446 <== mon.1 v2:172.21.15.31:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f2afc02c850 con 0x7f2b0c157720 2024-09-10T21:24:22.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.463+0000 7f643d800640 1 -- 172.21.15.17:0/340611349 wait complete. 2024-09-10T21:24:22.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.463+0000 7f1aa6400640 1 -- 172.21.15.17:0/2176925340 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1a84004810 msgr2=0x7f1a84004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:22.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.463+0000 7f1aa6400640 1 --2- 172.21.15.17:0/2176925340 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1a84004810 0x7f1a84004c90 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f1aa00c39b0 tx=0x7f1a9001c9c0 comp rx=0 tx=0).stop 2024-09-10T21:24:22.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.463+0000 7f1a9fe00640 1 -- 172.21.15.17:0/2176925340 reap_dead start 2024-09-10T21:24:22.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.463+0000 7f2afaa00640 1 -- 172.21.15.17:0/618969446 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b0c0c44a0 con 0x7f2b0c157720 2024-09-10T21:24:22.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.467+0000 7f1aa6400640 1 -- 172.21.15.17:0/2176925340 shutdown_connections 2024-09-10T21:24:22.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.467+0000 7f1aa6400640 1 -- 172.21.15.17:0/2176925340 >> 172.21.15.17:0/2176925340 conn(0x7f1aa00d0060 msgr2=0x7f1aa00d0470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:22.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.471+0000 7f2b09600640 1 -- 172.21.15.17:0/618969446 <== mon.1 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+76091 (secure 0 0 0) 0x7f2afc031060 con 0x7f2b0c157720 2024-09-10T21:24:22.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.475+0000 7f1aa6400640 1 -- 172.21.15.17:0/2176925340 shutdown_connections 2024-09-10T21:24:22.481 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.479+0000 7f1aa6400640 1 -- 172.21.15.17:0/2176925340 wait complete. 2024-09-10T21:24:22.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.483+0000 7f1aa6400640 1 Processor -- start 2024-09-10T21:24:22.488 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.483+0000 7f1aa6400640 1 -- start start 2024-09-10T21:24:22.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.487+0000 7f1aa6400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1a84004810 0x7f1aa00cf320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:22.490 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.487+0000 7f1aa6400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1aa0155d30 0x7f1aa01561b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:22.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.487+0000 7f1aa6400640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f1aa00c5bc0 0x7f1aa00c6040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:22.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.495+0000 7f1aa6400640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f1aa01692d0 con 0x7f1aa0155d30 2024-09-10T21:24:22.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.495+0000 7f1aa6400640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f1aa015c440 con 0x7f1a84004810 2024-09-10T21:24:22.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.495+0000 7f1aa6400640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f1aa0169450 con 0x7f1aa00c5bc0 2024-09-10T21:24:22.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.487+0000 7f1a9f400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1a84004810 0x7f1aa00cf320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:22.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.491+0000 7f1a96a00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1aa0155d30 0x7f1aa01561b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:22.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.495+0000 7f1a9f400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1a84004810 0x7f1aa00cf320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.17:47746/0 (socket says 172.21.15.17:47746) 2024-09-10T21:24:22.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.495+0000 7f1a9f400640 1 -- 172.21.15.17:0/1365788862 learned_addr learned my addr 172.21.15.17:0/1365788862 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:24:22.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.495+0000 7f1a9fe00640 1 --2- 172.21.15.17:0/1365788862 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f1aa00c5bc0 0x7f1aa00c6040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:22.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.495+0000 7f1a9f400640 1 -- 172.21.15.17:0/1365788862 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f1aa00c5bc0 msgr2=0x7f1aa00c6040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:22.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.495+0000 7f1a9f400640 1 --2- 172.21.15.17:0/1365788862 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f1aa00c5bc0 0x7f1aa00c6040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:22.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.495+0000 7f1a9f400640 1 -- 172.21.15.17:0/1365788862 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1aa0155d30 msgr2=0x7f1aa01561b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:22.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.495+0000 7f1a9f400640 1 --2- 172.21.15.17:0/1365788862 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1aa0155d30 0x7f1aa01561b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:22.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.495+0000 7f1a9f400640 1 -- 172.21.15.17:0/1365788862 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1aa00c4e70 con 0x7f1a84004810 2024-09-10T21:24:22.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.495+0000 7f1a9f400640 1 --2- 172.21.15.17:0/1365788862 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1a84004810 0x7f1aa00cf320 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f1a8800b550 tx=0x7f1a8800ba10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:22.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.495+0000 7f1a97e00640 1 -- 172.21.15.17:0/1365788862 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a88014070 con 0x7f1a84004810 2024-09-10T21:24:22.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.495+0000 7f1aa6400640 1 -- 172.21.15.17:0/1365788862 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1aa00c6580 con 0x7f1a84004810 2024-09-10T21:24:22.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.495+0000 7f1aa6400640 1 -- 172.21.15.17:0/1365788862 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1aa00c50e0 con 0x7f1a84004810 2024-09-10T21:24:22.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.495+0000 7f1a97e00640 1 -- 172.21.15.17:0/1365788862 <== mon.1 v2:172.21.15.31:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1a88010980 con 0x7f1a84004810 2024-09-10T21:24:22.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.499+0000 7f1a97e00640 1 -- 172.21.15.17:0/1365788862 <== mon.1 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a88010dd0 con 0x7f1a84004810 2024-09-10T21:24:22.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.499+0000 7f1aa6400640 1 -- 172.21.15.17:0/1365788862 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a780052a0 con 0x7f1a84004810 2024-09-10T21:24:22.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.503+0000 7f1a97e00640 1 -- 172.21.15.17:0/1365788862 <== mon.1 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7f1a88019450 con 0x7f1a84004810 2024-09-10T21:24:22.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.503+0000 7f1a97e00640 1 -- 172.21.15.17:0/1365788862 <== mon.1 v2:172.21.15.31:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f1a8800fd00 con 0x7f1a84004810 2024-09-10T21:24:22.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.503+0000 7f1a97e00640 1 -- 172.21.15.17:0/1365788862 <== mon.1 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+76091 (secure 0 0 0) 0x7f1aa00c6580 con 0x7f1a84004810 2024-09-10T21:24:22.537 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'debug_mds_balancer', '20'], check_status=True, hostname='smithi017') 2024-09-10T21:24:22.544 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'debug_ms', '2'], check_status=True, hostname='smithi017') 2024-09-10T21:24:22.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.555+0000 7f0c3c200640 1 -- 172.21.15.17:0/1237229646 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0c340c72b0 msgr2=0x7f0c3415e1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:22.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.555+0000 7f0c3c200640 1 --2- 172.21.15.17:0/1237229646 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0c340c72b0 0x7f0c3415e1e0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f0c28009890 tx=0x7f0c2801c880 comp rx=0 tx=0).stop 2024-09-10T21:24:22.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.555+0000 7f0c3c200640 1 -- 172.21.15.17:0/1237229646 shutdown_connections 2024-09-10T21:24:22.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.555+0000 7f0c3c200640 1 --2- 172.21.15.17:0/1237229646 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0c340c72b0 0x7f0c3415e1e0 secure :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f0c28009890 tx=0x7f0c2801c880 comp rx=0 tx=0).stop 2024-09-10T21:24:22.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.555+0000 7f0c3c200640 1 -- 172.21.15.17:0/1237229646 >> 172.21.15.17:0/1237229646 conn(0x7f0c34070110 msgr2=0x7f0c34070520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:22.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.559+0000 7fb1b1400640 1 -- 172.21.15.17:0/2757040539 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=debug_ms}] v 0) -- 0x7fb1a40355c0 con 0x7fb1a4045cd0 2024-09-10T21:24:22.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.559+0000 7f0c3c200640 1 -- 172.21.15.17:0/1237229646 shutdown_connections 2024-09-10T21:24:22.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.555+0000 7f1abaa00640 1 -- 172.21.15.17:0/3659959038 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1ab4073d00 msgr2=0x7f1ab415dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:22.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.555+0000 7f1abaa00640 1 --2- 172.21.15.17:0/3659959038 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1ab4073d00 0x7f1ab415dd30 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f1aac009840 tx=0x7f1aac01c7c0 comp rx=0 tx=0).stop 2024-09-10T21:24:22.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.559+0000 7f1abaa00640 1 -- 172.21.15.17:0/3659959038 shutdown_connections 2024-09-10T21:24:22.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.559+0000 7f1abaa00640 1 --2- 172.21.15.17:0/3659959038 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1ab4073d00 0x7f1ab415dd30 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:22.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.559+0000 7f1abaa00640 1 --2- 172.21.15.17:0/3659959038 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f1ab415bc30 0x7f1ab4154ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:22.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.559+0000 7f1abaa00640 1 -- 172.21.15.17:0/3659959038 >> 172.21.15.17:0/3659959038 conn(0x7f1ab406fc60 msgr2=0x7f1ab4070070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:22.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.559+0000 7f1abaa00640 1 -- 172.21.15.17:0/3659959038 shutdown_connections 2024-09-10T21:24:22.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.559+0000 7f1abaa00640 1 -- 172.21.15.17:0/3659959038 wait complete. 2024-09-10T21:24:22.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.559+0000 7f1abaa00640 1 Processor -- start 2024-09-10T21:24:22.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.559+0000 7f0c3c200640 1 -- 172.21.15.17:0/1237229646 wait complete. 2024-09-10T21:24:22.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.559+0000 7f0c3c200640 1 Processor -- start 2024-09-10T21:24:22.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.567+0000 7f0c3c200640 1 -- start start 2024-09-10T21:24:22.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.567+0000 7f0c3c200640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f0c340c6ab0 0x7f0c340b0a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:22.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.567+0000 7f0c3c200640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c3415de00 0x7f0c340c34b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:22.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.567+0000 7f0c3c200640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0c3414aa40 0x7f0c3414aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:22.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.567+0000 7f0c3c200640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f0c341545e0 con 0x7f0c3414aa40 2024-09-10T21:24:22.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.567+0000 7f0c3c200640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f0c34154460 con 0x7f0c3415de00 2024-09-10T21:24:22.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.567+0000 7f0c3c200640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f0c34154760 con 0x7f0c340c6ab0 2024-09-10T21:24:22.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.567+0000 7f0c3a200640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0c3414aa40 0x7f0c3414aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:22.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.567+0000 7f0c39800640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f0c340c6ab0 0x7f0c340b0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:22.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.567+0000 7f0c3a200640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0c3414aa40 0x7f0c3414aeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35716/0 (socket says 172.21.15.17:35716) 2024-09-10T21:24:22.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.567+0000 7f0c3a200640 1 -- 172.21.15.17:0/1635190625 learned_addr learned my addr 172.21.15.17:0/1635190625 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:24:22.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.567+0000 7f1abaa00640 1 -- start start 2024-09-10T21:24:22.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.567+0000 7f0c39800640 1 -- 172.21.15.17:0/1635190625 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c3415de00 msgr2=0x7f0c340c34b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-10T21:24:22.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.567+0000 7f0c39800640 1 --2- 172.21.15.17:0/1635190625 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c3415de00 0x7f0c340c34b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:22.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.567+0000 7f0c39800640 1 -- 172.21.15.17:0/1635190625 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0c3414aa40 msgr2=0x7f0c3414aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:22.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.567+0000 7f0c39800640 1 --2- 172.21.15.17:0/1635190625 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0c3414aa40 0x7f0c3414aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:22.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.567+0000 7f0c39800640 1 -- 172.21.15.17:0/1635190625 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c340c50f0 con 0x7f0c340c6ab0 2024-09-10T21:24:22.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.567+0000 7f0c39800640 1 --2- 172.21.15.17:0/1635190625 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f0c340c6ab0 0x7f0c340b0a20 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f0c2000c390 tx=0x7f0c2000c850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:22.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.567+0000 7f0c31600640 1 -- 172.21.15.17:0/1635190625 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c20011790 con 0x7f0c340c6ab0 2024-09-10T21:24:22.574 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.567+0000 7f0c31600640 1 -- 172.21.15.17:0/1635190625 <== mon.2 v2:172.21.15.31:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0c20017070 con 0x7f0c340c6ab0 2024-09-10T21:24:22.574 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.571+0000 7f0c31600640 1 -- 172.21.15.17:0/1635190625 <== mon.2 v2:172.21.15.31:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c20011d20 con 0x7f0c340c6ab0 2024-09-10T21:24:22.574 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.571+0000 7f0c3c200640 1 -- 172.21.15.17:0/1635190625 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c3416a2f0 con 0x7f0c340c6ab0 2024-09-10T21:24:22.574 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.571+0000 7f0c3c200640 1 -- 172.21.15.17:0/1635190625 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c340b1710 con 0x7f0c340c6ab0 2024-09-10T21:24:22.575 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.571+0000 7f0c31600640 1 -- 172.21.15.17:0/1635190625 <== mon.2 v2:172.21.15.31:3301/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7f0c2001c460 con 0x7f0c340c6ab0 2024-09-10T21:24:22.575 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.571+0000 7f0c31600640 1 -- 172.21.15.17:0/1635190625 <== mon.2 v2:172.21.15.31:3301/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f0c20012850 con 0x7f0c340c6ab0 2024-09-10T21:24:22.576 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.575+0000 7f1abaa00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f1ab4073d00 0x7f1ab40c72a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:22.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.575+0000 7f1abaa00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1ab40c5f60 0x7f1ab41f28a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:22.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.575+0000 7f1abaa00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1ab415bc30 0x7f1ab414cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:22.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.575+0000 7f1abaa00640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f1ab406e850 con 0x7f1ab415bc30 2024-09-10T21:24:22.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.575+0000 7f1abaa00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f1ab40737e0 con 0x7f1ab40c5f60 2024-09-10T21:24:22.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.575+0000 7f1abaa00640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f1ab406e9d0 con 0x7f1ab4073d00 2024-09-10T21:24:22.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.575+0000 7f0c26a00640 1 -- 172.21.15.17:0/1635190625 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c34154be0 con 0x7f0c340c6ab0 2024-09-10T21:24:22.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.575+0000 7f1ab9600640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f1ab4073d00 0x7f1ab40c72a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:22.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.575+0000 7f1ab9600640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f1ab4073d00 0x7f1ab40c72a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3301/0 says I am v2:172.21.15.17:51840/0 (socket says 172.21.15.17:51840) 2024-09-10T21:24:22.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.575+0000 7f1ab9600640 1 -- 172.21.15.17:0/1802547544 learned_addr learned my addr 172.21.15.17:0/1802547544 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:24:22.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.575+0000 7f1aba000640 1 --2- 172.21.15.17:0/1802547544 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1ab415bc30 0x7f1ab414cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:22.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.575+0000 7f1ab8c00640 1 --2- 172.21.15.17:0/1802547544 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1ab40c5f60 0x7f1ab41f28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:22.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.575+0000 7f1aba000640 1 -- 172.21.15.17:0/1802547544 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f1ab4073d00 msgr2=0x7f1ab40c72a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:22.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.575+0000 7f1aba000640 1 --2- 172.21.15.17:0/1802547544 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f1ab4073d00 0x7f1ab40c72a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:22.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.575+0000 7f1aba000640 1 -- 172.21.15.17:0/1802547544 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1ab40c5f60 msgr2=0x7f1ab41f28a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:22.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.575+0000 7f1aba000640 1 --2- 172.21.15.17:0/1802547544 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1ab40c5f60 0x7f1ab41f28a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:22.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.575+0000 7f1aba000640 1 -- 172.21.15.17:0/1802547544 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ab40c83d0 con 0x7f1ab415bc30 2024-09-10T21:24:22.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.575+0000 7f1aba000640 1 --2- 172.21.15.17:0/1802547544 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1ab415bc30 0x7f1ab414cc70 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f1aac0ac000 tx=0x7f1aac0a56d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:22.580 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.575+0000 7f0c31600640 1 -- 172.21.15.17:0/1635190625 <== mon.2 v2:172.21.15.31:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+76091 (secure 0 0 0) 0x7f0c20012aa0 con 0x7f0c340c6ab0 2024-09-10T21:24:22.581 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.579+0000 7f1ab1600640 1 -- 172.21.15.17:0/1802547544 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aac0b2040 con 0x7f1ab415bc30 2024-09-10T21:24:22.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.583+0000 7fb19fe00640 1 -- 172.21.15.17:0/2757040539 <== mon.1 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_ms}]=0 v9)=0 v9) ==== 106+0+0 (secure 0 0 0) 0x7fb198012b30 con 0x7fb1a4045cd0 2024-09-10T21:24:22.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.579+0000 7f1ab1600640 1 -- 172.21.15.17:0/1802547544 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1aac0bb070 con 0x7f1ab415bc30 2024-09-10T21:24:22.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.579+0000 7f1ab1600640 1 -- 172.21.15.17:0/1802547544 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aac0046d0 con 0x7f1ab415bc30 2024-09-10T21:24:22.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.579+0000 7f1abaa00640 1 -- 172.21.15.17:0/1802547544 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ab414c070 con 0x7f1ab415bc30 2024-09-10T21:24:22.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.579+0000 7f1abaa00640 1 -- 172.21.15.17:0/1802547544 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ab414ff10 con 0x7f1ab415bc30 2024-09-10T21:24:22.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.579+0000 7f1ab1600640 1 -- 172.21.15.17:0/1802547544 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7f1aac0b5070 con 0x7f1ab415bc30 2024-09-10T21:24:22.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.579+0000 7f1ab1600640 1 -- 172.21.15.17:0/1802547544 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f1aac0b6430 con 0x7f1ab415bc30 2024-09-10T21:24:22.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.591+0000 7f1aaaa00640 1 -- 172.21.15.17:0/1802547544 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ab4069e60 con 0x7f1ab415bc30 2024-09-10T21:24:22.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.591+0000 7fb1b1400640 1 -- 172.21.15.17:0/2757040539 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1a4045cd0 msgr2=0x7fb1a4047c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:22.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.591+0000 7fb1b1400640 1 --2- 172.21.15.17:0/2757040539 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1a4045cd0 0x7fb1a4047c40 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fb19800c4b0 tx=0x7fb19800c8b0 comp rx=0 tx=0).stop 2024-09-10T21:24:22.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.595+0000 7fb1b1400640 1 -- 172.21.15.17:0/2757040539 shutdown_connections 2024-09-10T21:24:22.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.595+0000 7fb1b1400640 1 --2- 172.21.15.17:0/2757040539 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb1a40d52e0 0x7fb1a40d3740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:22.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.595+0000 7fb1b1400640 1 --2- 172.21.15.17:0/2757040539 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fb1a4050730 0x7fb1a40ceab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:22.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.595+0000 7fb1b1400640 1 --2- 172.21.15.17:0/2757040539 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1a4045cd0 0x7fb1a4047c40 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:22.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.595+0000 7fb1b1400640 1 -- 172.21.15.17:0/2757040539 >> 172.21.15.17:0/2757040539 conn(0x7fb1a404fc10 msgr2=0x7fb1a40e5290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:22.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.599+0000 7fb1b1400640 1 -- 172.21.15.17:0/2757040539 shutdown_connections 2024-09-10T21:24:22.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.599+0000 7fb1b1400640 1 -- 172.21.15.17:0/2757040539 wait complete. 2024-09-10T21:24:22.602 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.599+0000 7f1ab1600640 1 -- 172.21.15.17:0/1802547544 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+76091 (secure 0 0 0) 0x7f1aac016410 con 0x7f1ab415bc30 2024-09-10T21:24:22.616 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.615+0000 7fd40ce00640 1 -- 172.21.15.17:0/3372881001 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd40806e140 msgr2=0x7fd40806e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:22.617 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.615+0000 7fd40ce00640 1 --2- 172.21.15.17:0/3372881001 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd40806e140 0x7fd40806e5d0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fd408071ce0 tx=0x7fd40001cd00 comp rx=0 tx=0).stop 2024-09-10T21:24:22.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.623+0000 7fd40ce00640 1 -- 172.21.15.17:0/3372881001 shutdown_connections 2024-09-10T21:24:22.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.623+0000 7fd40ce00640 1 --2- 172.21.15.17:0/3372881001 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd40806e140 0x7fd40806e5d0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:22.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.623+0000 7fd40ce00640 1 --2- 172.21.15.17:0/3372881001 >> v2:172.21.15.17:3300/0 conn(0x7fd408073d00 0x7fd408154ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:22.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.623+0000 7fd40ce00640 1 --2- 172.21.15.17:0/3372881001 >> v2:172.21.15.31:3300/0 conn(0x7fd4080c5f60 0x7fd40815bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:22.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.623+0000 7fd40ce00640 1 -- 172.21.15.17:0/3372881001 >> 172.21.15.17:0/3372881001 conn(0x7fd40806fc60 msgr2=0x7fd408070070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:22.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.627+0000 7fd40ce00640 1 -- 172.21.15.17:0/3372881001 shutdown_connections 2024-09-10T21:24:22.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.627+0000 7fd40ce00640 1 -- 172.21.15.17:0/3372881001 wait complete. 2024-09-10T21:24:22.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.627+0000 7fd40ce00640 1 Processor -- start 2024-09-10T21:24:22.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.639+0000 7fd40ce00640 1 -- start start 2024-09-10T21:24:22.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.647+0000 7fd40ce00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd40806e140 0x7fd408155130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:22.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.647+0000 7fd40ce00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd408073d00 0x7fd4081591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:22.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.647+0000 7fd40ce00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd4080c5f60 0x7fd408157e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:22.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.647+0000 7fd40ce00640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fd40815e540 con 0x7fd40806e140 2024-09-10T21:24:22.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.647+0000 7fd40ce00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fd408074100 con 0x7fd4080c5f60 2024-09-10T21:24:22.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.647+0000 7fd40ce00640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7fd40815e6c0 con 0x7fd408073d00 2024-09-10T21:24:22.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.647+0000 7fd406a00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd408073d00 0x7fd4081591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:22.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.647+0000 7fd406a00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd408073d00 0x7fd4081591e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3301/0 says I am v2:172.21.15.17:51858/0 (socket says 172.21.15.17:51858) 2024-09-10T21:24:22.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.647+0000 7fd406a00640 1 -- 172.21.15.17:0/2522117851 learned_addr learned my addr 172.21.15.17:0/2522117851 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:24:22.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.647+0000 7fd407400640 1 --2- 172.21.15.17:0/2522117851 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd40806e140 0x7fd408155130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:22.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.647+0000 7fd406a00640 1 -- 172.21.15.17:0/2522117851 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd4080c5f60 msgr2=0x7fd408157e80 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-10T21:24:22.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.647+0000 7fd406a00640 1 --2- 172.21.15.17:0/2522117851 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd4080c5f60 0x7fd408157e80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:22.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.647+0000 7fd406a00640 1 -- 172.21.15.17:0/2522117851 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd40806e140 msgr2=0x7fd408155130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:22.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.647+0000 7fd406a00640 1 --2- 172.21.15.17:0/2522117851 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd40806e140 0x7fd408155130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:22.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.647+0000 7fd406a00640 1 -- 172.21.15.17:0/2522117851 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4080b1170 con 0x7fd408073d00 2024-09-10T21:24:22.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.647+0000 7fd406a00640 1 --2- 172.21.15.17:0/2522117851 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd408073d00 0x7fd4081591e0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fd3f800c940 tx=0x7fd3f800ce00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:22.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.655+0000 7fd3ff400640 1 -- 172.21.15.17:0/2522117851 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3f8007bb0 con 0x7fd408073d00 2024-09-10T21:24:22.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.655+0000 7fd3ff400640 1 -- 172.21.15.17:0/2522117851 <== mon.2 v2:172.21.15.31:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd3f8007e40 con 0x7fd408073d00 2024-09-10T21:24:22.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.655+0000 7fd3ff400640 1 -- 172.21.15.17:0/2522117851 <== mon.2 v2:172.21.15.31:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3f800f690 con 0x7fd408073d00 2024-09-10T21:24:22.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.655+0000 7f614a400640 1 -- 172.21.15.17:0/2069229202 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f614415de00 msgr2=0x7f61440c8ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:22.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.655+0000 7f6142000640 1 -- 172.21.15.17:0/2069229202 <== mon.2 v2:172.21.15.31:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61340b2070 con 0x7f614415de00 2024-09-10T21:24:22.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.655+0000 7f614a400640 1 --2- 172.21.15.17:0/2069229202 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f614415de00 0x7f61440c8ea0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f6134009900 tx=0x7f613401c900 comp rx=0 tx=0).stop 2024-09-10T21:24:22.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.659+0000 7fd40ce00640 1 -- 172.21.15.17:0/2522117851 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4080b1450 con 0x7fd408073d00 2024-09-10T21:24:22.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.659+0000 7fd40ce00640 1 -- 172.21.15.17:0/2522117851 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4081589e0 con 0x7fd408073d00 2024-09-10T21:24:22.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.659+0000 7fd3ff400640 1 -- 172.21.15.17:0/2522117851 <== mon.2 v2:172.21.15.31:3301/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7fd3f800f830 con 0x7fd408073d00 2024-09-10T21:24:22.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.659+0000 7fd3ff400640 1 -- 172.21.15.17:0/2522117851 <== mon.2 v2:172.21.15.31:3301/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fd3f8004de0 con 0x7fd408073d00 2024-09-10T21:24:22.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.659+0000 7fd40ce00640 1 -- 172.21.15.17:0/2522117851 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd40806e5d0 con 0x7fd408073d00 2024-09-10T21:24:22.664 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.663+0000 7f614a400640 1 -- 172.21.15.17:0/2069229202 shutdown_connections 2024-09-10T21:24:22.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.663+0000 7f614a400640 1 --2- 172.21.15.17:0/2069229202 >> v2:172.21.15.17:3300/0 conn(0x7f61440c72b0 0x7f614415e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:22.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.663+0000 7f614a400640 1 --2- 172.21.15.17:0/2069229202 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f614415de00 0x7f61440c8ea0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:22.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.663+0000 7f614a400640 1 -- 172.21.15.17:0/2069229202 >> 172.21.15.17:0/2069229202 conn(0x7f6144070110 msgr2=0x7f6144070520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:22.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.663+0000 7fd3ff400640 1 -- 172.21.15.17:0/2522117851 <== mon.2 v2:172.21.15.31:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+76091 (secure 0 0 0) 0x7fd3f8026080 con 0x7fd408073d00 2024-09-10T21:24:22.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.675+0000 7f614a400640 1 -- 172.21.15.17:0/2069229202 shutdown_connections 2024-09-10T21:24:22.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.679+0000 7f614a400640 1 -- 172.21.15.17:0/2069229202 wait complete. 2024-09-10T21:24:22.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.679+0000 7f147a000640 1 -- 172.21.15.17:0/179152250 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f145c004df0 msgr2=0x7f145c0071e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:22.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.679+0000 7f147a000640 1 --2- 172.21.15.17:0/179152250 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f145c004df0 0x7f145c0071e0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f1460001750 tx=0x7f146001c860 comp rx=0 tx=0).stop 2024-09-10T21:24:22.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.679+0000 7f1473e00640 1 -- 172.21.15.17:0/179152250 reap_dead start 2024-09-10T21:24:22.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.679+0000 7f147a000640 1 -- 172.21.15.17:0/179152250 shutdown_connections 2024-09-10T21:24:22.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.679+0000 7f147a000640 1 -- 172.21.15.17:0/179152250 >> 172.21.15.17:0/179152250 conn(0x7f146c04fc40 msgr2=0x7f146c050050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:22.684 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'mds_bal_fragment_size_max', '10000'], check_status=True, hostname='smithi017') 2024-09-10T21:24:22.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.679+0000 7f614a400640 1 Processor -- start 2024-09-10T21:24:22.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.679+0000 7f614a400640 1 -- start start 2024-09-10T21:24:22.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.683+0000 7f614a400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f61440c6ab0 0x7f6144169480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:22.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.683+0000 7f614a400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f61440c72b0 0x7f61440b1600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:22.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.683+0000 7f6143400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f61440c6ab0 0x7f6144169480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:22.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.683+0000 7f6143400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f61440c6ab0 0x7f6144169480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.17:47810/0 (socket says 172.21.15.17:47810) 2024-09-10T21:24:22.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.687+0000 7f6143400640 1 -- 172.21.15.17:0/2923521647 learned_addr learned my addr 172.21.15.17:0/2923521647 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:24:22.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.687+0000 7f614a400640 1 --2- 172.21.15.17:0/2923521647 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f614415de00 0x7f614414cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:22.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.687+0000 7f614a400640 1 -- 172.21.15.17:0/2923521647 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f6144153450 con 0x7f61440c72b0 2024-09-10T21:24:22.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.687+0000 7f614a400640 1 -- 172.21.15.17:0/2923521647 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f61441535d0 con 0x7f61440c6ab0 2024-09-10T21:24:22.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.687+0000 7f614a400640 1 -- 172.21.15.17:0/2923521647 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f6144153750 con 0x7f614415de00 2024-09-10T21:24:22.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.687+0000 7f6142a00640 1 --2- 172.21.15.17:0/2923521647 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f61440c72b0 0x7f61440b1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:22.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.687+0000 7f6143e00640 1 --2- 172.21.15.17:0/2923521647 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f614415de00 0x7f614414cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:22.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.687+0000 7f6143400640 1 -- 172.21.15.17:0/2923521647 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f614415de00 msgr2=0x7f614414cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:22.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.687+0000 7f6143400640 1 --2- 172.21.15.17:0/2923521647 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f614415de00 0x7f614414cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:22.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.687+0000 7f6143400640 1 -- 172.21.15.17:0/2923521647 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f61440c72b0 msgr2=0x7f61440b1600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:22.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.687+0000 7f6143400640 1 --2- 172.21.15.17:0/2923521647 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f61440c72b0 0x7f61440b1600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:22.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.687+0000 7f6143400640 1 -- 172.21.15.17:0/2923521647 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61441f9440 con 0x7f61440c6ab0 2024-09-10T21:24:22.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.687+0000 7f6143400640 1 --2- 172.21.15.17:0/2923521647 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f61440c6ab0 0x7f6144169480 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f61340b0040 tx=0x7f6134005e70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:22.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.687+0000 7f613b400640 1 -- 172.21.15.17:0/2923521647 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61340b2040 con 0x7f61440c6ab0 2024-09-10T21:24:22.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.687+0000 7f613b400640 1 -- 172.21.15.17:0/2923521647 <== mon.1 v2:172.21.15.31:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f61340043f0 con 0x7f61440c6ab0 2024-09-10T21:24:22.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.687+0000 7f613b400640 1 -- 172.21.15.17:0/2923521647 <== mon.1 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6134005220 con 0x7f61440c6ab0 2024-09-10T21:24:22.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.687+0000 7f614a400640 1 -- 172.21.15.17:0/2923521647 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f614416a170 con 0x7f61440c6ab0 2024-09-10T21:24:22.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.687+0000 7f614a400640 1 -- 172.21.15.17:0/2923521647 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61440cc810 con 0x7f61440c6ab0 2024-09-10T21:24:22.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.691+0000 7f147a000640 1 -- 172.21.15.17:0/179152250 shutdown_connections 2024-09-10T21:24:22.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.691+0000 7f147a000640 1 -- 172.21.15.17:0/179152250 wait complete. 2024-09-10T21:24:22.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.687+0000 7f613b400640 1 -- 172.21.15.17:0/2923521647 <== mon.1 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7f61340a5850 con 0x7f61440c6ab0 2024-09-10T21:24:22.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.687+0000 7f613b400640 1 -- 172.21.15.17:0/2923521647 <== mon.1 v2:172.21.15.31:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f6134004d40 con 0x7f61440c6ab0 2024-09-10T21:24:22.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.691+0000 7f614a400640 1 -- 172.21.15.17:0/2923521647 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f614415e1e0 con 0x7f61440c6ab0 2024-09-10T21:24:22.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.691+0000 7f147a000640 1 Processor -- start 2024-09-10T21:24:22.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.691+0000 7f613b400640 1 -- 172.21.15.17:0/2923521647 <== mon.1 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+76091 (secure 0 0 0) 0x7f61340164f0 con 0x7f61440c6ab0 2024-09-10T21:24:22.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.691+0000 7f147a000640 1 -- start start 2024-09-10T21:24:22.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.695+0000 7f147a000640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f145c004df0 0x7f146c168eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:22.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.695+0000 7f147a000640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f146c0ca920 0x7f146c0c8a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:22.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.695+0000 7f147a000640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f146c0c8f80 0x7f146c0db740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:22.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.695+0000 7f1473400640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f145c004df0 0x7f146c168eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:22.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.695+0000 7f1473400640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f145c004df0 0x7f146c168eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3301/0 says I am v2:172.21.15.17:51904/0 (socket says 172.21.15.17:51904) 2024-09-10T21:24:22.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.695+0000 7f1472a00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f146c0ca920 0x7f146c0c8a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:22.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.695+0000 7f1472a00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f146c0ca920 0x7f146c0c8a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.17:47836/0 (socket says 172.21.15.17:47836) 2024-09-10T21:24:22.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.699+0000 7f147a000640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f146c0d49b0 con 0x7f146c0c8f80 2024-09-10T21:24:22.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.699+0000 7f147a000640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f146c007110 con 0x7f146c0ca920 2024-09-10T21:24:22.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.699+0000 7f147a000640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f146c033b20 con 0x7f145c004df0 2024-09-10T21:24:22.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.699+0000 7f3ba1600640 1 -- 172.21.15.17:0/1375525020 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3b9c0b1020 msgr2=0x7f3b9c14fb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:22.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.699+0000 7f3ba1600640 1 --2- 172.21.15.17:0/1375525020 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3b9c0b1020 0x7f3b9c14fb70 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f3b9c15e620 tx=0x7f3b8c02d630 comp rx=0 tx=0).stop 2024-09-10T21:24:22.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.699+0000 7f1473e00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f146c0c8f80 0x7f146c0db740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:22.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.699+0000 7f1473e00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f146c0c8f80 0x7f146c0db740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35816/0 (socket says 172.21.15.17:35816) 2024-09-10T21:24:22.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.699+0000 7f1473e00640 1 -- 172.21.15.17:0/2205813428 learned_addr learned my addr 172.21.15.17:0/2205813428 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:24:22.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.699+0000 7f1473e00640 1 -- 172.21.15.17:0/2205813428 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f145c004df0 msgr2=0x7f146c168eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:22.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.699+0000 7f1473e00640 1 --2- 172.21.15.17:0/2205813428 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f145c004df0 0x7f146c168eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:22.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.699+0000 7f1473e00640 1 -- 172.21.15.17:0/2205813428 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f146c0ca920 msgr2=0x7f146c0c8a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:22.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.699+0000 7f1473e00640 1 --2- 172.21.15.17:0/2205813428 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f146c0ca920 0x7f146c0c8a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:22.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.699+0000 7f1473e00640 1 -- 172.21.15.17:0/2205813428 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f146c169a90 con 0x7f146c0c8f80 2024-09-10T21:24:22.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.699+0000 7f1473400640 1 --2- 172.21.15.17:0/2205813428 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f145c004df0 0x7f146c168eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2024-09-10T21:24:22.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.699+0000 7f1473e00640 1 --2- 172.21.15.17:0/2205813428 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f146c0c8f80 0x7f146c0db740 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f146c009380 tx=0x7f1468010bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:22.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.699+0000 7f1472a00640 1 --2- 172.21.15.17:0/2205813428 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f146c0ca920 0x7f146c0c8a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2024-09-10T21:24:22.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.699+0000 7f1467400640 1 -- 172.21.15.17:0/2205813428 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1468004220 con 0x7f146c0c8f80 2024-09-10T21:24:22.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.699+0000 7f1467400640 1 -- 172.21.15.17:0/2205813428 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f14680043c0 con 0x7f146c0c8f80 2024-09-10T21:24:22.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.699+0000 7f1467400640 1 -- 172.21.15.17:0/2205813428 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f146800ea80 con 0x7f146c0c8f80 2024-09-10T21:24:22.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.699+0000 7f147a000640 1 -- 172.21.15.17:0/2205813428 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f146c0c9d70 con 0x7f146c0c8f80 2024-09-10T21:24:22.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.699+0000 7f147a000640 1 -- 172.21.15.17:0/2205813428 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f146c1697f0 con 0x7f146c0c8f80 2024-09-10T21:24:22.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.703+0000 7f1467400640 1 -- 172.21.15.17:0/2205813428 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7f1468023030 con 0x7f146c0c8f80 2024-09-10T21:24:22.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.703+0000 7f1467400640 1 -- 172.21.15.17:0/2205813428 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f1468011950 con 0x7f146c0c8f80 2024-09-10T21:24:22.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.711+0000 7f3ba1600640 1 -- 172.21.15.17:0/1375525020 shutdown_connections 2024-09-10T21:24:22.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.711+0000 7f3ba1600640 1 --2- 172.21.15.17:0/1375525020 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3b9c0b1020 0x7f3b9c14fb70 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:22.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.711+0000 7f3ba1600640 1 --2- 172.21.15.17:0/1375525020 >> v2:172.21.15.31:3300/0 conn(0x7f3b9c06fdf0 0x7f3b9c15aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:22.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.711+0000 7f3ba1600640 1 -- 172.21.15.17:0/1375525020 >> 172.21.15.17:0/1375525020 conn(0x7f3b9c06e410 msgr2=0x7f3b9c0cbdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:22.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.711+0000 7f1464c00640 1 -- 172.21.15.17:0/2205813428 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14500052a0 con 0x7f146c0c8f80 2024-09-10T21:24:22.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.711+0000 7f006a400640 1 -- 172.21.15.17:0/415774 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f004c005220 msgr2=0x7f004c00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:22.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.711+0000 7f006a400640 1 --2- 172.21.15.17:0/415774 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f004c005220 0x7f004c00b9d0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f0064000b70 tx=0x7f00640997e0 comp rx=0 tx=0).stop 2024-09-10T21:24:22.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.715+0000 7f11f3e00640 1 -- 172.21.15.17:0/2939076473 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f11ec0462a0 msgr2=0x7f11ec04b880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:22.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.715+0000 7f11f3e00640 1 --2- 172.21.15.17:0/2939076473 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f11ec0462a0 0x7f11ec04b880 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f11ec007590 tx=0x7f11e001c790 comp rx=0 tx=0).stop 2024-09-10T21:24:22.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.715+0000 7f11f3e00640 1 -- 172.21.15.17:0/2939076473 shutdown_connections 2024-09-10T21:24:22.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.715+0000 7f11f3e00640 1 --2- 172.21.15.17:0/2939076473 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f11ec0462a0 0x7f11ec04b880 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:22.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.715+0000 7f11f3e00640 1 --2- 172.21.15.17:0/2939076473 >> v2:172.21.15.31:3300/0 conn(0x7f11ec035690 0x7f11ec0def90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:22.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.715+0000 7f11f3e00640 1 -- 172.21.15.17:0/2939076473 >> 172.21.15.17:0/2939076473 conn(0x7f11ec04fc10 msgr2=0x7f11ec050020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:22.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.715+0000 7f11f3e00640 1 -- 172.21.15.17:0/2939076473 shutdown_connections 2024-09-10T21:24:22.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.715+0000 7f11f3e00640 1 -- 172.21.15.17:0/2939076473 wait complete. 2024-09-10T21:24:22.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.719+0000 7f1467400640 1 -- 172.21.15.17:0/2205813428 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+76091 (secure 0 0 0) 0x7f146c0c9d70 con 0x7f146c0c8f80 2024-09-10T21:24:22.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.719+0000 7f006a400640 1 -- 172.21.15.17:0/415774 shutdown_connections 2024-09-10T21:24:22.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.719+0000 7f006a400640 1 --2- 172.21.15.17:0/415774 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f004c005220 0x7f004c00b9d0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:22.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.719+0000 7f006a400640 1 -- 172.21.15.17:0/415774 >> 172.21.15.17:0/415774 conn(0x7f005c04fde0 msgr2=0x7f005c0501c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:22.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.719+0000 7f11f3e00640 1 Processor -- start 2024-09-10T21:24:22.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.723+0000 7f006a400640 1 -- 172.21.15.17:0/415774 shutdown_connections 2024-09-10T21:24:22.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.723+0000 7f006a400640 1 -- 172.21.15.17:0/415774 wait complete. 2024-09-10T21:24:22.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.723+0000 7f3ba1600640 1 -- 172.21.15.17:0/1375525020 shutdown_connections 2024-09-10T21:24:22.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.723+0000 7f3ba1600640 1 -- 172.21.15.17:0/1375525020 wait complete. 2024-09-10T21:24:22.732 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.723+0000 7f3ba1600640 1 Processor -- start 2024-09-10T21:24:22.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.723+0000 7f006a400640 1 Processor -- start 2024-09-10T21:24:22.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.727+0000 7f006a400640 1 -- start start 2024-09-10T21:24:22.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.723+0000 7f11f3e00640 1 -- start start 2024-09-10T21:24:22.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.727+0000 7f11f3e00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f11ec035690 0x7f11ec0db660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:22.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.727+0000 7f37dee00640 1 -- 172.21.15.17:0/3531386534 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f37c0003ab0 msgr2=0x7f37c0003ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:22.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.727+0000 7f37dee00640 1 --2- 172.21.15.17:0/3531386534 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f37c0003ab0 0x7f37c0003ed0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f37d000c640 tx=0x7f37d001fa20 comp rx=0 tx=0).stop 2024-09-10T21:24:22.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.727+0000 7f37de400640 1 -- 172.21.15.17:0/3531386534 reap_dead start 2024-09-10T21:24:22.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.727+0000 7f37dee00640 1 -- 172.21.15.17:0/3531386534 shutdown_connections 2024-09-10T21:24:22.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.727+0000 7f37dee00640 1 -- 172.21.15.17:0/3531386534 >> 172.21.15.17:0/3531386534 conn(0x7f37d806fc60 msgr2=0x7f37d8070070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:22.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.727+0000 7f37dee00640 1 -- 172.21.15.17:0/3531386534 shutdown_connections 2024-09-10T21:24:22.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.727+0000 7f37dee00640 1 -- 172.21.15.17:0/3531386534 wait complete. 2024-09-10T21:24:22.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.723+0000 7f3ba1600640 1 -- start start 2024-09-10T21:24:22.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.727+0000 7f3ba1600640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3b9c06fdf0 0x7f3b9c14d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:22.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.727+0000 7f3ba1600640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3b9c0701d0 0x7f3b9c1f2890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:22.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.727+0000 7f3ba1600640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3b9c0b1020 0x7f3b9c15c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:22.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.727+0000 7f3ba1600640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f3b9c15b400 con 0x7f3b9c06fdf0 2024-09-10T21:24:22.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.727+0000 7f3ba1600640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f3b9c0c6890 con 0x7f3b9c0b1020 2024-09-10T21:24:22.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.727+0000 7f3ba1600640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f3b9c15e930 con 0x7f3b9c0701d0 2024-09-10T21:24:22.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.727+0000 7f3b9aa00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3b9c06fdf0 0x7f3b9c14d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:22.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.727+0000 7f3b9aa00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3b9c06fdf0 0x7f3b9c14d3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35850/0 (socket says 172.21.15.17:35850) 2024-09-10T21:24:22.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.727+0000 7f3b9aa00640 1 -- 172.21.15.17:0/1784954793 learned_addr learned my addr 172.21.15.17:0/1784954793 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:24:22.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.727+0000 7f3b9aa00640 1 -- 172.21.15.17:0/1784954793 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3b9c0701d0 msgr2=0x7f3b9c1f2890 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:24:22.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.727+0000 7f3b9aa00640 1 --2- 172.21.15.17:0/1784954793 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3b9c0701d0 0x7f3b9c1f2890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:22.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.727+0000 7f3b9aa00640 1 -- 172.21.15.17:0/1784954793 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3b9c0b1020 msgr2=0x7f3b9c15c730 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-10T21:24:22.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.727+0000 7f3b9aa00640 1 --2- 172.21.15.17:0/1784954793 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3b9c0b1020 0x7f3b9c15c730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:22.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.727+0000 7f3b9aa00640 1 -- 172.21.15.17:0/1784954793 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b9c1575b0 con 0x7f3b9c06fdf0 2024-09-10T21:24:22.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.731+0000 7f3b9aa00640 1 --2- 172.21.15.17:0/1784954793 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3b9c06fdf0 0x7f3b9c14d3b0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f3b9400ba50 tx=0x7f3b9400bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:22.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.731+0000 7f11f3e00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f11ec0462a0 0x7f11ec0d61f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:22.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.731+0000 7f11f2a00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f11ec035690 0x7f11ec0db660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:22.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.731+0000 7f11f2a00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f11ec035690 0x7f11ec0db660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3301/0 says I am v2:172.21.15.17:51960/0 (socket says 172.21.15.17:51960) 2024-09-10T21:24:22.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.731+0000 7f11f3e00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f11ec04b440 0x7f11ec0d45b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:22.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.731+0000 7f11f3e00640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f11ec04b0e0 con 0x7f11ec0462a0 2024-09-10T21:24:22.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.731+0000 7f11f3e00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f11ec0d8b20 con 0x7f11ec04b440 2024-09-10T21:24:22.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.731+0000 7f11f3e00640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f11ec050730 con 0x7f11ec035690 2024-09-10T21:24:22.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.731+0000 7f11f2a00640 1 -- 172.21.15.17:0/675567414 learned_addr learned my addr 172.21.15.17:0/675567414 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:24:22.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.731+0000 7f3b92a00640 1 -- 172.21.15.17:0/1784954793 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b94019070 con 0x7f3b9c06fdf0 2024-09-10T21:24:22.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.731+0000 7f3b92a00640 1 -- 172.21.15.17:0/1784954793 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3b9400cc70 con 0x7f3b9c06fdf0 2024-09-10T21:24:22.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.731+0000 7f3b92a00640 1 -- 172.21.15.17:0/1784954793 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b9400ce60 con 0x7f3b9c06fdf0 2024-09-10T21:24:22.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.731+0000 7f5d03200640 1 -- 172.21.15.17:0/2439638658 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5d0406ef80 msgr2=0x7f5d041570a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:22.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.731+0000 7f11f3400640 1 --2- 172.21.15.17:0/675567414 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f11ec04b440 0x7f11ec0d45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:22.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.731+0000 7f5d03200640 1 --2- 172.21.15.17:0/2439638658 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5d0406ef80 0x7f5d041570a0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f5d0406f7c0 tx=0x7f5cf001d530 comp rx=0 tx=0).stop 2024-09-10T21:24:22.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.731+0000 7f11f2a00640 1 -- 172.21.15.17:0/675567414 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f11ec04b440 msgr2=0x7f11ec0d45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:22.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.731+0000 7f11f2a00640 1 --2- 172.21.15.17:0/675567414 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f11ec04b440 0x7f11ec0d45b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:22.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.731+0000 7f11f2a00640 1 -- 172.21.15.17:0/675567414 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f11ec0462a0 msgr2=0x7f11ec0d61f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:22.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.731+0000 7f11f2a00640 1 --2- 172.21.15.17:0/675567414 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f11ec0462a0 0x7f11ec0d61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:22.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.731+0000 7f006a400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f004c005220 0x7f005c0cbab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:22.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.731+0000 7f11f2a00640 1 -- 172.21.15.17:0/675567414 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11ec0d98b0 con 0x7f11ec035690 2024-09-10T21:24:22.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.731+0000 7f006a400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f005c032850 0x7f005c048470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:22.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.731+0000 7f006a400640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f005c049e50 0x7f005c0489b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:22.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.731+0000 7f006a400640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f005c04c2d0 con 0x7f004c005220 2024-09-10T21:24:22.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.731+0000 7f006a400640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f005c0debb0 con 0x7f005c032850 2024-09-10T21:24:22.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.731+0000 7f006a400640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f005c04c450 con 0x7f005c049e50 2024-09-10T21:24:22.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.731+0000 7f0062a00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f005c032850 0x7f005c048470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:22.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.731+0000 7f0062a00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f005c032850 0x7f005c048470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.17:47902/0 (socket says 172.21.15.17:47902) 2024-09-10T21:24:22.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.731+0000 7f0062a00640 1 -- 172.21.15.17:0/3250427795 learned_addr learned my addr 172.21.15.17:0/3250427795 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:24:22.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.731+0000 7f0068e00640 1 --2- 172.21.15.17:0/3250427795 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f005c049e50 0x7f005c0489b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:22.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.731+0000 7f0062a00640 1 -- 172.21.15.17:0/3250427795 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f005c049e50 msgr2=0x7f005c0489b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:22.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.731+0000 7f0062a00640 1 --2- 172.21.15.17:0/3250427795 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f005c049e50 0x7f005c0489b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:22.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.731+0000 7f0062a00640 1 -- 172.21.15.17:0/3250427795 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f004c005220 msgr2=0x7f005c0cbab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:22.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.731+0000 7f0062a00640 1 --2- 172.21.15.17:0/3250427795 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f004c005220 0x7f005c0cbab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:22.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.731+0000 7f0062a00640 1 -- 172.21.15.17:0/3250427795 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f005c0e6180 con 0x7f005c032850 2024-09-10T21:24:22.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.731+0000 7f0062a00640 1 --2- 172.21.15.17:0/3250427795 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f005c032850 0x7f005c048470 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f0050007890 tx=0x7f0050007d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:22.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.731+0000 7f0057400640 1 -- 172.21.15.17:0/3250427795 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0050017030 con 0x7f005c032850 2024-09-10T21:24:22.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.731+0000 7f0057400640 1 -- 172.21.15.17:0/3250427795 <== mon.1 v2:172.21.15.31:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f005000d8b0 con 0x7f005c032850 2024-09-10T21:24:22.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.731+0000 7f0057400640 1 -- 172.21.15.17:0/3250427795 <== mon.1 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f005000dd00 con 0x7f005c032850 2024-09-10T21:24:22.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.731+0000 7f006a400640 1 -- 172.21.15.17:0/3250427795 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f005c047180 con 0x7f005c032850 2024-09-10T21:24:22.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.731+0000 7f006a400640 1 -- 172.21.15.17:0/3250427795 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f005c04f5f0 con 0x7f005c032850 2024-09-10T21:24:22.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.731+0000 7f3ba1600640 1 -- 172.21.15.17:0/1784954793 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b9c1f01e0 con 0x7f3b9c06fdf0 2024-09-10T21:24:22.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.731+0000 7f3ba1600640 1 -- 172.21.15.17:0/1784954793 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b9c150f00 con 0x7f3b9c06fdf0 2024-09-10T21:24:22.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.731+0000 7f11f2a00640 1 --2- 172.21.15.17:0/675567414 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f11ec035690 0x7f11ec0db660 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f11f406c200 tx=0x7f11f406a670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:22.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.735+0000 7f5d03200640 1 -- 172.21.15.17:0/2439638658 shutdown_connections 2024-09-10T21:24:22.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.735+0000 7f5d03200640 1 --2- 172.21.15.17:0/2439638658 >> v2:172.21.15.17:3300/0 conn(0x7f5d04072e10 0x7f5d040c8370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:22.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.735+0000 7f5d03200640 1 --2- 172.21.15.17:0/2439638658 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5d0406ef80 0x7f5d041570a0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:22.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.735+0000 7f5d03200640 1 -- 172.21.15.17:0/2439638658 >> 172.21.15.17:0/2439638658 conn(0x7f5d0406fc60 msgr2=0x7f5d04070070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:22.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.735+0000 7f3b92a00640 1 -- 172.21.15.17:0/1784954793 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7f3b940146a0 con 0x7f3b9c06fdf0 2024-09-10T21:24:22.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.735+0000 7f3b92a00640 1 -- 172.21.15.17:0/1784954793 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f3b9401d3b0 con 0x7f3b9c06fdf0 2024-09-10T21:24:22.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.735+0000 7f5d03200640 1 -- 172.21.15.17:0/2439638658 shutdown_connections 2024-09-10T21:24:22.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.735+0000 7f37dee00640 1 Processor -- start 2024-09-10T21:24:22.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.735+0000 7f0057400640 1 -- 172.21.15.17:0/3250427795 <== mon.1 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7f0050005020 con 0x7f005c032850 2024-09-10T21:24:22.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.735+0000 7f0057400640 1 -- 172.21.15.17:0/3250427795 <== mon.1 v2:172.21.15.31:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f005001c750 con 0x7f005c032850 2024-09-10T21:24:22.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.735+0000 7f11eaa00640 1 -- 172.21.15.17:0/675567414 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11f4090070 con 0x7f11ec035690 2024-09-10T21:24:22.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.735+0000 7f11eaa00640 1 -- 172.21.15.17:0/675567414 <== mon.2 v2:172.21.15.31:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f11f40733e0 con 0x7f11ec035690 2024-09-10T21:24:22.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.735+0000 7f11eaa00640 1 -- 172.21.15.17:0/675567414 <== mon.2 v2:172.21.15.31:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11f40738b0 con 0x7f11ec035690 2024-09-10T21:24:22.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.735+0000 7f0054c00640 1 -- 172.21.15.17:0/3250427795 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00380052a0 con 0x7f005c032850 2024-09-10T21:24:22.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.735+0000 7f5d03200640 1 -- 172.21.15.17:0/2439638658 wait complete. 2024-09-10T21:24:22.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.735+0000 7f11f3e00640 1 -- 172.21.15.17:0/675567414 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11ec0da530 con 0x7f11ec035690 2024-09-10T21:24:22.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.735+0000 7f11f3e00640 1 -- 172.21.15.17:0/675567414 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f11ec0d5ab0 con 0x7f11ec035690 2024-09-10T21:24:22.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.739+0000 7f11eaa00640 1 -- 172.21.15.17:0/675567414 <== mon.2 v2:172.21.15.31:3301/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7f11f4073a50 con 0x7f11ec035690 2024-09-10T21:24:22.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.739+0000 7f11eaa00640 1 -- 172.21.15.17:0/675567414 <== mon.2 v2:172.21.15.31:3301/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f11f409e700 con 0x7f11ec035690 2024-09-10T21:24:22.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.739+0000 7f37dee00640 1 -- start start 2024-09-10T21:24:22.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.739+0000 7f11f3e00640 1 -- 172.21.15.17:0/675567414 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11ec0def90 con 0x7f11ec035690 2024-09-10T21:24:22.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.739+0000 7f5d03200640 1 Processor -- start 2024-09-10T21:24:22.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.739+0000 7f0057400640 1 -- 172.21.15.17:0/3250427795 <== mon.1 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+76091 (secure 0 0 0) 0x7f0050021070 con 0x7f005c032850 2024-09-10T21:24:22.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.739+0000 7f5d03200640 1 -- start start 2024-09-10T21:24:22.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.739+0000 7f37dee00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f37c0003ab0 0x7f37d81fd4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:22.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.739+0000 7f37dee00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f37d80c08c0 0x7f37d80c22c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:22.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.739+0000 7f37dda00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f37c0003ab0 0x7f37d81fd4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:22.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.739+0000 7f37dda00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f37c0003ab0 0x7f37d81fd4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.17:47914/0 (socket says 172.21.15.17:47914) 2024-09-10T21:24:22.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.743+0000 7f37dda00640 1 -- 172.21.15.17:0/1136092739 learned_addr learned my addr 172.21.15.17:0/1136092739 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:24:22.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.743+0000 7f3b87e00640 1 -- 172.21.15.17:0/1784954793 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b9c15aec0 con 0x7f3b9c06fdf0 2024-09-10T21:24:22.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.743+0000 7ff675400640 1 -- 172.21.15.17:0/785739415 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=mds_bal_split_size}] v 0) -- 0x7ff6700b2200 con 0x7ff67015de00 2024-09-10T21:24:22.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.743+0000 7f37dd000640 1 --2- 172.21.15.17:0/1136092739 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f37d80c08c0 0x7f37d80c22c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:22.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.743+0000 7f37dee00640 1 --2- 172.21.15.17:0/1136092739 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f37d80c2800 0x7f37d80c4aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:22.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.743+0000 7f37dee00640 1 -- 172.21.15.17:0/1136092739 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f37d8073220 con 0x7f37d80c2800 2024-09-10T21:24:22.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.743+0000 7f37dee00640 1 -- 172.21.15.17:0/1136092739 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f37d815c030 con 0x7f37c0003ab0 2024-09-10T21:24:22.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.743+0000 7f5d03200640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f5d0406ef80 0x7f5d040c0e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:22.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.743+0000 7f37dee00640 1 -- 172.21.15.17:0/1136092739 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f37d8073500 con 0x7f37d80c08c0 2024-09-10T21:24:22.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.743+0000 7f5d03200640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5d04072a30 0x7f5d041475c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:22.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.743+0000 7f11eaa00640 1 -- 172.21.15.17:0/675567414 <== mon.2 v2:172.21.15.31:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+76091 (secure 0 0 0) 0x7f11f4073d70 con 0x7f11ec035690 2024-09-10T21:24:22.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.743+0000 7f5d03200640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5d04072e10 0x7f5d040c1710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:22.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.743+0000 7f5d01e00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f5d0406ef80 0x7f5d040c0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:22.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.743+0000 7f5d01e00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f5d0406ef80 0x7f5d040c0e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3301/0 says I am v2:172.21.15.17:51978/0 (socket says 172.21.15.17:51978) 2024-09-10T21:24:22.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.743+0000 7f5d01400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5d04072a30 0x7f5d041475c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:22.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.743+0000 7f5d01400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5d04072a30 0x7f5d041475c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.17:47926/0 (socket says 172.21.15.17:47926) 2024-09-10T21:24:22.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.743+0000 7f37de400640 1 --2- 172.21.15.17:0/1136092739 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f37d80c2800 0x7f37d80c4aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:22.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.743+0000 7f37dd000640 1 -- 172.21.15.17:0/1136092739 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f37c0003ab0 msgr2=0x7f37d81fd4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:22.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.743+0000 7f37dd000640 1 --2- 172.21.15.17:0/1136092739 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f37c0003ab0 0x7f37d81fd4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:22.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.743+0000 7f37dd000640 1 -- 172.21.15.17:0/1136092739 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f37d80c2800 msgr2=0x7f37d80c4aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:22.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.743+0000 7f37dd000640 1 --2- 172.21.15.17:0/1136092739 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f37d80c2800 0x7f37d80c4aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:22.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.743+0000 7f37dd000640 1 -- 172.21.15.17:0/1136092739 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37d80c3b00 con 0x7f37d80c08c0 2024-09-10T21:24:22.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.743+0000 7f37de400640 1 --2- 172.21.15.17:0/1136092739 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f37d80c2800 0x7f37d80c4aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:24:22.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.743+0000 7f5d01e00640 1 -- 172.21.15.17:0/419032460 learned_addr learned my addr 172.21.15.17:0/419032460 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:24:22.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.743+0000 7f5d03200640 1 -- 172.21.15.17:0/419032460 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f5d040c6de0 con 0x7f5d04072e10 2024-09-10T21:24:22.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.743+0000 7f5d03200640 1 -- 172.21.15.17:0/419032460 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f5d041579f0 con 0x7f5d04072a30 2024-09-10T21:24:22.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.743+0000 7f5d03200640 1 -- 172.21.15.17:0/419032460 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f5d040c6f60 con 0x7f5d0406ef80 2024-09-10T21:24:22.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.743+0000 7f37dd000640 1 --2- 172.21.15.17:0/1136092739 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f37d80c08c0 0x7f37d80c22c0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f37d80b1930 tx=0x7f37c400da90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:22.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.747+0000 7f37d5600640 1 -- 172.21.15.17:0/1136092739 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37c4017070 con 0x7f37d80c08c0 2024-09-10T21:24:22.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.747+0000 7f37d5600640 1 -- 172.21.15.17:0/1136092739 <== mon.2 v2:172.21.15.31:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f37c4005310 con 0x7f37d80c08c0 2024-09-10T21:24:22.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.747+0000 7f37d5600640 1 -- 172.21.15.17:0/1136092739 <== mon.2 v2:172.21.15.31:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37c40047b0 con 0x7f37d80c08c0 2024-09-10T21:24:22.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.747+0000 7f5d01400640 1 -- 172.21.15.17:0/419032460 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f5d0406ef80 msgr2=0x7f5d040c0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:22.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.747+0000 7f5d01400640 1 --2- 172.21.15.17:0/419032460 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f5d0406ef80 0x7f5d040c0e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:22.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.747+0000 7f5d01400640 1 -- 172.21.15.17:0/419032460 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5d04072e10 msgr2=0x7f5d040c1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:22.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.747+0000 7f5d01400640 1 --2- 172.21.15.17:0/419032460 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5d04072e10 0x7f5d040c1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:22.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.747+0000 7f5d01400640 1 -- 172.21.15.17:0/419032460 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d041f3bf0 con 0x7f5d04072a30 2024-09-10T21:24:22.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.747+0000 7f5d01400640 1 --2- 172.21.15.17:0/419032460 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5d04072a30 0x7f5d041475c0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f5cf400b440 tx=0x7f5cf400b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:22.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.747+0000 7f5cfa000640 1 -- 172.21.15.17:0/419032460 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cf4015020 con 0x7f5d04072a30 2024-09-10T21:24:22.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.747+0000 7f37dee00640 1 -- 172.21.15.17:0/1136092739 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37d80be660 con 0x7f37d80c08c0 2024-09-10T21:24:22.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.747+0000 7f37dee00640 1 -- 172.21.15.17:0/1136092739 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f37d80c00c0 con 0x7f37d80c08c0 2024-09-10T21:24:22.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.747+0000 7f5cfa000640 1 -- 172.21.15.17:0/419032460 <== mon.1 v2:172.21.15.31:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5cf4011920 con 0x7f5d04072a30 2024-09-10T21:24:22.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.747+0000 7f5cfa000640 1 -- 172.21.15.17:0/419032460 <== mon.1 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cf4011d70 con 0x7f5d04072a30 2024-09-10T21:24:22.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.751+0000 7f3b92a00640 1 -- 172.21.15.17:0/1784954793 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+76091 (secure 0 0 0) 0x7f3b94023070 con 0x7f3b9c06fdf0 2024-09-10T21:24:22.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.747+0000 7f5d03200640 1 -- 172.21.15.17:0/419032460 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d040bfc70 con 0x7f5d04072a30 2024-09-10T21:24:22.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.747+0000 7f5d03200640 1 -- 172.21.15.17:0/419032460 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d041f3eb0 con 0x7f5d04072a30 2024-09-10T21:24:22.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.747+0000 7f37d5600640 1 -- 172.21.15.17:0/1136092739 <== mon.2 v2:172.21.15.31:3301/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7f37c4004950 con 0x7f37d80c08c0 2024-09-10T21:24:22.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.747+0000 7f37d5600640 1 -- 172.21.15.17:0/1136092739 <== mon.2 v2:172.21.15.31:3301/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f37c40105a0 con 0x7f37d80c08c0 2024-09-10T21:24:22.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.751+0000 7f5cfa000640 1 -- 172.21.15.17:0/419032460 <== mon.1 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7f5cf4022060 con 0x7f5d04072a30 2024-09-10T21:24:22.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.751+0000 7f5cfa000640 1 -- 172.21.15.17:0/419032460 <== mon.1 v2:172.21.15.31:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f5cf40199d0 con 0x7f5d04072a30 2024-09-10T21:24:22.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.751+0000 7f6126c00640 1 -- 172.21.15.17:0/1382104961 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f611804b4a0 msgr2=0x7f611804c4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:22.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.751+0000 7f6126c00640 1 --2- 172.21.15.17:0/1382104961 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f611804b4a0 0x7f611804c4a0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f610c009900 tx=0x7f610c01c850 comp rx=0 tx=0).stop 2024-09-10T21:24:22.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.751+0000 7f37caa00640 1 -- 172.21.15.17:0/1136092739 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37d806e140 con 0x7f37d80c08c0 2024-09-10T21:24:22.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.751+0000 7f6126c00640 1 -- 172.21.15.17:0/1382104961 shutdown_connections 2024-09-10T21:24:22.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.751+0000 7f6126c00640 1 --2- 172.21.15.17:0/1382104961 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f611804b4a0 0x7f611804c4a0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:22.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.751+0000 7f6126c00640 1 --2- 172.21.15.17:0/1382104961 >> v2:172.21.15.31:3300/0 conn(0x7f61180397e0 0x7f611804bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:22.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.751+0000 7f6126c00640 1 -- 172.21.15.17:0/1382104961 >> 172.21.15.17:0/1382104961 conn(0x7f6118054030 msgr2=0x7f6118054440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:22.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.751+0000 7f5cef400640 1 -- 172.21.15.17:0/419032460 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d0406eb50 con 0x7f5d04072a30 2024-09-10T21:24:22.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.755+0000 7f37d5600640 1 -- 172.21.15.17:0/1136092739 <== mon.2 v2:172.21.15.31:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+76091 (secure 0 0 0) 0x7f37c40096f0 con 0x7f37d80c08c0 2024-09-10T21:24:22.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.759+0000 7f6126c00640 1 -- 172.21.15.17:0/1382104961 shutdown_connections 2024-09-10T21:24:22.764 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.759+0000 7f5cfa000640 1 -- 172.21.15.17:0/419032460 <== mon.1 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+76091 (secure 0 0 0) 0x7f5cf4019460 con 0x7f5d04072a30 2024-09-10T21:24:22.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.759+0000 7f6126c00640 1 -- 172.21.15.17:0/1382104961 wait complete. 2024-09-10T21:24:22.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.759+0000 7f6126c00640 1 Processor -- start 2024-09-10T21:24:22.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.763+0000 7ff666a00640 1 -- 172.21.15.17:0/785739415 <== mon.1 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_split_size}]=0 v10)=0 v10) ==== 118+0+0 (secure 0 0 0) 0x7ff6680c4060 con 0x7ff67015de00 2024-09-10T21:24:22.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.771+0000 7f6126c00640 1 -- start start 2024-09-10T21:24:22.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.771+0000 7f6126c00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f61180397e0 0x7f6118053700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:22.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.771+0000 7f6126c00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f611804b4a0 0x7f6118047ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:22.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.771+0000 7f6126c00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f611804b910 0x7f61181815f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:22.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.771+0000 7f6126c00640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f61180eaef0 con 0x7f61180397e0 2024-09-10T21:24:22.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.771+0000 7f6126c00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f6118039dd0 con 0x7f611804b910 2024-09-10T21:24:22.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.771+0000 7f6126c00640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f61180eb070 con 0x7f611804b4a0 2024-09-10T21:24:22.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.771+0000 7ff675400640 1 -- 172.21.15.17:0/785739415 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff67015de00 msgr2=0x7ff6700c45c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:22.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.771+0000 7ff675400640 1 --2- 172.21.15.17:0/785739415 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff67015de00 0x7ff6700c45c0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7ff6680ac000 tx=0x7ff668005ae0 comp rx=0 tx=0).stop 2024-09-10T21:24:22.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.771+0000 7f6124c00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f611804b910 0x7f61181815f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:22.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.771+0000 7f611f400640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f611804b4a0 0x7f6118047ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:22.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.771+0000 7f6124c00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f611804b910 0x7f61181815f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.17:47930/0 (socket says 172.21.15.17:47930) 2024-09-10T21:24:22.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.771+0000 7f6124c00640 1 -- 172.21.15.17:0/3453483382 learned_addr learned my addr 172.21.15.17:0/3453483382 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:24:22.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.771+0000 7ff675400640 1 -- 172.21.15.17:0/785739415 shutdown_connections 2024-09-10T21:24:22.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.771+0000 7ff675400640 1 --2- 172.21.15.17:0/785739415 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff67015de00 0x7ff6700c45c0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:22.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.771+0000 7ff675400640 1 --2- 172.21.15.17:0/785739415 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff6700c72b0 0x7ff6700c5480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:22.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.771+0000 7ff675400640 1 --2- 172.21.15.17:0/785739415 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff6700c6ab0 0x7ff6700c3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:22.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.771+0000 7ff675400640 1 -- 172.21.15.17:0/785739415 >> 172.21.15.17:0/785739415 conn(0x7ff670070110 msgr2=0x7ff6700704f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:22.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.771+0000 7f6124c00640 1 -- 172.21.15.17:0/3453483382 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f611804b4a0 msgr2=0x7f6118047ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:22.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.771+0000 7f6124c00640 1 --2- 172.21.15.17:0/3453483382 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f611804b4a0 0x7f6118047ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:22.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.771+0000 7f6124c00640 1 -- 172.21.15.17:0/3453483382 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f61180397e0 msgr2=0x7f6118053700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:22.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.771+0000 7f6124c00640 1 --2- 172.21.15.17:0/3453483382 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f61180397e0 0x7f6118053700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:22.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.771+0000 7f6124c00640 1 -- 172.21.15.17:0/3453483382 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61180d6a00 con 0x7f611804b910 2024-09-10T21:24:22.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.771+0000 7f611f400640 1 --2- 172.21.15.17:0/3453483382 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f611804b4a0 0x7f6118047ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:24:22.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.771+0000 7ff675400640 1 -- 172.21.15.17:0/785739415 shutdown_connections 2024-09-10T21:24:22.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.771+0000 7f2afaa00640 1 -- 172.21.15.17:0/618969446 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=mds_debug_scatterstat}] v 0) -- 0x7f2b0c069e60 con 0x7f2b0c157720 2024-09-10T21:24:22.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.775+0000 7ff675400640 1 -- 172.21.15.17:0/785739415 wait complete. 2024-09-10T21:24:22.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.775+0000 7f6124c00640 1 --2- 172.21.15.17:0/3453483382 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f611804b910 0x7f61181815f0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f612006af00 tx=0x7f6120000b10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:22.780 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.779+0000 7f6113e00640 1 -- 172.21.15.17:0/3453483382 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6120090070 con 0x7f611804b910 2024-09-10T21:24:22.780 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.779+0000 7f6113e00640 1 -- 172.21.15.17:0/3453483382 <== mon.1 v2:172.21.15.31:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f61200943b0 con 0x7f611804b910 2024-09-10T21:24:22.781 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.779+0000 7f6113e00640 1 -- 172.21.15.17:0/3453483382 <== mon.1 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61200947e0 con 0x7f611804b910 2024-09-10T21:24:22.781 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.779+0000 7f6126c00640 1 -- 172.21.15.17:0/3453483382 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f611804c2a0 con 0x7f611804b910 2024-09-10T21:24:22.781 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.779+0000 7f6126c00640 1 -- 172.21.15.17:0/3453483382 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f611804a950 con 0x7f611804b910 2024-09-10T21:24:22.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.783+0000 7f6113e00640 1 -- 172.21.15.17:0/3453483382 <== mon.1 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7f6120094980 con 0x7f611804b910 2024-09-10T21:24:22.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.783+0000 7f6113e00640 1 -- 172.21.15.17:0/3453483382 <== mon.1 v2:172.21.15.31:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f612009e3a0 con 0x7f611804b910 2024-09-10T21:24:22.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.783+0000 7f6126c00640 1 -- 172.21.15.17:0/3453483382 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60fc0052a0 con 0x7f611804b910 2024-09-10T21:24:22.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.787+0000 7f6113e00640 1 -- 172.21.15.17:0/3453483382 <== mon.1 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+76091 (secure 0 0 0) 0x7f6120094ca0 con 0x7f611804b910 2024-09-10T21:24:22.792 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.787+0000 7f2b09600640 1 -- 172.21.15.17:0/618969446 <== mon.1 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_debug_scatterstat}]=0 v11)=0 v11) ==== 122+0+0 (secure 0 0 0) 0x7f2afc02ca50 con 0x7f2b0c157720 2024-09-10T21:24:22.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.799+0000 7f2b12800640 1 -- 172.21.15.17:0/618969446 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2b0c157720 msgr2=0x7f2b0c1514c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:22.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.799+0000 7f2b12800640 1 --2- 172.21.15.17:0/618969446 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2b0c157720 0x7f2b0c1514c0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f2afc013f70 tx=0x7f2afc007410 comp rx=0 tx=0).stop 2024-09-10T21:24:22.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.803+0000 7f2b12800640 1 -- 172.21.15.17:0/618969446 shutdown_connections 2024-09-10T21:24:22.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.803+0000 7f2b12800640 1 --2- 172.21.15.17:0/618969446 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2b0c153250 0x7f2b0c0bed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:22.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.803+0000 7f2b12800640 1 --2- 172.21.15.17:0/618969446 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2b0c157720 0x7f2b0c1514c0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:22.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.803+0000 7f2b12800640 1 --2- 172.21.15.17:0/618969446 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f2b0c06efb0 0x7f2b0c152520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:22.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.803+0000 7f2b12800640 1 -- 172.21.15.17:0/618969446 >> 172.21.15.17:0/618969446 conn(0x7f2b0c06fc60 msgr2=0x7f2b0c155fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:22.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.803+0000 7f2b12800640 1 -- 172.21.15.17:0/618969446 shutdown_connections 2024-09-10T21:24:22.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.803+0000 7f2b12800640 1 -- 172.21.15.17:0/618969446 wait complete. 2024-09-10T21:24:22.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.811+0000 7f1946200640 1 -- 172.21.15.17:0/3082106515 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f19400c91e0 msgr2=0x7f19400c3de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:22.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.811+0000 7f1946200640 1 --2- 172.21.15.17:0/3082106515 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f19400c91e0 0x7f19400c3de0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f1930009840 tx=0x7f193001f010 comp rx=0 tx=0).stop 2024-09-10T21:24:22.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.811+0000 7f1946200640 1 -- 172.21.15.17:0/3082106515 shutdown_connections 2024-09-10T21:24:22.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.811+0000 7f1946200640 1 --2- 172.21.15.17:0/3082106515 >> v2:172.21.15.17:3300/0 conn(0x7f19400cda00 0x7f19400cac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:22.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.811+0000 7f1946200640 1 --2- 172.21.15.17:0/3082106515 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f19400c91e0 0x7f19400c3de0 secure :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f1930009840 tx=0x7f193001f010 comp rx=0 tx=0).stop 2024-09-10T21:24:22.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.811+0000 7f1946200640 1 -- 172.21.15.17:0/3082106515 >> 172.21.15.17:0/3082106515 conn(0x7f19400ccd10 msgr2=0x7f19400cd0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:22.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.815+0000 7f1aa6400640 1 -- 172.21.15.17:0/1365788862 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=mds_bal_fragment_size_max}] v 0) -- 0x7f1a78005590 con 0x7f1a84004810 2024-09-10T21:24:22.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.819+0000 7f1946200640 1 -- 172.21.15.17:0/3082106515 shutdown_connections 2024-09-10T21:24:22.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.819+0000 7f1946200640 1 -- 172.21.15.17:0/3082106515 wait complete. 2024-09-10T21:24:22.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.819+0000 7f1946200640 1 Processor -- start 2024-09-10T21:24:22.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.819+0000 7f1946200640 1 -- start start 2024-09-10T21:24:22.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.823+0000 7f1946200640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f19400c7690 0x7f1940160dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:22.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.823+0000 7f1946200640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f194006ee80 0x7f19401f5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:22.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.823+0000 7f1946200640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f19401f6530 0x7f194006f500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:22.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.823+0000 7f1946200640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f19400b4130 con 0x7f19401f6530 2024-09-10T21:24:22.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.823+0000 7f1946200640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f19400b3e50 con 0x7f194006ee80 2024-09-10T21:24:22.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.823+0000 7f1946200640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f19400c9e20 con 0x7f19400c7690 2024-09-10T21:24:22.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.823+0000 7f193f400640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f19400c7690 0x7f1940160dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:22.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.823+0000 7f193f400640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f19400c7690 0x7f1940160dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3301/0 says I am v2:172.21.15.17:51994/0 (socket says 172.21.15.17:51994) 2024-09-10T21:24:22.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.823+0000 7f193f400640 1 -- 172.21.15.17:0/2708284904 learned_addr learned my addr 172.21.15.17:0/2708284904 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:24:22.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.823+0000 7f193ea00640 1 --2- 172.21.15.17:0/2708284904 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f194006ee80 0x7f19401f5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:22.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.823+0000 7f193fe00640 1 --2- 172.21.15.17:0/2708284904 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f19401f6530 0x7f194006f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:22.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.823+0000 7f193f400640 1 -- 172.21.15.17:0/2708284904 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f194006ee80 msgr2=0x7f19401f5ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:22.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.823+0000 7f193f400640 1 --2- 172.21.15.17:0/2708284904 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f194006ee80 0x7f19401f5ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:22.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.823+0000 7f193f400640 1 -- 172.21.15.17:0/2708284904 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f19401f6530 msgr2=0x7f194006f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:22.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.823+0000 7f193f400640 1 --2- 172.21.15.17:0/2708284904 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f19401f6530 0x7f194006f500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:22.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.823+0000 7f193f400640 1 -- 172.21.15.17:0/2708284904 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19401f69b0 con 0x7f19400c7690 2024-09-10T21:24:22.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.823+0000 7f193f400640 1 --2- 172.21.15.17:0/2708284904 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f19400c7690 0x7f1940160dd0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f193001cc40 tx=0x7f19300a8c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:22.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.823+0000 7f1937400640 1 -- 172.21.15.17:0/2708284904 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19300b5040 con 0x7f19400c7690 2024-09-10T21:24:22.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.823+0000 7f1937400640 1 -- 172.21.15.17:0/2708284904 <== mon.2 v2:172.21.15.31:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f19300b1680 con 0x7f19400c7690 2024-09-10T21:24:22.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.823+0000 7f1937400640 1 -- 172.21.15.17:0/2708284904 <== mon.2 v2:172.21.15.31:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19300b1ab0 con 0x7f19400c7690 2024-09-10T21:24:22.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.827+0000 7f1a97e00640 1 -- 172.21.15.17:0/1365788862 <== mon.1 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_fragment_size_max}]=0 v12)=0 v12) ==== 127+0+0 (secure 0 0 0) 0x7f1a8800f3e0 con 0x7f1a84004810 2024-09-10T21:24:22.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.827+0000 7f1946200640 1 -- 172.21.15.17:0/2708284904 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19400cb120 con 0x7f19400c7690 2024-09-10T21:24:22.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.827+0000 7f1946200640 1 -- 172.21.15.17:0/2708284904 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f19401e9470 con 0x7f19400c7690 2024-09-10T21:24:22.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.827+0000 7f1937400640 1 -- 172.21.15.17:0/2708284904 <== mon.2 v2:172.21.15.31:3301/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7f19300b8050 con 0x7f19400c7690 2024-09-10T21:24:22.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.827+0000 7f1937400640 1 -- 172.21.15.17:0/2708284904 <== mon.2 v2:172.21.15.31:3301/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f19300c36c0 con 0x7f19400c7690 2024-09-10T21:24:22.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.835+0000 7f1aa6400640 1 -- 172.21.15.17:0/1365788862 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1a84004810 msgr2=0x7f1aa00cf320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:22.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.835+0000 7f1aa6400640 1 --2- 172.21.15.17:0/1365788862 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1a84004810 0x7f1aa00cf320 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f1a8800b550 tx=0x7f1a8800ba10 comp rx=0 tx=0).stop 2024-09-10T21:24:22.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.831+0000 7f1934c00640 1 -- 172.21.15.17:0/2708284904 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19401e9260 con 0x7f19400c7690 2024-09-10T21:24:22.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.835+0000 7f1937400640 1 -- 172.21.15.17:0/2708284904 <== mon.2 v2:172.21.15.31:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+76091 (secure 0 0 0) 0x7f19300c8060 con 0x7f19400c7690 2024-09-10T21:24:22.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.835+0000 7f0c26a00640 1 -- 172.21.15.17:0/1635190625 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_command([{prefix=config set, name=mds_bal_merge_size}] v 0) -- 0x7f0c3414da70 con 0x7f0c340c6ab0 2024-09-10T21:24:22.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.839+0000 7f1aa6400640 1 -- 172.21.15.17:0/1365788862 shutdown_connections 2024-09-10T21:24:22.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.839+0000 7f1aa6400640 1 --2- 172.21.15.17:0/1365788862 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f1aa00c5bc0 0x7f1aa00c6040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:22.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.839+0000 7f1aa6400640 1 --2- 172.21.15.17:0/1365788862 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1aa0155d30 0x7f1aa01561b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:22.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.839+0000 7f1aa6400640 1 --2- 172.21.15.17:0/1365788862 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1a84004810 0x7f1aa00cf320 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:22.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.839+0000 7f1aa6400640 1 -- 172.21.15.17:0/1365788862 >> 172.21.15.17:0/1365788862 conn(0x7f1aa00d0060 msgr2=0x7f1aa0156dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:22.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.839+0000 7f1aa6400640 1 -- 172.21.15.17:0/1365788862 shutdown_connections 2024-09-10T21:24:22.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.839+0000 7f1aa6400640 1 -- 172.21.15.17:0/1365788862 wait complete. 2024-09-10T21:24:22.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.851+0000 7f0c31600640 1 -- 172.21.15.17:0/1635190625 <== mon.2 v2:172.21.15.31:3301/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_merge_size}]=0 v13)=0 v13) ==== 116+0+0 (secure 0 0 0) 0x7f0c20012c80 con 0x7f0c340c6ab0 2024-09-10T21:24:22.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.855+0000 7f1aaaa00640 1 -- 172.21.15.17:0/1802547544 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command([{prefix=config set, name=mds_op_complaint_time}] v 0) -- 0x7f1ab406e2e0 con 0x7f1ab415bc30 2024-09-10T21:24:22.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.855+0000 7f0c26a00640 1 -- 172.21.15.17:0/1635190625 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f0c340c6ab0 msgr2=0x7f0c340b0a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:22.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.855+0000 7f0c26a00640 1 --2- 172.21.15.17:0/1635190625 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f0c340c6ab0 0x7f0c340b0a20 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f0c2000c390 tx=0x7f0c2000c850 comp rx=0 tx=0).stop 2024-09-10T21:24:22.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.855+0000 7f0c26a00640 1 -- 172.21.15.17:0/1635190625 shutdown_connections 2024-09-10T21:24:22.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.855+0000 7f0c26a00640 1 --2- 172.21.15.17:0/1635190625 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0c3414aa40 0x7f0c3414aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:22.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.855+0000 7f0c26a00640 1 --2- 172.21.15.17:0/1635190625 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0c3415de00 0x7f0c340c34b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:22.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.855+0000 7f0c26a00640 1 --2- 172.21.15.17:0/1635190625 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f0c340c6ab0 0x7f0c340b0a20 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:22.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.855+0000 7f0c26a00640 1 -- 172.21.15.17:0/1635190625 >> 172.21.15.17:0/1635190625 conn(0x7f0c34070110 msgr2=0x7f0c3415b770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:22.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.855+0000 7f0c26a00640 1 -- 172.21.15.17:0/1635190625 shutdown_connections 2024-09-10T21:24:22.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.867+0000 7f1ab1600640 1 -- 172.21.15.17:0/1802547544 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_op_complaint_time}]=0 v14)=0 v14) ==== 121+0+0 (secure 0 0 0) 0x7f1aac004c00 con 0x7f1ab415bc30 2024-09-10T21:24:22.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.855+0000 7f0c26a00640 1 -- 172.21.15.17:0/1635190625 wait complete. 2024-09-10T21:24:22.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.879+0000 7f1aaaa00640 1 -- 172.21.15.17:0/1802547544 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1ab415bc30 msgr2=0x7f1ab414cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:22.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.879+0000 7f1aaaa00640 1 --2- 172.21.15.17:0/1802547544 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1ab415bc30 0x7f1ab414cc70 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f1aac0ac000 tx=0x7f1aac0a56d0 comp rx=0 tx=0).stop 2024-09-10T21:24:22.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.879+0000 7f1aaaa00640 1 -- 172.21.15.17:0/1802547544 shutdown_connections 2024-09-10T21:24:22.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.879+0000 7f1aaaa00640 1 --2- 172.21.15.17:0/1802547544 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1ab415bc30 0x7f1ab414cc70 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:22.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.879+0000 7f1aaaa00640 1 --2- 172.21.15.17:0/1802547544 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1ab40c5f60 0x7f1ab41f28a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:22.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.879+0000 7f1aaaa00640 1 --2- 172.21.15.17:0/1802547544 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f1ab4073d00 0x7f1ab40c72a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:22.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.879+0000 7f1aaaa00640 1 -- 172.21.15.17:0/1802547544 >> 172.21.15.17:0/1802547544 conn(0x7f1ab406fc60 msgr2=0x7f1ab40c9eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:22.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.883+0000 7f1aaaa00640 1 -- 172.21.15.17:0/1802547544 shutdown_connections 2024-09-10T21:24:22.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.883+0000 7f1aaaa00640 1 -- 172.21.15.17:0/1802547544 wait complete. 2024-09-10T21:24:22.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.887+0000 7fd40ce00640 1 -- 172.21.15.17:0/2522117851 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_command([{prefix=config set, name=mds_bal_split_bits}] v 0) -- 0x7fd408069e60 con 0x7fd408073d00 2024-09-10T21:24:22.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.891+0000 7fd3ff400640 1 -- 172.21.15.17:0/2522117851 <== mon.2 v2:172.21.15.31:3301/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_split_bits}]=0 v15)=0 v15) ==== 116+0+0 (secure 0 0 0) 0x7fd3f8014030 con 0x7fd408073d00 2024-09-10T21:24:22.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.891+0000 7fd40ce00640 1 -- 172.21.15.17:0/2522117851 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd408073d00 msgr2=0x7fd4081591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:22.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.891+0000 7fd40ce00640 1 --2- 172.21.15.17:0/2522117851 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd408073d00 0x7fd4081591e0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fd3f800c940 tx=0x7fd3f800ce00 comp rx=0 tx=0).stop 2024-09-10T21:24:22.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.895+0000 7fd40ce00640 1 -- 172.21.15.17:0/2522117851 shutdown_connections 2024-09-10T21:24:22.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.895+0000 7fd40ce00640 1 --2- 172.21.15.17:0/2522117851 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd4080c5f60 0x7fd408157e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:22.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.895+0000 7fd40ce00640 1 --2- 172.21.15.17:0/2522117851 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd408073d00 0x7fd4081591e0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:22.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.895+0000 7fd40ce00640 1 --2- 172.21.15.17:0/2522117851 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd40806e140 0x7fd408155130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:22.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.895+0000 7fd40ce00640 1 -- 172.21.15.17:0/2522117851 >> 172.21.15.17:0/2522117851 conn(0x7fd40806fc60 msgr2=0x7fd40815b760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:22.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.895+0000 7f1464c00640 1 -- 172.21.15.17:0/2205813428 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command([{prefix=config set, name=mds_dir_prefetch}] v 0) -- 0x7f1450005590 con 0x7f146c0c8f80 2024-09-10T21:24:22.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.903+0000 7fd40ce00640 1 -- 172.21.15.17:0/2522117851 shutdown_connections 2024-09-10T21:24:22.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.903+0000 7fd40ce00640 1 -- 172.21.15.17:0/2522117851 wait complete. 2024-09-10T21:24:22.909 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'mds_bal_merge_size', '5'], check_status=True, hostname='smithi017') 2024-09-10T21:24:22.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.919+0000 7f0054c00640 1 -- 172.21.15.17:0/3250427795 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=osd_op_complaint_time}] v 0) -- 0x7f0038005590 con 0x7f005c032850 2024-09-10T21:24:22.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.919+0000 7f1467400640 1 -- 172.21.15.17:0/2205813428 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_dir_prefetch}]=0 v16)=0 v16) ==== 117+0+0 (secure 0 0 0) 0x7f1468003600 con 0x7f146c0c8f80 2024-09-10T21:24:22.931 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'mds_bal_split_bits', '3'], check_status=True, hostname='smithi017') 2024-09-10T21:24:22.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.927+0000 7f1464c00640 1 -- 172.21.15.17:0/2205813428 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f146c0c8f80 msgr2=0x7f146c0db740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:22.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.927+0000 7f1464c00640 1 --2- 172.21.15.17:0/2205813428 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f146c0c8f80 0x7f146c0db740 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f146c009380 tx=0x7f1468010bd0 comp rx=0 tx=0).stop 2024-09-10T21:24:22.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.927+0000 7f1464c00640 1 -- 172.21.15.17:0/2205813428 shutdown_connections 2024-09-10T21:24:22.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.927+0000 7f1464c00640 1 --2- 172.21.15.17:0/2205813428 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f146c0c8f80 0x7f146c0db740 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:22.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.927+0000 7f1464c00640 1 --2- 172.21.15.17:0/2205813428 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f146c0ca920 0x7f146c0c8a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:22.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.927+0000 7f1464c00640 1 --2- 172.21.15.17:0/2205813428 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f145c004df0 0x7f146c168eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:22.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.927+0000 7f1464c00640 1 -- 172.21.15.17:0/2205813428 >> 172.21.15.17:0/2205813428 conn(0x7f146c04fc40 msgr2=0x7f146c0d3470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:22.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.935+0000 7f0057400640 1 -- 172.21.15.17:0/3250427795 <== mon.1 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_op_complaint_time}]=0 v17)=0 v17) ==== 121+0+0 (secure 0 0 0) 0x7f0050005520 con 0x7f005c032850 2024-09-10T21:24:22.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.935+0000 7f1464c00640 1 -- 172.21.15.17:0/2205813428 shutdown_connections 2024-09-10T21:24:22.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.935+0000 7f1464c00640 1 -- 172.21.15.17:0/2205813428 wait complete. 2024-09-10T21:24:22.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.939+0000 7f0054c00640 1 -- 172.21.15.17:0/3250427795 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f005c032850 msgr2=0x7f005c048470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:22.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.939+0000 7f0054c00640 1 --2- 172.21.15.17:0/3250427795 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f005c032850 0x7f005c048470 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f0050007890 tx=0x7f0050007d50 comp rx=0 tx=0).stop 2024-09-10T21:24:22.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.939+0000 7f0054c00640 1 -- 172.21.15.17:0/3250427795 shutdown_connections 2024-09-10T21:24:22.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.939+0000 7f0054c00640 1 --2- 172.21.15.17:0/3250427795 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f005c049e50 0x7f005c0489b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:22.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.939+0000 7f0054c00640 1 --2- 172.21.15.17:0/3250427795 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f005c032850 0x7f005c048470 secure :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f0050007890 tx=0x7f0050007d50 comp rx=0 tx=0).stop 2024-09-10T21:24:22.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.939+0000 7f0054c00640 1 --2- 172.21.15.17:0/3250427795 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f004c005220 0x7f005c0cbab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:22.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.939+0000 7f0054c00640 1 -- 172.21.15.17:0/3250427795 >> 172.21.15.17:0/3250427795 conn(0x7f005c04fde0 msgr2=0x7f005c0db7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:22.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.943+0000 7f0054c00640 1 -- 172.21.15.17:0/3250427795 shutdown_connections 2024-09-10T21:24:22.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.943+0000 7f0054c00640 1 -- 172.21.15.17:0/3250427795 wait complete. 2024-09-10T21:24:22.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.943+0000 7f614a400640 1 -- 172.21.15.17:0/2923521647 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_pool_no_app}] v 0) -- 0x7f614414da70 con 0x7f61440c6ab0 2024-09-10T21:24:22.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.947+0000 7f613b400640 1 -- 172.21.15.17:0/2923521647 <== mon.1 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_pool_no_app}]=0 v18)=0 v18) ==== 125+0+0 (secure 0 0 0) 0x7f61340046d0 con 0x7f61440c6ab0 2024-09-10T21:24:22.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.951+0000 7f614a400640 1 -- 172.21.15.17:0/2923521647 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f61440c6ab0 msgr2=0x7f6144169480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:22.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.951+0000 7f614a400640 1 --2- 172.21.15.17:0/2923521647 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f61440c6ab0 0x7f6144169480 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f61340b0040 tx=0x7f6134005e70 comp rx=0 tx=0).stop 2024-09-10T21:24:22.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.951+0000 7f614a400640 1 -- 172.21.15.17:0/2923521647 shutdown_connections 2024-09-10T21:24:22.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.951+0000 7f614a400640 1 --2- 172.21.15.17:0/2923521647 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f614415de00 0x7f614414cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:22.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.951+0000 7f614a400640 1 --2- 172.21.15.17:0/2923521647 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f61440c72b0 0x7f61440b1600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:22.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.951+0000 7f614a400640 1 --2- 172.21.15.17:0/2923521647 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f61440c6ab0 0x7f6144169480 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:22.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.951+0000 7f614a400640 1 -- 172.21.15.17:0/2923521647 >> 172.21.15.17:0/2923521647 conn(0x7f6144070110 msgr2=0x7f614414f4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:22.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.951+0000 7f614a400640 1 -- 172.21.15.17:0/2923521647 shutdown_connections 2024-09-10T21:24:22.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.951+0000 7f614a400640 1 -- 172.21.15.17:0/2923521647 wait complete. 2024-09-10T21:24:22.958 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'mds_bal_split_size', '100'], check_status=True, hostname='smithi017') 2024-09-10T21:24:22.958 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'mds_debug_frag', 'True'], check_status=True, hostname='smithi017') 2024-09-10T21:24:22.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.955+0000 7f11f3e00640 1 -- 172.21.15.17:0/675567414 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_command([{prefix=config set, name=rados_mon_op_timeout}] v 0) -- 0x7f11ec04d9f0 con 0x7f11ec035690 2024-09-10T21:24:22.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.959+0000 7f11eaa00640 1 -- 172.21.15.17:0/675567414 <== mon.2 v2:172.21.15.31:3301/0 7 ==== mon_command_ack([{prefix=config set, name=rados_mon_op_timeout}]=0 v19)=0 v19) ==== 120+0+0 (secure 0 0 0) 0x7f11f4095450 con 0x7f11ec035690 2024-09-10T21:24:22.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.967+0000 7f11f3e00640 1 -- 172.21.15.17:0/675567414 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f11ec035690 msgr2=0x7f11ec0db660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:22.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.967+0000 7f11f3e00640 1 --2- 172.21.15.17:0/675567414 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f11ec035690 0x7f11ec0db660 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f11f406c200 tx=0x7f11f406a670 comp rx=0 tx=0).stop 2024-09-10T21:24:22.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.967+0000 7f11f3e00640 1 -- 172.21.15.17:0/675567414 shutdown_connections 2024-09-10T21:24:22.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.967+0000 7f11f3e00640 1 --2- 172.21.15.17:0/675567414 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f11ec04b440 0x7f11ec0d45b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:22.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.967+0000 7f11f3e00640 1 --2- 172.21.15.17:0/675567414 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f11ec0462a0 0x7f11ec0d61f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:22.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.967+0000 7f11f3e00640 1 --2- 172.21.15.17:0/675567414 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f11ec035690 0x7f11ec0db660 secure :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f11f406c200 tx=0x7f11f406a670 comp rx=0 tx=0).stop 2024-09-10T21:24:22.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.967+0000 7f37caa00640 1 -- 172.21.15.17:0/1136092739 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_command([{prefix=config set, name=mds_debug_frag}] v 0) -- 0x7f37d8069e60 con 0x7f37d80c08c0 2024-09-10T21:24:22.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.967+0000 7f11f3e00640 1 -- 172.21.15.17:0/675567414 >> 172.21.15.17:0/675567414 conn(0x7f11ec04fc10 msgr2=0x7f11ec04e1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:22.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.967+0000 7f11f3e00640 1 -- 172.21.15.17:0/675567414 shutdown_connections 2024-09-10T21:24:22.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.967+0000 7f11f3e00640 1 -- 172.21.15.17:0/675567414 wait complete. 2024-09-10T21:24:22.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.971+0000 7f5cef400640 1 -- 172.21.15.17:0/419032460 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=mds_verify_scatter}] v 0) -- 0x7f5d04069e60 con 0x7f5d04072a30 2024-09-10T21:24:22.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.975+0000 7f37d5600640 1 -- 172.21.15.17:0/1136092739 <== mon.2 v2:172.21.15.31:3301/0 7 ==== mon_command_ack([{prefix=config set, name=mds_debug_frag}]=0 v20)=0 v20) ==== 115+0+0 (secure 0 0 0) 0x7f37c40098d0 con 0x7f37d80c08c0 2024-09-10T21:24:22.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.979+0000 7f37caa00640 1 -- 172.21.15.17:0/1136092739 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f37d80c08c0 msgr2=0x7f37d80c22c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:22.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.979+0000 7f37caa00640 1 --2- 172.21.15.17:0/1136092739 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f37d80c08c0 0x7f37d80c22c0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f37d80b1930 tx=0x7f37c400da90 comp rx=0 tx=0).stop 2024-09-10T21:24:22.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.979+0000 7f37caa00640 1 -- 172.21.15.17:0/1136092739 shutdown_connections 2024-09-10T21:24:22.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.979+0000 7f37caa00640 1 --2- 172.21.15.17:0/1136092739 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f37d80c2800 0x7f37d80c4aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:22.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.979+0000 7f37caa00640 1 --2- 172.21.15.17:0/1136092739 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f37d80c08c0 0x7f37d80c22c0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:22.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.979+0000 7f37caa00640 1 --2- 172.21.15.17:0/1136092739 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f37c0003ab0 0x7f37d81fd4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:22.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.979+0000 7f37caa00640 1 -- 172.21.15.17:0/1136092739 >> 172.21.15.17:0/1136092739 conn(0x7f37d806fc60 msgr2=0x7f37d8153f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:22.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.979+0000 7f37caa00640 1 -- 172.21.15.17:0/1136092739 shutdown_connections 2024-09-10T21:24:22.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.979+0000 7f37caa00640 1 -- 172.21.15.17:0/1136092739 wait complete. 2024-09-10T21:24:22.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.987+0000 7f5cfa000640 1 -- 172.21.15.17:0/419032460 <== mon.1 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_verify_scatter}]=0 v21)=0 v21) ==== 119+0+0 (secure 0 0 0) 0x7f5cf4019c20 con 0x7f5d04072a30 2024-09-10T21:24:22.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.987+0000 7f5cef400640 1 -- 172.21.15.17:0/419032460 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5d04072a30 msgr2=0x7f5d041475c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:22.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.987+0000 7f5cef400640 1 --2- 172.21.15.17:0/419032460 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5d04072a30 0x7f5d041475c0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f5cf400b440 tx=0x7f5cf400b840 comp rx=0 tx=0).stop 2024-09-10T21:24:22.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.987+0000 7f5cef400640 1 -- 172.21.15.17:0/419032460 shutdown_connections 2024-09-10T21:24:22.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.987+0000 7f5cef400640 1 --2- 172.21.15.17:0/419032460 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5d04072e10 0x7f5d040c1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:22.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.987+0000 7f5cef400640 1 --2- 172.21.15.17:0/419032460 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5d04072a30 0x7f5d041475c0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:22.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.987+0000 7f5cef400640 1 --2- 172.21.15.17:0/419032460 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f5d0406ef80 0x7f5d040c0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:22.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.987+0000 7f5cef400640 1 -- 172.21.15.17:0/419032460 >> 172.21.15.17:0/419032460 conn(0x7f5d0406fc60 msgr2=0x7f5d040c3b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:22.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.991+0000 7f5cef400640 1 -- 172.21.15.17:0/419032460 shutdown_connections 2024-09-10T21:24:22.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.991+0000 7f5cef400640 1 -- 172.21.15.17:0/419032460 wait complete. 2024-09-10T21:24:23.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:22.999+0000 7f6126c00640 1 -- 172.21.15.17:0/3453483382 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command([{prefix=config set, name=rados_osd_op_timeout}] v 0) -- 0x7f60fc005590 con 0x7f611804b910 2024-09-10T21:24:23.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:23.003+0000 7f6113e00640 1 -- 172.21.15.17:0/3453483382 <== mon.1 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_osd_op_timeout}]=0 v22)=0 v22) ==== 120+0+0 (secure 0 0 0) 0x7f6120095450 con 0x7f611804b910 2024-09-10T21:24:23.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:23.003+0000 7f6126c00640 1 -- 172.21.15.17:0/3453483382 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f611804b910 msgr2=0x7f61181815f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:23.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:23.003+0000 7f6126c00640 1 --2- 172.21.15.17:0/3453483382 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f611804b910 0x7f61181815f0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f612006af00 tx=0x7f6120000b10 comp rx=0 tx=0).stop 2024-09-10T21:24:23.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:23.003+0000 7f6126c00640 1 -- 172.21.15.17:0/3453483382 shutdown_connections 2024-09-10T21:24:23.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:23.003+0000 7f6126c00640 1 --2- 172.21.15.17:0/3453483382 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f611804b910 0x7f61181815f0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:23.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:23.003+0000 7f6126c00640 1 --2- 172.21.15.17:0/3453483382 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f611804b4a0 0x7f6118047ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:23.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:23.003+0000 7f6126c00640 1 --2- 172.21.15.17:0/3453483382 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f61180397e0 0x7f6118053700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:23.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:23.003+0000 7f6126c00640 1 -- 172.21.15.17:0/3453483382 >> 172.21.15.17:0/3453483382 conn(0x7f6118054030 msgr2=0x7f61180d57a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:23.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:23.003+0000 7f6126c00640 1 -- 172.21.15.17:0/3453483382 shutdown_connections 2024-09-10T21:24:23.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:23.003+0000 7f6126c00640 1 -- 172.21.15.17:0/3453483382 wait complete. 2024-09-10T21:24:23.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:23.007+0000 7f3b87e00640 1 -- 172.21.15.17:0/1784954793 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command([{prefix=config set, name=mon_op_complaint_time}] v 0) -- 0x7f3b9c069e60 con 0x7f3b9c06fdf0 2024-09-10T21:24:23.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:23.011+0000 7f3b92a00640 1 -- 172.21.15.17:0/1784954793 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_op_complaint_time}]=0 v23)=0 v23) ==== 121+0+0 (secure 0 0 0) 0x7f3b94014c80 con 0x7f3b9c06fdf0 2024-09-10T21:24:23.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:23.011+0000 7f3b87e00640 1 -- 172.21.15.17:0/1784954793 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3b9c06fdf0 msgr2=0x7f3b9c14d3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:23.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:23.011+0000 7f3b87e00640 1 --2- 172.21.15.17:0/1784954793 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3b9c06fdf0 0x7f3b9c14d3b0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f3b9400ba50 tx=0x7f3b9400bf10 comp rx=0 tx=0).stop 2024-09-10T21:24:23.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:23.011+0000 7f3b87e00640 1 -- 172.21.15.17:0/1784954793 shutdown_connections 2024-09-10T21:24:23.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:23.011+0000 7f3b87e00640 1 --2- 172.21.15.17:0/1784954793 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3b9c0b1020 0x7f3b9c15c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:23.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:23.011+0000 7f3b87e00640 1 --2- 172.21.15.17:0/1784954793 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3b9c0701d0 0x7f3b9c1f2890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:23.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:23.011+0000 7f3b87e00640 1 --2- 172.21.15.17:0/1784954793 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3b9c06fdf0 0x7f3b9c14d3b0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:23.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:23.011+0000 7f3b87e00640 1 -- 172.21.15.17:0/1784954793 >> 172.21.15.17:0/1784954793 conn(0x7f3b9c06e410 msgr2=0x7f3b9c154590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:23.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:23.011+0000 7f3b87e00640 1 -- 172.21.15.17:0/1784954793 shutdown_connections 2024-09-10T21:24:23.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:23.011+0000 7f3b87e00640 1 -- 172.21.15.17:0/1784954793 wait complete. 2024-09-10T21:24:23.018 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'mds_debug_scatterstat', 'True'], check_status=True, hostname='smithi017') 2024-09-10T21:24:23.018 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'mds_op_complaint_time', '180'], check_status=True, hostname='smithi017') 2024-09-10T21:24:23.018 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'mds_verify_scatter', 'True'], check_status=True, hostname='smithi017') 2024-09-10T21:24:23.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:23.015+0000 7f1934c00640 1 -- 172.21.15.17:0/2708284904 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_command([{prefix=config set, name=osd_op_complaint_time}] v 0) -- 0x7f19400cb900 con 0x7f19400c7690 2024-09-10T21:24:23.020 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'mds_dir_prefetch', 'True'], check_status=True, hostname='smithi017') 2024-09-10T21:24:23.020 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'osd_op_complaint_time', '180'], check_status=True, hostname='smithi017') 2024-09-10T21:24:23.020 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'rados_mon_op_timeout', '900'], check_status=True, hostname='smithi017') 2024-09-10T21:24:23.020 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'rados_osd_op_timeout', '900'], check_status=True, hostname='smithi017') 2024-09-10T21:24:23.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:23.019+0000 7f1937400640 1 -- 172.21.15.17:0/2708284904 <== mon.2 v2:172.21.15.31:3301/0 7 ==== mon_command_ack([{prefix=config set, name=osd_op_complaint_time}]=0 v24)=0 v24) ==== 121+0+0 (secure 0 0 0) 0x7f19401f69b0 con 0x7f19400c7690 2024-09-10T21:24:23.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:23.023+0000 7f1934c00640 1 -- 172.21.15.17:0/2708284904 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f19400c7690 msgr2=0x7f1940160dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:23.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:23.023+0000 7f1934c00640 1 --2- 172.21.15.17:0/2708284904 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f19400c7690 0x7f1940160dd0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f193001cc40 tx=0x7f19300a8c20 comp rx=0 tx=0).stop 2024-09-10T21:24:23.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:23.023+0000 7f1934c00640 1 -- 172.21.15.17:0/2708284904 shutdown_connections 2024-09-10T21:24:23.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:23.023+0000 7f1934c00640 1 --2- 172.21.15.17:0/2708284904 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f19401f6530 0x7f194006f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:23.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:23.023+0000 7f1934c00640 1 --2- 172.21.15.17:0/2708284904 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f194006ee80 0x7f19401f5ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:23.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:23.023+0000 7f1934c00640 1 --2- 172.21.15.17:0/2708284904 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f19400c7690 0x7f1940160dd0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:23.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:23.023+0000 7f1934c00640 1 -- 172.21.15.17:0/2708284904 >> 172.21.15.17:0/2708284904 conn(0x7f19400ccd10 msgr2=0x7f1940163330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:23.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:23.023+0000 7f1934c00640 1 -- 172.21.15.17:0/2708284904 shutdown_connections 2024-09-10T21:24:23.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:23.023+0000 7f1934c00640 1 -- 172.21.15.17:0/2708284904 wait complete. 2024-09-10T21:24:23.038 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mon', 'mon_op_complaint_time', '120'], check_status=True, hostname='smithi017') 2024-09-10T21:24:23.044 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mon', 'mon_warn_on_pool_no_app', 'False'], check_status=True, hostname='smithi017') 2024-09-10T21:24:23.044 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'osd', 'osd_op_complaint_time', '180'], check_status=True, hostname='smithi017') 2024-09-10T21:24:23.050 INFO:tasks.ceph:Setting crush tunables to default 2024-09-10T21:24:23.050 DEBUG:teuthology.orchestra.run.smithi017:> sudo ceph --cluster ceph osd crush tunables default 2024-09-10T21:24:23.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:23.231+0000 7fecd0800640 1 -- 172.21.15.17:0/1960667027 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fecc80c82f0 msgr2=0x7fecc8071960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:23.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:23.231+0000 7fecd0800640 1 --2- 172.21.15.17:0/1960667027 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fecc80c82f0 0x7fecc8071960 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fecb8009840 tx=0x7fecb801f010 comp rx=0 tx=0).stop 2024-09-10T21:24:23.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:23.235+0000 7fecd0800640 1 -- 172.21.15.17:0/1960667027 shutdown_connections 2024-09-10T21:24:23.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:23.235+0000 7fecd0800640 1 --2- 172.21.15.17:0/1960667027 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fecc80c82f0 0x7fecc8071960 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:23.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:23.235+0000 7fecd0800640 1 -- 172.21.15.17:0/1960667027 >> 172.21.15.17:0/1960667027 conn(0x7fecc806dd30 msgr2=0x7fecc806e140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:23.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:23.235+0000 7fecd0800640 1 -- 172.21.15.17:0/1960667027 shutdown_connections 2024-09-10T21:24:23.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:23.235+0000 7fecd0800640 1 -- 172.21.15.17:0/1960667027 wait complete. 2024-09-10T21:24:23.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:23.235+0000 7fecd0800640 1 Processor -- start 2024-09-10T21:24:23.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:23.235+0000 7fecd0800640 1 -- start start 2024-09-10T21:24:23.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:23.235+0000 7fecd0800640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fecc80c82f0 0x7fecc815a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:23.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:23.235+0000 7fecd0800640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fecc8152400 0x7fecc81e1c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:23.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:23.235+0000 7fecd0800640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fecc81527e0 0x7fecc81eef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:23.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:23.235+0000 7fecd0800640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fecc81594c0 con 0x7fecc8152400 2024-09-10T21:24:23.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:23.235+0000 7fecd0800640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fecc8073bf0 con 0x7fecc80c82f0 2024-09-10T21:24:23.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:23.235+0000 7fecd0800640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7fecc80c87c0 con 0x7fecc81527e0 2024-09-10T21:24:23.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:23.235+0000 7feccd400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fecc8152400 0x7fecc81e1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:23.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:23.235+0000 7feccd400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fecc8152400 0x7fecc81e1c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35936/0 (socket says 172.21.15.17:35936) 2024-09-10T21:24:23.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:23.235+0000 7feccd400640 1 -- 172.21.15.17:0/257442450 learned_addr learned my addr 172.21.15.17:0/257442450 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:24:23.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:23.235+0000 7feccde00640 1 --2- 172.21.15.17:0/257442450 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fecc80c82f0 0x7fecc815a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:23.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:23.235+0000 7fecce800640 1 --2- 172.21.15.17:0/257442450 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fecc81527e0 0x7fecc81eef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:23.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:23.235+0000 7feccd400640 1 -- 172.21.15.17:0/257442450 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fecc81527e0 msgr2=0x7fecc81eef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:23.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:23.235+0000 7feccd400640 1 --2- 172.21.15.17:0/257442450 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fecc81527e0 0x7fecc81eef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:23.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:23.235+0000 7feccd400640 1 -- 172.21.15.17:0/257442450 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fecc80c82f0 msgr2=0x7fecc815a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:23.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:23.235+0000 7feccd400640 1 --2- 172.21.15.17:0/257442450 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fecc80c82f0 0x7fecc815a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:23.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:23.235+0000 7feccd400640 1 -- 172.21.15.17:0/257442450 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecc81f1e00 con 0x7fecc8152400 2024-09-10T21:24:23.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:23.235+0000 7feccd400640 1 --2- 172.21.15.17:0/257442450 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fecc8152400 0x7fecc81e1c40 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fecb801bc20 tx=0x7fecb8005d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:23.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:23.239+0000 7fecc2000640 1 -- 172.21.15.17:0/257442450 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecb80b5040 con 0x7fecc8152400 2024-09-10T21:24:23.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:23.239+0000 7fecd0800640 1 -- 172.21.15.17:0/257442450 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fecc80c59f0 con 0x7fecc8152400 2024-09-10T21:24:23.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:23.239+0000 7fecc2000640 1 -- 172.21.15.17:0/257442450 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fecb8005220 con 0x7fecc8152400 2024-09-10T21:24:23.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:23.239+0000 7fecc2000640 1 -- 172.21.15.17:0/257442450 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecb80046d0 con 0x7fecc8152400 2024-09-10T21:24:23.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:23.239+0000 7fecd0800640 1 -- 172.21.15.17:0/257442450 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fecc80c5fb0 con 0x7fecc8152400 2024-09-10T21:24:23.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:23.239+0000 7fecc2000640 1 -- 172.21.15.17:0/257442450 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7fecb80c2060 con 0x7fecc8152400 2024-09-10T21:24:23.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:23.239+0000 7fecc2000640 1 -- 172.21.15.17:0/257442450 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fecb80b0440 con 0x7fecc8152400 2024-09-10T21:24:23.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:23.239+0000 7fecd0800640 1 -- 172.21.15.17:0/257442450 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fecc8071390 con 0x7fecc8152400 2024-09-10T21:24:23.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:23.239+0000 7fecc2000640 1 -- 172.21.15.17:0/257442450 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+76091 (secure 0 0 0) 0x7fecb8018b70 con 0x7fecc8152400 2024-09-10T21:24:23.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:23.307+0000 7fecd0800640 1 -- 172.21.15.17:0/257442450 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7fecc8155f80 con 0x7fecc8152400 2024-09-10T21:24:23.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:23.307+0000 7fecc2000640 1 -- 172.21.15.17:0/257442450 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) ==== 124+0+0 (secure 0 0 0) 0x7fecb8003600 con 0x7fecc8152400 2024-09-10T21:24:23.312 INFO:teuthology.orchestra.run.smithi017.stderr:adjusted tunables profile to default 2024-09-10T21:24:23.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:23.311+0000 7fecd0800640 1 -- 172.21.15.17:0/257442450 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fecc8152400 msgr2=0x7fecc81e1c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:23.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:23.311+0000 7fecd0800640 1 --2- 172.21.15.17:0/257442450 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fecc8152400 0x7fecc81e1c40 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fecb801bc20 tx=0x7fecb8005d20 comp rx=0 tx=0).stop 2024-09-10T21:24:23.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:23.311+0000 7fecd0800640 1 -- 172.21.15.17:0/257442450 shutdown_connections 2024-09-10T21:24:23.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:23.311+0000 7fecd0800640 1 --2- 172.21.15.17:0/257442450 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fecc81527e0 0x7fecc81eef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:23.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:23.311+0000 7fecd0800640 1 --2- 172.21.15.17:0/257442450 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fecc8152400 0x7fecc81e1c40 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:23.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:23.311+0000 7fecd0800640 1 --2- 172.21.15.17:0/257442450 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fecc80c82f0 0x7fecc815a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:23.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:23.311+0000 7fecd0800640 1 -- 172.21.15.17:0/257442450 >> 172.21.15.17:0/257442450 conn(0x7fecc806dd30 msgr2=0x7fecc8157680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:23.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:23.311+0000 7fecd0800640 1 -- 172.21.15.17:0/257442450 shutdown_connections 2024-09-10T21:24:23.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:23.311+0000 7fecd0800640 1 -- 172.21.15.17:0/257442450 wait complete. 2024-09-10T21:24:23.332 INFO:tasks.ceph:check_enable_crimson: False 2024-09-10T21:24:23.332 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-09-10T21:24:23.332 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-09-10T21:24:23.332 DEBUG:teuthology.orchestra.run.smithi017:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-09-10T21:24:23.352 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-09-10T21:24:23.352 DEBUG:teuthology.orchestra.run.smithi017:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-09-10T21:24:23.419 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-09-10T21:24:23.419 DEBUG:teuthology.orchestra.run.smithi017:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-09-10T21:24:23.483 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-09-10T21:24:23.483 DEBUG:teuthology.orchestra.run.smithi017:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-09-10T21:24:23.543 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-10T21:24:23.543 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-09-10T21:24:23.563 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-10T21:24:23.564 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-09-10T21:24:23.625 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-10T21:24:23.625 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-09-10T21:24:23.686 DEBUG:teuthology.orchestra.run.smithi031:> set -ex 2024-09-10T21:24:23.686 DEBUG:teuthology.orchestra.run.smithi031:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-09-10T21:24:23.745 DEBUG:teuthology.orchestra.run.smithi031:> sudo ceph --cluster ceph osd new 0f5fabc0-434e-4066-b430-233969a548f2 0 2024-09-10T21:24:23.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:23.928+0000 7f65f1800640 1 -- 172.21.15.31:0/2777726348 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f65ec0ccce0 msgr2=0x7f65ec161d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:23.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:23.928+0000 7f65f1800640 1 --2- 172.21.15.31:0/2777726348 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f65ec0ccce0 0x7f65ec161d60 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f65ec0c5340 tx=0x7f65e401cd80 comp rx=0 tx=0).stop 2024-09-10T21:24:23.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:23.928+0000 7f65f1800640 1 -- 172.21.15.31:0/2777726348 shutdown_connections 2024-09-10T21:24:23.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:23.928+0000 7f65f1800640 1 --2- 172.21.15.31:0/2777726348 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f65ec0ccce0 0x7f65ec161d60 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:23.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:23.928+0000 7f65f1800640 1 --2- 172.21.15.31:0/2777726348 >> v2:172.21.15.17:3300/0 conn(0x7f65ec0cc900 0x7f65ec0c8890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:23.934 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:23.928+0000 7f65f1800640 1 -- 172.21.15.31:0/2777726348 >> 172.21.15.31:0/2777726348 conn(0x7f65ec06cc20 msgr2=0x7f65ec06d030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:23.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:23.928+0000 7f65f1800640 1 -- 172.21.15.31:0/2777726348 shutdown_connections 2024-09-10T21:24:23.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:23.928+0000 7f65f1800640 1 -- 172.21.15.31:0/2777726348 wait complete. 2024-09-10T21:24:23.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:23.932+0000 7f65f1800640 1 Processor -- start 2024-09-10T21:24:23.935 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:23.932+0000 7f65f1800640 1 -- start start 2024-09-10T21:24:23.936 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:23.932+0000 7f65f1800640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f65ec068990 0x7f65ec14ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:23.937 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:23.932+0000 7f65f1800640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f65ec0cc900 0x7f65ec1f7760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:23.937 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:23.932+0000 7f65f1800640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f65ec0ccce0 0x7f65ec1f4070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:23.937 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:23.932+0000 7f65f1800640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f65ec073ff0 con 0x7f65ec068990 2024-09-10T21:24:23.937 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:23.932+0000 7f65f1800640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f65ec073e70 con 0x7f65ec0ccce0 2024-09-10T21:24:23.937 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:23.932+0000 7f65f1800640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f65ec074170 con 0x7f65ec0cc900 2024-09-10T21:24:23.937 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:23.932+0000 7f65eb400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f65ec0ccce0 0x7f65ec1f4070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:23.937 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:23.932+0000 7f65eaa00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f65ec068990 0x7f65ec14ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:23.937 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:23.932+0000 7f65e3e00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f65ec0cc900 0x7f65ec1f7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:23.937 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:23.932+0000 7f65eaa00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f65ec068990 0x7f65ec14ebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.31:35784/0 (socket says 172.21.15.31:35784) 2024-09-10T21:24:23.937 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:23.932+0000 7f65eaa00640 1 -- 172.21.15.31:0/1800696070 learned_addr learned my addr 172.21.15.31:0/1800696070 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-10T21:24:23.938 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:23.932+0000 7f65eaa00640 1 -- 172.21.15.31:0/1800696070 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f65ec0cc900 msgr2=0x7f65ec1f7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:23.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:23.932+0000 7f65eaa00640 1 --2- 172.21.15.31:0/1800696070 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f65ec0cc900 0x7f65ec1f7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:23.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:23.932+0000 7f65eaa00640 1 -- 172.21.15.31:0/1800696070 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f65ec0ccce0 msgr2=0x7f65ec1f4070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:23.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:23.932+0000 7f65eaa00640 1 --2- 172.21.15.31:0/1800696070 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f65ec0ccce0 0x7f65ec1f4070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:23.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:23.932+0000 7f65eaa00640 1 -- 172.21.15.31:0/1800696070 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65ec14cf00 con 0x7f65ec068990 2024-09-10T21:24:23.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:23.932+0000 7f65eaa00640 1 --2- 172.21.15.31:0/1800696070 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f65ec068990 0x7f65ec14ebc0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f65dc00ea20 tx=0x7f65dc00eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:23.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:23.932+0000 7f65e2a00640 1 -- 172.21.15.31:0/1800696070 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65dc00cd80 con 0x7f65ec068990 2024-09-10T21:24:23.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:23.932+0000 7f65e2a00640 1 -- 172.21.15.31:0/1800696070 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f65dc010430 con 0x7f65ec068990 2024-09-10T21:24:23.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:23.932+0000 7f65e2a00640 1 -- 172.21.15.31:0/1800696070 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65dc010860 con 0x7f65ec068990 2024-09-10T21:24:23.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:23.932+0000 7f65f1800640 1 -- 172.21.15.31:0/1800696070 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65ec14c940 con 0x7f65ec068990 2024-09-10T21:24:23.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:23.932+0000 7f65f1800640 1 -- 172.21.15.31:0/1800696070 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65ec1f6c80 con 0x7f65ec068990 2024-09-10T21:24:23.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:23.932+0000 7f65e2a00640 1 -- 172.21.15.31:0/1800696070 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7f65dc010a00 con 0x7f65ec068990 2024-09-10T21:24:23.939 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:23.932+0000 7f65e2a00640 1 -- 172.21.15.31:0/1800696070 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 725+0+0 (secure 0 0 0) 0x7f65dc0213f0 con 0x7f65ec068990 2024-09-10T21:24:23.940 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:23.936+0000 7f65f1800640 1 -- 172.21.15.31:0/1800696070 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65ec0c8890 con 0x7f65ec068990 2024-09-10T21:24:23.941 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:23.936+0000 7f65e2a00640 1 -- 172.21.15.31:0/1800696070 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+76091 (secure 0 0 0) 0x7f65dc018e20 con 0x7f65ec068990 2024-09-10T21:24:24.020 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.016+0000 7f65f1800640 1 -- 172.21.15.31:0/1800696070 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd new", "uuid": "0f5fabc0-434e-4066-b430-233969a548f2", "id": 0} v 0) -- 0x7f65ec1512e0 con 0x7f65ec068990 2024-09-10T21:24:24.022 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.016+0000 7f65e2a00640 1 -- 172.21.15.31:0/1800696070 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "0f5fabc0-434e-4066-b430-233969a548f2", "id": 0}]=0 v3) ==== 112+0+2 (secure 0 0 0) 0x7f65dc018420 con 0x7f65ec068990 2024-09-10T21:24:24.022 INFO:teuthology.orchestra.run.smithi031.stdout:0 2024-09-10T21:24:24.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.020+0000 7f65f1800640 1 -- 172.21.15.31:0/1800696070 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f65ec068990 msgr2=0x7f65ec14ebc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:24.023 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.020+0000 7f65f1800640 1 --2- 172.21.15.31:0/1800696070 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f65ec068990 0x7f65ec14ebc0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f65dc00ea20 tx=0x7f65dc00eee0 comp rx=0 tx=0).stop 2024-09-10T21:24:24.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.020+0000 7f65f1800640 1 -- 172.21.15.31:0/1800696070 shutdown_connections 2024-09-10T21:24:24.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.020+0000 7f65f1800640 1 --2- 172.21.15.31:0/1800696070 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f65ec0ccce0 0x7f65ec1f4070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:24.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.020+0000 7f65f1800640 1 --2- 172.21.15.31:0/1800696070 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f65ec0cc900 0x7f65ec1f7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:24.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.020+0000 7f65f1800640 1 --2- 172.21.15.31:0/1800696070 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f65ec068990 0x7f65ec14ebc0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:24.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.020+0000 7f65f1800640 1 -- 172.21.15.31:0/1800696070 >> 172.21.15.31:0/1800696070 conn(0x7f65ec06cc20 msgr2=0x7f65ec0c9210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:24.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.020+0000 7f65f1800640 1 -- 172.21.15.31:0/1800696070 shutdown_connections 2024-09-10T21:24:24.024 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.020+0000 7f65f1800640 1 -- 172.21.15.31:0/1800696070 wait complete. 2024-09-10T21:24:24.042 DEBUG:teuthology.orchestra.run.smithi031:> sudo ceph --cluster ceph osd new f6691cb3-2a91-4c90-a9fd-ece3c11dd6eb 1 2024-09-10T21:24:24.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.184+0000 7f8355400640 1 -- 172.21.15.31:0/3984806790 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8350154ba0 msgr2=0x7f835015b0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:24.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.184+0000 7f8355400640 1 --2- 172.21.15.31:0/3984806790 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8350154ba0 0x7f835015b0e0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f8348009bc0 tx=0x7f834801cba0 comp rx=0 tx=0).stop 2024-09-10T21:24:24.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.184+0000 7f8355400640 1 -- 172.21.15.31:0/3984806790 shutdown_connections 2024-09-10T21:24:24.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.184+0000 7f8355400640 1 --2- 172.21.15.31:0/3984806790 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8350154ba0 0x7f835015b0e0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:24.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.184+0000 7f8355400640 1 --2- 172.21.15.31:0/3984806790 >> v2:172.21.15.17:3300/0 conn(0x7f83500cd220 0x7f835006d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:24.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.184+0000 7f8355400640 1 -- 172.21.15.31:0/3984806790 >> 172.21.15.31:0/3984806790 conn(0x7f8350068f90 msgr2=0x7f835006dd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:24.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.184+0000 7f8355400640 1 -- 172.21.15.31:0/3984806790 shutdown_connections 2024-09-10T21:24:24.188 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.184+0000 7f8355400640 1 -- 172.21.15.31:0/3984806790 wait complete. 2024-09-10T21:24:24.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.184+0000 7f8355400640 1 Processor -- start 2024-09-10T21:24:24.189 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.184+0000 7f8355400640 1 -- start start 2024-09-10T21:24:24.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.184+0000 7f8355400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f835006d930 0x7f83500cbd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:24.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.184+0000 7f8355400640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f83500cd220 0x7f83500c61b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:24.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.184+0000 7f8355400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8350154ba0 0x7f83501f6f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:24.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.184+0000 7f8355400640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f8350167380 con 0x7f8350154ba0 2024-09-10T21:24:24.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.184+0000 7f8355400640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f8350167200 con 0x7f835006d930 2024-09-10T21:24:24.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.184+0000 7f8355400640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f8350167500 con 0x7f83500cd220 2024-09-10T21:24:24.191 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.184+0000 7f834e800640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f835006d930 0x7f83500cbd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:24.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.184+0000 7f834e800640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f835006d930 0x7f83500cbd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57526/0 (socket says 172.21.15.31:57526) 2024-09-10T21:24:24.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.184+0000 7f834e800640 1 -- 172.21.15.31:0/4100135079 learned_addr learned my addr 172.21.15.31:0/4100135079 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-10T21:24:24.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.184+0000 7f8345e00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f83500cd220 0x7f83500c61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:24.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.184+0000 7f834f200640 1 --2- 172.21.15.31:0/4100135079 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8350154ba0 0x7f83501f6f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:24.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.184+0000 7f834e800640 1 -- 172.21.15.31:0/4100135079 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f83500cd220 msgr2=0x7f83500c61b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:24.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.184+0000 7f834e800640 1 --2- 172.21.15.31:0/4100135079 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f83500cd220 0x7f83500c61b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:24.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.184+0000 7f834e800640 1 -- 172.21.15.31:0/4100135079 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8350154ba0 msgr2=0x7f83501f6f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:24.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.184+0000 7f834e800640 1 --2- 172.21.15.31:0/4100135079 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8350154ba0 0x7f83501f6f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:24.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.188+0000 7f834e800640 1 -- 172.21.15.31:0/4100135079 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83501f4310 con 0x7f835006d930 2024-09-10T21:24:24.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.188+0000 7f834e800640 1 --2- 172.21.15.31:0/4100135079 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f835006d930 0x7f83500cbd10 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f834000e9f0 tx=0x7f834000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:24.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.188+0000 7f8347400640 1 -- 172.21.15.31:0/4100135079 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f834000cd80 con 0x7f835006d930 2024-09-10T21:24:24.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.188+0000 7f8347400640 1 -- 172.21.15.31:0/4100135079 <== mon.1 v2:172.21.15.31:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8340010430 con 0x7f835006d930 2024-09-10T21:24:24.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.188+0000 7f8347400640 1 -- 172.21.15.31:0/4100135079 <== mon.1 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8340010860 con 0x7f835006d930 2024-09-10T21:24:24.192 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.188+0000 7f8355400640 1 -- 172.21.15.31:0/4100135079 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83501f45f0 con 0x7f835006d930 2024-09-10T21:24:24.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.188+0000 7f8355400640 1 -- 172.21.15.31:0/4100135079 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f835014f720 con 0x7f835006d930 2024-09-10T21:24:24.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.188+0000 7f8347400640 1 -- 172.21.15.31:0/4100135079 <== mon.1 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7f8340010a00 con 0x7f835006d930 2024-09-10T21:24:24.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.188+0000 7f8347400640 1 -- 172.21.15.31:0/4100135079 <== mon.1 v2:172.21.15.31:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 840+0+0 (secure 0 0 0) 0x7f83400217a0 con 0x7f835006d930 2024-09-10T21:24:24.193 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.188+0000 7f8355400640 1 -- 172.21.15.31:0/4100135079 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83500c6e30 con 0x7f835006d930 2024-09-10T21:24:24.194 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.188+0000 7f8347400640 1 -- 172.21.15.31:0/4100135079 <== mon.1 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+76091 (secure 0 0 0) 0x7f83400219a0 con 0x7f835006d930 2024-09-10T21:24:24.266 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.260+0000 7f8355400640 1 -- 172.21.15.31:0/4100135079 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd new", "uuid": "f6691cb3-2a91-4c90-a9fd-ece3c11dd6eb", "id": 1} v 0) -- 0x7f835006d140 con 0x7f835006d930 2024-09-10T21:24:24.273 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.268+0000 7f8347400640 1 -- 172.21.15.31:0/4100135079 <== mon.1 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "f6691cb3-2a91-4c90-a9fd-ece3c11dd6eb", "id": 1}]=0 v4) ==== 112+0+2 (secure 0 0 0) 0x7f8340021b80 con 0x7f835006d930 2024-09-10T21:24:24.273 INFO:teuthology.orchestra.run.smithi031.stdout:1 2024-09-10T21:24:24.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.268+0000 7f8355400640 1 -- 172.21.15.31:0/4100135079 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f835006d930 msgr2=0x7f83500cbd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:24.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.268+0000 7f8355400640 1 --2- 172.21.15.31:0/4100135079 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f835006d930 0x7f83500cbd10 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f834000e9f0 tx=0x7f834000eeb0 comp rx=0 tx=0).stop 2024-09-10T21:24:24.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.268+0000 7f8355400640 1 -- 172.21.15.31:0/4100135079 shutdown_connections 2024-09-10T21:24:24.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.268+0000 7f8355400640 1 --2- 172.21.15.31:0/4100135079 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8350154ba0 0x7f83501f6f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:24.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.268+0000 7f8355400640 1 --2- 172.21.15.31:0/4100135079 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f83500cd220 0x7f83500c61b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:24.274 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.268+0000 7f8355400640 1 --2- 172.21.15.31:0/4100135079 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f835006d930 0x7f83500cbd10 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:24.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.268+0000 7f8355400640 1 -- 172.21.15.31:0/4100135079 >> 172.21.15.31:0/4100135079 conn(0x7f8350068f90 msgr2=0x7f83501582c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:24.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.268+0000 7f8355400640 1 -- 172.21.15.31:0/4100135079 shutdown_connections 2024-09-10T21:24:24.275 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.268+0000 7f8355400640 1 -- 172.21.15.31:0/4100135079 wait complete. 2024-09-10T21:24:24.293 DEBUG:teuthology.orchestra.run.smithi031:> sudo ceph --cluster ceph osd new cb61adb0-a216-437b-9598-06fa7c84aa64 2 2024-09-10T21:24:24.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.432+0000 7f79a2400640 1 -- 172.21.15.31:0/1938148659 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f799c0c9310 msgr2=0x7f799c070960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:24.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.432+0000 7f79a2400640 1 --2- 172.21.15.31:0/1938148659 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f799c0c9310 0x7f799c070960 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f798c009870 tx=0x7f798c01f050 comp rx=0 tx=0).stop 2024-09-10T21:24:24.438 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.432+0000 7f79a2400640 1 -- 172.21.15.31:0/1938148659 shutdown_connections 2024-09-10T21:24:24.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.432+0000 7f79a2400640 1 --2- 172.21.15.31:0/1938148659 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f799c0c9310 0x7f799c070960 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:24.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.432+0000 7f79a2400640 1 -- 172.21.15.31:0/1938148659 >> 172.21.15.31:0/1938148659 conn(0x7f799c06d220 msgr2=0x7f799c068990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:24.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.432+0000 7f79a2400640 1 -- 172.21.15.31:0/1938148659 shutdown_connections 2024-09-10T21:24:24.439 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.432+0000 7f79a2400640 1 -- 172.21.15.31:0/1938148659 wait complete. 2024-09-10T21:24:24.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.436+0000 7f79a2400640 1 Processor -- start 2024-09-10T21:24:24.440 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.436+0000 7f79a2400640 1 -- start start 2024-09-10T21:24:24.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.436+0000 7f79a2400640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f799c0c9310 0x7f799c0ce270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:24.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.436+0000 7f79a2400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f799c0c96f0 0x7f799c1f4170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:24.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.436+0000 7f79a2400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f799c152b20 0x7f799c1f46b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:24.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.436+0000 7f79a2400640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f799c0c9d40 con 0x7f799c0c96f0 2024-09-10T21:24:24.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.436+0000 7f79a2400640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f799c070ea0 con 0x7f799c152b20 2024-09-10T21:24:24.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.436+0000 7f79a2400640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f799c070400 con 0x7f799c0c9310 2024-09-10T21:24:24.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.436+0000 7f799b400640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f799c0c9310 0x7f799c0ce270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:24.442 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.436+0000 7f799b400640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f799c0c9310 0x7f799c0ce270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3301/0 says I am v2:172.21.15.31:41440/0 (socket says 172.21.15.31:41440) 2024-09-10T21:24:24.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.436+0000 7f799b400640 1 -- 172.21.15.31:0/2983957229 learned_addr learned my addr 172.21.15.31:0/2983957229 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-10T21:24:24.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.436+0000 7f799aa00640 1 --2- 172.21.15.31:0/2983957229 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f799c0c96f0 0x7f799c1f4170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:24.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.436+0000 7f799be00640 1 --2- 172.21.15.31:0/2983957229 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f799c152b20 0x7f799c1f46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:24.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.436+0000 7f799b400640 1 -- 172.21.15.31:0/2983957229 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f799c152b20 msgr2=0x7f799c1f46b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:24.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.436+0000 7f799b400640 1 --2- 172.21.15.31:0/2983957229 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f799c152b20 0x7f799c1f46b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:24.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.436+0000 7f799b400640 1 -- 172.21.15.31:0/2983957229 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f799c0c96f0 msgr2=0x7f799c1f4170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:24.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.436+0000 7f799b400640 1 --2- 172.21.15.31:0/2983957229 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f799c0c96f0 0x7f799c1f4170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:24.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.436+0000 7f799b400640 1 -- 172.21.15.31:0/2983957229 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f799c06fd20 con 0x7f799c0c9310 2024-09-10T21:24:24.443 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.436+0000 7f799b400640 1 --2- 172.21.15.31:0/2983957229 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f799c0c9310 0x7f799c0ce270 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f798c009960 tx=0x7f798c005cc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:24.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.436+0000 7f7993400640 1 -- 172.21.15.31:0/2983957229 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f798c0be070 con 0x7f799c0c9310 2024-09-10T21:24:24.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.436+0000 7f7993400640 1 -- 172.21.15.31:0/2983957229 <== mon.2 v2:172.21.15.31:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f798c005320 con 0x7f799c0c9310 2024-09-10T21:24:24.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.440+0000 7f7993400640 1 -- 172.21.15.31:0/2983957229 <== mon.2 v2:172.21.15.31:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f798c0b5040 con 0x7f799c0c9310 2024-09-10T21:24:24.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.440+0000 7f79a2400640 1 -- 172.21.15.31:0/2983957229 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f799c1e2a60 con 0x7f799c0c9310 2024-09-10T21:24:24.444 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.440+0000 7f79a2400640 1 -- 172.21.15.31:0/2983957229 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f799c1e3150 con 0x7f799c0c9310 2024-09-10T21:24:24.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.440+0000 7f7993400640 1 -- 172.21.15.31:0/2983957229 <== mon.2 v2:172.21.15.31:3301/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7f798c0b8030 con 0x7f799c0c9310 2024-09-10T21:24:24.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.440+0000 7f7993400640 1 -- 172.21.15.31:0/2983957229 <== mon.2 v2:172.21.15.31:3301/0 5 ==== osd_map(4..4 src has 1..4) ==== 955+0+0 (secure 0 0 0) 0x7f798c004af0 con 0x7f799c0c9310 2024-09-10T21:24:24.445 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.440+0000 7f79a2400640 1 -- 172.21.15.31:0/2983957229 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f799c0c34f0 con 0x7f799c0c9310 2024-09-10T21:24:24.446 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.440+0000 7f7993400640 1 -- 172.21.15.31:0/2983957229 <== mon.2 v2:172.21.15.31:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+76091 (secure 0 0 0) 0x7f798c0c7020 con 0x7f799c0c9310 2024-09-10T21:24:24.517 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.512+0000 7f79a2400640 1 -- 172.21.15.31:0/2983957229 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_command({"prefix": "osd new", "uuid": "cb61adb0-a216-437b-9598-06fa7c84aa64", "id": 2} v 0) -- 0x7f799c0cef60 con 0x7f799c0c9310 2024-09-10T21:24:24.524 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.520+0000 7f7993400640 1 -- 172.21.15.31:0/2983957229 <== mon.2 v2:172.21.15.31:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "cb61adb0-a216-437b-9598-06fa7c84aa64", "id": 2}]=0 v5) ==== 112+0+2 (secure 0 0 0) 0x7f798c018bb0 con 0x7f799c0c9310 2024-09-10T21:24:24.524 INFO:teuthology.orchestra.run.smithi031.stdout:2 2024-09-10T21:24:24.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.520+0000 7f79a2400640 1 -- 172.21.15.31:0/2983957229 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f799c0c9310 msgr2=0x7f799c0ce270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:24.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.520+0000 7f79a2400640 1 --2- 172.21.15.31:0/2983957229 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f799c0c9310 0x7f799c0ce270 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f798c009960 tx=0x7f798c005cc0 comp rx=0 tx=0).stop 2024-09-10T21:24:24.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.520+0000 7f79a2400640 1 -- 172.21.15.31:0/2983957229 shutdown_connections 2024-09-10T21:24:24.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.520+0000 7f79a2400640 1 --2- 172.21.15.31:0/2983957229 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f799c152b20 0x7f799c1f46b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:24.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.520+0000 7f79a2400640 1 --2- 172.21.15.31:0/2983957229 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f799c0c96f0 0x7f799c1f4170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:24.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.520+0000 7f79a2400640 1 --2- 172.21.15.31:0/2983957229 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f799c0c9310 0x7f799c0ce270 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:24.525 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.520+0000 7f79a2400640 1 -- 172.21.15.31:0/2983957229 >> 172.21.15.31:0/2983957229 conn(0x7f799c06d220 msgr2=0x7f799c0cb0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:24.526 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.520+0000 7f79a2400640 1 -- 172.21.15.31:0/2983957229 shutdown_connections 2024-09-10T21:24:24.526 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.520+0000 7f79a2400640 1 -- 172.21.15.31:0/2983957229 wait complete. 2024-09-10T21:24:24.543 DEBUG:teuthology.orchestra.run.smithi031:> sudo ceph --cluster ceph osd new 6d2da26a-e0bf-47dc-b020-af86e6bcfccc 3 2024-09-10T21:24:24.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.684+0000 7f516c800640 1 -- 172.21.15.31:0/4226937113 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5164074860 msgr2=0x7f51640c10d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:24.690 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.684+0000 7f516c800640 1 --2- 172.21.15.31:0/4226937113 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5164074860 0x7f51640c10d0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f51540097c0 tx=0x7f515401c570 comp rx=0 tx=0).stop 2024-09-10T21:24:24.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.688+0000 7f516c800640 1 -- 172.21.15.31:0/4226937113 shutdown_connections 2024-09-10T21:24:24.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.688+0000 7f516c800640 1 --2- 172.21.15.31:0/4226937113 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5164074860 0x7f51640c10d0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:24.691 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.688+0000 7f516c800640 1 -- 172.21.15.31:0/4226937113 >> 172.21.15.31:0/4226937113 conn(0x7f5164070110 msgr2=0x7f5164070520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:24.693 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.688+0000 7f516c800640 1 -- 172.21.15.31:0/4226937113 shutdown_connections 2024-09-10T21:24:24.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.688+0000 7f516c800640 1 -- 172.21.15.31:0/4226937113 wait complete. 2024-09-10T21:24:24.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.688+0000 7f516c800640 1 Processor -- start 2024-09-10T21:24:24.694 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.688+0000 7f516c800640 1 -- start start 2024-09-10T21:24:24.696 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.692+0000 7f516c800640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5164074860 0x7f51641563b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:24.696 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.692+0000 7f516c800640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f51640c6dd0 0x7f51640c5a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:24.696 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.692+0000 7f516c800640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5164153f20 0x7f51640c5f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:24.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.692+0000 7f5169e00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5164074860 0x7f51641563b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:24.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.692+0000 7f5169e00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5164074860 0x7f51641563b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.31:35836/0 (socket says 172.21.15.31:35836) 2024-09-10T21:24:24.697 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.692+0000 7f516a800640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5164153f20 0x7f51640c5f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:24.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.692+0000 7f5169400640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f51640c6dd0 0x7f51640c5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:24.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.692+0000 7f5169400640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f51640c6dd0 0x7f51640c5a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3301/0 says I am v2:172.21.15.31:41450/0 (socket says 172.21.15.31:41450) 2024-09-10T21:24:24.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.692+0000 7f516a800640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5164153f20 0x7f51640c5f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57546/0 (socket says 172.21.15.31:57546) 2024-09-10T21:24:24.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.692+0000 7f516c800640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f5164073a80 con 0x7f5164074860 2024-09-10T21:24:24.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.692+0000 7f5169e00640 1 -- 172.21.15.31:0/3935110759 learned_addr learned my addr 172.21.15.31:0/3935110759 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-10T21:24:24.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.692+0000 7f516c800640 1 -- 172.21.15.31:0/3935110759 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f51640c8730 con 0x7f5164153f20 2024-09-10T21:24:24.698 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.692+0000 7f516c800640 1 -- 172.21.15.31:0/3935110759 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f5164073d60 con 0x7f51640c6dd0 2024-09-10T21:24:24.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.696+0000 7f516a800640 1 -- 172.21.15.31:0/3935110759 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f51640c6dd0 msgr2=0x7f51640c5a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:24.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.696+0000 7f516a800640 1 --2- 172.21.15.31:0/3935110759 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f51640c6dd0 0x7f51640c5a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:24.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.696+0000 7f516a800640 1 -- 172.21.15.31:0/3935110759 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5164074860 msgr2=0x7f51641563b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:24.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.696+0000 7f516a800640 1 --2- 172.21.15.31:0/3935110759 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5164074860 0x7f51641563b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:24.699 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.696+0000 7f516a800640 1 -- 172.21.15.31:0/3935110759 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51641ec0e0 con 0x7f5164153f20 2024-09-10T21:24:24.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.696+0000 7f516a800640 1 --2- 172.21.15.31:0/3935110759 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5164153f20 0x7f51640c5f60 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f516000cb80 tx=0x7f516000cbb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:24.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.696+0000 7f5169400640 1 --2- 172.21.15.31:0/3935110759 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f51640c6dd0 0x7f51640c5a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:24:24.700 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.696+0000 7f515e000640 1 -- 172.21.15.31:0/3935110759 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f516000a850 con 0x7f5164153f20 2024-09-10T21:24:24.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.696+0000 7f516c800640 1 -- 172.21.15.31:0/3935110759 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51641ec3c0 con 0x7f5164153f20 2024-09-10T21:24:24.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.696+0000 7f516c800640 1 -- 172.21.15.31:0/3935110759 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51641ec980 con 0x7f5164153f20 2024-09-10T21:24:24.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.696+0000 7f515e000640 1 -- 172.21.15.31:0/3935110759 <== mon.1 v2:172.21.15.31:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f516000aae0 con 0x7f5164153f20 2024-09-10T21:24:24.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.696+0000 7f515e000640 1 -- 172.21.15.31:0/3935110759 <== mon.1 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5160011470 con 0x7f5164153f20 2024-09-10T21:24:24.701 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.696+0000 7f515e000640 1 -- 172.21.15.31:0/3935110759 <== mon.1 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7f5160022030 con 0x7f5164153f20 2024-09-10T21:24:24.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.696+0000 7f515e000640 1 -- 172.21.15.31:0/3935110759 <== mon.1 v2:172.21.15.31:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1070+0+0 (secure 0 0 0) 0x7f516001ac00 con 0x7f5164153f20 2024-09-10T21:24:24.702 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.696+0000 7f516c800640 1 -- 172.21.15.31:0/3935110759 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51400052a0 con 0x7f5164153f20 2024-09-10T21:24:24.707 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.704+0000 7f515e000640 1 -- 172.21.15.31:0/3935110759 <== mon.1 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+76091 (secure 0 0 0) 0x7f5160016030 con 0x7f5164153f20 2024-09-10T21:24:24.812 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.808+0000 7f516c800640 1 -- 172.21.15.31:0/3935110759 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd new", "uuid": "6d2da26a-e0bf-47dc-b020-af86e6bcfccc", "id": 3} v 0) -- 0x7f5140005590 con 0x7f5164153f20 2024-09-10T21:24:24.816 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.812+0000 7f515e000640 1 -- 172.21.15.31:0/3935110759 <== mon.1 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "6d2da26a-e0bf-47dc-b020-af86e6bcfccc", "id": 3}]=0 v6) ==== 112+0+2 (secure 0 0 0) 0x7f516001a770 con 0x7f5164153f20 2024-09-10T21:24:24.816 INFO:teuthology.orchestra.run.smithi031.stdout:3 2024-09-10T21:24:24.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.812+0000 7f516c800640 1 -- 172.21.15.31:0/3935110759 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5164153f20 msgr2=0x7f51640c5f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:24.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.812+0000 7f516c800640 1 --2- 172.21.15.31:0/3935110759 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5164153f20 0x7f51640c5f60 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f516000cb80 tx=0x7f516000cbb0 comp rx=0 tx=0).stop 2024-09-10T21:24:24.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.812+0000 7f516c800640 1 -- 172.21.15.31:0/3935110759 shutdown_connections 2024-09-10T21:24:24.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.812+0000 7f516c800640 1 --2- 172.21.15.31:0/3935110759 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5164153f20 0x7f51640c5f60 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:24.818 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.812+0000 7f516c800640 1 --2- 172.21.15.31:0/3935110759 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f51640c6dd0 0x7f51640c5a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:24.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.812+0000 7f516c800640 1 --2- 172.21.15.31:0/3935110759 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5164074860 0x7f51641563b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:24.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.812+0000 7f516c800640 1 -- 172.21.15.31:0/3935110759 >> 172.21.15.31:0/3935110759 conn(0x7f5164070110 msgr2=0x7f5164075a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:24.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.812+0000 7f516c800640 1 -- 172.21.15.31:0/3935110759 shutdown_connections 2024-09-10T21:24:24.819 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.812+0000 7f516c800640 1 -- 172.21.15.31:0/3935110759 wait complete. 2024-09-10T21:24:24.835 DEBUG:teuthology.orchestra.run.smithi031:> sudo ceph --cluster ceph osd new c65071dd-9ca7-4f1d-a584-45dc121419a7 4 2024-09-10T21:24:24.980 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.976+0000 7f6915800640 1 -- 172.21.15.31:0/4026963400 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f69100c34f0 msgr2=0x7f6910152950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:24.980 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.976+0000 7f6915800640 1 --2- 172.21.15.31:0/4026963400 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f69100c34f0 0x7f6910152950 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f6908009c80 tx=0x7f690801cd60 comp rx=0 tx=0).stop 2024-09-10T21:24:24.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.976+0000 7f6915800640 1 -- 172.21.15.31:0/4026963400 shutdown_connections 2024-09-10T21:24:24.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.976+0000 7f6915800640 1 --2- 172.21.15.31:0/4026963400 >> v2:172.21.15.17:3300/0 conn(0x7f69100c96f0 0x7f6910152410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:24.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.976+0000 7f6915800640 1 --2- 172.21.15.31:0/4026963400 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f69100c34f0 0x7f6910152950 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:24.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.976+0000 7f6915800640 1 -- 172.21.15.31:0/4026963400 >> 172.21.15.31:0/4026963400 conn(0x7f691006d220 msgr2=0x7f6910068990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:24.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.976+0000 7f6915800640 1 -- 172.21.15.31:0/4026963400 shutdown_connections 2024-09-10T21:24:24.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.976+0000 7f6915800640 1 -- 172.21.15.31:0/4026963400 wait complete. 2024-09-10T21:24:24.981 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.976+0000 7f6915800640 1 Processor -- start 2024-09-10T21:24:24.982 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.976+0000 7f6915800640 1 -- start start 2024-09-10T21:24:24.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.980+0000 7f6915800640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f69100c34f0 0x7f69101546d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:24.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.980+0000 7f6915800640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f69100c9310 0x7f691014b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:24.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.980+0000 7f6915800640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f69100c96f0 0x7f691006eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:24.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.980+0000 7f6915800640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f6910070580 con 0x7f69100c96f0 2024-09-10T21:24:24.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.980+0000 7f6915800640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f6910070400 con 0x7f69100c34f0 2024-09-10T21:24:24.984 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.980+0000 7f6915800640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f69101657c0 con 0x7f69100c9310 2024-09-10T21:24:24.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.980+0000 7f690ea00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f69100c34f0 0x7f69101546d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:24.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.980+0000 7f6907e00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f69100c9310 0x7f691014b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:24.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.980+0000 7f690ea00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f69100c34f0 0x7f69101546d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57570/0 (socket says 172.21.15.31:57570) 2024-09-10T21:24:24.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.980+0000 7f690ea00640 1 -- 172.21.15.31:0/2127139756 learned_addr learned my addr 172.21.15.31:0/2127139756 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-10T21:24:24.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.980+0000 7f690f400640 1 --2- 172.21.15.31:0/2127139756 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f69100c96f0 0x7f691006eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:24.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.980+0000 7f690ea00640 1 -- 172.21.15.31:0/2127139756 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f69100c9310 msgr2=0x7f691014b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:24.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.980+0000 7f690ea00640 1 --2- 172.21.15.31:0/2127139756 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f69100c9310 0x7f691014b930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:24.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.980+0000 7f690ea00640 1 -- 172.21.15.31:0/2127139756 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f69100c96f0 msgr2=0x7f691006eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:24.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.980+0000 7f690ea00640 1 --2- 172.21.15.31:0/2127139756 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f69100c96f0 0x7f691006eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:24.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.980+0000 7f690ea00640 1 -- 172.21.15.31:0/2127139756 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f691006e070 con 0x7f69100c34f0 2024-09-10T21:24:24.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.980+0000 7f690ea00640 1 --2- 172.21.15.31:0/2127139756 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f69100c34f0 0x7f69101546d0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f690000c500 tx=0x7f690000c9c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:24.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.980+0000 7f6906a00640 1 -- 172.21.15.31:0/2127139756 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6900017070 con 0x7f69100c34f0 2024-09-10T21:24:24.985 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.980+0000 7f6906a00640 1 -- 172.21.15.31:0/2127139756 <== mon.1 v2:172.21.15.31:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f69000053c0 con 0x7f69100c34f0 2024-09-10T21:24:24.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.980+0000 7f6915800640 1 -- 172.21.15.31:0/2127139756 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f691014c490 con 0x7f69100c34f0 2024-09-10T21:24:24.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.980+0000 7f6906a00640 1 -- 172.21.15.31:0/2127139756 <== mon.1 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6900012890 con 0x7f69100c34f0 2024-09-10T21:24:24.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.980+0000 7f6915800640 1 -- 172.21.15.31:0/2127139756 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f691006fa60 con 0x7f69100c34f0 2024-09-10T21:24:24.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.980+0000 7f6906a00640 1 -- 172.21.15.31:0/2127139756 <== mon.1 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7f6900012a30 con 0x7f69100c34f0 2024-09-10T21:24:24.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.980+0000 7f6906a00640 1 -- 172.21.15.31:0/2127139756 <== mon.1 v2:172.21.15.31:3300/0 5 ==== osd_map(6..6 src has 1..6) ==== 1185+0+0 (secure 0 0 0) 0x7f6900012d50 con 0x7f69100c34f0 2024-09-10T21:24:24.986 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.980+0000 7f6915800640 1 -- 172.21.15.31:0/2127139756 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6910070960 con 0x7f69100c34f0 2024-09-10T21:24:24.987 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:24.984+0000 7f6906a00640 1 -- 172.21.15.31:0/2127139756 <== mon.1 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+76091 (secure 0 0 0) 0x7f6900021070 con 0x7f69100c34f0 2024-09-10T21:24:25.064 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:25.060+0000 7f6915800640 1 -- 172.21.15.31:0/2127139756 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd new", "uuid": "c65071dd-9ca7-4f1d-a584-45dc121419a7", "id": 4} v 0) -- 0x7f69100731c0 con 0x7f69100c34f0 2024-09-10T21:24:25.069 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:25.064+0000 7f6906a00640 1 -- 172.21.15.31:0/2127139756 <== mon.1 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "c65071dd-9ca7-4f1d-a584-45dc121419a7", "id": 4}]=0 v7) ==== 112+0+2 (secure 0 0 0) 0x7f6900003600 con 0x7f69100c34f0 2024-09-10T21:24:25.069 INFO:teuthology.orchestra.run.smithi031.stdout:4 2024-09-10T21:24:25.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:25.064+0000 7f6915800640 1 -- 172.21.15.31:0/2127139756 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f69100c34f0 msgr2=0x7f69101546d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:25.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:25.064+0000 7f6915800640 1 --2- 172.21.15.31:0/2127139756 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f69100c34f0 0x7f69101546d0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f690000c500 tx=0x7f690000c9c0 comp rx=0 tx=0).stop 2024-09-10T21:24:25.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:25.064+0000 7f6915800640 1 -- 172.21.15.31:0/2127139756 shutdown_connections 2024-09-10T21:24:25.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:25.064+0000 7f6915800640 1 --2- 172.21.15.31:0/2127139756 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f69100c96f0 0x7f691006eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:25.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:25.064+0000 7f6915800640 1 --2- 172.21.15.31:0/2127139756 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f69100c9310 0x7f691014b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:25.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:25.064+0000 7f6915800640 1 --2- 172.21.15.31:0/2127139756 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f69100c34f0 0x7f69101546d0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:25.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:25.064+0000 7f6915800640 1 -- 172.21.15.31:0/2127139756 >> 172.21.15.31:0/2127139756 conn(0x7f691006d220 msgr2=0x7f69100729c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:25.070 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:25.064+0000 7f6915800640 1 -- 172.21.15.31:0/2127139756 shutdown_connections 2024-09-10T21:24:25.071 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:25.064+0000 7f6915800640 1 -- 172.21.15.31:0/2127139756 wait complete. 2024-09-10T21:24:25.077 INFO:tasks.ceph.mgr.x.smithi031.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-09-10T21:24:25.077 INFO:tasks.ceph.mgr.x.smithi031.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-09-10T21:24:25.077 INFO:tasks.ceph.mgr.x.smithi031.stderr: from numpy import show_config as show_numpy_config 2024-09-10T21:24:25.088 DEBUG:teuthology.orchestra.run.smithi031:> sudo ceph --cluster ceph osd new 235389de-7777-4744-81b8-d0bfc11190e2 5 2024-09-10T21:24:25.234 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:25.228+0000 7fc701800640 1 -- 172.21.15.31:0/1322603022 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc6fc0c34f0 msgr2=0x7fc6fc152950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:25.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:25.228+0000 7fc701800640 1 --2- 172.21.15.31:0/1322603022 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc6fc0c34f0 0x7fc6fc152950 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fc6f4009bc0 tx=0x7fc6f401cba0 comp rx=0 tx=0).stop 2024-09-10T21:24:25.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:25.228+0000 7fc701800640 1 -- 172.21.15.31:0/1322603022 shutdown_connections 2024-09-10T21:24:25.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:25.228+0000 7fc701800640 1 --2- 172.21.15.31:0/1322603022 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc6fc0c34f0 0x7fc6fc152950 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:25.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:25.228+0000 7fc701800640 1 --2- 172.21.15.31:0/1322603022 >> v2:172.21.15.31:3300/0 conn(0x7fc6fc0c96f0 0x7fc6fc152410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:25.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:25.228+0000 7fc701800640 1 -- 172.21.15.31:0/1322603022 >> 172.21.15.31:0/1322603022 conn(0x7fc6fc06d220 msgr2=0x7fc6fc068990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:25.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:25.228+0000 7fc701800640 1 -- 172.21.15.31:0/1322603022 shutdown_connections 2024-09-10T21:24:25.235 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:25.228+0000 7fc701800640 1 -- 172.21.15.31:0/1322603022 wait complete. 2024-09-10T21:24:25.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:25.232+0000 7fc701800640 1 Processor -- start 2024-09-10T21:24:25.236 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:25.232+0000 7fc701800640 1 -- start start 2024-09-10T21:24:25.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:25.232+0000 7fc701800640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc6fc0c34f0 0x7fc6fc149b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:25.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:25.232+0000 7fc701800640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc6fc0c9310 0x7fc6fc1f7480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:25.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:25.232+0000 7fc701800640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc6fc0c96f0 0x7fc6fc14b960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:25.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:25.232+0000 7fc701800640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fc6fc070580 con 0x7fc6fc0c9310 2024-09-10T21:24:25.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:25.232+0000 7fc701800640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fc6fc070400 con 0x7fc6fc0c34f0 2024-09-10T21:24:25.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:25.232+0000 7fc701800640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7fc6fc1657c0 con 0x7fc6fc0c96f0 2024-09-10T21:24:25.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:25.232+0000 7fc6fb400640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc6fc0c96f0 0x7fc6fc14b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:25.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:25.232+0000 7fc6faa00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc6fc0c34f0 0x7fc6fc149b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:25.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:25.232+0000 7fc6fb400640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc6fc0c96f0 0x7fc6fc14b960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3301/0 says I am v2:172.21.15.31:41464/0 (socket says 172.21.15.31:41464) 2024-09-10T21:24:25.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:25.232+0000 7fc6fb400640 1 -- 172.21.15.31:0/39473756 learned_addr learned my addr 172.21.15.31:0/39473756 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-10T21:24:25.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:25.232+0000 7fc6faa00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc6fc0c34f0 0x7fc6fc149b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57594/0 (socket says 172.21.15.31:57594) 2024-09-10T21:24:25.238 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:25.232+0000 7fc6f3e00640 1 --2- 172.21.15.31:0/39473756 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc6fc0c9310 0x7fc6fc1f7480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:25.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:25.232+0000 7fc6fb400640 1 -- 172.21.15.31:0/39473756 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc6fc0c34f0 msgr2=0x7fc6fc149b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:25.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:25.232+0000 7fc6fb400640 1 --2- 172.21.15.31:0/39473756 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc6fc0c34f0 0x7fc6fc149b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:25.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:25.232+0000 7fc6fb400640 1 -- 172.21.15.31:0/39473756 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc6fc0c9310 msgr2=0x7fc6fc1f7480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:25.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:25.232+0000 7fc6fb400640 1 --2- 172.21.15.31:0/39473756 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc6fc0c9310 0x7fc6fc1f7480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:25.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:25.232+0000 7fc6fb400640 1 -- 172.21.15.31:0/39473756 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6fc06fcc0 con 0x7fc6fc0c96f0 2024-09-10T21:24:25.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:25.232+0000 7fc6f3e00640 1 --2- 172.21.15.31:0/39473756 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc6fc0c9310 0x7fc6fc1f7480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:24:25.239 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:25.232+0000 7fc6fb400640 1 --2- 172.21.15.31:0/39473756 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc6fc0c96f0 0x7fc6fc14b960 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fc6f401a7d0 tx=0x7fc6f4009280 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:25.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:25.232+0000 7fc6f2a00640 1 -- 172.21.15.31:0/39473756 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6f40bb070 con 0x7fc6fc0c96f0 2024-09-10T21:24:25.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:25.232+0000 7fc6f2a00640 1 -- 172.21.15.31:0/39473756 <== mon.2 v2:172.21.15.31:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc6f4004a50 con 0x7fc6fc0c96f0 2024-09-10T21:24:25.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:25.232+0000 7fc6f2a00640 1 -- 172.21.15.31:0/39473756 <== mon.2 v2:172.21.15.31:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6f40b2040 con 0x7fc6fc0c96f0 2024-09-10T21:24:25.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:25.232+0000 7fc701800640 1 -- 172.21.15.31:0/39473756 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6fc1f40b0 con 0x7fc6fc0c96f0 2024-09-10T21:24:25.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:25.236+0000 7fc701800640 1 -- 172.21.15.31:0/39473756 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6fc1f4e30 con 0x7fc6fc0c96f0 2024-09-10T21:24:25.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:25.236+0000 7fc6f2a00640 1 -- 172.21.15.31:0/39473756 <== mon.2 v2:172.21.15.31:3301/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7fc6f40b5030 con 0x7fc6fc0c96f0 2024-09-10T21:24:25.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:25.236+0000 7fc6f2a00640 1 -- 172.21.15.31:0/39473756 <== mon.2 v2:172.21.15.31:3301/0 5 ==== osd_map(7..7 src has 1..7) ==== 1300+0+0 (secure 0 0 0) 0x7fc6f40047c0 con 0x7fc6fc0c96f0 2024-09-10T21:24:25.240 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:25.236+0000 7fc701800640 1 -- 172.21.15.31:0/39473756 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6fc070960 con 0x7fc6fc0c96f0 2024-09-10T21:24:25.241 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:25.236+0000 7fc6f2a00640 1 -- 172.21.15.31:0/39473756 <== mon.2 v2:172.21.15.31:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+76091 (secure 0 0 0) 0x7fc6f40c4080 con 0x7fc6fc0c96f0 2024-09-10T21:24:25.314 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:25.308+0000 7fc701800640 1 -- 172.21.15.31:0/39473756 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_command({"prefix": "osd new", "uuid": "235389de-7777-4744-81b8-d0bfc11190e2", "id": 5} v 0) -- 0x7fc6fc06f9e0 con 0x7fc6fc0c96f0 2024-09-10T21:24:25.318 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:25.312+0000 7fc6f2a00640 1 -- 172.21.15.31:0/39473756 <== mon.2 v2:172.21.15.31:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "235389de-7777-4744-81b8-d0bfc11190e2", "id": 5}]=0 v8) ==== 112+0+2 (secure 0 0 0) 0x7fc6f4016790 con 0x7fc6fc0c96f0 2024-09-10T21:24:25.318 INFO:teuthology.orchestra.run.smithi031.stdout:5 2024-09-10T21:24:25.319 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:25.316+0000 7fc701800640 1 -- 172.21.15.31:0/39473756 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc6fc0c96f0 msgr2=0x7fc6fc14b960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:25.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:25.316+0000 7fc701800640 1 --2- 172.21.15.31:0/39473756 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc6fc0c96f0 0x7fc6fc14b960 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fc6f401a7d0 tx=0x7fc6f4009280 comp rx=0 tx=0).stop 2024-09-10T21:24:25.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:25.316+0000 7fc701800640 1 -- 172.21.15.31:0/39473756 shutdown_connections 2024-09-10T21:24:25.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:25.316+0000 7fc701800640 1 --2- 172.21.15.31:0/39473756 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc6fc0c96f0 0x7fc6fc14b960 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:25.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:25.316+0000 7fc701800640 1 --2- 172.21.15.31:0/39473756 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc6fc0c9310 0x7fc6fc1f7480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:25.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:25.316+0000 7fc701800640 1 --2- 172.21.15.31:0/39473756 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc6fc0c34f0 0x7fc6fc149b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:25.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:25.316+0000 7fc701800640 1 -- 172.21.15.31:0/39473756 >> 172.21.15.31:0/39473756 conn(0x7fc6fc06d220 msgr2=0x7fc6fc157f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:25.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:25.316+0000 7fc701800640 1 -- 172.21.15.31:0/39473756 shutdown_connections 2024-09-10T21:24:25.320 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:25.316+0000 7fc701800640 1 -- 172.21.15.31:0/39473756 wait complete. 2024-09-10T21:24:25.338 DEBUG:teuthology.orchestra.run.smithi031:> sudo ceph --cluster ceph osd new cea70a57-3715-4ebb-8511-a528a9df2b1b 6 2024-09-10T21:24:25.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:25.480+0000 7f91e3e00640 1 -- 172.21.15.31:0/2716023569 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f91dc068990 msgr2=0x7f91dc150ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:25.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:25.480+0000 7f91e3e00640 1 --2- 172.21.15.31:0/2716023569 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f91dc068990 0x7f91dc150ae0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f91c8009900 tx=0x7f91c801f0d0 comp rx=0 tx=0).stop 2024-09-10T21:24:25.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:25.480+0000 7f91e3e00640 1 -- 172.21.15.31:0/2716023569 shutdown_connections 2024-09-10T21:24:25.484 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:25.480+0000 7f91e3e00640 1 --2- 172.21.15.31:0/2716023569 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f91dc068990 0x7f91dc150ae0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:25.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:25.480+0000 7f91e3e00640 1 --2- 172.21.15.31:0/2716023569 >> v2:172.21.15.31:3300/0 conn(0x7f91dc0c33a0 0x7f91dc157590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:25.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:25.480+0000 7f91e3e00640 1 -- 172.21.15.31:0/2716023569 >> 172.21.15.31:0/2716023569 conn(0x7f91dc06cc20 msgr2=0x7f91dc06d030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:25.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:25.480+0000 7f91e3e00640 1 -- 172.21.15.31:0/2716023569 shutdown_connections 2024-09-10T21:24:25.485 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:25.480+0000 7f91e3e00640 1 -- 172.21.15.31:0/2716023569 wait complete. 2024-09-10T21:24:25.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:25.480+0000 7f91e3e00640 1 Processor -- start 2024-09-10T21:24:25.486 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:25.480+0000 7f91e3e00640 1 -- start start 2024-09-10T21:24:25.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:25.484+0000 7f91e3e00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f91dc068990 0x7f91dc145a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:25.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:25.484+0000 7f91e3e00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f91dc068db0 0x7f91dc1eef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:25.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:25.484+0000 7f91e3e00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f91dc0c33a0 0x7f91dc1ef470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:25.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:25.484+0000 7f91e3e00640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f91dc0cd4c0 con 0x7f91dc068990 2024-09-10T21:24:25.490 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:25.484+0000 7f91e3e00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f91dc0c7890 con 0x7f91dc068db0 2024-09-10T21:24:25.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:25.484+0000 7f91e3e00640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f91dc0c98b0 con 0x7f91dc0c33a0 2024-09-10T21:24:25.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:25.484+0000 7f91e0a00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f91dc068db0 0x7f91dc1eef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:25.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:25.484+0000 7f91e1e00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f91dc0c33a0 0x7f91dc1ef470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:25.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:25.484+0000 7f91e0a00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f91dc068db0 0x7f91dc1eef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57608/0 (socket says 172.21.15.31:57608) 2024-09-10T21:24:25.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:25.484+0000 7f91e0a00640 1 -- 172.21.15.31:0/783991630 learned_addr learned my addr 172.21.15.31:0/783991630 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-10T21:24:25.491 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:25.484+0000 7f91e1400640 1 --2- 172.21.15.31:0/783991630 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f91dc068990 0x7f91dc145a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:25.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:25.484+0000 7f91e0a00640 1 -- 172.21.15.31:0/783991630 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f91dc0c33a0 msgr2=0x7f91dc1ef470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:25.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:25.484+0000 7f91e0a00640 1 --2- 172.21.15.31:0/783991630 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f91dc0c33a0 0x7f91dc1ef470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:25.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:25.484+0000 7f91e0a00640 1 -- 172.21.15.31:0/783991630 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f91dc068990 msgr2=0x7f91dc145a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:25.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:25.484+0000 7f91e0a00640 1 --2- 172.21.15.31:0/783991630 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f91dc068990 0x7f91dc145a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:25.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:25.484+0000 7f91e0a00640 1 -- 172.21.15.31:0/783991630 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91dc14c6b0 con 0x7f91dc068db0 2024-09-10T21:24:25.492 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:25.484+0000 7f91e0a00640 1 --2- 172.21.15.31:0/783991630 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f91dc068db0 0x7f91dc1eef30 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f91c8009850 tx=0x7f91c8005e70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:25.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:25.484+0000 7f91d9600640 1 -- 172.21.15.31:0/783991630 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91c80be070 con 0x7f91dc068db0 2024-09-10T21:24:25.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:25.484+0000 7f91d9600640 1 -- 172.21.15.31:0/783991630 <== mon.1 v2:172.21.15.31:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f91c80053e0 con 0x7f91dc068db0 2024-09-10T21:24:25.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:25.484+0000 7f91e3e00640 1 -- 172.21.15.31:0/783991630 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91dc0c69b0 con 0x7f91dc068db0 2024-09-10T21:24:25.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:25.484+0000 7f91d9600640 1 -- 172.21.15.31:0/783991630 <== mon.1 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91c80b5040 con 0x7f91dc068db0 2024-09-10T21:24:25.493 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:25.484+0000 7f91e3e00640 1 -- 172.21.15.31:0/783991630 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91dc14c3d0 con 0x7f91dc068db0 2024-09-10T21:24:25.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:25.484+0000 7f91d9600640 1 -- 172.21.15.31:0/783991630 <== mon.1 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7f91c80b8030 con 0x7f91dc068db0 2024-09-10T21:24:25.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:25.484+0000 7f91d9600640 1 -- 172.21.15.31:0/783991630 <== mon.1 v2:172.21.15.31:3300/0 5 ==== osd_map(8..8 src has 1..8) ==== 1415+0+0 (secure 0 0 0) 0x7f91c80046d0 con 0x7f91dc068db0 2024-09-10T21:24:25.494 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:25.484+0000 7f91cea00640 1 -- 172.21.15.31:0/783991630 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91b00052a0 con 0x7f91dc068db0 2024-09-10T21:24:25.495 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:25.488+0000 7f91d9600640 1 -- 172.21.15.31:0/783991630 <== mon.1 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+76091 (secure 0 0 0) 0x7f91c8018c30 con 0x7f91dc068db0 2024-09-10T21:24:25.581 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:25.576+0000 7f91cea00640 1 -- 172.21.15.31:0/783991630 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd new", "uuid": "cea70a57-3715-4ebb-8511-a528a9df2b1b", "id": 6} v 0) -- 0x7f91b0005860 con 0x7f91dc068db0 2024-09-10T21:24:25.585 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:25.580+0000 7f91d9600640 1 -- 172.21.15.31:0/783991630 <== mon.1 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "cea70a57-3715-4ebb-8511-a528a9df2b1b", "id": 6}]=0 v9) ==== 112+0+2 (secure 0 0 0) 0x7f91c80038d0 con 0x7f91dc068db0 2024-09-10T21:24:25.585 INFO:teuthology.orchestra.run.smithi031.stdout:6 2024-09-10T21:24:25.586 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:25.580+0000 7f91cea00640 1 -- 172.21.15.31:0/783991630 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f91dc068db0 msgr2=0x7f91dc1eef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:25.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:25.580+0000 7f91cea00640 1 --2- 172.21.15.31:0/783991630 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f91dc068db0 0x7f91dc1eef30 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f91c8009850 tx=0x7f91c8005e70 comp rx=0 tx=0).stop 2024-09-10T21:24:25.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:25.580+0000 7f91cea00640 1 -- 172.21.15.31:0/783991630 shutdown_connections 2024-09-10T21:24:25.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:25.580+0000 7f91cea00640 1 --2- 172.21.15.31:0/783991630 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f91dc0c33a0 0x7f91dc1ef470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:25.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:25.580+0000 7f91cea00640 1 --2- 172.21.15.31:0/783991630 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f91dc068db0 0x7f91dc1eef30 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:25.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:25.580+0000 7f91cea00640 1 --2- 172.21.15.31:0/783991630 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f91dc068990 0x7f91dc145a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:25.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:25.580+0000 7f91cea00640 1 -- 172.21.15.31:0/783991630 >> 172.21.15.31:0/783991630 conn(0x7f91dc06cc20 msgr2=0x7f91dc0c8d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:25.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:25.580+0000 7f91cea00640 1 -- 172.21.15.31:0/783991630 shutdown_connections 2024-09-10T21:24:25.587 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:25.584+0000 7f91cea00640 1 -- 172.21.15.31:0/783991630 wait complete. 2024-09-10T21:24:25.604 DEBUG:teuthology.orchestra.run.smithi031:> sudo ceph --cluster ceph osd new 01cc111b-a45f-4ca3-9cc1-7fe0afbeba45 7 2024-09-10T21:24:25.750 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:25.744+0000 7fcd80a00640 1 -- 172.21.15.31:0/241581752 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fcd7c0c9310 msgr2=0x7fcd7c070960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:25.750 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:25.744+0000 7fcd80a00640 1 --2- 172.21.15.31:0/241581752 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fcd7c0c9310 0x7fcd7c070960 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fcd6c009900 tx=0x7fcd6c01f0d0 comp rx=0 tx=0).stop 2024-09-10T21:24:25.750 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:25.744+0000 7fcd80a00640 1 -- 172.21.15.31:0/241581752 shutdown_connections 2024-09-10T21:24:25.750 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:25.744+0000 7fcd80a00640 1 --2- 172.21.15.31:0/241581752 >> v2:172.21.15.31:3300/0 conn(0x7fcd7c152b20 0x7fcd7c162fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:25.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:25.744+0000 7fcd80a00640 1 --2- 172.21.15.31:0/241581752 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fcd7c0c9310 0x7fcd7c070960 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:25.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:25.744+0000 7fcd80a00640 1 -- 172.21.15.31:0/241581752 >> 172.21.15.31:0/241581752 conn(0x7fcd7c06d220 msgr2=0x7fcd7c068990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:25.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:25.744+0000 7fcd80a00640 1 -- 172.21.15.31:0/241581752 shutdown_connections 2024-09-10T21:24:25.751 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:25.744+0000 7fcd80a00640 1 -- 172.21.15.31:0/241581752 wait complete. 2024-09-10T21:24:25.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:25.748+0000 7fcd80a00640 1 Processor -- start 2024-09-10T21:24:25.752 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:25.748+0000 7fcd80a00640 1 -- start start 2024-09-10T21:24:25.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:25.748+0000 7fcd80a00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcd7c0c9310 0x7fcd7c14d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:25.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:25.748+0000 7fcd80a00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcd7c0c96f0 0x7fcd7c0cd150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:25.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:25.748+0000 7fcd80a00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fcd7c152b20 0x7fcd7c14b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:25.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:25.748+0000 7fcd80a00640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fcd7c0c9d40 con 0x7fcd7c0c96f0 2024-09-10T21:24:25.754 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:25.748+0000 7fcd80a00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fcd7c070ea0 con 0x7fcd7c0c9310 2024-09-10T21:24:25.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:25.748+0000 7fcd80a00640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7fcd7c070400 con 0x7fcd7c152b20 2024-09-10T21:24:25.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:25.748+0000 7fcd79e00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcd7c0c9310 0x7fcd7c14d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:25.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:25.748+0000 7fcd7a800640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fcd7c152b20 0x7fcd7c14b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:25.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:25.748+0000 7fcd79e00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcd7c0c9310 0x7fcd7c14d300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.31:57630/0 (socket says 172.21.15.31:57630) 2024-09-10T21:24:25.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:25.748+0000 7fcd79e00640 1 -- 172.21.15.31:0/3867178446 learned_addr learned my addr 172.21.15.31:0/3867178446 (peer_addr_for_me v2:172.21.15.31:0/0) 2024-09-10T21:24:25.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:25.748+0000 7fcd73e00640 1 --2- 172.21.15.31:0/3867178446 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcd7c0c96f0 0x7fcd7c0cd150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:25.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:25.748+0000 7fcd79e00640 1 -- 172.21.15.31:0/3867178446 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fcd7c152b20 msgr2=0x7fcd7c14b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:25.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:25.748+0000 7fcd79e00640 1 --2- 172.21.15.31:0/3867178446 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fcd7c152b20 0x7fcd7c14b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:25.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:25.748+0000 7fcd79e00640 1 -- 172.21.15.31:0/3867178446 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcd7c0c96f0 msgr2=0x7fcd7c0cd150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:25.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:25.748+0000 7fcd79e00640 1 --2- 172.21.15.31:0/3867178446 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcd7c0c96f0 0x7fcd7c0cd150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:25.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:25.748+0000 7fcd79e00640 1 -- 172.21.15.31:0/3867178446 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd7c14dc60 con 0x7fcd7c0c9310 2024-09-10T21:24:25.755 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:25.748+0000 7fcd79e00640 1 --2- 172.21.15.31:0/3867178446 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcd7c0c9310 0x7fcd7c14d300 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fcd6c0b3040 tx=0x7fcd6c005e70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:25.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:25.748+0000 7fcd72000640 1 -- 172.21.15.31:0/3867178446 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd6c0b5040 con 0x7fcd7c0c9310 2024-09-10T21:24:25.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:25.748+0000 7fcd72000640 1 -- 172.21.15.31:0/3867178446 <== mon.1 v2:172.21.15.31:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcd6c005310 con 0x7fcd7c0c9310 2024-09-10T21:24:25.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:25.748+0000 7fcd72000640 1 -- 172.21.15.31:0/3867178446 <== mon.1 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd6c0046d0 con 0x7fcd7c0c9310 2024-09-10T21:24:25.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:25.748+0000 7fcd80a00640 1 -- 172.21.15.31:0/3867178446 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd7c14e670 con 0x7fcd7c0c9310 2024-09-10T21:24:25.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:25.748+0000 7fcd80a00640 1 -- 172.21.15.31:0/3867178446 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd7c14cde0 con 0x7fcd7c0c9310 2024-09-10T21:24:25.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:25.748+0000 7fcd72000640 1 -- 172.21.15.31:0/3867178446 <== mon.1 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7fcd6c0b8070 con 0x7fcd7c0c9310 2024-09-10T21:24:25.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:25.748+0000 7fcd72000640 1 -- 172.21.15.31:0/3867178446 <== mon.1 v2:172.21.15.31:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1530+0+0 (secure 0 0 0) 0x7fcd6c004c00 con 0x7fcd7c0c9310 2024-09-10T21:24:25.756 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:25.752+0000 7fcd80a00640 1 -- 172.21.15.31:0/3867178446 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd7c0c34f0 con 0x7fcd7c0c9310 2024-09-10T21:24:25.757 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:25.752+0000 7fcd72000640 1 -- 172.21.15.31:0/3867178446 <== mon.1 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+76091 (secure 0 0 0) 0x7fcd6c0bf060 con 0x7fcd7c0c9310 2024-09-10T21:24:25.758 INFO:tasks.ceph.mgr.y.smithi017.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-09-10T21:24:25.758 INFO:tasks.ceph.mgr.y.smithi017.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-09-10T21:24:25.758 INFO:tasks.ceph.mgr.y.smithi017.stderr: from numpy import show_config as show_numpy_config 2024-09-10T21:24:25.831 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:25.828+0000 7fcd80a00640 1 -- 172.21.15.31:0/3867178446 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd new", "uuid": "01cc111b-a45f-4ca3-9cc1-7fe0afbeba45", "id": 7} v 0) -- 0x7fcd7c0c39c0 con 0x7fcd7c0c9310 2024-09-10T21:24:25.835 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:25.832+0000 7fcd72000640 1 -- 172.21.15.31:0/3867178446 <== mon.1 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "01cc111b-a45f-4ca3-9cc1-7fe0afbeba45", "id": 7}]=0 v10) ==== 112+0+2 (secure 0 0 0) 0x7fcd6c018c30 con 0x7fcd7c0c9310 2024-09-10T21:24:25.835 INFO:teuthology.orchestra.run.smithi031.stdout:7 2024-09-10T21:24:25.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:25.832+0000 7fcd80a00640 1 -- 172.21.15.31:0/3867178446 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcd7c0c9310 msgr2=0x7fcd7c14d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:25.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:25.832+0000 7fcd80a00640 1 --2- 172.21.15.31:0/3867178446 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcd7c0c9310 0x7fcd7c14d300 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fcd6c0b3040 tx=0x7fcd6c005e70 comp rx=0 tx=0).stop 2024-09-10T21:24:25.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:25.832+0000 7fcd80a00640 1 -- 172.21.15.31:0/3867178446 shutdown_connections 2024-09-10T21:24:25.836 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:25.832+0000 7fcd80a00640 1 --2- 172.21.15.31:0/3867178446 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fcd7c152b20 0x7fcd7c14b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:25.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:25.832+0000 7fcd80a00640 1 --2- 172.21.15.31:0/3867178446 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcd7c0c96f0 0x7fcd7c0cd150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:25.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:25.832+0000 7fcd80a00640 1 --2- 172.21.15.31:0/3867178446 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcd7c0c9310 0x7fcd7c14d300 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:25.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:25.832+0000 7fcd80a00640 1 -- 172.21.15.31:0/3867178446 >> 172.21.15.31:0/3867178446 conn(0x7fcd7c06d220 msgr2=0x7fcd7c072850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:25.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:25.832+0000 7fcd80a00640 1 -- 172.21.15.31:0/3867178446 shutdown_connections 2024-09-10T21:24:25.837 INFO:teuthology.orchestra.run.smithi031.stderr:2024-09-10T21:24:25.832+0000 7fcd80a00640 1 -- 172.21.15.31:0/3867178446 wait complete. 2024-09-10T21:24:25.854 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-10T21:24:25.854 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-09-10T21:24:25.857 INFO:tasks.ceph.osd.0:Started 2024-09-10T21:24:25.858 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-10T21:24:25.858 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-09-10T21:24:25.861 INFO:tasks.ceph.osd.1:Started 2024-09-10T21:24:25.861 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-10T21:24:25.861 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-09-10T21:24:25.864 INFO:tasks.ceph.osd.2:Started 2024-09-10T21:24:25.864 INFO:tasks.ceph.osd.3:Restarting daemon 2024-09-10T21:24:25.864 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-09-10T21:24:25.868 INFO:tasks.ceph.osd.3:Started 2024-09-10T21:24:25.868 INFO:tasks.ceph.osd.4:Restarting daemon 2024-09-10T21:24:25.868 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-09-10T21:24:25.871 INFO:tasks.ceph.osd.4:Started 2024-09-10T21:24:25.871 INFO:tasks.ceph.osd.5:Restarting daemon 2024-09-10T21:24:25.871 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-09-10T21:24:25.874 INFO:tasks.ceph.osd.5:Started 2024-09-10T21:24:25.874 INFO:tasks.ceph.osd.6:Restarting daemon 2024-09-10T21:24:25.874 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-09-10T21:24:25.877 INFO:tasks.ceph.osd.6:Started 2024-09-10T21:24:25.878 INFO:tasks.ceph.osd.7:Restarting daemon 2024-09-10T21:24:25.878 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-09-10T21:24:25.882 INFO:tasks.ceph.osd.7:Started 2024-09-10T21:24:25.882 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:24:26.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:26.051+0000 7f5f23400640 1 -- 172.21.15.17:0/1880887629 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5f1c152820 msgr2=0x7f5f1c15c280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:26.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:26.051+0000 7f5f23400640 1 --2- 172.21.15.17:0/1880887629 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5f1c152820 0x7f5f1c15c280 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c073220 tx=0x7f5f0401c830 comp rx=0 tx=0).stop 2024-09-10T21:24:26.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:26.051+0000 7f5f23400640 1 -- 172.21.15.17:0/1880887629 shutdown_connections 2024-09-10T21:24:26.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:26.051+0000 7f5f23400640 1 --2- 172.21.15.17:0/1880887629 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f5f1c0cdf20 0x7f5f1c0ce350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:26.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:26.051+0000 7f5f23400640 1 --2- 172.21.15.17:0/1880887629 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5f1c152820 0x7f5f1c15c280 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:26.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:26.051+0000 7f5f23400640 1 --2- 172.21.15.17:0/1880887629 >> v2:172.21.15.31:3300/0 conn(0x7f5f1c152440 0x7f5f1c0b3eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:26.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:26.051+0000 7f5f23400640 1 -- 172.21.15.17:0/1880887629 >> 172.21.15.17:0/1880887629 conn(0x7f5f1c0702f0 msgr2=0x7f5f1c070700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:26.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:26.051+0000 7f5f23400640 1 -- 172.21.15.17:0/1880887629 shutdown_connections 2024-09-10T21:24:26.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:26.051+0000 7f5f23400640 1 -- 172.21.15.17:0/1880887629 wait complete. 2024-09-10T21:24:26.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:26.055+0000 7f5f23400640 1 Processor -- start 2024-09-10T21:24:26.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:26.055+0000 7f5f23400640 1 -- start start 2024-09-10T21:24:26.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:26.055+0000 7f5f23400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f1c0cdf20 0x7f5f1c148ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:26.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:26.055+0000 7f5f23400640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f5f1c152440 0x7f5f1c14b150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:26.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:26.055+0000 7f5f23400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5f1c152820 0x7f5f1c1f4380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:26.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:26.055+0000 7f5f23400640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f5f1c0cb450 con 0x7f5f1c152820 2024-09-10T21:24:26.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:26.055+0000 7f5f23400640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f5f1c15c7c0 con 0x7f5f1c0cdf20 2024-09-10T21:24:26.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:26.055+0000 7f5f23400640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f5f1c0cb5d0 con 0x7f5f1c152440 2024-09-10T21:24:26.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:26.055+0000 7f5f20a00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f1c0cdf20 0x7f5f1c148ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:26.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:26.055+0000 7f5f21400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5f1c152820 0x7f5f1c1f4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:26.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:26.055+0000 7f5f13e00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f5f1c152440 0x7f5f1c14b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:26.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:26.055+0000 7f5f20a00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f1c0cdf20 0x7f5f1c148ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.17:51672/0 (socket says 172.21.15.17:51672) 2024-09-10T21:24:26.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:26.055+0000 7f5f20a00640 1 -- 172.21.15.17:0/1409567804 learned_addr learned my addr 172.21.15.17:0/1409567804 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:24:26.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:26.055+0000 7f5f21400640 1 -- 172.21.15.17:0/1409567804 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f5f1c152440 msgr2=0x7f5f1c14b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:26.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:26.055+0000 7f5f21400640 1 --2- 172.21.15.17:0/1409567804 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f5f1c152440 0x7f5f1c14b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:26.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:26.055+0000 7f5f21400640 1 -- 172.21.15.17:0/1409567804 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f1c0cdf20 msgr2=0x7f5f1c148ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:26.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:26.055+0000 7f5f21400640 1 --2- 172.21.15.17:0/1409567804 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f1c0cdf20 0x7f5f1c148ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:26.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:26.055+0000 7f5f21400640 1 -- 172.21.15.17:0/1409567804 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f1c1f0df0 con 0x7f5f1c152820 2024-09-10T21:24:26.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:26.055+0000 7f5f21400640 1 --2- 172.21.15.17:0/1409567804 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5f1c152820 0x7f5f1c1f4380 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f5f1400cab0 tx=0x7f5f1400cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:26.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:26.055+0000 7f5f19600640 1 -- 172.21.15.17:0/1409567804 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f14007d30 con 0x7f5f1c152820 2024-09-10T21:24:26.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:26.055+0000 7f5f19600640 1 -- 172.21.15.17:0/1409567804 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5f1400f450 con 0x7f5f1c152820 2024-09-10T21:24:26.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:26.055+0000 7f5f19600640 1 -- 172.21.15.17:0/1409567804 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f1400f8a0 con 0x7f5f1c152820 2024-09-10T21:24:26.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:26.055+0000 7f5f23400640 1 -- 172.21.15.17:0/1409567804 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f1c14a0b0 con 0x7f5f1c152820 2024-09-10T21:24:26.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:26.059+0000 7f5f23400640 1 -- 172.21.15.17:0/1409567804 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f1c1f1070 con 0x7f5f1c152820 2024-09-10T21:24:26.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:26.059+0000 7f5f19600640 1 -- 172.21.15.17:0/1409567804 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7f5f1400fa40 con 0x7f5f1c152820 2024-09-10T21:24:26.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:26.059+0000 7f5f19600640 1 -- 172.21.15.17:0/1409567804 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f5f14018ad0 con 0x7f5f1c152820 2024-09-10T21:24:26.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:26.059+0000 7f5f23400640 1 -- 172.21.15.17:0/1409567804 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f1c158360 con 0x7f5f1c152820 2024-09-10T21:24:26.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:26.059+0000 7f5f19600640 1 -- 172.21.15.17:0/1409567804 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+76091 (secure 0 0 0) 0x7f5f1c1f0df0 con 0x7f5f1c152820 2024-09-10T21:24:26.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:26.123+0000 7f5f23400640 1 -- 172.21.15.17:0/1409567804 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5f1c14b900 con 0x7f5f1c152820 2024-09-10T21:24:26.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:26.123+0000 7f5f19600640 1 -- 172.21.15.17:0/1409567804 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f5f14014030 con 0x7f5f1c152820 2024-09-10T21:24:26.127 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-10T21:24:26.127 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":10,"fsid":"5e2f023f-d6a6-49f4-a256-f4807dce3739","created":"2024-09-10T21:24:21.750712+0000","modified":"2024-09-10T21:24:25.831127+0000","last_up_change":"0.000000","last_in_change":"2024-09-10T21:24:25.831127+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"0f5fabc0-434e-4066-b430-233969a548f2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"f6691cb3-2a91-4c90-a9fd-ece3c11dd6eb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"cb61adb0-a216-437b-9598-06fa7c84aa64","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"6d2da26a-e0bf-47dc-b020-af86e6bcfccc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"c65071dd-9ca7-4f1d-a584-45dc121419a7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"235389de-7777-4744-81b8-d0bfc11190e2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"cea70a57-3715-4ebb-8511-a528a9df2b1b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"01cc111b-a45f-4ca3-9cc1-7fe0afbeba45","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:24:26.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:26.123+0000 7f5f23400640 1 -- 172.21.15.17:0/1409567804 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5f1c152820 msgr2=0x7f5f1c1f4380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:26.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:26.123+0000 7f5f23400640 1 --2- 172.21.15.17:0/1409567804 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5f1c152820 0x7f5f1c1f4380 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f5f1400cab0 tx=0x7f5f1400cf70 comp rx=0 tx=0).stop 2024-09-10T21:24:26.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:26.123+0000 7f5f23400640 1 -- 172.21.15.17:0/1409567804 shutdown_connections 2024-09-10T21:24:26.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:26.123+0000 7f5f23400640 1 --2- 172.21.15.17:0/1409567804 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5f1c152820 0x7f5f1c1f4380 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:26.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:26.123+0000 7f5f23400640 1 --2- 172.21.15.17:0/1409567804 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f5f1c152440 0x7f5f1c14b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:26.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:26.123+0000 7f5f23400640 1 --2- 172.21.15.17:0/1409567804 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f1c0cdf20 0x7f5f1c148ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:26.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:26.123+0000 7f5f23400640 1 -- 172.21.15.17:0/1409567804 >> 172.21.15.17:0/1409567804 conn(0x7f5f1c0702f0 msgr2=0x7f5f1c1519e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:26.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:26.123+0000 7f5f23400640 1 -- 172.21.15.17:0/1409567804 shutdown_connections 2024-09-10T21:24:26.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:26.123+0000 7f5f23400640 1 -- 172.21.15.17:0/1409567804 wait complete. 2024-09-10T21:24:26.146 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-09-10T21:24:26.147 INFO:tasks.ceph:Waiting for OSDs to come up 2024-09-10T21:24:26.953 DEBUG:teuthology.orchestra.run.smithi017:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-10T21:24:27.092 INFO:teuthology.misc.health.smithi017.stderr:2024-09-10T21:24:27.087+0000 7f85e0800640 1 -- 172.21.15.17:0/3067931234 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f85d80c9360 msgr2=0x7f85d80cb0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:27.092 INFO:teuthology.misc.health.smithi017.stderr:2024-09-10T21:24:27.087+0000 7f85e0800640 1 --2- 172.21.15.17:0/3067931234 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f85d80c9360 0x7f85d80cb0e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f85c8009900 tx=0x7f85c801c8e0 comp rx=0 tx=0).stop 2024-09-10T21:24:27.092 INFO:teuthology.misc.health.smithi017.stderr:2024-09-10T21:24:27.087+0000 7f85e0800640 1 -- 172.21.15.17:0/3067931234 shutdown_connections 2024-09-10T21:24:27.092 INFO:teuthology.misc.health.smithi017.stderr:2024-09-10T21:24:27.087+0000 7f85e0800640 1 --2- 172.21.15.17:0/3067931234 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f85d80c9360 0x7f85d80cb0e0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:27.092 INFO:teuthology.misc.health.smithi017.stderr:2024-09-10T21:24:27.087+0000 7f85e0800640 1 --2- 172.21.15.17:0/3067931234 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f85d8070bb0 0x7f85d80caba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:27.092 INFO:teuthology.misc.health.smithi017.stderr:2024-09-10T21:24:27.087+0000 7f85e0800640 1 -- 172.21.15.17:0/3067931234 >> 172.21.15.17:0/3067931234 conn(0x7f85d8074f20 msgr2=0x7f85d80715f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:27.092 INFO:teuthology.misc.health.smithi017.stderr:2024-09-10T21:24:27.087+0000 7f85e0800640 1 -- 172.21.15.17:0/3067931234 shutdown_connections 2024-09-10T21:24:27.092 INFO:teuthology.misc.health.smithi017.stderr:2024-09-10T21:24:27.087+0000 7f85e0800640 1 -- 172.21.15.17:0/3067931234 wait complete. 2024-09-10T21:24:27.093 INFO:teuthology.misc.health.smithi017.stderr:2024-09-10T21:24:27.091+0000 7f85e0800640 1 Processor -- start 2024-09-10T21:24:27.093 INFO:teuthology.misc.health.smithi017.stderr:2024-09-10T21:24:27.091+0000 7f85e0800640 1 -- start start 2024-09-10T21:24:27.095 INFO:teuthology.misc.health.smithi017.stderr:2024-09-10T21:24:27.091+0000 7f85e0800640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f85d8070bb0 0x7f85d81f4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:27.095 INFO:teuthology.misc.health.smithi017.stderr:2024-09-10T21:24:27.091+0000 7f85e0800640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f85d8070f90 0x7f85d81502c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:27.095 INFO:teuthology.misc.health.smithi017.stderr:2024-09-10T21:24:27.091+0000 7f85e0800640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f85d80c9360 0x7f85d806e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:27.095 INFO:teuthology.misc.health.smithi017.stderr:2024-09-10T21:24:27.091+0000 7f85e0800640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f85d80c8d60 con 0x7f85d8070bb0 2024-09-10T21:24:27.095 INFO:teuthology.misc.health.smithi017.stderr:2024-09-10T21:24:27.091+0000 7f85e0800640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f85d80c8be0 con 0x7f85d8070f90 2024-09-10T21:24:27.095 INFO:teuthology.misc.health.smithi017.stderr:2024-09-10T21:24:27.091+0000 7f85e0800640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f85d8165590 con 0x7f85d80c9360 2024-09-10T21:24:27.095 INFO:teuthology.misc.health.smithi017.stderr:2024-09-10T21:24:27.091+0000 7f85dde00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f85d8070bb0 0x7f85d81f4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:27.096 INFO:teuthology.misc.health.smithi017.stderr:2024-09-10T21:24:27.091+0000 7f85dde00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f85d8070bb0 0x7f85d81f4360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39436/0 (socket says 172.21.15.17:39436) 2024-09-10T21:24:27.096 INFO:teuthology.misc.health.smithi017.stderr:2024-09-10T21:24:27.091+0000 7f85dde00640 1 -- 172.21.15.17:0/1108863037 learned_addr learned my addr 172.21.15.17:0/1108863037 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:24:27.096 INFO:teuthology.misc.health.smithi017.stderr:2024-09-10T21:24:27.091+0000 7f85de800640 1 --2- 172.21.15.17:0/1108863037 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f85d80c9360 0x7f85d806e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:27.096 INFO:teuthology.misc.health.smithi017.stderr:2024-09-10T21:24:27.091+0000 7f85dd400640 1 --2- 172.21.15.17:0/1108863037 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f85d8070f90 0x7f85d81502c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:27.096 INFO:teuthology.misc.health.smithi017.stderr:2024-09-10T21:24:27.091+0000 7f85dde00640 1 -- 172.21.15.17:0/1108863037 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f85d80c9360 msgr2=0x7f85d806e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:27.096 INFO:teuthology.misc.health.smithi017.stderr:2024-09-10T21:24:27.091+0000 7f85dde00640 1 --2- 172.21.15.17:0/1108863037 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f85d80c9360 0x7f85d806e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:27.096 INFO:teuthology.misc.health.smithi017.stderr:2024-09-10T21:24:27.091+0000 7f85dde00640 1 -- 172.21.15.17:0/1108863037 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f85d8070f90 msgr2=0x7f85d81502c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:27.096 INFO:teuthology.misc.health.smithi017.stderr:2024-09-10T21:24:27.091+0000 7f85dde00640 1 --2- 172.21.15.17:0/1108863037 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f85d8070f90 0x7f85d81502c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:27.096 INFO:teuthology.misc.health.smithi017.stderr:2024-09-10T21:24:27.091+0000 7f85dde00640 1 -- 172.21.15.17:0/1108863037 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85d8150800 con 0x7f85d8070bb0 2024-09-10T21:24:27.096 INFO:teuthology.misc.health.smithi017.stderr:2024-09-10T21:24:27.091+0000 7f85dde00640 1 --2- 172.21.15.17:0/1108863037 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f85d8070bb0 0x7f85d81f4360 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f85cc00d9b0 tx=0x7f85cc00de70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:27.097 INFO:teuthology.misc.health.smithi017.stderr:2024-09-10T21:24:27.091+0000 7f85d2000640 1 -- 172.21.15.17:0/1108863037 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85cc017070 con 0x7f85d8070bb0 2024-09-10T21:24:27.097 INFO:teuthology.misc.health.smithi017.stderr:2024-09-10T21:24:27.091+0000 7f85d2000640 1 -- 172.21.15.17:0/1108863037 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f85cc004310 con 0x7f85d8070bb0 2024-09-10T21:24:27.097 INFO:teuthology.misc.health.smithi017.stderr:2024-09-10T21:24:27.091+0000 7f85e0800640 1 -- 172.21.15.17:0/1108863037 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85d80c56e0 con 0x7f85d8070bb0 2024-09-10T21:24:27.097 INFO:teuthology.misc.health.smithi017.stderr:2024-09-10T21:24:27.091+0000 7f85d2000640 1 -- 172.21.15.17:0/1108863037 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85cc009c30 con 0x7f85d8070bb0 2024-09-10T21:24:27.097 INFO:teuthology.misc.health.smithi017.stderr:2024-09-10T21:24:27.091+0000 7f85e0800640 1 -- 172.21.15.17:0/1108863037 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85d80c5ca0 con 0x7f85d8070bb0 2024-09-10T21:24:27.098 INFO:teuthology.misc.health.smithi017.stderr:2024-09-10T21:24:27.091+0000 7f85d2000640 1 -- 172.21.15.17:0/1108863037 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 1) ==== 811+0+0 (secure 0 0 0) 0x7f85cc023060 con 0x7f85d8070bb0 2024-09-10T21:24:27.098 INFO:teuthology.misc.health.smithi017.stderr:2024-09-10T21:24:27.091+0000 7f85d2000640 1 -- 172.21.15.17:0/1108863037 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f85cc009dd0 con 0x7f85d8070bb0 2024-09-10T21:24:27.098 INFO:teuthology.misc.health.smithi017.stderr:2024-09-10T21:24:27.095+0000 7f85e0800640 1 -- 172.21.15.17:0/1108863037 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85d8150f50 con 0x7f85d8070bb0 2024-09-10T21:24:27.099 INFO:teuthology.misc.health.smithi017.stderr:2024-09-10T21:24:27.095+0000 7f85d2000640 1 -- 172.21.15.17:0/1108863037 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+76091 (secure 0 0 0) 0x7f85d8150800 con 0x7f85d8070bb0 2024-09-10T21:24:27.161 INFO:teuthology.misc.health.smithi017.stderr:2024-09-10T21:24:27.159+0000 7f85e0800640 1 -- 172.21.15.17:0/1108863037 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f85d8071690 con 0x7f85d8070bb0 2024-09-10T21:24:27.161 INFO:teuthology.misc.health.smithi017.stderr:2024-09-10T21:24:27.159+0000 7f85d2000640 1 -- 172.21.15.17:0/1108863037 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f85cc005000 con 0x7f85d8070bb0 2024-09-10T21:24:27.161 INFO:teuthology.misc.health.smithi017.stdout: 2024-09-10T21:24:27.162 INFO:teuthology.misc.health.smithi017.stdout:{"epoch":10,"fsid":"5e2f023f-d6a6-49f4-a256-f4807dce3739","created":"2024-09-10T21:24:21.750712+0000","modified":"2024-09-10T21:24:25.831127+0000","last_up_change":"0.000000","last_in_change":"2024-09-10T21:24:25.831127+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"0f5fabc0-434e-4066-b430-233969a548f2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"f6691cb3-2a91-4c90-a9fd-ece3c11dd6eb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"cb61adb0-a216-437b-9598-06fa7c84aa64","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"6d2da26a-e0bf-47dc-b020-af86e6bcfccc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"c65071dd-9ca7-4f1d-a584-45dc121419a7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"235389de-7777-4744-81b8-d0bfc11190e2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"cea70a57-3715-4ebb-8511-a528a9df2b1b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"01cc111b-a45f-4ca3-9cc1-7fe0afbeba45","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:24:27.162 INFO:teuthology.misc.health.smithi017.stderr:2024-09-10T21:24:27.159+0000 7f85e0800640 1 -- 172.21.15.17:0/1108863037 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f85d8070bb0 msgr2=0x7f85d81f4360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:27.163 INFO:teuthology.misc.health.smithi017.stderr:2024-09-10T21:24:27.159+0000 7f85e0800640 1 --2- 172.21.15.17:0/1108863037 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f85d8070bb0 0x7f85d81f4360 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f85cc00d9b0 tx=0x7f85cc00de70 comp rx=0 tx=0).stop 2024-09-10T21:24:27.163 INFO:teuthology.misc.health.smithi017.stderr:2024-09-10T21:24:27.159+0000 7f85e0800640 1 -- 172.21.15.17:0/1108863037 shutdown_connections 2024-09-10T21:24:27.163 INFO:teuthology.misc.health.smithi017.stderr:2024-09-10T21:24:27.159+0000 7f85e0800640 1 --2- 172.21.15.17:0/1108863037 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f85d80c9360 0x7f85d806e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:27.163 INFO:teuthology.misc.health.smithi017.stderr:2024-09-10T21:24:27.159+0000 7f85e0800640 1 --2- 172.21.15.17:0/1108863037 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f85d8070f90 0x7f85d81502c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:27.163 INFO:teuthology.misc.health.smithi017.stderr:2024-09-10T21:24:27.159+0000 7f85e0800640 1 --2- 172.21.15.17:0/1108863037 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f85d8070bb0 0x7f85d81f4360 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:27.163 INFO:teuthology.misc.health.smithi017.stderr:2024-09-10T21:24:27.159+0000 7f85e0800640 1 -- 172.21.15.17:0/1108863037 >> 172.21.15.17:0/1108863037 conn(0x7f85d8074f20 msgr2=0x7f85d815b420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:27.163 INFO:teuthology.misc.health.smithi017.stderr:2024-09-10T21:24:27.159+0000 7f85e0800640 1 -- 172.21.15.17:0/1108863037 shutdown_connections 2024-09-10T21:24:27.163 INFO:teuthology.misc.health.smithi017.stderr:2024-09-10T21:24:27.159+0000 7f85e0800640 1 -- 172.21.15.17:0/1108863037 wait complete. 2024-09-10T21:24:27.179 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-09-10T21:24:27.328 INFO:tasks.ceph.osd.5.smithi031.stderr:2024-09-10T21:24:27.324+0000 7ff717aac7c0 -1 Falling back to public interface 2024-09-10T21:24:27.349 INFO:tasks.ceph.osd.3.smithi017.stderr:2024-09-10T21:24:27.347+0000 7f7d7a6807c0 -1 Falling back to public interface 2024-09-10T21:24:27.353 INFO:tasks.ceph.osd.0.smithi017.stderr:2024-09-10T21:24:27.351+0000 7f74132517c0 -1 Falling back to public interface 2024-09-10T21:24:27.577 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-10T21:24:27.575+0000 7feb7e2547c0 -1 Falling back to public interface 2024-09-10T21:24:27.580 INFO:tasks.ceph.osd.4.smithi031.stderr:2024-09-10T21:24:27.576+0000 7feb6bfb67c0 -1 Falling back to public interface 2024-09-10T21:24:27.591 INFO:tasks.ceph.osd.7.smithi031.stderr:2024-09-10T21:24:27.588+0000 7feb493297c0 -1 Falling back to public interface 2024-09-10T21:24:27.597 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-10T21:24:27.595+0000 7f7e5eb9b7c0 -1 Falling back to public interface 2024-09-10T21:24:27.616 INFO:tasks.ceph.osd.6.smithi031.stderr:2024-09-10T21:24:27.612+0000 7fe6518a07c0 -1 Falling back to public interface 2024-09-10T21:24:28.347 INFO:tasks.ceph.mgr.x.smithi031.stderr:2024-09-10T21:24:28.344+0000 7f04ac600640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-09-10T21:24:28.348 INFO:tasks.ceph.mgr.x.smithi031.stderr:2024-09-10T21:24:28.344+0000 7f04ac600640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-09-10T21:24:31.632 INFO:tasks.ceph.osd.0.smithi017.stderr:2024-09-10T21:24:31.627+0000 7f74132517c0 -1 osd.0 0 log_to_monitors true 2024-09-10T21:24:31.764 INFO:tasks.ceph.osd.4.smithi031.stderr:2024-09-10T21:24:31.760+0000 7feb6bfb67c0 -1 osd.4 0 log_to_monitors true 2024-09-10T21:24:31.778 INFO:tasks.ceph.osd.3.smithi017.stderr:2024-09-10T21:24:31.775+0000 7f7d7a6807c0 -1 osd.3 0 log_to_monitors true 2024-09-10T21:24:31.782 INFO:tasks.ceph.osd.5.smithi031.stderr:2024-09-10T21:24:31.776+0000 7ff717aac7c0 -1 osd.5 0 log_to_monitors true 2024-09-10T21:24:31.904 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-10T21:24:31.899+0000 7f7e5eb9b7c0 -1 osd.1 0 log_to_monitors true 2024-09-10T21:24:31.906 INFO:tasks.ceph.osd.7.smithi031.stderr:2024-09-10T21:24:31.900+0000 7feb493297c0 -1 osd.7 0 log_to_monitors true 2024-09-10T21:24:31.925 INFO:tasks.ceph.osd.6.smithi031.stderr:2024-09-10T21:24:31.920+0000 7fe6518a07c0 -1 osd.6 0 log_to_monitors true 2024-09-10T21:24:32.305 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-10T21:24:32.303+0000 7feb7e2547c0 -1 osd.2 0 log_to_monitors true 2024-09-10T21:24:33.767 INFO:tasks.ceph.osd.5.smithi031.stderr:2024-09-10T21:24:33.759+0000 7ff712a00640 -1 osd.5 0 waiting for initial osdmap 2024-09-10T21:24:33.797 INFO:tasks.ceph.osd.5.smithi031.stderr:2024-09-10T21:24:33.791+0000 7ff708a00640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T21:24:33.821 INFO:tasks.ceph.osd.3.smithi017.stderr:2024-09-10T21:24:33.819+0000 7f7d76000640 -1 osd.3 0 waiting for initial osdmap 2024-09-10T21:24:33.833 INFO:tasks.ceph.osd.0.smithi017.stderr:2024-09-10T21:24:33.831+0000 7f740e200640 -1 osd.0 0 waiting for initial osdmap 2024-09-10T21:24:33.841 INFO:tasks.ceph.osd.3.smithi017.stderr:2024-09-10T21:24:33.839+0000 7f7d6b600640 -1 osd.3 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T21:24:33.878 INFO:tasks.ceph.osd.0.smithi017.stderr:2024-09-10T21:24:33.875+0000 7f7404200640 -1 osd.0 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T21:24:33.895 INFO:tasks.ceph.osd.4.smithi031.stderr:2024-09-10T21:24:33.891+0000 7feb66e00640 -1 osd.4 0 waiting for initial osdmap 2024-09-10T21:24:33.924 INFO:tasks.ceph.osd.4.smithi031.stderr:2024-09-10T21:24:33.919+0000 7feb5ce00640 -1 osd.4 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T21:24:33.986 DEBUG:teuthology.orchestra.run.smithi017:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-10T21:24:34.171 INFO:teuthology.misc.health.smithi017.stderr:2024-09-10T21:24:34.167+0000 7f6188600640 1 -- 172.21.15.17:0/173609345 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6180070bb0 msgr2=0x7f61800caba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:34.172 INFO:teuthology.misc.health.smithi017.stderr:2024-09-10T21:24:34.167+0000 7f6188600640 1 --2- 172.21.15.17:0/173609345 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6180070bb0 0x7f61800caba0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f6174009840 tx=0x7f617401c820 comp rx=0 tx=0).stop 2024-09-10T21:24:34.172 INFO:teuthology.misc.health.smithi017.stderr:2024-09-10T21:24:34.167+0000 7f6188600640 1 -- 172.21.15.17:0/173609345 shutdown_connections 2024-09-10T21:24:34.172 INFO:teuthology.misc.health.smithi017.stderr:2024-09-10T21:24:34.167+0000 7f6188600640 1 --2- 172.21.15.17:0/173609345 >> v2:172.21.15.17:3300/0 conn(0x7f61800c9360 0x7f61800cb0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:34.172 INFO:teuthology.misc.health.smithi017.stderr:2024-09-10T21:24:34.167+0000 7f6188600640 1 --2- 172.21.15.17:0/173609345 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6180070bb0 0x7f61800caba0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:34.172 INFO:teuthology.misc.health.smithi017.stderr:2024-09-10T21:24:34.167+0000 7f6188600640 1 -- 172.21.15.17:0/173609345 >> 172.21.15.17:0/173609345 conn(0x7f6180074f20 msgr2=0x7f61800715f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:34.172 INFO:teuthology.misc.health.smithi017.stderr:2024-09-10T21:24:34.167+0000 7f6188600640 1 -- 172.21.15.17:0/173609345 shutdown_connections 2024-09-10T21:24:34.172 INFO:teuthology.misc.health.smithi017.stderr:2024-09-10T21:24:34.167+0000 7f6188600640 1 -- 172.21.15.17:0/173609345 wait complete. 2024-09-10T21:24:34.174 INFO:teuthology.misc.health.smithi017.stderr:2024-09-10T21:24:34.171+0000 7f6188600640 1 Processor -- start 2024-09-10T21:24:34.174 INFO:teuthology.misc.health.smithi017.stderr:2024-09-10T21:24:34.171+0000 7f6188600640 1 -- start start 2024-09-10T21:24:34.175 INFO:teuthology.misc.health.smithi017.stderr:2024-09-10T21:24:34.171+0000 7f6188600640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6180070bb0 0x7f6180158b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:34.176 INFO:teuthology.misc.health.smithi017.stderr:2024-09-10T21:24:34.171+0000 7f6188600640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6180070f90 0x7f61801590c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:34.176 INFO:teuthology.misc.health.smithi017.stderr:2024-09-10T21:24:34.171+0000 7f6188600640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f61800c9360 0x7f61800c6960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:34.176 INFO:teuthology.misc.health.smithi017.stderr:2024-09-10T21:24:34.171+0000 7f6188600640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f61800c8df0 con 0x7f61800c9360 2024-09-10T21:24:34.176 INFO:teuthology.misc.health.smithi017.stderr:2024-09-10T21:24:34.171+0000 7f6188600640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f61800c9910 con 0x7f6180070f90 2024-09-10T21:24:34.176 INFO:teuthology.misc.health.smithi017.stderr:2024-09-10T21:24:34.171+0000 7f6188600640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f6180165590 con 0x7f6180070bb0 2024-09-10T21:24:34.176 INFO:teuthology.misc.health.smithi017.stderr:2024-09-10T21:24:34.171+0000 7f6185c00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6180070bb0 0x7f6180158b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:34.176 INFO:teuthology.misc.health.smithi017.stderr:2024-09-10T21:24:34.171+0000 7f6185c00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6180070bb0 0x7f6180158b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3301/0 says I am v2:172.21.15.17:41634/0 (socket says 172.21.15.17:41634) 2024-09-10T21:24:34.176 INFO:teuthology.misc.health.smithi017.stderr:2024-09-10T21:24:34.171+0000 7f6185c00640 1 -- 172.21.15.17:0/1044362467 learned_addr learned my addr 172.21.15.17:0/1044362467 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:24:34.176 INFO:teuthology.misc.health.smithi017.stderr:2024-09-10T21:24:34.171+0000 7f6185200640 1 --2- 172.21.15.17:0/1044362467 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6180070f90 0x7f61801590c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:34.177 INFO:teuthology.misc.health.smithi017.stderr:2024-09-10T21:24:34.175+0000 7f6185c00640 1 -- 172.21.15.17:0/1044362467 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6180070f90 msgr2=0x7f61801590c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:34.177 INFO:teuthology.misc.health.smithi017.stderr:2024-09-10T21:24:34.175+0000 7f6185c00640 1 --2- 172.21.15.17:0/1044362467 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6180070f90 0x7f61801590c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:34.177 INFO:teuthology.misc.health.smithi017.stderr:2024-09-10T21:24:34.175+0000 7f6185c00640 1 -- 172.21.15.17:0/1044362467 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f61800c9360 msgr2=0x7f61800c6960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:34.177 INFO:teuthology.misc.health.smithi017.stderr:2024-09-10T21:24:34.175+0000 7f6185c00640 1 --2- 172.21.15.17:0/1044362467 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f61800c9360 0x7f61800c6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:34.177 INFO:teuthology.misc.health.smithi017.stderr:2024-09-10T21:24:34.175+0000 7f6185c00640 1 -- 172.21.15.17:0/1044362467 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61800c6340 con 0x7f6180070bb0 2024-09-10T21:24:34.177 INFO:teuthology.misc.health.smithi017.stderr:2024-09-10T21:24:34.175+0000 7f6185200640 1 --2- 172.21.15.17:0/1044362467 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6180070f90 0x7f61801590c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:24:34.178 INFO:teuthology.misc.health.smithi017.stderr:2024-09-10T21:24:34.175+0000 7f6185c00640 1 --2- 172.21.15.17:0/1044362467 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6180070bb0 0x7f6180158b80 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f6174009790 tx=0x7f61740040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:34.178 INFO:teuthology.misc.health.smithi017.stderr:2024-09-10T21:24:34.175+0000 7f617d600640 1 -- 172.21.15.17:0/1044362467 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61740bb070 con 0x7f6180070bb0 2024-09-10T21:24:34.178 INFO:teuthology.misc.health.smithi017.stderr:2024-09-10T21:24:34.175+0000 7f617d600640 1 -- 172.21.15.17:0/1044362467 <== mon.2 v2:172.21.15.31:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6174005030 con 0x7f6180070bb0 2024-09-10T21:24:34.178 INFO:teuthology.misc.health.smithi017.stderr:2024-09-10T21:24:34.175+0000 7f6188600640 1 -- 172.21.15.17:0/1044362467 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61800cdf30 con 0x7f6180070bb0 2024-09-10T21:24:34.178 INFO:teuthology.misc.health.smithi017.stderr:2024-09-10T21:24:34.175+0000 7f617d600640 1 -- 172.21.15.17:0/1044362467 <== mon.2 v2:172.21.15.31:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61740b2040 con 0x7f6180070bb0 2024-09-10T21:24:34.179 INFO:teuthology.misc.health.smithi017.stderr:2024-09-10T21:24:34.175+0000 7f6188600640 1 -- 172.21.15.17:0/1044362467 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f61800cd860 con 0x7f6180070bb0 2024-09-10T21:24:34.181 INFO:teuthology.misc.health.smithi017.stderr:2024-09-10T21:24:34.179+0000 7f617d600640 1 -- 172.21.15.17:0/1044362467 <== mon.2 v2:172.21.15.31:3301/0 4 ==== mgrmap(e 4) ==== 96338+0+0 (secure 0 0 0) 0x7f61740b5030 con 0x7f6180070bb0 2024-09-10T21:24:34.181 INFO:teuthology.misc.health.smithi017.stderr:2024-09-10T21:24:34.179+0000 7f6188600640 1 -- 172.21.15.17:0/1044362467 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61800c66f0 con 0x7f6180070bb0 2024-09-10T21:24:34.182 INFO:teuthology.misc.health.smithi017.stderr:2024-09-10T21:24:34.179+0000 7f617d600640 1 --2- 172.21.15.17:0/1044362467 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f6160072550 0x7f6160074a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:34.182 INFO:teuthology.misc.health.smithi017.stderr:2024-09-10T21:24:34.179+0000 7f617d600640 1 -- 172.21.15.17:0/1044362467 <== mon.2 v2:172.21.15.31:3301/0 5 ==== osd_map(13..13 src has 1..13) ==== 2511+0+0 (secure 0 0 0) 0x7f617412bbd0 con 0x7f6180070bb0 2024-09-10T21:24:34.182 INFO:teuthology.misc.health.smithi017.stderr:2024-09-10T21:24:34.179+0000 7f6185200640 1 --2- 172.21.15.17:0/1044362467 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f6160072550 0x7f6160074a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:34.183 INFO:teuthology.misc.health.smithi017.stderr:2024-09-10T21:24:34.179+0000 7f6185200640 1 --2- 172.21.15.17:0/1044362467 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f6160072550 0x7f6160074a10 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f61800c86f0 tx=0x7f616c006f90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:34.194 INFO:teuthology.misc.health.smithi017.stderr:2024-09-10T21:24:34.191+0000 7f617d600640 1 -- 172.21.15.17:0/1044362467 <== mon.2 v2:172.21.15.31:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f61800c66f0 con 0x7f6180070bb0 2024-09-10T21:24:34.493 INFO:teuthology.misc.health.smithi017.stderr:2024-09-10T21:24:34.491+0000 7f6188600640 1 -- 172.21.15.17:0/1044362467 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f618015c120 con 0x7f6180070bb0 2024-09-10T21:24:34.494 INFO:teuthology.misc.health.smithi017.stderr:2024-09-10T21:24:34.491+0000 7f617d600640 1 -- 172.21.15.17:0/1044362467 <== mon.2 v2:172.21.15.31:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v13) ==== 74+0+7555 (secure 0 0 0) 0x7f6174016410 con 0x7f6180070bb0 2024-09-10T21:24:34.497 INFO:teuthology.misc.health.smithi017.stdout: 2024-09-10T21:24:34.497 INFO:teuthology.misc.health.smithi017.stdout:{"epoch":13,"fsid":"5e2f023f-d6a6-49f4-a256-f4807dce3739","created":"2024-09-10T21:24:21.750712+0000","modified":"2024-09-10T21:24:33.809701+0000","last_up_change":"2024-09-10T21:24:33.809701+0000","last_in_change":"2024-09-10T21:24:25.831127+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"0f5fabc0-434e-4066-b430-233969a548f2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"f6691cb3-2a91-4c90-a9fd-ece3c11dd6eb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"cb61adb0-a216-437b-9598-06fa7c84aa64","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"6d2da26a-e0bf-47dc-b020-af86e6bcfccc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"c65071dd-9ca7-4f1d-a584-45dc121419a7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"235389de-7777-4744-81b8-d0bfc11190e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":1472711057},{"type":"v1","addr":"172.21.15.31:6801","nonce":1472711057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":1472711057},{"type":"v1","addr":"172.21.15.31:6803","nonce":1472711057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":1472711057},{"type":"v1","addr":"172.21.15.31:6807","nonce":1472711057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":1472711057},{"type":"v1","addr":"172.21.15.31:6805","nonce":1472711057}]},"public_addr":"172.21.15.31:6801/1472711057","cluster_addr":"172.21.15.31:6803/1472711057","heartbeat_back_addr":"172.21.15.31:6807/1472711057","heartbeat_front_addr":"172.21.15.31:6805/1472711057","state":["exists","up"]},{"osd":6,"uuid":"cea70a57-3715-4ebb-8511-a528a9df2b1b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"01cc111b-a45f-4ca3-9cc1-7fe0afbeba45","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:24:34.498 INFO:teuthology.misc.health.smithi017.stderr:2024-09-10T21:24:34.495+0000 7f6172a00640 1 -- 172.21.15.17:0/1044362467 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f6160072550 msgr2=0x7f6160074a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:34.498 INFO:teuthology.misc.health.smithi017.stderr:2024-09-10T21:24:34.495+0000 7f6172a00640 1 --2- 172.21.15.17:0/1044362467 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f6160072550 0x7f6160074a10 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f61800c86f0 tx=0x7f616c006f90 comp rx=0 tx=0).stop 2024-09-10T21:24:34.498 INFO:teuthology.misc.health.smithi017.stderr:2024-09-10T21:24:34.495+0000 7f6172a00640 1 -- 172.21.15.17:0/1044362467 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6180070bb0 msgr2=0x7f6180158b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:34.498 INFO:teuthology.misc.health.smithi017.stderr:2024-09-10T21:24:34.495+0000 7f6172a00640 1 --2- 172.21.15.17:0/1044362467 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6180070bb0 0x7f6180158b80 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f6174009790 tx=0x7f61740040c0 comp rx=0 tx=0).stop 2024-09-10T21:24:34.499 INFO:teuthology.misc.health.smithi017.stderr:2024-09-10T21:24:34.495+0000 7f6172a00640 1 -- 172.21.15.17:0/1044362467 shutdown_connections 2024-09-10T21:24:34.499 INFO:teuthology.misc.health.smithi017.stderr:2024-09-10T21:24:34.495+0000 7f6172a00640 1 --2- 172.21.15.17:0/1044362467 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f6160072550 0x7f6160074a10 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:34.499 INFO:teuthology.misc.health.smithi017.stderr:2024-09-10T21:24:34.495+0000 7f6172a00640 1 --2- 172.21.15.17:0/1044362467 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f61800c9360 0x7f61800c6960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:34.499 INFO:teuthology.misc.health.smithi017.stderr:2024-09-10T21:24:34.495+0000 7f6172a00640 1 --2- 172.21.15.17:0/1044362467 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6180070f90 0x7f61801590c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:34.499 INFO:teuthology.misc.health.smithi017.stderr:2024-09-10T21:24:34.495+0000 7f6172a00640 1 --2- 172.21.15.17:0/1044362467 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6180070bb0 0x7f6180158b80 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:34.499 INFO:teuthology.misc.health.smithi017.stderr:2024-09-10T21:24:34.495+0000 7f6172a00640 1 -- 172.21.15.17:0/1044362467 >> 172.21.15.17:0/1044362467 conn(0x7f6180074f20 msgr2=0x7f6180075a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:34.499 INFO:teuthology.misc.health.smithi017.stderr:2024-09-10T21:24:34.495+0000 7f6172a00640 1 -- 172.21.15.17:0/1044362467 shutdown_connections 2024-09-10T21:24:34.499 INFO:teuthology.misc.health.smithi017.stderr:2024-09-10T21:24:34.495+0000 7f6172a00640 1 -- 172.21.15.17:0/1044362467 wait complete. 2024-09-10T21:24:34.524 DEBUG:teuthology.misc:1 of 8 OSDs are up 2024-09-10T21:24:34.831 INFO:tasks.ceph.osd.7.smithi031.stderr:2024-09-10T21:24:34.827+0000 7feb44200640 -1 osd.7 0 waiting for initial osdmap 2024-09-10T21:24:34.846 INFO:tasks.ceph.osd.6.smithi031.stderr:2024-09-10T21:24:34.839+0000 7fe64d200640 -1 osd.6 0 waiting for initial osdmap 2024-09-10T21:24:34.877 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-10T21:24:34.875+0000 7f7e59a00640 -1 osd.1 0 waiting for initial osdmap 2024-09-10T21:24:34.884 INFO:tasks.ceph.osd.7.smithi031.stderr:2024-09-10T21:24:34.879+0000 7feb3a200640 -1 osd.7 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T21:24:34.892 INFO:tasks.ceph.osd.6.smithi031.stderr:2024-09-10T21:24:34.887+0000 7fe642800640 -1 osd.6 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T21:24:34.900 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-10T21:24:34.899+0000 7feb79200640 -1 osd.2 0 waiting for initial osdmap 2024-09-10T21:24:34.909 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-10T21:24:34.907+0000 7f7e4fa00640 -1 osd.1 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T21:24:34.933 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-10T21:24:34.931+0000 7feb6f200640 -1 osd.2 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T21:24:41.332 DEBUG:teuthology.orchestra.run.smithi017:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-10T21:24:41.480 INFO:teuthology.misc.health.smithi017.stderr:2024-09-10T21:24:41.474+0000 7fdd94600640 1 -- 172.21.15.17:0/105401409 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdd8c0ce1e0 msgr2=0x7fdd8c070d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:41.481 INFO:teuthology.misc.health.smithi017.stderr:2024-09-10T21:24:41.474+0000 7fdd94600640 1 --2- 172.21.15.17:0/105401409 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdd8c0ce1e0 0x7fdd8c070d20 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fdd8c074940 tx=0x7fdd8001c7e0 comp rx=0 tx=0).stop 2024-09-10T21:24:41.481 INFO:teuthology.misc.health.smithi017.stderr:2024-09-10T21:24:41.474+0000 7fdd94600640 1 -- 172.21.15.17:0/105401409 shutdown_connections 2024-09-10T21:24:41.481 INFO:teuthology.misc.health.smithi017.stderr:2024-09-10T21:24:41.474+0000 7fdd94600640 1 --2- 172.21.15.17:0/105401409 >> v2:172.21.15.31:3300/0 conn(0x7fdd8c0c5240 0x7fdd8c0ce5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:41.481 INFO:teuthology.misc.health.smithi017.stderr:2024-09-10T21:24:41.474+0000 7fdd94600640 1 --2- 172.21.15.17:0/105401409 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdd8c0ce1e0 0x7fdd8c070d20 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:41.481 INFO:teuthology.misc.health.smithi017.stderr:2024-09-10T21:24:41.474+0000 7fdd94600640 1 -- 172.21.15.17:0/105401409 >> 172.21.15.17:0/105401409 conn(0x7fdd8c074d90 msgr2=0x7fdd8c068ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:41.481 INFO:teuthology.misc.health.smithi017.stderr:2024-09-10T21:24:41.474+0000 7fdd94600640 1 -- 172.21.15.17:0/105401409 shutdown_connections 2024-09-10T21:24:41.481 INFO:teuthology.misc.health.smithi017.stderr:2024-09-10T21:24:41.474+0000 7fdd94600640 1 -- 172.21.15.17:0/105401409 wait complete. 2024-09-10T21:24:41.482 INFO:teuthology.misc.health.smithi017.stderr:2024-09-10T21:24:41.478+0000 7fdd94600640 1 Processor -- start 2024-09-10T21:24:41.482 INFO:teuthology.misc.health.smithi017.stderr:2024-09-10T21:24:41.478+0000 7fdd94600640 1 -- start start 2024-09-10T21:24:41.483 INFO:teuthology.misc.health.smithi017.stderr:2024-09-10T21:24:41.478+0000 7fdd94600640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdd8c0c3290 0x7fdd8c1eea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:41.483 INFO:teuthology.misc.health.smithi017.stderr:2024-09-10T21:24:41.478+0000 7fdd94600640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fdd8c0c5240 0x7fdd8c1eef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:41.483 INFO:teuthology.misc.health.smithi017.stderr:2024-09-10T21:24:41.478+0000 7fdd94600640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd8c0ce1e0 0x7fdd8c14ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:41.483 INFO:teuthology.misc.health.smithi017.stderr:2024-09-10T21:24:41.478+0000 7fdd94600640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fdd8c0cbc00 con 0x7fdd8c0c3290 2024-09-10T21:24:41.484 INFO:teuthology.misc.health.smithi017.stderr:2024-09-10T21:24:41.478+0000 7fdd94600640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fdd8c0cba80 con 0x7fdd8c0ce1e0 2024-09-10T21:24:41.484 INFO:teuthology.misc.health.smithi017.stderr:2024-09-10T21:24:41.478+0000 7fdd94600640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7fdd8c0714f0 con 0x7fdd8c0c5240 2024-09-10T21:24:41.484 INFO:teuthology.misc.health.smithi017.stderr:2024-09-10T21:24:41.478+0000 7fdd91c00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdd8c0c3290 0x7fdd8c1eea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:41.484 INFO:teuthology.misc.health.smithi017.stderr:2024-09-10T21:24:41.478+0000 7fdd91c00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdd8c0c3290 0x7fdd8c1eea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44500/0 (socket says 172.21.15.17:44500) 2024-09-10T21:24:41.484 INFO:teuthology.misc.health.smithi017.stderr:2024-09-10T21:24:41.478+0000 7fdd91c00640 1 -- 172.21.15.17:0/3976388431 learned_addr learned my addr 172.21.15.17:0/3976388431 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:24:41.484 INFO:teuthology.misc.health.smithi017.stderr:2024-09-10T21:24:41.478+0000 7fdd92600640 1 --2- 172.21.15.17:0/3976388431 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd8c0ce1e0 0x7fdd8c14ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:41.484 INFO:teuthology.misc.health.smithi017.stderr:2024-09-10T21:24:41.478+0000 7fdd91c00640 1 -- 172.21.15.17:0/3976388431 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fdd8c0c5240 msgr2=0x7fdd8c1eef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:41.484 INFO:teuthology.misc.health.smithi017.stderr:2024-09-10T21:24:41.478+0000 7fdd91c00640 1 --2- 172.21.15.17:0/3976388431 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fdd8c0c5240 0x7fdd8c1eef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:41.484 INFO:teuthology.misc.health.smithi017.stderr:2024-09-10T21:24:41.478+0000 7fdd91c00640 1 -- 172.21.15.17:0/3976388431 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd8c0ce1e0 msgr2=0x7fdd8c14ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:41.484 INFO:teuthology.misc.health.smithi017.stderr:2024-09-10T21:24:41.478+0000 7fdd91c00640 1 --2- 172.21.15.17:0/3976388431 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd8c0ce1e0 0x7fdd8c14ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:41.484 INFO:teuthology.misc.health.smithi017.stderr:2024-09-10T21:24:41.478+0000 7fdd91c00640 1 -- 172.21.15.17:0/3976388431 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd8c151030 con 0x7fdd8c0c3290 2024-09-10T21:24:41.485 INFO:teuthology.misc.health.smithi017.stderr:2024-09-10T21:24:41.478+0000 7fdd91c00640 1 --2- 172.21.15.17:0/3976388431 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdd8c0c3290 0x7fdd8c1eea50 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fdd800b0040 tx=0x7fdd800042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:41.485 INFO:teuthology.misc.health.smithi017.stderr:2024-09-10T21:24:41.478+0000 7fdd89600640 1 -- 172.21.15.17:0/3976388431 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd800b2040 con 0x7fdd8c0c3290 2024-09-10T21:24:41.485 INFO:teuthology.misc.health.smithi017.stderr:2024-09-10T21:24:41.478+0000 7fdd89600640 1 -- 172.21.15.17:0/3976388431 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdd80005030 con 0x7fdd8c0c3290 2024-09-10T21:24:41.485 INFO:teuthology.misc.health.smithi017.stderr:2024-09-10T21:24:41.478+0000 7fdd89600640 1 -- 172.21.15.17:0/3976388431 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd80005480 con 0x7fdd8c0c3290 2024-09-10T21:24:41.485 INFO:teuthology.misc.health.smithi017.stderr:2024-09-10T21:24:41.478+0000 7fdd94600640 1 -- 172.21.15.17:0/3976388431 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd8c150b70 con 0x7fdd8c0c3290 2024-09-10T21:24:41.486 INFO:teuthology.misc.health.smithi017.stderr:2024-09-10T21:24:41.482+0000 7fdd94600640 1 -- 172.21.15.17:0/3976388431 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd8c155d20 con 0x7fdd8c0c3290 2024-09-10T21:24:41.486 INFO:teuthology.misc.health.smithi017.stderr:2024-09-10T21:24:41.482+0000 7fdd94600640 1 -- 172.21.15.17:0/3976388431 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd8c070d20 con 0x7fdd8c0c3290 2024-09-10T21:24:41.486 INFO:teuthology.misc.health.smithi017.stderr:2024-09-10T21:24:41.482+0000 7fdd89600640 1 -- 172.21.15.17:0/3976388431 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fdd800bf060 con 0x7fdd8c0c3290 2024-09-10T21:24:41.487 INFO:teuthology.misc.health.smithi017.stderr:2024-09-10T21:24:41.482+0000 7fdd89600640 1 --2- 172.21.15.17:0/3976388431 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fdd6c0725e0 0x7fdd6c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:41.487 INFO:teuthology.misc.health.smithi017.stderr:2024-09-10T21:24:41.482+0000 7fdd89600640 1 -- 172.21.15.17:0/3976388431 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7fdd8012bfb0 con 0x7fdd8c0c3290 2024-09-10T21:24:41.488 INFO:teuthology.misc.health.smithi017.stderr:2024-09-10T21:24:41.482+0000 7fdd91200640 1 --2- 172.21.15.17:0/3976388431 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fdd6c0725e0 0x7fdd6c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:41.488 INFO:teuthology.misc.health.smithi017.stderr:2024-09-10T21:24:41.486+0000 7fdd91200640 1 --2- 172.21.15.17:0/3976388431 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fdd6c0725e0 0x7fdd6c074aa0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fdd780058d0 tx=0x7fdd78005860 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:41.492 INFO:teuthology.misc.health.smithi017.stderr:2024-09-10T21:24:41.486+0000 7fdd89600640 1 -- 172.21.15.17:0/3976388431 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fdd800163d0 con 0x7fdd8c0c3290 2024-09-10T21:24:41.666 INFO:teuthology.misc.health.smithi017.stderr:2024-09-10T21:24:41.662+0000 7fdd94600640 1 -- 172.21.15.17:0/3976388431 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdd8c0718d0 con 0x7fdd8c0c3290 2024-09-10T21:24:41.667 INFO:teuthology.misc.health.smithi017.stderr:2024-09-10T21:24:41.662+0000 7fdd89600640 1 -- 172.21.15.17:0/3976388431 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) ==== 74+0+12860 (secure 0 0 0) 0x7fdd800f72f0 con 0x7fdd8c0c3290 2024-09-10T21:24:41.668 INFO:teuthology.misc.health.smithi017.stdout: 2024-09-10T21:24:41.668 INFO:teuthology.misc.health.smithi017.stdout:{"epoch":17,"fsid":"5e2f023f-d6a6-49f4-a256-f4807dce3739","created":"2024-09-10T21:24:21.750712+0000","modified":"2024-09-10T21:24:37.887870+0000","last_up_change":"2024-09-10T21:24:35.877801+0000","last_in_change":"2024-09-10T21:24:25.831127+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:24:35.362670+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0f5fabc0-434e-4066-b430-233969a548f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":4105489867},{"type":"v1","addr":"172.21.15.17:6809","nonce":4105489867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":4105489867},{"type":"v1","addr":"172.21.15.17:6811","nonce":4105489867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":4105489867},{"type":"v1","addr":"172.21.15.17:6815","nonce":4105489867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":4105489867},{"type":"v1","addr":"172.21.15.17:6813","nonce":4105489867}]},"public_addr":"172.21.15.17:6809/4105489867","cluster_addr":"172.21.15.17:6811/4105489867","heartbeat_back_addr":"172.21.15.17:6815/4105489867","heartbeat_front_addr":"172.21.15.17:6813/4105489867","state":["exists","up"]},{"osd":1,"uuid":"f6691cb3-2a91-4c90-a9fd-ece3c11dd6eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":2599842027},{"type":"v1","addr":"172.21.15.17:6825","nonce":2599842027}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":2599842027},{"type":"v1","addr":"172.21.15.17:6827","nonce":2599842027}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":2599842027},{"type":"v1","addr":"172.21.15.17:6831","nonce":2599842027}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":2599842027},{"type":"v1","addr":"172.21.15.17:6829","nonce":2599842027}]},"public_addr":"172.21.15.17:6825/2599842027","cluster_addr":"172.21.15.17:6827/2599842027","heartbeat_back_addr":"172.21.15.17:6831/2599842027","heartbeat_front_addr":"172.21.15.17:6829/2599842027","state":["exists","up"]},{"osd":2,"uuid":"cb61adb0-a216-437b-9598-06fa7c84aa64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":1248243634},{"type":"v1","addr":"172.21.15.17:6817","nonce":1248243634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":1248243634},{"type":"v1","addr":"172.21.15.17:6819","nonce":1248243634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":1248243634},{"type":"v1","addr":"172.21.15.17:6823","nonce":1248243634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":1248243634},{"type":"v1","addr":"172.21.15.17:6821","nonce":1248243634}]},"public_addr":"172.21.15.17:6817/1248243634","cluster_addr":"172.21.15.17:6819/1248243634","heartbeat_back_addr":"172.21.15.17:6823/1248243634","heartbeat_front_addr":"172.21.15.17:6821/1248243634","state":["exists","up"]},{"osd":3,"uuid":"6d2da26a-e0bf-47dc-b020-af86e6bcfccc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":2314489522},{"type":"v1","addr":"172.21.15.17:6801","nonce":2314489522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":2314489522},{"type":"v1","addr":"172.21.15.17:6803","nonce":2314489522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":2314489522},{"type":"v1","addr":"172.21.15.17:6807","nonce":2314489522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":2314489522},{"type":"v1","addr":"172.21.15.17:6805","nonce":2314489522}]},"public_addr":"172.21.15.17:6801/2314489522","cluster_addr":"172.21.15.17:6803/2314489522","heartbeat_back_addr":"172.21.15.17:6807/2314489522","heartbeat_front_addr":"172.21.15.17:6805/2314489522","state":["exists","up"]},{"osd":4,"uuid":"c65071dd-9ca7-4f1d-a584-45dc121419a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":202520744},{"type":"v1","addr":"172.21.15.31:6811","nonce":202520744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":202520744},{"type":"v1","addr":"172.21.15.31:6813","nonce":202520744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":202520744},{"type":"v1","addr":"172.21.15.31:6817","nonce":202520744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":202520744},{"type":"v1","addr":"172.21.15.31:6815","nonce":202520744}]},"public_addr":"172.21.15.31:6811/202520744","cluster_addr":"172.21.15.31:6813/202520744","heartbeat_back_addr":"172.21.15.31:6817/202520744","heartbeat_front_addr":"172.21.15.31:6815/202520744","state":["exists","up"]},{"osd":5,"uuid":"235389de-7777-4744-81b8-d0bfc11190e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":1472711057},{"type":"v1","addr":"172.21.15.31:6801","nonce":1472711057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":1472711057},{"type":"v1","addr":"172.21.15.31:6803","nonce":1472711057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":1472711057},{"type":"v1","addr":"172.21.15.31:6807","nonce":1472711057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":1472711057},{"type":"v1","addr":"172.21.15.31:6805","nonce":1472711057}]},"public_addr":"172.21.15.31:6801/1472711057","cluster_addr":"172.21.15.31:6803/1472711057","heartbeat_back_addr":"172.21.15.31:6807/1472711057","heartbeat_front_addr":"172.21.15.31:6805/1472711057","state":["exists","up"]},{"osd":6,"uuid":"cea70a57-3715-4ebb-8511-a528a9df2b1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":3754251089},{"type":"v1","addr":"172.21.15.31:6827","nonce":3754251089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":3754251089},{"type":"v1","addr":"172.21.15.31:6829","nonce":3754251089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":3754251089},{"type":"v1","addr":"172.21.15.31:6833","nonce":3754251089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":3754251089},{"type":"v1","addr":"172.21.15.31:6831","nonce":3754251089}]},"public_addr":"172.21.15.31:6827/3754251089","cluster_addr":"172.21.15.31:6829/3754251089","heartbeat_back_addr":"172.21.15.31:6833/3754251089","heartbeat_front_addr":"172.21.15.31:6831/3754251089","state":["exists","up"]},{"osd":7,"uuid":"01cc111b-a45f-4ca3-9cc1-7fe0afbeba45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":3948648934},{"type":"v1","addr":"172.21.15.31:6819","nonce":3948648934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":3948648934},{"type":"v1","addr":"172.21.15.31:6821","nonce":3948648934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":3948648934},{"type":"v1","addr":"172.21.15.31:6825","nonce":3948648934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":3948648934},{"type":"v1","addr":"172.21.15.31:6823","nonce":3948648934}]},"public_addr":"172.21.15.31:6819/3948648934","cluster_addr":"172.21.15.31:6821/3948648934","heartbeat_back_addr":"172.21.15.31:6825/3948648934","heartbeat_front_addr":"172.21.15.31:6823/3948648934","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:24:32.606989+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:24:32.891535+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:24:33.330793+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:24:32.811533+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:24:33.894624+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:24:32.773817+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:24:32.901865+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:24:32.907207+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:24:41.670 INFO:teuthology.misc.health.smithi017.stderr:2024-09-10T21:24:41.666+0000 7fdd94600640 1 -- 172.21.15.17:0/3976388431 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fdd6c0725e0 msgr2=0x7fdd6c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:41.670 INFO:teuthology.misc.health.smithi017.stderr:2024-09-10T21:24:41.666+0000 7fdd94600640 1 --2- 172.21.15.17:0/3976388431 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fdd6c0725e0 0x7fdd6c074aa0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fdd780058d0 tx=0x7fdd78005860 comp rx=0 tx=0).stop 2024-09-10T21:24:41.670 INFO:teuthology.misc.health.smithi017.stderr:2024-09-10T21:24:41.666+0000 7fdd94600640 1 -- 172.21.15.17:0/3976388431 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdd8c0c3290 msgr2=0x7fdd8c1eea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:41.670 INFO:teuthology.misc.health.smithi017.stderr:2024-09-10T21:24:41.666+0000 7fdd94600640 1 --2- 172.21.15.17:0/3976388431 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdd8c0c3290 0x7fdd8c1eea50 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fdd800b0040 tx=0x7fdd800042f0 comp rx=0 tx=0).stop 2024-09-10T21:24:41.670 INFO:teuthology.misc.health.smithi017.stderr:2024-09-10T21:24:41.666+0000 7fdd94600640 1 -- 172.21.15.17:0/3976388431 shutdown_connections 2024-09-10T21:24:41.670 INFO:teuthology.misc.health.smithi017.stderr:2024-09-10T21:24:41.666+0000 7fdd94600640 1 --2- 172.21.15.17:0/3976388431 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fdd6c0725e0 0x7fdd6c074aa0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:41.670 INFO:teuthology.misc.health.smithi017.stderr:2024-09-10T21:24:41.666+0000 7fdd94600640 1 --2- 172.21.15.17:0/3976388431 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd8c0ce1e0 0x7fdd8c14ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:41.670 INFO:teuthology.misc.health.smithi017.stderr:2024-09-10T21:24:41.666+0000 7fdd94600640 1 --2- 172.21.15.17:0/3976388431 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fdd8c0c5240 0x7fdd8c1eef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:41.670 INFO:teuthology.misc.health.smithi017.stderr:2024-09-10T21:24:41.666+0000 7fdd94600640 1 --2- 172.21.15.17:0/3976388431 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdd8c0c3290 0x7fdd8c1eea50 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:41.670 INFO:teuthology.misc.health.smithi017.stderr:2024-09-10T21:24:41.666+0000 7fdd94600640 1 -- 172.21.15.17:0/3976388431 >> 172.21.15.17:0/3976388431 conn(0x7fdd8c074d90 msgr2=0x7fdd8c0ccb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:41.670 INFO:teuthology.misc.health.smithi017.stderr:2024-09-10T21:24:41.666+0000 7fdd94600640 1 -- 172.21.15.17:0/3976388431 shutdown_connections 2024-09-10T21:24:41.671 INFO:teuthology.misc.health.smithi017.stderr:2024-09-10T21:24:41.666+0000 7fdd94600640 1 -- 172.21.15.17:0/3976388431 wait complete. 2024-09-10T21:24:41.687 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-09-10T21:24:41.687 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-09-10T21:24:41.687 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-10T21:24:41.687 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-10T21:24:41.690 INFO:tasks.ceph.mds.a:Started 2024-09-10T21:24:41.691 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-10T21:24:41.691 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-10T21:24:41.693 INFO:tasks.ceph.mds.c:Started 2024-09-10T21:24:41.694 INFO:tasks.ceph.mds.e:Restarting daemon 2024-09-10T21:24:41.694 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i e 2024-09-10T21:24:41.697 INFO:tasks.ceph.mds.e:Started 2024-09-10T21:24:41.697 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-10T21:24:41.697 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-10T21:24:41.701 INFO:tasks.ceph.mds.b:Started 2024-09-10T21:24:41.701 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-10T21:24:41.701 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-10T21:24:41.704 INFO:tasks.ceph.mds.d:Started 2024-09-10T21:24:41.704 INFO:tasks.ceph.mds.f:Restarting daemon 2024-09-10T21:24:41.704 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i f 2024-09-10T21:24:41.707 INFO:tasks.ceph.mds.f:Started 2024-09-10T21:24:41.707 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-09-10T21:24:41.708 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:24:41.797 INFO:tasks.ceph.mds.a.smithi017.stdout:starting mds.a at 2024-09-10T21:24:41.807 INFO:tasks.ceph.mds.b.smithi031.stdout:starting mds.b at 2024-09-10T21:24:41.815 INFO:tasks.ceph.mds.c.smithi017.stdout:starting mds.c at 2024-09-10T21:24:41.817 INFO:tasks.ceph.mds.e.smithi017.stdout:starting mds.e at 2024-09-10T21:24:41.819 INFO:tasks.ceph.mds.f.smithi031.stdout:starting mds.f at 2024-09-10T21:24:41.820 INFO:tasks.ceph.mds.d.smithi031.stdout:starting mds.d at 2024-09-10T21:24:41.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:41.858+0000 7ff57c200640 1 -- 172.21.15.17:0/3033376879 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff5741546c0 msgr2=0x7ff5740ca430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:41.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:41.858+0000 7ff57c200640 1 --2- 172.21.15.17:0/3033376879 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff5741546c0 0x7ff5740ca430 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff57406f980 tx=0x7ff56801c890 comp rx=0 tx=0).stop 2024-09-10T21:24:41.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:41.858+0000 7ff57c200640 1 -- 172.21.15.17:0/3033376879 shutdown_connections 2024-09-10T21:24:41.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:41.858+0000 7ff57c200640 1 --2- 172.21.15.17:0/3033376879 >> v2:172.21.15.31:3300/0 conn(0x7ff5740c89d0 0x7ff5740cdf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:41.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:41.858+0000 7ff57c200640 1 --2- 172.21.15.17:0/3033376879 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff5741546c0 0x7ff5740ca430 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:41.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:41.858+0000 7ff57c200640 1 -- 172.21.15.17:0/3033376879 >> 172.21.15.17:0/3033376879 conn(0x7ff5740702f0 msgr2=0x7ff574070700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:41.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:41.858+0000 7ff57c200640 1 -- 172.21.15.17:0/3033376879 shutdown_connections 2024-09-10T21:24:41.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:41.858+0000 7ff57c200640 1 -- 172.21.15.17:0/3033376879 wait complete. 2024-09-10T21:24:41.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:41.862+0000 7ff57c200640 1 Processor -- start 2024-09-10T21:24:41.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:41.862+0000 7ff57c200640 1 -- start start 2024-09-10T21:24:41.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:41.862+0000 7ff57c200640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff5740c89d0 0x7ff5741511d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:41.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:41.862+0000 7ff57c200640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff5741546c0 0x7ff5741f6820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:41.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:41.862+0000 7ff57c200640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff574154aa0 0x7ff574068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:41.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:41.862+0000 7ff57c200640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7ff5740c9640 con 0x7ff5741546c0 2024-09-10T21:24:41.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:41.862+0000 7ff57c200640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7ff574072790 con 0x7ff5740c89d0 2024-09-10T21:24:41.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:41.862+0000 7ff57c200640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7ff5740c97c0 con 0x7ff574154aa0 2024-09-10T21:24:41.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:41.862+0000 7ff578e00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff5741546c0 0x7ff5741f6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:41.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:41.862+0000 7ff578e00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff5741546c0 0x7ff5741f6820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44560/0 (socket says 172.21.15.17:44560) 2024-09-10T21:24:41.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:41.862+0000 7ff578e00640 1 -- 172.21.15.17:0/3043595962 learned_addr learned my addr 172.21.15.17:0/3043595962 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:24:41.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:41.862+0000 7ff579800640 1 --2- 172.21.15.17:0/3043595962 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff5740c89d0 0x7ff5741511d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:41.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:41.862+0000 7ff57a200640 1 --2- 172.21.15.17:0/3043595962 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff574154aa0 0x7ff574068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:41.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:41.862+0000 7ff578e00640 1 -- 172.21.15.17:0/3043595962 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff574154aa0 msgr2=0x7ff574068bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:41.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:41.862+0000 7ff578e00640 1 --2- 172.21.15.17:0/3043595962 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff574154aa0 0x7ff574068bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:41.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:41.862+0000 7ff578e00640 1 -- 172.21.15.17:0/3043595962 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff5740c89d0 msgr2=0x7ff5741511d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:41.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:41.862+0000 7ff578e00640 1 --2- 172.21.15.17:0/3043595962 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff5740c89d0 0x7ff5741511d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:41.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:41.862+0000 7ff578e00640 1 -- 172.21.15.17:0/3043595962 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5740b3fc0 con 0x7ff5741546c0 2024-09-10T21:24:41.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:41.862+0000 7ff579800640 1 --2- 172.21.15.17:0/3043595962 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff5740c89d0 0x7ff5741511d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:24:41.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:41.862+0000 7ff578e00640 1 --2- 172.21.15.17:0/3043595962 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff5741546c0 0x7ff5741f6820 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff56000c3c0 tx=0x7ff56000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:41.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:41.862+0000 7ff571600640 1 -- 172.21.15.17:0/3043595962 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff560017070 con 0x7ff5741546c0 2024-09-10T21:24:41.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:41.862+0000 7ff571600640 1 -- 172.21.15.17:0/3043595962 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff5600053c0 con 0x7ff5741546c0 2024-09-10T21:24:41.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:41.862+0000 7ff57c200640 1 -- 172.21.15.17:0/3043595962 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5740c8390 con 0x7ff5741546c0 2024-09-10T21:24:41.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:41.862+0000 7ff571600640 1 -- 172.21.15.17:0/3043595962 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff560003860 con 0x7ff5741546c0 2024-09-10T21:24:41.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:41.862+0000 7ff57c200640 1 -- 172.21.15.17:0/3043595962 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5741500d0 con 0x7ff5741546c0 2024-09-10T21:24:41.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:41.866+0000 7ff57c200640 1 -- 172.21.15.17:0/3043595962 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5741508a0 con 0x7ff5741546c0 2024-09-10T21:24:41.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:41.866+0000 7ff571600640 1 -- 172.21.15.17:0/3043595962 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7ff560003a00 con 0x7ff5741546c0 2024-09-10T21:24:41.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:41.866+0000 7ff571600640 1 --2- 172.21.15.17:0/3043595962 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7ff5540725e0 0x7ff554074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:41.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:41.866+0000 7ff571600640 1 -- 172.21.15.17:0/3043595962 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7ff5600924f0 con 0x7ff5741546c0 2024-09-10T21:24:41.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:41.866+0000 7ff579800640 1 --2- 172.21.15.17:0/3043595962 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7ff5540725e0 0x7ff554074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:41.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:41.870+0000 7ff579800640 1 --2- 172.21.15.17:0/3043595962 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7ff5540725e0 0x7ff554074aa0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7ff5680af000 tx=0x7ff56801b0d0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:41.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:41.874+0000 7ff571600640 1 -- 172.21.15.17:0/3043595962 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7ff56005d8c0 con 0x7ff5741546c0 2024-09-10T21:24:42.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:42.082+0000 7ff57c200640 1 -- 172.21.15.17:0/3043595962 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff574158e70 con 0x7ff5741546c0 2024-09-10T21:24:42.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:42.082+0000 7ff571600640 1 -- 172.21.15.17:0/3043595962 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1434 (secure 0 0 0) 0x7ff5600622f0 con 0x7ff5741546c0 2024-09-10T21:24:42.086 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-10T21:24:42.086 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":2,"btime":"2024-09-10T21:24:41:805160+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4384,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/3592734043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":3592734043},{"type":"v1","addr":"172.21.15.17:6833","nonce":3592734043}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-10T21:24:42.086 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 2 2024-09-10T21:24:42.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:42.082+0000 7ff57c200640 1 -- 172.21.15.17:0/3043595962 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7ff5540725e0 msgr2=0x7ff554074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:42.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:42.082+0000 7ff57c200640 1 --2- 172.21.15.17:0/3043595962 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7ff5540725e0 0x7ff554074aa0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7ff5680af000 tx=0x7ff56801b0d0 comp rx=0 tx=0).stop 2024-09-10T21:24:42.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:42.082+0000 7ff57c200640 1 -- 172.21.15.17:0/3043595962 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff5741546c0 msgr2=0x7ff5741f6820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:42.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:42.082+0000 7ff57c200640 1 --2- 172.21.15.17:0/3043595962 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff5741546c0 0x7ff5741f6820 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff56000c3c0 tx=0x7ff56000c880 comp rx=0 tx=0).stop 2024-09-10T21:24:42.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:42.082+0000 7ff57c200640 1 -- 172.21.15.17:0/3043595962 shutdown_connections 2024-09-10T21:24:42.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:42.082+0000 7ff57c200640 1 --2- 172.21.15.17:0/3043595962 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7ff5540725e0 0x7ff554074aa0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:42.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:42.082+0000 7ff57c200640 1 --2- 172.21.15.17:0/3043595962 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff574154aa0 0x7ff574068bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:42.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:42.082+0000 7ff57c200640 1 --2- 172.21.15.17:0/3043595962 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff5741546c0 0x7ff5741f6820 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:42.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:42.082+0000 7ff57c200640 1 --2- 172.21.15.17:0/3043595962 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff5740c89d0 0x7ff5741511d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:42.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:42.082+0000 7ff57c200640 1 -- 172.21.15.17:0/3043595962 >> 172.21.15.17:0/3043595962 conn(0x7ff5740702f0 msgr2=0x7ff57414d940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:42.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:42.082+0000 7ff57c200640 1 -- 172.21.15.17:0/3043595962 shutdown_connections 2024-09-10T21:24:42.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:42.082+0000 7ff57c200640 1 -- 172.21.15.17:0/3043595962 wait complete. 2024-09-10T21:24:44.107 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:24:44.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:44.262+0000 7f61b0000640 1 -- 172.21.15.17:0/3585049514 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f61a80c8470 msgr2=0x7f61a8073d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:44.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:44.262+0000 7f61b0000640 1 --2- 172.21.15.17:0/3585049514 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f61a80c8470 0x7f61a8073d40 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6194009900 tx=0x7f619401c8b0 comp rx=0 tx=0).stop 2024-09-10T21:24:44.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:44.262+0000 7f61b0000640 1 -- 172.21.15.17:0/3585049514 shutdown_connections 2024-09-10T21:24:44.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:44.262+0000 7f61b0000640 1 --2- 172.21.15.17:0/3585049514 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f61a80c8470 0x7f61a8073d40 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:44.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:44.262+0000 7f61b0000640 1 --2- 172.21.15.17:0/3585049514 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f61a8150fc0 0x7f61a8157500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:44.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:44.262+0000 7f61b0000640 1 -- 172.21.15.17:0/3585049514 >> 172.21.15.17:0/3585049514 conn(0x7f61a8070160 msgr2=0x7f61a8070570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:44.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:44.262+0000 7f61b0000640 1 -- 172.21.15.17:0/3585049514 shutdown_connections 2024-09-10T21:24:44.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:44.262+0000 7f61b0000640 1 -- 172.21.15.17:0/3585049514 wait complete. 2024-09-10T21:24:44.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:44.262+0000 7f61b0000640 1 Processor -- start 2024-09-10T21:24:44.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:44.262+0000 7f61b0000640 1 -- start start 2024-09-10T21:24:44.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:44.266+0000 7f61b0000640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f61a8074280 0x7f61a80caf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:44.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:44.266+0000 7f61b0000640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f61a80c8470 0x7f61a80ca050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:44.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:44.266+0000 7f61b0000640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f61a8150fc0 0x7f61a80ca590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:44.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:44.266+0000 7f61b0000640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f61a80c8050 con 0x7f61a8074280 2024-09-10T21:24:44.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:44.266+0000 7f61b0000640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f61a80c7ed0 con 0x7f61a80c8470 2024-09-10T21:24:44.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:44.266+0000 7f61b0000640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f61a8161380 con 0x7f61a8150fc0 2024-09-10T21:24:44.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:44.266+0000 7f61ad600640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f61a8074280 0x7f61a80caf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:44.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:44.266+0000 7f61ad600640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f61a8074280 0x7f61a80caf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44570/0 (socket says 172.21.15.17:44570) 2024-09-10T21:24:44.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:44.266+0000 7f61ad600640 1 -- 172.21.15.17:0/3960606935 learned_addr learned my addr 172.21.15.17:0/3960606935 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:24:44.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:44.266+0000 7f61ae000640 1 --2- 172.21.15.17:0/3960606935 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f61a8150fc0 0x7f61a80ca590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:44.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:44.266+0000 7f61acc00640 1 --2- 172.21.15.17:0/3960606935 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f61a80c8470 0x7f61a80ca050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:44.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:44.266+0000 7f61ad600640 1 -- 172.21.15.17:0/3960606935 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f61a8150fc0 msgr2=0x7f61a80ca590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:44.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:44.266+0000 7f61ad600640 1 --2- 172.21.15.17:0/3960606935 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f61a8150fc0 0x7f61a80ca590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:44.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:44.266+0000 7f61ad600640 1 -- 172.21.15.17:0/3960606935 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f61a80c8470 msgr2=0x7f61a80ca050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:44.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:44.266+0000 7f61ad600640 1 --2- 172.21.15.17:0/3960606935 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f61a80c8470 0x7f61a80ca050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:44.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:44.266+0000 7f61ad600640 1 -- 172.21.15.17:0/3960606935 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61a814f7a0 con 0x7f61a8074280 2024-09-10T21:24:44.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:44.266+0000 7f61ae000640 1 --2- 172.21.15.17:0/3960606935 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f61a8150fc0 0x7f61a80ca590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:24:44.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:44.266+0000 7f61ad600640 1 --2- 172.21.15.17:0/3960606935 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f61a8074280 0x7f61a80caf70 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f619c00e9a0 tx=0x7f619c00ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:44.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:44.266+0000 7f61a5600640 1 -- 172.21.15.17:0/3960606935 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f619c00cd80 con 0x7f61a8074280 2024-09-10T21:24:44.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:44.266+0000 7f61a5600640 1 -- 172.21.15.17:0/3960606935 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f619c004020 con 0x7f61a8074280 2024-09-10T21:24:44.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:44.266+0000 7f61a5600640 1 -- 172.21.15.17:0/3960606935 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f619c004450 con 0x7f61a8074280 2024-09-10T21:24:44.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:44.266+0000 7f61b0000640 1 -- 172.21.15.17:0/3960606935 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61a814fa80 con 0x7f61a8074280 2024-09-10T21:24:44.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:44.266+0000 7f61b0000640 1 -- 172.21.15.17:0/3960606935 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61a80cb4b0 con 0x7f61a8074280 2024-09-10T21:24:44.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:44.266+0000 7f61b0000640 1 -- 172.21.15.17:0/3960606935 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61a8073d40 con 0x7f61a8074280 2024-09-10T21:24:44.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:44.270+0000 7f61a5600640 1 -- 172.21.15.17:0/3960606935 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f619c003600 con 0x7f61a8074280 2024-09-10T21:24:44.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:44.270+0000 7f61a5600640 1 --2- 172.21.15.17:0/3960606935 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f61880725e0 0x7f6188074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:44.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:44.270+0000 7f61a5600640 1 -- 172.21.15.17:0/3960606935 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7f619c093900 con 0x7f61a8074280 2024-09-10T21:24:44.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:44.270+0000 7f61acc00640 1 --2- 172.21.15.17:0/3960606935 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f61880725e0 0x7f6188074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:44.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:44.270+0000 7f61acc00640 1 --2- 172.21.15.17:0/3960606935 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f61880725e0 0x7f6188074aa0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f61940af000 tx=0x7f619401b0f0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:44.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:44.274+0000 7f61a5600640 1 -- 172.21.15.17:0/3960606935 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f619c05f320 con 0x7f61a8074280 2024-09-10T21:24:44.488 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:44.482+0000 7f61b0000640 1 -- 172.21.15.17:0/3960606935 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f61a8074dc0 con 0x7f61a8074280 2024-09-10T21:24:44.488 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:44.486+0000 7f61a5600640 1 -- 172.21.15.17:0/3960606935 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+5554 (secure 0 0 0) 0x7f619c05ec20 con 0x7f61a8074280 2024-09-10T21:24:44.489 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-10T21:24:44.489 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":3,"btime":"2024-09-10T21:24:42:806575+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4350,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/1643071167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":1643071167},{"type":"v1","addr":"172.21.15.31:6835","nonce":1643071167}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4356,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6837/1666225712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6836","nonce":1666225712},{"type":"v1","addr":"172.21.15.17:6837","nonce":1666225712}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4384,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/3592734043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":3592734043},{"type":"v1","addr":"172.21.15.17:6833","nonce":3592734043}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4405,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/1293665856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":1293665856},{"type":"v1","addr":"172.21.15.17:6835","nonce":1293665856}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4411,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6838/1944852172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":1944852172},{"type":"v1","addr":"172.21.15.31:6838","nonce":1944852172}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4414,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6839/3129062106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6837","nonce":3129062106},{"type":"v1","addr":"172.21.15.31:6839","nonce":3129062106}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-09-10T21:24:44.489 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 3 2024-09-10T21:24:44.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:44.486+0000 7f61b0000640 1 -- 172.21.15.17:0/3960606935 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f61880725e0 msgr2=0x7f6188074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:44.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:44.486+0000 7f61b0000640 1 --2- 172.21.15.17:0/3960606935 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f61880725e0 0x7f6188074aa0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f61940af000 tx=0x7f619401b0f0 comp rx=0 tx=0).stop 2024-09-10T21:24:44.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:44.486+0000 7f61b0000640 1 -- 172.21.15.17:0/3960606935 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f61a8074280 msgr2=0x7f61a80caf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:44.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:44.486+0000 7f61b0000640 1 --2- 172.21.15.17:0/3960606935 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f61a8074280 0x7f61a80caf70 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f619c00e9a0 tx=0x7f619c00ee60 comp rx=0 tx=0).stop 2024-09-10T21:24:44.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:44.486+0000 7f61b0000640 1 -- 172.21.15.17:0/3960606935 shutdown_connections 2024-09-10T21:24:44.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:44.486+0000 7f61b0000640 1 --2- 172.21.15.17:0/3960606935 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f61880725e0 0x7f6188074aa0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:44.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:44.486+0000 7f61b0000640 1 --2- 172.21.15.17:0/3960606935 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f61a8150fc0 0x7f61a80ca590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:44.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:44.486+0000 7f61b0000640 1 --2- 172.21.15.17:0/3960606935 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f61a80c8470 0x7f61a80ca050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:44.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:44.486+0000 7f61b0000640 1 --2- 172.21.15.17:0/3960606935 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f61a8074280 0x7f61a80caf70 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:44.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:44.486+0000 7f61b0000640 1 -- 172.21.15.17:0/3960606935 >> 172.21.15.17:0/3960606935 conn(0x7f61a8070160 msgr2=0x7f61a8153770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:44.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:44.486+0000 7f61b0000640 1 -- 172.21.15.17:0/3960606935 shutdown_connections 2024-09-10T21:24:44.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:44.486+0000 7f61b0000640 1 -- 172.21.15.17:0/3960606935 wait complete. 2024-09-10T21:24:44.510 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-10T21:24:44.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:44.650+0000 7f8093400640 1 -- 172.21.15.17:0/4243121624 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f808c154aa0 msgr2=0x7f808c0ca430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:44.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:44.650+0000 7f8093400640 1 --2- 172.21.15.17:0/4243121624 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f808c154aa0 0x7f808c0ca430 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f8074009840 tx=0x7f807401c7f0 comp rx=0 tx=0).stop 2024-09-10T21:24:44.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:44.650+0000 7f8093400640 1 -- 172.21.15.17:0/4243121624 shutdown_connections 2024-09-10T21:24:44.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:44.650+0000 7f8093400640 1 --2- 172.21.15.17:0/4243121624 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f808c154aa0 0x7f808c0ca430 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:44.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:44.650+0000 7f8093400640 1 -- 172.21.15.17:0/4243121624 >> 172.21.15.17:0/4243121624 conn(0x7f808c0702f0 msgr2=0x7f808c070700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:44.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:44.650+0000 7f8093400640 1 -- 172.21.15.17:0/4243121624 shutdown_connections 2024-09-10T21:24:44.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:44.650+0000 7f8093400640 1 -- 172.21.15.17:0/4243121624 wait complete. 2024-09-10T21:24:44.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:44.654+0000 7f8093400640 1 Processor -- start 2024-09-10T21:24:44.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:44.654+0000 7f8093400640 1 -- start start 2024-09-10T21:24:44.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:44.654+0000 7f8093400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f808c0c89d0 0x7f808c1572d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:44.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:44.654+0000 7f8093400640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f808c1546c0 0x7f808c152c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:44.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:44.654+0000 7f8093400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f808c154aa0 0x7f808c14c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:44.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:44.654+0000 7f8093400640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f808c0ce630 con 0x7f808c0c89d0 2024-09-10T21:24:44.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:44.654+0000 7f8093400640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f808c072790 con 0x7f808c154aa0 2024-09-10T21:24:44.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:44.654+0000 7f8093400640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f808c0ce7b0 con 0x7f808c1546c0 2024-09-10T21:24:44.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:44.654+0000 7f8090a00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f808c0c89d0 0x7f808c1572d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:44.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:44.654+0000 7f8090a00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f808c0c89d0 0x7f808c1572d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44572/0 (socket says 172.21.15.17:44572) 2024-09-10T21:24:44.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:44.654+0000 7f8090a00640 1 -- 172.21.15.17:0/3228900769 learned_addr learned my addr 172.21.15.17:0/3228900769 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:24:44.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:44.654+0000 7f8083e00640 1 --2- 172.21.15.17:0/3228900769 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f808c1546c0 0x7f808c152c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:44.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:44.654+0000 7f8091400640 1 --2- 172.21.15.17:0/3228900769 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f808c154aa0 0x7f808c14c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:44.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:44.654+0000 7f8090a00640 1 -- 172.21.15.17:0/3228900769 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f808c1546c0 msgr2=0x7f808c152c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:44.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:44.654+0000 7f8090a00640 1 --2- 172.21.15.17:0/3228900769 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f808c1546c0 0x7f808c152c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:44.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:44.654+0000 7f8090a00640 1 -- 172.21.15.17:0/3228900769 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f808c154aa0 msgr2=0x7f808c14c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:44.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:44.654+0000 7f8090a00640 1 --2- 172.21.15.17:0/3228900769 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f808c154aa0 0x7f808c14c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:44.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:44.654+0000 7f8090a00640 1 -- 172.21.15.17:0/3228900769 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f808c0c7dd0 con 0x7f808c0c89d0 2024-09-10T21:24:44.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:44.654+0000 7f8090a00640 1 --2- 172.21.15.17:0/3228900769 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f808c0c89d0 0x7f808c1572d0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f807c00c500 tx=0x7f807c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:44.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:44.654+0000 7f8089600640 1 -- 172.21.15.17:0/3228900769 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f807c017070 con 0x7f808c0c89d0 2024-09-10T21:24:44.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:44.654+0000 7f8089600640 1 -- 172.21.15.17:0/3228900769 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f807c003600 con 0x7f808c0c89d0 2024-09-10T21:24:44.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:44.654+0000 7f8089600640 1 -- 172.21.15.17:0/3228900769 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f807c003a50 con 0x7f808c0c89d0 2024-09-10T21:24:44.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:44.654+0000 7f8093400640 1 -- 172.21.15.17:0/3228900769 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f808c0ccc40 con 0x7f808c0c89d0 2024-09-10T21:24:44.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:44.654+0000 7f8093400640 1 -- 172.21.15.17:0/3228900769 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f808c0c8350 con 0x7f808c0c89d0 2024-09-10T21:24:44.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:44.658+0000 7f8093400640 1 -- 172.21.15.17:0/3228900769 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f808c0c9740 con 0x7f808c0c89d0 2024-09-10T21:24:44.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:44.658+0000 7f8089600640 1 -- 172.21.15.17:0/3228900769 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f807c003bf0 con 0x7f808c0c89d0 2024-09-10T21:24:44.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:44.658+0000 7f8089600640 1 --2- 172.21.15.17:0/3228900769 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f80680725e0 0x7f8068074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:44.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:44.658+0000 7f8089600640 1 -- 172.21.15.17:0/3228900769 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7f807c0926c0 con 0x7f808c0c89d0 2024-09-10T21:24:44.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:44.658+0000 7f8083e00640 1 --2- 172.21.15.17:0/3228900769 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f80680725e0 0x7f8068074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:44.664 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:44.662+0000 7f8083e00640 1 --2- 172.21.15.17:0/3228900769 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f80680725e0 0x7f8068074aa0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f80740af000 tx=0x7f807401b030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:44.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:44.662+0000 7f8089600640 1 -- 172.21.15.17:0/3228900769 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f807c05e1a0 con 0x7f808c0c89d0 2024-09-10T21:24:44.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:44.842+0000 7f8093400640 1 -- 172.21.15.17:0/3228900769 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f808c14ff30 con 0x7f808c0c89d0 2024-09-10T21:24:44.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:44.842+0000 7f8089600640 1 -- 172.21.15.17:0/3228900769 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v17) ==== 84+0+65 (secure 0 0 0) 0x7f807c05daa0 con 0x7f808c0c89d0 2024-09-10T21:24:44.845 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-10T21:24:44.845 INFO:teuthology.orchestra.run.smithi017.stdout:[ 2024-09-10T21:24:44.846 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:24:44.846 INFO:teuthology.orchestra.run.smithi017.stdout: "poolnum": 1, 2024-09-10T21:24:44.846 INFO:teuthology.orchestra.run.smithi017.stdout: "poolname": ".mgr" 2024-09-10T21:24:44.846 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:24:44.846 INFO:teuthology.orchestra.run.smithi017.stdout:] 2024-09-10T21:24:44.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:44.842+0000 7f8093400640 1 -- 172.21.15.17:0/3228900769 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f80680725e0 msgr2=0x7f8068074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:44.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:44.842+0000 7f8093400640 1 --2- 172.21.15.17:0/3228900769 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f80680725e0 0x7f8068074aa0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f80740af000 tx=0x7f807401b030 comp rx=0 tx=0).stop 2024-09-10T21:24:44.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:44.842+0000 7f8093400640 1 -- 172.21.15.17:0/3228900769 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f808c0c89d0 msgr2=0x7f808c1572d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:44.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:44.842+0000 7f8093400640 1 --2- 172.21.15.17:0/3228900769 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f808c0c89d0 0x7f808c1572d0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f807c00c500 tx=0x7f807c00c9c0 comp rx=0 tx=0).stop 2024-09-10T21:24:44.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:44.842+0000 7f8093400640 1 -- 172.21.15.17:0/3228900769 shutdown_connections 2024-09-10T21:24:44.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:44.842+0000 7f8093400640 1 --2- 172.21.15.17:0/3228900769 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f80680725e0 0x7f8068074aa0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:44.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:44.842+0000 7f8093400640 1 --2- 172.21.15.17:0/3228900769 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f808c154aa0 0x7f808c14c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:44.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:44.842+0000 7f8093400640 1 --2- 172.21.15.17:0/3228900769 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f808c1546c0 0x7f808c152c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:44.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:44.842+0000 7f8093400640 1 --2- 172.21.15.17:0/3228900769 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f808c0c89d0 0x7f808c1572d0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:44.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:44.842+0000 7f8093400640 1 -- 172.21.15.17:0/3228900769 >> 172.21.15.17:0/3228900769 conn(0x7f808c0702f0 msgr2=0x7f808c0706d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:44.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:44.842+0000 7f8093400640 1 -- 172.21.15.17:0/3228900769 shutdown_connections 2024-09-10T21:24:44.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:44.842+0000 7f8093400640 1 -- 172.21.15.17:0/3228900769 wait complete. 2024-09-10T21:24:44.866 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-10T21:24:44.866 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-10T21:24:45.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:45.014+0000 7f2ef7200640 1 -- 172.21.15.17:0/917773303 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2ef00ce620 msgr2=0x7f2ef00c98f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:45.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:45.014+0000 7f2ef7200640 1 --2- 172.21.15.17:0/917773303 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2ef00ce620 0x7f2ef00c98f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f2ee0009900 tx=0x7f2ee001f0d0 comp rx=0 tx=0).stop 2024-09-10T21:24:45.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:45.014+0000 7f2ef7200640 1 -- 172.21.15.17:0/917773303 shutdown_connections 2024-09-10T21:24:45.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:45.014+0000 7f2ef7200640 1 --2- 172.21.15.17:0/917773303 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f2ef01590c0 0x7f2ef015c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:45.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:45.014+0000 7f2ef7200640 1 --2- 172.21.15.17:0/917773303 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2ef00ce620 0x7f2ef00c98f0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:45.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:45.014+0000 7f2ef7200640 1 -- 172.21.15.17:0/917773303 >> 172.21.15.17:0/917773303 conn(0x7f2ef0068b60 msgr2=0x7f2ef0068f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:45.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:45.014+0000 7f2ef7200640 1 -- 172.21.15.17:0/917773303 shutdown_connections 2024-09-10T21:24:45.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:45.014+0000 7f2ef7200640 1 -- 172.21.15.17:0/917773303 wait complete. 2024-09-10T21:24:45.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:45.014+0000 7f2ef7200640 1 Processor -- start 2024-09-10T21:24:45.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:45.014+0000 7f2ef7200640 1 -- start start 2024-09-10T21:24:45.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:45.014+0000 7f2ef7200640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f2ef00ce620 0x7f2ef00c47c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:45.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:45.014+0000 7f2ef7200640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ef00cea00 0x7f2ef014c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:45.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:45.014+0000 7f2ef7200640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2ef01590c0 0x7f2ef00cd0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:45.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:45.014+0000 7f2ef7200640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f2ef00c5a00 con 0x7f2ef01590c0 2024-09-10T21:24:45.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:45.014+0000 7f2ef7200640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f2ef0159580 con 0x7f2ef00cea00 2024-09-10T21:24:45.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:45.014+0000 7f2ef7200640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f2ef00c5b80 con 0x7f2ef00ce620 2024-09-10T21:24:45.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:45.014+0000 7f2ef5200640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2ef01590c0 0x7f2ef00cd0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:45.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:45.014+0000 7f2ef5200640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2ef01590c0 0x7f2ef00cd0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:59002/0 (socket says 172.21.15.17:59002) 2024-09-10T21:24:45.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:45.014+0000 7f2ef5200640 1 -- 172.21.15.17:0/3777985600 learned_addr learned my addr 172.21.15.17:0/3777985600 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:24:45.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:45.014+0000 7f2eefe00640 1 --2- 172.21.15.17:0/3777985600 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f2ef00ce620 0x7f2ef00c47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:45.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:45.014+0000 7f2ee7e00640 1 --2- 172.21.15.17:0/3777985600 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ef00cea00 0x7f2ef014c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:45.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:45.018+0000 7f2ef5200640 1 -- 172.21.15.17:0/3777985600 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f2ef00ce620 msgr2=0x7f2ef00c47c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:45.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:45.018+0000 7f2ef5200640 1 --2- 172.21.15.17:0/3777985600 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f2ef00ce620 0x7f2ef00c47c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:45.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:45.018+0000 7f2ef5200640 1 -- 172.21.15.17:0/3777985600 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ef00cea00 msgr2=0x7f2ef014c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:45.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:45.018+0000 7f2ef5200640 1 --2- 172.21.15.17:0/3777985600 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ef00cea00 0x7f2ef014c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:45.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:45.018+0000 7f2ef5200640 1 -- 172.21.15.17:0/3777985600 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ef00c2a60 con 0x7f2ef01590c0 2024-09-10T21:24:45.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:45.018+0000 7f2ef5200640 1 --2- 172.21.15.17:0/3777985600 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2ef01590c0 0x7f2ef00cd0f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f2ee800eb20 tx=0x7f2ee800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:45.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:45.018+0000 7f2eecc00640 1 -- 172.21.15.17:0/3777985600 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ee8004b30 con 0x7f2ef01590c0 2024-09-10T21:24:45.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:45.018+0000 7f2eecc00640 1 -- 172.21.15.17:0/3777985600 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2ee8004cf0 con 0x7f2ef01590c0 2024-09-10T21:24:45.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:45.018+0000 7f2eecc00640 1 -- 172.21.15.17:0/3777985600 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ee8005280 con 0x7f2ef01590c0 2024-09-10T21:24:45.023 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:45.018+0000 7f2ef7200640 1 -- 172.21.15.17:0/3777985600 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ef00c2d40 con 0x7f2ef01590c0 2024-09-10T21:24:45.023 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:45.018+0000 7f2ef7200640 1 -- 172.21.15.17:0/3777985600 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ef00c3300 con 0x7f2ef01590c0 2024-09-10T21:24:45.023 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:45.018+0000 7f2ef7200640 1 -- 172.21.15.17:0/3777985600 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ef0070a90 con 0x7f2ef01590c0 2024-09-10T21:24:45.023 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:45.018+0000 7f2eecc00640 1 -- 172.21.15.17:0/3777985600 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f2ee8005420 con 0x7f2ef01590c0 2024-09-10T21:24:45.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:45.018+0000 7f2eecc00640 1 --2- 172.21.15.17:0/3777985600 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f2ecc0725e0 0x7f2ecc074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:45.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:45.018+0000 7f2eecc00640 1 -- 172.21.15.17:0/3777985600 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 4857+0+0 (secure 0 0 0) 0x7f2ee8093bf0 con 0x7f2ef01590c0 2024-09-10T21:24:45.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:45.022+0000 7f2eefe00640 1 --2- 172.21.15.17:0/3777985600 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f2ecc0725e0 0x7f2ecc074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:45.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:45.022+0000 7f2eefe00640 1 --2- 172.21.15.17:0/3777985600 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f2ecc0725e0 0x7f2ecc074aa0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f2ee00b2000 tx=0x7f2ee001d910 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:45.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:45.026+0000 7f2eecc00640 1 -- 172.21.15.17:0/3777985600 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f2ee805f580 con 0x7f2ef01590c0 2024-09-10T21:24:45.156 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:24:45.160 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:24:45.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:45.198+0000 7f2ef7200640 1 -- 172.21.15.17:0/3777985600 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f2ef01583f0 con 0x7f2ef01590c0 2024-09-10T21:24:45.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:45.822+0000 7f2eecc00640 1 -- 172.21.15.17:0/3777985600 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v18) ==== 138+0+0 (secure 0 0 0) 0x7f2ee805ee80 con 0x7f2ef01590c0 2024-09-10T21:24:45.826 INFO:teuthology.orchestra.run.smithi017.stderr:pool 'cephfs_metadata' created 2024-09-10T21:24:45.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:45.826+0000 7f2ef7200640 1 -- 172.21.15.17:0/3777985600 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f2ecc0725e0 msgr2=0x7f2ecc074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:45.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:45.826+0000 7f2ef7200640 1 --2- 172.21.15.17:0/3777985600 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f2ecc0725e0 0x7f2ecc074aa0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f2ee00b2000 tx=0x7f2ee001d910 comp rx=0 tx=0).stop 2024-09-10T21:24:45.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:45.826+0000 7f2ef7200640 1 -- 172.21.15.17:0/3777985600 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2ef01590c0 msgr2=0x7f2ef00cd0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:45.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:45.826+0000 7f2ef7200640 1 --2- 172.21.15.17:0/3777985600 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2ef01590c0 0x7f2ef00cd0f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f2ee800eb20 tx=0x7f2ee800c2d0 comp rx=0 tx=0).stop 2024-09-10T21:24:45.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:45.826+0000 7f2ef7200640 1 -- 172.21.15.17:0/3777985600 shutdown_connections 2024-09-10T21:24:45.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:45.826+0000 7f2ef7200640 1 --2- 172.21.15.17:0/3777985600 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f2ecc0725e0 0x7f2ecc074aa0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:45.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:45.826+0000 7f2ef7200640 1 --2- 172.21.15.17:0/3777985600 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2ef01590c0 0x7f2ef00cd0f0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:45.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:45.826+0000 7f2ef7200640 1 --2- 172.21.15.17:0/3777985600 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ef00cea00 0x7f2ef014c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:45.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:45.826+0000 7f2ef7200640 1 --2- 172.21.15.17:0/3777985600 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f2ef00ce620 0x7f2ef00c47c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:45.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:45.826+0000 7f2ef7200640 1 -- 172.21.15.17:0/3777985600 >> 172.21.15.17:0/3777985600 conn(0x7f2ef0068b60 msgr2=0x7f2ef0072a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:45.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:45.826+0000 7f2ef7200640 1 -- 172.21.15.17:0/3777985600 shutdown_connections 2024-09-10T21:24:45.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:45.826+0000 7f2ef7200640 1 -- 172.21.15.17:0/3777985600 wait complete. 2024-09-10T21:24:45.858 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-09-10T21:24:46.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:46.010+0000 7fe133e00640 1 -- 172.21.15.17:0/2804694653 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe12c15ad60 msgr2=0x7fe12c06dc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:46.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:46.010+0000 7fe133e00640 1 --2- 172.21.15.17:0/2804694653 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe12c15ad60 0x7fe12c06dc30 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fe12c0c9f40 tx=0x7fe11801c8c0 comp rx=0 tx=0).stop 2024-09-10T21:24:46.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:46.010+0000 7fe133e00640 1 -- 172.21.15.17:0/2804694653 shutdown_connections 2024-09-10T21:24:46.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:46.010+0000 7fe133e00640 1 --2- 172.21.15.17:0/2804694653 >> v2:172.21.15.31:3300/0 conn(0x7fe12c0c8410 0x7fe12c06e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:46.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:46.010+0000 7fe133e00640 1 --2- 172.21.15.17:0/2804694653 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe12c15ad60 0x7fe12c06dc30 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:46.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:46.010+0000 7fe133e00640 1 --2- 172.21.15.17:0/2804694653 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe12c150dc0 0x7fe12c15a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:46.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:46.010+0000 7fe133e00640 1 -- 172.21.15.17:0/2804694653 >> 172.21.15.17:0/2804694653 conn(0x7fe12c070110 msgr2=0x7fe12c070520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:46.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:46.010+0000 7fe133e00640 1 -- 172.21.15.17:0/2804694653 shutdown_connections 2024-09-10T21:24:46.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:46.010+0000 7fe133e00640 1 -- 172.21.15.17:0/2804694653 wait complete. 2024-09-10T21:24:46.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:46.010+0000 7fe133e00640 1 Processor -- start 2024-09-10T21:24:46.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:46.010+0000 7fe133e00640 1 -- start start 2024-09-10T21:24:46.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:46.014+0000 7fe133e00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe12c0c8410 0x7fe12c141cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:46.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:46.014+0000 7fe133e00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe12c150dc0 0x7fe12c0c7560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:46.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:46.014+0000 7fe133e00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe12c15ad60 0x7fe12c0c7aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:46.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:46.014+0000 7fe133e00640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fe12c0751f0 con 0x7fe12c15ad60 2024-09-10T21:24:46.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:46.014+0000 7fe133e00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fe12c075070 con 0x7fe12c0c8410 2024-09-10T21:24:46.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:46.014+0000 7fe133e00640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7fe12c1511a0 con 0x7fe12c150dc0 2024-09-10T21:24:46.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:46.014+0000 7fe131e00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe12c15ad60 0x7fe12c0c7aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:46.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:46.014+0000 7fe131e00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe12c15ad60 0x7fe12c0c7aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:59008/0 (socket says 172.21.15.17:59008) 2024-09-10T21:24:46.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:46.014+0000 7fe131e00640 1 -- 172.21.15.17:0/3446007066 learned_addr learned my addr 172.21.15.17:0/3446007066 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:24:46.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:46.014+0000 7fe131400640 1 --2- 172.21.15.17:0/3446007066 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe12c0c8410 0x7fe12c141cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:46.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:46.014+0000 7fe130a00640 1 --2- 172.21.15.17:0/3446007066 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe12c150dc0 0x7fe12c0c7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:46.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:46.014+0000 7fe131e00640 1 -- 172.21.15.17:0/3446007066 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe12c150dc0 msgr2=0x7fe12c0c7560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:46.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:46.014+0000 7fe131e00640 1 --2- 172.21.15.17:0/3446007066 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe12c150dc0 0x7fe12c0c7560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:46.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:46.014+0000 7fe131e00640 1 -- 172.21.15.17:0/3446007066 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe12c0c8410 msgr2=0x7fe12c141cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:46.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:46.014+0000 7fe131e00640 1 --2- 172.21.15.17:0/3446007066 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe12c0c8410 0x7fe12c141cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:46.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:46.014+0000 7fe131e00640 1 -- 172.21.15.17:0/3446007066 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe12c1f4190 con 0x7fe12c15ad60 2024-09-10T21:24:46.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:46.014+0000 7fe131e00640 1 --2- 172.21.15.17:0/3446007066 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe12c15ad60 0x7fe12c0c7aa0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe12400e990 tx=0x7fe12400ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:46.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:46.014+0000 7fe129600640 1 -- 172.21.15.17:0/3446007066 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe124004d30 con 0x7fe12c15ad60 2024-09-10T21:24:46.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:46.014+0000 7fe129600640 1 -- 172.21.15.17:0/3446007066 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe12400caa0 con 0x7fe12c15ad60 2024-09-10T21:24:46.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:46.014+0000 7fe129600640 1 -- 172.21.15.17:0/3446007066 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe124005020 con 0x7fe12c15ad60 2024-09-10T21:24:46.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:46.014+0000 7fe133e00640 1 -- 172.21.15.17:0/3446007066 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe12c144220 con 0x7fe12c15ad60 2024-09-10T21:24:46.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:46.014+0000 7fe133e00640 1 -- 172.21.15.17:0/3446007066 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe12c161af0 con 0x7fe12c15ad60 2024-09-10T21:24:46.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:46.014+0000 7fe133e00640 1 -- 172.21.15.17:0/3446007066 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe12c0c10d0 con 0x7fe12c15ad60 2024-09-10T21:24:46.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:46.014+0000 7fe129600640 1 -- 172.21.15.17:0/3446007066 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fe12400f040 con 0x7fe12c15ad60 2024-09-10T21:24:46.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:46.018+0000 7fe129600640 1 --2- 172.21.15.17:0/3446007066 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fe10c0725e0 0x7fe10c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:46.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:46.018+0000 7fe129600640 1 -- 172.21.15.17:0/3446007066 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 5227+0+0 (secure 0 0 0) 0x7fe1240935a0 con 0x7fe12c15ad60 2024-09-10T21:24:46.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:46.018+0000 7fe131400640 1 --2- 172.21.15.17:0/3446007066 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fe10c0725e0 0x7fe10c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:46.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:46.018+0000 7fe131400640 1 --2- 172.21.15.17:0/3446007066 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fe10c0725e0 0x7fe10c074aa0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fe1200045b0 tx=0x7fe12000b390 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:46.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:46.022+0000 7fe129600640 1 -- 172.21.15.17:0/3446007066 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fe124016030 con 0x7fe12c15ad60 2024-09-10T21:24:46.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:46.198+0000 7fe133e00640 1 -- 172.21.15.17:0/3446007066 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) -- 0x7fe12c0734e0 con 0x7fe12c15ad60 2024-09-10T21:24:46.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:46.830+0000 7fe129600640 1 -- 172.21.15.17:0/3446007066 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v19) ==== 171+0+0 (secure 0 0 0) 0x7fe12405e770 con 0x7fe12c15ad60 2024-09-10T21:24:46.833 INFO:teuthology.orchestra.run.smithi017.stderr:pool 'cephfs_data' created 2024-09-10T21:24:46.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:46.830+0000 7fe133e00640 1 -- 172.21.15.17:0/3446007066 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fe10c0725e0 msgr2=0x7fe10c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:46.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:46.830+0000 7fe133e00640 1 --2- 172.21.15.17:0/3446007066 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fe10c0725e0 0x7fe10c074aa0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fe1200045b0 tx=0x7fe12000b390 comp rx=0 tx=0).stop 2024-09-10T21:24:46.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:46.830+0000 7fe133e00640 1 -- 172.21.15.17:0/3446007066 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe12c15ad60 msgr2=0x7fe12c0c7aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:46.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:46.830+0000 7fe133e00640 1 --2- 172.21.15.17:0/3446007066 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe12c15ad60 0x7fe12c0c7aa0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe12400e990 tx=0x7fe12400ee50 comp rx=0 tx=0).stop 2024-09-10T21:24:46.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:46.834+0000 7fe133e00640 1 -- 172.21.15.17:0/3446007066 shutdown_connections 2024-09-10T21:24:46.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:46.834+0000 7fe133e00640 1 --2- 172.21.15.17:0/3446007066 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fe10c0725e0 0x7fe10c074aa0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:46.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:46.834+0000 7fe133e00640 1 --2- 172.21.15.17:0/3446007066 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe12c15ad60 0x7fe12c0c7aa0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:46.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:46.834+0000 7fe133e00640 1 --2- 172.21.15.17:0/3446007066 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe12c150dc0 0x7fe12c0c7560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:46.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:46.834+0000 7fe133e00640 1 --2- 172.21.15.17:0/3446007066 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe12c0c8410 0x7fe12c141cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:46.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:46.834+0000 7fe133e00640 1 -- 172.21.15.17:0/3446007066 >> 172.21.15.17:0/3446007066 conn(0x7fe12c070110 msgr2=0x7fe12c14f010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:46.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:46.834+0000 7fe133e00640 1 -- 172.21.15.17:0/3446007066 shutdown_connections 2024-09-10T21:24:46.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:46.834+0000 7fe133e00640 1 -- 172.21.15.17:0/3446007066 wait complete. 2024-09-10T21:24:46.881 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-10T21:24:47.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:47.030+0000 7f5287e00640 1 -- 172.21.15.17:0/307378642 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f52880c8bc0 msgr2=0x7f52880b3c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:47.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:47.030+0000 7f5287e00640 1 --2- 172.21.15.17:0/307378642 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f52880c8bc0 0x7f52880b3c90 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f52880b3a30 tx=0x7f527801eff0 comp rx=0 tx=0).stop 2024-09-10T21:24:47.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:47.030+0000 7f5287e00640 1 -- 172.21.15.17:0/307378642 shutdown_connections 2024-09-10T21:24:47.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:47.030+0000 7f5287e00640 1 --2- 172.21.15.17:0/307378642 >> v2:172.21.15.31:3300/0 conn(0x7f52880c3700 0x7f5288158e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:47.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:47.030+0000 7f5287e00640 1 --2- 172.21.15.17:0/307378642 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f52880c8bc0 0x7f52880b3c90 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:47.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:47.030+0000 7f5287e00640 1 -- 172.21.15.17:0/307378642 >> 172.21.15.17:0/307378642 conn(0x7f52880720a0 msgr2=0x7f52880724b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:47.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:47.030+0000 7f5287e00640 1 -- 172.21.15.17:0/307378642 shutdown_connections 2024-09-10T21:24:47.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:47.030+0000 7f5287e00640 1 -- 172.21.15.17:0/307378642 wait complete. 2024-09-10T21:24:47.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:47.034+0000 7f5287e00640 1 Processor -- start 2024-09-10T21:24:47.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:47.034+0000 7f5287e00640 1 -- start start 2024-09-10T21:24:47.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:47.034+0000 7f5287e00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f52880c3700 0x7f52881f37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:47.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:47.034+0000 7f5287e00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52880c8bc0 0x7f5288155490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:47.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:47.034+0000 7f5287e00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f52880c8fa0 0x7f5288154620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:47.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:47.034+0000 7f5287e00640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f52880c9650 con 0x7f52880c8fa0 2024-09-10T21:24:47.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:47.034+0000 7f5287e00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f52880c94d0 con 0x7f52880c8bc0 2024-09-10T21:24:47.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:47.034+0000 7f5287e00640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f528814e8a0 con 0x7f52880c3700 2024-09-10T21:24:47.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:47.034+0000 7f5287400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f52880c8fa0 0x7f5288154620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:47.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:47.034+0000 7f5287400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f52880c8fa0 0x7f5288154620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:59028/0 (socket says 172.21.15.17:59028) 2024-09-10T21:24:47.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:47.034+0000 7f5287400640 1 -- 172.21.15.17:0/2246815857 learned_addr learned my addr 172.21.15.17:0/2246815857 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:24:47.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:47.034+0000 7f5286a00640 1 --2- 172.21.15.17:0/2246815857 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f52880c3700 0x7f52881f37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:47.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:47.034+0000 7f527fe00640 1 --2- 172.21.15.17:0/2246815857 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52880c8bc0 0x7f5288155490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:47.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:47.034+0000 7f5287400640 1 -- 172.21.15.17:0/2246815857 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f52880c3700 msgr2=0x7f52881f37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:47.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:47.034+0000 7f5287400640 1 --2- 172.21.15.17:0/2246815857 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f52880c3700 0x7f52881f37d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:47.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:47.034+0000 7f5287400640 1 -- 172.21.15.17:0/2246815857 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52880c8bc0 msgr2=0x7f5288155490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:47.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:47.034+0000 7f5287400640 1 --2- 172.21.15.17:0/2246815857 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52880c8bc0 0x7f5288155490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:47.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:47.034+0000 7f5287400640 1 -- 172.21.15.17:0/2246815857 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5288153840 con 0x7f52880c8fa0 2024-09-10T21:24:47.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:47.034+0000 7f5287400640 1 --2- 172.21.15.17:0/2246815857 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f52880c8fa0 0x7f5288154620 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f528000eb50 tx=0x7f528000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:47.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:47.034+0000 7f527ea00640 1 -- 172.21.15.17:0/2246815857 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5280004d30 con 0x7f52880c8fa0 2024-09-10T21:24:47.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:47.034+0000 7f527ea00640 1 -- 172.21.15.17:0/2246815857 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f528000ce70 con 0x7f52880c8fa0 2024-09-10T21:24:47.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:47.034+0000 7f527ea00640 1 -- 172.21.15.17:0/2246815857 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5280005240 con 0x7f52880c8fa0 2024-09-10T21:24:47.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:47.034+0000 7f5287e00640 1 -- 172.21.15.17:0/2246815857 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52881f40a0 con 0x7f52880c8fa0 2024-09-10T21:24:47.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:47.034+0000 7f5287e00640 1 -- 172.21.15.17:0/2246815857 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5288071840 con 0x7f52880c8fa0 2024-09-10T21:24:47.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:47.034+0000 7f5287e00640 1 -- 172.21.15.17:0/2246815857 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52880b3c90 con 0x7f52880c8fa0 2024-09-10T21:24:47.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:47.038+0000 7f527ea00640 1 -- 172.21.15.17:0/2246815857 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f52800053e0 con 0x7f52880c8fa0 2024-09-10T21:24:47.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:47.038+0000 7f527ea00640 1 --2- 172.21.15.17:0/2246815857 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f52640725e0 0x7f5264074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:47.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:47.038+0000 7f527ea00640 1 -- 172.21.15.17:0/2246815857 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 5609+0+0 (secure 0 0 0) 0x7f5280093d60 con 0x7f52880c8fa0 2024-09-10T21:24:47.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:47.038+0000 7f5286a00640 1 --2- 172.21.15.17:0/2246815857 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f52640725e0 0x7f5264074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:47.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:47.038+0000 7f5286a00640 1 --2- 172.21.15.17:0/2246815857 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f52640725e0 0x7f5264074aa0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f52780b2000 tx=0x7f527801d830 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:47.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:47.042+0000 7f527ea00640 1 -- 172.21.15.17:0/2246815857 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f528005ee40 con 0x7f52880c8fa0 2024-09-10T21:24:47.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:47.250+0000 7f5287e00640 1 -- 172.21.15.17:0/2246815857 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f52880cb6e0 con 0x7f52880c8fa0 2024-09-10T21:24:47.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:47.846+0000 7f527ea00640 1 -- 172.21.15.17:0/2246815857 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) ==== 172+0+0 (secure 0 0 0) 0x7f5280063870 con 0x7f52880c8fa0 2024-09-10T21:24:47.851 INFO:teuthology.orchestra.run.smithi017.stderr:new fs with metadata pool 2 and data pool 3 2024-09-10T21:24:47.853 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:47.850+0000 7f5287e00640 1 -- 172.21.15.17:0/2246815857 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f52640725e0 msgr2=0x7f5264074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:47.853 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:47.850+0000 7f5287e00640 1 --2- 172.21.15.17:0/2246815857 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f52640725e0 0x7f5264074aa0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f52780b2000 tx=0x7f527801d830 comp rx=0 tx=0).stop 2024-09-10T21:24:47.853 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:47.850+0000 7f5287e00640 1 -- 172.21.15.17:0/2246815857 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f52880c8fa0 msgr2=0x7f5288154620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:47.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:47.850+0000 7f5287e00640 1 --2- 172.21.15.17:0/2246815857 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f52880c8fa0 0x7f5288154620 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f528000eb50 tx=0x7f528000c6a0 comp rx=0 tx=0).stop 2024-09-10T21:24:47.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:47.854+0000 7f5287e00640 1 -- 172.21.15.17:0/2246815857 shutdown_connections 2024-09-10T21:24:47.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:47.854+0000 7f5287e00640 1 --2- 172.21.15.17:0/2246815857 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f52640725e0 0x7f5264074aa0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:47.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:47.854+0000 7f5287e00640 1 --2- 172.21.15.17:0/2246815857 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f52880c8fa0 0x7f5288154620 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:47.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:47.854+0000 7f5287e00640 1 --2- 172.21.15.17:0/2246815857 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52880c8bc0 0x7f5288155490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:47.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:47.854+0000 7f5287e00640 1 --2- 172.21.15.17:0/2246815857 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f52880c3700 0x7f52881f37d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:47.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:47.854+0000 7f5287e00640 1 -- 172.21.15.17:0/2246815857 >> 172.21.15.17:0/2246815857 conn(0x7f52880720a0 msgr2=0x7f52880c9bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:47.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:47.854+0000 7f5287e00640 1 -- 172.21.15.17:0/2246815857 shutdown_connections 2024-09-10T21:24:47.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:47.858+0000 7f5287e00640 1 -- 172.21.15.17:0/2246815857 wait complete. 2024-09-10T21:24:47.889 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-09-10T21:24:47.889 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-09-10T21:24:48.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:48.042+0000 7ff987c00640 1 -- 172.21.15.17:0/3952605579 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff980071680 msgr2=0x7ff9800c4bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:48.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:48.042+0000 7ff987c00640 1 --2- 172.21.15.17:0/3952605579 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff980071680 0x7ff9800c4bc0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff96c009840 tx=0x7ff96c01f010 comp rx=0 tx=0).stop 2024-09-10T21:24:48.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:48.042+0000 7ff987c00640 1 -- 172.21.15.17:0/3952605579 shutdown_connections 2024-09-10T21:24:48.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:48.042+0000 7ff987c00640 1 --2- 172.21.15.17:0/3952605579 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff980071680 0x7ff9800c4bc0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:48.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:48.042+0000 7ff987c00640 1 --2- 172.21.15.17:0/3952605579 >> v2:172.21.15.31:3300/0 conn(0x7ff980152520 0x7ff980070dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:48.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:48.042+0000 7ff987c00640 1 -- 172.21.15.17:0/3952605579 >> 172.21.15.17:0/3952605579 conn(0x7ff9800748d0 msgr2=0x7ff980068990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:48.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:48.042+0000 7ff987c00640 1 -- 172.21.15.17:0/3952605579 shutdown_connections 2024-09-10T21:24:48.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:48.042+0000 7ff987c00640 1 -- 172.21.15.17:0/3952605579 wait complete. 2024-09-10T21:24:48.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:48.046+0000 7ff987c00640 1 Processor -- start 2024-09-10T21:24:48.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:48.046+0000 7ff987c00640 1 -- start start 2024-09-10T21:24:48.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:48.046+0000 7ff987c00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff980071680 0x7ff9800c9820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:48.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:48.046+0000 7ff987c00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff9800cdf50 0x7ff9800caa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:48.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:48.046+0000 7ff987c00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff980152520 0x7ff98006e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:48.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:48.046+0000 7ff987c00640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7ff9800c5100 con 0x7ff9800cdf50 2024-09-10T21:24:48.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:48.046+0000 7ff987c00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7ff9800c3270 con 0x7ff980071680 2024-09-10T21:24:48.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:48.046+0000 7ff987c00640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7ff980152900 con 0x7ff980152520 2024-09-10T21:24:48.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:48.046+0000 7ff97fe00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff9800cdf50 0x7ff9800caa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:48.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:48.046+0000 7ff97fe00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff9800cdf50 0x7ff9800caa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:59038/0 (socket says 172.21.15.17:59038) 2024-09-10T21:24:48.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:48.046+0000 7ff97fe00640 1 -- 172.21.15.17:0/2115997858 learned_addr learned my addr 172.21.15.17:0/2115997858 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:24:48.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:48.046+0000 7ff985200640 1 --2- 172.21.15.17:0/2115997858 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff980071680 0x7ff9800c9820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:48.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:48.046+0000 7ff985c00640 1 --2- 172.21.15.17:0/2115997858 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff980152520 0x7ff98006e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:48.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:48.046+0000 7ff97fe00640 1 -- 172.21.15.17:0/2115997858 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff980152520 msgr2=0x7ff98006e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:48.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:48.046+0000 7ff97fe00640 1 --2- 172.21.15.17:0/2115997858 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff980152520 0x7ff98006e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:48.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:48.046+0000 7ff97fe00640 1 -- 172.21.15.17:0/2115997858 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff980071680 msgr2=0x7ff9800c9820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:48.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:48.046+0000 7ff97fe00640 1 --2- 172.21.15.17:0/2115997858 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff980071680 0x7ff9800c9820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:48.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:48.046+0000 7ff97fe00640 1 -- 172.21.15.17:0/2115997858 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff98006fcf0 con 0x7ff9800cdf50 2024-09-10T21:24:48.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:48.046+0000 7ff985c00640 1 --2- 172.21.15.17:0/2115997858 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff980152520 0x7ff98006e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:24:48.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:48.046+0000 7ff97fe00640 1 --2- 172.21.15.17:0/2115997858 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff9800cdf50 0x7ff9800caa80 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff96c009930 tx=0x7ff96c004620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:48.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:48.046+0000 7ff97cc00640 1 -- 172.21.15.17:0/2115997858 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff96c0b5040 con 0x7ff9800cdf50 2024-09-10T21:24:48.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:48.046+0000 7ff97cc00640 1 -- 172.21.15.17:0/2115997858 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff96c0a8c20 con 0x7ff9800cdf50 2024-09-10T21:24:48.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:48.046+0000 7ff987c00640 1 -- 172.21.15.17:0/2115997858 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff98015ac00 con 0x7ff9800cdf50 2024-09-10T21:24:48.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:48.046+0000 7ff97cc00640 1 -- 172.21.15.17:0/2115997858 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff96c005220 con 0x7ff9800cdf50 2024-09-10T21:24:48.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:48.046+0000 7ff987c00640 1 -- 172.21.15.17:0/2115997858 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff98006fa10 con 0x7ff9800cdf50 2024-09-10T21:24:48.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:48.050+0000 7ff987c00640 1 -- 172.21.15.17:0/2115997858 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff980159810 con 0x7ff9800cdf50 2024-09-10T21:24:48.055 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:48.050+0000 7ff97cc00640 1 -- 172.21.15.17:0/2115997858 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7ff96c0b8070 con 0x7ff9800cdf50 2024-09-10T21:24:48.055 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:48.050+0000 7ff97cc00640 1 --2- 172.21.15.17:0/2115997858 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7ff9600725e0 0x7ff960074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:48.055 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:48.050+0000 7ff97cc00640 1 -- 172.21.15.17:0/2115997858 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5709+0+0 (secure 0 0 0) 0x7ff96c130610 con 0x7ff9800cdf50 2024-09-10T21:24:48.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:48.050+0000 7ff985200640 1 --2- 172.21.15.17:0/2115997858 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7ff9600725e0 0x7ff960074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:48.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:48.054+0000 7ff985200640 1 --2- 172.21.15.17:0/2115997858 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7ff9600725e0 0x7ff960074aa0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff974009ad0 tx=0x7ff974009260 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:48.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:48.054+0000 7ff97cc00640 1 -- 172.21.15.17:0/2115997858 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7ff96c018b70 con 0x7ff9800cdf50 2024-09-10T21:24:48.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:48.238+0000 7ff987c00640 1 -- 172.21.15.17:0/2115997858 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) -- 0x7ff9800c4b20 con 0x7ff9800cdf50 2024-09-10T21:24:48.853 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:48.850+0000 7ff97cc00640 1 -- 172.21.15.17:0/2115997858 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v21) ==== 157+0+0 (secure 0 0 0) 0x7ff96c0fbc50 con 0x7ff9800cdf50 2024-09-10T21:24:48.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:48.854+0000 7ff987c00640 1 -- 172.21.15.17:0/2115997858 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7ff9600725e0 msgr2=0x7ff960074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:48.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:48.854+0000 7ff987c00640 1 --2- 172.21.15.17:0/2115997858 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7ff9600725e0 0x7ff960074aa0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff974009ad0 tx=0x7ff974009260 comp rx=0 tx=0).stop 2024-09-10T21:24:48.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:48.854+0000 7ff987c00640 1 -- 172.21.15.17:0/2115997858 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff9800cdf50 msgr2=0x7ff9800caa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:48.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:48.854+0000 7ff987c00640 1 --2- 172.21.15.17:0/2115997858 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff9800cdf50 0x7ff9800caa80 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff96c009930 tx=0x7ff96c004620 comp rx=0 tx=0).stop 2024-09-10T21:24:48.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:48.854+0000 7ff987c00640 1 -- 172.21.15.17:0/2115997858 shutdown_connections 2024-09-10T21:24:48.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:48.854+0000 7ff987c00640 1 --2- 172.21.15.17:0/2115997858 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7ff9600725e0 0x7ff960074aa0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:48.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:48.854+0000 7ff987c00640 1 --2- 172.21.15.17:0/2115997858 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff980152520 0x7ff98006e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:48.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:48.854+0000 7ff987c00640 1 --2- 172.21.15.17:0/2115997858 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff9800cdf50 0x7ff9800caa80 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:48.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:48.854+0000 7ff987c00640 1 --2- 172.21.15.17:0/2115997858 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff980071680 0x7ff9800c9820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:48.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:48.854+0000 7ff987c00640 1 -- 172.21.15.17:0/2115997858 >> 172.21.15.17:0/2115997858 conn(0x7ff9800748d0 msgr2=0x7ff980157900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:48.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:48.854+0000 7ff987c00640 1 -- 172.21.15.17:0/2115997858 shutdown_connections 2024-09-10T21:24:48.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:48.854+0000 7ff987c00640 1 -- 172.21.15.17:0/2115997858 wait complete. 2024-09-10T21:24:48.882 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-09-10T21:24:49.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:49.038+0000 7efe6c000640 1 -- 172.21.15.17:0/3330513343 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efe640c34f0 msgr2=0x7efe64152950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:49.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:49.038+0000 7efe6c000640 1 --2- 172.21.15.17:0/3330513343 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efe640c34f0 0x7efe64152950 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7efe5c009c80 tx=0x7efe5c01cd10 comp rx=0 tx=0).stop 2024-09-10T21:24:49.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:49.038+0000 7efe6c000640 1 -- 172.21.15.17:0/3330513343 shutdown_connections 2024-09-10T21:24:49.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:49.038+0000 7efe6c000640 1 --2- 172.21.15.17:0/3330513343 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efe640c34f0 0x7efe64152950 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:49.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:49.038+0000 7efe6c000640 1 --2- 172.21.15.17:0/3330513343 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7efe640c96f0 0x7efe64152410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:49.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:49.038+0000 7efe6c000640 1 -- 172.21.15.17:0/3330513343 >> 172.21.15.17:0/3330513343 conn(0x7efe6406d220 msgr2=0x7efe64068990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:49.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:49.038+0000 7efe6c000640 1 -- 172.21.15.17:0/3330513343 shutdown_connections 2024-09-10T21:24:49.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:49.038+0000 7efe6c000640 1 -- 172.21.15.17:0/3330513343 wait complete. 2024-09-10T21:24:49.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:49.038+0000 7efe6c000640 1 Processor -- start 2024-09-10T21:24:49.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:49.038+0000 7efe6c000640 1 -- start start 2024-09-10T21:24:49.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:49.042+0000 7efe6c000640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7efe640c34f0 0x7efe641f7840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:49.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:49.042+0000 7efe6c000640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe640c9310 0x7efe6414b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:49.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:49.042+0000 7efe6c000640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efe640c96f0 0x7efe6414bde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:49.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:49.042+0000 7efe6c000640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7efe64070580 con 0x7efe640c96f0 2024-09-10T21:24:49.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:49.042+0000 7efe6c000640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7efe64070400 con 0x7efe640c9310 2024-09-10T21:24:49.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:49.042+0000 7efe6c000640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7efe641657c0 con 0x7efe640c34f0 2024-09-10T21:24:49.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:49.042+0000 7efe6a000640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efe640c96f0 0x7efe6414bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:49.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:49.042+0000 7efe6a000640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efe640c96f0 0x7efe6414bde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:59064/0 (socket says 172.21.15.17:59064) 2024-09-10T21:24:49.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:49.042+0000 7efe6a000640 1 -- 172.21.15.17:0/1682358942 learned_addr learned my addr 172.21.15.17:0/1682358942 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:24:49.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:49.042+0000 7efe69600640 1 --2- 172.21.15.17:0/1682358942 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7efe640c34f0 0x7efe641f7840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:49.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:49.042+0000 7efe68c00640 1 --2- 172.21.15.17:0/1682358942 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe640c9310 0x7efe6414b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:49.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:49.042+0000 7efe6a000640 1 -- 172.21.15.17:0/1682358942 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7efe640c34f0 msgr2=0x7efe641f7840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:49.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:49.042+0000 7efe6a000640 1 --2- 172.21.15.17:0/1682358942 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7efe640c34f0 0x7efe641f7840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:49.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:49.042+0000 7efe6a000640 1 -- 172.21.15.17:0/1682358942 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe640c9310 msgr2=0x7efe6414b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:49.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:49.042+0000 7efe6a000640 1 --2- 172.21.15.17:0/1682358942 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe640c9310 0x7efe6414b8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:49.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:49.042+0000 7efe6a000640 1 -- 172.21.15.17:0/1682358942 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe64151b70 con 0x7efe640c96f0 2024-09-10T21:24:49.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:49.042+0000 7efe69600640 1 --2- 172.21.15.17:0/1682358942 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7efe640c34f0 0x7efe641f7840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:24:49.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:49.042+0000 7efe6a000640 1 --2- 172.21.15.17:0/1682358942 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efe640c96f0 0x7efe6414bde0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7efe5c01cb40 tx=0x7efe5c0a5a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:49.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:49.042+0000 7efe61600640 1 -- 172.21.15.17:0/1682358942 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe5c0bb070 con 0x7efe640c96f0 2024-09-10T21:24:49.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:49.042+0000 7efe61600640 1 -- 172.21.15.17:0/1682358942 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efe5c004d60 con 0x7efe640c96f0 2024-09-10T21:24:49.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:49.042+0000 7efe6c000640 1 -- 172.21.15.17:0/1682358942 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe6406e170 con 0x7efe640c96f0 2024-09-10T21:24:49.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:49.042+0000 7efe61600640 1 -- 172.21.15.17:0/1682358942 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe5c0b2040 con 0x7efe640c96f0 2024-09-10T21:24:49.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:49.042+0000 7efe6c000640 1 -- 172.21.15.17:0/1682358942 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe6406ded0 con 0x7efe640c96f0 2024-09-10T21:24:49.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:49.042+0000 7efe6c000640 1 -- 172.21.15.17:0/1682358942 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe64070960 con 0x7efe640c96f0 2024-09-10T21:24:49.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:49.046+0000 7efe61600640 1 -- 172.21.15.17:0/1682358942 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7efe5c0b5030 con 0x7efe640c96f0 2024-09-10T21:24:49.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:49.046+0000 7efe61600640 1 --2- 172.21.15.17:0/1682358942 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7efe440725e0 0x7efe44074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:49.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:49.046+0000 7efe61600640 1 -- 172.21.15.17:0/1682358942 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 6007+0+0 (secure 0 0 0) 0x7efe5c12d2d0 con 0x7efe640c96f0 2024-09-10T21:24:49.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:49.046+0000 7efe69600640 1 --2- 172.21.15.17:0/1682358942 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7efe440725e0 0x7efe44074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:49.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:49.046+0000 7efe69600640 1 --2- 172.21.15.17:0/1682358942 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7efe440725e0 0x7efe44074aa0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7efe58001760 tx=0x7efe58009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:49.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:49.050+0000 7efe61600640 1 -- 172.21.15.17:0/1682358942 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7efe5c016990 con 0x7efe640c96f0 2024-09-10T21:24:49.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:49.226+0000 7efe6c000640 1 -- 172.21.15.17:0/1682358942 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7efe641f7d80 con 0x7efe640c96f0 2024-09-10T21:24:50.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:50.862+0000 7efe61600640 1 -- 172.21.15.17:0/1682358942 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v23) ==== 228+0+0 (secure 0 0 0) 0x7efe5c0f8980 con 0x7efe640c96f0 2024-09-10T21:24:50.866 INFO:teuthology.orchestra.run.smithi017.stderr:pool 'cephfs_data_ec' created 2024-09-10T21:24:50.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:50.866+0000 7efe6c000640 1 -- 172.21.15.17:0/1682358942 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7efe440725e0 msgr2=0x7efe44074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:50.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:50.866+0000 7efe6c000640 1 --2- 172.21.15.17:0/1682358942 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7efe440725e0 0x7efe44074aa0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7efe58001760 tx=0x7efe58009040 comp rx=0 tx=0).stop 2024-09-10T21:24:50.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:50.866+0000 7efe6c000640 1 -- 172.21.15.17:0/1682358942 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efe640c96f0 msgr2=0x7efe6414bde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:50.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:50.866+0000 7efe6c000640 1 --2- 172.21.15.17:0/1682358942 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efe640c96f0 0x7efe6414bde0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7efe5c01cb40 tx=0x7efe5c0a5a50 comp rx=0 tx=0).stop 2024-09-10T21:24:50.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:50.866+0000 7efe6c000640 1 -- 172.21.15.17:0/1682358942 shutdown_connections 2024-09-10T21:24:50.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:50.866+0000 7efe6c000640 1 --2- 172.21.15.17:0/1682358942 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7efe440725e0 0x7efe44074aa0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:50.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:50.866+0000 7efe6c000640 1 --2- 172.21.15.17:0/1682358942 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efe640c96f0 0x7efe6414bde0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:50.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:50.866+0000 7efe6c000640 1 --2- 172.21.15.17:0/1682358942 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe640c9310 0x7efe6414b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:50.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:50.866+0000 7efe6c000640 1 --2- 172.21.15.17:0/1682358942 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7efe640c34f0 0x7efe641f7840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:50.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:50.866+0000 7efe6c000640 1 -- 172.21.15.17:0/1682358942 >> 172.21.15.17:0/1682358942 conn(0x7efe6406d220 msgr2=0x7efe64068990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:50.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:50.866+0000 7efe6c000640 1 -- 172.21.15.17:0/1682358942 shutdown_connections 2024-09-10T21:24:50.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:50.866+0000 7efe6c000640 1 -- 172.21.15.17:0/1682358942 wait complete. 2024-09-10T21:24:50.905 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-09-10T21:24:51.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:51.054+0000 7f8f42a00640 1 -- 172.21.15.17:0/3129813648 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8f3c159010 msgr2=0x7f8f3c0c3060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:51.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:51.054+0000 7f8f42a00640 1 --2- 172.21.15.17:0/3129813648 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8f3c159010 0x7f8f3c0c3060 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f8f34009c80 tx=0x7f8f3401cda0 comp rx=0 tx=0).stop 2024-09-10T21:24:51.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:51.054+0000 7f8f42a00640 1 -- 172.21.15.17:0/3129813648 shutdown_connections 2024-09-10T21:24:51.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:51.054+0000 7f8f42a00640 1 --2- 172.21.15.17:0/3129813648 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8f3c159010 0x7f8f3c0c3060 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:51.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:51.054+0000 7f8f42a00640 1 --2- 172.21.15.17:0/3129813648 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8f3c0c8060 0x7f8f3c15ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:51.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:51.054+0000 7f8f42a00640 1 -- 172.21.15.17:0/3129813648 >> 172.21.15.17:0/3129813648 conn(0x7f8f3c06d330 msgr2=0x7f8f3c0747f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:51.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:51.054+0000 7f8f42a00640 1 -- 172.21.15.17:0/3129813648 shutdown_connections 2024-09-10T21:24:51.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:51.054+0000 7f8f42a00640 1 -- 172.21.15.17:0/3129813648 wait complete. 2024-09-10T21:24:51.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:51.058+0000 7f8f42a00640 1 Processor -- start 2024-09-10T21:24:51.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:51.058+0000 7f8f42a00640 1 -- start start 2024-09-10T21:24:51.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:51.058+0000 7f8f42a00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8f3c0c8060 0x7f8f3c153b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:51.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:51.058+0000 7f8f42a00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f3c1525b0 0x7f8f3c14dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:51.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:51.058+0000 7f8f42a00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8f3c159010 0x7f8f3c14e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:51.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:51.058+0000 7f8f42a00640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f8f3c152b10 con 0x7f8f3c0c8060 2024-09-10T21:24:51.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:51.058+0000 7f8f42a00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f8f3c152990 con 0x7f8f3c1525b0 2024-09-10T21:24:51.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:51.058+0000 7f8f42a00640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f8f3c0cb810 con 0x7f8f3c159010 2024-09-10T21:24:51.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:51.058+0000 7f8f3be00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8f3c0c8060 0x7f8f3c153b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:51.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:51.058+0000 7f8f3be00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8f3c0c8060 0x7f8f3c153b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:59076/0 (socket says 172.21.15.17:59076) 2024-09-10T21:24:51.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:51.058+0000 7f8f3be00640 1 -- 172.21.15.17:0/3951774598 learned_addr learned my addr 172.21.15.17:0/3951774598 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:24:51.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:51.058+0000 7f8f33e00640 1 --2- 172.21.15.17:0/3951774598 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f3c1525b0 0x7f8f3c14dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:51.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:51.058+0000 7f8f40a00640 1 --2- 172.21.15.17:0/3951774598 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8f3c159010 0x7f8f3c14e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:51.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:51.058+0000 7f8f3be00640 1 -- 172.21.15.17:0/3951774598 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8f3c159010 msgr2=0x7f8f3c14e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:51.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:51.058+0000 7f8f3be00640 1 --2- 172.21.15.17:0/3951774598 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8f3c159010 0x7f8f3c14e010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:51.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:51.058+0000 7f8f3be00640 1 -- 172.21.15.17:0/3951774598 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f3c1525b0 msgr2=0x7f8f3c14dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:51.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:51.058+0000 7f8f3be00640 1 --2- 172.21.15.17:0/3951774598 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f3c1525b0 0x7f8f3c14dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:51.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:51.058+0000 7f8f3be00640 1 -- 172.21.15.17:0/3951774598 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f3c15b5a0 con 0x7f8f3c0c8060 2024-09-10T21:24:51.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:51.058+0000 7f8f3be00640 1 --2- 172.21.15.17:0/3951774598 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8f3c0c8060 0x7f8f3c153b90 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f8f2c00c450 tx=0x7f8f2c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:51.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:51.058+0000 7f8f38c00640 1 -- 172.21.15.17:0/3951774598 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f2c011790 con 0x7f8f3c0c8060 2024-09-10T21:24:51.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:51.058+0000 7f8f38c00640 1 -- 172.21.15.17:0/3951774598 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8f2c017070 con 0x7f8f3c0c8060 2024-09-10T21:24:51.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:51.058+0000 7f8f42a00640 1 -- 172.21.15.17:0/3951774598 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f3c1f25e0 con 0x7f8f3c0c8060 2024-09-10T21:24:51.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:51.058+0000 7f8f38c00640 1 -- 172.21.15.17:0/3951774598 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f2c011ce0 con 0x7f8f3c0c8060 2024-09-10T21:24:51.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:51.058+0000 7f8f42a00640 1 -- 172.21.15.17:0/3951774598 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f3c152cf0 con 0x7f8f3c0c8060 2024-09-10T21:24:51.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:51.058+0000 7f8f42a00640 1 -- 172.21.15.17:0/3951774598 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f3c15ba10 con 0x7f8f3c0c8060 2024-09-10T21:24:51.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:51.062+0000 7f8f38c00640 1 -- 172.21.15.17:0/3951774598 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f8f2c01c460 con 0x7f8f3c0c8060 2024-09-10T21:24:51.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:51.062+0000 7f8f38c00640 1 --2- 172.21.15.17:0/3951774598 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f8f18072610 0x7f8f18074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:51.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:51.062+0000 7f8f38c00640 1 -- 172.21.15.17:0/3951774598 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 6500+0+0 (secure 0 0 0) 0x7f8f2c0932d0 con 0x7f8f3c0c8060 2024-09-10T21:24:51.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:51.062+0000 7f8f33e00640 1 --2- 172.21.15.17:0/3951774598 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f8f18072610 0x7f8f18074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:51.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:51.062+0000 7f8f33e00640 1 --2- 172.21.15.17:0/3951774598 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f8f18072610 0x7f8f18074ad0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8f3c14e9e0 tx=0x7f8f2400b390 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:51.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:51.066+0000 7f8f38c00640 1 -- 172.21.15.17:0/3951774598 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f8f2c05dfa0 con 0x7f8f3c0c8060 2024-09-10T21:24:51.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:51.238+0000 7f8f42a00640 1 -- 172.21.15.17:0/3951774598 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) -- 0x7f8f3c074ad0 con 0x7f8f3c0c8060 2024-09-10T21:24:51.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:51.866+0000 7f8f38c00640 1 -- 172.21.15.17:0/3951774598 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v24) ==== 169+0+0 (secure 0 0 0) 0x7f8f2c0629d0 con 0x7f8f3c0c8060 2024-09-10T21:24:51.872 INFO:teuthology.orchestra.run.smithi017.stderr:set pool 4 allow_ec_overwrites to true 2024-09-10T21:24:51.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:51.870+0000 7f8f42a00640 1 -- 172.21.15.17:0/3951774598 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f8f18072610 msgr2=0x7f8f18074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:51.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:51.870+0000 7f8f42a00640 1 --2- 172.21.15.17:0/3951774598 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f8f18072610 0x7f8f18074ad0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8f3c14e9e0 tx=0x7f8f2400b390 comp rx=0 tx=0).stop 2024-09-10T21:24:51.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:51.870+0000 7f8f42a00640 1 -- 172.21.15.17:0/3951774598 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8f3c0c8060 msgr2=0x7f8f3c153b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:51.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:51.870+0000 7f8f42a00640 1 --2- 172.21.15.17:0/3951774598 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8f3c0c8060 0x7f8f3c153b90 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f8f2c00c450 tx=0x7f8f2c00c910 comp rx=0 tx=0).stop 2024-09-10T21:24:51.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:51.870+0000 7f8f42a00640 1 -- 172.21.15.17:0/3951774598 shutdown_connections 2024-09-10T21:24:51.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:51.870+0000 7f8f42a00640 1 --2- 172.21.15.17:0/3951774598 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f8f18072610 0x7f8f18074ad0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:51.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:51.870+0000 7f8f42a00640 1 --2- 172.21.15.17:0/3951774598 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8f3c159010 0x7f8f3c14e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:51.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:51.870+0000 7f8f42a00640 1 --2- 172.21.15.17:0/3951774598 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f3c1525b0 0x7f8f3c14dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:51.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:51.870+0000 7f8f42a00640 1 --2- 172.21.15.17:0/3951774598 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8f3c0c8060 0x7f8f3c153b90 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:51.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:51.870+0000 7f8f42a00640 1 -- 172.21.15.17:0/3951774598 >> 172.21.15.17:0/3951774598 conn(0x7f8f3c06d330 msgr2=0x7f8f3c1515e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:51.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:51.870+0000 7f8f42a00640 1 -- 172.21.15.17:0/3951774598 shutdown_connections 2024-09-10T21:24:51.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:51.870+0000 7f8f42a00640 1 -- 172.21.15.17:0/3951774598 wait complete. 2024-09-10T21:24:51.916 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2024-09-10T21:24:52.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:52.066+0000 7f943c400640 1 -- 172.21.15.17:0/3635271625 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9434074860 msgr2=0x7f94340c10d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:52.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:52.066+0000 7f943c400640 1 --2- 172.21.15.17:0/3635271625 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9434074860 0x7f94340c10d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f943406fd90 tx=0x7f94280203e0 comp rx=0 tx=0).stop 2024-09-10T21:24:52.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:52.070+0000 7f943c400640 1 -- 172.21.15.17:0/3635271625 shutdown_connections 2024-09-10T21:24:52.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:52.070+0000 7f943c400640 1 --2- 172.21.15.17:0/3635271625 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f94340c6dd0 0x7f94340c71e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:52.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:52.070+0000 7f943c400640 1 --2- 172.21.15.17:0/3635271625 >> v2:172.21.15.31:3300/0 conn(0x7f9434072600 0x7f9434153f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:52.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:52.070+0000 7f943c400640 1 --2- 172.21.15.17:0/3635271625 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9434074860 0x7f94340c10d0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:52.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:52.070+0000 7f943c400640 1 -- 172.21.15.17:0/3635271625 >> 172.21.15.17:0/3635271625 conn(0x7f9434070110 msgr2=0x7f9434070520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:52.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:52.070+0000 7f943c400640 1 -- 172.21.15.17:0/3635271625 shutdown_connections 2024-09-10T21:24:52.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:52.070+0000 7f943c400640 1 -- 172.21.15.17:0/3635271625 wait complete. 2024-09-10T21:24:52.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:52.070+0000 7f943c400640 1 Processor -- start 2024-09-10T21:24:52.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:52.070+0000 7f943c400640 1 -- start start 2024-09-10T21:24:52.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:52.070+0000 7f943c400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9434072600 0x7f943414e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:52.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:52.070+0000 7f943c400640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f9434074860 0x7f94341e2480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:52.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:52.070+0000 7f943c400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f94340c6dd0 0x7f9434150bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:52.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:52.070+0000 7f943c400640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f9434074040 con 0x7f94340c6dd0 2024-09-10T21:24:52.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:52.070+0000 7f943c400640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f9434073d60 con 0x7f9434072600 2024-09-10T21:24:52.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:52.070+0000 7f943c400640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f94340c8110 con 0x7f9434074860 2024-09-10T21:24:52.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:52.070+0000 7f943a400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f94340c6dd0 0x7f9434150bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:52.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:52.070+0000 7f943a400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f94340c6dd0 0x7f9434150bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:59094/0 (socket says 172.21.15.17:59094) 2024-09-10T21:24:52.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:52.070+0000 7f943a400640 1 -- 172.21.15.17:0/2497369872 learned_addr learned my addr 172.21.15.17:0/2497369872 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:24:52.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:52.070+0000 7f9439a00640 1 --2- 172.21.15.17:0/2497369872 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9434072600 0x7f943414e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:52.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:52.070+0000 7f9439000640 1 --2- 172.21.15.17:0/2497369872 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f9434074860 0x7f94341e2480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:52.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:52.070+0000 7f943a400640 1 -- 172.21.15.17:0/2497369872 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f9434074860 msgr2=0x7f94341e2480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:52.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:52.070+0000 7f943a400640 1 --2- 172.21.15.17:0/2497369872 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f9434074860 0x7f94341e2480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:52.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:52.070+0000 7f943a400640 1 -- 172.21.15.17:0/2497369872 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9434072600 msgr2=0x7f943414e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:52.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:52.070+0000 7f943a400640 1 --2- 172.21.15.17:0/2497369872 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9434072600 0x7f943414e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:52.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:52.070+0000 7f943a400640 1 -- 172.21.15.17:0/2497369872 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94341ee120 con 0x7f94340c6dd0 2024-09-10T21:24:52.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:52.074+0000 7f943a400640 1 --2- 172.21.15.17:0/2497369872 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f94340c6dd0 0x7f9434150bb0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f942c00e990 tx=0x7f942c00ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:52.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:52.074+0000 7f9431600640 1 -- 172.21.15.17:0/2497369872 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f942c004d30 con 0x7f94340c6dd0 2024-09-10T21:24:52.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:52.074+0000 7f9431600640 1 -- 172.21.15.17:0/2497369872 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f942c00caa0 con 0x7f94340c6dd0 2024-09-10T21:24:52.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:52.074+0000 7f9431600640 1 -- 172.21.15.17:0/2497369872 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f942c005020 con 0x7f94340c6dd0 2024-09-10T21:24:52.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:52.074+0000 7f943c400640 1 -- 172.21.15.17:0/2497369872 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94341e3670 con 0x7f94340c6dd0 2024-09-10T21:24:52.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:52.074+0000 7f943c400640 1 -- 172.21.15.17:0/2497369872 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94341e3910 con 0x7f94340c6dd0 2024-09-10T21:24:52.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:52.074+0000 7f943c400640 1 -- 172.21.15.17:0/2497369872 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9434074540 con 0x7f94340c6dd0 2024-09-10T21:24:52.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:52.074+0000 7f9431600640 1 -- 172.21.15.17:0/2497369872 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f942c00f040 con 0x7f94340c6dd0 2024-09-10T21:24:52.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:52.074+0000 7f9431600640 1 --2- 172.21.15.17:0/2497369872 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f94140725e0 0x7f9414074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:52.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:52.074+0000 7f9431600640 1 -- 172.21.15.17:0/2497369872 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 6500+0+0 (secure 0 0 0) 0x7f942c093b90 con 0x7f94340c6dd0 2024-09-10T21:24:52.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:52.074+0000 7f9439a00640 1 --2- 172.21.15.17:0/2497369872 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f94140725e0 0x7f9414074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:52.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:52.078+0000 7f9439a00640 1 --2- 172.21.15.17:0/2497369872 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f94140725e0 0x7f9414074aa0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f94280b2000 tx=0x7f942801ec20 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:52.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:52.078+0000 7f9431600640 1 -- 172.21.15.17:0/2497369872 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f942c016030 con 0x7f94340c6dd0 2024-09-10T21:24:52.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:52.290+0000 7f943c400640 1 -- 172.21.15.17:0/2497369872 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) -- 0x7f94340c7ce0 con 0x7f94340c6dd0 2024-09-10T21:24:52.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:52.886+0000 7f9431600640 1 -- 172.21.15.17:0/2497369872 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v5) ==== 137+0+0 (secure 0 0 0) 0x7f942c05e7d0 con 0x7f94340c6dd0 2024-09-10T21:24:52.889 INFO:teuthology.orchestra.run.smithi017.stderr:added data pool 4 to fsmap 2024-09-10T21:24:52.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:52.886+0000 7f943c400640 1 -- 172.21.15.17:0/2497369872 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f94140725e0 msgr2=0x7f9414074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:52.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:52.886+0000 7f943c400640 1 --2- 172.21.15.17:0/2497369872 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f94140725e0 0x7f9414074aa0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f94280b2000 tx=0x7f942801ec20 comp rx=0 tx=0).stop 2024-09-10T21:24:52.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:52.886+0000 7f943c400640 1 -- 172.21.15.17:0/2497369872 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f94340c6dd0 msgr2=0x7f9434150bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:52.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:52.886+0000 7f943c400640 1 --2- 172.21.15.17:0/2497369872 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f94340c6dd0 0x7f9434150bb0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f942c00e990 tx=0x7f942c00ee50 comp rx=0 tx=0).stop 2024-09-10T21:24:52.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:52.886+0000 7f943c400640 1 -- 172.21.15.17:0/2497369872 shutdown_connections 2024-09-10T21:24:52.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:52.886+0000 7f943c400640 1 --2- 172.21.15.17:0/2497369872 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f94140725e0 0x7f9414074aa0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:52.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:52.886+0000 7f943c400640 1 --2- 172.21.15.17:0/2497369872 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f94340c6dd0 0x7f9434150bb0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:52.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:52.886+0000 7f943c400640 1 --2- 172.21.15.17:0/2497369872 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f9434074860 0x7f94341e2480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:52.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:52.886+0000 7f943c400640 1 --2- 172.21.15.17:0/2497369872 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9434072600 0x7f943414e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:52.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:52.886+0000 7f943c400640 1 -- 172.21.15.17:0/2497369872 >> 172.21.15.17:0/2497369872 conn(0x7f9434070110 msgr2=0x7f94340c2930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:52.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:52.886+0000 7f943c400640 1 -- 172.21.15.17:0/2497369872 shutdown_connections 2024-09-10T21:24:52.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:52.886+0000 7f943c400640 1 -- 172.21.15.17:0/2497369872 wait complete. 2024-09-10T21:24:52.914 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:24:53.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:53.058+0000 7f4863e00640 1 -- 172.21.15.17:0/2446577293 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f48640c64f0 msgr2=0x7f48640c8270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:53.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:53.058+0000 7f4863e00640 1 --2- 172.21.15.17:0/2446577293 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f48640c64f0 0x7f48640c8270 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f4850009900 tx=0x7f485001f0d0 comp rx=0 tx=0).stop 2024-09-10T21:24:53.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:53.058+0000 7f4863e00640 1 -- 172.21.15.17:0/2446577293 shutdown_connections 2024-09-10T21:24:53.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:53.058+0000 7f4863e00640 1 --2- 172.21.15.17:0/2446577293 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f48640c64f0 0x7f48640c8270 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:53.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:53.058+0000 7f4863e00640 1 --2- 172.21.15.17:0/2446577293 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f48640c6110 0x7f4864068bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:53.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:53.058+0000 7f4863e00640 1 -- 172.21.15.17:0/2446577293 >> 172.21.15.17:0/2446577293 conn(0x7f486406cdf0 msgr2=0x7f486406d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:53.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:53.058+0000 7f4863e00640 1 -- 172.21.15.17:0/2446577293 shutdown_connections 2024-09-10T21:24:53.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:53.058+0000 7f4863e00640 1 -- 172.21.15.17:0/2446577293 wait complete. 2024-09-10T21:24:53.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:53.058+0000 7f4863e00640 1 Processor -- start 2024-09-10T21:24:53.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:53.058+0000 7f4863e00640 1 -- start start 2024-09-10T21:24:53.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:53.062+0000 7f4863e00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f48640c6110 0x7f4864145cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:53.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:53.062+0000 7f4863e00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f48640c64f0 0x7f4864153360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:53.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:53.062+0000 7f4863e00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48640c87b0 0x7f486414fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:53.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:53.062+0000 7f4863e00640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f4864158ee0 con 0x7f48640c64f0 2024-09-10T21:24:53.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:53.062+0000 7f4863e00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f48640c9ab0 con 0x7f48640c87b0 2024-09-10T21:24:53.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:53.062+0000 7f4863e00640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f4864159060 con 0x7f48640c6110 2024-09-10T21:24:53.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:53.062+0000 7f4862000640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f48640c64f0 0x7f4864153360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:53.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:53.062+0000 7f4862000640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f48640c64f0 0x7f4864153360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:59118/0 (socket says 172.21.15.17:59118) 2024-09-10T21:24:53.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:53.062+0000 7f4862000640 1 -- 172.21.15.17:0/469623064 learned_addr learned my addr 172.21.15.17:0/469623064 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:24:53.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:53.062+0000 7f4862000640 1 -- 172.21.15.17:0/469623064 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f48640c6110 msgr2=0x7f4864145cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:53.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:53.062+0000 7f4862000640 1 --2- 172.21.15.17:0/469623064 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f48640c6110 0x7f4864145cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:53.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:53.062+0000 7f4862000640 1 -- 172.21.15.17:0/469623064 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48640c87b0 msgr2=0x7f486414fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:53.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:53.062+0000 7f4863400640 1 --2- 172.21.15.17:0/469623064 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48640c87b0 0x7f486414fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:53.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:53.062+0000 7f4862000640 1 --2- 172.21.15.17:0/469623064 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48640c87b0 0x7f486414fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:53.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:53.062+0000 7f4862000640 1 -- 172.21.15.17:0/469623064 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4864146200 con 0x7f48640c64f0 2024-09-10T21:24:53.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:53.062+0000 7f4863400640 1 --2- 172.21.15.17:0/469623064 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48640c87b0 0x7f486414fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:24:53.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:53.062+0000 7f4862000640 1 --2- 172.21.15.17:0/469623064 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f48640c64f0 0x7f4864153360 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f48500b3040 tx=0x7f4850005790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:53.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:53.062+0000 7f485aa00640 1 -- 172.21.15.17:0/469623064 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48500b5040 con 0x7f48640c64f0 2024-09-10T21:24:53.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:53.062+0000 7f485aa00640 1 -- 172.21.15.17:0/469623064 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f48500036f0 con 0x7f48640c64f0 2024-09-10T21:24:53.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:53.062+0000 7f485aa00640 1 -- 172.21.15.17:0/469623064 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4850003ac0 con 0x7f48640c64f0 2024-09-10T21:24:53.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:53.062+0000 7f4863e00640 1 -- 172.21.15.17:0/469623064 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4864152ba0 con 0x7f48640c64f0 2024-09-10T21:24:53.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:53.062+0000 7f4863e00640 1 -- 172.21.15.17:0/469623064 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48640cab90 con 0x7f48640c64f0 2024-09-10T21:24:53.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:53.062+0000 7f4863e00640 1 -- 172.21.15.17:0/469623064 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4864068bc0 con 0x7f48640c64f0 2024-09-10T21:24:53.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:53.062+0000 7f485aa00640 1 -- 172.21.15.17:0/469623064 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f4850003c60 con 0x7f48640c64f0 2024-09-10T21:24:53.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:53.066+0000 7f485aa00640 1 --2- 172.21.15.17:0/469623064 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f48440725e0 0x7f4844074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:53.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:53.066+0000 7f485aa00640 1 -- 172.21.15.17:0/469623064 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f48501311a0 con 0x7f48640c64f0 2024-09-10T21:24:53.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:53.066+0000 7f4862a00640 1 --2- 172.21.15.17:0/469623064 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f48440725e0 0x7f4844074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:53.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:53.066+0000 7f4862a00640 1 --2- 172.21.15.17:0/469623064 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f48440725e0 0x7f4844074aa0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f48540058d0 tx=0x7f4854005860 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:53.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:53.070+0000 7f485aa00640 1 -- 172.21.15.17:0/469623064 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f48500c9020 con 0x7f48640c64f0 2024-09-10T21:24:53.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:53.278+0000 7f4863e00640 1 -- 172.21.15.17:0/469623064 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4864069e60 con 0x7f48640c64f0 2024-09-10T21:24:53.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:53.278+0000 7f485aa00640 1 -- 172.21.15.17:0/469623064 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+6862 (secure 0 0 0) 0x7f48500a8e20 con 0x7f48640c64f0 2024-09-10T21:24:53.285 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-10T21:24:53.285 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":5,"btime":"2024-09-10T21:24:52:871404+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4350,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/1643071167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":1643071167},{"type":"v1","addr":"172.21.15.31:6835","nonce":1643071167}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4356,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6837/1666225712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6836","nonce":1666225712},{"type":"v1","addr":"172.21.15.17:6837","nonce":1666225712}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4384,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/3592734043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":3592734043},{"type":"v1","addr":"172.21.15.17:6833","nonce":3592734043}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4405,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/1293665856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":1293665856},{"type":"v1","addr":"172.21.15.17:6835","nonce":1293665856}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4411,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6838/1944852172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":1944852172},{"type":"v1","addr":"172.21.15.31:6838","nonce":1944852172}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":5,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:24:47.255397+0000","modified":"2024-09-10T21:24:52.295961+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4414},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4414":{"gid":4414,"name":"d","rank":0,"incarnation":4,"state":"up:creating","state_seq":1,"addr":"172.21.15.31:6839/3129062106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6837","nonce":3129062106},{"type":"v1","addr":"172.21.15.31:6839","nonce":3129062106}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-09-10T21:24:53.285 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 5 2024-09-10T21:24:53.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:53.282+0000 7f4863e00640 1 -- 172.21.15.17:0/469623064 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f48440725e0 msgr2=0x7f4844074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:53.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:53.282+0000 7f4863e00640 1 --2- 172.21.15.17:0/469623064 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f48440725e0 0x7f4844074aa0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f48540058d0 tx=0x7f4854005860 comp rx=0 tx=0).stop 2024-09-10T21:24:53.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:53.282+0000 7f4863e00640 1 -- 172.21.15.17:0/469623064 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f48640c64f0 msgr2=0x7f4864153360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:53.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:53.282+0000 7f4863e00640 1 --2- 172.21.15.17:0/469623064 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f48640c64f0 0x7f4864153360 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f48500b3040 tx=0x7f4850005790 comp rx=0 tx=0).stop 2024-09-10T21:24:53.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:53.282+0000 7f4863e00640 1 -- 172.21.15.17:0/469623064 shutdown_connections 2024-09-10T21:24:53.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:53.282+0000 7f4863e00640 1 --2- 172.21.15.17:0/469623064 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f48440725e0 0x7f4844074aa0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:53.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:53.282+0000 7f4863e00640 1 --2- 172.21.15.17:0/469623064 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48640c87b0 0x7f486414fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:53.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:53.282+0000 7f4863e00640 1 --2- 172.21.15.17:0/469623064 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f48640c64f0 0x7f4864153360 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:53.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:53.282+0000 7f4863e00640 1 --2- 172.21.15.17:0/469623064 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f48640c6110 0x7f4864145cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:53.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:53.282+0000 7f4863e00640 1 -- 172.21.15.17:0/469623064 >> 172.21.15.17:0/469623064 conn(0x7f486406cdf0 msgr2=0x7f4864147e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:53.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:53.282+0000 7f4863e00640 1 -- 172.21.15.17:0/469623064 shutdown_connections 2024-09-10T21:24:53.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:53.282+0000 7f4863e00640 1 -- 172.21.15.17:0/469623064 wait complete. 2024-09-10T21:24:53.306 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:24:53.453 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:53.450+0000 7fceace00640 1 -- 172.21.15.17:0/1490632967 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcea8152440 msgr2=0x7fcea80b3eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:53.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:53.450+0000 7fceace00640 1 --2- 172.21.15.17:0/1490632967 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcea8152440 0x7fcea80b3eb0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fce98009900 tx=0x7fce9801c850 comp rx=0 tx=0).stop 2024-09-10T21:24:53.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:53.450+0000 7fceace00640 1 -- 172.21.15.17:0/1490632967 shutdown_connections 2024-09-10T21:24:53.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:53.450+0000 7fceace00640 1 --2- 172.21.15.17:0/1490632967 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fcea80cdf20 0x7fcea80ce350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:53.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:53.450+0000 7fceace00640 1 --2- 172.21.15.17:0/1490632967 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcea8152440 0x7fcea80b3eb0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:53.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:53.450+0000 7fceace00640 1 -- 172.21.15.17:0/1490632967 >> 172.21.15.17:0/1490632967 conn(0x7fcea80702f0 msgr2=0x7fcea8070700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:53.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:53.450+0000 7fceace00640 1 -- 172.21.15.17:0/1490632967 shutdown_connections 2024-09-10T21:24:53.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:53.450+0000 7fceace00640 1 -- 172.21.15.17:0/1490632967 wait complete. 2024-09-10T21:24:53.454 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:53.450+0000 7fceace00640 1 Processor -- start 2024-09-10T21:24:53.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:53.450+0000 7fceace00640 1 -- start start 2024-09-10T21:24:53.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:53.450+0000 7fceace00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcea80cdf20 0x7fcea81471b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:53.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:53.450+0000 7fceace00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcea8152440 0x7fcea80cc910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:53.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:53.450+0000 7fceace00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fcea8152820 0x7fcea81462f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:53.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:53.450+0000 7fceace00640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fcea80cb5d0 con 0x7fcea8152440 2024-09-10T21:24:53.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:53.450+0000 7fceace00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fcea80cb450 con 0x7fcea80cdf20 2024-09-10T21:24:53.456 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:53.450+0000 7fceace00640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7fcea80c2ff0 con 0x7fcea8152820 2024-09-10T21:24:53.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:53.450+0000 7fce9fe00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcea8152440 0x7fcea80cc910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:53.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:53.450+0000 7fce9fe00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcea8152440 0x7fcea80cc910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:59136/0 (socket says 172.21.15.17:59136) 2024-09-10T21:24:53.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:53.450+0000 7fce9fe00640 1 -- 172.21.15.17:0/3278234124 learned_addr learned my addr 172.21.15.17:0/3278234124 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:24:53.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:53.450+0000 7fcea6800640 1 --2- 172.21.15.17:0/3278234124 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fcea8152820 0x7fcea81462f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:53.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:53.450+0000 7fcea5e00640 1 --2- 172.21.15.17:0/3278234124 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcea80cdf20 0x7fcea81471b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:53.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:53.454+0000 7fce9fe00640 1 -- 172.21.15.17:0/3278234124 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fcea8152820 msgr2=0x7fcea81462f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:53.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:53.454+0000 7fce9fe00640 1 --2- 172.21.15.17:0/3278234124 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fcea8152820 0x7fcea81462f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:53.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:53.454+0000 7fce9fe00640 1 -- 172.21.15.17:0/3278234124 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcea80cdf20 msgr2=0x7fcea81471b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:53.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:53.454+0000 7fce9fe00640 1 --2- 172.21.15.17:0/3278234124 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcea80cdf20 0x7fcea81471b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:53.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:53.454+0000 7fce9fe00640 1 -- 172.21.15.17:0/3278234124 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcea80c70a0 con 0x7fcea8152440 2024-09-10T21:24:53.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:53.454+0000 7fce9fe00640 1 --2- 172.21.15.17:0/3278234124 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcea8152440 0x7fcea80cc910 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fce9000c500 tx=0x7fce9000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:53.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:53.454+0000 7fce9e000640 1 -- 172.21.15.17:0/3278234124 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce90017070 con 0x7fcea8152440 2024-09-10T21:24:53.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:53.454+0000 7fce9e000640 1 -- 172.21.15.17:0/3278234124 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fce90003600 con 0x7fcea8152440 2024-09-10T21:24:53.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:53.454+0000 7fce9e000640 1 -- 172.21.15.17:0/3278234124 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce90003a90 con 0x7fcea8152440 2024-09-10T21:24:53.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:53.454+0000 7fceace00640 1 -- 172.21.15.17:0/3278234124 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcea8072d70 con 0x7fcea8152440 2024-09-10T21:24:53.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:53.454+0000 7fceace00640 1 -- 172.21.15.17:0/3278234124 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcea80c75f0 con 0x7fcea8152440 2024-09-10T21:24:53.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:53.454+0000 7fceace00640 1 -- 172.21.15.17:0/3278234124 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcea8158360 con 0x7fcea8152440 2024-09-10T21:24:53.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:53.454+0000 7fce9e000640 1 -- 172.21.15.17:0/3278234124 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fce90003c30 con 0x7fcea8152440 2024-09-10T21:24:53.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:53.454+0000 7fce9e000640 1 --2- 172.21.15.17:0/3278234124 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fce840725e0 0x7fce84074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:53.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:53.454+0000 7fce9e000640 1 -- 172.21.15.17:0/3278234124 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fce90092cc0 con 0x7fcea8152440 2024-09-10T21:24:53.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:53.458+0000 7fcea5e00640 1 --2- 172.21.15.17:0/3278234124 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fce840725e0 0x7fce84074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:53.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:53.458+0000 7fcea5e00640 1 --2- 172.21.15.17:0/3278234124 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fce840725e0 0x7fce84074aa0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fce980af000 tx=0x7fce9801b090 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:53.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:53.462+0000 7fce9e000640 1 -- 172.21.15.17:0/3278234124 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fce90099080 con 0x7fcea8152440 2024-09-10T21:24:53.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:53.634+0000 7fceace00640 1 -- 172.21.15.17:0/3278234124 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcea8160ff0 con 0x7fcea8152440 2024-09-10T21:24:53.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:53.638+0000 7fce9e000640 1 -- 172.21.15.17:0/3278234124 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18562 (secure 0 0 0) 0x7fce9005dac0 con 0x7fcea8152440 2024-09-10T21:24:53.641 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-10T21:24:53.641 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":25,"fsid":"5e2f023f-d6a6-49f4-a256-f4807dce3739","created":"2024-09-10T21:24:21.750712+0000","modified":"2024-09-10T21:24:52.295803+0000","last_up_change":"2024-09-10T21:24:35.877801+0000","last_in_change":"2024-09-10T21:24:25.831127+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:24:35.362670+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T21:24:45.211689+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T21:24:46.209784+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T21:24:49.871300+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0f5fabc0-434e-4066-b430-233969a548f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":4105489867},{"type":"v1","addr":"172.21.15.17:6809","nonce":4105489867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":4105489867},{"type":"v1","addr":"172.21.15.17:6811","nonce":4105489867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":4105489867},{"type":"v1","addr":"172.21.15.17:6815","nonce":4105489867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":4105489867},{"type":"v1","addr":"172.21.15.17:6813","nonce":4105489867}]},"public_addr":"172.21.15.17:6809/4105489867","cluster_addr":"172.21.15.17:6811/4105489867","heartbeat_back_addr":"172.21.15.17:6815/4105489867","heartbeat_front_addr":"172.21.15.17:6813/4105489867","state":["exists","up"]},{"osd":1,"uuid":"f6691cb3-2a91-4c90-a9fd-ece3c11dd6eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":2599842027},{"type":"v1","addr":"172.21.15.17:6825","nonce":2599842027}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":2599842027},{"type":"v1","addr":"172.21.15.17:6827","nonce":2599842027}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":2599842027},{"type":"v1","addr":"172.21.15.17:6831","nonce":2599842027}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":2599842027},{"type":"v1","addr":"172.21.15.17:6829","nonce":2599842027}]},"public_addr":"172.21.15.17:6825/2599842027","cluster_addr":"172.21.15.17:6827/2599842027","heartbeat_back_addr":"172.21.15.17:6831/2599842027","heartbeat_front_addr":"172.21.15.17:6829/2599842027","state":["exists","up"]},{"osd":2,"uuid":"cb61adb0-a216-437b-9598-06fa7c84aa64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":1248243634},{"type":"v1","addr":"172.21.15.17:6817","nonce":1248243634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":1248243634},{"type":"v1","addr":"172.21.15.17:6819","nonce":1248243634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":1248243634},{"type":"v1","addr":"172.21.15.17:6823","nonce":1248243634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":1248243634},{"type":"v1","addr":"172.21.15.17:6821","nonce":1248243634}]},"public_addr":"172.21.15.17:6817/1248243634","cluster_addr":"172.21.15.17:6819/1248243634","heartbeat_back_addr":"172.21.15.17:6823/1248243634","heartbeat_front_addr":"172.21.15.17:6821/1248243634","state":["exists","up"]},{"osd":3,"uuid":"6d2da26a-e0bf-47dc-b020-af86e6bcfccc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":2314489522},{"type":"v1","addr":"172.21.15.17:6801","nonce":2314489522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":2314489522},{"type":"v1","addr":"172.21.15.17:6803","nonce":2314489522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":2314489522},{"type":"v1","addr":"172.21.15.17:6807","nonce":2314489522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":2314489522},{"type":"v1","addr":"172.21.15.17:6805","nonce":2314489522}]},"public_addr":"172.21.15.17:6801/2314489522","cluster_addr":"172.21.15.17:6803/2314489522","heartbeat_back_addr":"172.21.15.17:6807/2314489522","heartbeat_front_addr":"172.21.15.17:6805/2314489522","state":["exists","up"]},{"osd":4,"uuid":"c65071dd-9ca7-4f1d-a584-45dc121419a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":202520744},{"type":"v1","addr":"172.21.15.31:6811","nonce":202520744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":202520744},{"type":"v1","addr":"172.21.15.31:6813","nonce":202520744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":202520744},{"type":"v1","addr":"172.21.15.31:6817","nonce":202520744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":202520744},{"type":"v1","addr":"172.21.15.31:6815","nonce":202520744}]},"public_addr":"172.21.15.31:6811/202520744","cluster_addr":"172.21.15.31:6813/202520744","heartbeat_back_addr":"172.21.15.31:6817/202520744","heartbeat_front_addr":"172.21.15.31:6815/202520744","state":["exists","up"]},{"osd":5,"uuid":"235389de-7777-4744-81b8-d0bfc11190e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":1472711057},{"type":"v1","addr":"172.21.15.31:6801","nonce":1472711057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":1472711057},{"type":"v1","addr":"172.21.15.31:6803","nonce":1472711057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":1472711057},{"type":"v1","addr":"172.21.15.31:6807","nonce":1472711057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":1472711057},{"type":"v1","addr":"172.21.15.31:6805","nonce":1472711057}]},"public_addr":"172.21.15.31:6801/1472711057","cluster_addr":"172.21.15.31:6803/1472711057","heartbeat_back_addr":"172.21.15.31:6807/1472711057","heartbeat_front_addr":"172.21.15.31:6805/1472711057","state":["exists","up"]},{"osd":6,"uuid":"cea70a57-3715-4ebb-8511-a528a9df2b1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":3754251089},{"type":"v1","addr":"172.21.15.31:6827","nonce":3754251089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":3754251089},{"type":"v1","addr":"172.21.15.31:6829","nonce":3754251089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":3754251089},{"type":"v1","addr":"172.21.15.31:6833","nonce":3754251089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":3754251089},{"type":"v1","addr":"172.21.15.31:6831","nonce":3754251089}]},"public_addr":"172.21.15.31:6827/3754251089","cluster_addr":"172.21.15.31:6829/3754251089","heartbeat_back_addr":"172.21.15.31:6833/3754251089","heartbeat_front_addr":"172.21.15.31:6831/3754251089","state":["exists","up"]},{"osd":7,"uuid":"01cc111b-a45f-4ca3-9cc1-7fe0afbeba45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":3948648934},{"type":"v1","addr":"172.21.15.31:6819","nonce":3948648934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":3948648934},{"type":"v1","addr":"172.21.15.31:6821","nonce":3948648934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":3948648934},{"type":"v1","addr":"172.21.15.31:6825","nonce":3948648934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":3948648934},{"type":"v1","addr":"172.21.15.31:6823","nonce":3948648934}]},"public_addr":"172.21.15.31:6819/3948648934","cluster_addr":"172.21.15.31:6821/3948648934","heartbeat_back_addr":"172.21.15.31:6825/3948648934","heartbeat_front_addr":"172.21.15.31:6823/3948648934","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:24:32.606989+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:24:32.891535+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:24:33.330793+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:24:32.811533+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:24:33.894624+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:24:32.773817+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:24:32.901865+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:24:32.907207+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:24:53.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:53.638+0000 7fceace00640 1 -- 172.21.15.17:0/3278234124 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fce840725e0 msgr2=0x7fce84074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:53.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:53.638+0000 7fceace00640 1 --2- 172.21.15.17:0/3278234124 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fce840725e0 0x7fce84074aa0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fce980af000 tx=0x7fce9801b090 comp rx=0 tx=0).stop 2024-09-10T21:24:53.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:53.638+0000 7fceace00640 1 -- 172.21.15.17:0/3278234124 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcea8152440 msgr2=0x7fcea80cc910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:53.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:53.638+0000 7fceace00640 1 --2- 172.21.15.17:0/3278234124 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcea8152440 0x7fcea80cc910 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fce9000c500 tx=0x7fce9000c9c0 comp rx=0 tx=0).stop 2024-09-10T21:24:53.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:53.638+0000 7fceace00640 1 -- 172.21.15.17:0/3278234124 shutdown_connections 2024-09-10T21:24:53.644 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:53.638+0000 7fceace00640 1 --2- 172.21.15.17:0/3278234124 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fce840725e0 0x7fce84074aa0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:53.644 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:53.638+0000 7fceace00640 1 --2- 172.21.15.17:0/3278234124 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fcea8152820 0x7fcea81462f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:53.644 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:53.638+0000 7fceace00640 1 --2- 172.21.15.17:0/3278234124 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcea8152440 0x7fcea80cc910 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:53.644 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:53.638+0000 7fceace00640 1 --2- 172.21.15.17:0/3278234124 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcea80cdf20 0x7fcea81471b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:53.644 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:53.638+0000 7fceace00640 1 -- 172.21.15.17:0/3278234124 >> 172.21.15.17:0/3278234124 conn(0x7fcea80702f0 msgr2=0x7fcea80c9a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:53.644 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:53.638+0000 7fceace00640 1 -- 172.21.15.17:0/3278234124 shutdown_connections 2024-09-10T21:24:53.644 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:53.638+0000 7fceace00640 1 -- 172.21.15.17:0/3278234124 wait complete. 2024-09-10T21:24:53.662 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:24:53.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:53.806+0000 7ff8df400640 1 -- 172.21.15.17:0/2336171477 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff8d80725c0 msgr2=0x7ff8d81522c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:53.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:53.806+0000 7ff8df400640 1 --2- 172.21.15.17:0/2336171477 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff8d80725c0 0x7ff8d81522c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff8d0009c80 tx=0x7ff8d001cda0 comp rx=0 tx=0).stop 2024-09-10T21:24:53.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:53.806+0000 7ff8df400640 1 -- 172.21.15.17:0/2336171477 shutdown_connections 2024-09-10T21:24:53.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:53.806+0000 7ff8df400640 1 --2- 172.21.15.17:0/2336171477 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff8d80725c0 0x7ff8d81522c0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:53.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:53.806+0000 7ff8df400640 1 --2- 172.21.15.17:0/2336171477 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff8d80c8d10 0x7ff8d80c6c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:53.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:53.806+0000 7ff8df400640 1 -- 172.21.15.17:0/2336171477 >> 172.21.15.17:0/2336171477 conn(0x7ff8d8070160 msgr2=0x7ff8d8070570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:53.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:53.806+0000 7ff8df400640 1 -- 172.21.15.17:0/2336171477 shutdown_connections 2024-09-10T21:24:53.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:53.806+0000 7ff8df400640 1 -- 172.21.15.17:0/2336171477 wait complete. 2024-09-10T21:24:53.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:53.806+0000 7ff8df400640 1 Processor -- start 2024-09-10T21:24:53.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:53.806+0000 7ff8df400640 1 -- start start 2024-09-10T21:24:53.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:53.806+0000 7ff8df400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff8d80725c0 0x7ff8d814b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:53.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:53.806+0000 7ff8df400640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff8d80c8d10 0x7ff8d813f340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:53.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:53.806+0000 7ff8df400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff8d80c90f0 0x7ff8d81f3c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:53.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:53.806+0000 7ff8df400640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7ff8d806ea00 con 0x7ff8d80c90f0 2024-09-10T21:24:53.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:53.806+0000 7ff8df400640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7ff8d806eb80 con 0x7ff8d80725c0 2024-09-10T21:24:53.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:53.806+0000 7ff8df400640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7ff8d81655b0 con 0x7ff8d80c8d10 2024-09-10T21:24:53.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:53.806+0000 7ff8dd400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff8d80c90f0 0x7ff8d81f3c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:53.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:53.806+0000 7ff8dd400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff8d80c90f0 0x7ff8d81f3c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:59154/0 (socket says 172.21.15.17:59154) 2024-09-10T21:24:53.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:53.806+0000 7ff8dd400640 1 -- 172.21.15.17:0/2774966360 learned_addr learned my addr 172.21.15.17:0/2774966360 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:24:53.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:53.810+0000 7ff8dca00640 1 --2- 172.21.15.17:0/2774966360 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff8d80725c0 0x7ff8d814b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:53.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:53.810+0000 7ff8cfe00640 1 --2- 172.21.15.17:0/2774966360 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff8d80c8d10 0x7ff8d813f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:53.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:53.810+0000 7ff8dd400640 1 -- 172.21.15.17:0/2774966360 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff8d80c8d10 msgr2=0x7ff8d813f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:53.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:53.810+0000 7ff8dd400640 1 --2- 172.21.15.17:0/2774966360 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff8d80c8d10 0x7ff8d813f340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:53.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:53.810+0000 7ff8dd400640 1 -- 172.21.15.17:0/2774966360 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff8d80725c0 msgr2=0x7ff8d814b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:53.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:53.810+0000 7ff8dd400640 1 --2- 172.21.15.17:0/2774966360 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff8d80725c0 0x7ff8d814b930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:53.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:53.810+0000 7ff8dd400640 1 -- 172.21.15.17:0/2774966360 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8d813fff0 con 0x7ff8d80c90f0 2024-09-10T21:24:53.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:53.810+0000 7ff8dd400640 1 --2- 172.21.15.17:0/2774966360 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff8d80c90f0 0x7ff8d81f3c20 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ff8d001b450 tx=0x7ff8d00092e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:53.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:53.810+0000 7ff8d5600640 1 -- 172.21.15.17:0/2774966360 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8d00bb070 con 0x7ff8d80c90f0 2024-09-10T21:24:53.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:53.810+0000 7ff8d5600640 1 -- 172.21.15.17:0/2774966360 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff8d0004c50 con 0x7ff8d80c90f0 2024-09-10T21:24:53.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:53.810+0000 7ff8d5600640 1 -- 172.21.15.17:0/2774966360 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8d00b2040 con 0x7ff8d80c90f0 2024-09-10T21:24:53.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:53.810+0000 7ff8df400640 1 -- 172.21.15.17:0/2774966360 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8d81454b0 con 0x7ff8d80c90f0 2024-09-10T21:24:53.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:53.810+0000 7ff8df400640 1 -- 172.21.15.17:0/2774966360 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8d8145750 con 0x7ff8d80c90f0 2024-09-10T21:24:53.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:53.810+0000 7ff8df400640 1 -- 172.21.15.17:0/2774966360 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8d81522c0 con 0x7ff8d80c90f0 2024-09-10T21:24:53.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:53.810+0000 7ff8d5600640 1 -- 172.21.15.17:0/2774966360 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7ff8d00b5030 con 0x7ff8d80c90f0 2024-09-10T21:24:53.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:53.810+0000 7ff8d5600640 1 --2- 172.21.15.17:0/2774966360 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7ff8b40725e0 0x7ff8b4074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:53.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:53.814+0000 7ff8d5600640 1 -- 172.21.15.17:0/2774966360 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7ff8d012d020 con 0x7ff8d80c90f0 2024-09-10T21:24:53.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:53.814+0000 7ff8dca00640 1 --2- 172.21.15.17:0/2774966360 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7ff8b40725e0 0x7ff8b4074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:53.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:53.814+0000 7ff8dca00640 1 --2- 172.21.15.17:0/2774966360 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7ff8b40725e0 0x7ff8b4074aa0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff8c80058d0 tx=0x7ff8c8005860 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:53.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:53.818+0000 7ff8d5600640 1 -- 172.21.15.17:0/2774966360 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7ff8d0016990 con 0x7ff8d80c90f0 2024-09-10T21:24:53.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:53.994+0000 7ff8df400640 1 -- 172.21.15.17:0/2774966360 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff8d80c7ba0 con 0x7ff8d80c90f0 2024-09-10T21:24:53.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:53.994+0000 7ff8d5600640 1 -- 172.21.15.17:0/2774966360 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18562 (secure 0 0 0) 0x7ff8d00f8330 con 0x7ff8d80c90f0 2024-09-10T21:24:54.000 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-10T21:24:54.000 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":25,"fsid":"5e2f023f-d6a6-49f4-a256-f4807dce3739","created":"2024-09-10T21:24:21.750712+0000","modified":"2024-09-10T21:24:52.295803+0000","last_up_change":"2024-09-10T21:24:35.877801+0000","last_in_change":"2024-09-10T21:24:25.831127+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:24:35.362670+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T21:24:45.211689+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T21:24:46.209784+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T21:24:49.871300+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0f5fabc0-434e-4066-b430-233969a548f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":4105489867},{"type":"v1","addr":"172.21.15.17:6809","nonce":4105489867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":4105489867},{"type":"v1","addr":"172.21.15.17:6811","nonce":4105489867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":4105489867},{"type":"v1","addr":"172.21.15.17:6815","nonce":4105489867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":4105489867},{"type":"v1","addr":"172.21.15.17:6813","nonce":4105489867}]},"public_addr":"172.21.15.17:6809/4105489867","cluster_addr":"172.21.15.17:6811/4105489867","heartbeat_back_addr":"172.21.15.17:6815/4105489867","heartbeat_front_addr":"172.21.15.17:6813/4105489867","state":["exists","up"]},{"osd":1,"uuid":"f6691cb3-2a91-4c90-a9fd-ece3c11dd6eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":2599842027},{"type":"v1","addr":"172.21.15.17:6825","nonce":2599842027}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":2599842027},{"type":"v1","addr":"172.21.15.17:6827","nonce":2599842027}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":2599842027},{"type":"v1","addr":"172.21.15.17:6831","nonce":2599842027}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":2599842027},{"type":"v1","addr":"172.21.15.17:6829","nonce":2599842027}]},"public_addr":"172.21.15.17:6825/2599842027","cluster_addr":"172.21.15.17:6827/2599842027","heartbeat_back_addr":"172.21.15.17:6831/2599842027","heartbeat_front_addr":"172.21.15.17:6829/2599842027","state":["exists","up"]},{"osd":2,"uuid":"cb61adb0-a216-437b-9598-06fa7c84aa64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":1248243634},{"type":"v1","addr":"172.21.15.17:6817","nonce":1248243634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":1248243634},{"type":"v1","addr":"172.21.15.17:6819","nonce":1248243634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":1248243634},{"type":"v1","addr":"172.21.15.17:6823","nonce":1248243634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":1248243634},{"type":"v1","addr":"172.21.15.17:6821","nonce":1248243634}]},"public_addr":"172.21.15.17:6817/1248243634","cluster_addr":"172.21.15.17:6819/1248243634","heartbeat_back_addr":"172.21.15.17:6823/1248243634","heartbeat_front_addr":"172.21.15.17:6821/1248243634","state":["exists","up"]},{"osd":3,"uuid":"6d2da26a-e0bf-47dc-b020-af86e6bcfccc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":2314489522},{"type":"v1","addr":"172.21.15.17:6801","nonce":2314489522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":2314489522},{"type":"v1","addr":"172.21.15.17:6803","nonce":2314489522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":2314489522},{"type":"v1","addr":"172.21.15.17:6807","nonce":2314489522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":2314489522},{"type":"v1","addr":"172.21.15.17:6805","nonce":2314489522}]},"public_addr":"172.21.15.17:6801/2314489522","cluster_addr":"172.21.15.17:6803/2314489522","heartbeat_back_addr":"172.21.15.17:6807/2314489522","heartbeat_front_addr":"172.21.15.17:6805/2314489522","state":["exists","up"]},{"osd":4,"uuid":"c65071dd-9ca7-4f1d-a584-45dc121419a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":202520744},{"type":"v1","addr":"172.21.15.31:6811","nonce":202520744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":202520744},{"type":"v1","addr":"172.21.15.31:6813","nonce":202520744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":202520744},{"type":"v1","addr":"172.21.15.31:6817","nonce":202520744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":202520744},{"type":"v1","addr":"172.21.15.31:6815","nonce":202520744}]},"public_addr":"172.21.15.31:6811/202520744","cluster_addr":"172.21.15.31:6813/202520744","heartbeat_back_addr":"172.21.15.31:6817/202520744","heartbeat_front_addr":"172.21.15.31:6815/202520744","state":["exists","up"]},{"osd":5,"uuid":"235389de-7777-4744-81b8-d0bfc11190e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":1472711057},{"type":"v1","addr":"172.21.15.31:6801","nonce":1472711057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":1472711057},{"type":"v1","addr":"172.21.15.31:6803","nonce":1472711057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":1472711057},{"type":"v1","addr":"172.21.15.31:6807","nonce":1472711057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":1472711057},{"type":"v1","addr":"172.21.15.31:6805","nonce":1472711057}]},"public_addr":"172.21.15.31:6801/1472711057","cluster_addr":"172.21.15.31:6803/1472711057","heartbeat_back_addr":"172.21.15.31:6807/1472711057","heartbeat_front_addr":"172.21.15.31:6805/1472711057","state":["exists","up"]},{"osd":6,"uuid":"cea70a57-3715-4ebb-8511-a528a9df2b1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":3754251089},{"type":"v1","addr":"172.21.15.31:6827","nonce":3754251089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":3754251089},{"type":"v1","addr":"172.21.15.31:6829","nonce":3754251089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":3754251089},{"type":"v1","addr":"172.21.15.31:6833","nonce":3754251089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":3754251089},{"type":"v1","addr":"172.21.15.31:6831","nonce":3754251089}]},"public_addr":"172.21.15.31:6827/3754251089","cluster_addr":"172.21.15.31:6829/3754251089","heartbeat_back_addr":"172.21.15.31:6833/3754251089","heartbeat_front_addr":"172.21.15.31:6831/3754251089","state":["exists","up"]},{"osd":7,"uuid":"01cc111b-a45f-4ca3-9cc1-7fe0afbeba45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":3948648934},{"type":"v1","addr":"172.21.15.31:6819","nonce":3948648934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":3948648934},{"type":"v1","addr":"172.21.15.31:6821","nonce":3948648934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":3948648934},{"type":"v1","addr":"172.21.15.31:6825","nonce":3948648934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":3948648934},{"type":"v1","addr":"172.21.15.31:6823","nonce":3948648934}]},"public_addr":"172.21.15.31:6819/3948648934","cluster_addr":"172.21.15.31:6821/3948648934","heartbeat_back_addr":"172.21.15.31:6825/3948648934","heartbeat_front_addr":"172.21.15.31:6823/3948648934","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:24:32.606989+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:24:32.891535+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:24:33.330793+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:24:32.811533+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:24:33.894624+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:24:32.773817+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:24:32.901865+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:24:32.907207+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:24:54.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:53.998+0000 7ff8df400640 1 -- 172.21.15.17:0/2774966360 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7ff8b40725e0 msgr2=0x7ff8b4074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:54.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:53.998+0000 7ff8df400640 1 --2- 172.21.15.17:0/2774966360 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7ff8b40725e0 0x7ff8b4074aa0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff8c80058d0 tx=0x7ff8c8005860 comp rx=0 tx=0).stop 2024-09-10T21:24:54.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:53.998+0000 7ff8df400640 1 -- 172.21.15.17:0/2774966360 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff8d80c90f0 msgr2=0x7ff8d81f3c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:54.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:53.998+0000 7ff8df400640 1 --2- 172.21.15.17:0/2774966360 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff8d80c90f0 0x7ff8d81f3c20 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ff8d001b450 tx=0x7ff8d00092e0 comp rx=0 tx=0).stop 2024-09-10T21:24:54.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:53.998+0000 7ff8df400640 1 -- 172.21.15.17:0/2774966360 shutdown_connections 2024-09-10T21:24:54.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:53.998+0000 7ff8df400640 1 --2- 172.21.15.17:0/2774966360 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7ff8b40725e0 0x7ff8b4074aa0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:54.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:53.998+0000 7ff8df400640 1 --2- 172.21.15.17:0/2774966360 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff8d80c90f0 0x7ff8d81f3c20 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:54.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:53.998+0000 7ff8df400640 1 --2- 172.21.15.17:0/2774966360 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff8d80c8d10 0x7ff8d813f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:54.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:53.998+0000 7ff8df400640 1 --2- 172.21.15.17:0/2774966360 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff8d80725c0 0x7ff8d814b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:54.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:53.998+0000 7ff8df400640 1 -- 172.21.15.17:0/2774966360 >> 172.21.15.17:0/2774966360 conn(0x7ff8d8070160 msgr2=0x7ff8d814c790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:54.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:53.998+0000 7ff8df400640 1 -- 172.21.15.17:0/2774966360 shutdown_connections 2024-09-10T21:24:54.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:53.998+0000 7ff8df400640 1 -- 172.21.15.17:0/2774966360 wait complete. 2024-09-10T21:24:54.039 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:24:54.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:54.182+0000 7f8917c00640 1 -- 172.21.15.17:0/197237801 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8910152440 msgr2=0x7f89100b3eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:54.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:54.182+0000 7f8917c00640 1 --2- 172.21.15.17:0/197237801 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8910152440 0x7f89100b3eb0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f8904009900 tx=0x7f890401c850 comp rx=0 tx=0).stop 2024-09-10T21:24:54.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:54.182+0000 7f8917c00640 1 -- 172.21.15.17:0/197237801 shutdown_connections 2024-09-10T21:24:54.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:54.182+0000 7f8917c00640 1 --2- 172.21.15.17:0/197237801 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f89100cdf20 0x7f89100ce350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:54.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:54.182+0000 7f8917c00640 1 --2- 172.21.15.17:0/197237801 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8910152440 0x7f89100b3eb0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:54.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:54.182+0000 7f8917c00640 1 -- 172.21.15.17:0/197237801 >> 172.21.15.17:0/197237801 conn(0x7f89100702f0 msgr2=0x7f8910070700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:54.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:54.182+0000 7f8917c00640 1 -- 172.21.15.17:0/197237801 shutdown_connections 2024-09-10T21:24:54.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:54.182+0000 7f8917c00640 1 -- 172.21.15.17:0/197237801 wait complete. 2024-09-10T21:24:54.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:54.182+0000 7f8917c00640 1 Processor -- start 2024-09-10T21:24:54.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:54.182+0000 7f8917c00640 1 -- start start 2024-09-10T21:24:54.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:54.182+0000 7f8917c00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f89100cdf20 0x7f89101ea470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:54.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:54.182+0000 7f8917c00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8910152440 0x7f89101f3a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:54.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:54.182+0000 7f8917c00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8910152820 0x7f89101f3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:54.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:54.182+0000 7f8917c00640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f89100cb5d0 con 0x7f8910152440 2024-09-10T21:24:54.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:54.182+0000 7f8917c00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f89100cb450 con 0x7f89100cdf20 2024-09-10T21:24:54.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:54.182+0000 7f8917c00640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f89100c2ff0 con 0x7f8910152820 2024-09-10T21:24:54.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:54.182+0000 7f890fe00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8910152440 0x7f89101f3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:54.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:54.182+0000 7f890fe00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8910152440 0x7f89101f3a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:59162/0 (socket says 172.21.15.17:59162) 2024-09-10T21:24:54.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:54.182+0000 7f890fe00640 1 -- 172.21.15.17:0/1759334475 learned_addr learned my addr 172.21.15.17:0/1759334475 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:24:54.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:54.182+0000 7f8915200640 1 --2- 172.21.15.17:0/1759334475 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f89100cdf20 0x7f89101ea470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:54.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:54.182+0000 7f8915c00640 1 --2- 172.21.15.17:0/1759334475 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8910152820 0x7f89101f3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:54.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:54.186+0000 7f890fe00640 1 -- 172.21.15.17:0/1759334475 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8910152820 msgr2=0x7f89101f3fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:54.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:54.186+0000 7f890fe00640 1 --2- 172.21.15.17:0/1759334475 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8910152820 0x7f89101f3fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:54.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:54.186+0000 7f890fe00640 1 -- 172.21.15.17:0/1759334475 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f89100cdf20 msgr2=0x7f89101ea470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:54.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:54.186+0000 7f890fe00640 1 --2- 172.21.15.17:0/1759334475 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f89100cdf20 0x7f89101ea470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:54.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:54.186+0000 7f890fe00640 1 -- 172.21.15.17:0/1759334475 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8910162ab0 con 0x7f8910152440 2024-09-10T21:24:54.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:54.186+0000 7f890fe00640 1 --2- 172.21.15.17:0/1759334475 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8910152440 0x7f89101f3a80 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f88fc00c450 tx=0x7f88fc00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:54.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:54.186+0000 7f890cc00640 1 -- 172.21.15.17:0/1759334475 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88fc012830 con 0x7f8910152440 2024-09-10T21:24:54.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:54.186+0000 7f890cc00640 1 -- 172.21.15.17:0/1759334475 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f88fc017070 con 0x7f8910152440 2024-09-10T21:24:54.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:54.186+0000 7f890cc00640 1 -- 172.21.15.17:0/1759334475 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88fc012d80 con 0x7f8910152440 2024-09-10T21:24:54.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:54.186+0000 7f8917c00640 1 -- 172.21.15.17:0/1759334475 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89101f6ff0 con 0x7f8910152440 2024-09-10T21:24:54.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:54.186+0000 7f8917c00640 1 -- 172.21.15.17:0/1759334475 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89101f75b0 con 0x7f8910152440 2024-09-10T21:24:54.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:54.186+0000 7f8917c00640 1 -- 172.21.15.17:0/1759334475 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8910158360 con 0x7f8910152440 2024-09-10T21:24:54.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:54.186+0000 7f890cc00640 1 -- 172.21.15.17:0/1759334475 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f88fc01c460 con 0x7f8910152440 2024-09-10T21:24:54.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:54.186+0000 7f890cc00640 1 --2- 172.21.15.17:0/1759334475 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f88f00725e0 0x7f88f0074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:54.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:54.186+0000 7f890cc00640 1 -- 172.21.15.17:0/1759334475 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f88fc093270 con 0x7f8910152440 2024-09-10T21:24:54.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:54.190+0000 7f8915200640 1 --2- 172.21.15.17:0/1759334475 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f88f00725e0 0x7f88f0074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:54.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:54.190+0000 7f8915200640 1 --2- 172.21.15.17:0/1759334475 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f88f00725e0 0x7f88f0074aa0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f89040af000 tx=0x7f890401b090 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:54.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:54.190+0000 7f890cc00640 1 -- 172.21.15.17:0/1759334475 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f88fc05df20 con 0x7f8910152440 2024-09-10T21:24:54.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:54.402+0000 7f8917c00640 1 -- 172.21.15.17:0/1759334475 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f89100c9e90 con 0x7f8910152440 2024-09-10T21:24:54.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:54.402+0000 7f890cc00640 1 -- 172.21.15.17:0/1759334475 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+6862 (secure 0 0 0) 0x7f88fc062950 con 0x7f8910152440 2024-09-10T21:24:54.406 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-10T21:24:54.406 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":5,"btime":"2024-09-10T21:24:52:871404+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4350,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/1643071167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":1643071167},{"type":"v1","addr":"172.21.15.31:6835","nonce":1643071167}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4356,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6837/1666225712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6836","nonce":1666225712},{"type":"v1","addr":"172.21.15.17:6837","nonce":1666225712}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4384,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/3592734043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":3592734043},{"type":"v1","addr":"172.21.15.17:6833","nonce":3592734043}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4405,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/1293665856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":1293665856},{"type":"v1","addr":"172.21.15.17:6835","nonce":1293665856}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4411,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6838/1944852172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":1944852172},{"type":"v1","addr":"172.21.15.31:6838","nonce":1944852172}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":5,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:24:47.255397+0000","modified":"2024-09-10T21:24:52.295961+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4414},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4414":{"gid":4414,"name":"d","rank":0,"incarnation":4,"state":"up:creating","state_seq":1,"addr":"172.21.15.31:6839/3129062106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6837","nonce":3129062106},{"type":"v1","addr":"172.21.15.31:6839","nonce":3129062106}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-09-10T21:24:54.407 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 5 2024-09-10T21:24:54.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:54.406+0000 7f8917c00640 1 -- 172.21.15.17:0/1759334475 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f88f00725e0 msgr2=0x7f88f0074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:54.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:54.406+0000 7f8917c00640 1 --2- 172.21.15.17:0/1759334475 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f88f00725e0 0x7f88f0074aa0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f89040af000 tx=0x7f890401b090 comp rx=0 tx=0).stop 2024-09-10T21:24:54.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:54.406+0000 7f8917c00640 1 -- 172.21.15.17:0/1759334475 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8910152440 msgr2=0x7f89101f3a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:54.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:54.406+0000 7f8917c00640 1 --2- 172.21.15.17:0/1759334475 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8910152440 0x7f89101f3a80 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f88fc00c450 tx=0x7f88fc00c910 comp rx=0 tx=0).stop 2024-09-10T21:24:54.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:54.406+0000 7f8917c00640 1 -- 172.21.15.17:0/1759334475 shutdown_connections 2024-09-10T21:24:54.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:54.406+0000 7f8917c00640 1 --2- 172.21.15.17:0/1759334475 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f88f00725e0 0x7f88f0074aa0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:54.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:54.406+0000 7f8917c00640 1 --2- 172.21.15.17:0/1759334475 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8910152820 0x7f89101f3fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:54.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:54.406+0000 7f8917c00640 1 --2- 172.21.15.17:0/1759334475 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8910152440 0x7f89101f3a80 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:54.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:54.406+0000 7f8917c00640 1 --2- 172.21.15.17:0/1759334475 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f89100cdf20 0x7f89101ea470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:54.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:54.406+0000 7f8917c00640 1 -- 172.21.15.17:0/1759334475 >> 172.21.15.17:0/1759334475 conn(0x7f89100702f0 msgr2=0x7f89100ca3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:54.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:54.406+0000 7f8917c00640 1 -- 172.21.15.17:0/1759334475 shutdown_connections 2024-09-10T21:24:54.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:54.406+0000 7f8917c00640 1 -- 172.21.15.17:0/1759334475 wait complete. 2024-09-10T21:24:54.427 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:24:54.574 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:54.570+0000 7fb91d600640 1 -- 172.21.15.17:0/3924430462 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb9180b3eb0 msgr2=0x7fb918163cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:54.574 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:54.570+0000 7fb91d600640 1 --2- 172.21.15.17:0/3924430462 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb9180b3eb0 0x7fb918163cf0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fb9181594a0 tx=0x7fb91001cae0 comp rx=0 tx=0).stop 2024-09-10T21:24:54.575 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:54.570+0000 7fb91d600640 1 -- 172.21.15.17:0/3924430462 shutdown_connections 2024-09-10T21:24:54.575 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:54.570+0000 7fb91d600640 1 --2- 172.21.15.17:0/3924430462 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb9180b3eb0 0x7fb918163cf0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:54.575 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:54.570+0000 7fb91d600640 1 -- 172.21.15.17:0/3924430462 >> 172.21.15.17:0/3924430462 conn(0x7fb9180702f0 msgr2=0x7fb918070700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:54.575 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:54.570+0000 7fb91d600640 1 -- 172.21.15.17:0/3924430462 shutdown_connections 2024-09-10T21:24:54.575 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:54.570+0000 7fb91d600640 1 -- 172.21.15.17:0/3924430462 wait complete. 2024-09-10T21:24:54.575 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:54.570+0000 7fb91d600640 1 Processor -- start 2024-09-10T21:24:54.576 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:54.570+0000 7fb91d600640 1 -- start start 2024-09-10T21:24:54.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:54.574+0000 7fb91d600640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fb9180b3eb0 0x7fb918154800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:54.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:54.574+0000 7fb91d600640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb918152440 0x7fb918157960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:54.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:54.574+0000 7fb91d600640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb918152820 0x7fb91815f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:54.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:54.574+0000 7fb91d600640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fb9180ce630 con 0x7fb918152820 2024-09-10T21:24:54.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:54.574+0000 7fb91d600640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fb9180b42e0 con 0x7fb918152440 2024-09-10T21:24:54.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:54.574+0000 7fb91d600640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7fb9180ce7b0 con 0x7fb9180b3eb0 2024-09-10T21:24:54.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:54.574+0000 7fb917400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb918152820 0x7fb91815f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:54.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:54.574+0000 7fb917400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb918152820 0x7fb91815f560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:59178/0 (socket says 172.21.15.17:59178) 2024-09-10T21:24:54.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:54.574+0000 7fb917400640 1 -- 172.21.15.17:0/3016564130 learned_addr learned my addr 172.21.15.17:0/3016564130 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:24:54.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:54.574+0000 7fb90e000640 1 --2- 172.21.15.17:0/3016564130 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb918152440 0x7fb918157960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:54.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:54.574+0000 7fb916a00640 1 --2- 172.21.15.17:0/3016564130 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fb9180b3eb0 0x7fb918154800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:54.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:54.574+0000 7fb917400640 1 -- 172.21.15.17:0/3016564130 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fb9180b3eb0 msgr2=0x7fb918154800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:54.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:54.574+0000 7fb917400640 1 --2- 172.21.15.17:0/3016564130 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fb9180b3eb0 0x7fb918154800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:54.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:54.574+0000 7fb917400640 1 -- 172.21.15.17:0/3016564130 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb918152440 msgr2=0x7fb918157960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:54.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:54.574+0000 7fb917400640 1 --2- 172.21.15.17:0/3016564130 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb918152440 0x7fb918157960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:54.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:54.574+0000 7fb917400640 1 -- 172.21.15.17:0/3016564130 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb918154f00 con 0x7fb918152820 2024-09-10T21:24:54.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:54.574+0000 7fb917400640 1 --2- 172.21.15.17:0/3016564130 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb918152820 0x7fb91815f560 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fb9100196f0 tx=0x7fb9100a5b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:54.580 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:54.574+0000 7fb90f400640 1 -- 172.21.15.17:0/3016564130 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9100b2070 con 0x7fb918152820 2024-09-10T21:24:54.580 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:54.574+0000 7fb90f400640 1 -- 172.21.15.17:0/3016564130 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb9100bc070 con 0x7fb918152820 2024-09-10T21:24:54.580 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:54.574+0000 7fb91d600640 1 -- 172.21.15.17:0/3016564130 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9181551e0 con 0x7fb918152820 2024-09-10T21:24:54.580 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:54.574+0000 7fb90f400640 1 -- 172.21.15.17:0/3016564130 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9100a6e60 con 0x7fb918152820 2024-09-10T21:24:54.580 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:54.574+0000 7fb91d600640 1 -- 172.21.15.17:0/3016564130 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb918153ae0 con 0x7fb918152820 2024-09-10T21:24:54.580 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:54.574+0000 7fb91d600640 1 -- 172.21.15.17:0/3016564130 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb918153ce0 con 0x7fb918152820 2024-09-10T21:24:54.580 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:54.578+0000 7fb90f400640 1 -- 172.21.15.17:0/3016564130 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fb9100056f0 con 0x7fb918152820 2024-09-10T21:24:54.582 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:54.578+0000 7fb90f400640 1 --2- 172.21.15.17:0/3016564130 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fb8f40725e0 0x7fb8f4074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:54.582 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:54.578+0000 7fb90f400640 1 -- 172.21.15.17:0/3016564130 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fb91012dbf0 con 0x7fb918152820 2024-09-10T21:24:54.582 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:54.578+0000 7fb916a00640 1 --2- 172.21.15.17:0/3016564130 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fb8f40725e0 0x7fb8f4074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:54.582 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:54.578+0000 7fb916a00640 1 --2- 172.21.15.17:0/3016564130 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fb8f40725e0 0x7fb8f4074aa0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb908007fd0 tx=0x7fb908009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:54.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:54.578+0000 7fb90f400640 1 -- 172.21.15.17:0/3016564130 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fb918153ce0 con 0x7fb918152820 2024-09-10T21:24:54.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:54.770+0000 7fb91d600640 1 -- 172.21.15.17:0/3016564130 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb9180c2ff0 con 0x7fb918152820 2024-09-10T21:24:54.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:54.770+0000 7fb90f400640 1 -- 172.21.15.17:0/3016564130 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18562 (secure 0 0 0) 0x7fb910016790 con 0x7fb918152820 2024-09-10T21:24:54.776 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-10T21:24:54.776 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":25,"fsid":"5e2f023f-d6a6-49f4-a256-f4807dce3739","created":"2024-09-10T21:24:21.750712+0000","modified":"2024-09-10T21:24:52.295803+0000","last_up_change":"2024-09-10T21:24:35.877801+0000","last_in_change":"2024-09-10T21:24:25.831127+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:24:35.362670+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T21:24:45.211689+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T21:24:46.209784+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T21:24:49.871300+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0f5fabc0-434e-4066-b430-233969a548f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":4105489867},{"type":"v1","addr":"172.21.15.17:6809","nonce":4105489867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":4105489867},{"type":"v1","addr":"172.21.15.17:6811","nonce":4105489867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":4105489867},{"type":"v1","addr":"172.21.15.17:6815","nonce":4105489867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":4105489867},{"type":"v1","addr":"172.21.15.17:6813","nonce":4105489867}]},"public_addr":"172.21.15.17:6809/4105489867","cluster_addr":"172.21.15.17:6811/4105489867","heartbeat_back_addr":"172.21.15.17:6815/4105489867","heartbeat_front_addr":"172.21.15.17:6813/4105489867","state":["exists","up"]},{"osd":1,"uuid":"f6691cb3-2a91-4c90-a9fd-ece3c11dd6eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":2599842027},{"type":"v1","addr":"172.21.15.17:6825","nonce":2599842027}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":2599842027},{"type":"v1","addr":"172.21.15.17:6827","nonce":2599842027}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":2599842027},{"type":"v1","addr":"172.21.15.17:6831","nonce":2599842027}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":2599842027},{"type":"v1","addr":"172.21.15.17:6829","nonce":2599842027}]},"public_addr":"172.21.15.17:6825/2599842027","cluster_addr":"172.21.15.17:6827/2599842027","heartbeat_back_addr":"172.21.15.17:6831/2599842027","heartbeat_front_addr":"172.21.15.17:6829/2599842027","state":["exists","up"]},{"osd":2,"uuid":"cb61adb0-a216-437b-9598-06fa7c84aa64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":1248243634},{"type":"v1","addr":"172.21.15.17:6817","nonce":1248243634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":1248243634},{"type":"v1","addr":"172.21.15.17:6819","nonce":1248243634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":1248243634},{"type":"v1","addr":"172.21.15.17:6823","nonce":1248243634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":1248243634},{"type":"v1","addr":"172.21.15.17:6821","nonce":1248243634}]},"public_addr":"172.21.15.17:6817/1248243634","cluster_addr":"172.21.15.17:6819/1248243634","heartbeat_back_addr":"172.21.15.17:6823/1248243634","heartbeat_front_addr":"172.21.15.17:6821/1248243634","state":["exists","up"]},{"osd":3,"uuid":"6d2da26a-e0bf-47dc-b020-af86e6bcfccc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":2314489522},{"type":"v1","addr":"172.21.15.17:6801","nonce":2314489522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":2314489522},{"type":"v1","addr":"172.21.15.17:6803","nonce":2314489522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":2314489522},{"type":"v1","addr":"172.21.15.17:6807","nonce":2314489522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":2314489522},{"type":"v1","addr":"172.21.15.17:6805","nonce":2314489522}]},"public_addr":"172.21.15.17:6801/2314489522","cluster_addr":"172.21.15.17:6803/2314489522","heartbeat_back_addr":"172.21.15.17:6807/2314489522","heartbeat_front_addr":"172.21.15.17:6805/2314489522","state":["exists","up"]},{"osd":4,"uuid":"c65071dd-9ca7-4f1d-a584-45dc121419a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":202520744},{"type":"v1","addr":"172.21.15.31:6811","nonce":202520744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":202520744},{"type":"v1","addr":"172.21.15.31:6813","nonce":202520744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":202520744},{"type":"v1","addr":"172.21.15.31:6817","nonce":202520744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":202520744},{"type":"v1","addr":"172.21.15.31:6815","nonce":202520744}]},"public_addr":"172.21.15.31:6811/202520744","cluster_addr":"172.21.15.31:6813/202520744","heartbeat_back_addr":"172.21.15.31:6817/202520744","heartbeat_front_addr":"172.21.15.31:6815/202520744","state":["exists","up"]},{"osd":5,"uuid":"235389de-7777-4744-81b8-d0bfc11190e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":1472711057},{"type":"v1","addr":"172.21.15.31:6801","nonce":1472711057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":1472711057},{"type":"v1","addr":"172.21.15.31:6803","nonce":1472711057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":1472711057},{"type":"v1","addr":"172.21.15.31:6807","nonce":1472711057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":1472711057},{"type":"v1","addr":"172.21.15.31:6805","nonce":1472711057}]},"public_addr":"172.21.15.31:6801/1472711057","cluster_addr":"172.21.15.31:6803/1472711057","heartbeat_back_addr":"172.21.15.31:6807/1472711057","heartbeat_front_addr":"172.21.15.31:6805/1472711057","state":["exists","up"]},{"osd":6,"uuid":"cea70a57-3715-4ebb-8511-a528a9df2b1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":3754251089},{"type":"v1","addr":"172.21.15.31:6827","nonce":3754251089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":3754251089},{"type":"v1","addr":"172.21.15.31:6829","nonce":3754251089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":3754251089},{"type":"v1","addr":"172.21.15.31:6833","nonce":3754251089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":3754251089},{"type":"v1","addr":"172.21.15.31:6831","nonce":3754251089}]},"public_addr":"172.21.15.31:6827/3754251089","cluster_addr":"172.21.15.31:6829/3754251089","heartbeat_back_addr":"172.21.15.31:6833/3754251089","heartbeat_front_addr":"172.21.15.31:6831/3754251089","state":["exists","up"]},{"osd":7,"uuid":"01cc111b-a45f-4ca3-9cc1-7fe0afbeba45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":3948648934},{"type":"v1","addr":"172.21.15.31:6819","nonce":3948648934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":3948648934},{"type":"v1","addr":"172.21.15.31:6821","nonce":3948648934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":3948648934},{"type":"v1","addr":"172.21.15.31:6825","nonce":3948648934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":3948648934},{"type":"v1","addr":"172.21.15.31:6823","nonce":3948648934}]},"public_addr":"172.21.15.31:6819/3948648934","cluster_addr":"172.21.15.31:6821/3948648934","heartbeat_back_addr":"172.21.15.31:6825/3948648934","heartbeat_front_addr":"172.21.15.31:6823/3948648934","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:24:32.606989+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:24:32.891535+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:24:33.330793+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:24:32.811533+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:24:33.894624+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:24:32.773817+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:24:32.901865+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:24:32.907207+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:24:54.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:54.774+0000 7fb91d600640 1 -- 172.21.15.17:0/3016564130 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fb8f40725e0 msgr2=0x7fb8f4074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:54.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:54.774+0000 7fb91d600640 1 --2- 172.21.15.17:0/3016564130 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fb8f40725e0 0x7fb8f4074aa0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb908007fd0 tx=0x7fb908009040 comp rx=0 tx=0).stop 2024-09-10T21:24:54.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:54.774+0000 7fb91d600640 1 -- 172.21.15.17:0/3016564130 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb918152820 msgr2=0x7fb91815f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:54.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:54.774+0000 7fb91d600640 1 --2- 172.21.15.17:0/3016564130 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb918152820 0x7fb91815f560 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fb9100196f0 tx=0x7fb9100a5b10 comp rx=0 tx=0).stop 2024-09-10T21:24:54.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:54.774+0000 7fb91d600640 1 -- 172.21.15.17:0/3016564130 shutdown_connections 2024-09-10T21:24:54.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:54.774+0000 7fb91d600640 1 --2- 172.21.15.17:0/3016564130 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fb8f40725e0 0x7fb8f4074aa0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:54.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:54.774+0000 7fb91d600640 1 --2- 172.21.15.17:0/3016564130 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb918152820 0x7fb91815f560 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:54.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:54.774+0000 7fb91d600640 1 --2- 172.21.15.17:0/3016564130 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb918152440 0x7fb918157960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:54.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:54.774+0000 7fb91d600640 1 --2- 172.21.15.17:0/3016564130 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fb9180b3eb0 0x7fb918154800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:54.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:54.774+0000 7fb91d600640 1 -- 172.21.15.17:0/3016564130 >> 172.21.15.17:0/3016564130 conn(0x7fb9180702f0 msgr2=0x7fb918159ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:54.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:54.774+0000 7fb91d600640 1 -- 172.21.15.17:0/3016564130 shutdown_connections 2024-09-10T21:24:54.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:54.774+0000 7fb91d600640 1 -- 172.21.15.17:0/3016564130 wait complete. 2024-09-10T21:24:55.401 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 5, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T21:24:47.255397+0000', 'modified': '2024-09-10T21:24:52.295961+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4414}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4414': {'gid': 4414, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:creating', 'state_seq': 1, 'addr': '172.21.15.31:6839/3129062106', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6837', 'nonce': 3129062106}, {'type': 'v1', 'addr': '172.21.15.31:6839', 'nonce': 3129062106}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-09-10T21:24:55.401 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_4414:up:creating 2024-09-10T21:24:56.402 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:24:56.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:56.558+0000 7f3d95800640 1 -- 172.21.15.17:0/1496055729 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3d901517d0 msgr2=0x7f3d900c1040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:56.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:56.558+0000 7f3d95800640 1 --2- 172.21.15.17:0/1496055729 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3d901517d0 0x7f3d900c1040 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3d78009840 tx=0x7f3d7801c7f0 comp rx=0 tx=0).stop 2024-09-10T21:24:56.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:56.558+0000 7f3d95800640 1 -- 172.21.15.17:0/1496055729 shutdown_connections 2024-09-10T21:24:56.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:56.558+0000 7f3d95800640 1 --2- 172.21.15.17:0/1496055729 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3d901517d0 0x7f3d900c1040 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:56.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:56.558+0000 7f3d95800640 1 -- 172.21.15.17:0/1496055729 >> 172.21.15.17:0/1496055729 conn(0x7f3d90070160 msgr2=0x7f3d90070570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:56.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:56.558+0000 7f3d95800640 1 -- 172.21.15.17:0/1496055729 shutdown_connections 2024-09-10T21:24:56.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:56.558+0000 7f3d95800640 1 -- 172.21.15.17:0/1496055729 wait complete. 2024-09-10T21:24:56.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:56.558+0000 7f3d95800640 1 Processor -- start 2024-09-10T21:24:56.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:56.558+0000 7f3d95800640 1 -- start start 2024-09-10T21:24:56.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:56.558+0000 7f3d95800640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3d900c1580 0x7f3d90144170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:56.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:56.558+0000 7f3d95800640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3d901513f0 0x7f3d900c3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:56.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:56.558+0000 7f3d95800640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3d901517d0 0x7f3d900c52f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:56.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:56.558+0000 7f3d95800640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f3d900c8030 con 0x7f3d901513f0 2024-09-10T21:24:56.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:56.558+0000 7f3d95800640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f3d90158150 con 0x7f3d900c1580 2024-09-10T21:24:56.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:56.558+0000 7f3d95800640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f3d900c81b0 con 0x7f3d901517d0 2024-09-10T21:24:56.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:56.558+0000 7f3d87e00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3d901513f0 0x7f3d900c3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:56.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:56.558+0000 7f3d87e00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3d901513f0 0x7f3d900c3650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49906/0 (socket says 172.21.15.17:49906) 2024-09-10T21:24:56.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:56.558+0000 7f3d87e00640 1 -- 172.21.15.17:0/2861140614 learned_addr learned my addr 172.21.15.17:0/2861140614 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:24:56.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:56.558+0000 7f3d8ea00640 1 --2- 172.21.15.17:0/2861140614 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3d900c1580 0x7f3d90144170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:56.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:56.558+0000 7f3d8f400640 1 --2- 172.21.15.17:0/2861140614 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3d901517d0 0x7f3d900c52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:56.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:56.558+0000 7f3d87e00640 1 -- 172.21.15.17:0/2861140614 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3d901517d0 msgr2=0x7f3d900c52f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:56.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:56.558+0000 7f3d87e00640 1 --2- 172.21.15.17:0/2861140614 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3d901517d0 0x7f3d900c52f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:56.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:56.558+0000 7f3d87e00640 1 -- 172.21.15.17:0/2861140614 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3d900c1580 msgr2=0x7f3d90144170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:56.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:56.558+0000 7f3d87e00640 1 --2- 172.21.15.17:0/2861140614 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3d900c1580 0x7f3d90144170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:56.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:56.558+0000 7f3d87e00640 1 -- 172.21.15.17:0/2861140614 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d900c4b30 con 0x7f3d901513f0 2024-09-10T21:24:56.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:56.562+0000 7f3d87e00640 1 --2- 172.21.15.17:0/2861140614 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3d901513f0 0x7f3d900c3650 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f3d78009790 tx=0x7f3d78004220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:56.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:56.562+0000 7f3d86a00640 1 -- 172.21.15.17:0/2861140614 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d780bb070 con 0x7f3d901513f0 2024-09-10T21:24:56.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:56.562+0000 7f3d86a00640 1 -- 172.21.15.17:0/2861140614 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3d78005320 con 0x7f3d901513f0 2024-09-10T21:24:56.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:56.562+0000 7f3d86a00640 1 -- 172.21.15.17:0/2861140614 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d780b2040 con 0x7f3d901513f0 2024-09-10T21:24:56.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:56.562+0000 7f3d95800640 1 -- 172.21.15.17:0/2861140614 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d901454f0 con 0x7f3d901513f0 2024-09-10T21:24:56.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:56.562+0000 7f3d95800640 1 -- 172.21.15.17:0/2861140614 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d90145ab0 con 0x7f3d901513f0 2024-09-10T21:24:56.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:56.562+0000 7f3d95800640 1 -- 172.21.15.17:0/2861140614 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d900c1040 con 0x7f3d901513f0 2024-09-10T21:24:56.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:56.562+0000 7f3d86a00640 1 -- 172.21.15.17:0/2861140614 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f3d780b5030 con 0x7f3d901513f0 2024-09-10T21:24:56.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:56.562+0000 7f3d86a00640 1 --2- 172.21.15.17:0/2861140614 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f3d6c0725e0 0x7f3d6c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:56.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:56.562+0000 7f3d86a00640 1 -- 172.21.15.17:0/2861140614 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f3d7812ce20 con 0x7f3d901513f0 2024-09-10T21:24:56.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:56.566+0000 7f3d8ea00640 1 --2- 172.21.15.17:0/2861140614 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f3d6c0725e0 0x7f3d6c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:56.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:56.566+0000 7f3d8ea00640 1 --2- 172.21.15.17:0/2861140614 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f3d6c0725e0 0x7f3d6c074aa0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f3d80001690 tx=0x7f3d80009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:56.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:56.570+0000 7f3d86a00640 1 -- 172.21.15.17:0/2861140614 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f3d78016410 con 0x7f3d901513f0 2024-09-10T21:24:56.780 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:56.774+0000 7f3d95800640 1 -- 172.21.15.17:0/2861140614 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3d900c2210 con 0x7f3d901513f0 2024-09-10T21:24:56.781 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:56.778+0000 7f3d86a00640 1 -- 172.21.15.17:0/2861140614 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+6862 (secure 0 0 0) 0x7f3d780f81b0 con 0x7f3d901513f0 2024-09-10T21:24:56.781 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-10T21:24:56.781 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":5,"btime":"2024-09-10T21:24:52:871404+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4350,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/1643071167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":1643071167},{"type":"v1","addr":"172.21.15.31:6835","nonce":1643071167}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4356,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6837/1666225712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6836","nonce":1666225712},{"type":"v1","addr":"172.21.15.17:6837","nonce":1666225712}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4384,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/3592734043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":3592734043},{"type":"v1","addr":"172.21.15.17:6833","nonce":3592734043}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4405,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/1293665856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":1293665856},{"type":"v1","addr":"172.21.15.17:6835","nonce":1293665856}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4411,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6838/1944852172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":1944852172},{"type":"v1","addr":"172.21.15.31:6838","nonce":1944852172}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":5,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:24:47.255397+0000","modified":"2024-09-10T21:24:52.295961+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4414},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4414":{"gid":4414,"name":"d","rank":0,"incarnation":4,"state":"up:creating","state_seq":1,"addr":"172.21.15.31:6839/3129062106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6837","nonce":3129062106},{"type":"v1","addr":"172.21.15.31:6839","nonce":3129062106}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-09-10T21:24:56.781 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 5 2024-09-10T21:24:56.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:56.778+0000 7f3d95800640 1 -- 172.21.15.17:0/2861140614 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f3d6c0725e0 msgr2=0x7f3d6c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:56.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:56.778+0000 7f3d95800640 1 --2- 172.21.15.17:0/2861140614 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f3d6c0725e0 0x7f3d6c074aa0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f3d80001690 tx=0x7f3d80009040 comp rx=0 tx=0).stop 2024-09-10T21:24:56.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:56.778+0000 7f3d95800640 1 -- 172.21.15.17:0/2861140614 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3d901513f0 msgr2=0x7f3d900c3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:56.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:56.778+0000 7f3d95800640 1 --2- 172.21.15.17:0/2861140614 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3d901513f0 0x7f3d900c3650 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f3d78009790 tx=0x7f3d78004220 comp rx=0 tx=0).stop 2024-09-10T21:24:56.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:56.778+0000 7f3d95800640 1 -- 172.21.15.17:0/2861140614 shutdown_connections 2024-09-10T21:24:56.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:56.778+0000 7f3d95800640 1 --2- 172.21.15.17:0/2861140614 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f3d6c0725e0 0x7f3d6c074aa0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:56.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:56.778+0000 7f3d95800640 1 --2- 172.21.15.17:0/2861140614 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3d901517d0 0x7f3d900c52f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:56.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:56.778+0000 7f3d95800640 1 --2- 172.21.15.17:0/2861140614 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3d901513f0 0x7f3d900c3650 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:56.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:56.778+0000 7f3d95800640 1 --2- 172.21.15.17:0/2861140614 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3d900c1580 0x7f3d90144170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:56.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:56.778+0000 7f3d95800640 1 -- 172.21.15.17:0/2861140614 >> 172.21.15.17:0/2861140614 conn(0x7f3d90070160 msgr2=0x7f3d9006dd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:56.784 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:56.778+0000 7f3d95800640 1 -- 172.21.15.17:0/2861140614 shutdown_connections 2024-09-10T21:24:56.784 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:56.778+0000 7f3d95800640 1 -- 172.21.15.17:0/2861140614 wait complete. 2024-09-10T21:24:57.406 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 5, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T21:24:47.255397+0000', 'modified': '2024-09-10T21:24:52.295961+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4414}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4414': {'gid': 4414, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:creating', 'state_seq': 1, 'addr': '172.21.15.31:6839/3129062106', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6837', 'nonce': 3129062106}, {'type': 'v1', 'addr': '172.21.15.31:6839', 'nonce': 3129062106}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 0, 'qdb_cluster': []} 2024-09-10T21:24:57.406 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_4414:up:creating 2024-09-10T21:24:58.407 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:24:58.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:58.562+0000 7f6023a00640 1 -- 172.21.15.17:0/934838381 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f60040052f0 msgr2=0x7f600400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:58.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:58.562+0000 7f6023a00640 1 --2- 172.21.15.17:0/934838381 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f60040052f0 0x7f600400b9d0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f6014009ee0 tx=0x7f601401d630 comp rx=0 tx=0).stop 2024-09-10T21:24:58.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:58.562+0000 7f6023a00640 1 -- 172.21.15.17:0/934838381 shutdown_connections 2024-09-10T21:24:58.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:58.562+0000 7f6023a00640 1 --2- 172.21.15.17:0/934838381 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f60040052f0 0x7f600400b9d0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:58.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:58.562+0000 7f6023a00640 1 -- 172.21.15.17:0/934838381 >> 172.21.15.17:0/934838381 conn(0x7f601c068990 msgr2=0x7f601c068da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:58.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:58.562+0000 7f6023a00640 1 -- 172.21.15.17:0/934838381 shutdown_connections 2024-09-10T21:24:58.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:58.562+0000 7f6023a00640 1 -- 172.21.15.17:0/934838381 wait complete. 2024-09-10T21:24:58.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:58.562+0000 7f6023a00640 1 Processor -- start 2024-09-10T21:24:58.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:58.562+0000 7f6023a00640 1 -- start start 2024-09-10T21:24:58.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:58.566+0000 7f6023a00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f60040052f0 0x7f601c0724f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:58.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:58.566+0000 7f6023a00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f601c153640 0x7f601c149a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:58.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:58.566+0000 7f6023a00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f601c153a60 0x7f601c149f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:58.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:58.566+0000 7f6023a00640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f601c071770 con 0x7f601c153640 2024-09-10T21:24:58.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:58.566+0000 7f6023a00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f601c0715f0 con 0x7f601c153a60 2024-09-10T21:24:58.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:58.566+0000 7f6023a00640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f601c0718f0 con 0x7f60040052f0 2024-09-10T21:24:58.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:58.566+0000 7f601be00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f601c153640 0x7f601c149a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:58.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:58.566+0000 7f601be00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f601c153640 0x7f601c149a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49932/0 (socket says 172.21.15.17:49932) 2024-09-10T21:24:58.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:58.566+0000 7f601be00640 1 -- 172.21.15.17:0/3750066552 learned_addr learned my addr 172.21.15.17:0/3750066552 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:24:58.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:58.566+0000 7f6021a00640 1 --2- 172.21.15.17:0/3750066552 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f601c153a60 0x7f601c149f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:58.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:58.566+0000 7f6021000640 1 --2- 172.21.15.17:0/3750066552 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f60040052f0 0x7f601c0724f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:58.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:58.566+0000 7f601be00640 1 -- 172.21.15.17:0/3750066552 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f60040052f0 msgr2=0x7f601c0724f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:58.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:58.566+0000 7f601be00640 1 --2- 172.21.15.17:0/3750066552 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f60040052f0 0x7f601c0724f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:58.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:58.566+0000 7f601be00640 1 -- 172.21.15.17:0/3750066552 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f601c153a60 msgr2=0x7f601c149f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:58.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:58.566+0000 7f601be00640 1 --2- 172.21.15.17:0/3750066552 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f601c153a60 0x7f601c149f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:58.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:58.566+0000 7f601be00640 1 -- 172.21.15.17:0/3750066552 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f601c155c20 con 0x7f601c153640 2024-09-10T21:24:58.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:58.566+0000 7f6021a00640 1 --2- 172.21.15.17:0/3750066552 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f601c153a60 0x7f601c149f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:24:58.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:58.566+0000 7f601be00640 1 --2- 172.21.15.17:0/3750066552 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f601c153640 0x7f601c149a40 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f6008003eb0 tx=0x7f6008001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:58.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:58.566+0000 7f6018c00640 1 -- 172.21.15.17:0/3750066552 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6008015070 con 0x7f601c153640 2024-09-10T21:24:58.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:58.566+0000 7f6018c00640 1 -- 172.21.15.17:0/3750066552 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6008011840 con 0x7f601c153640 2024-09-10T21:24:58.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:58.566+0000 7f6023a00640 1 -- 172.21.15.17:0/3750066552 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f601c06deb0 con 0x7f601c153640 2024-09-10T21:24:58.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:58.566+0000 7f6018c00640 1 -- 172.21.15.17:0/3750066552 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6008011c70 con 0x7f601c153640 2024-09-10T21:24:58.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:58.566+0000 7f6023a00640 1 -- 172.21.15.17:0/3750066552 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f601c06e3d0 con 0x7f601c153640 2024-09-10T21:24:58.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:58.566+0000 7f6023a00640 1 -- 172.21.15.17:0/3750066552 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f601c0c3060 con 0x7f601c153640 2024-09-10T21:24:58.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:58.570+0000 7f6018c00640 1 -- 172.21.15.17:0/3750066552 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f60080103c0 con 0x7f601c153640 2024-09-10T21:24:58.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:58.570+0000 7f6018c00640 1 --2- 172.21.15.17:0/3750066552 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f5ffc0725e0 0x7f5ffc074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:58.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:58.570+0000 7f6018c00640 1 -- 172.21.15.17:0/3750066552 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f60080945b0 con 0x7f601c153640 2024-09-10T21:24:58.574 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:58.570+0000 7f6021000640 1 --2- 172.21.15.17:0/3750066552 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f5ffc0725e0 0x7f5ffc074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:58.574 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:58.570+0000 7f6021000640 1 --2- 172.21.15.17:0/3750066552 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f5ffc0725e0 0x7f5ffc074aa0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f60100016c0 tx=0x7f60100067b0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:58.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:58.574+0000 7f6018c00640 1 -- 172.21.15.17:0/3750066552 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f600805f8c0 con 0x7f601c153640 2024-09-10T21:24:58.785 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:58.782+0000 7f6023a00640 1 -- 172.21.15.17:0/3750066552 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f601c0c99f0 con 0x7f601c153640 2024-09-10T21:24:58.785 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:58.782+0000 7f6018c00640 1 -- 172.21.15.17:0/3750066552 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+6867 (secure 0 0 0) 0x7f600805f1c0 con 0x7f601c153640 2024-09-10T21:24:58.786 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-10T21:24:58.786 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":7,"btime":"2024-09-10T21:24:56:889652+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4350,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/1643071167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":1643071167},{"type":"v1","addr":"172.21.15.31:6835","nonce":1643071167}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4356,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6837/1666225712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6836","nonce":1666225712},{"type":"v1","addr":"172.21.15.17:6837","nonce":1666225712}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4384,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/3592734043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":3592734043},{"type":"v1","addr":"172.21.15.17:6833","nonce":3592734043}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4405,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/1293665856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":1293665856},{"type":"v1","addr":"172.21.15.17:6835","nonce":1293665856}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4411,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6838/1944852172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":1944852172},{"type":"v1","addr":"172.21.15.31:6838","nonce":1944852172}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:24:47.255397+0000","modified":"2024-09-10T21:24:56.885702+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4414},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4414":{"gid":4414,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.31:6839/3129062106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6837","nonce":3129062106},{"type":"v1","addr":"172.21.15.31:6839","nonce":3129062106}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":4414,"qdb_cluster":[4414]},"id":1}]} 2024-09-10T21:24:58.786 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 7 2024-09-10T21:24:58.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:58.782+0000 7f6023a00640 1 -- 172.21.15.17:0/3750066552 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f5ffc0725e0 msgr2=0x7f5ffc074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:58.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:58.782+0000 7f6023a00640 1 --2- 172.21.15.17:0/3750066552 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f5ffc0725e0 0x7f5ffc074aa0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f60100016c0 tx=0x7f60100067b0 comp rx=0 tx=0).stop 2024-09-10T21:24:58.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:58.782+0000 7f6023a00640 1 -- 172.21.15.17:0/3750066552 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f601c153640 msgr2=0x7f601c149a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:58.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:58.782+0000 7f6023a00640 1 --2- 172.21.15.17:0/3750066552 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f601c153640 0x7f601c149a40 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f6008003eb0 tx=0x7f6008001640 comp rx=0 tx=0).stop 2024-09-10T21:24:58.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:58.782+0000 7f6023a00640 1 -- 172.21.15.17:0/3750066552 shutdown_connections 2024-09-10T21:24:58.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:58.782+0000 7f6023a00640 1 --2- 172.21.15.17:0/3750066552 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f5ffc0725e0 0x7f5ffc074aa0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:58.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:58.782+0000 7f6023a00640 1 --2- 172.21.15.17:0/3750066552 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f601c153a60 0x7f601c149f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:58.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:58.782+0000 7f6023a00640 1 --2- 172.21.15.17:0/3750066552 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f601c153640 0x7f601c149a40 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:58.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:58.782+0000 7f6023a00640 1 --2- 172.21.15.17:0/3750066552 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f60040052f0 0x7f601c0724f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:58.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:58.782+0000 7f6023a00640 1 -- 172.21.15.17:0/3750066552 >> 172.21.15.17:0/3750066552 conn(0x7f601c068990 msgr2=0x7f601c15bdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:58.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:58.782+0000 7f6023a00640 1 -- 172.21.15.17:0/3750066552 shutdown_connections 2024-09-10T21:24:58.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:58.782+0000 7f6023a00640 1 -- 172.21.15.17:0/3750066552 wait complete. 2024-09-10T21:24:59.411 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 6, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T21:24:47.255397+0000', 'modified': '2024-09-10T21:24:56.885702+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4414}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4414': {'gid': 4414, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.31:6839/3129062106', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6837', 'nonce': 3129062106}, {'type': 'v1', 'addr': '172.21.15.31:6839', 'nonce': 3129062106}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 4414, 'qdb_cluster': [4414]} 2024-09-10T21:24:59.411 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-10T21:24:59.411 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-09-10T21:24:59.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:59.562+0000 7f6562e00640 1 -- 172.21.15.17:0/265085326 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f655c0c77e0 msgr2=0x7f655c162bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:59.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:59.562+0000 7f6562e00640 1 --2- 172.21.15.17:0/265085326 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f655c0c77e0 0x7f655c162bd0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f655c152b10 tx=0x7f655401cc60 comp rx=0 tx=0).stop 2024-09-10T21:24:59.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:59.562+0000 7f6562e00640 1 -- 172.21.15.17:0/265085326 shutdown_connections 2024-09-10T21:24:59.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:59.562+0000 7f6562e00640 1 --2- 172.21.15.17:0/265085326 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f655c0c77e0 0x7f655c162bd0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:59.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:59.562+0000 7f6562e00640 1 --2- 172.21.15.17:0/265085326 >> v2:172.21.15.31:3300/0 conn(0x7f655c152460 0x7f655c0c8770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:59.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:59.562+0000 7f6562e00640 1 -- 172.21.15.17:0/265085326 >> 172.21.15.17:0/265085326 conn(0x7f655c0cd690 msgr2=0x7f655c0c98e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:59.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:59.562+0000 7f6562e00640 1 -- 172.21.15.17:0/265085326 shutdown_connections 2024-09-10T21:24:59.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:59.562+0000 7f6562e00640 1 -- 172.21.15.17:0/265085326 wait complete. 2024-09-10T21:24:59.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:59.566+0000 7f6562e00640 1 Processor -- start 2024-09-10T21:24:59.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:59.566+0000 7f6562e00640 1 -- start start 2024-09-10T21:24:59.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:59.566+0000 7f6562e00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f655c06ec30 0x7f655c0c52f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:59.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:59.566+0000 7f6562e00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f655c0c77e0 0x7f655c0c36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:59.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:59.566+0000 7f6562e00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f655c152460 0x7f655c0af760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:59.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:59.566+0000 7f6562e00640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f655c06fbe0 con 0x7f655c0c77e0 2024-09-10T21:24:59.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:59.566+0000 7f6562e00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f655c06fa60 con 0x7f655c06ec30 2024-09-10T21:24:59.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:59.566+0000 7f6562e00640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f655c0ca5f0 con 0x7f655c152460 2024-09-10T21:24:59.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:59.566+0000 7f6561000640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f655c0c77e0 0x7f655c0c36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:59.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:59.566+0000 7f6561000640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f655c0c77e0 0x7f655c0c36d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49948/0 (socket says 172.21.15.17:49948) 2024-09-10T21:24:59.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:59.566+0000 7f6561000640 1 -- 172.21.15.17:0/1251834602 learned_addr learned my addr 172.21.15.17:0/1251834602 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:24:59.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:59.566+0000 7f6561a00640 1 --2- 172.21.15.17:0/1251834602 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f655c06ec30 0x7f655c0c52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:59.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:59.566+0000 7f6562400640 1 --2- 172.21.15.17:0/1251834602 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f655c152460 0x7f655c0af760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:59.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:59.566+0000 7f6561000640 1 -- 172.21.15.17:0/1251834602 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f655c152460 msgr2=0x7f655c0af760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:59.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:59.566+0000 7f6561000640 1 --2- 172.21.15.17:0/1251834602 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f655c152460 0x7f655c0af760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:59.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:59.566+0000 7f6561000640 1 -- 172.21.15.17:0/1251834602 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f655c06ec30 msgr2=0x7f655c0c52f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:59.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:59.566+0000 7f6561000640 1 --2- 172.21.15.17:0/1251834602 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f655c06ec30 0x7f655c0c52f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:59.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:59.566+0000 7f6561000640 1 -- 172.21.15.17:0/1251834602 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f655c074960 con 0x7f655c0c77e0 2024-09-10T21:24:59.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:59.566+0000 7f6561000640 1 --2- 172.21.15.17:0/1251834602 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f655c0c77e0 0x7f655c0c36d0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f654800c480 tx=0x7f654800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:59.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:59.566+0000 7f6559600640 1 -- 172.21.15.17:0/1251834602 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6548012830 con 0x7f655c0c77e0 2024-09-10T21:24:59.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:59.566+0000 7f6559600640 1 -- 172.21.15.17:0/1251834602 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6548017070 con 0x7f655c0c77e0 2024-09-10T21:24:59.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:59.566+0000 7f6559600640 1 -- 172.21.15.17:0/1251834602 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6548012d80 con 0x7f655c0c77e0 2024-09-10T21:24:59.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:59.566+0000 7f6562e00640 1 -- 172.21.15.17:0/1251834602 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f655c151c50 con 0x7f655c0c77e0 2024-09-10T21:24:59.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:59.566+0000 7f6562e00640 1 -- 172.21.15.17:0/1251834602 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f655c0c4560 con 0x7f655c0c77e0 2024-09-10T21:24:59.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:59.570+0000 7f6559600640 1 -- 172.21.15.17:0/1251834602 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f654801c460 con 0x7f655c0c77e0 2024-09-10T21:24:59.575 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:59.570+0000 7f6559600640 1 --2- 172.21.15.17:0/1251834602 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f653c072610 0x7f653c074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:59.575 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:59.570+0000 7f6559600640 1 -- 172.21.15.17:0/1251834602 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f65480932f0 con 0x7f655c0c77e0 2024-09-10T21:24:59.575 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:59.570+0000 7f65678d3000 1 Processor -- start 2024-09-10T21:24:59.576 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:59.570+0000 7f6561a00640 1 --2- 172.21.15.17:0/1251834602 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f653c072610 0x7f653c074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:59.576 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:59.570+0000 7f65678d3000 1 -- start start 2024-09-10T21:24:59.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:59.574+0000 7f6561a00640 1 --2- 172.21.15.17:0/1251834602 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f653c072610 0x7f653c074ad0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f6550004630 tx=0x7f655000b2e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:59.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:59.574+0000 7f65678d3000 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x560e25887190 0x560e25887570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:59.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:59.574+0000 7f65678d3000 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560e25850950 0x560e25850d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:59.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:59.574+0000 7f65678d3000 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560e25851270 0x560e2588c040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:59.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:59.574+0000 7f65678d3000 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x560e257693f0 con 0x560e25851270 2024-09-10T21:24:59.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:59.574+0000 7f65678d3000 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x560e256a6b20 con 0x560e25850950 2024-09-10T21:24:59.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:59.574+0000 7f65678d3000 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x560e25695f50 con 0x560e25887190 2024-09-10T21:24:59.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:59.574+0000 7f6562400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560e25851270 0x560e2588c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:59.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:59.574+0000 7f6562400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560e25851270 0x560e2588c040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49952/0 (socket says 172.21.15.17:49952) 2024-09-10T21:24:59.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:59.574+0000 7f6562400640 1 -- 172.21.15.17:0/3381982038 learned_addr learned my addr 172.21.15.17:0/3381982038 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:24:59.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:59.574+0000 7f6561a00640 1 --2- 172.21.15.17:0/3381982038 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x560e25887190 0x560e25887570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:59.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:59.574+0000 7f6561000640 1 --2- 172.21.15.17:0/3381982038 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560e25850950 0x560e25850d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:59.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:59.574+0000 7f6562400640 1 -- 172.21.15.17:0/3381982038 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x560e25887190 msgr2=0x560e25887570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:24:59.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:59.574+0000 7f6562400640 1 --2- 172.21.15.17:0/3381982038 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x560e25887190 0x560e25887570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:59.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:59.574+0000 7f6562400640 1 -- 172.21.15.17:0/3381982038 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560e25850950 msgr2=0x560e25850d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:24:59.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:59.574+0000 7f6562400640 1 --2- 172.21.15.17:0/3381982038 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560e25850950 0x560e25850d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:59.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:59.574+0000 7f6562400640 1 -- 172.21.15.17:0/3381982038 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e2565d590 con 0x560e25851270 2024-09-10T21:24:59.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:59.574+0000 7f6562400640 1 --2- 172.21.15.17:0/3381982038 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560e25851270 0x560e2588c040 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f6554009bd0 tx=0x7f6554019a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b74ce40254e9f96f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:59.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:59.574+0000 7f654e000640 1 -- 172.21.15.17:0/3381982038 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65540b2040 con 0x560e25851270 2024-09-10T21:24:59.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:59.574+0000 7f654e000640 1 -- 172.21.15.17:0/3381982038 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f65540bc070 con 0x560e25851270 2024-09-10T21:24:59.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:59.574+0000 7f654e000640 1 -- 172.21.15.17:0/3381982038 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6554004cb0 con 0x560e25851270 2024-09-10T21:24:59.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:59.574+0000 7f65678d3000 1 -- 172.21.15.17:0/3381982038 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560e25851270 msgr2=0x560e2588c040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:24:59.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:59.574+0000 7f65678d3000 1 --2- 172.21.15.17:0/3381982038 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560e25851270 0x560e2588c040 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f6554009bd0 tx=0x7f6554019a00 comp rx=0 tx=0).stop 2024-09-10T21:24:59.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:59.574+0000 7f65678d3000 1 -- 172.21.15.17:0/3381982038 shutdown_connections 2024-09-10T21:24:59.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:59.574+0000 7f65678d3000 1 --2- 172.21.15.17:0/3381982038 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560e25851270 0x560e2588c040 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:59.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:59.574+0000 7f65678d3000 1 --2- 172.21.15.17:0/3381982038 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560e25850950 0x560e25850d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:59.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:59.574+0000 7f65678d3000 1 --2- 172.21.15.17:0/3381982038 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x560e25887190 0x560e25887570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:59.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:59.574+0000 7f65678d3000 1 -- 172.21.15.17:0/3381982038 >> 172.21.15.17:0/3381982038 conn(0x560e25732130 msgr2=0x560e25889e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:24:59.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:59.574+0000 7f65678d3000 1 -- 172.21.15.17:0/3381982038 shutdown_connections 2024-09-10T21:24:59.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:59.574+0000 7f65678d3000 1 -- 172.21.15.17:0/3381982038 wait complete. 2024-09-10T21:24:59.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:59.574+0000 7f65678d3000 1 -- start start 2024-09-10T21:24:59.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:59.574+0000 7f65678d3000 1 Processor -- start 2024-09-10T21:24:59.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:59.574+0000 7f65678d3000 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560e25850950 0x560e2584f550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:59.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:59.574+0000 7f65678d3000 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x560e25851270 0x560e2584fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:59.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:59.574+0000 7f65678d3000 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560e25887190 0x560e258923e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:59.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:59.574+0000 7f65678d3000 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x560e256305d0 con 0x560e25887190 2024-09-10T21:24:59.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:59.574+0000 7f65678d3000 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x560e25699d60 con 0x560e25850950 2024-09-10T21:24:59.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:59.574+0000 7f65678d3000 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x560e2563c500 con 0x560e25851270 2024-09-10T21:24:59.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:59.578+0000 7f65678d3000 10 client.0 _ll_register_callbacks cb 0x560e25640810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:24:59.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:59.578+0000 7f6562400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560e25887190 0x560e258923e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:59.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:59.578+0000 7f6562400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560e25887190 0x560e258923e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49954/0 (socket says 172.21.15.17:49954) 2024-09-10T21:24:59.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:59.578+0000 7f6562400640 1 -- 172.21.15.17:0/4085223850 learned_addr learned my addr 172.21.15.17:0/4085223850 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:24:59.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:59.578+0000 7f6561000640 1 --2- 172.21.15.17:0/4085223850 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x560e25851270 0x560e2584fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:59.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:59.578+0000 7f6561a00640 1 --2- 172.21.15.17:0/4085223850 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560e25850950 0x560e2584f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:59.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:59.578+0000 7f6562400640 1 -- 172.21.15.17:0/4085223850 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x560e25851270 msgr2=0x560e2584fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:24:59.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:59.578+0000 7f6562400640 1 --2- 172.21.15.17:0/4085223850 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x560e25851270 0x560e2584fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:59.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:59.578+0000 7f6562400640 1 -- 172.21.15.17:0/4085223850 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560e25850950 msgr2=0x560e2584f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:24:59.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:59.578+0000 7f6562400640 1 --2- 172.21.15.17:0/4085223850 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560e25850950 0x560e2584f550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:24:59.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:59.578+0000 7f6562400640 1 -- 172.21.15.17:0/4085223850 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e25634170 con 0x560e25887190 2024-09-10T21:24:59.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:59.578+0000 7f6561a00640 1 --2- 172.21.15.17:0/4085223850 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560e25850950 0x560e2584f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:24:59.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:59.578+0000 7f6562400640 1 --2- 172.21.15.17:0/4085223850 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560e25887190 0x560e258923e0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f65540b9040 tx=0x7f6554004e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=648950dad412de07 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:24:59.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:59.578+0000 7f6542a00640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-09-10T21:24:59.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:59.578+0000 7f6542a00640 1 -- 172.21.15.17:0/4085223850 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65540b2070 con 0x560e25887190 2024-09-10T21:24:59.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:59.578+0000 7f6542a00640 1 -- 172.21.15.17:0/4085223850 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f65540c1df0 con 0x560e25887190 2024-09-10T21:24:59.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:59.578+0000 7f65678d3000 1 -- 172.21.15.17:0/4085223850 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x560e2562e0f0 con 0x560e25887190 2024-09-10T21:24:59.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:59.578+0000 7f6542a00640 1 -- 172.21.15.17:0/4085223850 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) ==== 24+0+0 (secure 0 0 0) 0x7f6554003980 con 0x560e25887190 2024-09-10T21:24:59.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:59.578+0000 7f65678d3000 10 client.4485 fetch_fsmap learned FSMap version 7 2024-09-10T21:24:59.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:59.578+0000 7f65678d3000 1 -- 172.21.15.17:0/4085223850 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) -- 0x560e2571aef0 con 0x560e25887190 2024-09-10T21:24:59.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:59.578+0000 7f6542a00640 1 -- 172.21.15.17:0/4085223850 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x560e2562e0f0 con 0x560e25887190 2024-09-10T21:24:59.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:59.578+0000 7f6542a00640 1 -- 172.21.15.17:0/4085223850 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 7) ==== 4720+0+0 (secure 0 0 0) 0x7f6554003c10 con 0x560e25887190 2024-09-10T21:24:59.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:59.578+0000 7f65678d3000 10 client.4485 fetch_fsmap finished waiting for FSMap version 7 2024-09-10T21:24:59.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:59.578+0000 7f65678d3000 10 client.4485 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-10T21:24:59.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:59.578+0000 7f65678d3000 20 client.4485 populate_metadata read hostname 'smithi017' 2024-09-10T21:24:59.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:59.578+0000 7f65678d3000 1 --2- 172.21.15.17:0/4085223850 >> [v2:172.21.15.31:6837/3129062106,v1:172.21.15.31:6839/3129062106] conn(0x560e25885f70 0x560e25886350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:24:59.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:59.578+0000 7f65678d3000 4 client.4485 mds_command: new command op to 4414 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:24:59.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:59.578+0000 7f65678d3000 1 -- 172.21.15.17:0/4085223850 --> [v2:172.21.15.31:6837/3129062106,v1:172.21.15.31:6839/3129062106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560e256a8a70 con 0x560e25885f70 2024-09-10T21:24:59.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:59.578+0000 7f6561a00640 1 --2- 172.21.15.17:0/4085223850 >> [v2:172.21.15.31:6837/3129062106,v1:172.21.15.31:6839/3129062106] conn(0x560e25885f70 0x560e25886350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:24:59.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:59.578+0000 7f6561a00640 1 --2- 172.21.15.17:0/4085223850 >> [v2:172.21.15.31:6837/3129062106,v1:172.21.15.31:6839/3129062106] conn(0x560e25885f70 0x560e25886350 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a4312f40a180bc6 server_cookie=be06b2185b14d4b1 in_seq=0 out_seq=0 2024-09-10T21:24:59.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:24:59.578+0000 7f6542a00640 10 client.4485 ms_handle_connect on v2:172.21.15.31:6837/3129062106 2024-09-10T21:25:01.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:01.890+0000 7f6542a00640 1 -- 172.21.15.17:0/4085223850 <== mds.0 v2:172.21.15.31:6837/3129062106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x560e256a8a70 con 0x560e25885f70 2024-09-10T21:25:01.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:01.890+0000 7f6542a00640 10 client.4485 handle_command_reply: tid=0 2024-09-10T21:25:01.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:01.890+0000 7f65678d3000 1 client.4485 shutdown 2024-09-10T21:25:01.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:01.890+0000 7f65678d3000 1 -- 172.21.15.17:0/4085223850 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560e25887190 msgr2=0x560e258923e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:01.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:01.890+0000 7f65678d3000 1 --2- 172.21.15.17:0/4085223850 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560e25887190 0x560e258923e0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f65540b9040 tx=0x7f6554004e30 comp rx=0 tx=0).stop 2024-09-10T21:25:01.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:01.890+0000 7f65678d3000 1 -- 172.21.15.17:0/4085223850 shutdown_connections 2024-09-10T21:25:01.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:01.890+0000 7f65678d3000 1 --2- 172.21.15.17:0/4085223850 >> [v2:172.21.15.31:6837/3129062106,v1:172.21.15.31:6839/3129062106] conn(0x560e25885f70 0x560e25886350 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:01.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:01.890+0000 7f65678d3000 1 --2- 172.21.15.17:0/4085223850 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560e25887190 0x560e258923e0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:01.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:01.890+0000 7f65678d3000 1 --2- 172.21.15.17:0/4085223850 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x560e25851270 0x560e2584fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:01.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:01.890+0000 7f65678d3000 1 --2- 172.21.15.17:0/4085223850 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560e25850950 0x560e2584f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:01.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:01.890+0000 7f6542a00640 0 client.4485 ms_handle_reset on v2:172.21.15.31:6837/3129062106 2024-09-10T21:25:01.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:01.890+0000 7f65678d3000 1 -- 172.21.15.17:0/4085223850 >> 172.21.15.17:0/4085223850 conn(0x560e25732130 msgr2=0x560e258490b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:01.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:01.890+0000 7f65678d3000 1 -- 172.21.15.17:0/4085223850 shutdown_connections 2024-09-10T21:25:01.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:01.890+0000 7f65678d3000 1 -- 172.21.15.17:0/4085223850 wait complete. 2024-09-10T21:25:01.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:01.890+0000 7f65678d3000 20 client.4485 trim_cache size 0 max 16384 2024-09-10T21:25:01.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:01.910+0000 7f65678d3000 1 Processor -- start 2024-09-10T21:25:01.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:01.910+0000 7f65678d3000 1 -- start start 2024-09-10T21:25:01.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:01.914+0000 7f65678d3000 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x560e25850950 0x560e258a7e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:01.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:01.914+0000 7f65678d3000 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560e25851270 0x560e258a8350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:01.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:01.914+0000 7f65678d3000 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560e25887190 0x560e258a2ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:01.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:01.914+0000 7f65678d3000 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f655c201d70 con 0x560e25851270 2024-09-10T21:25:01.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:01.914+0000 7f65678d3000 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f655c200c50 con 0x560e25887190 2024-09-10T21:25:01.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:01.914+0000 7f65678d3000 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f655c150670 con 0x560e25850950 2024-09-10T21:25:01.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:01.914+0000 7f6561000640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560e25851270 0x560e258a8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:01.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:01.914+0000 7f6561000640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560e25851270 0x560e258a8350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49958/0 (socket says 172.21.15.17:49958) 2024-09-10T21:25:01.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:01.914+0000 7f6561000640 1 -- 172.21.15.17:0/324845830 learned_addr learned my addr 172.21.15.17:0/324845830 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:25:01.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:01.914+0000 7f6562400640 1 --2- 172.21.15.17:0/324845830 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560e25887190 0x560e258a2ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:01.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:01.914+0000 7f6561a00640 1 --2- 172.21.15.17:0/324845830 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x560e25850950 0x560e258a7e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:01.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:01.914+0000 7f6561000640 1 -- 172.21.15.17:0/324845830 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x560e25850950 msgr2=0x560e258a7e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:25:01.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:01.914+0000 7f6561000640 1 --2- 172.21.15.17:0/324845830 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x560e25850950 0x560e258a7e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:01.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:01.914+0000 7f6561000640 1 -- 172.21.15.17:0/324845830 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560e25887190 msgr2=0x560e258a2ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:25:01.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:01.914+0000 7f6561000640 1 --2- 172.21.15.17:0/324845830 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560e25887190 0x560e258a2ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:01.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:01.914+0000 7f6561000640 1 -- 172.21.15.17:0/324845830 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e25892750 con 0x560e25851270 2024-09-10T21:25:01.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:01.914+0000 7f6561000640 1 --2- 172.21.15.17:0/324845830 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560e25851270 0x560e258a8350 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f654800ce40 tx=0x7f6548093040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8498d9ca0ba3a5f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:01.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:01.914+0000 7f654d600640 1 -- 172.21.15.17:0/324845830 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f654805e030 con 0x560e25851270 2024-09-10T21:25:01.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:01.914+0000 7f654d600640 1 -- 172.21.15.17:0/324845830 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6548062a60 con 0x560e25851270 2024-09-10T21:25:01.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:01.914+0000 7f654d600640 1 -- 172.21.15.17:0/324845830 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f654805d4d0 con 0x560e25851270 2024-09-10T21:25:01.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:01.914+0000 7f65678d3000 1 -- 172.21.15.17:0/324845830 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560e25851270 msgr2=0x560e258a8350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:01.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:01.914+0000 7f65678d3000 1 --2- 172.21.15.17:0/324845830 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560e25851270 0x560e258a8350 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f654800ce40 tx=0x7f6548093040 comp rx=0 tx=0).stop 2024-09-10T21:25:01.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:01.914+0000 7f65678d3000 1 -- 172.21.15.17:0/324845830 shutdown_connections 2024-09-10T21:25:01.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:01.914+0000 7f65678d3000 1 --2- 172.21.15.17:0/324845830 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560e25887190 0x560e258a2ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:01.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:01.914+0000 7f65678d3000 1 --2- 172.21.15.17:0/324845830 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560e25851270 0x560e258a8350 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:01.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:01.914+0000 7f65678d3000 1 --2- 172.21.15.17:0/324845830 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x560e25850950 0x560e258a7e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:01.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:01.914+0000 7f65678d3000 1 -- 172.21.15.17:0/324845830 >> 172.21.15.17:0/324845830 conn(0x560e25732130 msgr2=0x560e2584b8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:01.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:01.914+0000 7f65678d3000 1 -- 172.21.15.17:0/324845830 shutdown_connections 2024-09-10T21:25:01.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:01.914+0000 7f65678d3000 1 -- 172.21.15.17:0/324845830 wait complete. 2024-09-10T21:25:01.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:01.914+0000 7f65678d3000 1 -- start start 2024-09-10T21:25:01.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:01.914+0000 7f65678d3000 1 Processor -- start 2024-09-10T21:25:01.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:01.918+0000 7f65678d3000 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x560e25850950 0x560e258a69a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:01.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:01.918+0000 7f65678d3000 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560e25851270 0x560e258a6ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:01.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:01.918+0000 7f65678d3000 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560e25887190 0x560e258a7420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:01.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:01.918+0000 7f65678d3000 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f655c1ff940 con 0x560e25851270 2024-09-10T21:25:01.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:01.918+0000 7f65678d3000 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f655c200580 con 0x560e25887190 2024-09-10T21:25:01.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:01.918+0000 7f65678d3000 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f655c1ffed0 con 0x560e25850950 2024-09-10T21:25:01.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:01.918+0000 7f65678d3000 10 client.0 _ll_register_callbacks cb 0x7f6548059ed0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:25:01.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:01.918+0000 7f6561000640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560e25851270 0x560e258a6ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:01.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:01.918+0000 7f6561000640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560e25851270 0x560e258a6ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49974/0 (socket says 172.21.15.17:49974) 2024-09-10T21:25:01.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:01.918+0000 7f6561000640 1 -- 172.21.15.17:0/1089270408 learned_addr learned my addr 172.21.15.17:0/1089270408 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:25:01.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:01.918+0000 7f6562400640 1 --2- 172.21.15.17:0/1089270408 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560e25887190 0x560e258a7420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:01.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:01.918+0000 7f6561a00640 1 --2- 172.21.15.17:0/1089270408 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x560e25850950 0x560e258a69a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:01.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:01.918+0000 7f6561000640 1 -- 172.21.15.17:0/1089270408 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x560e25850950 msgr2=0x560e258a69a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:25:01.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:01.918+0000 7f6561000640 1 --2- 172.21.15.17:0/1089270408 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x560e25850950 0x560e258a69a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:01.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:01.918+0000 7f6561000640 1 -- 172.21.15.17:0/1089270408 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560e25887190 msgr2=0x560e258a7420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:25:01.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:01.918+0000 7f6561000640 1 --2- 172.21.15.17:0/1089270408 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560e25887190 0x560e258a7420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:01.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:01.918+0000 7f6561000640 1 -- 172.21.15.17:0/1089270408 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e256f1d90 con 0x560e25851270 2024-09-10T21:25:01.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:01.918+0000 7f6561000640 1 --2- 172.21.15.17:0/1089270408 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560e25851270 0x560e258a6ee0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f654805dd30 tx=0x7f654805a550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2bcb0f9391d9927 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:01.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:01.918+0000 7f6542a00640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-09-10T21:25:01.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:01.918+0000 7f6542a00640 1 -- 172.21.15.17:0/1089270408 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f654808e030 con 0x560e25851270 2024-09-10T21:25:01.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:01.918+0000 7f6542a00640 1 -- 172.21.15.17:0/1089270408 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f654805d4a0 con 0x560e25851270 2024-09-10T21:25:01.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:01.918+0000 7f65678d3000 1 -- 172.21.15.17:0/1089270408 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6548062a60 con 0x560e25851270 2024-09-10T21:25:01.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:01.918+0000 7f6542a00640 1 -- 172.21.15.17:0/1089270408 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) ==== 24+0+0 (secure 0 0 0) 0x7f654805a680 con 0x560e25851270 2024-09-10T21:25:01.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:01.918+0000 7f65678d3000 10 client.4491 fetch_fsmap learned FSMap version 7 2024-09-10T21:25:01.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:01.918+0000 7f65678d3000 1 -- 172.21.15.17:0/1089270408 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) -- 0x560e2565ca90 con 0x560e25851270 2024-09-10T21:25:01.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:01.918+0000 7f6542a00640 1 -- 172.21.15.17:0/1089270408 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6548062a60 con 0x560e25851270 2024-09-10T21:25:01.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:01.918+0000 7f6542a00640 1 -- 172.21.15.17:0/1089270408 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 7) ==== 4720+0+0 (secure 0 0 0) 0x7f654808a190 con 0x560e25851270 2024-09-10T21:25:01.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:01.918+0000 7f65678d3000 10 client.4491 fetch_fsmap finished waiting for FSMap version 7 2024-09-10T21:25:01.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:01.918+0000 7f65678d3000 10 client.4491 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-10T21:25:01.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:01.918+0000 7f65678d3000 20 client.4491 populate_metadata read hostname 'smithi017' 2024-09-10T21:25:01.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:01.918+0000 7f65678d3000 1 --2- 172.21.15.17:0/1089270408 >> [v2:172.21.15.31:6837/3129062106,v1:172.21.15.31:6839/3129062106] conn(0x560e2590cfc0 0x560e2590f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:01.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:01.918+0000 7f65678d3000 4 client.4491 mds_command: new command op to 4414 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T21:25:01.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:01.918+0000 7f65678d3000 1 -- 172.21.15.17:0/1089270408 --> [v2:172.21.15.31:6837/3129062106,v1:172.21.15.31:6839/3129062106] -- command(tid 0: {"prefix": "status"}) -- 0x560e256a4520 con 0x560e2590cfc0 2024-09-10T21:25:01.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:01.918+0000 7f6561a00640 1 --2- 172.21.15.17:0/1089270408 >> [v2:172.21.15.31:6837/3129062106,v1:172.21.15.31:6839/3129062106] conn(0x560e2590cfc0 0x560e2590f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:01.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:01.922+0000 7f6561a00640 1 --2- 172.21.15.17:0/1089270408 >> [v2:172.21.15.31:6837/3129062106,v1:172.21.15.31:6839/3129062106] conn(0x560e2590cfc0 0x560e2590f3b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a3850dceda895e1c server_cookie=e54d53e5da4c6e2e in_seq=0 out_seq=0 2024-09-10T21:25:01.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:01.922+0000 7f6542a00640 10 client.4491 ms_handle_connect on v2:172.21.15.31:6837/3129062106 2024-09-10T21:25:01.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:01.922+0000 7f6542a00640 1 -- 172.21.15.17:0/1089270408 <== mds.0 v2:172.21.15.31:6837/3129062106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+323 (crc 0 0 0) 0x560e256a4520 con 0x560e2590cfc0 2024-09-10T21:25:01.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:01.922+0000 7f6542a00640 10 client.4491 handle_command_reply: tid=0 2024-09-10T21:25:01.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:01.922+0000 7f65678d3000 1 client.4491 shutdown 2024-09-10T21:25:01.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:01.922+0000 7f65678d3000 1 -- 172.21.15.17:0/1089270408 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560e25851270 msgr2=0x560e258a6ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:01.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:01.922+0000 7f65678d3000 1 --2- 172.21.15.17:0/1089270408 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560e25851270 0x560e258a6ee0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f654805dd30 tx=0x7f654805a550 comp rx=0 tx=0).stop 2024-09-10T21:25:01.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:01.922+0000 7f65678d3000 1 -- 172.21.15.17:0/1089270408 shutdown_connections 2024-09-10T21:25:01.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:01.922+0000 7f65678d3000 1 --2- 172.21.15.17:0/1089270408 >> [v2:172.21.15.31:6837/3129062106,v1:172.21.15.31:6839/3129062106] conn(0x560e2590cfc0 0x560e2590f3b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:01.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:01.922+0000 7f65678d3000 1 --2- 172.21.15.17:0/1089270408 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560e25887190 0x560e258a7420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:01.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:01.922+0000 7f65678d3000 1 --2- 172.21.15.17:0/1089270408 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560e25851270 0x560e258a6ee0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:01.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:01.922+0000 7f6542a00640 0 client.4491 ms_handle_reset on v2:172.21.15.31:6837/3129062106 2024-09-10T21:25:01.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:01.922+0000 7f65678d3000 1 --2- 172.21.15.17:0/1089270408 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x560e25850950 0x560e258a69a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:01.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:01.922+0000 7f65678d3000 1 -- 172.21.15.17:0/1089270408 >> 172.21.15.17:0/1089270408 conn(0x560e25732130 msgr2=0x560e2584b0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:01.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:01.922+0000 7f65678d3000 1 -- 172.21.15.17:0/1089270408 shutdown_connections 2024-09-10T21:25:01.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:01.922+0000 7f65678d3000 1 -- 172.21.15.17:0/1089270408 wait complete. 2024-09-10T21:25:01.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:01.922+0000 7f65678d3000 20 client.4491 trim_cache size 0 max 16384 2024-09-10T21:25:01.930 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-10T21:25:01.930 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_fsid": "5e2f023f-d6a6-49f4-a256-f4807dce3739", 2024-09-10T21:25:01.930 INFO:teuthology.orchestra.run.smithi017.stdout: "whoami": 0, 2024-09-10T21:25:01.930 INFO:teuthology.orchestra.run.smithi017.stdout: "id": 4414, 2024-09-10T21:25:01.930 INFO:teuthology.orchestra.run.smithi017.stdout: "want_state": "up:active", 2024-09-10T21:25:01.930 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "up:active", 2024-09-10T21:25:01.930 INFO:teuthology.orchestra.run.smithi017.stdout: "fs_name": "cephfs", 2024-09-10T21:25:01.930 INFO:teuthology.orchestra.run.smithi017.stdout: "rank_uptime": 14.066477795000001, 2024-09-10T21:25:01.930 INFO:teuthology.orchestra.run.smithi017.stdout: "mdsmap_epoch": 6, 2024-09-10T21:25:01.930 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_epoch": 25, 2024-09-10T21:25:01.931 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_epoch_barrier": 25, 2024-09-10T21:25:01.931 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 20.09763135 2024-09-10T21:25:01.931 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-10T21:25:01.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:01.922+0000 7f6562e00640 1 -- 172.21.15.17:0/1251834602 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f653c072610 msgr2=0x7f653c074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:01.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:01.922+0000 7f6562e00640 1 --2- 172.21.15.17:0/1251834602 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f653c072610 0x7f653c074ad0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f6550004630 tx=0x7f655000b2e0 comp rx=0 tx=0).stop 2024-09-10T21:25:01.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:01.922+0000 7f6562e00640 1 -- 172.21.15.17:0/1251834602 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f655c0c77e0 msgr2=0x7f655c0c36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:01.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:01.922+0000 7f6562e00640 1 --2- 172.21.15.17:0/1251834602 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f655c0c77e0 0x7f655c0c36d0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f654800c480 tx=0x7f654800c940 comp rx=0 tx=0).stop 2024-09-10T21:25:01.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:01.922+0000 7f6562e00640 1 -- 172.21.15.17:0/1251834602 shutdown_connections 2024-09-10T21:25:01.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:01.922+0000 7f6562e00640 1 --2- 172.21.15.17:0/1251834602 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f653c072610 0x7f653c074ad0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:01.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:01.922+0000 7f6562e00640 1 --2- 172.21.15.17:0/1251834602 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f655c152460 0x7f655c0af760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:01.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:01.922+0000 7f6562e00640 1 --2- 172.21.15.17:0/1251834602 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f655c0c77e0 0x7f655c0c36d0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:01.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:01.922+0000 7f6562e00640 1 --2- 172.21.15.17:0/1251834602 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f655c06ec30 0x7f655c0c52f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:01.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:01.922+0000 7f6562e00640 1 -- 172.21.15.17:0/1251834602 >> 172.21.15.17:0/1251834602 conn(0x7f655c0cd690 msgr2=0x7f655c14ce90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:01.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:01.922+0000 7f6562e00640 1 -- 172.21.15.17:0/1251834602 shutdown_connections 2024-09-10T21:25:01.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:01.922+0000 7f6562e00640 1 -- 172.21.15.17:0/1251834602 wait complete. 2024-09-10T21:25:01.950 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-10T21:25:01.950 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-09-10T21:25:01.950 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T21:25:02.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:02.098+0000 7ff836400640 1 -- 172.21.15.17:0/2113106664 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff8301524d0 msgr2=0x7ff83015bf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:02.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:02.098+0000 7ff836400640 1 --2- 172.21.15.17:0/2113106664 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff8301524d0 0x7ff83015bf00 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7ff820009840 tx=0x7ff82001f010 comp rx=0 tx=0).stop 2024-09-10T21:25:02.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:02.098+0000 7ff836400640 1 -- 172.21.15.17:0/2113106664 shutdown_connections 2024-09-10T21:25:02.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:02.098+0000 7ff836400640 1 --2- 172.21.15.17:0/2113106664 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff83015c440 0x7ff830070d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:02.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:02.098+0000 7ff836400640 1 --2- 172.21.15.17:0/2113106664 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff8301524d0 0x7ff83015bf00 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:02.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:02.098+0000 7ff836400640 1 -- 172.21.15.17:0/2113106664 >> 172.21.15.17:0/2113106664 conn(0x7ff8300cdef0 msgr2=0x7ff8300ce2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:02.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:02.098+0000 7ff836400640 1 -- 172.21.15.17:0/2113106664 shutdown_connections 2024-09-10T21:25:02.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:02.098+0000 7ff836400640 1 -- 172.21.15.17:0/2113106664 wait complete. 2024-09-10T21:25:02.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:02.098+0000 7ff836400640 1 Processor -- start 2024-09-10T21:25:02.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:02.098+0000 7ff836400640 1 -- start start 2024-09-10T21:25:02.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:02.102+0000 7ff836400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff8301524d0 0x7ff8300c59a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:02.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:02.102+0000 7ff836400640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff830158c20 0x7ff8301515f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:02.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:02.102+0000 7ff836400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff83015c440 0x7ff830144e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:02.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:02.102+0000 7ff836400640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7ff8300c9350 con 0x7ff83015c440 2024-09-10T21:25:02.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:02.102+0000 7ff836400640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7ff8300c9070 con 0x7ff8301524d0 2024-09-10T21:25:02.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:02.102+0000 7ff836400640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7ff83015b9e0 con 0x7ff830158c20 2024-09-10T21:25:02.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:02.102+0000 7ff835a00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff83015c440 0x7ff830144e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:02.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:02.102+0000 7ff835a00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff83015c440 0x7ff830144e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49982/0 (socket says 172.21.15.17:49982) 2024-09-10T21:25:02.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:02.102+0000 7ff835a00640 1 -- 172.21.15.17:0/1599910400 learned_addr learned my addr 172.21.15.17:0/1599910400 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:25:02.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:02.102+0000 7ff835000640 1 --2- 172.21.15.17:0/1599910400 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff8301524d0 0x7ff8300c59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:02.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:02.102+0000 7ff827e00640 1 --2- 172.21.15.17:0/1599910400 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff830158c20 0x7ff8301515f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:02.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:02.102+0000 7ff835a00640 1 -- 172.21.15.17:0/1599910400 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff830158c20 msgr2=0x7ff8301515f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:02.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:02.102+0000 7ff835a00640 1 --2- 172.21.15.17:0/1599910400 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff830158c20 0x7ff8301515f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:02.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:02.102+0000 7ff835a00640 1 -- 172.21.15.17:0/1599910400 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff8301524d0 msgr2=0x7ff8300c59a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:02.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:02.102+0000 7ff835a00640 1 --2- 172.21.15.17:0/1599910400 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff8301524d0 0x7ff8300c59a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:02.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:02.102+0000 7ff835a00640 1 -- 172.21.15.17:0/1599910400 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8300c6250 con 0x7ff83015c440 2024-09-10T21:25:02.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:02.102+0000 7ff835000640 1 --2- 172.21.15.17:0/1599910400 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff8301524d0 0x7ff8300c59a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:25:02.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:02.102+0000 7ff835a00640 1 --2- 172.21.15.17:0/1599910400 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff83015c440 0x7ff830144e70 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ff828009fd0 tx=0x7ff82800ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:02.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:02.102+0000 7ff82d600640 1 -- 172.21.15.17:0/1599910400 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff82800ce20 con 0x7ff83015c440 2024-09-10T21:25:02.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:02.102+0000 7ff82d600640 1 -- 172.21.15.17:0/1599910400 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff828004b30 con 0x7ff83015c440 2024-09-10T21:25:02.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:02.102+0000 7ff836400640 1 -- 172.21.15.17:0/1599910400 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff83015a4b0 con 0x7ff83015c440 2024-09-10T21:25:02.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:02.102+0000 7ff82d600640 1 -- 172.21.15.17:0/1599910400 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff828005020 con 0x7ff83015c440 2024-09-10T21:25:02.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:02.102+0000 7ff836400640 1 -- 172.21.15.17:0/1599910400 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff83015aa70 con 0x7ff83015c440 2024-09-10T21:25:02.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:02.102+0000 7ff836400640 1 -- 172.21.15.17:0/1599910400 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8300b3e00 con 0x7ff83015c440 2024-09-10T21:25:02.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:02.102+0000 7ff82d600640 1 -- 172.21.15.17:0/1599910400 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7ff8280051c0 con 0x7ff83015c440 2024-09-10T21:25:02.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:02.106+0000 7ff82d600640 1 --2- 172.21.15.17:0/1599910400 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7ff80c0725e0 0x7ff80c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:02.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:02.106+0000 7ff82d600640 1 -- 172.21.15.17:0/1599910400 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7ff828094310 con 0x7ff83015c440 2024-09-10T21:25:02.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:02.106+0000 7ff835000640 1 --2- 172.21.15.17:0/1599910400 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7ff80c0725e0 0x7ff80c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:02.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:02.106+0000 7ff835000640 1 --2- 172.21.15.17:0/1599910400 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7ff80c0725e0 0x7ff80c074aa0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ff8200b2000 tx=0x7ff82001d850 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:02.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:02.110+0000 7ff82d600640 1 -- 172.21.15.17:0/1599910400 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7ff82805f590 con 0x7ff83015c440 2024-09-10T21:25:02.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:02.322+0000 7ff836400640 1 -- 172.21.15.17:0/1599910400 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7ff8300b39d0 con 0x7ff83015c440 2024-09-10T21:25:02.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:02.322+0000 7ff82d600640 1 -- 172.21.15.17:0/1599910400 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) ==== 53+0+88 (secure 0 0 0) 0x7ff82805ee90 con 0x7ff83015c440 2024-09-10T21:25:02.326 INFO:teuthology.orchestra.run.smithi017.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-09-10T21:25:02.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:02.322+0000 7ff836400640 1 -- 172.21.15.17:0/1599910400 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7ff80c0725e0 msgr2=0x7ff80c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:02.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:02.322+0000 7ff836400640 1 --2- 172.21.15.17:0/1599910400 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7ff80c0725e0 0x7ff80c074aa0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ff8200b2000 tx=0x7ff82001d850 comp rx=0 tx=0).stop 2024-09-10T21:25:02.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:02.322+0000 7ff836400640 1 -- 172.21.15.17:0/1599910400 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff83015c440 msgr2=0x7ff830144e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:02.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:02.322+0000 7ff836400640 1 --2- 172.21.15.17:0/1599910400 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff83015c440 0x7ff830144e70 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ff828009fd0 tx=0x7ff82800ef10 comp rx=0 tx=0).stop 2024-09-10T21:25:02.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:02.322+0000 7ff836400640 1 -- 172.21.15.17:0/1599910400 shutdown_connections 2024-09-10T21:25:02.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:02.322+0000 7ff836400640 1 --2- 172.21.15.17:0/1599910400 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7ff80c0725e0 0x7ff80c074aa0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:02.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:02.322+0000 7ff836400640 1 --2- 172.21.15.17:0/1599910400 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff83015c440 0x7ff830144e70 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:02.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:02.322+0000 7ff836400640 1 --2- 172.21.15.17:0/1599910400 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff830158c20 0x7ff8301515f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:02.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:02.322+0000 7ff836400640 1 --2- 172.21.15.17:0/1599910400 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff8301524d0 0x7ff8300c59a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:02.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:02.326+0000 7ff836400640 1 -- 172.21.15.17:0/1599910400 >> 172.21.15.17:0/1599910400 conn(0x7ff8300cdef0 msgr2=0x7ff83014f1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:02.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:02.326+0000 7ff836400640 1 -- 172.21.15.17:0/1599910400 shutdown_connections 2024-09-10T21:25:02.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:02.326+0000 7ff836400640 1 -- 172.21.15.17:0/1599910400 wait complete. 2024-09-10T21:25:02.348 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-10T21:25:02.348 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T21:25:02.348 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi017.front.sepia.ceph.com 2024-09-10T21:25:02.348 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-09-10T21:25:02.348 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-09-10T21:25:02.349 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-10T21:25:02.349 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-10T21:25:02.349 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T21:25:02.349 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-09-10T21:25:02.349 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:25:02.349 DEBUG:teuthology.orchestra.run.smithi017:> ip addr 2024-09-10T21:25:02.358 INFO:teuthology.orchestra.run.smithi017.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T21:25:02.359 INFO:teuthology.orchestra.run.smithi017.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T21:25:02.359 INFO:teuthology.orchestra.run.smithi017.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T21:25:02.359 INFO:teuthology.orchestra.run.smithi017.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:25:02.359 INFO:teuthology.orchestra.run.smithi017.stdout: inet6 ::1/128 scope host 2024-09-10T21:25:02.359 INFO:teuthology.orchestra.run.smithi017.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:25:02.359 INFO:teuthology.orchestra.run.smithi017.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T21:25:02.359 INFO:teuthology.orchestra.run.smithi017.stdout: link/ether 0c:c4:7a:6b:fe:00 brd ff:ff:ff:ff:ff:ff 2024-09-10T21:25:02.359 INFO:teuthology.orchestra.run.smithi017.stdout: altname enp7s0f0 2024-09-10T21:25:02.359 INFO:teuthology.orchestra.run.smithi017.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T21:25:02.359 INFO:teuthology.orchestra.run.smithi017.stdout: link/ether 0c:c4:7a:6b:fe:01 brd ff:ff:ff:ff:ff:ff 2024-09-10T21:25:02.359 INFO:teuthology.orchestra.run.smithi017.stdout: altname enp7s0f1 2024-09-10T21:25:02.359 INFO:teuthology.orchestra.run.smithi017.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T21:25:02.360 INFO:teuthology.orchestra.run.smithi017.stdout: link/ether 0c:c4:7a:59:d9:92 brd ff:ff:ff:ff:ff:ff 2024-09-10T21:25:02.360 INFO:teuthology.orchestra.run.smithi017.stdout: altname enp1s0f0 2024-09-10T21:25:02.360 INFO:teuthology.orchestra.run.smithi017.stdout: inet 172.21.15.17/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-09-10T21:25:02.360 INFO:teuthology.orchestra.run.smithi017.stdout: valid_lft 42415sec preferred_lft 42415sec 2024-09-10T21:25:02.360 INFO:teuthology.orchestra.run.smithi017.stdout: inet6 fe80::ec4:7aff:fe59:d992/64 scope link 2024-09-10T21:25:02.360 INFO:teuthology.orchestra.run.smithi017.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:25:02.360 INFO:teuthology.orchestra.run.smithi017.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T21:25:02.360 INFO:teuthology.orchestra.run.smithi017.stdout: link/ether 0c:c4:7a:59:d9:93 brd ff:ff:ff:ff:ff:ff 2024-09-10T21:25:02.360 INFO:teuthology.orchestra.run.smithi017.stdout: altname enp1s0f1 2024-09-10T21:25:02.360 INFO:teuthology.orchestra.run.smithi017.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-10T21:25:02.360 INFO:teuthology.orchestra.run.smithi017.stdout: link/ether 02:42:96:fe:6a:38 brd ff:ff:ff:ff:ff:ff 2024-09-10T21:25:02.360 INFO:teuthology.orchestra.run.smithi017.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-10T21:25:02.361 INFO:teuthology.orchestra.run.smithi017.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:25:02.361 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-10T21:25:02.361 DEBUG:teuthology.orchestra.run.smithi017:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T21:25:02.361 DEBUG:teuthology.orchestra.run.smithi017:> set -e 2024-09-10T21:25:02.362 DEBUG:teuthology.orchestra.run.smithi017:> sudo ip link add name ceph-brx type bridge 2024-09-10T21:25:02.362 DEBUG:teuthology.orchestra.run.smithi017:> sudo ip addr flush dev ceph-brx 2024-09-10T21:25:02.362 DEBUG:teuthology.orchestra.run.smithi017:> sudo ip link set ceph-brx up 2024-09-10T21:25:02.362 DEBUG:teuthology.orchestra.run.smithi017:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-10T21:25:02.362 DEBUG:teuthology.orchestra.run.smithi017:> ') 2024-09-10T21:25:02.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:02 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T21:25:02.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:25:02.518 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:25:02.518 DEBUG:teuthology.orchestra.run.smithi017:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T21:25:02.582 INFO:teuthology.orchestra.run.smithi017.stdout:1 2024-09-10T21:25:02.584 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:25:02.584 DEBUG:teuthology.orchestra.run.smithi017:> route 2024-09-10T21:25:02.653 INFO:teuthology.orchestra.run.smithi017.stdout:Kernel IP routing table 2024-09-10T21:25:02.653 INFO:teuthology.orchestra.run.smithi017.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T21:25:02.653 INFO:teuthology.orchestra.run.smithi017.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-09-10T21:25:02.653 INFO:teuthology.orchestra.run.smithi017.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-10T21:25:02.653 INFO:teuthology.orchestra.run.smithi017.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-09-10T21:25:02.653 INFO:teuthology.orchestra.run.smithi017.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T21:25:02.654 DEBUG:teuthology.orchestra.run.smithi017:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T21:25:02.654 DEBUG:teuthology.orchestra.run.smithi017:> set -e 2024-09-10T21:25:02.654 DEBUG:teuthology.orchestra.run.smithi017:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-10T21:25:02.654 DEBUG:teuthology.orchestra.run.smithi017:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-10T21:25:02.654 DEBUG:teuthology.orchestra.run.smithi017:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-10T21:25:02.654 DEBUG:teuthology.orchestra.run.smithi017:> ') 2024-09-10T21:25:02.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:02 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T21:25:02.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:25:02.795 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:25:02.795 DEBUG:teuthology.orchestra.run.smithi017:> ip netns list 2024-09-10T21:25:02.805 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:25:02.805 DEBUG:teuthology.orchestra.run.smithi017:> ip netns list-id 2024-09-10T21:25:02.821 INFO:tasks.ceph.mds.d.smithi031.stderr:2024-09-10T21:25:02.814+0000 7ffa47400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-10T21:25:02.857 DEBUG:teuthology.orchestra.run.smithi017:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T21:25:02.857 DEBUG:teuthology.orchestra.run.smithi017:> set -e 2024-09-10T21:25:02.857 DEBUG:teuthology.orchestra.run.smithi017:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-10T21:25:02.857 DEBUG:teuthology.orchestra.run.smithi017:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-09-10T21:25:02.857 DEBUG:teuthology.orchestra.run.smithi017:> ') 2024-09-10T21:25:02.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:02 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T21:25:02.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:25:02.985 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-09-10T21:25:02.985 DEBUG:teuthology.orchestra.run.smithi017:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T21:25:02.985 DEBUG:teuthology.orchestra.run.smithi017:> set -e 2024-09-10T21:25:02.986 DEBUG:teuthology.orchestra.run.smithi017:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-09-10T21:25:02.986 DEBUG:teuthology.orchestra.run.smithi017:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-10T21:25:02.986 DEBUG:teuthology.orchestra.run.smithi017:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-09-10T21:25:02.986 DEBUG:teuthology.orchestra.run.smithi017:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-09-10T21:25:02.986 DEBUG:teuthology.orchestra.run.smithi017:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-09-10T21:25:02.986 DEBUG:teuthology.orchestra.run.smithi017:> ') 2024-09-10T21:25:03.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:03 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T21:25:03.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:25:03.165 DEBUG:teuthology.orchestra.run.smithi017:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T21:25:03.165 DEBUG:teuthology.orchestra.run.smithi017:> set -e 2024-09-10T21:25:03.165 DEBUG:teuthology.orchestra.run.smithi017:> sudo ip link set brx.0 up 2024-09-10T21:25:03.166 DEBUG:teuthology.orchestra.run.smithi017:> sudo ip link set dev brx.0 master ceph-brx 2024-09-10T21:25:03.166 DEBUG:teuthology.orchestra.run.smithi017:> ') 2024-09-10T21:25:03.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:03 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T21:25:03.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:25:03.251 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-09-10T21:25:03.251 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T21:25:03.251 DEBUG:teuthology.orchestra.run.smithi017:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-09-10T21:25:03.260 INFO:teuthology.orchestra.run.smithi017.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-09-10T21:25:03.260 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T21:25:03.260 DEBUG:teuthology.orchestra.run.smithi017:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-09-10T21:25:03.312 DEBUG:teuthology.orchestra.run.smithi017:> sudo modprobe fuse 2024-09-10T21:25:03.380 DEBUG:teuthology.orchestra.run.smithi017:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T21:25:03.434 INFO:teuthology.orchestra.run.smithi017.stdout:/sys 2024-09-10T21:25:03.434 INFO:teuthology.orchestra.run.smithi017.stdout:/proc 2024-09-10T21:25:03.434 INFO:teuthology.orchestra.run.smithi017.stdout:/dev 2024-09-10T21:25:03.434 INFO:teuthology.orchestra.run.smithi017.stdout:/dev/pts 2024-09-10T21:25:03.434 INFO:teuthology.orchestra.run.smithi017.stdout:/run 2024-09-10T21:25:03.434 INFO:teuthology.orchestra.run.smithi017.stdout:/ 2024-09-10T21:25:03.434 INFO:teuthology.orchestra.run.smithi017.stdout:/sys/kernel/security 2024-09-10T21:25:03.434 INFO:teuthology.orchestra.run.smithi017.stdout:/dev/shm 2024-09-10T21:25:03.434 INFO:teuthology.orchestra.run.smithi017.stdout:/run/lock 2024-09-10T21:25:03.434 INFO:teuthology.orchestra.run.smithi017.stdout:/sys/fs/cgroup 2024-09-10T21:25:03.434 INFO:teuthology.orchestra.run.smithi017.stdout:/sys/fs/pstore 2024-09-10T21:25:03.434 INFO:teuthology.orchestra.run.smithi017.stdout:/sys/fs/bpf 2024-09-10T21:25:03.435 INFO:teuthology.orchestra.run.smithi017.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T21:25:03.435 INFO:teuthology.orchestra.run.smithi017.stdout:/dev/hugepages 2024-09-10T21:25:03.435 INFO:teuthology.orchestra.run.smithi017.stdout:/dev/mqueue 2024-09-10T21:25:03.435 INFO:teuthology.orchestra.run.smithi017.stdout:/sys/kernel/debug 2024-09-10T21:25:03.435 INFO:teuthology.orchestra.run.smithi017.stdout:/sys/kernel/tracing 2024-09-10T21:25:03.435 INFO:teuthology.orchestra.run.smithi017.stdout:/proc/fs/nfsd 2024-09-10T21:25:03.435 INFO:teuthology.orchestra.run.smithi017.stdout:/sys/fs/fuse/connections 2024-09-10T21:25:03.435 INFO:teuthology.orchestra.run.smithi017.stdout:/sys/kernel/config 2024-09-10T21:25:03.435 INFO:teuthology.orchestra.run.smithi017.stdout:/run/credentials/systemd-sysusers.service 2024-09-10T21:25:03.435 INFO:teuthology.orchestra.run.smithi017.stdout:/run/rpc_pipefs 2024-09-10T21:25:03.435 INFO:teuthology.orchestra.run.smithi017.stdout:/run/user/1000 2024-09-10T21:25:03.435 INFO:teuthology.orchestra.run.smithi017.stdout:/var/lib/ceph 2024-09-10T21:25:03.435 INFO:teuthology.orchestra.run.smithi017.stdout:/var/lib/ceph/osd/ceph-0 2024-09-10T21:25:03.435 INFO:teuthology.orchestra.run.smithi017.stdout:/var/lib/ceph/osd/ceph-1 2024-09-10T21:25:03.436 INFO:teuthology.orchestra.run.smithi017.stdout:/var/lib/ceph/osd/ceph-2 2024-09-10T21:25:03.436 INFO:teuthology.orchestra.run.smithi017.stdout:/var/lib/ceph/osd/ceph-3 2024-09-10T21:25:03.436 INFO:teuthology.orchestra.run.smithi017.stdout:/run/netns 2024-09-10T21:25:03.436 INFO:teuthology.orchestra.run.smithi017.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-10T21:25:03.436 INFO:teuthology.orchestra.run.smithi017.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-10T21:25:03.437 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:25:03.437 DEBUG:teuthology.orchestra.run.smithi017:> ls /sys/fs/fuse/connections 2024-09-10T21:25:03.488 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-10T21:25:03.488 DEBUG:teuthology.orchestra.run.smithi017:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-09-10T21:25:03.535 DEBUG:teuthology.orchestra.run.smithi017:> sudo modprobe fuse 2024-09-10T21:25:03.556 DEBUG:teuthology.orchestra.run.smithi017:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T21:25:03.609 INFO:teuthology.orchestra.run.smithi017.stdout:/sys 2024-09-10T21:25:03.609 INFO:teuthology.orchestra.run.smithi017.stdout:/proc 2024-09-10T21:25:03.609 INFO:teuthology.orchestra.run.smithi017.stdout:/dev 2024-09-10T21:25:03.609 INFO:teuthology.orchestra.run.smithi017.stdout:/dev/pts 2024-09-10T21:25:03.609 INFO:teuthology.orchestra.run.smithi017.stdout:/run 2024-09-10T21:25:03.610 INFO:teuthology.orchestra.run.smithi017.stdout:/ 2024-09-10T21:25:03.610 INFO:teuthology.orchestra.run.smithi017.stdout:/sys/kernel/security 2024-09-10T21:25:03.610 INFO:teuthology.orchestra.run.smithi017.stdout:/dev/shm 2024-09-10T21:25:03.610 INFO:teuthology.orchestra.run.smithi017.stdout:/run/lock 2024-09-10T21:25:03.610 INFO:teuthology.orchestra.run.smithi017.stdout:/sys/fs/cgroup 2024-09-10T21:25:03.610 INFO:teuthology.orchestra.run.smithi017.stdout:/sys/fs/pstore 2024-09-10T21:25:03.610 INFO:teuthology.orchestra.run.smithi017.stdout:/sys/fs/bpf 2024-09-10T21:25:03.610 INFO:teuthology.orchestra.run.smithi017.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T21:25:03.610 INFO:teuthology.orchestra.run.smithi017.stdout:/dev/hugepages 2024-09-10T21:25:03.610 INFO:teuthology.orchestra.run.smithi017.stdout:/dev/mqueue 2024-09-10T21:25:03.610 INFO:teuthology.orchestra.run.smithi017.stdout:/sys/kernel/debug 2024-09-10T21:25:03.610 INFO:teuthology.orchestra.run.smithi017.stdout:/sys/kernel/tracing 2024-09-10T21:25:03.610 INFO:teuthology.orchestra.run.smithi017.stdout:/proc/fs/nfsd 2024-09-10T21:25:03.611 INFO:teuthology.orchestra.run.smithi017.stdout:/sys/fs/fuse/connections 2024-09-10T21:25:03.611 INFO:teuthology.orchestra.run.smithi017.stdout:/sys/kernel/config 2024-09-10T21:25:03.611 INFO:teuthology.orchestra.run.smithi017.stdout:/run/credentials/systemd-sysusers.service 2024-09-10T21:25:03.611 INFO:teuthology.orchestra.run.smithi017.stdout:/run/rpc_pipefs 2024-09-10T21:25:03.611 INFO:teuthology.orchestra.run.smithi017.stdout:/run/user/1000 2024-09-10T21:25:03.611 INFO:teuthology.orchestra.run.smithi017.stdout:/var/lib/ceph 2024-09-10T21:25:03.611 INFO:teuthology.orchestra.run.smithi017.stdout:/var/lib/ceph/osd/ceph-0 2024-09-10T21:25:03.611 INFO:teuthology.orchestra.run.smithi017.stdout:/var/lib/ceph/osd/ceph-1 2024-09-10T21:25:03.611 INFO:teuthology.orchestra.run.smithi017.stdout:/var/lib/ceph/osd/ceph-2 2024-09-10T21:25:03.611 INFO:teuthology.orchestra.run.smithi017.stdout:/var/lib/ceph/osd/ceph-3 2024-09-10T21:25:03.611 INFO:teuthology.orchestra.run.smithi017.stdout:/run/netns 2024-09-10T21:25:03.611 INFO:teuthology.orchestra.run.smithi017.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-10T21:25:03.611 INFO:teuthology.orchestra.run.smithi017.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-10T21:25:03.612 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:25:03.612 DEBUG:teuthology.orchestra.run.smithi017:> ls /sys/fs/fuse/connections 2024-09-10T21:25:03.634 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi017.stderr:2024-09-10T21:25:03.630+0000 7f0b4890f580 -1 init, newargv = 0x555e228132a0 newargc=15 2024-09-10T21:25:03.634 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi017.stderr:ceph-fuse[33694]: starting ceph client 2024-09-10T21:25:03.645 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi017.stderr:ceph-fuse[33694]: starting fuse 2024-09-10T21:25:04.744 DEBUG:teuthology.orchestra.run.smithi017:> sudo modprobe fuse 2024-09-10T21:25:04.769 DEBUG:teuthology.orchestra.run.smithi017:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T21:25:04.827 INFO:teuthology.orchestra.run.smithi017.stdout:/sys 2024-09-10T21:25:04.827 INFO:teuthology.orchestra.run.smithi017.stdout:/proc 2024-09-10T21:25:04.827 INFO:teuthology.orchestra.run.smithi017.stdout:/dev 2024-09-10T21:25:04.827 INFO:teuthology.orchestra.run.smithi017.stdout:/dev/pts 2024-09-10T21:25:04.827 INFO:teuthology.orchestra.run.smithi017.stdout:/run 2024-09-10T21:25:04.827 INFO:teuthology.orchestra.run.smithi017.stdout:/ 2024-09-10T21:25:04.827 INFO:teuthology.orchestra.run.smithi017.stdout:/sys/kernel/security 2024-09-10T21:25:04.827 INFO:teuthology.orchestra.run.smithi017.stdout:/dev/shm 2024-09-10T21:25:04.827 INFO:teuthology.orchestra.run.smithi017.stdout:/run/lock 2024-09-10T21:25:04.828 INFO:teuthology.orchestra.run.smithi017.stdout:/sys/fs/cgroup 2024-09-10T21:25:04.828 INFO:teuthology.orchestra.run.smithi017.stdout:/sys/fs/pstore 2024-09-10T21:25:04.828 INFO:teuthology.orchestra.run.smithi017.stdout:/sys/fs/bpf 2024-09-10T21:25:04.828 INFO:teuthology.orchestra.run.smithi017.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T21:25:04.828 INFO:teuthology.orchestra.run.smithi017.stdout:/dev/hugepages 2024-09-10T21:25:04.828 INFO:teuthology.orchestra.run.smithi017.stdout:/dev/mqueue 2024-09-10T21:25:04.828 INFO:teuthology.orchestra.run.smithi017.stdout:/sys/kernel/debug 2024-09-10T21:25:04.828 INFO:teuthology.orchestra.run.smithi017.stdout:/sys/kernel/tracing 2024-09-10T21:25:04.828 INFO:teuthology.orchestra.run.smithi017.stdout:/proc/fs/nfsd 2024-09-10T21:25:04.828 INFO:teuthology.orchestra.run.smithi017.stdout:/sys/fs/fuse/connections 2024-09-10T21:25:04.828 INFO:teuthology.orchestra.run.smithi017.stdout:/sys/kernel/config 2024-09-10T21:25:04.828 INFO:teuthology.orchestra.run.smithi017.stdout:/run/credentials/systemd-sysusers.service 2024-09-10T21:25:04.828 INFO:teuthology.orchestra.run.smithi017.stdout:/run/rpc_pipefs 2024-09-10T21:25:04.829 INFO:teuthology.orchestra.run.smithi017.stdout:/run/user/1000 2024-09-10T21:25:04.829 INFO:teuthology.orchestra.run.smithi017.stdout:/var/lib/ceph 2024-09-10T21:25:04.829 INFO:teuthology.orchestra.run.smithi017.stdout:/var/lib/ceph/osd/ceph-0 2024-09-10T21:25:04.829 INFO:teuthology.orchestra.run.smithi017.stdout:/var/lib/ceph/osd/ceph-1 2024-09-10T21:25:04.829 INFO:teuthology.orchestra.run.smithi017.stdout:/var/lib/ceph/osd/ceph-2 2024-09-10T21:25:04.829 INFO:teuthology.orchestra.run.smithi017.stdout:/var/lib/ceph/osd/ceph-3 2024-09-10T21:25:04.829 INFO:teuthology.orchestra.run.smithi017.stdout:/run/netns 2024-09-10T21:25:04.829 INFO:teuthology.orchestra.run.smithi017.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-10T21:25:04.829 INFO:teuthology.orchestra.run.smithi017.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-10T21:25:04.829 INFO:teuthology.orchestra.run.smithi017.stdout:/home/ubuntu/cephtest/mnt.admin 2024-09-10T21:25:04.830 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:25:04.830 DEBUG:teuthology.orchestra.run.smithi017:> ls /sys/fs/fuse/connections 2024-09-10T21:25:04.881 INFO:teuthology.orchestra.run.smithi017.stdout:44 2024-09-10T21:25:04.881 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [44] 2024-09-10T21:25:04.882 DEBUG:teuthology.orchestra.run.smithi017:> sudo stdin-killer -- python3 -c ' 2024-09-10T21:25:04.882 DEBUG:teuthology.orchestra.run.smithi017:> import glob 2024-09-10T21:25:04.882 DEBUG:teuthology.orchestra.run.smithi017:> import re 2024-09-10T21:25:04.882 DEBUG:teuthology.orchestra.run.smithi017:> import os 2024-09-10T21:25:04.882 DEBUG:teuthology.orchestra.run.smithi017:> import subprocess 2024-09-10T21:25:04.882 DEBUG:teuthology.orchestra.run.smithi017:> 2024-09-10T21:25:04.882 DEBUG:teuthology.orchestra.run.smithi017:> def _find_admin_socket(client_name): 2024-09-10T21:25:04.882 DEBUG:teuthology.orchestra.run.smithi017:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-09-10T21:25:04.882 DEBUG:teuthology.orchestra.run.smithi017:> files = glob.glob(asok_path) 2024-09-10T21:25:04.882 DEBUG:teuthology.orchestra.run.smithi017:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-09-10T21:25:04.882 DEBUG:teuthology.orchestra.run.smithi017:> 2024-09-10T21:25:04.882 DEBUG:teuthology.orchestra.run.smithi017:> # Given a non-glob path, it better be there 2024-09-10T21:25:04.882 DEBUG:teuthology.orchestra.run.smithi017:> if "*" not in asok_path: 2024-09-10T21:25:04.883 DEBUG:teuthology.orchestra.run.smithi017:> assert(len(files) == 1) 2024-09-10T21:25:04.883 DEBUG:teuthology.orchestra.run.smithi017:> return files[0] 2024-09-10T21:25:04.883 DEBUG:teuthology.orchestra.run.smithi017:> 2024-09-10T21:25:04.883 DEBUG:teuthology.orchestra.run.smithi017:> for f in files: 2024-09-10T21:25:04.883 DEBUG:teuthology.orchestra.run.smithi017:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-10T21:25:04.883 DEBUG:teuthology.orchestra.run.smithi017:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-10T21:25:04.883 DEBUG:teuthology.orchestra.run.smithi017:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-10T21:25:04.883 DEBUG:teuthology.orchestra.run.smithi017:> contents = proc_f.read() 2024-09-10T21:25:04.883 DEBUG:teuthology.orchestra.run.smithi017:> if mountpoint in contents: 2024-09-10T21:25:04.883 DEBUG:teuthology.orchestra.run.smithi017:> return f 2024-09-10T21:25:04.883 DEBUG:teuthology.orchestra.run.smithi017:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-10T21:25:04.883 DEBUG:teuthology.orchestra.run.smithi017:> 2024-09-10T21:25:04.883 DEBUG:teuthology.orchestra.run.smithi017:> print(_find_admin_socket("client.admin")) 2024-09-10T21:25:04.883 DEBUG:teuthology.orchestra.run.smithi017:> ' 2024-09-10T21:25:05.016 INFO:teuthology.orchestra.run.smithi017.stdout:/var/run/ceph/ceph-client.admin.33694.asok 2024-09-10T21:25:05.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:25:05.029 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.33694.asok 2024-09-10T21:25:05.029 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:25:05.029 DEBUG:teuthology.orchestra.run.smithi017:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.33694.asok status 2024-09-10T21:25:05.207 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-10T21:25:05.207 INFO:teuthology.orchestra.run.smithi017.stdout: "metadata": { 2024-09-10T21:25:05.207 INFO:teuthology.orchestra.run.smithi017.stdout: "ceph_sha1": "8f219c476cb4f26fc5719c661c0323719a45cb30", 2024-09-10T21:25:05.207 INFO:teuthology.orchestra.run.smithi017.stdout: "ceph_version": "ceph version 19.3.0-4854-g8f219c47 (8f219c476cb4f26fc5719c661c0323719a45cb30) squid (dev)", 2024-09-10T21:25:05.207 INFO:teuthology.orchestra.run.smithi017.stdout: "entity_id": "admin", 2024-09-10T21:25:05.208 INFO:teuthology.orchestra.run.smithi017.stdout: "hostname": "smithi017", 2024-09-10T21:25:05.208 INFO:teuthology.orchestra.run.smithi017.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-09-10T21:25:05.208 INFO:teuthology.orchestra.run.smithi017.stdout: "pid": "33694", 2024-09-10T21:25:05.208 INFO:teuthology.orchestra.run.smithi017.stdout: "root": "/" 2024-09-10T21:25:05.208 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:25:05.208 INFO:teuthology.orchestra.run.smithi017.stdout: "dentry_count": 0, 2024-09-10T21:25:05.208 INFO:teuthology.orchestra.run.smithi017.stdout: "dentry_pinned_count": 0, 2024-09-10T21:25:05.208 INFO:teuthology.orchestra.run.smithi017.stdout: "id": 4503, 2024-09-10T21:25:05.208 INFO:teuthology.orchestra.run.smithi017.stdout: "inst": { 2024-09-10T21:25:05.208 INFO:teuthology.orchestra.run.smithi017.stdout: "name": { 2024-09-10T21:25:05.208 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "client", 2024-09-10T21:25:05.208 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 4503 2024-09-10T21:25:05.208 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:25:05.208 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": { 2024-09-10T21:25:05.209 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:25:05.209 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "192.168.0.1:0", 2024-09-10T21:25:05.209 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 344279998 2024-09-10T21:25:05.209 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:25:05.209 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:25:05.209 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": { 2024-09-10T21:25:05.209 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:25:05.209 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "192.168.0.1:0", 2024-09-10T21:25:05.209 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 344279998 2024-09-10T21:25:05.209 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:25:05.209 INFO:teuthology.orchestra.run.smithi017.stdout: "inst_str": "client.4503 192.168.0.1:0/344279998", 2024-09-10T21:25:05.209 INFO:teuthology.orchestra.run.smithi017.stdout: "addr_str": "192.168.0.1:0/344279998", 2024-09-10T21:25:05.209 INFO:teuthology.orchestra.run.smithi017.stdout: "inode_count": 1, 2024-09-10T21:25:05.210 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_epoch": 6, 2024-09-10T21:25:05.210 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_epoch": 25, 2024-09-10T21:25:05.210 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_epoch_barrier": 0, 2024-09-10T21:25:05.210 INFO:teuthology.orchestra.run.smithi017.stdout: "blocklisted": false, 2024-09-10T21:25:05.210 INFO:teuthology.orchestra.run.smithi017.stdout: "fs_name": "cephfs" 2024-09-10T21:25:05.210 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-10T21:25:05.222 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:25:05.223 DEBUG:teuthology.orchestra.run.smithi017:> stat --file-system '--printf=%T 2024-09-10T21:25:05.223 DEBUG:teuthology.orchestra.run.smithi017:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-09-10T21:25:05.232 INFO:teuthology.orchestra.run.smithi017.stdout:fuseblk 2024-09-10T21:25:05.232 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-09-10T21:25:05.232 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:25:05.232 DEBUG:teuthology.orchestra.run.smithi017:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-09-10T21:25:05.301 DEBUG:teuthology.orchestra.run.smithi017:> (cd /home/ubuntu/cephtest/mnt.admin && exec stdin-killer --timeout=900 -- bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-09-10T21:25:05.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:05 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-10T21:25:05.428 INFO:teuthology.orchestra.run.smithi017.stdout:# file: . 2024-09-10T21:25:05.428 INFO:teuthology.orchestra.run.smithi017.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-09-10T21:25:05.429 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-10T21:25:05.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:25:05.435 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-09-10T21:25:05.435 DEBUG:teuthology.orchestra.run.smithi017:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:25:05.444 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T21:25:05.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:05.630+0000 7fba06000640 1 -- 172.21.15.17:0/243769982 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fba001526b0 msgr2=0x7fba00159850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:05.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:05.630+0000 7fba06000640 1 --2- 172.21.15.17:0/243769982 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fba001526b0 0x7fba00159850 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fb9f0009900 tx=0x7fb9f001f070 comp rx=0 tx=0).stop 2024-09-10T21:25:05.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:05.630+0000 7fba06000640 1 -- 172.21.15.17:0/243769982 shutdown_connections 2024-09-10T21:25:05.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:05.630+0000 7fba06000640 1 --2- 172.21.15.17:0/243769982 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fba00071760 0x7fba000c4dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:05.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:05.630+0000 7fba06000640 1 --2- 172.21.15.17:0/243769982 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fba001526b0 0x7fba00159850 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:05.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:05.630+0000 7fba06000640 1 -- 172.21.15.17:0/243769982 >> 172.21.15.17:0/243769982 conn(0x7fba00068990 msgr2=0x7fba00068da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:05.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:05.630+0000 7fba06000640 1 -- 172.21.15.17:0/243769982 shutdown_connections 2024-09-10T21:25:05.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:05.630+0000 7fba06000640 1 -- 172.21.15.17:0/243769982 wait complete. 2024-09-10T21:25:05.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:05.630+0000 7fba06000640 1 Processor -- start 2024-09-10T21:25:05.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:05.630+0000 7fba06000640 1 -- start start 2024-09-10T21:25:05.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:05.634+0000 7fba06000640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba00070dd0 0x7fba0014e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:05.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:05.634+0000 7fba06000640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fba00071760 0x7fba0014d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:05.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:05.634+0000 7fba06000640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fba001526b0 0x7fba0006e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:05.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:05.634+0000 7fba06000640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fba00158f80 con 0x7fba001526b0 2024-09-10T21:25:05.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:05.634+0000 7fba06000640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fba00152a90 con 0x7fba00070dd0 2024-09-10T21:25:05.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:05.634+0000 7fba06000640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7fba00159260 con 0x7fba00071760 2024-09-10T21:25:05.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:05.634+0000 7fb9ffe00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fba001526b0 0x7fba0006e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:05.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:05.634+0000 7fb9ffe00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fba001526b0 0x7fba0006e990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40132/0 (socket says 172.21.15.17:40132) 2024-09-10T21:25:05.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:05.634+0000 7fb9ffe00640 1 -- 172.21.15.17:0/2699291421 learned_addr learned my addr 172.21.15.17:0/2699291421 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:25:05.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:05.634+0000 7fb9f7e00640 1 --2- 172.21.15.17:0/2699291421 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fba00071760 0x7fba0014d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:05.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:05.634+0000 7fb9ff400640 1 --2- 172.21.15.17:0/2699291421 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba00070dd0 0x7fba0014e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:05.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:05.634+0000 7fb9ffe00640 1 -- 172.21.15.17:0/2699291421 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fba00071760 msgr2=0x7fba0014d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:05.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:05.634+0000 7fb9ffe00640 1 --2- 172.21.15.17:0/2699291421 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fba00071760 0x7fba0014d570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:05.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:05.634+0000 7fb9ffe00640 1 -- 172.21.15.17:0/2699291421 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba00070dd0 msgr2=0x7fba0014e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:05.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:05.634+0000 7fb9ffe00640 1 --2- 172.21.15.17:0/2699291421 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba00070dd0 0x7fba0014e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:05.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:05.634+0000 7fb9ffe00640 1 -- 172.21.15.17:0/2699291421 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba001f1250 con 0x7fba001526b0 2024-09-10T21:25:05.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:05.634+0000 7fb9ffe00640 1 --2- 172.21.15.17:0/2699291421 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fba001526b0 0x7fba0006e990 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fb9f800c7b0 tx=0x7fb9f800cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:05.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:05.634+0000 7fb9f7400640 1 -- 172.21.15.17:0/2699291421 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9f8017070 con 0x7fba001526b0 2024-09-10T21:25:05.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:05.634+0000 7fb9f7400640 1 -- 172.21.15.17:0/2699291421 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb9f8005070 con 0x7fba001526b0 2024-09-10T21:25:05.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:05.634+0000 7fba06000640 1 -- 172.21.15.17:0/2699291421 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba001f1530 con 0x7fba001526b0 2024-09-10T21:25:05.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:05.634+0000 7fb9f7400640 1 -- 172.21.15.17:0/2699291421 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9f80054c0 con 0x7fba001526b0 2024-09-10T21:25:05.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:05.634+0000 7fba06000640 1 -- 172.21.15.17:0/2699291421 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba0006e190 con 0x7fba001526b0 2024-09-10T21:25:05.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:05.634+0000 7fba06000640 1 -- 172.21.15.17:0/2699291421 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba00071260 con 0x7fba001526b0 2024-09-10T21:25:05.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:05.638+0000 7fb9f7400640 1 -- 172.21.15.17:0/2699291421 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fb9f8005660 con 0x7fba001526b0 2024-09-10T21:25:05.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:05.638+0000 7fb9f7400640 1 --2- 172.21.15.17:0/2699291421 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fb9dc0725e0 0x7fb9dc074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:05.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:05.638+0000 7fb9f7400640 1 -- 172.21.15.17:0/2699291421 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fb9f8092fb0 con 0x7fba001526b0 2024-09-10T21:25:05.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:05.638+0000 7fb9ff400640 1 --2- 172.21.15.17:0/2699291421 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fb9dc0725e0 0x7fb9dc074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:05.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:05.638+0000 7fb9ff400640 1 --2- 172.21.15.17:0/2699291421 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fb9dc0725e0 0x7fb9dc074aa0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb9f00b2000 tx=0x7fb9f001d910 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:05.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:05.642+0000 7fb9f7400640 1 -- 172.21.15.17:0/2699291421 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fb9f805dbd0 con 0x7fba001526b0 2024-09-10T21:25:05.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:05.826+0000 7fba06000640 1 -- 172.21.15.17:0/2699291421 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fba00148230 con 0x7fba001526b0 2024-09-10T21:25:05.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:05.826+0000 7fb9f7400640 1 -- 172.21.15.17:0/2699291421 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v25) ==== 80+0+0 (secure 0 0 0) 0x7fb9f8062600 con 0x7fba001526b0 2024-09-10T21:25:05.832 INFO:teuthology.orchestra.run.smithi017.stderr:listed 0 entries 2024-09-10T21:25:05.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:05.830+0000 7fba06000640 1 -- 172.21.15.17:0/2699291421 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fb9dc0725e0 msgr2=0x7fb9dc074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:05.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:05.830+0000 7fba06000640 1 --2- 172.21.15.17:0/2699291421 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fb9dc0725e0 0x7fb9dc074aa0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fb9f00b2000 tx=0x7fb9f001d910 comp rx=0 tx=0).stop 2024-09-10T21:25:05.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:05.830+0000 7fba06000640 1 -- 172.21.15.17:0/2699291421 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fba001526b0 msgr2=0x7fba0006e990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:05.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:05.830+0000 7fba06000640 1 --2- 172.21.15.17:0/2699291421 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fba001526b0 0x7fba0006e990 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fb9f800c7b0 tx=0x7fb9f800cc70 comp rx=0 tx=0).stop 2024-09-10T21:25:05.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:05.830+0000 7fba06000640 1 -- 172.21.15.17:0/2699291421 shutdown_connections 2024-09-10T21:25:05.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:05.830+0000 7fba06000640 1 --2- 172.21.15.17:0/2699291421 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fb9dc0725e0 0x7fb9dc074aa0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:05.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:05.830+0000 7fba06000640 1 --2- 172.21.15.17:0/2699291421 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fba001526b0 0x7fba0006e990 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:05.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:05.830+0000 7fba06000640 1 --2- 172.21.15.17:0/2699291421 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fba00071760 0x7fba0014d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:05.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:05.830+0000 7fba06000640 1 --2- 172.21.15.17:0/2699291421 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba00070dd0 0x7fba0014e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:05.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:05.830+0000 7fba06000640 1 -- 172.21.15.17:0/2699291421 >> 172.21.15.17:0/2699291421 conn(0x7fba00068990 msgr2=0x7fba00068d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:05.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:05.830+0000 7fba06000640 1 -- 172.21.15.17:0/2699291421 shutdown_connections 2024-09-10T21:25:05.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:05.830+0000 7fba06000640 1 -- 172.21.15.17:0/2699291421 wait complete. 2024-09-10T21:25:05.854 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-09-10T21:25:05.854 DEBUG:teuthology.orchestra.run.smithi017:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:25:05.863 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T21:25:06.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:06.050+0000 7f222ce00640 1 -- 172.21.15.17:0/4225252748 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f22280cdf20 msgr2=0x7f22280ce350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:06.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:06.050+0000 7f222ce00640 1 --2- 172.21.15.17:0/4225252748 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f22280cdf20 0x7f22280ce350 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f2220009c80 tx=0x7f222001cd10 comp rx=0 tx=0).stop 2024-09-10T21:25:06.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:06.050+0000 7f222ce00640 1 -- 172.21.15.17:0/4225252748 shutdown_connections 2024-09-10T21:25:06.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:06.050+0000 7f222ce00640 1 --2- 172.21.15.17:0/4225252748 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f22280cdf20 0x7f22280ce350 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:06.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:06.050+0000 7f222ce00640 1 --2- 172.21.15.17:0/4225252748 >> v2:172.21.15.31:3300/0 conn(0x7f2228152440 0x7f22280b3eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:06.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:06.050+0000 7f222ce00640 1 -- 172.21.15.17:0/4225252748 >> 172.21.15.17:0/4225252748 conn(0x7f22280702f0 msgr2=0x7f2228070700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:06.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:06.050+0000 7f222ce00640 1 -- 172.21.15.17:0/4225252748 shutdown_connections 2024-09-10T21:25:06.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:06.050+0000 7f222ce00640 1 -- 172.21.15.17:0/4225252748 wait complete. 2024-09-10T21:25:06.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:06.054+0000 7f222ce00640 1 Processor -- start 2024-09-10T21:25:06.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:06.054+0000 7f222ce00640 1 -- start start 2024-09-10T21:25:06.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:06.054+0000 7f222ce00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22280cdf20 0x7f22281f4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:06.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:06.054+0000 7f222ce00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2228152440 0x7f22281556e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:06.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:06.054+0000 7f222ce00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f2228152820 0x7f222814aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:06.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:06.054+0000 7f222ce00640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f22280cb5d0 con 0x7f2228152440 2024-09-10T21:25:06.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:06.054+0000 7f222ce00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f22280cb450 con 0x7f22280cdf20 2024-09-10T21:25:06.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:06.054+0000 7f222ce00640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f22280c2ff0 con 0x7f2228152820 2024-09-10T21:25:06.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:06.054+0000 7f221d400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2228152440 0x7f22281556e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:06.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:06.054+0000 7f221d400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2228152440 0x7f22281556e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40138/0 (socket says 172.21.15.17:40138) 2024-09-10T21:25:06.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:06.054+0000 7f221d400640 1 -- 172.21.15.17:0/1495359431 learned_addr learned my addr 172.21.15.17:0/1495359431 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:25:06.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:06.054+0000 7f2225e00640 1 --2- 172.21.15.17:0/1495359431 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22280cdf20 0x7f22281f4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:06.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:06.054+0000 7f2226800640 1 --2- 172.21.15.17:0/1495359431 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f2228152820 0x7f222814aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:06.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:06.054+0000 7f221d400640 1 -- 172.21.15.17:0/1495359431 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f2228152820 msgr2=0x7f222814aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:06.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:06.054+0000 7f221d400640 1 --2- 172.21.15.17:0/1495359431 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f2228152820 0x7f222814aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:06.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:06.054+0000 7f221d400640 1 -- 172.21.15.17:0/1495359431 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22280cdf20 msgr2=0x7f22281f4ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:06.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:06.054+0000 7f221d400640 1 --2- 172.21.15.17:0/1495359431 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22280cdf20 0x7f22281f4ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:06.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:06.054+0000 7f221d400640 1 -- 172.21.15.17:0/1495359431 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22280727c0 con 0x7f2228152440 2024-09-10T21:25:06.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:06.054+0000 7f221d400640 1 --2- 172.21.15.17:0/1495359431 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2228152440 0x7f22281556e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f221000c530 tx=0x7f221000c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:06.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:06.054+0000 7f221ea00640 1 -- 172.21.15.17:0/1495359431 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2210017070 con 0x7f2228152440 2024-09-10T21:25:06.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:06.054+0000 7f221ea00640 1 -- 172.21.15.17:0/1495359431 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2210003600 con 0x7f2228152440 2024-09-10T21:25:06.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:06.054+0000 7f222ce00640 1 -- 172.21.15.17:0/1495359431 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22280cd400 con 0x7f2228152440 2024-09-10T21:25:06.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:06.054+0000 7f221ea00640 1 -- 172.21.15.17:0/1495359431 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2210003a50 con 0x7f2228152440 2024-09-10T21:25:06.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:06.054+0000 7f222ce00640 1 -- 172.21.15.17:0/1495359431 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22280cd9c0 con 0x7f2228152440 2024-09-10T21:25:06.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:06.058+0000 7f222ce00640 1 -- 172.21.15.17:0/1495359431 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2228158360 con 0x7f2228152440 2024-09-10T21:25:06.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:06.058+0000 7f221ea00640 1 -- 172.21.15.17:0/1495359431 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f2210003bf0 con 0x7f2228152440 2024-09-10T21:25:06.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:06.058+0000 7f221ea00640 1 --2- 172.21.15.17:0/1495359431 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f22040725e0 0x7f2204074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:06.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:06.058+0000 7f221ea00640 1 -- 172.21.15.17:0/1495359431 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f2210092c80 con 0x7f2228152440 2024-09-10T21:25:06.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:06.058+0000 7f2225e00640 1 --2- 172.21.15.17:0/1495359431 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f22040725e0 0x7f2204074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:06.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:06.058+0000 7f2225e00640 1 --2- 172.21.15.17:0/1495359431 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f22040725e0 0x7f2204074aa0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f2218009be0 tx=0x7f22180091e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:06.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:06.062+0000 7f221ea00640 1 -- 172.21.15.17:0/1495359431 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f221005d9f0 con 0x7f2228152440 2024-09-10T21:25:06.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:06.254+0000 7f222ce00640 1 -- 172.21.15.17:0/1495359431 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f2228072f70 con 0x7f2228152440 2024-09-10T21:25:06.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:06.254+0000 7f221ea00640 1 -- 172.21.15.17:0/1495359431 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v25) ==== 80+0+0 (secure 0 0 0) 0x7f2210062420 con 0x7f2228152440 2024-09-10T21:25:06.257 INFO:teuthology.orchestra.run.smithi017.stderr:listed 0 entries 2024-09-10T21:25:06.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:06.254+0000 7f222ce00640 1 -- 172.21.15.17:0/1495359431 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f22040725e0 msgr2=0x7f2204074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:06.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:06.254+0000 7f222ce00640 1 --2- 172.21.15.17:0/1495359431 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f22040725e0 0x7f2204074aa0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f2218009be0 tx=0x7f22180091e0 comp rx=0 tx=0).stop 2024-09-10T21:25:06.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:06.254+0000 7f222ce00640 1 -- 172.21.15.17:0/1495359431 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2228152440 msgr2=0x7f22281556e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:06.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:06.254+0000 7f222ce00640 1 --2- 172.21.15.17:0/1495359431 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2228152440 0x7f22281556e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f221000c530 tx=0x7f221000c9f0 comp rx=0 tx=0).stop 2024-09-10T21:25:06.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:06.254+0000 7f222ce00640 1 -- 172.21.15.17:0/1495359431 shutdown_connections 2024-09-10T21:25:06.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:06.254+0000 7f222ce00640 1 --2- 172.21.15.17:0/1495359431 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f22040725e0 0x7f2204074aa0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:06.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:06.254+0000 7f222ce00640 1 --2- 172.21.15.17:0/1495359431 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f2228152820 0x7f222814aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:06.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:06.254+0000 7f222ce00640 1 --2- 172.21.15.17:0/1495359431 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2228152440 0x7f22281556e0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:06.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:06.254+0000 7f222ce00640 1 --2- 172.21.15.17:0/1495359431 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f22280cdf20 0x7f22281f4ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:06.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:06.254+0000 7f222ce00640 1 -- 172.21.15.17:0/1495359431 >> 172.21.15.17:0/1495359431 conn(0x7f22280702f0 msgr2=0x7f2228155fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:06.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:06.254+0000 7f222ce00640 1 -- 172.21.15.17:0/1495359431 shutdown_connections 2024-09-10T21:25:06.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:06.254+0000 7f222ce00640 1 -- 172.21.15.17:0/1495359431 wait complete. 2024-09-10T21:25:06.279 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi017.front.sepia.ceph.com... 2024-09-10T21:25:06.279 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:25:06.280 DEBUG:teuthology.orchestra.run.smithi017:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-09-10T21:25:06.307 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi017.stderr:ceph-fuse[33694]: fuse finished with error 0 and tester_r 0 2024-09-10T21:25:06.310 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:25:12.412 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi017.front.sepia.ceph.com 2024-09-10T21:25:12.413 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:25:12.413 DEBUG:teuthology.orchestra.run.smithi017:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-09-10T21:25:12.433 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:25:12.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:12.630+0000 7fc07cc00640 1 -- 172.21.15.17:0/2005055469 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc07815b950 msgr2=0x7fc078158d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:12.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:12.630+0000 7fc07cc00640 1 --2- 172.21.15.17:0/2005055469 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc07815b950 0x7fc078158d70 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fc060009900 tx=0x7fc06001c670 comp rx=0 tx=0).stop 2024-09-10T21:25:12.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:12.630+0000 7fc07cc00640 1 -- 172.21.15.17:0/2005055469 shutdown_connections 2024-09-10T21:25:12.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:12.630+0000 7fc07cc00640 1 --2- 172.21.15.17:0/2005055469 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc07815b950 0x7fc078158d70 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:12.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:12.630+0000 7fc07cc00640 1 --2- 172.21.15.17:0/2005055469 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc078152520 0x7fc0780715f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:12.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:12.630+0000 7fc07cc00640 1 -- 172.21.15.17:0/2005055469 >> 172.21.15.17:0/2005055469 conn(0x7fc078068990 msgr2=0x7fc078068da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:12.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:12.630+0000 7fc07cc00640 1 -- 172.21.15.17:0/2005055469 shutdown_connections 2024-09-10T21:25:12.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:12.630+0000 7fc07cc00640 1 -- 172.21.15.17:0/2005055469 wait complete. 2024-09-10T21:25:12.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:12.630+0000 7fc07cc00640 1 Processor -- start 2024-09-10T21:25:12.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:12.630+0000 7fc07cc00640 1 -- start start 2024-09-10T21:25:12.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:12.630+0000 7fc07cc00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc0780c8a40 0x7fc07814a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:12.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:12.630+0000 7fc07cc00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc078152520 0x7fc0780c7e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:12.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:12.630+0000 7fc07cc00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc07815b950 0x7fc0781544e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:12.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:12.630+0000 7fc07cc00640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fc078165680 con 0x7fc0780c8a40 2024-09-10T21:25:12.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:12.630+0000 7fc07cc00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fc078165500 con 0x7fc07815b950 2024-09-10T21:25:12.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:12.630+0000 7fc07cc00640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7fc078165800 con 0x7fc078152520 2024-09-10T21:25:12.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:12.630+0000 7fc075e00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc0780c8a40 0x7fc07814a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:12.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:12.630+0000 7fc075e00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc0780c8a40 0x7fc07814a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40156/0 (socket says 172.21.15.17:40156) 2024-09-10T21:25:12.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:12.630+0000 7fc075e00640 1 -- 172.21.15.17:0/2970874414 learned_addr learned my addr 172.21.15.17:0/2970874414 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:25:12.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:12.630+0000 7fc06d400640 1 --2- 172.21.15.17:0/2970874414 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc078152520 0x7fc0780c7e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:12.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:12.630+0000 7fc076800640 1 --2- 172.21.15.17:0/2970874414 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc07815b950 0x7fc0781544e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:12.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:12.630+0000 7fc075e00640 1 -- 172.21.15.17:0/2970874414 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc078152520 msgr2=0x7fc0780c7e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:12.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:12.630+0000 7fc075e00640 1 --2- 172.21.15.17:0/2970874414 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc078152520 0x7fc0780c7e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:12.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:12.630+0000 7fc075e00640 1 -- 172.21.15.17:0/2970874414 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc07815b950 msgr2=0x7fc0781544e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:12.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:12.630+0000 7fc075e00640 1 --2- 172.21.15.17:0/2970874414 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc07815b950 0x7fc0781544e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:12.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:12.630+0000 7fc075e00640 1 -- 172.21.15.17:0/2970874414 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0780ce030 con 0x7fc0780c8a40 2024-09-10T21:25:12.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:12.630+0000 7fc076800640 1 --2- 172.21.15.17:0/2970874414 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc07815b950 0x7fc0781544e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:25:12.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:12.630+0000 7fc075e00640 1 --2- 172.21.15.17:0/2970874414 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc0780c8a40 0x7fc07814a650 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fc06800c9a0 tx=0x7fc06800ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:12.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:12.634+0000 7fc06ea00640 1 -- 172.21.15.17:0/2970874414 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc068007c70 con 0x7fc0780c8a40 2024-09-10T21:25:12.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:12.634+0000 7fc06ea00640 1 -- 172.21.15.17:0/2970874414 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc06800f450 con 0x7fc0780c8a40 2024-09-10T21:25:12.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:12.634+0000 7fc06ea00640 1 -- 172.21.15.17:0/2970874414 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc06800f8a0 con 0x7fc0780c8a40 2024-09-10T21:25:12.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:12.634+0000 7fc07cc00640 1 -- 172.21.15.17:0/2970874414 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc078153d20 con 0x7fc0780c8a40 2024-09-10T21:25:12.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:12.634+0000 7fc07cc00640 1 -- 172.21.15.17:0/2970874414 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc078153fc0 con 0x7fc0780c8a40 2024-09-10T21:25:12.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:12.634+0000 7fc07cc00640 1 -- 172.21.15.17:0/2970874414 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc078158d70 con 0x7fc0780c8a40 2024-09-10T21:25:12.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:12.634+0000 7fc06ea00640 1 -- 172.21.15.17:0/2970874414 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fc06800fa40 con 0x7fc0780c8a40 2024-09-10T21:25:12.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:12.634+0000 7fc06ea00640 1 --2- 172.21.15.17:0/2970874414 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fc054072610 0x7fc054074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:12.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:12.634+0000 7fc06ea00640 1 -- 172.21.15.17:0/2970874414 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fc068093d80 con 0x7fc0780c8a40 2024-09-10T21:25:12.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:12.634+0000 7fc06d400640 1 --2- 172.21.15.17:0/2970874414 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fc054072610 0x7fc054074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:12.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:12.638+0000 7fc06d400640 1 --2- 172.21.15.17:0/2970874414 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fc054072610 0x7fc054074ad0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc0600af000 tx=0x7fc06001aeb0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:12.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:12.638+0000 7fc06ea00640 1 -- 172.21.15.17:0/2970874414 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fc068014030 con 0x7fc0780c8a40 2024-09-10T21:25:12.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:12.814+0000 7fc07cc00640 1 -- 172.21.15.17:0/2970874414 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc07814b250 con 0x7fc0780c8a40 2024-09-10T21:25:12.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:12.814+0000 7fc06ea00640 1 -- 172.21.15.17:0/2970874414 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18562 (secure 0 0 0) 0x7fc06805f090 con 0x7fc0780c8a40 2024-09-10T21:25:12.819 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-10T21:25:12.819 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":25,"fsid":"5e2f023f-d6a6-49f4-a256-f4807dce3739","created":"2024-09-10T21:24:21.750712+0000","modified":"2024-09-10T21:24:52.295803+0000","last_up_change":"2024-09-10T21:24:35.877801+0000","last_in_change":"2024-09-10T21:24:25.831127+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:24:35.362670+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T21:24:45.211689+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T21:24:46.209784+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T21:24:49.871300+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0f5fabc0-434e-4066-b430-233969a548f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":4105489867},{"type":"v1","addr":"172.21.15.17:6809","nonce":4105489867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":4105489867},{"type":"v1","addr":"172.21.15.17:6811","nonce":4105489867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":4105489867},{"type":"v1","addr":"172.21.15.17:6815","nonce":4105489867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":4105489867},{"type":"v1","addr":"172.21.15.17:6813","nonce":4105489867}]},"public_addr":"172.21.15.17:6809/4105489867","cluster_addr":"172.21.15.17:6811/4105489867","heartbeat_back_addr":"172.21.15.17:6815/4105489867","heartbeat_front_addr":"172.21.15.17:6813/4105489867","state":["exists","up"]},{"osd":1,"uuid":"f6691cb3-2a91-4c90-a9fd-ece3c11dd6eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":2599842027},{"type":"v1","addr":"172.21.15.17:6825","nonce":2599842027}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":2599842027},{"type":"v1","addr":"172.21.15.17:6827","nonce":2599842027}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":2599842027},{"type":"v1","addr":"172.21.15.17:6831","nonce":2599842027}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":2599842027},{"type":"v1","addr":"172.21.15.17:6829","nonce":2599842027}]},"public_addr":"172.21.15.17:6825/2599842027","cluster_addr":"172.21.15.17:6827/2599842027","heartbeat_back_addr":"172.21.15.17:6831/2599842027","heartbeat_front_addr":"172.21.15.17:6829/2599842027","state":["exists","up"]},{"osd":2,"uuid":"cb61adb0-a216-437b-9598-06fa7c84aa64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":1248243634},{"type":"v1","addr":"172.21.15.17:6817","nonce":1248243634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":1248243634},{"type":"v1","addr":"172.21.15.17:6819","nonce":1248243634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":1248243634},{"type":"v1","addr":"172.21.15.17:6823","nonce":1248243634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":1248243634},{"type":"v1","addr":"172.21.15.17:6821","nonce":1248243634}]},"public_addr":"172.21.15.17:6817/1248243634","cluster_addr":"172.21.15.17:6819/1248243634","heartbeat_back_addr":"172.21.15.17:6823/1248243634","heartbeat_front_addr":"172.21.15.17:6821/1248243634","state":["exists","up"]},{"osd":3,"uuid":"6d2da26a-e0bf-47dc-b020-af86e6bcfccc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":2314489522},{"type":"v1","addr":"172.21.15.17:6801","nonce":2314489522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":2314489522},{"type":"v1","addr":"172.21.15.17:6803","nonce":2314489522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":2314489522},{"type":"v1","addr":"172.21.15.17:6807","nonce":2314489522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":2314489522},{"type":"v1","addr":"172.21.15.17:6805","nonce":2314489522}]},"public_addr":"172.21.15.17:6801/2314489522","cluster_addr":"172.21.15.17:6803/2314489522","heartbeat_back_addr":"172.21.15.17:6807/2314489522","heartbeat_front_addr":"172.21.15.17:6805/2314489522","state":["exists","up"]},{"osd":4,"uuid":"c65071dd-9ca7-4f1d-a584-45dc121419a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":202520744},{"type":"v1","addr":"172.21.15.31:6811","nonce":202520744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":202520744},{"type":"v1","addr":"172.21.15.31:6813","nonce":202520744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":202520744},{"type":"v1","addr":"172.21.15.31:6817","nonce":202520744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":202520744},{"type":"v1","addr":"172.21.15.31:6815","nonce":202520744}]},"public_addr":"172.21.15.31:6811/202520744","cluster_addr":"172.21.15.31:6813/202520744","heartbeat_back_addr":"172.21.15.31:6817/202520744","heartbeat_front_addr":"172.21.15.31:6815/202520744","state":["exists","up"]},{"osd":5,"uuid":"235389de-7777-4744-81b8-d0bfc11190e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":1472711057},{"type":"v1","addr":"172.21.15.31:6801","nonce":1472711057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":1472711057},{"type":"v1","addr":"172.21.15.31:6803","nonce":1472711057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":1472711057},{"type":"v1","addr":"172.21.15.31:6807","nonce":1472711057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":1472711057},{"type":"v1","addr":"172.21.15.31:6805","nonce":1472711057}]},"public_addr":"172.21.15.31:6801/1472711057","cluster_addr":"172.21.15.31:6803/1472711057","heartbeat_back_addr":"172.21.15.31:6807/1472711057","heartbeat_front_addr":"172.21.15.31:6805/1472711057","state":["exists","up"]},{"osd":6,"uuid":"cea70a57-3715-4ebb-8511-a528a9df2b1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":3754251089},{"type":"v1","addr":"172.21.15.31:6827","nonce":3754251089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":3754251089},{"type":"v1","addr":"172.21.15.31:6829","nonce":3754251089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":3754251089},{"type":"v1","addr":"172.21.15.31:6833","nonce":3754251089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":3754251089},{"type":"v1","addr":"172.21.15.31:6831","nonce":3754251089}]},"public_addr":"172.21.15.31:6827/3754251089","cluster_addr":"172.21.15.31:6829/3754251089","heartbeat_back_addr":"172.21.15.31:6833/3754251089","heartbeat_front_addr":"172.21.15.31:6831/3754251089","state":["exists","up"]},{"osd":7,"uuid":"01cc111b-a45f-4ca3-9cc1-7fe0afbeba45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":3948648934},{"type":"v1","addr":"172.21.15.31:6819","nonce":3948648934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":3948648934},{"type":"v1","addr":"172.21.15.31:6821","nonce":3948648934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":3948648934},{"type":"v1","addr":"172.21.15.31:6825","nonce":3948648934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":3948648934},{"type":"v1","addr":"172.21.15.31:6823","nonce":3948648934}]},"public_addr":"172.21.15.31:6819/3948648934","cluster_addr":"172.21.15.31:6821/3948648934","heartbeat_back_addr":"172.21.15.31:6825/3948648934","heartbeat_front_addr":"172.21.15.31:6823/3948648934","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:24:32.606989+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:24:32.891535+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:24:33.330793+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:24:32.811533+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:24:33.894624+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:24:32.773817+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:24:32.901865+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:24:32.907207+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:25:12.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:12.818+0000 7fc07cc00640 1 -- 172.21.15.17:0/2970874414 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fc054072610 msgr2=0x7fc054074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:12.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:12.818+0000 7fc07cc00640 1 --2- 172.21.15.17:0/2970874414 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fc054072610 0x7fc054074ad0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc0600af000 tx=0x7fc06001aeb0 comp rx=0 tx=0).stop 2024-09-10T21:25:12.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:12.818+0000 7fc07cc00640 1 -- 172.21.15.17:0/2970874414 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc0780c8a40 msgr2=0x7fc07814a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:12.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:12.818+0000 7fc07cc00640 1 --2- 172.21.15.17:0/2970874414 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc0780c8a40 0x7fc07814a650 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fc06800c9a0 tx=0x7fc06800ce60 comp rx=0 tx=0).stop 2024-09-10T21:25:12.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:12.818+0000 7fc07cc00640 1 -- 172.21.15.17:0/2970874414 shutdown_connections 2024-09-10T21:25:12.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:12.818+0000 7fc07cc00640 1 --2- 172.21.15.17:0/2970874414 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fc054072610 0x7fc054074ad0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:12.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:12.818+0000 7fc07cc00640 1 --2- 172.21.15.17:0/2970874414 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc07815b950 0x7fc0781544e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:12.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:12.818+0000 7fc07cc00640 1 --2- 172.21.15.17:0/2970874414 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc078152520 0x7fc0780c7e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:12.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:12.818+0000 7fc07cc00640 1 --2- 172.21.15.17:0/2970874414 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc0780c8a40 0x7fc07814a650 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:12.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:12.818+0000 7fc07cc00640 1 -- 172.21.15.17:0/2970874414 >> 172.21.15.17:0/2970874414 conn(0x7fc078068990 msgr2=0x7fc0781507c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:12.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:12.818+0000 7fc07cc00640 1 -- 172.21.15.17:0/2970874414 shutdown_connections 2024-09-10T21:25:12.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:12.818+0000 7fc07cc00640 1 -- 172.21.15.17:0/2970874414 wait complete. 2024-09-10T21:25:12.842 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:25:12.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:12.982+0000 7ff362a00640 1 -- 172.21.15.17:0/259996064 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff35c152440 msgr2=0x7ff35c0b3eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:12.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:12.982+0000 7ff362a00640 1 --2- 172.21.15.17:0/259996064 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff35c152440 0x7ff35c0b3eb0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7ff35c1594a0 tx=0x7ff34c01c850 comp rx=0 tx=0).stop 2024-09-10T21:25:12.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:12.982+0000 7ff362a00640 1 -- 172.21.15.17:0/259996064 shutdown_connections 2024-09-10T21:25:12.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:12.982+0000 7ff362a00640 1 --2- 172.21.15.17:0/259996064 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff35c152440 0x7ff35c0b3eb0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:12.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:12.982+0000 7ff362a00640 1 -- 172.21.15.17:0/259996064 >> 172.21.15.17:0/259996064 conn(0x7ff35c0702f0 msgr2=0x7ff35c070700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:12.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:12.982+0000 7ff362a00640 1 -- 172.21.15.17:0/259996064 shutdown_connections 2024-09-10T21:25:12.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:12.982+0000 7ff362a00640 1 -- 172.21.15.17:0/259996064 wait complete. 2024-09-10T21:25:12.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:12.986+0000 7ff362a00640 1 Processor -- start 2024-09-10T21:25:12.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:12.986+0000 7ff362a00640 1 -- start start 2024-09-10T21:25:12.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:12.986+0000 7ff362a00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff35c0cdf20 0x7ff35c14b200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:12.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:12.986+0000 7ff362a00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff35c152440 0x7ff35c145b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:12.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:12.986+0000 7ff362a00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff35c152820 0x7ff35c0c6d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:12.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:12.986+0000 7ff362a00640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7ff35c0cb450 con 0x7ff35c152440 2024-09-10T21:25:12.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:12.986+0000 7ff362a00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7ff35c0ce760 con 0x7ff35c152820 2024-09-10T21:25:12.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:12.986+0000 7ff362a00640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7ff35c0cb5d0 con 0x7ff35c0cdf20 2024-09-10T21:25:12.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:12.986+0000 7ff353e00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff35c152440 0x7ff35c145b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:12.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:12.986+0000 7ff353e00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff35c152440 0x7ff35c145b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40164/0 (socket says 172.21.15.17:40164) 2024-09-10T21:25:12.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:12.986+0000 7ff353e00640 1 -- 172.21.15.17:0/422946561 learned_addr learned my addr 172.21.15.17:0/422946561 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:25:12.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:12.986+0000 7ff360a00640 1 --2- 172.21.15.17:0/422946561 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff35c152820 0x7ff35c0c6d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:12.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:12.986+0000 7ff35be00640 1 --2- 172.21.15.17:0/422946561 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff35c0cdf20 0x7ff35c14b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:12.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:12.986+0000 7ff353e00640 1 -- 172.21.15.17:0/422946561 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff35c0cdf20 msgr2=0x7ff35c14b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:12.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:12.986+0000 7ff353e00640 1 --2- 172.21.15.17:0/422946561 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff35c0cdf20 0x7ff35c14b200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:12.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:12.986+0000 7ff353e00640 1 -- 172.21.15.17:0/422946561 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff35c152820 msgr2=0x7ff35c0c6d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:12.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:12.986+0000 7ff353e00640 1 --2- 172.21.15.17:0/422946561 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff35c152820 0x7ff35c0c6d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:12.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:12.986+0000 7ff353e00640 1 -- 172.21.15.17:0/422946561 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff35c1580b0 con 0x7ff35c152440 2024-09-10T21:25:12.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:12.986+0000 7ff360a00640 1 --2- 172.21.15.17:0/422946561 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff35c152820 0x7ff35c0c6d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:25:12.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:12.986+0000 7ff353e00640 1 --2- 172.21.15.17:0/422946561 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff35c152440 0x7ff35c145b20 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7ff34400c500 tx=0x7ff34400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:12.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:12.986+0000 7ff358c00640 1 -- 172.21.15.17:0/422946561 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff344018070 con 0x7ff35c152440 2024-09-10T21:25:12.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:12.986+0000 7ff358c00640 1 -- 172.21.15.17:0/422946561 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff344003600 con 0x7ff35c152440 2024-09-10T21:25:12.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:12.986+0000 7ff362a00640 1 -- 172.21.15.17:0/422946561 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff35c156270 con 0x7ff35c152440 2024-09-10T21:25:12.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:12.986+0000 7ff358c00640 1 -- 172.21.15.17:0/422946561 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff344003ad0 con 0x7ff35c152440 2024-09-10T21:25:12.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:12.986+0000 7ff362a00640 1 -- 172.21.15.17:0/422946561 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff35c0c5910 con 0x7ff35c152440 2024-09-10T21:25:12.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:12.990+0000 7ff362a00640 1 -- 172.21.15.17:0/422946561 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff35c14b800 con 0x7ff35c152440 2024-09-10T21:25:12.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:12.990+0000 7ff358c00640 1 -- 172.21.15.17:0/422946561 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7ff344003c70 con 0x7ff35c152440 2024-09-10T21:25:12.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:12.990+0000 7ff358c00640 1 --2- 172.21.15.17:0/422946561 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7ff3380725e0 0x7ff338074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:12.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:12.990+0000 7ff358c00640 1 -- 172.21.15.17:0/422946561 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7ff344092ca0 con 0x7ff35c152440 2024-09-10T21:25:12.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:12.990+0000 7ff35be00640 1 --2- 172.21.15.17:0/422946561 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7ff3380725e0 0x7ff338074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:12.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:12.990+0000 7ff35be00640 1 --2- 172.21.15.17:0/422946561 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7ff3380725e0 0x7ff338074aa0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff34c0af000 tx=0x7ff34c01b090 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:12.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:12.994+0000 7ff358c00640 1 -- 172.21.15.17:0/422946561 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7ff34405e070 con 0x7ff35c152440 2024-09-10T21:25:13.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:13.170+0000 7ff362a00640 1 -- 172.21.15.17:0/422946561 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff35c14ffc0 con 0x7ff35c152440 2024-09-10T21:25:13.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:13.170+0000 7ff358c00640 1 -- 172.21.15.17:0/422946561 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18562 (secure 0 0 0) 0x7ff34405d970 con 0x7ff35c152440 2024-09-10T21:25:13.174 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-10T21:25:13.174 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":25,"fsid":"5e2f023f-d6a6-49f4-a256-f4807dce3739","created":"2024-09-10T21:24:21.750712+0000","modified":"2024-09-10T21:24:52.295803+0000","last_up_change":"2024-09-10T21:24:35.877801+0000","last_in_change":"2024-09-10T21:24:25.831127+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:24:35.362670+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T21:24:45.211689+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T21:24:46.209784+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T21:24:49.871300+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0f5fabc0-434e-4066-b430-233969a548f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":4105489867},{"type":"v1","addr":"172.21.15.17:6809","nonce":4105489867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":4105489867},{"type":"v1","addr":"172.21.15.17:6811","nonce":4105489867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":4105489867},{"type":"v1","addr":"172.21.15.17:6815","nonce":4105489867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":4105489867},{"type":"v1","addr":"172.21.15.17:6813","nonce":4105489867}]},"public_addr":"172.21.15.17:6809/4105489867","cluster_addr":"172.21.15.17:6811/4105489867","heartbeat_back_addr":"172.21.15.17:6815/4105489867","heartbeat_front_addr":"172.21.15.17:6813/4105489867","state":["exists","up"]},{"osd":1,"uuid":"f6691cb3-2a91-4c90-a9fd-ece3c11dd6eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":2599842027},{"type":"v1","addr":"172.21.15.17:6825","nonce":2599842027}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":2599842027},{"type":"v1","addr":"172.21.15.17:6827","nonce":2599842027}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":2599842027},{"type":"v1","addr":"172.21.15.17:6831","nonce":2599842027}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":2599842027},{"type":"v1","addr":"172.21.15.17:6829","nonce":2599842027}]},"public_addr":"172.21.15.17:6825/2599842027","cluster_addr":"172.21.15.17:6827/2599842027","heartbeat_back_addr":"172.21.15.17:6831/2599842027","heartbeat_front_addr":"172.21.15.17:6829/2599842027","state":["exists","up"]},{"osd":2,"uuid":"cb61adb0-a216-437b-9598-06fa7c84aa64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":1248243634},{"type":"v1","addr":"172.21.15.17:6817","nonce":1248243634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":1248243634},{"type":"v1","addr":"172.21.15.17:6819","nonce":1248243634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":1248243634},{"type":"v1","addr":"172.21.15.17:6823","nonce":1248243634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":1248243634},{"type":"v1","addr":"172.21.15.17:6821","nonce":1248243634}]},"public_addr":"172.21.15.17:6817/1248243634","cluster_addr":"172.21.15.17:6819/1248243634","heartbeat_back_addr":"172.21.15.17:6823/1248243634","heartbeat_front_addr":"172.21.15.17:6821/1248243634","state":["exists","up"]},{"osd":3,"uuid":"6d2da26a-e0bf-47dc-b020-af86e6bcfccc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":2314489522},{"type":"v1","addr":"172.21.15.17:6801","nonce":2314489522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":2314489522},{"type":"v1","addr":"172.21.15.17:6803","nonce":2314489522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":2314489522},{"type":"v1","addr":"172.21.15.17:6807","nonce":2314489522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":2314489522},{"type":"v1","addr":"172.21.15.17:6805","nonce":2314489522}]},"public_addr":"172.21.15.17:6801/2314489522","cluster_addr":"172.21.15.17:6803/2314489522","heartbeat_back_addr":"172.21.15.17:6807/2314489522","heartbeat_front_addr":"172.21.15.17:6805/2314489522","state":["exists","up"]},{"osd":4,"uuid":"c65071dd-9ca7-4f1d-a584-45dc121419a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":202520744},{"type":"v1","addr":"172.21.15.31:6811","nonce":202520744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":202520744},{"type":"v1","addr":"172.21.15.31:6813","nonce":202520744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":202520744},{"type":"v1","addr":"172.21.15.31:6817","nonce":202520744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":202520744},{"type":"v1","addr":"172.21.15.31:6815","nonce":202520744}]},"public_addr":"172.21.15.31:6811/202520744","cluster_addr":"172.21.15.31:6813/202520744","heartbeat_back_addr":"172.21.15.31:6817/202520744","heartbeat_front_addr":"172.21.15.31:6815/202520744","state":["exists","up"]},{"osd":5,"uuid":"235389de-7777-4744-81b8-d0bfc11190e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":1472711057},{"type":"v1","addr":"172.21.15.31:6801","nonce":1472711057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":1472711057},{"type":"v1","addr":"172.21.15.31:6803","nonce":1472711057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":1472711057},{"type":"v1","addr":"172.21.15.31:6807","nonce":1472711057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":1472711057},{"type":"v1","addr":"172.21.15.31:6805","nonce":1472711057}]},"public_addr":"172.21.15.31:6801/1472711057","cluster_addr":"172.21.15.31:6803/1472711057","heartbeat_back_addr":"172.21.15.31:6807/1472711057","heartbeat_front_addr":"172.21.15.31:6805/1472711057","state":["exists","up"]},{"osd":6,"uuid":"cea70a57-3715-4ebb-8511-a528a9df2b1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":3754251089},{"type":"v1","addr":"172.21.15.31:6827","nonce":3754251089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":3754251089},{"type":"v1","addr":"172.21.15.31:6829","nonce":3754251089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":3754251089},{"type":"v1","addr":"172.21.15.31:6833","nonce":3754251089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":3754251089},{"type":"v1","addr":"172.21.15.31:6831","nonce":3754251089}]},"public_addr":"172.21.15.31:6827/3754251089","cluster_addr":"172.21.15.31:6829/3754251089","heartbeat_back_addr":"172.21.15.31:6833/3754251089","heartbeat_front_addr":"172.21.15.31:6831/3754251089","state":["exists","up"]},{"osd":7,"uuid":"01cc111b-a45f-4ca3-9cc1-7fe0afbeba45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":3948648934},{"type":"v1","addr":"172.21.15.31:6819","nonce":3948648934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":3948648934},{"type":"v1","addr":"172.21.15.31:6821","nonce":3948648934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":3948648934},{"type":"v1","addr":"172.21.15.31:6825","nonce":3948648934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":3948648934},{"type":"v1","addr":"172.21.15.31:6823","nonce":3948648934}]},"public_addr":"172.21.15.31:6819/3948648934","cluster_addr":"172.21.15.31:6821/3948648934","heartbeat_back_addr":"172.21.15.31:6825/3948648934","heartbeat_front_addr":"172.21.15.31:6823/3948648934","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:24:32.606989+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:24:32.891535+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:24:33.330793+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:24:32.811533+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:24:33.894624+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:24:32.773817+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:24:32.901865+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:24:32.907207+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:25:13.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:13.170+0000 7ff362a00640 1 -- 172.21.15.17:0/422946561 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7ff3380725e0 msgr2=0x7ff338074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:13.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:13.170+0000 7ff362a00640 1 --2- 172.21.15.17:0/422946561 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7ff3380725e0 0x7ff338074aa0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ff34c0af000 tx=0x7ff34c01b090 comp rx=0 tx=0).stop 2024-09-10T21:25:13.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:13.170+0000 7ff362a00640 1 -- 172.21.15.17:0/422946561 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff35c152440 msgr2=0x7ff35c145b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:13.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:13.170+0000 7ff362a00640 1 --2- 172.21.15.17:0/422946561 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff35c152440 0x7ff35c145b20 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7ff34400c500 tx=0x7ff34400c9c0 comp rx=0 tx=0).stop 2024-09-10T21:25:13.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:13.170+0000 7ff362a00640 1 -- 172.21.15.17:0/422946561 shutdown_connections 2024-09-10T21:25:13.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:13.170+0000 7ff362a00640 1 --2- 172.21.15.17:0/422946561 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7ff3380725e0 0x7ff338074aa0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:13.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:13.170+0000 7ff362a00640 1 --2- 172.21.15.17:0/422946561 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff35c152820 0x7ff35c0c6d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:13.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:13.170+0000 7ff362a00640 1 --2- 172.21.15.17:0/422946561 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff35c152440 0x7ff35c145b20 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:13.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:13.170+0000 7ff362a00640 1 --2- 172.21.15.17:0/422946561 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff35c0cdf20 0x7ff35c14b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:13.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:13.170+0000 7ff362a00640 1 -- 172.21.15.17:0/422946561 >> 172.21.15.17:0/422946561 conn(0x7ff35c0702f0 msgr2=0x7ff35c0c3240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:13.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:13.170+0000 7ff362a00640 1 -- 172.21.15.17:0/422946561 shutdown_connections 2024-09-10T21:25:13.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:13.170+0000 7ff362a00640 1 -- 172.21.15.17:0/422946561 wait complete. 2024-09-10T21:25:13.196 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-10T21:25:13.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:13.342+0000 7fb1b9a00640 1 -- 172.21.15.17:0/2990151103 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb198004810 msgr2=0x7fb198004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:13.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:13.342+0000 7fb1b9a00640 1 --2- 172.21.15.17:0/2990151103 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb198004810 0x7fb198004c90 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fb1b4070c60 tx=0x7fb1a001d1f0 comp rx=0 tx=0).stop 2024-09-10T21:25:13.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:13.342+0000 7fb1b3400640 1 -- 172.21.15.17:0/2990151103 reap_dead start 2024-09-10T21:25:13.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:13.342+0000 7fb1b9a00640 1 -- 172.21.15.17:0/2990151103 shutdown_connections 2024-09-10T21:25:13.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:13.342+0000 7fb1b9a00640 1 -- 172.21.15.17:0/2990151103 >> 172.21.15.17:0/2990151103 conn(0x7fb1b406dd30 msgr2=0x7fb1b406e140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:13.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:13.342+0000 7fb1b9a00640 1 -- 172.21.15.17:0/2990151103 shutdown_connections 2024-09-10T21:25:13.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:13.342+0000 7fb1b9a00640 1 -- 172.21.15.17:0/2990151103 wait complete. 2024-09-10T21:25:13.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:13.346+0000 7fb1b9a00640 1 Processor -- start 2024-09-10T21:25:13.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:13.346+0000 7fb1b9a00640 1 -- start start 2024-09-10T21:25:13.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:13.346+0000 7fb1b9a00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fb198004810 0x7fb1b41f6d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:13.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:13.346+0000 7fb1b9a00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb1b406ce80 0x7fb1b406d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:13.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:13.346+0000 7fb1b9a00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1b4151840 0x7fb1b4151cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:13.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:13.346+0000 7fb1b9a00640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fb1b41650f0 con 0x7fb1b406ce80 2024-09-10T21:25:13.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:13.346+0000 7fb1b9a00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fb1b4071e70 con 0x7fb1b4151840 2024-09-10T21:25:13.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:13.346+0000 7fb1b9a00640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7fb1b4165270 con 0x7fb198004810 2024-09-10T21:25:13.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:13.346+0000 7fb1b2000640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb1b406ce80 0x7fb1b406d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:13.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:13.346+0000 7fb1b2a00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fb198004810 0x7fb1b41f6d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:13.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:13.346+0000 7fb1b2000640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb1b406ce80 0x7fb1b406d300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40174/0 (socket says 172.21.15.17:40174) 2024-09-10T21:25:13.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:13.346+0000 7fb1b2000640 1 -- 172.21.15.17:0/2754127257 learned_addr learned my addr 172.21.15.17:0/2754127257 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:25:13.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:13.346+0000 7fb1b3400640 1 --2- 172.21.15.17:0/2754127257 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1b4151840 0x7fb1b4151cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:13.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:13.346+0000 7fb1b2000640 1 -- 172.21.15.17:0/2754127257 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fb198004810 msgr2=0x7fb1b41f6d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:13.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:13.346+0000 7fb1b2000640 1 --2- 172.21.15.17:0/2754127257 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fb198004810 0x7fb1b41f6d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:13.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:13.346+0000 7fb1b2000640 1 -- 172.21.15.17:0/2754127257 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1b4151840 msgr2=0x7fb1b4151cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:13.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:13.346+0000 7fb1b2000640 1 --2- 172.21.15.17:0/2754127257 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1b4151840 0x7fb1b4151cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:13.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:13.346+0000 7fb1b2000640 1 -- 172.21.15.17:0/2754127257 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1b40c4700 con 0x7fb1b406ce80 2024-09-10T21:25:13.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:13.346+0000 7fb1b2000640 1 --2- 172.21.15.17:0/2754127257 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb1b406ce80 0x7fb1b406d300 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fb1a001cc60 tx=0x7fb1a00097e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:13.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:13.346+0000 7fb1aaa00640 1 -- 172.21.15.17:0/2754127257 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1a00b1040 con 0x7fb1b406ce80 2024-09-10T21:25:13.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:13.346+0000 7fb1aaa00640 1 -- 172.21.15.17:0/2754127257 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb1a0005110 con 0x7fb1b406ce80 2024-09-10T21:25:13.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:13.346+0000 7fb1aaa00640 1 -- 172.21.15.17:0/2754127257 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1a0005560 con 0x7fb1b406ce80 2024-09-10T21:25:13.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:13.346+0000 7fb1b9a00640 1 -- 172.21.15.17:0/2754127257 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1b4152200 con 0x7fb1b406ce80 2024-09-10T21:25:13.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:13.346+0000 7fb1b9a00640 1 -- 172.21.15.17:0/2754127257 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1b41584b0 con 0x7fb1b406ce80 2024-09-10T21:25:13.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:13.350+0000 7fb1b9a00640 1 -- 172.21.15.17:0/2754127257 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1b4158680 con 0x7fb1b406ce80 2024-09-10T21:25:13.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:13.350+0000 7fb1aaa00640 1 -- 172.21.15.17:0/2754127257 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fb1a00047b0 con 0x7fb1b406ce80 2024-09-10T21:25:13.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:13.350+0000 7fb1aaa00640 1 --2- 172.21.15.17:0/2754127257 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fb1940725e0 0x7fb194074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:13.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:13.350+0000 7fb1aaa00640 1 -- 172.21.15.17:0/2754127257 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fb1a012e560 con 0x7fb1b406ce80 2024-09-10T21:25:13.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:13.350+0000 7fb1b2a00640 1 --2- 172.21.15.17:0/2754127257 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fb1940725e0 0x7fb194074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:13.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:13.350+0000 7fb1b2a00640 1 --2- 172.21.15.17:0/2754127257 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fb1940725e0 0x7fb194074aa0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb1a4001ec0 tx=0x7fb1a4006040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:13.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:13.354+0000 7fb1aaa00640 1 -- 172.21.15.17:0/2754127257 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fb1a0016f00 con 0x7fb1b406ce80 2024-09-10T21:25:13.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:13.562+0000 7fb1b9a00640 1 -- 172.21.15.17:0/2754127257 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fb1b40c9b30 con 0x7fb1b406ce80 2024-09-10T21:25:14.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:14.422+0000 7fb1aaa00640 1 -- 172.21.15.17:0/2754127257 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v8) ==== 118+0+0 (secure 0 0 0) 0x7fb1a00f9210 con 0x7fb1b406ce80 2024-09-10T21:25:14.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:14.426+0000 7fb1b9a00640 1 -- 172.21.15.17:0/2754127257 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fb1940725e0 msgr2=0x7fb194074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:14.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:14.426+0000 7fb1b9a00640 1 --2- 172.21.15.17:0/2754127257 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fb1940725e0 0x7fb194074aa0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb1a4001ec0 tx=0x7fb1a4006040 comp rx=0 tx=0).stop 2024-09-10T21:25:14.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:14.426+0000 7fb1b9a00640 1 -- 172.21.15.17:0/2754127257 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb1b406ce80 msgr2=0x7fb1b406d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:14.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:14.426+0000 7fb1b9a00640 1 --2- 172.21.15.17:0/2754127257 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb1b406ce80 0x7fb1b406d300 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fb1a001cc60 tx=0x7fb1a00097e0 comp rx=0 tx=0).stop 2024-09-10T21:25:14.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:14.426+0000 7fb1b9a00640 1 -- 172.21.15.17:0/2754127257 shutdown_connections 2024-09-10T21:25:14.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:14.426+0000 7fb1b9a00640 1 --2- 172.21.15.17:0/2754127257 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fb1940725e0 0x7fb194074aa0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:14.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:14.426+0000 7fb1b9a00640 1 --2- 172.21.15.17:0/2754127257 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1b4151840 0x7fb1b4151cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:14.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:14.426+0000 7fb1b9a00640 1 --2- 172.21.15.17:0/2754127257 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb1b406ce80 0x7fb1b406d300 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:14.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:14.426+0000 7fb1b9a00640 1 --2- 172.21.15.17:0/2754127257 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fb198004810 0x7fb1b41f6d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:14.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:14.426+0000 7fb1b9a00640 1 -- 172.21.15.17:0/2754127257 >> 172.21.15.17:0/2754127257 conn(0x7fb1b406dd30 msgr2=0x7fb1b4153b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:14.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:14.426+0000 7fb1b9a00640 1 -- 172.21.15.17:0/2754127257 shutdown_connections 2024-09-10T21:25:14.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:14.426+0000 7fb1b9a00640 1 -- 172.21.15.17:0/2754127257 wait complete. 2024-09-10T21:25:14.456 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 1, 'session_timeout': 300} 2024-09-10T21:25:14.457 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-10T21:25:14.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:14.606+0000 7f32f7600640 1 -- 172.21.15.17:0/4133600371 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f32f01548c0 msgr2=0x7f32f00c3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:14.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:14.606+0000 7f32f7600640 1 --2- 172.21.15.17:0/4133600371 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f32f01548c0 0x7f32f00c3600 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f32dc009840 tx=0x7f32dc01f010 comp rx=0 tx=0).stop 2024-09-10T21:25:14.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:14.606+0000 7f32f7600640 1 -- 172.21.15.17:0/4133600371 shutdown_connections 2024-09-10T21:25:14.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:14.606+0000 7f32f7600640 1 --2- 172.21.15.17:0/4133600371 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f32f01548c0 0x7f32f00c3600 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:14.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:14.606+0000 7f32f7600640 1 -- 172.21.15.17:0/4133600371 >> 172.21.15.17:0/4133600371 conn(0x7f32f0074e70 msgr2=0x7f32f006cc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:14.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:14.606+0000 7f32f7600640 1 -- 172.21.15.17:0/4133600371 shutdown_connections 2024-09-10T21:25:14.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:14.606+0000 7f32f7600640 1 -- 172.21.15.17:0/4133600371 wait complete. 2024-09-10T21:25:14.612 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:14.606+0000 7f32f7600640 1 Processor -- start 2024-09-10T21:25:14.612 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:14.606+0000 7f32f7600640 1 -- start start 2024-09-10T21:25:14.614 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:14.610+0000 7f32f7600640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f32f00c8a40 0x7f32f014ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:14.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:14.610+0000 7f32f7600640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f32f01544e0 0x7f32f0146e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:14.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:14.610+0000 7f32f7600640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f32f01548c0 0x7f32f00cabd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:14.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:14.610+0000 7f32f7600640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f32f00cea40 con 0x7f32f01544e0 2024-09-10T21:25:14.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:14.610+0000 7f32f7600640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f32f006e770 con 0x7f32f01548c0 2024-09-10T21:25:14.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:14.610+0000 7f32f7600640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f32f00c3b40 con 0x7f32f00c8a40 2024-09-10T21:25:14.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:14.610+0000 7f32efe00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f32f01544e0 0x7f32f0146e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:14.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:14.610+0000 7f32efe00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f32f01544e0 0x7f32f0146e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40200/0 (socket says 172.21.15.17:40200) 2024-09-10T21:25:14.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:14.610+0000 7f32efe00640 1 -- 172.21.15.17:0/3910318734 learned_addr learned my addr 172.21.15.17:0/3910318734 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:25:14.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:14.610+0000 7f32f4c00640 1 --2- 172.21.15.17:0/3910318734 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f32f00c8a40 0x7f32f014ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:14.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:14.610+0000 7f32efe00640 1 -- 172.21.15.17:0/3910318734 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f32f00c8a40 msgr2=0x7f32f014ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:14.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:14.610+0000 7f32efe00640 1 --2- 172.21.15.17:0/3910318734 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f32f00c8a40 0x7f32f014ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:14.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:14.610+0000 7f32efe00640 1 -- 172.21.15.17:0/3910318734 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f32f01548c0 msgr2=0x7f32f00cabd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:14.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:14.610+0000 7f32efe00640 1 --2- 172.21.15.17:0/3910318734 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f32f01548c0 0x7f32f00cabd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:14.616 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:14.610+0000 7f32efe00640 1 -- 172.21.15.17:0/3910318734 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32f00c9180 con 0x7f32f01544e0 2024-09-10T21:25:14.616 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:14.610+0000 7f32f4c00640 1 --2- 172.21.15.17:0/3910318734 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f32f00c8a40 0x7f32f014ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:25:14.616 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:14.610+0000 7f32efe00640 1 --2- 172.21.15.17:0/3910318734 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f32f01544e0 0x7f32f0146e30 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f32dc009790 tx=0x7f32dc0a8d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:14.616 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:14.610+0000 7f32ecc00640 1 -- 172.21.15.17:0/3910318734 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32dc0b5040 con 0x7f32f01544e0 2024-09-10T21:25:14.616 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:14.610+0000 7f32ecc00640 1 -- 172.21.15.17:0/3910318734 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f32dc005030 con 0x7f32f01544e0 2024-09-10T21:25:14.616 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:14.610+0000 7f32f7600640 1 -- 172.21.15.17:0/3910318734 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32f00c9460 con 0x7f32f01544e0 2024-09-10T21:25:14.616 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:14.610+0000 7f32ecc00640 1 -- 172.21.15.17:0/3910318734 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32dc005460 con 0x7f32f01544e0 2024-09-10T21:25:14.616 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:14.610+0000 7f32f7600640 1 -- 172.21.15.17:0/3910318734 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32f00c9a20 con 0x7f32f01544e0 2024-09-10T21:25:14.617 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:14.610+0000 7f32f7600640 1 -- 172.21.15.17:0/3910318734 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32f006ec90 con 0x7f32f01544e0 2024-09-10T21:25:14.617 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:14.614+0000 7f32ecc00640 1 -- 172.21.15.17:0/3910318734 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f32dc0b8050 con 0x7f32f01544e0 2024-09-10T21:25:14.617 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:14.614+0000 7f32ecc00640 1 --2- 172.21.15.17:0/3910318734 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f32d00725e0 0x7f32d0074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:14.617 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:14.614+0000 7f32ecc00640 1 -- 172.21.15.17:0/3910318734 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f32dc1307e0 con 0x7f32f01544e0 2024-09-10T21:25:14.617 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:14.614+0000 7f32f4c00640 1 --2- 172.21.15.17:0/3910318734 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f32d00725e0 0x7f32d0074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:14.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:14.614+0000 7f32f4c00640 1 --2- 172.21.15.17:0/3910318734 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f32d00725e0 0x7f32d0074aa0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f32e4000f70 tx=0x7f32e4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:14.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:14.618+0000 7f32ecc00640 1 -- 172.21.15.17:0/3910318734 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f32dc0c7020 con 0x7f32f01544e0 2024-09-10T21:25:14.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:14.826+0000 7f32f7600640 1 -- 172.21.15.17:0/3910318734 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f32f006f110 con 0x7f32f01544e0 2024-09-10T21:25:15.304 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:25:15.307 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:25:15.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:15.430+0000 7f32ecc00640 1 -- 172.21.15.17:0/3910318734 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v9) ==== 122+0+0 (secure 0 0 0) 0x7f32dc018b70 con 0x7f32f01544e0 2024-09-10T21:25:15.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:15.434+0000 7f32f7600640 1 -- 172.21.15.17:0/3910318734 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f32d00725e0 msgr2=0x7f32d0074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:15.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:15.434+0000 7f32f7600640 1 --2- 172.21.15.17:0/3910318734 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f32d00725e0 0x7f32d0074aa0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f32e4000f70 tx=0x7f32e4009040 comp rx=0 tx=0).stop 2024-09-10T21:25:15.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:15.434+0000 7f32f7600640 1 -- 172.21.15.17:0/3910318734 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f32f01544e0 msgr2=0x7f32f0146e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:15.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:15.434+0000 7f32f7600640 1 --2- 172.21.15.17:0/3910318734 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f32f01544e0 0x7f32f0146e30 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f32dc009790 tx=0x7f32dc0a8d30 comp rx=0 tx=0).stop 2024-09-10T21:25:15.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:15.434+0000 7f32f7600640 1 -- 172.21.15.17:0/3910318734 shutdown_connections 2024-09-10T21:25:15.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:15.434+0000 7f32f7600640 1 --2- 172.21.15.17:0/3910318734 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f32d00725e0 0x7f32d0074aa0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:15.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:15.434+0000 7f32f7600640 1 --2- 172.21.15.17:0/3910318734 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f32f01548c0 0x7f32f00cabd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:15.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:15.434+0000 7f32f7600640 1 --2- 172.21.15.17:0/3910318734 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f32f01544e0 0x7f32f0146e30 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:15.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:15.434+0000 7f32f7600640 1 --2- 172.21.15.17:0/3910318734 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f32f00c8a40 0x7f32f014ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:15.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:15.434+0000 7f32f7600640 1 -- 172.21.15.17:0/3910318734 >> 172.21.15.17:0/3910318734 conn(0x7f32f0074e70 msgr2=0x7f32f0153610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:15.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:15.434+0000 7f32f7600640 1 -- 172.21.15.17:0/3910318734 shutdown_connections 2024-09-10T21:25:15.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:15.434+0000 7f32f7600640 1 -- 172.21.15.17:0/3910318734 wait complete. 2024-09-10T21:25:15.462 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-09-10T21:25:15.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:15.730+0000 7ff84e400640 1 -- 172.21.15.17:0/2734418677 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff8480b0bd0 msgr2=0x7ff8480cb580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:15.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:15.730+0000 7ff84e400640 1 --2- 172.21.15.17:0/2734418677 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff8480b0bd0 0x7ff8480cb580 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff840009bf0 tx=0x7ff84001f2f0 comp rx=0 tx=0).stop 2024-09-10T21:25:15.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:15.730+0000 7ff84e400640 1 -- 172.21.15.17:0/2734418677 shutdown_connections 2024-09-10T21:25:15.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:15.730+0000 7ff84e400640 1 --2- 172.21.15.17:0/2734418677 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff8480b0bd0 0x7ff8480cb580 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:15.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:15.730+0000 7ff84e400640 1 -- 172.21.15.17:0/2734418677 >> 172.21.15.17:0/2734418677 conn(0x7ff8480708a0 msgr2=0x7ff848070cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:15.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:15.734+0000 7ff84e400640 1 -- 172.21.15.17:0/2734418677 shutdown_connections 2024-09-10T21:25:15.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:15.734+0000 7ff84e400640 1 -- 172.21.15.17:0/2734418677 wait complete. 2024-09-10T21:25:15.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:15.734+0000 7ff84e400640 1 Processor -- start 2024-09-10T21:25:15.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:15.734+0000 7ff84e400640 1 -- start start 2024-09-10T21:25:15.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:15.734+0000 7ff84e400640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff8480b0bd0 0x7ff848068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:15.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:15.734+0000 7ff84e400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff848152680 0x7ff8481518a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:15.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:15.734+0000 7ff84e400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff848152a60 0x7ff848151de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:15.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:15.734+0000 7ff84e400640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7ff8480c3ab0 con 0x7ff848152680 2024-09-10T21:25:15.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:15.734+0000 7ff84e400640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7ff8480c37d0 con 0x7ff848152a60 2024-09-10T21:25:15.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:15.734+0000 7ff84e400640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7ff8480c3d90 con 0x7ff8480b0bd0 2024-09-10T21:25:15.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:15.734+0000 7ff83fe00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff848152680 0x7ff8481518a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:15.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:15.734+0000 7ff83fe00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff848152680 0x7ff8481518a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44044/0 (socket says 172.21.15.17:44044) 2024-09-10T21:25:15.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:15.734+0000 7ff83fe00640 1 -- 172.21.15.17:0/871575386 learned_addr learned my addr 172.21.15.17:0/871575386 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:25:15.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:15.734+0000 7ff847e00640 1 --2- 172.21.15.17:0/871575386 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff848152a60 0x7ff848151de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:15.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:15.734+0000 7ff847400640 1 --2- 172.21.15.17:0/871575386 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff8480b0bd0 0x7ff848068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:15.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:15.734+0000 7ff83fe00640 1 -- 172.21.15.17:0/871575386 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff8480b0bd0 msgr2=0x7ff848068bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:15.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:15.734+0000 7ff83fe00640 1 --2- 172.21.15.17:0/871575386 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff8480b0bd0 0x7ff848068bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:15.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:15.734+0000 7ff83fe00640 1 -- 172.21.15.17:0/871575386 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff848152a60 msgr2=0x7ff848151de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:15.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:15.734+0000 7ff83fe00640 1 --2- 172.21.15.17:0/871575386 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff848152a60 0x7ff848151de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:15.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:15.734+0000 7ff83fe00640 1 -- 172.21.15.17:0/871575386 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8481e23c0 con 0x7ff848152680 2024-09-10T21:25:15.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:15.734+0000 7ff847400640 1 --2- 172.21.15.17:0/871575386 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff8480b0bd0 0x7ff848068bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:25:15.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:15.734+0000 7ff83fe00640 1 --2- 172.21.15.17:0/871575386 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff848152680 0x7ff8481518a0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ff83000c390 tx=0x7ff83000c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:15.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:15.734+0000 7ff83f400640 1 -- 172.21.15.17:0/871575386 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff830017070 con 0x7ff848152680 2024-09-10T21:25:15.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:15.734+0000 7ff84e400640 1 -- 172.21.15.17:0/871575386 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8481e2680 con 0x7ff848152680 2024-09-10T21:25:15.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:15.734+0000 7ff83f400640 1 -- 172.21.15.17:0/871575386 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff8300053c0 con 0x7ff848152680 2024-09-10T21:25:15.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:15.738+0000 7ff83f400640 1 -- 172.21.15.17:0/871575386 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff830003860 con 0x7ff848152680 2024-09-10T21:25:15.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:15.738+0000 7ff84e400640 1 -- 172.21.15.17:0/871575386 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff848159c50 con 0x7ff848152680 2024-09-10T21:25:15.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:15.738+0000 7ff84e400640 1 -- 172.21.15.17:0/871575386 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8480cb580 con 0x7ff848152680 2024-09-10T21:25:15.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:15.738+0000 7ff83f400640 1 -- 172.21.15.17:0/871575386 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7ff830003a00 con 0x7ff848152680 2024-09-10T21:25:15.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:15.738+0000 7ff83f400640 1 --2- 172.21.15.17:0/871575386 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7ff8240725e0 0x7ff824074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:15.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:15.738+0000 7ff83f400640 1 -- 172.21.15.17:0/871575386 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7ff830092c60 con 0x7ff848152680 2024-09-10T21:25:15.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:15.738+0000 7ff847400640 1 --2- 172.21.15.17:0/871575386 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7ff8240725e0 0x7ff824074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:15.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:15.742+0000 7ff847400640 1 --2- 172.21.15.17:0/871575386 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7ff8240725e0 0x7ff824074aa0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff838000c10 tx=0x7ff838009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:15.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:15.742+0000 7ff83f400640 1 -- 172.21.15.17:0/871575386 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7ff83005d890 con 0x7ff848152680 2024-09-10T21:25:15.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:15.958+0000 7ff84e400640 1 -- 172.21.15.17:0/871575386 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7ff8481584b0 con 0x7ff848152680 2024-09-10T21:25:16.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:16.430+0000 7ff83f400640 1 -- 172.21.15.17:0/871575386 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v10) ==== 115+0+0 (secure 0 0 0) 0x7ff8300622c0 con 0x7ff848152680 2024-09-10T21:25:16.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:16.434+0000 7ff84e400640 1 -- 172.21.15.17:0/871575386 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7ff8240725e0 msgr2=0x7ff824074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:16.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:16.434+0000 7ff84e400640 1 --2- 172.21.15.17:0/871575386 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7ff8240725e0 0x7ff824074aa0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff838000c10 tx=0x7ff838009040 comp rx=0 tx=0).stop 2024-09-10T21:25:16.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:16.434+0000 7ff84e400640 1 -- 172.21.15.17:0/871575386 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff848152680 msgr2=0x7ff8481518a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:16.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:16.434+0000 7ff84e400640 1 --2- 172.21.15.17:0/871575386 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff848152680 0x7ff8481518a0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ff83000c390 tx=0x7ff83000c850 comp rx=0 tx=0).stop 2024-09-10T21:25:16.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:16.434+0000 7ff84e400640 1 -- 172.21.15.17:0/871575386 shutdown_connections 2024-09-10T21:25:16.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:16.434+0000 7ff84e400640 1 --2- 172.21.15.17:0/871575386 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7ff8240725e0 0x7ff824074aa0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:16.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:16.434+0000 7ff84e400640 1 --2- 172.21.15.17:0/871575386 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff848152a60 0x7ff848151de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:16.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:16.434+0000 7ff84e400640 1 --2- 172.21.15.17:0/871575386 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff848152680 0x7ff8481518a0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:16.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:16.434+0000 7ff84e400640 1 --2- 172.21.15.17:0/871575386 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff8480b0bd0 0x7ff848068bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:16.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:16.434+0000 7ff84e400640 1 -- 172.21.15.17:0/871575386 >> 172.21.15.17:0/871575386 conn(0x7ff8480708a0 msgr2=0x7ff848070c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:16.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:16.434+0000 7ff84e400640 1 -- 172.21.15.17:0/871575386 shutdown_connections 2024-09-10T21:25:16.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:16.434+0000 7ff84e400640 1 -- 172.21.15.17:0/871575386 wait complete. 2024-09-10T21:25:16.463 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-10T21:25:16.464 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:25:16.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:16.614+0000 7f58ce200640 1 -- 172.21.15.17:0/732223620 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f58ac004810 msgr2=0x7f58ac004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:16.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:16.614+0000 7f58ce200640 1 --2- 172.21.15.17:0/732223620 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f58ac004810 0x7f58ac004c90 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f58b0001890 tx=0x7f58b001c890 comp rx=0 tx=0).stop 2024-09-10T21:25:16.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:16.614+0000 7f58c7e00640 1 -- 172.21.15.17:0/732223620 reap_dead start 2024-09-10T21:25:16.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:16.614+0000 7f58ce200640 1 -- 172.21.15.17:0/732223620 shutdown_connections 2024-09-10T21:25:16.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:16.614+0000 7f58ce200640 1 -- 172.21.15.17:0/732223620 >> 172.21.15.17:0/732223620 conn(0x7f58c80702f0 msgr2=0x7f58c8070700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:16.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:16.614+0000 7f58ce200640 1 -- 172.21.15.17:0/732223620 shutdown_connections 2024-09-10T21:25:16.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:16.614+0000 7f58ce200640 1 -- 172.21.15.17:0/732223620 wait complete. 2024-09-10T21:25:16.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:16.614+0000 7f58ce200640 1 Processor -- start 2024-09-10T21:25:16.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:16.614+0000 7f58ce200640 1 -- start start 2024-09-10T21:25:16.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:16.618+0000 7f58ce200640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f58ac004810 0x7f58c80c7180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:16.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:16.618+0000 7f58ce200640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f58c81f3a30 0x7f58c80c60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:16.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:16.618+0000 7f58ce200640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f58c815ad40 0x7f58c8159780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:16.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:16.618+0000 7f58ce200640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f58c80ce740 con 0x7f58c815ad40 2024-09-10T21:25:16.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:16.618+0000 7f58ce200640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f58c80b42c0 con 0x7f58ac004810 2024-09-10T21:25:16.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:16.618+0000 7f58ce200640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f58c80cb450 con 0x7f58c81f3a30 2024-09-10T21:25:16.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:16.618+0000 7f58c7e00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f58c815ad40 0x7f58c8159780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:16.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:16.618+0000 7f58c7400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f58ac004810 0x7f58c80c7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:16.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:16.618+0000 7f58c7e00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f58c815ad40 0x7f58c8159780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44060/0 (socket says 172.21.15.17:44060) 2024-09-10T21:25:16.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:16.618+0000 7f58c7e00640 1 -- 172.21.15.17:0/3812388307 learned_addr learned my addr 172.21.15.17:0/3812388307 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:25:16.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:16.618+0000 7f58bfe00640 1 --2- 172.21.15.17:0/3812388307 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f58c81f3a30 0x7f58c80c60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:16.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:16.618+0000 7f58c7e00640 1 -- 172.21.15.17:0/3812388307 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f58c81f3a30 msgr2=0x7f58c80c60f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:16.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:16.618+0000 7f58c7e00640 1 --2- 172.21.15.17:0/3812388307 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f58c81f3a30 0x7f58c80c60f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:16.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:16.618+0000 7f58c7e00640 1 -- 172.21.15.17:0/3812388307 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f58ac004810 msgr2=0x7f58c80c7180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:16.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:16.618+0000 7f58c7e00640 1 --2- 172.21.15.17:0/3812388307 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f58ac004810 0x7f58c80c7180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:16.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:16.618+0000 7f58c7e00640 1 -- 172.21.15.17:0/3812388307 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58c80c56c0 con 0x7f58c815ad40 2024-09-10T21:25:16.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:16.618+0000 7f58c7e00640 1 --2- 172.21.15.17:0/3812388307 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f58c815ad40 0x7f58c8159780 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f58c80cbe60 tx=0x7f58c000ea20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:16.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:16.618+0000 7f58bf400640 1 -- 172.21.15.17:0/3812388307 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58c0015070 con 0x7f58c815ad40 2024-09-10T21:25:16.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:16.618+0000 7f58bf400640 1 -- 172.21.15.17:0/3812388307 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f58c0010980 con 0x7f58c815ad40 2024-09-10T21:25:16.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:16.618+0000 7f58ce200640 1 -- 172.21.15.17:0/3812388307 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58c8150eb0 con 0x7f58c815ad40 2024-09-10T21:25:16.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:16.618+0000 7f58bf400640 1 -- 172.21.15.17:0/3812388307 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58c0010dd0 con 0x7f58c815ad40 2024-09-10T21:25:16.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:16.618+0000 7f58ce200640 1 -- 172.21.15.17:0/3812388307 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58c8151470 con 0x7f58c815ad40 2024-09-10T21:25:16.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:16.618+0000 7f58ce200640 1 -- 172.21.15.17:0/3812388307 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58c80c2ff0 con 0x7f58c815ad40 2024-09-10T21:25:16.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:16.622+0000 7f58bf400640 1 -- 172.21.15.17:0/3812388307 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f58c0011460 con 0x7f58c815ad40 2024-09-10T21:25:16.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:16.622+0000 7f58bf400640 1 --2- 172.21.15.17:0/3812388307 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f58a4072610 0x7f58a4074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:16.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:16.622+0000 7f58bf400640 1 -- 172.21.15.17:0/3812388307 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f58c0093b10 con 0x7f58c815ad40 2024-09-10T21:25:16.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:16.622+0000 7f58c7400640 1 --2- 172.21.15.17:0/3812388307 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f58a4072610 0x7f58a4074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:16.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:16.622+0000 7f58c7400640 1 --2- 172.21.15.17:0/3812388307 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f58a4072610 0x7f58a4074ad0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f58b8009b80 tx=0x7f58b80091e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:16.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:16.626+0000 7f58bf400640 1 -- 172.21.15.17:0/3812388307 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f58c005e910 con 0x7f58c815ad40 2024-09-10T21:25:16.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:16.842+0000 7f58ce200640 1 -- 172.21.15.17:0/3812388307 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f58c80c4870 con 0x7f58c815ad40 2024-09-10T21:25:16.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:16.842+0000 7f58bf400640 1 -- 172.21.15.17:0/3812388307 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+6870 (secure 0 0 0) 0x7f58c0063340 con 0x7f58c815ad40 2024-09-10T21:25:16.846 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-10T21:25:16.846 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":10,"btime":"2024-09-10T21:25:16:432823+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4350,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/1643071167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":1643071167},{"type":"v1","addr":"172.21.15.31:6835","nonce":1643071167}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4356,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6837/1666225712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6836","nonce":1666225712},{"type":"v1","addr":"172.21.15.17:6837","nonce":1666225712}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4384,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/3592734043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":3592734043},{"type":"v1","addr":"172.21.15.17:6833","nonce":3592734043}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4405,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/1293665856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":1293665856},{"type":"v1","addr":"172.21.15.17:6835","nonce":1293665856}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4411,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6838/1944852172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":1944852172},{"type":"v1","addr":"172.21.15.31:6838","nonce":1944852172}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":10,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:24:47.255397+0000","modified":"2024-09-10T21:25:15.961582+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4414},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4414":{"gid":4414,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.31:6839/3129062106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6837","nonce":3129062106},{"type":"v1","addr":"172.21.15.31:6839","nonce":3129062106}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4414,"qdb_cluster":[4414]},"id":1}]} 2024-09-10T21:25:16.846 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 10 2024-09-10T21:25:16.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:16.842+0000 7f58ce200640 1 -- 172.21.15.17:0/3812388307 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f58a4072610 msgr2=0x7f58a4074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:16.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:16.842+0000 7f58ce200640 1 --2- 172.21.15.17:0/3812388307 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f58a4072610 0x7f58a4074ad0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f58b8009b80 tx=0x7f58b80091e0 comp rx=0 tx=0).stop 2024-09-10T21:25:16.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:16.842+0000 7f58ce200640 1 -- 172.21.15.17:0/3812388307 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f58c815ad40 msgr2=0x7f58c8159780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:16.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:16.842+0000 7f58ce200640 1 --2- 172.21.15.17:0/3812388307 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f58c815ad40 0x7f58c8159780 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f58c80cbe60 tx=0x7f58c000ea20 comp rx=0 tx=0).stop 2024-09-10T21:25:16.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:16.842+0000 7f58ce200640 1 -- 172.21.15.17:0/3812388307 shutdown_connections 2024-09-10T21:25:16.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:16.842+0000 7f58ce200640 1 --2- 172.21.15.17:0/3812388307 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f58a4072610 0x7f58a4074ad0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:16.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:16.842+0000 7f58ce200640 1 --2- 172.21.15.17:0/3812388307 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f58c815ad40 0x7f58c8159780 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:16.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:16.842+0000 7f58ce200640 1 --2- 172.21.15.17:0/3812388307 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f58c81f3a30 0x7f58c80c60f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:16.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:16.842+0000 7f58ce200640 1 --2- 172.21.15.17:0/3812388307 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f58ac004810 0x7f58c80c7180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:16.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:16.842+0000 7f58ce200640 1 -- 172.21.15.17:0/3812388307 >> 172.21.15.17:0/3812388307 conn(0x7f58c80702f0 msgr2=0x7f58c8072640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:16.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:16.842+0000 7f58ce200640 1 -- 172.21.15.17:0/3812388307 shutdown_connections 2024-09-10T21:25:16.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:16.842+0000 7f58ce200640 1 -- 172.21.15.17:0/3812388307 wait complete. 2024-09-10T21:25:16.868 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:25:17.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:17.010+0000 7f263aa00640 1 -- 172.21.15.17:0/3329817231 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2634072b60 msgr2=0x7f26340c60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:17.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:17.010+0000 7f263aa00640 1 --2- 172.21.15.17:0/3329817231 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2634072b60 0x7f26340c60f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f2620009900 tx=0x7f262001f0d0 comp rx=0 tx=0).stop 2024-09-10T21:25:17.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:17.010+0000 7f263aa00640 1 -- 172.21.15.17:0/3329817231 shutdown_connections 2024-09-10T21:25:17.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:17.010+0000 7f263aa00640 1 --2- 172.21.15.17:0/3329817231 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2634072b60 0x7f26340c60f0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:17.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:17.010+0000 7f263aa00640 1 --2- 172.21.15.17:0/3329817231 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f2634151250 0x7f2634157bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:17.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:17.010+0000 7f263aa00640 1 -- 172.21.15.17:0/3329817231 >> 172.21.15.17:0/3329817231 conn(0x7f263406cdf0 msgr2=0x7f263406d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:17.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:17.010+0000 7f263aa00640 1 -- 172.21.15.17:0/3329817231 shutdown_connections 2024-09-10T21:25:17.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:17.010+0000 7f263aa00640 1 -- 172.21.15.17:0/3329817231 wait complete. 2024-09-10T21:25:17.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:17.010+0000 7f263aa00640 1 Processor -- start 2024-09-10T21:25:17.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:17.010+0000 7f263aa00640 1 -- start start 2024-09-10T21:25:17.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:17.014+0000 7f263aa00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f2634072b60 0x7f26341588a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:17.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:17.014+0000 7f263aa00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f26340c9100 0x7f26341ef590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:17.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:17.014+0000 7f263aa00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2634151250 0x7f2634144e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:17.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:17.014+0000 7f263aa00640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f26340c5610 con 0x7f2634151250 2024-09-10T21:25:17.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:17.014+0000 7f263aa00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f26340c5330 con 0x7f26340c9100 2024-09-10T21:25:17.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:17.014+0000 7f263aa00640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f26340c58f0 con 0x7f2634072b60 2024-09-10T21:25:17.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:17.014+0000 7f263a000640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2634151250 0x7f2634144e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:17.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:17.014+0000 7f263a000640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2634151250 0x7f2634144e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44086/0 (socket says 172.21.15.17:44086) 2024-09-10T21:25:17.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:17.014+0000 7f263a000640 1 -- 172.21.15.17:0/1412844731 learned_addr learned my addr 172.21.15.17:0/1412844731 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:25:17.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:17.014+0000 7f2639600640 1 --2- 172.21.15.17:0/1412844731 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f2634072b60 0x7f26341588a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:17.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:17.014+0000 7f2638c00640 1 --2- 172.21.15.17:0/1412844731 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f26340c9100 0x7f26341ef590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:17.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:17.014+0000 7f263a000640 1 -- 172.21.15.17:0/1412844731 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f2634072b60 msgr2=0x7f26341588a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:17.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:17.014+0000 7f263a000640 1 --2- 172.21.15.17:0/1412844731 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f2634072b60 0x7f26341588a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:17.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:17.014+0000 7f263a000640 1 -- 172.21.15.17:0/1412844731 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f26340c9100 msgr2=0x7f26341ef590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:17.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:17.014+0000 7f263a000640 1 --2- 172.21.15.17:0/1412844731 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f26340c9100 0x7f26341ef590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:17.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:17.014+0000 7f263a000640 1 -- 172.21.15.17:0/1412844731 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f263414aca0 con 0x7f2634151250 2024-09-10T21:25:17.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:17.014+0000 7f2638c00640 1 --2- 172.21.15.17:0/1412844731 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f26340c9100 0x7f26341ef590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:25:17.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:17.014+0000 7f263a000640 1 --2- 172.21.15.17:0/1412844731 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2634151250 0x7f2634144e30 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f262c00c7c0 tx=0x7f262c00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:17.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:17.014+0000 7f2631600640 1 -- 172.21.15.17:0/1412844731 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f262c003b80 con 0x7f2634151250 2024-09-10T21:25:17.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:17.014+0000 7f2631600640 1 -- 172.21.15.17:0/1412844731 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f262c003d20 con 0x7f2634151250 2024-09-10T21:25:17.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:17.014+0000 7f2631600640 1 -- 172.21.15.17:0/1412844731 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f262c00aa50 con 0x7f2634151250 2024-09-10T21:25:17.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:17.014+0000 7f263aa00640 1 -- 172.21.15.17:0/1412844731 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f263414af60 con 0x7f2634151250 2024-09-10T21:25:17.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:17.014+0000 7f263aa00640 1 -- 172.21.15.17:0/1412844731 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f263414b520 con 0x7f2634151250 2024-09-10T21:25:17.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:17.014+0000 7f263aa00640 1 -- 172.21.15.17:0/1412844731 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2634068e00 con 0x7f2634151250 2024-09-10T21:25:17.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:17.018+0000 7f2631600640 1 -- 172.21.15.17:0/1412844731 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f262c00abf0 con 0x7f2634151250 2024-09-10T21:25:17.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:17.018+0000 7f2631600640 1 --2- 172.21.15.17:0/1412844731 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f26140725e0 0x7f2614074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:17.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:17.018+0000 7f2631600640 1 -- 172.21.15.17:0/1412844731 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f262c0934c0 con 0x7f2634151250 2024-09-10T21:25:17.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:17.018+0000 7f2639600640 1 --2- 172.21.15.17:0/1412844731 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f26140725e0 0x7f2614074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:17.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:17.018+0000 7f2639600640 1 --2- 172.21.15.17:0/1412844731 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f26140725e0 0x7f2614074aa0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f2628009b40 tx=0x7f26280091e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:17.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:17.022+0000 7f2631600640 1 -- 172.21.15.17:0/1412844731 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f262c05e730 con 0x7f2634151250 2024-09-10T21:25:17.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:17.198+0000 7f263aa00640 1 -- 172.21.15.17:0/1412844731 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2634069e60 con 0x7f2634151250 2024-09-10T21:25:17.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:17.198+0000 7f2631600640 1 -- 172.21.15.17:0/1412844731 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18562 (secure 0 0 0) 0x7f262c05e030 con 0x7f2634151250 2024-09-10T21:25:17.203 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-10T21:25:17.204 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":25,"fsid":"5e2f023f-d6a6-49f4-a256-f4807dce3739","created":"2024-09-10T21:24:21.750712+0000","modified":"2024-09-10T21:24:52.295803+0000","last_up_change":"2024-09-10T21:24:35.877801+0000","last_in_change":"2024-09-10T21:24:25.831127+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:24:35.362670+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T21:24:45.211689+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T21:24:46.209784+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T21:24:49.871300+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0f5fabc0-434e-4066-b430-233969a548f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":4105489867},{"type":"v1","addr":"172.21.15.17:6809","nonce":4105489867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":4105489867},{"type":"v1","addr":"172.21.15.17:6811","nonce":4105489867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":4105489867},{"type":"v1","addr":"172.21.15.17:6815","nonce":4105489867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":4105489867},{"type":"v1","addr":"172.21.15.17:6813","nonce":4105489867}]},"public_addr":"172.21.15.17:6809/4105489867","cluster_addr":"172.21.15.17:6811/4105489867","heartbeat_back_addr":"172.21.15.17:6815/4105489867","heartbeat_front_addr":"172.21.15.17:6813/4105489867","state":["exists","up"]},{"osd":1,"uuid":"f6691cb3-2a91-4c90-a9fd-ece3c11dd6eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":2599842027},{"type":"v1","addr":"172.21.15.17:6825","nonce":2599842027}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":2599842027},{"type":"v1","addr":"172.21.15.17:6827","nonce":2599842027}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":2599842027},{"type":"v1","addr":"172.21.15.17:6831","nonce":2599842027}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":2599842027},{"type":"v1","addr":"172.21.15.17:6829","nonce":2599842027}]},"public_addr":"172.21.15.17:6825/2599842027","cluster_addr":"172.21.15.17:6827/2599842027","heartbeat_back_addr":"172.21.15.17:6831/2599842027","heartbeat_front_addr":"172.21.15.17:6829/2599842027","state":["exists","up"]},{"osd":2,"uuid":"cb61adb0-a216-437b-9598-06fa7c84aa64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":1248243634},{"type":"v1","addr":"172.21.15.17:6817","nonce":1248243634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":1248243634},{"type":"v1","addr":"172.21.15.17:6819","nonce":1248243634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":1248243634},{"type":"v1","addr":"172.21.15.17:6823","nonce":1248243634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":1248243634},{"type":"v1","addr":"172.21.15.17:6821","nonce":1248243634}]},"public_addr":"172.21.15.17:6817/1248243634","cluster_addr":"172.21.15.17:6819/1248243634","heartbeat_back_addr":"172.21.15.17:6823/1248243634","heartbeat_front_addr":"172.21.15.17:6821/1248243634","state":["exists","up"]},{"osd":3,"uuid":"6d2da26a-e0bf-47dc-b020-af86e6bcfccc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":2314489522},{"type":"v1","addr":"172.21.15.17:6801","nonce":2314489522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":2314489522},{"type":"v1","addr":"172.21.15.17:6803","nonce":2314489522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":2314489522},{"type":"v1","addr":"172.21.15.17:6807","nonce":2314489522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":2314489522},{"type":"v1","addr":"172.21.15.17:6805","nonce":2314489522}]},"public_addr":"172.21.15.17:6801/2314489522","cluster_addr":"172.21.15.17:6803/2314489522","heartbeat_back_addr":"172.21.15.17:6807/2314489522","heartbeat_front_addr":"172.21.15.17:6805/2314489522","state":["exists","up"]},{"osd":4,"uuid":"c65071dd-9ca7-4f1d-a584-45dc121419a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":202520744},{"type":"v1","addr":"172.21.15.31:6811","nonce":202520744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":202520744},{"type":"v1","addr":"172.21.15.31:6813","nonce":202520744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":202520744},{"type":"v1","addr":"172.21.15.31:6817","nonce":202520744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":202520744},{"type":"v1","addr":"172.21.15.31:6815","nonce":202520744}]},"public_addr":"172.21.15.31:6811/202520744","cluster_addr":"172.21.15.31:6813/202520744","heartbeat_back_addr":"172.21.15.31:6817/202520744","heartbeat_front_addr":"172.21.15.31:6815/202520744","state":["exists","up"]},{"osd":5,"uuid":"235389de-7777-4744-81b8-d0bfc11190e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":1472711057},{"type":"v1","addr":"172.21.15.31:6801","nonce":1472711057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":1472711057},{"type":"v1","addr":"172.21.15.31:6803","nonce":1472711057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":1472711057},{"type":"v1","addr":"172.21.15.31:6807","nonce":1472711057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":1472711057},{"type":"v1","addr":"172.21.15.31:6805","nonce":1472711057}]},"public_addr":"172.21.15.31:6801/1472711057","cluster_addr":"172.21.15.31:6803/1472711057","heartbeat_back_addr":"172.21.15.31:6807/1472711057","heartbeat_front_addr":"172.21.15.31:6805/1472711057","state":["exists","up"]},{"osd":6,"uuid":"cea70a57-3715-4ebb-8511-a528a9df2b1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":3754251089},{"type":"v1","addr":"172.21.15.31:6827","nonce":3754251089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":3754251089},{"type":"v1","addr":"172.21.15.31:6829","nonce":3754251089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":3754251089},{"type":"v1","addr":"172.21.15.31:6833","nonce":3754251089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":3754251089},{"type":"v1","addr":"172.21.15.31:6831","nonce":3754251089}]},"public_addr":"172.21.15.31:6827/3754251089","cluster_addr":"172.21.15.31:6829/3754251089","heartbeat_back_addr":"172.21.15.31:6833/3754251089","heartbeat_front_addr":"172.21.15.31:6831/3754251089","state":["exists","up"]},{"osd":7,"uuid":"01cc111b-a45f-4ca3-9cc1-7fe0afbeba45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":3948648934},{"type":"v1","addr":"172.21.15.31:6819","nonce":3948648934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":3948648934},{"type":"v1","addr":"172.21.15.31:6821","nonce":3948648934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":3948648934},{"type":"v1","addr":"172.21.15.31:6825","nonce":3948648934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":3948648934},{"type":"v1","addr":"172.21.15.31:6823","nonce":3948648934}]},"public_addr":"172.21.15.31:6819/3948648934","cluster_addr":"172.21.15.31:6821/3948648934","heartbeat_back_addr":"172.21.15.31:6825/3948648934","heartbeat_front_addr":"172.21.15.31:6823/3948648934","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:24:32.606989+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:24:32.891535+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:24:33.330793+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:24:32.811533+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:24:33.894624+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:24:32.773817+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:24:32.901865+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:24:32.907207+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:25:17.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:17.202+0000 7f263aa00640 1 -- 172.21.15.17:0/1412844731 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f26140725e0 msgr2=0x7f2614074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:17.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:17.202+0000 7f263aa00640 1 --2- 172.21.15.17:0/1412844731 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f26140725e0 0x7f2614074aa0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f2628009b40 tx=0x7f26280091e0 comp rx=0 tx=0).stop 2024-09-10T21:25:17.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:17.202+0000 7f263aa00640 1 -- 172.21.15.17:0/1412844731 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2634151250 msgr2=0x7f2634144e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:17.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:17.202+0000 7f263aa00640 1 --2- 172.21.15.17:0/1412844731 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2634151250 0x7f2634144e30 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f262c00c7c0 tx=0x7f262c00cc80 comp rx=0 tx=0).stop 2024-09-10T21:25:17.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:17.202+0000 7f263aa00640 1 -- 172.21.15.17:0/1412844731 shutdown_connections 2024-09-10T21:25:17.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:17.202+0000 7f263aa00640 1 --2- 172.21.15.17:0/1412844731 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f26140725e0 0x7f2614074aa0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:17.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:17.202+0000 7f263aa00640 1 --2- 172.21.15.17:0/1412844731 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2634151250 0x7f2634144e30 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:17.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:17.202+0000 7f263aa00640 1 --2- 172.21.15.17:0/1412844731 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f26340c9100 0x7f26341ef590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:17.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:17.202+0000 7f263aa00640 1 --2- 172.21.15.17:0/1412844731 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f2634072b60 0x7f26341588a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:17.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:17.202+0000 7f263aa00640 1 -- 172.21.15.17:0/1412844731 >> 172.21.15.17:0/1412844731 conn(0x7f263406cdf0 msgr2=0x7f26340cccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:17.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:17.202+0000 7f263aa00640 1 -- 172.21.15.17:0/1412844731 shutdown_connections 2024-09-10T21:25:17.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:17.202+0000 7f263aa00640 1 -- 172.21.15.17:0/1412844731 wait complete. 2024-09-10T21:25:17.226 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-10T21:25:17.226 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T21:25:17.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:17.370+0000 7fa3cbe00640 1 -- 172.21.15.17:0/3052802607 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa3c4152440 msgr2=0x7fa3c40b3eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:17.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:17.370+0000 7fa3cbe00640 1 --2- 172.21.15.17:0/3052802607 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa3c4152440 0x7fa3c40b3eb0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fa3c40cbaa0 tx=0x7fa3b801c830 comp rx=0 tx=0).stop 2024-09-10T21:25:17.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:17.370+0000 7fa3cbe00640 1 -- 172.21.15.17:0/3052802607 shutdown_connections 2024-09-10T21:25:17.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:17.370+0000 7fa3cbe00640 1 --2- 172.21.15.17:0/3052802607 >> v2:172.21.15.31:3300/0 conn(0x7fa3c4152820 0x7fa3c415c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:17.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:17.370+0000 7fa3cbe00640 1 --2- 172.21.15.17:0/3052802607 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa3c4152440 0x7fa3c40b3eb0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:17.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:17.370+0000 7fa3cbe00640 1 -- 172.21.15.17:0/3052802607 >> 172.21.15.17:0/3052802607 conn(0x7fa3c40702f0 msgr2=0x7fa3c4070700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:17.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:17.370+0000 7fa3cbe00640 1 -- 172.21.15.17:0/3052802607 shutdown_connections 2024-09-10T21:25:17.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:17.370+0000 7fa3cbe00640 1 -- 172.21.15.17:0/3052802607 wait complete. 2024-09-10T21:25:17.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:17.370+0000 7fa3cbe00640 1 Processor -- start 2024-09-10T21:25:17.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:17.370+0000 7fa3cbe00640 1 -- start start 2024-09-10T21:25:17.376 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:17.370+0000 7fa3cbe00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa3c40cdf20 0x7fa3c41574f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:17.376 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:17.370+0000 7fa3cbe00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa3c4152440 0x7fa3c415b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:17.376 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:17.370+0000 7fa3cbe00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa3c4152820 0x7fa3c41f3530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:17.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:17.370+0000 7fa3cbe00640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fa3c415c7c0 con 0x7fa3c4152440 2024-09-10T21:25:17.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:17.370+0000 7fa3cbe00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fa3c40ce740 con 0x7fa3c40cdf20 2024-09-10T21:25:17.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:17.370+0000 7fa3cbe00640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7fa3c40cb450 con 0x7fa3c4152820 2024-09-10T21:25:17.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:17.370+0000 7fa3c9e00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa3c4152820 0x7fa3c41f3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:17.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:17.370+0000 7fa3c9e00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa3c4152820 0x7fa3c41f3530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3301/0 says I am v2:172.21.15.17:42760/0 (socket says 172.21.15.17:42760) 2024-09-10T21:25:17.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:17.370+0000 7fa3c9e00640 1 -- 172.21.15.17:0/4040700709 learned_addr learned my addr 172.21.15.17:0/4040700709 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:25:17.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:17.370+0000 7fa3c8a00640 1 --2- 172.21.15.17:0/4040700709 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa3c4152440 0x7fa3c415b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:17.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:17.370+0000 7fa3c9400640 1 --2- 172.21.15.17:0/4040700709 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa3c40cdf20 0x7fa3c41574f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:17.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:17.374+0000 7fa3c9e00640 1 -- 172.21.15.17:0/4040700709 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa3c40cdf20 msgr2=0x7fa3c41574f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:17.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:17.374+0000 7fa3c9e00640 1 --2- 172.21.15.17:0/4040700709 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa3c40cdf20 0x7fa3c41574f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:17.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:17.374+0000 7fa3c9e00640 1 -- 172.21.15.17:0/4040700709 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa3c4152440 msgr2=0x7fa3c415b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:17.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:17.374+0000 7fa3c9e00640 1 --2- 172.21.15.17:0/4040700709 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa3c4152440 0x7fa3c415b010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:17.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:17.374+0000 7fa3c9e00640 1 -- 172.21.15.17:0/4040700709 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3c4157f20 con 0x7fa3c4152820 2024-09-10T21:25:17.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:17.374+0000 7fa3c9e00640 1 --2- 172.21.15.17:0/4040700709 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa3c4152820 0x7fa3c41f3530 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fa3bc00c760 tx=0x7fa3bc00cc20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:17.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:17.374+0000 7fa3c1600640 1 -- 172.21.15.17:0/4040700709 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3bc003db0 con 0x7fa3c4152820 2024-09-10T21:25:17.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:17.374+0000 7fa3c1600640 1 -- 172.21.15.17:0/4040700709 <== mon.2 v2:172.21.15.31:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa3bc00ce10 con 0x7fa3c4152820 2024-09-10T21:25:17.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:17.374+0000 7fa3cbe00640 1 -- 172.21.15.17:0/4040700709 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3c406e540 con 0x7fa3c4152820 2024-09-10T21:25:17.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:17.374+0000 7fa3c1600640 1 -- 172.21.15.17:0/4040700709 <== mon.2 v2:172.21.15.31:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3bc0052d0 con 0x7fa3c4152820 2024-09-10T21:25:17.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:17.374+0000 7fa3cbe00640 1 -- 172.21.15.17:0/4040700709 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3c4162c60 con 0x7fa3c4152820 2024-09-10T21:25:17.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:17.374+0000 7fa3cbe00640 1 -- 172.21.15.17:0/4040700709 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3c40c2ff0 con 0x7fa3c4152820 2024-09-10T21:25:17.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:17.374+0000 7fa3c1600640 1 -- 172.21.15.17:0/4040700709 <== mon.2 v2:172.21.15.31:3301/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fa3bc005470 con 0x7fa3c4152820 2024-09-10T21:25:17.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:17.378+0000 7fa3c1600640 1 --2- 172.21.15.17:0/4040700709 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fa3a40725e0 0x7fa3a4074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:17.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:17.378+0000 7fa3c1600640 1 -- 172.21.15.17:0/4040700709 <== mon.2 v2:172.21.15.31:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fa3bc093e10 con 0x7fa3c4152820 2024-09-10T21:25:17.381 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:17.378+0000 7fa3c9400640 1 --2- 172.21.15.17:0/4040700709 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fa3a40725e0 0x7fa3a4074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:17.381 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:17.378+0000 7fa3c9400640 1 --2- 172.21.15.17:0/4040700709 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fa3a40725e0 0x7fa3a4074aa0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa3b80af000 tx=0x7fa3b801b070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:17.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:17.382+0000 7fa3c1600640 1 -- 172.21.15.17:0/4040700709 <== mon.2 v2:172.21.15.31:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fa3bc05eb40 con 0x7fa3c4152820 2024-09-10T21:25:17.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:17.554+0000 7fa3cbe00640 1 -- 172.21.15.17:0/4040700709 --> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa3c4068a30 con 0x7fa3a40725e0 2024-09-10T21:25:17.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:17.562+0000 7fa3c1600640 1 -- 172.21.15.17:0/4040700709 <== mgr.4100 v2:172.21.15.31:6808/3101308586 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332600 (secure 0 0 0) 0x7fa3c4068a30 con 0x7fa3a40725e0 2024-09-10T21:25:17.568 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-10T21:25:17.572 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2024-09-10T21:25:17.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:17.566+0000 7fa3cbe00640 1 -- 172.21.15.17:0/4040700709 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fa3a40725e0 msgr2=0x7fa3a4074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:17.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:17.566+0000 7fa3cbe00640 1 --2- 172.21.15.17:0/4040700709 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fa3a40725e0 0x7fa3a4074aa0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa3b80af000 tx=0x7fa3b801b070 comp rx=0 tx=0).stop 2024-09-10T21:25:17.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:17.566+0000 7fa3cbe00640 1 -- 172.21.15.17:0/4040700709 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa3c4152820 msgr2=0x7fa3c41f3530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:17.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:17.566+0000 7fa3cbe00640 1 --2- 172.21.15.17:0/4040700709 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa3c4152820 0x7fa3c41f3530 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fa3bc00c760 tx=0x7fa3bc00cc20 comp rx=0 tx=0).stop 2024-09-10T21:25:17.574 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:17.566+0000 7fa3cbe00640 1 -- 172.21.15.17:0/4040700709 shutdown_connections 2024-09-10T21:25:17.574 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:17.566+0000 7fa3cbe00640 1 --2- 172.21.15.17:0/4040700709 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fa3a40725e0 0x7fa3a4074aa0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:17.574 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:17.566+0000 7fa3cbe00640 1 --2- 172.21.15.17:0/4040700709 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa3c4152820 0x7fa3c41f3530 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:17.574 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:17.566+0000 7fa3cbe00640 1 --2- 172.21.15.17:0/4040700709 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa3c4152440 0x7fa3c415b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:17.574 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:17.566+0000 7fa3cbe00640 1 --2- 172.21.15.17:0/4040700709 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa3c40cdf20 0x7fa3c41574f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:17.574 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:17.566+0000 7fa3cbe00640 1 -- 172.21.15.17:0/4040700709 >> 172.21.15.17:0/4040700709 conn(0x7fa3c40702f0 msgr2=0x7fa3c4072080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:17.574 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:17.566+0000 7fa3cbe00640 1 -- 172.21.15.17:0/4040700709 shutdown_connections 2024-09-10T21:25:17.574 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:17.566+0000 7fa3cbe00640 1 -- 172.21.15.17:0/4040700709 wait complete. 2024-09-10T21:25:17.593 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":40,"stamp":"2024-09-10T21:25:17.345280+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":149,"num_write_kb":1646,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":114,"ondisk_log_size":114,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":222608,"kb_used_data":2208,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754752112,"statfs":{"total":773094113280,"available":772866162688,"internally_reserved":0,"allocated":2260992,"data_stored":2009992,"data_compressed":66932,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12752,"internal_metadata":225496624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2230,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003112"},"pg_stats":[{"pgid":"2.1f","version":"24'4","reported_seq":27,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174456+0000","last_change":"2024-09-10T21:24:46.864556+0000","last_active":"2024-09-10T21:24:53.174456+0000","last_peered":"2024-09-10T21:24:53.174456+0000","last_clean":"2024-09-10T21:24:53.174456+0000","last_became_active":"2024-09-10T21:24:46.864288+0000","last_became_peered":"2024-09-10T21:24:46.864288+0000","last_unstale":"2024-09-10T21:24:53.174456+0000","last_undegraded":"2024-09-10T21:24:53.174456+0000","last_fullsized":"2024-09-10T21:24:53.174456+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:00:38.734807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.885137+0000","last_change":"2024-09-10T21:24:47.864452+0000","last_active":"2024-09-10T21:24:52.885137+0000","last_peered":"2024-09-10T21:24:52.885137+0000","last_clean":"2024-09-10T21:24:52.885137+0000","last_became_active":"2024-09-10T21:24:47.864168+0000","last_became_peered":"2024-09-10T21:24:47.864168+0000","last_unstale":"2024-09-10T21:24:52.885137+0000","last_undegraded":"2024-09-10T21:24:52.885137+0000","last_fullsized":"2024-09-10T21:24:52.885137+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:51:31.417845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.884507+0000","last_change":"2024-09-10T21:24:52.070054+0000","last_active":"2024-09-10T21:24:52.884507+0000","last_peered":"2024-09-10T21:24:52.884507+0000","last_clean":"2024-09-10T21:24:52.884507+0000","last_became_active":"2024-09-10T21:24:52.067258+0000","last_became_peered":"2024-09-10T21:24:52.067258+0000","last_unstale":"2024-09-10T21:24:52.884507+0000","last_undegraded":"2024-09-10T21:24:52.884507+0000","last_fullsized":"2024-09-10T21:24:52.884507+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:21:53.080013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"24'1","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.885320+0000","last_change":"2024-09-10T21:24:46.860648+0000","last_active":"2024-09-10T21:24:52.885320+0000","last_peered":"2024-09-10T21:24:52.885320+0000","last_clean":"2024-09-10T21:24:52.885320+0000","last_became_active":"2024-09-10T21:24:46.860341+0000","last_became_peered":"2024-09-10T21:24:46.860341+0000","last_unstale":"2024-09-10T21:24:52.885320+0000","last_undegraded":"2024-09-10T21:24:52.885320+0000","last_fullsized":"2024-09-10T21:24:52.885320+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:39:24.819979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174137+0000","last_change":"2024-09-10T21:24:47.865617+0000","last_active":"2024-09-10T21:24:53.174137+0000","last_peered":"2024-09-10T21:24:53.174137+0000","last_clean":"2024-09-10T21:24:53.174137+0000","last_became_active":"2024-09-10T21:24:47.865365+0000","last_became_peered":"2024-09-10T21:24:47.865365+0000","last_unstale":"2024-09-10T21:24:53.174137+0000","last_undegraded":"2024-09-10T21:24:53.174137+0000","last_fullsized":"2024-09-10T21:24:53.174137+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:10:05.971996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.883672+0000","last_change":"2024-09-10T21:24:51.900077+0000","last_active":"2024-09-10T21:24:52.883672+0000","last_peered":"2024-09-10T21:24:52.883672+0000","last_clean":"2024-09-10T21:24:52.883672+0000","last_became_active":"2024-09-10T21:24:51.899652+0000","last_became_peered":"2024-09-10T21:24:51.899652+0000","last_unstale":"2024-09-10T21:24:52.883672+0000","last_undegraded":"2024-09-10T21:24:52.883672+0000","last_fullsized":"2024-09-10T21:24:52.883672+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:40:49.585857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"24'2","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.175383+0000","last_change":"2024-09-10T21:24:46.861040+0000","last_active":"2024-09-10T21:24:53.175383+0000","last_peered":"2024-09-10T21:24:53.175383+0000","last_clean":"2024-09-10T21:24:53.175383+0000","last_became_active":"2024-09-10T21:24:46.860780+0000","last_became_peered":"2024-09-10T21:24:46.860780+0000","last_unstale":"2024-09-10T21:24:53.175383+0000","last_undegraded":"2024-09-10T21:24:53.175383+0000","last_fullsized":"2024-09-10T21:24:53.175383+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:53:31.454472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174596+0000","last_change":"2024-09-10T21:24:47.877997+0000","last_active":"2024-09-10T21:24:53.174596+0000","last_peered":"2024-09-10T21:24:53.174596+0000","last_clean":"2024-09-10T21:24:53.174596+0000","last_became_active":"2024-09-10T21:24:47.877765+0000","last_became_peered":"2024-09-10T21:24:47.877765+0000","last_unstale":"2024-09-10T21:24:53.174596+0000","last_undegraded":"2024-09-10T21:24:53.174596+0000","last_fullsized":"2024-09-10T21:24:53.174596+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:33:20.590209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.886528+0000","last_change":"2024-09-10T21:24:51.908333+0000","last_active":"2024-09-10T21:24:52.886528+0000","last_peered":"2024-09-10T21:24:52.886528+0000","last_clean":"2024-09-10T21:24:52.886528+0000","last_became_active":"2024-09-10T21:24:51.907933+0000","last_became_peered":"2024-09-10T21:24:51.907933+0000","last_unstale":"2024-09-10T21:24:52.886528+0000","last_undegraded":"2024-09-10T21:24:52.886528+0000","last_fullsized":"2024-09-10T21:24:52.886528+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:07:33.193555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"24'2","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174867+0000","last_change":"2024-09-10T21:24:46.873044+0000","last_active":"2024-09-10T21:24:53.174867+0000","last_peered":"2024-09-10T21:24:53.174867+0000","last_clean":"2024-09-10T21:24:53.174867+0000","last_became_active":"2024-09-10T21:24:46.872075+0000","last_became_peered":"2024-09-10T21:24:46.872075+0000","last_unstale":"2024-09-10T21:24:53.174867+0000","last_undegraded":"2024-09-10T21:24:53.174867+0000","last_fullsized":"2024-09-10T21:24:53.174867+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:40:33.632846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174402+0000","last_change":"2024-09-10T21:24:47.878094+0000","last_active":"2024-09-10T21:24:53.174402+0000","last_peered":"2024-09-10T21:24:53.174402+0000","last_clean":"2024-09-10T21:24:53.174402+0000","last_became_active":"2024-09-10T21:24:47.876688+0000","last_became_peered":"2024-09-10T21:24:47.876688+0000","last_unstale":"2024-09-10T21:24:53.174402+0000","last_undegraded":"2024-09-10T21:24:53.174402+0000","last_fullsized":"2024-09-10T21:24:53.174402+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:03:37.076080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.884328+0000","last_change":"2024-09-10T21:24:51.900208+0000","last_active":"2024-09-10T21:24:52.884328+0000","last_peered":"2024-09-10T21:24:52.884328+0000","last_clean":"2024-09-10T21:24:52.884328+0000","last_became_active":"2024-09-10T21:24:51.899576+0000","last_became_peered":"2024-09-10T21:24:51.899576+0000","last_unstale":"2024-09-10T21:24:52.884328+0000","last_undegraded":"2024-09-10T21:24:52.884328+0000","last_fullsized":"2024-09-10T21:24:52.884328+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:52:49.479932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.883746+0000","last_change":"2024-09-10T21:24:47.869830+0000","last_active":"2024-09-10T21:24:52.883746+0000","last_peered":"2024-09-10T21:24:52.883746+0000","last_clean":"2024-09-10T21:24:52.883746+0000","last_became_active":"2024-09-10T21:24:47.869569+0000","last_became_peered":"2024-09-10T21:24:47.869569+0000","last_unstale":"2024-09-10T21:24:52.883746+0000","last_undegraded":"2024-09-10T21:24:52.883746+0000","last_fullsized":"2024-09-10T21:24:52.883746+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:42:15.320518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"24'2","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174248+0000","last_change":"2024-09-10T21:24:46.867297+0000","last_active":"2024-09-10T21:24:53.174248+0000","last_peered":"2024-09-10T21:24:53.174248+0000","last_clean":"2024-09-10T21:24:53.174248+0000","last_became_active":"2024-09-10T21:24:46.866893+0000","last_became_peered":"2024-09-10T21:24:46.866893+0000","last_unstale":"2024-09-10T21:24:53.174248+0000","last_undegraded":"2024-09-10T21:24:53.174248+0000","last_fullsized":"2024-09-10T21:24:53.174248+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:53.197666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.885044+0000","last_change":"2024-09-10T21:24:51.908412+0000","last_active":"2024-09-10T21:24:52.885044+0000","last_peered":"2024-09-10T21:24:52.885044+0000","last_clean":"2024-09-10T21:24:52.885044+0000","last_became_active":"2024-09-10T21:24:51.907923+0000","last_became_peered":"2024-09-10T21:24:51.907923+0000","last_unstale":"2024-09-10T21:24:52.885044+0000","last_undegraded":"2024-09-10T21:24:52.885044+0000","last_fullsized":"2024-09-10T21:24:52.885044+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:26:46.479501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.884755+0000","last_change":"2024-09-10T21:24:47.862479+0000","last_active":"2024-09-10T21:24:52.884755+0000","last_peered":"2024-09-10T21:24:52.884755+0000","last_clean":"2024-09-10T21:24:52.884755+0000","last_became_active":"2024-09-10T21:24:47.861842+0000","last_became_peered":"2024-09-10T21:24:47.861842+0000","last_unstale":"2024-09-10T21:24:52.884755+0000","last_undegraded":"2024-09-10T21:24:52.884755+0000","last_fullsized":"2024-09-10T21:24:52.884755+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:53:09.082512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.878911+0000","last_change":"2024-09-10T21:24:46.852467+0000","last_active":"2024-09-10T21:24:52.878911+0000","last_peered":"2024-09-10T21:24:52.878911+0000","last_clean":"2024-09-10T21:24:52.878911+0000","last_became_active":"2024-09-10T21:24:46.851027+0000","last_became_peered":"2024-09-10T21:24:46.851027+0000","last_unstale":"2024-09-10T21:24:52.878911+0000","last_undegraded":"2024-09-10T21:24:52.878911+0000","last_fullsized":"2024-09-10T21:24:52.878911+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:39:27.441714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.886564+0000","last_change":"2024-09-10T21:24:51.903577+0000","last_active":"2024-09-10T21:24:52.886564+0000","last_peered":"2024-09-10T21:24:52.886564+0000","last_clean":"2024-09-10T21:24:52.886564+0000","last_became_active":"2024-09-10T21:24:51.903031+0000","last_became_peered":"2024-09-10T21:24:51.903031+0000","last_unstale":"2024-09-10T21:24:52.886564+0000","last_undegraded":"2024-09-10T21:24:52.886564+0000","last_fullsized":"2024-09-10T21:24:52.886564+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:08:52.787720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.884612+0000","last_change":"2024-09-10T21:24:47.856334+0000","last_active":"2024-09-10T21:24:52.884612+0000","last_peered":"2024-09-10T21:24:52.884612+0000","last_clean":"2024-09-10T21:24:52.884612+0000","last_became_active":"2024-09-10T21:24:47.856003+0000","last_became_peered":"2024-09-10T21:24:47.856003+0000","last_unstale":"2024-09-10T21:24:52.884612+0000","last_undegraded":"2024-09-10T21:24:52.884612+0000","last_fullsized":"2024-09-10T21:24:52.884612+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:58:53.698584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.878371+0000","last_change":"2024-09-10T21:24:46.853373+0000","last_active":"2024-09-10T21:24:52.878371+0000","last_peered":"2024-09-10T21:24:52.878371+0000","last_clean":"2024-09-10T21:24:52.878371+0000","last_became_active":"2024-09-10T21:24:46.852993+0000","last_became_peered":"2024-09-10T21:24:46.852993+0000","last_unstale":"2024-09-10T21:24:52.878371+0000","last_undegraded":"2024-09-10T21:24:52.878371+0000","last_fullsized":"2024-09-10T21:24:52.878371+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:53:31.454472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.879271+0000","last_change":"2024-09-10T21:24:52.065246+0000","last_active":"2024-09-10T21:24:52.879271+0000","last_peered":"2024-09-10T21:24:52.879271+0000","last_clean":"2024-09-10T21:24:52.879271+0000","last_became_active":"2024-09-10T21:24:52.064884+0000","last_became_peered":"2024-09-10T21:24:52.064884+0000","last_unstale":"2024-09-10T21:24:52.879271+0000","last_undegraded":"2024-09-10T21:24:52.879271+0000","last_fullsized":"2024-09-10T21:24:52.879271+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:54:54.393305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.884318+0000","last_change":"2024-09-10T21:24:47.856517+0000","last_active":"2024-09-10T21:24:52.884318+0000","last_peered":"2024-09-10T21:24:52.884318+0000","last_clean":"2024-09-10T21:24:52.884318+0000","last_became_active":"2024-09-10T21:24:47.856185+0000","last_became_peered":"2024-09-10T21:24:47.856185+0000","last_unstale":"2024-09-10T21:24:52.884318+0000","last_undegraded":"2024-09-10T21:24:52.884318+0000","last_fullsized":"2024-09-10T21:24:52.884318+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:26:00.852818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.173635+0000","last_change":"2024-09-10T21:24:46.867888+0000","last_active":"2024-09-10T21:24:53.173635+0000","last_peered":"2024-09-10T21:24:53.173635+0000","last_clean":"2024-09-10T21:24:53.173635+0000","last_became_active":"2024-09-10T21:24:46.866477+0000","last_became_peered":"2024-09-10T21:24:46.866477+0000","last_unstale":"2024-09-10T21:24:53.173635+0000","last_undegraded":"2024-09-10T21:24:53.173635+0000","last_fullsized":"2024-09-10T21:24:53.173635+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:39:27.441714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.885185+0000","last_change":"2024-09-10T21:24:51.908376+0000","last_active":"2024-09-10T21:24:52.885185+0000","last_peered":"2024-09-10T21:24:52.885185+0000","last_clean":"2024-09-10T21:24:52.885185+0000","last_became_active":"2024-09-10T21:24:51.907878+0000","last_became_peered":"2024-09-10T21:24:51.907878+0000","last_unstale":"2024-09-10T21:24:52.885185+0000","last_undegraded":"2024-09-10T21:24:52.885185+0000","last_fullsized":"2024-09-10T21:24:52.885185+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:39:02.492952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174342+0000","last_change":"2024-09-10T21:24:47.844823+0000","last_active":"2024-09-10T21:24:53.174342+0000","last_peered":"2024-09-10T21:24:53.174342+0000","last_clean":"2024-09-10T21:24:53.174342+0000","last_became_active":"2024-09-10T21:24:47.844505+0000","last_became_peered":"2024-09-10T21:24:47.844505+0000","last_unstale":"2024-09-10T21:24:53.174342+0000","last_undegraded":"2024-09-10T21:24:53.174342+0000","last_fullsized":"2024-09-10T21:24:53.174342+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:37:51.784258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"24'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.885358+0000","last_change":"2024-09-10T21:24:46.862172+0000","last_active":"2024-09-10T21:24:52.885358+0000","last_peered":"2024-09-10T21:24:52.885358+0000","last_clean":"2024-09-10T21:24:52.885358+0000","last_became_active":"2024-09-10T21:24:46.861473+0000","last_became_peered":"2024-09-10T21:24:46.861473+0000","last_unstale":"2024-09-10T21:24:52.885358+0000","last_undegraded":"2024-09-10T21:24:52.885358+0000","last_fullsized":"2024-09-10T21:24:52.885358+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:53:31.454472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.883945+0000","last_change":"2024-09-10T21:24:52.062841+0000","last_active":"2024-09-10T21:24:52.883945+0000","last_peered":"2024-09-10T21:24:52.883945+0000","last_clean":"2024-09-10T21:24:52.883945+0000","last_became_active":"2024-09-10T21:24:52.061581+0000","last_became_peered":"2024-09-10T21:24:52.061581+0000","last_unstale":"2024-09-10T21:24:52.883945+0000","last_undegraded":"2024-09-10T21:24:52.883945+0000","last_fullsized":"2024-09-10T21:24:52.883945+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:34:03.325781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.885543+0000","last_change":"2024-09-10T21:24:47.859801+0000","last_active":"2024-09-10T21:24:52.885543+0000","last_peered":"2024-09-10T21:24:52.885543+0000","last_clean":"2024-09-10T21:24:52.885543+0000","last_became_active":"2024-09-10T21:24:47.859483+0000","last_became_peered":"2024-09-10T21:24:47.859483+0000","last_unstale":"2024-09-10T21:24:52.885543+0000","last_undegraded":"2024-09-10T21:24:52.885543+0000","last_fullsized":"2024-09-10T21:24:52.885543+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:38:21.368499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"25'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.879175+0000","last_change":"2024-09-10T21:24:46.860624+0000","last_active":"2024-09-10T21:24:52.879175+0000","last_peered":"2024-09-10T21:24:52.879175+0000","last_clean":"2024-09-10T21:24:52.879175+0000","last_became_active":"2024-09-10T21:24:46.860309+0000","last_became_peered":"2024-09-10T21:24:46.860309+0000","last_unstale":"2024-09-10T21:24:52.879175+0000","last_undegraded":"2024-09-10T21:24:52.879175+0000","last_fullsized":"2024-09-10T21:24:52.879175+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:37:12.332013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.884414+0000","last_change":"2024-09-10T21:24:52.068581+0000","last_active":"2024-09-10T21:24:52.884414+0000","last_peered":"2024-09-10T21:24:52.884414+0000","last_clean":"2024-09-10T21:24:52.884414+0000","last_became_active":"2024-09-10T21:24:52.068250+0000","last_became_peered":"2024-09-10T21:24:52.068250+0000","last_unstale":"2024-09-10T21:24:52.884414+0000","last_undegraded":"2024-09-10T21:24:52.884414+0000","last_fullsized":"2024-09-10T21:24:52.884414+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:52:01.990009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.879000+0000","last_change":"2024-09-10T21:24:47.854042+0000","last_active":"2024-09-10T21:24:52.879000+0000","last_peered":"2024-09-10T21:24:52.879000+0000","last_clean":"2024-09-10T21:24:52.879000+0000","last_became_active":"2024-09-10T21:24:47.853750+0000","last_became_peered":"2024-09-10T21:24:47.853750+0000","last_unstale":"2024-09-10T21:24:52.879000+0000","last_undegraded":"2024-09-10T21:24:52.879000+0000","last_fullsized":"2024-09-10T21:24:52.879000+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:06:50.775132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.175510+0000","last_change":"2024-09-10T21:24:46.864617+0000","last_active":"2024-09-10T21:24:53.175510+0000","last_peered":"2024-09-10T21:24:53.175510+0000","last_clean":"2024-09-10T21:24:53.175510+0000","last_became_active":"2024-09-10T21:24:46.864149+0000","last_became_peered":"2024-09-10T21:24:46.864149+0000","last_unstale":"2024-09-10T21:24:53.175510+0000","last_undegraded":"2024-09-10T21:24:53.175510+0000","last_fullsized":"2024-09-10T21:24:53.175510+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:24:15.673107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174530+0000","last_change":"2024-09-10T21:24:52.061485+0000","last_active":"2024-09-10T21:24:53.174530+0000","last_peered":"2024-09-10T21:24:53.174530+0000","last_clean":"2024-09-10T21:24:53.174530+0000","last_became_active":"2024-09-10T21:24:52.060953+0000","last_became_peered":"2024-09-10T21:24:52.060953+0000","last_unstale":"2024-09-10T21:24:53.174530+0000","last_undegraded":"2024-09-10T21:24:53.174530+0000","last_fullsized":"2024-09-10T21:24:53.174530+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:49:30.147202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174576+0000","last_change":"2024-09-10T21:24:47.867241+0000","last_active":"2024-09-10T21:24:53.174576+0000","last_peered":"2024-09-10T21:24:53.174576+0000","last_clean":"2024-09-10T21:24:53.174576+0000","last_became_active":"2024-09-10T21:24:47.866907+0000","last_became_peered":"2024-09-10T21:24:47.866907+0000","last_unstale":"2024-09-10T21:24:53.174576+0000","last_undegraded":"2024-09-10T21:24:53.174576+0000","last_fullsized":"2024-09-10T21:24:53.174576+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:07.620800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.879175+0000","last_change":"2024-09-10T21:24:46.856848+0000","last_active":"2024-09-10T21:24:52.879175+0000","last_peered":"2024-09-10T21:24:52.879175+0000","last_clean":"2024-09-10T21:24:52.879175+0000","last_became_active":"2024-09-10T21:24:46.856489+0000","last_became_peered":"2024-09-10T21:24:46.856489+0000","last_unstale":"2024-09-10T21:24:52.879175+0000","last_undegraded":"2024-09-10T21:24:52.879175+0000","last_fullsized":"2024-09-10T21:24:52.879175+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:46:55.286321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.878563+0000","last_change":"2024-09-10T21:24:51.901016+0000","last_active":"2024-09-10T21:24:52.878563+0000","last_peered":"2024-09-10T21:24:52.878563+0000","last_clean":"2024-09-10T21:24:52.878563+0000","last_became_active":"2024-09-10T21:24:51.900479+0000","last_became_peered":"2024-09-10T21:24:51.900479+0000","last_unstale":"2024-09-10T21:24:52.878563+0000","last_undegraded":"2024-09-10T21:24:52.878563+0000","last_fullsized":"2024-09-10T21:24:52.878563+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:36.741810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.886797+0000","last_change":"2024-09-10T21:24:47.858753+0000","last_active":"2024-09-10T21:24:52.886797+0000","last_peered":"2024-09-10T21:24:52.886797+0000","last_clean":"2024-09-10T21:24:52.886797+0000","last_became_active":"2024-09-10T21:24:47.858438+0000","last_became_peered":"2024-09-10T21:24:47.858438+0000","last_unstale":"2024-09-10T21:24:52.886797+0000","last_undegraded":"2024-09-10T21:24:52.886797+0000","last_fullsized":"2024-09-10T21:24:52.886797+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:12:09.198013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.175120+0000","last_change":"2024-09-10T21:24:46.863610+0000","last_active":"2024-09-10T21:24:53.175120+0000","last_peered":"2024-09-10T21:24:53.175120+0000","last_clean":"2024-09-10T21:24:53.175120+0000","last_became_active":"2024-09-10T21:24:46.863150+0000","last_became_peered":"2024-09-10T21:24:46.863150+0000","last_unstale":"2024-09-10T21:24:53.175120+0000","last_undegraded":"2024-09-10T21:24:53.175120+0000","last_fullsized":"2024-09-10T21:24:53.175120+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:27:35.473995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.878912+0000","last_change":"2024-09-10T21:24:51.907366+0000","last_active":"2024-09-10T21:24:52.878912+0000","last_peered":"2024-09-10T21:24:52.878912+0000","last_clean":"2024-09-10T21:24:52.878912+0000","last_became_active":"2024-09-10T21:24:51.906888+0000","last_became_peered":"2024-09-10T21:24:51.906888+0000","last_unstale":"2024-09-10T21:24:52.878912+0000","last_undegraded":"2024-09-10T21:24:52.878912+0000","last_fullsized":"2024-09-10T21:24:52.878912+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:24:54.278514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.878445+0000","last_change":"2024-09-10T21:24:47.850155+0000","last_active":"2024-09-10T21:24:52.878445+0000","last_peered":"2024-09-10T21:24:52.878445+0000","last_clean":"2024-09-10T21:24:52.878445+0000","last_became_active":"2024-09-10T21:24:47.849846+0000","last_became_peered":"2024-09-10T21:24:47.849846+0000","last_unstale":"2024-09-10T21:24:52.878445+0000","last_undegraded":"2024-09-10T21:24:52.878445+0000","last_fullsized":"2024-09-10T21:24:52.878445+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:13:40.032808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.173931+0000","last_change":"2024-09-10T21:24:46.851800+0000","last_active":"2024-09-10T21:24:53.173931+0000","last_peered":"2024-09-10T21:24:53.173931+0000","last_clean":"2024-09-10T21:24:53.173931+0000","last_became_active":"2024-09-10T21:24:46.851200+0000","last_became_peered":"2024-09-10T21:24:46.851200+0000","last_unstale":"2024-09-10T21:24:53.173931+0000","last_undegraded":"2024-09-10T21:24:53.173931+0000","last_fullsized":"2024-09-10T21:24:53.173931+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:00:22.839361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.879296+0000","last_change":"2024-09-10T21:24:51.902529+0000","last_active":"2024-09-10T21:24:52.879296+0000","last_peered":"2024-09-10T21:24:52.879296+0000","last_clean":"2024-09-10T21:24:52.879296+0000","last_became_active":"2024-09-10T21:24:51.901141+0000","last_became_peered":"2024-09-10T21:24:51.901141+0000","last_unstale":"2024-09-10T21:24:52.879296+0000","last_undegraded":"2024-09-10T21:24:52.879296+0000","last_fullsized":"2024-09-10T21:24:52.879296+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:08:40.203348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.173878+0000","last_change":"2024-09-10T21:24:47.846582+0000","last_active":"2024-09-10T21:24:53.173878+0000","last_peered":"2024-09-10T21:24:53.173878+0000","last_clean":"2024-09-10T21:24:53.173878+0000","last_became_active":"2024-09-10T21:24:47.846358+0000","last_became_peered":"2024-09-10T21:24:47.846358+0000","last_unstale":"2024-09-10T21:24:53.173878+0000","last_undegraded":"2024-09-10T21:24:53.173878+0000","last_fullsized":"2024-09-10T21:24:53.173878+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:58:53.698584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.173773+0000","last_change":"2024-09-10T21:24:46.870908+0000","last_active":"2024-09-10T21:24:53.173773+0000","last_peered":"2024-09-10T21:24:53.173773+0000","last_clean":"2024-09-10T21:24:53.173773+0000","last_became_active":"2024-09-10T21:24:46.868375+0000","last_became_peered":"2024-09-10T21:24:46.868375+0000","last_unstale":"2024-09-10T21:24:53.173773+0000","last_undegraded":"2024-09-10T21:24:53.173773+0000","last_fullsized":"2024-09-10T21:24:53.173773+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:37:12.332013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174362+0000","last_change":"2024-09-10T21:24:52.069018+0000","last_active":"2024-09-10T21:24:53.174362+0000","last_peered":"2024-09-10T21:24:53.174362+0000","last_clean":"2024-09-10T21:24:53.174362+0000","last_became_active":"2024-09-10T21:24:52.067308+0000","last_became_peered":"2024-09-10T21:24:52.067308+0000","last_unstale":"2024-09-10T21:24:53.174362+0000","last_undegraded":"2024-09-10T21:24:53.174362+0000","last_fullsized":"2024-09-10T21:24:53.174362+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:07:25.162168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.884767+0000","last_change":"2024-09-10T21:24:47.871542+0000","last_active":"2024-09-10T21:24:52.884767+0000","last_peered":"2024-09-10T21:24:52.884767+0000","last_clean":"2024-09-10T21:24:52.884767+0000","last_became_active":"2024-09-10T21:24:47.871128+0000","last_became_peered":"2024-09-10T21:24:47.871128+0000","last_unstale":"2024-09-10T21:24:52.884767+0000","last_undegraded":"2024-09-10T21:24:52.884767+0000","last_fullsized":"2024-09-10T21:24:52.884767+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:51:22.876126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174204+0000","last_change":"2024-09-10T21:24:46.859218+0000","last_active":"2024-09-10T21:24:53.174204+0000","last_peered":"2024-09-10T21:24:53.174204+0000","last_clean":"2024-09-10T21:24:53.174204+0000","last_became_active":"2024-09-10T21:24:46.853190+0000","last_became_peered":"2024-09-10T21:24:46.853190+0000","last_unstale":"2024-09-10T21:24:53.174204+0000","last_undegraded":"2024-09-10T21:24:53.174204+0000","last_fullsized":"2024-09-10T21:24:53.174204+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:26:34.848759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":119,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.173768+0000","last_change":"2024-09-10T21:24:37.216542+0000","last_active":"2024-09-10T21:24:53.173768+0000","last_peered":"2024-09-10T21:24:53.173768+0000","last_clean":"2024-09-10T21:24:53.173768+0000","last_became_active":"2024-09-10T21:24:37.215443+0000","last_became_peered":"2024-09-10T21:24:37.215443+0000","last_unstale":"2024-09-10T21:24:53.173768+0000","last_undegraded":"2024-09-10T21:24:53.173768+0000","last_fullsized":"2024-09-10T21:24:53.173768+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:35.877801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:35.877801+0000","last_clean_scrub_stamp":"2024-09-10T21:24:35.877801+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:14:20.260329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.884231+0000","last_change":"2024-09-10T21:24:52.067645+0000","last_active":"2024-09-10T21:24:52.884231+0000","last_peered":"2024-09-10T21:24:52.884231+0000","last_clean":"2024-09-10T21:24:52.884231+0000","last_became_active":"2024-09-10T21:24:52.067148+0000","last_became_peered":"2024-09-10T21:24:52.067148+0000","last_unstale":"2024-09-10T21:24:52.884231+0000","last_undegraded":"2024-09-10T21:24:52.884231+0000","last_fullsized":"2024-09-10T21:24:52.884231+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:54:42.435011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.175128+0000","last_change":"2024-09-10T21:24:47.867282+0000","last_active":"2024-09-10T21:24:53.175128+0000","last_peered":"2024-09-10T21:24:53.175128+0000","last_clean":"2024-09-10T21:24:53.175128+0000","last_became_active":"2024-09-10T21:24:47.865582+0000","last_became_peered":"2024-09-10T21:24:47.865582+0000","last_unstale":"2024-09-10T21:24:53.175128+0000","last_undegraded":"2024-09-10T21:24:53.175128+0000","last_fullsized":"2024-09-10T21:24:53.175128+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:38:55.009670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174808+0000","last_change":"2024-09-10T21:24:46.859532+0000","last_active":"2024-09-10T21:24:53.174808+0000","last_peered":"2024-09-10T21:24:53.174808+0000","last_clean":"2024-09-10T21:24:53.174808+0000","last_became_active":"2024-09-10T21:24:46.859144+0000","last_became_peered":"2024-09-10T21:24:46.859144+0000","last_unstale":"2024-09-10T21:24:53.174808+0000","last_undegraded":"2024-09-10T21:24:53.174808+0000","last_fullsized":"2024-09-10T21:24:53.174808+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:48:41.992441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.883627+0000","last_change":"2024-09-10T21:24:52.061029+0000","last_active":"2024-09-10T21:24:52.883627+0000","last_peered":"2024-09-10T21:24:52.883627+0000","last_clean":"2024-09-10T21:24:52.883627+0000","last_became_active":"2024-09-10T21:24:52.060605+0000","last_became_peered":"2024-09-10T21:24:52.060605+0000","last_unstale":"2024-09-10T21:24:52.883627+0000","last_undegraded":"2024-09-10T21:24:52.883627+0000","last_fullsized":"2024-09-10T21:24:52.883627+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:09:48.077266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174744+0000","last_change":"2024-09-10T21:24:47.866927+0000","last_active":"2024-09-10T21:24:53.174744+0000","last_peered":"2024-09-10T21:24:53.174744+0000","last_clean":"2024-09-10T21:24:53.174744+0000","last_became_active":"2024-09-10T21:24:47.866650+0000","last_became_peered":"2024-09-10T21:24:47.866650+0000","last_unstale":"2024-09-10T21:24:53.174744+0000","last_undegraded":"2024-09-10T21:24:53.174744+0000","last_fullsized":"2024-09-10T21:24:53.174744+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:47:37.304624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"24'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.879000+0000","last_change":"2024-09-10T21:24:46.857082+0000","last_active":"2024-09-10T21:24:52.879000+0000","last_peered":"2024-09-10T21:24:52.879000+0000","last_clean":"2024-09-10T21:24:52.879000+0000","last_became_active":"2024-09-10T21:24:46.856799+0000","last_became_peered":"2024-09-10T21:24:46.856799+0000","last_unstale":"2024-09-10T21:24:52.879000+0000","last_undegraded":"2024-09-10T21:24:52.879000+0000","last_fullsized":"2024-09-10T21:24:52.879000+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:40:33.632846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174743+0000","last_change":"2024-09-10T21:24:51.911880+0000","last_active":"2024-09-10T21:24:53.174743+0000","last_peered":"2024-09-10T21:24:53.174743+0000","last_clean":"2024-09-10T21:24:53.174743+0000","last_became_active":"2024-09-10T21:24:51.911088+0000","last_became_peered":"2024-09-10T21:24:51.911088+0000","last_unstale":"2024-09-10T21:24:53.174743+0000","last_undegraded":"2024-09-10T21:24:53.174743+0000","last_fullsized":"2024-09-10T21:24:53.174743+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:38:57.030655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.175370+0000","last_change":"2024-09-10T21:24:47.876946+0000","last_active":"2024-09-10T21:24:53.175370+0000","last_peered":"2024-09-10T21:24:53.175370+0000","last_clean":"2024-09-10T21:24:53.175370+0000","last_became_active":"2024-09-10T21:24:47.876690+0000","last_became_peered":"2024-09-10T21:24:47.876690+0000","last_unstale":"2024-09-10T21:24:53.175370+0000","last_undegraded":"2024-09-10T21:24:53.175370+0000","last_fullsized":"2024-09-10T21:24:53.175370+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:14:06.368275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"24'2","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.175105+0000","last_change":"2024-09-10T21:24:46.855248+0000","last_active":"2024-09-10T21:24:53.175105+0000","last_peered":"2024-09-10T21:24:53.175105+0000","last_clean":"2024-09-10T21:24:53.175105+0000","last_became_active":"2024-09-10T21:24:46.854930+0000","last_became_peered":"2024-09-10T21:24:46.854930+0000","last_unstale":"2024-09-10T21:24:53.175105+0000","last_undegraded":"2024-09-10T21:24:53.175105+0000","last_fullsized":"2024-09-10T21:24:53.175105+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:46:55.286321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.173528+0000","last_change":"2024-09-10T21:24:52.063112+0000","last_active":"2024-09-10T21:24:53.173528+0000","last_peered":"2024-09-10T21:24:53.173528+0000","last_clean":"2024-09-10T21:24:53.173528+0000","last_became_active":"2024-09-10T21:24:52.062395+0000","last_became_peered":"2024-09-10T21:24:52.062395+0000","last_unstale":"2024-09-10T21:24:53.173528+0000","last_undegraded":"2024-09-10T21:24:53.173528+0000","last_fullsized":"2024-09-10T21:24:53.173528+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:21:53.080013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.175254+0000","last_change":"2024-09-10T21:24:47.858593+0000","last_active":"2024-09-10T21:24:53.175254+0000","last_peered":"2024-09-10T21:24:53.175254+0000","last_clean":"2024-09-10T21:24:53.175254+0000","last_became_active":"2024-09-10T21:24:47.858216+0000","last_became_peered":"2024-09-10T21:24:47.858216+0000","last_unstale":"2024-09-10T21:24:53.175254+0000","last_undegraded":"2024-09-10T21:24:53.175254+0000","last_fullsized":"2024-09-10T21:24:53.175254+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:42.961291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"24'1","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.886950+0000","last_change":"2024-09-10T21:24:46.860330+0000","last_active":"2024-09-10T21:24:52.886950+0000","last_peered":"2024-09-10T21:24:52.886950+0000","last_clean":"2024-09-10T21:24:52.886950+0000","last_became_active":"2024-09-10T21:24:46.860053+0000","last_became_peered":"2024-09-10T21:24:46.860053+0000","last_unstale":"2024-09-10T21:24:52.886950+0000","last_undegraded":"2024-09-10T21:24:52.886950+0000","last_fullsized":"2024-09-10T21:24:52.886950+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:58:53.177740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.883701+0000","last_change":"2024-09-10T21:24:51.901673+0000","last_active":"2024-09-10T21:24:52.883701+0000","last_peered":"2024-09-10T21:24:52.883701+0000","last_clean":"2024-09-10T21:24:52.883701+0000","last_became_active":"2024-09-10T21:24:51.901222+0000","last_became_peered":"2024-09-10T21:24:51.901222+0000","last_unstale":"2024-09-10T21:24:52.883701+0000","last_undegraded":"2024-09-10T21:24:52.883701+0000","last_fullsized":"2024-09-10T21:24:52.883701+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:07:52.120299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.173790+0000","last_change":"2024-09-10T21:24:47.870589+0000","last_active":"2024-09-10T21:24:53.173790+0000","last_peered":"2024-09-10T21:24:53.173790+0000","last_clean":"2024-09-10T21:24:53.173790+0000","last_became_active":"2024-09-10T21:24:47.861834+0000","last_became_peered":"2024-09-10T21:24:47.861834+0000","last_unstale":"2024-09-10T21:24:53.173790+0000","last_undegraded":"2024-09-10T21:24:53.173790+0000","last_fullsized":"2024-09-10T21:24:53.173790+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:43.363013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"24'8","reported_seq":31,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.173490+0000","last_change":"2024-09-10T21:24:46.852064+0000","last_active":"2024-09-10T21:24:53.173490+0000","last_peered":"2024-09-10T21:24:53.173490+0000","last_clean":"2024-09-10T21:24:53.173490+0000","last_became_active":"2024-09-10T21:24:46.851776+0000","last_became_peered":"2024-09-10T21:24:46.851776+0000","last_unstale":"2024-09-10T21:24:53.173490+0000","last_undegraded":"2024-09-10T21:24:53.173490+0000","last_fullsized":"2024-09-10T21:24:53.173490+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:26:24.371844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174153+0000","last_change":"2024-09-10T21:24:51.904449+0000","last_active":"2024-09-10T21:24:53.174153+0000","last_peered":"2024-09-10T21:24:53.174153+0000","last_clean":"2024-09-10T21:24:53.174153+0000","last_became_active":"2024-09-10T21:24:51.903533+0000","last_became_peered":"2024-09-10T21:24:51.903533+0000","last_unstale":"2024-09-10T21:24:53.174153+0000","last_undegraded":"2024-09-10T21:24:53.174153+0000","last_fullsized":"2024-09-10T21:24:53.174153+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:11.654935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.883479+0000","last_change":"2024-09-10T21:24:47.857736+0000","last_active":"2024-09-10T21:24:52.883479+0000","last_peered":"2024-09-10T21:24:52.883479+0000","last_clean":"2024-09-10T21:24:52.883479+0000","last_became_active":"2024-09-10T21:24:47.857437+0000","last_became_peered":"2024-09-10T21:24:47.857437+0000","last_unstale":"2024-09-10T21:24:52.883479+0000","last_undegraded":"2024-09-10T21:24:52.883479+0000","last_fullsized":"2024-09-10T21:24:52.883479+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:13:40.032808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.883979+0000","last_change":"2024-09-10T21:24:46.860662+0000","last_active":"2024-09-10T21:24:52.883979+0000","last_peered":"2024-09-10T21:24:52.883979+0000","last_clean":"2024-09-10T21:24:52.883979+0000","last_became_active":"2024-09-10T21:24:46.859796+0000","last_became_peered":"2024-09-10T21:24:46.859796+0000","last_unstale":"2024-09-10T21:24:52.883979+0000","last_undegraded":"2024-09-10T21:24:52.883979+0000","last_fullsized":"2024-09-10T21:24:52.883979+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:48:27.758481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.885543+0000","last_change":"2024-09-10T21:24:52.069770+0000","last_active":"2024-09-10T21:24:52.885543+0000","last_peered":"2024-09-10T21:24:52.885543+0000","last_clean":"2024-09-10T21:24:52.885543+0000","last_became_active":"2024-09-10T21:24:52.067409+0000","last_became_peered":"2024-09-10T21:24:52.067409+0000","last_unstale":"2024-09-10T21:24:52.885543+0000","last_undegraded":"2024-09-10T21:24:52.885543+0000","last_fullsized":"2024-09-10T21:24:52.885543+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:23:39.622612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174136+0000","last_change":"2024-09-10T21:24:47.858692+0000","last_active":"2024-09-10T21:24:53.174136+0000","last_peered":"2024-09-10T21:24:53.174136+0000","last_clean":"2024-09-10T21:24:53.174136+0000","last_became_active":"2024-09-10T21:24:47.858343+0000","last_became_peered":"2024-09-10T21:24:47.858343+0000","last_unstale":"2024-09-10T21:24:53.174136+0000","last_undegraded":"2024-09-10T21:24:53.174136+0000","last_fullsized":"2024-09-10T21:24:53.174136+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:04:36.661791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.175219+0000","last_change":"2024-09-10T21:24:46.864592+0000","last_active":"2024-09-10T21:24:53.175219+0000","last_peered":"2024-09-10T21:24:53.175219+0000","last_clean":"2024-09-10T21:24:53.175219+0000","last_became_active":"2024-09-10T21:24:46.858696+0000","last_became_peered":"2024-09-10T21:24:46.858696+0000","last_unstale":"2024-09-10T21:24:53.175219+0000","last_undegraded":"2024-09-10T21:24:53.175219+0000","last_fullsized":"2024-09-10T21:24:53.175219+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:54:19.356867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.885074+0000","last_change":"2024-09-10T21:24:52.061572+0000","last_active":"2024-09-10T21:24:52.885074+0000","last_peered":"2024-09-10T21:24:52.885074+0000","last_clean":"2024-09-10T21:24:52.885074+0000","last_became_active":"2024-09-10T21:24:52.061061+0000","last_became_peered":"2024-09-10T21:24:52.061061+0000","last_unstale":"2024-09-10T21:24:52.885074+0000","last_undegraded":"2024-09-10T21:24:52.885074+0000","last_fullsized":"2024-09-10T21:24:52.885074+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:04:36.399811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174340+0000","last_change":"2024-09-10T21:24:47.857699+0000","last_active":"2024-09-10T21:24:53.174340+0000","last_peered":"2024-09-10T21:24:53.174340+0000","last_clean":"2024-09-10T21:24:53.174340+0000","last_became_active":"2024-09-10T21:24:47.857429+0000","last_became_peered":"2024-09-10T21:24:47.857429+0000","last_unstale":"2024-09-10T21:24:53.174340+0000","last_undegraded":"2024-09-10T21:24:53.174340+0000","last_fullsized":"2024-09-10T21:24:53.174340+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:07:11.880228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"24'2","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174839+0000","last_change":"2024-09-10T21:24:46.859753+0000","last_active":"2024-09-10T21:24:53.174839+0000","last_peered":"2024-09-10T21:24:53.174839+0000","last_clean":"2024-09-10T21:24:53.174839+0000","last_became_active":"2024-09-10T21:24:46.859299+0000","last_became_peered":"2024-09-10T21:24:46.859299+0000","last_unstale":"2024-09-10T21:24:53.174839+0000","last_undegraded":"2024-09-10T21:24:53.174839+0000","last_fullsized":"2024-09-10T21:24:53.174839+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:06:41.965052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174673+0000","last_change":"2024-09-10T21:24:52.064718+0000","last_active":"2024-09-10T21:24:53.174673+0000","last_peered":"2024-09-10T21:24:53.174673+0000","last_clean":"2024-09-10T21:24:53.174673+0000","last_became_active":"2024-09-10T21:24:52.062327+0000","last_became_peered":"2024-09-10T21:24:52.062327+0000","last_unstale":"2024-09-10T21:24:53.174673+0000","last_undegraded":"2024-09-10T21:24:53.174673+0000","last_fullsized":"2024-09-10T21:24:53.174673+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:43:17.173095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.175009+0000","last_change":"2024-09-10T21:24:47.870766+0000","last_active":"2024-09-10T21:24:53.175009+0000","last_peered":"2024-09-10T21:24:53.175009+0000","last_clean":"2024-09-10T21:24:53.175009+0000","last_became_active":"2024-09-10T21:24:47.870276+0000","last_became_peered":"2024-09-10T21:24:47.870276+0000","last_unstale":"2024-09-10T21:24:53.175009+0000","last_undegraded":"2024-09-10T21:24:53.175009+0000","last_fullsized":"2024-09-10T21:24:53.175009+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:54:38.400877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"25'9","reported_seq":31,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:25:05.421191+0000","last_change":"2024-09-10T21:24:46.867790+0000","last_active":"2024-09-10T21:25:05.421191+0000","last_peered":"2024-09-10T21:25:05.421191+0000","last_clean":"2024-09-10T21:25:05.421191+0000","last_became_active":"2024-09-10T21:24:46.866899+0000","last_became_peered":"2024-09-10T21:24:46.866899+0000","last_unstale":"2024-09-10T21:25:05.421191+0000","last_undegraded":"2024-09-10T21:25:05.421191+0000","last_fullsized":"2024-09-10T21:25:05.421191+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:25:04.923730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.878572+0000","last_change":"2024-09-10T21:24:51.899803+0000","last_active":"2024-09-10T21:24:52.878572+0000","last_peered":"2024-09-10T21:24:52.878572+0000","last_clean":"2024-09-10T21:24:52.878572+0000","last_became_active":"2024-09-10T21:24:51.895398+0000","last_became_peered":"2024-09-10T21:24:51.895398+0000","last_unstale":"2024-09-10T21:24:52.878572+0000","last_undegraded":"2024-09-10T21:24:52.878572+0000","last_fullsized":"2024-09-10T21:24:52.878572+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:52:34.689860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.886661+0000","last_change":"2024-09-10T21:24:47.857544+0000","last_active":"2024-09-10T21:24:52.886661+0000","last_peered":"2024-09-10T21:24:52.886661+0000","last_clean":"2024-09-10T21:24:52.886661+0000","last_became_active":"2024-09-10T21:24:47.857284+0000","last_became_peered":"2024-09-10T21:24:47.857284+0000","last_unstale":"2024-09-10T21:24:52.886661+0000","last_undegraded":"2024-09-10T21:24:52.886661+0000","last_fullsized":"2024-09-10T21:24:52.886661+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:10:22.747606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"24'3","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.878942+0000","last_change":"2024-09-10T21:24:46.859633+0000","last_active":"2024-09-10T21:24:52.878942+0000","last_peered":"2024-09-10T21:24:52.878942+0000","last_clean":"2024-09-10T21:24:52.878942+0000","last_became_active":"2024-09-10T21:24:46.859332+0000","last_became_peered":"2024-09-10T21:24:46.859332+0000","last_unstale":"2024-09-10T21:24:52.878942+0000","last_undegraded":"2024-09-10T21:24:52.878942+0000","last_fullsized":"2024-09-10T21:24:52.878942+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:56:46.975225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.886734+0000","last_change":"2024-09-10T21:24:51.899100+0000","last_active":"2024-09-10T21:24:52.886734+0000","last_peered":"2024-09-10T21:24:52.886734+0000","last_clean":"2024-09-10T21:24:52.886734+0000","last_became_active":"2024-09-10T21:24:51.898642+0000","last_became_peered":"2024-09-10T21:24:51.898642+0000","last_unstale":"2024-09-10T21:24:52.886734+0000","last_undegraded":"2024-09-10T21:24:52.886734+0000","last_fullsized":"2024-09-10T21:24:52.886734+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:06.394081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174291+0000","last_change":"2024-09-10T21:24:47.867890+0000","last_active":"2024-09-10T21:24:53.174291+0000","last_peered":"2024-09-10T21:24:53.174291+0000","last_clean":"2024-09-10T21:24:53.174291+0000","last_became_active":"2024-09-10T21:24:47.865016+0000","last_became_peered":"2024-09-10T21:24:47.865016+0000","last_unstale":"2024-09-10T21:24:53.174291+0000","last_undegraded":"2024-09-10T21:24:53.174291+0000","last_fullsized":"2024-09-10T21:24:53.174291+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:53:09.082512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174673+0000","last_change":"2024-09-10T21:24:46.861400+0000","last_active":"2024-09-10T21:24:53.174673+0000","last_peered":"2024-09-10T21:24:53.174673+0000","last_clean":"2024-09-10T21:24:53.174673+0000","last_became_active":"2024-09-10T21:24:46.861136+0000","last_became_peered":"2024-09-10T21:24:46.861136+0000","last_unstale":"2024-09-10T21:24:53.174673+0000","last_undegraded":"2024-09-10T21:24:53.174673+0000","last_fullsized":"2024-09-10T21:24:53.174673+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:35:31.682448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.884671+0000","last_change":"2024-09-10T21:24:51.907850+0000","last_active":"2024-09-10T21:24:52.884671+0000","last_peered":"2024-09-10T21:24:52.884671+0000","last_clean":"2024-09-10T21:24:52.884671+0000","last_became_active":"2024-09-10T21:24:51.907062+0000","last_became_peered":"2024-09-10T21:24:51.907062+0000","last_unstale":"2024-09-10T21:24:52.884671+0000","last_undegraded":"2024-09-10T21:24:52.884671+0000","last_fullsized":"2024-09-10T21:24:52.884671+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:32:52.519627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174521+0000","last_change":"2024-09-10T21:24:47.861800+0000","last_active":"2024-09-10T21:24:53.174521+0000","last_peered":"2024-09-10T21:24:53.174521+0000","last_clean":"2024-09-10T21:24:53.174521+0000","last_became_active":"2024-09-10T21:24:47.861498+0000","last_became_peered":"2024-09-10T21:24:47.861498+0000","last_unstale":"2024-09-10T21:24:53.174521+0000","last_undegraded":"2024-09-10T21:24:53.174521+0000","last_fullsized":"2024-09-10T21:24:53.174521+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:07:21.128033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.885153+0000","last_change":"2024-09-10T21:24:46.871677+0000","last_active":"2024-09-10T21:24:52.885153+0000","last_peered":"2024-09-10T21:24:52.885153+0000","last_clean":"2024-09-10T21:24:52.885153+0000","last_became_active":"2024-09-10T21:24:46.871367+0000","last_became_peered":"2024-09-10T21:24:46.871367+0000","last_unstale":"2024-09-10T21:24:52.885153+0000","last_undegraded":"2024-09-10T21:24:52.885153+0000","last_fullsized":"2024-09-10T21:24:52.885153+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:40:33.632846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.884646+0000","last_change":"2024-09-10T21:24:52.068637+0000","last_active":"2024-09-10T21:24:52.884646+0000","last_peered":"2024-09-10T21:24:52.884646+0000","last_clean":"2024-09-10T21:24:52.884646+0000","last_became_active":"2024-09-10T21:24:52.068348+0000","last_became_peered":"2024-09-10T21:24:52.068348+0000","last_unstale":"2024-09-10T21:24:52.884646+0000","last_undegraded":"2024-09-10T21:24:52.884646+0000","last_fullsized":"2024-09-10T21:24:52.884646+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:43:17.173095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174107+0000","last_change":"2024-09-10T21:24:52.067732+0000","last_active":"2024-09-10T21:24:53.174107+0000","last_peered":"2024-09-10T21:24:53.174107+0000","last_clean":"2024-09-10T21:24:53.174107+0000","last_became_active":"2024-09-10T21:24:52.067330+0000","last_became_peered":"2024-09-10T21:24:52.067330+0000","last_unstale":"2024-09-10T21:24:53.174107+0000","last_undegraded":"2024-09-10T21:24:53.174107+0000","last_fullsized":"2024-09-10T21:24:53.174107+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:03:41.110214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.878246+0000","last_change":"2024-09-10T21:24:47.862774+0000","last_active":"2024-09-10T21:24:52.878246+0000","last_peered":"2024-09-10T21:24:52.878246+0000","last_clean":"2024-09-10T21:24:52.878246+0000","last_became_active":"2024-09-10T21:24:47.862556+0000","last_became_peered":"2024-09-10T21:24:47.862556+0000","last_unstale":"2024-09-10T21:24:52.878246+0000","last_undegraded":"2024-09-10T21:24:52.878246+0000","last_fullsized":"2024-09-10T21:24:52.878246+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:17:52.343553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.173610+0000","last_change":"2024-09-10T21:24:46.862847+0000","last_active":"2024-09-10T21:24:53.173610+0000","last_peered":"2024-09-10T21:24:53.173610+0000","last_clean":"2024-09-10T21:24:53.173610+0000","last_became_active":"2024-09-10T21:24:46.862540+0000","last_became_peered":"2024-09-10T21:24:46.862540+0000","last_unstale":"2024-09-10T21:24:53.173610+0000","last_undegraded":"2024-09-10T21:24:53.173610+0000","last_fullsized":"2024-09-10T21:24:53.173610+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:52:31.628568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.884678+0000","last_change":"2024-09-10T21:24:51.901224+0000","last_active":"2024-09-10T21:24:52.884678+0000","last_peered":"2024-09-10T21:24:52.884678+0000","last_clean":"2024-09-10T21:24:52.884678+0000","last_became_active":"2024-09-10T21:24:51.900610+0000","last_became_peered":"2024-09-10T21:24:51.900610+0000","last_unstale":"2024-09-10T21:24:52.884678+0000","last_undegraded":"2024-09-10T21:24:52.884678+0000","last_fullsized":"2024-09-10T21:24:52.884678+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:52:34.689860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.884489+0000","last_change":"2024-09-10T21:24:47.861255+0000","last_active":"2024-09-10T21:24:52.884489+0000","last_peered":"2024-09-10T21:24:52.884489+0000","last_clean":"2024-09-10T21:24:52.884489+0000","last_became_active":"2024-09-10T21:24:47.860930+0000","last_became_peered":"2024-09-10T21:24:47.860930+0000","last_unstale":"2024-09-10T21:24:52.884489+0000","last_undegraded":"2024-09-10T21:24:52.884489+0000","last_fullsized":"2024-09-10T21:24:52.884489+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:37:51.784258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.884796+0000","last_change":"2024-09-10T21:24:46.858561+0000","last_active":"2024-09-10T21:24:52.884796+0000","last_peered":"2024-09-10T21:24:52.884796+0000","last_clean":"2024-09-10T21:24:52.884796+0000","last_became_active":"2024-09-10T21:24:46.858265+0000","last_became_peered":"2024-09-10T21:24:46.858265+0000","last_unstale":"2024-09-10T21:24:52.884796+0000","last_undegraded":"2024-09-10T21:24:52.884796+0000","last_fullsized":"2024-09-10T21:24:52.884796+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:49:00.454609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"24'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.884271+0000","last_change":"2024-09-10T21:24:46.856886+0000","last_active":"2024-09-10T21:24:52.884271+0000","last_peered":"2024-09-10T21:24:52.884271+0000","last_clean":"2024-09-10T21:24:52.884271+0000","last_became_active":"2024-09-10T21:24:46.856506+0000","last_became_peered":"2024-09-10T21:24:46.856506+0000","last_unstale":"2024-09-10T21:24:52.884271+0000","last_undegraded":"2024-09-10T21:24:52.884271+0000","last_fullsized":"2024-09-10T21:24:52.884271+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:56:46.975225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174721+0000","last_change":"2024-09-10T21:24:47.848602+0000","last_active":"2024-09-10T21:24:53.174721+0000","last_peered":"2024-09-10T21:24:53.174721+0000","last_clean":"2024-09-10T21:24:53.174721+0000","last_became_active":"2024-09-10T21:24:47.848160+0000","last_became_peered":"2024-09-10T21:24:47.848160+0000","last_unstale":"2024-09-10T21:24:53.174721+0000","last_undegraded":"2024-09-10T21:24:53.174721+0000","last_fullsized":"2024-09-10T21:24:53.174721+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:32:50.027827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174649+0000","last_change":"2024-09-10T21:24:51.903553+0000","last_active":"2024-09-10T21:24:53.174649+0000","last_peered":"2024-09-10T21:24:53.174649+0000","last_clean":"2024-09-10T21:24:53.174649+0000","last_became_active":"2024-09-10T21:24:51.903120+0000","last_became_peered":"2024-09-10T21:24:51.903120+0000","last_unstale":"2024-09-10T21:24:53.174649+0000","last_undegraded":"2024-09-10T21:24:53.174649+0000","last_fullsized":"2024-09-10T21:24:53.174649+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:58.240380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"24'1","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.884865+0000","last_change":"2024-09-10T21:24:46.853550+0000","last_active":"2024-09-10T21:24:52.884865+0000","last_peered":"2024-09-10T21:24:52.884865+0000","last_clean":"2024-09-10T21:24:52.884865+0000","last_became_active":"2024-09-10T21:24:46.852204+0000","last_became_peered":"2024-09-10T21:24:46.852204+0000","last_unstale":"2024-09-10T21:24:52.884865+0000","last_undegraded":"2024-09-10T21:24:52.884865+0000","last_fullsized":"2024-09-10T21:24:52.884865+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:06:37.669444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.884581+0000","last_change":"2024-09-10T21:24:47.849151+0000","last_active":"2024-09-10T21:24:52.884581+0000","last_peered":"2024-09-10T21:24:52.884581+0000","last_clean":"2024-09-10T21:24:52.884581+0000","last_became_active":"2024-09-10T21:24:47.848851+0000","last_became_peered":"2024-09-10T21:24:47.848851+0000","last_unstale":"2024-09-10T21:24:52.884581+0000","last_undegraded":"2024-09-10T21:24:52.884581+0000","last_fullsized":"2024-09-10T21:24:52.884581+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:39:25.828558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.885276+0000","last_change":"2024-09-10T21:24:51.907549+0000","last_active":"2024-09-10T21:24:52.885276+0000","last_peered":"2024-09-10T21:24:52.885276+0000","last_clean":"2024-09-10T21:24:52.885276+0000","last_became_active":"2024-09-10T21:24:51.907082+0000","last_became_peered":"2024-09-10T21:24:51.907082+0000","last_unstale":"2024-09-10T21:24:52.885276+0000","last_undegraded":"2024-09-10T21:24:52.885276+0000","last_fullsized":"2024-09-10T21:24:52.885276+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:44:30.813754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.883360+0000","last_change":"2024-09-10T21:24:47.857804+0000","last_active":"2024-09-10T21:24:52.883360+0000","last_peered":"2024-09-10T21:24:52.883360+0000","last_clean":"2024-09-10T21:24:52.883360+0000","last_became_active":"2024-09-10T21:24:47.857501+0000","last_became_peered":"2024-09-10T21:24:47.857501+0000","last_unstale":"2024-09-10T21:24:52.883360+0000","last_undegraded":"2024-09-10T21:24:52.883360+0000","last_fullsized":"2024-09-10T21:24:52.883360+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:19:20.807422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.173632+0000","last_change":"2024-09-10T21:24:47.867803+0000","last_active":"2024-09-10T21:24:53.173632+0000","last_peered":"2024-09-10T21:24:53.173632+0000","last_clean":"2024-09-10T21:24:53.173632+0000","last_became_active":"2024-09-10T21:24:47.867504+0000","last_became_peered":"2024-09-10T21:24:47.867504+0000","last_unstale":"2024-09-10T21:24:53.173632+0000","last_undegraded":"2024-09-10T21:24:53.173632+0000","last_fullsized":"2024-09-10T21:24:53.173632+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:53:30.420454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174959+0000","last_change":"2024-09-10T21:24:47.868901+0000","last_active":"2024-09-10T21:24:53.174959+0000","last_peered":"2024-09-10T21:24:53.174959+0000","last_clean":"2024-09-10T21:24:53.174959+0000","last_became_active":"2024-09-10T21:24:47.868168+0000","last_became_peered":"2024-09-10T21:24:47.868168+0000","last_unstale":"2024-09-10T21:24:53.174959+0000","last_undegraded":"2024-09-10T21:24:53.174959+0000","last_fullsized":"2024-09-10T21:24:53.174959+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:06:21.461089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174633+0000","last_change":"2024-09-10T21:24:47.864561+0000","last_active":"2024-09-10T21:24:53.174633+0000","last_peered":"2024-09-10T21:24:53.174633+0000","last_clean":"2024-09-10T21:24:53.174633+0000","last_became_active":"2024-09-10T21:24:47.864100+0000","last_became_peered":"2024-09-10T21:24:47.864100+0000","last_unstale":"2024-09-10T21:24:53.174633+0000","last_undegraded":"2024-09-10T21:24:53.174633+0000","last_fullsized":"2024-09-10T21:24:53.174633+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:34:13.053494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.885174+0000","last_change":"2024-09-10T21:24:47.864303+0000","last_active":"2024-09-10T21:24:52.885174+0000","last_peered":"2024-09-10T21:24:52.885174+0000","last_clean":"2024-09-10T21:24:52.885174+0000","last_became_active":"2024-09-10T21:24:47.863970+0000","last_became_peered":"2024-09-10T21:24:47.863970+0000","last_unstale":"2024-09-10T21:24:52.885174+0000","last_undegraded":"2024-09-10T21:24:52.885174+0000","last_fullsized":"2024-09-10T21:24:52.885174+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:46:36.918539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.878793+0000","last_change":"2024-09-10T21:24:47.860734+0000","last_active":"2024-09-10T21:24:52.878793+0000","last_peered":"2024-09-10T21:24:52.878793+0000","last_clean":"2024-09-10T21:24:52.878793+0000","last_became_active":"2024-09-10T21:24:47.860471+0000","last_became_peered":"2024-09-10T21:24:47.860471+0000","last_unstale":"2024-09-10T21:24:52.878793+0000","last_undegraded":"2024-09-10T21:24:52.878793+0000","last_fullsized":"2024-09-10T21:24:52.878793+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:54:13.200082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.885049+0000","last_change":"2024-09-10T21:24:47.872504+0000","last_active":"2024-09-10T21:24:52.885049+0000","last_peered":"2024-09-10T21:24:52.885049+0000","last_clean":"2024-09-10T21:24:52.885049+0000","last_became_active":"2024-09-10T21:24:47.872218+0000","last_became_peered":"2024-09-10T21:24:47.872218+0000","last_unstale":"2024-09-10T21:24:52.885049+0000","last_undegraded":"2024-09-10T21:24:52.885049+0000","last_fullsized":"2024-09-10T21:24:52.885049+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:21:32.400520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.885478+0000","last_change":"2024-09-10T21:24:47.862167+0000","last_active":"2024-09-10T21:24:52.885478+0000","last_peered":"2024-09-10T21:24:52.885478+0000","last_clean":"2024-09-10T21:24:52.885478+0000","last_became_active":"2024-09-10T21:24:47.859649+0000","last_became_peered":"2024-09-10T21:24:47.859649+0000","last_unstale":"2024-09-10T21:24:52.885478+0000","last_undegraded":"2024-09-10T21:24:52.885478+0000","last_fullsized":"2024-09-10T21:24:52.885478+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:42:59.204284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.173544+0000","last_change":"2024-09-10T21:24:47.865322+0000","last_active":"2024-09-10T21:24:53.173544+0000","last_peered":"2024-09-10T21:24:53.173544+0000","last_clean":"2024-09-10T21:24:53.173544+0000","last_became_active":"2024-09-10T21:24:47.864933+0000","last_became_peered":"2024-09-10T21:24:47.864933+0000","last_unstale":"2024-09-10T21:24:53.173544+0000","last_undegraded":"2024-09-10T21:24:53.173544+0000","last_fullsized":"2024-09-10T21:24:53.173544+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:40:45.551723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.884760+0000","last_change":"2024-09-10T21:24:47.860155+0000","last_active":"2024-09-10T21:24:52.884760+0000","last_peered":"2024-09-10T21:24:52.884760+0000","last_clean":"2024-09-10T21:24:52.884760+0000","last_became_active":"2024-09-10T21:24:47.859857+0000","last_became_peered":"2024-09-10T21:24:47.859857+0000","last_unstale":"2024-09-10T21:24:52.884760+0000","last_undegraded":"2024-09-10T21:24:52.884760+0000","last_fullsized":"2024-09-10T21:24:52.884760+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:32:50.027827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174935+0000","last_change":"2024-09-10T21:24:47.872848+0000","last_active":"2024-09-10T21:24:53.174935+0000","last_peered":"2024-09-10T21:24:53.174935+0000","last_clean":"2024-09-10T21:24:53.174935+0000","last_became_active":"2024-09-10T21:24:47.868671+0000","last_became_peered":"2024-09-10T21:24:47.868671+0000","last_unstale":"2024-09-10T21:24:53.174935+0000","last_undegraded":"2024-09-10T21:24:53.174935+0000","last_fullsized":"2024-09-10T21:24:53.174935+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:33:59.291647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174519+0000","last_change":"2024-09-10T21:24:47.857464+0000","last_active":"2024-09-10T21:24:53.174519+0000","last_peered":"2024-09-10T21:24:53.174519+0000","last_clean":"2024-09-10T21:24:53.174519+0000","last_became_active":"2024-09-10T21:24:47.857116+0000","last_became_peered":"2024-09-10T21:24:47.857116+0000","last_unstale":"2024-09-10T21:24:53.174519+0000","last_undegraded":"2024-09-10T21:24:53.174519+0000","last_fullsized":"2024-09-10T21:24:53.174519+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:56:43.269026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.884005+0000","last_change":"2024-09-10T21:24:47.871619+0000","last_active":"2024-09-10T21:24:52.884005+0000","last_peered":"2024-09-10T21:24:52.884005+0000","last_clean":"2024-09-10T21:24:52.884005+0000","last_became_active":"2024-09-10T21:24:47.871350+0000","last_became_peered":"2024-09-10T21:24:47.871350+0000","last_unstale":"2024-09-10T21:24:52.884005+0000","last_undegraded":"2024-09-10T21:24:52.884005+0000","last_fullsized":"2024-09-10T21:24:52.884005+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:41.305116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.175386+0000","last_change":"2024-09-10T21:24:47.861352+0000","last_active":"2024-09-10T21:24:53.175386+0000","last_peered":"2024-09-10T21:24:53.175386+0000","last_clean":"2024-09-10T21:24:53.175386+0000","last_became_active":"2024-09-10T21:24:47.861007+0000","last_became_peered":"2024-09-10T21:24:47.861007+0000","last_unstale":"2024-09-10T21:24:53.175386+0000","last_undegraded":"2024-09-10T21:24:53.175386+0000","last_fullsized":"2024-09-10T21:24:53.175386+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:25.273458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.175077+0000","last_change":"2024-09-10T21:24:47.864205+0000","last_active":"2024-09-10T21:24:53.175077+0000","last_peered":"2024-09-10T21:24:53.175077+0000","last_clean":"2024-09-10T21:24:53.175077+0000","last_became_active":"2024-09-10T21:24:47.863932+0000","last_became_peered":"2024-09-10T21:24:47.863932+0000","last_unstale":"2024-09-10T21:24:53.175077+0000","last_undegraded":"2024-09-10T21:24:53.175077+0000","last_fullsized":"2024-09-10T21:24:53.175077+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:53:17.669629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.175063+0000","last_change":"2024-09-10T21:24:47.866225+0000","last_active":"2024-09-10T21:24:53.175063+0000","last_peered":"2024-09-10T21:24:53.175063+0000","last_clean":"2024-09-10T21:24:53.175063+0000","last_became_active":"2024-09-10T21:24:47.865587+0000","last_became_peered":"2024-09-10T21:24:47.865587+0000","last_unstale":"2024-09-10T21:24:53.175063+0000","last_undegraded":"2024-09-10T21:24:53.175063+0000","last_fullsized":"2024-09-10T21:24:53.175063+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:49:26.113068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.173421+0000","last_change":"2024-09-10T21:24:47.872669+0000","last_active":"2024-09-10T21:24:53.173421+0000","last_peered":"2024-09-10T21:24:53.173421+0000","last_clean":"2024-09-10T21:24:53.173421+0000","last_became_active":"2024-09-10T21:24:47.865837+0000","last_became_peered":"2024-09-10T21:24:47.865837+0000","last_unstale":"2024-09-10T21:24:53.173421+0000","last_undegraded":"2024-09-10T21:24:53.173421+0000","last_fullsized":"2024-09-10T21:24:53.173421+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:53:09.082512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.878332+0000","last_change":"2024-09-10T21:24:47.846567+0000","last_active":"2024-09-10T21:24:52.878332+0000","last_peered":"2024-09-10T21:24:52.878332+0000","last_clean":"2024-09-10T21:24:52.878332+0000","last_became_active":"2024-09-10T21:24:47.846283+0000","last_became_peered":"2024-09-10T21:24:47.846283+0000","last_unstale":"2024-09-10T21:24:52.878332+0000","last_undegraded":"2024-09-10T21:24:52.878332+0000","last_fullsized":"2024-09-10T21:24:52.878332+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:42:15.320518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174247+0000","last_change":"2024-09-10T21:24:47.844888+0000","last_active":"2024-09-10T21:24:53.174247+0000","last_peered":"2024-09-10T21:24:53.174247+0000","last_clean":"2024-09-10T21:24:53.174247+0000","last_became_active":"2024-09-10T21:24:47.844517+0000","last_became_peered":"2024-09-10T21:24:47.844517+0000","last_unstale":"2024-09-10T21:24:53.174247+0000","last_undegraded":"2024-09-10T21:24:53.174247+0000","last_fullsized":"2024-09-10T21:24:53.174247+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:00:14.988282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.884642+0000","last_change":"2024-09-10T21:24:47.862746+0000","last_active":"2024-09-10T21:24:52.884642+0000","last_peered":"2024-09-10T21:24:52.884642+0000","last_clean":"2024-09-10T21:24:52.884642+0000","last_became_active":"2024-09-10T21:24:47.862185+0000","last_became_peered":"2024-09-10T21:24:47.862185+0000","last_unstale":"2024-09-10T21:24:52.884642+0000","last_undegraded":"2024-09-10T21:24:52.884642+0000","last_fullsized":"2024-09-10T21:24:52.884642+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:06:21.461089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.886651+0000","last_change":"2024-09-10T21:24:47.867727+0000","last_active":"2024-09-10T21:24:52.886651+0000","last_peered":"2024-09-10T21:24:52.886651+0000","last_clean":"2024-09-10T21:24:52.886651+0000","last_became_active":"2024-09-10T21:24:47.861045+0000","last_became_peered":"2024-09-10T21:24:47.861045+0000","last_unstale":"2024-09-10T21:24:52.886651+0000","last_undegraded":"2024-09-10T21:24:52.886651+0000","last_fullsized":"2024-09-10T21:24:52.886651+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:08:30.065968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.884547+0000","last_change":"2024-09-10T21:24:47.867410+0000","last_active":"2024-09-10T21:24:52.884547+0000","last_peered":"2024-09-10T21:24:52.884547+0000","last_clean":"2024-09-10T21:24:52.884547+0000","last_became_active":"2024-09-10T21:24:47.867113+0000","last_became_peered":"2024-09-10T21:24:47.867113+0000","last_unstale":"2024-09-10T21:24:52.884547+0000","last_undegraded":"2024-09-10T21:24:52.884547+0000","last_fullsized":"2024-09-10T21:24:52.884547+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:49:53.502472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.884956+0000","last_change":"2024-09-10T21:24:47.867473+0000","last_active":"2024-09-10T21:24:52.884956+0000","last_peered":"2024-09-10T21:24:52.884956+0000","last_clean":"2024-09-10T21:24:52.884956+0000","last_became_active":"2024-09-10T21:24:47.867126+0000","last_became_peered":"2024-09-10T21:24:47.867126+0000","last_unstale":"2024-09-10T21:24:52.884956+0000","last_undegraded":"2024-09-10T21:24:52.884956+0000","last_fullsized":"2024-09-10T21:24:52.884956+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:54:32.307507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.175341+0000","last_change":"2024-09-10T21:24:47.865450+0000","last_active":"2024-09-10T21:24:53.175341+0000","last_peered":"2024-09-10T21:24:53.175341+0000","last_clean":"2024-09-10T21:24:53.175341+0000","last_became_active":"2024-09-10T21:24:47.865093+0000","last_became_peered":"2024-09-10T21:24:47.865093+0000","last_unstale":"2024-09-10T21:24:53.175341+0000","last_undegraded":"2024-09-10T21:24:53.175341+0000","last_fullsized":"2024-09-10T21:24:53.175341+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:12:18.854783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.884322+0000","last_change":"2024-09-10T21:24:47.868093+0000","last_active":"2024-09-10T21:24:52.884322+0000","last_peered":"2024-09-10T21:24:52.884322+0000","last_clean":"2024-09-10T21:24:52.884322+0000","last_became_active":"2024-09-10T21:24:47.867782+0000","last_became_peered":"2024-09-10T21:24:47.867782+0000","last_unstale":"2024-09-10T21:24:52.884322+0000","last_undegraded":"2024-09-10T21:24:52.884322+0000","last_fullsized":"2024-09-10T21:24:52.884322+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:34:13.053494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.173787+0000","last_change":"2024-09-10T21:24:47.862204+0000","last_active":"2024-09-10T21:24:53.173787+0000","last_peered":"2024-09-10T21:24:53.173787+0000","last_clean":"2024-09-10T21:24:53.173787+0000","last_became_active":"2024-09-10T21:24:47.861434+0000","last_became_peered":"2024-09-10T21:24:47.861434+0000","last_unstale":"2024-09-10T21:24:53.173787+0000","last_undegraded":"2024-09-10T21:24:53.173787+0000","last_fullsized":"2024-09-10T21:24:53.173787+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:50:03.245285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174760+0000","last_change":"2024-09-10T21:24:47.869248+0000","last_active":"2024-09-10T21:24:53.174760+0000","last_peered":"2024-09-10T21:24:53.174760+0000","last_clean":"2024-09-10T21:24:53.174760+0000","last_became_active":"2024-09-10T21:24:47.868503+0000","last_became_peered":"2024-09-10T21:24:47.868503+0000","last_unstale":"2024-09-10T21:24:53.174760+0000","last_undegraded":"2024-09-10T21:24:53.174760+0000","last_fullsized":"2024-09-10T21:24:53.174760+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:51:31.417845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174007+0000","last_change":"2024-09-10T21:24:47.864802+0000","last_active":"2024-09-10T21:24:53.174007+0000","last_peered":"2024-09-10T21:24:53.174007+0000","last_clean":"2024-09-10T21:24:53.174007+0000","last_became_active":"2024-09-10T21:24:47.863976+0000","last_became_peered":"2024-09-10T21:24:47.863976+0000","last_unstale":"2024-09-10T21:24:53.174007+0000","last_undegraded":"2024-09-10T21:24:53.174007+0000","last_fullsized":"2024-09-10T21:24:53.174007+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:55:08.544075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174437+0000","last_change":"2024-09-10T21:24:47.873364+0000","last_active":"2024-09-10T21:24:53.174437+0000","last_peered":"2024-09-10T21:24:53.174437+0000","last_clean":"2024-09-10T21:24:53.174437+0000","last_became_active":"2024-09-10T21:24:47.873069+0000","last_became_peered":"2024-09-10T21:24:47.873069+0000","last_unstale":"2024-09-10T21:24:53.174437+0000","last_undegraded":"2024-09-10T21:24:53.174437+0000","last_fullsized":"2024-09-10T21:24:53.174437+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:38:21.368499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174196+0000","last_change":"2024-09-10T21:24:47.856936+0000","last_active":"2024-09-10T21:24:53.174196+0000","last_peered":"2024-09-10T21:24:53.174196+0000","last_clean":"2024-09-10T21:24:53.174196+0000","last_became_active":"2024-09-10T21:24:47.854713+0000","last_became_peered":"2024-09-10T21:24:47.854713+0000","last_unstale":"2024-09-10T21:24:53.174196+0000","last_undegraded":"2024-09-10T21:24:53.174196+0000","last_fullsized":"2024-09-10T21:24:53.174196+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:46:36.918539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.878924+0000","last_change":"2024-09-10T21:24:47.846492+0000","last_active":"2024-09-10T21:24:52.878924+0000","last_peered":"2024-09-10T21:24:52.878924+0000","last_clean":"2024-09-10T21:24:52.878924+0000","last_became_active":"2024-09-10T21:24:47.846227+0000","last_became_peered":"2024-09-10T21:24:47.846227+0000","last_unstale":"2024-09-10T21:24:52.878924+0000","last_undegraded":"2024-09-10T21:24:52.878924+0000","last_fullsized":"2024-09-10T21:24:52.878924+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:13:58.432819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.175219+0000","last_change":"2024-09-10T21:24:47.871469+0000","last_active":"2024-09-10T21:24:53.175219+0000","last_peered":"2024-09-10T21:24:53.175219+0000","last_clean":"2024-09-10T21:24:53.175219+0000","last_became_active":"2024-09-10T21:24:47.871177+0000","last_became_peered":"2024-09-10T21:24:47.871177+0000","last_unstale":"2024-09-10T21:24:53.175219+0000","last_undegraded":"2024-09-10T21:24:53.175219+0000","last_fullsized":"2024-09-10T21:24:53.175219+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:19:24.226916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":66932,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509449,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":320,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":327680,"data_stored":693447,"data_compressed":33466,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509450,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":270336,"data_stored":106339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509449,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":258048,"data_stored":102497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509450,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31456,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340384,"statfs":{"total":96636764160,"available":96604553216,"internally_reserved":0,"allocated":266240,"data_stored":102657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542154,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27296,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":270336,"data_stored":102623,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542153,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":270336,"data_stored":106339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542156,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27364,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344476,"statfs":{"total":96636764160,"available":96608743424,"internally_reserved":0,"allocated":335872,"data_stored":693503,"data_compressed":33466,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574857,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":262144,"data_stored":102587,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33466,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33466,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T21:25:17.599 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T21:25:17.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:17.778+0000 7fa4ea800640 1 -- 172.21.15.17:0/1203774233 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa4e40c89d0 msgr2=0x7fa4e40ca430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:17.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:17.778+0000 7fa4ea800640 1 --2- 172.21.15.17:0/1203774233 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa4e40c89d0 0x7fa4e40ca430 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fa4dc009bf0 tx=0x7fa4dc01cbb0 comp rx=0 tx=0).stop 2024-09-10T21:25:17.784 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:17.778+0000 7fa4ea800640 1 -- 172.21.15.17:0/1203774233 shutdown_connections 2024-09-10T21:25:17.784 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:17.778+0000 7fa4ea800640 1 --2- 172.21.15.17:0/1203774233 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa4e40c89d0 0x7fa4e40ca430 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:17.784 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:17.778+0000 7fa4ea800640 1 -- 172.21.15.17:0/1203774233 >> 172.21.15.17:0/1203774233 conn(0x7fa4e40702f0 msgr2=0x7fa4e4070700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:17.784 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:17.778+0000 7fa4ea800640 1 -- 172.21.15.17:0/1203774233 shutdown_connections 2024-09-10T21:25:17.784 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:17.778+0000 7fa4ea800640 1 -- 172.21.15.17:0/1203774233 wait complete. 2024-09-10T21:25:17.784 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:17.782+0000 7fa4ea800640 1 Processor -- start 2024-09-10T21:25:17.785 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:17.782+0000 7fa4ea800640 1 -- start start 2024-09-10T21:25:17.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:17.782+0000 7fa4ea800640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa4e40c89d0 0x7fa4e41f3210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:17.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:17.782+0000 7fa4ea800640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa4e41546c0 0x7fa4e4068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:17.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:17.782+0000 7fa4ea800640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa4e4154aa0 0x7fa4e4156a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:17.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:17.782+0000 7fa4ea800640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fa4e40ce630 con 0x7fa4e40c89d0 2024-09-10T21:25:17.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:17.782+0000 7fa4ea800640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fa4e4072790 con 0x7fa4e41546c0 2024-09-10T21:25:17.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:17.782+0000 7fa4ea800640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7fa4e40ce7b0 con 0x7fa4e4154aa0 2024-09-10T21:25:17.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:17.782+0000 7fa4e3400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa4e40c89d0 0x7fa4e41f3210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:17.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:17.782+0000 7fa4e3400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa4e40c89d0 0x7fa4e41f3210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44122/0 (socket says 172.21.15.17:44122) 2024-09-10T21:25:17.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:17.782+0000 7fa4e3400640 1 -- 172.21.15.17:0/4100035985 learned_addr learned my addr 172.21.15.17:0/4100035985 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:25:17.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:17.782+0000 7fa4e3400640 1 -- 172.21.15.17:0/4100035985 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa4e4154aa0 msgr2=0x7fa4e4156a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:17.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:17.782+0000 7fa4e3400640 1 --2- 172.21.15.17:0/4100035985 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa4e4154aa0 0x7fa4e4156a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:17.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:17.782+0000 7fa4e3400640 1 -- 172.21.15.17:0/4100035985 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa4e41546c0 msgr2=0x7fa4e4068bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:17.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:17.782+0000 7fa4e3400640 1 --2- 172.21.15.17:0/4100035985 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa4e41546c0 0x7fa4e4068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:17.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:17.782+0000 7fa4e3400640 1 -- 172.21.15.17:0/4100035985 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4e4157780 con 0x7fa4e40c89d0 2024-09-10T21:25:17.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:17.782+0000 7fa4e3400640 1 --2- 172.21.15.17:0/4100035985 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa4e40c89d0 0x7fa4e41f3210 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fa4d400c500 tx=0x7fa4d400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:17.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:17.782+0000 7fa4db400640 1 -- 172.21.15.17:0/4100035985 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4d4017070 con 0x7fa4e40c89d0 2024-09-10T21:25:17.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:17.782+0000 7fa4db400640 1 -- 172.21.15.17:0/4100035985 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa4d4003600 con 0x7fa4e40c89d0 2024-09-10T21:25:17.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:17.782+0000 7fa4db400640 1 -- 172.21.15.17:0/4100035985 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4d4003a90 con 0x7fa4e40c89d0 2024-09-10T21:25:17.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:17.782+0000 7fa4ea800640 1 -- 172.21.15.17:0/4100035985 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4e4157a20 con 0x7fa4e40c89d0 2024-09-10T21:25:17.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:17.782+0000 7fa4ea800640 1 -- 172.21.15.17:0/4100035985 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4e4155f90 con 0x7fa4e40c89d0 2024-09-10T21:25:17.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:17.782+0000 7fa4ea800640 1 -- 172.21.15.17:0/4100035985 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4e40c9720 con 0x7fa4e40c89d0 2024-09-10T21:25:17.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:17.786+0000 7fa4db400640 1 -- 172.21.15.17:0/4100035985 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fa4d4003c30 con 0x7fa4e40c89d0 2024-09-10T21:25:17.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:17.786+0000 7fa4db400640 1 --2- 172.21.15.17:0/4100035985 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fa4c0072610 0x7fa4c0074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:17.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:17.786+0000 7fa4db400640 1 -- 172.21.15.17:0/4100035985 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fa4d4092d10 con 0x7fa4e40c89d0 2024-09-10T21:25:17.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:17.786+0000 7fa4dbe00640 1 --2- 172.21.15.17:0/4100035985 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fa4c0072610 0x7fa4c0074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:17.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:17.786+0000 7fa4dbe00640 1 --2- 172.21.15.17:0/4100035985 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fa4c0072610 0x7fa4c0074ad0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fa4cc0046c0 tx=0x7fa4cc007910 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:17.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:17.790+0000 7fa4db400640 1 -- 172.21.15.17:0/4100035985 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fa4d4098020 con 0x7fa4e40c89d0 2024-09-10T21:25:17.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:17.962+0000 7fa4ea800640 1 -- 172.21.15.17:0/4100035985 --> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa4e40694f0 con 0x7fa4c0072610 2024-09-10T21:25:17.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:17.966+0000 7fa4db400640 1 -- 172.21.15.17:0/4100035985 <== mgr.4100 v2:172.21.15.31:6808/3101308586 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332600 (secure 0 0 0) 0x7fa4e40694f0 con 0x7fa4c0072610 2024-09-10T21:25:17.975 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-10T21:25:17.977 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2024-09-10T21:25:17.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:17.970+0000 7fa4ea800640 1 -- 172.21.15.17:0/4100035985 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fa4c0072610 msgr2=0x7fa4c0074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:17.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:17.970+0000 7fa4ea800640 1 --2- 172.21.15.17:0/4100035985 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fa4c0072610 0x7fa4c0074ad0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fa4cc0046c0 tx=0x7fa4cc007910 comp rx=0 tx=0).stop 2024-09-10T21:25:17.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:17.970+0000 7fa4ea800640 1 -- 172.21.15.17:0/4100035985 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa4e40c89d0 msgr2=0x7fa4e41f3210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:17.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:17.970+0000 7fa4ea800640 1 --2- 172.21.15.17:0/4100035985 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa4e40c89d0 0x7fa4e41f3210 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fa4d400c500 tx=0x7fa4d400c9c0 comp rx=0 tx=0).stop 2024-09-10T21:25:17.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:17.970+0000 7fa4ea800640 1 -- 172.21.15.17:0/4100035985 shutdown_connections 2024-09-10T21:25:17.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:17.970+0000 7fa4ea800640 1 --2- 172.21.15.17:0/4100035985 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fa4c0072610 0x7fa4c0074ad0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:17.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:17.970+0000 7fa4ea800640 1 --2- 172.21.15.17:0/4100035985 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa4e4154aa0 0x7fa4e4156a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:17.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:17.970+0000 7fa4ea800640 1 --2- 172.21.15.17:0/4100035985 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa4e41546c0 0x7fa4e4068bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:17.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:17.970+0000 7fa4ea800640 1 --2- 172.21.15.17:0/4100035985 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa4e40c89d0 0x7fa4e41f3210 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:17.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:17.970+0000 7fa4ea800640 1 -- 172.21.15.17:0/4100035985 >> 172.21.15.17:0/4100035985 conn(0x7fa4e40702f0 msgr2=0x7fa4e4150150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:17.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:17.970+0000 7fa4ea800640 1 -- 172.21.15.17:0/4100035985 shutdown_connections 2024-09-10T21:25:17.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:17.974+0000 7fa4ea800640 1 -- 172.21.15.17:0/4100035985 wait complete. 2024-09-10T21:25:17.995 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":40,"stamp":"2024-09-10T21:25:17.345280+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":149,"num_write_kb":1646,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":114,"ondisk_log_size":114,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":222608,"kb_used_data":2208,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754752112,"statfs":{"total":773094113280,"available":772866162688,"internally_reserved":0,"allocated":2260992,"data_stored":2009992,"data_compressed":66932,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12752,"internal_metadata":225496624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2230,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003112"},"pg_stats":[{"pgid":"2.1f","version":"24'4","reported_seq":27,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174456+0000","last_change":"2024-09-10T21:24:46.864556+0000","last_active":"2024-09-10T21:24:53.174456+0000","last_peered":"2024-09-10T21:24:53.174456+0000","last_clean":"2024-09-10T21:24:53.174456+0000","last_became_active":"2024-09-10T21:24:46.864288+0000","last_became_peered":"2024-09-10T21:24:46.864288+0000","last_unstale":"2024-09-10T21:24:53.174456+0000","last_undegraded":"2024-09-10T21:24:53.174456+0000","last_fullsized":"2024-09-10T21:24:53.174456+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:00:38.734807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.885137+0000","last_change":"2024-09-10T21:24:47.864452+0000","last_active":"2024-09-10T21:24:52.885137+0000","last_peered":"2024-09-10T21:24:52.885137+0000","last_clean":"2024-09-10T21:24:52.885137+0000","last_became_active":"2024-09-10T21:24:47.864168+0000","last_became_peered":"2024-09-10T21:24:47.864168+0000","last_unstale":"2024-09-10T21:24:52.885137+0000","last_undegraded":"2024-09-10T21:24:52.885137+0000","last_fullsized":"2024-09-10T21:24:52.885137+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:51:31.417845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.884507+0000","last_change":"2024-09-10T21:24:52.070054+0000","last_active":"2024-09-10T21:24:52.884507+0000","last_peered":"2024-09-10T21:24:52.884507+0000","last_clean":"2024-09-10T21:24:52.884507+0000","last_became_active":"2024-09-10T21:24:52.067258+0000","last_became_peered":"2024-09-10T21:24:52.067258+0000","last_unstale":"2024-09-10T21:24:52.884507+0000","last_undegraded":"2024-09-10T21:24:52.884507+0000","last_fullsized":"2024-09-10T21:24:52.884507+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:21:53.080013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"24'1","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.885320+0000","last_change":"2024-09-10T21:24:46.860648+0000","last_active":"2024-09-10T21:24:52.885320+0000","last_peered":"2024-09-10T21:24:52.885320+0000","last_clean":"2024-09-10T21:24:52.885320+0000","last_became_active":"2024-09-10T21:24:46.860341+0000","last_became_peered":"2024-09-10T21:24:46.860341+0000","last_unstale":"2024-09-10T21:24:52.885320+0000","last_undegraded":"2024-09-10T21:24:52.885320+0000","last_fullsized":"2024-09-10T21:24:52.885320+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:39:24.819979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174137+0000","last_change":"2024-09-10T21:24:47.865617+0000","last_active":"2024-09-10T21:24:53.174137+0000","last_peered":"2024-09-10T21:24:53.174137+0000","last_clean":"2024-09-10T21:24:53.174137+0000","last_became_active":"2024-09-10T21:24:47.865365+0000","last_became_peered":"2024-09-10T21:24:47.865365+0000","last_unstale":"2024-09-10T21:24:53.174137+0000","last_undegraded":"2024-09-10T21:24:53.174137+0000","last_fullsized":"2024-09-10T21:24:53.174137+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:10:05.971996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.883672+0000","last_change":"2024-09-10T21:24:51.900077+0000","last_active":"2024-09-10T21:24:52.883672+0000","last_peered":"2024-09-10T21:24:52.883672+0000","last_clean":"2024-09-10T21:24:52.883672+0000","last_became_active":"2024-09-10T21:24:51.899652+0000","last_became_peered":"2024-09-10T21:24:51.899652+0000","last_unstale":"2024-09-10T21:24:52.883672+0000","last_undegraded":"2024-09-10T21:24:52.883672+0000","last_fullsized":"2024-09-10T21:24:52.883672+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:40:49.585857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"24'2","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.175383+0000","last_change":"2024-09-10T21:24:46.861040+0000","last_active":"2024-09-10T21:24:53.175383+0000","last_peered":"2024-09-10T21:24:53.175383+0000","last_clean":"2024-09-10T21:24:53.175383+0000","last_became_active":"2024-09-10T21:24:46.860780+0000","last_became_peered":"2024-09-10T21:24:46.860780+0000","last_unstale":"2024-09-10T21:24:53.175383+0000","last_undegraded":"2024-09-10T21:24:53.175383+0000","last_fullsized":"2024-09-10T21:24:53.175383+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:53:31.454472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174596+0000","last_change":"2024-09-10T21:24:47.877997+0000","last_active":"2024-09-10T21:24:53.174596+0000","last_peered":"2024-09-10T21:24:53.174596+0000","last_clean":"2024-09-10T21:24:53.174596+0000","last_became_active":"2024-09-10T21:24:47.877765+0000","last_became_peered":"2024-09-10T21:24:47.877765+0000","last_unstale":"2024-09-10T21:24:53.174596+0000","last_undegraded":"2024-09-10T21:24:53.174596+0000","last_fullsized":"2024-09-10T21:24:53.174596+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:33:20.590209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.886528+0000","last_change":"2024-09-10T21:24:51.908333+0000","last_active":"2024-09-10T21:24:52.886528+0000","last_peered":"2024-09-10T21:24:52.886528+0000","last_clean":"2024-09-10T21:24:52.886528+0000","last_became_active":"2024-09-10T21:24:51.907933+0000","last_became_peered":"2024-09-10T21:24:51.907933+0000","last_unstale":"2024-09-10T21:24:52.886528+0000","last_undegraded":"2024-09-10T21:24:52.886528+0000","last_fullsized":"2024-09-10T21:24:52.886528+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:07:33.193555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"24'2","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174867+0000","last_change":"2024-09-10T21:24:46.873044+0000","last_active":"2024-09-10T21:24:53.174867+0000","last_peered":"2024-09-10T21:24:53.174867+0000","last_clean":"2024-09-10T21:24:53.174867+0000","last_became_active":"2024-09-10T21:24:46.872075+0000","last_became_peered":"2024-09-10T21:24:46.872075+0000","last_unstale":"2024-09-10T21:24:53.174867+0000","last_undegraded":"2024-09-10T21:24:53.174867+0000","last_fullsized":"2024-09-10T21:24:53.174867+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:40:33.632846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174402+0000","last_change":"2024-09-10T21:24:47.878094+0000","last_active":"2024-09-10T21:24:53.174402+0000","last_peered":"2024-09-10T21:24:53.174402+0000","last_clean":"2024-09-10T21:24:53.174402+0000","last_became_active":"2024-09-10T21:24:47.876688+0000","last_became_peered":"2024-09-10T21:24:47.876688+0000","last_unstale":"2024-09-10T21:24:53.174402+0000","last_undegraded":"2024-09-10T21:24:53.174402+0000","last_fullsized":"2024-09-10T21:24:53.174402+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:03:37.076080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.884328+0000","last_change":"2024-09-10T21:24:51.900208+0000","last_active":"2024-09-10T21:24:52.884328+0000","last_peered":"2024-09-10T21:24:52.884328+0000","last_clean":"2024-09-10T21:24:52.884328+0000","last_became_active":"2024-09-10T21:24:51.899576+0000","last_became_peered":"2024-09-10T21:24:51.899576+0000","last_unstale":"2024-09-10T21:24:52.884328+0000","last_undegraded":"2024-09-10T21:24:52.884328+0000","last_fullsized":"2024-09-10T21:24:52.884328+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:52:49.479932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.883746+0000","last_change":"2024-09-10T21:24:47.869830+0000","last_active":"2024-09-10T21:24:52.883746+0000","last_peered":"2024-09-10T21:24:52.883746+0000","last_clean":"2024-09-10T21:24:52.883746+0000","last_became_active":"2024-09-10T21:24:47.869569+0000","last_became_peered":"2024-09-10T21:24:47.869569+0000","last_unstale":"2024-09-10T21:24:52.883746+0000","last_undegraded":"2024-09-10T21:24:52.883746+0000","last_fullsized":"2024-09-10T21:24:52.883746+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:42:15.320518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"24'2","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174248+0000","last_change":"2024-09-10T21:24:46.867297+0000","last_active":"2024-09-10T21:24:53.174248+0000","last_peered":"2024-09-10T21:24:53.174248+0000","last_clean":"2024-09-10T21:24:53.174248+0000","last_became_active":"2024-09-10T21:24:46.866893+0000","last_became_peered":"2024-09-10T21:24:46.866893+0000","last_unstale":"2024-09-10T21:24:53.174248+0000","last_undegraded":"2024-09-10T21:24:53.174248+0000","last_fullsized":"2024-09-10T21:24:53.174248+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:53.197666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.885044+0000","last_change":"2024-09-10T21:24:51.908412+0000","last_active":"2024-09-10T21:24:52.885044+0000","last_peered":"2024-09-10T21:24:52.885044+0000","last_clean":"2024-09-10T21:24:52.885044+0000","last_became_active":"2024-09-10T21:24:51.907923+0000","last_became_peered":"2024-09-10T21:24:51.907923+0000","last_unstale":"2024-09-10T21:24:52.885044+0000","last_undegraded":"2024-09-10T21:24:52.885044+0000","last_fullsized":"2024-09-10T21:24:52.885044+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:26:46.479501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.884755+0000","last_change":"2024-09-10T21:24:47.862479+0000","last_active":"2024-09-10T21:24:52.884755+0000","last_peered":"2024-09-10T21:24:52.884755+0000","last_clean":"2024-09-10T21:24:52.884755+0000","last_became_active":"2024-09-10T21:24:47.861842+0000","last_became_peered":"2024-09-10T21:24:47.861842+0000","last_unstale":"2024-09-10T21:24:52.884755+0000","last_undegraded":"2024-09-10T21:24:52.884755+0000","last_fullsized":"2024-09-10T21:24:52.884755+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:53:09.082512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.878911+0000","last_change":"2024-09-10T21:24:46.852467+0000","last_active":"2024-09-10T21:24:52.878911+0000","last_peered":"2024-09-10T21:24:52.878911+0000","last_clean":"2024-09-10T21:24:52.878911+0000","last_became_active":"2024-09-10T21:24:46.851027+0000","last_became_peered":"2024-09-10T21:24:46.851027+0000","last_unstale":"2024-09-10T21:24:52.878911+0000","last_undegraded":"2024-09-10T21:24:52.878911+0000","last_fullsized":"2024-09-10T21:24:52.878911+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:39:27.441714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.886564+0000","last_change":"2024-09-10T21:24:51.903577+0000","last_active":"2024-09-10T21:24:52.886564+0000","last_peered":"2024-09-10T21:24:52.886564+0000","last_clean":"2024-09-10T21:24:52.886564+0000","last_became_active":"2024-09-10T21:24:51.903031+0000","last_became_peered":"2024-09-10T21:24:51.903031+0000","last_unstale":"2024-09-10T21:24:52.886564+0000","last_undegraded":"2024-09-10T21:24:52.886564+0000","last_fullsized":"2024-09-10T21:24:52.886564+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:08:52.787720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.884612+0000","last_change":"2024-09-10T21:24:47.856334+0000","last_active":"2024-09-10T21:24:52.884612+0000","last_peered":"2024-09-10T21:24:52.884612+0000","last_clean":"2024-09-10T21:24:52.884612+0000","last_became_active":"2024-09-10T21:24:47.856003+0000","last_became_peered":"2024-09-10T21:24:47.856003+0000","last_unstale":"2024-09-10T21:24:52.884612+0000","last_undegraded":"2024-09-10T21:24:52.884612+0000","last_fullsized":"2024-09-10T21:24:52.884612+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:58:53.698584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.878371+0000","last_change":"2024-09-10T21:24:46.853373+0000","last_active":"2024-09-10T21:24:52.878371+0000","last_peered":"2024-09-10T21:24:52.878371+0000","last_clean":"2024-09-10T21:24:52.878371+0000","last_became_active":"2024-09-10T21:24:46.852993+0000","last_became_peered":"2024-09-10T21:24:46.852993+0000","last_unstale":"2024-09-10T21:24:52.878371+0000","last_undegraded":"2024-09-10T21:24:52.878371+0000","last_fullsized":"2024-09-10T21:24:52.878371+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:53:31.454472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.879271+0000","last_change":"2024-09-10T21:24:52.065246+0000","last_active":"2024-09-10T21:24:52.879271+0000","last_peered":"2024-09-10T21:24:52.879271+0000","last_clean":"2024-09-10T21:24:52.879271+0000","last_became_active":"2024-09-10T21:24:52.064884+0000","last_became_peered":"2024-09-10T21:24:52.064884+0000","last_unstale":"2024-09-10T21:24:52.879271+0000","last_undegraded":"2024-09-10T21:24:52.879271+0000","last_fullsized":"2024-09-10T21:24:52.879271+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:54:54.393305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.884318+0000","last_change":"2024-09-10T21:24:47.856517+0000","last_active":"2024-09-10T21:24:52.884318+0000","last_peered":"2024-09-10T21:24:52.884318+0000","last_clean":"2024-09-10T21:24:52.884318+0000","last_became_active":"2024-09-10T21:24:47.856185+0000","last_became_peered":"2024-09-10T21:24:47.856185+0000","last_unstale":"2024-09-10T21:24:52.884318+0000","last_undegraded":"2024-09-10T21:24:52.884318+0000","last_fullsized":"2024-09-10T21:24:52.884318+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:26:00.852818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.173635+0000","last_change":"2024-09-10T21:24:46.867888+0000","last_active":"2024-09-10T21:24:53.173635+0000","last_peered":"2024-09-10T21:24:53.173635+0000","last_clean":"2024-09-10T21:24:53.173635+0000","last_became_active":"2024-09-10T21:24:46.866477+0000","last_became_peered":"2024-09-10T21:24:46.866477+0000","last_unstale":"2024-09-10T21:24:53.173635+0000","last_undegraded":"2024-09-10T21:24:53.173635+0000","last_fullsized":"2024-09-10T21:24:53.173635+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:39:27.441714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.885185+0000","last_change":"2024-09-10T21:24:51.908376+0000","last_active":"2024-09-10T21:24:52.885185+0000","last_peered":"2024-09-10T21:24:52.885185+0000","last_clean":"2024-09-10T21:24:52.885185+0000","last_became_active":"2024-09-10T21:24:51.907878+0000","last_became_peered":"2024-09-10T21:24:51.907878+0000","last_unstale":"2024-09-10T21:24:52.885185+0000","last_undegraded":"2024-09-10T21:24:52.885185+0000","last_fullsized":"2024-09-10T21:24:52.885185+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:39:02.492952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174342+0000","last_change":"2024-09-10T21:24:47.844823+0000","last_active":"2024-09-10T21:24:53.174342+0000","last_peered":"2024-09-10T21:24:53.174342+0000","last_clean":"2024-09-10T21:24:53.174342+0000","last_became_active":"2024-09-10T21:24:47.844505+0000","last_became_peered":"2024-09-10T21:24:47.844505+0000","last_unstale":"2024-09-10T21:24:53.174342+0000","last_undegraded":"2024-09-10T21:24:53.174342+0000","last_fullsized":"2024-09-10T21:24:53.174342+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:37:51.784258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"24'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.885358+0000","last_change":"2024-09-10T21:24:46.862172+0000","last_active":"2024-09-10T21:24:52.885358+0000","last_peered":"2024-09-10T21:24:52.885358+0000","last_clean":"2024-09-10T21:24:52.885358+0000","last_became_active":"2024-09-10T21:24:46.861473+0000","last_became_peered":"2024-09-10T21:24:46.861473+0000","last_unstale":"2024-09-10T21:24:52.885358+0000","last_undegraded":"2024-09-10T21:24:52.885358+0000","last_fullsized":"2024-09-10T21:24:52.885358+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:53:31.454472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.883945+0000","last_change":"2024-09-10T21:24:52.062841+0000","last_active":"2024-09-10T21:24:52.883945+0000","last_peered":"2024-09-10T21:24:52.883945+0000","last_clean":"2024-09-10T21:24:52.883945+0000","last_became_active":"2024-09-10T21:24:52.061581+0000","last_became_peered":"2024-09-10T21:24:52.061581+0000","last_unstale":"2024-09-10T21:24:52.883945+0000","last_undegraded":"2024-09-10T21:24:52.883945+0000","last_fullsized":"2024-09-10T21:24:52.883945+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:34:03.325781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.885543+0000","last_change":"2024-09-10T21:24:47.859801+0000","last_active":"2024-09-10T21:24:52.885543+0000","last_peered":"2024-09-10T21:24:52.885543+0000","last_clean":"2024-09-10T21:24:52.885543+0000","last_became_active":"2024-09-10T21:24:47.859483+0000","last_became_peered":"2024-09-10T21:24:47.859483+0000","last_unstale":"2024-09-10T21:24:52.885543+0000","last_undegraded":"2024-09-10T21:24:52.885543+0000","last_fullsized":"2024-09-10T21:24:52.885543+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:38:21.368499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"25'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.879175+0000","last_change":"2024-09-10T21:24:46.860624+0000","last_active":"2024-09-10T21:24:52.879175+0000","last_peered":"2024-09-10T21:24:52.879175+0000","last_clean":"2024-09-10T21:24:52.879175+0000","last_became_active":"2024-09-10T21:24:46.860309+0000","last_became_peered":"2024-09-10T21:24:46.860309+0000","last_unstale":"2024-09-10T21:24:52.879175+0000","last_undegraded":"2024-09-10T21:24:52.879175+0000","last_fullsized":"2024-09-10T21:24:52.879175+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:37:12.332013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.884414+0000","last_change":"2024-09-10T21:24:52.068581+0000","last_active":"2024-09-10T21:24:52.884414+0000","last_peered":"2024-09-10T21:24:52.884414+0000","last_clean":"2024-09-10T21:24:52.884414+0000","last_became_active":"2024-09-10T21:24:52.068250+0000","last_became_peered":"2024-09-10T21:24:52.068250+0000","last_unstale":"2024-09-10T21:24:52.884414+0000","last_undegraded":"2024-09-10T21:24:52.884414+0000","last_fullsized":"2024-09-10T21:24:52.884414+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:52:01.990009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.879000+0000","last_change":"2024-09-10T21:24:47.854042+0000","last_active":"2024-09-10T21:24:52.879000+0000","last_peered":"2024-09-10T21:24:52.879000+0000","last_clean":"2024-09-10T21:24:52.879000+0000","last_became_active":"2024-09-10T21:24:47.853750+0000","last_became_peered":"2024-09-10T21:24:47.853750+0000","last_unstale":"2024-09-10T21:24:52.879000+0000","last_undegraded":"2024-09-10T21:24:52.879000+0000","last_fullsized":"2024-09-10T21:24:52.879000+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:06:50.775132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.175510+0000","last_change":"2024-09-10T21:24:46.864617+0000","last_active":"2024-09-10T21:24:53.175510+0000","last_peered":"2024-09-10T21:24:53.175510+0000","last_clean":"2024-09-10T21:24:53.175510+0000","last_became_active":"2024-09-10T21:24:46.864149+0000","last_became_peered":"2024-09-10T21:24:46.864149+0000","last_unstale":"2024-09-10T21:24:53.175510+0000","last_undegraded":"2024-09-10T21:24:53.175510+0000","last_fullsized":"2024-09-10T21:24:53.175510+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:24:15.673107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174530+0000","last_change":"2024-09-10T21:24:52.061485+0000","last_active":"2024-09-10T21:24:53.174530+0000","last_peered":"2024-09-10T21:24:53.174530+0000","last_clean":"2024-09-10T21:24:53.174530+0000","last_became_active":"2024-09-10T21:24:52.060953+0000","last_became_peered":"2024-09-10T21:24:52.060953+0000","last_unstale":"2024-09-10T21:24:53.174530+0000","last_undegraded":"2024-09-10T21:24:53.174530+0000","last_fullsized":"2024-09-10T21:24:53.174530+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:49:30.147202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174576+0000","last_change":"2024-09-10T21:24:47.867241+0000","last_active":"2024-09-10T21:24:53.174576+0000","last_peered":"2024-09-10T21:24:53.174576+0000","last_clean":"2024-09-10T21:24:53.174576+0000","last_became_active":"2024-09-10T21:24:47.866907+0000","last_became_peered":"2024-09-10T21:24:47.866907+0000","last_unstale":"2024-09-10T21:24:53.174576+0000","last_undegraded":"2024-09-10T21:24:53.174576+0000","last_fullsized":"2024-09-10T21:24:53.174576+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:07.620800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.879175+0000","last_change":"2024-09-10T21:24:46.856848+0000","last_active":"2024-09-10T21:24:52.879175+0000","last_peered":"2024-09-10T21:24:52.879175+0000","last_clean":"2024-09-10T21:24:52.879175+0000","last_became_active":"2024-09-10T21:24:46.856489+0000","last_became_peered":"2024-09-10T21:24:46.856489+0000","last_unstale":"2024-09-10T21:24:52.879175+0000","last_undegraded":"2024-09-10T21:24:52.879175+0000","last_fullsized":"2024-09-10T21:24:52.879175+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:46:55.286321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.878563+0000","last_change":"2024-09-10T21:24:51.901016+0000","last_active":"2024-09-10T21:24:52.878563+0000","last_peered":"2024-09-10T21:24:52.878563+0000","last_clean":"2024-09-10T21:24:52.878563+0000","last_became_active":"2024-09-10T21:24:51.900479+0000","last_became_peered":"2024-09-10T21:24:51.900479+0000","last_unstale":"2024-09-10T21:24:52.878563+0000","last_undegraded":"2024-09-10T21:24:52.878563+0000","last_fullsized":"2024-09-10T21:24:52.878563+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:36.741810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.886797+0000","last_change":"2024-09-10T21:24:47.858753+0000","last_active":"2024-09-10T21:24:52.886797+0000","last_peered":"2024-09-10T21:24:52.886797+0000","last_clean":"2024-09-10T21:24:52.886797+0000","last_became_active":"2024-09-10T21:24:47.858438+0000","last_became_peered":"2024-09-10T21:24:47.858438+0000","last_unstale":"2024-09-10T21:24:52.886797+0000","last_undegraded":"2024-09-10T21:24:52.886797+0000","last_fullsized":"2024-09-10T21:24:52.886797+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:12:09.198013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.175120+0000","last_change":"2024-09-10T21:24:46.863610+0000","last_active":"2024-09-10T21:24:53.175120+0000","last_peered":"2024-09-10T21:24:53.175120+0000","last_clean":"2024-09-10T21:24:53.175120+0000","last_became_active":"2024-09-10T21:24:46.863150+0000","last_became_peered":"2024-09-10T21:24:46.863150+0000","last_unstale":"2024-09-10T21:24:53.175120+0000","last_undegraded":"2024-09-10T21:24:53.175120+0000","last_fullsized":"2024-09-10T21:24:53.175120+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:27:35.473995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.878912+0000","last_change":"2024-09-10T21:24:51.907366+0000","last_active":"2024-09-10T21:24:52.878912+0000","last_peered":"2024-09-10T21:24:52.878912+0000","last_clean":"2024-09-10T21:24:52.878912+0000","last_became_active":"2024-09-10T21:24:51.906888+0000","last_became_peered":"2024-09-10T21:24:51.906888+0000","last_unstale":"2024-09-10T21:24:52.878912+0000","last_undegraded":"2024-09-10T21:24:52.878912+0000","last_fullsized":"2024-09-10T21:24:52.878912+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:24:54.278514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.878445+0000","last_change":"2024-09-10T21:24:47.850155+0000","last_active":"2024-09-10T21:24:52.878445+0000","last_peered":"2024-09-10T21:24:52.878445+0000","last_clean":"2024-09-10T21:24:52.878445+0000","last_became_active":"2024-09-10T21:24:47.849846+0000","last_became_peered":"2024-09-10T21:24:47.849846+0000","last_unstale":"2024-09-10T21:24:52.878445+0000","last_undegraded":"2024-09-10T21:24:52.878445+0000","last_fullsized":"2024-09-10T21:24:52.878445+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:13:40.032808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.173931+0000","last_change":"2024-09-10T21:24:46.851800+0000","last_active":"2024-09-10T21:24:53.173931+0000","last_peered":"2024-09-10T21:24:53.173931+0000","last_clean":"2024-09-10T21:24:53.173931+0000","last_became_active":"2024-09-10T21:24:46.851200+0000","last_became_peered":"2024-09-10T21:24:46.851200+0000","last_unstale":"2024-09-10T21:24:53.173931+0000","last_undegraded":"2024-09-10T21:24:53.173931+0000","last_fullsized":"2024-09-10T21:24:53.173931+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:00:22.839361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.879296+0000","last_change":"2024-09-10T21:24:51.902529+0000","last_active":"2024-09-10T21:24:52.879296+0000","last_peered":"2024-09-10T21:24:52.879296+0000","last_clean":"2024-09-10T21:24:52.879296+0000","last_became_active":"2024-09-10T21:24:51.901141+0000","last_became_peered":"2024-09-10T21:24:51.901141+0000","last_unstale":"2024-09-10T21:24:52.879296+0000","last_undegraded":"2024-09-10T21:24:52.879296+0000","last_fullsized":"2024-09-10T21:24:52.879296+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:08:40.203348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.173878+0000","last_change":"2024-09-10T21:24:47.846582+0000","last_active":"2024-09-10T21:24:53.173878+0000","last_peered":"2024-09-10T21:24:53.173878+0000","last_clean":"2024-09-10T21:24:53.173878+0000","last_became_active":"2024-09-10T21:24:47.846358+0000","last_became_peered":"2024-09-10T21:24:47.846358+0000","last_unstale":"2024-09-10T21:24:53.173878+0000","last_undegraded":"2024-09-10T21:24:53.173878+0000","last_fullsized":"2024-09-10T21:24:53.173878+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:58:53.698584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.173773+0000","last_change":"2024-09-10T21:24:46.870908+0000","last_active":"2024-09-10T21:24:53.173773+0000","last_peered":"2024-09-10T21:24:53.173773+0000","last_clean":"2024-09-10T21:24:53.173773+0000","last_became_active":"2024-09-10T21:24:46.868375+0000","last_became_peered":"2024-09-10T21:24:46.868375+0000","last_unstale":"2024-09-10T21:24:53.173773+0000","last_undegraded":"2024-09-10T21:24:53.173773+0000","last_fullsized":"2024-09-10T21:24:53.173773+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:37:12.332013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174362+0000","last_change":"2024-09-10T21:24:52.069018+0000","last_active":"2024-09-10T21:24:53.174362+0000","last_peered":"2024-09-10T21:24:53.174362+0000","last_clean":"2024-09-10T21:24:53.174362+0000","last_became_active":"2024-09-10T21:24:52.067308+0000","last_became_peered":"2024-09-10T21:24:52.067308+0000","last_unstale":"2024-09-10T21:24:53.174362+0000","last_undegraded":"2024-09-10T21:24:53.174362+0000","last_fullsized":"2024-09-10T21:24:53.174362+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:07:25.162168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.884767+0000","last_change":"2024-09-10T21:24:47.871542+0000","last_active":"2024-09-10T21:24:52.884767+0000","last_peered":"2024-09-10T21:24:52.884767+0000","last_clean":"2024-09-10T21:24:52.884767+0000","last_became_active":"2024-09-10T21:24:47.871128+0000","last_became_peered":"2024-09-10T21:24:47.871128+0000","last_unstale":"2024-09-10T21:24:52.884767+0000","last_undegraded":"2024-09-10T21:24:52.884767+0000","last_fullsized":"2024-09-10T21:24:52.884767+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:51:22.876126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174204+0000","last_change":"2024-09-10T21:24:46.859218+0000","last_active":"2024-09-10T21:24:53.174204+0000","last_peered":"2024-09-10T21:24:53.174204+0000","last_clean":"2024-09-10T21:24:53.174204+0000","last_became_active":"2024-09-10T21:24:46.853190+0000","last_became_peered":"2024-09-10T21:24:46.853190+0000","last_unstale":"2024-09-10T21:24:53.174204+0000","last_undegraded":"2024-09-10T21:24:53.174204+0000","last_fullsized":"2024-09-10T21:24:53.174204+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:26:34.848759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":119,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.173768+0000","last_change":"2024-09-10T21:24:37.216542+0000","last_active":"2024-09-10T21:24:53.173768+0000","last_peered":"2024-09-10T21:24:53.173768+0000","last_clean":"2024-09-10T21:24:53.173768+0000","last_became_active":"2024-09-10T21:24:37.215443+0000","last_became_peered":"2024-09-10T21:24:37.215443+0000","last_unstale":"2024-09-10T21:24:53.173768+0000","last_undegraded":"2024-09-10T21:24:53.173768+0000","last_fullsized":"2024-09-10T21:24:53.173768+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:35.877801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:35.877801+0000","last_clean_scrub_stamp":"2024-09-10T21:24:35.877801+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:14:20.260329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.884231+0000","last_change":"2024-09-10T21:24:52.067645+0000","last_active":"2024-09-10T21:24:52.884231+0000","last_peered":"2024-09-10T21:24:52.884231+0000","last_clean":"2024-09-10T21:24:52.884231+0000","last_became_active":"2024-09-10T21:24:52.067148+0000","last_became_peered":"2024-09-10T21:24:52.067148+0000","last_unstale":"2024-09-10T21:24:52.884231+0000","last_undegraded":"2024-09-10T21:24:52.884231+0000","last_fullsized":"2024-09-10T21:24:52.884231+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:54:42.435011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.175128+0000","last_change":"2024-09-10T21:24:47.867282+0000","last_active":"2024-09-10T21:24:53.175128+0000","last_peered":"2024-09-10T21:24:53.175128+0000","last_clean":"2024-09-10T21:24:53.175128+0000","last_became_active":"2024-09-10T21:24:47.865582+0000","last_became_peered":"2024-09-10T21:24:47.865582+0000","last_unstale":"2024-09-10T21:24:53.175128+0000","last_undegraded":"2024-09-10T21:24:53.175128+0000","last_fullsized":"2024-09-10T21:24:53.175128+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:38:55.009670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174808+0000","last_change":"2024-09-10T21:24:46.859532+0000","last_active":"2024-09-10T21:24:53.174808+0000","last_peered":"2024-09-10T21:24:53.174808+0000","last_clean":"2024-09-10T21:24:53.174808+0000","last_became_active":"2024-09-10T21:24:46.859144+0000","last_became_peered":"2024-09-10T21:24:46.859144+0000","last_unstale":"2024-09-10T21:24:53.174808+0000","last_undegraded":"2024-09-10T21:24:53.174808+0000","last_fullsized":"2024-09-10T21:24:53.174808+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:48:41.992441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.883627+0000","last_change":"2024-09-10T21:24:52.061029+0000","last_active":"2024-09-10T21:24:52.883627+0000","last_peered":"2024-09-10T21:24:52.883627+0000","last_clean":"2024-09-10T21:24:52.883627+0000","last_became_active":"2024-09-10T21:24:52.060605+0000","last_became_peered":"2024-09-10T21:24:52.060605+0000","last_unstale":"2024-09-10T21:24:52.883627+0000","last_undegraded":"2024-09-10T21:24:52.883627+0000","last_fullsized":"2024-09-10T21:24:52.883627+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:09:48.077266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174744+0000","last_change":"2024-09-10T21:24:47.866927+0000","last_active":"2024-09-10T21:24:53.174744+0000","last_peered":"2024-09-10T21:24:53.174744+0000","last_clean":"2024-09-10T21:24:53.174744+0000","last_became_active":"2024-09-10T21:24:47.866650+0000","last_became_peered":"2024-09-10T21:24:47.866650+0000","last_unstale":"2024-09-10T21:24:53.174744+0000","last_undegraded":"2024-09-10T21:24:53.174744+0000","last_fullsized":"2024-09-10T21:24:53.174744+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:47:37.304624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"24'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.879000+0000","last_change":"2024-09-10T21:24:46.857082+0000","last_active":"2024-09-10T21:24:52.879000+0000","last_peered":"2024-09-10T21:24:52.879000+0000","last_clean":"2024-09-10T21:24:52.879000+0000","last_became_active":"2024-09-10T21:24:46.856799+0000","last_became_peered":"2024-09-10T21:24:46.856799+0000","last_unstale":"2024-09-10T21:24:52.879000+0000","last_undegraded":"2024-09-10T21:24:52.879000+0000","last_fullsized":"2024-09-10T21:24:52.879000+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:40:33.632846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174743+0000","last_change":"2024-09-10T21:24:51.911880+0000","last_active":"2024-09-10T21:24:53.174743+0000","last_peered":"2024-09-10T21:24:53.174743+0000","last_clean":"2024-09-10T21:24:53.174743+0000","last_became_active":"2024-09-10T21:24:51.911088+0000","last_became_peered":"2024-09-10T21:24:51.911088+0000","last_unstale":"2024-09-10T21:24:53.174743+0000","last_undegraded":"2024-09-10T21:24:53.174743+0000","last_fullsized":"2024-09-10T21:24:53.174743+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:38:57.030655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.175370+0000","last_change":"2024-09-10T21:24:47.876946+0000","last_active":"2024-09-10T21:24:53.175370+0000","last_peered":"2024-09-10T21:24:53.175370+0000","last_clean":"2024-09-10T21:24:53.175370+0000","last_became_active":"2024-09-10T21:24:47.876690+0000","last_became_peered":"2024-09-10T21:24:47.876690+0000","last_unstale":"2024-09-10T21:24:53.175370+0000","last_undegraded":"2024-09-10T21:24:53.175370+0000","last_fullsized":"2024-09-10T21:24:53.175370+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:14:06.368275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"24'2","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.175105+0000","last_change":"2024-09-10T21:24:46.855248+0000","last_active":"2024-09-10T21:24:53.175105+0000","last_peered":"2024-09-10T21:24:53.175105+0000","last_clean":"2024-09-10T21:24:53.175105+0000","last_became_active":"2024-09-10T21:24:46.854930+0000","last_became_peered":"2024-09-10T21:24:46.854930+0000","last_unstale":"2024-09-10T21:24:53.175105+0000","last_undegraded":"2024-09-10T21:24:53.175105+0000","last_fullsized":"2024-09-10T21:24:53.175105+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:46:55.286321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.173528+0000","last_change":"2024-09-10T21:24:52.063112+0000","last_active":"2024-09-10T21:24:53.173528+0000","last_peered":"2024-09-10T21:24:53.173528+0000","last_clean":"2024-09-10T21:24:53.173528+0000","last_became_active":"2024-09-10T21:24:52.062395+0000","last_became_peered":"2024-09-10T21:24:52.062395+0000","last_unstale":"2024-09-10T21:24:53.173528+0000","last_undegraded":"2024-09-10T21:24:53.173528+0000","last_fullsized":"2024-09-10T21:24:53.173528+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:21:53.080013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.175254+0000","last_change":"2024-09-10T21:24:47.858593+0000","last_active":"2024-09-10T21:24:53.175254+0000","last_peered":"2024-09-10T21:24:53.175254+0000","last_clean":"2024-09-10T21:24:53.175254+0000","last_became_active":"2024-09-10T21:24:47.858216+0000","last_became_peered":"2024-09-10T21:24:47.858216+0000","last_unstale":"2024-09-10T21:24:53.175254+0000","last_undegraded":"2024-09-10T21:24:53.175254+0000","last_fullsized":"2024-09-10T21:24:53.175254+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:42.961291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"24'1","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.886950+0000","last_change":"2024-09-10T21:24:46.860330+0000","last_active":"2024-09-10T21:24:52.886950+0000","last_peered":"2024-09-10T21:24:52.886950+0000","last_clean":"2024-09-10T21:24:52.886950+0000","last_became_active":"2024-09-10T21:24:46.860053+0000","last_became_peered":"2024-09-10T21:24:46.860053+0000","last_unstale":"2024-09-10T21:24:52.886950+0000","last_undegraded":"2024-09-10T21:24:52.886950+0000","last_fullsized":"2024-09-10T21:24:52.886950+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:58:53.177740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.883701+0000","last_change":"2024-09-10T21:24:51.901673+0000","last_active":"2024-09-10T21:24:52.883701+0000","last_peered":"2024-09-10T21:24:52.883701+0000","last_clean":"2024-09-10T21:24:52.883701+0000","last_became_active":"2024-09-10T21:24:51.901222+0000","last_became_peered":"2024-09-10T21:24:51.901222+0000","last_unstale":"2024-09-10T21:24:52.883701+0000","last_undegraded":"2024-09-10T21:24:52.883701+0000","last_fullsized":"2024-09-10T21:24:52.883701+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:07:52.120299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.173790+0000","last_change":"2024-09-10T21:24:47.870589+0000","last_active":"2024-09-10T21:24:53.173790+0000","last_peered":"2024-09-10T21:24:53.173790+0000","last_clean":"2024-09-10T21:24:53.173790+0000","last_became_active":"2024-09-10T21:24:47.861834+0000","last_became_peered":"2024-09-10T21:24:47.861834+0000","last_unstale":"2024-09-10T21:24:53.173790+0000","last_undegraded":"2024-09-10T21:24:53.173790+0000","last_fullsized":"2024-09-10T21:24:53.173790+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:43.363013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"24'8","reported_seq":31,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.173490+0000","last_change":"2024-09-10T21:24:46.852064+0000","last_active":"2024-09-10T21:24:53.173490+0000","last_peered":"2024-09-10T21:24:53.173490+0000","last_clean":"2024-09-10T21:24:53.173490+0000","last_became_active":"2024-09-10T21:24:46.851776+0000","last_became_peered":"2024-09-10T21:24:46.851776+0000","last_unstale":"2024-09-10T21:24:53.173490+0000","last_undegraded":"2024-09-10T21:24:53.173490+0000","last_fullsized":"2024-09-10T21:24:53.173490+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:26:24.371844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174153+0000","last_change":"2024-09-10T21:24:51.904449+0000","last_active":"2024-09-10T21:24:53.174153+0000","last_peered":"2024-09-10T21:24:53.174153+0000","last_clean":"2024-09-10T21:24:53.174153+0000","last_became_active":"2024-09-10T21:24:51.903533+0000","last_became_peered":"2024-09-10T21:24:51.903533+0000","last_unstale":"2024-09-10T21:24:53.174153+0000","last_undegraded":"2024-09-10T21:24:53.174153+0000","last_fullsized":"2024-09-10T21:24:53.174153+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:11.654935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.883479+0000","last_change":"2024-09-10T21:24:47.857736+0000","last_active":"2024-09-10T21:24:52.883479+0000","last_peered":"2024-09-10T21:24:52.883479+0000","last_clean":"2024-09-10T21:24:52.883479+0000","last_became_active":"2024-09-10T21:24:47.857437+0000","last_became_peered":"2024-09-10T21:24:47.857437+0000","last_unstale":"2024-09-10T21:24:52.883479+0000","last_undegraded":"2024-09-10T21:24:52.883479+0000","last_fullsized":"2024-09-10T21:24:52.883479+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:13:40.032808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.883979+0000","last_change":"2024-09-10T21:24:46.860662+0000","last_active":"2024-09-10T21:24:52.883979+0000","last_peered":"2024-09-10T21:24:52.883979+0000","last_clean":"2024-09-10T21:24:52.883979+0000","last_became_active":"2024-09-10T21:24:46.859796+0000","last_became_peered":"2024-09-10T21:24:46.859796+0000","last_unstale":"2024-09-10T21:24:52.883979+0000","last_undegraded":"2024-09-10T21:24:52.883979+0000","last_fullsized":"2024-09-10T21:24:52.883979+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:48:27.758481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.885543+0000","last_change":"2024-09-10T21:24:52.069770+0000","last_active":"2024-09-10T21:24:52.885543+0000","last_peered":"2024-09-10T21:24:52.885543+0000","last_clean":"2024-09-10T21:24:52.885543+0000","last_became_active":"2024-09-10T21:24:52.067409+0000","last_became_peered":"2024-09-10T21:24:52.067409+0000","last_unstale":"2024-09-10T21:24:52.885543+0000","last_undegraded":"2024-09-10T21:24:52.885543+0000","last_fullsized":"2024-09-10T21:24:52.885543+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:23:39.622612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174136+0000","last_change":"2024-09-10T21:24:47.858692+0000","last_active":"2024-09-10T21:24:53.174136+0000","last_peered":"2024-09-10T21:24:53.174136+0000","last_clean":"2024-09-10T21:24:53.174136+0000","last_became_active":"2024-09-10T21:24:47.858343+0000","last_became_peered":"2024-09-10T21:24:47.858343+0000","last_unstale":"2024-09-10T21:24:53.174136+0000","last_undegraded":"2024-09-10T21:24:53.174136+0000","last_fullsized":"2024-09-10T21:24:53.174136+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:04:36.661791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.175219+0000","last_change":"2024-09-10T21:24:46.864592+0000","last_active":"2024-09-10T21:24:53.175219+0000","last_peered":"2024-09-10T21:24:53.175219+0000","last_clean":"2024-09-10T21:24:53.175219+0000","last_became_active":"2024-09-10T21:24:46.858696+0000","last_became_peered":"2024-09-10T21:24:46.858696+0000","last_unstale":"2024-09-10T21:24:53.175219+0000","last_undegraded":"2024-09-10T21:24:53.175219+0000","last_fullsized":"2024-09-10T21:24:53.175219+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:54:19.356867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.885074+0000","last_change":"2024-09-10T21:24:52.061572+0000","last_active":"2024-09-10T21:24:52.885074+0000","last_peered":"2024-09-10T21:24:52.885074+0000","last_clean":"2024-09-10T21:24:52.885074+0000","last_became_active":"2024-09-10T21:24:52.061061+0000","last_became_peered":"2024-09-10T21:24:52.061061+0000","last_unstale":"2024-09-10T21:24:52.885074+0000","last_undegraded":"2024-09-10T21:24:52.885074+0000","last_fullsized":"2024-09-10T21:24:52.885074+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:04:36.399811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174340+0000","last_change":"2024-09-10T21:24:47.857699+0000","last_active":"2024-09-10T21:24:53.174340+0000","last_peered":"2024-09-10T21:24:53.174340+0000","last_clean":"2024-09-10T21:24:53.174340+0000","last_became_active":"2024-09-10T21:24:47.857429+0000","last_became_peered":"2024-09-10T21:24:47.857429+0000","last_unstale":"2024-09-10T21:24:53.174340+0000","last_undegraded":"2024-09-10T21:24:53.174340+0000","last_fullsized":"2024-09-10T21:24:53.174340+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:07:11.880228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"24'2","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174839+0000","last_change":"2024-09-10T21:24:46.859753+0000","last_active":"2024-09-10T21:24:53.174839+0000","last_peered":"2024-09-10T21:24:53.174839+0000","last_clean":"2024-09-10T21:24:53.174839+0000","last_became_active":"2024-09-10T21:24:46.859299+0000","last_became_peered":"2024-09-10T21:24:46.859299+0000","last_unstale":"2024-09-10T21:24:53.174839+0000","last_undegraded":"2024-09-10T21:24:53.174839+0000","last_fullsized":"2024-09-10T21:24:53.174839+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:06:41.965052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174673+0000","last_change":"2024-09-10T21:24:52.064718+0000","last_active":"2024-09-10T21:24:53.174673+0000","last_peered":"2024-09-10T21:24:53.174673+0000","last_clean":"2024-09-10T21:24:53.174673+0000","last_became_active":"2024-09-10T21:24:52.062327+0000","last_became_peered":"2024-09-10T21:24:52.062327+0000","last_unstale":"2024-09-10T21:24:53.174673+0000","last_undegraded":"2024-09-10T21:24:53.174673+0000","last_fullsized":"2024-09-10T21:24:53.174673+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:43:17.173095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.175009+0000","last_change":"2024-09-10T21:24:47.870766+0000","last_active":"2024-09-10T21:24:53.175009+0000","last_peered":"2024-09-10T21:24:53.175009+0000","last_clean":"2024-09-10T21:24:53.175009+0000","last_became_active":"2024-09-10T21:24:47.870276+0000","last_became_peered":"2024-09-10T21:24:47.870276+0000","last_unstale":"2024-09-10T21:24:53.175009+0000","last_undegraded":"2024-09-10T21:24:53.175009+0000","last_fullsized":"2024-09-10T21:24:53.175009+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:54:38.400877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"25'9","reported_seq":31,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:25:05.421191+0000","last_change":"2024-09-10T21:24:46.867790+0000","last_active":"2024-09-10T21:25:05.421191+0000","last_peered":"2024-09-10T21:25:05.421191+0000","last_clean":"2024-09-10T21:25:05.421191+0000","last_became_active":"2024-09-10T21:24:46.866899+0000","last_became_peered":"2024-09-10T21:24:46.866899+0000","last_unstale":"2024-09-10T21:25:05.421191+0000","last_undegraded":"2024-09-10T21:25:05.421191+0000","last_fullsized":"2024-09-10T21:25:05.421191+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:25:04.923730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.878572+0000","last_change":"2024-09-10T21:24:51.899803+0000","last_active":"2024-09-10T21:24:52.878572+0000","last_peered":"2024-09-10T21:24:52.878572+0000","last_clean":"2024-09-10T21:24:52.878572+0000","last_became_active":"2024-09-10T21:24:51.895398+0000","last_became_peered":"2024-09-10T21:24:51.895398+0000","last_unstale":"2024-09-10T21:24:52.878572+0000","last_undegraded":"2024-09-10T21:24:52.878572+0000","last_fullsized":"2024-09-10T21:24:52.878572+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:52:34.689860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.886661+0000","last_change":"2024-09-10T21:24:47.857544+0000","last_active":"2024-09-10T21:24:52.886661+0000","last_peered":"2024-09-10T21:24:52.886661+0000","last_clean":"2024-09-10T21:24:52.886661+0000","last_became_active":"2024-09-10T21:24:47.857284+0000","last_became_peered":"2024-09-10T21:24:47.857284+0000","last_unstale":"2024-09-10T21:24:52.886661+0000","last_undegraded":"2024-09-10T21:24:52.886661+0000","last_fullsized":"2024-09-10T21:24:52.886661+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:10:22.747606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"24'3","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.878942+0000","last_change":"2024-09-10T21:24:46.859633+0000","last_active":"2024-09-10T21:24:52.878942+0000","last_peered":"2024-09-10T21:24:52.878942+0000","last_clean":"2024-09-10T21:24:52.878942+0000","last_became_active":"2024-09-10T21:24:46.859332+0000","last_became_peered":"2024-09-10T21:24:46.859332+0000","last_unstale":"2024-09-10T21:24:52.878942+0000","last_undegraded":"2024-09-10T21:24:52.878942+0000","last_fullsized":"2024-09-10T21:24:52.878942+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:56:46.975225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.886734+0000","last_change":"2024-09-10T21:24:51.899100+0000","last_active":"2024-09-10T21:24:52.886734+0000","last_peered":"2024-09-10T21:24:52.886734+0000","last_clean":"2024-09-10T21:24:52.886734+0000","last_became_active":"2024-09-10T21:24:51.898642+0000","last_became_peered":"2024-09-10T21:24:51.898642+0000","last_unstale":"2024-09-10T21:24:52.886734+0000","last_undegraded":"2024-09-10T21:24:52.886734+0000","last_fullsized":"2024-09-10T21:24:52.886734+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:06.394081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174291+0000","last_change":"2024-09-10T21:24:47.867890+0000","last_active":"2024-09-10T21:24:53.174291+0000","last_peered":"2024-09-10T21:24:53.174291+0000","last_clean":"2024-09-10T21:24:53.174291+0000","last_became_active":"2024-09-10T21:24:47.865016+0000","last_became_peered":"2024-09-10T21:24:47.865016+0000","last_unstale":"2024-09-10T21:24:53.174291+0000","last_undegraded":"2024-09-10T21:24:53.174291+0000","last_fullsized":"2024-09-10T21:24:53.174291+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:53:09.082512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174673+0000","last_change":"2024-09-10T21:24:46.861400+0000","last_active":"2024-09-10T21:24:53.174673+0000","last_peered":"2024-09-10T21:24:53.174673+0000","last_clean":"2024-09-10T21:24:53.174673+0000","last_became_active":"2024-09-10T21:24:46.861136+0000","last_became_peered":"2024-09-10T21:24:46.861136+0000","last_unstale":"2024-09-10T21:24:53.174673+0000","last_undegraded":"2024-09-10T21:24:53.174673+0000","last_fullsized":"2024-09-10T21:24:53.174673+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:35:31.682448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.884671+0000","last_change":"2024-09-10T21:24:51.907850+0000","last_active":"2024-09-10T21:24:52.884671+0000","last_peered":"2024-09-10T21:24:52.884671+0000","last_clean":"2024-09-10T21:24:52.884671+0000","last_became_active":"2024-09-10T21:24:51.907062+0000","last_became_peered":"2024-09-10T21:24:51.907062+0000","last_unstale":"2024-09-10T21:24:52.884671+0000","last_undegraded":"2024-09-10T21:24:52.884671+0000","last_fullsized":"2024-09-10T21:24:52.884671+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:32:52.519627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174521+0000","last_change":"2024-09-10T21:24:47.861800+0000","last_active":"2024-09-10T21:24:53.174521+0000","last_peered":"2024-09-10T21:24:53.174521+0000","last_clean":"2024-09-10T21:24:53.174521+0000","last_became_active":"2024-09-10T21:24:47.861498+0000","last_became_peered":"2024-09-10T21:24:47.861498+0000","last_unstale":"2024-09-10T21:24:53.174521+0000","last_undegraded":"2024-09-10T21:24:53.174521+0000","last_fullsized":"2024-09-10T21:24:53.174521+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:07:21.128033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.885153+0000","last_change":"2024-09-10T21:24:46.871677+0000","last_active":"2024-09-10T21:24:52.885153+0000","last_peered":"2024-09-10T21:24:52.885153+0000","last_clean":"2024-09-10T21:24:52.885153+0000","last_became_active":"2024-09-10T21:24:46.871367+0000","last_became_peered":"2024-09-10T21:24:46.871367+0000","last_unstale":"2024-09-10T21:24:52.885153+0000","last_undegraded":"2024-09-10T21:24:52.885153+0000","last_fullsized":"2024-09-10T21:24:52.885153+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:40:33.632846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.884646+0000","last_change":"2024-09-10T21:24:52.068637+0000","last_active":"2024-09-10T21:24:52.884646+0000","last_peered":"2024-09-10T21:24:52.884646+0000","last_clean":"2024-09-10T21:24:52.884646+0000","last_became_active":"2024-09-10T21:24:52.068348+0000","last_became_peered":"2024-09-10T21:24:52.068348+0000","last_unstale":"2024-09-10T21:24:52.884646+0000","last_undegraded":"2024-09-10T21:24:52.884646+0000","last_fullsized":"2024-09-10T21:24:52.884646+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:43:17.173095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174107+0000","last_change":"2024-09-10T21:24:52.067732+0000","last_active":"2024-09-10T21:24:53.174107+0000","last_peered":"2024-09-10T21:24:53.174107+0000","last_clean":"2024-09-10T21:24:53.174107+0000","last_became_active":"2024-09-10T21:24:52.067330+0000","last_became_peered":"2024-09-10T21:24:52.067330+0000","last_unstale":"2024-09-10T21:24:53.174107+0000","last_undegraded":"2024-09-10T21:24:53.174107+0000","last_fullsized":"2024-09-10T21:24:53.174107+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:03:41.110214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.878246+0000","last_change":"2024-09-10T21:24:47.862774+0000","last_active":"2024-09-10T21:24:52.878246+0000","last_peered":"2024-09-10T21:24:52.878246+0000","last_clean":"2024-09-10T21:24:52.878246+0000","last_became_active":"2024-09-10T21:24:47.862556+0000","last_became_peered":"2024-09-10T21:24:47.862556+0000","last_unstale":"2024-09-10T21:24:52.878246+0000","last_undegraded":"2024-09-10T21:24:52.878246+0000","last_fullsized":"2024-09-10T21:24:52.878246+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:17:52.343553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.173610+0000","last_change":"2024-09-10T21:24:46.862847+0000","last_active":"2024-09-10T21:24:53.173610+0000","last_peered":"2024-09-10T21:24:53.173610+0000","last_clean":"2024-09-10T21:24:53.173610+0000","last_became_active":"2024-09-10T21:24:46.862540+0000","last_became_peered":"2024-09-10T21:24:46.862540+0000","last_unstale":"2024-09-10T21:24:53.173610+0000","last_undegraded":"2024-09-10T21:24:53.173610+0000","last_fullsized":"2024-09-10T21:24:53.173610+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:52:31.628568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.884678+0000","last_change":"2024-09-10T21:24:51.901224+0000","last_active":"2024-09-10T21:24:52.884678+0000","last_peered":"2024-09-10T21:24:52.884678+0000","last_clean":"2024-09-10T21:24:52.884678+0000","last_became_active":"2024-09-10T21:24:51.900610+0000","last_became_peered":"2024-09-10T21:24:51.900610+0000","last_unstale":"2024-09-10T21:24:52.884678+0000","last_undegraded":"2024-09-10T21:24:52.884678+0000","last_fullsized":"2024-09-10T21:24:52.884678+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:52:34.689860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.884489+0000","last_change":"2024-09-10T21:24:47.861255+0000","last_active":"2024-09-10T21:24:52.884489+0000","last_peered":"2024-09-10T21:24:52.884489+0000","last_clean":"2024-09-10T21:24:52.884489+0000","last_became_active":"2024-09-10T21:24:47.860930+0000","last_became_peered":"2024-09-10T21:24:47.860930+0000","last_unstale":"2024-09-10T21:24:52.884489+0000","last_undegraded":"2024-09-10T21:24:52.884489+0000","last_fullsized":"2024-09-10T21:24:52.884489+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:37:51.784258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.884796+0000","last_change":"2024-09-10T21:24:46.858561+0000","last_active":"2024-09-10T21:24:52.884796+0000","last_peered":"2024-09-10T21:24:52.884796+0000","last_clean":"2024-09-10T21:24:52.884796+0000","last_became_active":"2024-09-10T21:24:46.858265+0000","last_became_peered":"2024-09-10T21:24:46.858265+0000","last_unstale":"2024-09-10T21:24:52.884796+0000","last_undegraded":"2024-09-10T21:24:52.884796+0000","last_fullsized":"2024-09-10T21:24:52.884796+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:49:00.454609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"24'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.884271+0000","last_change":"2024-09-10T21:24:46.856886+0000","last_active":"2024-09-10T21:24:52.884271+0000","last_peered":"2024-09-10T21:24:52.884271+0000","last_clean":"2024-09-10T21:24:52.884271+0000","last_became_active":"2024-09-10T21:24:46.856506+0000","last_became_peered":"2024-09-10T21:24:46.856506+0000","last_unstale":"2024-09-10T21:24:52.884271+0000","last_undegraded":"2024-09-10T21:24:52.884271+0000","last_fullsized":"2024-09-10T21:24:52.884271+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:56:46.975225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174721+0000","last_change":"2024-09-10T21:24:47.848602+0000","last_active":"2024-09-10T21:24:53.174721+0000","last_peered":"2024-09-10T21:24:53.174721+0000","last_clean":"2024-09-10T21:24:53.174721+0000","last_became_active":"2024-09-10T21:24:47.848160+0000","last_became_peered":"2024-09-10T21:24:47.848160+0000","last_unstale":"2024-09-10T21:24:53.174721+0000","last_undegraded":"2024-09-10T21:24:53.174721+0000","last_fullsized":"2024-09-10T21:24:53.174721+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:32:50.027827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174649+0000","last_change":"2024-09-10T21:24:51.903553+0000","last_active":"2024-09-10T21:24:53.174649+0000","last_peered":"2024-09-10T21:24:53.174649+0000","last_clean":"2024-09-10T21:24:53.174649+0000","last_became_active":"2024-09-10T21:24:51.903120+0000","last_became_peered":"2024-09-10T21:24:51.903120+0000","last_unstale":"2024-09-10T21:24:53.174649+0000","last_undegraded":"2024-09-10T21:24:53.174649+0000","last_fullsized":"2024-09-10T21:24:53.174649+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:58.240380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"24'1","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.884865+0000","last_change":"2024-09-10T21:24:46.853550+0000","last_active":"2024-09-10T21:24:52.884865+0000","last_peered":"2024-09-10T21:24:52.884865+0000","last_clean":"2024-09-10T21:24:52.884865+0000","last_became_active":"2024-09-10T21:24:46.852204+0000","last_became_peered":"2024-09-10T21:24:46.852204+0000","last_unstale":"2024-09-10T21:24:52.884865+0000","last_undegraded":"2024-09-10T21:24:52.884865+0000","last_fullsized":"2024-09-10T21:24:52.884865+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:06:37.669444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.884581+0000","last_change":"2024-09-10T21:24:47.849151+0000","last_active":"2024-09-10T21:24:52.884581+0000","last_peered":"2024-09-10T21:24:52.884581+0000","last_clean":"2024-09-10T21:24:52.884581+0000","last_became_active":"2024-09-10T21:24:47.848851+0000","last_became_peered":"2024-09-10T21:24:47.848851+0000","last_unstale":"2024-09-10T21:24:52.884581+0000","last_undegraded":"2024-09-10T21:24:52.884581+0000","last_fullsized":"2024-09-10T21:24:52.884581+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:39:25.828558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.885276+0000","last_change":"2024-09-10T21:24:51.907549+0000","last_active":"2024-09-10T21:24:52.885276+0000","last_peered":"2024-09-10T21:24:52.885276+0000","last_clean":"2024-09-10T21:24:52.885276+0000","last_became_active":"2024-09-10T21:24:51.907082+0000","last_became_peered":"2024-09-10T21:24:51.907082+0000","last_unstale":"2024-09-10T21:24:52.885276+0000","last_undegraded":"2024-09-10T21:24:52.885276+0000","last_fullsized":"2024-09-10T21:24:52.885276+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:44:30.813754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.883360+0000","last_change":"2024-09-10T21:24:47.857804+0000","last_active":"2024-09-10T21:24:52.883360+0000","last_peered":"2024-09-10T21:24:52.883360+0000","last_clean":"2024-09-10T21:24:52.883360+0000","last_became_active":"2024-09-10T21:24:47.857501+0000","last_became_peered":"2024-09-10T21:24:47.857501+0000","last_unstale":"2024-09-10T21:24:52.883360+0000","last_undegraded":"2024-09-10T21:24:52.883360+0000","last_fullsized":"2024-09-10T21:24:52.883360+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:19:20.807422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.173632+0000","last_change":"2024-09-10T21:24:47.867803+0000","last_active":"2024-09-10T21:24:53.173632+0000","last_peered":"2024-09-10T21:24:53.173632+0000","last_clean":"2024-09-10T21:24:53.173632+0000","last_became_active":"2024-09-10T21:24:47.867504+0000","last_became_peered":"2024-09-10T21:24:47.867504+0000","last_unstale":"2024-09-10T21:24:53.173632+0000","last_undegraded":"2024-09-10T21:24:53.173632+0000","last_fullsized":"2024-09-10T21:24:53.173632+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:53:30.420454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174959+0000","last_change":"2024-09-10T21:24:47.868901+0000","last_active":"2024-09-10T21:24:53.174959+0000","last_peered":"2024-09-10T21:24:53.174959+0000","last_clean":"2024-09-10T21:24:53.174959+0000","last_became_active":"2024-09-10T21:24:47.868168+0000","last_became_peered":"2024-09-10T21:24:47.868168+0000","last_unstale":"2024-09-10T21:24:53.174959+0000","last_undegraded":"2024-09-10T21:24:53.174959+0000","last_fullsized":"2024-09-10T21:24:53.174959+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:06:21.461089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174633+0000","last_change":"2024-09-10T21:24:47.864561+0000","last_active":"2024-09-10T21:24:53.174633+0000","last_peered":"2024-09-10T21:24:53.174633+0000","last_clean":"2024-09-10T21:24:53.174633+0000","last_became_active":"2024-09-10T21:24:47.864100+0000","last_became_peered":"2024-09-10T21:24:47.864100+0000","last_unstale":"2024-09-10T21:24:53.174633+0000","last_undegraded":"2024-09-10T21:24:53.174633+0000","last_fullsized":"2024-09-10T21:24:53.174633+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:34:13.053494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.885174+0000","last_change":"2024-09-10T21:24:47.864303+0000","last_active":"2024-09-10T21:24:52.885174+0000","last_peered":"2024-09-10T21:24:52.885174+0000","last_clean":"2024-09-10T21:24:52.885174+0000","last_became_active":"2024-09-10T21:24:47.863970+0000","last_became_peered":"2024-09-10T21:24:47.863970+0000","last_unstale":"2024-09-10T21:24:52.885174+0000","last_undegraded":"2024-09-10T21:24:52.885174+0000","last_fullsized":"2024-09-10T21:24:52.885174+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:46:36.918539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.878793+0000","last_change":"2024-09-10T21:24:47.860734+0000","last_active":"2024-09-10T21:24:52.878793+0000","last_peered":"2024-09-10T21:24:52.878793+0000","last_clean":"2024-09-10T21:24:52.878793+0000","last_became_active":"2024-09-10T21:24:47.860471+0000","last_became_peered":"2024-09-10T21:24:47.860471+0000","last_unstale":"2024-09-10T21:24:52.878793+0000","last_undegraded":"2024-09-10T21:24:52.878793+0000","last_fullsized":"2024-09-10T21:24:52.878793+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:54:13.200082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.885049+0000","last_change":"2024-09-10T21:24:47.872504+0000","last_active":"2024-09-10T21:24:52.885049+0000","last_peered":"2024-09-10T21:24:52.885049+0000","last_clean":"2024-09-10T21:24:52.885049+0000","last_became_active":"2024-09-10T21:24:47.872218+0000","last_became_peered":"2024-09-10T21:24:47.872218+0000","last_unstale":"2024-09-10T21:24:52.885049+0000","last_undegraded":"2024-09-10T21:24:52.885049+0000","last_fullsized":"2024-09-10T21:24:52.885049+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:21:32.400520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.885478+0000","last_change":"2024-09-10T21:24:47.862167+0000","last_active":"2024-09-10T21:24:52.885478+0000","last_peered":"2024-09-10T21:24:52.885478+0000","last_clean":"2024-09-10T21:24:52.885478+0000","last_became_active":"2024-09-10T21:24:47.859649+0000","last_became_peered":"2024-09-10T21:24:47.859649+0000","last_unstale":"2024-09-10T21:24:52.885478+0000","last_undegraded":"2024-09-10T21:24:52.885478+0000","last_fullsized":"2024-09-10T21:24:52.885478+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:42:59.204284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.173544+0000","last_change":"2024-09-10T21:24:47.865322+0000","last_active":"2024-09-10T21:24:53.173544+0000","last_peered":"2024-09-10T21:24:53.173544+0000","last_clean":"2024-09-10T21:24:53.173544+0000","last_became_active":"2024-09-10T21:24:47.864933+0000","last_became_peered":"2024-09-10T21:24:47.864933+0000","last_unstale":"2024-09-10T21:24:53.173544+0000","last_undegraded":"2024-09-10T21:24:53.173544+0000","last_fullsized":"2024-09-10T21:24:53.173544+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:40:45.551723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.884760+0000","last_change":"2024-09-10T21:24:47.860155+0000","last_active":"2024-09-10T21:24:52.884760+0000","last_peered":"2024-09-10T21:24:52.884760+0000","last_clean":"2024-09-10T21:24:52.884760+0000","last_became_active":"2024-09-10T21:24:47.859857+0000","last_became_peered":"2024-09-10T21:24:47.859857+0000","last_unstale":"2024-09-10T21:24:52.884760+0000","last_undegraded":"2024-09-10T21:24:52.884760+0000","last_fullsized":"2024-09-10T21:24:52.884760+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:32:50.027827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174935+0000","last_change":"2024-09-10T21:24:47.872848+0000","last_active":"2024-09-10T21:24:53.174935+0000","last_peered":"2024-09-10T21:24:53.174935+0000","last_clean":"2024-09-10T21:24:53.174935+0000","last_became_active":"2024-09-10T21:24:47.868671+0000","last_became_peered":"2024-09-10T21:24:47.868671+0000","last_unstale":"2024-09-10T21:24:53.174935+0000","last_undegraded":"2024-09-10T21:24:53.174935+0000","last_fullsized":"2024-09-10T21:24:53.174935+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:33:59.291647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174519+0000","last_change":"2024-09-10T21:24:47.857464+0000","last_active":"2024-09-10T21:24:53.174519+0000","last_peered":"2024-09-10T21:24:53.174519+0000","last_clean":"2024-09-10T21:24:53.174519+0000","last_became_active":"2024-09-10T21:24:47.857116+0000","last_became_peered":"2024-09-10T21:24:47.857116+0000","last_unstale":"2024-09-10T21:24:53.174519+0000","last_undegraded":"2024-09-10T21:24:53.174519+0000","last_fullsized":"2024-09-10T21:24:53.174519+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:56:43.269026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.884005+0000","last_change":"2024-09-10T21:24:47.871619+0000","last_active":"2024-09-10T21:24:52.884005+0000","last_peered":"2024-09-10T21:24:52.884005+0000","last_clean":"2024-09-10T21:24:52.884005+0000","last_became_active":"2024-09-10T21:24:47.871350+0000","last_became_peered":"2024-09-10T21:24:47.871350+0000","last_unstale":"2024-09-10T21:24:52.884005+0000","last_undegraded":"2024-09-10T21:24:52.884005+0000","last_fullsized":"2024-09-10T21:24:52.884005+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:41.305116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.175386+0000","last_change":"2024-09-10T21:24:47.861352+0000","last_active":"2024-09-10T21:24:53.175386+0000","last_peered":"2024-09-10T21:24:53.175386+0000","last_clean":"2024-09-10T21:24:53.175386+0000","last_became_active":"2024-09-10T21:24:47.861007+0000","last_became_peered":"2024-09-10T21:24:47.861007+0000","last_unstale":"2024-09-10T21:24:53.175386+0000","last_undegraded":"2024-09-10T21:24:53.175386+0000","last_fullsized":"2024-09-10T21:24:53.175386+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:25.273458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.175077+0000","last_change":"2024-09-10T21:24:47.864205+0000","last_active":"2024-09-10T21:24:53.175077+0000","last_peered":"2024-09-10T21:24:53.175077+0000","last_clean":"2024-09-10T21:24:53.175077+0000","last_became_active":"2024-09-10T21:24:47.863932+0000","last_became_peered":"2024-09-10T21:24:47.863932+0000","last_unstale":"2024-09-10T21:24:53.175077+0000","last_undegraded":"2024-09-10T21:24:53.175077+0000","last_fullsized":"2024-09-10T21:24:53.175077+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:53:17.669629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.175063+0000","last_change":"2024-09-10T21:24:47.866225+0000","last_active":"2024-09-10T21:24:53.175063+0000","last_peered":"2024-09-10T21:24:53.175063+0000","last_clean":"2024-09-10T21:24:53.175063+0000","last_became_active":"2024-09-10T21:24:47.865587+0000","last_became_peered":"2024-09-10T21:24:47.865587+0000","last_unstale":"2024-09-10T21:24:53.175063+0000","last_undegraded":"2024-09-10T21:24:53.175063+0000","last_fullsized":"2024-09-10T21:24:53.175063+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:49:26.113068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.173421+0000","last_change":"2024-09-10T21:24:47.872669+0000","last_active":"2024-09-10T21:24:53.173421+0000","last_peered":"2024-09-10T21:24:53.173421+0000","last_clean":"2024-09-10T21:24:53.173421+0000","last_became_active":"2024-09-10T21:24:47.865837+0000","last_became_peered":"2024-09-10T21:24:47.865837+0000","last_unstale":"2024-09-10T21:24:53.173421+0000","last_undegraded":"2024-09-10T21:24:53.173421+0000","last_fullsized":"2024-09-10T21:24:53.173421+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:53:09.082512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.878332+0000","last_change":"2024-09-10T21:24:47.846567+0000","last_active":"2024-09-10T21:24:52.878332+0000","last_peered":"2024-09-10T21:24:52.878332+0000","last_clean":"2024-09-10T21:24:52.878332+0000","last_became_active":"2024-09-10T21:24:47.846283+0000","last_became_peered":"2024-09-10T21:24:47.846283+0000","last_unstale":"2024-09-10T21:24:52.878332+0000","last_undegraded":"2024-09-10T21:24:52.878332+0000","last_fullsized":"2024-09-10T21:24:52.878332+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:42:15.320518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174247+0000","last_change":"2024-09-10T21:24:47.844888+0000","last_active":"2024-09-10T21:24:53.174247+0000","last_peered":"2024-09-10T21:24:53.174247+0000","last_clean":"2024-09-10T21:24:53.174247+0000","last_became_active":"2024-09-10T21:24:47.844517+0000","last_became_peered":"2024-09-10T21:24:47.844517+0000","last_unstale":"2024-09-10T21:24:53.174247+0000","last_undegraded":"2024-09-10T21:24:53.174247+0000","last_fullsized":"2024-09-10T21:24:53.174247+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:00:14.988282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.884642+0000","last_change":"2024-09-10T21:24:47.862746+0000","last_active":"2024-09-10T21:24:52.884642+0000","last_peered":"2024-09-10T21:24:52.884642+0000","last_clean":"2024-09-10T21:24:52.884642+0000","last_became_active":"2024-09-10T21:24:47.862185+0000","last_became_peered":"2024-09-10T21:24:47.862185+0000","last_unstale":"2024-09-10T21:24:52.884642+0000","last_undegraded":"2024-09-10T21:24:52.884642+0000","last_fullsized":"2024-09-10T21:24:52.884642+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:06:21.461089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.886651+0000","last_change":"2024-09-10T21:24:47.867727+0000","last_active":"2024-09-10T21:24:52.886651+0000","last_peered":"2024-09-10T21:24:52.886651+0000","last_clean":"2024-09-10T21:24:52.886651+0000","last_became_active":"2024-09-10T21:24:47.861045+0000","last_became_peered":"2024-09-10T21:24:47.861045+0000","last_unstale":"2024-09-10T21:24:52.886651+0000","last_undegraded":"2024-09-10T21:24:52.886651+0000","last_fullsized":"2024-09-10T21:24:52.886651+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:08:30.065968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.884547+0000","last_change":"2024-09-10T21:24:47.867410+0000","last_active":"2024-09-10T21:24:52.884547+0000","last_peered":"2024-09-10T21:24:52.884547+0000","last_clean":"2024-09-10T21:24:52.884547+0000","last_became_active":"2024-09-10T21:24:47.867113+0000","last_became_peered":"2024-09-10T21:24:47.867113+0000","last_unstale":"2024-09-10T21:24:52.884547+0000","last_undegraded":"2024-09-10T21:24:52.884547+0000","last_fullsized":"2024-09-10T21:24:52.884547+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:49:53.502472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.884956+0000","last_change":"2024-09-10T21:24:47.867473+0000","last_active":"2024-09-10T21:24:52.884956+0000","last_peered":"2024-09-10T21:24:52.884956+0000","last_clean":"2024-09-10T21:24:52.884956+0000","last_became_active":"2024-09-10T21:24:47.867126+0000","last_became_peered":"2024-09-10T21:24:47.867126+0000","last_unstale":"2024-09-10T21:24:52.884956+0000","last_undegraded":"2024-09-10T21:24:52.884956+0000","last_fullsized":"2024-09-10T21:24:52.884956+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:54:32.307507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.175341+0000","last_change":"2024-09-10T21:24:47.865450+0000","last_active":"2024-09-10T21:24:53.175341+0000","last_peered":"2024-09-10T21:24:53.175341+0000","last_clean":"2024-09-10T21:24:53.175341+0000","last_became_active":"2024-09-10T21:24:47.865093+0000","last_became_peered":"2024-09-10T21:24:47.865093+0000","last_unstale":"2024-09-10T21:24:53.175341+0000","last_undegraded":"2024-09-10T21:24:53.175341+0000","last_fullsized":"2024-09-10T21:24:53.175341+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:12:18.854783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.884322+0000","last_change":"2024-09-10T21:24:47.868093+0000","last_active":"2024-09-10T21:24:52.884322+0000","last_peered":"2024-09-10T21:24:52.884322+0000","last_clean":"2024-09-10T21:24:52.884322+0000","last_became_active":"2024-09-10T21:24:47.867782+0000","last_became_peered":"2024-09-10T21:24:47.867782+0000","last_unstale":"2024-09-10T21:24:52.884322+0000","last_undegraded":"2024-09-10T21:24:52.884322+0000","last_fullsized":"2024-09-10T21:24:52.884322+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:34:13.053494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.173787+0000","last_change":"2024-09-10T21:24:47.862204+0000","last_active":"2024-09-10T21:24:53.173787+0000","last_peered":"2024-09-10T21:24:53.173787+0000","last_clean":"2024-09-10T21:24:53.173787+0000","last_became_active":"2024-09-10T21:24:47.861434+0000","last_became_peered":"2024-09-10T21:24:47.861434+0000","last_unstale":"2024-09-10T21:24:53.173787+0000","last_undegraded":"2024-09-10T21:24:53.173787+0000","last_fullsized":"2024-09-10T21:24:53.173787+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:50:03.245285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174760+0000","last_change":"2024-09-10T21:24:47.869248+0000","last_active":"2024-09-10T21:24:53.174760+0000","last_peered":"2024-09-10T21:24:53.174760+0000","last_clean":"2024-09-10T21:24:53.174760+0000","last_became_active":"2024-09-10T21:24:47.868503+0000","last_became_peered":"2024-09-10T21:24:47.868503+0000","last_unstale":"2024-09-10T21:24:53.174760+0000","last_undegraded":"2024-09-10T21:24:53.174760+0000","last_fullsized":"2024-09-10T21:24:53.174760+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:51:31.417845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174007+0000","last_change":"2024-09-10T21:24:47.864802+0000","last_active":"2024-09-10T21:24:53.174007+0000","last_peered":"2024-09-10T21:24:53.174007+0000","last_clean":"2024-09-10T21:24:53.174007+0000","last_became_active":"2024-09-10T21:24:47.863976+0000","last_became_peered":"2024-09-10T21:24:47.863976+0000","last_unstale":"2024-09-10T21:24:53.174007+0000","last_undegraded":"2024-09-10T21:24:53.174007+0000","last_fullsized":"2024-09-10T21:24:53.174007+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:55:08.544075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174437+0000","last_change":"2024-09-10T21:24:47.873364+0000","last_active":"2024-09-10T21:24:53.174437+0000","last_peered":"2024-09-10T21:24:53.174437+0000","last_clean":"2024-09-10T21:24:53.174437+0000","last_became_active":"2024-09-10T21:24:47.873069+0000","last_became_peered":"2024-09-10T21:24:47.873069+0000","last_unstale":"2024-09-10T21:24:53.174437+0000","last_undegraded":"2024-09-10T21:24:53.174437+0000","last_fullsized":"2024-09-10T21:24:53.174437+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:38:21.368499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174196+0000","last_change":"2024-09-10T21:24:47.856936+0000","last_active":"2024-09-10T21:24:53.174196+0000","last_peered":"2024-09-10T21:24:53.174196+0000","last_clean":"2024-09-10T21:24:53.174196+0000","last_became_active":"2024-09-10T21:24:47.854713+0000","last_became_peered":"2024-09-10T21:24:47.854713+0000","last_unstale":"2024-09-10T21:24:53.174196+0000","last_undegraded":"2024-09-10T21:24:53.174196+0000","last_fullsized":"2024-09-10T21:24:53.174196+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:46:36.918539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.878924+0000","last_change":"2024-09-10T21:24:47.846492+0000","last_active":"2024-09-10T21:24:52.878924+0000","last_peered":"2024-09-10T21:24:52.878924+0000","last_clean":"2024-09-10T21:24:52.878924+0000","last_became_active":"2024-09-10T21:24:47.846227+0000","last_became_peered":"2024-09-10T21:24:47.846227+0000","last_unstale":"2024-09-10T21:24:52.878924+0000","last_undegraded":"2024-09-10T21:24:52.878924+0000","last_fullsized":"2024-09-10T21:24:52.878924+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:13:58.432819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.175219+0000","last_change":"2024-09-10T21:24:47.871469+0000","last_active":"2024-09-10T21:24:53.175219+0000","last_peered":"2024-09-10T21:24:53.175219+0000","last_clean":"2024-09-10T21:24:53.175219+0000","last_became_active":"2024-09-10T21:24:47.871177+0000","last_became_peered":"2024-09-10T21:24:47.871177+0000","last_unstale":"2024-09-10T21:24:53.175219+0000","last_undegraded":"2024-09-10T21:24:53.175219+0000","last_fullsized":"2024-09-10T21:24:53.175219+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:19:24.226916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":66932,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509449,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":320,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":327680,"data_stored":693447,"data_compressed":33466,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509450,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":270336,"data_stored":106339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509449,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":258048,"data_stored":102497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509450,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31456,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340384,"statfs":{"total":96636764160,"available":96604553216,"internally_reserved":0,"allocated":266240,"data_stored":102657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542154,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27296,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":270336,"data_stored":102623,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542153,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":270336,"data_stored":106339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542156,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27364,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344476,"statfs":{"total":96636764160,"available":96608743424,"internally_reserved":0,"allocated":335872,"data_stored":693503,"data_compressed":33466,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574857,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":262144,"data_stored":102587,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33466,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33466,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T21:25:18.003 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-10T21:25:18.003 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config log 1 --format=json 2024-09-10T21:25:18.004 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-09-10T21:25:18.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:18.182+0000 7f94ac400640 1 -- 172.21.15.17:0/4061139739 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f94a415abb0 msgr2=0x7f94a41613a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:18.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:18.182+0000 7f94ac400640 1 --2- 172.21.15.17:0/4061139739 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f94a415abb0 0x7f94a41613a0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f94a4074860 tx=0x7f949c01deb0 comp rx=0 tx=0).stop 2024-09-10T21:25:18.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:18.182+0000 7f94ac400640 1 -- 172.21.15.17:0/4061139739 shutdown_connections 2024-09-10T21:25:18.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:18.182+0000 7f94ac400640 1 --2- 172.21.15.17:0/4061139739 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f94a415abb0 0x7f94a41613a0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:18.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:18.182+0000 7f94ac400640 1 --2- 172.21.15.17:0/4061139739 >> v2:172.21.15.31:3300/0 conn(0x7f94a4073cf0 0x7f94a40740d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:18.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:18.182+0000 7f94ac400640 1 -- 172.21.15.17:0/4061139739 >> 172.21.15.17:0/4061139739 conn(0x7f94a40700d0 msgr2=0x7f94a40704e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:18.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:18.182+0000 7f94ac400640 1 -- 172.21.15.17:0/4061139739 shutdown_connections 2024-09-10T21:25:18.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:18.182+0000 7f94ac400640 1 -- 172.21.15.17:0/4061139739 wait complete. 2024-09-10T21:25:18.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:18.186+0000 7f94ac400640 1 Processor -- start 2024-09-10T21:25:18.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:18.186+0000 7f94ac400640 1 -- start start 2024-09-10T21:25:18.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:18.186+0000 7f94ac400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f94a4073cf0 0x7f94a4153800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:18.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:18.186+0000 7f94ac400640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f94a40c0fd0 0x7f94a41f7210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:18.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:18.186+0000 7f94ac400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f94a415abb0 0x7f94a4148c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:18.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:18.186+0000 7f94ac400640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f94a40c7dd0 con 0x7f94a415abb0 2024-09-10T21:25:18.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:18.186+0000 7f94ac400640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f94a40c8370 con 0x7f94a4073cf0 2024-09-10T21:25:18.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:18.186+0000 7f94ac400640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f94a40c7f50 con 0x7f94a40c0fd0 2024-09-10T21:25:18.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:18.186+0000 7f94aa400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f94a415abb0 0x7f94a4148c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:18.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:18.186+0000 7f94aa400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f94a415abb0 0x7f94a4148c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44146/0 (socket says 172.21.15.17:44146) 2024-09-10T21:25:18.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:18.186+0000 7f94aa400640 1 -- 172.21.15.17:0/4077822956 learned_addr learned my addr 172.21.15.17:0/4077822956 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:25:18.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:18.186+0000 7f94a9a00640 1 --2- 172.21.15.17:0/4077822956 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f94a4073cf0 0x7f94a4153800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:18.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:18.186+0000 7f94a9000640 1 --2- 172.21.15.17:0/4077822956 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f94a40c0fd0 0x7f94a41f7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:18.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:18.186+0000 7f94aa400640 1 -- 172.21.15.17:0/4077822956 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f94a40c0fd0 msgr2=0x7f94a41f7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:18.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:18.186+0000 7f94aa400640 1 --2- 172.21.15.17:0/4077822956 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f94a40c0fd0 0x7f94a41f7210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:18.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:18.186+0000 7f94aa400640 1 -- 172.21.15.17:0/4077822956 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f94a4073cf0 msgr2=0x7f94a4153800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:18.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:18.186+0000 7f94aa400640 1 --2- 172.21.15.17:0/4077822956 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f94a4073cf0 0x7f94a4153800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:18.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:18.186+0000 7f94aa400640 1 -- 172.21.15.17:0/4077822956 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94a4155090 con 0x7f94a415abb0 2024-09-10T21:25:18.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:18.186+0000 7f94a9000640 1 --2- 172.21.15.17:0/4077822956 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f94a40c0fd0 0x7f94a41f7210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:25:18.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:18.186+0000 7f94aa400640 1 --2- 172.21.15.17:0/4077822956 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f94a415abb0 0x7f94a4148c70 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f949c00ac20 tx=0x7f949c00a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:18.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:18.186+0000 7f94a1600640 1 -- 172.21.15.17:0/4077822956 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f949c0bd070 con 0x7f94a415abb0 2024-09-10T21:25:18.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:18.186+0000 7f94a1600640 1 -- 172.21.15.17:0/4077822956 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f949c004d40 con 0x7f94a415abb0 2024-09-10T21:25:18.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:18.186+0000 7f94a1600640 1 -- 172.21.15.17:0/4077822956 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f949c0b3040 con 0x7f94a415abb0 2024-09-10T21:25:18.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:18.186+0000 7f94ac400640 1 -- 172.21.15.17:0/4077822956 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94a4155370 con 0x7f94a415abb0 2024-09-10T21:25:18.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:18.186+0000 7f94ac400640 1 -- 172.21.15.17:0/4077822956 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94a4155930 con 0x7f94a415abb0 2024-09-10T21:25:18.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:18.190+0000 7f94ac400640 1 -- 172.21.15.17:0/4077822956 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94a4151130 con 0x7f94a415abb0 2024-09-10T21:25:18.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:18.190+0000 7f94a1600640 1 -- 172.21.15.17:0/4077822956 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f949c00b040 con 0x7f94a415abb0 2024-09-10T21:25:18.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:18.190+0000 7f94a1600640 1 --2- 172.21.15.17:0/4077822956 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f94840725e0 0x7f9484074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:18.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:18.190+0000 7f94a1600640 1 -- 172.21.15.17:0/4077822956 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f949c12d1e0 con 0x7f94a415abb0 2024-09-10T21:25:18.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:18.190+0000 7f94a9a00640 1 --2- 172.21.15.17:0/4077822956 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f94840725e0 0x7f9484074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:18.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:18.190+0000 7f94a9a00640 1 --2- 172.21.15.17:0/4077822956 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f94840725e0 0x7f9484074aa0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f94980058d0 tx=0x7f9498005860 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:18.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:18.194+0000 7f94a1600640 1 -- 172.21.15.17:0/4077822956 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f949c017aa0 con 0x7f94a415abb0 2024-09-10T21:25:18.376 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:18.370+0000 7f94ac400640 1 -- 172.21.15.17:0/4077822956 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "config log", "num": 1, "format": "json"} v 0) -- 0x7f94a4074e50 con 0x7f94a415abb0 2024-09-10T21:25:18.376 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:18.374+0000 7f94a1600640 1 -- 172.21.15.17:0/4077822956 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "config log", "num": 1, "format": "json"}]=0 v32) ==== 86+0+160 (secure 0 0 0) 0x7f949c0f84e0 con 0x7f94a415abb0 2024-09-10T21:25:18.377 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-10T21:25:18.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:18.374+0000 7f94ac400640 1 -- 172.21.15.17:0/4077822956 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f94840725e0 msgr2=0x7f9484074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:18.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:18.374+0000 7f94ac400640 1 --2- 172.21.15.17:0/4077822956 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f94840725e0 0x7f9484074aa0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f94980058d0 tx=0x7f9498005860 comp rx=0 tx=0).stop 2024-09-10T21:25:18.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:18.374+0000 7f94ac400640 1 -- 172.21.15.17:0/4077822956 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f94a415abb0 msgr2=0x7f94a4148c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:18.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:18.374+0000 7f94ac400640 1 --2- 172.21.15.17:0/4077822956 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f94a415abb0 0x7f94a4148c70 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f949c00ac20 tx=0x7f949c00a410 comp rx=0 tx=0).stop 2024-09-10T21:25:18.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:18.374+0000 7f94ac400640 1 -- 172.21.15.17:0/4077822956 shutdown_connections 2024-09-10T21:25:18.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:18.374+0000 7f94ac400640 1 --2- 172.21.15.17:0/4077822956 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f94840725e0 0x7f9484074aa0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:18.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:18.374+0000 7f94ac400640 1 --2- 172.21.15.17:0/4077822956 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f94a415abb0 0x7f94a4148c70 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:18.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:18.374+0000 7f94ac400640 1 --2- 172.21.15.17:0/4077822956 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f94a40c0fd0 0x7f94a41f7210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:18.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:18.374+0000 7f94ac400640 1 --2- 172.21.15.17:0/4077822956 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f94a4073cf0 0x7f94a4153800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:18.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:18.374+0000 7f94ac400640 1 -- 172.21.15.17:0/4077822956 >> 172.21.15.17:0/4077822956 conn(0x7f94a40700d0 msgr2=0x7f94a414b910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:18.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:18.374+0000 7f94ac400640 1 -- 172.21.15.17:0/4077822956 shutdown_connections 2024-09-10T21:25:18.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:18.374+0000 7f94ac400640 1 -- 172.21.15.17:0/4077822956 wait complete. 2024-09-10T21:25:18.399 INFO:teuthology.orchestra.run.smithi017.stdout:[{"version":32,"timestamp":"2024-09-10T21:24:34.900120+0000","name":"","changes":[{"name":"osd.2/osd_mclock_max_capacity_iops_ssd","new_value":"5407.103450"}]}] 2024-09-10T21:25:18.399 INFO:tasks.ceph_manager:config epoch is 32 2024-09-10T21:25:18.399 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-10T21:25:18.400 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-09-10T21:25:18.400 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-09-10T21:25:18.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:18.542+0000 7f2240000640 1 -- 172.21.15.17:0/701758960 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f22380c7210 msgr2=0x7f2238157470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:18.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:18.542+0000 7f2240000640 1 --2- 172.21.15.17:0/701758960 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f22380c7210 0x7f2238157470 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f223815baf0 tx=0x7f222c01f090 comp rx=0 tx=0).stop 2024-09-10T21:25:18.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:18.542+0000 7f2240000640 1 -- 172.21.15.17:0/701758960 shutdown_connections 2024-09-10T21:25:18.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:18.542+0000 7f2240000640 1 --2- 172.21.15.17:0/701758960 >> v2:172.21.15.31:3300/0 conn(0x7f2238070450 0x7f223806e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:18.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:18.542+0000 7f2240000640 1 --2- 172.21.15.17:0/701758960 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f22380c7210 0x7f2238157470 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:18.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:18.542+0000 7f2240000640 1 -- 172.21.15.17:0/701758960 >> 172.21.15.17:0/701758960 conn(0x7f223806dd30 msgr2=0x7f223806e140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:18.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:18.542+0000 7f2240000640 1 -- 172.21.15.17:0/701758960 shutdown_connections 2024-09-10T21:25:18.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:18.542+0000 7f2240000640 1 -- 172.21.15.17:0/701758960 wait complete. 2024-09-10T21:25:18.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:18.542+0000 7f2240000640 1 Processor -- start 2024-09-10T21:25:18.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:18.542+0000 7f2240000640 1 -- start start 2024-09-10T21:25:18.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:18.546+0000 7f2240000640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f223806ed90 0x7f22381523f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:18.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:18.546+0000 7f2240000640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f2238070450 0x7f22380c8b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:18.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:18.546+0000 7f2240000640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f22380c7210 0x7f22380c90a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:18.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:18.546+0000 7f2240000640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f223815b690 con 0x7f22380c7210 2024-09-10T21:25:18.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:18.546+0000 7f2240000640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f223815b3b0 con 0x7f223806ed90 2024-09-10T21:25:18.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:18.546+0000 7f2240000640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f2238069b90 con 0x7f2238070450 2024-09-10T21:25:18.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:18.546+0000 7f223e000640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f22380c7210 0x7f22380c90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:18.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:18.546+0000 7f223e000640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f22380c7210 0x7f22380c90a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44174/0 (socket says 172.21.15.17:44174) 2024-09-10T21:25:18.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:18.546+0000 7f223e000640 1 -- 172.21.15.17:0/3211726990 learned_addr learned my addr 172.21.15.17:0/3211726990 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:25:18.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:18.546+0000 7f223d600640 1 --2- 172.21.15.17:0/3211726990 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f223806ed90 0x7f22381523f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:18.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:18.546+0000 7f223cc00640 1 --2- 172.21.15.17:0/3211726990 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f2238070450 0x7f22380c8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:18.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:18.546+0000 7f223e000640 1 -- 172.21.15.17:0/3211726990 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f2238070450 msgr2=0x7f22380c8b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:18.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:18.546+0000 7f223e000640 1 --2- 172.21.15.17:0/3211726990 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f2238070450 0x7f22380c8b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:18.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:18.546+0000 7f223e000640 1 -- 172.21.15.17:0/3211726990 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f223806ed90 msgr2=0x7f22381523f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:18.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:18.546+0000 7f223e000640 1 --2- 172.21.15.17:0/3211726990 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f223806ed90 0x7f22381523f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:18.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:18.546+0000 7f223e000640 1 -- 172.21.15.17:0/3211726990 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2238159ff0 con 0x7f22380c7210 2024-09-10T21:25:18.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:18.546+0000 7f223cc00640 1 --2- 172.21.15.17:0/3211726990 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f2238070450 0x7f22380c8b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:25:18.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:18.546+0000 7f223e000640 1 --2- 172.21.15.17:0/3211726990 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f22380c7210 0x7f22380c90a0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f223000c7f0 tx=0x7f223000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:18.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:18.546+0000 7f2235600640 1 -- 172.21.15.17:0/3211726990 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2230003b80 con 0x7f22380c7210 2024-09-10T21:25:18.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:18.546+0000 7f2235600640 1 -- 172.21.15.17:0/3211726990 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2230003d20 con 0x7f22380c7210 2024-09-10T21:25:18.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:18.546+0000 7f2235600640 1 -- 172.21.15.17:0/3211726990 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f223000aab0 con 0x7f22380c7210 2024-09-10T21:25:18.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:18.546+0000 7f2240000640 1 -- 172.21.15.17:0/3211726990 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f223815a780 con 0x7f22380c7210 2024-09-10T21:25:18.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:18.546+0000 7f2240000640 1 -- 172.21.15.17:0/3211726990 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22381f4ae0 con 0x7f22380c7210 2024-09-10T21:25:18.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:18.546+0000 7f2240000640 1 -- 172.21.15.17:0/3211726990 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22380c56a0 con 0x7f22380c7210 2024-09-10T21:25:18.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:18.550+0000 7f2235600640 1 -- 172.21.15.17:0/3211726990 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f223000ac50 con 0x7f22380c7210 2024-09-10T21:25:18.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:18.550+0000 7f2235600640 1 --2- 172.21.15.17:0/3211726990 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f22180725e0 0x7f2218074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:18.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:18.550+0000 7f2235600640 1 -- 172.21.15.17:0/3211726990 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f2230093400 con 0x7f22380c7210 2024-09-10T21:25:18.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:18.550+0000 7f223d600640 1 --2- 172.21.15.17:0/3211726990 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f22180725e0 0x7f2218074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:18.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:18.550+0000 7f223d600640 1 --2- 172.21.15.17:0/3211726990 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f22180725e0 0x7f2218074aa0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f222c0b2000 tx=0x7f222c01d8d0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:18.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:18.554+0000 7f2235600640 1 -- 172.21.15.17:0/3211726990 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f223005e790 con 0x7f22380c7210 2024-09-10T21:25:18.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:18.770+0000 7f2240000640 1 -- 172.21.15.17:0/3211726990 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f2238074ed0 con 0x7f22380c7210 2024-09-10T21:25:18.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:18.774+0000 7f2235600640 1 -- 172.21.15.17:0/3211726990 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) ==== 74+0+184984 (secure 0 0 0) 0x7f223005e090 con 0x7f22380c7210 2024-09-10T21:25:18.779 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-10T21:25:18.781 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:18.778+0000 7f2240000640 1 -- 172.21.15.17:0/3211726990 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f22180725e0 msgr2=0x7f2218074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:18.781 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:18.778+0000 7f2240000640 1 --2- 172.21.15.17:0/3211726990 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f22180725e0 0x7f2218074aa0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f222c0b2000 tx=0x7f222c01d8d0 comp rx=0 tx=0).stop 2024-09-10T21:25:18.781 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:18.778+0000 7f2240000640 1 -- 172.21.15.17:0/3211726990 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f22380c7210 msgr2=0x7f22380c90a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:18.781 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:18.778+0000 7f2240000640 1 --2- 172.21.15.17:0/3211726990 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f22380c7210 0x7f22380c90a0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f223000c7f0 tx=0x7f223000ccb0 comp rx=0 tx=0).stop 2024-09-10T21:25:18.781 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:18.778+0000 7f2240000640 1 -- 172.21.15.17:0/3211726990 shutdown_connections 2024-09-10T21:25:18.781 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:18.778+0000 7f2240000640 1 --2- 172.21.15.17:0/3211726990 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f22180725e0 0x7f2218074aa0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:18.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:18.778+0000 7f2240000640 1 --2- 172.21.15.17:0/3211726990 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f22380c7210 0x7f22380c90a0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:18.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:18.778+0000 7f2240000640 1 --2- 172.21.15.17:0/3211726990 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f2238070450 0x7f22380c8b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:18.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:18.778+0000 7f2240000640 1 --2- 172.21.15.17:0/3211726990 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f223806ed90 0x7f22381523f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:18.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:18.778+0000 7f2240000640 1 -- 172.21.15.17:0/3211726990 >> 172.21.15.17:0/3211726990 conn(0x7f223806dd30 msgr2=0x7f22380cc730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:18.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:18.778+0000 7f2240000640 1 -- 172.21.15.17:0/3211726990 shutdown_connections 2024-09-10T21:25:18.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:18.778+0000 7f2240000640 1 -- 172.21.15.17:0/3211726990 wait complete. 2024-09-10T21:25:18.801 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":5,"flags":0,"active_gid":4100,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6808","nonce":3101308586},{"type":"v1","addr":"172.21.15.31:6809","nonce":3101308586}]},"active_addr":"172.21.15.31:6809/3101308586","active_change":"2024-09-10T21:24:27.325355+0000","active_mgr_features":4540719139924082687,"available":true,"standbys":[{"gid":4111,"name":"y","mgr_features":4540719139924082687,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:1.26.1","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:1.26.1","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"force_disabled_modules":{},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.31:0","nonce":718123590}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.31:0","nonce":3626379860}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.31:0","nonce":3939910846}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.31:0","nonce":2981782162}]}]} 2024-09-10T21:25:18.806 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-09-10T21:25:18.806 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-10T21:25:18.806 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:25:18.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:18.950+0000 7f2eac600640 1 -- 172.21.15.17:0/706527076 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2e8c003fe0 msgr2=0x7f2e8c004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:18.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:18.950+0000 7f2eac600640 1 --2- 172.21.15.17:0/706527076 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2e8c003fe0 0x7f2e8c004810 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f2e9000c2e0 tx=0x7f2e9001cfa0 comp rx=0 tx=0).stop 2024-09-10T21:25:18.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:18.950+0000 7f2eaa600640 1 -- 172.21.15.17:0/706527076 reap_dead start 2024-09-10T21:25:18.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:18.950+0000 7f2eac600640 1 -- 172.21.15.17:0/706527076 shutdown_connections 2024-09-10T21:25:18.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:18.950+0000 7f2eac600640 1 -- 172.21.15.17:0/706527076 >> 172.21.15.17:0/706527076 conn(0x7f2ea406dd30 msgr2=0x7f2ea406e140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:18.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:18.950+0000 7f2eac600640 1 -- 172.21.15.17:0/706527076 shutdown_connections 2024-09-10T21:25:18.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:18.950+0000 7f2eac600640 1 -- 172.21.15.17:0/706527076 wait complete. 2024-09-10T21:25:18.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:18.954+0000 7f2eac600640 1 Processor -- start 2024-09-10T21:25:18.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:18.954+0000 7f2eac600640 1 -- start start 2024-09-10T21:25:18.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:18.954+0000 7f2eac600640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e8c003fe0 0x7f2ea41f22f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:18.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:18.954+0000 7f2eac600640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2ea41f2830 0x7f2ea40b03d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:18.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:18.954+0000 7f2eac600640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f2ea40ad9f0 0x7f2ea40b0910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:18.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:18.954+0000 7f2eac600640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f2ea415b3b0 con 0x7f2ea41f2830 2024-09-10T21:25:18.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:18.954+0000 7f2eac600640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f2ea415b0d0 con 0x7f2e8c003fe0 2024-09-10T21:25:18.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:18.954+0000 7f2eac600640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f2ea415b690 con 0x7f2ea40ad9f0 2024-09-10T21:25:18.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:18.954+0000 7f2ea9200640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2ea41f2830 0x7f2ea40b03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:18.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:18.954+0000 7f2ea9200640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2ea41f2830 0x7f2ea40b03d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44184/0 (socket says 172.21.15.17:44184) 2024-09-10T21:25:18.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:18.954+0000 7f2ea9200640 1 -- 172.21.15.17:0/798613211 learned_addr learned my addr 172.21.15.17:0/798613211 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:25:18.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:18.954+0000 7f2ea9c00640 1 --2- 172.21.15.17:0/798613211 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e8c003fe0 0x7f2ea41f22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:18.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:18.954+0000 7f2eaa600640 1 --2- 172.21.15.17:0/798613211 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f2ea40ad9f0 0x7f2ea40b0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:18.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:18.954+0000 7f2ea9200640 1 -- 172.21.15.17:0/798613211 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f2ea40ad9f0 msgr2=0x7f2ea40b0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:18.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:18.954+0000 7f2ea9200640 1 --2- 172.21.15.17:0/798613211 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f2ea40ad9f0 0x7f2ea40b0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:18.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:18.954+0000 7f2ea9200640 1 -- 172.21.15.17:0/798613211 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e8c003fe0 msgr2=0x7f2ea41f22f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:18.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:18.954+0000 7f2ea9200640 1 --2- 172.21.15.17:0/798613211 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e8c003fe0 0x7f2ea41f22f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:18.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:18.954+0000 7f2ea9200640 1 -- 172.21.15.17:0/798613211 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ea40c91d0 con 0x7f2ea41f2830 2024-09-10T21:25:18.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:18.954+0000 7f2ea9200640 1 --2- 172.21.15.17:0/798613211 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2ea41f2830 0x7f2ea40b03d0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f2e9001b6a0 tx=0x7f2e90006d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:18.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:18.954+0000 7f2ea1600640 1 -- 172.21.15.17:0/798613211 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e90003600 con 0x7f2ea41f2830 2024-09-10T21:25:18.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:18.954+0000 7f2ea1600640 1 -- 172.21.15.17:0/798613211 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2e90003890 con 0x7f2ea41f2830 2024-09-10T21:25:18.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:18.954+0000 7f2ea1600640 1 -- 172.21.15.17:0/798613211 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e90003cc0 con 0x7f2ea41f2830 2024-09-10T21:25:18.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:18.954+0000 7f2eac600640 1 -- 172.21.15.17:0/798613211 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ea40cc640 con 0x7f2ea41f2830 2024-09-10T21:25:18.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:18.954+0000 7f2eac600640 1 -- 172.21.15.17:0/798613211 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ea40af750 con 0x7f2ea41f2830 2024-09-10T21:25:18.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:18.958+0000 7f2eac600640 1 -- 172.21.15.17:0/798613211 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ea40c56a0 con 0x7f2ea41f2830 2024-09-10T21:25:18.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:18.958+0000 7f2ea1600640 1 -- 172.21.15.17:0/798613211 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f2e900c1450 con 0x7f2ea41f2830 2024-09-10T21:25:18.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:18.958+0000 7f2ea1600640 1 --2- 172.21.15.17:0/798613211 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f2e840725e0 0x7f2e84074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:18.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:18.958+0000 7f2ea1600640 1 -- 172.21.15.17:0/798613211 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f2e9012db80 con 0x7f2ea41f2830 2024-09-10T21:25:18.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:18.958+0000 7f2ea9c00640 1 --2- 172.21.15.17:0/798613211 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f2e840725e0 0x7f2e84074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:18.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:18.958+0000 7f2ea9c00640 1 --2- 172.21.15.17:0/798613211 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f2e840725e0 0x7f2e84074aa0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f2e98009b80 tx=0x7f2e980091e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:18.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:18.962+0000 7f2ea1600640 1 -- 172.21.15.17:0/798613211 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f2e90016d50 con 0x7f2ea41f2830 2024-09-10T21:25:19.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.138+0000 7f2eac600640 1 -- 172.21.15.17:0/798613211 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2ea4074ed0 con 0x7f2ea41f2830 2024-09-10T21:25:19.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.138+0000 7f2ea1600640 1 -- 172.21.15.17:0/798613211 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18562 (secure 0 0 0) 0x7f2e900f8fd0 con 0x7f2ea41f2830 2024-09-10T21:25:19.143 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-10T21:25:19.143 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":25,"fsid":"5e2f023f-d6a6-49f4-a256-f4807dce3739","created":"2024-09-10T21:24:21.750712+0000","modified":"2024-09-10T21:24:52.295803+0000","last_up_change":"2024-09-10T21:24:35.877801+0000","last_in_change":"2024-09-10T21:24:25.831127+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:24:35.362670+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T21:24:45.211689+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T21:24:46.209784+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T21:24:49.871300+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0f5fabc0-434e-4066-b430-233969a548f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":4105489867},{"type":"v1","addr":"172.21.15.17:6809","nonce":4105489867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":4105489867},{"type":"v1","addr":"172.21.15.17:6811","nonce":4105489867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":4105489867},{"type":"v1","addr":"172.21.15.17:6815","nonce":4105489867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":4105489867},{"type":"v1","addr":"172.21.15.17:6813","nonce":4105489867}]},"public_addr":"172.21.15.17:6809/4105489867","cluster_addr":"172.21.15.17:6811/4105489867","heartbeat_back_addr":"172.21.15.17:6815/4105489867","heartbeat_front_addr":"172.21.15.17:6813/4105489867","state":["exists","up"]},{"osd":1,"uuid":"f6691cb3-2a91-4c90-a9fd-ece3c11dd6eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":2599842027},{"type":"v1","addr":"172.21.15.17:6825","nonce":2599842027}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":2599842027},{"type":"v1","addr":"172.21.15.17:6827","nonce":2599842027}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":2599842027},{"type":"v1","addr":"172.21.15.17:6831","nonce":2599842027}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":2599842027},{"type":"v1","addr":"172.21.15.17:6829","nonce":2599842027}]},"public_addr":"172.21.15.17:6825/2599842027","cluster_addr":"172.21.15.17:6827/2599842027","heartbeat_back_addr":"172.21.15.17:6831/2599842027","heartbeat_front_addr":"172.21.15.17:6829/2599842027","state":["exists","up"]},{"osd":2,"uuid":"cb61adb0-a216-437b-9598-06fa7c84aa64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":1248243634},{"type":"v1","addr":"172.21.15.17:6817","nonce":1248243634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":1248243634},{"type":"v1","addr":"172.21.15.17:6819","nonce":1248243634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":1248243634},{"type":"v1","addr":"172.21.15.17:6823","nonce":1248243634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":1248243634},{"type":"v1","addr":"172.21.15.17:6821","nonce":1248243634}]},"public_addr":"172.21.15.17:6817/1248243634","cluster_addr":"172.21.15.17:6819/1248243634","heartbeat_back_addr":"172.21.15.17:6823/1248243634","heartbeat_front_addr":"172.21.15.17:6821/1248243634","state":["exists","up"]},{"osd":3,"uuid":"6d2da26a-e0bf-47dc-b020-af86e6bcfccc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":2314489522},{"type":"v1","addr":"172.21.15.17:6801","nonce":2314489522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":2314489522},{"type":"v1","addr":"172.21.15.17:6803","nonce":2314489522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":2314489522},{"type":"v1","addr":"172.21.15.17:6807","nonce":2314489522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":2314489522},{"type":"v1","addr":"172.21.15.17:6805","nonce":2314489522}]},"public_addr":"172.21.15.17:6801/2314489522","cluster_addr":"172.21.15.17:6803/2314489522","heartbeat_back_addr":"172.21.15.17:6807/2314489522","heartbeat_front_addr":"172.21.15.17:6805/2314489522","state":["exists","up"]},{"osd":4,"uuid":"c65071dd-9ca7-4f1d-a584-45dc121419a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":202520744},{"type":"v1","addr":"172.21.15.31:6811","nonce":202520744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":202520744},{"type":"v1","addr":"172.21.15.31:6813","nonce":202520744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":202520744},{"type":"v1","addr":"172.21.15.31:6817","nonce":202520744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":202520744},{"type":"v1","addr":"172.21.15.31:6815","nonce":202520744}]},"public_addr":"172.21.15.31:6811/202520744","cluster_addr":"172.21.15.31:6813/202520744","heartbeat_back_addr":"172.21.15.31:6817/202520744","heartbeat_front_addr":"172.21.15.31:6815/202520744","state":["exists","up"]},{"osd":5,"uuid":"235389de-7777-4744-81b8-d0bfc11190e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":1472711057},{"type":"v1","addr":"172.21.15.31:6801","nonce":1472711057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":1472711057},{"type":"v1","addr":"172.21.15.31:6803","nonce":1472711057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":1472711057},{"type":"v1","addr":"172.21.15.31:6807","nonce":1472711057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":1472711057},{"type":"v1","addr":"172.21.15.31:6805","nonce":1472711057}]},"public_addr":"172.21.15.31:6801/1472711057","cluster_addr":"172.21.15.31:6803/1472711057","heartbeat_back_addr":"172.21.15.31:6807/1472711057","heartbeat_front_addr":"172.21.15.31:6805/1472711057","state":["exists","up"]},{"osd":6,"uuid":"cea70a57-3715-4ebb-8511-a528a9df2b1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":3754251089},{"type":"v1","addr":"172.21.15.31:6827","nonce":3754251089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":3754251089},{"type":"v1","addr":"172.21.15.31:6829","nonce":3754251089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":3754251089},{"type":"v1","addr":"172.21.15.31:6833","nonce":3754251089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":3754251089},{"type":"v1","addr":"172.21.15.31:6831","nonce":3754251089}]},"public_addr":"172.21.15.31:6827/3754251089","cluster_addr":"172.21.15.31:6829/3754251089","heartbeat_back_addr":"172.21.15.31:6833/3754251089","heartbeat_front_addr":"172.21.15.31:6831/3754251089","state":["exists","up"]},{"osd":7,"uuid":"01cc111b-a45f-4ca3-9cc1-7fe0afbeba45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":3948648934},{"type":"v1","addr":"172.21.15.31:6819","nonce":3948648934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":3948648934},{"type":"v1","addr":"172.21.15.31:6821","nonce":3948648934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":3948648934},{"type":"v1","addr":"172.21.15.31:6825","nonce":3948648934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":3948648934},{"type":"v1","addr":"172.21.15.31:6823","nonce":3948648934}]},"public_addr":"172.21.15.31:6819/3948648934","cluster_addr":"172.21.15.31:6821/3948648934","heartbeat_back_addr":"172.21.15.31:6825/3948648934","heartbeat_front_addr":"172.21.15.31:6823/3948648934","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:24:32.606989+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:24:32.891535+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:24:33.330793+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:24:32.811533+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:24:33.894624+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:24:32.773817+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:24:32.901865+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:24:32.907207+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:25:19.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.142+0000 7f2eac600640 1 -- 172.21.15.17:0/798613211 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f2e840725e0 msgr2=0x7f2e84074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:19.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.142+0000 7f2eac600640 1 --2- 172.21.15.17:0/798613211 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f2e840725e0 0x7f2e84074aa0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f2e98009b80 tx=0x7f2e980091e0 comp rx=0 tx=0).stop 2024-09-10T21:25:19.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.142+0000 7f2eac600640 1 -- 172.21.15.17:0/798613211 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2ea41f2830 msgr2=0x7f2ea40b03d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:19.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.142+0000 7f2eac600640 1 --2- 172.21.15.17:0/798613211 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2ea41f2830 0x7f2ea40b03d0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f2e9001b6a0 tx=0x7f2e90006d00 comp rx=0 tx=0).stop 2024-09-10T21:25:19.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.142+0000 7f2eac600640 1 -- 172.21.15.17:0/798613211 shutdown_connections 2024-09-10T21:25:19.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.142+0000 7f2eac600640 1 --2- 172.21.15.17:0/798613211 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f2e840725e0 0x7f2e84074aa0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:19.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.142+0000 7f2eac600640 1 --2- 172.21.15.17:0/798613211 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f2ea40ad9f0 0x7f2ea40b0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:19.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.142+0000 7f2eac600640 1 --2- 172.21.15.17:0/798613211 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2ea41f2830 0x7f2ea40b03d0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:19.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.142+0000 7f2eac600640 1 --2- 172.21.15.17:0/798613211 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2e8c003fe0 0x7f2ea41f22f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:19.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.142+0000 7f2eac600640 1 -- 172.21.15.17:0/798613211 >> 172.21.15.17:0/798613211 conn(0x7f2ea406dd30 msgr2=0x7f2ea41570e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:19.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.142+0000 7f2eac600640 1 -- 172.21.15.17:0/798613211 shutdown_connections 2024-09-10T21:25:19.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.142+0000 7f2eac600640 1 -- 172.21.15.17:0/798613211 wait complete. 2024-09-10T21:25:19.165 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-10T21:25:19.165 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:25:19.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.306+0000 7f2668a00640 1 -- 172.21.15.17:0/4258143728 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2648004810 msgr2=0x7f2648004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:19.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.306+0000 7f2668a00640 1 --2- 172.21.15.17:0/4258143728 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2648004810 0x7f2648004c90 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f264c009b60 tx=0x7f264c01d240 comp rx=0 tx=0).stop 2024-09-10T21:25:19.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.306+0000 7f2662800640 1 -- 172.21.15.17:0/4258143728 reap_dead start 2024-09-10T21:25:19.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.306+0000 7f2668a00640 1 -- 172.21.15.17:0/4258143728 shutdown_connections 2024-09-10T21:25:19.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.306+0000 7f2668a00640 1 -- 172.21.15.17:0/4258143728 >> 172.21.15.17:0/4258143728 conn(0x7f26640702f0 msgr2=0x7f2664070700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:19.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.306+0000 7f2668a00640 1 -- 172.21.15.17:0/4258143728 shutdown_connections 2024-09-10T21:25:19.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.306+0000 7f2668a00640 1 -- 172.21.15.17:0/4258143728 wait complete. 2024-09-10T21:25:19.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.310+0000 7f2668a00640 1 Processor -- start 2024-09-10T21:25:19.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.310+0000 7f2668a00640 1 -- start start 2024-09-10T21:25:19.316 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.310+0000 7f2668a00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2648004810 0x7f26640c58e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:19.316 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.310+0000 7f2668a00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f266414d540 0x7f2664156f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:19.316 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.310+0000 7f2668a00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f26641574c0 0x7f266414dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:19.316 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.310+0000 7f2668a00640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f26640ce740 con 0x7f266414d540 2024-09-10T21:25:19.316 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.310+0000 7f2668a00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f26640b42c0 con 0x7f2648004810 2024-09-10T21:25:19.316 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.310+0000 7f2668a00640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f26640cb450 con 0x7f26641574c0 2024-09-10T21:25:19.316 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.310+0000 7f2659400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f266414d540 0x7f2664156f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:19.316 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.310+0000 7f2659400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f266414d540 0x7f2664156f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44204/0 (socket says 172.21.15.17:44204) 2024-09-10T21:25:19.316 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.310+0000 7f2659400640 1 -- 172.21.15.17:0/556118120 learned_addr learned my addr 172.21.15.17:0/556118120 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:25:19.316 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.310+0000 7f2661e00640 1 --2- 172.21.15.17:0/556118120 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2648004810 0x7f26640c58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:19.316 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.310+0000 7f2662800640 1 --2- 172.21.15.17:0/556118120 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f26641574c0 0x7f266414dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:19.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.310+0000 7f2659400640 1 -- 172.21.15.17:0/556118120 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f26641574c0 msgr2=0x7f266414dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:19.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.310+0000 7f2659400640 1 --2- 172.21.15.17:0/556118120 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f26641574c0 0x7f266414dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:19.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.310+0000 7f2659400640 1 -- 172.21.15.17:0/556118120 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2648004810 msgr2=0x7f26640c58e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:19.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.310+0000 7f2659400640 1 --2- 172.21.15.17:0/556118120 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2648004810 0x7f26640c58e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:19.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.310+0000 7f2659400640 1 -- 172.21.15.17:0/556118120 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26640c6f10 con 0x7f266414d540 2024-09-10T21:25:19.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.310+0000 7f2662800640 1 --2- 172.21.15.17:0/556118120 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f26641574c0 0x7f266414dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:25:19.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.310+0000 7f2661e00640 1 --2- 172.21.15.17:0/556118120 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2648004810 0x7f26640c58e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:25:19.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.310+0000 7f2659400640 1 --2- 172.21.15.17:0/556118120 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f266414d540 0x7f2664156f80 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f264c0af040 tx=0x7f264c005620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:19.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.310+0000 7f265aa00640 1 -- 172.21.15.17:0/556118120 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f264c0bd070 con 0x7f266414d540 2024-09-10T21:25:19.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.310+0000 7f265aa00640 1 -- 172.21.15.17:0/556118120 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f264c0ac8a0 con 0x7f266414d540 2024-09-10T21:25:19.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.310+0000 7f265aa00640 1 -- 172.21.15.17:0/556118120 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f264c0b1040 con 0x7f266414d540 2024-09-10T21:25:19.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.310+0000 7f2668a00640 1 -- 172.21.15.17:0/556118120 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26641f56a0 con 0x7f266414d540 2024-09-10T21:25:19.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.310+0000 7f2668a00640 1 -- 172.21.15.17:0/556118120 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26641f5970 con 0x7f266414d540 2024-09-10T21:25:19.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.314+0000 7f2668a00640 1 -- 172.21.15.17:0/556118120 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26640c2ff0 con 0x7f266414d540 2024-09-10T21:25:19.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.314+0000 7f265aa00640 1 -- 172.21.15.17:0/556118120 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f264c0046d0 con 0x7f266414d540 2024-09-10T21:25:19.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.314+0000 7f265aa00640 1 --2- 172.21.15.17:0/556118120 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f2640072610 0x7f2640074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:19.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.314+0000 7f265aa00640 1 -- 172.21.15.17:0/556118120 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f264c12dbd0 con 0x7f266414d540 2024-09-10T21:25:19.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.314+0000 7f2661e00640 1 --2- 172.21.15.17:0/556118120 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f2640072610 0x7f2640074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:19.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.314+0000 7f2661e00640 1 --2- 172.21.15.17:0/556118120 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f2640072610 0x7f2640074ad0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f2654003980 tx=0x7f2654001d20 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:19.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.318+0000 7f265aa00640 1 -- 172.21.15.17:0/556118120 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f264c016f80 con 0x7f266414d540 2024-09-10T21:25:19.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.494+0000 7f2668a00640 1 -- 172.21.15.17:0/556118120 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2664155c60 con 0x7f266414d540 2024-09-10T21:25:19.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.494+0000 7f265aa00640 1 -- 172.21.15.17:0/556118120 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18562 (secure 0 0 0) 0x7f264c0f87f0 con 0x7f266414d540 2024-09-10T21:25:19.500 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-10T21:25:19.500 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":25,"fsid":"5e2f023f-d6a6-49f4-a256-f4807dce3739","created":"2024-09-10T21:24:21.750712+0000","modified":"2024-09-10T21:24:52.295803+0000","last_up_change":"2024-09-10T21:24:35.877801+0000","last_in_change":"2024-09-10T21:24:25.831127+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:24:35.362670+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T21:24:45.211689+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T21:24:46.209784+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T21:24:49.871300+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0f5fabc0-434e-4066-b430-233969a548f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":4105489867},{"type":"v1","addr":"172.21.15.17:6809","nonce":4105489867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":4105489867},{"type":"v1","addr":"172.21.15.17:6811","nonce":4105489867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":4105489867},{"type":"v1","addr":"172.21.15.17:6815","nonce":4105489867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":4105489867},{"type":"v1","addr":"172.21.15.17:6813","nonce":4105489867}]},"public_addr":"172.21.15.17:6809/4105489867","cluster_addr":"172.21.15.17:6811/4105489867","heartbeat_back_addr":"172.21.15.17:6815/4105489867","heartbeat_front_addr":"172.21.15.17:6813/4105489867","state":["exists","up"]},{"osd":1,"uuid":"f6691cb3-2a91-4c90-a9fd-ece3c11dd6eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":2599842027},{"type":"v1","addr":"172.21.15.17:6825","nonce":2599842027}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":2599842027},{"type":"v1","addr":"172.21.15.17:6827","nonce":2599842027}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":2599842027},{"type":"v1","addr":"172.21.15.17:6831","nonce":2599842027}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":2599842027},{"type":"v1","addr":"172.21.15.17:6829","nonce":2599842027}]},"public_addr":"172.21.15.17:6825/2599842027","cluster_addr":"172.21.15.17:6827/2599842027","heartbeat_back_addr":"172.21.15.17:6831/2599842027","heartbeat_front_addr":"172.21.15.17:6829/2599842027","state":["exists","up"]},{"osd":2,"uuid":"cb61adb0-a216-437b-9598-06fa7c84aa64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":1248243634},{"type":"v1","addr":"172.21.15.17:6817","nonce":1248243634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":1248243634},{"type":"v1","addr":"172.21.15.17:6819","nonce":1248243634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":1248243634},{"type":"v1","addr":"172.21.15.17:6823","nonce":1248243634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":1248243634},{"type":"v1","addr":"172.21.15.17:6821","nonce":1248243634}]},"public_addr":"172.21.15.17:6817/1248243634","cluster_addr":"172.21.15.17:6819/1248243634","heartbeat_back_addr":"172.21.15.17:6823/1248243634","heartbeat_front_addr":"172.21.15.17:6821/1248243634","state":["exists","up"]},{"osd":3,"uuid":"6d2da26a-e0bf-47dc-b020-af86e6bcfccc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":2314489522},{"type":"v1","addr":"172.21.15.17:6801","nonce":2314489522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":2314489522},{"type":"v1","addr":"172.21.15.17:6803","nonce":2314489522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":2314489522},{"type":"v1","addr":"172.21.15.17:6807","nonce":2314489522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":2314489522},{"type":"v1","addr":"172.21.15.17:6805","nonce":2314489522}]},"public_addr":"172.21.15.17:6801/2314489522","cluster_addr":"172.21.15.17:6803/2314489522","heartbeat_back_addr":"172.21.15.17:6807/2314489522","heartbeat_front_addr":"172.21.15.17:6805/2314489522","state":["exists","up"]},{"osd":4,"uuid":"c65071dd-9ca7-4f1d-a584-45dc121419a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":202520744},{"type":"v1","addr":"172.21.15.31:6811","nonce":202520744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":202520744},{"type":"v1","addr":"172.21.15.31:6813","nonce":202520744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":202520744},{"type":"v1","addr":"172.21.15.31:6817","nonce":202520744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":202520744},{"type":"v1","addr":"172.21.15.31:6815","nonce":202520744}]},"public_addr":"172.21.15.31:6811/202520744","cluster_addr":"172.21.15.31:6813/202520744","heartbeat_back_addr":"172.21.15.31:6817/202520744","heartbeat_front_addr":"172.21.15.31:6815/202520744","state":["exists","up"]},{"osd":5,"uuid":"235389de-7777-4744-81b8-d0bfc11190e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":1472711057},{"type":"v1","addr":"172.21.15.31:6801","nonce":1472711057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":1472711057},{"type":"v1","addr":"172.21.15.31:6803","nonce":1472711057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":1472711057},{"type":"v1","addr":"172.21.15.31:6807","nonce":1472711057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":1472711057},{"type":"v1","addr":"172.21.15.31:6805","nonce":1472711057}]},"public_addr":"172.21.15.31:6801/1472711057","cluster_addr":"172.21.15.31:6803/1472711057","heartbeat_back_addr":"172.21.15.31:6807/1472711057","heartbeat_front_addr":"172.21.15.31:6805/1472711057","state":["exists","up"]},{"osd":6,"uuid":"cea70a57-3715-4ebb-8511-a528a9df2b1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":3754251089},{"type":"v1","addr":"172.21.15.31:6827","nonce":3754251089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":3754251089},{"type":"v1","addr":"172.21.15.31:6829","nonce":3754251089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":3754251089},{"type":"v1","addr":"172.21.15.31:6833","nonce":3754251089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":3754251089},{"type":"v1","addr":"172.21.15.31:6831","nonce":3754251089}]},"public_addr":"172.21.15.31:6827/3754251089","cluster_addr":"172.21.15.31:6829/3754251089","heartbeat_back_addr":"172.21.15.31:6833/3754251089","heartbeat_front_addr":"172.21.15.31:6831/3754251089","state":["exists","up"]},{"osd":7,"uuid":"01cc111b-a45f-4ca3-9cc1-7fe0afbeba45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":3948648934},{"type":"v1","addr":"172.21.15.31:6819","nonce":3948648934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":3948648934},{"type":"v1","addr":"172.21.15.31:6821","nonce":3948648934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":3948648934},{"type":"v1","addr":"172.21.15.31:6825","nonce":3948648934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":3948648934},{"type":"v1","addr":"172.21.15.31:6823","nonce":3948648934}]},"public_addr":"172.21.15.31:6819/3948648934","cluster_addr":"172.21.15.31:6821/3948648934","heartbeat_back_addr":"172.21.15.31:6825/3948648934","heartbeat_front_addr":"172.21.15.31:6823/3948648934","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:24:32.606989+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:24:32.891535+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:24:33.330793+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:24:32.811533+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:24:33.894624+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:24:32.773817+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:24:32.901865+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:24:32.907207+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:25:19.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.498+0000 7f2668a00640 1 -- 172.21.15.17:0/556118120 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f2640072610 msgr2=0x7f2640074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:19.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.498+0000 7f2668a00640 1 --2- 172.21.15.17:0/556118120 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f2640072610 0x7f2640074ad0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f2654003980 tx=0x7f2654001d20 comp rx=0 tx=0).stop 2024-09-10T21:25:19.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.498+0000 7f2668a00640 1 -- 172.21.15.17:0/556118120 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f266414d540 msgr2=0x7f2664156f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:19.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.498+0000 7f2668a00640 1 --2- 172.21.15.17:0/556118120 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f266414d540 0x7f2664156f80 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f264c0af040 tx=0x7f264c005620 comp rx=0 tx=0).stop 2024-09-10T21:25:19.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.498+0000 7f2668a00640 1 -- 172.21.15.17:0/556118120 shutdown_connections 2024-09-10T21:25:19.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.498+0000 7f2668a00640 1 --2- 172.21.15.17:0/556118120 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f2640072610 0x7f2640074ad0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:19.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.498+0000 7f2668a00640 1 --2- 172.21.15.17:0/556118120 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f26641574c0 0x7f266414dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:19.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.498+0000 7f2668a00640 1 --2- 172.21.15.17:0/556118120 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f266414d540 0x7f2664156f80 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:19.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.498+0000 7f2668a00640 1 --2- 172.21.15.17:0/556118120 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2648004810 0x7f26640c58e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:19.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.498+0000 7f2668a00640 1 -- 172.21.15.17:0/556118120 >> 172.21.15.17:0/556118120 conn(0x7f26640702f0 msgr2=0x7f26640c3200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:19.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.498+0000 7f2668a00640 1 -- 172.21.15.17:0/556118120 shutdown_connections 2024-09-10T21:25:19.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.498+0000 7f2668a00640 1 -- 172.21.15.17:0/556118120 wait complete. 2024-09-10T21:25:19.522 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-10T21:25:19.523 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-10T21:25:19.523 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-10T21:25:19.523 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-10T21:25:19.524 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-10T21:25:19.524 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-10T21:25:19.525 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-09-10T21:25:19.525 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-09-10T21:25:19.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.774+0000 7fceae800640 1 -- 172.21.15.17:0/3341203200 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcea00d6300 msgr2=0x7fcea00dc2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:19.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.774+0000 7fceae800640 1 --2- 172.21.15.17:0/3341203200 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcea00d6300 0x7fcea00dc2a0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fce9c009900 tx=0x7fce9c01f0d0 comp rx=0 tx=0).stop 2024-09-10T21:25:19.780 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.778+0000 7fceae800640 1 -- 172.21.15.17:0/3341203200 shutdown_connections 2024-09-10T21:25:19.781 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.778+0000 7fceae800640 1 --2- 172.21.15.17:0/3341203200 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fcea004db70 0x7fcea00521f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:19.781 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.778+0000 7fceae800640 1 --2- 172.21.15.17:0/3341203200 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcea00d6300 0x7fcea00dc2a0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:19.781 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.778+0000 7fceae800640 1 -- 172.21.15.17:0/3341203200 >> 172.21.15.17:0/3341203200 conn(0x7fcea000e420 msgr2=0x7fcea000ba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:19.781 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.778+0000 7fceae800640 1 -- 172.21.15.17:0/3341203200 shutdown_connections 2024-09-10T21:25:19.784 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.778+0000 7fceae800640 1 -- 172.21.15.17:0/3341203200 wait complete. 2024-09-10T21:25:19.784 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.778+0000 7fceae800640 1 Processor -- start 2024-09-10T21:25:19.785 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.782+0000 7fceae800640 1 -- start start 2024-09-10T21:25:19.785 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.782+0000 7f448be00640 1 -- 172.21.15.17:0/4142155696 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f44700039c0 msgr2=0x7f4470003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:19.785 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.782+0000 7f448be00640 1 --2- 172.21.15.17:0/4142155696 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f44700039c0 0x7f4470003de0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f447c00ab00 tx=0x7f447c01cb10 comp rx=0 tx=0).stop 2024-09-10T21:25:19.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.782+0000 7f448be00640 1 -- 172.21.15.17:0/4142155696 shutdown_connections 2024-09-10T21:25:19.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.782+0000 7f448be00640 1 --2- 172.21.15.17:0/4142155696 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f44700039c0 0x7f4470003de0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:19.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.782+0000 7f448be00640 1 -- 172.21.15.17:0/4142155696 >> 172.21.15.17:0/4142155696 conn(0x7f4484054000 msgr2=0x7f4484054410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:19.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.782+0000 7f448be00640 1 -- 172.21.15.17:0/4142155696 shutdown_connections 2024-09-10T21:25:19.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.782+0000 7f448be00640 1 -- 172.21.15.17:0/4142155696 wait complete. 2024-09-10T21:25:19.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.782+0000 7fceae800640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcea004db70 0x7fcea00cf3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:19.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.782+0000 7fceae800640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcea0051da0 0x7fcea0181680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:19.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.782+0000 7fceae800640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fcea00d6300 0x7fcea0176ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:19.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.782+0000 7fceae800640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fcea00d66e0 con 0x7fcea004db70 2024-09-10T21:25:19.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.782+0000 7fceae800640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fcea004cf40 con 0x7fcea0051da0 2024-09-10T21:25:19.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.782+0000 7fceae800640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7fcea00df890 con 0x7fcea00d6300 2024-09-10T21:25:19.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.782+0000 7fcead400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcea004db70 0x7fcea00cf3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:19.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.782+0000 7fceaca00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcea0051da0 0x7fcea0181680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:19.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.782+0000 7fceaca00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcea0051da0 0x7fcea0181680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.17:51326/0 (socket says 172.21.15.17:51326) 2024-09-10T21:25:19.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.782+0000 7fceaca00640 1 -- 172.21.15.17:0/3392999051 learned_addr learned my addr 172.21.15.17:0/3392999051 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:25:19.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.782+0000 7fceade00640 1 --2- 172.21.15.17:0/3392999051 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fcea00d6300 0x7fcea0176ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:19.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.782+0000 7fceaca00640 1 -- 172.21.15.17:0/3392999051 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fcea00d6300 msgr2=0x7fcea0176ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:19.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.782+0000 7fceaca00640 1 --2- 172.21.15.17:0/3392999051 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fcea00d6300 0x7fcea0176ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:19.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.782+0000 7fceaca00640 1 -- 172.21.15.17:0/3392999051 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcea004db70 msgr2=0x7fcea00cf3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:19.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.782+0000 7fceaca00640 1 --2- 172.21.15.17:0/3392999051 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcea004db70 0x7fcea00cf3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:19.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.782+0000 7fceaca00640 1 -- 172.21.15.17:0/3392999051 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcea00cf8f0 con 0x7fcea0051da0 2024-09-10T21:25:19.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.786+0000 7fceaca00640 1 --2- 172.21.15.17:0/3392999051 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcea0051da0 0x7fcea0181680 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fce9400c4b0 tx=0x7fce9400c970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:19.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.786+0000 7fcea5600640 1 -- 172.21.15.17:0/3392999051 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce940052d0 con 0x7fcea0051da0 2024-09-10T21:25:19.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.786+0000 7fcea5600640 1 -- 172.21.15.17:0/3392999051 <== mon.1 v2:172.21.15.31:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fce94005470 con 0x7fcea0051da0 2024-09-10T21:25:19.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.786+0000 7fcea5600640 1 -- 172.21.15.17:0/3392999051 <== mon.1 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce94012870 con 0x7fcea0051da0 2024-09-10T21:25:19.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.786+0000 7fceae800640 1 -- 172.21.15.17:0/3392999051 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcea0172750 con 0x7fcea0051da0 2024-09-10T21:25:19.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.786+0000 7fceae800640 1 -- 172.21.15.17:0/3392999051 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcea0172d10 con 0x7fcea0051da0 2024-09-10T21:25:19.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.786+0000 7f448be00640 1 Processor -- start 2024-09-10T21:25:19.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.786+0000 7f448be00640 1 -- start start 2024-09-10T21:25:19.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.786+0000 7fceae800640 1 -- 172.21.15.17:0/3392999051 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fcea0052f80 con 0x7fcea0051da0 2024-09-10T21:25:19.792 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.786+0000 7f514f000640 1 -- 172.21.15.17:0/3936384214 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f51481548b0 msgr2=0x7f514815afb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:19.792 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.786+0000 7f514f000640 1 --2- 172.21.15.17:0/3936384214 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f51481548b0 0x7f514815afb0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f5138009840 tx=0x7f513801f010 comp rx=0 tx=0).stop 2024-09-10T21:25:19.792 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.790+0000 7f9c6b200640 1 -- 172.21.15.17:0/314613312 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9c64153d40 msgr2=0x7f9c640c8010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:19.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.790+0000 7f514f000640 1 -- 172.21.15.17:0/3936384214 shutdown_connections 2024-09-10T21:25:19.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.790+0000 7f514f000640 1 --2- 172.21.15.17:0/3936384214 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f51481548b0 0x7f514815afb0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:19.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.790+0000 7f514f000640 1 -- 172.21.15.17:0/3936384214 >> 172.21.15.17:0/3936384214 conn(0x7f51480700d0 msgr2=0x7f51480704e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:19.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.790+0000 7f9c6b200640 1 --2- 172.21.15.17:0/314613312 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9c64153d40 0x7f9c640c8010 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f9c64073750 tx=0x7f9c5401fc60 comp rx=0 tx=0).stop 2024-09-10T21:25:19.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.790+0000 7f448be00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f44700039c0 0x7f44840534d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:19.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.790+0000 7fcea5600640 1 -- 172.21.15.17:0/3392999051 <== mon.1 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fce94039b80 con 0x7fcea0051da0 2024-09-10T21:25:19.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.790+0000 7f9c6b200640 1 -- 172.21.15.17:0/314613312 shutdown_connections 2024-09-10T21:25:19.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.790+0000 7f9c6b200640 1 --2- 172.21.15.17:0/314613312 >> v2:172.21.15.31:3300/0 conn(0x7f9c6406dd10 0x7f9c6406e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:19.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.790+0000 7f9c6b200640 1 --2- 172.21.15.17:0/314613312 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9c64153d40 0x7f9c640c8010 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:19.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.790+0000 7f9c6b200640 1 -- 172.21.15.17:0/314613312 >> 172.21.15.17:0/314613312 conn(0x7f9c640700d0 msgr2=0x7f9c640704e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:19.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.790+0000 7f448be00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f4484175db0 0x7f44841761f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:19.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.790+0000 7f448be00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4484052040 0x7f44840524a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:19.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.790+0000 7f448aa00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f44700039c0 0x7f44840534d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:19.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.790+0000 7f448aa00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f44700039c0 0x7f44840534d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44274/0 (socket says 172.21.15.17:44274) 2024-09-10T21:25:19.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.790+0000 7f448aa00640 1 -- 172.21.15.17:0/1053768786 learned_addr learned my addr 172.21.15.17:0/1053768786 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:25:19.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.790+0000 7f448be00640 1 -- 172.21.15.17:0/1053768786 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f44840e1b20 con 0x7f44700039c0 2024-09-10T21:25:19.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.790+0000 7f448be00640 1 -- 172.21.15.17:0/1053768786 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f4484039da0 con 0x7f4484052040 2024-09-10T21:25:19.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.790+0000 7f514f000640 1 -- 172.21.15.17:0/3936384214 shutdown_connections 2024-09-10T21:25:19.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.790+0000 7f514f000640 1 -- 172.21.15.17:0/3936384214 wait complete. 2024-09-10T21:25:19.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.790+0000 7f448be00640 1 -- 172.21.15.17:0/1053768786 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f44840e1ca0 con 0x7f4484175db0 2024-09-10T21:25:19.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.790+0000 7f448a000640 1 --2- 172.21.15.17:0/1053768786 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f4484175db0 0x7f44841761f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:19.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.790+0000 7f448b400640 1 --2- 172.21.15.17:0/1053768786 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4484052040 0x7f44840524a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:19.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.790+0000 7fcea5600640 1 --2- 172.21.15.17:0/3392999051 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fce880725e0 0x7fce88074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:19.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.790+0000 7fcea5600640 1 -- 172.21.15.17:0/3392999051 <== mon.1 v2:172.21.15.31:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fce94093380 con 0x7fcea0051da0 2024-09-10T21:25:19.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.790+0000 7fcea5600640 1 --2- 172.21.15.17:0/3392999051 >> [v2:172.21.15.31:6826/3754251089,v1:172.21.15.31:6827/3754251089] conn(0x7fce8807bd50 0x7fce8807e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:19.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.790+0000 7fcea5600640 1 -- 172.21.15.17:0/3392999051 --> [v2:172.21.15.31:6826/3754251089,v1:172.21.15.31:6827/3754251089] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fce8807e860 con 0x7fce8807bd50 2024-09-10T21:25:19.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.790+0000 7fcea5600640 1 -- 172.21.15.17:0/3392999051 <== mon.1 v2:172.21.15.31:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fce94093770 con 0x7fcea0051da0 2024-09-10T21:25:19.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.790+0000 7f448b400640 1 -- 172.21.15.17:0/1053768786 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f4484175db0 msgr2=0x7f44841761f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:19.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.790+0000 7f448b400640 1 --2- 172.21.15.17:0/1053768786 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f4484175db0 0x7f44841761f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:19.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.790+0000 7f448b400640 1 -- 172.21.15.17:0/1053768786 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f44700039c0 msgr2=0x7f44840534d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:19.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.790+0000 7f448b400640 1 --2- 172.21.15.17:0/1053768786 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f44700039c0 0x7f44840534d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:19.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.790+0000 7fceade00640 1 --2- 172.21.15.17:0/3392999051 >> [v2:172.21.15.31:6826/3754251089,v1:172.21.15.31:6827/3754251089] conn(0x7fce8807bd50 0x7fce8807e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:19.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.790+0000 7f448b400640 1 -- 172.21.15.17:0/1053768786 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4484053a10 con 0x7f4484052040 2024-09-10T21:25:19.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.790+0000 7f448a000640 1 --2- 172.21.15.17:0/1053768786 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f4484175db0 0x7f44841761f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:25:19.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.790+0000 7f448b400640 1 --2- 172.21.15.17:0/1053768786 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4484052040 0x7f44840524a0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f448c06ba20 tx=0x7f448c06c890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:19.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.790+0000 7f9c6b200640 1 -- 172.21.15.17:0/314613312 shutdown_connections 2024-09-10T21:25:19.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.790+0000 7f9c6b200640 1 -- 172.21.15.17:0/314613312 wait complete. 2024-09-10T21:25:19.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.794+0000 7f4482a00640 1 -- 172.21.15.17:0/1053768786 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f448c092070 con 0x7f4484052040 2024-09-10T21:25:19.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.794+0000 7f448be00640 1 -- 172.21.15.17:0/1053768786 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44841752f0 con 0x7f4484052040 2024-09-10T21:25:19.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.794+0000 7f4482a00640 1 -- 172.21.15.17:0/1053768786 <== mon.1 v2:172.21.15.31:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f448c051e80 con 0x7f4484052040 2024-09-10T21:25:19.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.794+0000 7f448be00640 1 -- 172.21.15.17:0/1053768786 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4484184fd0 con 0x7f4484052040 2024-09-10T21:25:19.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.794+0000 7fceade00640 1 --2- 172.21.15.17:0/3392999051 >> [v2:172.21.15.31:6826/3754251089,v1:172.21.15.31:6827/3754251089] conn(0x7fce8807bd50 0x7fce8807e190 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:19.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.794+0000 7f4482a00640 1 -- 172.21.15.17:0/1053768786 <== mon.1 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f448c075a40 con 0x7f4484052040 2024-09-10T21:25:19.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.794+0000 7fcead400640 1 --2- 172.21.15.17:0/3392999051 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fce880725e0 0x7fce88074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:19.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.794+0000 7f514f000640 1 Processor -- start 2024-09-10T21:25:19.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.794+0000 7f514f000640 1 -- start start 2024-09-10T21:25:19.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.794+0000 7f94cc200640 1 -- 172.21.15.17:0/1141710644 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f94c4150290 msgr2=0x7f94c41506a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:19.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.794+0000 7f94cc200640 1 --2- 172.21.15.17:0/1141710644 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f94c4150290 0x7f94c41506a0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f94c4153ef0 tx=0x7f94bc01c9c0 comp rx=0 tx=0).stop 2024-09-10T21:25:19.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.794+0000 7f94cc200640 1 -- 172.21.15.17:0/1141710644 shutdown_connections 2024-09-10T21:25:19.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.794+0000 7f94cc200640 1 --2- 172.21.15.17:0/1141710644 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f94c4150290 0x7f94c41506a0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:19.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.794+0000 7f94cc200640 1 --2- 172.21.15.17:0/1141710644 >> v2:172.21.15.17:3300/0 conn(0x7f94c4156e00 0x7f94c40c7e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:19.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.794+0000 7f94cc200640 1 -- 172.21.15.17:0/1141710644 >> 172.21.15.17:0/1141710644 conn(0x7f94c406fe10 msgr2=0x7f94c40701f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:19.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.794+0000 7fcea5600640 1 -- 172.21.15.17:0/3392999051 <== osd.6 v2:172.21.15.31:6826/3754251089 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fce8807e860 con 0x7fce8807bd50 2024-09-10T21:25:19.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.794+0000 7f9c6b200640 1 Processor -- start 2024-09-10T21:25:19.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.794+0000 7fcead400640 1 --2- 172.21.15.17:0/3392999051 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fce880725e0 0x7fce88074aa0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fce9c0b2000 tx=0x7fce9c01be70 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:19.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.794+0000 7f94cc200640 1 -- 172.21.15.17:0/1141710644 shutdown_connections 2024-09-10T21:25:19.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.794+0000 7f94cc200640 1 -- 172.21.15.17:0/1141710644 wait complete. 2024-09-10T21:25:19.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.794+0000 7f9c6b200640 1 -- start start 2024-09-10T21:25:19.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.794+0000 7f514f000640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f514806e730 0x7f51481521c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:19.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.794+0000 7f514f000640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f51480c6f40 0x7f51480c4480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:19.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.794+0000 7f514f000640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f51481548b0 0x7f514814cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:19.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.794+0000 7f514f000640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f51480c82e0 con 0x7f51481548b0 2024-09-10T21:25:19.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.794+0000 7f514f000640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f51480c7460 con 0x7f514806e730 2024-09-10T21:25:19.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.794+0000 7f514f000640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f51480c85c0 con 0x7f51480c6f40 2024-09-10T21:25:19.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.794+0000 7f5147400640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f51480c6f40 0x7f51480c4480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:19.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.794+0000 7f5147400640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f51480c6f40 0x7f51480c4480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3301/0 says I am v2:172.21.15.17:42932/0 (socket says 172.21.15.17:42932) 2024-09-10T21:25:19.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.794+0000 7f5147400640 1 -- 172.21.15.17:0/2653535512 learned_addr learned my addr 172.21.15.17:0/2653535512 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:25:19.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.794+0000 7f5147400640 1 -- 172.21.15.17:0/2653535512 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f514806e730 msgr2=0x7f51481521c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:25:19.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.798+0000 7f5147400640 1 --2- 172.21.15.17:0/2653535512 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f514806e730 0x7f51481521c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:19.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.798+0000 7f5147400640 1 -- 172.21.15.17:0/2653535512 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f51481548b0 msgr2=0x7f514814cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:19.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.798+0000 7f5147400640 1 --2- 172.21.15.17:0/2653535512 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f51481548b0 0x7f514814cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:19.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.798+0000 7f5147400640 1 -- 172.21.15.17:0/2653535512 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51480c3930 con 0x7f51480c6f40 2024-09-10T21:25:19.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.794+0000 7f9c6b200640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9c6406dd10 0x7f9c64147dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:19.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.794+0000 7f9c6b200640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f9c640c0fd0 0x7f9c640c2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:19.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.798+0000 7f9c6b200640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c64153d40 0x7f9c640c26a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:19.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.798+0000 7f448be00640 1 -- 172.21.15.17:0/1053768786 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f44840e2160 con 0x7f4484052040 2024-09-10T21:25:19.806 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.798+0000 7f9c63e00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9c6406dd10 0x7f9c64147dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:19.806 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.798+0000 7f9c63400640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f9c640c0fd0 0x7f9c640c2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:19.806 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.798+0000 7f9c63e00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9c6406dd10 0x7f9c64147dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44284/0 (socket says 172.21.15.17:44284) 2024-09-10T21:25:19.806 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.798+0000 7f5147400640 1 --2- 172.21.15.17:0/2653535512 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f51480c6f40 0x7f51480c4480 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f513400c4b0 tx=0x7f513400c970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:19.806 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.798+0000 7f9c63400640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f9c640c0fd0 0x7f9c640c2160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3301/0 says I am v2:172.21.15.17:42936/0 (socket says 172.21.15.17:42936) 2024-09-10T21:25:19.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.798+0000 7f9c6b200640 1 -- 172.21.15.17:0/483186392 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f9c640c92c0 con 0x7f9c6406dd10 2024-09-10T21:25:19.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.798+0000 7f513fe00640 1 -- 172.21.15.17:0/2653535512 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5134016020 con 0x7f51480c6f40 2024-09-10T21:25:19.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.798+0000 7f513fe00640 1 -- 172.21.15.17:0/2653535512 <== mon.2 v2:172.21.15.31:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5134012920 con 0x7f51480c6f40 2024-09-10T21:25:19.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.798+0000 7f513fe00640 1 -- 172.21.15.17:0/2653535512 <== mon.2 v2:172.21.15.31:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5134012d50 con 0x7f51480c6f40 2024-09-10T21:25:19.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.798+0000 7f514f000640 1 -- 172.21.15.17:0/2653535512 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5148151bb0 con 0x7f51480c6f40 2024-09-10T21:25:19.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.798+0000 7f9c63e00640 1 -- 172.21.15.17:0/483186392 learned_addr learned my addr 172.21.15.17:0/483186392 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:25:19.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.798+0000 7f514f000640 1 -- 172.21.15.17:0/2653535512 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f51480cc280 con 0x7f51480c6f40 2024-09-10T21:25:19.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.798+0000 7fcd40400640 1 -- 172.21.15.17:0/2820167842 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fcd381548b0 msgr2=0x7fcd3815afb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:19.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.798+0000 7fcd40400640 1 --2- 172.21.15.17:0/2820167842 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fcd381548b0 0x7fcd3815afb0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fcd28009930 tx=0x7fcd2801f110 comp rx=0 tx=0).stop 2024-09-10T21:25:19.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.798+0000 7f9c69200640 1 --2- 172.21.15.17:0/483186392 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c64153d40 0x7f9c640c26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:19.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.798+0000 7f94cc200640 1 Processor -- start 2024-09-10T21:25:19.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.798+0000 7f9c6b200640 1 -- 172.21.15.17:0/483186392 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f9c640c8fe0 con 0x7f9c64153d40 2024-09-10T21:25:19.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.798+0000 7fcd40400640 1 -- 172.21.15.17:0/2820167842 shutdown_connections 2024-09-10T21:25:19.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.798+0000 7fcd40400640 1 --2- 172.21.15.17:0/2820167842 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fcd381548b0 0x7fcd3815afb0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:19.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.798+0000 7fcd40400640 1 -- 172.21.15.17:0/2820167842 >> 172.21.15.17:0/2820167842 conn(0x7fcd380700d0 msgr2=0x7fcd380704e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:19.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.798+0000 7fcd40400640 1 -- 172.21.15.17:0/2820167842 shutdown_connections 2024-09-10T21:25:19.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.798+0000 7fcd40400640 1 -- 172.21.15.17:0/2820167842 wait complete. 2024-09-10T21:25:19.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.798+0000 7f514f000640 1 -- 172.21.15.17:0/2653535512 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5120001040 con 0x7f51480c6f40 2024-09-10T21:25:19.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.798+0000 7f9c6b200640 1 -- 172.21.15.17:0/483186392 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f9c640c95a0 con 0x7f9c640c0fd0 2024-09-10T21:25:19.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.798+0000 7f9c63e00640 1 -- 172.21.15.17:0/483186392 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f9c640c0fd0 msgr2=0x7f9c640c2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:19.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.798+0000 7f9c63e00640 1 --2- 172.21.15.17:0/483186392 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f9c640c0fd0 0x7f9c640c2160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:19.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.798+0000 7f9c63e00640 1 -- 172.21.15.17:0/483186392 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c64153d40 msgr2=0x7f9c640c26a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:19.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.798+0000 7f9c63e00640 1 --2- 172.21.15.17:0/483186392 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c64153d40 0x7f9c640c26a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:19.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.798+0000 7f9c63e00640 1 -- 172.21.15.17:0/483186392 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c64072d70 con 0x7f9c6406dd10 2024-09-10T21:25:19.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.798+0000 7f94cc200640 1 -- start start 2024-09-10T21:25:19.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.802+0000 7f9c69200640 1 --2- 172.21.15.17:0/483186392 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c64153d40 0x7f9c640c26a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:25:19.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.802+0000 7fcd40400640 1 Processor -- start 2024-09-10T21:25:19.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.802+0000 7f4482a00640 1 -- 172.21.15.17:0/1053768786 <== mon.1 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f448c075be0 con 0x7f4484052040 2024-09-10T21:25:19.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.802+0000 7fcd40400640 1 -- start start 2024-09-10T21:25:19.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.802+0000 7f94cc200640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f94c4072470 0x7f94c41446d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:19.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.802+0000 7f94cc200640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f94c4150290 0x7f94c41453a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:19.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.802+0000 7f94cc200640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f94c4156e00 0x7f94c413ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:19.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.802+0000 7fcd40400640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fcd3806e730 0x7fcd381ed950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:19.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.802+0000 7fcd40400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcd380c6f40 0x7fcd381ec750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:19.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.802+0000 7fcd40400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcd381548b0 0x7fcd3814ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:19.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.802+0000 7f94c9800640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f94c4072470 0x7f94c41446d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:19.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.802+0000 7f94c9800640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f94c4072470 0x7f94c41446d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3301/0 says I am v2:172.21.15.17:42952/0 (socket says 172.21.15.17:42952) 2024-09-10T21:25:19.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.802+0000 7fcd40400640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fcd380c82e0 con 0x7fcd381548b0 2024-09-10T21:25:19.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.802+0000 7fcd40400640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fcd380c7460 con 0x7fcd380c6f40 2024-09-10T21:25:19.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.802+0000 7fcd40400640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7fcd380c85c0 con 0x7fcd3806e730 2024-09-10T21:25:19.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.802+0000 7fcd3da00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fcd3806e730 0x7fcd381ed950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:19.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.802+0000 7fcd3d000640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcd380c6f40 0x7fcd381ec750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:19.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.802+0000 7fcd3da00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fcd3806e730 0x7fcd381ed950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3301/0 says I am v2:172.21.15.17:42960/0 (socket says 172.21.15.17:42960) 2024-09-10T21:25:19.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.802+0000 7fcd3da00640 1 -- 172.21.15.17:0/3359527135 learned_addr learned my addr 172.21.15.17:0/3359527135 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:25:19.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.802+0000 7f94cc200640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f94c40c7440 con 0x7f94c4150290 2024-09-10T21:25:19.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.802+0000 7f94cc200640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f94c40c72c0 con 0x7f94c4156e00 2024-09-10T21:25:19.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.802+0000 7f94cc200640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f94c40c75c0 con 0x7f94c4072470 2024-09-10T21:25:19.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.802+0000 7f94c9800640 1 -- 172.21.15.17:0/1458225254 learned_addr learned my addr 172.21.15.17:0/1458225254 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:25:19.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.802+0000 7f513fe00640 1 -- 172.21.15.17:0/2653535512 <== mon.2 v2:172.21.15.31:3301/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f513401b400 con 0x7f51480c6f40 2024-09-10T21:25:19.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.802+0000 7f513fe00640 1 --2- 172.21.15.17:0/2653535512 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f51280726e0 0x7f5128074ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:19.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.802+0000 7f513fe00640 1 -- 172.21.15.17:0/2653535512 <== mon.2 v2:172.21.15.31:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f5134093c80 con 0x7f51480c6f40 2024-09-10T21:25:19.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.802+0000 7f513fe00640 1 --2- 172.21.15.17:0/2653535512 >> [v2:172.21.15.17:6816/1248243634,v1:172.21.15.17:6817/1248243634] conn(0x7f512807be60 0x7f512807e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:19.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.802+0000 7f513fe00640 1 -- 172.21.15.17:0/2653535512 --> [v2:172.21.15.17:6816/1248243634,v1:172.21.15.17:6817/1248243634] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f513403eea0 con 0x7f512807be60 2024-09-10T21:25:19.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.802+0000 7f513fe00640 1 -- 172.21.15.17:0/2653535512 <== mon.2 v2:172.21.15.31:3301/0 6 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f5134094070 con 0x7f51480c6f40 2024-09-10T21:25:19.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.802+0000 7f5147e00640 1 --2- 172.21.15.17:0/2653535512 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f51280726e0 0x7f5128074ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:19.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.802+0000 7f94c9800640 1 -- 172.21.15.17:0/1458225254 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f94c4156e00 msgr2=0x7f94c413ec80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:25:19.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.802+0000 7f94c9800640 1 --2- 172.21.15.17:0/1458225254 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f94c4156e00 0x7f94c413ec80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:19.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.802+0000 7f94c9800640 1 -- 172.21.15.17:0/1458225254 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f94c4150290 msgr2=0x7f94c41453a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:19.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.802+0000 7f94c9800640 1 --2- 172.21.15.17:0/1458225254 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f94c4150290 0x7f94c41453a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:19.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.802+0000 7f94c9800640 1 -- 172.21.15.17:0/1458225254 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94c4070be0 con 0x7f94c4072470 2024-09-10T21:25:19.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.806+0000 7f514d000640 1 --2- 172.21.15.17:0/2653535512 >> [v2:172.21.15.17:6816/1248243634,v1:172.21.15.17:6817/1248243634] conn(0x7f512807be60 0x7f512807e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:19.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.806+0000 7f94c9800640 1 --2- 172.21.15.17:0/1458225254 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f94c4072470 0x7f94c41446d0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f94b000e9a0 tx=0x7f94b000ee60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:19.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.806+0000 7f5147e00640 1 --2- 172.21.15.17:0/2653535512 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f51280726e0 0x7f5128074ba0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f51380b2000 tx=0x7f513801bdb0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:19.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.806+0000 7f4482a00640 1 --2- 172.21.15.17:0/1053768786 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f446c0725e0 0x7f446c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:19.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.806+0000 7f4482a00640 1 -- 172.21.15.17:0/1053768786 <== mon.1 v2:172.21.15.31:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f448c10e250 con 0x7f4484052040 2024-09-10T21:25:19.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.806+0000 7f4482a00640 1 --2- 172.21.15.17:0/1053768786 >> [v2:172.21.15.17:6808/4105489867,v1:172.21.15.17:6809/4105489867] conn(0x7f446c07bd50 0x7f446c07e190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:19.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.806+0000 7f4482a00640 1 -- 172.21.15.17:0/1053768786 --> [v2:172.21.15.17:6808/4105489867,v1:172.21.15.17:6809/4105489867] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f448c093e50 con 0x7f446c07bd50 2024-09-10T21:25:19.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.806+0000 7f4482a00640 1 -- 172.21.15.17:0/1053768786 <== mon.1 v2:172.21.15.31:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f448c115050 con 0x7f4484052040 2024-09-10T21:25:19.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.802+0000 7fcd3da00640 1 -- 172.21.15.17:0/3359527135 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcd380c6f40 msgr2=0x7fcd381ec750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:19.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.802+0000 7fcd3da00640 1 --2- 172.21.15.17:0/3359527135 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcd380c6f40 0x7fcd381ec750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:19.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.802+0000 7fcd3da00640 1 -- 172.21.15.17:0/3359527135 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcd381548b0 msgr2=0x7fcd3814ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:19.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.802+0000 7fcd3da00640 1 --2- 172.21.15.17:0/3359527135 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcd381548b0 0x7fcd3814ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:19.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.802+0000 7fcd3da00640 1 -- 172.21.15.17:0/3359527135 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd381ed120 con 0x7fcd3806e730 2024-09-10T21:25:19.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.802+0000 7fcd3d000640 1 --2- 172.21.15.17:0/3359527135 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcd380c6f40 0x7fcd381ec750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:25:19.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.802+0000 7fcd3da00640 1 --2- 172.21.15.17:0/3359527135 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fcd3806e730 0x7fcd381ed950 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fcd280b3040 tx=0x7fcd28005760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:19.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.802+0000 7fcd35600640 1 -- 172.21.15.17:0/3359527135 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd280b5040 con 0x7fcd3806e730 2024-09-10T21:25:19.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.806+0000 7fcd35600640 1 -- 172.21.15.17:0/3359527135 <== mon.2 v2:172.21.15.31:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcd280b09c0 con 0x7fcd3806e730 2024-09-10T21:25:19.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.806+0000 7fcd35600640 1 -- 172.21.15.17:0/3359527135 <== mon.2 v2:172.21.15.31:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd280b0e10 con 0x7fcd3806e730 2024-09-10T21:25:19.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.806+0000 7fcd40400640 1 -- 172.21.15.17:0/3359527135 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd38155c90 con 0x7fcd3806e730 2024-09-10T21:25:19.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.806+0000 7fcd40400640 1 -- 172.21.15.17:0/3359527135 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd38156250 con 0x7fcd3806e730 2024-09-10T21:25:19.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.806+0000 7f448aa00640 1 --2- 172.21.15.17:0/1053768786 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f446c0725e0 0x7f446c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:19.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.806+0000 7f448a000640 1 --2- 172.21.15.17:0/1053768786 >> [v2:172.21.15.17:6808/4105489867,v1:172.21.15.17:6809/4105489867] conn(0x7f446c07bd50 0x7f446c07e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:19.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.806+0000 7f448a000640 1 --2- 172.21.15.17:0/1053768786 >> [v2:172.21.15.17:6808/4105489867,v1:172.21.15.17:6809/4105489867] conn(0x7f446c07bd50 0x7f446c07e190 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:19.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.806+0000 7f448aa00640 1 --2- 172.21.15.17:0/1053768786 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f446c0725e0 0x7f446c074aa0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f447c01c940 tx=0x7f447c0198b0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:19.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.806+0000 7f514d000640 1 --2- 172.21.15.17:0/2653535512 >> [v2:172.21.15.17:6816/1248243634,v1:172.21.15.17:6817/1248243634] conn(0x7f512807be60 0x7f512807e2a0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:19.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.806+0000 7f9c63e00640 1 --2- 172.21.15.17:0/483186392 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9c6406dd10 0x7f9c64147dc0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f9c54007320 tx=0x7f9c54008bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:19.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.806+0000 7f94c1600640 1 -- 172.21.15.17:0/1458225254 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94b000cd80 con 0x7f94c4072470 2024-09-10T21:25:19.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.806+0000 7f94c1600640 1 -- 172.21.15.17:0/1458225254 <== mon.2 v2:172.21.15.31:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f94b0004020 con 0x7f94c4072470 2024-09-10T21:25:19.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.806+0000 7f94c1600640 1 -- 172.21.15.17:0/1458225254 <== mon.2 v2:172.21.15.31:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94b00043f0 con 0x7f94c4072470 2024-09-10T21:25:19.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.806+0000 7f94cc200640 1 -- 172.21.15.17:0/1458225254 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94c413e010 con 0x7f94c4072470 2024-09-10T21:25:19.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.806+0000 7f94cc200640 1 -- 172.21.15.17:0/1458225254 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f94c40cb720 con 0x7f94c4072470 2024-09-10T21:25:19.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.806+0000 7fcd40400640 1 -- 172.21.15.17:0/3359527135 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fcd380c0fd0 con 0x7fcd3806e730 2024-09-10T21:25:19.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.806+0000 7fcd35600640 1 -- 172.21.15.17:0/3359527135 <== mon.2 v2:172.21.15.31:3301/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fcd280b8070 con 0x7fcd3806e730 2024-09-10T21:25:19.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.810+0000 7f4482a00640 1 -- 172.21.15.17:0/1053768786 <== osd.0 v2:172.21.15.17:6808/4105489867 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f448c093e50 con 0x7f446c07bd50 2024-09-10T21:25:19.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.810+0000 7f513fe00640 1 -- 172.21.15.17:0/2653535512 <== osd.2 v2:172.21.15.17:6816/1248243634 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f513403eea0 con 0x7f512807be60 2024-09-10T21:25:19.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.810+0000 7f9c5be00640 1 -- 172.21.15.17:0/483186392 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c540a9cb0 con 0x7f9c6406dd10 2024-09-10T21:25:19.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.810+0000 7f9c5be00640 1 -- 172.21.15.17:0/483186392 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9c54003600 con 0x7f9c6406dd10 2024-09-10T21:25:19.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.810+0000 7f9c5be00640 1 -- 172.21.15.17:0/483186392 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c54003a30 con 0x7f9c6406dd10 2024-09-10T21:25:19.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.810+0000 7fcd35600640 1 --2- 172.21.15.17:0/3359527135 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fcd1c0725e0 0x7fcd1c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:19.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.810+0000 7fcd35600640 1 -- 172.21.15.17:0/3359527135 <== mon.2 v2:172.21.15.31:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fcd281306f0 con 0x7fcd3806e730 2024-09-10T21:25:19.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.810+0000 7fcd35600640 1 --2- 172.21.15.17:0/3359527135 >> [v2:172.21.15.31:6800/1472711057,v1:172.21.15.31:6801/1472711057] conn(0x7fcd1c07bcd0 0x7fcd1c07e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:19.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.810+0000 7fcd35600640 1 -- 172.21.15.17:0/3359527135 --> [v2:172.21.15.31:6800/1472711057,v1:172.21.15.31:6801/1472711057] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fcd280b2e90 con 0x7fcd1c07bcd0 2024-09-10T21:25:19.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.810+0000 7fcd35600640 1 -- 172.21.15.17:0/3359527135 <== mon.2 v2:172.21.15.31:3301/0 6 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fcd28130aa0 con 0x7fcd3806e730 2024-09-10T21:25:19.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.810+0000 7f11aec00640 1 -- 172.21.15.17:0/865370934 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f118c0052f0 msgr2=0x7f118c00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:19.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.810+0000 7f11aec00640 1 --2- 172.21.15.17:0/865370934 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f118c0052f0 0x7f118c00b9d0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f11a80c7390 tx=0x7f11a001ca20 comp rx=0 tx=0).stop 2024-09-10T21:25:19.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.810+0000 7f11aec00640 1 -- 172.21.15.17:0/865370934 shutdown_connections 2024-09-10T21:25:19.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.810+0000 7f11aec00640 1 --2- 172.21.15.17:0/865370934 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f118c0052f0 0x7f118c00b9d0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:19.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.810+0000 7f11aec00640 1 -- 172.21.15.17:0/865370934 >> 172.21.15.17:0/865370934 conn(0x7f11a80700d0 msgr2=0x7f11a80704e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:19.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.810+0000 7fcd3d000640 1 --2- 172.21.15.17:0/3359527135 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fcd1c0725e0 0x7fcd1c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:19.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.810+0000 7fcd3e400640 1 --2- 172.21.15.17:0/3359527135 >> [v2:172.21.15.31:6800/1472711057,v1:172.21.15.31:6801/1472711057] conn(0x7fcd1c07bcd0 0x7fcd1c07e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:19.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.810+0000 7f11aec00640 1 -- 172.21.15.17:0/865370934 shutdown_connections 2024-09-10T21:25:19.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.810+0000 7fcd3d000640 1 --2- 172.21.15.17:0/3359527135 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fcd1c0725e0 0x7fcd1c074aa0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fcd380c1420 tx=0x7fcd2c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:19.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.810+0000 7fcd3e400640 1 --2- 172.21.15.17:0/3359527135 >> [v2:172.21.15.31:6800/1472711057,v1:172.21.15.31:6801/1472711057] conn(0x7fcd1c07bcd0 0x7fcd1c07e110 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:19.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.810+0000 7f9c6b200640 1 -- 172.21.15.17:0/483186392 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c641e0da0 con 0x7f9c6406dd10 2024-09-10T21:25:19.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.810+0000 7f11aec00640 1 -- 172.21.15.17:0/865370934 wait complete. 2024-09-10T21:25:19.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.810+0000 7fcd35600640 1 -- 172.21.15.17:0/3359527135 <== osd.5 v2:172.21.15.31:6800/1472711057 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fcd280b2e90 con 0x7fcd1c07bcd0 2024-09-10T21:25:19.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.810+0000 7f9c6b200640 1 -- 172.21.15.17:0/483186392 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c641ecef0 con 0x7f9c6406dd10 2024-09-10T21:25:19.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.814+0000 7f11aec00640 1 Processor -- start 2024-09-10T21:25:19.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.814+0000 7f11aec00640 1 -- start start 2024-09-10T21:25:19.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.814+0000 7f9c6b200640 1 -- 172.21.15.17:0/483186392 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9c641ecca0 con 0x7f9c6406dd10 2024-09-10T21:25:19.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.810+0000 7f94b6a00640 1 -- 172.21.15.17:0/1458225254 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9498001040 con 0x7f94c4072470 2024-09-10T21:25:19.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.818+0000 7f11aec00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f118c0052f0 0x7f11a81e27b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:19.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.818+0000 7fceae800640 1 -- 172.21.15.17:0/3392999051 --> [v2:172.21.15.31:6826/3754251089,v1:172.21.15.31:6827/3754251089] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fcea00d0040 con 0x7fce8807bd50 2024-09-10T21:25:19.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.818+0000 7f11aec00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f11a81e3190 0x7f11a81e1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:19.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.818+0000 7f11aec00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f11a81e1530 0x7f11a80c5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:19.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.818+0000 7f11aec00640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f11a80c88a0 con 0x7f118c0052f0 2024-09-10T21:25:19.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.818+0000 7f11aec00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f11a80c85c0 con 0x7f11a81e1530 2024-09-10T21:25:19.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.818+0000 7f11aec00640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f11a80c0fd0 con 0x7f11a81e3190 2024-09-10T21:25:19.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.818+0000 7fcea5600640 1 -- 172.21.15.17:0/3392999051 <== osd.6 v2:172.21.15.31:6826/3754251089 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fcea00d0040 con 0x7fce8807bd50 2024-09-10T21:25:19.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.818+0000 7f11a7400640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f11a81e3190 0x7f11a81e1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:19.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.818+0000 7f11a7400640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f11a81e3190 0x7f11a81e1aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3301/0 says I am v2:172.21.15.17:42968/0 (socket says 172.21.15.17:42968) 2024-09-10T21:25:19.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.818+0000 7f11a7400640 1 -- 172.21.15.17:0/3538369118 learned_addr learned my addr 172.21.15.17:0/3538369118 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:25:19.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.818+0000 7f11acc00640 1 --2- 172.21.15.17:0/3538369118 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f11a81e1530 0x7f11a80c5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:19.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.818+0000 7f11a7e00640 1 --2- 172.21.15.17:0/3538369118 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f118c0052f0 0x7f11a81e27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:19.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.818+0000 7fceae800640 1 -- 172.21.15.17:0/3392999051 >> [v2:172.21.15.31:6826/3754251089,v1:172.21.15.31:6827/3754251089] conn(0x7fce8807bd50 msgr2=0x7fce8807e190 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:19.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.818+0000 7fceae800640 1 --2- 172.21.15.17:0/3392999051 >> [v2:172.21.15.31:6826/3754251089,v1:172.21.15.31:6827/3754251089] conn(0x7fce8807bd50 0x7fce8807e190 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:19.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.818+0000 7f11a7400640 1 -- 172.21.15.17:0/3538369118 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f11a81e1530 msgr2=0x7f11a80c5580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:19.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.818+0000 7f11a7400640 1 --2- 172.21.15.17:0/3538369118 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f11a81e1530 0x7f11a80c5580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:19.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.818+0000 7f11a7400640 1 -- 172.21.15.17:0/3538369118 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f118c0052f0 msgr2=0x7f11a81e27b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:19.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.818+0000 7f11a7400640 1 --2- 172.21.15.17:0/3538369118 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f118c0052f0 0x7f11a81e27b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:19.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.818+0000 7f11a7400640 1 -- 172.21.15.17:0/3538369118 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11a81566a0 con 0x7f11a81e3190 2024-09-10T21:25:19.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.818+0000 7fceae800640 1 -- 172.21.15.17:0/3392999051 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fce880725e0 msgr2=0x7fce88074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:19.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.818+0000 7fceae800640 1 --2- 172.21.15.17:0/3392999051 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fce880725e0 0x7fce88074aa0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fce9c0b2000 tx=0x7fce9c01be70 comp rx=0 tx=0).stop 2024-09-10T21:25:19.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.818+0000 7fceae800640 1 -- 172.21.15.17:0/3392999051 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcea0051da0 msgr2=0x7fcea0181680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:19.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.818+0000 7fceae800640 1 --2- 172.21.15.17:0/3392999051 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcea0051da0 0x7fcea0181680 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fce9400c4b0 tx=0x7fce9400c970 comp rx=0 tx=0).stop 2024-09-10T21:25:19.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.818+0000 7f9c5be00640 1 -- 172.21.15.17:0/483186392 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f9c54003bd0 con 0x7f9c6406dd10 2024-09-10T21:25:19.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.818+0000 7f94c1600640 1 -- 172.21.15.17:0/1458225254 <== mon.2 v2:172.21.15.31:3301/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f94b0036be0 con 0x7f94c4072470 2024-09-10T21:25:19.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.818+0000 7f94c1600640 1 --2- 172.21.15.17:0/1458225254 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f94a40726b0 0x7f94a4074b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:19.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.818+0000 7f94c1600640 1 -- 172.21.15.17:0/1458225254 <== mon.2 v2:172.21.15.31:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f94b0093a40 con 0x7f94c4072470 2024-09-10T21:25:19.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.818+0000 7f94c1600640 1 --2- 172.21.15.17:0/1458225254 >> [v2:172.21.15.31:6810/202520744,v1:172.21.15.31:6811/202520744] conn(0x7f94a407be30 0x7f94a407e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:19.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.818+0000 7f94c1600640 1 -- 172.21.15.17:0/1458225254 --> [v2:172.21.15.31:6810/202520744,v1:172.21.15.31:6811/202520744] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f94a407e940 con 0x7f94a407be30 2024-09-10T21:25:19.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.818+0000 7f94c1600640 1 -- 172.21.15.17:0/1458225254 <== mon.2 v2:172.21.15.31:3301/0 6 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f94b00982a0 con 0x7f94c4072470 2024-09-10T21:25:19.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.818+0000 7fceade00640 1 -- 172.21.15.17:0/3392999051 reap_dead start 2024-09-10T21:25:19.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.818+0000 7fceae800640 1 -- 172.21.15.17:0/3392999051 shutdown_connections 2024-09-10T21:25:19.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.818+0000 7fceae800640 1 -- 172.21.15.17:0/3392999051 >> 172.21.15.17:0/3392999051 conn(0x7fcea000e420 msgr2=0x7fcea00d3b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:19.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.818+0000 7f11a7400640 1 --2- 172.21.15.17:0/3538369118 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f11a81e3190 0x7f11a81e1aa0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f1194006480 tx=0x7f11940039d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:19.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.818+0000 7f94ca200640 1 --2- 172.21.15.17:0/1458225254 >> [v2:172.21.15.31:6810/202520744,v1:172.21.15.31:6811/202520744] conn(0x7f94a407be30 0x7f94a407e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:19.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.818+0000 7fceae800640 1 -- 172.21.15.17:0/3392999051 shutdown_connections 2024-09-10T21:25:19.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.818+0000 7fceae800640 1 -- 172.21.15.17:0/3392999051 wait complete. 2024-09-10T21:25:19.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.818+0000 7f4ecc800640 1 -- 172.21.15.17:0/1873496170 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4ec406dd10 msgr2=0x7f4ec406e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:19.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.818+0000 7f4ecc800640 1 --2- 172.21.15.17:0/1873496170 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4ec406dd10 0x7f4ec406e1a0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f4ec406e860 tx=0x7f4eb801ede0 comp rx=0 tx=0).stop 2024-09-10T21:25:19.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.818+0000 7f119fe00640 1 -- 172.21.15.17:0/3538369118 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1194015070 con 0x7f11a81e3190 2024-09-10T21:25:19.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.818+0000 7f119fe00640 1 -- 172.21.15.17:0/3538369118 <== mon.2 v2:172.21.15.31:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f11940119e0 con 0x7f11a81e3190 2024-09-10T21:25:19.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.818+0000 7f119fe00640 1 -- 172.21.15.17:0/3538369118 <== mon.2 v2:172.21.15.31:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1194011e30 con 0x7f11a81e3190 2024-09-10T21:25:19.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.818+0000 7f11aec00640 1 -- 172.21.15.17:0/3538369118 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11a81e0770 con 0x7f11a81e3190 2024-09-10T21:25:19.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.818+0000 7f11aec00640 1 -- 172.21.15.17:0/3538369118 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f11a81e0d30 con 0x7f11a81e3190 2024-09-10T21:25:19.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.818+0000 7f94c8e00640 1 --2- 172.21.15.17:0/1458225254 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f94a40726b0 0x7f94a4074b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:19.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.818+0000 7f94ca200640 1 --2- 172.21.15.17:0/1458225254 >> [v2:172.21.15.31:6810/202520744,v1:172.21.15.31:6811/202520744] conn(0x7f94a407be30 0x7f94a407e270 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:19.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.822+0000 7f94c8e00640 1 --2- 172.21.15.17:0/1458225254 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f94a40726b0 0x7f94a4074b70 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f94b8001890 tx=0x7f94b8001600 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:19.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.822+0000 7f11aec00640 1 -- 172.21.15.17:0/3538369118 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f11a8156980 con 0x7f11a81e3190 2024-09-10T21:25:19.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.822+0000 7f9c5be00640 1 --2- 172.21.15.17:0/483186392 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f9c440726e0 0x7f9c44074ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:19.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.822+0000 7f4ecc800640 1 -- 172.21.15.17:0/1873496170 shutdown_connections 2024-09-10T21:25:19.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.822+0000 7f4ecc800640 1 --2- 172.21.15.17:0/1873496170 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4ec406dd10 0x7f4ec406e1a0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:19.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.822+0000 7f4ecc800640 1 --2- 172.21.15.17:0/1873496170 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f4ec4153d40 0x7f4ec40c8010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:19.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.822+0000 7f4ecc800640 1 -- 172.21.15.17:0/1873496170 >> 172.21.15.17:0/1873496170 conn(0x7f4ec40700d0 msgr2=0x7f4ec40704e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:19.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.822+0000 7f9c5be00640 1 -- 172.21.15.17:0/483186392 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f9c541307a0 con 0x7f9c6406dd10 2024-09-10T21:25:19.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.822+0000 7f94c1600640 1 -- 172.21.15.17:0/1458225254 <== osd.4 v2:172.21.15.31:6810/202520744 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f94a407e940 con 0x7f94a407be30 2024-09-10T21:25:19.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.822+0000 7f4ecc800640 1 -- 172.21.15.17:0/1873496170 shutdown_connections 2024-09-10T21:25:19.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.822+0000 7f4ecc800640 1 -- 172.21.15.17:0/1873496170 wait complete. 2024-09-10T21:25:19.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.822+0000 7f9c63400640 1 --2- 172.21.15.17:0/483186392 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f9c440726e0 0x7f9c44074ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:19.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.822+0000 7f9c5be00640 1 --2- 172.21.15.17:0/483186392 >> [v2:172.21.15.17:6800/2314489522,v1:172.21.15.17:6801/2314489522] conn(0x7f9c4407be60 0x7f9c4407e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:19.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.822+0000 7f9c5be00640 1 -- 172.21.15.17:0/483186392 --> [v2:172.21.15.17:6800/2314489522,v1:172.21.15.17:6801/2314489522] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9c540aee50 con 0x7f9c4407be60 2024-09-10T21:25:19.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.822+0000 7f9c5be00640 1 -- 172.21.15.17:0/483186392 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f9c54137050 con 0x7f9c6406dd10 2024-09-10T21:25:19.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.822+0000 7f9c63400640 1 --2- 172.21.15.17:0/483186392 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f9c440726e0 0x7f9c44074ba0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f9c641506d0 tx=0x7f9c50007f20 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:19.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.822+0000 7f9c69200640 1 --2- 172.21.15.17:0/483186392 >> [v2:172.21.15.17:6800/2314489522,v1:172.21.15.17:6801/2314489522] conn(0x7f9c4407be60 0x7f9c4407e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:19.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.822+0000 7f119fe00640 1 -- 172.21.15.17:0/3538369118 <== mon.2 v2:172.21.15.31:3301/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f11940103c0 con 0x7f11a81e3190 2024-09-10T21:25:19.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.822+0000 7f4ecc800640 1 Processor -- start 2024-09-10T21:25:19.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.822+0000 7f4ecc800640 1 -- start start 2024-09-10T21:25:19.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.826+0000 7f119fe00640 1 --2- 172.21.15.17:0/3538369118 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f1188072630 0x7f1188074af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:19.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.826+0000 7f119fe00640 1 -- 172.21.15.17:0/3538369118 <== mon.2 v2:172.21.15.31:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f11940947f0 con 0x7f11a81e3190 2024-09-10T21:25:19.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.826+0000 7f11a7e00640 1 --2- 172.21.15.17:0/3538369118 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f1188072630 0x7f1188074af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:19.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.826+0000 7f119fe00640 1 --2- 172.21.15.17:0/3538369118 >> [v2:172.21.15.17:6824/2599842027,v1:172.21.15.17:6825/2599842027] conn(0x7f118807bda0 0x7f118807e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:19.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.826+0000 7f119fe00640 1 -- 172.21.15.17:0/3538369118 --> [v2:172.21.15.17:6824/2599842027,v1:172.21.15.17:6825/2599842027] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f118807e8b0 con 0x7f118807bda0 2024-09-10T21:25:19.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.826+0000 7f119fe00640 1 -- 172.21.15.17:0/3538369118 <== mon.2 v2:172.21.15.31:3301/0 6 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1194094ba0 con 0x7f11a81e3190 2024-09-10T21:25:19.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.826+0000 7f9c69200640 1 --2- 172.21.15.17:0/483186392 >> [v2:172.21.15.17:6800/2314489522,v1:172.21.15.17:6801/2314489522] conn(0x7f9c4407be60 0x7f9c4407e2a0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:19.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.826+0000 7f4ecc800640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f4ec406dd10 0x7f4ec40c4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:19.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.826+0000 7f4ecc800640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4ec40c0fd0 0x7f4ec41517d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:19.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.826+0000 7f4ecc800640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4ec4153d40 0x7f4ec4140b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:19.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.826+0000 7f4ecc800640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f4ec40c95a0 con 0x7f4ec40c0fd0 2024-09-10T21:25:19.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.826+0000 7f4ecc800640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f4ec40c13b0 con 0x7f4ec4153d40 2024-09-10T21:25:19.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.826+0000 7f4ecc800640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f4ec4154120 con 0x7f4ec406dd10 2024-09-10T21:25:19.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.826+0000 7f4eca800640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4ec4153d40 0x7f4ec4140b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:19.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.826+0000 7f4eca800640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4ec4153d40 0x7f4ec4140b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.17:51388/0 (socket says 172.21.15.17:51388) 2024-09-10T21:25:19.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.826+0000 7f4eca800640 1 -- 172.21.15.17:0/1926711127 learned_addr learned my addr 172.21.15.17:0/1926711127 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:25:19.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.826+0000 7f11a7e00640 1 --2- 172.21.15.17:0/3538369118 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f1188072630 0x7f1188074af0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f1198006fd0 tx=0x7f1198006b80 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:19.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.826+0000 7f11acc00640 1 --2- 172.21.15.17:0/3538369118 >> [v2:172.21.15.17:6824/2599842027,v1:172.21.15.17:6825/2599842027] conn(0x7f118807bda0 0x7f118807e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:19.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.830+0000 7f11acc00640 1 --2- 172.21.15.17:0/3538369118 >> [v2:172.21.15.17:6824/2599842027,v1:172.21.15.17:6825/2599842027] conn(0x7f118807bda0 0x7f118807e1e0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:19.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.826+0000 7f4eca800640 1 -- 172.21.15.17:0/1926711127 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f4ec406dd10 msgr2=0x7f4ec40c4d60 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-10T21:25:19.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.826+0000 7f4eca800640 1 --2- 172.21.15.17:0/1926711127 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f4ec406dd10 0x7f4ec40c4d60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:19.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.826+0000 7f4eca800640 1 -- 172.21.15.17:0/1926711127 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4ec40c0fd0 msgr2=0x7f4ec41517d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:19.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.826+0000 7f4eca800640 1 --2- 172.21.15.17:0/1926711127 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4ec40c0fd0 0x7f4ec41517d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:19.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.826+0000 7f4eca800640 1 -- 172.21.15.17:0/1926711127 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ec40c3810 con 0x7f4ec4153d40 2024-09-10T21:25:19.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.826+0000 7f4eca800640 1 --2- 172.21.15.17:0/1926711127 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4ec4153d40 0x7f4ec4140b60 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f4eb8008330 tx=0x7f4eb8009e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:19.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.830+0000 7f4ebe000640 1 -- 172.21.15.17:0/1926711127 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eb80be070 con 0x7f4ec4153d40 2024-09-10T21:25:19.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.830+0000 7f4ecc800640 1 -- 172.21.15.17:0/1926711127 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ec41dcd30 con 0x7f4ec4153d40 2024-09-10T21:25:19.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.830+0000 7f4ecc800640 1 -- 172.21.15.17:0/1926711127 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ec41ddc10 con 0x7f4ec4153d40 2024-09-10T21:25:19.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.830+0000 7f4ecc800640 1 -- 172.21.15.17:0/1926711127 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4ec40c74c0 con 0x7f4ec4153d40 2024-09-10T21:25:19.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.830+0000 7f9c5be00640 1 -- 172.21.15.17:0/483186392 <== osd.3 v2:172.21.15.17:6800/2314489522 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9c540aee50 con 0x7f9c4407be60 2024-09-10T21:25:19.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.830+0000 7f4ebe000640 1 -- 172.21.15.17:0/1926711127 <== mon.1 v2:172.21.15.31:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4eb80a89f0 con 0x7f4ec4153d40 2024-09-10T21:25:19.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.830+0000 7f4ebe000640 1 -- 172.21.15.17:0/1926711127 <== mon.1 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4eb8005070 con 0x7f4ec4153d40 2024-09-10T21:25:19.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.834+0000 7f119fe00640 1 -- 172.21.15.17:0/3538369118 <== osd.1 v2:172.21.15.17:6824/2599842027 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f118807e8b0 con 0x7f118807bda0 2024-09-10T21:25:19.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.834+0000 7fcd40400640 1 -- 172.21.15.17:0/3359527135 --> [v2:172.21.15.31:6800/1472711057,v1:172.21.15.31:6801/1472711057] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fcd380c3160 con 0x7fcd1c07bcd0 2024-09-10T21:25:19.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.834+0000 7fcd35600640 1 -- 172.21.15.17:0/3359527135 <== osd.5 v2:172.21.15.31:6800/1472711057 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fcd380c3160 con 0x7fcd1c07bcd0 2024-09-10T21:25:19.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.834+0000 7fcd26a00640 1 -- 172.21.15.17:0/3359527135 >> [v2:172.21.15.31:6800/1472711057,v1:172.21.15.31:6801/1472711057] conn(0x7fcd1c07bcd0 msgr2=0x7fcd1c07e110 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:19.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.834+0000 7fcd26a00640 1 --2- 172.21.15.17:0/3359527135 >> [v2:172.21.15.31:6800/1472711057,v1:172.21.15.31:6801/1472711057] conn(0x7fcd1c07bcd0 0x7fcd1c07e110 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:19.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.834+0000 7fcd26a00640 1 -- 172.21.15.17:0/3359527135 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fcd1c0725e0 msgr2=0x7fcd1c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:19.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.834+0000 7fcd26a00640 1 --2- 172.21.15.17:0/3359527135 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fcd1c0725e0 0x7fcd1c074aa0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fcd380c1420 tx=0x7fcd2c009040 comp rx=0 tx=0).stop 2024-09-10T21:25:19.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.834+0000 7fcd26a00640 1 -- 172.21.15.17:0/3359527135 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fcd3806e730 msgr2=0x7fcd381ed950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:19.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.834+0000 7fcd26a00640 1 --2- 172.21.15.17:0/3359527135 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fcd3806e730 0x7fcd381ed950 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fcd280b3040 tx=0x7fcd28005760 comp rx=0 tx=0).stop 2024-09-10T21:25:19.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.834+0000 7fcd3e400640 1 -- 172.21.15.17:0/3359527135 reap_dead start 2024-09-10T21:25:19.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.838+0000 7f4ebe000640 1 -- 172.21.15.17:0/1926711127 <== mon.1 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f4eb8005230 con 0x7f4ec4153d40 2024-09-10T21:25:19.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.838+0000 7f514f000640 1 -- 172.21.15.17:0/2653535512 --> [v2:172.21.15.17:6816/1248243634,v1:172.21.15.17:6817/1248243634] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5120002de0 con 0x7f512807be60 2024-09-10T21:25:19.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.838+0000 7f4ebe000640 1 --2- 172.21.15.17:0/1926711127 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f4ea8072610 0x7f4ea8074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:19.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.838+0000 7fcd26a00640 1 -- 172.21.15.17:0/3359527135 shutdown_connections 2024-09-10T21:25:19.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.838+0000 7fcd26a00640 1 -- 172.21.15.17:0/3359527135 >> 172.21.15.17:0/3359527135 conn(0x7fcd380700d0 msgr2=0x7fcd38159570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:19.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.838+0000 7f513fe00640 1 -- 172.21.15.17:0/2653535512 <== osd.2 v2:172.21.15.17:6816/1248243634 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f5120002de0 con 0x7f512807be60 2024-09-10T21:25:19.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.838+0000 7f4ebe000640 1 -- 172.21.15.17:0/1926711127 <== mon.1 v2:172.21.15.31:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f4eb8131320 con 0x7f4ec4153d40 2024-09-10T21:25:19.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.838+0000 7fcd26a00640 1 -- 172.21.15.17:0/3359527135 shutdown_connections 2024-09-10T21:25:19.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.838+0000 7fcd26a00640 1 -- 172.21.15.17:0/3359527135 wait complete. 2024-09-10T21:25:19.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.838+0000 7f4ec9e00640 1 --2- 172.21.15.17:0/1926711127 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f4ea8072610 0x7f4ea8074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:19.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.838+0000 7f4ebe000640 1 --2- 172.21.15.17:0/1926711127 >> [v2:172.21.15.31:6818/3948648934,v1:172.21.15.31:6819/3948648934] conn(0x7f4ea807bd10 0x7f4ea807e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:19.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.838+0000 7f4ebe000640 1 -- 172.21.15.17:0/1926711127 --> [v2:172.21.15.31:6818/3948648934,v1:172.21.15.31:6819/3948648934] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4eb80bee80 con 0x7f4ea807bd10 2024-09-10T21:25:19.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.838+0000 7f4ebe000640 1 -- 172.21.15.17:0/1926711127 <== mon.1 v2:172.21.15.31:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4eb8131710 con 0x7f4ec4153d40 2024-09-10T21:25:19.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.838+0000 7f514f000640 1 -- 172.21.15.17:0/2653535512 >> [v2:172.21.15.17:6816/1248243634,v1:172.21.15.17:6817/1248243634] conn(0x7f512807be60 msgr2=0x7f512807e2a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:19.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.838+0000 7f514f000640 1 --2- 172.21.15.17:0/2653535512 >> [v2:172.21.15.17:6816/1248243634,v1:172.21.15.17:6817/1248243634] conn(0x7f512807be60 0x7f512807e2a0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:19.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.838+0000 7f514f000640 1 -- 172.21.15.17:0/2653535512 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f51280726e0 msgr2=0x7f5128074ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:19.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.838+0000 7f514f000640 1 --2- 172.21.15.17:0/2653535512 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f51280726e0 0x7f5128074ba0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f51380b2000 tx=0x7f513801bdb0 comp rx=0 tx=0).stop 2024-09-10T21:25:19.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.838+0000 7f514f000640 1 -- 172.21.15.17:0/2653535512 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f51480c6f40 msgr2=0x7f51480c4480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:19.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.838+0000 7f514f000640 1 --2- 172.21.15.17:0/2653535512 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f51480c6f40 0x7f51480c4480 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f513400c4b0 tx=0x7f513400c970 comp rx=0 tx=0).stop 2024-09-10T21:25:19.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.838+0000 7f514d000640 1 -- 172.21.15.17:0/2653535512 reap_dead start 2024-09-10T21:25:19.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.838+0000 7f514f000640 1 -- 172.21.15.17:0/2653535512 shutdown_connections 2024-09-10T21:25:19.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.838+0000 7f514f000640 1 -- 172.21.15.17:0/2653535512 >> 172.21.15.17:0/2653535512 conn(0x7f51480700d0 msgr2=0x7f51481e2f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:19.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.838+0000 7f4ec9e00640 1 --2- 172.21.15.17:0/1926711127 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f4ea8072610 0x7f4ea8074ad0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f4ec0007b80 tx=0x7f4ec0005ef0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:19.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.842+0000 7f4ec9400640 1 --2- 172.21.15.17:0/1926711127 >> [v2:172.21.15.31:6818/3948648934,v1:172.21.15.31:6819/3948648934] conn(0x7f4ea807bd10 0x7f4ea807e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:19.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.842+0000 7f514f000640 1 -- 172.21.15.17:0/2653535512 shutdown_connections 2024-09-10T21:25:19.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.842+0000 7f514f000640 1 -- 172.21.15.17:0/2653535512 wait complete. 2024-09-10T21:25:19.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.842+0000 7f4ec9400640 1 --2- 172.21.15.17:0/1926711127 >> [v2:172.21.15.31:6818/3948648934,v1:172.21.15.31:6819/3948648934] conn(0x7f4ea807bd10 0x7f4ea807e150 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:19.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.842+0000 7f4ebe000640 1 -- 172.21.15.17:0/1926711127 <== osd.7 v2:172.21.15.31:6818/3948648934 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4eb80bee80 con 0x7f4ea807bd10 2024-09-10T21:25:19.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.846+0000 7f448be00640 1 -- 172.21.15.17:0/1053768786 --> [v2:172.21.15.17:6808/4105489867,v1:172.21.15.17:6809/4105489867] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f44840dec50 con 0x7f446c07bd50 2024-09-10T21:25:19.855 INFO:teuthology.orchestra.run.smithi017.stdout:64424509451 2024-09-10T21:25:19.856 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-10T21:25:19.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.850+0000 7f4482a00640 1 -- 172.21.15.17:0/1053768786 <== osd.0 v2:172.21.15.17:6808/4105489867 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f44840dec50 con 0x7f446c07bd50 2024-09-10T21:25:19.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.854+0000 7f94b6a00640 1 -- 172.21.15.17:0/1458225254 --> [v2:172.21.15.31:6810/202520744,v1:172.21.15.31:6811/202520744] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9498002de0 con 0x7f94a407be30 2024-09-10T21:25:19.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.858+0000 7f94c1600640 1 -- 172.21.15.17:0/1458225254 <== osd.4 v2:172.21.15.31:6810/202520744 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f9498002de0 con 0x7f94a407be30 2024-09-10T21:25:19.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.858+0000 7f448be00640 1 -- 172.21.15.17:0/1053768786 >> [v2:172.21.15.17:6808/4105489867,v1:172.21.15.17:6809/4105489867] conn(0x7f446c07bd50 msgr2=0x7f446c07e190 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:19.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.858+0000 7f448be00640 1 --2- 172.21.15.17:0/1053768786 >> [v2:172.21.15.17:6808/4105489867,v1:172.21.15.17:6809/4105489867] conn(0x7f446c07bd50 0x7f446c07e190 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:19.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.858+0000 7f448be00640 1 -- 172.21.15.17:0/1053768786 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f446c0725e0 msgr2=0x7f446c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:19.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.858+0000 7f448be00640 1 --2- 172.21.15.17:0/1053768786 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f446c0725e0 0x7f446c074aa0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f447c01c940 tx=0x7f447c0198b0 comp rx=0 tx=0).stop 2024-09-10T21:25:19.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.858+0000 7f448be00640 1 -- 172.21.15.17:0/1053768786 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4484052040 msgr2=0x7f44840524a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:19.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.858+0000 7f448be00640 1 --2- 172.21.15.17:0/1053768786 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4484052040 0x7f44840524a0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f448c06ba20 tx=0x7f448c06c890 comp rx=0 tx=0).stop 2024-09-10T21:25:19.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.858+0000 7f448b400640 1 -- 172.21.15.17:0/1053768786 reap_dead start 2024-09-10T21:25:19.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.858+0000 7f94b6a00640 1 -- 172.21.15.17:0/1458225254 >> [v2:172.21.15.31:6810/202520744,v1:172.21.15.31:6811/202520744] conn(0x7f94a407be30 msgr2=0x7f94a407e270 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:19.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.858+0000 7f94b6a00640 1 --2- 172.21.15.17:0/1458225254 >> [v2:172.21.15.31:6810/202520744,v1:172.21.15.31:6811/202520744] conn(0x7f94a407be30 0x7f94a407e270 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:19.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.858+0000 7f94b6a00640 1 -- 172.21.15.17:0/1458225254 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f94a40726b0 msgr2=0x7f94a4074b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:19.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.858+0000 7f94b6a00640 1 --2- 172.21.15.17:0/1458225254 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f94a40726b0 0x7f94a4074b70 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f94b8001890 tx=0x7f94b8001600 comp rx=0 tx=0).stop 2024-09-10T21:25:19.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.858+0000 7f94b6a00640 1 -- 172.21.15.17:0/1458225254 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f94c4072470 msgr2=0x7f94c41446d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:19.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.858+0000 7f94b6a00640 1 --2- 172.21.15.17:0/1458225254 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f94c4072470 0x7f94c41446d0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f94b000e9a0 tx=0x7f94b000ee60 comp rx=0 tx=0).stop 2024-09-10T21:25:19.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.862+0000 7f94ca200640 1 -- 172.21.15.17:0/1458225254 reap_dead start 2024-09-10T21:25:19.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.862+0000 7f448be00640 1 -- 172.21.15.17:0/1053768786 shutdown_connections 2024-09-10T21:25:19.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.862+0000 7f448be00640 1 -- 172.21.15.17:0/1053768786 >> 172.21.15.17:0/1053768786 conn(0x7f4484054000 msgr2=0x7f44840e2370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:19.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.862+0000 7f94b6a00640 1 -- 172.21.15.17:0/1458225254 shutdown_connections 2024-09-10T21:25:19.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.862+0000 7f94b6a00640 1 -- 172.21.15.17:0/1458225254 >> 172.21.15.17:0/1458225254 conn(0x7f94c406fe10 msgr2=0x7f94c4148d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:19.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.862+0000 7f94b6a00640 1 -- 172.21.15.17:0/1458225254 shutdown_connections 2024-09-10T21:25:19.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.862+0000 7f94b6a00640 1 -- 172.21.15.17:0/1458225254 wait complete. 2024-09-10T21:25:19.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.866+0000 7f448be00640 1 -- 172.21.15.17:0/1053768786 shutdown_connections 2024-09-10T21:25:19.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.866+0000 7f448be00640 1 -- 172.21.15.17:0/1053768786 wait complete. 2024-09-10T21:25:19.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.870+0000 7f9c6b200640 1 -- 172.21.15.17:0/483186392 --> [v2:172.21.15.17:6800/2314489522,v1:172.21.15.17:6801/2314489522] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9c640747d0 con 0x7f9c4407be60 2024-09-10T21:25:19.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.870+0000 7f11aec00640 1 -- 172.21.15.17:0/3538369118 --> [v2:172.21.15.17:6824/2599842027,v1:172.21.15.17:6825/2599842027] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f11a80c73c0 con 0x7f118807bda0 2024-09-10T21:25:19.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.870+0000 7f119fe00640 1 -- 172.21.15.17:0/3538369118 <== osd.1 v2:172.21.15.17:6824/2599842027 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f11a80c73c0 con 0x7f118807bda0 2024-09-10T21:25:19.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.874+0000 7f9c5be00640 1 -- 172.21.15.17:0/483186392 <== osd.3 v2:172.21.15.17:6800/2314489522 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f9c640747d0 con 0x7f9c4407be60 2024-09-10T21:25:19.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.874+0000 7f9c6b200640 1 -- 172.21.15.17:0/483186392 >> [v2:172.21.15.17:6800/2314489522,v1:172.21.15.17:6801/2314489522] conn(0x7f9c4407be60 msgr2=0x7f9c4407e2a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:19.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.874+0000 7f9c6b200640 1 --2- 172.21.15.17:0/483186392 >> [v2:172.21.15.17:6800/2314489522,v1:172.21.15.17:6801/2314489522] conn(0x7f9c4407be60 0x7f9c4407e2a0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:19.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.874+0000 7f11aec00640 1 -- 172.21.15.17:0/3538369118 >> [v2:172.21.15.17:6824/2599842027,v1:172.21.15.17:6825/2599842027] conn(0x7f118807bda0 msgr2=0x7f118807e1e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:19.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.874+0000 7f11aec00640 1 --2- 172.21.15.17:0/3538369118 >> [v2:172.21.15.17:6824/2599842027,v1:172.21.15.17:6825/2599842027] conn(0x7f118807bda0 0x7f118807e1e0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:19.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.874+0000 7f11aec00640 1 -- 172.21.15.17:0/3538369118 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f1188072630 msgr2=0x7f1188074af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:19.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.874+0000 7f11aec00640 1 --2- 172.21.15.17:0/3538369118 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f1188072630 0x7f1188074af0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f1198006fd0 tx=0x7f1198006b80 comp rx=0 tx=0).stop 2024-09-10T21:25:19.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.874+0000 7f11aec00640 1 -- 172.21.15.17:0/3538369118 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f11a81e3190 msgr2=0x7f11a81e1aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:19.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.874+0000 7f11aec00640 1 --2- 172.21.15.17:0/3538369118 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f11a81e3190 0x7f11a81e1aa0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f1194006480 tx=0x7f11940039d0 comp rx=0 tx=0).stop 2024-09-10T21:25:19.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.874+0000 7f11acc00640 1 -- 172.21.15.17:0/3538369118 reap_dead start 2024-09-10T21:25:19.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.874+0000 7f11aec00640 1 -- 172.21.15.17:0/3538369118 shutdown_connections 2024-09-10T21:25:19.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.874+0000 7f11aec00640 1 -- 172.21.15.17:0/3538369118 >> 172.21.15.17:0/3538369118 conn(0x7f11a80700d0 msgr2=0x7f11a80704b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:19.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.874+0000 7f9c6b200640 1 -- 172.21.15.17:0/483186392 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f9c440726e0 msgr2=0x7f9c44074ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:19.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.874+0000 7f9c6b200640 1 --2- 172.21.15.17:0/483186392 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f9c440726e0 0x7f9c44074ba0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f9c641506d0 tx=0x7f9c50007f20 comp rx=0 tx=0).stop 2024-09-10T21:25:19.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.874+0000 7f9c6b200640 1 -- 172.21.15.17:0/483186392 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9c6406dd10 msgr2=0x7f9c64147dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:19.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.874+0000 7f9c6b200640 1 --2- 172.21.15.17:0/483186392 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9c6406dd10 0x7f9c64147dc0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f9c54007320 tx=0x7f9c54008bc0 comp rx=0 tx=0).stop 2024-09-10T21:25:19.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.874+0000 7f9c69200640 1 -- 172.21.15.17:0/483186392 reap_dead start 2024-09-10T21:25:19.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.874+0000 7f9c6b200640 1 -- 172.21.15.17:0/483186392 shutdown_connections 2024-09-10T21:25:19.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.874+0000 7f9c6b200640 1 -- 172.21.15.17:0/483186392 >> 172.21.15.17:0/483186392 conn(0x7f9c640700d0 msgr2=0x7f9c640cab40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:19.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.874+0000 7f9c6b200640 1 -- 172.21.15.17:0/483186392 shutdown_connections 2024-09-10T21:25:19.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.874+0000 7f9c6b200640 1 -- 172.21.15.17:0/483186392 wait complete. 2024-09-10T21:25:19.881 INFO:teuthology.orchestra.run.smithi017.stdout:55834574859 2024-09-10T21:25:19.881 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-10T21:25:19.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.878+0000 7f4ecc800640 1 -- 172.21.15.17:0/1926711127 --> [v2:172.21.15.31:6818/3948648934,v1:172.21.15.31:6819/3948648934] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4ec40c78a0 con 0x7f4ea807bd10 2024-09-10T21:25:19.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.878+0000 7f11aec00640 1 -- 172.21.15.17:0/3538369118 shutdown_connections 2024-09-10T21:25:19.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.878+0000 7f11aec00640 1 -- 172.21.15.17:0/3538369118 wait complete. 2024-09-10T21:25:19.882 INFO:teuthology.orchestra.run.smithi017.stdout:64424509450 2024-09-10T21:25:19.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.878+0000 7f4ebe000640 1 -- 172.21.15.17:0/1926711127 <== osd.7 v2:172.21.15.31:6818/3948648934 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f4ec40c78a0 con 0x7f4ea807bd10 2024-09-10T21:25:19.883 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-10T21:25:19.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.878+0000 7f4ecc800640 1 -- 172.21.15.17:0/1926711127 >> [v2:172.21.15.31:6818/3948648934,v1:172.21.15.31:6819/3948648934] conn(0x7f4ea807bd10 msgr2=0x7f4ea807e150 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:19.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.878+0000 7f4ecc800640 1 --2- 172.21.15.17:0/1926711127 >> [v2:172.21.15.31:6818/3948648934,v1:172.21.15.31:6819/3948648934] conn(0x7f4ea807bd10 0x7f4ea807e150 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:19.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.878+0000 7f4ecc800640 1 -- 172.21.15.17:0/1926711127 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f4ea8072610 msgr2=0x7f4ea8074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:19.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.878+0000 7f4ecc800640 1 --2- 172.21.15.17:0/1926711127 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f4ea8072610 0x7f4ea8074ad0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f4ec0007b80 tx=0x7f4ec0005ef0 comp rx=0 tx=0).stop 2024-09-10T21:25:19.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.878+0000 7f4ecc800640 1 -- 172.21.15.17:0/1926711127 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4ec4153d40 msgr2=0x7f4ec4140b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:19.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.878+0000 7f4ecc800640 1 --2- 172.21.15.17:0/1926711127 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4ec4153d40 0x7f4ec4140b60 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f4eb8008330 tx=0x7f4eb8009e30 comp rx=0 tx=0).stop 2024-09-10T21:25:19.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.878+0000 7f4eca800640 1 -- 172.21.15.17:0/1926711127 reap_dead start 2024-09-10T21:25:19.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.878+0000 7f4ecc800640 1 -- 172.21.15.17:0/1926711127 shutdown_connections 2024-09-10T21:25:19.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.878+0000 7f4ecc800640 1 -- 172.21.15.17:0/1926711127 >> 172.21.15.17:0/1926711127 conn(0x7f4ec40700d0 msgr2=0x7f4ec4074010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:19.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.878+0000 7f4ecc800640 1 -- 172.21.15.17:0/1926711127 shutdown_connections 2024-09-10T21:25:19.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:19.878+0000 7f4ecc800640 1 -- 172.21.15.17:0/1926711127 wait complete. 2024-09-10T21:25:19.902 INFO:teuthology.orchestra.run.smithi017.stdout:60129542157 2024-09-10T21:25:19.902 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-10T21:25:19.904 INFO:teuthology.orchestra.run.smithi017.stdout:60129542155 2024-09-10T21:25:19.905 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-10T21:25:19.916 INFO:teuthology.orchestra.run.smithi017.stdout:60129542154 2024-09-10T21:25:19.916 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-10T21:25:19.922 INFO:teuthology.orchestra.run.smithi017.stdout:64424509451 2024-09-10T21:25:19.922 INFO:teuthology.orchestra.run.smithi017.stdout:64424509450 2024-09-10T21:25:19.922 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-10T21:25:19.923 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-10T21:25:20.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.106+0000 7fee96000640 1 -- 172.21.15.17:0/851243084 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fee74004810 msgr2=0x7fee74004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:20.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.106+0000 7fee96000640 1 --2- 172.21.15.17:0/851243084 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fee74004810 0x7fee74004c90 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fee80006eb0 tx=0x7fee8001ce40 comp rx=0 tx=0).stop 2024-09-10T21:25:20.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.106+0000 7fee96000640 1 -- 172.21.15.17:0/851243084 shutdown_connections 2024-09-10T21:25:20.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.106+0000 7fee96000640 1 --2- 172.21.15.17:0/851243084 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fee74004810 0x7fee74004c90 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:20.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.106+0000 7fee96000640 1 -- 172.21.15.17:0/851243084 >> 172.21.15.17:0/851243084 conn(0x7fee9006fe60 msgr2=0x7fee90070270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:20.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.106+0000 7fee96000640 1 -- 172.21.15.17:0/851243084 shutdown_connections 2024-09-10T21:25:20.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.106+0000 7fee96000640 1 -- 172.21.15.17:0/851243084 wait complete. 2024-09-10T21:25:20.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.110+0000 7fee96000640 1 Processor -- start 2024-09-10T21:25:20.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.110+0000 7fee96000640 1 -- start start 2024-09-10T21:25:20.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.114+0000 7fee96000640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fee74004810 0x7fee9014bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:20.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.114+0000 7fee96000640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fee90152160 0x7fee901525a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:20.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.114+0000 7fee94c00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fee74004810 0x7fee9014bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:20.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.114+0000 7fee94c00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fee74004810 0x7fee9014bc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.17:51400/0 (socket says 172.21.15.17:51400) 2024-09-10T21:25:20.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.114+0000 7fee94c00640 1 -- 172.21.15.17:0/1995537471 learned_addr learned my addr 172.21.15.17:0/1995537471 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:25:20.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.114+0000 7fee96000640 1 --2- 172.21.15.17:0/1995537471 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fee90148900 0x7fee90148d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:20.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.114+0000 7fee96000640 1 -- 172.21.15.17:0/1995537471 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fee90157c10 con 0x7fee90148900 2024-09-10T21:25:20.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.114+0000 7fee96000640 1 -- 172.21.15.17:0/1995537471 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fee90157930 con 0x7fee74004810 2024-09-10T21:25:20.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.114+0000 7fee96000640 1 -- 172.21.15.17:0/1995537471 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7fee901518e0 con 0x7fee90152160 2024-09-10T21:25:20.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.114+0000 7fee8fe00640 1 --2- 172.21.15.17:0/1995537471 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fee90152160 0x7fee901525a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:20.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.114+0000 7fee94c00640 1 -- 172.21.15.17:0/1995537471 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fee90152160 msgr2=0x7fee901525a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:20.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.114+0000 7fee94c00640 1 --2- 172.21.15.17:0/1995537471 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fee90152160 0x7fee901525a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:20.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.114+0000 7fee94c00640 1 -- 172.21.15.17:0/1995537471 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fee90148900 msgr2=0x7fee90148d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:20.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.114+0000 7fee94c00640 1 --2- 172.21.15.17:0/1995537471 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fee90148900 0x7fee90148d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:20.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.114+0000 7fee94c00640 1 -- 172.21.15.17:0/1995537471 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee9014cb80 con 0x7fee74004810 2024-09-10T21:25:20.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.114+0000 7fee8fe00640 1 --2- 172.21.15.17:0/1995537471 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fee90152160 0x7fee901525a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:25:20.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.114+0000 7fee94c00640 1 --2- 172.21.15.17:0/1995537471 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fee74004810 0x7fee9014bc00 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fee7c004da0 tx=0x7fee7c001640 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:20.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.114+0000 7fee8cc00640 1 -- 172.21.15.17:0/1995537471 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee7c015020 con 0x7fee74004810 2024-09-10T21:25:20.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.114+0000 7fee8cc00640 1 -- 172.21.15.17:0/1995537471 <== mon.1 v2:172.21.15.31:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fee7c011760 con 0x7fee74004810 2024-09-10T21:25:20.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.114+0000 7fee8cc00640 1 -- 172.21.15.17:0/1995537471 <== mon.1 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee7c011bb0 con 0x7fee74004810 2024-09-10T21:25:20.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.118+0000 7fee96000640 1 -- 172.21.15.17:0/1995537471 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee9014c8e0 con 0x7fee74004810 2024-09-10T21:25:20.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.118+0000 7fee96000640 1 -- 172.21.15.17:0/1995537471 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee900c7070 con 0x7fee74004810 2024-09-10T21:25:20.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.118+0000 7fee96000640 1 -- 172.21.15.17:0/1995537471 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee90074e10 con 0x7fee74004810 2024-09-10T21:25:20.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.122+0000 7fee8cc00640 1 -- 172.21.15.17:0/1995537471 <== mon.1 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fee7c024440 con 0x7fee74004810 2024-09-10T21:25:20.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.122+0000 7fee8cc00640 1 --2- 172.21.15.17:0/1995537471 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fee70072610 0x7fee70074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:20.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.122+0000 7fee8cc00640 1 -- 172.21.15.17:0/1995537471 <== mon.1 v2:172.21.15.31:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fee7c094760 con 0x7fee74004810 2024-09-10T21:25:20.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.122+0000 7fee8fe00640 1 --2- 172.21.15.17:0/1995537471 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fee70072610 0x7fee70074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:20.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.126+0000 7fee8cc00640 1 -- 172.21.15.17:0/1995537471 <== mon.1 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fee7c01f020 con 0x7fee74004810 2024-09-10T21:25:20.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.126+0000 7fee8fe00640 1 --2- 172.21.15.17:0/1995537471 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fee70072610 0x7fee70074ad0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fee800ab000 tx=0x7fee80019c80 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:20.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.126+0000 7f299f600640 1 -- 172.21.15.17:0/1466139217 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f29981548b0 msgr2=0x7f299815afb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:20.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.126+0000 7f299f600640 1 --2- 172.21.15.17:0/1466139217 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f29981548b0 0x7f299815afb0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f2988009900 tx=0x7f298801f0f0 comp rx=0 tx=0).stop 2024-09-10T21:25:20.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.130+0000 7f299f600640 1 -- 172.21.15.17:0/1466139217 shutdown_connections 2024-09-10T21:25:20.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.130+0000 7f299f600640 1 --2- 172.21.15.17:0/1466139217 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f299806e730 0x7f29980c8dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:20.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.130+0000 7f299f600640 1 --2- 172.21.15.17:0/1466139217 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f29981548b0 0x7f299815afb0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:20.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.130+0000 7f299f600640 1 -- 172.21.15.17:0/1466139217 >> 172.21.15.17:0/1466139217 conn(0x7f29980700d0 msgr2=0x7f29980704e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:20.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.130+0000 7f299f600640 1 -- 172.21.15.17:0/1466139217 shutdown_connections 2024-09-10T21:25:20.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.130+0000 7f299f600640 1 -- 172.21.15.17:0/1466139217 wait complete. 2024-09-10T21:25:20.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.130+0000 7f25fd400640 1 -- 172.21.15.17:0/1622957893 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f25f8152820 msgr2=0x7f25f80b3eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:20.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.130+0000 7f25fd400640 1 --2- 172.21.15.17:0/1622957893 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f25f8152820 0x7f25f80b3eb0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f25e8009870 tx=0x7f25e801c7f0 comp rx=0 tx=0).stop 2024-09-10T21:25:20.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.134+0000 7f299f600640 1 Processor -- start 2024-09-10T21:25:20.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.134+0000 7f299f600640 1 -- start start 2024-09-10T21:25:20.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.134+0000 7f299f600640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f299806e730 0x7f29980cb210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:20.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.134+0000 7f299f600640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f29980c6f40 0x7f29981f88d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:20.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.134+0000 7f299f600640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f29981548b0 0x7f29981e4080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:20.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.134+0000 7f25fd400640 1 -- 172.21.15.17:0/1622957893 shutdown_connections 2024-09-10T21:25:20.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.134+0000 7f25fd400640 1 --2- 172.21.15.17:0/1622957893 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f25f8152820 0x7f25f80b3eb0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:20.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.134+0000 7f25fd400640 1 -- 172.21.15.17:0/1622957893 >> 172.21.15.17:0/1622957893 conn(0x7f25f80702f0 msgr2=0x7f25f8070700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:20.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.134+0000 7f25fd400640 1 -- 172.21.15.17:0/1622957893 shutdown_connections 2024-09-10T21:25:20.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.134+0000 7f2997e00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f29980c6f40 0x7f29981f88d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:20.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.134+0000 7f299d600640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f29981548b0 0x7f29981e4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:20.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.134+0000 7f299d600640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f29981548b0 0x7f29981e4080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3301/0 says I am v2:172.21.15.17:43038/0 (socket says 172.21.15.17:43038) 2024-09-10T21:25:20.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.134+0000 7f299d600640 1 -- 172.21.15.17:0/4102610328 learned_addr learned my addr 172.21.15.17:0/4102610328 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:25:20.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.134+0000 7f25fd400640 1 -- 172.21.15.17:0/1622957893 wait complete. 2024-09-10T21:25:20.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.138+0000 7f299f600640 1 -- 172.21.15.17:0/4102610328 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f29980c85c0 con 0x7f299806e730 2024-09-10T21:25:20.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.138+0000 7f299f600640 1 -- 172.21.15.17:0/4102610328 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f29980c82e0 con 0x7f29980c6f40 2024-09-10T21:25:20.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.138+0000 7f299f600640 1 -- 172.21.15.17:0/4102610328 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f29980c88a0 con 0x7f29981548b0 2024-09-10T21:25:20.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.138+0000 7f299cc00640 1 --2- 172.21.15.17:0/4102610328 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f299806e730 0x7f29980cb210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:20.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.138+0000 7f299d600640 1 -- 172.21.15.17:0/4102610328 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f29980c6f40 msgr2=0x7f29981f88d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:20.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.138+0000 7f25fd400640 1 Processor -- start 2024-09-10T21:25:20.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.138+0000 7f299d600640 1 --2- 172.21.15.17:0/4102610328 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f29980c6f40 0x7f29981f88d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:20.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.138+0000 7f299d600640 1 -- 172.21.15.17:0/4102610328 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f299806e730 msgr2=0x7f29980cb210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:20.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.138+0000 7f299d600640 1 --2- 172.21.15.17:0/4102610328 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f299806e730 0x7f29980cb210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:20.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.138+0000 7f299d600640 1 -- 172.21.15.17:0/4102610328 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29980cc560 con 0x7f29981548b0 2024-09-10T21:25:20.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.138+0000 7f299cc00640 1 --2- 172.21.15.17:0/4102610328 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f299806e730 0x7f29980cb210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:25:20.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.138+0000 7f299d600640 1 --2- 172.21.15.17:0/4102610328 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f29981548b0 0x7f29981e4080 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f299000c7a0 tx=0x7f299000cc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:20.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.138+0000 7f25fd400640 1 -- start start 2024-09-10T21:25:20.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.138+0000 7f2994c00640 1 -- 172.21.15.17:0/4102610328 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2990003b80 con 0x7f29981548b0 2024-09-10T21:25:20.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.138+0000 7f299f600640 1 -- 172.21.15.17:0/4102610328 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29981f93c0 con 0x7f29981548b0 2024-09-10T21:25:20.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.138+0000 7f299f600640 1 -- 172.21.15.17:0/4102610328 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f29980cacf0 con 0x7f29981548b0 2024-09-10T21:25:20.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.138+0000 7f25fd400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f25f80cdf20 0x7f25f81fb950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:20.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.138+0000 7f2994c00640 1 -- 172.21.15.17:0/4102610328 <== mon.2 v2:172.21.15.31:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2990003d20 con 0x7f29981548b0 2024-09-10T21:25:20.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.138+0000 7f2994c00640 1 -- 172.21.15.17:0/4102610328 <== mon.2 v2:172.21.15.31:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2990003b80 con 0x7f29981548b0 2024-09-10T21:25:20.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.142+0000 7f25f6800640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f25f80cdf20 0x7f25f81fb950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:20.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.142+0000 7f25f6800640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f25f80cdf20 0x7f25f81fb950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44398/0 (socket says 172.21.15.17:44398) 2024-09-10T21:25:20.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.142+0000 7f299f600640 1 -- 172.21.15.17:0/4102610328 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29700052a0 con 0x7f29981548b0 2024-09-10T21:25:20.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.142+0000 7f2994c00640 1 -- 172.21.15.17:0/4102610328 <== mon.2 v2:172.21.15.31:3301/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f299000a850 con 0x7f29981548b0 2024-09-10T21:25:20.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.146+0000 7f2994c00640 1 --2- 172.21.15.17:0/4102610328 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f2978072810 0x7f2978074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:20.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.146+0000 7f2994c00640 1 -- 172.21.15.17:0/4102610328 <== mon.2 v2:172.21.15.31:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f2990092e20 con 0x7f29981548b0 2024-09-10T21:25:20.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.146+0000 7f299cc00640 1 --2- 172.21.15.17:0/4102610328 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f2978072810 0x7f2978074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:20.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.142+0000 7f25fd400640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f25f8152440 0x7f25f81f8210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:20.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.142+0000 7f25fd400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f25f8152820 0x7f25f806eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:20.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.142+0000 7f25fd400640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f25f80cb5d0 con 0x7f25f80cdf20 2024-09-10T21:25:20.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.142+0000 7f25fd400640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f25f80cb450 con 0x7f25f8152820 2024-09-10T21:25:20.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.142+0000 7f25fd400640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f25f80c2ff0 con 0x7f25f8152440 2024-09-10T21:25:20.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.142+0000 7f25f6800640 1 -- 172.21.15.17:0/296409905 learned_addr learned my addr 172.21.15.17:0/296409905 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:25:20.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.142+0000 7f25f7200640 1 --2- 172.21.15.17:0/296409905 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f25f8152820 0x7f25f806eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:20.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.142+0000 7f25f6800640 1 -- 172.21.15.17:0/296409905 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f25f8152440 msgr2=0x7f25f81f8210 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-10T21:25:20.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.142+0000 7f25f6800640 1 --2- 172.21.15.17:0/296409905 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f25f8152440 0x7f25f81f8210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:20.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.142+0000 7f25f6800640 1 -- 172.21.15.17:0/296409905 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f25f8152820 msgr2=0x7f25f806eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:20.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.142+0000 7f25f6800640 1 --2- 172.21.15.17:0/296409905 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f25f8152820 0x7f25f806eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:20.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.142+0000 7f25f6800640 1 -- 172.21.15.17:0/296409905 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25f8159870 con 0x7f25f80cdf20 2024-09-10T21:25:20.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.142+0000 7f25f6800640 1 --2- 172.21.15.17:0/296409905 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f25f80cdf20 0x7f25f81fb950 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f25e000c4b0 tx=0x7f25e000c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:20.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.142+0000 7f25ef400640 1 -- 172.21.15.17:0/296409905 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25e00052d0 con 0x7f25f80cdf20 2024-09-10T21:25:20.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.142+0000 7f25ef400640 1 -- 172.21.15.17:0/296409905 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f25e0005470 con 0x7f25f80cdf20 2024-09-10T21:25:20.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.142+0000 7f25ef400640 1 -- 172.21.15.17:0/296409905 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25e0012870 con 0x7f25f80cdf20 2024-09-10T21:25:20.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.142+0000 7f25fd400640 1 -- 172.21.15.17:0/296409905 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25f8159b10 con 0x7f25f80cdf20 2024-09-10T21:25:20.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.142+0000 7f25fd400640 1 -- 172.21.15.17:0/296409905 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25f81f7a10 con 0x7f25f80cdf20 2024-09-10T21:25:20.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.146+0000 7f299cc00640 1 --2- 172.21.15.17:0/4102610328 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f2978072810 0x7f2978074cd0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f29880b2000 tx=0x7f298801d930 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:20.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.146+0000 7f25fd400640 1 -- 172.21.15.17:0/296409905 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25d00052a0 con 0x7f25f80cdf20 2024-09-10T21:25:20.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.150+0000 7f2994c00640 1 -- 172.21.15.17:0/4102610328 <== mon.2 v2:172.21.15.31:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f299005e1b0 con 0x7f29981548b0 2024-09-10T21:25:20.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.150+0000 7f25ef400640 1 -- 172.21.15.17:0/296409905 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f25e0012a10 con 0x7f25f80cdf20 2024-09-10T21:25:20.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.154+0000 7f25ef400640 1 --2- 172.21.15.17:0/296409905 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f25d40726e0 0x7f25d4074ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:20.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.154+0000 7f25ef400640 1 -- 172.21.15.17:0/296409905 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f25e0093020 con 0x7f25f80cdf20 2024-09-10T21:25:20.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.154+0000 7f25ede00640 1 --2- 172.21.15.17:0/296409905 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f25d40726e0 0x7f25d4074ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:20.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.154+0000 7f25ede00640 1 --2- 172.21.15.17:0/296409905 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f25d40726e0 0x7f25d4074ba0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f25e80af000 tx=0x7f25e801b030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:20.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.154+0000 7f94c5800640 1 -- 172.21.15.17:0/972498786 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f94c00b2d00 msgr2=0x7f94a4004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:20.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.154+0000 7f94c5800640 1 --2- 172.21.15.17:0/972498786 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f94c00b2d00 0x7f94a4004850 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f94b000bb10 tx=0x7f94b001ce00 comp rx=0 tx=0).stop 2024-09-10T21:25:20.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.154+0000 7f94bf400640 1 -- 172.21.15.17:0/972498786 reap_dead start 2024-09-10T21:25:20.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.154+0000 7f94c5800640 1 -- 172.21.15.17:0/972498786 shutdown_connections 2024-09-10T21:25:20.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.154+0000 7f94c5800640 1 -- 172.21.15.17:0/972498786 >> 172.21.15.17:0/972498786 conn(0x7f94c0068990 msgr2=0x7f94c0068da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:20.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.154+0000 7f94c5800640 1 -- 172.21.15.17:0/972498786 shutdown_connections 2024-09-10T21:25:20.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.154+0000 7f94c5800640 1 -- 172.21.15.17:0/972498786 wait complete. 2024-09-10T21:25:20.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.158+0000 7f25ef400640 1 -- 172.21.15.17:0/296409905 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f25e005e3a0 con 0x7f25f80cdf20 2024-09-10T21:25:20.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.158+0000 7f336dc00640 1 -- 172.21.15.17:0/4033312787 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3360004a40 msgr2=0x7f336004ad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:20.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.158+0000 7f336dc00640 1 --2- 172.21.15.17:0/4033312787 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3360004a40 0x7f336004ad00 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f336004ac20 tx=0x7f335c027430 comp rx=0 tx=0).stop 2024-09-10T21:25:20.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.158+0000 7f94c5800640 1 Processor -- start 2024-09-10T21:25:20.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.158+0000 7f336dc00640 1 -- 172.21.15.17:0/4033312787 shutdown_connections 2024-09-10T21:25:20.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.158+0000 7f336dc00640 1 --2- 172.21.15.17:0/4033312787 >> v2:172.21.15.31:3300/0 conn(0x7f336004c3b0 0x7f336004b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:20.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.158+0000 7f336dc00640 1 --2- 172.21.15.17:0/4033312787 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3360004a40 0x7f336004ad00 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:20.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.158+0000 7f336dc00640 1 --2- 172.21.15.17:0/4033312787 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f336004baf0 0x7f33600db010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:20.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.158+0000 7f336dc00640 1 -- 172.21.15.17:0/4033312787 >> 172.21.15.17:0/4033312787 conn(0x7f3360050060 msgr2=0x7f3360050470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:20.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.158+0000 7f336dc00640 1 -- 172.21.15.17:0/4033312787 shutdown_connections 2024-09-10T21:25:20.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.158+0000 7f336dc00640 1 -- 172.21.15.17:0/4033312787 wait complete. 2024-09-10T21:25:20.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.162+0000 7f336dc00640 1 Processor -- start 2024-09-10T21:25:20.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.158+0000 7f94c5800640 1 -- start start 2024-09-10T21:25:20.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.162+0000 7f94c5800640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f94c00b2d00 0x7f94c0149d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:20.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.162+0000 7f94c5800640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f94c01f4750 0x7f94c01f4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:20.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.162+0000 7f94c5800640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f94c01f7b80 0x7f94c01f8000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:20.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.162+0000 7f94c5800640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f94c00d0b50 con 0x7f94c00b2d00 2024-09-10T21:25:20.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.162+0000 7f94c5800640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f94c00d09d0 con 0x7f94c01f7b80 2024-09-10T21:25:20.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.162+0000 7f94c5800640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f94c0166690 con 0x7f94c01f4750 2024-09-10T21:25:20.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.162+0000 7f94bf400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f94c01f7b80 0x7f94c01f8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:20.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.162+0000 7f94bf400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f94c01f7b80 0x7f94c01f8000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.17:51450/0 (socket says 172.21.15.17:51450) 2024-09-10T21:25:20.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.162+0000 7f94bf400640 1 -- 172.21.15.17:0/1662589028 learned_addr learned my addr 172.21.15.17:0/1662589028 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:25:20.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.162+0000 7f94be000640 1 --2- 172.21.15.17:0/1662589028 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f94c01f4750 0x7f94c01f4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:20.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.162+0000 7f94bea00640 1 --2- 172.21.15.17:0/1662589028 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f94c00b2d00 0x7f94c0149d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:20.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.162+0000 7f336dc00640 1 -- start start 2024-09-10T21:25:20.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.162+0000 7f94be000640 1 -- 172.21.15.17:0/1662589028 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f94c01f7b80 msgr2=0x7f94c01f8000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:20.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.162+0000 7f94be000640 1 --2- 172.21.15.17:0/1662589028 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f94c01f7b80 0x7f94c01f8000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:20.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.162+0000 7f94be000640 1 -- 172.21.15.17:0/1662589028 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f94c00b2d00 msgr2=0x7f94c0149d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:20.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.162+0000 7f94be000640 1 --2- 172.21.15.17:0/1662589028 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f94c00b2d00 0x7f94c0149d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:20.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.162+0000 7f94be000640 1 -- 172.21.15.17:0/1662589028 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94c01609f0 con 0x7f94c01f4750 2024-09-10T21:25:20.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.162+0000 7f94bea00640 1 --2- 172.21.15.17:0/1662589028 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f94c00b2d00 0x7f94c0149d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:25:20.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.166+0000 7f336dc00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3360004a40 0x7f33600d7b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:20.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.166+0000 7f94be000640 1 --2- 172.21.15.17:0/1662589028 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f94c01f4750 0x7f94c01f4bd0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f94a4005780 tx=0x7f94b0005880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:20.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.166+0000 7f336dc00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f336004baf0 0x7f33600cff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:20.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.166+0000 7f3367e00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3360004a40 0x7f33600d7b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:20.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.166+0000 7f94b6a00640 1 -- 172.21.15.17:0/1662589028 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94b00bc070 con 0x7f94c01f4750 2024-09-10T21:25:20.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.166+0000 7f94b6a00640 1 -- 172.21.15.17:0/1662589028 <== mon.2 v2:172.21.15.31:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f94b0005320 con 0x7f94c01f4750 2024-09-10T21:25:20.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.166+0000 7f94b6a00640 1 -- 172.21.15.17:0/1662589028 <== mon.2 v2:172.21.15.31:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94b00ab070 con 0x7f94c01f4750 2024-09-10T21:25:20.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.166+0000 7f3367e00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3360004a40 0x7f33600d7b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44442/0 (socket says 172.21.15.17:44442) 2024-09-10T21:25:20.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.166+0000 7f3367e00640 1 -- 172.21.15.17:0/1623411877 learned_addr learned my addr 172.21.15.17:0/1623411877 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:25:20.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.166+0000 7f336dc00640 1 --2- 172.21.15.17:0/1623411877 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f336004c3b0 0x7f3360162450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:20.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.166+0000 7f3367400640 1 --2- 172.21.15.17:0/1623411877 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f336004baf0 0x7f33600cff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:20.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.166+0000 7f336dc00640 1 -- 172.21.15.17:0/1623411877 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f33600066c0 con 0x7f3360004a40 2024-09-10T21:25:20.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.166+0000 7f336d200640 1 --2- 172.21.15.17:0/1623411877 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f336004c3b0 0x7f3360162450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:20.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.166+0000 7f336dc00640 1 -- 172.21.15.17:0/1623411877 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f33600063e0 con 0x7f336004c3b0 2024-09-10T21:25:20.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.166+0000 7f94c5800640 1 -- 172.21.15.17:0/1662589028 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94c00c7750 con 0x7f94c01f4750 2024-09-10T21:25:20.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.166+0000 7f94c5800640 1 -- 172.21.15.17:0/1662589028 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f94c00b5780 con 0x7f94c01f4750 2024-09-10T21:25:20.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.166+0000 7f336dc00640 1 -- 172.21.15.17:0/1623411877 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f33600069a0 con 0x7f336004baf0 2024-09-10T21:25:20.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.170+0000 7f3367400640 1 -- 172.21.15.17:0/1623411877 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f336004c3b0 msgr2=0x7f3360162450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:20.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.170+0000 7f3367400640 1 --2- 172.21.15.17:0/1623411877 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f336004c3b0 0x7f3360162450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:20.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.170+0000 7f94c5800640 1 -- 172.21.15.17:0/1662589028 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94c006cc20 con 0x7f94c01f4750 2024-09-10T21:25:20.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.170+0000 7f3367400640 1 -- 172.21.15.17:0/1623411877 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3360004a40 msgr2=0x7f33600d7b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:20.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.170+0000 7f3367400640 1 --2- 172.21.15.17:0/1623411877 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3360004a40 0x7f33600d7b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:20.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.170+0000 7f3367400640 1 -- 172.21.15.17:0/1623411877 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33600d4eb0 con 0x7f336004baf0 2024-09-10T21:25:20.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.170+0000 7f3367e00640 1 --2- 172.21.15.17:0/1623411877 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3360004a40 0x7f33600d7b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:25:20.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.170+0000 7f3367400640 1 --2- 172.21.15.17:0/1623411877 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f336004baf0 0x7f33600cff80 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f335c0b6000 tx=0x7f335c0046d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:20.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.170+0000 7f335be00640 1 -- 172.21.15.17:0/1623411877 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f335c0c7070 con 0x7f336004baf0 2024-09-10T21:25:20.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.170+0000 7f336dc00640 1 -- 172.21.15.17:0/1623411877 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33601742e0 con 0x7f336004baf0 2024-09-10T21:25:20.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.170+0000 7f335be00640 1 -- 172.21.15.17:0/1623411877 <== mon.2 v2:172.21.15.31:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f335c0bd040 con 0x7f336004baf0 2024-09-10T21:25:20.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.170+0000 7f335be00640 1 -- 172.21.15.17:0/1623411877 <== mon.2 v2:172.21.15.31:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f335c0c7070 con 0x7f336004baf0 2024-09-10T21:25:20.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.174+0000 7f336dc00640 1 -- 172.21.15.17:0/1623411877 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f33601748a0 con 0x7f336004baf0 2024-09-10T21:25:20.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.170+0000 7f94b6a00640 1 -- 172.21.15.17:0/1662589028 <== mon.2 v2:172.21.15.31:3301/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f94b00046d0 con 0x7f94c01f4750 2024-09-10T21:25:20.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.174+0000 7f94b6a00640 1 --2- 172.21.15.17:0/1662589028 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f94a00726b0 0x7f94a0074b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:20.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.174+0000 7f94b6a00640 1 -- 172.21.15.17:0/1662589028 <== mon.2 v2:172.21.15.31:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f94b012d210 con 0x7f94c01f4750 2024-09-10T21:25:20.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.174+0000 7f94bea00640 1 --2- 172.21.15.17:0/1662589028 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f94a00726b0 0x7f94a0074b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:20.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.174+0000 7f336dc00640 1 -- 172.21.15.17:0/1623411877 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3360007c70 con 0x7f336004baf0 2024-09-10T21:25:20.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.174+0000 7f94bea00640 1 --2- 172.21.15.17:0/1662589028 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f94a00726b0 0x7f94a0074b70 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f94b8001bd0 tx=0x7f94b8002100 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:20.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.178+0000 7f94b6a00640 1 -- 172.21.15.17:0/1662589028 <== mon.2 v2:172.21.15.31:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f94b0016b10 con 0x7f94c01f4750 2024-09-10T21:25:20.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.182+0000 7f335be00640 1 -- 172.21.15.17:0/1623411877 <== mon.2 v2:172.21.15.31:3301/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f335c00d190 con 0x7f336004baf0 2024-09-10T21:25:20.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.186+0000 7f335be00640 1 --2- 172.21.15.17:0/1623411877 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f33480727e0 0x7f3348074ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:20.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.186+0000 7f335be00640 1 -- 172.21.15.17:0/1623411877 <== mon.2 v2:172.21.15.31:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f335c1383e0 con 0x7f336004baf0 2024-09-10T21:25:20.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.186+0000 7f335be00640 1 -- 172.21.15.17:0/1623411877 <== mon.2 v2:172.21.15.31:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f335c020f90 con 0x7f336004baf0 2024-09-10T21:25:20.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.186+0000 7f3367e00640 1 --2- 172.21.15.17:0/1623411877 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f33480727e0 0x7f3348074ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:20.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.194+0000 7f3367e00640 1 --2- 172.21.15.17:0/1623411877 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f33480727e0 0x7f3348074ca0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f336806ba20 tx=0x7f336806a7f0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:20.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.230+0000 7ff7b7400640 1 -- 172.21.15.17:0/3167290125 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff7b015b280 msgr2=0x7ff7b00d00b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:20.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.230+0000 7ff7b7400640 1 --2- 172.21.15.17:0/3167290125 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff7b015b280 0x7ff7b00d00b0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ff7a8009c80 tx=0x7ff7a801cda0 comp rx=0 tx=0).stop 2024-09-10T21:25:20.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.234+0000 7ff7b7400640 1 -- 172.21.15.17:0/3167290125 shutdown_connections 2024-09-10T21:25:20.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.234+0000 7ff7b7400640 1 --2- 172.21.15.17:0/3167290125 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff7b015b280 0x7ff7b00d00b0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:20.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.234+0000 7ff7b7400640 1 --2- 172.21.15.17:0/3167290125 >> v2:172.21.15.17:3300/0 conn(0x7ff7b00b2d00 0x7ff7b00c6580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:20.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.234+0000 7ff7b7400640 1 -- 172.21.15.17:0/3167290125 >> 172.21.15.17:0/3167290125 conn(0x7ff7b0068990 msgr2=0x7ff7b0068da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:20.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.234+0000 7ff7b7400640 1 -- 172.21.15.17:0/3167290125 shutdown_connections 2024-09-10T21:25:20.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.234+0000 7ff7b7400640 1 -- 172.21.15.17:0/3167290125 wait complete. 2024-09-10T21:25:20.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.234+0000 7ff7b7400640 1 Processor -- start 2024-09-10T21:25:20.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.238+0000 7ff7b7400640 1 -- start start 2024-09-10T21:25:20.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.238+0000 7ff7b7400640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff7b00b2d00 0x7ff7b00b5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:20.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.238+0000 7ff7b7400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff7b0154a20 0x7ff7b00b3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:20.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.238+0000 7ff7b4a00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff7b00b2d00 0x7ff7b00b5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:20.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.238+0000 7ff7b4a00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff7b00b2d00 0x7ff7b00b5240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3301/0 says I am v2:172.21.15.17:43112/0 (socket says 172.21.15.17:43112) 2024-09-10T21:25:20.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.242+0000 7fbd20600640 1 -- 172.21.15.17:0/483168262 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbd18153d40 msgr2=0x7fbd180c8010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:20.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.242+0000 7fbd20600640 1 --2- 172.21.15.17:0/483168262 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbd18153d40 0x7fbd180c8010 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fbd180c9b70 tx=0x7fbd0c01f0b0 comp rx=0 tx=0).stop 2024-09-10T21:25:20.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.238+0000 7ff7b7400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff7b015b280 0x7ff7b0202850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:20.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.238+0000 7ff7b7400640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7ff7b0167a20 con 0x7ff7b0154a20 2024-09-10T21:25:20.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.238+0000 7ff7b7400640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7ff7b01678a0 con 0x7ff7b015b280 2024-09-10T21:25:20.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.238+0000 7ff7b7400640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7ff7b0167ba0 con 0x7ff7b00b2d00 2024-09-10T21:25:20.246 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.242+0000 7fbd20600640 1 -- 172.21.15.17:0/483168262 shutdown_connections 2024-09-10T21:25:20.246 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.242+0000 7fbd20600640 1 --2- 172.21.15.17:0/483168262 >> v2:172.21.15.17:3300/0 conn(0x7fbd180c0fd0 0x7fbd180c6cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:20.246 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.242+0000 7ff7b4a00640 1 -- 172.21.15.17:0/2258491875 learned_addr learned my addr 172.21.15.17:0/2258491875 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:25:20.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.242+0000 7fbd20600640 1 --2- 172.21.15.17:0/483168262 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbd18153d40 0x7fbd180c8010 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:20.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.242+0000 7fbd20600640 1 -- 172.21.15.17:0/483168262 >> 172.21.15.17:0/483168262 conn(0x7fbd180700d0 msgr2=0x7fbd180704e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:20.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.242+0000 7ff7afe00640 1 --2- 172.21.15.17:0/2258491875 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff7b0154a20 0x7ff7b00b3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:20.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.242+0000 7fbd20600640 1 -- 172.21.15.17:0/483168262 shutdown_connections 2024-09-10T21:25:20.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.242+0000 7fbd20600640 1 -- 172.21.15.17:0/483168262 wait complete. 2024-09-10T21:25:20.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.242+0000 7ff7b5400640 1 --2- 172.21.15.17:0/2258491875 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff7b015b280 0x7ff7b0202850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:20.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.242+0000 7ff7b4a00640 1 -- 172.21.15.17:0/2258491875 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff7b015b280 msgr2=0x7ff7b0202850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:20.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.242+0000 7ff7b4a00640 1 --2- 172.21.15.17:0/2258491875 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff7b015b280 0x7ff7b0202850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:20.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.242+0000 7ff7b4a00640 1 -- 172.21.15.17:0/2258491875 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff7b0154a20 msgr2=0x7ff7b00b3ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:20.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.242+0000 7ff7b4a00640 1 --2- 172.21.15.17:0/2258491875 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff7b0154a20 0x7ff7b00b3ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:20.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.242+0000 7ff7b4a00640 1 -- 172.21.15.17:0/2258491875 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7b0150c90 con 0x7ff7b00b2d00 2024-09-10T21:25:20.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.242+0000 7ff7b4a00640 1 --2- 172.21.15.17:0/2258491875 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff7b00b2d00 0x7ff7b00b5240 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ff7a400b4d0 tx=0x7ff7a400b990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:20.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.242+0000 7ff7acc00640 1 -- 172.21.15.17:0/2258491875 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7a4014070 con 0x7ff7b00b2d00 2024-09-10T21:25:20.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.242+0000 7ff7acc00640 1 -- 172.21.15.17:0/2258491875 <== mon.2 v2:172.21.15.31:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff7a4010980 con 0x7ff7b00b2d00 2024-09-10T21:25:20.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.242+0000 7ff7acc00640 1 -- 172.21.15.17:0/2258491875 <== mon.2 v2:172.21.15.31:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7a4010dd0 con 0x7ff7b00b2d00 2024-09-10T21:25:20.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.242+0000 7ff7b7400640 1 -- 172.21.15.17:0/2258491875 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7b0202350 con 0x7ff7b00b2d00 2024-09-10T21:25:20.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.242+0000 7ff7b7400640 1 -- 172.21.15.17:0/2258491875 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7b0150140 con 0x7ff7b00b2d00 2024-09-10T21:25:20.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.246+0000 7fc54e400640 1 -- 172.21.15.17:0/838663375 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc5480d0630 msgr2=0x7fc54815e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:20.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.246+0000 7fc54e400640 1 --2- 172.21.15.17:0/838663375 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc5480d0630 0x7fc54815e160 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fc5480698f0 tx=0x7fc53401c8c0 comp rx=0 tx=0).stop 2024-09-10T21:25:20.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.246+0000 7fc54e400640 1 -- 172.21.15.17:0/838663375 shutdown_connections 2024-09-10T21:25:20.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.246+0000 7fc54e400640 1 --2- 172.21.15.17:0/838663375 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc5480d0630 0x7fc54815e160 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:20.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.246+0000 7fc54e400640 1 --2- 172.21.15.17:0/838663375 >> v2:172.21.15.17:3300/0 conn(0x7fc5480d0250 0x7fc5480cad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:20.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.246+0000 7fc54e400640 1 -- 172.21.15.17:0/838663375 >> 172.21.15.17:0/838663375 conn(0x7fc548068990 msgr2=0x7fc548068da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:20.251 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.246+0000 7ff7a2000640 1 -- 172.21.15.17:0/2258491875 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7b00c6580 con 0x7ff7b00b2d00 2024-09-10T21:25:20.251 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.246+0000 7fbd20600640 1 Processor -- start 2024-09-10T21:25:20.251 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.246+0000 7fc54e400640 1 -- 172.21.15.17:0/838663375 shutdown_connections 2024-09-10T21:25:20.251 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.246+0000 7fc54e400640 1 -- 172.21.15.17:0/838663375 wait complete. 2024-09-10T21:25:20.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.246+0000 7ff7acc00640 1 -- 172.21.15.17:0/2258491875 <== mon.2 v2:172.21.15.31:3301/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7ff7a400f3c0 con 0x7ff7b00b2d00 2024-09-10T21:25:20.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.250+0000 7ff7acc00640 1 --2- 172.21.15.17:0/2258491875 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7ff790072630 0x7ff790074af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:20.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.250+0000 7fc54e400640 1 Processor -- start 2024-09-10T21:25:20.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.250+0000 7fc54e400640 1 -- start start 2024-09-10T21:25:20.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.250+0000 7fbd20600640 1 -- start start 2024-09-10T21:25:20.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.250+0000 7fbd20600640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fbd180c0fd0 0x7fbd181490c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:20.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.250+0000 7fbd20600640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbd180c7210 0x7fbd18139b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:20.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.250+0000 7ff7acc00640 1 -- 172.21.15.17:0/2258491875 <== mon.2 v2:172.21.15.31:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7ff7a4093730 con 0x7ff7b00b2d00 2024-09-10T21:25:20.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.250+0000 7fbd20600640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbd18153d40 0x7fbd18145b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:20.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.250+0000 7fbd1dc00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fbd180c0fd0 0x7fbd181490c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:20.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.250+0000 7fbd1dc00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fbd180c0fd0 0x7fbd181490c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3301/0 says I am v2:172.21.15.17:43122/0 (socket says 172.21.15.17:43122) 2024-09-10T21:25:20.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.250+0000 7fbd20600640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fbd180c95a0 con 0x7fbd180c7210 2024-09-10T21:25:20.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.250+0000 7fbd20600640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fbd180c92c0 con 0x7fbd18153d40 2024-09-10T21:25:20.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.250+0000 7fbd20600640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7fbd18154120 con 0x7fbd180c0fd0 2024-09-10T21:25:20.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.250+0000 7ff7afe00640 1 --2- 172.21.15.17:0/2258491875 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7ff790072630 0x7ff790074af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:20.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.250+0000 7fbd1dc00640 1 -- 172.21.15.17:0/3135444326 learned_addr learned my addr 172.21.15.17:0/3135444326 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:25:20.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.250+0000 7fbd1e600640 1 --2- 172.21.15.17:0/3135444326 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbd18153d40 0x7fbd18145b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:20.261 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.250+0000 7fc54e400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc5480d0250 0x7fc54814c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:20.261 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.250+0000 7fc54e400640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc5480d0630 0x7fc5481f3c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:20.261 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.254+0000 7ff7afe00640 1 --2- 172.21.15.17:0/2258491875 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7ff790072630 0x7ff790074af0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7ff7b00b2a40 tx=0x7ff79c0091e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:20.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.254+0000 7fc54e400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc54815e6a0 0x7fc5481f4140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:20.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.254+0000 7fc547e00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc5480d0630 0x7fc5481f3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:20.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.254+0000 7fc547e00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc5480d0630 0x7fc5481f3c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3301/0 says I am v2:172.21.15.17:43136/0 (socket says 172.21.15.17:43136) 2024-09-10T21:25:20.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.254+0000 7fc54e400640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fc54815a5b0 con 0x7fc5480d0250 2024-09-10T21:25:20.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.254+0000 7fc54e400640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fc5480cb2b0 con 0x7fc54815e6a0 2024-09-10T21:25:20.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.254+0000 7fc54e400640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7fc54815d9c0 con 0x7fc5480d0630 2024-09-10T21:25:20.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.254+0000 7ff7acc00640 1 -- 172.21.15.17:0/2258491875 <== mon.2 v2:172.21.15.31:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7ff7a405eab0 con 0x7ff7b00b2d00 2024-09-10T21:25:20.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.254+0000 7fc54da00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc54815e6a0 0x7fc5481f4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:20.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.254+0000 7fc54da00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc54815e6a0 0x7fc5481f4140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.17:51506/0 (socket says 172.21.15.17:51506) 2024-09-10T21:25:20.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.254+0000 7fc54da00640 1 -- 172.21.15.17:0/1840779579 learned_addr learned my addr 172.21.15.17:0/1840779579 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:25:20.264 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.254+0000 7fc54d000640 1 --2- 172.21.15.17:0/1840779579 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc5480d0250 0x7fc54814c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:20.265 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.254+0000 7fbd1d200640 1 --2- 172.21.15.17:0/3135444326 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbd180c7210 0x7fbd18139b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:20.265 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.254+0000 7fbd1e600640 1 -- 172.21.15.17:0/3135444326 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fbd180c0fd0 msgr2=0x7fbd181490c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:20.265 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.254+0000 7fbd1e600640 1 --2- 172.21.15.17:0/3135444326 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fbd180c0fd0 0x7fbd181490c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:20.265 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.254+0000 7fbd1e600640 1 -- 172.21.15.17:0/3135444326 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbd180c7210 msgr2=0x7fbd18139b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:20.265 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.254+0000 7fbd1e600640 1 --2- 172.21.15.17:0/3135444326 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbd180c7210 0x7fbd18139b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:20.266 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.254+0000 7fbd1e600640 1 -- 172.21.15.17:0/3135444326 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd18149600 con 0x7fbd18153d40 2024-09-10T21:25:20.266 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.254+0000 7fbd1d200640 1 --2- 172.21.15.17:0/3135444326 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbd180c7210 0x7fbd18139b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:25:20.266 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.254+0000 7fc547e00640 1 -- 172.21.15.17:0/1840779579 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc54815e6a0 msgr2=0x7fc5481f4140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:20.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.254+0000 7fc547e00640 1 --2- 172.21.15.17:0/1840779579 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc54815e6a0 0x7fc5481f4140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:20.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.254+0000 7fc547e00640 1 -- 172.21.15.17:0/1840779579 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc5480d0250 msgr2=0x7fc54814c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:20.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.254+0000 7fc547e00640 1 --2- 172.21.15.17:0/1840779579 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc5480d0250 0x7fc54814c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:20.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.254+0000 7fc547e00640 1 -- 172.21.15.17:0/1840779579 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5481557e0 con 0x7fc5480d0630 2024-09-10T21:25:20.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.254+0000 7fbd1dc00640 1 --2- 172.21.15.17:0/3135444326 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fbd180c0fd0 0x7fbd181490c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:25:20.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.254+0000 7fc547e00640 1 --2- 172.21.15.17:0/1840779579 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc5480d0630 0x7fc5481f3c00 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fc54815b660 tx=0x7fc5340a56d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:20.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.254+0000 7fc544c00640 1 -- 172.21.15.17:0/1840779579 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5340b2040 con 0x7fc5480d0630 2024-09-10T21:25:20.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.254+0000 7fc544c00640 1 -- 172.21.15.17:0/1840779579 <== mon.2 v2:172.21.15.31:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc534005020 con 0x7fc5480d0630 2024-09-10T21:25:20.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.254+0000 7fc544c00640 1 -- 172.21.15.17:0/1840779579 <== mon.2 v2:172.21.15.31:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc534005470 con 0x7fc5480d0630 2024-09-10T21:25:20.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.254+0000 7fc54e400640 1 -- 172.21.15.17:0/1840779579 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5481f7820 con 0x7fc5480d0630 2024-09-10T21:25:20.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.254+0000 7fc54e400640 1 -- 172.21.15.17:0/1840779579 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fc54815d6e0 con 0x7fc5480d0630 2024-09-10T21:25:20.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.258+0000 7fbd1e600640 1 --2- 172.21.15.17:0/3135444326 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbd18153d40 0x7fbd18145b00 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fbd10002170 tx=0x7fbd1000cbb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:20.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.258+0000 7fc54e400640 1 -- 172.21.15.17:0/1840779579 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc54814ec70 con 0x7fc5480d0630 2024-09-10T21:25:20.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.258+0000 7fbd15600640 1 -- 172.21.15.17:0/3135444326 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd10003bb0 con 0x7fbd18153d40 2024-09-10T21:25:20.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.258+0000 7fbd20600640 1 -- 172.21.15.17:0/3135444326 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd181ec0c0 con 0x7fbd18153d40 2024-09-10T21:25:20.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.258+0000 7fbd20600640 1 -- 172.21.15.17:0/3135444326 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd181ec380 con 0x7fbd18153d40 2024-09-10T21:25:20.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.258+0000 7fbd15600640 1 -- 172.21.15.17:0/3135444326 <== mon.1 v2:172.21.15.31:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbd10015070 con 0x7fbd18153d40 2024-09-10T21:25:20.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.258+0000 7fbd15600640 1 -- 172.21.15.17:0/3135444326 <== mon.1 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd100052d0 con 0x7fbd18153d40 2024-09-10T21:25:20.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.262+0000 7fbd15600640 1 -- 172.21.15.17:0/3135444326 <== mon.1 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fbd10022030 con 0x7fbd18153d40 2024-09-10T21:25:20.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.262+0000 7fc544c00640 1 -- 172.21.15.17:0/1840779579 <== mon.2 v2:172.21.15.31:3301/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fc5340047b0 con 0x7fc5480d0630 2024-09-10T21:25:20.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.262+0000 7fc544c00640 1 --2- 172.21.15.17:0/1840779579 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fc5280725e0 0x7fc528074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:20.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.262+0000 7fc544c00640 1 -- 172.21.15.17:0/1840779579 <== mon.2 v2:172.21.15.31:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fc53412d930 con 0x7fc5480d0630 2024-09-10T21:25:20.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.262+0000 7fc54d000640 1 --2- 172.21.15.17:0/1840779579 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fc5280725e0 0x7fc528074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:20.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.262+0000 7fbd20600640 1 -- 172.21.15.17:0/3135444326 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbcf40052a0 con 0x7fbd18153d40 2024-09-10T21:25:20.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.262+0000 7fbd15600640 1 --2- 172.21.15.17:0/3135444326 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fbcf80726b0 0x7fbcf8074b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:20.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.266+0000 7fbd15600640 1 -- 172.21.15.17:0/3135444326 <== mon.1 v2:172.21.15.31:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fbd10096df0 con 0x7fbd18153d40 2024-09-10T21:25:20.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.266+0000 7fbd1dc00640 1 --2- 172.21.15.17:0/3135444326 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fbcf80726b0 0x7fbcf8074b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:20.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.266+0000 7fc54d000640 1 --2- 172.21.15.17:0/1840779579 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fc5280725e0 0x7fc528074aa0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fc538009f20 tx=0x7fc5380094f0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:20.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.266+0000 7fbd1dc00640 1 --2- 172.21.15.17:0/3135444326 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fbcf80726b0 0x7fbcf8074b70 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fbd0c000c10 tx=0x7fbd0c01d8f0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:20.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.266+0000 7fc544c00640 1 -- 172.21.15.17:0/1840779579 <== mon.2 v2:172.21.15.31:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fc5340164b0 con 0x7fc5480d0630 2024-09-10T21:25:20.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.270+0000 7fbd15600640 1 -- 172.21.15.17:0/3135444326 <== mon.1 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fbd10062280 con 0x7fbd18153d40 2024-09-10T21:25:20.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.498+0000 7fee96000640 1 -- 172.21.15.17:0/1995537471 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7fee90156980 con 0x7fee74004810 2024-09-10T21:25:20.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.498+0000 7fee8cc00640 1 -- 172.21.15.17:0/1995537471 <== mon.1 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fee7c05fb70 con 0x7fee74004810 2024-09-10T21:25:20.503 INFO:teuthology.orchestra.run.smithi017.stdout:64424509450 2024-09-10T21:25:20.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.502+0000 7fee96000640 1 -- 172.21.15.17:0/1995537471 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fee70072610 msgr2=0x7fee70074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:20.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.502+0000 7fee96000640 1 --2- 172.21.15.17:0/1995537471 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fee70072610 0x7fee70074ad0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fee800ab000 tx=0x7fee80019c80 comp rx=0 tx=0).stop 2024-09-10T21:25:20.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.502+0000 7fee96000640 1 -- 172.21.15.17:0/1995537471 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fee74004810 msgr2=0x7fee9014bc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:20.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.502+0000 7fee96000640 1 --2- 172.21.15.17:0/1995537471 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fee74004810 0x7fee9014bc00 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fee7c004da0 tx=0x7fee7c001640 comp rx=0 tx=0).stop 2024-09-10T21:25:20.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.502+0000 7fee96000640 1 -- 172.21.15.17:0/1995537471 shutdown_connections 2024-09-10T21:25:20.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.502+0000 7fee96000640 1 --2- 172.21.15.17:0/1995537471 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fee70072610 0x7fee70074ad0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:20.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.502+0000 7fee96000640 1 --2- 172.21.15.17:0/1995537471 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fee90148900 0x7fee90148d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:20.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.502+0000 7fee96000640 1 --2- 172.21.15.17:0/1995537471 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fee90152160 0x7fee901525a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:20.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.502+0000 7fee96000640 1 --2- 172.21.15.17:0/1995537471 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fee74004810 0x7fee9014bc00 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:20.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.502+0000 7fee96000640 1 -- 172.21.15.17:0/1995537471 >> 172.21.15.17:0/1995537471 conn(0x7fee9006fe60 msgr2=0x7fee90155610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:20.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.502+0000 7fee96000640 1 -- 172.21.15.17:0/1995537471 shutdown_connections 2024-09-10T21:25:20.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.510+0000 7fee96000640 1 -- 172.21.15.17:0/1995537471 wait complete. 2024-09-10T21:25:20.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.522+0000 7f25fd400640 1 -- 172.21.15.17:0/296409905 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f25d0005590 con 0x7f25f80cdf20 2024-09-10T21:25:20.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.526+0000 7f299f600640 1 -- 172.21.15.17:0/4102610328 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f2970005590 con 0x7f29981548b0 2024-09-10T21:25:20.531 INFO:teuthology.orchestra.run.smithi017.stdout:60129542156 2024-09-10T21:25:20.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.526+0000 7f25ef400640 1 -- 172.21.15.17:0/296409905 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f25e005dca0 con 0x7f25f80cdf20 2024-09-10T21:25:20.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.526+0000 7f2994c00640 1 -- 172.21.15.17:0/4102610328 <== mon.2 v2:172.21.15.31:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f299005dab0 con 0x7f29981548b0 2024-09-10T21:25:20.532 INFO:teuthology.orchestra.run.smithi017.stdout:64424509449 2024-09-10T21:25:20.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.530+0000 7f299f600640 1 -- 172.21.15.17:0/4102610328 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f2978072810 msgr2=0x7f2978074cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:20.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.530+0000 7f299f600640 1 --2- 172.21.15.17:0/4102610328 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f2978072810 0x7f2978074cd0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f29880b2000 tx=0x7f298801d930 comp rx=0 tx=0).stop 2024-09-10T21:25:20.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.530+0000 7f299f600640 1 -- 172.21.15.17:0/4102610328 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f29981548b0 msgr2=0x7f29981e4080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:20.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.530+0000 7f299f600640 1 --2- 172.21.15.17:0/4102610328 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f29981548b0 0x7f29981e4080 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f299000c7a0 tx=0x7f299000cc60 comp rx=0 tx=0).stop 2024-09-10T21:25:20.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.530+0000 7f299f600640 1 -- 172.21.15.17:0/4102610328 shutdown_connections 2024-09-10T21:25:20.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.530+0000 7f299f600640 1 --2- 172.21.15.17:0/4102610328 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f2978072810 0x7f2978074cd0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:20.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.530+0000 7f299f600640 1 --2- 172.21.15.17:0/4102610328 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f29981548b0 0x7f29981e4080 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:20.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.530+0000 7f299f600640 1 --2- 172.21.15.17:0/4102610328 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f29980c6f40 0x7f29981f88d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:20.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.530+0000 7f299f600640 1 --2- 172.21.15.17:0/4102610328 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f299806e730 0x7f29980cb210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:20.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.530+0000 7f299f600640 1 -- 172.21.15.17:0/4102610328 >> 172.21.15.17:0/4102610328 conn(0x7f29980700d0 msgr2=0x7f2998156e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:20.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.530+0000 7f299f600640 1 -- 172.21.15.17:0/4102610328 shutdown_connections 2024-09-10T21:25:20.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.530+0000 7f25fd400640 1 -- 172.21.15.17:0/296409905 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f25d40726e0 msgr2=0x7f25d4074ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:20.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.530+0000 7f25fd400640 1 --2- 172.21.15.17:0/296409905 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f25d40726e0 0x7f25d4074ba0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f25e80af000 tx=0x7f25e801b030 comp rx=0 tx=0).stop 2024-09-10T21:25:20.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.530+0000 7f25fd400640 1 -- 172.21.15.17:0/296409905 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f25f80cdf20 msgr2=0x7f25f81fb950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:20.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.530+0000 7f25fd400640 1 --2- 172.21.15.17:0/296409905 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f25f80cdf20 0x7f25f81fb950 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f25e000c4b0 tx=0x7f25e000c970 comp rx=0 tx=0).stop 2024-09-10T21:25:20.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.530+0000 7f25fd400640 1 -- 172.21.15.17:0/296409905 shutdown_connections 2024-09-10T21:25:20.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.530+0000 7f25fd400640 1 --2- 172.21.15.17:0/296409905 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f25d40726e0 0x7f25d4074ba0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:20.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.530+0000 7f25fd400640 1 --2- 172.21.15.17:0/296409905 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f25f8152820 0x7f25f806eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:20.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.530+0000 7f25fd400640 1 --2- 172.21.15.17:0/296409905 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f25f8152440 0x7f25f81f8210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:20.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.530+0000 7f25fd400640 1 --2- 172.21.15.17:0/296409905 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f25f80cdf20 0x7f25f81fb950 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:20.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.530+0000 7f25fd400640 1 -- 172.21.15.17:0/296409905 >> 172.21.15.17:0/296409905 conn(0x7f25f80702f0 msgr2=0x7f25f81561d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:20.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.530+0000 7f25fd400640 1 -- 172.21.15.17:0/296409905 shutdown_connections 2024-09-10T21:25:20.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.530+0000 7f25fd400640 1 -- 172.21.15.17:0/296409905 wait complete. 2024-09-10T21:25:20.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.530+0000 7f299f600640 1 -- 172.21.15.17:0/4102610328 wait complete. 2024-09-10T21:25:20.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.534+0000 7f94c5800640 1 -- 172.21.15.17:0/1662589028 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f94c01522e0 con 0x7f94c01f4750 2024-09-10T21:25:20.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.534+0000 7f94b6a00640 1 -- 172.21.15.17:0/1662589028 <== mon.2 v2:172.21.15.31:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f94b00f8500 con 0x7f94c01f4750 2024-09-10T21:25:20.540 INFO:teuthology.orchestra.run.smithi017.stdout:55834574858 2024-09-10T21:25:20.541 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509451 got 64424509450 for osd.6 2024-09-10T21:25:20.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.538+0000 7f94c5800640 1 -- 172.21.15.17:0/1662589028 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f94a00726b0 msgr2=0x7f94a0074b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:20.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.538+0000 7f94c5800640 1 --2- 172.21.15.17:0/1662589028 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f94a00726b0 0x7f94a0074b70 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f94b8001bd0 tx=0x7f94b8002100 comp rx=0 tx=0).stop 2024-09-10T21:25:20.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.538+0000 7f94c5800640 1 -- 172.21.15.17:0/1662589028 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f94c01f4750 msgr2=0x7f94c01f4bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:20.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.538+0000 7f94c5800640 1 --2- 172.21.15.17:0/1662589028 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f94c01f4750 0x7f94c01f4bd0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f94a4005780 tx=0x7f94b0005880 comp rx=0 tx=0).stop 2024-09-10T21:25:20.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.538+0000 7f94c5800640 1 -- 172.21.15.17:0/1662589028 shutdown_connections 2024-09-10T21:25:20.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.538+0000 7f94c5800640 1 --2- 172.21.15.17:0/1662589028 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f94a00726b0 0x7f94a0074b70 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:20.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.538+0000 7f94c5800640 1 --2- 172.21.15.17:0/1662589028 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f94c01f7b80 0x7f94c01f8000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:20.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.538+0000 7f94c5800640 1 --2- 172.21.15.17:0/1662589028 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f94c01f4750 0x7f94c01f4bd0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:20.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.538+0000 7f94c5800640 1 --2- 172.21.15.17:0/1662589028 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f94c00b2d00 0x7f94c0149d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:20.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.538+0000 7f94c5800640 1 -- 172.21.15.17:0/1662589028 >> 172.21.15.17:0/1662589028 conn(0x7f94c0068990 msgr2=0x7f94c014ab90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:20.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.538+0000 7f94c5800640 1 -- 172.21.15.17:0/1662589028 shutdown_connections 2024-09-10T21:25:20.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.538+0000 7f94c5800640 1 -- 172.21.15.17:0/1662589028 wait complete. 2024-09-10T21:25:20.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.550+0000 7f336dc00640 1 -- 172.21.15.17:0/1623411877 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f3360050920 con 0x7f336004baf0 2024-09-10T21:25:20.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.554+0000 7f335be00640 1 -- 172.21.15.17:0/1623411877 <== mon.2 v2:172.21.15.31:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f335c1031a0 con 0x7f336004baf0 2024-09-10T21:25:20.557 INFO:teuthology.orchestra.run.smithi017.stdout:60129542154 2024-09-10T21:25:20.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.558+0000 7f336dc00640 1 -- 172.21.15.17:0/1623411877 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f33480727e0 msgr2=0x7f3348074ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:20.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.558+0000 7f336dc00640 1 --2- 172.21.15.17:0/1623411877 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f33480727e0 0x7f3348074ca0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f336806ba20 tx=0x7f336806a7f0 comp rx=0 tx=0).stop 2024-09-10T21:25:20.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.558+0000 7f336dc00640 1 -- 172.21.15.17:0/1623411877 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f336004baf0 msgr2=0x7f33600cff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:20.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.558+0000 7f336dc00640 1 --2- 172.21.15.17:0/1623411877 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f336004baf0 0x7f33600cff80 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f335c0b6000 tx=0x7f335c0046d0 comp rx=0 tx=0).stop 2024-09-10T21:25:20.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.558+0000 7f336dc00640 1 -- 172.21.15.17:0/1623411877 shutdown_connections 2024-09-10T21:25:20.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.558+0000 7f336dc00640 1 --2- 172.21.15.17:0/1623411877 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f33480727e0 0x7f3348074ca0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:20.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.558+0000 7f336dc00640 1 --2- 172.21.15.17:0/1623411877 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f336004c3b0 0x7f3360162450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:20.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.558+0000 7f336dc00640 1 --2- 172.21.15.17:0/1623411877 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f336004baf0 0x7f33600cff80 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:20.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.558+0000 7f336dc00640 1 --2- 172.21.15.17:0/1623411877 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3360004a40 0x7f33600d7b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:20.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.558+0000 7f336dc00640 1 -- 172.21.15.17:0/1623411877 >> 172.21.15.17:0/1623411877 conn(0x7f3360050060 msgr2=0x7f33600d3c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:20.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.558+0000 7f336dc00640 1 -- 172.21.15.17:0/1623411877 shutdown_connections 2024-09-10T21:25:20.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.558+0000 7f336dc00640 1 -- 172.21.15.17:0/1623411877 wait complete. 2024-09-10T21:25:20.573 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542157 got 60129542156 for osd.0 2024-09-10T21:25:20.574 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509450 got 64424509449 for osd.2 2024-09-10T21:25:20.577 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574858 for osd.5 2024-09-10T21:25:20.589 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542155 got 60129542154 for osd.4 2024-09-10T21:25:20.607 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.602+0000 7fc54e400640 1 -- 172.21.15.17:0/1840779579 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7fc548150470 con 0x7fc5480d0630 2024-09-10T21:25:20.608 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.602+0000 7fc544c00640 1 -- 172.21.15.17:0/1840779579 <== mon.2 v2:172.21.15.31:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fc5340f8ba0 con 0x7fc5480d0630 2024-09-10T21:25:20.608 INFO:teuthology.orchestra.run.smithi017.stdout:60129542153 2024-09-10T21:25:20.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.606+0000 7fc54e400640 1 -- 172.21.15.17:0/1840779579 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fc5280725e0 msgr2=0x7fc528074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:20.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.606+0000 7fc54e400640 1 --2- 172.21.15.17:0/1840779579 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fc5280725e0 0x7fc528074aa0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fc538009f20 tx=0x7fc5380094f0 comp rx=0 tx=0).stop 2024-09-10T21:25:20.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.606+0000 7fc54e400640 1 -- 172.21.15.17:0/1840779579 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc5480d0630 msgr2=0x7fc5481f3c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:20.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.606+0000 7fc54e400640 1 --2- 172.21.15.17:0/1840779579 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc5480d0630 0x7fc5481f3c00 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fc54815b660 tx=0x7fc5340a56d0 comp rx=0 tx=0).stop 2024-09-10T21:25:20.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.606+0000 7fc54e400640 1 -- 172.21.15.17:0/1840779579 shutdown_connections 2024-09-10T21:25:20.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.606+0000 7fc54e400640 1 --2- 172.21.15.17:0/1840779579 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fc5280725e0 0x7fc528074aa0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:20.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.606+0000 7fc54e400640 1 --2- 172.21.15.17:0/1840779579 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc54815e6a0 0x7fc5481f4140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:20.612 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.606+0000 7fc54e400640 1 --2- 172.21.15.17:0/1840779579 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc5480d0630 0x7fc5481f3c00 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:20.612 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.606+0000 7fc54e400640 1 --2- 172.21.15.17:0/1840779579 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc5480d0250 0x7fc54814c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:20.612 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.606+0000 7fc54e400640 1 -- 172.21.15.17:0/1840779579 >> 172.21.15.17:0/1840779579 conn(0x7fc548068990 msgr2=0x7fc548150ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:20.612 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.606+0000 7fc54e400640 1 -- 172.21.15.17:0/1840779579 shutdown_connections 2024-09-10T21:25:20.612 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.606+0000 7fc54e400640 1 -- 172.21.15.17:0/1840779579 wait complete. 2024-09-10T21:25:20.613 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.610+0000 7fbd20600640 1 -- 172.21.15.17:0/3135444326 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7fbcf4005590 con 0x7fbd18153d40 2024-09-10T21:25:20.613 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.610+0000 7fbd15600640 1 -- 172.21.15.17:0/3135444326 <== mon.1 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fbd10061b80 con 0x7fbd18153d40 2024-09-10T21:25:20.614 INFO:teuthology.orchestra.run.smithi017.stdout:64424509449 2024-09-10T21:25:20.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.610+0000 7ff7a2000640 1 -- 172.21.15.17:0/2258491875 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7ff7b006a200 con 0x7ff7b00b2d00 2024-09-10T21:25:20.616 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.614+0000 7ff7acc00640 1 -- 172.21.15.17:0/2258491875 <== mon.2 v2:172.21.15.31:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7ff7a405e3b0 con 0x7ff7b00b2d00 2024-09-10T21:25:20.617 INFO:teuthology.orchestra.run.smithi017.stdout:64424509450 2024-09-10T21:25:20.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.614+0000 7fbd20600640 1 -- 172.21.15.17:0/3135444326 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fbcf80726b0 msgr2=0x7fbcf8074b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:20.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.614+0000 7fbd20600640 1 --2- 172.21.15.17:0/3135444326 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fbcf80726b0 0x7fbcf8074b70 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fbd0c000c10 tx=0x7fbd0c01d8f0 comp rx=0 tx=0).stop 2024-09-10T21:25:20.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.614+0000 7fbd20600640 1 -- 172.21.15.17:0/3135444326 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbd18153d40 msgr2=0x7fbd18145b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:20.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.614+0000 7fbd20600640 1 --2- 172.21.15.17:0/3135444326 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbd18153d40 0x7fbd18145b00 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fbd10002170 tx=0x7fbd1000cbb0 comp rx=0 tx=0).stop 2024-09-10T21:25:20.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.614+0000 7fbd20600640 1 -- 172.21.15.17:0/3135444326 shutdown_connections 2024-09-10T21:25:20.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.614+0000 7fbd20600640 1 --2- 172.21.15.17:0/3135444326 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fbcf80726b0 0x7fbcf8074b70 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:20.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.614+0000 7fbd20600640 1 --2- 172.21.15.17:0/3135444326 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbd18153d40 0x7fbd18145b00 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:20.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.614+0000 7fbd20600640 1 --2- 172.21.15.17:0/3135444326 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbd180c7210 0x7fbd18139b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:20.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.614+0000 7fbd20600640 1 --2- 172.21.15.17:0/3135444326 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fbd180c0fd0 0x7fbd181490c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:20.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.614+0000 7fbd20600640 1 -- 172.21.15.17:0/3135444326 >> 172.21.15.17:0/3135444326 conn(0x7fbd180700d0 msgr2=0x7fbd18150dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:20.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.614+0000 7fbd20600640 1 -- 172.21.15.17:0/3135444326 shutdown_connections 2024-09-10T21:25:20.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.614+0000 7fbd20600640 1 -- 172.21.15.17:0/3135444326 wait complete. 2024-09-10T21:25:20.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.614+0000 7ff7a2000640 1 -- 172.21.15.17:0/2258491875 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7ff790072630 msgr2=0x7ff790074af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:20.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.614+0000 7ff7a2000640 1 --2- 172.21.15.17:0/2258491875 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7ff790072630 0x7ff790074af0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7ff7b00b2a40 tx=0x7ff79c0091e0 comp rx=0 tx=0).stop 2024-09-10T21:25:20.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.614+0000 7ff7a2000640 1 -- 172.21.15.17:0/2258491875 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff7b00b2d00 msgr2=0x7ff7b00b5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:20.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.614+0000 7ff7a2000640 1 --2- 172.21.15.17:0/2258491875 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff7b00b2d00 0x7ff7b00b5240 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ff7a400b4d0 tx=0x7ff7a400b990 comp rx=0 tx=0).stop 2024-09-10T21:25:20.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.614+0000 7ff7a2000640 1 -- 172.21.15.17:0/2258491875 shutdown_connections 2024-09-10T21:25:20.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.614+0000 7ff7a2000640 1 --2- 172.21.15.17:0/2258491875 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7ff790072630 0x7ff790074af0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:20.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.614+0000 7ff7a2000640 1 --2- 172.21.15.17:0/2258491875 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff7b015b280 0x7ff7b0202850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:20.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.614+0000 7ff7a2000640 1 --2- 172.21.15.17:0/2258491875 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff7b0154a20 0x7ff7b00b3ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:20.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.614+0000 7ff7a2000640 1 --2- 172.21.15.17:0/2258491875 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff7b00b2d00 0x7ff7b00b5240 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:20.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.614+0000 7ff7a2000640 1 -- 172.21.15.17:0/2258491875 >> 172.21.15.17:0/2258491875 conn(0x7ff7b0068990 msgr2=0x7ff7b0153970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:20.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.614+0000 7ff7a2000640 1 -- 172.21.15.17:0/2258491875 shutdown_connections 2024-09-10T21:25:20.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:20.614+0000 7ff7a2000640 1 -- 172.21.15.17:0/2258491875 wait complete. 2024-09-10T21:25:20.635 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542154 got 60129542153 for osd.3 2024-09-10T21:25:20.642 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509450 got 64424509449 for osd.7 2024-09-10T21:25:20.643 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509451 got 64424509450 for osd.1 2024-09-10T21:25:21.542 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-10T21:25:21.574 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-10T21:25:21.575 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-10T21:25:21.578 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-10T21:25:21.590 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-10T21:25:21.636 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-10T21:25:21.643 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-10T21:25:21.643 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-10T21:25:21.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.746+0000 7f7613a00640 1 -- 172.21.15.17:0/3130818714 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f760c0c12b0 msgr2=0x7f760c15a420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:21.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.746+0000 7f7613a00640 1 --2- 172.21.15.17:0/3130818714 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f760c0c12b0 0x7f760c15a420 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f75f8009840 tx=0x7f75f801efb0 comp rx=0 tx=0).stop 2024-09-10T21:25:21.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.746+0000 7f7613a00640 1 -- 172.21.15.17:0/3130818714 shutdown_connections 2024-09-10T21:25:21.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.746+0000 7f7613a00640 1 --2- 172.21.15.17:0/3130818714 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f760c0c12b0 0x7f760c15a420 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:21.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.746+0000 7f7613a00640 1 --2- 172.21.15.17:0/3130818714 >> v2:172.21.15.17:3300/0 conn(0x7f760c0c6860 0x7f760c06ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:21.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.746+0000 7f7613a00640 1 -- 172.21.15.17:0/3130818714 >> 172.21.15.17:0/3130818714 conn(0x7f760c06fe10 msgr2=0x7f760c0701f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:21.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.746+0000 7f7613a00640 1 -- 172.21.15.17:0/3130818714 shutdown_connections 2024-09-10T21:25:21.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.746+0000 7f7613a00640 1 -- 172.21.15.17:0/3130818714 wait complete. 2024-09-10T21:25:21.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.750+0000 7f7613a00640 1 Processor -- start 2024-09-10T21:25:21.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.750+0000 7f7613a00640 1 -- start start 2024-09-10T21:25:21.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.750+0000 7f7613a00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f760c0c12b0 0x7f760c148d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:21.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.750+0000 7f7613a00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f760c0c6860 0x7f760c0c3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:21.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.750+0000 7f7613a00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f760c15dc70 0x7f760c0c4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:21.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.750+0000 7f7613a00640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f760c06e7a0 con 0x7f760c0c6860 2024-09-10T21:25:21.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.750+0000 7f760be00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f760c0c6860 0x7f760c0c3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:21.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.750+0000 7f7611000640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f760c0c12b0 0x7f760c148d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:21.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.750+0000 7f760be00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f760c0c6860 0x7f760c0c3ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44502/0 (socket says 172.21.15.17:44502) 2024-09-10T21:25:21.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.750+0000 7f760be00640 1 -- 172.21.15.17:0/1845533753 learned_addr learned my addr 172.21.15.17:0/1845533753 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:25:21.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.750+0000 7f7611a00640 1 --2- 172.21.15.17:0/1845533753 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f760c15dc70 0x7f760c0c4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:21.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.750+0000 7f7613a00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f760c06e4c0 con 0x7f760c0c12b0 2024-09-10T21:25:21.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.754+0000 7f7613a00640 1 -- 172.21.15.17:0/1845533753 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f760c06ea80 con 0x7f760c15dc70 2024-09-10T21:25:21.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.754+0000 7f760be00640 1 -- 172.21.15.17:0/1845533753 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f760c15dc70 msgr2=0x7f760c0c4010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:21.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.754+0000 7f760be00640 1 --2- 172.21.15.17:0/1845533753 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f760c15dc70 0x7f760c0c4010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:21.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.754+0000 7f760be00640 1 -- 172.21.15.17:0/1845533753 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f760c0c12b0 msgr2=0x7f760c148d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:21.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.754+0000 7f760be00640 1 --2- 172.21.15.17:0/1845533753 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f760c0c12b0 0x7f760c148d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:21.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.754+0000 7f760be00640 1 -- 172.21.15.17:0/1845533753 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f760c1572e0 con 0x7f760c0c6860 2024-09-10T21:25:21.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.754+0000 7f7611000640 1 --2- 172.21.15.17:0/1845533753 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f760c0c12b0 0x7f760c148d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:25:21.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.754+0000 7f7611a00640 1 --2- 172.21.15.17:0/1845533753 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f760c15dc70 0x7f760c0c4010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:25:21.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.754+0000 7f760be00640 1 --2- 172.21.15.17:0/1845533753 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f760c0c6860 0x7f760c0c3ad0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f760400cce0 tx=0x7f760400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:21.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.754+0000 7f7608c00640 1 -- 172.21.15.17:0/1845533753 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7604004b30 con 0x7f760c0c6860 2024-09-10T21:25:21.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.758+0000 7f7613a00640 1 -- 172.21.15.17:0/1845533753 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f760c149250 con 0x7f760c0c6860 2024-09-10T21:25:21.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.758+0000 7f7613a00640 1 -- 172.21.15.17:0/1845533753 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f760c2016a0 con 0x7f760c0c6860 2024-09-10T21:25:21.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.758+0000 7f7613a00640 1 -- 172.21.15.17:0/1845533753 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f760c0c2340 con 0x7f760c0c6860 2024-09-10T21:25:21.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.758+0000 7f7608c00640 1 -- 172.21.15.17:0/1845533753 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7604004cd0 con 0x7f760c0c6860 2024-09-10T21:25:21.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.758+0000 7f7608c00640 1 -- 172.21.15.17:0/1845533753 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f760401d060 con 0x7f760c0c6860 2024-09-10T21:25:21.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.762+0000 7f7608c00640 1 -- 172.21.15.17:0/1845533753 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f76040052e0 con 0x7f760c0c6860 2024-09-10T21:25:21.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.762+0000 7f7608c00640 1 --2- 172.21.15.17:0/1845533753 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f75ec072610 0x7f75ec074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:21.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.762+0000 7f7611000640 1 --2- 172.21.15.17:0/1845533753 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f75ec072610 0x7f75ec074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:21.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.766+0000 7f7608c00640 1 -- 172.21.15.17:0/1845533753 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f7604091520 con 0x7f760c0c6860 2024-09-10T21:25:21.769 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.766+0000 7f7611000640 1 --2- 172.21.15.17:0/1845533753 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f75ec072610 0x7f75ec074ad0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f75fc000f70 tx=0x7f75fc009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:21.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.770+0000 7f7608c00640 1 -- 172.21.15.17:0/1845533753 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f760c149250 con 0x7f760c0c6860 2024-09-10T21:25:21.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.810+0000 7fdad8800640 1 -- 172.21.15.17:0/3679717037 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdad015dc70 msgr2=0x7fdad006ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:21.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.810+0000 7fdad8800640 1 --2- 172.21.15.17:0/3679717037 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdad015dc70 0x7fdad006ef80 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fdad00c9990 tx=0x7fdacc01f370 comp rx=0 tx=0).stop 2024-09-10T21:25:21.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.810+0000 7fdad8800640 1 -- 172.21.15.17:0/3679717037 shutdown_connections 2024-09-10T21:25:21.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.810+0000 7fdad8800640 1 --2- 172.21.15.17:0/3679717037 >> v2:172.21.15.17:3300/0 conn(0x7fdad00c6860 0x7fdad015e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:21.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.810+0000 7fdad8800640 1 --2- 172.21.15.17:0/3679717037 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdad015dc70 0x7fdad006ef80 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:21.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.810+0000 7fdad8800640 1 -- 172.21.15.17:0/3679717037 >> 172.21.15.17:0/3679717037 conn(0x7fdad006fe10 msgr2=0x7fdad00701f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:21.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.810+0000 7fdad8800640 1 -- 172.21.15.17:0/3679717037 shutdown_connections 2024-09-10T21:25:21.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.810+0000 7fdad8800640 1 -- 172.21.15.17:0/3679717037 wait complete. 2024-09-10T21:25:21.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.814+0000 7fdad8800640 1 Processor -- start 2024-09-10T21:25:21.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.814+0000 7fdad8800640 1 -- start start 2024-09-10T21:25:21.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.814+0000 7fdad8800640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdad00c6860 0x7fdad01583d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:21.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.814+0000 7fdad8800640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fdad015dc70 0x7fdad0149f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:21.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.814+0000 7fdad8800640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdad014db20 0x7fdad014ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:21.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.814+0000 7fdad5e00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdad00c6860 0x7fdad01583d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:21.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.818+0000 7fdad5e00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdad00c6860 0x7fdad01583d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.17:51528/0 (socket says 172.21.15.17:51528) 2024-09-10T21:25:21.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.818+0000 7fdad5400640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fdad015dc70 0x7fdad0149f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:21.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.818+0000 7fdad5400640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fdad015dc70 0x7fdad0149f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3301/0 says I am v2:172.21.15.17:43184/0 (socket says 172.21.15.17:43184) 2024-09-10T21:25:21.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.818+0000 7fdad6800640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdad014db20 0x7fdad014ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:21.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.818+0000 7fdad6800640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdad014db20 0x7fdad014ecf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44542/0 (socket says 172.21.15.17:44542) 2024-09-10T21:25:21.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.818+0000 7fdad5e00640 1 -- 172.21.15.17:0/3289112185 learned_addr learned my addr 172.21.15.17:0/3289112185 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:25:21.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.818+0000 7fdad8800640 1 -- 172.21.15.17:0/3289112185 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fdad006e7a0 con 0x7fdad014db20 2024-09-10T21:25:21.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.818+0000 7fdad8800640 1 -- 172.21.15.17:0/3289112185 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fdad006e2a0 con 0x7fdad00c6860 2024-09-10T21:25:21.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.818+0000 7fdad8800640 1 -- 172.21.15.17:0/3289112185 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7fdad006ea80 con 0x7fdad015dc70 2024-09-10T21:25:21.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.818+0000 7fdad6800640 1 -- 172.21.15.17:0/3289112185 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fdad015dc70 msgr2=0x7fdad0149f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:21.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.818+0000 7fdad6800640 1 --2- 172.21.15.17:0/3289112185 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fdad015dc70 0x7fdad0149f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:21.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.818+0000 7fdad6800640 1 -- 172.21.15.17:0/3289112185 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdad00c6860 msgr2=0x7fdad01583d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:21.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.818+0000 7fdad6800640 1 --2- 172.21.15.17:0/3289112185 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdad00c6860 0x7fdad01583d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:21.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.818+0000 7fdad6800640 1 -- 172.21.15.17:0/3289112185 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdad0150580 con 0x7fdad014db20 2024-09-10T21:25:21.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.818+0000 7fdad5e00640 1 --2- 172.21.15.17:0/3289112185 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdad00c6860 0x7fdad01583d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:25:21.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.818+0000 7fdad6800640 1 --2- 172.21.15.17:0/3289112185 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdad014db20 0x7fdad014ecf0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fdac4009420 tx=0x7fdac40098e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:21.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.818+0000 7fdaca000640 1 -- 172.21.15.17:0/3289112185 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdac40107b0 con 0x7fdad014db20 2024-09-10T21:25:21.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.818+0000 7fdad8800640 1 -- 172.21.15.17:0/3289112185 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdad015d400 con 0x7fdad014db20 2024-09-10T21:25:21.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.818+0000 7fdaca000640 1 -- 172.21.15.17:0/3289112185 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdac4016070 con 0x7fdad014db20 2024-09-10T21:25:21.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.818+0000 7fdaca000640 1 -- 172.21.15.17:0/3289112185 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdac4010ce0 con 0x7fdad014db20 2024-09-10T21:25:21.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.822+0000 7fdad8800640 1 -- 172.21.15.17:0/3289112185 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdad014f960 con 0x7fdad014db20 2024-09-10T21:25:21.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.822+0000 7fdabf400640 1 -- 172.21.15.17:0/3289112185 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdad00c2180 con 0x7fdad014db20 2024-09-10T21:25:21.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.822+0000 7f479ca00640 1 -- 172.21.15.17:0/2405085508 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f477c004170 msgr2=0x7f477c00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:21.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.822+0000 7f479ca00640 1 --2- 172.21.15.17:0/2405085508 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f477c004170 0x7f477c00b9d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f4790001f80 tx=0x7f479001da40 comp rx=0 tx=0).stop 2024-09-10T21:25:21.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.822+0000 7f4796800640 1 -- 172.21.15.17:0/2405085508 reap_dead start 2024-09-10T21:25:21.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.822+0000 7f4e5fe00640 1 -- 172.21.15.17:0/1995164998 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4e60071190 msgr2=0x7f4e600c8e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:21.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.822+0000 7f4e5fe00640 1 --2- 172.21.15.17:0/1995164998 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4e60071190 0x7f4e600c8e80 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f4e4c009840 tx=0x7f4e4c01f030 comp rx=0 tx=0).stop 2024-09-10T21:25:21.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.826+0000 7f4e5fe00640 1 -- 172.21.15.17:0/1995164998 shutdown_connections 2024-09-10T21:25:21.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.826+0000 7f4e5fe00640 1 --2- 172.21.15.17:0/1995164998 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4e60071190 0x7f4e600c8e80 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:21.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.826+0000 7f4e5fe00640 1 --2- 172.21.15.17:0/1995164998 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f4e600cac50 0x7f4e6015afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:21.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.826+0000 7f4e5fe00640 1 -- 172.21.15.17:0/1995164998 >> 172.21.15.17:0/1995164998 conn(0x7f4e60068990 msgr2=0x7f4e60068da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:21.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.826+0000 7fdaca000640 1 -- 172.21.15.17:0/3289112185 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fdac401b460 con 0x7fdad014db20 2024-09-10T21:25:21.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.826+0000 7f479ca00640 1 -- 172.21.15.17:0/2405085508 shutdown_connections 2024-09-10T21:25:21.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.826+0000 7f479ca00640 1 -- 172.21.15.17:0/2405085508 >> 172.21.15.17:0/2405085508 conn(0x7f479806fe10 msgr2=0x7f47980701f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:21.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.826+0000 7f4e5fe00640 1 -- 172.21.15.17:0/1995164998 shutdown_connections 2024-09-10T21:25:21.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.826+0000 7f479ca00640 1 -- 172.21.15.17:0/2405085508 shutdown_connections 2024-09-10T21:25:21.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.826+0000 7f479ca00640 1 -- 172.21.15.17:0/2405085508 wait complete. 2024-09-10T21:25:21.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.826+0000 7fdaca000640 1 --2- 172.21.15.17:0/3289112185 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fdab40725e0 0x7fdab4074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:21.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.826+0000 7fdad5e00640 1 --2- 172.21.15.17:0/3289112185 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fdab40725e0 0x7fdab4074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:21.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.826+0000 7f4e5fe00640 1 -- 172.21.15.17:0/1995164998 wait complete. 2024-09-10T21:25:21.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.826+0000 7f4e5fe00640 1 Processor -- start 2024-09-10T21:25:21.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.826+0000 7fdaca000640 1 -- 172.21.15.17:0/3289112185 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fdac4048bb0 con 0x7fdad014db20 2024-09-10T21:25:21.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.826+0000 7fdad5e00640 1 --2- 172.21.15.17:0/3289112185 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fdab40725e0 0x7fdab4074aa0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fdacc009b10 tx=0x7fdacc01dbb0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:21.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.826+0000 7f479ca00640 1 Processor -- start 2024-09-10T21:25:21.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.830+0000 7f479ca00640 1 -- start start 2024-09-10T21:25:21.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.830+0000 7f4e5fe00640 1 -- start start 2024-09-10T21:25:21.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.830+0000 7f4e5fe00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4e600cac50 0x7f4e601ee8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:21.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.830+0000 7f4e5fe00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4e600cb030 0x7f4e601fd620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:21.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.830+0000 7f4e5fe00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f4e6006fb60 0x7f4e60200320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:21.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.830+0000 7f4e5fe00640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f4e60072220 con 0x7f4e600cac50 2024-09-10T21:25:21.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.830+0000 7f4e5fe00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f4e60071f40 con 0x7f4e600cb030 2024-09-10T21:25:21.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.830+0000 7f4e5fe00640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f4e60072500 con 0x7f4e6006fb60 2024-09-10T21:25:21.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.830+0000 7f4e5ea00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4e600cac50 0x7f4e601ee8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:21.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.830+0000 7f4e5e000640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4e600cb030 0x7f4e601fd620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:21.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.830+0000 7f4e5f400640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f4e6006fb60 0x7f4e60200320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:21.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.830+0000 7f4e5ea00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4e600cac50 0x7f4e601ee8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44554/0 (socket says 172.21.15.17:44554) 2024-09-10T21:25:21.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.830+0000 7f4e5ea00640 1 -- 172.21.15.17:0/1315142383 learned_addr learned my addr 172.21.15.17:0/1315142383 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:25:21.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.830+0000 7f479ca00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f477c004170 0x7f479814b590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:21.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.830+0000 7f479ca00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f47981e1f30 0x7f479814de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:21.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.830+0000 7f479ca00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f47981e7480 0x7f479814a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:21.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.830+0000 7f479ca00640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f47980c7d80 con 0x7f477c004170 2024-09-10T21:25:21.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.830+0000 7f479ca00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f4798073590 con 0x7f47981e1f30 2024-09-10T21:25:21.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.830+0000 7f479ca00640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f47980c7f00 con 0x7f47981e7480 2024-09-10T21:25:21.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.830+0000 7f4e5e000640 1 -- 172.21.15.17:0/1315142383 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f4e6006fb60 msgr2=0x7f4e60200320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:21.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.830+0000 7f4e5e000640 1 --2- 172.21.15.17:0/1315142383 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f4e6006fb60 0x7f4e60200320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:21.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.830+0000 7f4e5e000640 1 -- 172.21.15.17:0/1315142383 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4e600cac50 msgr2=0x7f4e601ee8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:21.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.830+0000 7f4e5e000640 1 --2- 172.21.15.17:0/1315142383 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4e600cac50 0x7f4e601ee8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:21.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.830+0000 7f4e5e000640 1 -- 172.21.15.17:0/1315142383 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e601594d0 con 0x7f4e600cb030 2024-09-10T21:25:21.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.830+0000 7f4795400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f47981e1f30 0x7f479814de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:21.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.830+0000 7f4796800640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f47981e7480 0x7f479814a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:21.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.830+0000 7f4796800640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f47981e7480 0x7f479814a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3301/0 says I am v2:172.21.15.17:43202/0 (socket says 172.21.15.17:43202) 2024-09-10T21:25:21.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.830+0000 7f4796800640 1 -- 172.21.15.17:0/3485133911 learned_addr learned my addr 172.21.15.17:0/3485133911 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:25:21.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.830+0000 7f4795e00640 1 --2- 172.21.15.17:0/3485133911 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f477c004170 0x7f479814b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:21.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.830+0000 7f4e5e000640 1 --2- 172.21.15.17:0/1315142383 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4e600cb030 0x7f4e601fd620 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f4e4c0af000 tx=0x7f4e4c005cc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:21.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.830+0000 7f4796800640 1 -- 172.21.15.17:0/3485133911 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f47981e1f30 msgr2=0x7f479814de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:21.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.834+0000 7f4796800640 1 --2- 172.21.15.17:0/3485133911 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f47981e1f30 0x7f479814de30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:21.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.834+0000 7f4796800640 1 -- 172.21.15.17:0/3485133911 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f477c004170 msgr2=0x7f479814b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:21.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.834+0000 7f4796800640 1 --2- 172.21.15.17:0/3485133911 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f477c004170 0x7f479814b590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:21.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.834+0000 7f4796800640 1 -- 172.21.15.17:0/3485133911 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47981599b0 con 0x7f47981e7480 2024-09-10T21:25:21.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.830+0000 7f4e57400640 1 -- 172.21.15.17:0/1315142383 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e4c0bd020 con 0x7f4e600cb030 2024-09-10T21:25:21.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.834+0000 7f4e57400640 1 -- 172.21.15.17:0/1315142383 <== mon.1 v2:172.21.15.31:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4e4c0043e0 con 0x7f4e600cb030 2024-09-10T21:25:21.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.834+0000 7f4e57400640 1 -- 172.21.15.17:0/1315142383 <== mon.1 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e4c0b5040 con 0x7f4e600cb030 2024-09-10T21:25:21.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.834+0000 7f4796800640 1 --2- 172.21.15.17:0/3485133911 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f47981e7480 0x7f479814a780 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f479001c140 tx=0x7f4790007c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:21.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.834+0000 7f4e5fe00640 1 -- 172.21.15.17:0/1315142383 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e60159770 con 0x7f4e600cb030 2024-09-10T21:25:21.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.834+0000 7f4e5fe00640 1 -- 172.21.15.17:0/1315142383 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e60159d30 con 0x7f4e600cb030 2024-09-10T21:25:21.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.834+0000 7f478e000640 1 -- 172.21.15.17:0/3485133911 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4790004e40 con 0x7f47981e7480 2024-09-10T21:25:21.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.834+0000 7f478e000640 1 -- 172.21.15.17:0/3485133911 <== mon.2 v2:172.21.15.31:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4790004a90 con 0x7f47981e7480 2024-09-10T21:25:21.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.834+0000 7f478e000640 1 -- 172.21.15.17:0/3485133911 <== mon.2 v2:172.21.15.31:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4790001aa0 con 0x7f47981e7480 2024-09-10T21:25:21.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.834+0000 7f479ca00640 1 -- 172.21.15.17:0/3485133911 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f479815bdb0 con 0x7f47981e7480 2024-09-10T21:25:21.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.834+0000 7f479ca00640 1 -- 172.21.15.17:0/3485133911 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f4798159df0 con 0x7f47981e7480 2024-09-10T21:25:21.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.838+0000 7f479ca00640 1 -- 172.21.15.17:0/3485133911 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4798152690 con 0x7f47981e7480 2024-09-10T21:25:21.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.838+0000 7f4e57400640 1 -- 172.21.15.17:0/1315142383 <== mon.1 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f4e4c0b8070 con 0x7f4e600cb030 2024-09-10T21:25:21.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.838+0000 7f478e000640 1 -- 172.21.15.17:0/3485133911 <== mon.2 v2:172.21.15.31:3301/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f4790001ce0 con 0x7f47981e7480 2024-09-10T21:25:21.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.838+0000 7f4e57400640 1 --2- 172.21.15.17:0/1315142383 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f4e400725e0 0x7f4e40074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:21.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.838+0000 7fdaca000640 1 -- 172.21.15.17:0/3289112185 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fdac405dda0 con 0x7fdad014db20 2024-09-10T21:25:21.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.838+0000 7f4e5fe00640 1 -- 172.21.15.17:0/1315142383 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e600c94f0 con 0x7f4e600cb030 2024-09-10T21:25:21.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.838+0000 7f4e5ea00640 1 --2- 172.21.15.17:0/1315142383 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f4e400725e0 0x7f4e40074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:21.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.838+0000 7f4e57400640 1 -- 172.21.15.17:0/1315142383 <== mon.1 v2:172.21.15.31:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f4e4c130510 con 0x7f4e600cb030 2024-09-10T21:25:21.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.842+0000 7f478e000640 1 --2- 172.21.15.17:0/3485133911 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f4774072630 0x7f4774074af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:21.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.842+0000 7f478e000640 1 -- 172.21.15.17:0/3485133911 <== mon.2 v2:172.21.15.31:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f479012cee0 con 0x7f47981e7480 2024-09-10T21:25:21.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.842+0000 7f4795e00640 1 --2- 172.21.15.17:0/3485133911 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f4774072630 0x7f4774074af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:21.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.842+0000 7f4e5ea00640 1 --2- 172.21.15.17:0/1315142383 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f4e400725e0 0x7f4e40074aa0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f4e500058d0 tx=0x7f4e5000b480 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:21.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.842+0000 7f4795e00640 1 --2- 172.21.15.17:0/3485133911 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f4774072630 0x7f4774074af0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f47980ccec0 tx=0x7f47880091e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:21.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.846+0000 7f478e000640 1 -- 172.21.15.17:0/3485133911 <== mon.2 v2:172.21.15.31:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f47900c0020 con 0x7f47981e7480 2024-09-10T21:25:21.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.846+0000 7f4e57400640 1 -- 172.21.15.17:0/1315142383 <== mon.1 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f4e4c018b90 con 0x7f4e600cb030 2024-09-10T21:25:21.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.902+0000 7f5c20800640 1 -- 172.21.15.17:0/826224398 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5c18151530 msgr2=0x7f5c180c2bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:21.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.902+0000 7f5c20800640 1 --2- 172.21.15.17:0/826224398 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5c18151530 0x7f5c180c2bf0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f5c0c009840 tx=0x7f5c0c01ee50 comp rx=0 tx=0).stop 2024-09-10T21:25:21.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.902+0000 7f5c20800640 1 -- 172.21.15.17:0/826224398 shutdown_connections 2024-09-10T21:25:21.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.902+0000 7f5c20800640 1 --2- 172.21.15.17:0/826224398 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f5c180b1a50 0x7f5c181571b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:21.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.902+0000 7f5c20800640 1 --2- 172.21.15.17:0/826224398 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5c18151530 0x7f5c180c2bf0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:21.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.902+0000 7f5c20800640 1 -- 172.21.15.17:0/826224398 >> 172.21.15.17:0/826224398 conn(0x7f5c180700d0 msgr2=0x7f5c180704e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:21.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.906+0000 7f5c20800640 1 -- 172.21.15.17:0/826224398 shutdown_connections 2024-09-10T21:25:21.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.906+0000 7f1ad1400640 1 -- 172.21.15.17:0/373893795 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1ac404baf0 msgr2=0x7f1ac4006ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:21.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.906+0000 7f1ad1400640 1 --2- 172.21.15.17:0/373893795 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1ac404baf0 0x7f1ac4006ec0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f1ab800dc90 tx=0x7f1ab8023040 comp rx=0 tx=0).stop 2024-09-10T21:25:21.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.906+0000 7f1ad1400640 1 -- 172.21.15.17:0/373893795 shutdown_connections 2024-09-10T21:25:21.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.906+0000 7f1ad1400640 1 --2- 172.21.15.17:0/373893795 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f1ac404c3b0 0x7f1ac4007400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:21.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.906+0000 7f1ad1400640 1 --2- 172.21.15.17:0/373893795 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1ac404baf0 0x7f1ac4006ec0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:21.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.906+0000 7f1ad1400640 1 -- 172.21.15.17:0/373893795 >> 172.21.15.17:0/373893795 conn(0x7f1ac4050060 msgr2=0x7f1ac4050470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:21.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.906+0000 7f1ad1400640 1 -- 172.21.15.17:0/373893795 shutdown_connections 2024-09-10T21:25:21.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.906+0000 7f1ad1400640 1 -- 172.21.15.17:0/373893795 wait complete. 2024-09-10T21:25:21.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.906+0000 7f5c20800640 1 -- 172.21.15.17:0/826224398 wait complete. 2024-09-10T21:25:21.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.906+0000 7f5c20800640 1 Processor -- start 2024-09-10T21:25:21.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.910+0000 7f5c20800640 1 -- start start 2024-09-10T21:25:21.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.910+0000 7f1ad1400640 1 Processor -- start 2024-09-10T21:25:21.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.910+0000 7f5c20800640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f5c180725c0 0x7f5c1806df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:21.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.910+0000 7f5c20800640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5c180b1a50 0x7f5c18200c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:21.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.910+0000 7f5c20800640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5c18151530 0x7f5c1814ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:21.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.910+0000 7f5c1de00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f5c180725c0 0x7f5c1806df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:21.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.910+0000 7f5c20800640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f5c18074b30 con 0x7f5c18151530 2024-09-10T21:25:21.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.910+0000 7f5c20800640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f5c18074850 con 0x7f5c180b1a50 2024-09-10T21:25:21.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.910+0000 7f5c20800640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f5c180c3130 con 0x7f5c180725c0 2024-09-10T21:25:21.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.910+0000 7f5c1d400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5c180b1a50 0x7f5c18200c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:21.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.910+0000 7f5c1de00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f5c180725c0 0x7f5c1806df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3301/0 says I am v2:172.21.15.17:43230/0 (socket says 172.21.15.17:43230) 2024-09-10T21:25:21.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.910+0000 7f5c1de00640 1 -- 172.21.15.17:0/3555673444 learned_addr learned my addr 172.21.15.17:0/3555673444 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:25:21.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.910+0000 7f1ad1400640 1 -- start start 2024-09-10T21:25:21.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.914+0000 7f5c1e800640 1 --2- 172.21.15.17:0/3555673444 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5c18151530 0x7f5c1814ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:21.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.914+0000 7f1ad1400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1ac404baf0 0x7f1ac40d2570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:21.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.914+0000 7f1ad1400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1ac404c3b0 0x7f1ac4179a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:21.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.914+0000 7f1ad1400640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f1ac40d4980 0x7f1ac40d0270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:21.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.914+0000 7f5c1de00640 1 -- 172.21.15.17:0/3555673444 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5c180b1a50 msgr2=0x7f5c18200c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:21.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.914+0000 7f5c1de00640 1 --2- 172.21.15.17:0/3555673444 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5c180b1a50 0x7f5c18200c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:21.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.914+0000 7f1acb400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1ac404c3b0 0x7f1ac4179a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:21.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.914+0000 7f1acb400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1ac404c3b0 0x7f1ac4179a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44614/0 (socket says 172.21.15.17:44614) 2024-09-10T21:25:21.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.914+0000 7f1acbe00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1ac404baf0 0x7f1ac40d2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:21.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.914+0000 7f1ad1400640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f1ac40066c0 con 0x7f1ac404c3b0 2024-09-10T21:25:21.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.914+0000 7f1ad1400640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f1ac40063e0 con 0x7f1ac404baf0 2024-09-10T21:25:21.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.914+0000 7f1acbe00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1ac404baf0 0x7f1ac40d2570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.17:51578/0 (socket says 172.21.15.17:51578) 2024-09-10T21:25:21.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.914+0000 7f1acbe00640 1 -- 172.21.15.17:0/3992653565 learned_addr learned my addr 172.21.15.17:0/3992653565 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:25:21.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.914+0000 7f5c1de00640 1 -- 172.21.15.17:0/3555673444 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5c18151530 msgr2=0x7f5c1814ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:21.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.914+0000 7f5c1d400640 1 --2- 172.21.15.17:0/3555673444 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5c180b1a50 0x7f5c18200c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:25:21.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.914+0000 7f5c1de00640 1 --2- 172.21.15.17:0/3555673444 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5c18151530 0x7f5c1814ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:21.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.914+0000 7f5c1de00640 1 -- 172.21.15.17:0/3555673444 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c181f2c80 con 0x7f5c180725c0 2024-09-10T21:25:21.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.914+0000 7f5c1e800640 1 --2- 172.21.15.17:0/3555673444 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5c18151530 0x7f5c1814ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:25:21.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.914+0000 7f1ad0a00640 1 --2- 172.21.15.17:0/3992653565 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f1ac40d4980 0x7f1ac40d0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:21.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.914+0000 7f5c1de00640 1 --2- 172.21.15.17:0/3555673444 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f5c180725c0 0x7f5c1806df70 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f5c0c01f6f0 tx=0x7f5c0c004620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:21.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.914+0000 7f1ad1400640 1 -- 172.21.15.17:0/3992653565 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f1ac40069a0 con 0x7f1ac40d4980 2024-09-10T21:25:21.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.914+0000 7f1acbe00640 1 -- 172.21.15.17:0/3992653565 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f1ac40d4980 msgr2=0x7f1ac40d0270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:21.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.914+0000 7f1acbe00640 1 --2- 172.21.15.17:0/3992653565 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f1ac40d4980 0x7f1ac40d0270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:21.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.914+0000 7f1acbe00640 1 -- 172.21.15.17:0/3992653565 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1ac404c3b0 msgr2=0x7f1ac4179a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:21.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.914+0000 7f5c12000640 1 -- 172.21.15.17:0/3555673444 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c0c0be070 con 0x7f5c180725c0 2024-09-10T21:25:21.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.914+0000 7f1acbe00640 1 --2- 172.21.15.17:0/3992653565 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1ac404c3b0 0x7f1ac4179a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:21.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.914+0000 7f5c20800640 1 -- 172.21.15.17:0/3555673444 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c181f2f20 con 0x7f5c180725c0 2024-09-10T21:25:21.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.914+0000 7f5c12000640 1 -- 172.21.15.17:0/3555673444 <== mon.2 v2:172.21.15.31:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5c0c0053e0 con 0x7f5c180725c0 2024-09-10T21:25:21.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.914+0000 7f5c20800640 1 -- 172.21.15.17:0/3555673444 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c18200750 con 0x7f5c180725c0 2024-09-10T21:25:21.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.914+0000 7f1acbe00640 1 -- 172.21.15.17:0/3992653565 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ac4178f50 con 0x7f1ac404baf0 2024-09-10T21:25:21.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.918+0000 7f1acb400640 1 --2- 172.21.15.17:0/3992653565 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1ac404c3b0 0x7f1ac4179a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:25:21.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.918+0000 7f5c12000640 1 -- 172.21.15.17:0/3555673444 <== mon.2 v2:172.21.15.31:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c0c0be070 con 0x7f5c180725c0 2024-09-10T21:25:21.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.918+0000 7f1acbe00640 1 --2- 172.21.15.17:0/3992653565 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1ac404baf0 0x7f1ac40d2570 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f1ac000c450 tx=0x7f1ac000c910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:21.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.918+0000 7f1abfe00640 1 -- 172.21.15.17:0/3992653565 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ac00052d0 con 0x7f1ac404baf0 2024-09-10T21:25:21.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.918+0000 7f1ad1400640 1 -- 172.21.15.17:0/3992653565 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ac40d1a00 con 0x7f1ac404baf0 2024-09-10T21:25:21.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.918+0000 7f5c20800640 1 -- 172.21.15.17:0/3555673444 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c18075150 con 0x7f5c180725c0 2024-09-10T21:25:21.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.918+0000 7f1ad1400640 1 -- 172.21.15.17:0/3992653565 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ac40d1fc0 con 0x7f1ac404baf0 2024-09-10T21:25:21.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.918+0000 7f1abfe00640 1 -- 172.21.15.17:0/3992653565 <== mon.1 v2:172.21.15.31:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1ac0005470 con 0x7f1ac404baf0 2024-09-10T21:25:21.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.918+0000 7fc41f200640 1 -- 172.21.15.17:0/106905991 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc4180c7ca0 msgr2=0x7fc4180c80b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:21.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.918+0000 7fc41f200640 1 --2- 172.21.15.17:0/106905991 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc4180c7ca0 0x7fc4180c80b0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fc414009bc0 tx=0x7fc41401cb30 comp rx=0 tx=0).stop 2024-09-10T21:25:21.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.918+0000 7f1abfe00640 1 -- 172.21.15.17:0/3992653565 <== mon.1 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ac0012870 con 0x7f1ac404baf0 2024-09-10T21:25:21.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.918+0000 7fc41f200640 1 -- 172.21.15.17:0/106905991 shutdown_connections 2024-09-10T21:25:21.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.918+0000 7fc41f200640 1 --2- 172.21.15.17:0/106905991 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc4180c7ca0 0x7fc4180c80b0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:21.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.918+0000 7fc41f200640 1 --2- 172.21.15.17:0/106905991 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc418152510 0x7fc418074ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:21.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.918+0000 7fc41f200640 1 -- 172.21.15.17:0/106905991 >> 172.21.15.17:0/106905991 conn(0x7fc41806fe60 msgr2=0x7fc418070270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:21.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.922+0000 7fc41f200640 1 -- 172.21.15.17:0/106905991 shutdown_connections 2024-09-10T21:25:21.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.922+0000 7fc41f200640 1 -- 172.21.15.17:0/106905991 wait complete. 2024-09-10T21:25:21.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.922+0000 7f5c12000640 1 -- 172.21.15.17:0/3555673444 <== mon.2 v2:172.21.15.31:3301/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f5c0c0b8030 con 0x7f5c180725c0 2024-09-10T21:25:21.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.922+0000 7f5c12000640 1 --2- 172.21.15.17:0/3555673444 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f5bfc072a10 0x7f5bfc074ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:21.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.922+0000 7f5c12000640 1 -- 172.21.15.17:0/3555673444 <== mon.2 v2:172.21.15.31:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f5c0c0050f0 con 0x7f5c180725c0 2024-09-10T21:25:21.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.922+0000 7f5c1d400640 1 --2- 172.21.15.17:0/3555673444 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f5bfc072a10 0x7f5bfc074ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:21.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.922+0000 7f5c12000640 1 -- 172.21.15.17:0/3555673444 <== mon.2 v2:172.21.15.31:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f5c0c0c7020 con 0x7f5c180725c0 2024-09-10T21:25:21.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.922+0000 7f5c1d400640 1 --2- 172.21.15.17:0/3555673444 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f5bfc072a10 0x7f5bfc074ed0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f5c180c2520 tx=0x7f5c080091e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:21.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.922+0000 7f1abfe00640 1 -- 172.21.15.17:0/3992653565 <== mon.1 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f1ac0012a10 con 0x7f1ac404baf0 2024-09-10T21:25:21.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.922+0000 7f1abfe00640 1 --2- 172.21.15.17:0/3992653565 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f1aac0725e0 0x7f1aac074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:21.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.922+0000 7f1abfe00640 1 -- 172.21.15.17:0/3992653565 <== mon.1 v2:172.21.15.31:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f1ac0093090 con 0x7f1ac404baf0 2024-09-10T21:25:21.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.926+0000 7f1ad1400640 1 -- 172.21.15.17:0/3992653565 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1aa80052a0 con 0x7f1ac404baf0 2024-09-10T21:25:21.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.926+0000 7fc41f200640 1 Processor -- start 2024-09-10T21:25:21.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.926+0000 7f8020200640 1 -- 172.21.15.17:0/1462595895 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f801806dd10 msgr2=0x7f801806e150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:21.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.926+0000 7f8020200640 1 --2- 172.21.15.17:0/1462595895 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f801806dd10 0x7f801806e150 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f801000afd0 tx=0x7f8010020000 comp rx=0 tx=0).stop 2024-09-10T21:25:21.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.926+0000 7f8020200640 1 -- 172.21.15.17:0/1462595895 shutdown_connections 2024-09-10T21:25:21.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.926+0000 7f8020200640 1 --2- 172.21.15.17:0/1462595895 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f801806dd10 0x7f801806e150 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:21.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.926+0000 7f8020200640 1 --2- 172.21.15.17:0/1462595895 >> v2:172.21.15.17:3300/0 conn(0x7f80180c8010 0x7f80180c0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:21.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.926+0000 7f8020200640 1 -- 172.21.15.17:0/1462595895 >> 172.21.15.17:0/1462595895 conn(0x7f80180700d0 msgr2=0x7f80180704e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:21.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.926+0000 7f1acb400640 1 --2- 172.21.15.17:0/3992653565 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f1aac0725e0 0x7f1aac074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:21.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.926+0000 7f8020200640 1 -- 172.21.15.17:0/1462595895 shutdown_connections 2024-09-10T21:25:21.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.926+0000 7f8020200640 1 -- 172.21.15.17:0/1462595895 wait complete. 2024-09-10T21:25:21.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.926+0000 7f8020200640 1 Processor -- start 2024-09-10T21:25:21.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.926+0000 7f8020200640 1 -- start start 2024-09-10T21:25:21.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.926+0000 7fc41f200640 1 -- start start 2024-09-10T21:25:21.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.926+0000 7fc41f200640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc418070980 0x7fc4181572f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:21.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.926+0000 7fc41f200640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc4180c7ca0 0x7fc418147ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:21.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.926+0000 7fc41f200640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc418152510 0x7fc4181481e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:21.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.926+0000 7fc41f200640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fc4180c6f70 con 0x7fc4180c7ca0 2024-09-10T21:25:21.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.926+0000 7fc41f200640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fc4181528f0 con 0x7fc418070980 2024-09-10T21:25:21.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.926+0000 7fc41f200640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7fc4181501e0 con 0x7fc418152510 2024-09-10T21:25:21.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.926+0000 7fc41e800640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc418152510 0x7fc4181481e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:21.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.930+0000 7fc41de00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc418070980 0x7fc4181572f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:21.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.930+0000 7fc41d400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc4180c7ca0 0x7fc418147ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:21.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.930+0000 7fc41e800640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc418152510 0x7fc4181481e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3301/0 says I am v2:172.21.15.17:43260/0 (socket says 172.21.15.17:43260) 2024-09-10T21:25:21.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.930+0000 7fc41e800640 1 -- 172.21.15.17:0/2663958597 learned_addr learned my addr 172.21.15.17:0/2663958597 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:25:21.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.930+0000 7f1acb400640 1 --2- 172.21.15.17:0/3992653565 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f1aac0725e0 0x7f1aac074aa0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f1ab80b5000 tx=0x7f1ab8021880 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:21.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.930+0000 7fc41e800640 1 -- 172.21.15.17:0/2663958597 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc418070980 msgr2=0x7fc4181572f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:21.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.930+0000 7fc41e800640 1 --2- 172.21.15.17:0/2663958597 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc418070980 0x7fc4181572f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:21.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.930+0000 7fc41e800640 1 -- 172.21.15.17:0/2663958597 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc4180c7ca0 msgr2=0x7fc418147ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:21.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.930+0000 7f8020200640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f801806dd10 0x7f80181f16d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:21.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.930+0000 7f8020200640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f80180c8010 0x7f8018151840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:21.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.930+0000 7f8020200640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8018153d40 0x7f80181469a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:21.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.930+0000 7f8020200640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f80180c92c0 con 0x7f801806dd10 2024-09-10T21:25:21.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.930+0000 7f8020200640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f80180c8fe0 con 0x7f80180c8010 2024-09-10T21:25:21.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.930+0000 7f8020200640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f80180c95a0 con 0x7f8018153d40 2024-09-10T21:25:21.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.930+0000 7f801d800640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f801806dd10 0x7f80181f16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:21.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.930+0000 7f801e200640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8018153d40 0x7f80181469a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:21.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.930+0000 7f801d800640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f801806dd10 0x7f80181f16d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44634/0 (socket says 172.21.15.17:44634) 2024-09-10T21:25:21.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.930+0000 7f801d800640 1 -- 172.21.15.17:0/4137244861 learned_addr learned my addr 172.21.15.17:0/4137244861 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:25:21.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.930+0000 7f801ce00640 1 --2- 172.21.15.17:0/4137244861 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f80180c8010 0x7f8018151840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:21.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.930+0000 7f801e200640 1 -- 172.21.15.17:0/4137244861 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f80180c8010 msgr2=0x7f8018151840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:21.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.930+0000 7f801e200640 1 --2- 172.21.15.17:0/4137244861 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f80180c8010 0x7f8018151840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:21.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.930+0000 7f801e200640 1 -- 172.21.15.17:0/4137244861 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f801806dd10 msgr2=0x7f80181f16d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:21.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.930+0000 7f801e200640 1 --2- 172.21.15.17:0/4137244861 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f801806dd10 0x7f80181f16d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:21.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.930+0000 7f801e200640 1 -- 172.21.15.17:0/4137244861 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80180c3960 con 0x7f8018153d40 2024-09-10T21:25:21.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.930+0000 7f801d800640 1 --2- 172.21.15.17:0/4137244861 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f801806dd10 0x7f80181f16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:25:21.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.930+0000 7f1abfe00640 1 -- 172.21.15.17:0/3992653565 <== mon.1 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f1ac005e280 con 0x7f1ac404baf0 2024-09-10T21:25:21.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.930+0000 7fc41e800640 1 --2- 172.21.15.17:0/2663958597 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc4180c7ca0 0x7fc418147ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:21.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.930+0000 7fc41e800640 1 -- 172.21.15.17:0/2663958597 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc418150440 con 0x7fc418152510 2024-09-10T21:25:21.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.930+0000 7fc41d400640 1 --2- 172.21.15.17:0/2663958597 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc4180c7ca0 0x7fc418147ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:25:21.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.930+0000 7f801e200640 1 --2- 172.21.15.17:0/4137244861 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8018153d40 0x7f80181469a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f8010004000 tx=0x7f8010009fe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:21.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.930+0000 7f8015600640 1 -- 172.21.15.17:0/4137244861 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8010003cd0 con 0x7f8018153d40 2024-09-10T21:25:21.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.930+0000 7f8015600640 1 -- 172.21.15.17:0/4137244861 <== mon.2 v2:172.21.15.31:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8010004030 con 0x7f8018153d40 2024-09-10T21:25:21.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.930+0000 7f8015600640 1 -- 172.21.15.17:0/4137244861 <== mon.2 v2:172.21.15.31:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8010004220 con 0x7f8018153d40 2024-09-10T21:25:21.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.930+0000 7f8020200640 1 -- 172.21.15.17:0/4137244861 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80181ee160 con 0x7f8018153d40 2024-09-10T21:25:21.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.930+0000 7fc41e800640 1 --2- 172.21.15.17:0/2663958597 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc418152510 0x7fc4181481e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc41401d3d0 tx=0x7fc4140a5780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:21.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.934+0000 7fc412000640 1 -- 172.21.15.17:0/2663958597 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4140ba020 con 0x7fc418152510 2024-09-10T21:25:21.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.934+0000 7fc412000640 1 -- 172.21.15.17:0/2663958597 <== mon.2 v2:172.21.15.31:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc414004d40 con 0x7fc418152510 2024-09-10T21:25:21.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.934+0000 7fc412000640 1 -- 172.21.15.17:0/2663958597 <== mon.2 v2:172.21.15.31:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4140b2040 con 0x7fc418152510 2024-09-10T21:25:21.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.934+0000 7fc41f200640 1 -- 172.21.15.17:0/2663958597 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4181fb450 con 0x7fc418152510 2024-09-10T21:25:21.939 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.930+0000 7f8020200640 1 -- 172.21.15.17:0/4137244861 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f80180c3ee0 con 0x7f8018153d40 2024-09-10T21:25:21.939 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.934+0000 7f8020200640 1 -- 172.21.15.17:0/4137244861 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80180747d0 con 0x7f8018153d40 2024-09-10T21:25:21.939 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.934+0000 7fc41f200640 1 -- 172.21.15.17:0/2663958597 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fc418150970 con 0x7fc418152510 2024-09-10T21:25:21.939 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.934+0000 7f8015600640 1 -- 172.21.15.17:0/4137244861 <== mon.2 v2:172.21.15.31:3301/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f8010008b60 con 0x7f8018153d40 2024-09-10T21:25:21.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.934+0000 7f8015600640 1 --2- 172.21.15.17:0/4137244861 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f7ff80725e0 0x7f7ff8074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:21.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.938+0000 7f8015600640 1 -- 172.21.15.17:0/4137244861 <== mon.2 v2:172.21.15.31:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f8010130bb0 con 0x7f8018153d40 2024-09-10T21:25:21.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.934+0000 7fc407400640 1 -- 172.21.15.17:0/2663958597 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3f00052a0 con 0x7fc418152510 2024-09-10T21:25:21.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.938+0000 7f801d800640 1 --2- 172.21.15.17:0/4137244861 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f7ff80725e0 0x7f7ff8074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:21.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.938+0000 7f801d800640 1 --2- 172.21.15.17:0/4137244861 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f7ff80725e0 0x7f7ff8074aa0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f800c001be0 tx=0x7f800c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:21.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.938+0000 7f8015600640 1 -- 172.21.15.17:0/4137244861 <== mon.2 v2:172.21.15.31:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f8010019b60 con 0x7f8018153d40 2024-09-10T21:25:21.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.938+0000 7fc412000640 1 -- 172.21.15.17:0/2663958597 <== mon.2 v2:172.21.15.31:3301/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fc4140b5070 con 0x7fc418152510 2024-09-10T21:25:21.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.938+0000 7fc412000640 1 --2- 172.21.15.17:0/2663958597 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fc3fc072660 0x7fc3fc074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:21.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.938+0000 7fc412000640 1 -- 172.21.15.17:0/2663958597 <== mon.2 v2:172.21.15.31:3301/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fc41412d6e0 con 0x7fc418152510 2024-09-10T21:25:21.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.942+0000 7fc41de00640 1 --2- 172.21.15.17:0/2663958597 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fc3fc072660 0x7fc3fc074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:21.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.946+0000 7fc41de00640 1 --2- 172.21.15.17:0/2663958597 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fc3fc072660 0x7fc3fc074b20 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fc4080045f0 tx=0x7fc40800b2e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:21.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:21.946+0000 7fc412000640 1 -- 172.21.15.17:0/2663958597 <== mon.2 v2:172.21.15.31:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fc4140dc020 con 0x7fc418152510 2024-09-10T21:25:22.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.146+0000 7f7613a00640 1 -- 172.21.15.17:0/1845533753 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f760c0c0e80 con 0x7f760c0c6860 2024-09-10T21:25:22.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.150+0000 7f7608c00640 1 -- 172.21.15.17:0/1845533753 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f7604014030 con 0x7f760c0c6860 2024-09-10T21:25:22.153 INFO:teuthology.orchestra.run.smithi017.stdout:64424509451 2024-09-10T21:25:22.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.154+0000 7f7613a00640 1 -- 172.21.15.17:0/1845533753 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f75ec072610 msgr2=0x7f75ec074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:22.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.154+0000 7f7613a00640 1 --2- 172.21.15.17:0/1845533753 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f75ec072610 0x7f75ec074ad0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f75fc000f70 tx=0x7f75fc009040 comp rx=0 tx=0).stop 2024-09-10T21:25:22.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.154+0000 7f7613a00640 1 -- 172.21.15.17:0/1845533753 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f760c0c6860 msgr2=0x7f760c0c3ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:22.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.154+0000 7f7613a00640 1 --2- 172.21.15.17:0/1845533753 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f760c0c6860 0x7f760c0c3ad0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f760400cce0 tx=0x7f760400a620 comp rx=0 tx=0).stop 2024-09-10T21:25:22.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.154+0000 7f7613a00640 1 -- 172.21.15.17:0/1845533753 shutdown_connections 2024-09-10T21:25:22.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.154+0000 7f7613a00640 1 --2- 172.21.15.17:0/1845533753 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f75ec072610 0x7f75ec074ad0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:22.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.154+0000 7f7613a00640 1 --2- 172.21.15.17:0/1845533753 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f760c15dc70 0x7f760c0c4010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:22.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.154+0000 7f7613a00640 1 --2- 172.21.15.17:0/1845533753 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f760c0c6860 0x7f760c0c3ad0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:22.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.154+0000 7f7613a00640 1 --2- 172.21.15.17:0/1845533753 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f760c0c12b0 0x7f760c148d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:22.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.154+0000 7f7613a00640 1 -- 172.21.15.17:0/1845533753 >> 172.21.15.17:0/1845533753 conn(0x7f760c06fe10 msgr2=0x7f760c0cb0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:22.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.154+0000 7f7613a00640 1 -- 172.21.15.17:0/1845533753 shutdown_connections 2024-09-10T21:25:22.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.154+0000 7f7613a00640 1 -- 172.21.15.17:0/1845533753 wait complete. 2024-09-10T21:25:22.196 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509451 got 64424509451 for osd.6 2024-09-10T21:25:22.196 DEBUG:teuthology.parallel:result is None 2024-09-10T21:25:22.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.198+0000 7f479ca00640 1 -- 172.21.15.17:0/3485133911 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f47980c34d0 con 0x7f47981e7480 2024-09-10T21:25:22.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.198+0000 7f478e000640 1 -- 172.21.15.17:0/3485133911 <== mon.2 v2:172.21.15.31:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f47900177f0 con 0x7f47981e7480 2024-09-10T21:25:22.203 INFO:teuthology.orchestra.run.smithi017.stdout:64424509450 2024-09-10T21:25:22.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.202+0000 7fdabf400640 1 -- 172.21.15.17:0/3289112185 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7fdad00c0e80 con 0x7fdad014db20 2024-09-10T21:25:22.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.202+0000 7f479ca00640 1 -- 172.21.15.17:0/3485133911 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f4774072630 msgr2=0x7f4774074af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:22.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.202+0000 7f479ca00640 1 --2- 172.21.15.17:0/3485133911 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f4774072630 0x7f4774074af0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f47980ccec0 tx=0x7f47880091e0 comp rx=0 tx=0).stop 2024-09-10T21:25:22.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.202+0000 7f479ca00640 1 -- 172.21.15.17:0/3485133911 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f47981e7480 msgr2=0x7f479814a780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:22.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.202+0000 7f479ca00640 1 --2- 172.21.15.17:0/3485133911 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f47981e7480 0x7f479814a780 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f479001c140 tx=0x7f4790007c10 comp rx=0 tx=0).stop 2024-09-10T21:25:22.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.202+0000 7f479ca00640 1 -- 172.21.15.17:0/3485133911 shutdown_connections 2024-09-10T21:25:22.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.202+0000 7f479ca00640 1 --2- 172.21.15.17:0/3485133911 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f4774072630 0x7f4774074af0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:22.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.202+0000 7f479ca00640 1 --2- 172.21.15.17:0/3485133911 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f47981e7480 0x7f479814a780 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:22.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.202+0000 7f479ca00640 1 --2- 172.21.15.17:0/3485133911 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f47981e1f30 0x7f479814de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:22.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.202+0000 7f479ca00640 1 --2- 172.21.15.17:0/3485133911 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f477c004170 0x7f479814b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:22.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.202+0000 7f479ca00640 1 -- 172.21.15.17:0/3485133911 >> 172.21.15.17:0/3485133911 conn(0x7f479806fe10 msgr2=0x7f4798075a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:22.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.202+0000 7fdaca000640 1 -- 172.21.15.17:0/3289112185 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fdac405d6a0 con 0x7fdad014db20 2024-09-10T21:25:22.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.202+0000 7f479ca00640 1 -- 172.21.15.17:0/3485133911 shutdown_connections 2024-09-10T21:25:22.208 INFO:teuthology.orchestra.run.smithi017.stdout:60129542158 2024-09-10T21:25:22.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.202+0000 7f479ca00640 1 -- 172.21.15.17:0/3485133911 wait complete. 2024-09-10T21:25:22.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.206+0000 7fdabf400640 1 -- 172.21.15.17:0/3289112185 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fdab40725e0 msgr2=0x7fdab4074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:22.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.206+0000 7fdabf400640 1 --2- 172.21.15.17:0/3289112185 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fdab40725e0 0x7fdab4074aa0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fdacc009b10 tx=0x7fdacc01dbb0 comp rx=0 tx=0).stop 2024-09-10T21:25:22.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.206+0000 7fdabf400640 1 -- 172.21.15.17:0/3289112185 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdad014db20 msgr2=0x7fdad014ecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:22.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.206+0000 7fdabf400640 1 --2- 172.21.15.17:0/3289112185 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdad014db20 0x7fdad014ecf0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fdac4009420 tx=0x7fdac40098e0 comp rx=0 tx=0).stop 2024-09-10T21:25:22.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.206+0000 7fdabf400640 1 -- 172.21.15.17:0/3289112185 shutdown_connections 2024-09-10T21:25:22.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.206+0000 7fdabf400640 1 --2- 172.21.15.17:0/3289112185 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fdab40725e0 0x7fdab4074aa0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:22.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.206+0000 7fdabf400640 1 --2- 172.21.15.17:0/3289112185 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdad014db20 0x7fdad014ecf0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:22.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.206+0000 7fdabf400640 1 --2- 172.21.15.17:0/3289112185 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fdad015dc70 0x7fdad0149f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:22.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.206+0000 7fdabf400640 1 --2- 172.21.15.17:0/3289112185 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdad00c6860 0x7fdad01583d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:22.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.206+0000 7fdabf400640 1 -- 172.21.15.17:0/3289112185 >> 172.21.15.17:0/3289112185 conn(0x7fdad006fe10 msgr2=0x7fdad00c1550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:22.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.206+0000 7fdabf400640 1 -- 172.21.15.17:0/3289112185 shutdown_connections 2024-09-10T21:25:22.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.206+0000 7fdabf400640 1 -- 172.21.15.17:0/3289112185 wait complete. 2024-09-10T21:25:22.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.218+0000 7f4e5fe00640 1 -- 172.21.15.17:0/1315142383 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f4e600b2820 con 0x7f4e600cb030 2024-09-10T21:25:22.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.218+0000 7f4e57400640 1 -- 172.21.15.17:0/1315142383 <== mon.1 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f4e4c0fb130 con 0x7f4e600cb030 2024-09-10T21:25:22.224 INFO:teuthology.orchestra.run.smithi017.stdout:55834574859 2024-09-10T21:25:22.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.222+0000 7f4e5fe00640 1 -- 172.21.15.17:0/1315142383 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f4e400725e0 msgr2=0x7f4e40074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:22.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.222+0000 7f4e5fe00640 1 --2- 172.21.15.17:0/1315142383 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f4e400725e0 0x7f4e40074aa0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f4e500058d0 tx=0x7f4e5000b480 comp rx=0 tx=0).stop 2024-09-10T21:25:22.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.222+0000 7f4e5fe00640 1 -- 172.21.15.17:0/1315142383 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4e600cb030 msgr2=0x7f4e601fd620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:22.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.222+0000 7f4e5fe00640 1 --2- 172.21.15.17:0/1315142383 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4e600cb030 0x7f4e601fd620 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f4e4c0af000 tx=0x7f4e4c005cc0 comp rx=0 tx=0).stop 2024-09-10T21:25:22.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.226+0000 7f4e5fe00640 1 -- 172.21.15.17:0/1315142383 shutdown_connections 2024-09-10T21:25:22.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.226+0000 7f4e5fe00640 1 --2- 172.21.15.17:0/1315142383 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f4e400725e0 0x7f4e40074aa0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:22.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.226+0000 7f4e5fe00640 1 --2- 172.21.15.17:0/1315142383 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f4e6006fb60 0x7f4e60200320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:22.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.226+0000 7f4e5fe00640 1 --2- 172.21.15.17:0/1315142383 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4e600cb030 0x7f4e601fd620 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:22.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.226+0000 7f4e5fe00640 1 --2- 172.21.15.17:0/1315142383 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4e600cac50 0x7f4e601ee8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:22.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.226+0000 7f4e5fe00640 1 -- 172.21.15.17:0/1315142383 >> 172.21.15.17:0/1315142383 conn(0x7f4e60068990 msgr2=0x7f4e601f3a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:22.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.226+0000 7f4e5fe00640 1 -- 172.21.15.17:0/1315142383 shutdown_connections 2024-09-10T21:25:22.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.226+0000 7f4e5fe00640 1 -- 172.21.15.17:0/1315142383 wait complete. 2024-09-10T21:25:22.242 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542157 got 60129542158 for osd.0 2024-09-10T21:25:22.242 DEBUG:teuthology.parallel:result is None 2024-09-10T21:25:22.243 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509450 got 64424509450 for osd.2 2024-09-10T21:25:22.243 DEBUG:teuthology.parallel:result is None 2024-09-10T21:25:22.254 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574859 for osd.5 2024-09-10T21:25:22.254 DEBUG:teuthology.parallel:result is None 2024-09-10T21:25:22.265 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.262+0000 7f5c20800640 1 -- 172.21.15.17:0/3555673444 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f5c180b2130 con 0x7f5c180725c0 2024-09-10T21:25:22.266 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.262+0000 7f5c12000640 1 -- 172.21.15.17:0/3555673444 <== mon.2 v2:172.21.15.31:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f5c0c018a70 con 0x7f5c180725c0 2024-09-10T21:25:22.266 INFO:teuthology.orchestra.run.smithi017.stdout:60129542155 2024-09-10T21:25:22.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.262+0000 7f5c20800640 1 -- 172.21.15.17:0/3555673444 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f5bfc072a10 msgr2=0x7f5bfc074ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:22.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.262+0000 7f5c20800640 1 --2- 172.21.15.17:0/3555673444 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f5bfc072a10 0x7f5bfc074ed0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f5c180c2520 tx=0x7f5c080091e0 comp rx=0 tx=0).stop 2024-09-10T21:25:22.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.262+0000 7f5c20800640 1 -- 172.21.15.17:0/3555673444 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f5c180725c0 msgr2=0x7f5c1806df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:22.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.262+0000 7f5c20800640 1 --2- 172.21.15.17:0/3555673444 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f5c180725c0 0x7f5c1806df70 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f5c0c01f6f0 tx=0x7f5c0c004620 comp rx=0 tx=0).stop 2024-09-10T21:25:22.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.266+0000 7f5c20800640 1 -- 172.21.15.17:0/3555673444 shutdown_connections 2024-09-10T21:25:22.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.266+0000 7f5c20800640 1 --2- 172.21.15.17:0/3555673444 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f5bfc072a10 0x7f5bfc074ed0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:22.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.266+0000 7f5c20800640 1 --2- 172.21.15.17:0/3555673444 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5c18151530 0x7f5c1814ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:22.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.266+0000 7f5c20800640 1 --2- 172.21.15.17:0/3555673444 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5c180b1a50 0x7f5c18200c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:22.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.266+0000 7f5c20800640 1 --2- 172.21.15.17:0/3555673444 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f5c180725c0 0x7f5c1806df70 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:22.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.266+0000 7f5c20800640 1 -- 172.21.15.17:0/3555673444 >> 172.21.15.17:0/3555673444 conn(0x7f5c180700d0 msgr2=0x7f5c180c19a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:22.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.266+0000 7f5c20800640 1 -- 172.21.15.17:0/3555673444 shutdown_connections 2024-09-10T21:25:22.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.266+0000 7f5c20800640 1 -- 172.21.15.17:0/3555673444 wait complete. 2024-09-10T21:25:22.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.270+0000 7f1ad1400640 1 -- 172.21.15.17:0/3992653565 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f1aa8005590 con 0x7f1ac404baf0 2024-09-10T21:25:22.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.270+0000 7f1abfe00640 1 -- 172.21.15.17:0/3992653565 <== mon.1 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f1ac005db80 con 0x7f1ac404baf0 2024-09-10T21:25:22.275 INFO:teuthology.orchestra.run.smithi017.stdout:60129542156 2024-09-10T21:25:22.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.274+0000 7f1ad1400640 1 -- 172.21.15.17:0/3992653565 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f1aac0725e0 msgr2=0x7f1aac074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:22.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.274+0000 7f1ad1400640 1 --2- 172.21.15.17:0/3992653565 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f1aac0725e0 0x7f1aac074aa0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f1ab80b5000 tx=0x7f1ab8021880 comp rx=0 tx=0).stop 2024-09-10T21:25:22.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.274+0000 7f1ad1400640 1 -- 172.21.15.17:0/3992653565 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1ac404baf0 msgr2=0x7f1ac40d2570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:22.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.274+0000 7f1ad1400640 1 --2- 172.21.15.17:0/3992653565 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1ac404baf0 0x7f1ac40d2570 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f1ac000c450 tx=0x7f1ac000c910 comp rx=0 tx=0).stop 2024-09-10T21:25:22.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.274+0000 7f1ad1400640 1 -- 172.21.15.17:0/3992653565 shutdown_connections 2024-09-10T21:25:22.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.274+0000 7f1ad1400640 1 --2- 172.21.15.17:0/3992653565 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f1aac0725e0 0x7f1aac074aa0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:22.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.274+0000 7f1ad1400640 1 --2- 172.21.15.17:0/3992653565 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f1ac40d4980 0x7f1ac40d0270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:22.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.274+0000 7f1ad1400640 1 --2- 172.21.15.17:0/3992653565 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1ac404c3b0 0x7f1ac4179a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:22.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.274+0000 7f1ad1400640 1 --2- 172.21.15.17:0/3992653565 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1ac404baf0 0x7f1ac40d2570 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:22.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.274+0000 7f1ad1400640 1 -- 172.21.15.17:0/3992653565 >> 172.21.15.17:0/3992653565 conn(0x7f1ac4050060 msgr2=0x7f1ac41628a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:22.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.274+0000 7f1ad1400640 1 -- 172.21.15.17:0/3992653565 shutdown_connections 2024-09-10T21:25:22.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.274+0000 7f1ad1400640 1 -- 172.21.15.17:0/3992653565 wait complete. 2024-09-10T21:25:22.289 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542154 got 60129542155 for osd.3 2024-09-10T21:25:22.289 DEBUG:teuthology.parallel:result is None 2024-09-10T21:25:22.301 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542155 got 60129542156 for osd.4 2024-09-10T21:25:22.301 DEBUG:teuthology.parallel:result is None 2024-09-10T21:25:22.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.302+0000 7fc407400640 1 -- 172.21.15.17:0/2663958597 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7fc3f0005860 con 0x7fc418152510 2024-09-10T21:25:22.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.302+0000 7fc412000640 1 -- 172.21.15.17:0/2663958597 <== mon.2 v2:172.21.15.31:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fc4140167b0 con 0x7fc418152510 2024-09-10T21:25:22.307 INFO:teuthology.orchestra.run.smithi017.stdout:64424509450 2024-09-10T21:25:22.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.306+0000 7fc407400640 1 -- 172.21.15.17:0/2663958597 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fc3fc072660 msgr2=0x7fc3fc074b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:22.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.306+0000 7fc407400640 1 --2- 172.21.15.17:0/2663958597 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fc3fc072660 0x7fc3fc074b20 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fc4080045f0 tx=0x7fc40800b2e0 comp rx=0 tx=0).stop 2024-09-10T21:25:22.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.306+0000 7fc407400640 1 -- 172.21.15.17:0/2663958597 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc418152510 msgr2=0x7fc4181481e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:22.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.306+0000 7fc407400640 1 --2- 172.21.15.17:0/2663958597 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc418152510 0x7fc4181481e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc41401d3d0 tx=0x7fc4140a5780 comp rx=0 tx=0).stop 2024-09-10T21:25:22.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.306+0000 7fc407400640 1 -- 172.21.15.17:0/2663958597 shutdown_connections 2024-09-10T21:25:22.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.306+0000 7fc407400640 1 --2- 172.21.15.17:0/2663958597 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fc3fc072660 0x7fc3fc074b20 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:22.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.306+0000 7fc407400640 1 --2- 172.21.15.17:0/2663958597 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc418152510 0x7fc4181481e0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:22.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.306+0000 7fc407400640 1 --2- 172.21.15.17:0/2663958597 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc4180c7ca0 0x7fc418147ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:22.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.306+0000 7fc407400640 1 --2- 172.21.15.17:0/2663958597 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc418070980 0x7fc4181572f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:22.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.306+0000 7fc407400640 1 -- 172.21.15.17:0/2663958597 >> 172.21.15.17:0/2663958597 conn(0x7fc41806fe60 msgr2=0x7fc418161ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:22.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.306+0000 7fc407400640 1 -- 172.21.15.17:0/2663958597 shutdown_connections 2024-09-10T21:25:22.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.306+0000 7fc407400640 1 -- 172.21.15.17:0/2663958597 wait complete. 2024-09-10T21:25:22.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.306+0000 7f8020200640 1 -- 172.21.15.17:0/4137244861 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f8018074580 con 0x7f8018153d40 2024-09-10T21:25:22.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.306+0000 7f8015600640 1 -- 172.21.15.17:0/4137244861 <== mon.2 v2:172.21.15.31:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f80100fb970 con 0x7f8018153d40 2024-09-10T21:25:22.311 INFO:teuthology.orchestra.run.smithi017.stdout:64424509451 2024-09-10T21:25:22.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.310+0000 7f8020200640 1 -- 172.21.15.17:0/4137244861 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f7ff80725e0 msgr2=0x7f7ff8074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:22.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.310+0000 7f8020200640 1 --2- 172.21.15.17:0/4137244861 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f7ff80725e0 0x7f7ff8074aa0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f800c001be0 tx=0x7f800c009040 comp rx=0 tx=0).stop 2024-09-10T21:25:22.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.310+0000 7f8020200640 1 -- 172.21.15.17:0/4137244861 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8018153d40 msgr2=0x7f80181469a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:22.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.310+0000 7f8020200640 1 --2- 172.21.15.17:0/4137244861 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8018153d40 0x7f80181469a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f8010004000 tx=0x7f8010009fe0 comp rx=0 tx=0).stop 2024-09-10T21:25:22.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.310+0000 7f8020200640 1 -- 172.21.15.17:0/4137244861 shutdown_connections 2024-09-10T21:25:22.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.310+0000 7f8020200640 1 --2- 172.21.15.17:0/4137244861 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f7ff80725e0 0x7f7ff8074aa0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:22.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.310+0000 7f8020200640 1 --2- 172.21.15.17:0/4137244861 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8018153d40 0x7f80181469a0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:22.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.310+0000 7f8020200640 1 --2- 172.21.15.17:0/4137244861 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f80180c8010 0x7f8018151840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:22.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.310+0000 7f8020200640 1 --2- 172.21.15.17:0/4137244861 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f801806dd10 0x7f80181f16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:22.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.310+0000 7f8020200640 1 -- 172.21.15.17:0/4137244861 >> 172.21.15.17:0/4137244861 conn(0x7f80180700d0 msgr2=0x7f80180704b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:22.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.310+0000 7f8020200640 1 -- 172.21.15.17:0/4137244861 shutdown_connections 2024-09-10T21:25:22.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.310+0000 7f8020200640 1 -- 172.21.15.17:0/4137244861 wait complete. 2024-09-10T21:25:22.330 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509450 got 64424509450 for osd.7 2024-09-10T21:25:22.330 DEBUG:teuthology.parallel:result is None 2024-09-10T21:25:22.336 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509451 got 64424509451 for osd.1 2024-09-10T21:25:22.336 DEBUG:teuthology.parallel:result is None 2024-09-10T21:25:22.336 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-10T21:25:22.336 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T21:25:22.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.522+0000 7fde29a00640 1 -- 172.21.15.17:0/2431202649 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fde08003c40 msgr2=0x7fde08004040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:22.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.522+0000 7fde29a00640 1 --2- 172.21.15.17:0/2431202649 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fde08003c40 0x7fde08004040 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fde14009b60 tx=0x7fde1401d270 comp rx=0 tx=0).stop 2024-09-10T21:25:22.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.522+0000 7fde29a00640 1 -- 172.21.15.17:0/2431202649 shutdown_connections 2024-09-10T21:25:22.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.522+0000 7fde29a00640 1 --2- 172.21.15.17:0/2431202649 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fde08003c40 0x7fde08004040 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:22.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.522+0000 7fde29a00640 1 -- 172.21.15.17:0/2431202649 >> 172.21.15.17:0/2431202649 conn(0x7fde2406dd30 msgr2=0x7fde2406e140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:22.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.522+0000 7fde29a00640 1 -- 172.21.15.17:0/2431202649 shutdown_connections 2024-09-10T21:25:22.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.522+0000 7fde29a00640 1 -- 172.21.15.17:0/2431202649 wait complete. 2024-09-10T21:25:22.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.526+0000 7fde29a00640 1 Processor -- start 2024-09-10T21:25:22.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.526+0000 7fde29a00640 1 -- start start 2024-09-10T21:25:22.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.526+0000 7fde29a00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fde08003c40 0x7fde240aff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:22.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.526+0000 7fde29a00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fde240c9330 0x7fde240c1b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:22.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.526+0000 7fde29a00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde240c28b0 0x7fde24156f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:22.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.526+0000 7fde29a00640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fde2415b690 con 0x7fde08003c40 2024-09-10T21:25:22.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.526+0000 7fde29a00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fde2415b3b0 con 0x7fde240c28b0 2024-09-10T21:25:22.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.526+0000 7fde29a00640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7fde24069b90 con 0x7fde240c9330 2024-09-10T21:25:22.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.526+0000 7fde22a00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fde08003c40 0x7fde240aff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:22.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.526+0000 7fde22a00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fde08003c40 0x7fde240aff40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44650/0 (socket says 172.21.15.17:44650) 2024-09-10T21:25:22.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.526+0000 7fde22a00640 1 -- 172.21.15.17:0/3088374386 learned_addr learned my addr 172.21.15.17:0/3088374386 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:25:22.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.526+0000 7fde1a000640 1 --2- 172.21.15.17:0/3088374386 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fde240c9330 0x7fde240c1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:22.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.526+0000 7fde23400640 1 --2- 172.21.15.17:0/3088374386 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde240c28b0 0x7fde24156f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:22.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.526+0000 7fde22a00640 1 -- 172.21.15.17:0/3088374386 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fde240c9330 msgr2=0x7fde240c1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:22.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.526+0000 7fde22a00640 1 --2- 172.21.15.17:0/3088374386 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fde240c9330 0x7fde240c1b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:22.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.526+0000 7fde22a00640 1 -- 172.21.15.17:0/3088374386 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde240c28b0 msgr2=0x7fde24156f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:22.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.526+0000 7fde22a00640 1 --2- 172.21.15.17:0/3088374386 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde240c28b0 0x7fde24156f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:22.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.526+0000 7fde22a00640 1 -- 172.21.15.17:0/3088374386 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde24156ac0 con 0x7fde08003c40 2024-09-10T21:25:22.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.526+0000 7fde22a00640 1 --2- 172.21.15.17:0/3088374386 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fde08003c40 0x7fde240aff40 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fde1401d0a0 tx=0x7fde14005560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:22.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.526+0000 7fde1b400640 1 -- 172.21.15.17:0/3088374386 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde140b1040 con 0x7fde08003c40 2024-09-10T21:25:22.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.526+0000 7fde1b400640 1 -- 172.21.15.17:0/3088374386 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fde140bb070 con 0x7fde08003c40 2024-09-10T21:25:22.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.526+0000 7fde29a00640 1 -- 172.21.15.17:0/3088374386 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde240c2060 con 0x7fde08003c40 2024-09-10T21:25:22.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.526+0000 7fde1b400640 1 -- 172.21.15.17:0/3088374386 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde140043a0 con 0x7fde08003c40 2024-09-10T21:25:22.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.526+0000 7fde29a00640 1 -- 172.21.15.17:0/3088374386 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde240c1040 con 0x7fde08003c40 2024-09-10T21:25:22.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.526+0000 7fde29a00640 1 -- 172.21.15.17:0/3088374386 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde240c56a0 con 0x7fde08003c40 2024-09-10T21:25:22.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.530+0000 7fde1b400640 1 -- 172.21.15.17:0/3088374386 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fde140046d0 con 0x7fde08003c40 2024-09-10T21:25:22.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.530+0000 7fde1b400640 1 --2- 172.21.15.17:0/3088374386 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fde000725e0 0x7fde00074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:22.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.530+0000 7fde1b400640 1 -- 172.21.15.17:0/3088374386 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fde1412e710 con 0x7fde08003c40 2024-09-10T21:25:22.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.530+0000 7fde1a000640 1 --2- 172.21.15.17:0/3088374386 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fde000725e0 0x7fde00074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:22.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.530+0000 7fde1a000640 1 --2- 172.21.15.17:0/3088374386 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fde000725e0 0x7fde00074aa0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fde24152630 tx=0x7fde0c006b40 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:22.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.534+0000 7fde1b400640 1 -- 172.21.15.17:0/3088374386 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fde14016f80 con 0x7fde08003c40 2024-09-10T21:25:22.709 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.706+0000 7fde29a00640 1 -- 172.21.15.17:0/3088374386 --> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fde240746d0 con 0x7fde000725e0 2024-09-10T21:25:22.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.710+0000 7fde1b400640 1 -- 172.21.15.17:0/3088374386 <== mgr.4100 v2:172.21.15.31:6808/3101308586 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332597 (secure 0 0 0) 0x7fde240746d0 con 0x7fde000725e0 2024-09-10T21:25:22.719 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-10T21:25:22.721 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2024-09-10T21:25:22.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.718+0000 7fde29a00640 1 -- 172.21.15.17:0/3088374386 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fde000725e0 msgr2=0x7fde00074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:22.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.718+0000 7fde29a00640 1 --2- 172.21.15.17:0/3088374386 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fde000725e0 0x7fde00074aa0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fde24152630 tx=0x7fde0c006b40 comp rx=0 tx=0).stop 2024-09-10T21:25:22.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.718+0000 7fde29a00640 1 -- 172.21.15.17:0/3088374386 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fde08003c40 msgr2=0x7fde240aff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:22.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.718+0000 7fde29a00640 1 --2- 172.21.15.17:0/3088374386 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fde08003c40 0x7fde240aff40 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fde1401d0a0 tx=0x7fde14005560 comp rx=0 tx=0).stop 2024-09-10T21:25:22.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.718+0000 7fde29a00640 1 -- 172.21.15.17:0/3088374386 shutdown_connections 2024-09-10T21:25:22.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.718+0000 7fde29a00640 1 --2- 172.21.15.17:0/3088374386 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fde000725e0 0x7fde00074aa0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:22.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.718+0000 7fde29a00640 1 --2- 172.21.15.17:0/3088374386 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde240c28b0 0x7fde24156f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:22.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.718+0000 7fde29a00640 1 --2- 172.21.15.17:0/3088374386 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fde240c9330 0x7fde240c1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:22.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.718+0000 7fde29a00640 1 --2- 172.21.15.17:0/3088374386 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fde08003c40 0x7fde240aff40 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:22.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.718+0000 7fde29a00640 1 -- 172.21.15.17:0/3088374386 >> 172.21.15.17:0/3088374386 conn(0x7fde2406dd30 msgr2=0x7fde2406eac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:22.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.718+0000 7fde29a00640 1 -- 172.21.15.17:0/3088374386 shutdown_connections 2024-09-10T21:25:22.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.718+0000 7fde29a00640 1 -- 172.21.15.17:0/3088374386 wait complete. 2024-09-10T21:25:22.740 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":42,"stamp":"2024-09-10T21:25:21.346548+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":149,"num_write_kb":1646,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":114,"ondisk_log_size":114,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":222608,"kb_used_data":2208,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754752112,"statfs":{"total":773094113280,"available":772866162688,"internally_reserved":0,"allocated":2260992,"data_stored":2009992,"data_compressed":66932,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12752,"internal_metadata":225496624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003364"},"pg_stats":[{"pgid":"2.1f","version":"24'4","reported_seq":27,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174456+0000","last_change":"2024-09-10T21:24:46.864556+0000","last_active":"2024-09-10T21:24:53.174456+0000","last_peered":"2024-09-10T21:24:53.174456+0000","last_clean":"2024-09-10T21:24:53.174456+0000","last_became_active":"2024-09-10T21:24:46.864288+0000","last_became_peered":"2024-09-10T21:24:46.864288+0000","last_unstale":"2024-09-10T21:24:53.174456+0000","last_undegraded":"2024-09-10T21:24:53.174456+0000","last_fullsized":"2024-09-10T21:24:53.174456+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:00:38.734807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.885137+0000","last_change":"2024-09-10T21:24:47.864452+0000","last_active":"2024-09-10T21:24:52.885137+0000","last_peered":"2024-09-10T21:24:52.885137+0000","last_clean":"2024-09-10T21:24:52.885137+0000","last_became_active":"2024-09-10T21:24:47.864168+0000","last_became_peered":"2024-09-10T21:24:47.864168+0000","last_unstale":"2024-09-10T21:24:52.885137+0000","last_undegraded":"2024-09-10T21:24:52.885137+0000","last_fullsized":"2024-09-10T21:24:52.885137+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:51:31.417845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.884507+0000","last_change":"2024-09-10T21:24:52.070054+0000","last_active":"2024-09-10T21:24:52.884507+0000","last_peered":"2024-09-10T21:24:52.884507+0000","last_clean":"2024-09-10T21:24:52.884507+0000","last_became_active":"2024-09-10T21:24:52.067258+0000","last_became_peered":"2024-09-10T21:24:52.067258+0000","last_unstale":"2024-09-10T21:24:52.884507+0000","last_undegraded":"2024-09-10T21:24:52.884507+0000","last_fullsized":"2024-09-10T21:24:52.884507+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:21:53.080013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"24'1","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.885320+0000","last_change":"2024-09-10T21:24:46.860648+0000","last_active":"2024-09-10T21:24:52.885320+0000","last_peered":"2024-09-10T21:24:52.885320+0000","last_clean":"2024-09-10T21:24:52.885320+0000","last_became_active":"2024-09-10T21:24:46.860341+0000","last_became_peered":"2024-09-10T21:24:46.860341+0000","last_unstale":"2024-09-10T21:24:52.885320+0000","last_undegraded":"2024-09-10T21:24:52.885320+0000","last_fullsized":"2024-09-10T21:24:52.885320+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:39:24.819979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174137+0000","last_change":"2024-09-10T21:24:47.865617+0000","last_active":"2024-09-10T21:24:53.174137+0000","last_peered":"2024-09-10T21:24:53.174137+0000","last_clean":"2024-09-10T21:24:53.174137+0000","last_became_active":"2024-09-10T21:24:47.865365+0000","last_became_peered":"2024-09-10T21:24:47.865365+0000","last_unstale":"2024-09-10T21:24:53.174137+0000","last_undegraded":"2024-09-10T21:24:53.174137+0000","last_fullsized":"2024-09-10T21:24:53.174137+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:10:05.971996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.883672+0000","last_change":"2024-09-10T21:24:51.900077+0000","last_active":"2024-09-10T21:24:52.883672+0000","last_peered":"2024-09-10T21:24:52.883672+0000","last_clean":"2024-09-10T21:24:52.883672+0000","last_became_active":"2024-09-10T21:24:51.899652+0000","last_became_peered":"2024-09-10T21:24:51.899652+0000","last_unstale":"2024-09-10T21:24:52.883672+0000","last_undegraded":"2024-09-10T21:24:52.883672+0000","last_fullsized":"2024-09-10T21:24:52.883672+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:40:49.585857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"24'2","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.175383+0000","last_change":"2024-09-10T21:24:46.861040+0000","last_active":"2024-09-10T21:24:53.175383+0000","last_peered":"2024-09-10T21:24:53.175383+0000","last_clean":"2024-09-10T21:24:53.175383+0000","last_became_active":"2024-09-10T21:24:46.860780+0000","last_became_peered":"2024-09-10T21:24:46.860780+0000","last_unstale":"2024-09-10T21:24:53.175383+0000","last_undegraded":"2024-09-10T21:24:53.175383+0000","last_fullsized":"2024-09-10T21:24:53.175383+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:53:31.454472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174596+0000","last_change":"2024-09-10T21:24:47.877997+0000","last_active":"2024-09-10T21:24:53.174596+0000","last_peered":"2024-09-10T21:24:53.174596+0000","last_clean":"2024-09-10T21:24:53.174596+0000","last_became_active":"2024-09-10T21:24:47.877765+0000","last_became_peered":"2024-09-10T21:24:47.877765+0000","last_unstale":"2024-09-10T21:24:53.174596+0000","last_undegraded":"2024-09-10T21:24:53.174596+0000","last_fullsized":"2024-09-10T21:24:53.174596+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:33:20.590209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.886528+0000","last_change":"2024-09-10T21:24:51.908333+0000","last_active":"2024-09-10T21:24:52.886528+0000","last_peered":"2024-09-10T21:24:52.886528+0000","last_clean":"2024-09-10T21:24:52.886528+0000","last_became_active":"2024-09-10T21:24:51.907933+0000","last_became_peered":"2024-09-10T21:24:51.907933+0000","last_unstale":"2024-09-10T21:24:52.886528+0000","last_undegraded":"2024-09-10T21:24:52.886528+0000","last_fullsized":"2024-09-10T21:24:52.886528+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:07:33.193555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"24'2","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174867+0000","last_change":"2024-09-10T21:24:46.873044+0000","last_active":"2024-09-10T21:24:53.174867+0000","last_peered":"2024-09-10T21:24:53.174867+0000","last_clean":"2024-09-10T21:24:53.174867+0000","last_became_active":"2024-09-10T21:24:46.872075+0000","last_became_peered":"2024-09-10T21:24:46.872075+0000","last_unstale":"2024-09-10T21:24:53.174867+0000","last_undegraded":"2024-09-10T21:24:53.174867+0000","last_fullsized":"2024-09-10T21:24:53.174867+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:40:33.632846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174402+0000","last_change":"2024-09-10T21:24:47.878094+0000","last_active":"2024-09-10T21:24:53.174402+0000","last_peered":"2024-09-10T21:24:53.174402+0000","last_clean":"2024-09-10T21:24:53.174402+0000","last_became_active":"2024-09-10T21:24:47.876688+0000","last_became_peered":"2024-09-10T21:24:47.876688+0000","last_unstale":"2024-09-10T21:24:53.174402+0000","last_undegraded":"2024-09-10T21:24:53.174402+0000","last_fullsized":"2024-09-10T21:24:53.174402+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:03:37.076080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.884328+0000","last_change":"2024-09-10T21:24:51.900208+0000","last_active":"2024-09-10T21:24:52.884328+0000","last_peered":"2024-09-10T21:24:52.884328+0000","last_clean":"2024-09-10T21:24:52.884328+0000","last_became_active":"2024-09-10T21:24:51.899576+0000","last_became_peered":"2024-09-10T21:24:51.899576+0000","last_unstale":"2024-09-10T21:24:52.884328+0000","last_undegraded":"2024-09-10T21:24:52.884328+0000","last_fullsized":"2024-09-10T21:24:52.884328+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:52:49.479932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.883746+0000","last_change":"2024-09-10T21:24:47.869830+0000","last_active":"2024-09-10T21:24:52.883746+0000","last_peered":"2024-09-10T21:24:52.883746+0000","last_clean":"2024-09-10T21:24:52.883746+0000","last_became_active":"2024-09-10T21:24:47.869569+0000","last_became_peered":"2024-09-10T21:24:47.869569+0000","last_unstale":"2024-09-10T21:24:52.883746+0000","last_undegraded":"2024-09-10T21:24:52.883746+0000","last_fullsized":"2024-09-10T21:24:52.883746+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:42:15.320518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"24'2","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174248+0000","last_change":"2024-09-10T21:24:46.867297+0000","last_active":"2024-09-10T21:24:53.174248+0000","last_peered":"2024-09-10T21:24:53.174248+0000","last_clean":"2024-09-10T21:24:53.174248+0000","last_became_active":"2024-09-10T21:24:46.866893+0000","last_became_peered":"2024-09-10T21:24:46.866893+0000","last_unstale":"2024-09-10T21:24:53.174248+0000","last_undegraded":"2024-09-10T21:24:53.174248+0000","last_fullsized":"2024-09-10T21:24:53.174248+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:53.197666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.885044+0000","last_change":"2024-09-10T21:24:51.908412+0000","last_active":"2024-09-10T21:24:52.885044+0000","last_peered":"2024-09-10T21:24:52.885044+0000","last_clean":"2024-09-10T21:24:52.885044+0000","last_became_active":"2024-09-10T21:24:51.907923+0000","last_became_peered":"2024-09-10T21:24:51.907923+0000","last_unstale":"2024-09-10T21:24:52.885044+0000","last_undegraded":"2024-09-10T21:24:52.885044+0000","last_fullsized":"2024-09-10T21:24:52.885044+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:26:46.479501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.884755+0000","last_change":"2024-09-10T21:24:47.862479+0000","last_active":"2024-09-10T21:24:52.884755+0000","last_peered":"2024-09-10T21:24:52.884755+0000","last_clean":"2024-09-10T21:24:52.884755+0000","last_became_active":"2024-09-10T21:24:47.861842+0000","last_became_peered":"2024-09-10T21:24:47.861842+0000","last_unstale":"2024-09-10T21:24:52.884755+0000","last_undegraded":"2024-09-10T21:24:52.884755+0000","last_fullsized":"2024-09-10T21:24:52.884755+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:53:09.082512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.878911+0000","last_change":"2024-09-10T21:24:46.852467+0000","last_active":"2024-09-10T21:24:52.878911+0000","last_peered":"2024-09-10T21:24:52.878911+0000","last_clean":"2024-09-10T21:24:52.878911+0000","last_became_active":"2024-09-10T21:24:46.851027+0000","last_became_peered":"2024-09-10T21:24:46.851027+0000","last_unstale":"2024-09-10T21:24:52.878911+0000","last_undegraded":"2024-09-10T21:24:52.878911+0000","last_fullsized":"2024-09-10T21:24:52.878911+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:39:27.441714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.886564+0000","last_change":"2024-09-10T21:24:51.903577+0000","last_active":"2024-09-10T21:24:52.886564+0000","last_peered":"2024-09-10T21:24:52.886564+0000","last_clean":"2024-09-10T21:24:52.886564+0000","last_became_active":"2024-09-10T21:24:51.903031+0000","last_became_peered":"2024-09-10T21:24:51.903031+0000","last_unstale":"2024-09-10T21:24:52.886564+0000","last_undegraded":"2024-09-10T21:24:52.886564+0000","last_fullsized":"2024-09-10T21:24:52.886564+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:08:52.787720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.884612+0000","last_change":"2024-09-10T21:24:47.856334+0000","last_active":"2024-09-10T21:24:52.884612+0000","last_peered":"2024-09-10T21:24:52.884612+0000","last_clean":"2024-09-10T21:24:52.884612+0000","last_became_active":"2024-09-10T21:24:47.856003+0000","last_became_peered":"2024-09-10T21:24:47.856003+0000","last_unstale":"2024-09-10T21:24:52.884612+0000","last_undegraded":"2024-09-10T21:24:52.884612+0000","last_fullsized":"2024-09-10T21:24:52.884612+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:58:53.698584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.878371+0000","last_change":"2024-09-10T21:24:46.853373+0000","last_active":"2024-09-10T21:24:52.878371+0000","last_peered":"2024-09-10T21:24:52.878371+0000","last_clean":"2024-09-10T21:24:52.878371+0000","last_became_active":"2024-09-10T21:24:46.852993+0000","last_became_peered":"2024-09-10T21:24:46.852993+0000","last_unstale":"2024-09-10T21:24:52.878371+0000","last_undegraded":"2024-09-10T21:24:52.878371+0000","last_fullsized":"2024-09-10T21:24:52.878371+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:53:31.454472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.879271+0000","last_change":"2024-09-10T21:24:52.065246+0000","last_active":"2024-09-10T21:24:52.879271+0000","last_peered":"2024-09-10T21:24:52.879271+0000","last_clean":"2024-09-10T21:24:52.879271+0000","last_became_active":"2024-09-10T21:24:52.064884+0000","last_became_peered":"2024-09-10T21:24:52.064884+0000","last_unstale":"2024-09-10T21:24:52.879271+0000","last_undegraded":"2024-09-10T21:24:52.879271+0000","last_fullsized":"2024-09-10T21:24:52.879271+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:54:54.393305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.884318+0000","last_change":"2024-09-10T21:24:47.856517+0000","last_active":"2024-09-10T21:24:52.884318+0000","last_peered":"2024-09-10T21:24:52.884318+0000","last_clean":"2024-09-10T21:24:52.884318+0000","last_became_active":"2024-09-10T21:24:47.856185+0000","last_became_peered":"2024-09-10T21:24:47.856185+0000","last_unstale":"2024-09-10T21:24:52.884318+0000","last_undegraded":"2024-09-10T21:24:52.884318+0000","last_fullsized":"2024-09-10T21:24:52.884318+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:26:00.852818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.173635+0000","last_change":"2024-09-10T21:24:46.867888+0000","last_active":"2024-09-10T21:24:53.173635+0000","last_peered":"2024-09-10T21:24:53.173635+0000","last_clean":"2024-09-10T21:24:53.173635+0000","last_became_active":"2024-09-10T21:24:46.866477+0000","last_became_peered":"2024-09-10T21:24:46.866477+0000","last_unstale":"2024-09-10T21:24:53.173635+0000","last_undegraded":"2024-09-10T21:24:53.173635+0000","last_fullsized":"2024-09-10T21:24:53.173635+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:39:27.441714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.885185+0000","last_change":"2024-09-10T21:24:51.908376+0000","last_active":"2024-09-10T21:24:52.885185+0000","last_peered":"2024-09-10T21:24:52.885185+0000","last_clean":"2024-09-10T21:24:52.885185+0000","last_became_active":"2024-09-10T21:24:51.907878+0000","last_became_peered":"2024-09-10T21:24:51.907878+0000","last_unstale":"2024-09-10T21:24:52.885185+0000","last_undegraded":"2024-09-10T21:24:52.885185+0000","last_fullsized":"2024-09-10T21:24:52.885185+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:39:02.492952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174342+0000","last_change":"2024-09-10T21:24:47.844823+0000","last_active":"2024-09-10T21:24:53.174342+0000","last_peered":"2024-09-10T21:24:53.174342+0000","last_clean":"2024-09-10T21:24:53.174342+0000","last_became_active":"2024-09-10T21:24:47.844505+0000","last_became_peered":"2024-09-10T21:24:47.844505+0000","last_unstale":"2024-09-10T21:24:53.174342+0000","last_undegraded":"2024-09-10T21:24:53.174342+0000","last_fullsized":"2024-09-10T21:24:53.174342+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:37:51.784258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"24'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.885358+0000","last_change":"2024-09-10T21:24:46.862172+0000","last_active":"2024-09-10T21:24:52.885358+0000","last_peered":"2024-09-10T21:24:52.885358+0000","last_clean":"2024-09-10T21:24:52.885358+0000","last_became_active":"2024-09-10T21:24:46.861473+0000","last_became_peered":"2024-09-10T21:24:46.861473+0000","last_unstale":"2024-09-10T21:24:52.885358+0000","last_undegraded":"2024-09-10T21:24:52.885358+0000","last_fullsized":"2024-09-10T21:24:52.885358+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:53:31.454472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.883945+0000","last_change":"2024-09-10T21:24:52.062841+0000","last_active":"2024-09-10T21:24:52.883945+0000","last_peered":"2024-09-10T21:24:52.883945+0000","last_clean":"2024-09-10T21:24:52.883945+0000","last_became_active":"2024-09-10T21:24:52.061581+0000","last_became_peered":"2024-09-10T21:24:52.061581+0000","last_unstale":"2024-09-10T21:24:52.883945+0000","last_undegraded":"2024-09-10T21:24:52.883945+0000","last_fullsized":"2024-09-10T21:24:52.883945+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:34:03.325781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.885543+0000","last_change":"2024-09-10T21:24:47.859801+0000","last_active":"2024-09-10T21:24:52.885543+0000","last_peered":"2024-09-10T21:24:52.885543+0000","last_clean":"2024-09-10T21:24:52.885543+0000","last_became_active":"2024-09-10T21:24:47.859483+0000","last_became_peered":"2024-09-10T21:24:47.859483+0000","last_unstale":"2024-09-10T21:24:52.885543+0000","last_undegraded":"2024-09-10T21:24:52.885543+0000","last_fullsized":"2024-09-10T21:24:52.885543+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:38:21.368499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"25'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.879175+0000","last_change":"2024-09-10T21:24:46.860624+0000","last_active":"2024-09-10T21:24:52.879175+0000","last_peered":"2024-09-10T21:24:52.879175+0000","last_clean":"2024-09-10T21:24:52.879175+0000","last_became_active":"2024-09-10T21:24:46.860309+0000","last_became_peered":"2024-09-10T21:24:46.860309+0000","last_unstale":"2024-09-10T21:24:52.879175+0000","last_undegraded":"2024-09-10T21:24:52.879175+0000","last_fullsized":"2024-09-10T21:24:52.879175+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:37:12.332013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.884414+0000","last_change":"2024-09-10T21:24:52.068581+0000","last_active":"2024-09-10T21:24:52.884414+0000","last_peered":"2024-09-10T21:24:52.884414+0000","last_clean":"2024-09-10T21:24:52.884414+0000","last_became_active":"2024-09-10T21:24:52.068250+0000","last_became_peered":"2024-09-10T21:24:52.068250+0000","last_unstale":"2024-09-10T21:24:52.884414+0000","last_undegraded":"2024-09-10T21:24:52.884414+0000","last_fullsized":"2024-09-10T21:24:52.884414+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:52:01.990009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.879000+0000","last_change":"2024-09-10T21:24:47.854042+0000","last_active":"2024-09-10T21:24:52.879000+0000","last_peered":"2024-09-10T21:24:52.879000+0000","last_clean":"2024-09-10T21:24:52.879000+0000","last_became_active":"2024-09-10T21:24:47.853750+0000","last_became_peered":"2024-09-10T21:24:47.853750+0000","last_unstale":"2024-09-10T21:24:52.879000+0000","last_undegraded":"2024-09-10T21:24:52.879000+0000","last_fullsized":"2024-09-10T21:24:52.879000+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:06:50.775132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.175510+0000","last_change":"2024-09-10T21:24:46.864617+0000","last_active":"2024-09-10T21:24:53.175510+0000","last_peered":"2024-09-10T21:24:53.175510+0000","last_clean":"2024-09-10T21:24:53.175510+0000","last_became_active":"2024-09-10T21:24:46.864149+0000","last_became_peered":"2024-09-10T21:24:46.864149+0000","last_unstale":"2024-09-10T21:24:53.175510+0000","last_undegraded":"2024-09-10T21:24:53.175510+0000","last_fullsized":"2024-09-10T21:24:53.175510+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:24:15.673107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174530+0000","last_change":"2024-09-10T21:24:52.061485+0000","last_active":"2024-09-10T21:24:53.174530+0000","last_peered":"2024-09-10T21:24:53.174530+0000","last_clean":"2024-09-10T21:24:53.174530+0000","last_became_active":"2024-09-10T21:24:52.060953+0000","last_became_peered":"2024-09-10T21:24:52.060953+0000","last_unstale":"2024-09-10T21:24:53.174530+0000","last_undegraded":"2024-09-10T21:24:53.174530+0000","last_fullsized":"2024-09-10T21:24:53.174530+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:49:30.147202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174576+0000","last_change":"2024-09-10T21:24:47.867241+0000","last_active":"2024-09-10T21:24:53.174576+0000","last_peered":"2024-09-10T21:24:53.174576+0000","last_clean":"2024-09-10T21:24:53.174576+0000","last_became_active":"2024-09-10T21:24:47.866907+0000","last_became_peered":"2024-09-10T21:24:47.866907+0000","last_unstale":"2024-09-10T21:24:53.174576+0000","last_undegraded":"2024-09-10T21:24:53.174576+0000","last_fullsized":"2024-09-10T21:24:53.174576+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:07.620800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.879175+0000","last_change":"2024-09-10T21:24:46.856848+0000","last_active":"2024-09-10T21:24:52.879175+0000","last_peered":"2024-09-10T21:24:52.879175+0000","last_clean":"2024-09-10T21:24:52.879175+0000","last_became_active":"2024-09-10T21:24:46.856489+0000","last_became_peered":"2024-09-10T21:24:46.856489+0000","last_unstale":"2024-09-10T21:24:52.879175+0000","last_undegraded":"2024-09-10T21:24:52.879175+0000","last_fullsized":"2024-09-10T21:24:52.879175+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:46:55.286321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.878563+0000","last_change":"2024-09-10T21:24:51.901016+0000","last_active":"2024-09-10T21:24:52.878563+0000","last_peered":"2024-09-10T21:24:52.878563+0000","last_clean":"2024-09-10T21:24:52.878563+0000","last_became_active":"2024-09-10T21:24:51.900479+0000","last_became_peered":"2024-09-10T21:24:51.900479+0000","last_unstale":"2024-09-10T21:24:52.878563+0000","last_undegraded":"2024-09-10T21:24:52.878563+0000","last_fullsized":"2024-09-10T21:24:52.878563+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:36.741810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.886797+0000","last_change":"2024-09-10T21:24:47.858753+0000","last_active":"2024-09-10T21:24:52.886797+0000","last_peered":"2024-09-10T21:24:52.886797+0000","last_clean":"2024-09-10T21:24:52.886797+0000","last_became_active":"2024-09-10T21:24:47.858438+0000","last_became_peered":"2024-09-10T21:24:47.858438+0000","last_unstale":"2024-09-10T21:24:52.886797+0000","last_undegraded":"2024-09-10T21:24:52.886797+0000","last_fullsized":"2024-09-10T21:24:52.886797+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:12:09.198013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.175120+0000","last_change":"2024-09-10T21:24:46.863610+0000","last_active":"2024-09-10T21:24:53.175120+0000","last_peered":"2024-09-10T21:24:53.175120+0000","last_clean":"2024-09-10T21:24:53.175120+0000","last_became_active":"2024-09-10T21:24:46.863150+0000","last_became_peered":"2024-09-10T21:24:46.863150+0000","last_unstale":"2024-09-10T21:24:53.175120+0000","last_undegraded":"2024-09-10T21:24:53.175120+0000","last_fullsized":"2024-09-10T21:24:53.175120+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:27:35.473995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.878912+0000","last_change":"2024-09-10T21:24:51.907366+0000","last_active":"2024-09-10T21:24:52.878912+0000","last_peered":"2024-09-10T21:24:52.878912+0000","last_clean":"2024-09-10T21:24:52.878912+0000","last_became_active":"2024-09-10T21:24:51.906888+0000","last_became_peered":"2024-09-10T21:24:51.906888+0000","last_unstale":"2024-09-10T21:24:52.878912+0000","last_undegraded":"2024-09-10T21:24:52.878912+0000","last_fullsized":"2024-09-10T21:24:52.878912+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:24:54.278514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.878445+0000","last_change":"2024-09-10T21:24:47.850155+0000","last_active":"2024-09-10T21:24:52.878445+0000","last_peered":"2024-09-10T21:24:52.878445+0000","last_clean":"2024-09-10T21:24:52.878445+0000","last_became_active":"2024-09-10T21:24:47.849846+0000","last_became_peered":"2024-09-10T21:24:47.849846+0000","last_unstale":"2024-09-10T21:24:52.878445+0000","last_undegraded":"2024-09-10T21:24:52.878445+0000","last_fullsized":"2024-09-10T21:24:52.878445+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:13:40.032808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.173931+0000","last_change":"2024-09-10T21:24:46.851800+0000","last_active":"2024-09-10T21:24:53.173931+0000","last_peered":"2024-09-10T21:24:53.173931+0000","last_clean":"2024-09-10T21:24:53.173931+0000","last_became_active":"2024-09-10T21:24:46.851200+0000","last_became_peered":"2024-09-10T21:24:46.851200+0000","last_unstale":"2024-09-10T21:24:53.173931+0000","last_undegraded":"2024-09-10T21:24:53.173931+0000","last_fullsized":"2024-09-10T21:24:53.173931+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:00:22.839361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.879296+0000","last_change":"2024-09-10T21:24:51.902529+0000","last_active":"2024-09-10T21:24:52.879296+0000","last_peered":"2024-09-10T21:24:52.879296+0000","last_clean":"2024-09-10T21:24:52.879296+0000","last_became_active":"2024-09-10T21:24:51.901141+0000","last_became_peered":"2024-09-10T21:24:51.901141+0000","last_unstale":"2024-09-10T21:24:52.879296+0000","last_undegraded":"2024-09-10T21:24:52.879296+0000","last_fullsized":"2024-09-10T21:24:52.879296+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:08:40.203348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.173878+0000","last_change":"2024-09-10T21:24:47.846582+0000","last_active":"2024-09-10T21:24:53.173878+0000","last_peered":"2024-09-10T21:24:53.173878+0000","last_clean":"2024-09-10T21:24:53.173878+0000","last_became_active":"2024-09-10T21:24:47.846358+0000","last_became_peered":"2024-09-10T21:24:47.846358+0000","last_unstale":"2024-09-10T21:24:53.173878+0000","last_undegraded":"2024-09-10T21:24:53.173878+0000","last_fullsized":"2024-09-10T21:24:53.173878+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:58:53.698584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.173773+0000","last_change":"2024-09-10T21:24:46.870908+0000","last_active":"2024-09-10T21:24:53.173773+0000","last_peered":"2024-09-10T21:24:53.173773+0000","last_clean":"2024-09-10T21:24:53.173773+0000","last_became_active":"2024-09-10T21:24:46.868375+0000","last_became_peered":"2024-09-10T21:24:46.868375+0000","last_unstale":"2024-09-10T21:24:53.173773+0000","last_undegraded":"2024-09-10T21:24:53.173773+0000","last_fullsized":"2024-09-10T21:24:53.173773+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:37:12.332013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174362+0000","last_change":"2024-09-10T21:24:52.069018+0000","last_active":"2024-09-10T21:24:53.174362+0000","last_peered":"2024-09-10T21:24:53.174362+0000","last_clean":"2024-09-10T21:24:53.174362+0000","last_became_active":"2024-09-10T21:24:52.067308+0000","last_became_peered":"2024-09-10T21:24:52.067308+0000","last_unstale":"2024-09-10T21:24:53.174362+0000","last_undegraded":"2024-09-10T21:24:53.174362+0000","last_fullsized":"2024-09-10T21:24:53.174362+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:07:25.162168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.884767+0000","last_change":"2024-09-10T21:24:47.871542+0000","last_active":"2024-09-10T21:24:52.884767+0000","last_peered":"2024-09-10T21:24:52.884767+0000","last_clean":"2024-09-10T21:24:52.884767+0000","last_became_active":"2024-09-10T21:24:47.871128+0000","last_became_peered":"2024-09-10T21:24:47.871128+0000","last_unstale":"2024-09-10T21:24:52.884767+0000","last_undegraded":"2024-09-10T21:24:52.884767+0000","last_fullsized":"2024-09-10T21:24:52.884767+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:51:22.876126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174204+0000","last_change":"2024-09-10T21:24:46.859218+0000","last_active":"2024-09-10T21:24:53.174204+0000","last_peered":"2024-09-10T21:24:53.174204+0000","last_clean":"2024-09-10T21:24:53.174204+0000","last_became_active":"2024-09-10T21:24:46.853190+0000","last_became_peered":"2024-09-10T21:24:46.853190+0000","last_unstale":"2024-09-10T21:24:53.174204+0000","last_undegraded":"2024-09-10T21:24:53.174204+0000","last_fullsized":"2024-09-10T21:24:53.174204+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:26:34.848759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":119,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.173768+0000","last_change":"2024-09-10T21:24:37.216542+0000","last_active":"2024-09-10T21:24:53.173768+0000","last_peered":"2024-09-10T21:24:53.173768+0000","last_clean":"2024-09-10T21:24:53.173768+0000","last_became_active":"2024-09-10T21:24:37.215443+0000","last_became_peered":"2024-09-10T21:24:37.215443+0000","last_unstale":"2024-09-10T21:24:53.173768+0000","last_undegraded":"2024-09-10T21:24:53.173768+0000","last_fullsized":"2024-09-10T21:24:53.173768+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:35.877801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:35.877801+0000","last_clean_scrub_stamp":"2024-09-10T21:24:35.877801+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:14:20.260329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.884231+0000","last_change":"2024-09-10T21:24:52.067645+0000","last_active":"2024-09-10T21:24:52.884231+0000","last_peered":"2024-09-10T21:24:52.884231+0000","last_clean":"2024-09-10T21:24:52.884231+0000","last_became_active":"2024-09-10T21:24:52.067148+0000","last_became_peered":"2024-09-10T21:24:52.067148+0000","last_unstale":"2024-09-10T21:24:52.884231+0000","last_undegraded":"2024-09-10T21:24:52.884231+0000","last_fullsized":"2024-09-10T21:24:52.884231+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:54:42.435011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.175128+0000","last_change":"2024-09-10T21:24:47.867282+0000","last_active":"2024-09-10T21:24:53.175128+0000","last_peered":"2024-09-10T21:24:53.175128+0000","last_clean":"2024-09-10T21:24:53.175128+0000","last_became_active":"2024-09-10T21:24:47.865582+0000","last_became_peered":"2024-09-10T21:24:47.865582+0000","last_unstale":"2024-09-10T21:24:53.175128+0000","last_undegraded":"2024-09-10T21:24:53.175128+0000","last_fullsized":"2024-09-10T21:24:53.175128+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:38:55.009670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174808+0000","last_change":"2024-09-10T21:24:46.859532+0000","last_active":"2024-09-10T21:24:53.174808+0000","last_peered":"2024-09-10T21:24:53.174808+0000","last_clean":"2024-09-10T21:24:53.174808+0000","last_became_active":"2024-09-10T21:24:46.859144+0000","last_became_peered":"2024-09-10T21:24:46.859144+0000","last_unstale":"2024-09-10T21:24:53.174808+0000","last_undegraded":"2024-09-10T21:24:53.174808+0000","last_fullsized":"2024-09-10T21:24:53.174808+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:48:41.992441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.883627+0000","last_change":"2024-09-10T21:24:52.061029+0000","last_active":"2024-09-10T21:24:52.883627+0000","last_peered":"2024-09-10T21:24:52.883627+0000","last_clean":"2024-09-10T21:24:52.883627+0000","last_became_active":"2024-09-10T21:24:52.060605+0000","last_became_peered":"2024-09-10T21:24:52.060605+0000","last_unstale":"2024-09-10T21:24:52.883627+0000","last_undegraded":"2024-09-10T21:24:52.883627+0000","last_fullsized":"2024-09-10T21:24:52.883627+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:09:48.077266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174744+0000","last_change":"2024-09-10T21:24:47.866927+0000","last_active":"2024-09-10T21:24:53.174744+0000","last_peered":"2024-09-10T21:24:53.174744+0000","last_clean":"2024-09-10T21:24:53.174744+0000","last_became_active":"2024-09-10T21:24:47.866650+0000","last_became_peered":"2024-09-10T21:24:47.866650+0000","last_unstale":"2024-09-10T21:24:53.174744+0000","last_undegraded":"2024-09-10T21:24:53.174744+0000","last_fullsized":"2024-09-10T21:24:53.174744+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:47:37.304624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"24'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.879000+0000","last_change":"2024-09-10T21:24:46.857082+0000","last_active":"2024-09-10T21:24:52.879000+0000","last_peered":"2024-09-10T21:24:52.879000+0000","last_clean":"2024-09-10T21:24:52.879000+0000","last_became_active":"2024-09-10T21:24:46.856799+0000","last_became_peered":"2024-09-10T21:24:46.856799+0000","last_unstale":"2024-09-10T21:24:52.879000+0000","last_undegraded":"2024-09-10T21:24:52.879000+0000","last_fullsized":"2024-09-10T21:24:52.879000+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:40:33.632846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174743+0000","last_change":"2024-09-10T21:24:51.911880+0000","last_active":"2024-09-10T21:24:53.174743+0000","last_peered":"2024-09-10T21:24:53.174743+0000","last_clean":"2024-09-10T21:24:53.174743+0000","last_became_active":"2024-09-10T21:24:51.911088+0000","last_became_peered":"2024-09-10T21:24:51.911088+0000","last_unstale":"2024-09-10T21:24:53.174743+0000","last_undegraded":"2024-09-10T21:24:53.174743+0000","last_fullsized":"2024-09-10T21:24:53.174743+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:38:57.030655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.175370+0000","last_change":"2024-09-10T21:24:47.876946+0000","last_active":"2024-09-10T21:24:53.175370+0000","last_peered":"2024-09-10T21:24:53.175370+0000","last_clean":"2024-09-10T21:24:53.175370+0000","last_became_active":"2024-09-10T21:24:47.876690+0000","last_became_peered":"2024-09-10T21:24:47.876690+0000","last_unstale":"2024-09-10T21:24:53.175370+0000","last_undegraded":"2024-09-10T21:24:53.175370+0000","last_fullsized":"2024-09-10T21:24:53.175370+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:14:06.368275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"24'2","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.175105+0000","last_change":"2024-09-10T21:24:46.855248+0000","last_active":"2024-09-10T21:24:53.175105+0000","last_peered":"2024-09-10T21:24:53.175105+0000","last_clean":"2024-09-10T21:24:53.175105+0000","last_became_active":"2024-09-10T21:24:46.854930+0000","last_became_peered":"2024-09-10T21:24:46.854930+0000","last_unstale":"2024-09-10T21:24:53.175105+0000","last_undegraded":"2024-09-10T21:24:53.175105+0000","last_fullsized":"2024-09-10T21:24:53.175105+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:46:55.286321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.173528+0000","last_change":"2024-09-10T21:24:52.063112+0000","last_active":"2024-09-10T21:24:53.173528+0000","last_peered":"2024-09-10T21:24:53.173528+0000","last_clean":"2024-09-10T21:24:53.173528+0000","last_became_active":"2024-09-10T21:24:52.062395+0000","last_became_peered":"2024-09-10T21:24:52.062395+0000","last_unstale":"2024-09-10T21:24:53.173528+0000","last_undegraded":"2024-09-10T21:24:53.173528+0000","last_fullsized":"2024-09-10T21:24:53.173528+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:21:53.080013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.175254+0000","last_change":"2024-09-10T21:24:47.858593+0000","last_active":"2024-09-10T21:24:53.175254+0000","last_peered":"2024-09-10T21:24:53.175254+0000","last_clean":"2024-09-10T21:24:53.175254+0000","last_became_active":"2024-09-10T21:24:47.858216+0000","last_became_peered":"2024-09-10T21:24:47.858216+0000","last_unstale":"2024-09-10T21:24:53.175254+0000","last_undegraded":"2024-09-10T21:24:53.175254+0000","last_fullsized":"2024-09-10T21:24:53.175254+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:42.961291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"24'1","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.886950+0000","last_change":"2024-09-10T21:24:46.860330+0000","last_active":"2024-09-10T21:24:52.886950+0000","last_peered":"2024-09-10T21:24:52.886950+0000","last_clean":"2024-09-10T21:24:52.886950+0000","last_became_active":"2024-09-10T21:24:46.860053+0000","last_became_peered":"2024-09-10T21:24:46.860053+0000","last_unstale":"2024-09-10T21:24:52.886950+0000","last_undegraded":"2024-09-10T21:24:52.886950+0000","last_fullsized":"2024-09-10T21:24:52.886950+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:58:53.177740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.883701+0000","last_change":"2024-09-10T21:24:51.901673+0000","last_active":"2024-09-10T21:24:52.883701+0000","last_peered":"2024-09-10T21:24:52.883701+0000","last_clean":"2024-09-10T21:24:52.883701+0000","last_became_active":"2024-09-10T21:24:51.901222+0000","last_became_peered":"2024-09-10T21:24:51.901222+0000","last_unstale":"2024-09-10T21:24:52.883701+0000","last_undegraded":"2024-09-10T21:24:52.883701+0000","last_fullsized":"2024-09-10T21:24:52.883701+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:07:52.120299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.173790+0000","last_change":"2024-09-10T21:24:47.870589+0000","last_active":"2024-09-10T21:24:53.173790+0000","last_peered":"2024-09-10T21:24:53.173790+0000","last_clean":"2024-09-10T21:24:53.173790+0000","last_became_active":"2024-09-10T21:24:47.861834+0000","last_became_peered":"2024-09-10T21:24:47.861834+0000","last_unstale":"2024-09-10T21:24:53.173790+0000","last_undegraded":"2024-09-10T21:24:53.173790+0000","last_fullsized":"2024-09-10T21:24:53.173790+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:43.363013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"24'8","reported_seq":31,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.173490+0000","last_change":"2024-09-10T21:24:46.852064+0000","last_active":"2024-09-10T21:24:53.173490+0000","last_peered":"2024-09-10T21:24:53.173490+0000","last_clean":"2024-09-10T21:24:53.173490+0000","last_became_active":"2024-09-10T21:24:46.851776+0000","last_became_peered":"2024-09-10T21:24:46.851776+0000","last_unstale":"2024-09-10T21:24:53.173490+0000","last_undegraded":"2024-09-10T21:24:53.173490+0000","last_fullsized":"2024-09-10T21:24:53.173490+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:26:24.371844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174153+0000","last_change":"2024-09-10T21:24:51.904449+0000","last_active":"2024-09-10T21:24:53.174153+0000","last_peered":"2024-09-10T21:24:53.174153+0000","last_clean":"2024-09-10T21:24:53.174153+0000","last_became_active":"2024-09-10T21:24:51.903533+0000","last_became_peered":"2024-09-10T21:24:51.903533+0000","last_unstale":"2024-09-10T21:24:53.174153+0000","last_undegraded":"2024-09-10T21:24:53.174153+0000","last_fullsized":"2024-09-10T21:24:53.174153+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:11.654935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.883479+0000","last_change":"2024-09-10T21:24:47.857736+0000","last_active":"2024-09-10T21:24:52.883479+0000","last_peered":"2024-09-10T21:24:52.883479+0000","last_clean":"2024-09-10T21:24:52.883479+0000","last_became_active":"2024-09-10T21:24:47.857437+0000","last_became_peered":"2024-09-10T21:24:47.857437+0000","last_unstale":"2024-09-10T21:24:52.883479+0000","last_undegraded":"2024-09-10T21:24:52.883479+0000","last_fullsized":"2024-09-10T21:24:52.883479+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:13:40.032808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.883979+0000","last_change":"2024-09-10T21:24:46.860662+0000","last_active":"2024-09-10T21:24:52.883979+0000","last_peered":"2024-09-10T21:24:52.883979+0000","last_clean":"2024-09-10T21:24:52.883979+0000","last_became_active":"2024-09-10T21:24:46.859796+0000","last_became_peered":"2024-09-10T21:24:46.859796+0000","last_unstale":"2024-09-10T21:24:52.883979+0000","last_undegraded":"2024-09-10T21:24:52.883979+0000","last_fullsized":"2024-09-10T21:24:52.883979+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:48:27.758481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.885543+0000","last_change":"2024-09-10T21:24:52.069770+0000","last_active":"2024-09-10T21:24:52.885543+0000","last_peered":"2024-09-10T21:24:52.885543+0000","last_clean":"2024-09-10T21:24:52.885543+0000","last_became_active":"2024-09-10T21:24:52.067409+0000","last_became_peered":"2024-09-10T21:24:52.067409+0000","last_unstale":"2024-09-10T21:24:52.885543+0000","last_undegraded":"2024-09-10T21:24:52.885543+0000","last_fullsized":"2024-09-10T21:24:52.885543+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:23:39.622612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174136+0000","last_change":"2024-09-10T21:24:47.858692+0000","last_active":"2024-09-10T21:24:53.174136+0000","last_peered":"2024-09-10T21:24:53.174136+0000","last_clean":"2024-09-10T21:24:53.174136+0000","last_became_active":"2024-09-10T21:24:47.858343+0000","last_became_peered":"2024-09-10T21:24:47.858343+0000","last_unstale":"2024-09-10T21:24:53.174136+0000","last_undegraded":"2024-09-10T21:24:53.174136+0000","last_fullsized":"2024-09-10T21:24:53.174136+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:04:36.661791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.175219+0000","last_change":"2024-09-10T21:24:46.864592+0000","last_active":"2024-09-10T21:24:53.175219+0000","last_peered":"2024-09-10T21:24:53.175219+0000","last_clean":"2024-09-10T21:24:53.175219+0000","last_became_active":"2024-09-10T21:24:46.858696+0000","last_became_peered":"2024-09-10T21:24:46.858696+0000","last_unstale":"2024-09-10T21:24:53.175219+0000","last_undegraded":"2024-09-10T21:24:53.175219+0000","last_fullsized":"2024-09-10T21:24:53.175219+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:54:19.356867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.885074+0000","last_change":"2024-09-10T21:24:52.061572+0000","last_active":"2024-09-10T21:24:52.885074+0000","last_peered":"2024-09-10T21:24:52.885074+0000","last_clean":"2024-09-10T21:24:52.885074+0000","last_became_active":"2024-09-10T21:24:52.061061+0000","last_became_peered":"2024-09-10T21:24:52.061061+0000","last_unstale":"2024-09-10T21:24:52.885074+0000","last_undegraded":"2024-09-10T21:24:52.885074+0000","last_fullsized":"2024-09-10T21:24:52.885074+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:04:36.399811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174340+0000","last_change":"2024-09-10T21:24:47.857699+0000","last_active":"2024-09-10T21:24:53.174340+0000","last_peered":"2024-09-10T21:24:53.174340+0000","last_clean":"2024-09-10T21:24:53.174340+0000","last_became_active":"2024-09-10T21:24:47.857429+0000","last_became_peered":"2024-09-10T21:24:47.857429+0000","last_unstale":"2024-09-10T21:24:53.174340+0000","last_undegraded":"2024-09-10T21:24:53.174340+0000","last_fullsized":"2024-09-10T21:24:53.174340+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:07:11.880228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"24'2","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174839+0000","last_change":"2024-09-10T21:24:46.859753+0000","last_active":"2024-09-10T21:24:53.174839+0000","last_peered":"2024-09-10T21:24:53.174839+0000","last_clean":"2024-09-10T21:24:53.174839+0000","last_became_active":"2024-09-10T21:24:46.859299+0000","last_became_peered":"2024-09-10T21:24:46.859299+0000","last_unstale":"2024-09-10T21:24:53.174839+0000","last_undegraded":"2024-09-10T21:24:53.174839+0000","last_fullsized":"2024-09-10T21:24:53.174839+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:06:41.965052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174673+0000","last_change":"2024-09-10T21:24:52.064718+0000","last_active":"2024-09-10T21:24:53.174673+0000","last_peered":"2024-09-10T21:24:53.174673+0000","last_clean":"2024-09-10T21:24:53.174673+0000","last_became_active":"2024-09-10T21:24:52.062327+0000","last_became_peered":"2024-09-10T21:24:52.062327+0000","last_unstale":"2024-09-10T21:24:53.174673+0000","last_undegraded":"2024-09-10T21:24:53.174673+0000","last_fullsized":"2024-09-10T21:24:53.174673+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:43:17.173095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.175009+0000","last_change":"2024-09-10T21:24:47.870766+0000","last_active":"2024-09-10T21:24:53.175009+0000","last_peered":"2024-09-10T21:24:53.175009+0000","last_clean":"2024-09-10T21:24:53.175009+0000","last_became_active":"2024-09-10T21:24:47.870276+0000","last_became_peered":"2024-09-10T21:24:47.870276+0000","last_unstale":"2024-09-10T21:24:53.175009+0000","last_undegraded":"2024-09-10T21:24:53.175009+0000","last_fullsized":"2024-09-10T21:24:53.175009+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:54:38.400877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"25'9","reported_seq":31,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:25:05.421191+0000","last_change":"2024-09-10T21:24:46.867790+0000","last_active":"2024-09-10T21:25:05.421191+0000","last_peered":"2024-09-10T21:25:05.421191+0000","last_clean":"2024-09-10T21:25:05.421191+0000","last_became_active":"2024-09-10T21:24:46.866899+0000","last_became_peered":"2024-09-10T21:24:46.866899+0000","last_unstale":"2024-09-10T21:25:05.421191+0000","last_undegraded":"2024-09-10T21:25:05.421191+0000","last_fullsized":"2024-09-10T21:25:05.421191+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:25:04.923730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.878572+0000","last_change":"2024-09-10T21:24:51.899803+0000","last_active":"2024-09-10T21:24:52.878572+0000","last_peered":"2024-09-10T21:24:52.878572+0000","last_clean":"2024-09-10T21:24:52.878572+0000","last_became_active":"2024-09-10T21:24:51.895398+0000","last_became_peered":"2024-09-10T21:24:51.895398+0000","last_unstale":"2024-09-10T21:24:52.878572+0000","last_undegraded":"2024-09-10T21:24:52.878572+0000","last_fullsized":"2024-09-10T21:24:52.878572+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:52:34.689860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.886661+0000","last_change":"2024-09-10T21:24:47.857544+0000","last_active":"2024-09-10T21:24:52.886661+0000","last_peered":"2024-09-10T21:24:52.886661+0000","last_clean":"2024-09-10T21:24:52.886661+0000","last_became_active":"2024-09-10T21:24:47.857284+0000","last_became_peered":"2024-09-10T21:24:47.857284+0000","last_unstale":"2024-09-10T21:24:52.886661+0000","last_undegraded":"2024-09-10T21:24:52.886661+0000","last_fullsized":"2024-09-10T21:24:52.886661+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:10:22.747606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"24'3","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.878942+0000","last_change":"2024-09-10T21:24:46.859633+0000","last_active":"2024-09-10T21:24:52.878942+0000","last_peered":"2024-09-10T21:24:52.878942+0000","last_clean":"2024-09-10T21:24:52.878942+0000","last_became_active":"2024-09-10T21:24:46.859332+0000","last_became_peered":"2024-09-10T21:24:46.859332+0000","last_unstale":"2024-09-10T21:24:52.878942+0000","last_undegraded":"2024-09-10T21:24:52.878942+0000","last_fullsized":"2024-09-10T21:24:52.878942+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:56:46.975225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.886734+0000","last_change":"2024-09-10T21:24:51.899100+0000","last_active":"2024-09-10T21:24:52.886734+0000","last_peered":"2024-09-10T21:24:52.886734+0000","last_clean":"2024-09-10T21:24:52.886734+0000","last_became_active":"2024-09-10T21:24:51.898642+0000","last_became_peered":"2024-09-10T21:24:51.898642+0000","last_unstale":"2024-09-10T21:24:52.886734+0000","last_undegraded":"2024-09-10T21:24:52.886734+0000","last_fullsized":"2024-09-10T21:24:52.886734+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:06.394081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174291+0000","last_change":"2024-09-10T21:24:47.867890+0000","last_active":"2024-09-10T21:24:53.174291+0000","last_peered":"2024-09-10T21:24:53.174291+0000","last_clean":"2024-09-10T21:24:53.174291+0000","last_became_active":"2024-09-10T21:24:47.865016+0000","last_became_peered":"2024-09-10T21:24:47.865016+0000","last_unstale":"2024-09-10T21:24:53.174291+0000","last_undegraded":"2024-09-10T21:24:53.174291+0000","last_fullsized":"2024-09-10T21:24:53.174291+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:53:09.082512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174673+0000","last_change":"2024-09-10T21:24:46.861400+0000","last_active":"2024-09-10T21:24:53.174673+0000","last_peered":"2024-09-10T21:24:53.174673+0000","last_clean":"2024-09-10T21:24:53.174673+0000","last_became_active":"2024-09-10T21:24:46.861136+0000","last_became_peered":"2024-09-10T21:24:46.861136+0000","last_unstale":"2024-09-10T21:24:53.174673+0000","last_undegraded":"2024-09-10T21:24:53.174673+0000","last_fullsized":"2024-09-10T21:24:53.174673+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:35:31.682448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.884671+0000","last_change":"2024-09-10T21:24:51.907850+0000","last_active":"2024-09-10T21:24:52.884671+0000","last_peered":"2024-09-10T21:24:52.884671+0000","last_clean":"2024-09-10T21:24:52.884671+0000","last_became_active":"2024-09-10T21:24:51.907062+0000","last_became_peered":"2024-09-10T21:24:51.907062+0000","last_unstale":"2024-09-10T21:24:52.884671+0000","last_undegraded":"2024-09-10T21:24:52.884671+0000","last_fullsized":"2024-09-10T21:24:52.884671+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:32:52.519627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174521+0000","last_change":"2024-09-10T21:24:47.861800+0000","last_active":"2024-09-10T21:24:53.174521+0000","last_peered":"2024-09-10T21:24:53.174521+0000","last_clean":"2024-09-10T21:24:53.174521+0000","last_became_active":"2024-09-10T21:24:47.861498+0000","last_became_peered":"2024-09-10T21:24:47.861498+0000","last_unstale":"2024-09-10T21:24:53.174521+0000","last_undegraded":"2024-09-10T21:24:53.174521+0000","last_fullsized":"2024-09-10T21:24:53.174521+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:07:21.128033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.885153+0000","last_change":"2024-09-10T21:24:46.871677+0000","last_active":"2024-09-10T21:24:52.885153+0000","last_peered":"2024-09-10T21:24:52.885153+0000","last_clean":"2024-09-10T21:24:52.885153+0000","last_became_active":"2024-09-10T21:24:46.871367+0000","last_became_peered":"2024-09-10T21:24:46.871367+0000","last_unstale":"2024-09-10T21:24:52.885153+0000","last_undegraded":"2024-09-10T21:24:52.885153+0000","last_fullsized":"2024-09-10T21:24:52.885153+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:40:33.632846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.884646+0000","last_change":"2024-09-10T21:24:52.068637+0000","last_active":"2024-09-10T21:24:52.884646+0000","last_peered":"2024-09-10T21:24:52.884646+0000","last_clean":"2024-09-10T21:24:52.884646+0000","last_became_active":"2024-09-10T21:24:52.068348+0000","last_became_peered":"2024-09-10T21:24:52.068348+0000","last_unstale":"2024-09-10T21:24:52.884646+0000","last_undegraded":"2024-09-10T21:24:52.884646+0000","last_fullsized":"2024-09-10T21:24:52.884646+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:43:17.173095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174107+0000","last_change":"2024-09-10T21:24:52.067732+0000","last_active":"2024-09-10T21:24:53.174107+0000","last_peered":"2024-09-10T21:24:53.174107+0000","last_clean":"2024-09-10T21:24:53.174107+0000","last_became_active":"2024-09-10T21:24:52.067330+0000","last_became_peered":"2024-09-10T21:24:52.067330+0000","last_unstale":"2024-09-10T21:24:53.174107+0000","last_undegraded":"2024-09-10T21:24:53.174107+0000","last_fullsized":"2024-09-10T21:24:53.174107+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:03:41.110214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.878246+0000","last_change":"2024-09-10T21:24:47.862774+0000","last_active":"2024-09-10T21:24:52.878246+0000","last_peered":"2024-09-10T21:24:52.878246+0000","last_clean":"2024-09-10T21:24:52.878246+0000","last_became_active":"2024-09-10T21:24:47.862556+0000","last_became_peered":"2024-09-10T21:24:47.862556+0000","last_unstale":"2024-09-10T21:24:52.878246+0000","last_undegraded":"2024-09-10T21:24:52.878246+0000","last_fullsized":"2024-09-10T21:24:52.878246+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:17:52.343553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.173610+0000","last_change":"2024-09-10T21:24:46.862847+0000","last_active":"2024-09-10T21:24:53.173610+0000","last_peered":"2024-09-10T21:24:53.173610+0000","last_clean":"2024-09-10T21:24:53.173610+0000","last_became_active":"2024-09-10T21:24:46.862540+0000","last_became_peered":"2024-09-10T21:24:46.862540+0000","last_unstale":"2024-09-10T21:24:53.173610+0000","last_undegraded":"2024-09-10T21:24:53.173610+0000","last_fullsized":"2024-09-10T21:24:53.173610+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:52:31.628568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.884678+0000","last_change":"2024-09-10T21:24:51.901224+0000","last_active":"2024-09-10T21:24:52.884678+0000","last_peered":"2024-09-10T21:24:52.884678+0000","last_clean":"2024-09-10T21:24:52.884678+0000","last_became_active":"2024-09-10T21:24:51.900610+0000","last_became_peered":"2024-09-10T21:24:51.900610+0000","last_unstale":"2024-09-10T21:24:52.884678+0000","last_undegraded":"2024-09-10T21:24:52.884678+0000","last_fullsized":"2024-09-10T21:24:52.884678+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:52:34.689860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.884489+0000","last_change":"2024-09-10T21:24:47.861255+0000","last_active":"2024-09-10T21:24:52.884489+0000","last_peered":"2024-09-10T21:24:52.884489+0000","last_clean":"2024-09-10T21:24:52.884489+0000","last_became_active":"2024-09-10T21:24:47.860930+0000","last_became_peered":"2024-09-10T21:24:47.860930+0000","last_unstale":"2024-09-10T21:24:52.884489+0000","last_undegraded":"2024-09-10T21:24:52.884489+0000","last_fullsized":"2024-09-10T21:24:52.884489+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:37:51.784258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.884796+0000","last_change":"2024-09-10T21:24:46.858561+0000","last_active":"2024-09-10T21:24:52.884796+0000","last_peered":"2024-09-10T21:24:52.884796+0000","last_clean":"2024-09-10T21:24:52.884796+0000","last_became_active":"2024-09-10T21:24:46.858265+0000","last_became_peered":"2024-09-10T21:24:46.858265+0000","last_unstale":"2024-09-10T21:24:52.884796+0000","last_undegraded":"2024-09-10T21:24:52.884796+0000","last_fullsized":"2024-09-10T21:24:52.884796+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:49:00.454609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"24'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.884271+0000","last_change":"2024-09-10T21:24:46.856886+0000","last_active":"2024-09-10T21:24:52.884271+0000","last_peered":"2024-09-10T21:24:52.884271+0000","last_clean":"2024-09-10T21:24:52.884271+0000","last_became_active":"2024-09-10T21:24:46.856506+0000","last_became_peered":"2024-09-10T21:24:46.856506+0000","last_unstale":"2024-09-10T21:24:52.884271+0000","last_undegraded":"2024-09-10T21:24:52.884271+0000","last_fullsized":"2024-09-10T21:24:52.884271+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:56:46.975225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174721+0000","last_change":"2024-09-10T21:24:47.848602+0000","last_active":"2024-09-10T21:24:53.174721+0000","last_peered":"2024-09-10T21:24:53.174721+0000","last_clean":"2024-09-10T21:24:53.174721+0000","last_became_active":"2024-09-10T21:24:47.848160+0000","last_became_peered":"2024-09-10T21:24:47.848160+0000","last_unstale":"2024-09-10T21:24:53.174721+0000","last_undegraded":"2024-09-10T21:24:53.174721+0000","last_fullsized":"2024-09-10T21:24:53.174721+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:32:50.027827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174649+0000","last_change":"2024-09-10T21:24:51.903553+0000","last_active":"2024-09-10T21:24:53.174649+0000","last_peered":"2024-09-10T21:24:53.174649+0000","last_clean":"2024-09-10T21:24:53.174649+0000","last_became_active":"2024-09-10T21:24:51.903120+0000","last_became_peered":"2024-09-10T21:24:51.903120+0000","last_unstale":"2024-09-10T21:24:53.174649+0000","last_undegraded":"2024-09-10T21:24:53.174649+0000","last_fullsized":"2024-09-10T21:24:53.174649+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:58.240380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"24'1","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.884865+0000","last_change":"2024-09-10T21:24:46.853550+0000","last_active":"2024-09-10T21:24:52.884865+0000","last_peered":"2024-09-10T21:24:52.884865+0000","last_clean":"2024-09-10T21:24:52.884865+0000","last_became_active":"2024-09-10T21:24:46.852204+0000","last_became_peered":"2024-09-10T21:24:46.852204+0000","last_unstale":"2024-09-10T21:24:52.884865+0000","last_undegraded":"2024-09-10T21:24:52.884865+0000","last_fullsized":"2024-09-10T21:24:52.884865+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:06:37.669444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.884581+0000","last_change":"2024-09-10T21:24:47.849151+0000","last_active":"2024-09-10T21:24:52.884581+0000","last_peered":"2024-09-10T21:24:52.884581+0000","last_clean":"2024-09-10T21:24:52.884581+0000","last_became_active":"2024-09-10T21:24:47.848851+0000","last_became_peered":"2024-09-10T21:24:47.848851+0000","last_unstale":"2024-09-10T21:24:52.884581+0000","last_undegraded":"2024-09-10T21:24:52.884581+0000","last_fullsized":"2024-09-10T21:24:52.884581+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:39:25.828558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.885276+0000","last_change":"2024-09-10T21:24:51.907549+0000","last_active":"2024-09-10T21:24:52.885276+0000","last_peered":"2024-09-10T21:24:52.885276+0000","last_clean":"2024-09-10T21:24:52.885276+0000","last_became_active":"2024-09-10T21:24:51.907082+0000","last_became_peered":"2024-09-10T21:24:51.907082+0000","last_unstale":"2024-09-10T21:24:52.885276+0000","last_undegraded":"2024-09-10T21:24:52.885276+0000","last_fullsized":"2024-09-10T21:24:52.885276+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:44:30.813754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.883360+0000","last_change":"2024-09-10T21:24:47.857804+0000","last_active":"2024-09-10T21:24:52.883360+0000","last_peered":"2024-09-10T21:24:52.883360+0000","last_clean":"2024-09-10T21:24:52.883360+0000","last_became_active":"2024-09-10T21:24:47.857501+0000","last_became_peered":"2024-09-10T21:24:47.857501+0000","last_unstale":"2024-09-10T21:24:52.883360+0000","last_undegraded":"2024-09-10T21:24:52.883360+0000","last_fullsized":"2024-09-10T21:24:52.883360+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:19:20.807422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.173632+0000","last_change":"2024-09-10T21:24:47.867803+0000","last_active":"2024-09-10T21:24:53.173632+0000","last_peered":"2024-09-10T21:24:53.173632+0000","last_clean":"2024-09-10T21:24:53.173632+0000","last_became_active":"2024-09-10T21:24:47.867504+0000","last_became_peered":"2024-09-10T21:24:47.867504+0000","last_unstale":"2024-09-10T21:24:53.173632+0000","last_undegraded":"2024-09-10T21:24:53.173632+0000","last_fullsized":"2024-09-10T21:24:53.173632+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:53:30.420454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174959+0000","last_change":"2024-09-10T21:24:47.868901+0000","last_active":"2024-09-10T21:24:53.174959+0000","last_peered":"2024-09-10T21:24:53.174959+0000","last_clean":"2024-09-10T21:24:53.174959+0000","last_became_active":"2024-09-10T21:24:47.868168+0000","last_became_peered":"2024-09-10T21:24:47.868168+0000","last_unstale":"2024-09-10T21:24:53.174959+0000","last_undegraded":"2024-09-10T21:24:53.174959+0000","last_fullsized":"2024-09-10T21:24:53.174959+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:06:21.461089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174633+0000","last_change":"2024-09-10T21:24:47.864561+0000","last_active":"2024-09-10T21:24:53.174633+0000","last_peered":"2024-09-10T21:24:53.174633+0000","last_clean":"2024-09-10T21:24:53.174633+0000","last_became_active":"2024-09-10T21:24:47.864100+0000","last_became_peered":"2024-09-10T21:24:47.864100+0000","last_unstale":"2024-09-10T21:24:53.174633+0000","last_undegraded":"2024-09-10T21:24:53.174633+0000","last_fullsized":"2024-09-10T21:24:53.174633+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:34:13.053494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.885174+0000","last_change":"2024-09-10T21:24:47.864303+0000","last_active":"2024-09-10T21:24:52.885174+0000","last_peered":"2024-09-10T21:24:52.885174+0000","last_clean":"2024-09-10T21:24:52.885174+0000","last_became_active":"2024-09-10T21:24:47.863970+0000","last_became_peered":"2024-09-10T21:24:47.863970+0000","last_unstale":"2024-09-10T21:24:52.885174+0000","last_undegraded":"2024-09-10T21:24:52.885174+0000","last_fullsized":"2024-09-10T21:24:52.885174+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:46:36.918539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.878793+0000","last_change":"2024-09-10T21:24:47.860734+0000","last_active":"2024-09-10T21:24:52.878793+0000","last_peered":"2024-09-10T21:24:52.878793+0000","last_clean":"2024-09-10T21:24:52.878793+0000","last_became_active":"2024-09-10T21:24:47.860471+0000","last_became_peered":"2024-09-10T21:24:47.860471+0000","last_unstale":"2024-09-10T21:24:52.878793+0000","last_undegraded":"2024-09-10T21:24:52.878793+0000","last_fullsized":"2024-09-10T21:24:52.878793+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:54:13.200082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.885049+0000","last_change":"2024-09-10T21:24:47.872504+0000","last_active":"2024-09-10T21:24:52.885049+0000","last_peered":"2024-09-10T21:24:52.885049+0000","last_clean":"2024-09-10T21:24:52.885049+0000","last_became_active":"2024-09-10T21:24:47.872218+0000","last_became_peered":"2024-09-10T21:24:47.872218+0000","last_unstale":"2024-09-10T21:24:52.885049+0000","last_undegraded":"2024-09-10T21:24:52.885049+0000","last_fullsized":"2024-09-10T21:24:52.885049+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:21:32.400520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.885478+0000","last_change":"2024-09-10T21:24:47.862167+0000","last_active":"2024-09-10T21:24:52.885478+0000","last_peered":"2024-09-10T21:24:52.885478+0000","last_clean":"2024-09-10T21:24:52.885478+0000","last_became_active":"2024-09-10T21:24:47.859649+0000","last_became_peered":"2024-09-10T21:24:47.859649+0000","last_unstale":"2024-09-10T21:24:52.885478+0000","last_undegraded":"2024-09-10T21:24:52.885478+0000","last_fullsized":"2024-09-10T21:24:52.885478+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:42:59.204284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.173544+0000","last_change":"2024-09-10T21:24:47.865322+0000","last_active":"2024-09-10T21:24:53.173544+0000","last_peered":"2024-09-10T21:24:53.173544+0000","last_clean":"2024-09-10T21:24:53.173544+0000","last_became_active":"2024-09-10T21:24:47.864933+0000","last_became_peered":"2024-09-10T21:24:47.864933+0000","last_unstale":"2024-09-10T21:24:53.173544+0000","last_undegraded":"2024-09-10T21:24:53.173544+0000","last_fullsized":"2024-09-10T21:24:53.173544+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:40:45.551723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.884760+0000","last_change":"2024-09-10T21:24:47.860155+0000","last_active":"2024-09-10T21:24:52.884760+0000","last_peered":"2024-09-10T21:24:52.884760+0000","last_clean":"2024-09-10T21:24:52.884760+0000","last_became_active":"2024-09-10T21:24:47.859857+0000","last_became_peered":"2024-09-10T21:24:47.859857+0000","last_unstale":"2024-09-10T21:24:52.884760+0000","last_undegraded":"2024-09-10T21:24:52.884760+0000","last_fullsized":"2024-09-10T21:24:52.884760+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:32:50.027827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174935+0000","last_change":"2024-09-10T21:24:47.872848+0000","last_active":"2024-09-10T21:24:53.174935+0000","last_peered":"2024-09-10T21:24:53.174935+0000","last_clean":"2024-09-10T21:24:53.174935+0000","last_became_active":"2024-09-10T21:24:47.868671+0000","last_became_peered":"2024-09-10T21:24:47.868671+0000","last_unstale":"2024-09-10T21:24:53.174935+0000","last_undegraded":"2024-09-10T21:24:53.174935+0000","last_fullsized":"2024-09-10T21:24:53.174935+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:33:59.291647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174519+0000","last_change":"2024-09-10T21:24:47.857464+0000","last_active":"2024-09-10T21:24:53.174519+0000","last_peered":"2024-09-10T21:24:53.174519+0000","last_clean":"2024-09-10T21:24:53.174519+0000","last_became_active":"2024-09-10T21:24:47.857116+0000","last_became_peered":"2024-09-10T21:24:47.857116+0000","last_unstale":"2024-09-10T21:24:53.174519+0000","last_undegraded":"2024-09-10T21:24:53.174519+0000","last_fullsized":"2024-09-10T21:24:53.174519+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:56:43.269026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.884005+0000","last_change":"2024-09-10T21:24:47.871619+0000","last_active":"2024-09-10T21:24:52.884005+0000","last_peered":"2024-09-10T21:24:52.884005+0000","last_clean":"2024-09-10T21:24:52.884005+0000","last_became_active":"2024-09-10T21:24:47.871350+0000","last_became_peered":"2024-09-10T21:24:47.871350+0000","last_unstale":"2024-09-10T21:24:52.884005+0000","last_undegraded":"2024-09-10T21:24:52.884005+0000","last_fullsized":"2024-09-10T21:24:52.884005+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:41.305116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.175386+0000","last_change":"2024-09-10T21:24:47.861352+0000","last_active":"2024-09-10T21:24:53.175386+0000","last_peered":"2024-09-10T21:24:53.175386+0000","last_clean":"2024-09-10T21:24:53.175386+0000","last_became_active":"2024-09-10T21:24:47.861007+0000","last_became_peered":"2024-09-10T21:24:47.861007+0000","last_unstale":"2024-09-10T21:24:53.175386+0000","last_undegraded":"2024-09-10T21:24:53.175386+0000","last_fullsized":"2024-09-10T21:24:53.175386+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:25.273458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.175077+0000","last_change":"2024-09-10T21:24:47.864205+0000","last_active":"2024-09-10T21:24:53.175077+0000","last_peered":"2024-09-10T21:24:53.175077+0000","last_clean":"2024-09-10T21:24:53.175077+0000","last_became_active":"2024-09-10T21:24:47.863932+0000","last_became_peered":"2024-09-10T21:24:47.863932+0000","last_unstale":"2024-09-10T21:24:53.175077+0000","last_undegraded":"2024-09-10T21:24:53.175077+0000","last_fullsized":"2024-09-10T21:24:53.175077+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:53:17.669629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.175063+0000","last_change":"2024-09-10T21:24:47.866225+0000","last_active":"2024-09-10T21:24:53.175063+0000","last_peered":"2024-09-10T21:24:53.175063+0000","last_clean":"2024-09-10T21:24:53.175063+0000","last_became_active":"2024-09-10T21:24:47.865587+0000","last_became_peered":"2024-09-10T21:24:47.865587+0000","last_unstale":"2024-09-10T21:24:53.175063+0000","last_undegraded":"2024-09-10T21:24:53.175063+0000","last_fullsized":"2024-09-10T21:24:53.175063+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:49:26.113068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.173421+0000","last_change":"2024-09-10T21:24:47.872669+0000","last_active":"2024-09-10T21:24:53.173421+0000","last_peered":"2024-09-10T21:24:53.173421+0000","last_clean":"2024-09-10T21:24:53.173421+0000","last_became_active":"2024-09-10T21:24:47.865837+0000","last_became_peered":"2024-09-10T21:24:47.865837+0000","last_unstale":"2024-09-10T21:24:53.173421+0000","last_undegraded":"2024-09-10T21:24:53.173421+0000","last_fullsized":"2024-09-10T21:24:53.173421+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:53:09.082512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.878332+0000","last_change":"2024-09-10T21:24:47.846567+0000","last_active":"2024-09-10T21:24:52.878332+0000","last_peered":"2024-09-10T21:24:52.878332+0000","last_clean":"2024-09-10T21:24:52.878332+0000","last_became_active":"2024-09-10T21:24:47.846283+0000","last_became_peered":"2024-09-10T21:24:47.846283+0000","last_unstale":"2024-09-10T21:24:52.878332+0000","last_undegraded":"2024-09-10T21:24:52.878332+0000","last_fullsized":"2024-09-10T21:24:52.878332+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:42:15.320518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174247+0000","last_change":"2024-09-10T21:24:47.844888+0000","last_active":"2024-09-10T21:24:53.174247+0000","last_peered":"2024-09-10T21:24:53.174247+0000","last_clean":"2024-09-10T21:24:53.174247+0000","last_became_active":"2024-09-10T21:24:47.844517+0000","last_became_peered":"2024-09-10T21:24:47.844517+0000","last_unstale":"2024-09-10T21:24:53.174247+0000","last_undegraded":"2024-09-10T21:24:53.174247+0000","last_fullsized":"2024-09-10T21:24:53.174247+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:00:14.988282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.884642+0000","last_change":"2024-09-10T21:24:47.862746+0000","last_active":"2024-09-10T21:24:52.884642+0000","last_peered":"2024-09-10T21:24:52.884642+0000","last_clean":"2024-09-10T21:24:52.884642+0000","last_became_active":"2024-09-10T21:24:47.862185+0000","last_became_peered":"2024-09-10T21:24:47.862185+0000","last_unstale":"2024-09-10T21:24:52.884642+0000","last_undegraded":"2024-09-10T21:24:52.884642+0000","last_fullsized":"2024-09-10T21:24:52.884642+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:06:21.461089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.886651+0000","last_change":"2024-09-10T21:24:47.867727+0000","last_active":"2024-09-10T21:24:52.886651+0000","last_peered":"2024-09-10T21:24:52.886651+0000","last_clean":"2024-09-10T21:24:52.886651+0000","last_became_active":"2024-09-10T21:24:47.861045+0000","last_became_peered":"2024-09-10T21:24:47.861045+0000","last_unstale":"2024-09-10T21:24:52.886651+0000","last_undegraded":"2024-09-10T21:24:52.886651+0000","last_fullsized":"2024-09-10T21:24:52.886651+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:08:30.065968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.884547+0000","last_change":"2024-09-10T21:24:47.867410+0000","last_active":"2024-09-10T21:24:52.884547+0000","last_peered":"2024-09-10T21:24:52.884547+0000","last_clean":"2024-09-10T21:24:52.884547+0000","last_became_active":"2024-09-10T21:24:47.867113+0000","last_became_peered":"2024-09-10T21:24:47.867113+0000","last_unstale":"2024-09-10T21:24:52.884547+0000","last_undegraded":"2024-09-10T21:24:52.884547+0000","last_fullsized":"2024-09-10T21:24:52.884547+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:49:53.502472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.884956+0000","last_change":"2024-09-10T21:24:47.867473+0000","last_active":"2024-09-10T21:24:52.884956+0000","last_peered":"2024-09-10T21:24:52.884956+0000","last_clean":"2024-09-10T21:24:52.884956+0000","last_became_active":"2024-09-10T21:24:47.867126+0000","last_became_peered":"2024-09-10T21:24:47.867126+0000","last_unstale":"2024-09-10T21:24:52.884956+0000","last_undegraded":"2024-09-10T21:24:52.884956+0000","last_fullsized":"2024-09-10T21:24:52.884956+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:54:32.307507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.175341+0000","last_change":"2024-09-10T21:24:47.865450+0000","last_active":"2024-09-10T21:24:53.175341+0000","last_peered":"2024-09-10T21:24:53.175341+0000","last_clean":"2024-09-10T21:24:53.175341+0000","last_became_active":"2024-09-10T21:24:47.865093+0000","last_became_peered":"2024-09-10T21:24:47.865093+0000","last_unstale":"2024-09-10T21:24:53.175341+0000","last_undegraded":"2024-09-10T21:24:53.175341+0000","last_fullsized":"2024-09-10T21:24:53.175341+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:12:18.854783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.884322+0000","last_change":"2024-09-10T21:24:47.868093+0000","last_active":"2024-09-10T21:24:52.884322+0000","last_peered":"2024-09-10T21:24:52.884322+0000","last_clean":"2024-09-10T21:24:52.884322+0000","last_became_active":"2024-09-10T21:24:47.867782+0000","last_became_peered":"2024-09-10T21:24:47.867782+0000","last_unstale":"2024-09-10T21:24:52.884322+0000","last_undegraded":"2024-09-10T21:24:52.884322+0000","last_fullsized":"2024-09-10T21:24:52.884322+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:34:13.053494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.173787+0000","last_change":"2024-09-10T21:24:47.862204+0000","last_active":"2024-09-10T21:24:53.173787+0000","last_peered":"2024-09-10T21:24:53.173787+0000","last_clean":"2024-09-10T21:24:53.173787+0000","last_became_active":"2024-09-10T21:24:47.861434+0000","last_became_peered":"2024-09-10T21:24:47.861434+0000","last_unstale":"2024-09-10T21:24:53.173787+0000","last_undegraded":"2024-09-10T21:24:53.173787+0000","last_fullsized":"2024-09-10T21:24:53.173787+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:50:03.245285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174760+0000","last_change":"2024-09-10T21:24:47.869248+0000","last_active":"2024-09-10T21:24:53.174760+0000","last_peered":"2024-09-10T21:24:53.174760+0000","last_clean":"2024-09-10T21:24:53.174760+0000","last_became_active":"2024-09-10T21:24:47.868503+0000","last_became_peered":"2024-09-10T21:24:47.868503+0000","last_unstale":"2024-09-10T21:24:53.174760+0000","last_undegraded":"2024-09-10T21:24:53.174760+0000","last_fullsized":"2024-09-10T21:24:53.174760+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:51:31.417845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174007+0000","last_change":"2024-09-10T21:24:47.864802+0000","last_active":"2024-09-10T21:24:53.174007+0000","last_peered":"2024-09-10T21:24:53.174007+0000","last_clean":"2024-09-10T21:24:53.174007+0000","last_became_active":"2024-09-10T21:24:47.863976+0000","last_became_peered":"2024-09-10T21:24:47.863976+0000","last_unstale":"2024-09-10T21:24:53.174007+0000","last_undegraded":"2024-09-10T21:24:53.174007+0000","last_fullsized":"2024-09-10T21:24:53.174007+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:55:08.544075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174437+0000","last_change":"2024-09-10T21:24:47.873364+0000","last_active":"2024-09-10T21:24:53.174437+0000","last_peered":"2024-09-10T21:24:53.174437+0000","last_clean":"2024-09-10T21:24:53.174437+0000","last_became_active":"2024-09-10T21:24:47.873069+0000","last_became_peered":"2024-09-10T21:24:47.873069+0000","last_unstale":"2024-09-10T21:24:53.174437+0000","last_undegraded":"2024-09-10T21:24:53.174437+0000","last_fullsized":"2024-09-10T21:24:53.174437+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:38:21.368499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174196+0000","last_change":"2024-09-10T21:24:47.856936+0000","last_active":"2024-09-10T21:24:53.174196+0000","last_peered":"2024-09-10T21:24:53.174196+0000","last_clean":"2024-09-10T21:24:53.174196+0000","last_became_active":"2024-09-10T21:24:47.854713+0000","last_became_peered":"2024-09-10T21:24:47.854713+0000","last_unstale":"2024-09-10T21:24:53.174196+0000","last_undegraded":"2024-09-10T21:24:53.174196+0000","last_fullsized":"2024-09-10T21:24:53.174196+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:46:36.918539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.878924+0000","last_change":"2024-09-10T21:24:47.846492+0000","last_active":"2024-09-10T21:24:52.878924+0000","last_peered":"2024-09-10T21:24:52.878924+0000","last_clean":"2024-09-10T21:24:52.878924+0000","last_became_active":"2024-09-10T21:24:47.846227+0000","last_became_peered":"2024-09-10T21:24:47.846227+0000","last_unstale":"2024-09-10T21:24:52.878924+0000","last_undegraded":"2024-09-10T21:24:52.878924+0000","last_fullsized":"2024-09-10T21:24:52.878924+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:13:58.432819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.175219+0000","last_change":"2024-09-10T21:24:47.871469+0000","last_active":"2024-09-10T21:24:53.175219+0000","last_peered":"2024-09-10T21:24:53.175219+0000","last_clean":"2024-09-10T21:24:53.175219+0000","last_became_active":"2024-09-10T21:24:47.871177+0000","last_became_peered":"2024-09-10T21:24:47.871177+0000","last_unstale":"2024-09-10T21:24:53.175219+0000","last_undegraded":"2024-09-10T21:24:53.175219+0000","last_fullsized":"2024-09-10T21:24:53.175219+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:19:24.226916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":66932,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509450,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":320,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":327680,"data_stored":693447,"data_compressed":33466,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509451,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":270336,"data_stored":106339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509450,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":258048,"data_stored":102497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509451,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31456,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340384,"statfs":{"total":96636764160,"available":96604553216,"internally_reserved":0,"allocated":266240,"data_stored":102657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542156,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27296,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":270336,"data_stored":102623,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542155,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":270336,"data_stored":106339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542158,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27364,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344476,"statfs":{"total":96636764160,"available":96608743424,"internally_reserved":0,"allocated":335872,"data_stored":693503,"data_compressed":33466,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574859,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":262144,"data_stored":102587,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33466,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33466,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T21:25:22.745 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T21:25:22.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.926+0000 7f8cea400640 1 -- 172.21.15.17:0/4262096224 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8ce40b2070 msgr2=0x7f8ce40725c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:22.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.926+0000 7f8cea400640 1 --2- 172.21.15.17:0/4262096224 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8ce40b2070 0x7f8ce40725c0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f8ce40c7c70 tx=0x7f8cdc01f580 comp rx=0 tx=0).stop 2024-09-10T21:25:22.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.926+0000 7f8cea400640 1 -- 172.21.15.17:0/4262096224 shutdown_connections 2024-09-10T21:25:22.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.926+0000 7f8cea400640 1 --2- 172.21.15.17:0/4262096224 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8ce40b2070 0x7f8ce40725c0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:22.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.926+0000 7f8cea400640 1 --2- 172.21.15.17:0/4262096224 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8ce41572d0 0x7f8ce415b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:22.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.926+0000 7f8cea400640 1 --2- 172.21.15.17:0/4262096224 >> v2:172.21.15.31:3300/0 conn(0x7f8ce41516c0 0x7f8ce40c25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:22.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.926+0000 7f8cea400640 1 -- 172.21.15.17:0/4262096224 >> 172.21.15.17:0/4262096224 conn(0x7f8ce4070160 msgr2=0x7f8ce4070570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:22.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.926+0000 7f8cea400640 1 -- 172.21.15.17:0/4262096224 shutdown_connections 2024-09-10T21:25:22.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.926+0000 7f8cea400640 1 -- 172.21.15.17:0/4262096224 wait complete. 2024-09-10T21:25:22.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.926+0000 7f8cea400640 1 Processor -- start 2024-09-10T21:25:22.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.926+0000 7f8cea400640 1 -- start start 2024-09-10T21:25:22.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.926+0000 7f8cea400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8ce40b2070 0x7f8ce41f0420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:22.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.926+0000 7f8cea400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8ce41516c0 0x7f8ce40b0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:22.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.926+0000 7f8cea400640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8ce4156540 0x7f8ce414e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:22.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.926+0000 7f8cea400640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f8ce415b780 con 0x7f8ce40b2070 2024-09-10T21:25:22.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.926+0000 7f8cea400640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f8ce415ad20 con 0x7f8ce41516c0 2024-09-10T21:25:22.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.926+0000 7f8cea400640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f8ce40c2cd0 con 0x7f8ce4156540 2024-09-10T21:25:22.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.926+0000 7f8ce3400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8ce40b2070 0x7f8ce41f0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:22.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.926+0000 7f8ce3400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8ce40b2070 0x7f8ce41f0420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44682/0 (socket says 172.21.15.17:44682) 2024-09-10T21:25:22.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.926+0000 7f8ce3400640 1 -- 172.21.15.17:0/585226355 learned_addr learned my addr 172.21.15.17:0/585226355 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:25:22.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.926+0000 7f8cdbe00640 1 --2- 172.21.15.17:0/585226355 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8ce41516c0 0x7f8ce40b0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:22.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.930+0000 7f8ce3e00640 1 --2- 172.21.15.17:0/585226355 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8ce4156540 0x7f8ce414e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:22.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.930+0000 7f8ce3400640 1 -- 172.21.15.17:0/585226355 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8ce4156540 msgr2=0x7f8ce414e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:22.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.930+0000 7f8ce3400640 1 --2- 172.21.15.17:0/585226355 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8ce4156540 0x7f8ce414e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:22.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.930+0000 7f8ce3400640 1 -- 172.21.15.17:0/585226355 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8ce41516c0 msgr2=0x7f8ce40b0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:22.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.930+0000 7f8ce3400640 1 --2- 172.21.15.17:0/585226355 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8ce41516c0 0x7f8ce40b0680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:22.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.930+0000 7f8ce3400640 1 -- 172.21.15.17:0/585226355 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ce40b0bc0 con 0x7f8ce40b2070 2024-09-10T21:25:22.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.930+0000 7f8ce3e00640 1 --2- 172.21.15.17:0/585226355 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8ce4156540 0x7f8ce414e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:25:22.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.930+0000 7f8cdbe00640 1 --2- 172.21.15.17:0/585226355 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8ce41516c0 0x7f8ce40b0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:25:22.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.930+0000 7f8ce3400640 1 --2- 172.21.15.17:0/585226355 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8ce40b2070 0x7f8ce41f0420 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f8cd400d770 tx=0x7f8cd400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:22.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.930+0000 7f8cdb400640 1 -- 172.21.15.17:0/585226355 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cd4017070 con 0x7f8ce40b2070 2024-09-10T21:25:22.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.930+0000 7f8cdb400640 1 -- 172.21.15.17:0/585226355 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8cd4009d20 con 0x7f8ce40b2070 2024-09-10T21:25:22.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.930+0000 7f8cea400640 1 -- 172.21.15.17:0/585226355 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ce414f280 con 0x7f8ce40b2070 2024-09-10T21:25:22.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.930+0000 7f8cdb400640 1 -- 172.21.15.17:0/585226355 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cd40106c0 con 0x7f8ce40b2070 2024-09-10T21:25:22.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.930+0000 7f8cea400640 1 -- 172.21.15.17:0/585226355 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ce40b1220 con 0x7f8ce40b2070 2024-09-10T21:25:22.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.930+0000 7f8cea400640 1 -- 172.21.15.17:0/585226355 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ce40c25d0 con 0x7f8ce40b2070 2024-09-10T21:25:22.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.930+0000 7f8cdb400640 1 -- 172.21.15.17:0/585226355 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f8cd4010860 con 0x7f8ce40b2070 2024-09-10T21:25:22.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.930+0000 7f8cdb400640 1 --2- 172.21.15.17:0/585226355 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f8cc00725e0 0x7f8cc0074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:22.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.930+0000 7f8cdb400640 1 -- 172.21.15.17:0/585226355 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f8cd4093aa0 con 0x7f8ce40b2070 2024-09-10T21:25:22.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.934+0000 7f8cdbe00640 1 --2- 172.21.15.17:0/585226355 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f8cc00725e0 0x7f8cc0074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:22.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.934+0000 7f8cdbe00640 1 --2- 172.21.15.17:0/585226355 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f8cc00725e0 0x7f8cc0074aa0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f8ce414ff00 tx=0x7f8ccc00b5b0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:22.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:22.934+0000 7f8cdb400640 1 -- 172.21.15.17:0/585226355 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f8ce414f280 con 0x7f8ce40b2070 2024-09-10T21:25:23.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:23.106+0000 7f8cea400640 1 -- 172.21.15.17:0/585226355 --> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8ce40c10a0 con 0x7f8cc00725e0 2024-09-10T21:25:23.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:23.114+0000 7f8cdb400640 1 -- 172.21.15.17:0/585226355 <== mgr.4100 v2:172.21.15.31:6808/3101308586 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332597 (secure 0 0 0) 0x7f8ce40c10a0 con 0x7f8cc00725e0 2024-09-10T21:25:23.122 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-10T21:25:23.124 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2024-09-10T21:25:23.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:23.118+0000 7f8cea400640 1 -- 172.21.15.17:0/585226355 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f8cc00725e0 msgr2=0x7f8cc0074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:23.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:23.118+0000 7f8cea400640 1 --2- 172.21.15.17:0/585226355 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f8cc00725e0 0x7f8cc0074aa0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f8ce414ff00 tx=0x7f8ccc00b5b0 comp rx=0 tx=0).stop 2024-09-10T21:25:23.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:23.118+0000 7f8cea400640 1 -- 172.21.15.17:0/585226355 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8ce40b2070 msgr2=0x7f8ce41f0420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:23.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:23.118+0000 7f8cea400640 1 --2- 172.21.15.17:0/585226355 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8ce40b2070 0x7f8ce41f0420 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f8cd400d770 tx=0x7f8cd400dc30 comp rx=0 tx=0).stop 2024-09-10T21:25:23.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:23.118+0000 7f8cea400640 1 -- 172.21.15.17:0/585226355 shutdown_connections 2024-09-10T21:25:23.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:23.118+0000 7f8cea400640 1 --2- 172.21.15.17:0/585226355 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f8cc00725e0 0x7f8cc0074aa0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:23.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:23.118+0000 7f8cea400640 1 --2- 172.21.15.17:0/585226355 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8ce4156540 0x7f8ce414e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:23.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:23.118+0000 7f8cea400640 1 --2- 172.21.15.17:0/585226355 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8ce41516c0 0x7f8ce40b0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:23.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:23.118+0000 7f8cea400640 1 --2- 172.21.15.17:0/585226355 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8ce40b2070 0x7f8ce41f0420 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:23.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:23.118+0000 7f8cea400640 1 -- 172.21.15.17:0/585226355 >> 172.21.15.17:0/585226355 conn(0x7f8ce4070160 msgr2=0x7f8ce4155e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:23.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:23.118+0000 7f8cea400640 1 -- 172.21.15.17:0/585226355 shutdown_connections 2024-09-10T21:25:23.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:23.118+0000 7f8cea400640 1 -- 172.21.15.17:0/585226355 wait complete. 2024-09-10T21:25:23.139 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":42,"stamp":"2024-09-10T21:25:21.346548+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":149,"num_write_kb":1646,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":114,"ondisk_log_size":114,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":222608,"kb_used_data":2208,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754752112,"statfs":{"total":773094113280,"available":772866162688,"internally_reserved":0,"allocated":2260992,"data_stored":2009992,"data_compressed":66932,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12752,"internal_metadata":225496624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003364"},"pg_stats":[{"pgid":"2.1f","version":"24'4","reported_seq":27,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174456+0000","last_change":"2024-09-10T21:24:46.864556+0000","last_active":"2024-09-10T21:24:53.174456+0000","last_peered":"2024-09-10T21:24:53.174456+0000","last_clean":"2024-09-10T21:24:53.174456+0000","last_became_active":"2024-09-10T21:24:46.864288+0000","last_became_peered":"2024-09-10T21:24:46.864288+0000","last_unstale":"2024-09-10T21:24:53.174456+0000","last_undegraded":"2024-09-10T21:24:53.174456+0000","last_fullsized":"2024-09-10T21:24:53.174456+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:00:38.734807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.885137+0000","last_change":"2024-09-10T21:24:47.864452+0000","last_active":"2024-09-10T21:24:52.885137+0000","last_peered":"2024-09-10T21:24:52.885137+0000","last_clean":"2024-09-10T21:24:52.885137+0000","last_became_active":"2024-09-10T21:24:47.864168+0000","last_became_peered":"2024-09-10T21:24:47.864168+0000","last_unstale":"2024-09-10T21:24:52.885137+0000","last_undegraded":"2024-09-10T21:24:52.885137+0000","last_fullsized":"2024-09-10T21:24:52.885137+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:51:31.417845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.884507+0000","last_change":"2024-09-10T21:24:52.070054+0000","last_active":"2024-09-10T21:24:52.884507+0000","last_peered":"2024-09-10T21:24:52.884507+0000","last_clean":"2024-09-10T21:24:52.884507+0000","last_became_active":"2024-09-10T21:24:52.067258+0000","last_became_peered":"2024-09-10T21:24:52.067258+0000","last_unstale":"2024-09-10T21:24:52.884507+0000","last_undegraded":"2024-09-10T21:24:52.884507+0000","last_fullsized":"2024-09-10T21:24:52.884507+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:21:53.080013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"24'1","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.885320+0000","last_change":"2024-09-10T21:24:46.860648+0000","last_active":"2024-09-10T21:24:52.885320+0000","last_peered":"2024-09-10T21:24:52.885320+0000","last_clean":"2024-09-10T21:24:52.885320+0000","last_became_active":"2024-09-10T21:24:46.860341+0000","last_became_peered":"2024-09-10T21:24:46.860341+0000","last_unstale":"2024-09-10T21:24:52.885320+0000","last_undegraded":"2024-09-10T21:24:52.885320+0000","last_fullsized":"2024-09-10T21:24:52.885320+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:39:24.819979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174137+0000","last_change":"2024-09-10T21:24:47.865617+0000","last_active":"2024-09-10T21:24:53.174137+0000","last_peered":"2024-09-10T21:24:53.174137+0000","last_clean":"2024-09-10T21:24:53.174137+0000","last_became_active":"2024-09-10T21:24:47.865365+0000","last_became_peered":"2024-09-10T21:24:47.865365+0000","last_unstale":"2024-09-10T21:24:53.174137+0000","last_undegraded":"2024-09-10T21:24:53.174137+0000","last_fullsized":"2024-09-10T21:24:53.174137+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:10:05.971996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.883672+0000","last_change":"2024-09-10T21:24:51.900077+0000","last_active":"2024-09-10T21:24:52.883672+0000","last_peered":"2024-09-10T21:24:52.883672+0000","last_clean":"2024-09-10T21:24:52.883672+0000","last_became_active":"2024-09-10T21:24:51.899652+0000","last_became_peered":"2024-09-10T21:24:51.899652+0000","last_unstale":"2024-09-10T21:24:52.883672+0000","last_undegraded":"2024-09-10T21:24:52.883672+0000","last_fullsized":"2024-09-10T21:24:52.883672+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:40:49.585857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"24'2","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.175383+0000","last_change":"2024-09-10T21:24:46.861040+0000","last_active":"2024-09-10T21:24:53.175383+0000","last_peered":"2024-09-10T21:24:53.175383+0000","last_clean":"2024-09-10T21:24:53.175383+0000","last_became_active":"2024-09-10T21:24:46.860780+0000","last_became_peered":"2024-09-10T21:24:46.860780+0000","last_unstale":"2024-09-10T21:24:53.175383+0000","last_undegraded":"2024-09-10T21:24:53.175383+0000","last_fullsized":"2024-09-10T21:24:53.175383+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:53:31.454472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174596+0000","last_change":"2024-09-10T21:24:47.877997+0000","last_active":"2024-09-10T21:24:53.174596+0000","last_peered":"2024-09-10T21:24:53.174596+0000","last_clean":"2024-09-10T21:24:53.174596+0000","last_became_active":"2024-09-10T21:24:47.877765+0000","last_became_peered":"2024-09-10T21:24:47.877765+0000","last_unstale":"2024-09-10T21:24:53.174596+0000","last_undegraded":"2024-09-10T21:24:53.174596+0000","last_fullsized":"2024-09-10T21:24:53.174596+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:33:20.590209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.886528+0000","last_change":"2024-09-10T21:24:51.908333+0000","last_active":"2024-09-10T21:24:52.886528+0000","last_peered":"2024-09-10T21:24:52.886528+0000","last_clean":"2024-09-10T21:24:52.886528+0000","last_became_active":"2024-09-10T21:24:51.907933+0000","last_became_peered":"2024-09-10T21:24:51.907933+0000","last_unstale":"2024-09-10T21:24:52.886528+0000","last_undegraded":"2024-09-10T21:24:52.886528+0000","last_fullsized":"2024-09-10T21:24:52.886528+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:07:33.193555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"24'2","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174867+0000","last_change":"2024-09-10T21:24:46.873044+0000","last_active":"2024-09-10T21:24:53.174867+0000","last_peered":"2024-09-10T21:24:53.174867+0000","last_clean":"2024-09-10T21:24:53.174867+0000","last_became_active":"2024-09-10T21:24:46.872075+0000","last_became_peered":"2024-09-10T21:24:46.872075+0000","last_unstale":"2024-09-10T21:24:53.174867+0000","last_undegraded":"2024-09-10T21:24:53.174867+0000","last_fullsized":"2024-09-10T21:24:53.174867+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:40:33.632846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174402+0000","last_change":"2024-09-10T21:24:47.878094+0000","last_active":"2024-09-10T21:24:53.174402+0000","last_peered":"2024-09-10T21:24:53.174402+0000","last_clean":"2024-09-10T21:24:53.174402+0000","last_became_active":"2024-09-10T21:24:47.876688+0000","last_became_peered":"2024-09-10T21:24:47.876688+0000","last_unstale":"2024-09-10T21:24:53.174402+0000","last_undegraded":"2024-09-10T21:24:53.174402+0000","last_fullsized":"2024-09-10T21:24:53.174402+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:03:37.076080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.884328+0000","last_change":"2024-09-10T21:24:51.900208+0000","last_active":"2024-09-10T21:24:52.884328+0000","last_peered":"2024-09-10T21:24:52.884328+0000","last_clean":"2024-09-10T21:24:52.884328+0000","last_became_active":"2024-09-10T21:24:51.899576+0000","last_became_peered":"2024-09-10T21:24:51.899576+0000","last_unstale":"2024-09-10T21:24:52.884328+0000","last_undegraded":"2024-09-10T21:24:52.884328+0000","last_fullsized":"2024-09-10T21:24:52.884328+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:52:49.479932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.883746+0000","last_change":"2024-09-10T21:24:47.869830+0000","last_active":"2024-09-10T21:24:52.883746+0000","last_peered":"2024-09-10T21:24:52.883746+0000","last_clean":"2024-09-10T21:24:52.883746+0000","last_became_active":"2024-09-10T21:24:47.869569+0000","last_became_peered":"2024-09-10T21:24:47.869569+0000","last_unstale":"2024-09-10T21:24:52.883746+0000","last_undegraded":"2024-09-10T21:24:52.883746+0000","last_fullsized":"2024-09-10T21:24:52.883746+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:42:15.320518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"24'2","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174248+0000","last_change":"2024-09-10T21:24:46.867297+0000","last_active":"2024-09-10T21:24:53.174248+0000","last_peered":"2024-09-10T21:24:53.174248+0000","last_clean":"2024-09-10T21:24:53.174248+0000","last_became_active":"2024-09-10T21:24:46.866893+0000","last_became_peered":"2024-09-10T21:24:46.866893+0000","last_unstale":"2024-09-10T21:24:53.174248+0000","last_undegraded":"2024-09-10T21:24:53.174248+0000","last_fullsized":"2024-09-10T21:24:53.174248+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:53.197666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.885044+0000","last_change":"2024-09-10T21:24:51.908412+0000","last_active":"2024-09-10T21:24:52.885044+0000","last_peered":"2024-09-10T21:24:52.885044+0000","last_clean":"2024-09-10T21:24:52.885044+0000","last_became_active":"2024-09-10T21:24:51.907923+0000","last_became_peered":"2024-09-10T21:24:51.907923+0000","last_unstale":"2024-09-10T21:24:52.885044+0000","last_undegraded":"2024-09-10T21:24:52.885044+0000","last_fullsized":"2024-09-10T21:24:52.885044+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:26:46.479501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.884755+0000","last_change":"2024-09-10T21:24:47.862479+0000","last_active":"2024-09-10T21:24:52.884755+0000","last_peered":"2024-09-10T21:24:52.884755+0000","last_clean":"2024-09-10T21:24:52.884755+0000","last_became_active":"2024-09-10T21:24:47.861842+0000","last_became_peered":"2024-09-10T21:24:47.861842+0000","last_unstale":"2024-09-10T21:24:52.884755+0000","last_undegraded":"2024-09-10T21:24:52.884755+0000","last_fullsized":"2024-09-10T21:24:52.884755+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:53:09.082512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.878911+0000","last_change":"2024-09-10T21:24:46.852467+0000","last_active":"2024-09-10T21:24:52.878911+0000","last_peered":"2024-09-10T21:24:52.878911+0000","last_clean":"2024-09-10T21:24:52.878911+0000","last_became_active":"2024-09-10T21:24:46.851027+0000","last_became_peered":"2024-09-10T21:24:46.851027+0000","last_unstale":"2024-09-10T21:24:52.878911+0000","last_undegraded":"2024-09-10T21:24:52.878911+0000","last_fullsized":"2024-09-10T21:24:52.878911+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:39:27.441714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.886564+0000","last_change":"2024-09-10T21:24:51.903577+0000","last_active":"2024-09-10T21:24:52.886564+0000","last_peered":"2024-09-10T21:24:52.886564+0000","last_clean":"2024-09-10T21:24:52.886564+0000","last_became_active":"2024-09-10T21:24:51.903031+0000","last_became_peered":"2024-09-10T21:24:51.903031+0000","last_unstale":"2024-09-10T21:24:52.886564+0000","last_undegraded":"2024-09-10T21:24:52.886564+0000","last_fullsized":"2024-09-10T21:24:52.886564+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:08:52.787720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.884612+0000","last_change":"2024-09-10T21:24:47.856334+0000","last_active":"2024-09-10T21:24:52.884612+0000","last_peered":"2024-09-10T21:24:52.884612+0000","last_clean":"2024-09-10T21:24:52.884612+0000","last_became_active":"2024-09-10T21:24:47.856003+0000","last_became_peered":"2024-09-10T21:24:47.856003+0000","last_unstale":"2024-09-10T21:24:52.884612+0000","last_undegraded":"2024-09-10T21:24:52.884612+0000","last_fullsized":"2024-09-10T21:24:52.884612+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:58:53.698584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.878371+0000","last_change":"2024-09-10T21:24:46.853373+0000","last_active":"2024-09-10T21:24:52.878371+0000","last_peered":"2024-09-10T21:24:52.878371+0000","last_clean":"2024-09-10T21:24:52.878371+0000","last_became_active":"2024-09-10T21:24:46.852993+0000","last_became_peered":"2024-09-10T21:24:46.852993+0000","last_unstale":"2024-09-10T21:24:52.878371+0000","last_undegraded":"2024-09-10T21:24:52.878371+0000","last_fullsized":"2024-09-10T21:24:52.878371+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:53:31.454472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.879271+0000","last_change":"2024-09-10T21:24:52.065246+0000","last_active":"2024-09-10T21:24:52.879271+0000","last_peered":"2024-09-10T21:24:52.879271+0000","last_clean":"2024-09-10T21:24:52.879271+0000","last_became_active":"2024-09-10T21:24:52.064884+0000","last_became_peered":"2024-09-10T21:24:52.064884+0000","last_unstale":"2024-09-10T21:24:52.879271+0000","last_undegraded":"2024-09-10T21:24:52.879271+0000","last_fullsized":"2024-09-10T21:24:52.879271+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:54:54.393305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.884318+0000","last_change":"2024-09-10T21:24:47.856517+0000","last_active":"2024-09-10T21:24:52.884318+0000","last_peered":"2024-09-10T21:24:52.884318+0000","last_clean":"2024-09-10T21:24:52.884318+0000","last_became_active":"2024-09-10T21:24:47.856185+0000","last_became_peered":"2024-09-10T21:24:47.856185+0000","last_unstale":"2024-09-10T21:24:52.884318+0000","last_undegraded":"2024-09-10T21:24:52.884318+0000","last_fullsized":"2024-09-10T21:24:52.884318+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:26:00.852818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.173635+0000","last_change":"2024-09-10T21:24:46.867888+0000","last_active":"2024-09-10T21:24:53.173635+0000","last_peered":"2024-09-10T21:24:53.173635+0000","last_clean":"2024-09-10T21:24:53.173635+0000","last_became_active":"2024-09-10T21:24:46.866477+0000","last_became_peered":"2024-09-10T21:24:46.866477+0000","last_unstale":"2024-09-10T21:24:53.173635+0000","last_undegraded":"2024-09-10T21:24:53.173635+0000","last_fullsized":"2024-09-10T21:24:53.173635+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:39:27.441714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.885185+0000","last_change":"2024-09-10T21:24:51.908376+0000","last_active":"2024-09-10T21:24:52.885185+0000","last_peered":"2024-09-10T21:24:52.885185+0000","last_clean":"2024-09-10T21:24:52.885185+0000","last_became_active":"2024-09-10T21:24:51.907878+0000","last_became_peered":"2024-09-10T21:24:51.907878+0000","last_unstale":"2024-09-10T21:24:52.885185+0000","last_undegraded":"2024-09-10T21:24:52.885185+0000","last_fullsized":"2024-09-10T21:24:52.885185+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:39:02.492952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174342+0000","last_change":"2024-09-10T21:24:47.844823+0000","last_active":"2024-09-10T21:24:53.174342+0000","last_peered":"2024-09-10T21:24:53.174342+0000","last_clean":"2024-09-10T21:24:53.174342+0000","last_became_active":"2024-09-10T21:24:47.844505+0000","last_became_peered":"2024-09-10T21:24:47.844505+0000","last_unstale":"2024-09-10T21:24:53.174342+0000","last_undegraded":"2024-09-10T21:24:53.174342+0000","last_fullsized":"2024-09-10T21:24:53.174342+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:37:51.784258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"24'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.885358+0000","last_change":"2024-09-10T21:24:46.862172+0000","last_active":"2024-09-10T21:24:52.885358+0000","last_peered":"2024-09-10T21:24:52.885358+0000","last_clean":"2024-09-10T21:24:52.885358+0000","last_became_active":"2024-09-10T21:24:46.861473+0000","last_became_peered":"2024-09-10T21:24:46.861473+0000","last_unstale":"2024-09-10T21:24:52.885358+0000","last_undegraded":"2024-09-10T21:24:52.885358+0000","last_fullsized":"2024-09-10T21:24:52.885358+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:53:31.454472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.883945+0000","last_change":"2024-09-10T21:24:52.062841+0000","last_active":"2024-09-10T21:24:52.883945+0000","last_peered":"2024-09-10T21:24:52.883945+0000","last_clean":"2024-09-10T21:24:52.883945+0000","last_became_active":"2024-09-10T21:24:52.061581+0000","last_became_peered":"2024-09-10T21:24:52.061581+0000","last_unstale":"2024-09-10T21:24:52.883945+0000","last_undegraded":"2024-09-10T21:24:52.883945+0000","last_fullsized":"2024-09-10T21:24:52.883945+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:34:03.325781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.885543+0000","last_change":"2024-09-10T21:24:47.859801+0000","last_active":"2024-09-10T21:24:52.885543+0000","last_peered":"2024-09-10T21:24:52.885543+0000","last_clean":"2024-09-10T21:24:52.885543+0000","last_became_active":"2024-09-10T21:24:47.859483+0000","last_became_peered":"2024-09-10T21:24:47.859483+0000","last_unstale":"2024-09-10T21:24:52.885543+0000","last_undegraded":"2024-09-10T21:24:52.885543+0000","last_fullsized":"2024-09-10T21:24:52.885543+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:38:21.368499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"25'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.879175+0000","last_change":"2024-09-10T21:24:46.860624+0000","last_active":"2024-09-10T21:24:52.879175+0000","last_peered":"2024-09-10T21:24:52.879175+0000","last_clean":"2024-09-10T21:24:52.879175+0000","last_became_active":"2024-09-10T21:24:46.860309+0000","last_became_peered":"2024-09-10T21:24:46.860309+0000","last_unstale":"2024-09-10T21:24:52.879175+0000","last_undegraded":"2024-09-10T21:24:52.879175+0000","last_fullsized":"2024-09-10T21:24:52.879175+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:37:12.332013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.884414+0000","last_change":"2024-09-10T21:24:52.068581+0000","last_active":"2024-09-10T21:24:52.884414+0000","last_peered":"2024-09-10T21:24:52.884414+0000","last_clean":"2024-09-10T21:24:52.884414+0000","last_became_active":"2024-09-10T21:24:52.068250+0000","last_became_peered":"2024-09-10T21:24:52.068250+0000","last_unstale":"2024-09-10T21:24:52.884414+0000","last_undegraded":"2024-09-10T21:24:52.884414+0000","last_fullsized":"2024-09-10T21:24:52.884414+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:52:01.990009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.879000+0000","last_change":"2024-09-10T21:24:47.854042+0000","last_active":"2024-09-10T21:24:52.879000+0000","last_peered":"2024-09-10T21:24:52.879000+0000","last_clean":"2024-09-10T21:24:52.879000+0000","last_became_active":"2024-09-10T21:24:47.853750+0000","last_became_peered":"2024-09-10T21:24:47.853750+0000","last_unstale":"2024-09-10T21:24:52.879000+0000","last_undegraded":"2024-09-10T21:24:52.879000+0000","last_fullsized":"2024-09-10T21:24:52.879000+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:06:50.775132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.175510+0000","last_change":"2024-09-10T21:24:46.864617+0000","last_active":"2024-09-10T21:24:53.175510+0000","last_peered":"2024-09-10T21:24:53.175510+0000","last_clean":"2024-09-10T21:24:53.175510+0000","last_became_active":"2024-09-10T21:24:46.864149+0000","last_became_peered":"2024-09-10T21:24:46.864149+0000","last_unstale":"2024-09-10T21:24:53.175510+0000","last_undegraded":"2024-09-10T21:24:53.175510+0000","last_fullsized":"2024-09-10T21:24:53.175510+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:24:15.673107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174530+0000","last_change":"2024-09-10T21:24:52.061485+0000","last_active":"2024-09-10T21:24:53.174530+0000","last_peered":"2024-09-10T21:24:53.174530+0000","last_clean":"2024-09-10T21:24:53.174530+0000","last_became_active":"2024-09-10T21:24:52.060953+0000","last_became_peered":"2024-09-10T21:24:52.060953+0000","last_unstale":"2024-09-10T21:24:53.174530+0000","last_undegraded":"2024-09-10T21:24:53.174530+0000","last_fullsized":"2024-09-10T21:24:53.174530+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:49:30.147202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174576+0000","last_change":"2024-09-10T21:24:47.867241+0000","last_active":"2024-09-10T21:24:53.174576+0000","last_peered":"2024-09-10T21:24:53.174576+0000","last_clean":"2024-09-10T21:24:53.174576+0000","last_became_active":"2024-09-10T21:24:47.866907+0000","last_became_peered":"2024-09-10T21:24:47.866907+0000","last_unstale":"2024-09-10T21:24:53.174576+0000","last_undegraded":"2024-09-10T21:24:53.174576+0000","last_fullsized":"2024-09-10T21:24:53.174576+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:07.620800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.879175+0000","last_change":"2024-09-10T21:24:46.856848+0000","last_active":"2024-09-10T21:24:52.879175+0000","last_peered":"2024-09-10T21:24:52.879175+0000","last_clean":"2024-09-10T21:24:52.879175+0000","last_became_active":"2024-09-10T21:24:46.856489+0000","last_became_peered":"2024-09-10T21:24:46.856489+0000","last_unstale":"2024-09-10T21:24:52.879175+0000","last_undegraded":"2024-09-10T21:24:52.879175+0000","last_fullsized":"2024-09-10T21:24:52.879175+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:46:55.286321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.878563+0000","last_change":"2024-09-10T21:24:51.901016+0000","last_active":"2024-09-10T21:24:52.878563+0000","last_peered":"2024-09-10T21:24:52.878563+0000","last_clean":"2024-09-10T21:24:52.878563+0000","last_became_active":"2024-09-10T21:24:51.900479+0000","last_became_peered":"2024-09-10T21:24:51.900479+0000","last_unstale":"2024-09-10T21:24:52.878563+0000","last_undegraded":"2024-09-10T21:24:52.878563+0000","last_fullsized":"2024-09-10T21:24:52.878563+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:36.741810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.886797+0000","last_change":"2024-09-10T21:24:47.858753+0000","last_active":"2024-09-10T21:24:52.886797+0000","last_peered":"2024-09-10T21:24:52.886797+0000","last_clean":"2024-09-10T21:24:52.886797+0000","last_became_active":"2024-09-10T21:24:47.858438+0000","last_became_peered":"2024-09-10T21:24:47.858438+0000","last_unstale":"2024-09-10T21:24:52.886797+0000","last_undegraded":"2024-09-10T21:24:52.886797+0000","last_fullsized":"2024-09-10T21:24:52.886797+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:12:09.198013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.175120+0000","last_change":"2024-09-10T21:24:46.863610+0000","last_active":"2024-09-10T21:24:53.175120+0000","last_peered":"2024-09-10T21:24:53.175120+0000","last_clean":"2024-09-10T21:24:53.175120+0000","last_became_active":"2024-09-10T21:24:46.863150+0000","last_became_peered":"2024-09-10T21:24:46.863150+0000","last_unstale":"2024-09-10T21:24:53.175120+0000","last_undegraded":"2024-09-10T21:24:53.175120+0000","last_fullsized":"2024-09-10T21:24:53.175120+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:27:35.473995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.878912+0000","last_change":"2024-09-10T21:24:51.907366+0000","last_active":"2024-09-10T21:24:52.878912+0000","last_peered":"2024-09-10T21:24:52.878912+0000","last_clean":"2024-09-10T21:24:52.878912+0000","last_became_active":"2024-09-10T21:24:51.906888+0000","last_became_peered":"2024-09-10T21:24:51.906888+0000","last_unstale":"2024-09-10T21:24:52.878912+0000","last_undegraded":"2024-09-10T21:24:52.878912+0000","last_fullsized":"2024-09-10T21:24:52.878912+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:24:54.278514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.878445+0000","last_change":"2024-09-10T21:24:47.850155+0000","last_active":"2024-09-10T21:24:52.878445+0000","last_peered":"2024-09-10T21:24:52.878445+0000","last_clean":"2024-09-10T21:24:52.878445+0000","last_became_active":"2024-09-10T21:24:47.849846+0000","last_became_peered":"2024-09-10T21:24:47.849846+0000","last_unstale":"2024-09-10T21:24:52.878445+0000","last_undegraded":"2024-09-10T21:24:52.878445+0000","last_fullsized":"2024-09-10T21:24:52.878445+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:13:40.032808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.173931+0000","last_change":"2024-09-10T21:24:46.851800+0000","last_active":"2024-09-10T21:24:53.173931+0000","last_peered":"2024-09-10T21:24:53.173931+0000","last_clean":"2024-09-10T21:24:53.173931+0000","last_became_active":"2024-09-10T21:24:46.851200+0000","last_became_peered":"2024-09-10T21:24:46.851200+0000","last_unstale":"2024-09-10T21:24:53.173931+0000","last_undegraded":"2024-09-10T21:24:53.173931+0000","last_fullsized":"2024-09-10T21:24:53.173931+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:00:22.839361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.879296+0000","last_change":"2024-09-10T21:24:51.902529+0000","last_active":"2024-09-10T21:24:52.879296+0000","last_peered":"2024-09-10T21:24:52.879296+0000","last_clean":"2024-09-10T21:24:52.879296+0000","last_became_active":"2024-09-10T21:24:51.901141+0000","last_became_peered":"2024-09-10T21:24:51.901141+0000","last_unstale":"2024-09-10T21:24:52.879296+0000","last_undegraded":"2024-09-10T21:24:52.879296+0000","last_fullsized":"2024-09-10T21:24:52.879296+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:08:40.203348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.173878+0000","last_change":"2024-09-10T21:24:47.846582+0000","last_active":"2024-09-10T21:24:53.173878+0000","last_peered":"2024-09-10T21:24:53.173878+0000","last_clean":"2024-09-10T21:24:53.173878+0000","last_became_active":"2024-09-10T21:24:47.846358+0000","last_became_peered":"2024-09-10T21:24:47.846358+0000","last_unstale":"2024-09-10T21:24:53.173878+0000","last_undegraded":"2024-09-10T21:24:53.173878+0000","last_fullsized":"2024-09-10T21:24:53.173878+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:58:53.698584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.173773+0000","last_change":"2024-09-10T21:24:46.870908+0000","last_active":"2024-09-10T21:24:53.173773+0000","last_peered":"2024-09-10T21:24:53.173773+0000","last_clean":"2024-09-10T21:24:53.173773+0000","last_became_active":"2024-09-10T21:24:46.868375+0000","last_became_peered":"2024-09-10T21:24:46.868375+0000","last_unstale":"2024-09-10T21:24:53.173773+0000","last_undegraded":"2024-09-10T21:24:53.173773+0000","last_fullsized":"2024-09-10T21:24:53.173773+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:37:12.332013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174362+0000","last_change":"2024-09-10T21:24:52.069018+0000","last_active":"2024-09-10T21:24:53.174362+0000","last_peered":"2024-09-10T21:24:53.174362+0000","last_clean":"2024-09-10T21:24:53.174362+0000","last_became_active":"2024-09-10T21:24:52.067308+0000","last_became_peered":"2024-09-10T21:24:52.067308+0000","last_unstale":"2024-09-10T21:24:53.174362+0000","last_undegraded":"2024-09-10T21:24:53.174362+0000","last_fullsized":"2024-09-10T21:24:53.174362+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:07:25.162168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.884767+0000","last_change":"2024-09-10T21:24:47.871542+0000","last_active":"2024-09-10T21:24:52.884767+0000","last_peered":"2024-09-10T21:24:52.884767+0000","last_clean":"2024-09-10T21:24:52.884767+0000","last_became_active":"2024-09-10T21:24:47.871128+0000","last_became_peered":"2024-09-10T21:24:47.871128+0000","last_unstale":"2024-09-10T21:24:52.884767+0000","last_undegraded":"2024-09-10T21:24:52.884767+0000","last_fullsized":"2024-09-10T21:24:52.884767+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:51:22.876126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174204+0000","last_change":"2024-09-10T21:24:46.859218+0000","last_active":"2024-09-10T21:24:53.174204+0000","last_peered":"2024-09-10T21:24:53.174204+0000","last_clean":"2024-09-10T21:24:53.174204+0000","last_became_active":"2024-09-10T21:24:46.853190+0000","last_became_peered":"2024-09-10T21:24:46.853190+0000","last_unstale":"2024-09-10T21:24:53.174204+0000","last_undegraded":"2024-09-10T21:24:53.174204+0000","last_fullsized":"2024-09-10T21:24:53.174204+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:26:34.848759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":119,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.173768+0000","last_change":"2024-09-10T21:24:37.216542+0000","last_active":"2024-09-10T21:24:53.173768+0000","last_peered":"2024-09-10T21:24:53.173768+0000","last_clean":"2024-09-10T21:24:53.173768+0000","last_became_active":"2024-09-10T21:24:37.215443+0000","last_became_peered":"2024-09-10T21:24:37.215443+0000","last_unstale":"2024-09-10T21:24:53.173768+0000","last_undegraded":"2024-09-10T21:24:53.173768+0000","last_fullsized":"2024-09-10T21:24:53.173768+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:35.877801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:35.877801+0000","last_clean_scrub_stamp":"2024-09-10T21:24:35.877801+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:14:20.260329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.884231+0000","last_change":"2024-09-10T21:24:52.067645+0000","last_active":"2024-09-10T21:24:52.884231+0000","last_peered":"2024-09-10T21:24:52.884231+0000","last_clean":"2024-09-10T21:24:52.884231+0000","last_became_active":"2024-09-10T21:24:52.067148+0000","last_became_peered":"2024-09-10T21:24:52.067148+0000","last_unstale":"2024-09-10T21:24:52.884231+0000","last_undegraded":"2024-09-10T21:24:52.884231+0000","last_fullsized":"2024-09-10T21:24:52.884231+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:54:42.435011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.175128+0000","last_change":"2024-09-10T21:24:47.867282+0000","last_active":"2024-09-10T21:24:53.175128+0000","last_peered":"2024-09-10T21:24:53.175128+0000","last_clean":"2024-09-10T21:24:53.175128+0000","last_became_active":"2024-09-10T21:24:47.865582+0000","last_became_peered":"2024-09-10T21:24:47.865582+0000","last_unstale":"2024-09-10T21:24:53.175128+0000","last_undegraded":"2024-09-10T21:24:53.175128+0000","last_fullsized":"2024-09-10T21:24:53.175128+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:38:55.009670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174808+0000","last_change":"2024-09-10T21:24:46.859532+0000","last_active":"2024-09-10T21:24:53.174808+0000","last_peered":"2024-09-10T21:24:53.174808+0000","last_clean":"2024-09-10T21:24:53.174808+0000","last_became_active":"2024-09-10T21:24:46.859144+0000","last_became_peered":"2024-09-10T21:24:46.859144+0000","last_unstale":"2024-09-10T21:24:53.174808+0000","last_undegraded":"2024-09-10T21:24:53.174808+0000","last_fullsized":"2024-09-10T21:24:53.174808+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:48:41.992441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.883627+0000","last_change":"2024-09-10T21:24:52.061029+0000","last_active":"2024-09-10T21:24:52.883627+0000","last_peered":"2024-09-10T21:24:52.883627+0000","last_clean":"2024-09-10T21:24:52.883627+0000","last_became_active":"2024-09-10T21:24:52.060605+0000","last_became_peered":"2024-09-10T21:24:52.060605+0000","last_unstale":"2024-09-10T21:24:52.883627+0000","last_undegraded":"2024-09-10T21:24:52.883627+0000","last_fullsized":"2024-09-10T21:24:52.883627+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:09:48.077266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174744+0000","last_change":"2024-09-10T21:24:47.866927+0000","last_active":"2024-09-10T21:24:53.174744+0000","last_peered":"2024-09-10T21:24:53.174744+0000","last_clean":"2024-09-10T21:24:53.174744+0000","last_became_active":"2024-09-10T21:24:47.866650+0000","last_became_peered":"2024-09-10T21:24:47.866650+0000","last_unstale":"2024-09-10T21:24:53.174744+0000","last_undegraded":"2024-09-10T21:24:53.174744+0000","last_fullsized":"2024-09-10T21:24:53.174744+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:47:37.304624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"24'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.879000+0000","last_change":"2024-09-10T21:24:46.857082+0000","last_active":"2024-09-10T21:24:52.879000+0000","last_peered":"2024-09-10T21:24:52.879000+0000","last_clean":"2024-09-10T21:24:52.879000+0000","last_became_active":"2024-09-10T21:24:46.856799+0000","last_became_peered":"2024-09-10T21:24:46.856799+0000","last_unstale":"2024-09-10T21:24:52.879000+0000","last_undegraded":"2024-09-10T21:24:52.879000+0000","last_fullsized":"2024-09-10T21:24:52.879000+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:40:33.632846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174743+0000","last_change":"2024-09-10T21:24:51.911880+0000","last_active":"2024-09-10T21:24:53.174743+0000","last_peered":"2024-09-10T21:24:53.174743+0000","last_clean":"2024-09-10T21:24:53.174743+0000","last_became_active":"2024-09-10T21:24:51.911088+0000","last_became_peered":"2024-09-10T21:24:51.911088+0000","last_unstale":"2024-09-10T21:24:53.174743+0000","last_undegraded":"2024-09-10T21:24:53.174743+0000","last_fullsized":"2024-09-10T21:24:53.174743+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:38:57.030655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.175370+0000","last_change":"2024-09-10T21:24:47.876946+0000","last_active":"2024-09-10T21:24:53.175370+0000","last_peered":"2024-09-10T21:24:53.175370+0000","last_clean":"2024-09-10T21:24:53.175370+0000","last_became_active":"2024-09-10T21:24:47.876690+0000","last_became_peered":"2024-09-10T21:24:47.876690+0000","last_unstale":"2024-09-10T21:24:53.175370+0000","last_undegraded":"2024-09-10T21:24:53.175370+0000","last_fullsized":"2024-09-10T21:24:53.175370+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:14:06.368275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"24'2","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.175105+0000","last_change":"2024-09-10T21:24:46.855248+0000","last_active":"2024-09-10T21:24:53.175105+0000","last_peered":"2024-09-10T21:24:53.175105+0000","last_clean":"2024-09-10T21:24:53.175105+0000","last_became_active":"2024-09-10T21:24:46.854930+0000","last_became_peered":"2024-09-10T21:24:46.854930+0000","last_unstale":"2024-09-10T21:24:53.175105+0000","last_undegraded":"2024-09-10T21:24:53.175105+0000","last_fullsized":"2024-09-10T21:24:53.175105+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:46:55.286321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.173528+0000","last_change":"2024-09-10T21:24:52.063112+0000","last_active":"2024-09-10T21:24:53.173528+0000","last_peered":"2024-09-10T21:24:53.173528+0000","last_clean":"2024-09-10T21:24:53.173528+0000","last_became_active":"2024-09-10T21:24:52.062395+0000","last_became_peered":"2024-09-10T21:24:52.062395+0000","last_unstale":"2024-09-10T21:24:53.173528+0000","last_undegraded":"2024-09-10T21:24:53.173528+0000","last_fullsized":"2024-09-10T21:24:53.173528+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:21:53.080013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.175254+0000","last_change":"2024-09-10T21:24:47.858593+0000","last_active":"2024-09-10T21:24:53.175254+0000","last_peered":"2024-09-10T21:24:53.175254+0000","last_clean":"2024-09-10T21:24:53.175254+0000","last_became_active":"2024-09-10T21:24:47.858216+0000","last_became_peered":"2024-09-10T21:24:47.858216+0000","last_unstale":"2024-09-10T21:24:53.175254+0000","last_undegraded":"2024-09-10T21:24:53.175254+0000","last_fullsized":"2024-09-10T21:24:53.175254+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:42.961291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"24'1","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.886950+0000","last_change":"2024-09-10T21:24:46.860330+0000","last_active":"2024-09-10T21:24:52.886950+0000","last_peered":"2024-09-10T21:24:52.886950+0000","last_clean":"2024-09-10T21:24:52.886950+0000","last_became_active":"2024-09-10T21:24:46.860053+0000","last_became_peered":"2024-09-10T21:24:46.860053+0000","last_unstale":"2024-09-10T21:24:52.886950+0000","last_undegraded":"2024-09-10T21:24:52.886950+0000","last_fullsized":"2024-09-10T21:24:52.886950+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:58:53.177740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.883701+0000","last_change":"2024-09-10T21:24:51.901673+0000","last_active":"2024-09-10T21:24:52.883701+0000","last_peered":"2024-09-10T21:24:52.883701+0000","last_clean":"2024-09-10T21:24:52.883701+0000","last_became_active":"2024-09-10T21:24:51.901222+0000","last_became_peered":"2024-09-10T21:24:51.901222+0000","last_unstale":"2024-09-10T21:24:52.883701+0000","last_undegraded":"2024-09-10T21:24:52.883701+0000","last_fullsized":"2024-09-10T21:24:52.883701+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:07:52.120299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.173790+0000","last_change":"2024-09-10T21:24:47.870589+0000","last_active":"2024-09-10T21:24:53.173790+0000","last_peered":"2024-09-10T21:24:53.173790+0000","last_clean":"2024-09-10T21:24:53.173790+0000","last_became_active":"2024-09-10T21:24:47.861834+0000","last_became_peered":"2024-09-10T21:24:47.861834+0000","last_unstale":"2024-09-10T21:24:53.173790+0000","last_undegraded":"2024-09-10T21:24:53.173790+0000","last_fullsized":"2024-09-10T21:24:53.173790+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:43.363013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"24'8","reported_seq":31,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.173490+0000","last_change":"2024-09-10T21:24:46.852064+0000","last_active":"2024-09-10T21:24:53.173490+0000","last_peered":"2024-09-10T21:24:53.173490+0000","last_clean":"2024-09-10T21:24:53.173490+0000","last_became_active":"2024-09-10T21:24:46.851776+0000","last_became_peered":"2024-09-10T21:24:46.851776+0000","last_unstale":"2024-09-10T21:24:53.173490+0000","last_undegraded":"2024-09-10T21:24:53.173490+0000","last_fullsized":"2024-09-10T21:24:53.173490+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:26:24.371844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174153+0000","last_change":"2024-09-10T21:24:51.904449+0000","last_active":"2024-09-10T21:24:53.174153+0000","last_peered":"2024-09-10T21:24:53.174153+0000","last_clean":"2024-09-10T21:24:53.174153+0000","last_became_active":"2024-09-10T21:24:51.903533+0000","last_became_peered":"2024-09-10T21:24:51.903533+0000","last_unstale":"2024-09-10T21:24:53.174153+0000","last_undegraded":"2024-09-10T21:24:53.174153+0000","last_fullsized":"2024-09-10T21:24:53.174153+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:11.654935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.883479+0000","last_change":"2024-09-10T21:24:47.857736+0000","last_active":"2024-09-10T21:24:52.883479+0000","last_peered":"2024-09-10T21:24:52.883479+0000","last_clean":"2024-09-10T21:24:52.883479+0000","last_became_active":"2024-09-10T21:24:47.857437+0000","last_became_peered":"2024-09-10T21:24:47.857437+0000","last_unstale":"2024-09-10T21:24:52.883479+0000","last_undegraded":"2024-09-10T21:24:52.883479+0000","last_fullsized":"2024-09-10T21:24:52.883479+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:13:40.032808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.883979+0000","last_change":"2024-09-10T21:24:46.860662+0000","last_active":"2024-09-10T21:24:52.883979+0000","last_peered":"2024-09-10T21:24:52.883979+0000","last_clean":"2024-09-10T21:24:52.883979+0000","last_became_active":"2024-09-10T21:24:46.859796+0000","last_became_peered":"2024-09-10T21:24:46.859796+0000","last_unstale":"2024-09-10T21:24:52.883979+0000","last_undegraded":"2024-09-10T21:24:52.883979+0000","last_fullsized":"2024-09-10T21:24:52.883979+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:48:27.758481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.885543+0000","last_change":"2024-09-10T21:24:52.069770+0000","last_active":"2024-09-10T21:24:52.885543+0000","last_peered":"2024-09-10T21:24:52.885543+0000","last_clean":"2024-09-10T21:24:52.885543+0000","last_became_active":"2024-09-10T21:24:52.067409+0000","last_became_peered":"2024-09-10T21:24:52.067409+0000","last_unstale":"2024-09-10T21:24:52.885543+0000","last_undegraded":"2024-09-10T21:24:52.885543+0000","last_fullsized":"2024-09-10T21:24:52.885543+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:23:39.622612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174136+0000","last_change":"2024-09-10T21:24:47.858692+0000","last_active":"2024-09-10T21:24:53.174136+0000","last_peered":"2024-09-10T21:24:53.174136+0000","last_clean":"2024-09-10T21:24:53.174136+0000","last_became_active":"2024-09-10T21:24:47.858343+0000","last_became_peered":"2024-09-10T21:24:47.858343+0000","last_unstale":"2024-09-10T21:24:53.174136+0000","last_undegraded":"2024-09-10T21:24:53.174136+0000","last_fullsized":"2024-09-10T21:24:53.174136+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:04:36.661791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.175219+0000","last_change":"2024-09-10T21:24:46.864592+0000","last_active":"2024-09-10T21:24:53.175219+0000","last_peered":"2024-09-10T21:24:53.175219+0000","last_clean":"2024-09-10T21:24:53.175219+0000","last_became_active":"2024-09-10T21:24:46.858696+0000","last_became_peered":"2024-09-10T21:24:46.858696+0000","last_unstale":"2024-09-10T21:24:53.175219+0000","last_undegraded":"2024-09-10T21:24:53.175219+0000","last_fullsized":"2024-09-10T21:24:53.175219+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:54:19.356867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.885074+0000","last_change":"2024-09-10T21:24:52.061572+0000","last_active":"2024-09-10T21:24:52.885074+0000","last_peered":"2024-09-10T21:24:52.885074+0000","last_clean":"2024-09-10T21:24:52.885074+0000","last_became_active":"2024-09-10T21:24:52.061061+0000","last_became_peered":"2024-09-10T21:24:52.061061+0000","last_unstale":"2024-09-10T21:24:52.885074+0000","last_undegraded":"2024-09-10T21:24:52.885074+0000","last_fullsized":"2024-09-10T21:24:52.885074+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:04:36.399811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174340+0000","last_change":"2024-09-10T21:24:47.857699+0000","last_active":"2024-09-10T21:24:53.174340+0000","last_peered":"2024-09-10T21:24:53.174340+0000","last_clean":"2024-09-10T21:24:53.174340+0000","last_became_active":"2024-09-10T21:24:47.857429+0000","last_became_peered":"2024-09-10T21:24:47.857429+0000","last_unstale":"2024-09-10T21:24:53.174340+0000","last_undegraded":"2024-09-10T21:24:53.174340+0000","last_fullsized":"2024-09-10T21:24:53.174340+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:07:11.880228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"24'2","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174839+0000","last_change":"2024-09-10T21:24:46.859753+0000","last_active":"2024-09-10T21:24:53.174839+0000","last_peered":"2024-09-10T21:24:53.174839+0000","last_clean":"2024-09-10T21:24:53.174839+0000","last_became_active":"2024-09-10T21:24:46.859299+0000","last_became_peered":"2024-09-10T21:24:46.859299+0000","last_unstale":"2024-09-10T21:24:53.174839+0000","last_undegraded":"2024-09-10T21:24:53.174839+0000","last_fullsized":"2024-09-10T21:24:53.174839+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:06:41.965052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174673+0000","last_change":"2024-09-10T21:24:52.064718+0000","last_active":"2024-09-10T21:24:53.174673+0000","last_peered":"2024-09-10T21:24:53.174673+0000","last_clean":"2024-09-10T21:24:53.174673+0000","last_became_active":"2024-09-10T21:24:52.062327+0000","last_became_peered":"2024-09-10T21:24:52.062327+0000","last_unstale":"2024-09-10T21:24:53.174673+0000","last_undegraded":"2024-09-10T21:24:53.174673+0000","last_fullsized":"2024-09-10T21:24:53.174673+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:43:17.173095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.175009+0000","last_change":"2024-09-10T21:24:47.870766+0000","last_active":"2024-09-10T21:24:53.175009+0000","last_peered":"2024-09-10T21:24:53.175009+0000","last_clean":"2024-09-10T21:24:53.175009+0000","last_became_active":"2024-09-10T21:24:47.870276+0000","last_became_peered":"2024-09-10T21:24:47.870276+0000","last_unstale":"2024-09-10T21:24:53.175009+0000","last_undegraded":"2024-09-10T21:24:53.175009+0000","last_fullsized":"2024-09-10T21:24:53.175009+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:54:38.400877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"25'9","reported_seq":31,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:25:05.421191+0000","last_change":"2024-09-10T21:24:46.867790+0000","last_active":"2024-09-10T21:25:05.421191+0000","last_peered":"2024-09-10T21:25:05.421191+0000","last_clean":"2024-09-10T21:25:05.421191+0000","last_became_active":"2024-09-10T21:24:46.866899+0000","last_became_peered":"2024-09-10T21:24:46.866899+0000","last_unstale":"2024-09-10T21:25:05.421191+0000","last_undegraded":"2024-09-10T21:25:05.421191+0000","last_fullsized":"2024-09-10T21:25:05.421191+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:25:04.923730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.878572+0000","last_change":"2024-09-10T21:24:51.899803+0000","last_active":"2024-09-10T21:24:52.878572+0000","last_peered":"2024-09-10T21:24:52.878572+0000","last_clean":"2024-09-10T21:24:52.878572+0000","last_became_active":"2024-09-10T21:24:51.895398+0000","last_became_peered":"2024-09-10T21:24:51.895398+0000","last_unstale":"2024-09-10T21:24:52.878572+0000","last_undegraded":"2024-09-10T21:24:52.878572+0000","last_fullsized":"2024-09-10T21:24:52.878572+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:52:34.689860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.886661+0000","last_change":"2024-09-10T21:24:47.857544+0000","last_active":"2024-09-10T21:24:52.886661+0000","last_peered":"2024-09-10T21:24:52.886661+0000","last_clean":"2024-09-10T21:24:52.886661+0000","last_became_active":"2024-09-10T21:24:47.857284+0000","last_became_peered":"2024-09-10T21:24:47.857284+0000","last_unstale":"2024-09-10T21:24:52.886661+0000","last_undegraded":"2024-09-10T21:24:52.886661+0000","last_fullsized":"2024-09-10T21:24:52.886661+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:10:22.747606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"24'3","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.878942+0000","last_change":"2024-09-10T21:24:46.859633+0000","last_active":"2024-09-10T21:24:52.878942+0000","last_peered":"2024-09-10T21:24:52.878942+0000","last_clean":"2024-09-10T21:24:52.878942+0000","last_became_active":"2024-09-10T21:24:46.859332+0000","last_became_peered":"2024-09-10T21:24:46.859332+0000","last_unstale":"2024-09-10T21:24:52.878942+0000","last_undegraded":"2024-09-10T21:24:52.878942+0000","last_fullsized":"2024-09-10T21:24:52.878942+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:56:46.975225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.886734+0000","last_change":"2024-09-10T21:24:51.899100+0000","last_active":"2024-09-10T21:24:52.886734+0000","last_peered":"2024-09-10T21:24:52.886734+0000","last_clean":"2024-09-10T21:24:52.886734+0000","last_became_active":"2024-09-10T21:24:51.898642+0000","last_became_peered":"2024-09-10T21:24:51.898642+0000","last_unstale":"2024-09-10T21:24:52.886734+0000","last_undegraded":"2024-09-10T21:24:52.886734+0000","last_fullsized":"2024-09-10T21:24:52.886734+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:06.394081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174291+0000","last_change":"2024-09-10T21:24:47.867890+0000","last_active":"2024-09-10T21:24:53.174291+0000","last_peered":"2024-09-10T21:24:53.174291+0000","last_clean":"2024-09-10T21:24:53.174291+0000","last_became_active":"2024-09-10T21:24:47.865016+0000","last_became_peered":"2024-09-10T21:24:47.865016+0000","last_unstale":"2024-09-10T21:24:53.174291+0000","last_undegraded":"2024-09-10T21:24:53.174291+0000","last_fullsized":"2024-09-10T21:24:53.174291+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:53:09.082512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174673+0000","last_change":"2024-09-10T21:24:46.861400+0000","last_active":"2024-09-10T21:24:53.174673+0000","last_peered":"2024-09-10T21:24:53.174673+0000","last_clean":"2024-09-10T21:24:53.174673+0000","last_became_active":"2024-09-10T21:24:46.861136+0000","last_became_peered":"2024-09-10T21:24:46.861136+0000","last_unstale":"2024-09-10T21:24:53.174673+0000","last_undegraded":"2024-09-10T21:24:53.174673+0000","last_fullsized":"2024-09-10T21:24:53.174673+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:35:31.682448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.884671+0000","last_change":"2024-09-10T21:24:51.907850+0000","last_active":"2024-09-10T21:24:52.884671+0000","last_peered":"2024-09-10T21:24:52.884671+0000","last_clean":"2024-09-10T21:24:52.884671+0000","last_became_active":"2024-09-10T21:24:51.907062+0000","last_became_peered":"2024-09-10T21:24:51.907062+0000","last_unstale":"2024-09-10T21:24:52.884671+0000","last_undegraded":"2024-09-10T21:24:52.884671+0000","last_fullsized":"2024-09-10T21:24:52.884671+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:32:52.519627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174521+0000","last_change":"2024-09-10T21:24:47.861800+0000","last_active":"2024-09-10T21:24:53.174521+0000","last_peered":"2024-09-10T21:24:53.174521+0000","last_clean":"2024-09-10T21:24:53.174521+0000","last_became_active":"2024-09-10T21:24:47.861498+0000","last_became_peered":"2024-09-10T21:24:47.861498+0000","last_unstale":"2024-09-10T21:24:53.174521+0000","last_undegraded":"2024-09-10T21:24:53.174521+0000","last_fullsized":"2024-09-10T21:24:53.174521+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:07:21.128033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.885153+0000","last_change":"2024-09-10T21:24:46.871677+0000","last_active":"2024-09-10T21:24:52.885153+0000","last_peered":"2024-09-10T21:24:52.885153+0000","last_clean":"2024-09-10T21:24:52.885153+0000","last_became_active":"2024-09-10T21:24:46.871367+0000","last_became_peered":"2024-09-10T21:24:46.871367+0000","last_unstale":"2024-09-10T21:24:52.885153+0000","last_undegraded":"2024-09-10T21:24:52.885153+0000","last_fullsized":"2024-09-10T21:24:52.885153+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:40:33.632846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.884646+0000","last_change":"2024-09-10T21:24:52.068637+0000","last_active":"2024-09-10T21:24:52.884646+0000","last_peered":"2024-09-10T21:24:52.884646+0000","last_clean":"2024-09-10T21:24:52.884646+0000","last_became_active":"2024-09-10T21:24:52.068348+0000","last_became_peered":"2024-09-10T21:24:52.068348+0000","last_unstale":"2024-09-10T21:24:52.884646+0000","last_undegraded":"2024-09-10T21:24:52.884646+0000","last_fullsized":"2024-09-10T21:24:52.884646+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:43:17.173095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174107+0000","last_change":"2024-09-10T21:24:52.067732+0000","last_active":"2024-09-10T21:24:53.174107+0000","last_peered":"2024-09-10T21:24:53.174107+0000","last_clean":"2024-09-10T21:24:53.174107+0000","last_became_active":"2024-09-10T21:24:52.067330+0000","last_became_peered":"2024-09-10T21:24:52.067330+0000","last_unstale":"2024-09-10T21:24:53.174107+0000","last_undegraded":"2024-09-10T21:24:53.174107+0000","last_fullsized":"2024-09-10T21:24:53.174107+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:03:41.110214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.878246+0000","last_change":"2024-09-10T21:24:47.862774+0000","last_active":"2024-09-10T21:24:52.878246+0000","last_peered":"2024-09-10T21:24:52.878246+0000","last_clean":"2024-09-10T21:24:52.878246+0000","last_became_active":"2024-09-10T21:24:47.862556+0000","last_became_peered":"2024-09-10T21:24:47.862556+0000","last_unstale":"2024-09-10T21:24:52.878246+0000","last_undegraded":"2024-09-10T21:24:52.878246+0000","last_fullsized":"2024-09-10T21:24:52.878246+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:17:52.343553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.173610+0000","last_change":"2024-09-10T21:24:46.862847+0000","last_active":"2024-09-10T21:24:53.173610+0000","last_peered":"2024-09-10T21:24:53.173610+0000","last_clean":"2024-09-10T21:24:53.173610+0000","last_became_active":"2024-09-10T21:24:46.862540+0000","last_became_peered":"2024-09-10T21:24:46.862540+0000","last_unstale":"2024-09-10T21:24:53.173610+0000","last_undegraded":"2024-09-10T21:24:53.173610+0000","last_fullsized":"2024-09-10T21:24:53.173610+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:52:31.628568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.884678+0000","last_change":"2024-09-10T21:24:51.901224+0000","last_active":"2024-09-10T21:24:52.884678+0000","last_peered":"2024-09-10T21:24:52.884678+0000","last_clean":"2024-09-10T21:24:52.884678+0000","last_became_active":"2024-09-10T21:24:51.900610+0000","last_became_peered":"2024-09-10T21:24:51.900610+0000","last_unstale":"2024-09-10T21:24:52.884678+0000","last_undegraded":"2024-09-10T21:24:52.884678+0000","last_fullsized":"2024-09-10T21:24:52.884678+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:52:34.689860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.884489+0000","last_change":"2024-09-10T21:24:47.861255+0000","last_active":"2024-09-10T21:24:52.884489+0000","last_peered":"2024-09-10T21:24:52.884489+0000","last_clean":"2024-09-10T21:24:52.884489+0000","last_became_active":"2024-09-10T21:24:47.860930+0000","last_became_peered":"2024-09-10T21:24:47.860930+0000","last_unstale":"2024-09-10T21:24:52.884489+0000","last_undegraded":"2024-09-10T21:24:52.884489+0000","last_fullsized":"2024-09-10T21:24:52.884489+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:37:51.784258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.884796+0000","last_change":"2024-09-10T21:24:46.858561+0000","last_active":"2024-09-10T21:24:52.884796+0000","last_peered":"2024-09-10T21:24:52.884796+0000","last_clean":"2024-09-10T21:24:52.884796+0000","last_became_active":"2024-09-10T21:24:46.858265+0000","last_became_peered":"2024-09-10T21:24:46.858265+0000","last_unstale":"2024-09-10T21:24:52.884796+0000","last_undegraded":"2024-09-10T21:24:52.884796+0000","last_fullsized":"2024-09-10T21:24:52.884796+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:49:00.454609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"24'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.884271+0000","last_change":"2024-09-10T21:24:46.856886+0000","last_active":"2024-09-10T21:24:52.884271+0000","last_peered":"2024-09-10T21:24:52.884271+0000","last_clean":"2024-09-10T21:24:52.884271+0000","last_became_active":"2024-09-10T21:24:46.856506+0000","last_became_peered":"2024-09-10T21:24:46.856506+0000","last_unstale":"2024-09-10T21:24:52.884271+0000","last_undegraded":"2024-09-10T21:24:52.884271+0000","last_fullsized":"2024-09-10T21:24:52.884271+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:56:46.975225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174721+0000","last_change":"2024-09-10T21:24:47.848602+0000","last_active":"2024-09-10T21:24:53.174721+0000","last_peered":"2024-09-10T21:24:53.174721+0000","last_clean":"2024-09-10T21:24:53.174721+0000","last_became_active":"2024-09-10T21:24:47.848160+0000","last_became_peered":"2024-09-10T21:24:47.848160+0000","last_unstale":"2024-09-10T21:24:53.174721+0000","last_undegraded":"2024-09-10T21:24:53.174721+0000","last_fullsized":"2024-09-10T21:24:53.174721+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:32:50.027827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174649+0000","last_change":"2024-09-10T21:24:51.903553+0000","last_active":"2024-09-10T21:24:53.174649+0000","last_peered":"2024-09-10T21:24:53.174649+0000","last_clean":"2024-09-10T21:24:53.174649+0000","last_became_active":"2024-09-10T21:24:51.903120+0000","last_became_peered":"2024-09-10T21:24:51.903120+0000","last_unstale":"2024-09-10T21:24:53.174649+0000","last_undegraded":"2024-09-10T21:24:53.174649+0000","last_fullsized":"2024-09-10T21:24:53.174649+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:58.240380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"24'1","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.884865+0000","last_change":"2024-09-10T21:24:46.853550+0000","last_active":"2024-09-10T21:24:52.884865+0000","last_peered":"2024-09-10T21:24:52.884865+0000","last_clean":"2024-09-10T21:24:52.884865+0000","last_became_active":"2024-09-10T21:24:46.852204+0000","last_became_peered":"2024-09-10T21:24:46.852204+0000","last_unstale":"2024-09-10T21:24:52.884865+0000","last_undegraded":"2024-09-10T21:24:52.884865+0000","last_fullsized":"2024-09-10T21:24:52.884865+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:06:37.669444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.884581+0000","last_change":"2024-09-10T21:24:47.849151+0000","last_active":"2024-09-10T21:24:52.884581+0000","last_peered":"2024-09-10T21:24:52.884581+0000","last_clean":"2024-09-10T21:24:52.884581+0000","last_became_active":"2024-09-10T21:24:47.848851+0000","last_became_peered":"2024-09-10T21:24:47.848851+0000","last_unstale":"2024-09-10T21:24:52.884581+0000","last_undegraded":"2024-09-10T21:24:52.884581+0000","last_fullsized":"2024-09-10T21:24:52.884581+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:39:25.828558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.885276+0000","last_change":"2024-09-10T21:24:51.907549+0000","last_active":"2024-09-10T21:24:52.885276+0000","last_peered":"2024-09-10T21:24:52.885276+0000","last_clean":"2024-09-10T21:24:52.885276+0000","last_became_active":"2024-09-10T21:24:51.907082+0000","last_became_peered":"2024-09-10T21:24:51.907082+0000","last_unstale":"2024-09-10T21:24:52.885276+0000","last_undegraded":"2024-09-10T21:24:52.885276+0000","last_fullsized":"2024-09-10T21:24:52.885276+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:44:30.813754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.883360+0000","last_change":"2024-09-10T21:24:47.857804+0000","last_active":"2024-09-10T21:24:52.883360+0000","last_peered":"2024-09-10T21:24:52.883360+0000","last_clean":"2024-09-10T21:24:52.883360+0000","last_became_active":"2024-09-10T21:24:47.857501+0000","last_became_peered":"2024-09-10T21:24:47.857501+0000","last_unstale":"2024-09-10T21:24:52.883360+0000","last_undegraded":"2024-09-10T21:24:52.883360+0000","last_fullsized":"2024-09-10T21:24:52.883360+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:19:20.807422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.173632+0000","last_change":"2024-09-10T21:24:47.867803+0000","last_active":"2024-09-10T21:24:53.173632+0000","last_peered":"2024-09-10T21:24:53.173632+0000","last_clean":"2024-09-10T21:24:53.173632+0000","last_became_active":"2024-09-10T21:24:47.867504+0000","last_became_peered":"2024-09-10T21:24:47.867504+0000","last_unstale":"2024-09-10T21:24:53.173632+0000","last_undegraded":"2024-09-10T21:24:53.173632+0000","last_fullsized":"2024-09-10T21:24:53.173632+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:53:30.420454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174959+0000","last_change":"2024-09-10T21:24:47.868901+0000","last_active":"2024-09-10T21:24:53.174959+0000","last_peered":"2024-09-10T21:24:53.174959+0000","last_clean":"2024-09-10T21:24:53.174959+0000","last_became_active":"2024-09-10T21:24:47.868168+0000","last_became_peered":"2024-09-10T21:24:47.868168+0000","last_unstale":"2024-09-10T21:24:53.174959+0000","last_undegraded":"2024-09-10T21:24:53.174959+0000","last_fullsized":"2024-09-10T21:24:53.174959+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:06:21.461089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174633+0000","last_change":"2024-09-10T21:24:47.864561+0000","last_active":"2024-09-10T21:24:53.174633+0000","last_peered":"2024-09-10T21:24:53.174633+0000","last_clean":"2024-09-10T21:24:53.174633+0000","last_became_active":"2024-09-10T21:24:47.864100+0000","last_became_peered":"2024-09-10T21:24:47.864100+0000","last_unstale":"2024-09-10T21:24:53.174633+0000","last_undegraded":"2024-09-10T21:24:53.174633+0000","last_fullsized":"2024-09-10T21:24:53.174633+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:34:13.053494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.885174+0000","last_change":"2024-09-10T21:24:47.864303+0000","last_active":"2024-09-10T21:24:52.885174+0000","last_peered":"2024-09-10T21:24:52.885174+0000","last_clean":"2024-09-10T21:24:52.885174+0000","last_became_active":"2024-09-10T21:24:47.863970+0000","last_became_peered":"2024-09-10T21:24:47.863970+0000","last_unstale":"2024-09-10T21:24:52.885174+0000","last_undegraded":"2024-09-10T21:24:52.885174+0000","last_fullsized":"2024-09-10T21:24:52.885174+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:46:36.918539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.878793+0000","last_change":"2024-09-10T21:24:47.860734+0000","last_active":"2024-09-10T21:24:52.878793+0000","last_peered":"2024-09-10T21:24:52.878793+0000","last_clean":"2024-09-10T21:24:52.878793+0000","last_became_active":"2024-09-10T21:24:47.860471+0000","last_became_peered":"2024-09-10T21:24:47.860471+0000","last_unstale":"2024-09-10T21:24:52.878793+0000","last_undegraded":"2024-09-10T21:24:52.878793+0000","last_fullsized":"2024-09-10T21:24:52.878793+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:54:13.200082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.885049+0000","last_change":"2024-09-10T21:24:47.872504+0000","last_active":"2024-09-10T21:24:52.885049+0000","last_peered":"2024-09-10T21:24:52.885049+0000","last_clean":"2024-09-10T21:24:52.885049+0000","last_became_active":"2024-09-10T21:24:47.872218+0000","last_became_peered":"2024-09-10T21:24:47.872218+0000","last_unstale":"2024-09-10T21:24:52.885049+0000","last_undegraded":"2024-09-10T21:24:52.885049+0000","last_fullsized":"2024-09-10T21:24:52.885049+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:21:32.400520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.885478+0000","last_change":"2024-09-10T21:24:47.862167+0000","last_active":"2024-09-10T21:24:52.885478+0000","last_peered":"2024-09-10T21:24:52.885478+0000","last_clean":"2024-09-10T21:24:52.885478+0000","last_became_active":"2024-09-10T21:24:47.859649+0000","last_became_peered":"2024-09-10T21:24:47.859649+0000","last_unstale":"2024-09-10T21:24:52.885478+0000","last_undegraded":"2024-09-10T21:24:52.885478+0000","last_fullsized":"2024-09-10T21:24:52.885478+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:42:59.204284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.173544+0000","last_change":"2024-09-10T21:24:47.865322+0000","last_active":"2024-09-10T21:24:53.173544+0000","last_peered":"2024-09-10T21:24:53.173544+0000","last_clean":"2024-09-10T21:24:53.173544+0000","last_became_active":"2024-09-10T21:24:47.864933+0000","last_became_peered":"2024-09-10T21:24:47.864933+0000","last_unstale":"2024-09-10T21:24:53.173544+0000","last_undegraded":"2024-09-10T21:24:53.173544+0000","last_fullsized":"2024-09-10T21:24:53.173544+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:40:45.551723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.884760+0000","last_change":"2024-09-10T21:24:47.860155+0000","last_active":"2024-09-10T21:24:52.884760+0000","last_peered":"2024-09-10T21:24:52.884760+0000","last_clean":"2024-09-10T21:24:52.884760+0000","last_became_active":"2024-09-10T21:24:47.859857+0000","last_became_peered":"2024-09-10T21:24:47.859857+0000","last_unstale":"2024-09-10T21:24:52.884760+0000","last_undegraded":"2024-09-10T21:24:52.884760+0000","last_fullsized":"2024-09-10T21:24:52.884760+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:32:50.027827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174935+0000","last_change":"2024-09-10T21:24:47.872848+0000","last_active":"2024-09-10T21:24:53.174935+0000","last_peered":"2024-09-10T21:24:53.174935+0000","last_clean":"2024-09-10T21:24:53.174935+0000","last_became_active":"2024-09-10T21:24:47.868671+0000","last_became_peered":"2024-09-10T21:24:47.868671+0000","last_unstale":"2024-09-10T21:24:53.174935+0000","last_undegraded":"2024-09-10T21:24:53.174935+0000","last_fullsized":"2024-09-10T21:24:53.174935+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:33:59.291647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174519+0000","last_change":"2024-09-10T21:24:47.857464+0000","last_active":"2024-09-10T21:24:53.174519+0000","last_peered":"2024-09-10T21:24:53.174519+0000","last_clean":"2024-09-10T21:24:53.174519+0000","last_became_active":"2024-09-10T21:24:47.857116+0000","last_became_peered":"2024-09-10T21:24:47.857116+0000","last_unstale":"2024-09-10T21:24:53.174519+0000","last_undegraded":"2024-09-10T21:24:53.174519+0000","last_fullsized":"2024-09-10T21:24:53.174519+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:56:43.269026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.884005+0000","last_change":"2024-09-10T21:24:47.871619+0000","last_active":"2024-09-10T21:24:52.884005+0000","last_peered":"2024-09-10T21:24:52.884005+0000","last_clean":"2024-09-10T21:24:52.884005+0000","last_became_active":"2024-09-10T21:24:47.871350+0000","last_became_peered":"2024-09-10T21:24:47.871350+0000","last_unstale":"2024-09-10T21:24:52.884005+0000","last_undegraded":"2024-09-10T21:24:52.884005+0000","last_fullsized":"2024-09-10T21:24:52.884005+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:41.305116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.175386+0000","last_change":"2024-09-10T21:24:47.861352+0000","last_active":"2024-09-10T21:24:53.175386+0000","last_peered":"2024-09-10T21:24:53.175386+0000","last_clean":"2024-09-10T21:24:53.175386+0000","last_became_active":"2024-09-10T21:24:47.861007+0000","last_became_peered":"2024-09-10T21:24:47.861007+0000","last_unstale":"2024-09-10T21:24:53.175386+0000","last_undegraded":"2024-09-10T21:24:53.175386+0000","last_fullsized":"2024-09-10T21:24:53.175386+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:25.273458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.175077+0000","last_change":"2024-09-10T21:24:47.864205+0000","last_active":"2024-09-10T21:24:53.175077+0000","last_peered":"2024-09-10T21:24:53.175077+0000","last_clean":"2024-09-10T21:24:53.175077+0000","last_became_active":"2024-09-10T21:24:47.863932+0000","last_became_peered":"2024-09-10T21:24:47.863932+0000","last_unstale":"2024-09-10T21:24:53.175077+0000","last_undegraded":"2024-09-10T21:24:53.175077+0000","last_fullsized":"2024-09-10T21:24:53.175077+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:53:17.669629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.175063+0000","last_change":"2024-09-10T21:24:47.866225+0000","last_active":"2024-09-10T21:24:53.175063+0000","last_peered":"2024-09-10T21:24:53.175063+0000","last_clean":"2024-09-10T21:24:53.175063+0000","last_became_active":"2024-09-10T21:24:47.865587+0000","last_became_peered":"2024-09-10T21:24:47.865587+0000","last_unstale":"2024-09-10T21:24:53.175063+0000","last_undegraded":"2024-09-10T21:24:53.175063+0000","last_fullsized":"2024-09-10T21:24:53.175063+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:49:26.113068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.173421+0000","last_change":"2024-09-10T21:24:47.872669+0000","last_active":"2024-09-10T21:24:53.173421+0000","last_peered":"2024-09-10T21:24:53.173421+0000","last_clean":"2024-09-10T21:24:53.173421+0000","last_became_active":"2024-09-10T21:24:47.865837+0000","last_became_peered":"2024-09-10T21:24:47.865837+0000","last_unstale":"2024-09-10T21:24:53.173421+0000","last_undegraded":"2024-09-10T21:24:53.173421+0000","last_fullsized":"2024-09-10T21:24:53.173421+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:53:09.082512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.878332+0000","last_change":"2024-09-10T21:24:47.846567+0000","last_active":"2024-09-10T21:24:52.878332+0000","last_peered":"2024-09-10T21:24:52.878332+0000","last_clean":"2024-09-10T21:24:52.878332+0000","last_became_active":"2024-09-10T21:24:47.846283+0000","last_became_peered":"2024-09-10T21:24:47.846283+0000","last_unstale":"2024-09-10T21:24:52.878332+0000","last_undegraded":"2024-09-10T21:24:52.878332+0000","last_fullsized":"2024-09-10T21:24:52.878332+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:42:15.320518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174247+0000","last_change":"2024-09-10T21:24:47.844888+0000","last_active":"2024-09-10T21:24:53.174247+0000","last_peered":"2024-09-10T21:24:53.174247+0000","last_clean":"2024-09-10T21:24:53.174247+0000","last_became_active":"2024-09-10T21:24:47.844517+0000","last_became_peered":"2024-09-10T21:24:47.844517+0000","last_unstale":"2024-09-10T21:24:53.174247+0000","last_undegraded":"2024-09-10T21:24:53.174247+0000","last_fullsized":"2024-09-10T21:24:53.174247+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:00:14.988282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.884642+0000","last_change":"2024-09-10T21:24:47.862746+0000","last_active":"2024-09-10T21:24:52.884642+0000","last_peered":"2024-09-10T21:24:52.884642+0000","last_clean":"2024-09-10T21:24:52.884642+0000","last_became_active":"2024-09-10T21:24:47.862185+0000","last_became_peered":"2024-09-10T21:24:47.862185+0000","last_unstale":"2024-09-10T21:24:52.884642+0000","last_undegraded":"2024-09-10T21:24:52.884642+0000","last_fullsized":"2024-09-10T21:24:52.884642+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:06:21.461089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.886651+0000","last_change":"2024-09-10T21:24:47.867727+0000","last_active":"2024-09-10T21:24:52.886651+0000","last_peered":"2024-09-10T21:24:52.886651+0000","last_clean":"2024-09-10T21:24:52.886651+0000","last_became_active":"2024-09-10T21:24:47.861045+0000","last_became_peered":"2024-09-10T21:24:47.861045+0000","last_unstale":"2024-09-10T21:24:52.886651+0000","last_undegraded":"2024-09-10T21:24:52.886651+0000","last_fullsized":"2024-09-10T21:24:52.886651+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:08:30.065968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.884547+0000","last_change":"2024-09-10T21:24:47.867410+0000","last_active":"2024-09-10T21:24:52.884547+0000","last_peered":"2024-09-10T21:24:52.884547+0000","last_clean":"2024-09-10T21:24:52.884547+0000","last_became_active":"2024-09-10T21:24:47.867113+0000","last_became_peered":"2024-09-10T21:24:47.867113+0000","last_unstale":"2024-09-10T21:24:52.884547+0000","last_undegraded":"2024-09-10T21:24:52.884547+0000","last_fullsized":"2024-09-10T21:24:52.884547+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:49:53.502472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.884956+0000","last_change":"2024-09-10T21:24:47.867473+0000","last_active":"2024-09-10T21:24:52.884956+0000","last_peered":"2024-09-10T21:24:52.884956+0000","last_clean":"2024-09-10T21:24:52.884956+0000","last_became_active":"2024-09-10T21:24:47.867126+0000","last_became_peered":"2024-09-10T21:24:47.867126+0000","last_unstale":"2024-09-10T21:24:52.884956+0000","last_undegraded":"2024-09-10T21:24:52.884956+0000","last_fullsized":"2024-09-10T21:24:52.884956+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:54:32.307507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.175341+0000","last_change":"2024-09-10T21:24:47.865450+0000","last_active":"2024-09-10T21:24:53.175341+0000","last_peered":"2024-09-10T21:24:53.175341+0000","last_clean":"2024-09-10T21:24:53.175341+0000","last_became_active":"2024-09-10T21:24:47.865093+0000","last_became_peered":"2024-09-10T21:24:47.865093+0000","last_unstale":"2024-09-10T21:24:53.175341+0000","last_undegraded":"2024-09-10T21:24:53.175341+0000","last_fullsized":"2024-09-10T21:24:53.175341+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:12:18.854783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.884322+0000","last_change":"2024-09-10T21:24:47.868093+0000","last_active":"2024-09-10T21:24:52.884322+0000","last_peered":"2024-09-10T21:24:52.884322+0000","last_clean":"2024-09-10T21:24:52.884322+0000","last_became_active":"2024-09-10T21:24:47.867782+0000","last_became_peered":"2024-09-10T21:24:47.867782+0000","last_unstale":"2024-09-10T21:24:52.884322+0000","last_undegraded":"2024-09-10T21:24:52.884322+0000","last_fullsized":"2024-09-10T21:24:52.884322+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:34:13.053494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.173787+0000","last_change":"2024-09-10T21:24:47.862204+0000","last_active":"2024-09-10T21:24:53.173787+0000","last_peered":"2024-09-10T21:24:53.173787+0000","last_clean":"2024-09-10T21:24:53.173787+0000","last_became_active":"2024-09-10T21:24:47.861434+0000","last_became_peered":"2024-09-10T21:24:47.861434+0000","last_unstale":"2024-09-10T21:24:53.173787+0000","last_undegraded":"2024-09-10T21:24:53.173787+0000","last_fullsized":"2024-09-10T21:24:53.173787+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:50:03.245285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174760+0000","last_change":"2024-09-10T21:24:47.869248+0000","last_active":"2024-09-10T21:24:53.174760+0000","last_peered":"2024-09-10T21:24:53.174760+0000","last_clean":"2024-09-10T21:24:53.174760+0000","last_became_active":"2024-09-10T21:24:47.868503+0000","last_became_peered":"2024-09-10T21:24:47.868503+0000","last_unstale":"2024-09-10T21:24:53.174760+0000","last_undegraded":"2024-09-10T21:24:53.174760+0000","last_fullsized":"2024-09-10T21:24:53.174760+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:51:31.417845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174007+0000","last_change":"2024-09-10T21:24:47.864802+0000","last_active":"2024-09-10T21:24:53.174007+0000","last_peered":"2024-09-10T21:24:53.174007+0000","last_clean":"2024-09-10T21:24:53.174007+0000","last_became_active":"2024-09-10T21:24:47.863976+0000","last_became_peered":"2024-09-10T21:24:47.863976+0000","last_unstale":"2024-09-10T21:24:53.174007+0000","last_undegraded":"2024-09-10T21:24:53.174007+0000","last_fullsized":"2024-09-10T21:24:53.174007+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:55:08.544075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174437+0000","last_change":"2024-09-10T21:24:47.873364+0000","last_active":"2024-09-10T21:24:53.174437+0000","last_peered":"2024-09-10T21:24:53.174437+0000","last_clean":"2024-09-10T21:24:53.174437+0000","last_became_active":"2024-09-10T21:24:47.873069+0000","last_became_peered":"2024-09-10T21:24:47.873069+0000","last_unstale":"2024-09-10T21:24:53.174437+0000","last_undegraded":"2024-09-10T21:24:53.174437+0000","last_fullsized":"2024-09-10T21:24:53.174437+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:38:21.368499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174196+0000","last_change":"2024-09-10T21:24:47.856936+0000","last_active":"2024-09-10T21:24:53.174196+0000","last_peered":"2024-09-10T21:24:53.174196+0000","last_clean":"2024-09-10T21:24:53.174196+0000","last_became_active":"2024-09-10T21:24:47.854713+0000","last_became_peered":"2024-09-10T21:24:47.854713+0000","last_unstale":"2024-09-10T21:24:53.174196+0000","last_undegraded":"2024-09-10T21:24:53.174196+0000","last_fullsized":"2024-09-10T21:24:53.174196+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:46:36.918539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.878924+0000","last_change":"2024-09-10T21:24:47.846492+0000","last_active":"2024-09-10T21:24:52.878924+0000","last_peered":"2024-09-10T21:24:52.878924+0000","last_clean":"2024-09-10T21:24:52.878924+0000","last_became_active":"2024-09-10T21:24:47.846227+0000","last_became_peered":"2024-09-10T21:24:47.846227+0000","last_unstale":"2024-09-10T21:24:52.878924+0000","last_undegraded":"2024-09-10T21:24:52.878924+0000","last_fullsized":"2024-09-10T21:24:52.878924+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:13:58.432819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.175219+0000","last_change":"2024-09-10T21:24:47.871469+0000","last_active":"2024-09-10T21:24:53.175219+0000","last_peered":"2024-09-10T21:24:53.175219+0000","last_clean":"2024-09-10T21:24:53.175219+0000","last_became_active":"2024-09-10T21:24:47.871177+0000","last_became_peered":"2024-09-10T21:24:47.871177+0000","last_unstale":"2024-09-10T21:24:53.175219+0000","last_undegraded":"2024-09-10T21:24:53.175219+0000","last_fullsized":"2024-09-10T21:24:53.175219+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:19:24.226916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":66932,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509450,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":320,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":327680,"data_stored":693447,"data_compressed":33466,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509451,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":270336,"data_stored":106339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509450,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":258048,"data_stored":102497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509451,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31456,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340384,"statfs":{"total":96636764160,"available":96604553216,"internally_reserved":0,"allocated":266240,"data_stored":102657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542156,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27296,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":270336,"data_stored":102623,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542155,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":270336,"data_stored":106339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542158,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27364,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344476,"statfs":{"total":96636764160,"available":96608743424,"internally_reserved":0,"allocated":335872,"data_stored":693503,"data_compressed":33466,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574859,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":262144,"data_stored":102587,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33466,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33466,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T21:25:23.144 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-10T21:25:23.144 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-10T21:25:23.145 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-09-10T21:25:23.145 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-10T21:25:23.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:23.326+0000 7f9e87200640 1 -- 172.21.15.17:0/1499152684 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9e80152440 msgr2=0x7f9e800b3eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:23.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:23.326+0000 7f9e87200640 1 --2- 172.21.15.17:0/1499152684 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9e80152440 0x7f9e800b3eb0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f9e70009900 tx=0x7f9e7001c850 comp rx=0 tx=0).stop 2024-09-10T21:25:23.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:23.326+0000 7f9e87200640 1 -- 172.21.15.17:0/1499152684 shutdown_connections 2024-09-10T21:25:23.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:23.326+0000 7f9e87200640 1 --2- 172.21.15.17:0/1499152684 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f9e80152820 0x7f9e8015c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:23.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:23.326+0000 7f9e87200640 1 --2- 172.21.15.17:0/1499152684 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9e80152440 0x7f9e800b3eb0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:23.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:23.326+0000 7f9e87200640 1 -- 172.21.15.17:0/1499152684 >> 172.21.15.17:0/1499152684 conn(0x7f9e800702f0 msgr2=0x7f9e80070700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:23.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:23.326+0000 7f9e87200640 1 -- 172.21.15.17:0/1499152684 shutdown_connections 2024-09-10T21:25:23.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:23.326+0000 7f9e87200640 1 -- 172.21.15.17:0/1499152684 wait complete. 2024-09-10T21:25:23.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:23.326+0000 7f9e87200640 1 Processor -- start 2024-09-10T21:25:23.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:23.326+0000 7f9e87200640 1 -- start start 2024-09-10T21:25:23.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:23.330+0000 7f9e87200640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9e800cdf20 0x7f9e801574c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:23.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:23.330+0000 7f9e87200640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f9e80152440 0x7f9e80150e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:23.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:23.330+0000 7f9e87200640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9e80152820 0x7f9e801f3610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:23.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:23.330+0000 7f9e87200640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f9e800cb450 con 0x7f9e800cdf20 2024-09-10T21:25:23.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:23.330+0000 7f9e87200640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f9e8015c7c0 con 0x7f9e80152820 2024-09-10T21:25:23.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:23.330+0000 7f9e87200640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f9e800cb5d0 con 0x7f9e80152440 2024-09-10T21:25:23.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:23.330+0000 7f9e7fe00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9e800cdf20 0x7f9e801574c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:23.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:23.330+0000 7f9e7fe00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9e800cdf20 0x7f9e801574c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44702/0 (socket says 172.21.15.17:44702) 2024-09-10T21:25:23.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:23.330+0000 7f9e7fe00640 1 -- 172.21.15.17:0/4268649066 learned_addr learned my addr 172.21.15.17:0/4268649066 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:25:23.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:23.330+0000 7f9e7fe00640 1 -- 172.21.15.17:0/4268649066 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f9e80152440 msgr2=0x7f9e80150e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:23.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:23.330+0000 7f9e7fe00640 1 --2- 172.21.15.17:0/4268649066 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f9e80152440 0x7f9e80150e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:23.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:23.330+0000 7f9e7fe00640 1 -- 172.21.15.17:0/4268649066 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9e80152820 msgr2=0x7f9e801f3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:23.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:23.330+0000 7f9e7fe00640 1 --2- 172.21.15.17:0/4268649066 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9e80152820 0x7f9e801f3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:23.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:23.330+0000 7f9e7fe00640 1 -- 172.21.15.17:0/4268649066 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e8015a940 con 0x7f9e800cdf20 2024-09-10T21:25:23.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:23.330+0000 7f9e7fe00640 1 --2- 172.21.15.17:0/4268649066 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9e800cdf20 0x7f9e801574c0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f9e7001cd30 tx=0x7f9e700a6520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:23.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:23.330+0000 7f9e7cc00640 1 -- 172.21.15.17:0/4268649066 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e700ac070 con 0x7f9e800cdf20 2024-09-10T21:25:23.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:23.330+0000 7f9e7cc00640 1 -- 172.21.15.17:0/4268649066 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9e700043e0 con 0x7f9e800cdf20 2024-09-10T21:25:23.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:23.330+0000 7f9e7cc00640 1 -- 172.21.15.17:0/4268649066 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e700a6b30 con 0x7f9e800cdf20 2024-09-10T21:25:23.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:23.330+0000 7f9e87200640 1 -- 172.21.15.17:0/4268649066 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e80069250 con 0x7f9e800cdf20 2024-09-10T21:25:23.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:23.330+0000 7f9e87200640 1 -- 172.21.15.17:0/4268649066 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e8015ab30 con 0x7f9e800cdf20 2024-09-10T21:25:23.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:23.330+0000 7f9e87200640 1 -- 172.21.15.17:0/4268649066 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e800c2ff0 con 0x7f9e800cdf20 2024-09-10T21:25:23.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:23.330+0000 7f9e7cc00640 1 -- 172.21.15.17:0/4268649066 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f9e700a6cd0 con 0x7f9e800cdf20 2024-09-10T21:25:23.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:23.334+0000 7f9e7cc00640 1 --2- 172.21.15.17:0/4268649066 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f9e5c0725e0 0x7f9e5c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:23.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:23.334+0000 7f9e7cc00640 1 -- 172.21.15.17:0/4268649066 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f9e7012dfd0 con 0x7f9e800cdf20 2024-09-10T21:25:23.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:23.334+0000 7f9e77e00640 1 --2- 172.21.15.17:0/4268649066 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f9e5c0725e0 0x7f9e5c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:23.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:23.334+0000 7f9e77e00640 1 --2- 172.21.15.17:0/4268649066 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f9e5c0725e0 0x7f9e5c074aa0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f9e800c7b00 tx=0x7f9e68005cc0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:23.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:23.338+0000 7f9e7cc00640 1 -- 172.21.15.17:0/4268649066 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f9e700164d0 con 0x7f9e800cdf20 2024-09-10T21:25:23.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:23.550+0000 7f9e87200640 1 -- 172.21.15.17:0/4268649066 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f9e80070850 con 0x7f9e800cdf20 2024-09-10T21:25:23.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:23.550+0000 7f9e7cc00640 1 -- 172.21.15.17:0/4268649066 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f9e700f92e0 con 0x7f9e800cdf20 2024-09-10T21:25:23.556 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-10T21:25:23.556 INFO:teuthology.orchestra.run.smithi017.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-10T21:25:23.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:23.554+0000 7f9e87200640 1 -- 172.21.15.17:0/4268649066 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f9e5c0725e0 msgr2=0x7f9e5c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:23.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:23.554+0000 7f9e87200640 1 --2- 172.21.15.17:0/4268649066 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f9e5c0725e0 0x7f9e5c074aa0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f9e800c7b00 tx=0x7f9e68005cc0 comp rx=0 tx=0).stop 2024-09-10T21:25:23.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:23.554+0000 7f9e87200640 1 -- 172.21.15.17:0/4268649066 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9e800cdf20 msgr2=0x7f9e801574c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:23.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:23.554+0000 7f9e87200640 1 --2- 172.21.15.17:0/4268649066 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9e800cdf20 0x7f9e801574c0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f9e7001cd30 tx=0x7f9e700a6520 comp rx=0 tx=0).stop 2024-09-10T21:25:23.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:23.554+0000 7f9e87200640 1 -- 172.21.15.17:0/4268649066 shutdown_connections 2024-09-10T21:25:23.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:23.554+0000 7f9e87200640 1 --2- 172.21.15.17:0/4268649066 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f9e5c0725e0 0x7f9e5c074aa0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:23.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:23.554+0000 7f9e87200640 1 --2- 172.21.15.17:0/4268649066 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9e80152820 0x7f9e801f3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:23.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:23.554+0000 7f9e87200640 1 --2- 172.21.15.17:0/4268649066 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f9e80152440 0x7f9e80150e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:23.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:23.554+0000 7f9e87200640 1 --2- 172.21.15.17:0/4268649066 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9e800cdf20 0x7f9e801574c0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:23.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:23.554+0000 7f9e87200640 1 -- 172.21.15.17:0/4268649066 >> 172.21.15.17:0/4268649066 conn(0x7f9e800702f0 msgr2=0x7f9e800ca540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:23.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:23.554+0000 7f9e87200640 1 -- 172.21.15.17:0/4268649066 shutdown_connections 2024-09-10T21:25:23.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:23.554+0000 7f9e87200640 1 -- 172.21.15.17:0/4268649066 wait complete. 2024-09-10T21:25:23.578 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-09-10T21:25:23.578 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:25:23.724 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:23.718+0000 7ffb80200640 1 -- 172.21.15.17:0/2058183731 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffb780c8a40 msgr2=0x7ffb780ce110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:23.724 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:23.718+0000 7ffb80200640 1 --2- 172.21.15.17:0/2058183731 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffb780c8a40 0x7ffb780ce110 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ffb70009c80 tx=0x7ffb7001cd10 comp rx=0 tx=0).stop 2024-09-10T21:25:23.724 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:23.718+0000 7ffb80200640 1 -- 172.21.15.17:0/2058183731 shutdown_connections 2024-09-10T21:25:23.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:23.718+0000 7ffb80200640 1 --2- 172.21.15.17:0/2058183731 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffb780c8a40 0x7ffb780ce110 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:23.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:23.718+0000 7ffb80200640 1 --2- 172.21.15.17:0/2058183731 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ffb78152520 0x7ffb780715f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:23.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:23.718+0000 7ffb80200640 1 -- 172.21.15.17:0/2058183731 >> 172.21.15.17:0/2058183731 conn(0x7ffb78068990 msgr2=0x7ffb78068da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:23.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:23.718+0000 7ffb80200640 1 -- 172.21.15.17:0/2058183731 shutdown_connections 2024-09-10T21:25:23.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:23.718+0000 7ffb80200640 1 -- 172.21.15.17:0/2058183731 wait complete. 2024-09-10T21:25:23.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:23.722+0000 7ffb80200640 1 Processor -- start 2024-09-10T21:25:23.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:23.722+0000 7ffb80200640 1 -- start start 2024-09-10T21:25:23.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:23.722+0000 7ffb80200640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffb780c8a40 0x7ffb7814c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:23.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:23.722+0000 7ffb80200640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffb78152520 0x7ffb780cd160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:23.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:23.722+0000 7ffb80200640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ffb7815b950 0x7ffb78161680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:23.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:23.722+0000 7ffb80200640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7ffb78165890 con 0x7ffb780c8a40 2024-09-10T21:25:23.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:23.722+0000 7ffb80200640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7ffb78165710 con 0x7ffb78152520 2024-09-10T21:25:23.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:23.722+0000 7ffb80200640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7ffb78165a10 con 0x7ffb7815b950 2024-09-10T21:25:23.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:23.722+0000 7ffb7d800640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffb780c8a40 0x7ffb7814c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:23.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:23.722+0000 7ffb7d800640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffb780c8a40 0x7ffb7814c1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44726/0 (socket says 172.21.15.17:44726) 2024-09-10T21:25:23.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:23.722+0000 7ffb7d800640 1 -- 172.21.15.17:0/1380353916 learned_addr learned my addr 172.21.15.17:0/1380353916 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:25:23.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:23.722+0000 7ffb7ce00640 1 --2- 172.21.15.17:0/1380353916 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffb78152520 0x7ffb780cd160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:23.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:23.722+0000 7ffb7e200640 1 --2- 172.21.15.17:0/1380353916 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ffb7815b950 0x7ffb78161680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:23.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:23.722+0000 7ffb7d800640 1 -- 172.21.15.17:0/1380353916 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ffb7815b950 msgr2=0x7ffb78161680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:23.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:23.722+0000 7ffb7d800640 1 --2- 172.21.15.17:0/1380353916 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ffb7815b950 0x7ffb78161680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:23.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:23.722+0000 7ffb7d800640 1 -- 172.21.15.17:0/1380353916 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffb78152520 msgr2=0x7ffb780cd160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:23.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:23.722+0000 7ffb7d800640 1 --2- 172.21.15.17:0/1380353916 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffb78152520 0x7ffb780cd160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:23.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:23.722+0000 7ffb7d800640 1 -- 172.21.15.17:0/1380353916 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb78160ef0 con 0x7ffb780c8a40 2024-09-10T21:25:23.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:23.722+0000 7ffb7d800640 1 --2- 172.21.15.17:0/1380353916 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffb780c8a40 0x7ffb7814c1d0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7ffb6c00e9f0 tx=0x7ffb6c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:23.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:23.722+0000 7ffb75600640 1 -- 172.21.15.17:0/1380353916 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb6c00cd80 con 0x7ffb780c8a40 2024-09-10T21:25:23.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:23.722+0000 7ffb75600640 1 -- 172.21.15.17:0/1380353916 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ffb6c010430 con 0x7ffb780c8a40 2024-09-10T21:25:23.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:23.722+0000 7ffb75600640 1 -- 172.21.15.17:0/1380353916 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb6c010860 con 0x7ffb780c8a40 2024-09-10T21:25:23.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:23.722+0000 7ffb80200640 1 -- 172.21.15.17:0/1380353916 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb781f4bc0 con 0x7ffb780c8a40 2024-09-10T21:25:23.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:23.722+0000 7ffb80200640 1 -- 172.21.15.17:0/1380353916 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb78155e70 con 0x7ffb780c8a40 2024-09-10T21:25:23.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:23.726+0000 7ffb80200640 1 -- 172.21.15.17:0/1380353916 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb780ce110 con 0x7ffb780c8a40 2024-09-10T21:25:23.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:23.726+0000 7ffb75600640 1 -- 172.21.15.17:0/1380353916 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7ffb6c010a00 con 0x7ffb780c8a40 2024-09-10T21:25:23.732 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:23.726+0000 7ffb75600640 1 --2- 172.21.15.17:0/1380353916 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7ffb58072610 0x7ffb58074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:23.732 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:23.726+0000 7ffb75600640 1 -- 172.21.15.17:0/1380353916 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7ffb6c094100 con 0x7ffb780c8a40 2024-09-10T21:25:23.732 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:23.726+0000 7ffb7ce00640 1 --2- 172.21.15.17:0/1380353916 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7ffb58072610 0x7ffb58074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:23.732 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:23.730+0000 7ffb7ce00640 1 --2- 172.21.15.17:0/1380353916 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7ffb58072610 0x7ffb58074ad0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ffb78162460 tx=0x7ffb64009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:23.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:23.730+0000 7ffb75600640 1 -- 172.21.15.17:0/1380353916 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7ffb6c05ef00 con 0x7ffb780c8a40 2024-09-10T21:25:23.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:23.946+0000 7ffb80200640 1 -- 172.21.15.17:0/1380353916 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ffb78150420 con 0x7ffb780c8a40 2024-09-10T21:25:23.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:23.946+0000 7ffb75600640 1 -- 172.21.15.17:0/1380353916 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+6870 (secure 0 0 0) 0x7ffb6c063930 con 0x7ffb780c8a40 2024-09-10T21:25:23.949 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-10T21:25:23.949 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":10,"btime":"2024-09-10T21:25:16:432823+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4350,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/1643071167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":1643071167},{"type":"v1","addr":"172.21.15.31:6835","nonce":1643071167}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4356,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6837/1666225712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6836","nonce":1666225712},{"type":"v1","addr":"172.21.15.17:6837","nonce":1666225712}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4384,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/3592734043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":3592734043},{"type":"v1","addr":"172.21.15.17:6833","nonce":3592734043}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4405,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/1293665856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":1293665856},{"type":"v1","addr":"172.21.15.17:6835","nonce":1293665856}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4411,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6838/1944852172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":1944852172},{"type":"v1","addr":"172.21.15.31:6838","nonce":1944852172}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":10,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:24:47.255397+0000","modified":"2024-09-10T21:25:15.961582+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4414},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4414":{"gid":4414,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.31:6839/3129062106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6837","nonce":3129062106},{"type":"v1","addr":"172.21.15.31:6839","nonce":3129062106}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4414,"qdb_cluster":[4414]},"id":1}]} 2024-09-10T21:25:23.950 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 10 2024-09-10T21:25:23.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:23.946+0000 7ffb80200640 1 -- 172.21.15.17:0/1380353916 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7ffb58072610 msgr2=0x7ffb58074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:23.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:23.946+0000 7ffb80200640 1 --2- 172.21.15.17:0/1380353916 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7ffb58072610 0x7ffb58074ad0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ffb78162460 tx=0x7ffb64009040 comp rx=0 tx=0).stop 2024-09-10T21:25:23.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:23.946+0000 7ffb80200640 1 -- 172.21.15.17:0/1380353916 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffb780c8a40 msgr2=0x7ffb7814c1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:23.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:23.946+0000 7ffb80200640 1 --2- 172.21.15.17:0/1380353916 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffb780c8a40 0x7ffb7814c1d0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7ffb6c00e9f0 tx=0x7ffb6c00eeb0 comp rx=0 tx=0).stop 2024-09-10T21:25:23.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:23.946+0000 7ffb80200640 1 -- 172.21.15.17:0/1380353916 shutdown_connections 2024-09-10T21:25:23.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:23.946+0000 7ffb80200640 1 --2- 172.21.15.17:0/1380353916 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7ffb58072610 0x7ffb58074ad0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:23.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:23.946+0000 7ffb80200640 1 --2- 172.21.15.17:0/1380353916 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ffb7815b950 0x7ffb78161680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:23.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:23.946+0000 7ffb80200640 1 --2- 172.21.15.17:0/1380353916 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffb78152520 0x7ffb780cd160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:23.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:23.946+0000 7ffb80200640 1 --2- 172.21.15.17:0/1380353916 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffb780c8a40 0x7ffb7814c1d0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:23.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:23.946+0000 7ffb80200640 1 -- 172.21.15.17:0/1380353916 >> 172.21.15.17:0/1380353916 conn(0x7ffb78068990 msgr2=0x7ffb78150940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:23.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:23.946+0000 7ffb80200640 1 -- 172.21.15.17:0/1380353916 shutdown_connections 2024-09-10T21:25:23.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:23.946+0000 7ffb80200640 1 -- 172.21.15.17:0/1380353916 wait complete. 2024-09-10T21:25:23.971 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:25:24.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:24.114+0000 7f02c8200640 1 -- 172.21.15.17:0/1710044574 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f02c0152820 msgr2=0x7f02c00b3eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:24.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:24.114+0000 7f02c8200640 1 --2- 172.21.15.17:0/1710044574 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f02c0152820 0x7f02c00b3eb0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f02c00c9850 tx=0x7f02ac01c830 comp rx=0 tx=0).stop 2024-09-10T21:25:24.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:24.114+0000 7f02c8200640 1 -- 172.21.15.17:0/1710044574 shutdown_connections 2024-09-10T21:25:24.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:24.114+0000 7f02c8200640 1 --2- 172.21.15.17:0/1710044574 >> v2:172.21.15.31:3300/0 conn(0x7f02c00cdf20 0x7f02c00ce350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:24.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:24.114+0000 7f02c8200640 1 --2- 172.21.15.17:0/1710044574 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f02c0152820 0x7f02c00b3eb0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:24.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:24.114+0000 7f02c8200640 1 -- 172.21.15.17:0/1710044574 >> 172.21.15.17:0/1710044574 conn(0x7f02c00702f0 msgr2=0x7f02c0070700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:24.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:24.114+0000 7f02c8200640 1 -- 172.21.15.17:0/1710044574 shutdown_connections 2024-09-10T21:25:24.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:24.114+0000 7f02c8200640 1 -- 172.21.15.17:0/1710044574 wait complete. 2024-09-10T21:25:24.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:24.118+0000 7f02c8200640 1 Processor -- start 2024-09-10T21:25:24.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:24.118+0000 7f02c8200640 1 -- start start 2024-09-10T21:25:24.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:24.118+0000 7f02c8200640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f02c00cdf20 0x7f02c0157380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:24.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:24.118+0000 7f02c8200640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f02c0152440 0x7f02c014aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:24.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:24.118+0000 7f02c8200640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f02c0152820 0x7f02c00c6780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:24.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:24.118+0000 7f02c8200640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f02c00cb5d0 con 0x7f02c00cdf20 2024-09-10T21:25:24.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:24.118+0000 7f02c8200640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f02c00cb450 con 0x7f02c0152440 2024-09-10T21:25:24.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:24.118+0000 7f02c8200640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f02c00c2ff0 con 0x7f02c0152820 2024-09-10T21:25:24.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:24.118+0000 7f02c5800640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f02c00cdf20 0x7f02c0157380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:24.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:24.118+0000 7f02c5800640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f02c00cdf20 0x7f02c0157380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44742/0 (socket says 172.21.15.17:44742) 2024-09-10T21:25:24.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:24.118+0000 7f02c5800640 1 -- 172.21.15.17:0/3707982340 learned_addr learned my addr 172.21.15.17:0/3707982340 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:25:24.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:24.118+0000 7f02c4e00640 1 --2- 172.21.15.17:0/3707982340 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f02c0152440 0x7f02c014aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:24.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:24.118+0000 7f02c6200640 1 --2- 172.21.15.17:0/3707982340 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f02c0152820 0x7f02c00c6780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:24.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:24.118+0000 7f02c5800640 1 -- 172.21.15.17:0/3707982340 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f02c0152820 msgr2=0x7f02c00c6780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:24.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:24.118+0000 7f02c5800640 1 --2- 172.21.15.17:0/3707982340 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f02c0152820 0x7f02c00c6780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:24.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:24.118+0000 7f02c5800640 1 -- 172.21.15.17:0/3707982340 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f02c0152440 msgr2=0x7f02c014aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:24.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:24.118+0000 7f02c5800640 1 --2- 172.21.15.17:0/3707982340 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f02c0152440 0x7f02c014aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:24.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:24.118+0000 7f02c5800640 1 -- 172.21.15.17:0/3707982340 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02c01578c0 con 0x7f02c00cdf20 2024-09-10T21:25:24.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:24.118+0000 7f02c5800640 1 --2- 172.21.15.17:0/3707982340 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f02c00cdf20 0x7f02c0157380 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f02b400c480 tx=0x7f02b400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:24.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:24.118+0000 7f02bd600640 1 -- 172.21.15.17:0/3707982340 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02b4012830 con 0x7f02c00cdf20 2024-09-10T21:25:24.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:24.118+0000 7f02bd600640 1 -- 172.21.15.17:0/3707982340 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f02b4017070 con 0x7f02c00cdf20 2024-09-10T21:25:24.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:24.118+0000 7f02bd600640 1 -- 172.21.15.17:0/3707982340 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02b4012d80 con 0x7f02c00cdf20 2024-09-10T21:25:24.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:24.118+0000 7f02c8200640 1 -- 172.21.15.17:0/3707982340 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02c00c7610 con 0x7f02c00cdf20 2024-09-10T21:25:24.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:24.118+0000 7f02c8200640 1 -- 172.21.15.17:0/3707982340 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02c00cd440 con 0x7f02c00cdf20 2024-09-10T21:25:24.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:24.118+0000 7f02c8200640 1 -- 172.21.15.17:0/3707982340 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02c00b38f0 con 0x7f02c00cdf20 2024-09-10T21:25:24.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:24.122+0000 7f02bd600640 1 -- 172.21.15.17:0/3707982340 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f02b401c460 con 0x7f02c00cdf20 2024-09-10T21:25:24.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:24.122+0000 7f02bd600640 1 --2- 172.21.15.17:0/3707982340 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f02a00725e0 0x7f02a0074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:24.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:24.122+0000 7f02bd600640 1 -- 172.21.15.17:0/3707982340 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f02b40932b0 con 0x7f02c00cdf20 2024-09-10T21:25:24.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:24.122+0000 7f02c4e00640 1 --2- 172.21.15.17:0/3707982340 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f02a00725e0 0x7f02a0074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:24.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:24.122+0000 7f02c4e00640 1 --2- 172.21.15.17:0/3707982340 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f02a00725e0 0x7f02a0074aa0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f02ac0af000 tx=0x7f02ac01b070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:24.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:24.126+0000 7f02bd600640 1 -- 172.21.15.17:0/3707982340 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f02b405dff0 con 0x7f02c00cdf20 2024-09-10T21:25:24.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:24.334+0000 7f02c8200640 1 -- 172.21.15.17:0/3707982340 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f02c00c8bf0 con 0x7f02c00cdf20 2024-09-10T21:25:24.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:24.334+0000 7f02bd600640 1 -- 172.21.15.17:0/3707982340 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+6870 (secure 0 0 0) 0x7f02b4062a20 con 0x7f02c00cdf20 2024-09-10T21:25:24.339 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-10T21:25:24.340 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":10,"btime":"2024-09-10T21:25:16:432823+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4350,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/1643071167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":1643071167},{"type":"v1","addr":"172.21.15.31:6835","nonce":1643071167}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4356,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6837/1666225712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6836","nonce":1666225712},{"type":"v1","addr":"172.21.15.17:6837","nonce":1666225712}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4384,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/3592734043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":3592734043},{"type":"v1","addr":"172.21.15.17:6833","nonce":3592734043}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4405,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/1293665856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":1293665856},{"type":"v1","addr":"172.21.15.17:6835","nonce":1293665856}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4411,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6838/1944852172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":1944852172},{"type":"v1","addr":"172.21.15.31:6838","nonce":1944852172}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":10,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:24:47.255397+0000","modified":"2024-09-10T21:25:15.961582+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4414},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4414":{"gid":4414,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.31:6839/3129062106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6837","nonce":3129062106},{"type":"v1","addr":"172.21.15.31:6839","nonce":3129062106}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4414,"qdb_cluster":[4414]},"id":1}]} 2024-09-10T21:25:24.340 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 10 2024-09-10T21:25:24.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:24.338+0000 7f02c8200640 1 -- 172.21.15.17:0/3707982340 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f02a00725e0 msgr2=0x7f02a0074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:24.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:24.338+0000 7f02c8200640 1 --2- 172.21.15.17:0/3707982340 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f02a00725e0 0x7f02a0074aa0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f02ac0af000 tx=0x7f02ac01b070 comp rx=0 tx=0).stop 2024-09-10T21:25:24.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:24.338+0000 7f02c8200640 1 -- 172.21.15.17:0/3707982340 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f02c00cdf20 msgr2=0x7f02c0157380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:24.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:24.338+0000 7f02c8200640 1 --2- 172.21.15.17:0/3707982340 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f02c00cdf20 0x7f02c0157380 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f02b400c480 tx=0x7f02b400c940 comp rx=0 tx=0).stop 2024-09-10T21:25:24.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:24.338+0000 7f02c8200640 1 -- 172.21.15.17:0/3707982340 shutdown_connections 2024-09-10T21:25:24.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:24.338+0000 7f02c8200640 1 --2- 172.21.15.17:0/3707982340 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f02a00725e0 0x7f02a0074aa0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:24.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:24.338+0000 7f02c8200640 1 --2- 172.21.15.17:0/3707982340 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f02c0152820 0x7f02c00c6780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:24.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:24.338+0000 7f02c8200640 1 --2- 172.21.15.17:0/3707982340 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f02c0152440 0x7f02c014aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:24.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:24.338+0000 7f02c8200640 1 --2- 172.21.15.17:0/3707982340 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f02c00cdf20 0x7f02c0157380 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:24.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:24.338+0000 7f02c8200640 1 -- 172.21.15.17:0/3707982340 >> 172.21.15.17:0/3707982340 conn(0x7f02c00702f0 msgr2=0x7f02c0162070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:24.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:24.338+0000 7f02c8200640 1 -- 172.21.15.17:0/3707982340 shutdown_connections 2024-09-10T21:25:24.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:24.338+0000 7f02c8200640 1 -- 172.21.15.17:0/3707982340 wait complete. 2024-09-10T21:25:24.362 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:25:24.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:24.506+0000 7f9499e00640 1 -- 172.21.15.17:0/4176362291 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9494152820 msgr2=0x7f949415c280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:24.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:24.506+0000 7f9499e00640 1 --2- 172.21.15.17:0/4176362291 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9494152820 0x7f949415c280 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f9494073220 tx=0x7f948001c770 comp rx=0 tx=0).stop 2024-09-10T21:25:24.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:24.506+0000 7f9499e00640 1 -- 172.21.15.17:0/4176362291 shutdown_connections 2024-09-10T21:25:24.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:24.506+0000 7f9499e00640 1 --2- 172.21.15.17:0/4176362291 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9494152820 0x7f949415c280 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:24.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:24.506+0000 7f9499e00640 1 --2- 172.21.15.17:0/4176362291 >> v2:172.21.15.31:3300/0 conn(0x7f9494152440 0x7f94940b3eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:24.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:24.506+0000 7f9499e00640 1 -- 172.21.15.17:0/4176362291 >> 172.21.15.17:0/4176362291 conn(0x7f94940702f0 msgr2=0x7f9494070700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:24.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:24.506+0000 7f9499e00640 1 -- 172.21.15.17:0/4176362291 shutdown_connections 2024-09-10T21:25:24.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:24.506+0000 7f9499e00640 1 -- 172.21.15.17:0/4176362291 wait complete. 2024-09-10T21:25:24.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:24.506+0000 7f9499e00640 1 Processor -- start 2024-09-10T21:25:24.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:24.506+0000 7f9499e00640 1 -- start start 2024-09-10T21:25:24.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:24.506+0000 7f9499e00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f94940cdf20 0x7f949406df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:24.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:24.506+0000 7f9499e00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9494152440 0x7f94941f6b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:24.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:24.506+0000 7f9499e00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9494152820 0x7f94941f70d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:24.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:24.506+0000 7f9499e00640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f949415c7c0 con 0x7f9494152820 2024-09-10T21:25:24.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:24.506+0000 7f9499e00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f94940ce740 con 0x7f9494152440 2024-09-10T21:25:24.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:24.506+0000 7f9499e00640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f94940cb450 con 0x7f94940cdf20 2024-09-10T21:25:24.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:24.506+0000 7f9493400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9494152820 0x7f94941f70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:24.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:24.506+0000 7f9493400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9494152820 0x7f94941f70d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44772/0 (socket says 172.21.15.17:44772) 2024-09-10T21:25:24.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:24.506+0000 7f9493400640 1 -- 172.21.15.17:0/1013227529 learned_addr learned my addr 172.21.15.17:0/1013227529 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:25:24.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:24.506+0000 7f9492000640 1 --2- 172.21.15.17:0/1013227529 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9494152440 0x7f94941f6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:24.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:24.506+0000 7f9492a00640 1 --2- 172.21.15.17:0/1013227529 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f94940cdf20 0x7f949406df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:24.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:24.510+0000 7f9493400640 1 -- 172.21.15.17:0/1013227529 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f94940cdf20 msgr2=0x7f949406df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:24.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:24.510+0000 7f9493400640 1 --2- 172.21.15.17:0/1013227529 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f94940cdf20 0x7f949406df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:24.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:24.510+0000 7f9493400640 1 -- 172.21.15.17:0/1013227529 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9494152440 msgr2=0x7f94941f6b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:24.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:24.510+0000 7f9493400640 1 --2- 172.21.15.17:0/1013227529 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9494152440 0x7f94941f6b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:24.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:24.510+0000 7f9493400640 1 -- 172.21.15.17:0/1013227529 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f949406edc0 con 0x7f9494152820 2024-09-10T21:25:24.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:24.510+0000 7f9492a00640 1 --2- 172.21.15.17:0/1013227529 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f94940cdf20 0x7f949406df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:25:24.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:24.510+0000 7f9493400640 1 --2- 172.21.15.17:0/1013227529 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9494152820 0x7f94941f70d0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f948c00cb10 tx=0x7f948c00cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:24.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:24.510+0000 7f948b400640 1 -- 172.21.15.17:0/1013227529 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f948c016020 con 0x7f9494152820 2024-09-10T21:25:24.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:24.510+0000 7f948b400640 1 -- 172.21.15.17:0/1013227529 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f948c00ce50 con 0x7f9494152820 2024-09-10T21:25:24.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:24.510+0000 7f9499e00640 1 -- 172.21.15.17:0/1013227529 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f949406f9e0 con 0x7f9494152820 2024-09-10T21:25:24.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:24.510+0000 7f948b400640 1 -- 172.21.15.17:0/1013227529 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f948c00ae00 con 0x7f9494152820 2024-09-10T21:25:24.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:24.510+0000 7f9499e00640 1 -- 172.21.15.17:0/1013227529 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f949406f070 con 0x7f9494152820 2024-09-10T21:25:24.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:24.510+0000 7f9499e00640 1 -- 172.21.15.17:0/1013227529 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94940c3040 con 0x7f9494152820 2024-09-10T21:25:24.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:24.510+0000 7f948b400640 1 -- 172.21.15.17:0/1013227529 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f948c012420 con 0x7f9494152820 2024-09-10T21:25:24.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:24.510+0000 7f948b400640 1 --2- 172.21.15.17:0/1013227529 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f94740725e0 0x7f9474074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:24.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:24.510+0000 7f948b400640 1 -- 172.21.15.17:0/1013227529 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f948c093a20 con 0x7f9494152820 2024-09-10T21:25:24.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:24.510+0000 7f9492a00640 1 --2- 172.21.15.17:0/1013227529 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f94740725e0 0x7f9474074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:24.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:24.514+0000 7f9492a00640 1 --2- 172.21.15.17:0/1013227529 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f94740725e0 0x7f9474074aa0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f9484009c30 tx=0x7f94840091e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:24.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:24.514+0000 7f948b400640 1 -- 172.21.15.17:0/1013227529 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f948c020020 con 0x7f9494152820 2024-09-10T21:25:24.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:24.694+0000 7f9499e00640 1 -- 172.21.15.17:0/1013227529 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f94940708e0 con 0x7f9494152820 2024-09-10T21:25:24.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:24.694+0000 7f948b400640 1 -- 172.21.15.17:0/1013227529 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) ==== 74+0+18562 (secure 0 0 0) 0x7f948c05edb0 con 0x7f9494152820 2024-09-10T21:25:24.698 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-10T21:25:24.698 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":25,"fsid":"5e2f023f-d6a6-49f4-a256-f4807dce3739","created":"2024-09-10T21:24:21.750712+0000","modified":"2024-09-10T21:24:52.295803+0000","last_up_change":"2024-09-10T21:24:35.877801+0000","last_in_change":"2024-09-10T21:24:25.831127+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:24:35.362670+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T21:24:45.211689+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T21:24:46.209784+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T21:24:49.871300+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0f5fabc0-434e-4066-b430-233969a548f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":4105489867},{"type":"v1","addr":"172.21.15.17:6809","nonce":4105489867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":4105489867},{"type":"v1","addr":"172.21.15.17:6811","nonce":4105489867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":4105489867},{"type":"v1","addr":"172.21.15.17:6815","nonce":4105489867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":4105489867},{"type":"v1","addr":"172.21.15.17:6813","nonce":4105489867}]},"public_addr":"172.21.15.17:6809/4105489867","cluster_addr":"172.21.15.17:6811/4105489867","heartbeat_back_addr":"172.21.15.17:6815/4105489867","heartbeat_front_addr":"172.21.15.17:6813/4105489867","state":["exists","up"]},{"osd":1,"uuid":"f6691cb3-2a91-4c90-a9fd-ece3c11dd6eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":2599842027},{"type":"v1","addr":"172.21.15.17:6825","nonce":2599842027}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":2599842027},{"type":"v1","addr":"172.21.15.17:6827","nonce":2599842027}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":2599842027},{"type":"v1","addr":"172.21.15.17:6831","nonce":2599842027}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":2599842027},{"type":"v1","addr":"172.21.15.17:6829","nonce":2599842027}]},"public_addr":"172.21.15.17:6825/2599842027","cluster_addr":"172.21.15.17:6827/2599842027","heartbeat_back_addr":"172.21.15.17:6831/2599842027","heartbeat_front_addr":"172.21.15.17:6829/2599842027","state":["exists","up"]},{"osd":2,"uuid":"cb61adb0-a216-437b-9598-06fa7c84aa64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":1248243634},{"type":"v1","addr":"172.21.15.17:6817","nonce":1248243634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":1248243634},{"type":"v1","addr":"172.21.15.17:6819","nonce":1248243634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":1248243634},{"type":"v1","addr":"172.21.15.17:6823","nonce":1248243634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":1248243634},{"type":"v1","addr":"172.21.15.17:6821","nonce":1248243634}]},"public_addr":"172.21.15.17:6817/1248243634","cluster_addr":"172.21.15.17:6819/1248243634","heartbeat_back_addr":"172.21.15.17:6823/1248243634","heartbeat_front_addr":"172.21.15.17:6821/1248243634","state":["exists","up"]},{"osd":3,"uuid":"6d2da26a-e0bf-47dc-b020-af86e6bcfccc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":2314489522},{"type":"v1","addr":"172.21.15.17:6801","nonce":2314489522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":2314489522},{"type":"v1","addr":"172.21.15.17:6803","nonce":2314489522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":2314489522},{"type":"v1","addr":"172.21.15.17:6807","nonce":2314489522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":2314489522},{"type":"v1","addr":"172.21.15.17:6805","nonce":2314489522}]},"public_addr":"172.21.15.17:6801/2314489522","cluster_addr":"172.21.15.17:6803/2314489522","heartbeat_back_addr":"172.21.15.17:6807/2314489522","heartbeat_front_addr":"172.21.15.17:6805/2314489522","state":["exists","up"]},{"osd":4,"uuid":"c65071dd-9ca7-4f1d-a584-45dc121419a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":202520744},{"type":"v1","addr":"172.21.15.31:6811","nonce":202520744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":202520744},{"type":"v1","addr":"172.21.15.31:6813","nonce":202520744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":202520744},{"type":"v1","addr":"172.21.15.31:6817","nonce":202520744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":202520744},{"type":"v1","addr":"172.21.15.31:6815","nonce":202520744}]},"public_addr":"172.21.15.31:6811/202520744","cluster_addr":"172.21.15.31:6813/202520744","heartbeat_back_addr":"172.21.15.31:6817/202520744","heartbeat_front_addr":"172.21.15.31:6815/202520744","state":["exists","up"]},{"osd":5,"uuid":"235389de-7777-4744-81b8-d0bfc11190e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":1472711057},{"type":"v1","addr":"172.21.15.31:6801","nonce":1472711057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":1472711057},{"type":"v1","addr":"172.21.15.31:6803","nonce":1472711057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":1472711057},{"type":"v1","addr":"172.21.15.31:6807","nonce":1472711057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":1472711057},{"type":"v1","addr":"172.21.15.31:6805","nonce":1472711057}]},"public_addr":"172.21.15.31:6801/1472711057","cluster_addr":"172.21.15.31:6803/1472711057","heartbeat_back_addr":"172.21.15.31:6807/1472711057","heartbeat_front_addr":"172.21.15.31:6805/1472711057","state":["exists","up"]},{"osd":6,"uuid":"cea70a57-3715-4ebb-8511-a528a9df2b1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":3754251089},{"type":"v1","addr":"172.21.15.31:6827","nonce":3754251089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":3754251089},{"type":"v1","addr":"172.21.15.31:6829","nonce":3754251089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":3754251089},{"type":"v1","addr":"172.21.15.31:6833","nonce":3754251089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":3754251089},{"type":"v1","addr":"172.21.15.31:6831","nonce":3754251089}]},"public_addr":"172.21.15.31:6827/3754251089","cluster_addr":"172.21.15.31:6829/3754251089","heartbeat_back_addr":"172.21.15.31:6833/3754251089","heartbeat_front_addr":"172.21.15.31:6831/3754251089","state":["exists","up"]},{"osd":7,"uuid":"01cc111b-a45f-4ca3-9cc1-7fe0afbeba45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":3948648934},{"type":"v1","addr":"172.21.15.31:6819","nonce":3948648934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":3948648934},{"type":"v1","addr":"172.21.15.31:6821","nonce":3948648934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":3948648934},{"type":"v1","addr":"172.21.15.31:6825","nonce":3948648934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":3948648934},{"type":"v1","addr":"172.21.15.31:6823","nonce":3948648934}]},"public_addr":"172.21.15.31:6819/3948648934","cluster_addr":"172.21.15.31:6821/3948648934","heartbeat_back_addr":"172.21.15.31:6825/3948648934","heartbeat_front_addr":"172.21.15.31:6823/3948648934","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:24:32.606989+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:24:32.891535+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:24:33.330793+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:24:32.811533+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:24:33.894624+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:24:32.773817+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:24:32.901865+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:24:32.907207+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:25:24.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:24.694+0000 7f9499e00640 1 -- 172.21.15.17:0/1013227529 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f94740725e0 msgr2=0x7f9474074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:24.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:24.694+0000 7f9499e00640 1 --2- 172.21.15.17:0/1013227529 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f94740725e0 0x7f9474074aa0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f9484009c30 tx=0x7f94840091e0 comp rx=0 tx=0).stop 2024-09-10T21:25:24.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:24.694+0000 7f9499e00640 1 -- 172.21.15.17:0/1013227529 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9494152820 msgr2=0x7f94941f70d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:24.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:24.694+0000 7f9499e00640 1 --2- 172.21.15.17:0/1013227529 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9494152820 0x7f94941f70d0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f948c00cb10 tx=0x7f948c00cc00 comp rx=0 tx=0).stop 2024-09-10T21:25:24.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:24.694+0000 7f9499e00640 1 -- 172.21.15.17:0/1013227529 shutdown_connections 2024-09-10T21:25:24.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:24.694+0000 7f9499e00640 1 --2- 172.21.15.17:0/1013227529 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f94740725e0 0x7f9474074aa0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:24.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:24.694+0000 7f9499e00640 1 --2- 172.21.15.17:0/1013227529 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9494152820 0x7f94941f70d0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:24.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:24.694+0000 7f9499e00640 1 --2- 172.21.15.17:0/1013227529 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9494152440 0x7f94941f6b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:24.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:24.694+0000 7f9499e00640 1 --2- 172.21.15.17:0/1013227529 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f94940cdf20 0x7f949406df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:24.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:24.694+0000 7f9499e00640 1 -- 172.21.15.17:0/1013227529 >> 172.21.15.17:0/1013227529 conn(0x7f94940702f0 msgr2=0x7f949415a0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:24.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:24.698+0000 7f9499e00640 1 -- 172.21.15.17:0/1013227529 shutdown_connections 2024-09-10T21:25:24.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:24.698+0000 7f9499e00640 1 -- 172.21.15.17:0/1013227529 wait complete. 2024-09-10T21:25:24.720 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:25:24.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:24.862+0000 7ff039400640 1 -- 172.21.15.17:0/2047195657 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff018003c20 msgr2=0x7ff018004020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:24.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:24.862+0000 7ff039400640 1 --2- 172.21.15.17:0/2047195657 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff018003c20 0x7ff018004020 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7ff024009b60 tx=0x7ff02401d240 comp rx=0 tx=0).stop 2024-09-10T21:25:24.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:24.862+0000 7ff033200640 1 -- 172.21.15.17:0/2047195657 reap_dead start 2024-09-10T21:25:24.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:24.862+0000 7ff039400640 1 -- 172.21.15.17:0/2047195657 shutdown_connections 2024-09-10T21:25:24.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:24.862+0000 7ff039400640 1 -- 172.21.15.17:0/2047195657 >> 172.21.15.17:0/2047195657 conn(0x7ff0340702f0 msgr2=0x7ff034070700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:24.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:24.862+0000 7ff039400640 1 -- 172.21.15.17:0/2047195657 shutdown_connections 2024-09-10T21:25:24.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:24.862+0000 7ff039400640 1 -- 172.21.15.17:0/2047195657 wait complete. 2024-09-10T21:25:24.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:24.866+0000 7ff039400640 1 Processor -- start 2024-09-10T21:25:24.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:24.866+0000 7ff039400640 1 -- start start 2024-09-10T21:25:24.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:24.866+0000 7ff039400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff018003c20 0x7ff034148dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:24.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:24.866+0000 7ff039400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff03406df40 0x7ff034143710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:24.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:24.866+0000 7ff039400640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff03406e3c0 0x7ff034143c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:24.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:24.866+0000 7ff039400640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7ff0340ce740 con 0x7ff018003c20 2024-09-10T21:25:24.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:24.866+0000 7ff039400640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7ff0340b42c0 con 0x7ff03406df40 2024-09-10T21:25:24.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:24.866+0000 7ff039400640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7ff0340cb450 con 0x7ff03406e3c0 2024-09-10T21:25:24.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:24.866+0000 7ff032800640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff018003c20 0x7ff034148dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:24.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:24.866+0000 7ff032800640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff018003c20 0x7ff034148dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43586/0 (socket says 172.21.15.17:43586) 2024-09-10T21:25:24.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:24.866+0000 7ff032800640 1 -- 172.21.15.17:0/3283677767 learned_addr learned my addr 172.21.15.17:0/3283677767 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:25:24.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:24.866+0000 7ff029e00640 1 --2- 172.21.15.17:0/3283677767 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff03406df40 0x7ff034143710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:24.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:24.866+0000 7ff033200640 1 --2- 172.21.15.17:0/3283677767 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff03406e3c0 0x7ff034143c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:24.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:24.866+0000 7ff032800640 1 -- 172.21.15.17:0/3283677767 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff03406e3c0 msgr2=0x7ff034143c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:24.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:24.866+0000 7ff032800640 1 --2- 172.21.15.17:0/3283677767 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff03406e3c0 0x7ff034143c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:24.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:24.866+0000 7ff032800640 1 -- 172.21.15.17:0/3283677767 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff03406df40 msgr2=0x7ff034143710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:24.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:24.866+0000 7ff032800640 1 --2- 172.21.15.17:0/3283677767 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff03406df40 0x7ff034143710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:24.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:24.866+0000 7ff032800640 1 -- 172.21.15.17:0/3283677767 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff034144190 con 0x7ff018003c20 2024-09-10T21:25:24.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:24.866+0000 7ff029e00640 1 --2- 172.21.15.17:0/3283677767 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff03406df40 0x7ff034143710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:25:24.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:24.866+0000 7ff032800640 1 --2- 172.21.15.17:0/3283677767 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff018003c20 0x7ff034148dc0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7ff02401d070 tx=0x7ff024005590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:24.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:24.866+0000 7ff02b400640 1 -- 172.21.15.17:0/3283677767 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0240b1040 con 0x7ff018003c20 2024-09-10T21:25:24.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:24.866+0000 7ff02b400640 1 -- 172.21.15.17:0/3283677767 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff0240ac400 con 0x7ff018003c20 2024-09-10T21:25:24.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:24.866+0000 7ff02b400640 1 -- 172.21.15.17:0/3283677767 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff024004290 con 0x7ff018003c20 2024-09-10T21:25:24.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:24.866+0000 7ff039400640 1 -- 172.21.15.17:0/3283677767 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff034151a30 con 0x7ff018003c20 2024-09-10T21:25:24.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:24.866+0000 7ff039400640 1 -- 172.21.15.17:0/3283677767 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff03406f930 con 0x7ff018003c20 2024-09-10T21:25:24.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:24.870+0000 7ff039400640 1 -- 172.21.15.17:0/3283677767 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0340c2ff0 con 0x7ff018003c20 2024-09-10T21:25:24.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:24.870+0000 7ff02b400640 1 -- 172.21.15.17:0/3283677767 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7ff0240046d0 con 0x7ff018003c20 2024-09-10T21:25:24.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:24.870+0000 7ff02b400640 1 --2- 172.21.15.17:0/3283677767 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7ff010072610 0x7ff010074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:24.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:24.870+0000 7ff02b400640 1 -- 172.21.15.17:0/3283677767 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7ff02412df80 con 0x7ff018003c20 2024-09-10T21:25:24.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:24.870+0000 7ff029e00640 1 --2- 172.21.15.17:0/3283677767 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7ff010072610 0x7ff010074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:24.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:24.870+0000 7ff029e00640 1 --2- 172.21.15.17:0/3283677767 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7ff010072610 0x7ff010074ad0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7ff01c000f70 tx=0x7ff01c000ee0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:24.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:24.874+0000 7ff02b400640 1 -- 172.21.15.17:0/3283677767 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7ff0240c4020 con 0x7ff018003c20 2024-09-10T21:25:25.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.082+0000 7ff039400640 1 -- 172.21.15.17:0/3283677767 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff0340c3580 con 0x7ff018003c20 2024-09-10T21:25:25.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.086+0000 7ff02b400640 1 -- 172.21.15.17:0/3283677767 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+6870 (secure 0 0 0) 0x7ff024016f80 con 0x7ff018003c20 2024-09-10T21:25:25.089 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-10T21:25:25.089 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":10,"btime":"2024-09-10T21:25:16:432823+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4350,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/1643071167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":1643071167},{"type":"v1","addr":"172.21.15.31:6835","nonce":1643071167}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4356,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6837/1666225712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6836","nonce":1666225712},{"type":"v1","addr":"172.21.15.17:6837","nonce":1666225712}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4384,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/3592734043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":3592734043},{"type":"v1","addr":"172.21.15.17:6833","nonce":3592734043}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4405,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/1293665856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":1293665856},{"type":"v1","addr":"172.21.15.17:6835","nonce":1293665856}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4411,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6838/1944852172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":1944852172},{"type":"v1","addr":"172.21.15.31:6838","nonce":1944852172}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":10,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:24:47.255397+0000","modified":"2024-09-10T21:25:15.961582+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4414},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4414":{"gid":4414,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.31:6839/3129062106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6837","nonce":3129062106},{"type":"v1","addr":"172.21.15.31:6839","nonce":3129062106}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4414,"qdb_cluster":[4414]},"id":1}]} 2024-09-10T21:25:25.090 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 10 2024-09-10T21:25:25.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.086+0000 7ff039400640 1 -- 172.21.15.17:0/3283677767 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7ff010072610 msgr2=0x7ff010074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:25.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.086+0000 7ff039400640 1 --2- 172.21.15.17:0/3283677767 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7ff010072610 0x7ff010074ad0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7ff01c000f70 tx=0x7ff01c000ee0 comp rx=0 tx=0).stop 2024-09-10T21:25:25.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.086+0000 7ff039400640 1 -- 172.21.15.17:0/3283677767 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff018003c20 msgr2=0x7ff034148dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:25.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.086+0000 7ff039400640 1 --2- 172.21.15.17:0/3283677767 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff018003c20 0x7ff034148dc0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7ff02401d070 tx=0x7ff024005590 comp rx=0 tx=0).stop 2024-09-10T21:25:25.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.086+0000 7ff039400640 1 -- 172.21.15.17:0/3283677767 shutdown_connections 2024-09-10T21:25:25.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.086+0000 7ff039400640 1 --2- 172.21.15.17:0/3283677767 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7ff010072610 0x7ff010074ad0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:25.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.086+0000 7ff039400640 1 --2- 172.21.15.17:0/3283677767 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff03406e3c0 0x7ff034143c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:25.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.086+0000 7ff039400640 1 --2- 172.21.15.17:0/3283677767 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff03406df40 0x7ff034143710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:25.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.086+0000 7ff039400640 1 --2- 172.21.15.17:0/3283677767 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff018003c20 0x7ff034148dc0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:25.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.086+0000 7ff039400640 1 -- 172.21.15.17:0/3283677767 >> 172.21.15.17:0/3283677767 conn(0x7ff0340702f0 msgr2=0x7ff0340cc290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:25.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.086+0000 7ff039400640 1 -- 172.21.15.17:0/3283677767 shutdown_connections 2024-09-10T21:25:25.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.086+0000 7ff039400640 1 -- 172.21.15.17:0/3283677767 wait complete. 2024-09-10T21:25:25.716 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 10, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-10T21:24:47.255397+0000', 'modified': '2024-09-10T21:25:15.961582+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4414}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4414': {'gid': 4414, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.31:6839/3129062106', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.31:6837', 'nonce': 3129062106}, {'type': 'v1', 'addr': '172.21.15.31:6839', 'nonce': 3129062106}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4414, 'qdb_cluster': [4414]} 2024-09-10T21:25:25.717 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-10T21:25:25.717 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-09-10T21:25:25.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.866+0000 7fba49200640 1 -- 172.21.15.17:0/3294027697 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fba44151530 msgr2=0x7fba440c2bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:25.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.866+0000 7fba49200640 1 --2- 172.21.15.17:0/3294027697 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fba44151530 0x7fba440c2bf0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fba34009900 tx=0x7fba3401f0d0 comp rx=0 tx=0).stop 2024-09-10T21:25:25.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.866+0000 7fba49200640 1 -- 172.21.15.17:0/3294027697 shutdown_connections 2024-09-10T21:25:25.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.866+0000 7fba49200640 1 --2- 172.21.15.17:0/3294027697 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fba440b1a50 0x7fba441571b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:25.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.866+0000 7fba49200640 1 --2- 172.21.15.17:0/3294027697 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fba44151530 0x7fba440c2bf0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:25.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.866+0000 7fba49200640 1 -- 172.21.15.17:0/3294027697 >> 172.21.15.17:0/3294027697 conn(0x7fba440700d0 msgr2=0x7fba440704e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:25.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.866+0000 7fba49200640 1 -- 172.21.15.17:0/3294027697 shutdown_connections 2024-09-10T21:25:25.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.866+0000 7fba49200640 1 -- 172.21.15.17:0/3294027697 wait complete. 2024-09-10T21:25:25.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.866+0000 7fba49200640 1 Processor -- start 2024-09-10T21:25:25.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.866+0000 7fba49200640 1 -- start start 2024-09-10T21:25:25.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.870+0000 7fba49200640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba440725c0 0x7fba441f1580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:25.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.870+0000 7fba49200640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fba440b1a50 0x7fba44153940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:25.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.870+0000 7fba49200640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fba44151530 0x7fba441e1f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:25.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.870+0000 7fba49200640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fba44074b30 con 0x7fba44151530 2024-09-10T21:25:25.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.870+0000 7fba49200640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fba44074850 con 0x7fba440725c0 2024-09-10T21:25:25.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.870+0000 7fba49200640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7fba440c3130 con 0x7fba440b1a50 2024-09-10T21:25:25.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.870+0000 7fba42800640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fba44151530 0x7fba441e1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:25.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.870+0000 7fba42800640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fba44151530 0x7fba441e1f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43600/0 (socket says 172.21.15.17:43600) 2024-09-10T21:25:25.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.870+0000 7fba42800640 1 -- 172.21.15.17:0/2767305419 learned_addr learned my addr 172.21.15.17:0/2767305419 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:25:25.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.870+0000 7fba41e00640 1 --2- 172.21.15.17:0/2767305419 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba440725c0 0x7fba441f1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:25.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.870+0000 7fba3be00640 1 --2- 172.21.15.17:0/2767305419 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fba440b1a50 0x7fba44153940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:25.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.870+0000 7fba42800640 1 -- 172.21.15.17:0/2767305419 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fba440b1a50 msgr2=0x7fba44153940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:25.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.870+0000 7fba42800640 1 --2- 172.21.15.17:0/2767305419 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fba440b1a50 0x7fba44153940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:25.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.870+0000 7fba42800640 1 -- 172.21.15.17:0/2767305419 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba440725c0 msgr2=0x7fba441f1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:25.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.870+0000 7fba42800640 1 --2- 172.21.15.17:0/2767305419 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba440725c0 0x7fba441f1580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:25.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.870+0000 7fba42800640 1 -- 172.21.15.17:0/2767305419 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba441f1ac0 con 0x7fba44151530 2024-09-10T21:25:25.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.870+0000 7fba42800640 1 --2- 172.21.15.17:0/2767305419 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fba44151530 0x7fba441e1f00 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fba3c00c710 tx=0x7fba3c00cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:25.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.870+0000 7fba3a000640 1 -- 172.21.15.17:0/2767305419 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba3c003b80 con 0x7fba44151530 2024-09-10T21:25:25.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.870+0000 7fba3a000640 1 -- 172.21.15.17:0/2767305419 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fba3c003d20 con 0x7fba44151530 2024-09-10T21:25:25.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.870+0000 7fba3a000640 1 -- 172.21.15.17:0/2767305419 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba3c00aab0 con 0x7fba44151530 2024-09-10T21:25:25.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.870+0000 7fba49200640 1 -- 172.21.15.17:0/2767305419 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba4414d150 con 0x7fba44151530 2024-09-10T21:25:25.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.870+0000 7fba49200640 1 -- 172.21.15.17:0/2767305419 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba44151ec0 con 0x7fba44151530 2024-09-10T21:25:25.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.874+0000 7fba3a000640 1 -- 172.21.15.17:0/2767305419 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fba3c00ac50 con 0x7fba44151530 2024-09-10T21:25:25.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.874+0000 7fba3a000640 1 --2- 172.21.15.17:0/2767305419 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fba200725e0 0x7fba20074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:25.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.874+0000 7fba4c5c3000 1 Processor -- start 2024-09-10T21:25:25.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.874+0000 7fba3a000640 1 -- 172.21.15.17:0/2767305419 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fba3c093370 con 0x7fba44151530 2024-09-10T21:25:25.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.874+0000 7fba4c5c3000 1 -- start start 2024-09-10T21:25:25.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.874+0000 7fba41e00640 1 --2- 172.21.15.17:0/2767305419 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fba200725e0 0x7fba20074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:25.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.874+0000 7fba4c5c3000 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560eb4570190 0x560eb4570570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:25.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.874+0000 7fba4c5c3000 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x560eb4539950 0x560eb4539d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:25.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.874+0000 7fba4c5c3000 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560eb453a270 0x560eb4575040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:25.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.874+0000 7fba4c5c3000 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x560eb44523f0 con 0x560eb4570190 2024-09-10T21:25:25.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.874+0000 7fba4c5c3000 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x560eb438fb20 con 0x560eb453a270 2024-09-10T21:25:25.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.874+0000 7fba4c5c3000 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x560eb437ef50 con 0x560eb4539950 2024-09-10T21:25:25.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.874+0000 7fba41e00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560eb4570190 0x560eb4570570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:25.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.874+0000 7fba41e00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560eb4570190 0x560eb4570570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43610/0 (socket says 172.21.15.17:43610) 2024-09-10T21:25:25.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.874+0000 7fba41e00640 1 -- 172.21.15.17:0/2504344604 learned_addr learned my addr 172.21.15.17:0/2504344604 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:25:25.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.874+0000 7fba41e00640 1 --2- 172.21.15.17:0/2767305419 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fba200725e0 0x7fba20074aa0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fba340b2000 tx=0x7fba340aa040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:25.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.874+0000 7fba42800640 1 --2- 172.21.15.17:0/2504344604 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560eb453a270 0x560eb4575040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:25.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.874+0000 7fba3be00640 1 --2- 172.21.15.17:0/2504344604 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x560eb4539950 0x560eb4539d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:25.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.874+0000 7fba41e00640 1 -- 172.21.15.17:0/2504344604 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x560eb4539950 msgr2=0x560eb4539d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:25:25.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.874+0000 7fba41e00640 1 --2- 172.21.15.17:0/2504344604 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x560eb4539950 0x560eb4539d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:25.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.874+0000 7fba41e00640 1 -- 172.21.15.17:0/2504344604 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560eb453a270 msgr2=0x560eb4575040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:25:25.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.874+0000 7fba41e00640 1 --2- 172.21.15.17:0/2504344604 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560eb453a270 0x560eb4575040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:25.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.874+0000 7fba41e00640 1 -- 172.21.15.17:0/2504344604 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560eb4346590 con 0x560eb4570190 2024-09-10T21:25:25.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.874+0000 7fba3be00640 1 --2- 172.21.15.17:0/2504344604 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x560eb4539950 0x560eb4539d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:25:25.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.874+0000 7fba42800640 1 --2- 172.21.15.17:0/2504344604 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560eb453a270 0x560eb4575040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:25:25.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.874+0000 7fba41e00640 1 --2- 172.21.15.17:0/2504344604 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560eb4570190 0x560eb4570570 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fba34009230 tx=0x7fba340b9000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92296c807281801d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:25.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.874+0000 7fba32a00640 1 -- 172.21.15.17:0/2504344604 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba340b5040 con 0x560eb4570190 2024-09-10T21:25:25.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.874+0000 7fba32a00640 1 -- 172.21.15.17:0/2504344604 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fba340c0070 con 0x560eb4570190 2024-09-10T21:25:25.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.874+0000 7fba32a00640 1 -- 172.21.15.17:0/2504344604 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba34005020 con 0x560eb4570190 2024-09-10T21:25:25.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.878+0000 7fba4c5c3000 1 -- 172.21.15.17:0/2504344604 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560eb4570190 msgr2=0x560eb4570570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:25.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.878+0000 7fba4c5c3000 1 --2- 172.21.15.17:0/2504344604 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560eb4570190 0x560eb4570570 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fba34009230 tx=0x7fba340b9000 comp rx=0 tx=0).stop 2024-09-10T21:25:25.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.878+0000 7fba4c5c3000 1 -- 172.21.15.17:0/2504344604 shutdown_connections 2024-09-10T21:25:25.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.878+0000 7fba4c5c3000 1 --2- 172.21.15.17:0/2504344604 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560eb453a270 0x560eb4575040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:25.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.878+0000 7fba4c5c3000 1 --2- 172.21.15.17:0/2504344604 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x560eb4539950 0x560eb4539d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:25.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.878+0000 7fba4c5c3000 1 --2- 172.21.15.17:0/2504344604 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560eb4570190 0x560eb4570570 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:25.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.878+0000 7fba4c5c3000 1 -- 172.21.15.17:0/2504344604 >> 172.21.15.17:0/2504344604 conn(0x560eb441b130 msgr2=0x560eb4572e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:25.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.878+0000 7fba4c5c3000 1 -- 172.21.15.17:0/2504344604 shutdown_connections 2024-09-10T21:25:25.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.878+0000 7fba4c5c3000 1 -- 172.21.15.17:0/2504344604 wait complete. 2024-09-10T21:25:25.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.878+0000 7fba4c5c3000 1 -- start start 2024-09-10T21:25:25.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.878+0000 7fba4c5c3000 1 Processor -- start 2024-09-10T21:25:25.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.878+0000 7fba4c5c3000 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560eb4539950 0x560eb45385a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:25.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.878+0000 7fba4c5c3000 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x560eb453a270 0x560eb4538ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:25.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.878+0000 7fba4c5c3000 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560eb4570190 0x560eb457b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:25.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.878+0000 7fba4c5c3000 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x560eb43195d0 con 0x560eb4539950 2024-09-10T21:25:25.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.878+0000 7fba4c5c3000 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x560eb4382d60 con 0x560eb4570190 2024-09-10T21:25:25.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.878+0000 7fba4c5c3000 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x560eb4325500 con 0x560eb453a270 2024-09-10T21:25:25.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.878+0000 7fba4c5c3000 10 client.0 _ll_register_callbacks cb 0x560eb4329810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:25:25.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.878+0000 7fba41e00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560eb4539950 0x560eb45385a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:25.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.878+0000 7fba41e00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560eb4539950 0x560eb45385a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43624/0 (socket says 172.21.15.17:43624) 2024-09-10T21:25:25.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.878+0000 7fba41e00640 1 -- 172.21.15.17:0/270576355 learned_addr learned my addr 172.21.15.17:0/270576355 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:25:25.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.878+0000 7fba42800640 1 --2- 172.21.15.17:0/270576355 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560eb4570190 0x560eb457b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:25.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.878+0000 7fba3be00640 1 --2- 172.21.15.17:0/270576355 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x560eb453a270 0x560eb4538ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:25.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.878+0000 7fba41e00640 1 -- 172.21.15.17:0/270576355 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x560eb453a270 msgr2=0x560eb4538ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:25:25.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.878+0000 7fba41e00640 1 --2- 172.21.15.17:0/270576355 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x560eb453a270 0x560eb4538ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:25.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.878+0000 7fba41e00640 1 -- 172.21.15.17:0/270576355 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560eb4570190 msgr2=0x560eb457b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:25:25.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.878+0000 7fba41e00640 1 --2- 172.21.15.17:0/270576355 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560eb4570190 0x560eb457b3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:25.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.878+0000 7fba41e00640 1 -- 172.21.15.17:0/270576355 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560eb431d170 con 0x560eb4539950 2024-09-10T21:25:25.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.878+0000 7fba41e00640 1 --2- 172.21.15.17:0/270576355 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560eb4539950 0x560eb45385a0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fba3409ba00 tx=0x7fba34004400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=737ad6613d39a8a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:25.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.878+0000 7fba27400640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-09-10T21:25:25.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.878+0000 7fba27400640 1 -- 172.21.15.17:0/270576355 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba340050e0 con 0x560eb4539950 2024-09-10T21:25:25.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.878+0000 7fba27400640 1 -- 172.21.15.17:0/270576355 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fba34005280 con 0x560eb4539950 2024-09-10T21:25:25.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.878+0000 7fba4c5c3000 1 -- 172.21.15.17:0/270576355 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x560eb43170f0 con 0x560eb4539950 2024-09-10T21:25:25.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.878+0000 7fba27400640 1 -- 172.21.15.17:0/270576355 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=10) ==== 24+0+0 (secure 0 0 0) 0x7fba340dc050 con 0x560eb4539950 2024-09-10T21:25:25.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.878+0000 7fba4c5c3000 10 client.4773 fetch_fsmap learned FSMap version 10 2024-09-10T21:25:25.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.878+0000 7fba4c5c3000 1 -- 172.21.15.17:0/270576355 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=10,monmap=2+}) -- 0x560eb4403ef0 con 0x560eb4539950 2024-09-10T21:25:25.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.882+0000 7fba27400640 1 -- 172.21.15.17:0/270576355 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x560eb43170f0 con 0x560eb4539950 2024-09-10T21:25:25.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.882+0000 7fba27400640 1 -- 172.21.15.17:0/270576355 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 10) ==== 4720+0+0 (secure 0 0 0) 0x7fba340df020 con 0x560eb4539950 2024-09-10T21:25:25.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.882+0000 7fba4c5c3000 10 client.4773 fetch_fsmap finished waiting for FSMap version 10 2024-09-10T21:25:25.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.882+0000 7fba4c5c3000 10 client.4773 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-10T21:25:25.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.882+0000 7fba4c5c3000 20 client.4773 populate_metadata read hostname 'smithi017' 2024-09-10T21:25:25.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.882+0000 7fba4c5c3000 1 --2- 172.21.15.17:0/270576355 >> [v2:172.21.15.31:6837/3129062106,v1:172.21.15.31:6839/3129062106] conn(0x560eb456ef70 0x560eb456f350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:25.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.882+0000 7fba4c5c3000 4 client.4773 mds_command: new command op to 4414 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-10T21:25:25.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.882+0000 7fba4c5c3000 1 -- 172.21.15.17:0/270576355 --> [v2:172.21.15.31:6837/3129062106,v1:172.21.15.31:6839/3129062106] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560eb4391a70 con 0x560eb456ef70 2024-09-10T21:25:25.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.882+0000 7fba3be00640 1 --2- 172.21.15.17:0/270576355 >> [v2:172.21.15.31:6837/3129062106,v1:172.21.15.31:6839/3129062106] conn(0x560eb456ef70 0x560eb456f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:25.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.882+0000 7fba3be00640 1 --2- 172.21.15.17:0/270576355 >> [v2:172.21.15.31:6837/3129062106,v1:172.21.15.31:6839/3129062106] conn(0x560eb456ef70 0x560eb456f350 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cea86a6e6b28b3e9 server_cookie=db96df7aadfbfc4a in_seq=0 out_seq=0 2024-09-10T21:25:25.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.882+0000 7fba27400640 10 client.4773 ms_handle_connect on v2:172.21.15.31:6837/3129062106 2024-09-10T21:25:25.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.882+0000 7fba27400640 1 -- 172.21.15.17:0/270576355 <== mds.0 v2:172.21.15.31:6837/3129062106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x560eb4391a70 con 0x560eb456ef70 2024-09-10T21:25:25.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.882+0000 7fba27400640 10 client.4773 handle_command_reply: tid=0 2024-09-10T21:25:25.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.882+0000 7fba4c5c3000 1 client.4773 shutdown 2024-09-10T21:25:25.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.882+0000 7fba4c5c3000 1 -- 172.21.15.17:0/270576355 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560eb4539950 msgr2=0x560eb45385a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:25.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.882+0000 7fba4c5c3000 1 --2- 172.21.15.17:0/270576355 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560eb4539950 0x560eb45385a0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fba3409ba00 tx=0x7fba34004400 comp rx=0 tx=0).stop 2024-09-10T21:25:25.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.882+0000 7fba4c5c3000 1 -- 172.21.15.17:0/270576355 shutdown_connections 2024-09-10T21:25:25.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.882+0000 7fba4c5c3000 1 --2- 172.21.15.17:0/270576355 >> [v2:172.21.15.31:6837/3129062106,v1:172.21.15.31:6839/3129062106] conn(0x560eb456ef70 0x560eb456f350 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:25.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.882+0000 7fba4c5c3000 1 --2- 172.21.15.17:0/270576355 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560eb4570190 0x560eb457b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:25.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.882+0000 7fba4c5c3000 1 --2- 172.21.15.17:0/270576355 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x560eb453a270 0x560eb4538ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:25.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.882+0000 7fba4c5c3000 1 --2- 172.21.15.17:0/270576355 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560eb4539950 0x560eb45385a0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:25.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.882+0000 7fba4c5c3000 1 -- 172.21.15.17:0/270576355 >> 172.21.15.17:0/270576355 conn(0x560eb441b130 msgr2=0x560eb45320b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:25.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.882+0000 7fba27400640 0 client.4773 ms_handle_reset on v2:172.21.15.31:6837/3129062106 2024-09-10T21:25:25.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.882+0000 7fba4c5c3000 1 -- 172.21.15.17:0/270576355 shutdown_connections 2024-09-10T21:25:25.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.882+0000 7fba4c5c3000 1 -- 172.21.15.17:0/270576355 wait complete. 2024-09-10T21:25:25.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.882+0000 7fba4c5c3000 20 client.4773 trim_cache size 0 max 16384 2024-09-10T21:25:25.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.902+0000 7fba4c5c3000 1 Processor -- start 2024-09-10T21:25:25.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.902+0000 7fba4c5c3000 1 -- start start 2024-09-10T21:25:25.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.902+0000 7fba4c5c3000 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560eb4539950 0x560eb45ebf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:25.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.902+0000 7fba4c5c3000 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560eb453a270 0x560eb45ec490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:25.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.902+0000 7fba4c5c3000 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x560eb4570190 0x560eb45ec9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:25.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.902+0000 7fba4c5c3000 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fba441ff390 con 0x560eb4539950 2024-09-10T21:25:25.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.902+0000 7fba4c5c3000 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fba441fe270 con 0x560eb453a270 2024-09-10T21:25:25.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.902+0000 7fba4c5c3000 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7fba441f4a00 con 0x560eb4570190 2024-09-10T21:25:25.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.902+0000 7fba41e00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560eb4539950 0x560eb45ebf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:25.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.902+0000 7fba41e00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560eb4539950 0x560eb45ebf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43632/0 (socket says 172.21.15.17:43632) 2024-09-10T21:25:25.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.902+0000 7fba41e00640 1 -- 172.21.15.17:0/3218150696 learned_addr learned my addr 172.21.15.17:0/3218150696 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:25:25.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.902+0000 7fba3be00640 1 --2- 172.21.15.17:0/3218150696 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560eb453a270 0x560eb45ec490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:25.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.902+0000 7fba42800640 1 --2- 172.21.15.17:0/3218150696 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x560eb4570190 0x560eb45ec9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:25.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.902+0000 7fba41e00640 1 -- 172.21.15.17:0/3218150696 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x560eb4570190 msgr2=0x560eb45ec9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:25:25.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.902+0000 7fba41e00640 1 --2- 172.21.15.17:0/3218150696 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x560eb4570190 0x560eb45ec9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:25.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.902+0000 7fba41e00640 1 -- 172.21.15.17:0/3218150696 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560eb453a270 msgr2=0x560eb45ec490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:25:25.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.902+0000 7fba41e00640 1 --2- 172.21.15.17:0/3218150696 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560eb453a270 0x560eb45ec490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:25.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.902+0000 7fba41e00640 1 -- 172.21.15.17:0/3218150696 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560eb43dad90 con 0x560eb4539950 2024-09-10T21:25:25.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.902+0000 7fba42800640 1 --2- 172.21.15.17:0/3218150696 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x560eb4570190 0x560eb45ec9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:25:25.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.902+0000 7fba41e00640 1 --2- 172.21.15.17:0/3218150696 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560eb4539950 0x560eb45ebf50 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fba34000c10 tx=0x7fba34009290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=952451efb26fe3e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:25.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.902+0000 7fba32000640 1 -- 172.21.15.17:0/3218150696 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba340b5020 con 0x560eb4539950 2024-09-10T21:25:25.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.902+0000 7fba32000640 1 -- 172.21.15.17:0/3218150696 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fba340d4070 con 0x560eb4539950 2024-09-10T21:25:25.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.902+0000 7fba32000640 1 -- 172.21.15.17:0/3218150696 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba34005800 con 0x560eb4539950 2024-09-10T21:25:25.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.906+0000 7fba4c5c3000 1 -- 172.21.15.17:0/3218150696 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560eb4539950 msgr2=0x560eb45ebf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:25.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.906+0000 7fba4c5c3000 1 --2- 172.21.15.17:0/3218150696 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560eb4539950 0x560eb45ebf50 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fba34000c10 tx=0x7fba34009290 comp rx=0 tx=0).stop 2024-09-10T21:25:25.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.906+0000 7fba4c5c3000 1 -- 172.21.15.17:0/3218150696 shutdown_connections 2024-09-10T21:25:25.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.906+0000 7fba4c5c3000 1 --2- 172.21.15.17:0/3218150696 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x560eb4570190 0x560eb45ec9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:25.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.906+0000 7fba4c5c3000 1 --2- 172.21.15.17:0/3218150696 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560eb453a270 0x560eb45ec490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:25.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.906+0000 7fba4c5c3000 1 --2- 172.21.15.17:0/3218150696 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560eb4539950 0x560eb45ebf50 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:25.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.906+0000 7fba4c5c3000 1 -- 172.21.15.17:0/3218150696 >> 172.21.15.17:0/3218150696 conn(0x560eb441b130 msgr2=0x560eb4591020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:25.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.906+0000 7fba4c5c3000 1 -- 172.21.15.17:0/3218150696 shutdown_connections 2024-09-10T21:25:25.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.906+0000 7fba4c5c3000 1 -- 172.21.15.17:0/3218150696 wait complete. 2024-09-10T21:25:25.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.906+0000 7fba4c5c3000 1 -- start start 2024-09-10T21:25:25.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.906+0000 7fba4c5c3000 1 Processor -- start 2024-09-10T21:25:25.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.906+0000 7fba4c5c3000 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560eb4539950 0x560eb45ecf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:25.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.906+0000 7fba4c5c3000 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560eb453a270 0x560eb45ed4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:25.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.906+0000 7fba4c5c3000 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x560eb4570190 0x560eb45ed9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:25.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.906+0000 7fba4c5c3000 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fba441fcfa0 con 0x560eb4539950 2024-09-10T21:25:25.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.906+0000 7fba4c5c3000 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fba441fdba0 con 0x560eb453a270 2024-09-10T21:25:25.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.906+0000 7fba4c5c3000 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7fba441fd4f0 con 0x560eb4570190 2024-09-10T21:25:25.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.906+0000 7fba4c5c3000 10 client.0 _ll_register_callbacks cb 0x7fba340041d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-10T21:25:25.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.906+0000 7fba41e00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560eb4539950 0x560eb45ecf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:25.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.906+0000 7fba41e00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560eb4539950 0x560eb45ecf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43646/0 (socket says 172.21.15.17:43646) 2024-09-10T21:25:25.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.906+0000 7fba41e00640 1 -- 172.21.15.17:0/2650023119 learned_addr learned my addr 172.21.15.17:0/2650023119 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:25:25.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.906+0000 7fba3be00640 1 --2- 172.21.15.17:0/2650023119 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560eb453a270 0x560eb45ed4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:25.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.906+0000 7fba42800640 1 --2- 172.21.15.17:0/2650023119 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x560eb4570190 0x560eb45ed9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:25.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.906+0000 7fba41e00640 1 -- 172.21.15.17:0/2650023119 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x560eb4570190 msgr2=0x560eb45ed9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:25:25.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.906+0000 7fba41e00640 1 --2- 172.21.15.17:0/2650023119 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x560eb4570190 0x560eb45ed9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:25.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.906+0000 7fba41e00640 1 -- 172.21.15.17:0/2650023119 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560eb453a270 msgr2=0x560eb45ed4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-10T21:25:25.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.906+0000 7fba41e00640 1 --2- 172.21.15.17:0/2650023119 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560eb453a270 0x560eb45ed4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:25.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.906+0000 7fba41e00640 1 -- 172.21.15.17:0/2650023119 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560eb4345a90 con 0x560eb4539950 2024-09-10T21:25:25.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.906+0000 7fba41e00640 1 --2- 172.21.15.17:0/2650023119 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560eb4539950 0x560eb45ecf60 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fba340206e0 tx=0x7fba34004580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0864fc321a68937 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:25.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.906+0000 7fba27400640 10 client.0 ms_handle_connect on v2:172.21.15.17:3300/0 2024-09-10T21:25:25.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.906+0000 7fba27400640 1 -- 172.21.15.17:0/2650023119 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba340050e0 con 0x560eb4539950 2024-09-10T21:25:25.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.906+0000 7fba27400640 1 -- 172.21.15.17:0/2650023119 <== mon.0 v2:172.21.15.17:3300/0 2 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7fba34005280 con 0x560eb4539950 2024-09-10T21:25:25.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.906+0000 7fba4c5c3000 1 -- 172.21.15.17:0/2650023119 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fba340d4070 con 0x560eb4539950 2024-09-10T21:25:25.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.906+0000 7fba27400640 1 -- 172.21.15.17:0/2650023119 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_get_version_reply(handle=1 version=10) ==== 24+0+0 (secure 0 0 0) 0x7fba340f6050 con 0x560eb4539950 2024-09-10T21:25:25.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.906+0000 7fba4c5c3000 10 client.4779 fetch_fsmap learned FSMap version 10 2024-09-10T21:25:25.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.906+0000 7fba4c5c3000 1 -- 172.21.15.17:0/2650023119 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,fsmap=10,monmap=2+}) -- 0x560eb4324050 con 0x560eb4539950 2024-09-10T21:25:25.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.906+0000 7fba27400640 1 -- 172.21.15.17:0/2650023119 <== mon.0 v2:172.21.15.17:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fba340d4070 con 0x560eb4539950 2024-09-10T21:25:25.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.906+0000 7fba27400640 1 -- 172.21.15.17:0/2650023119 <== mon.0 v2:172.21.15.17:3300/0 5 ==== fsmap(e 10) ==== 4720+0+0 (secure 0 0 0) 0x7fba340f9020 con 0x560eb4539950 2024-09-10T21:25:25.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.906+0000 7fba4c5c3000 10 client.4779 fetch_fsmap finished waiting for FSMap version 10 2024-09-10T21:25:25.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.906+0000 7fba4c5c3000 10 client.4779 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-10T21:25:25.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.906+0000 7fba4c5c3000 20 client.4779 populate_metadata read hostname 'smithi017' 2024-09-10T21:25:25.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.906+0000 7fba4c5c3000 1 --2- 172.21.15.17:0/2650023119 >> [v2:172.21.15.31:6837/3129062106,v1:172.21.15.31:6839/3129062106] conn(0x560eb458ed20 0x560eb458f100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:25.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.910+0000 7fba4c5c3000 4 client.4779 mds_command: new command op to 4414 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-10T21:25:25.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.910+0000 7fba4c5c3000 1 -- 172.21.15.17:0/2650023119 --> [v2:172.21.15.31:6837/3129062106,v1:172.21.15.31:6839/3129062106] -- command(tid 0: {"prefix": "status"}) -- 0x560eb438d520 con 0x560eb458ed20 2024-09-10T21:25:25.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.910+0000 7fba3be00640 1 --2- 172.21.15.17:0/2650023119 >> [v2:172.21.15.31:6837/3129062106,v1:172.21.15.31:6839/3129062106] conn(0x560eb458ed20 0x560eb458f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:25.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.910+0000 7fba3be00640 1 --2- 172.21.15.17:0/2650023119 >> [v2:172.21.15.31:6837/3129062106,v1:172.21.15.31:6839/3129062106] conn(0x560eb458ed20 0x560eb458f100 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c2e6da10786cac0 server_cookie=6b79b55858fc6b92 in_seq=0 out_seq=0 2024-09-10T21:25:25.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.910+0000 7fba27400640 10 client.4779 ms_handle_connect on v2:172.21.15.31:6837/3129062106 2024-09-10T21:25:25.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.910+0000 7fba27400640 1 -- 172.21.15.17:0/2650023119 <== mds.0 v2:172.21.15.31:6837/3129062106 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x560eb438d520 con 0x560eb458ed20 2024-09-10T21:25:25.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.910+0000 7fba27400640 10 client.4779 handle_command_reply: tid=0 2024-09-10T21:25:25.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.910+0000 7fba4c5c3000 1 client.4779 shutdown 2024-09-10T21:25:25.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.910+0000 7fba4c5c3000 1 -- 172.21.15.17:0/2650023119 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560eb4539950 msgr2=0x560eb45ecf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:25.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.910+0000 7fba4c5c3000 1 --2- 172.21.15.17:0/2650023119 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560eb4539950 0x560eb45ecf60 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fba340206e0 tx=0x7fba34004580 comp rx=0 tx=0).stop 2024-09-10T21:25:25.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.910+0000 7fba4c5c3000 1 -- 172.21.15.17:0/2650023119 shutdown_connections 2024-09-10T21:25:25.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.910+0000 7fba4c5c3000 1 --2- 172.21.15.17:0/2650023119 >> [v2:172.21.15.31:6837/3129062106,v1:172.21.15.31:6839/3129062106] conn(0x560eb458ed20 0x560eb458f100 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:25.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.910+0000 7fba4c5c3000 1 --2- 172.21.15.17:0/2650023119 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x560eb4570190 0x560eb45ed9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:25.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.910+0000 7fba4c5c3000 1 --2- 172.21.15.17:0/2650023119 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x560eb453a270 0x560eb45ed4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:25.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.910+0000 7fba4c5c3000 1 --2- 172.21.15.17:0/2650023119 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x560eb4539950 0x560eb45ecf60 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:25.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.910+0000 7fba4c5c3000 1 -- 172.21.15.17:0/2650023119 >> 172.21.15.17:0/2650023119 conn(0x560eb441b130 msgr2=0x560eb45912c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:25.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.910+0000 7fba27400640 0 client.4779 ms_handle_reset on v2:172.21.15.31:6837/3129062106 2024-09-10T21:25:25.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.910+0000 7fba4c5c3000 1 -- 172.21.15.17:0/2650023119 shutdown_connections 2024-09-10T21:25:25.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.910+0000 7fba4c5c3000 1 -- 172.21.15.17:0/2650023119 wait complete. 2024-09-10T21:25:25.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.910+0000 7fba4c5c3000 20 client.4779 trim_cache size 0 max 16384 2024-09-10T21:25:25.924 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-10T21:25:25.924 INFO:teuthology.orchestra.run.smithi017.stdout: "cluster_fsid": "5e2f023f-d6a6-49f4-a256-f4807dce3739", 2024-09-10T21:25:25.924 INFO:teuthology.orchestra.run.smithi017.stdout: "whoami": 0, 2024-09-10T21:25:25.924 INFO:teuthology.orchestra.run.smithi017.stdout: "id": 4414, 2024-09-10T21:25:25.924 INFO:teuthology.orchestra.run.smithi017.stdout: "want_state": "up:active", 2024-09-10T21:25:25.925 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "up:active", 2024-09-10T21:25:25.925 INFO:teuthology.orchestra.run.smithi017.stdout: "fs_name": "cephfs", 2024-09-10T21:25:25.925 INFO:teuthology.orchestra.run.smithi017.stdout: "rank_uptime": 38.054912510000001, 2024-09-10T21:25:25.925 INFO:teuthology.orchestra.run.smithi017.stdout: "mdsmap_epoch": 10, 2024-09-10T21:25:25.925 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_epoch": 25, 2024-09-10T21:25:25.925 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap_epoch_barrier": 25, 2024-09-10T21:25:25.925 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 44.086066354000003 2024-09-10T21:25:25.925 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-10T21:25:25.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.910+0000 7fba49200640 1 -- 172.21.15.17:0/2767305419 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fba200725e0 msgr2=0x7fba20074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:25.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.910+0000 7fba49200640 1 --2- 172.21.15.17:0/2767305419 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fba200725e0 0x7fba20074aa0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fba340b2000 tx=0x7fba340aa040 comp rx=0 tx=0).stop 2024-09-10T21:25:25.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.910+0000 7fba49200640 1 -- 172.21.15.17:0/2767305419 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fba44151530 msgr2=0x7fba441e1f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:25.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.910+0000 7fba49200640 1 --2- 172.21.15.17:0/2767305419 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fba44151530 0x7fba441e1f00 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fba3c00c710 tx=0x7fba3c00cbd0 comp rx=0 tx=0).stop 2024-09-10T21:25:25.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.910+0000 7fba49200640 1 -- 172.21.15.17:0/2767305419 shutdown_connections 2024-09-10T21:25:25.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.910+0000 7fba49200640 1 --2- 172.21.15.17:0/2767305419 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fba200725e0 0x7fba20074aa0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:25.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.910+0000 7fba49200640 1 --2- 172.21.15.17:0/2767305419 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fba44151530 0x7fba441e1f00 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:25.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.910+0000 7fba49200640 1 --2- 172.21.15.17:0/2767305419 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fba440b1a50 0x7fba44153940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:25.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.910+0000 7fba49200640 1 --2- 172.21.15.17:0/2767305419 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba440725c0 0x7fba441f1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:25.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.910+0000 7fba49200640 1 -- 172.21.15.17:0/2767305419 >> 172.21.15.17:0/2767305419 conn(0x7fba440700d0 msgr2=0x7fba441501e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:25.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.910+0000 7fba49200640 1 -- 172.21.15.17:0/2767305419 shutdown_connections 2024-09-10T21:25:25.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:25.910+0000 7fba49200640 1 -- 172.21.15.17:0/2767305419 wait complete. 2024-09-10T21:25:25.934 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-09-10T21:25:25.946 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-09-10T21:25:25.946 INFO:tasks.ceph_fuse:config is {'client.0': {}} 2024-09-10T21:25:25.946 INFO:tasks.ceph_fuse:client.0 config is {} 2024-09-10T21:25:25.946 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-10T21:25:25.946 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-10T21:25:25.947 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:25:25.947 DEBUG:teuthology.orchestra.run.smithi017:> ip netns list 2024-09-10T21:25:25.956 INFO:teuthology.orchestra.run.smithi017.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-09-10T21:25:25.957 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:25:25.957 DEBUG:teuthology.orchestra.run.smithi017:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-10T21:25:26.020 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:25:26.021 DEBUG:teuthology.orchestra.run.smithi017:> sudo ip link delete ceph-brx 2024-09-10T21:25:26.591 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-09-10T21:25:26.591 DEBUG:tasks.ceph_fuse:passing mntargs=[] 2024-09-10T21:25:26.591 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-10T21:25:26.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:26.734+0000 7f3e70400640 1 -- 172.21.15.17:0/3190920336 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3e50003a60 msgr2=0x7f3e50003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:26.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:26.734+0000 7f3e70400640 1 --2- 172.21.15.17:0/3190920336 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3e50003a60 0x7f3e50003e80 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f3e5c0016f0 tx=0x7f3e5c01c750 comp rx=0 tx=0).stop 2024-09-10T21:25:26.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:26.734+0000 7f3e70400640 1 -- 172.21.15.17:0/3190920336 shutdown_connections 2024-09-10T21:25:26.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:26.734+0000 7f3e70400640 1 --2- 172.21.15.17:0/3190920336 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3e50003a60 0x7f3e50003e80 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:26.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:26.734+0000 7f3e70400640 1 -- 172.21.15.17:0/3190920336 >> 172.21.15.17:0/3190920336 conn(0x7f3e6806dd30 msgr2=0x7f3e6806e140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:26.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:26.734+0000 7f3e70400640 1 -- 172.21.15.17:0/3190920336 shutdown_connections 2024-09-10T21:25:26.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:26.734+0000 7f3e70400640 1 -- 172.21.15.17:0/3190920336 wait complete. 2024-09-10T21:25:26.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:26.738+0000 7f3e70400640 1 Processor -- start 2024-09-10T21:25:26.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:26.738+0000 7f3e70400640 1 -- start start 2024-09-10T21:25:26.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:26.738+0000 7f3e70400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3e50003a60 0x7f3e681f4e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:26.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:26.738+0000 7f3e70400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e6815a930 0x7f3e681f5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:26.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:26.738+0000 7f3e70400640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3e681f5db0 0x7f3e680b0b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:26.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:26.738+0000 7f3e70400640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f3e68069b90 con 0x7f3e50003a60 2024-09-10T21:25:26.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:26.738+0000 7f3e70400640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f3e6806f060 con 0x7f3e6815a930 2024-09-10T21:25:26.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:26.738+0000 7f3e70400640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f3e6815c330 con 0x7f3e681f5db0 2024-09-10T21:25:26.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:26.738+0000 7f3e6da00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3e50003a60 0x7f3e681f4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:26.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:26.738+0000 7f3e6da00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3e50003a60 0x7f3e681f4e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43662/0 (socket says 172.21.15.17:43662) 2024-09-10T21:25:26.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:26.738+0000 7f3e6da00640 1 -- 172.21.15.17:0/757183770 learned_addr learned my addr 172.21.15.17:0/757183770 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:25:26.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:26.738+0000 7f3e6d000640 1 --2- 172.21.15.17:0/757183770 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e6815a930 0x7f3e681f5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:26.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:26.738+0000 7f3e6e400640 1 --2- 172.21.15.17:0/757183770 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3e681f5db0 0x7f3e680b0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:26.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:26.738+0000 7f3e6da00640 1 -- 172.21.15.17:0/757183770 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3e681f5db0 msgr2=0x7f3e680b0b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:26.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:26.738+0000 7f3e6da00640 1 --2- 172.21.15.17:0/757183770 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3e681f5db0 0x7f3e680b0b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:26.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:26.738+0000 7f3e6da00640 1 -- 172.21.15.17:0/757183770 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e6815a930 msgr2=0x7f3e681f5870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:26.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:26.738+0000 7f3e6da00640 1 --2- 172.21.15.17:0/757183770 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e6815a930 0x7f3e681f5870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:26.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:26.738+0000 7f3e6da00640 1 -- 172.21.15.17:0/757183770 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e6806a1b0 con 0x7f3e50003a60 2024-09-10T21:25:26.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:26.738+0000 7f3e6da00640 1 --2- 172.21.15.17:0/757183770 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3e50003a60 0x7f3e681f4e00 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f3e5c01cc30 tx=0x7f3e5c006a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:26.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:26.738+0000 7f3e65600640 1 -- 172.21.15.17:0/757183770 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e5c003600 con 0x7f3e50003a60 2024-09-10T21:25:26.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:26.738+0000 7f3e65600640 1 -- 172.21.15.17:0/757183770 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3e5c003890 con 0x7f3e50003a60 2024-09-10T21:25:26.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:26.738+0000 7f3e65600640 1 -- 172.21.15.17:0/757183770 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e5c003c60 con 0x7f3e50003a60 2024-09-10T21:25:26.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:26.738+0000 7f3e70400640 1 -- 172.21.15.17:0/757183770 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e680c3e80 con 0x7f3e50003a60 2024-09-10T21:25:26.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:26.738+0000 7f3e70400640 1 -- 172.21.15.17:0/757183770 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e680c1a30 con 0x7f3e50003a60 2024-09-10T21:25:26.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:26.742+0000 7f3e70400640 1 -- 172.21.15.17:0/757183770 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e68069e60 con 0x7f3e50003a60 2024-09-10T21:25:26.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:26.742+0000 7f3e65600640 1 -- 172.21.15.17:0/757183770 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f3e5c0a64b0 con 0x7f3e50003a60 2024-09-10T21:25:26.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:26.742+0000 7f3e65600640 1 --2- 172.21.15.17:0/757183770 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f3e480725e0 0x7f3e48074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:26.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:26.742+0000 7f3e65600640 1 -- 172.21.15.17:0/757183770 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 6532+0+0 (secure 0 0 0) 0x7f3e5c12cf40 con 0x7f3e50003a60 2024-09-10T21:25:26.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:26.742+0000 7f3e6d000640 1 --2- 172.21.15.17:0/757183770 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f3e480725e0 0x7f3e48074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:26.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:26.742+0000 7f3e6d000640 1 --2- 172.21.15.17:0/757183770 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f3e480725e0 0x7f3e48074aa0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f3e54001870 tx=0x7f3e54006040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:26.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:26.746+0000 7f3e65600640 1 -- 172.21.15.17:0/757183770 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f3e5c0aa070 con 0x7f3e50003a60 2024-09-10T21:25:26.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:26.958+0000 7f3e70400640 1 -- 172.21.15.17:0/757183770 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f3e680c8a80 con 0x7f3e50003a60 2024-09-10T21:25:26.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:26.958+0000 7f3e65600640 1 -- 172.21.15.17:0/757183770 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v10) ==== 53+0+88 (secure 0 0 0) 0x7f3e5c0164b0 con 0x7f3e50003a60 2024-09-10T21:25:26.964 INFO:teuthology.orchestra.run.smithi017.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-09-10T21:25:26.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:26.962+0000 7f3e70400640 1 -- 172.21.15.17:0/757183770 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f3e480725e0 msgr2=0x7f3e48074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:26.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:26.962+0000 7f3e70400640 1 --2- 172.21.15.17:0/757183770 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f3e480725e0 0x7f3e48074aa0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f3e54001870 tx=0x7f3e54006040 comp rx=0 tx=0).stop 2024-09-10T21:25:26.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:26.962+0000 7f3e70400640 1 -- 172.21.15.17:0/757183770 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3e50003a60 msgr2=0x7f3e681f4e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:26.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:26.962+0000 7f3e70400640 1 --2- 172.21.15.17:0/757183770 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3e50003a60 0x7f3e681f4e00 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f3e5c01cc30 tx=0x7f3e5c006a20 comp rx=0 tx=0).stop 2024-09-10T21:25:26.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:26.962+0000 7f3e70400640 1 -- 172.21.15.17:0/757183770 shutdown_connections 2024-09-10T21:25:26.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:26.962+0000 7f3e70400640 1 --2- 172.21.15.17:0/757183770 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f3e480725e0 0x7f3e48074aa0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:26.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:26.962+0000 7f3e70400640 1 --2- 172.21.15.17:0/757183770 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3e681f5db0 0x7f3e680b0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:26.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:26.962+0000 7f3e70400640 1 --2- 172.21.15.17:0/757183770 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e6815a930 0x7f3e681f5870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:26.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:26.962+0000 7f3e70400640 1 --2- 172.21.15.17:0/757183770 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3e50003a60 0x7f3e681f4e00 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:26.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:26.962+0000 7f3e70400640 1 -- 172.21.15.17:0/757183770 >> 172.21.15.17:0/757183770 conn(0x7f3e6806dd30 msgr2=0x7f3e68157bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:26.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:26.962+0000 7f3e70400640 1 -- 172.21.15.17:0/757183770 shutdown_connections 2024-09-10T21:25:26.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:26.962+0000 7f3e70400640 1 -- 172.21.15.17:0/757183770 wait complete. 2024-09-10T21:25:26.986 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-10T21:25:26.986 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T21:25:26.986 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi017.front.sepia.ceph.com 2024-09-10T21:25:26.986 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-10T21:25:26.986 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-10T21:25:26.986 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-10T21:25:26.986 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-10T21:25:26.986 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T21:25:26.986 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T21:25:26.987 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:25:26.987 DEBUG:teuthology.orchestra.run.smithi017:> ip addr 2024-09-10T21:25:26.997 INFO:teuthology.orchestra.run.smithi017.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T21:25:26.997 INFO:teuthology.orchestra.run.smithi017.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T21:25:26.997 INFO:teuthology.orchestra.run.smithi017.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T21:25:26.997 INFO:teuthology.orchestra.run.smithi017.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:25:26.997 INFO:teuthology.orchestra.run.smithi017.stdout: inet6 ::1/128 scope host 2024-09-10T21:25:26.997 INFO:teuthology.orchestra.run.smithi017.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:25:26.997 INFO:teuthology.orchestra.run.smithi017.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T21:25:26.997 INFO:teuthology.orchestra.run.smithi017.stdout: link/ether 0c:c4:7a:6b:fe:00 brd ff:ff:ff:ff:ff:ff 2024-09-10T21:25:26.997 INFO:teuthology.orchestra.run.smithi017.stdout: altname enp7s0f0 2024-09-10T21:25:26.997 INFO:teuthology.orchestra.run.smithi017.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T21:25:26.998 INFO:teuthology.orchestra.run.smithi017.stdout: link/ether 0c:c4:7a:6b:fe:01 brd ff:ff:ff:ff:ff:ff 2024-09-10T21:25:26.998 INFO:teuthology.orchestra.run.smithi017.stdout: altname enp7s0f1 2024-09-10T21:25:26.998 INFO:teuthology.orchestra.run.smithi017.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T21:25:26.998 INFO:teuthology.orchestra.run.smithi017.stdout: link/ether 0c:c4:7a:59:d9:92 brd ff:ff:ff:ff:ff:ff 2024-09-10T21:25:26.998 INFO:teuthology.orchestra.run.smithi017.stdout: altname enp1s0f0 2024-09-10T21:25:26.998 INFO:teuthology.orchestra.run.smithi017.stdout: inet 172.21.15.17/20 brd 172.21.15.255 scope global dynamic ens1f0 2024-09-10T21:25:26.998 INFO:teuthology.orchestra.run.smithi017.stdout: valid_lft 42390sec preferred_lft 42390sec 2024-09-10T21:25:26.998 INFO:teuthology.orchestra.run.smithi017.stdout: inet6 fe80::ec4:7aff:fe59:d992/64 scope link 2024-09-10T21:25:26.998 INFO:teuthology.orchestra.run.smithi017.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:25:26.998 INFO:teuthology.orchestra.run.smithi017.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2024-09-10T21:25:26.998 INFO:teuthology.orchestra.run.smithi017.stdout: link/ether 0c:c4:7a:59:d9:93 brd ff:ff:ff:ff:ff:ff 2024-09-10T21:25:26.998 INFO:teuthology.orchestra.run.smithi017.stdout: altname enp1s0f1 2024-09-10T21:25:26.998 INFO:teuthology.orchestra.run.smithi017.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2024-09-10T21:25:26.998 INFO:teuthology.orchestra.run.smithi017.stdout: link/ether 02:42:96:fe:6a:38 brd ff:ff:ff:ff:ff:ff 2024-09-10T21:25:26.999 INFO:teuthology.orchestra.run.smithi017.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2024-09-10T21:25:26.999 INFO:teuthology.orchestra.run.smithi017.stdout: valid_lft forever preferred_lft forever 2024-09-10T21:25:26.999 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-10T21:25:26.999 DEBUG:teuthology.orchestra.run.smithi017:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T21:25:26.999 DEBUG:teuthology.orchestra.run.smithi017:> set -e 2024-09-10T21:25:26.999 DEBUG:teuthology.orchestra.run.smithi017:> sudo ip link add name ceph-brx type bridge 2024-09-10T21:25:26.999 DEBUG:teuthology.orchestra.run.smithi017:> sudo ip addr flush dev ceph-brx 2024-09-10T21:25:26.999 DEBUG:teuthology.orchestra.run.smithi017:> sudo ip link set ceph-brx up 2024-09-10T21:25:26.999 DEBUG:teuthology.orchestra.run.smithi017:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-10T21:25:27.000 DEBUG:teuthology.orchestra.run.smithi017:> ') 2024-09-10T21:25:27.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:27 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T21:25:27.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:25:27.157 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:25:27.157 DEBUG:teuthology.orchestra.run.smithi017:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T21:25:27.175 INFO:teuthology.orchestra.run.smithi017.stdout:1 2024-09-10T21:25:27.177 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:25:27.177 DEBUG:teuthology.orchestra.run.smithi017:> route 2024-09-10T21:25:27.233 INFO:teuthology.orchestra.run.smithi017.stdout:Kernel IP routing table 2024-09-10T21:25:27.234 INFO:teuthology.orchestra.run.smithi017.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T21:25:27.234 INFO:teuthology.orchestra.run.smithi017.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2024-09-10T21:25:27.234 INFO:teuthology.orchestra.run.smithi017.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2024-09-10T21:25:27.234 INFO:teuthology.orchestra.run.smithi017.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2024-09-10T21:25:27.234 INFO:teuthology.orchestra.run.smithi017.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T21:25:27.234 DEBUG:teuthology.orchestra.run.smithi017:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T21:25:27.234 DEBUG:teuthology.orchestra.run.smithi017:> set -e 2024-09-10T21:25:27.234 DEBUG:teuthology.orchestra.run.smithi017:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-10T21:25:27.234 DEBUG:teuthology.orchestra.run.smithi017:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-10T21:25:27.235 DEBUG:teuthology.orchestra.run.smithi017:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-10T21:25:27.235 DEBUG:teuthology.orchestra.run.smithi017:> ') 2024-09-10T21:25:27.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:27 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T21:25:27.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:25:27.377 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:25:27.378 DEBUG:teuthology.orchestra.run.smithi017:> ip netns list 2024-09-10T21:25:27.387 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:25:27.388 DEBUG:teuthology.orchestra.run.smithi017:> ip netns list-id 2024-09-10T21:25:27.437 DEBUG:teuthology.orchestra.run.smithi017:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T21:25:27.437 DEBUG:teuthology.orchestra.run.smithi017:> set -e 2024-09-10T21:25:27.437 DEBUG:teuthology.orchestra.run.smithi017:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T21:25:27.437 DEBUG:teuthology.orchestra.run.smithi017:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-09-10T21:25:27.437 DEBUG:teuthology.orchestra.run.smithi017:> ') 2024-09-10T21:25:27.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:27 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T21:25:27.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:25:27.570 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-09-10T21:25:27.570 DEBUG:teuthology.orchestra.run.smithi017:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T21:25:27.570 DEBUG:teuthology.orchestra.run.smithi017:> set -e 2024-09-10T21:25:27.570 DEBUG:teuthology.orchestra.run.smithi017:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-09-10T21:25:27.570 DEBUG:teuthology.orchestra.run.smithi017:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-10T21:25:27.570 DEBUG:teuthology.orchestra.run.smithi017:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-09-10T21:25:27.570 DEBUG:teuthology.orchestra.run.smithi017:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-09-10T21:25:27.571 DEBUG:teuthology.orchestra.run.smithi017:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-09-10T21:25:27.571 DEBUG:teuthology.orchestra.run.smithi017:> ') 2024-09-10T21:25:27.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:27 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T21:25:27.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:25:27.713 DEBUG:teuthology.orchestra.run.smithi017:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T21:25:27.713 DEBUG:teuthology.orchestra.run.smithi017:> set -e 2024-09-10T21:25:27.714 DEBUG:teuthology.orchestra.run.smithi017:> sudo ip link set brx.0 up 2024-09-10T21:25:27.714 DEBUG:teuthology.orchestra.run.smithi017:> sudo ip link set dev brx.0 master ceph-brx 2024-09-10T21:25:27.714 DEBUG:teuthology.orchestra.run.smithi017:> ') 2024-09-10T21:25:27.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:27 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T21:25:27.792 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:25:27.799 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-09-10T21:25:27.799 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T21:25:27.799 DEBUG:teuthology.orchestra.run.smithi017:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-10T21:25:27.853 INFO:teuthology.orchestra.run.smithi017.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-10T21:25:27.853 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T21:25:27.853 DEBUG:teuthology.orchestra.run.smithi017:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-10T21:25:27.904 DEBUG:teuthology.orchestra.run.smithi017:> sudo modprobe fuse 2024-09-10T21:25:27.968 DEBUG:teuthology.orchestra.run.smithi017:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T21:25:28.021 INFO:teuthology.orchestra.run.smithi017.stdout:/sys 2024-09-10T21:25:28.022 INFO:teuthology.orchestra.run.smithi017.stdout:/proc 2024-09-10T21:25:28.022 INFO:teuthology.orchestra.run.smithi017.stdout:/dev 2024-09-10T21:25:28.022 INFO:teuthology.orchestra.run.smithi017.stdout:/dev/pts 2024-09-10T21:25:28.022 INFO:teuthology.orchestra.run.smithi017.stdout:/run 2024-09-10T21:25:28.022 INFO:teuthology.orchestra.run.smithi017.stdout:/ 2024-09-10T21:25:28.022 INFO:teuthology.orchestra.run.smithi017.stdout:/sys/kernel/security 2024-09-10T21:25:28.022 INFO:teuthology.orchestra.run.smithi017.stdout:/dev/shm 2024-09-10T21:25:28.022 INFO:teuthology.orchestra.run.smithi017.stdout:/run/lock 2024-09-10T21:25:28.022 INFO:teuthology.orchestra.run.smithi017.stdout:/sys/fs/cgroup 2024-09-10T21:25:28.022 INFO:teuthology.orchestra.run.smithi017.stdout:/sys/fs/pstore 2024-09-10T21:25:28.022 INFO:teuthology.orchestra.run.smithi017.stdout:/sys/fs/bpf 2024-09-10T21:25:28.023 INFO:teuthology.orchestra.run.smithi017.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T21:25:28.023 INFO:teuthology.orchestra.run.smithi017.stdout:/dev/hugepages 2024-09-10T21:25:28.023 INFO:teuthology.orchestra.run.smithi017.stdout:/dev/mqueue 2024-09-10T21:25:28.023 INFO:teuthology.orchestra.run.smithi017.stdout:/sys/kernel/debug 2024-09-10T21:25:28.023 INFO:teuthology.orchestra.run.smithi017.stdout:/sys/kernel/tracing 2024-09-10T21:25:28.023 INFO:teuthology.orchestra.run.smithi017.stdout:/proc/fs/nfsd 2024-09-10T21:25:28.023 INFO:teuthology.orchestra.run.smithi017.stdout:/sys/fs/fuse/connections 2024-09-10T21:25:28.023 INFO:teuthology.orchestra.run.smithi017.stdout:/sys/kernel/config 2024-09-10T21:25:28.023 INFO:teuthology.orchestra.run.smithi017.stdout:/run/credentials/systemd-sysusers.service 2024-09-10T21:25:28.023 INFO:teuthology.orchestra.run.smithi017.stdout:/run/rpc_pipefs 2024-09-10T21:25:28.023 INFO:teuthology.orchestra.run.smithi017.stdout:/run/user/1000 2024-09-10T21:25:28.023 INFO:teuthology.orchestra.run.smithi017.stdout:/var/lib/ceph 2024-09-10T21:25:28.024 INFO:teuthology.orchestra.run.smithi017.stdout:/var/lib/ceph/osd/ceph-0 2024-09-10T21:25:28.024 INFO:teuthology.orchestra.run.smithi017.stdout:/var/lib/ceph/osd/ceph-1 2024-09-10T21:25:28.024 INFO:teuthology.orchestra.run.smithi017.stdout:/var/lib/ceph/osd/ceph-2 2024-09-10T21:25:28.024 INFO:teuthology.orchestra.run.smithi017.stdout:/var/lib/ceph/osd/ceph-3 2024-09-10T21:25:28.024 INFO:teuthology.orchestra.run.smithi017.stdout:/run/netns 2024-09-10T21:25:28.024 INFO:teuthology.orchestra.run.smithi017.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T21:25:28.024 INFO:teuthology.orchestra.run.smithi017.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T21:25:28.025 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:25:28.025 DEBUG:teuthology.orchestra.run.smithi017:> ls /sys/fs/fuse/connections 2024-09-10T21:25:28.076 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-10T21:25:28.076 DEBUG:teuthology.orchestra.run.smithi017:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-09-10T21:25:28.122 DEBUG:teuthology.orchestra.run.smithi017:> sudo modprobe fuse 2024-09-10T21:25:28.144 DEBUG:teuthology.orchestra.run.smithi017:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T21:25:28.197 INFO:teuthology.orchestra.run.smithi017.stdout:/sys 2024-09-10T21:25:28.198 INFO:teuthology.orchestra.run.smithi017.stdout:/proc 2024-09-10T21:25:28.198 INFO:teuthology.orchestra.run.smithi017.stdout:/dev 2024-09-10T21:25:28.198 INFO:teuthology.orchestra.run.smithi017.stdout:/dev/pts 2024-09-10T21:25:28.198 INFO:teuthology.orchestra.run.smithi017.stdout:/run 2024-09-10T21:25:28.198 INFO:teuthology.orchestra.run.smithi017.stdout:/ 2024-09-10T21:25:28.198 INFO:teuthology.orchestra.run.smithi017.stdout:/sys/kernel/security 2024-09-10T21:25:28.198 INFO:teuthology.orchestra.run.smithi017.stdout:/dev/shm 2024-09-10T21:25:28.198 INFO:teuthology.orchestra.run.smithi017.stdout:/run/lock 2024-09-10T21:25:28.198 INFO:teuthology.orchestra.run.smithi017.stdout:/sys/fs/cgroup 2024-09-10T21:25:28.198 INFO:teuthology.orchestra.run.smithi017.stdout:/sys/fs/pstore 2024-09-10T21:25:28.199 INFO:teuthology.orchestra.run.smithi017.stdout:/sys/fs/bpf 2024-09-10T21:25:28.199 INFO:teuthology.orchestra.run.smithi017.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T21:25:28.199 INFO:teuthology.orchestra.run.smithi017.stdout:/dev/hugepages 2024-09-10T21:25:28.199 INFO:teuthology.orchestra.run.smithi017.stdout:/dev/mqueue 2024-09-10T21:25:28.199 INFO:teuthology.orchestra.run.smithi017.stdout:/sys/kernel/debug 2024-09-10T21:25:28.199 INFO:teuthology.orchestra.run.smithi017.stdout:/sys/kernel/tracing 2024-09-10T21:25:28.199 INFO:teuthology.orchestra.run.smithi017.stdout:/proc/fs/nfsd 2024-09-10T21:25:28.199 INFO:teuthology.orchestra.run.smithi017.stdout:/sys/fs/fuse/connections 2024-09-10T21:25:28.199 INFO:teuthology.orchestra.run.smithi017.stdout:/sys/kernel/config 2024-09-10T21:25:28.199 INFO:teuthology.orchestra.run.smithi017.stdout:/run/credentials/systemd-sysusers.service 2024-09-10T21:25:28.199 INFO:teuthology.orchestra.run.smithi017.stdout:/run/rpc_pipefs 2024-09-10T21:25:28.199 INFO:teuthology.orchestra.run.smithi017.stdout:/run/user/1000 2024-09-10T21:25:28.199 INFO:teuthology.orchestra.run.smithi017.stdout:/var/lib/ceph 2024-09-10T21:25:28.199 INFO:teuthology.orchestra.run.smithi017.stdout:/var/lib/ceph/osd/ceph-0 2024-09-10T21:25:28.200 INFO:teuthology.orchestra.run.smithi017.stdout:/var/lib/ceph/osd/ceph-1 2024-09-10T21:25:28.200 INFO:teuthology.orchestra.run.smithi017.stdout:/var/lib/ceph/osd/ceph-2 2024-09-10T21:25:28.200 INFO:teuthology.orchestra.run.smithi017.stdout:/var/lib/ceph/osd/ceph-3 2024-09-10T21:25:28.200 INFO:teuthology.orchestra.run.smithi017.stdout:/run/netns 2024-09-10T21:25:28.200 INFO:teuthology.orchestra.run.smithi017.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T21:25:28.200 INFO:teuthology.orchestra.run.smithi017.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T21:25:28.201 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:25:28.201 DEBUG:teuthology.orchestra.run.smithi017:> ls /sys/fs/fuse/connections 2024-09-10T21:25:28.219 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi017.stderr:2024-09-10T21:25:28.214+0000 7f35c87c9580 -1 init, newargv = 0x5650f56ae670 newargc=15 2024-09-10T21:25:28.219 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi017.stderr:ceph-fuse[35141]: starting ceph client 2024-09-10T21:25:28.227 INFO:teuthology.orchestra.run.smithi017.stdout:44 2024-09-10T21:25:28.228 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi017.stderr:ceph-fuse[35141]: starting fuse 2024-09-10T21:25:28.228 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [44] 2024-09-10T21:25:28.228 DEBUG:teuthology.orchestra.run.smithi017:> sudo stdin-killer -- python3 -c ' 2024-09-10T21:25:28.228 DEBUG:teuthology.orchestra.run.smithi017:> import glob 2024-09-10T21:25:28.228 DEBUG:teuthology.orchestra.run.smithi017:> import re 2024-09-10T21:25:28.228 DEBUG:teuthology.orchestra.run.smithi017:> import os 2024-09-10T21:25:28.229 DEBUG:teuthology.orchestra.run.smithi017:> import subprocess 2024-09-10T21:25:28.229 DEBUG:teuthology.orchestra.run.smithi017:> 2024-09-10T21:25:28.229 DEBUG:teuthology.orchestra.run.smithi017:> def _find_admin_socket(client_name): 2024-09-10T21:25:28.229 DEBUG:teuthology.orchestra.run.smithi017:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-10T21:25:28.229 DEBUG:teuthology.orchestra.run.smithi017:> files = glob.glob(asok_path) 2024-09-10T21:25:28.229 DEBUG:teuthology.orchestra.run.smithi017:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-10T21:25:28.229 DEBUG:teuthology.orchestra.run.smithi017:> 2024-09-10T21:25:28.229 DEBUG:teuthology.orchestra.run.smithi017:> # Given a non-glob path, it better be there 2024-09-10T21:25:28.229 DEBUG:teuthology.orchestra.run.smithi017:> if "*" not in asok_path: 2024-09-10T21:25:28.229 DEBUG:teuthology.orchestra.run.smithi017:> assert(len(files) == 1) 2024-09-10T21:25:28.229 DEBUG:teuthology.orchestra.run.smithi017:> return files[0] 2024-09-10T21:25:28.229 DEBUG:teuthology.orchestra.run.smithi017:> 2024-09-10T21:25:28.229 DEBUG:teuthology.orchestra.run.smithi017:> for f in files: 2024-09-10T21:25:28.230 DEBUG:teuthology.orchestra.run.smithi017:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-10T21:25:28.230 DEBUG:teuthology.orchestra.run.smithi017:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-10T21:25:28.230 DEBUG:teuthology.orchestra.run.smithi017:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-10T21:25:28.230 DEBUG:teuthology.orchestra.run.smithi017:> contents = proc_f.read() 2024-09-10T21:25:28.230 DEBUG:teuthology.orchestra.run.smithi017:> if mountpoint in contents: 2024-09-10T21:25:28.230 DEBUG:teuthology.orchestra.run.smithi017:> return f 2024-09-10T21:25:28.230 DEBUG:teuthology.orchestra.run.smithi017:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-10T21:25:28.230 DEBUG:teuthology.orchestra.run.smithi017:> 2024-09-10T21:25:28.230 DEBUG:teuthology.orchestra.run.smithi017:> print(_find_admin_socket("client.0")) 2024-09-10T21:25:28.230 DEBUG:teuthology.orchestra.run.smithi017:> ' 2024-09-10T21:25:28.354 INFO:teuthology.orchestra.run.smithi017.stdout:/var/run/ceph/ceph-client.0.35141.asok 2024-09-10T21:25:28.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T21:25:28.366 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.35141.asok 2024-09-10T21:25:28.366 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:25:28.366 DEBUG:teuthology.orchestra.run.smithi017:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.35141.asok status 2024-09-10T21:25:28.546 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-10T21:25:28.546 INFO:teuthology.orchestra.run.smithi017.stdout: "metadata": { 2024-09-10T21:25:28.546 INFO:teuthology.orchestra.run.smithi017.stdout: "ceph_sha1": "8f219c476cb4f26fc5719c661c0323719a45cb30", 2024-09-10T21:25:28.546 INFO:teuthology.orchestra.run.smithi017.stdout: "ceph_version": "ceph version 19.3.0-4854-g8f219c47 (8f219c476cb4f26fc5719c661c0323719a45cb30) squid (dev)", 2024-09-10T21:25:28.546 INFO:teuthology.orchestra.run.smithi017.stdout: "entity_id": "0", 2024-09-10T21:25:28.546 INFO:teuthology.orchestra.run.smithi017.stdout: "hostname": "smithi017", 2024-09-10T21:25:28.547 INFO:teuthology.orchestra.run.smithi017.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-10T21:25:28.547 INFO:teuthology.orchestra.run.smithi017.stdout: "pid": "35141", 2024-09-10T21:25:28.547 INFO:teuthology.orchestra.run.smithi017.stdout: "root": "/" 2024-09-10T21:25:28.547 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:25:28.547 INFO:teuthology.orchestra.run.smithi017.stdout: "dentry_count": 0, 2024-09-10T21:25:28.547 INFO:teuthology.orchestra.run.smithi017.stdout: "dentry_pinned_count": 0, 2024-09-10T21:25:28.547 INFO:teuthology.orchestra.run.smithi017.stdout: "id": 4791, 2024-09-10T21:25:28.547 INFO:teuthology.orchestra.run.smithi017.stdout: "inst": { 2024-09-10T21:25:28.547 INFO:teuthology.orchestra.run.smithi017.stdout: "name": { 2024-09-10T21:25:28.547 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "client", 2024-09-10T21:25:28.547 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 4791 2024-09-10T21:25:28.547 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:25:28.547 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": { 2024-09-10T21:25:28.548 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:25:28.548 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "192.168.0.1:0", 2024-09-10T21:25:28.548 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 1603017719 2024-09-10T21:25:28.548 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:25:28.548 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:25:28.548 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": { 2024-09-10T21:25:28.548 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:25:28.548 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "192.168.0.1:0", 2024-09-10T21:25:28.548 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 1603017719 2024-09-10T21:25:28.548 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:25:28.548 INFO:teuthology.orchestra.run.smithi017.stdout: "inst_str": "client.4791 192.168.0.1:0/1603017719", 2024-09-10T21:25:28.548 INFO:teuthology.orchestra.run.smithi017.stdout: "addr_str": "192.168.0.1:0/1603017719", 2024-09-10T21:25:28.548 INFO:teuthology.orchestra.run.smithi017.stdout: "inode_count": 1, 2024-09-10T21:25:28.548 INFO:teuthology.orchestra.run.smithi017.stdout: "mds_epoch": 10, 2024-09-10T21:25:28.549 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_epoch": 26, 2024-09-10T21:25:28.549 INFO:teuthology.orchestra.run.smithi017.stdout: "osd_epoch_barrier": 0, 2024-09-10T21:25:28.549 INFO:teuthology.orchestra.run.smithi017.stdout: "blocklisted": false, 2024-09-10T21:25:28.549 INFO:teuthology.orchestra.run.smithi017.stdout: "fs_name": "cephfs" 2024-09-10T21:25:28.549 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-10T21:25:28.561 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:25:28.561 DEBUG:teuthology.orchestra.run.smithi017:> stat --file-system '--printf=%T 2024-09-10T21:25:28.561 DEBUG:teuthology.orchestra.run.smithi017:> ' -- /home/ubuntu/cephtest/mnt.0 2024-09-10T21:25:28.570 INFO:teuthology.orchestra.run.smithi017.stdout:fuseblk 2024-09-10T21:25:28.571 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-09-10T21:25:28.571 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:25:28.571 DEBUG:teuthology.orchestra.run.smithi017:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-10T21:25:28.632 INFO:teuthology.run_tasks:Running task mon_thrash... 2024-09-10T21:25:28.650 INFO:tasks.mon_thrash:Beginning mon_thrash... 2024-09-10T21:25:28.650 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:25:28.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:28.822+0000 7fad3e200640 1 -- 172.21.15.17:0/817822389 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fad38152440 msgr2=0x7fad380b3eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:28.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:28.822+0000 7fad3e200640 1 --2- 172.21.15.17:0/817822389 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fad38152440 0x7fad380b3eb0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fad28009840 tx=0x7fad2801c790 comp rx=0 tx=0).stop 2024-09-10T21:25:28.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:28.822+0000 7fad3e200640 1 -- 172.21.15.17:0/817822389 shutdown_connections 2024-09-10T21:25:28.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:28.822+0000 7fad3e200640 1 --2- 172.21.15.17:0/817822389 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fad380cdf20 0x7fad380ce350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:28.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:28.822+0000 7fad3e200640 1 --2- 172.21.15.17:0/817822389 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fad38152440 0x7fad380b3eb0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:28.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:28.822+0000 7fad3e200640 1 -- 172.21.15.17:0/817822389 >> 172.21.15.17:0/817822389 conn(0x7fad380702f0 msgr2=0x7fad38070700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:28.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:28.822+0000 7fad3e200640 1 -- 172.21.15.17:0/817822389 shutdown_connections 2024-09-10T21:25:28.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:28.822+0000 7fad3e200640 1 -- 172.21.15.17:0/817822389 wait complete. 2024-09-10T21:25:28.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:28.826+0000 7fad3e200640 1 Processor -- start 2024-09-10T21:25:28.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:28.826+0000 7fad3e200640 1 -- start start 2024-09-10T21:25:28.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:28.826+0000 7fad3e200640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fad380cdf20 0x7fad38150b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:28.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:28.826+0000 7fad3e200640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fad38152440 0x7fad38151720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:28.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:28.826+0000 7fad3e200640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fad38152820 0x7fad381f6b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:28.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:28.826+0000 7fad3e200640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fad380cb5d0 con 0x7fad38152440 2024-09-10T21:25:28.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:28.826+0000 7fad3e200640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fad380cb450 con 0x7fad38152820 2024-09-10T21:25:28.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:28.826+0000 7fad3e200640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7fad380c2ff0 con 0x7fad380cdf20 2024-09-10T21:25:28.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:28.826+0000 7fad2fe00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fad38152440 0x7fad38151720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:28.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:28.826+0000 7fad2fe00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fad38152440 0x7fad38151720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43670/0 (socket says 172.21.15.17:43670) 2024-09-10T21:25:28.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:28.826+0000 7fad2fe00640 1 -- 172.21.15.17:0/3160754122 learned_addr learned my addr 172.21.15.17:0/3160754122 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:25:28.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:28.826+0000 7fad37e00640 1 --2- 172.21.15.17:0/3160754122 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fad38152820 0x7fad381f6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:28.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:28.826+0000 7fad37400640 1 --2- 172.21.15.17:0/3160754122 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fad380cdf20 0x7fad38150b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:28.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:28.826+0000 7fad2fe00640 1 -- 172.21.15.17:0/3160754122 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fad380cdf20 msgr2=0x7fad38150b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:28.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:28.826+0000 7fad2fe00640 1 --2- 172.21.15.17:0/3160754122 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fad380cdf20 0x7fad38150b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:28.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:28.826+0000 7fad2fe00640 1 -- 172.21.15.17:0/3160754122 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fad38152820 msgr2=0x7fad381f6b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:28.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:28.826+0000 7fad2fe00640 1 --2- 172.21.15.17:0/3160754122 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fad38152820 0x7fad381f6b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:28.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:28.826+0000 7fad2fe00640 1 -- 172.21.15.17:0/3160754122 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad380b38a0 con 0x7fad38152440 2024-09-10T21:25:28.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:28.826+0000 7fad2fe00640 1 --2- 172.21.15.17:0/3160754122 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fad38152440 0x7fad38151720 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fad2000c450 tx=0x7fad2000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:28.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:28.826+0000 7fad2f400640 1 -- 172.21.15.17:0/3160754122 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad20011790 con 0x7fad38152440 2024-09-10T21:25:28.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:28.826+0000 7fad2f400640 1 -- 172.21.15.17:0/3160754122 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fad20017070 con 0x7fad38152440 2024-09-10T21:25:28.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:28.826+0000 7fad2f400640 1 -- 172.21.15.17:0/3160754122 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad20011cc0 con 0x7fad38152440 2024-09-10T21:25:28.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:28.826+0000 7fad3e200640 1 -- 172.21.15.17:0/3160754122 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad381513d0 con 0x7fad38152440 2024-09-10T21:25:28.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:28.826+0000 7fad3e200640 1 -- 172.21.15.17:0/3160754122 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad3814c130 con 0x7fad38152440 2024-09-10T21:25:28.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:28.826+0000 7fad3e200640 1 -- 172.21.15.17:0/3160754122 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fad38158360 con 0x7fad38152440 2024-09-10T21:25:28.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:28.830+0000 7fad2f400640 1 -- 172.21.15.17:0/3160754122 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fad2001c460 con 0x7fad38152440 2024-09-10T21:25:28.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:28.830+0000 7fad2f400640 1 --2- 172.21.15.17:0/3160754122 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fad140725e0 0x7fad14074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:28.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:28.830+0000 7fad2f400640 1 -- 172.21.15.17:0/3160754122 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fad20093560 con 0x7fad38152440 2024-09-10T21:25:28.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:28.830+0000 7fad37400640 1 --2- 172.21.15.17:0/3160754122 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fad140725e0 0x7fad14074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:28.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:28.830+0000 7fad37400640 1 --2- 172.21.15.17:0/3160754122 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fad140725e0 0x7fad14074aa0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fad280af000 tx=0x7fad2801afd0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:28.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:28.834+0000 7fad2f400640 1 -- 172.21.15.17:0/3160754122 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fad2005e7a0 con 0x7fad38152440 2024-09-10T21:25:29.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:29.006+0000 7fad3e200640 1 -- 172.21.15.17:0/3160754122 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fad380ca7c0 con 0x7fad38152440 2024-09-10T21:25:29.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:29.006+0000 7fad2f400640 1 -- 172.21.15.17:0/3160754122 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18950 (secure 0 0 0) 0x7fad2005e0a0 con 0x7fad38152440 2024-09-10T21:25:29.012 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-10T21:25:29.012 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":27,"fsid":"5e2f023f-d6a6-49f4-a256-f4807dce3739","created":"2024-09-10T21:24:21.750712+0000","modified":"2024-09-10T21:25:28.499315+0000","last_up_change":"2024-09-10T21:24:35.877801+0000","last_in_change":"2024-09-10T21:24:25.831127+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:24:35.362670+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T21:24:45.211689+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T21:24:46.209784+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T21:24:49.871300+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0f5fabc0-434e-4066-b430-233969a548f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":4105489867},{"type":"v1","addr":"172.21.15.17:6809","nonce":4105489867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":4105489867},{"type":"v1","addr":"172.21.15.17:6811","nonce":4105489867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":4105489867},{"type":"v1","addr":"172.21.15.17:6815","nonce":4105489867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":4105489867},{"type":"v1","addr":"172.21.15.17:6813","nonce":4105489867}]},"public_addr":"172.21.15.17:6809/4105489867","cluster_addr":"172.21.15.17:6811/4105489867","heartbeat_back_addr":"172.21.15.17:6815/4105489867","heartbeat_front_addr":"172.21.15.17:6813/4105489867","state":["exists","up"]},{"osd":1,"uuid":"f6691cb3-2a91-4c90-a9fd-ece3c11dd6eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":2599842027},{"type":"v1","addr":"172.21.15.17:6825","nonce":2599842027}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":2599842027},{"type":"v1","addr":"172.21.15.17:6827","nonce":2599842027}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":2599842027},{"type":"v1","addr":"172.21.15.17:6831","nonce":2599842027}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":2599842027},{"type":"v1","addr":"172.21.15.17:6829","nonce":2599842027}]},"public_addr":"172.21.15.17:6825/2599842027","cluster_addr":"172.21.15.17:6827/2599842027","heartbeat_back_addr":"172.21.15.17:6831/2599842027","heartbeat_front_addr":"172.21.15.17:6829/2599842027","state":["exists","up"]},{"osd":2,"uuid":"cb61adb0-a216-437b-9598-06fa7c84aa64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":1248243634},{"type":"v1","addr":"172.21.15.17:6817","nonce":1248243634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":1248243634},{"type":"v1","addr":"172.21.15.17:6819","nonce":1248243634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":1248243634},{"type":"v1","addr":"172.21.15.17:6823","nonce":1248243634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":1248243634},{"type":"v1","addr":"172.21.15.17:6821","nonce":1248243634}]},"public_addr":"172.21.15.17:6817/1248243634","cluster_addr":"172.21.15.17:6819/1248243634","heartbeat_back_addr":"172.21.15.17:6823/1248243634","heartbeat_front_addr":"172.21.15.17:6821/1248243634","state":["exists","up"]},{"osd":3,"uuid":"6d2da26a-e0bf-47dc-b020-af86e6bcfccc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":2314489522},{"type":"v1","addr":"172.21.15.17:6801","nonce":2314489522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":2314489522},{"type":"v1","addr":"172.21.15.17:6803","nonce":2314489522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":2314489522},{"type":"v1","addr":"172.21.15.17:6807","nonce":2314489522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":2314489522},{"type":"v1","addr":"172.21.15.17:6805","nonce":2314489522}]},"public_addr":"172.21.15.17:6801/2314489522","cluster_addr":"172.21.15.17:6803/2314489522","heartbeat_back_addr":"172.21.15.17:6807/2314489522","heartbeat_front_addr":"172.21.15.17:6805/2314489522","state":["exists","up"]},{"osd":4,"uuid":"c65071dd-9ca7-4f1d-a584-45dc121419a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":202520744},{"type":"v1","addr":"172.21.15.31:6811","nonce":202520744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":202520744},{"type":"v1","addr":"172.21.15.31:6813","nonce":202520744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":202520744},{"type":"v1","addr":"172.21.15.31:6817","nonce":202520744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":202520744},{"type":"v1","addr":"172.21.15.31:6815","nonce":202520744}]},"public_addr":"172.21.15.31:6811/202520744","cluster_addr":"172.21.15.31:6813/202520744","heartbeat_back_addr":"172.21.15.31:6817/202520744","heartbeat_front_addr":"172.21.15.31:6815/202520744","state":["exists","up"]},{"osd":5,"uuid":"235389de-7777-4744-81b8-d0bfc11190e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":1472711057},{"type":"v1","addr":"172.21.15.31:6801","nonce":1472711057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":1472711057},{"type":"v1","addr":"172.21.15.31:6803","nonce":1472711057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":1472711057},{"type":"v1","addr":"172.21.15.31:6807","nonce":1472711057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":1472711057},{"type":"v1","addr":"172.21.15.31:6805","nonce":1472711057}]},"public_addr":"172.21.15.31:6801/1472711057","cluster_addr":"172.21.15.31:6803/1472711057","heartbeat_back_addr":"172.21.15.31:6807/1472711057","heartbeat_front_addr":"172.21.15.31:6805/1472711057","state":["exists","up"]},{"osd":6,"uuid":"cea70a57-3715-4ebb-8511-a528a9df2b1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":3754251089},{"type":"v1","addr":"172.21.15.31:6827","nonce":3754251089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":3754251089},{"type":"v1","addr":"172.21.15.31:6829","nonce":3754251089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":3754251089},{"type":"v1","addr":"172.21.15.31:6833","nonce":3754251089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":3754251089},{"type":"v1","addr":"172.21.15.31:6831","nonce":3754251089}]},"public_addr":"172.21.15.31:6827/3754251089","cluster_addr":"172.21.15.31:6829/3754251089","heartbeat_back_addr":"172.21.15.31:6833/3754251089","heartbeat_front_addr":"172.21.15.31:6831/3754251089","state":["exists","up"]},{"osd":7,"uuid":"01cc111b-a45f-4ca3-9cc1-7fe0afbeba45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":3948648934},{"type":"v1","addr":"172.21.15.31:6819","nonce":3948648934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":3948648934},{"type":"v1","addr":"172.21.15.31:6821","nonce":3948648934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":3948648934},{"type":"v1","addr":"172.21.15.31:6825","nonce":3948648934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":3948648934},{"type":"v1","addr":"172.21.15.31:6823","nonce":3948648934}]},"public_addr":"172.21.15.31:6819/3948648934","cluster_addr":"172.21.15.31:6821/3948648934","heartbeat_back_addr":"172.21.15.31:6825/3948648934","heartbeat_front_addr":"172.21.15.31:6823/3948648934","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:24:32.606989+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:24:32.891535+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:24:33.330793+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:24:32.811533+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:24:33.894624+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:24:32.773817+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:24:32.901865+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:24:32.907207+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.d","mappings":[{"from":5,"to":1}]},{"pgid":"3.e","mappings":[{"from":7,"to":2}]},{"pgid":"3.28","mappings":[{"from":5,"to":2}]},{"pgid":"3.29","mappings":[{"from":3,"to":2}]},{"pgid":"4.a","mappings":[{"from":1,"to":2}]},{"pgid":"4.14","mappings":[{"from":1,"to":5}]},{"pgid":"4.1b","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:25:29.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:29.010+0000 7fad3e200640 1 -- 172.21.15.17:0/3160754122 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fad140725e0 msgr2=0x7fad14074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:29.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:29.010+0000 7fad3e200640 1 --2- 172.21.15.17:0/3160754122 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fad140725e0 0x7fad14074aa0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fad280af000 tx=0x7fad2801afd0 comp rx=0 tx=0).stop 2024-09-10T21:25:29.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:29.010+0000 7fad3e200640 1 -- 172.21.15.17:0/3160754122 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fad38152440 msgr2=0x7fad38151720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:29.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:29.010+0000 7fad3e200640 1 --2- 172.21.15.17:0/3160754122 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fad38152440 0x7fad38151720 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fad2000c450 tx=0x7fad2000c910 comp rx=0 tx=0).stop 2024-09-10T21:25:29.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:29.010+0000 7fad3e200640 1 -- 172.21.15.17:0/3160754122 shutdown_connections 2024-09-10T21:25:29.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:29.010+0000 7fad3e200640 1 --2- 172.21.15.17:0/3160754122 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fad140725e0 0x7fad14074aa0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:29.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:29.010+0000 7fad3e200640 1 --2- 172.21.15.17:0/3160754122 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fad38152820 0x7fad381f6b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:29.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:29.010+0000 7fad3e200640 1 --2- 172.21.15.17:0/3160754122 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fad38152440 0x7fad38151720 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:29.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:29.010+0000 7fad3e200640 1 --2- 172.21.15.17:0/3160754122 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fad380cdf20 0x7fad38150b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:29.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:29.010+0000 7fad3e200640 1 -- 172.21.15.17:0/3160754122 >> 172.21.15.17:0/3160754122 conn(0x7fad380702f0 msgr2=0x7fad38157580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:29.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:29.010+0000 7fad3e200640 1 -- 172.21.15.17:0/3160754122 shutdown_connections 2024-09-10T21:25:29.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:29.010+0000 7fad3e200640 1 -- 172.21.15.17:0/3160754122 wait complete. 2024-09-10T21:25:29.034 INFO:tasks.mon_thrash.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-10T21:24:35.362670+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-10T21:24:45.211689+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-10T21:24:46.209784+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-09-10T21:24:49.871300+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-09-10T21:25:29.034 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-10T21:25:29.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:29.178+0000 7f8ef9000640 1 -- 172.21.15.17:0/3532718378 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8ef40c96f0 msgr2=0x7f8ef4152410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:29.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:29.178+0000 7f8ef9000640 1 --2- 172.21.15.17:0/3532718378 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8ef40c96f0 0x7f8ef4152410 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f8edc009900 tx=0x7f8edc01c8e0 comp rx=0 tx=0).stop 2024-09-10T21:25:29.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:29.178+0000 7f8ef9000640 1 -- 172.21.15.17:0/3532718378 shutdown_connections 2024-09-10T21:25:29.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:29.178+0000 7f8ef9000640 1 --2- 172.21.15.17:0/3532718378 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8ef40c34f0 0x7f8ef4152950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:29.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:29.178+0000 7f8ef9000640 1 --2- 172.21.15.17:0/3532718378 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8ef40c96f0 0x7f8ef4152410 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:29.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:29.178+0000 7f8ef9000640 1 -- 172.21.15.17:0/3532718378 >> 172.21.15.17:0/3532718378 conn(0x7f8ef406d220 msgr2=0x7f8ef4068990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:29.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:29.178+0000 7f8ef9000640 1 -- 172.21.15.17:0/3532718378 shutdown_connections 2024-09-10T21:25:29.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:29.178+0000 7f8ef9000640 1 -- 172.21.15.17:0/3532718378 wait complete. 2024-09-10T21:25:29.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:29.182+0000 7f8ef9000640 1 Processor -- start 2024-09-10T21:25:29.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:29.182+0000 7f8ef9000640 1 -- start start 2024-09-10T21:25:29.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:29.182+0000 7f8ef9000640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8ef40c34f0 0x7f8ef414cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:29.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:29.182+0000 7f8ef9000640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8ef40c9310 0x7f8ef40c5440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:29.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:29.182+0000 7f8ef9000640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8ef40c96f0 0x7f8ef41f7100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:29.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:29.182+0000 7f8ef9000640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f8ef4070580 con 0x7f8ef40c34f0 2024-09-10T21:25:29.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:29.182+0000 7f8ef9000640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f8ef4070400 con 0x7f8ef40c9310 2024-09-10T21:25:29.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:29.182+0000 7f8ef9000640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f8ef41657c0 con 0x7f8ef40c96f0 2024-09-10T21:25:29.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:29.182+0000 7f8ef1e00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8ef40c34f0 0x7f8ef414cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:29.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:29.182+0000 7f8ef1e00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8ef40c34f0 0x7f8ef414cbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43688/0 (socket says 172.21.15.17:43688) 2024-09-10T21:25:29.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:29.182+0000 7f8ef1e00640 1 -- 172.21.15.17:0/3553077566 learned_addr learned my addr 172.21.15.17:0/3553077566 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:25:29.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:29.182+0000 7f8eebe00640 1 --2- 172.21.15.17:0/3553077566 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8ef40c9310 0x7f8ef40c5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:29.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:29.182+0000 7f8ef2800640 1 --2- 172.21.15.17:0/3553077566 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8ef40c96f0 0x7f8ef41f7100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:29.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:29.182+0000 7f8ef1e00640 1 -- 172.21.15.17:0/3553077566 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8ef40c96f0 msgr2=0x7f8ef41f7100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:29.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:29.182+0000 7f8ef1e00640 1 --2- 172.21.15.17:0/3553077566 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8ef40c96f0 0x7f8ef41f7100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:29.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:29.182+0000 7f8ef1e00640 1 -- 172.21.15.17:0/3553077566 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8ef40c9310 msgr2=0x7f8ef40c5440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:29.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:29.182+0000 7f8ef1e00640 1 --2- 172.21.15.17:0/3553077566 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8ef40c9310 0x7f8ef40c5440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:29.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:29.182+0000 7f8ef1e00640 1 -- 172.21.15.17:0/3553077566 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ef4156490 con 0x7f8ef40c34f0 2024-09-10T21:25:29.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:29.182+0000 7f8eebe00640 1 --2- 172.21.15.17:0/3553077566 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8ef40c9310 0x7f8ef40c5440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:25:29.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:29.182+0000 7f8ef1e00640 1 --2- 172.21.15.17:0/3553077566 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8ef40c34f0 0x7f8ef414cbc0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f8ee400c480 tx=0x7f8ee400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:29.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:29.182+0000 7f8eea000640 1 -- 172.21.15.17:0/3553077566 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ee4012830 con 0x7f8ef40c34f0 2024-09-10T21:25:29.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:29.182+0000 7f8eea000640 1 -- 172.21.15.17:0/3553077566 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8ee4017070 con 0x7f8ef40c34f0 2024-09-10T21:25:29.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:29.182+0000 7f8eea000640 1 -- 172.21.15.17:0/3553077566 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ee4012d80 con 0x7f8ef40c34f0 2024-09-10T21:25:29.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:29.182+0000 7f8ef9000640 1 -- 172.21.15.17:0/3553077566 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ef41558d0 con 0x7f8ef40c34f0 2024-09-10T21:25:29.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:29.182+0000 7f8ef9000640 1 -- 172.21.15.17:0/3553077566 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ef41f7ad0 con 0x7f8ef40c34f0 2024-09-10T21:25:29.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:29.182+0000 7f8ef9000640 1 -- 172.21.15.17:0/3553077566 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ef4070960 con 0x7f8ef40c34f0 2024-09-10T21:25:29.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:29.186+0000 7f8eea000640 1 -- 172.21.15.17:0/3553077566 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f8ee401c460 con 0x7f8ef40c34f0 2024-09-10T21:25:29.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:29.186+0000 7f8eea000640 1 --2- 172.21.15.17:0/3553077566 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f8ed0072610 0x7f8ed0074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:29.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:29.186+0000 7f8eea000640 1 -- 172.21.15.17:0/3553077566 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f8ee4093460 con 0x7f8ef40c34f0 2024-09-10T21:25:29.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:29.186+0000 7f8eebe00640 1 --2- 172.21.15.17:0/3553077566 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f8ed0072610 0x7f8ed0074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:29.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:29.186+0000 7f8eebe00640 1 --2- 172.21.15.17:0/3553077566 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f8ed0072610 0x7f8ed0074ad0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f8edc0af000 tx=0x7f8edc01b120 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:29.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:29.190+0000 7f8eea000640 1 -- 172.21.15.17:0/3553077566 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f8ee405e730 con 0x7f8ef40c34f0 2024-09-10T21:25:29.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:29.362+0000 7f8ef9000640 1 -- 172.21.15.17:0/3553077566 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f8ef40c39c0 con 0x7f8ef40c34f0 2024-09-10T21:25:29.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:29.362+0000 7f8eea000640 1 -- 172.21.15.17:0/3553077566 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) ==== 93+0+10 (secure 0 0 0) 0x7f8ee405e030 con 0x7f8ef40c34f0 2024-09-10T21:25:29.368 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 1 2024-09-10T21:25:29.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:29.366+0000 7f8ef9000640 1 -- 172.21.15.17:0/3553077566 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f8ed0072610 msgr2=0x7f8ed0074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:29.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:29.366+0000 7f8ef9000640 1 --2- 172.21.15.17:0/3553077566 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f8ed0072610 0x7f8ed0074ad0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f8edc0af000 tx=0x7f8edc01b120 comp rx=0 tx=0).stop 2024-09-10T21:25:29.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:29.366+0000 7f8ef9000640 1 -- 172.21.15.17:0/3553077566 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8ef40c34f0 msgr2=0x7f8ef414cbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:29.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:29.366+0000 7f8ef9000640 1 --2- 172.21.15.17:0/3553077566 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8ef40c34f0 0x7f8ef414cbc0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f8ee400c480 tx=0x7f8ee400c940 comp rx=0 tx=0).stop 2024-09-10T21:25:29.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:29.366+0000 7f8ef9000640 1 -- 172.21.15.17:0/3553077566 shutdown_connections 2024-09-10T21:25:29.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:29.366+0000 7f8ef9000640 1 --2- 172.21.15.17:0/3553077566 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f8ed0072610 0x7f8ed0074ad0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:29.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:29.366+0000 7f8ef9000640 1 --2- 172.21.15.17:0/3553077566 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8ef40c96f0 0x7f8ef41f7100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:29.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:29.366+0000 7f8ef9000640 1 --2- 172.21.15.17:0/3553077566 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8ef40c9310 0x7f8ef40c5440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:29.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:29.366+0000 7f8ef9000640 1 --2- 172.21.15.17:0/3553077566 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8ef40c34f0 0x7f8ef414cbc0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:29.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:29.366+0000 7f8ef9000640 1 -- 172.21.15.17:0/3553077566 >> 172.21.15.17:0/3553077566 conn(0x7f8ef406d220 msgr2=0x7f8ef414bb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:29.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:29.366+0000 7f8ef9000640 1 -- 172.21.15.17:0/3553077566 shutdown_connections 2024-09-10T21:25:29.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:29.366+0000 7f8ef9000640 1 -- 172.21.15.17:0/3553077566 wait complete. 2024-09-10T21:25:29.389 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-10T21:25:29.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:29.534+0000 7f71d0800640 1 -- 172.21.15.17:0/816350410 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f71c80c3f70 msgr2=0x7f71c8158520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:29.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:29.534+0000 7f71d0800640 1 --2- 172.21.15.17:0/816350410 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f71c80c3f70 0x7f71c8158520 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f71c4009c80 tx=0x7f71c401cda0 comp rx=0 tx=0).stop 2024-09-10T21:25:29.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:29.534+0000 7f71d0800640 1 -- 172.21.15.17:0/816350410 shutdown_connections 2024-09-10T21:25:29.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:29.534+0000 7f71d0800640 1 --2- 172.21.15.17:0/816350410 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f71c80c3f70 0x7f71c8158520 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:29.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:29.534+0000 7f71d0800640 1 --2- 172.21.15.17:0/816350410 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f71c8152760 0x7f71c80ca8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:29.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:29.534+0000 7f71d0800640 1 -- 172.21.15.17:0/816350410 >> 172.21.15.17:0/816350410 conn(0x7f71c8072270 msgr2=0x7f71c8072680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:29.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:29.534+0000 7f71d0800640 1 -- 172.21.15.17:0/816350410 shutdown_connections 2024-09-10T21:25:29.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:29.534+0000 7f71d0800640 1 -- 172.21.15.17:0/816350410 wait complete. 2024-09-10T21:25:29.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:29.538+0000 7f71d0800640 1 Processor -- start 2024-09-10T21:25:29.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:29.538+0000 7f71d0800640 1 -- start start 2024-09-10T21:25:29.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:29.538+0000 7f71d0800640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f71c80b0bb0 0x7f71c814a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:29.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:29.538+0000 7f71d0800640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f71c80c3f70 0x7f71c80c7980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:29.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:29.538+0000 7f71d0800640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f71c8152760 0x7f71c81f70b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:29.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:29.538+0000 7f71d0800640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f71c80c9590 con 0x7f71c80c3f70 2024-09-10T21:25:29.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:29.538+0000 7f71d0800640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f71c8158e70 con 0x7f71c8152760 2024-09-10T21:25:29.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:29.538+0000 7f71d0800640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f71c80c9710 con 0x7f71c80b0bb0 2024-09-10T21:25:29.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:29.538+0000 7f71cd400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f71c80c3f70 0x7f71c80c7980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:29.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:29.538+0000 7f71cde00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f71c80b0bb0 0x7f71c814a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:29.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:29.538+0000 7f71cd400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f71c80c3f70 0x7f71c80c7980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43690/0 (socket says 172.21.15.17:43690) 2024-09-10T21:25:29.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:29.538+0000 7f71cd400640 1 -- 172.21.15.17:0/3756439147 learned_addr learned my addr 172.21.15.17:0/3756439147 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:25:29.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:29.538+0000 7f71ce800640 1 --2- 172.21.15.17:0/3756439147 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f71c8152760 0x7f71c81f70b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:29.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:29.538+0000 7f71cd400640 1 -- 172.21.15.17:0/3756439147 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f71c80b0bb0 msgr2=0x7f71c814a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:29.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:29.538+0000 7f71cd400640 1 --2- 172.21.15.17:0/3756439147 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f71c80b0bb0 0x7f71c814a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:29.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:29.538+0000 7f71cd400640 1 -- 172.21.15.17:0/3756439147 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f71c8152760 msgr2=0x7f71c81f70b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:29.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:29.538+0000 7f71cd400640 1 --2- 172.21.15.17:0/3756439147 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f71c8152760 0x7f71c81f70b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:29.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:29.538+0000 7f71cd400640 1 -- 172.21.15.17:0/3756439147 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71c80c7ec0 con 0x7f71c80c3f70 2024-09-10T21:25:29.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:29.538+0000 7f71cd400640 1 --2- 172.21.15.17:0/3756439147 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f71c80c3f70 0x7f71c80c7980 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f71b800c3f0 tx=0x7f71b800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:29.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:29.538+0000 7f71c2000640 1 -- 172.21.15.17:0/3756439147 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71b8017070 con 0x7f71c80c3f70 2024-09-10T21:25:29.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:29.538+0000 7f71c2000640 1 -- 172.21.15.17:0/3756439147 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f71b80053c0 con 0x7f71c80c3f70 2024-09-10T21:25:29.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:29.538+0000 7f71d0800640 1 -- 172.21.15.17:0/3756439147 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71c814b250 con 0x7f71c80c3f70 2024-09-10T21:25:29.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:29.538+0000 7f71c2000640 1 -- 172.21.15.17:0/3756439147 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71b8003860 con 0x7f71c80c3f70 2024-09-10T21:25:29.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:29.538+0000 7f71d0800640 1 -- 172.21.15.17:0/3756439147 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71c81f7ec0 con 0x7f71c80c3f70 2024-09-10T21:25:29.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:29.542+0000 7f71d0800640 1 -- 172.21.15.17:0/3756439147 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71c8158520 con 0x7f71c80c3f70 2024-09-10T21:25:29.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:29.542+0000 7f71c2000640 1 -- 172.21.15.17:0/3756439147 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f71b8003a00 con 0x7f71c80c3f70 2024-09-10T21:25:29.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:29.542+0000 7f71c2000640 1 --2- 172.21.15.17:0/3756439147 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f71ac0725e0 0x7f71ac074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:29.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:29.542+0000 7f71c2000640 1 -- 172.21.15.17:0/3756439147 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f71b8092ba0 con 0x7f71c80c3f70 2024-09-10T21:25:29.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:29.542+0000 7f71cde00640 1 --2- 172.21.15.17:0/3756439147 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f71ac0725e0 0x7f71ac074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:29.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:29.546+0000 7f71cde00640 1 --2- 172.21.15.17:0/3756439147 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f71ac0725e0 0x7f71ac074aa0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f71bc009be0 tx=0x7f71bc0091e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:29.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:29.546+0000 7f71c2000640 1 -- 172.21.15.17:0/3756439147 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f71c80c7ec0 con 0x7f71c80c3f70 2024-09-10T21:25:29.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:29.722+0000 7f71d0800640 1 -- 172.21.15.17:0/3756439147 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f71c814e360 con 0x7f71c80c3f70 2024-09-10T21:25:29.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:29.722+0000 7f71c2000640 1 -- 172.21.15.17:0/3756439147 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) ==== 104+0+11 (secure 0 0 0) 0x7f71b8001090 con 0x7f71c80c3f70 2024-09-10T21:25:29.726 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 32 2024-09-10T21:25:29.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:29.722+0000 7f71d0800640 1 -- 172.21.15.17:0/3756439147 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f71ac0725e0 msgr2=0x7f71ac074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:29.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:29.722+0000 7f71d0800640 1 --2- 172.21.15.17:0/3756439147 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f71ac0725e0 0x7f71ac074aa0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f71bc009be0 tx=0x7f71bc0091e0 comp rx=0 tx=0).stop 2024-09-10T21:25:29.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:29.722+0000 7f71d0800640 1 -- 172.21.15.17:0/3756439147 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f71c80c3f70 msgr2=0x7f71c80c7980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:29.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:29.722+0000 7f71d0800640 1 --2- 172.21.15.17:0/3756439147 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f71c80c3f70 0x7f71c80c7980 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f71b800c3f0 tx=0x7f71b800c8b0 comp rx=0 tx=0).stop 2024-09-10T21:25:29.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:29.722+0000 7f71d0800640 1 -- 172.21.15.17:0/3756439147 shutdown_connections 2024-09-10T21:25:29.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:29.722+0000 7f71d0800640 1 --2- 172.21.15.17:0/3756439147 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f71ac0725e0 0x7f71ac074aa0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:29.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:29.722+0000 7f71d0800640 1 --2- 172.21.15.17:0/3756439147 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f71c8152760 0x7f71c81f70b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:29.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:29.722+0000 7f71d0800640 1 --2- 172.21.15.17:0/3756439147 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f71c80c3f70 0x7f71c80c7980 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:29.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:29.722+0000 7f71d0800640 1 --2- 172.21.15.17:0/3756439147 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f71c80b0bb0 0x7f71c814a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:29.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:29.722+0000 7f71d0800640 1 -- 172.21.15.17:0/3756439147 >> 172.21.15.17:0/3756439147 conn(0x7f71c8072270 msgr2=0x7f71c8161050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:29.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:29.722+0000 7f71d0800640 1 -- 172.21.15.17:0/3756439147 shutdown_connections 2024-09-10T21:25:29.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:29.722+0000 7f71d0800640 1 -- 172.21.15.17:0/3756439147 wait complete. 2024-09-10T21:25:29.747 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-10T21:25:29.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:29.894+0000 7f4ff3400640 1 -- 172.21.15.17:0/3098784676 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4fec152760 msgr2=0x7f4fec0ca8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:29.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:29.894+0000 7f4ff3400640 1 --2- 172.21.15.17:0/3098784676 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4fec152760 0x7f4fec0ca8c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f4fe0009900 tx=0x7f4fe001c8e0 comp rx=0 tx=0).stop 2024-09-10T21:25:29.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:29.894+0000 7f4ff3400640 1 -- 172.21.15.17:0/3098784676 shutdown_connections 2024-09-10T21:25:29.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:29.894+0000 7f4ff3400640 1 --2- 172.21.15.17:0/3098784676 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f4fec0b0bb0 0x7f4fec0c8e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:29.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:29.894+0000 7f4ff3400640 1 --2- 172.21.15.17:0/3098784676 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4fec152760 0x7f4fec0ca8c0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:29.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:29.894+0000 7f4ff3400640 1 -- 172.21.15.17:0/3098784676 >> 172.21.15.17:0/3098784676 conn(0x7f4fec072270 msgr2=0x7f4fec072680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:29.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:29.894+0000 7f4ff3400640 1 -- 172.21.15.17:0/3098784676 shutdown_connections 2024-09-10T21:25:29.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:29.894+0000 7f4ff3400640 1 -- 172.21.15.17:0/3098784676 wait complete. 2024-09-10T21:25:29.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:29.894+0000 7f4ff3400640 1 Processor -- start 2024-09-10T21:25:29.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:29.894+0000 7f4ff3400640 1 -- start start 2024-09-10T21:25:29.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:29.894+0000 7f4ff3400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4fec0b0bb0 0x7f4fec14fee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:29.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:29.894+0000 7f4ff3400640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f4fec0c3f70 0x7f4fec157530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:29.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:29.894+0000 7f4ff3400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4fec152760 0x7f4fec157a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:29.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:29.894+0000 7f4ff3400640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f4fec165840 con 0x7f4fec0b0bb0 2024-09-10T21:25:29.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:29.894+0000 7f4ff3400640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f4fec1656c0 con 0x7f4fec152760 2024-09-10T21:25:29.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:29.894+0000 7f4ff3400640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f4fec1659c0 con 0x7f4fec0c3f70 2024-09-10T21:25:29.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:29.894+0000 7f4ff0a00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4fec0b0bb0 0x7f4fec14fee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:29.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:29.894+0000 7f4ff0a00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4fec0b0bb0 0x7f4fec14fee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43720/0 (socket says 172.21.15.17:43720) 2024-09-10T21:25:29.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:29.894+0000 7f4ff0a00640 1 -- 172.21.15.17:0/219364177 learned_addr learned my addr 172.21.15.17:0/219364177 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:25:29.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:29.894+0000 7f4febe00640 1 --2- 172.21.15.17:0/219364177 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f4fec0c3f70 0x7f4fec157530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:29.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:29.894+0000 7f4ff1400640 1 --2- 172.21.15.17:0/219364177 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4fec152760 0x7f4fec157a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:29.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:29.894+0000 7f4ff0a00640 1 -- 172.21.15.17:0/219364177 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f4fec0c3f70 msgr2=0x7f4fec157530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:29.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:29.894+0000 7f4ff0a00640 1 --2- 172.21.15.17:0/219364177 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f4fec0c3f70 0x7f4fec157530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:29.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:29.894+0000 7f4ff0a00640 1 -- 172.21.15.17:0/219364177 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4fec152760 msgr2=0x7f4fec157a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:29.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:29.894+0000 7f4ff0a00640 1 --2- 172.21.15.17:0/219364177 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4fec152760 0x7f4fec157a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:29.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:29.894+0000 7f4ff0a00640 1 -- 172.21.15.17:0/219364177 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fec151b40 con 0x7f4fec0b0bb0 2024-09-10T21:25:29.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:29.894+0000 7f4ff1400640 1 --2- 172.21.15.17:0/219364177 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4fec152760 0x7f4fec157a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:25:29.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:29.898+0000 7f4ff0a00640 1 --2- 172.21.15.17:0/219364177 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4fec0b0bb0 0x7f4fec14fee0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f4fe0009850 tx=0x7f4fe00a5730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:29.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:29.898+0000 7f4fe8c00640 1 -- 172.21.15.17:0/219364177 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fe00b2040 con 0x7f4fec0b0bb0 2024-09-10T21:25:29.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:29.898+0000 7f4fe8c00640 1 -- 172.21.15.17:0/219364177 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4fe00043b0 con 0x7f4fec0b0bb0 2024-09-10T21:25:29.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:29.898+0000 7f4fe8c00640 1 -- 172.21.15.17:0/219364177 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fe0005290 con 0x7f4fec0b0bb0 2024-09-10T21:25:29.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:29.898+0000 7f4ff3400640 1 -- 172.21.15.17:0/219364177 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fec14f6b0 con 0x7f4fec0b0bb0 2024-09-10T21:25:29.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:29.898+0000 7f4ff3400640 1 -- 172.21.15.17:0/219364177 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fec153e60 con 0x7f4fec0b0bb0 2024-09-10T21:25:29.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:29.898+0000 7f4ff3400640 1 -- 172.21.15.17:0/219364177 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4fec0cde00 con 0x7f4fec0b0bb0 2024-09-10T21:25:29.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:29.898+0000 7f4fe8c00640 1 -- 172.21.15.17:0/219364177 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f4fe00bf060 con 0x7f4fec0b0bb0 2024-09-10T21:25:29.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:29.898+0000 7f4fe8c00640 1 --2- 172.21.15.17:0/219364177 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f4fcc072610 0x7f4fcc074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:29.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:29.898+0000 7f4fe8c00640 1 -- 172.21.15.17:0/219364177 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f4fe012c4f0 con 0x7f4fec0b0bb0 2024-09-10T21:25:29.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:29.902+0000 7f4febe00640 1 --2- 172.21.15.17:0/219364177 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f4fcc072610 0x7f4fcc074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:29.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:29.902+0000 7f4febe00640 1 --2- 172.21.15.17:0/219364177 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f4fcc072610 0x7f4fcc074ad0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f4fec1f6050 tx=0x7f4fd80091e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:29.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:29.902+0000 7f4fe8c00640 1 -- 172.21.15.17:0/219364177 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f4fe00164d0 con 0x7f4fec0b0bb0 2024-09-10T21:25:30.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:30.078+0000 7f4ff3400640 1 -- 172.21.15.17:0/219364177 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f4fec0b2b80 con 0x7f4fec0b0bb0 2024-09-10T21:25:30.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:30.078+0000 7f4fe8c00640 1 -- 172.21.15.17:0/219364177 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) ==== 100+0+11 (secure 0 0 0) 0x7f4fe00f70f0 con 0x7f4fec0b0bb0 2024-09-10T21:25:30.083 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 64 2024-09-10T21:25:30.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:30.082+0000 7f4ff3400640 1 -- 172.21.15.17:0/219364177 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f4fcc072610 msgr2=0x7f4fcc074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:30.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:30.082+0000 7f4ff3400640 1 --2- 172.21.15.17:0/219364177 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f4fcc072610 0x7f4fcc074ad0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f4fec1f6050 tx=0x7f4fd80091e0 comp rx=0 tx=0).stop 2024-09-10T21:25:30.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:30.082+0000 7f4ff3400640 1 -- 172.21.15.17:0/219364177 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4fec0b0bb0 msgr2=0x7f4fec14fee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:30.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:30.082+0000 7f4ff3400640 1 --2- 172.21.15.17:0/219364177 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4fec0b0bb0 0x7f4fec14fee0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f4fe0009850 tx=0x7f4fe00a5730 comp rx=0 tx=0).stop 2024-09-10T21:25:30.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:30.082+0000 7f4ff3400640 1 -- 172.21.15.17:0/219364177 shutdown_connections 2024-09-10T21:25:30.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:30.082+0000 7f4ff3400640 1 --2- 172.21.15.17:0/219364177 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f4fcc072610 0x7f4fcc074ad0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:30.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:30.082+0000 7f4ff3400640 1 --2- 172.21.15.17:0/219364177 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4fec152760 0x7f4fec157a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:30.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:30.082+0000 7f4ff3400640 1 --2- 172.21.15.17:0/219364177 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f4fec0c3f70 0x7f4fec157530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:30.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:30.082+0000 7f4ff3400640 1 --2- 172.21.15.17:0/219364177 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4fec0b0bb0 0x7f4fec14fee0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:30.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:30.082+0000 7f4ff3400640 1 -- 172.21.15.17:0/219364177 >> 172.21.15.17:0/219364177 conn(0x7f4fec072270 msgr2=0x7f4fec0b3380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:30.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:30.082+0000 7f4ff3400640 1 -- 172.21.15.17:0/219364177 shutdown_connections 2024-09-10T21:25:30.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:30.082+0000 7f4ff3400640 1 -- 172.21.15.17:0/219364177 wait complete. 2024-09-10T21:25:30.105 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-09-10T21:25:30.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:30.254+0000 7f7874600640 1 -- 172.21.15.17:0/2140470434 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f786c151030 msgr2=0x7f786c073780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:30.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:30.254+0000 7f7874600640 1 --2- 172.21.15.17:0/2140470434 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f786c151030 0x7f786c073780 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f786000a950 tx=0x7f786001dae0 comp rx=0 tx=0).stop 2024-09-10T21:25:30.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:30.254+0000 7f7874600640 1 -- 172.21.15.17:0/2140470434 shutdown_connections 2024-09-10T21:25:30.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:30.254+0000 7f7874600640 1 --2- 172.21.15.17:0/2140470434 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f786c151030 0x7f786c073780 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:30.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:30.254+0000 7f7874600640 1 -- 172.21.15.17:0/2140470434 >> 172.21.15.17:0/2140470434 conn(0x7f786c0ccd10 msgr2=0x7f786c0cd120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:30.261 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:30.254+0000 7f7874600640 1 -- 172.21.15.17:0/2140470434 shutdown_connections 2024-09-10T21:25:30.261 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:30.254+0000 7f7874600640 1 -- 172.21.15.17:0/2140470434 wait complete. 2024-09-10T21:25:30.261 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:30.258+0000 7f7874600640 1 Processor -- start 2024-09-10T21:25:30.261 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:30.258+0000 7f7874600640 1 -- start start 2024-09-10T21:25:30.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:30.258+0000 7f7874600640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f786c070190 0x7f786c14bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:30.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:30.258+0000 7f7874600640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f786c0c7890 0x7f786c14d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:30.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:30.258+0000 7f7874600640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f786c151030 0x7f786c146070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:30.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:30.258+0000 7f7874600640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f786c070b30 con 0x7f786c070190 2024-09-10T21:25:30.264 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:30.258+0000 7f7874600640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f786c0709b0 con 0x7f786c0c7890 2024-09-10T21:25:30.264 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:30.258+0000 7f7874600640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f786c06d860 con 0x7f786c151030 2024-09-10T21:25:30.264 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:30.258+0000 7f7871c00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f786c070190 0x7f786c14bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:30.264 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:30.258+0000 7f7871c00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f786c070190 0x7f786c14bdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43726/0 (socket says 172.21.15.17:43726) 2024-09-10T21:25:30.264 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:30.258+0000 7f7871c00640 1 -- 172.21.15.17:0/758143548 learned_addr learned my addr 172.21.15.17:0/758143548 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:25:30.264 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:30.258+0000 7f7872600640 1 --2- 172.21.15.17:0/758143548 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f786c151030 0x7f786c146070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:30.264 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:30.258+0000 7f7871200640 1 --2- 172.21.15.17:0/758143548 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f786c0c7890 0x7f786c14d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:30.264 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:30.258+0000 7f7871c00640 1 -- 172.21.15.17:0/758143548 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f786c151030 msgr2=0x7f786c146070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:30.264 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:30.258+0000 7f7871c00640 1 --2- 172.21.15.17:0/758143548 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f786c151030 0x7f786c146070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:30.264 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:30.258+0000 7f7871c00640 1 -- 172.21.15.17:0/758143548 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f786c0c7890 msgr2=0x7f786c14d0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:30.264 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:30.258+0000 7f7871c00640 1 --2- 172.21.15.17:0/758143548 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f786c0c7890 0x7f786c14d0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:30.265 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:30.258+0000 7f7871c00640 1 -- 172.21.15.17:0/758143548 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f786c14c680 con 0x7f786c070190 2024-09-10T21:25:30.265 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:30.258+0000 7f7871c00640 1 --2- 172.21.15.17:0/758143548 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f786c070190 0x7f786c14bdb0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f786001a6f0 tx=0x7f7860005ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:30.265 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:30.258+0000 7f7869600640 1 -- 172.21.15.17:0/758143548 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78600b3040 con 0x7f786c070190 2024-09-10T21:25:30.265 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:30.258+0000 7f7869600640 1 -- 172.21.15.17:0/758143548 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f78600038f0 con 0x7f786c070190 2024-09-10T21:25:30.265 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:30.258+0000 7f7874600640 1 -- 172.21.15.17:0/758143548 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f786c150820 con 0x7f786c070190 2024-09-10T21:25:30.265 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:30.258+0000 7f7869600640 1 -- 172.21.15.17:0/758143548 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7860003cc0 con 0x7f786c070190 2024-09-10T21:25:30.265 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:30.258+0000 7f7874600640 1 -- 172.21.15.17:0/758143548 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f786c14cbd0 con 0x7f786c070190 2024-09-10T21:25:30.266 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:30.262+0000 7f7874600640 1 -- 172.21.15.17:0/758143548 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f786c0c8dc0 con 0x7f786c070190 2024-09-10T21:25:30.266 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:30.262+0000 7f7869600640 1 -- 172.21.15.17:0/758143548 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f786000b040 con 0x7f786c070190 2024-09-10T21:25:30.266 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:30.262+0000 7f7869600640 1 --2- 172.21.15.17:0/758143548 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f784c0725e0 0x7f784c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:30.266 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:30.262+0000 7f7869600640 1 -- 172.21.15.17:0/758143548 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f786012dc00 con 0x7f786c070190 2024-09-10T21:25:30.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:30.262+0000 7f7871200640 1 --2- 172.21.15.17:0/758143548 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f784c0725e0 0x7f784c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:30.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:30.262+0000 7f7871200640 1 --2- 172.21.15.17:0/758143548 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f784c0725e0 0x7f784c074aa0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f786c0738e0 tx=0x7f7858009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:30.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:30.266+0000 7f7869600640 1 -- 172.21.15.17:0/758143548 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f7860017760 con 0x7f786c070190 2024-09-10T21:25:30.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:30.442+0000 7f7874600640 1 -- 172.21.15.17:0/758143548 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f786c073630 con 0x7f786c070190 2024-09-10T21:25:30.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:30.442+0000 7f7869600640 1 -- 172.21.15.17:0/758143548 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) ==== 103+0+11 (secure 0 0 0) 0x7f78600f8ed0 con 0x7f786c070190 2024-09-10T21:25:30.447 INFO:teuthology.orchestra.run.smithi017.stdout:pg_num: 32 2024-09-10T21:25:30.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:30.446+0000 7f7874600640 1 -- 172.21.15.17:0/758143548 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f784c0725e0 msgr2=0x7f784c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:30.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:30.446+0000 7f7874600640 1 --2- 172.21.15.17:0/758143548 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f784c0725e0 0x7f784c074aa0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f786c0738e0 tx=0x7f7858009040 comp rx=0 tx=0).stop 2024-09-10T21:25:30.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:30.446+0000 7f7874600640 1 -- 172.21.15.17:0/758143548 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f786c070190 msgr2=0x7f786c14bdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:30.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:30.446+0000 7f7874600640 1 --2- 172.21.15.17:0/758143548 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f786c070190 0x7f786c14bdb0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f786001a6f0 tx=0x7f7860005ca0 comp rx=0 tx=0).stop 2024-09-10T21:25:30.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:30.446+0000 7f7874600640 1 -- 172.21.15.17:0/758143548 shutdown_connections 2024-09-10T21:25:30.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:30.446+0000 7f7874600640 1 --2- 172.21.15.17:0/758143548 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f784c0725e0 0x7f784c074aa0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:30.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:30.446+0000 7f7874600640 1 --2- 172.21.15.17:0/758143548 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f786c151030 0x7f786c146070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:30.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:30.446+0000 7f7874600640 1 --2- 172.21.15.17:0/758143548 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f786c0c7890 0x7f786c14d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:30.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:30.446+0000 7f7874600640 1 --2- 172.21.15.17:0/758143548 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f786c070190 0x7f786c14bdb0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:30.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:30.446+0000 7f7874600640 1 -- 172.21.15.17:0/758143548 >> 172.21.15.17:0/758143548 conn(0x7f786c0ccd10 msgr2=0x7f786c14e690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:30.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:30.446+0000 7f7874600640 1 -- 172.21.15.17:0/758143548 shutdown_connections 2024-09-10T21:25:30.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:30.446+0000 7f7874600640 1 -- 172.21.15.17:0/758143548 wait complete. 2024-09-10T21:25:30.468 INFO:tasks.mon_thrash.ceph_manager:waiting for clean 2024-09-10T21:25:30.468 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T21:25:30.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:30.610+0000 7f21e6a00640 1 -- 172.21.15.17:0/2310747736 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f21e0070450 msgr2=0x7f21e00c7130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:30.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:30.610+0000 7f21e6a00640 1 --2- 172.21.15.17:0/2310747736 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f21e0070450 0x7f21e00c7130 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f21cc009840 tx=0x7f21cc01c820 comp rx=0 tx=0).stop 2024-09-10T21:25:30.616 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:30.610+0000 7f21e6a00640 1 -- 172.21.15.17:0/2310747736 shutdown_connections 2024-09-10T21:25:30.616 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:30.610+0000 7f21e6a00640 1 --2- 172.21.15.17:0/2310747736 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f21e0070450 0x7f21e00c7130 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:30.616 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:30.610+0000 7f21e6a00640 1 -- 172.21.15.17:0/2310747736 >> 172.21.15.17:0/2310747736 conn(0x7f21e006dd30 msgr2=0x7f21e006e140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:30.616 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:30.610+0000 7f21e6a00640 1 -- 172.21.15.17:0/2310747736 shutdown_connections 2024-09-10T21:25:30.616 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:30.610+0000 7f21e6a00640 1 -- 172.21.15.17:0/2310747736 wait complete. 2024-09-10T21:25:30.616 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:30.614+0000 7f21e6a00640 1 Processor -- start 2024-09-10T21:25:30.617 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:30.614+0000 7f21e6a00640 1 -- start start 2024-09-10T21:25:30.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:30.614+0000 7f21e6a00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f21e0070450 0x7f21e0156d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:30.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:30.614+0000 7f21e6a00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f21e0074f70 0x7f21e014b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:30.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:30.614+0000 7f21e6a00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f21e015bf40 0x7f21e014b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:30.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:30.614+0000 7f21e6a00640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f21e006ef60 con 0x7f21e015bf40 2024-09-10T21:25:30.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:30.614+0000 7f21e6a00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f21e0069b90 con 0x7f21e0074f70 2024-09-10T21:25:30.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:30.614+0000 7f21e6a00640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f21e006f0e0 con 0x7f21e0070450 2024-09-10T21:25:30.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:30.614+0000 7f21e4a00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f21e015bf40 0x7f21e014b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:30.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:30.614+0000 7f21dfe00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f21e0070450 0x7f21e0156d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:30.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:30.614+0000 7f21dfe00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f21e0070450 0x7f21e0156d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3301/0 says I am v2:172.21.15.17:47182/0 (socket says 172.21.15.17:47182) 2024-09-10T21:25:30.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:30.614+0000 7f21dfe00640 1 -- 172.21.15.17:0/2657819711 learned_addr learned my addr 172.21.15.17:0/2657819711 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:25:30.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:30.614+0000 7f21df400640 1 --2- 172.21.15.17:0/2657819711 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f21e0074f70 0x7f21e014b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:30.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:30.614+0000 7f21dfe00640 1 -- 172.21.15.17:0/2657819711 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f21e0074f70 msgr2=0x7f21e014b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:30.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:30.614+0000 7f21dfe00640 1 --2- 172.21.15.17:0/2657819711 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f21e0074f70 0x7f21e014b460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:30.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:30.614+0000 7f21dfe00640 1 -- 172.21.15.17:0/2657819711 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f21e015bf40 msgr2=0x7f21e014b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:30.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:30.614+0000 7f21dfe00640 1 --2- 172.21.15.17:0/2657819711 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f21e015bf40 0x7f21e014b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:30.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:30.614+0000 7f21dfe00640 1 -- 172.21.15.17:0/2657819711 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21e01577f0 con 0x7f21e0070450 2024-09-10T21:25:30.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:30.614+0000 7f21e4a00640 1 --2- 172.21.15.17:0/2657819711 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f21e015bf40 0x7f21e014b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:25:30.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:30.614+0000 7f21dfe00640 1 --2- 172.21.15.17:0/2657819711 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f21e0070450 0x7f21e0156d20 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f21d000c4e0 tx=0x7f21d000c9a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:30.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:30.614+0000 7f21d7e00640 1 -- 172.21.15.17:0/2657819711 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21d00052d0 con 0x7f21e0070450 2024-09-10T21:25:30.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:30.614+0000 7f21d7e00640 1 -- 172.21.15.17:0/2657819711 <== mon.2 v2:172.21.15.31:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f21d0005470 con 0x7f21e0070450 2024-09-10T21:25:30.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:30.614+0000 7f21e6a00640 1 -- 172.21.15.17:0/2657819711 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21e0157a90 con 0x7f21e0070450 2024-09-10T21:25:30.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:30.614+0000 7f21d7e00640 1 -- 172.21.15.17:0/2657819711 <== mon.2 v2:172.21.15.31:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21d0012890 con 0x7f21e0070450 2024-09-10T21:25:30.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:30.614+0000 7f21e6a00640 1 -- 172.21.15.17:0/2657819711 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f21e01515c0 con 0x7f21e0070450 2024-09-10T21:25:30.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:30.618+0000 7f21e6a00640 1 -- 172.21.15.17:0/2657819711 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21e00c7130 con 0x7f21e0070450 2024-09-10T21:25:30.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:30.618+0000 7f21d7e00640 1 -- 172.21.15.17:0/2657819711 <== mon.2 v2:172.21.15.31:3301/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f21d0003600 con 0x7f21e0070450 2024-09-10T21:25:30.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:30.618+0000 7f21d7e00640 1 --2- 172.21.15.17:0/2657819711 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f21c0072610 0x7f21c0074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:30.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:30.618+0000 7f21d7e00640 1 -- 172.21.15.17:0/2657819711 <== mon.2 v2:172.21.15.31:3301/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f21d0093110 con 0x7f21e0070450 2024-09-10T21:25:30.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:30.618+0000 7f21df400640 1 --2- 172.21.15.17:0/2657819711 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f21c0072610 0x7f21c0074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:30.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:30.618+0000 7f21df400640 1 --2- 172.21.15.17:0/2657819711 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f21c0072610 0x7f21c0074ad0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f21cc0af000 tx=0x7f21cc01b060 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:30.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:30.622+0000 7f21d7e00640 1 -- 172.21.15.17:0/2657819711 <== mon.2 v2:172.21.15.31:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f21d005dd90 con 0x7f21e0070450 2024-09-10T21:25:30.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:30.798+0000 7f21e6a00640 1 -- 172.21.15.17:0/2657819711 --> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f21e0069e60 con 0x7f21c0072610 2024-09-10T21:25:30.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:30.802+0000 7f21d7e00640 1 -- 172.21.15.17:0/2657819711 <== mgr.4100 v2:172.21.15.31:6808/3101308586 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332620 (secure 0 0 0) 0x7f21e0069e60 con 0x7f21c0072610 2024-09-10T21:25:30.811 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-10T21:25:30.813 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2024-09-10T21:25:30.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:30.806+0000 7f21e6a00640 1 -- 172.21.15.17:0/2657819711 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f21c0072610 msgr2=0x7f21c0074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:30.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:30.806+0000 7f21e6a00640 1 --2- 172.21.15.17:0/2657819711 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f21c0072610 0x7f21c0074ad0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f21cc0af000 tx=0x7f21cc01b060 comp rx=0 tx=0).stop 2024-09-10T21:25:30.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:30.806+0000 7f21e6a00640 1 -- 172.21.15.17:0/2657819711 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f21e0070450 msgr2=0x7f21e0156d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:30.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:30.806+0000 7f21e6a00640 1 --2- 172.21.15.17:0/2657819711 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f21e0070450 0x7f21e0156d20 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f21d000c4e0 tx=0x7f21d000c9a0 comp rx=0 tx=0).stop 2024-09-10T21:25:30.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:30.806+0000 7f21e6a00640 1 -- 172.21.15.17:0/2657819711 shutdown_connections 2024-09-10T21:25:30.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:30.806+0000 7f21e6a00640 1 --2- 172.21.15.17:0/2657819711 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f21c0072610 0x7f21c0074ad0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:30.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:30.806+0000 7f21e6a00640 1 --2- 172.21.15.17:0/2657819711 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f21e015bf40 0x7f21e014b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:30.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:30.806+0000 7f21e6a00640 1 --2- 172.21.15.17:0/2657819711 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f21e0074f70 0x7f21e014b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:30.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:30.806+0000 7f21e6a00640 1 --2- 172.21.15.17:0/2657819711 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f21e0070450 0x7f21e0156d20 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:30.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:30.806+0000 7f21e6a00640 1 -- 172.21.15.17:0/2657819711 >> 172.21.15.17:0/2657819711 conn(0x7f21e006dd30 msgr2=0x7f21e006f340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:30.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:30.806+0000 7f21e6a00640 1 -- 172.21.15.17:0/2657819711 shutdown_connections 2024-09-10T21:25:30.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:30.806+0000 7f21e6a00640 1 -- 172.21.15.17:0/2657819711 wait complete. 2024-09-10T21:25:30.833 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":48,"stamp":"2024-09-10T21:25:29.348856+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":149,"num_write_kb":1646,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":114,"ondisk_log_size":114,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":222616,"kb_used_data":2220,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754752104,"statfs":{"total":773094113280,"available":772866154496,"internally_reserved":0,"allocated":2273280,"data_stored":2017130,"data_compressed":66932,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12752,"internal_metadata":225496624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002308"},"pg_stats":[{"pgid":"2.1f","version":"24'4","reported_seq":27,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174456+0000","last_change":"2024-09-10T21:24:46.864556+0000","last_active":"2024-09-10T21:24:53.174456+0000","last_peered":"2024-09-10T21:24:53.174456+0000","last_clean":"2024-09-10T21:24:53.174456+0000","last_became_active":"2024-09-10T21:24:46.864288+0000","last_became_peered":"2024-09-10T21:24:46.864288+0000","last_unstale":"2024-09-10T21:24:53.174456+0000","last_undegraded":"2024-09-10T21:24:53.174456+0000","last_fullsized":"2024-09-10T21:24:53.174456+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:00:38.734807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.885137+0000","last_change":"2024-09-10T21:24:47.864452+0000","last_active":"2024-09-10T21:24:52.885137+0000","last_peered":"2024-09-10T21:24:52.885137+0000","last_clean":"2024-09-10T21:24:52.885137+0000","last_became_active":"2024-09-10T21:24:47.864168+0000","last_became_peered":"2024-09-10T21:24:47.864168+0000","last_unstale":"2024-09-10T21:24:52.885137+0000","last_undegraded":"2024-09-10T21:24:52.885137+0000","last_fullsized":"2024-09-10T21:24:52.885137+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:51:31.417845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.884507+0000","last_change":"2024-09-10T21:24:52.070054+0000","last_active":"2024-09-10T21:24:52.884507+0000","last_peered":"2024-09-10T21:24:52.884507+0000","last_clean":"2024-09-10T21:24:52.884507+0000","last_became_active":"2024-09-10T21:24:52.067258+0000","last_became_peered":"2024-09-10T21:24:52.067258+0000","last_unstale":"2024-09-10T21:24:52.884507+0000","last_undegraded":"2024-09-10T21:24:52.884507+0000","last_fullsized":"2024-09-10T21:24:52.884507+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:21:53.080013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"24'1","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.885320+0000","last_change":"2024-09-10T21:24:46.860648+0000","last_active":"2024-09-10T21:24:52.885320+0000","last_peered":"2024-09-10T21:24:52.885320+0000","last_clean":"2024-09-10T21:24:52.885320+0000","last_became_active":"2024-09-10T21:24:46.860341+0000","last_became_peered":"2024-09-10T21:24:46.860341+0000","last_unstale":"2024-09-10T21:24:52.885320+0000","last_undegraded":"2024-09-10T21:24:52.885320+0000","last_fullsized":"2024-09-10T21:24:52.885320+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:39:24.819979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174137+0000","last_change":"2024-09-10T21:24:47.865617+0000","last_active":"2024-09-10T21:24:53.174137+0000","last_peered":"2024-09-10T21:24:53.174137+0000","last_clean":"2024-09-10T21:24:53.174137+0000","last_became_active":"2024-09-10T21:24:47.865365+0000","last_became_peered":"2024-09-10T21:24:47.865365+0000","last_unstale":"2024-09-10T21:24:53.174137+0000","last_undegraded":"2024-09-10T21:24:53.174137+0000","last_fullsized":"2024-09-10T21:24:53.174137+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:10:05.971996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.883672+0000","last_change":"2024-09-10T21:24:51.900077+0000","last_active":"2024-09-10T21:24:52.883672+0000","last_peered":"2024-09-10T21:24:52.883672+0000","last_clean":"2024-09-10T21:24:52.883672+0000","last_became_active":"2024-09-10T21:24:51.899652+0000","last_became_peered":"2024-09-10T21:24:51.899652+0000","last_unstale":"2024-09-10T21:24:52.883672+0000","last_undegraded":"2024-09-10T21:24:52.883672+0000","last_fullsized":"2024-09-10T21:24:52.883672+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:40:49.585857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"24'2","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.175383+0000","last_change":"2024-09-10T21:24:46.861040+0000","last_active":"2024-09-10T21:24:53.175383+0000","last_peered":"2024-09-10T21:24:53.175383+0000","last_clean":"2024-09-10T21:24:53.175383+0000","last_became_active":"2024-09-10T21:24:46.860780+0000","last_became_peered":"2024-09-10T21:24:46.860780+0000","last_unstale":"2024-09-10T21:24:53.175383+0000","last_undegraded":"2024-09-10T21:24:53.175383+0000","last_fullsized":"2024-09-10T21:24:53.175383+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:53:31.454472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:25:28.798529+0000","last_change":"2024-09-10T21:24:47.877997+0000","last_active":"2024-09-10T21:25:28.798529+0000","last_peered":"2024-09-10T21:25:28.798529+0000","last_clean":"2024-09-10T21:25:28.798529+0000","last_became_active":"2024-09-10T21:24:47.877765+0000","last_became_peered":"2024-09-10T21:24:47.877765+0000","last_unstale":"2024-09-10T21:25:28.798529+0000","last_undegraded":"2024-09-10T21:25:28.798529+0000","last_fullsized":"2024-09-10T21:25:28.798529+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:33:20.590209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.886528+0000","last_change":"2024-09-10T21:24:51.908333+0000","last_active":"2024-09-10T21:24:52.886528+0000","last_peered":"2024-09-10T21:24:52.886528+0000","last_clean":"2024-09-10T21:24:52.886528+0000","last_became_active":"2024-09-10T21:24:51.907933+0000","last_became_peered":"2024-09-10T21:24:51.907933+0000","last_unstale":"2024-09-10T21:24:52.886528+0000","last_undegraded":"2024-09-10T21:24:52.886528+0000","last_fullsized":"2024-09-10T21:24:52.886528+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:07:33.193555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"24'2","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174867+0000","last_change":"2024-09-10T21:24:46.873044+0000","last_active":"2024-09-10T21:24:53.174867+0000","last_peered":"2024-09-10T21:24:53.174867+0000","last_clean":"2024-09-10T21:24:53.174867+0000","last_became_active":"2024-09-10T21:24:46.872075+0000","last_became_peered":"2024-09-10T21:24:46.872075+0000","last_unstale":"2024-09-10T21:24:53.174867+0000","last_undegraded":"2024-09-10T21:24:53.174867+0000","last_fullsized":"2024-09-10T21:24:53.174867+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:40:33.632846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:25:28.798704+0000","last_change":"2024-09-10T21:24:47.878094+0000","last_active":"2024-09-10T21:25:28.798704+0000","last_peered":"2024-09-10T21:25:28.798704+0000","last_clean":"2024-09-10T21:25:28.798704+0000","last_became_active":"2024-09-10T21:24:47.876688+0000","last_became_peered":"2024-09-10T21:24:47.876688+0000","last_unstale":"2024-09-10T21:25:28.798704+0000","last_undegraded":"2024-09-10T21:25:28.798704+0000","last_fullsized":"2024-09-10T21:25:28.798704+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:03:37.076080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.884328+0000","last_change":"2024-09-10T21:24:51.900208+0000","last_active":"2024-09-10T21:24:52.884328+0000","last_peered":"2024-09-10T21:24:52.884328+0000","last_clean":"2024-09-10T21:24:52.884328+0000","last_became_active":"2024-09-10T21:24:51.899576+0000","last_became_peered":"2024-09-10T21:24:51.899576+0000","last_unstale":"2024-09-10T21:24:52.884328+0000","last_undegraded":"2024-09-10T21:24:52.884328+0000","last_fullsized":"2024-09-10T21:24:52.884328+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:52:49.479932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.883746+0000","last_change":"2024-09-10T21:24:47.869830+0000","last_active":"2024-09-10T21:24:52.883746+0000","last_peered":"2024-09-10T21:24:52.883746+0000","last_clean":"2024-09-10T21:24:52.883746+0000","last_became_active":"2024-09-10T21:24:47.869569+0000","last_became_peered":"2024-09-10T21:24:47.869569+0000","last_unstale":"2024-09-10T21:24:52.883746+0000","last_undegraded":"2024-09-10T21:24:52.883746+0000","last_fullsized":"2024-09-10T21:24:52.883746+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:42:15.320518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"24'2","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174248+0000","last_change":"2024-09-10T21:24:46.867297+0000","last_active":"2024-09-10T21:24:53.174248+0000","last_peered":"2024-09-10T21:24:53.174248+0000","last_clean":"2024-09-10T21:24:53.174248+0000","last_became_active":"2024-09-10T21:24:46.866893+0000","last_became_peered":"2024-09-10T21:24:46.866893+0000","last_unstale":"2024-09-10T21:24:53.174248+0000","last_undegraded":"2024-09-10T21:24:53.174248+0000","last_fullsized":"2024-09-10T21:24:53.174248+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:53.197666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.885044+0000","last_change":"2024-09-10T21:24:51.908412+0000","last_active":"2024-09-10T21:24:52.885044+0000","last_peered":"2024-09-10T21:24:52.885044+0000","last_clean":"2024-09-10T21:24:52.885044+0000","last_became_active":"2024-09-10T21:24:51.907923+0000","last_became_peered":"2024-09-10T21:24:51.907923+0000","last_unstale":"2024-09-10T21:24:52.885044+0000","last_undegraded":"2024-09-10T21:24:52.885044+0000","last_fullsized":"2024-09-10T21:24:52.885044+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:26:46.479501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.884755+0000","last_change":"2024-09-10T21:24:47.862479+0000","last_active":"2024-09-10T21:24:52.884755+0000","last_peered":"2024-09-10T21:24:52.884755+0000","last_clean":"2024-09-10T21:24:52.884755+0000","last_became_active":"2024-09-10T21:24:47.861842+0000","last_became_peered":"2024-09-10T21:24:47.861842+0000","last_unstale":"2024-09-10T21:24:52.884755+0000","last_undegraded":"2024-09-10T21:24:52.884755+0000","last_fullsized":"2024-09-10T21:24:52.884755+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:53:09.082512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.878911+0000","last_change":"2024-09-10T21:24:46.852467+0000","last_active":"2024-09-10T21:24:52.878911+0000","last_peered":"2024-09-10T21:24:52.878911+0000","last_clean":"2024-09-10T21:24:52.878911+0000","last_became_active":"2024-09-10T21:24:46.851027+0000","last_became_peered":"2024-09-10T21:24:46.851027+0000","last_unstale":"2024-09-10T21:24:52.878911+0000","last_undegraded":"2024-09-10T21:24:52.878911+0000","last_fullsized":"2024-09-10T21:24:52.878911+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:39:27.441714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.886564+0000","last_change":"2024-09-10T21:24:51.903577+0000","last_active":"2024-09-10T21:24:52.886564+0000","last_peered":"2024-09-10T21:24:52.886564+0000","last_clean":"2024-09-10T21:24:52.886564+0000","last_became_active":"2024-09-10T21:24:51.903031+0000","last_became_peered":"2024-09-10T21:24:51.903031+0000","last_unstale":"2024-09-10T21:24:52.886564+0000","last_undegraded":"2024-09-10T21:24:52.886564+0000","last_fullsized":"2024-09-10T21:24:52.886564+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:08:52.787720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.884612+0000","last_change":"2024-09-10T21:24:47.856334+0000","last_active":"2024-09-10T21:24:52.884612+0000","last_peered":"2024-09-10T21:24:52.884612+0000","last_clean":"2024-09-10T21:24:52.884612+0000","last_became_active":"2024-09-10T21:24:47.856003+0000","last_became_peered":"2024-09-10T21:24:47.856003+0000","last_unstale":"2024-09-10T21:24:52.884612+0000","last_undegraded":"2024-09-10T21:24:52.884612+0000","last_fullsized":"2024-09-10T21:24:52.884612+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:58:53.698584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.878371+0000","last_change":"2024-09-10T21:24:46.853373+0000","last_active":"2024-09-10T21:24:52.878371+0000","last_peered":"2024-09-10T21:24:52.878371+0000","last_clean":"2024-09-10T21:24:52.878371+0000","last_became_active":"2024-09-10T21:24:46.852993+0000","last_became_peered":"2024-09-10T21:24:46.852993+0000","last_unstale":"2024-09-10T21:24:52.878371+0000","last_undegraded":"2024-09-10T21:24:52.878371+0000","last_fullsized":"2024-09-10T21:24:52.878371+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:53:31.454472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.879271+0000","last_change":"2024-09-10T21:24:52.065246+0000","last_active":"2024-09-10T21:24:52.879271+0000","last_peered":"2024-09-10T21:24:52.879271+0000","last_clean":"2024-09-10T21:24:52.879271+0000","last_became_active":"2024-09-10T21:24:52.064884+0000","last_became_peered":"2024-09-10T21:24:52.064884+0000","last_unstale":"2024-09-10T21:24:52.879271+0000","last_undegraded":"2024-09-10T21:24:52.879271+0000","last_fullsized":"2024-09-10T21:24:52.879271+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:54:54.393305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.884318+0000","last_change":"2024-09-10T21:24:47.856517+0000","last_active":"2024-09-10T21:24:52.884318+0000","last_peered":"2024-09-10T21:24:52.884318+0000","last_clean":"2024-09-10T21:24:52.884318+0000","last_became_active":"2024-09-10T21:24:47.856185+0000","last_became_peered":"2024-09-10T21:24:47.856185+0000","last_unstale":"2024-09-10T21:24:52.884318+0000","last_undegraded":"2024-09-10T21:24:52.884318+0000","last_fullsized":"2024-09-10T21:24:52.884318+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:26:00.852818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.173635+0000","last_change":"2024-09-10T21:24:46.867888+0000","last_active":"2024-09-10T21:24:53.173635+0000","last_peered":"2024-09-10T21:24:53.173635+0000","last_clean":"2024-09-10T21:24:53.173635+0000","last_became_active":"2024-09-10T21:24:46.866477+0000","last_became_peered":"2024-09-10T21:24:46.866477+0000","last_unstale":"2024-09-10T21:24:53.173635+0000","last_undegraded":"2024-09-10T21:24:53.173635+0000","last_fullsized":"2024-09-10T21:24:53.173635+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:39:27.441714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.885185+0000","last_change":"2024-09-10T21:24:51.908376+0000","last_active":"2024-09-10T21:24:52.885185+0000","last_peered":"2024-09-10T21:24:52.885185+0000","last_clean":"2024-09-10T21:24:52.885185+0000","last_became_active":"2024-09-10T21:24:51.907878+0000","last_became_peered":"2024-09-10T21:24:51.907878+0000","last_unstale":"2024-09-10T21:24:52.885185+0000","last_undegraded":"2024-09-10T21:24:52.885185+0000","last_fullsized":"2024-09-10T21:24:52.885185+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:39:02.492952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174342+0000","last_change":"2024-09-10T21:24:47.844823+0000","last_active":"2024-09-10T21:24:53.174342+0000","last_peered":"2024-09-10T21:24:53.174342+0000","last_clean":"2024-09-10T21:24:53.174342+0000","last_became_active":"2024-09-10T21:24:47.844505+0000","last_became_peered":"2024-09-10T21:24:47.844505+0000","last_unstale":"2024-09-10T21:24:53.174342+0000","last_undegraded":"2024-09-10T21:24:53.174342+0000","last_fullsized":"2024-09-10T21:24:53.174342+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:37:51.784258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"24'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.885358+0000","last_change":"2024-09-10T21:24:46.862172+0000","last_active":"2024-09-10T21:24:52.885358+0000","last_peered":"2024-09-10T21:24:52.885358+0000","last_clean":"2024-09-10T21:24:52.885358+0000","last_became_active":"2024-09-10T21:24:46.861473+0000","last_became_peered":"2024-09-10T21:24:46.861473+0000","last_unstale":"2024-09-10T21:24:52.885358+0000","last_undegraded":"2024-09-10T21:24:52.885358+0000","last_fullsized":"2024-09-10T21:24:52.885358+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:53:31.454472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.883945+0000","last_change":"2024-09-10T21:24:52.062841+0000","last_active":"2024-09-10T21:24:52.883945+0000","last_peered":"2024-09-10T21:24:52.883945+0000","last_clean":"2024-09-10T21:24:52.883945+0000","last_became_active":"2024-09-10T21:24:52.061581+0000","last_became_peered":"2024-09-10T21:24:52.061581+0000","last_unstale":"2024-09-10T21:24:52.883945+0000","last_undegraded":"2024-09-10T21:24:52.883945+0000","last_fullsized":"2024-09-10T21:24:52.883945+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:34:03.325781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.885543+0000","last_change":"2024-09-10T21:24:47.859801+0000","last_active":"2024-09-10T21:24:52.885543+0000","last_peered":"2024-09-10T21:24:52.885543+0000","last_clean":"2024-09-10T21:24:52.885543+0000","last_became_active":"2024-09-10T21:24:47.859483+0000","last_became_peered":"2024-09-10T21:24:47.859483+0000","last_unstale":"2024-09-10T21:24:52.885543+0000","last_undegraded":"2024-09-10T21:24:52.885543+0000","last_fullsized":"2024-09-10T21:24:52.885543+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:38:21.368499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"25'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.879175+0000","last_change":"2024-09-10T21:24:46.860624+0000","last_active":"2024-09-10T21:24:52.879175+0000","last_peered":"2024-09-10T21:24:52.879175+0000","last_clean":"2024-09-10T21:24:52.879175+0000","last_became_active":"2024-09-10T21:24:46.860309+0000","last_became_peered":"2024-09-10T21:24:46.860309+0000","last_unstale":"2024-09-10T21:24:52.879175+0000","last_undegraded":"2024-09-10T21:24:52.879175+0000","last_fullsized":"2024-09-10T21:24:52.879175+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:37:12.332013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.884414+0000","last_change":"2024-09-10T21:24:52.068581+0000","last_active":"2024-09-10T21:24:52.884414+0000","last_peered":"2024-09-10T21:24:52.884414+0000","last_clean":"2024-09-10T21:24:52.884414+0000","last_became_active":"2024-09-10T21:24:52.068250+0000","last_became_peered":"2024-09-10T21:24:52.068250+0000","last_unstale":"2024-09-10T21:24:52.884414+0000","last_undegraded":"2024-09-10T21:24:52.884414+0000","last_fullsized":"2024-09-10T21:24:52.884414+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:52:01.990009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.879000+0000","last_change":"2024-09-10T21:24:47.854042+0000","last_active":"2024-09-10T21:24:52.879000+0000","last_peered":"2024-09-10T21:24:52.879000+0000","last_clean":"2024-09-10T21:24:52.879000+0000","last_became_active":"2024-09-10T21:24:47.853750+0000","last_became_peered":"2024-09-10T21:24:47.853750+0000","last_unstale":"2024-09-10T21:24:52.879000+0000","last_undegraded":"2024-09-10T21:24:52.879000+0000","last_fullsized":"2024-09-10T21:24:52.879000+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:06:50.775132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.175510+0000","last_change":"2024-09-10T21:24:46.864617+0000","last_active":"2024-09-10T21:24:53.175510+0000","last_peered":"2024-09-10T21:24:53.175510+0000","last_clean":"2024-09-10T21:24:53.175510+0000","last_became_active":"2024-09-10T21:24:46.864149+0000","last_became_peered":"2024-09-10T21:24:46.864149+0000","last_unstale":"2024-09-10T21:24:53.175510+0000","last_undegraded":"2024-09-10T21:24:53.175510+0000","last_fullsized":"2024-09-10T21:24:53.175510+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:24:15.673107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174530+0000","last_change":"2024-09-10T21:24:52.061485+0000","last_active":"2024-09-10T21:24:53.174530+0000","last_peered":"2024-09-10T21:24:53.174530+0000","last_clean":"2024-09-10T21:24:53.174530+0000","last_became_active":"2024-09-10T21:24:52.060953+0000","last_became_peered":"2024-09-10T21:24:52.060953+0000","last_unstale":"2024-09-10T21:24:53.174530+0000","last_undegraded":"2024-09-10T21:24:53.174530+0000","last_fullsized":"2024-09-10T21:24:53.174530+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:49:30.147202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:25:28.798884+0000","last_change":"2024-09-10T21:24:47.867241+0000","last_active":"2024-09-10T21:25:28.798884+0000","last_peered":"2024-09-10T21:25:28.798884+0000","last_clean":"2024-09-10T21:25:28.798884+0000","last_became_active":"2024-09-10T21:24:47.866907+0000","last_became_peered":"2024-09-10T21:24:47.866907+0000","last_unstale":"2024-09-10T21:25:28.798884+0000","last_undegraded":"2024-09-10T21:25:28.798884+0000","last_fullsized":"2024-09-10T21:25:28.798884+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:07.620800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.879175+0000","last_change":"2024-09-10T21:24:46.856848+0000","last_active":"2024-09-10T21:24:52.879175+0000","last_peered":"2024-09-10T21:24:52.879175+0000","last_clean":"2024-09-10T21:24:52.879175+0000","last_became_active":"2024-09-10T21:24:46.856489+0000","last_became_peered":"2024-09-10T21:24:46.856489+0000","last_unstale":"2024-09-10T21:24:52.879175+0000","last_undegraded":"2024-09-10T21:24:52.879175+0000","last_fullsized":"2024-09-10T21:24:52.879175+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:46:55.286321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.878563+0000","last_change":"2024-09-10T21:24:51.901016+0000","last_active":"2024-09-10T21:24:52.878563+0000","last_peered":"2024-09-10T21:24:52.878563+0000","last_clean":"2024-09-10T21:24:52.878563+0000","last_became_active":"2024-09-10T21:24:51.900479+0000","last_became_peered":"2024-09-10T21:24:51.900479+0000","last_unstale":"2024-09-10T21:24:52.878563+0000","last_undegraded":"2024-09-10T21:24:52.878563+0000","last_fullsized":"2024-09-10T21:24:52.878563+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:36.741810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.886797+0000","last_change":"2024-09-10T21:24:47.858753+0000","last_active":"2024-09-10T21:24:52.886797+0000","last_peered":"2024-09-10T21:24:52.886797+0000","last_clean":"2024-09-10T21:24:52.886797+0000","last_became_active":"2024-09-10T21:24:47.858438+0000","last_became_peered":"2024-09-10T21:24:47.858438+0000","last_unstale":"2024-09-10T21:24:52.886797+0000","last_undegraded":"2024-09-10T21:24:52.886797+0000","last_fullsized":"2024-09-10T21:24:52.886797+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:12:09.198013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:25:28.798402+0000","last_change":"2024-09-10T21:24:46.863610+0000","last_active":"2024-09-10T21:25:28.798402+0000","last_peered":"2024-09-10T21:25:28.798402+0000","last_clean":"2024-09-10T21:25:28.798402+0000","last_became_active":"2024-09-10T21:24:46.863150+0000","last_became_peered":"2024-09-10T21:24:46.863150+0000","last_unstale":"2024-09-10T21:25:28.798402+0000","last_undegraded":"2024-09-10T21:25:28.798402+0000","last_fullsized":"2024-09-10T21:25:28.798402+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:27:35.473995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.878912+0000","last_change":"2024-09-10T21:24:51.907366+0000","last_active":"2024-09-10T21:24:52.878912+0000","last_peered":"2024-09-10T21:24:52.878912+0000","last_clean":"2024-09-10T21:24:52.878912+0000","last_became_active":"2024-09-10T21:24:51.906888+0000","last_became_peered":"2024-09-10T21:24:51.906888+0000","last_unstale":"2024-09-10T21:24:52.878912+0000","last_undegraded":"2024-09-10T21:24:52.878912+0000","last_fullsized":"2024-09-10T21:24:52.878912+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:24:54.278514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.878445+0000","last_change":"2024-09-10T21:24:47.850155+0000","last_active":"2024-09-10T21:24:52.878445+0000","last_peered":"2024-09-10T21:24:52.878445+0000","last_clean":"2024-09-10T21:24:52.878445+0000","last_became_active":"2024-09-10T21:24:47.849846+0000","last_became_peered":"2024-09-10T21:24:47.849846+0000","last_unstale":"2024-09-10T21:24:52.878445+0000","last_undegraded":"2024-09-10T21:24:52.878445+0000","last_fullsized":"2024-09-10T21:24:52.878445+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:13:40.032808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.173931+0000","last_change":"2024-09-10T21:24:46.851800+0000","last_active":"2024-09-10T21:24:53.173931+0000","last_peered":"2024-09-10T21:24:53.173931+0000","last_clean":"2024-09-10T21:24:53.173931+0000","last_became_active":"2024-09-10T21:24:46.851200+0000","last_became_peered":"2024-09-10T21:24:46.851200+0000","last_unstale":"2024-09-10T21:24:53.173931+0000","last_undegraded":"2024-09-10T21:24:53.173931+0000","last_fullsized":"2024-09-10T21:24:53.173931+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:00:22.839361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.879296+0000","last_change":"2024-09-10T21:24:51.902529+0000","last_active":"2024-09-10T21:24:52.879296+0000","last_peered":"2024-09-10T21:24:52.879296+0000","last_clean":"2024-09-10T21:24:52.879296+0000","last_became_active":"2024-09-10T21:24:51.901141+0000","last_became_peered":"2024-09-10T21:24:51.901141+0000","last_unstale":"2024-09-10T21:24:52.879296+0000","last_undegraded":"2024-09-10T21:24:52.879296+0000","last_fullsized":"2024-09-10T21:24:52.879296+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:08:40.203348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.173878+0000","last_change":"2024-09-10T21:24:47.846582+0000","last_active":"2024-09-10T21:24:53.173878+0000","last_peered":"2024-09-10T21:24:53.173878+0000","last_clean":"2024-09-10T21:24:53.173878+0000","last_became_active":"2024-09-10T21:24:47.846358+0000","last_became_peered":"2024-09-10T21:24:47.846358+0000","last_unstale":"2024-09-10T21:24:53.173878+0000","last_undegraded":"2024-09-10T21:24:53.173878+0000","last_fullsized":"2024-09-10T21:24:53.173878+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:58:53.698584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.173773+0000","last_change":"2024-09-10T21:24:46.870908+0000","last_active":"2024-09-10T21:24:53.173773+0000","last_peered":"2024-09-10T21:24:53.173773+0000","last_clean":"2024-09-10T21:24:53.173773+0000","last_became_active":"2024-09-10T21:24:46.868375+0000","last_became_peered":"2024-09-10T21:24:46.868375+0000","last_unstale":"2024-09-10T21:24:53.173773+0000","last_undegraded":"2024-09-10T21:24:53.173773+0000","last_fullsized":"2024-09-10T21:24:53.173773+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:37:12.332013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174362+0000","last_change":"2024-09-10T21:24:52.069018+0000","last_active":"2024-09-10T21:24:53.174362+0000","last_peered":"2024-09-10T21:24:53.174362+0000","last_clean":"2024-09-10T21:24:53.174362+0000","last_became_active":"2024-09-10T21:24:52.067308+0000","last_became_peered":"2024-09-10T21:24:52.067308+0000","last_unstale":"2024-09-10T21:24:53.174362+0000","last_undegraded":"2024-09-10T21:24:53.174362+0000","last_fullsized":"2024-09-10T21:24:53.174362+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:07:25.162168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.884767+0000","last_change":"2024-09-10T21:24:47.871542+0000","last_active":"2024-09-10T21:24:52.884767+0000","last_peered":"2024-09-10T21:24:52.884767+0000","last_clean":"2024-09-10T21:24:52.884767+0000","last_became_active":"2024-09-10T21:24:47.871128+0000","last_became_peered":"2024-09-10T21:24:47.871128+0000","last_unstale":"2024-09-10T21:24:52.884767+0000","last_undegraded":"2024-09-10T21:24:52.884767+0000","last_fullsized":"2024-09-10T21:24:52.884767+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:51:22.876126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:25:28.798353+0000","last_change":"2024-09-10T21:24:46.859218+0000","last_active":"2024-09-10T21:25:28.798353+0000","last_peered":"2024-09-10T21:25:28.798353+0000","last_clean":"2024-09-10T21:25:28.798353+0000","last_became_active":"2024-09-10T21:24:46.853190+0000","last_became_peered":"2024-09-10T21:24:46.853190+0000","last_unstale":"2024-09-10T21:25:28.798353+0000","last_undegraded":"2024-09-10T21:25:28.798353+0000","last_fullsized":"2024-09-10T21:25:28.798353+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:26:34.848759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":119,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.173768+0000","last_change":"2024-09-10T21:24:37.216542+0000","last_active":"2024-09-10T21:24:53.173768+0000","last_peered":"2024-09-10T21:24:53.173768+0000","last_clean":"2024-09-10T21:24:53.173768+0000","last_became_active":"2024-09-10T21:24:37.215443+0000","last_became_peered":"2024-09-10T21:24:37.215443+0000","last_unstale":"2024-09-10T21:24:53.173768+0000","last_undegraded":"2024-09-10T21:24:53.173768+0000","last_fullsized":"2024-09-10T21:24:53.173768+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:35.877801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:35.877801+0000","last_clean_scrub_stamp":"2024-09-10T21:24:35.877801+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:14:20.260329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.884231+0000","last_change":"2024-09-10T21:24:52.067645+0000","last_active":"2024-09-10T21:24:52.884231+0000","last_peered":"2024-09-10T21:24:52.884231+0000","last_clean":"2024-09-10T21:24:52.884231+0000","last_became_active":"2024-09-10T21:24:52.067148+0000","last_became_peered":"2024-09-10T21:24:52.067148+0000","last_unstale":"2024-09-10T21:24:52.884231+0000","last_undegraded":"2024-09-10T21:24:52.884231+0000","last_fullsized":"2024-09-10T21:24:52.884231+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:54:42.435011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.175128+0000","last_change":"2024-09-10T21:24:47.867282+0000","last_active":"2024-09-10T21:24:53.175128+0000","last_peered":"2024-09-10T21:24:53.175128+0000","last_clean":"2024-09-10T21:24:53.175128+0000","last_became_active":"2024-09-10T21:24:47.865582+0000","last_became_peered":"2024-09-10T21:24:47.865582+0000","last_unstale":"2024-09-10T21:24:53.175128+0000","last_undegraded":"2024-09-10T21:24:53.175128+0000","last_fullsized":"2024-09-10T21:24:53.175128+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:38:55.009670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174808+0000","last_change":"2024-09-10T21:24:46.859532+0000","last_active":"2024-09-10T21:24:53.174808+0000","last_peered":"2024-09-10T21:24:53.174808+0000","last_clean":"2024-09-10T21:24:53.174808+0000","last_became_active":"2024-09-10T21:24:46.859144+0000","last_became_peered":"2024-09-10T21:24:46.859144+0000","last_unstale":"2024-09-10T21:24:53.174808+0000","last_undegraded":"2024-09-10T21:24:53.174808+0000","last_fullsized":"2024-09-10T21:24:53.174808+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:48:41.992441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.883627+0000","last_change":"2024-09-10T21:24:52.061029+0000","last_active":"2024-09-10T21:24:52.883627+0000","last_peered":"2024-09-10T21:24:52.883627+0000","last_clean":"2024-09-10T21:24:52.883627+0000","last_became_active":"2024-09-10T21:24:52.060605+0000","last_became_peered":"2024-09-10T21:24:52.060605+0000","last_unstale":"2024-09-10T21:24:52.883627+0000","last_undegraded":"2024-09-10T21:24:52.883627+0000","last_fullsized":"2024-09-10T21:24:52.883627+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:09:48.077266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:25:28.798667+0000","last_change":"2024-09-10T21:24:47.866927+0000","last_active":"2024-09-10T21:25:28.798667+0000","last_peered":"2024-09-10T21:25:28.798667+0000","last_clean":"2024-09-10T21:25:28.798667+0000","last_became_active":"2024-09-10T21:24:47.866650+0000","last_became_peered":"2024-09-10T21:24:47.866650+0000","last_unstale":"2024-09-10T21:25:28.798667+0000","last_undegraded":"2024-09-10T21:25:28.798667+0000","last_fullsized":"2024-09-10T21:25:28.798667+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:47:37.304624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"24'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.879000+0000","last_change":"2024-09-10T21:24:46.857082+0000","last_active":"2024-09-10T21:24:52.879000+0000","last_peered":"2024-09-10T21:24:52.879000+0000","last_clean":"2024-09-10T21:24:52.879000+0000","last_became_active":"2024-09-10T21:24:46.856799+0000","last_became_peered":"2024-09-10T21:24:46.856799+0000","last_unstale":"2024-09-10T21:24:52.879000+0000","last_undegraded":"2024-09-10T21:24:52.879000+0000","last_fullsized":"2024-09-10T21:24:52.879000+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:40:33.632846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174743+0000","last_change":"2024-09-10T21:24:51.911880+0000","last_active":"2024-09-10T21:24:53.174743+0000","last_peered":"2024-09-10T21:24:53.174743+0000","last_clean":"2024-09-10T21:24:53.174743+0000","last_became_active":"2024-09-10T21:24:51.911088+0000","last_became_peered":"2024-09-10T21:24:51.911088+0000","last_unstale":"2024-09-10T21:24:53.174743+0000","last_undegraded":"2024-09-10T21:24:53.174743+0000","last_fullsized":"2024-09-10T21:24:53.174743+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:38:57.030655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.175370+0000","last_change":"2024-09-10T21:24:47.876946+0000","last_active":"2024-09-10T21:24:53.175370+0000","last_peered":"2024-09-10T21:24:53.175370+0000","last_clean":"2024-09-10T21:24:53.175370+0000","last_became_active":"2024-09-10T21:24:47.876690+0000","last_became_peered":"2024-09-10T21:24:47.876690+0000","last_unstale":"2024-09-10T21:24:53.175370+0000","last_undegraded":"2024-09-10T21:24:53.175370+0000","last_fullsized":"2024-09-10T21:24:53.175370+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:14:06.368275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"24'2","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.175105+0000","last_change":"2024-09-10T21:24:46.855248+0000","last_active":"2024-09-10T21:24:53.175105+0000","last_peered":"2024-09-10T21:24:53.175105+0000","last_clean":"2024-09-10T21:24:53.175105+0000","last_became_active":"2024-09-10T21:24:46.854930+0000","last_became_peered":"2024-09-10T21:24:46.854930+0000","last_unstale":"2024-09-10T21:24:53.175105+0000","last_undegraded":"2024-09-10T21:24:53.175105+0000","last_fullsized":"2024-09-10T21:24:53.175105+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:46:55.286321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:25:28.797831+0000","last_change":"2024-09-10T21:24:52.063112+0000","last_active":"2024-09-10T21:25:28.797831+0000","last_peered":"2024-09-10T21:25:28.797831+0000","last_clean":"2024-09-10T21:25:28.797831+0000","last_became_active":"2024-09-10T21:24:52.062395+0000","last_became_peered":"2024-09-10T21:24:52.062395+0000","last_unstale":"2024-09-10T21:25:28.797831+0000","last_undegraded":"2024-09-10T21:25:28.797831+0000","last_fullsized":"2024-09-10T21:25:28.797831+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:21:53.080013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.175254+0000","last_change":"2024-09-10T21:24:47.858593+0000","last_active":"2024-09-10T21:24:53.175254+0000","last_peered":"2024-09-10T21:24:53.175254+0000","last_clean":"2024-09-10T21:24:53.175254+0000","last_became_active":"2024-09-10T21:24:47.858216+0000","last_became_peered":"2024-09-10T21:24:47.858216+0000","last_unstale":"2024-09-10T21:24:53.175254+0000","last_undegraded":"2024-09-10T21:24:53.175254+0000","last_fullsized":"2024-09-10T21:24:53.175254+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:42.961291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"24'1","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.886950+0000","last_change":"2024-09-10T21:24:46.860330+0000","last_active":"2024-09-10T21:24:52.886950+0000","last_peered":"2024-09-10T21:24:52.886950+0000","last_clean":"2024-09-10T21:24:52.886950+0000","last_became_active":"2024-09-10T21:24:46.860053+0000","last_became_peered":"2024-09-10T21:24:46.860053+0000","last_unstale":"2024-09-10T21:24:52.886950+0000","last_undegraded":"2024-09-10T21:24:52.886950+0000","last_fullsized":"2024-09-10T21:24:52.886950+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:58:53.177740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.883701+0000","last_change":"2024-09-10T21:24:51.901673+0000","last_active":"2024-09-10T21:24:52.883701+0000","last_peered":"2024-09-10T21:24:52.883701+0000","last_clean":"2024-09-10T21:24:52.883701+0000","last_became_active":"2024-09-10T21:24:51.901222+0000","last_became_peered":"2024-09-10T21:24:51.901222+0000","last_unstale":"2024-09-10T21:24:52.883701+0000","last_undegraded":"2024-09-10T21:24:52.883701+0000","last_fullsized":"2024-09-10T21:24:52.883701+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:07:52.120299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.173790+0000","last_change":"2024-09-10T21:24:47.870589+0000","last_active":"2024-09-10T21:24:53.173790+0000","last_peered":"2024-09-10T21:24:53.173790+0000","last_clean":"2024-09-10T21:24:53.173790+0000","last_became_active":"2024-09-10T21:24:47.861834+0000","last_became_peered":"2024-09-10T21:24:47.861834+0000","last_unstale":"2024-09-10T21:24:53.173790+0000","last_undegraded":"2024-09-10T21:24:53.173790+0000","last_fullsized":"2024-09-10T21:24:53.173790+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:43.363013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"24'8","reported_seq":31,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.173490+0000","last_change":"2024-09-10T21:24:46.852064+0000","last_active":"2024-09-10T21:24:53.173490+0000","last_peered":"2024-09-10T21:24:53.173490+0000","last_clean":"2024-09-10T21:24:53.173490+0000","last_became_active":"2024-09-10T21:24:46.851776+0000","last_became_peered":"2024-09-10T21:24:46.851776+0000","last_unstale":"2024-09-10T21:24:53.173490+0000","last_undegraded":"2024-09-10T21:24:53.173490+0000","last_fullsized":"2024-09-10T21:24:53.173490+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:26:24.371844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174153+0000","last_change":"2024-09-10T21:24:51.904449+0000","last_active":"2024-09-10T21:24:53.174153+0000","last_peered":"2024-09-10T21:24:53.174153+0000","last_clean":"2024-09-10T21:24:53.174153+0000","last_became_active":"2024-09-10T21:24:51.903533+0000","last_became_peered":"2024-09-10T21:24:51.903533+0000","last_unstale":"2024-09-10T21:24:53.174153+0000","last_undegraded":"2024-09-10T21:24:53.174153+0000","last_fullsized":"2024-09-10T21:24:53.174153+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:11.654935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.883479+0000","last_change":"2024-09-10T21:24:47.857736+0000","last_active":"2024-09-10T21:24:52.883479+0000","last_peered":"2024-09-10T21:24:52.883479+0000","last_clean":"2024-09-10T21:24:52.883479+0000","last_became_active":"2024-09-10T21:24:47.857437+0000","last_became_peered":"2024-09-10T21:24:47.857437+0000","last_unstale":"2024-09-10T21:24:52.883479+0000","last_undegraded":"2024-09-10T21:24:52.883479+0000","last_fullsized":"2024-09-10T21:24:52.883479+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:13:40.032808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.883979+0000","last_change":"2024-09-10T21:24:46.860662+0000","last_active":"2024-09-10T21:24:52.883979+0000","last_peered":"2024-09-10T21:24:52.883979+0000","last_clean":"2024-09-10T21:24:52.883979+0000","last_became_active":"2024-09-10T21:24:46.859796+0000","last_became_peered":"2024-09-10T21:24:46.859796+0000","last_unstale":"2024-09-10T21:24:52.883979+0000","last_undegraded":"2024-09-10T21:24:52.883979+0000","last_fullsized":"2024-09-10T21:24:52.883979+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:48:27.758481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.885543+0000","last_change":"2024-09-10T21:24:52.069770+0000","last_active":"2024-09-10T21:24:52.885543+0000","last_peered":"2024-09-10T21:24:52.885543+0000","last_clean":"2024-09-10T21:24:52.885543+0000","last_became_active":"2024-09-10T21:24:52.067409+0000","last_became_peered":"2024-09-10T21:24:52.067409+0000","last_unstale":"2024-09-10T21:24:52.885543+0000","last_undegraded":"2024-09-10T21:24:52.885543+0000","last_fullsized":"2024-09-10T21:24:52.885543+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:23:39.622612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174136+0000","last_change":"2024-09-10T21:24:47.858692+0000","last_active":"2024-09-10T21:24:53.174136+0000","last_peered":"2024-09-10T21:24:53.174136+0000","last_clean":"2024-09-10T21:24:53.174136+0000","last_became_active":"2024-09-10T21:24:47.858343+0000","last_became_peered":"2024-09-10T21:24:47.858343+0000","last_unstale":"2024-09-10T21:24:53.174136+0000","last_undegraded":"2024-09-10T21:24:53.174136+0000","last_fullsized":"2024-09-10T21:24:53.174136+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:04:36.661791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:25:28.798320+0000","last_change":"2024-09-10T21:24:46.864592+0000","last_active":"2024-09-10T21:25:28.798320+0000","last_peered":"2024-09-10T21:25:28.798320+0000","last_clean":"2024-09-10T21:25:28.798320+0000","last_became_active":"2024-09-10T21:24:46.858696+0000","last_became_peered":"2024-09-10T21:24:46.858696+0000","last_unstale":"2024-09-10T21:25:28.798320+0000","last_undegraded":"2024-09-10T21:25:28.798320+0000","last_fullsized":"2024-09-10T21:25:28.798320+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:54:19.356867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.885074+0000","last_change":"2024-09-10T21:24:52.061572+0000","last_active":"2024-09-10T21:24:52.885074+0000","last_peered":"2024-09-10T21:24:52.885074+0000","last_clean":"2024-09-10T21:24:52.885074+0000","last_became_active":"2024-09-10T21:24:52.061061+0000","last_became_peered":"2024-09-10T21:24:52.061061+0000","last_unstale":"2024-09-10T21:24:52.885074+0000","last_undegraded":"2024-09-10T21:24:52.885074+0000","last_fullsized":"2024-09-10T21:24:52.885074+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:04:36.399811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174340+0000","last_change":"2024-09-10T21:24:47.857699+0000","last_active":"2024-09-10T21:24:53.174340+0000","last_peered":"2024-09-10T21:24:53.174340+0000","last_clean":"2024-09-10T21:24:53.174340+0000","last_became_active":"2024-09-10T21:24:47.857429+0000","last_became_peered":"2024-09-10T21:24:47.857429+0000","last_unstale":"2024-09-10T21:24:53.174340+0000","last_undegraded":"2024-09-10T21:24:53.174340+0000","last_fullsized":"2024-09-10T21:24:53.174340+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:07:11.880228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"24'2","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174839+0000","last_change":"2024-09-10T21:24:46.859753+0000","last_active":"2024-09-10T21:24:53.174839+0000","last_peered":"2024-09-10T21:24:53.174839+0000","last_clean":"2024-09-10T21:24:53.174839+0000","last_became_active":"2024-09-10T21:24:46.859299+0000","last_became_peered":"2024-09-10T21:24:46.859299+0000","last_unstale":"2024-09-10T21:24:53.174839+0000","last_undegraded":"2024-09-10T21:24:53.174839+0000","last_fullsized":"2024-09-10T21:24:53.174839+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:06:41.965052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:25:28.798655+0000","last_change":"2024-09-10T21:24:52.064718+0000","last_active":"2024-09-10T21:25:28.798655+0000","last_peered":"2024-09-10T21:25:28.798655+0000","last_clean":"2024-09-10T21:25:28.798655+0000","last_became_active":"2024-09-10T21:24:52.062327+0000","last_became_peered":"2024-09-10T21:24:52.062327+0000","last_unstale":"2024-09-10T21:25:28.798655+0000","last_undegraded":"2024-09-10T21:25:28.798655+0000","last_fullsized":"2024-09-10T21:25:28.798655+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:43:17.173095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.175009+0000","last_change":"2024-09-10T21:24:47.870766+0000","last_active":"2024-09-10T21:24:53.175009+0000","last_peered":"2024-09-10T21:24:53.175009+0000","last_clean":"2024-09-10T21:24:53.175009+0000","last_became_active":"2024-09-10T21:24:47.870276+0000","last_became_peered":"2024-09-10T21:24:47.870276+0000","last_unstale":"2024-09-10T21:24:53.175009+0000","last_undegraded":"2024-09-10T21:24:53.175009+0000","last_fullsized":"2024-09-10T21:24:53.175009+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:54:38.400877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"25'9","reported_seq":31,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:25:05.421191+0000","last_change":"2024-09-10T21:24:46.867790+0000","last_active":"2024-09-10T21:25:05.421191+0000","last_peered":"2024-09-10T21:25:05.421191+0000","last_clean":"2024-09-10T21:25:05.421191+0000","last_became_active":"2024-09-10T21:24:46.866899+0000","last_became_peered":"2024-09-10T21:24:46.866899+0000","last_unstale":"2024-09-10T21:25:05.421191+0000","last_undegraded":"2024-09-10T21:25:05.421191+0000","last_fullsized":"2024-09-10T21:25:05.421191+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:25:04.923730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.878572+0000","last_change":"2024-09-10T21:24:51.899803+0000","last_active":"2024-09-10T21:24:52.878572+0000","last_peered":"2024-09-10T21:24:52.878572+0000","last_clean":"2024-09-10T21:24:52.878572+0000","last_became_active":"2024-09-10T21:24:51.895398+0000","last_became_peered":"2024-09-10T21:24:51.895398+0000","last_unstale":"2024-09-10T21:24:52.878572+0000","last_undegraded":"2024-09-10T21:24:52.878572+0000","last_fullsized":"2024-09-10T21:24:52.878572+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:52:34.689860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.886661+0000","last_change":"2024-09-10T21:24:47.857544+0000","last_active":"2024-09-10T21:24:52.886661+0000","last_peered":"2024-09-10T21:24:52.886661+0000","last_clean":"2024-09-10T21:24:52.886661+0000","last_became_active":"2024-09-10T21:24:47.857284+0000","last_became_peered":"2024-09-10T21:24:47.857284+0000","last_unstale":"2024-09-10T21:24:52.886661+0000","last_undegraded":"2024-09-10T21:24:52.886661+0000","last_fullsized":"2024-09-10T21:24:52.886661+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:10:22.747606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"24'3","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.878942+0000","last_change":"2024-09-10T21:24:46.859633+0000","last_active":"2024-09-10T21:24:52.878942+0000","last_peered":"2024-09-10T21:24:52.878942+0000","last_clean":"2024-09-10T21:24:52.878942+0000","last_became_active":"2024-09-10T21:24:46.859332+0000","last_became_peered":"2024-09-10T21:24:46.859332+0000","last_unstale":"2024-09-10T21:24:52.878942+0000","last_undegraded":"2024-09-10T21:24:52.878942+0000","last_fullsized":"2024-09-10T21:24:52.878942+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:56:46.975225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.886734+0000","last_change":"2024-09-10T21:24:51.899100+0000","last_active":"2024-09-10T21:24:52.886734+0000","last_peered":"2024-09-10T21:24:52.886734+0000","last_clean":"2024-09-10T21:24:52.886734+0000","last_became_active":"2024-09-10T21:24:51.898642+0000","last_became_peered":"2024-09-10T21:24:51.898642+0000","last_unstale":"2024-09-10T21:24:52.886734+0000","last_undegraded":"2024-09-10T21:24:52.886734+0000","last_fullsized":"2024-09-10T21:24:52.886734+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:06.394081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174291+0000","last_change":"2024-09-10T21:24:47.867890+0000","last_active":"2024-09-10T21:24:53.174291+0000","last_peered":"2024-09-10T21:24:53.174291+0000","last_clean":"2024-09-10T21:24:53.174291+0000","last_became_active":"2024-09-10T21:24:47.865016+0000","last_became_peered":"2024-09-10T21:24:47.865016+0000","last_unstale":"2024-09-10T21:24:53.174291+0000","last_undegraded":"2024-09-10T21:24:53.174291+0000","last_fullsized":"2024-09-10T21:24:53.174291+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:53:09.082512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:25:28.798864+0000","last_change":"2024-09-10T21:24:46.861400+0000","last_active":"2024-09-10T21:25:28.798864+0000","last_peered":"2024-09-10T21:25:28.798864+0000","last_clean":"2024-09-10T21:25:28.798864+0000","last_became_active":"2024-09-10T21:24:46.861136+0000","last_became_peered":"2024-09-10T21:24:46.861136+0000","last_unstale":"2024-09-10T21:25:28.798864+0000","last_undegraded":"2024-09-10T21:25:28.798864+0000","last_fullsized":"2024-09-10T21:25:28.798864+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:35:31.682448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.884671+0000","last_change":"2024-09-10T21:24:51.907850+0000","last_active":"2024-09-10T21:24:52.884671+0000","last_peered":"2024-09-10T21:24:52.884671+0000","last_clean":"2024-09-10T21:24:52.884671+0000","last_became_active":"2024-09-10T21:24:51.907062+0000","last_became_peered":"2024-09-10T21:24:51.907062+0000","last_unstale":"2024-09-10T21:24:52.884671+0000","last_undegraded":"2024-09-10T21:24:52.884671+0000","last_fullsized":"2024-09-10T21:24:52.884671+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:32:52.519627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:25:28.798854+0000","last_change":"2024-09-10T21:24:47.861800+0000","last_active":"2024-09-10T21:25:28.798854+0000","last_peered":"2024-09-10T21:25:28.798854+0000","last_clean":"2024-09-10T21:25:28.798854+0000","last_became_active":"2024-09-10T21:24:47.861498+0000","last_became_peered":"2024-09-10T21:24:47.861498+0000","last_unstale":"2024-09-10T21:25:28.798854+0000","last_undegraded":"2024-09-10T21:25:28.798854+0000","last_fullsized":"2024-09-10T21:25:28.798854+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:07:21.128033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.885153+0000","last_change":"2024-09-10T21:24:46.871677+0000","last_active":"2024-09-10T21:24:52.885153+0000","last_peered":"2024-09-10T21:24:52.885153+0000","last_clean":"2024-09-10T21:24:52.885153+0000","last_became_active":"2024-09-10T21:24:46.871367+0000","last_became_peered":"2024-09-10T21:24:46.871367+0000","last_unstale":"2024-09-10T21:24:52.885153+0000","last_undegraded":"2024-09-10T21:24:52.885153+0000","last_fullsized":"2024-09-10T21:24:52.885153+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:40:33.632846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.884646+0000","last_change":"2024-09-10T21:24:52.068637+0000","last_active":"2024-09-10T21:24:52.884646+0000","last_peered":"2024-09-10T21:24:52.884646+0000","last_clean":"2024-09-10T21:24:52.884646+0000","last_became_active":"2024-09-10T21:24:52.068348+0000","last_became_peered":"2024-09-10T21:24:52.068348+0000","last_unstale":"2024-09-10T21:24:52.884646+0000","last_undegraded":"2024-09-10T21:24:52.884646+0000","last_fullsized":"2024-09-10T21:24:52.884646+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:43:17.173095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174107+0000","last_change":"2024-09-10T21:24:52.067732+0000","last_active":"2024-09-10T21:24:53.174107+0000","last_peered":"2024-09-10T21:24:53.174107+0000","last_clean":"2024-09-10T21:24:53.174107+0000","last_became_active":"2024-09-10T21:24:52.067330+0000","last_became_peered":"2024-09-10T21:24:52.067330+0000","last_unstale":"2024-09-10T21:24:53.174107+0000","last_undegraded":"2024-09-10T21:24:53.174107+0000","last_fullsized":"2024-09-10T21:24:53.174107+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:03:41.110214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.878246+0000","last_change":"2024-09-10T21:24:47.862774+0000","last_active":"2024-09-10T21:24:52.878246+0000","last_peered":"2024-09-10T21:24:52.878246+0000","last_clean":"2024-09-10T21:24:52.878246+0000","last_became_active":"2024-09-10T21:24:47.862556+0000","last_became_peered":"2024-09-10T21:24:47.862556+0000","last_unstale":"2024-09-10T21:24:52.878246+0000","last_undegraded":"2024-09-10T21:24:52.878246+0000","last_fullsized":"2024-09-10T21:24:52.878246+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:17:52.343553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:25:28.798072+0000","last_change":"2024-09-10T21:24:46.862847+0000","last_active":"2024-09-10T21:25:28.798072+0000","last_peered":"2024-09-10T21:25:28.798072+0000","last_clean":"2024-09-10T21:25:28.798072+0000","last_became_active":"2024-09-10T21:24:46.862540+0000","last_became_peered":"2024-09-10T21:24:46.862540+0000","last_unstale":"2024-09-10T21:25:28.798072+0000","last_undegraded":"2024-09-10T21:25:28.798072+0000","last_fullsized":"2024-09-10T21:25:28.798072+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:52:31.628568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.884678+0000","last_change":"2024-09-10T21:24:51.901224+0000","last_active":"2024-09-10T21:24:52.884678+0000","last_peered":"2024-09-10T21:24:52.884678+0000","last_clean":"2024-09-10T21:24:52.884678+0000","last_became_active":"2024-09-10T21:24:51.900610+0000","last_became_peered":"2024-09-10T21:24:51.900610+0000","last_unstale":"2024-09-10T21:24:52.884678+0000","last_undegraded":"2024-09-10T21:24:52.884678+0000","last_fullsized":"2024-09-10T21:24:52.884678+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:52:34.689860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.884489+0000","last_change":"2024-09-10T21:24:47.861255+0000","last_active":"2024-09-10T21:24:52.884489+0000","last_peered":"2024-09-10T21:24:52.884489+0000","last_clean":"2024-09-10T21:24:52.884489+0000","last_became_active":"2024-09-10T21:24:47.860930+0000","last_became_peered":"2024-09-10T21:24:47.860930+0000","last_unstale":"2024-09-10T21:24:52.884489+0000","last_undegraded":"2024-09-10T21:24:52.884489+0000","last_fullsized":"2024-09-10T21:24:52.884489+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:37:51.784258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.884796+0000","last_change":"2024-09-10T21:24:46.858561+0000","last_active":"2024-09-10T21:24:52.884796+0000","last_peered":"2024-09-10T21:24:52.884796+0000","last_clean":"2024-09-10T21:24:52.884796+0000","last_became_active":"2024-09-10T21:24:46.858265+0000","last_became_peered":"2024-09-10T21:24:46.858265+0000","last_unstale":"2024-09-10T21:24:52.884796+0000","last_undegraded":"2024-09-10T21:24:52.884796+0000","last_fullsized":"2024-09-10T21:24:52.884796+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:49:00.454609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"24'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.884271+0000","last_change":"2024-09-10T21:24:46.856886+0000","last_active":"2024-09-10T21:24:52.884271+0000","last_peered":"2024-09-10T21:24:52.884271+0000","last_clean":"2024-09-10T21:24:52.884271+0000","last_became_active":"2024-09-10T21:24:46.856506+0000","last_became_peered":"2024-09-10T21:24:46.856506+0000","last_unstale":"2024-09-10T21:24:52.884271+0000","last_undegraded":"2024-09-10T21:24:52.884271+0000","last_fullsized":"2024-09-10T21:24:52.884271+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:56:46.975225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174721+0000","last_change":"2024-09-10T21:24:47.848602+0000","last_active":"2024-09-10T21:24:53.174721+0000","last_peered":"2024-09-10T21:24:53.174721+0000","last_clean":"2024-09-10T21:24:53.174721+0000","last_became_active":"2024-09-10T21:24:47.848160+0000","last_became_peered":"2024-09-10T21:24:47.848160+0000","last_unstale":"2024-09-10T21:24:53.174721+0000","last_undegraded":"2024-09-10T21:24:53.174721+0000","last_fullsized":"2024-09-10T21:24:53.174721+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:32:50.027827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174649+0000","last_change":"2024-09-10T21:24:51.903553+0000","last_active":"2024-09-10T21:24:53.174649+0000","last_peered":"2024-09-10T21:24:53.174649+0000","last_clean":"2024-09-10T21:24:53.174649+0000","last_became_active":"2024-09-10T21:24:51.903120+0000","last_became_peered":"2024-09-10T21:24:51.903120+0000","last_unstale":"2024-09-10T21:24:53.174649+0000","last_undegraded":"2024-09-10T21:24:53.174649+0000","last_fullsized":"2024-09-10T21:24:53.174649+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:58.240380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"24'1","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.884865+0000","last_change":"2024-09-10T21:24:46.853550+0000","last_active":"2024-09-10T21:24:52.884865+0000","last_peered":"2024-09-10T21:24:52.884865+0000","last_clean":"2024-09-10T21:24:52.884865+0000","last_became_active":"2024-09-10T21:24:46.852204+0000","last_became_peered":"2024-09-10T21:24:46.852204+0000","last_unstale":"2024-09-10T21:24:52.884865+0000","last_undegraded":"2024-09-10T21:24:52.884865+0000","last_fullsized":"2024-09-10T21:24:52.884865+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:06:37.669444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.884581+0000","last_change":"2024-09-10T21:24:47.849151+0000","last_active":"2024-09-10T21:24:52.884581+0000","last_peered":"2024-09-10T21:24:52.884581+0000","last_clean":"2024-09-10T21:24:52.884581+0000","last_became_active":"2024-09-10T21:24:47.848851+0000","last_became_peered":"2024-09-10T21:24:47.848851+0000","last_unstale":"2024-09-10T21:24:52.884581+0000","last_undegraded":"2024-09-10T21:24:52.884581+0000","last_fullsized":"2024-09-10T21:24:52.884581+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:39:25.828558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.885276+0000","last_change":"2024-09-10T21:24:51.907549+0000","last_active":"2024-09-10T21:24:52.885276+0000","last_peered":"2024-09-10T21:24:52.885276+0000","last_clean":"2024-09-10T21:24:52.885276+0000","last_became_active":"2024-09-10T21:24:51.907082+0000","last_became_peered":"2024-09-10T21:24:51.907082+0000","last_unstale":"2024-09-10T21:24:52.885276+0000","last_undegraded":"2024-09-10T21:24:52.885276+0000","last_fullsized":"2024-09-10T21:24:52.885276+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:44:30.813754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.883360+0000","last_change":"2024-09-10T21:24:47.857804+0000","last_active":"2024-09-10T21:24:52.883360+0000","last_peered":"2024-09-10T21:24:52.883360+0000","last_clean":"2024-09-10T21:24:52.883360+0000","last_became_active":"2024-09-10T21:24:47.857501+0000","last_became_peered":"2024-09-10T21:24:47.857501+0000","last_unstale":"2024-09-10T21:24:52.883360+0000","last_undegraded":"2024-09-10T21:24:52.883360+0000","last_fullsized":"2024-09-10T21:24:52.883360+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:19:20.807422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.173632+0000","last_change":"2024-09-10T21:24:47.867803+0000","last_active":"2024-09-10T21:24:53.173632+0000","last_peered":"2024-09-10T21:24:53.173632+0000","last_clean":"2024-09-10T21:24:53.173632+0000","last_became_active":"2024-09-10T21:24:47.867504+0000","last_became_peered":"2024-09-10T21:24:47.867504+0000","last_unstale":"2024-09-10T21:24:53.173632+0000","last_undegraded":"2024-09-10T21:24:53.173632+0000","last_fullsized":"2024-09-10T21:24:53.173632+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:53:30.420454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:25:28.798278+0000","last_change":"2024-09-10T21:24:47.868901+0000","last_active":"2024-09-10T21:25:28.798278+0000","last_peered":"2024-09-10T21:25:28.798278+0000","last_clean":"2024-09-10T21:25:28.798278+0000","last_became_active":"2024-09-10T21:24:47.868168+0000","last_became_peered":"2024-09-10T21:24:47.868168+0000","last_unstale":"2024-09-10T21:25:28.798278+0000","last_undegraded":"2024-09-10T21:25:28.798278+0000","last_fullsized":"2024-09-10T21:25:28.798278+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:06:21.461089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174633+0000","last_change":"2024-09-10T21:24:47.864561+0000","last_active":"2024-09-10T21:24:53.174633+0000","last_peered":"2024-09-10T21:24:53.174633+0000","last_clean":"2024-09-10T21:24:53.174633+0000","last_became_active":"2024-09-10T21:24:47.864100+0000","last_became_peered":"2024-09-10T21:24:47.864100+0000","last_unstale":"2024-09-10T21:24:53.174633+0000","last_undegraded":"2024-09-10T21:24:53.174633+0000","last_fullsized":"2024-09-10T21:24:53.174633+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:34:13.053494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.885174+0000","last_change":"2024-09-10T21:24:47.864303+0000","last_active":"2024-09-10T21:24:52.885174+0000","last_peered":"2024-09-10T21:24:52.885174+0000","last_clean":"2024-09-10T21:24:52.885174+0000","last_became_active":"2024-09-10T21:24:47.863970+0000","last_became_peered":"2024-09-10T21:24:47.863970+0000","last_unstale":"2024-09-10T21:24:52.885174+0000","last_undegraded":"2024-09-10T21:24:52.885174+0000","last_fullsized":"2024-09-10T21:24:52.885174+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:46:36.918539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.878793+0000","last_change":"2024-09-10T21:24:47.860734+0000","last_active":"2024-09-10T21:24:52.878793+0000","last_peered":"2024-09-10T21:24:52.878793+0000","last_clean":"2024-09-10T21:24:52.878793+0000","last_became_active":"2024-09-10T21:24:47.860471+0000","last_became_peered":"2024-09-10T21:24:47.860471+0000","last_unstale":"2024-09-10T21:24:52.878793+0000","last_undegraded":"2024-09-10T21:24:52.878793+0000","last_fullsized":"2024-09-10T21:24:52.878793+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:54:13.200082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.885049+0000","last_change":"2024-09-10T21:24:47.872504+0000","last_active":"2024-09-10T21:24:52.885049+0000","last_peered":"2024-09-10T21:24:52.885049+0000","last_clean":"2024-09-10T21:24:52.885049+0000","last_became_active":"2024-09-10T21:24:47.872218+0000","last_became_peered":"2024-09-10T21:24:47.872218+0000","last_unstale":"2024-09-10T21:24:52.885049+0000","last_undegraded":"2024-09-10T21:24:52.885049+0000","last_fullsized":"2024-09-10T21:24:52.885049+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:21:32.400520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.885478+0000","last_change":"2024-09-10T21:24:47.862167+0000","last_active":"2024-09-10T21:24:52.885478+0000","last_peered":"2024-09-10T21:24:52.885478+0000","last_clean":"2024-09-10T21:24:52.885478+0000","last_became_active":"2024-09-10T21:24:47.859649+0000","last_became_peered":"2024-09-10T21:24:47.859649+0000","last_unstale":"2024-09-10T21:24:52.885478+0000","last_undegraded":"2024-09-10T21:24:52.885478+0000","last_fullsized":"2024-09-10T21:24:52.885478+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:42:59.204284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.173544+0000","last_change":"2024-09-10T21:24:47.865322+0000","last_active":"2024-09-10T21:24:53.173544+0000","last_peered":"2024-09-10T21:24:53.173544+0000","last_clean":"2024-09-10T21:24:53.173544+0000","last_became_active":"2024-09-10T21:24:47.864933+0000","last_became_peered":"2024-09-10T21:24:47.864933+0000","last_unstale":"2024-09-10T21:24:53.173544+0000","last_undegraded":"2024-09-10T21:24:53.173544+0000","last_fullsized":"2024-09-10T21:24:53.173544+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:40:45.551723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.884760+0000","last_change":"2024-09-10T21:24:47.860155+0000","last_active":"2024-09-10T21:24:52.884760+0000","last_peered":"2024-09-10T21:24:52.884760+0000","last_clean":"2024-09-10T21:24:52.884760+0000","last_became_active":"2024-09-10T21:24:47.859857+0000","last_became_peered":"2024-09-10T21:24:47.859857+0000","last_unstale":"2024-09-10T21:24:52.884760+0000","last_undegraded":"2024-09-10T21:24:52.884760+0000","last_fullsized":"2024-09-10T21:24:52.884760+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:32:50.027827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:25:28.798192+0000","last_change":"2024-09-10T21:24:47.872848+0000","last_active":"2024-09-10T21:25:28.798192+0000","last_peered":"2024-09-10T21:25:28.798192+0000","last_clean":"2024-09-10T21:25:28.798192+0000","last_became_active":"2024-09-10T21:24:47.868671+0000","last_became_peered":"2024-09-10T21:24:47.868671+0000","last_unstale":"2024-09-10T21:25:28.798192+0000","last_undegraded":"2024-09-10T21:25:28.798192+0000","last_fullsized":"2024-09-10T21:25:28.798192+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:33:59.291647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174519+0000","last_change":"2024-09-10T21:24:47.857464+0000","last_active":"2024-09-10T21:24:53.174519+0000","last_peered":"2024-09-10T21:24:53.174519+0000","last_clean":"2024-09-10T21:24:53.174519+0000","last_became_active":"2024-09-10T21:24:47.857116+0000","last_became_peered":"2024-09-10T21:24:47.857116+0000","last_unstale":"2024-09-10T21:24:53.174519+0000","last_undegraded":"2024-09-10T21:24:53.174519+0000","last_fullsized":"2024-09-10T21:24:53.174519+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:56:43.269026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.884005+0000","last_change":"2024-09-10T21:24:47.871619+0000","last_active":"2024-09-10T21:24:52.884005+0000","last_peered":"2024-09-10T21:24:52.884005+0000","last_clean":"2024-09-10T21:24:52.884005+0000","last_became_active":"2024-09-10T21:24:47.871350+0000","last_became_peered":"2024-09-10T21:24:47.871350+0000","last_unstale":"2024-09-10T21:24:52.884005+0000","last_undegraded":"2024-09-10T21:24:52.884005+0000","last_fullsized":"2024-09-10T21:24:52.884005+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:41.305116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.175386+0000","last_change":"2024-09-10T21:24:47.861352+0000","last_active":"2024-09-10T21:24:53.175386+0000","last_peered":"2024-09-10T21:24:53.175386+0000","last_clean":"2024-09-10T21:24:53.175386+0000","last_became_active":"2024-09-10T21:24:47.861007+0000","last_became_peered":"2024-09-10T21:24:47.861007+0000","last_unstale":"2024-09-10T21:24:53.175386+0000","last_undegraded":"2024-09-10T21:24:53.175386+0000","last_fullsized":"2024-09-10T21:24:53.175386+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:25.273458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.175077+0000","last_change":"2024-09-10T21:24:47.864205+0000","last_active":"2024-09-10T21:24:53.175077+0000","last_peered":"2024-09-10T21:24:53.175077+0000","last_clean":"2024-09-10T21:24:53.175077+0000","last_became_active":"2024-09-10T21:24:47.863932+0000","last_became_peered":"2024-09-10T21:24:47.863932+0000","last_unstale":"2024-09-10T21:24:53.175077+0000","last_undegraded":"2024-09-10T21:24:53.175077+0000","last_fullsized":"2024-09-10T21:24:53.175077+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:53:17.669629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:25:28.799060+0000","last_change":"2024-09-10T21:24:47.866225+0000","last_active":"2024-09-10T21:25:28.799060+0000","last_peered":"2024-09-10T21:25:28.799060+0000","last_clean":"2024-09-10T21:25:28.799060+0000","last_became_active":"2024-09-10T21:24:47.865587+0000","last_became_peered":"2024-09-10T21:24:47.865587+0000","last_unstale":"2024-09-10T21:25:28.799060+0000","last_undegraded":"2024-09-10T21:25:28.799060+0000","last_fullsized":"2024-09-10T21:25:28.799060+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:49:26.113068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:25:28.797907+0000","last_change":"2024-09-10T21:24:47.872669+0000","last_active":"2024-09-10T21:25:28.797907+0000","last_peered":"2024-09-10T21:25:28.797907+0000","last_clean":"2024-09-10T21:25:28.797907+0000","last_became_active":"2024-09-10T21:24:47.865837+0000","last_became_peered":"2024-09-10T21:24:47.865837+0000","last_unstale":"2024-09-10T21:25:28.797907+0000","last_undegraded":"2024-09-10T21:25:28.797907+0000","last_fullsized":"2024-09-10T21:25:28.797907+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:53:09.082512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.878332+0000","last_change":"2024-09-10T21:24:47.846567+0000","last_active":"2024-09-10T21:24:52.878332+0000","last_peered":"2024-09-10T21:24:52.878332+0000","last_clean":"2024-09-10T21:24:52.878332+0000","last_became_active":"2024-09-10T21:24:47.846283+0000","last_became_peered":"2024-09-10T21:24:47.846283+0000","last_unstale":"2024-09-10T21:24:52.878332+0000","last_undegraded":"2024-09-10T21:24:52.878332+0000","last_fullsized":"2024-09-10T21:24:52.878332+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:42:15.320518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174247+0000","last_change":"2024-09-10T21:24:47.844888+0000","last_active":"2024-09-10T21:24:53.174247+0000","last_peered":"2024-09-10T21:24:53.174247+0000","last_clean":"2024-09-10T21:24:53.174247+0000","last_became_active":"2024-09-10T21:24:47.844517+0000","last_became_peered":"2024-09-10T21:24:47.844517+0000","last_unstale":"2024-09-10T21:24:53.174247+0000","last_undegraded":"2024-09-10T21:24:53.174247+0000","last_fullsized":"2024-09-10T21:24:53.174247+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:00:14.988282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.884642+0000","last_change":"2024-09-10T21:24:47.862746+0000","last_active":"2024-09-10T21:24:52.884642+0000","last_peered":"2024-09-10T21:24:52.884642+0000","last_clean":"2024-09-10T21:24:52.884642+0000","last_became_active":"2024-09-10T21:24:47.862185+0000","last_became_peered":"2024-09-10T21:24:47.862185+0000","last_unstale":"2024-09-10T21:24:52.884642+0000","last_undegraded":"2024-09-10T21:24:52.884642+0000","last_fullsized":"2024-09-10T21:24:52.884642+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:06:21.461089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.886651+0000","last_change":"2024-09-10T21:24:47.867727+0000","last_active":"2024-09-10T21:24:52.886651+0000","last_peered":"2024-09-10T21:24:52.886651+0000","last_clean":"2024-09-10T21:24:52.886651+0000","last_became_active":"2024-09-10T21:24:47.861045+0000","last_became_peered":"2024-09-10T21:24:47.861045+0000","last_unstale":"2024-09-10T21:24:52.886651+0000","last_undegraded":"2024-09-10T21:24:52.886651+0000","last_fullsized":"2024-09-10T21:24:52.886651+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:08:30.065968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.884547+0000","last_change":"2024-09-10T21:24:47.867410+0000","last_active":"2024-09-10T21:24:52.884547+0000","last_peered":"2024-09-10T21:24:52.884547+0000","last_clean":"2024-09-10T21:24:52.884547+0000","last_became_active":"2024-09-10T21:24:47.867113+0000","last_became_peered":"2024-09-10T21:24:47.867113+0000","last_unstale":"2024-09-10T21:24:52.884547+0000","last_undegraded":"2024-09-10T21:24:52.884547+0000","last_fullsized":"2024-09-10T21:24:52.884547+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:49:53.502472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.884956+0000","last_change":"2024-09-10T21:24:47.867473+0000","last_active":"2024-09-10T21:24:52.884956+0000","last_peered":"2024-09-10T21:24:52.884956+0000","last_clean":"2024-09-10T21:24:52.884956+0000","last_became_active":"2024-09-10T21:24:47.867126+0000","last_became_peered":"2024-09-10T21:24:47.867126+0000","last_unstale":"2024-09-10T21:24:52.884956+0000","last_undegraded":"2024-09-10T21:24:52.884956+0000","last_fullsized":"2024-09-10T21:24:52.884956+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:54:32.307507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.175341+0000","last_change":"2024-09-10T21:24:47.865450+0000","last_active":"2024-09-10T21:24:53.175341+0000","last_peered":"2024-09-10T21:24:53.175341+0000","last_clean":"2024-09-10T21:24:53.175341+0000","last_became_active":"2024-09-10T21:24:47.865093+0000","last_became_peered":"2024-09-10T21:24:47.865093+0000","last_unstale":"2024-09-10T21:24:53.175341+0000","last_undegraded":"2024-09-10T21:24:53.175341+0000","last_fullsized":"2024-09-10T21:24:53.175341+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:12:18.854783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.884322+0000","last_change":"2024-09-10T21:24:47.868093+0000","last_active":"2024-09-10T21:24:52.884322+0000","last_peered":"2024-09-10T21:24:52.884322+0000","last_clean":"2024-09-10T21:24:52.884322+0000","last_became_active":"2024-09-10T21:24:47.867782+0000","last_became_peered":"2024-09-10T21:24:47.867782+0000","last_unstale":"2024-09-10T21:24:52.884322+0000","last_undegraded":"2024-09-10T21:24:52.884322+0000","last_fullsized":"2024-09-10T21:24:52.884322+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:34:13.053494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.173787+0000","last_change":"2024-09-10T21:24:47.862204+0000","last_active":"2024-09-10T21:24:53.173787+0000","last_peered":"2024-09-10T21:24:53.173787+0000","last_clean":"2024-09-10T21:24:53.173787+0000","last_became_active":"2024-09-10T21:24:47.861434+0000","last_became_peered":"2024-09-10T21:24:47.861434+0000","last_unstale":"2024-09-10T21:24:53.173787+0000","last_undegraded":"2024-09-10T21:24:53.173787+0000","last_fullsized":"2024-09-10T21:24:53.173787+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:50:03.245285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:25:28.798174+0000","last_change":"2024-09-10T21:24:47.869248+0000","last_active":"2024-09-10T21:25:28.798174+0000","last_peered":"2024-09-10T21:25:28.798174+0000","last_clean":"2024-09-10T21:25:28.798174+0000","last_became_active":"2024-09-10T21:24:47.868503+0000","last_became_peered":"2024-09-10T21:24:47.868503+0000","last_unstale":"2024-09-10T21:25:28.798174+0000","last_undegraded":"2024-09-10T21:25:28.798174+0000","last_fullsized":"2024-09-10T21:25:28.798174+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:51:31.417845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174007+0000","last_change":"2024-09-10T21:24:47.864802+0000","last_active":"2024-09-10T21:24:53.174007+0000","last_peered":"2024-09-10T21:24:53.174007+0000","last_clean":"2024-09-10T21:24:53.174007+0000","last_became_active":"2024-09-10T21:24:47.863976+0000","last_became_peered":"2024-09-10T21:24:47.863976+0000","last_unstale":"2024-09-10T21:24:53.174007+0000","last_undegraded":"2024-09-10T21:24:53.174007+0000","last_fullsized":"2024-09-10T21:24:53.174007+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:55:08.544075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:25:28.798522+0000","last_change":"2024-09-10T21:24:47.873364+0000","last_active":"2024-09-10T21:25:28.798522+0000","last_peered":"2024-09-10T21:25:28.798522+0000","last_clean":"2024-09-10T21:25:28.798522+0000","last_became_active":"2024-09-10T21:24:47.873069+0000","last_became_peered":"2024-09-10T21:24:47.873069+0000","last_unstale":"2024-09-10T21:25:28.798522+0000","last_undegraded":"2024-09-10T21:25:28.798522+0000","last_fullsized":"2024-09-10T21:25:28.798522+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:38:21.368499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174196+0000","last_change":"2024-09-10T21:24:47.856936+0000","last_active":"2024-09-10T21:24:53.174196+0000","last_peered":"2024-09-10T21:24:53.174196+0000","last_clean":"2024-09-10T21:24:53.174196+0000","last_became_active":"2024-09-10T21:24:47.854713+0000","last_became_peered":"2024-09-10T21:24:47.854713+0000","last_unstale":"2024-09-10T21:24:53.174196+0000","last_undegraded":"2024-09-10T21:24:53.174196+0000","last_fullsized":"2024-09-10T21:24:53.174196+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:46:36.918539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.878924+0000","last_change":"2024-09-10T21:24:47.846492+0000","last_active":"2024-09-10T21:24:52.878924+0000","last_peered":"2024-09-10T21:24:52.878924+0000","last_clean":"2024-09-10T21:24:52.878924+0000","last_became_active":"2024-09-10T21:24:47.846227+0000","last_became_peered":"2024-09-10T21:24:47.846227+0000","last_unstale":"2024-09-10T21:24:52.878924+0000","last_undegraded":"2024-09-10T21:24:52.878924+0000","last_fullsized":"2024-09-10T21:24:52.878924+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:13:58.432819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.175219+0000","last_change":"2024-09-10T21:24:47.871469+0000","last_active":"2024-09-10T21:24:53.175219+0000","last_peered":"2024-09-10T21:24:53.175219+0000","last_clean":"2024-09-10T21:24:53.175219+0000","last_became_active":"2024-09-10T21:24:47.871177+0000","last_became_peered":"2024-09-10T21:24:47.871177+0000","last_unstale":"2024-09-10T21:24:53.175219+0000","last_undegraded":"2024-09-10T21:24:53.175219+0000","last_fullsized":"2024-09-10T21:24:53.175219+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:19:24.226916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":66932,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509452,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":320,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":327680,"data_stored":693447,"data_compressed":33466,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509453,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":270336,"data_stored":106339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509452,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":258048,"data_stored":102497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509453,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31456,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340384,"statfs":{"total":96636764160,"available":96604553216,"internally_reserved":0,"allocated":266240,"data_stored":102657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542157,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27296,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":270336,"data_stored":102623,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542156,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":270336,"data_stored":106339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542159,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27364,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344476,"statfs":{"total":96636764160,"available":96608743424,"internally_reserved":0,"allocated":335872,"data_stored":693503,"data_compressed":33466,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574861,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27296,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":274432,"data_stored":109725,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33466,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33466,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T21:25:30.838 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T21:25:31.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:31.018+0000 7ff226e00640 1 -- 172.21.15.17:0/40110247 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff220157f80 msgr2=0x7ff2200c1040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:31.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:31.018+0000 7ff226e00640 1 --2- 172.21.15.17:0/40110247 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff220157f80 0x7ff2200c1040 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7ff208009840 tx=0x7ff20801c820 comp rx=0 tx=0).stop 2024-09-10T21:25:31.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:31.018+0000 7ff226e00640 1 -- 172.21.15.17:0/40110247 shutdown_connections 2024-09-10T21:25:31.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:31.018+0000 7ff226e00640 1 --2- 172.21.15.17:0/40110247 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff220157f80 0x7ff2200c1040 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:31.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:31.018+0000 7ff226e00640 1 --2- 172.21.15.17:0/40110247 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff2201515f0 0x7ff220157a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:31.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:31.018+0000 7ff226e00640 1 -- 172.21.15.17:0/40110247 >> 172.21.15.17:0/40110247 conn(0x7ff220070160 msgr2=0x7ff220070570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:31.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:31.018+0000 7ff226e00640 1 -- 172.21.15.17:0/40110247 shutdown_connections 2024-09-10T21:25:31.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:31.018+0000 7ff226e00640 1 -- 172.21.15.17:0/40110247 wait complete. 2024-09-10T21:25:31.023 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:31.018+0000 7ff226e00640 1 Processor -- start 2024-09-10T21:25:31.023 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:31.018+0000 7ff226e00640 1 -- start start 2024-09-10T21:25:31.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:31.022+0000 7ff226e00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff2200c1580 0x7ff22014ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:31.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:31.022+0000 7ff226e00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff2201515f0 0x7ff22013ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:31.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:31.022+0000 7ff226e00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff220157f80 0x7ff2201495d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:31.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:31.022+0000 7ff226e00640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7ff2200c7fe0 con 0x7ff2201515f0 2024-09-10T21:25:31.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:31.022+0000 7ff226e00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7ff220157520 con 0x7ff2200c1580 2024-09-10T21:25:31.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:31.022+0000 7ff226e00640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7ff2200c8160 con 0x7ff220157f80 2024-09-10T21:25:31.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:31.022+0000 7ff217400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff2201515f0 0x7ff22013ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:31.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:31.022+0000 7ff217400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff2201515f0 0x7ff22013ee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43744/0 (socket says 172.21.15.17:43744) 2024-09-10T21:25:31.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:31.022+0000 7ff217400640 1 -- 172.21.15.17:0/1136626289 learned_addr learned my addr 172.21.15.17:0/1136626289 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:25:31.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:31.022+0000 7ff21fe00640 1 --2- 172.21.15.17:0/1136626289 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff2200c1580 0x7ff22014ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:31.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:31.022+0000 7ff224e00640 1 --2- 172.21.15.17:0/1136626289 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff220157f80 0x7ff2201495d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:31.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:31.022+0000 7ff217400640 1 -- 172.21.15.17:0/1136626289 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff220157f80 msgr2=0x7ff2201495d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:31.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:31.022+0000 7ff217400640 1 --2- 172.21.15.17:0/1136626289 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff220157f80 0x7ff2201495d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:31.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:31.022+0000 7ff217400640 1 -- 172.21.15.17:0/1136626289 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff2200c1580 msgr2=0x7ff22014ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:31.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:31.022+0000 7ff217400640 1 --2- 172.21.15.17:0/1136626289 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff2200c1580 0x7ff22014ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:31.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:31.022+0000 7ff217400640 1 -- 172.21.15.17:0/1136626289 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff220070540 con 0x7ff2201515f0 2024-09-10T21:25:31.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:31.022+0000 7ff21fe00640 1 --2- 172.21.15.17:0/1136626289 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff2200c1580 0x7ff22014ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:25:31.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:31.022+0000 7ff217400640 1 --2- 172.21.15.17:0/1136626289 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff2201515f0 0x7ff22013ee30 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7ff208019430 tx=0x7ff2080a5750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:31.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:31.022+0000 7ff21cc00640 1 -- 172.21.15.17:0/1136626289 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2080b2040 con 0x7ff2201515f0 2024-09-10T21:25:31.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:31.022+0000 7ff21cc00640 1 -- 172.21.15.17:0/1136626289 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff2080041e0 con 0x7ff2201515f0 2024-09-10T21:25:31.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:31.022+0000 7ff226e00640 1 -- 172.21.15.17:0/1136626289 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff22014e900 con 0x7ff2201515f0 2024-09-10T21:25:31.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:31.022+0000 7ff21cc00640 1 -- 172.21.15.17:0/1136626289 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff208005140 con 0x7ff2201515f0 2024-09-10T21:25:31.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:31.022+0000 7ff226e00640 1 -- 172.21.15.17:0/1136626289 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2200c5860 con 0x7ff2201515f0 2024-09-10T21:25:31.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:31.022+0000 7ff226e00640 1 -- 172.21.15.17:0/1136626289 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff220157a40 con 0x7ff2201515f0 2024-09-10T21:25:31.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:31.022+0000 7ff21cc00640 1 -- 172.21.15.17:0/1136626289 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7ff2080b5070 con 0x7ff2201515f0 2024-09-10T21:25:31.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:31.026+0000 7ff21cc00640 1 --2- 172.21.15.17:0/1136626289 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7ff1fc072610 0x7ff1fc074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:31.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:31.026+0000 7ff21cc00640 1 -- 172.21.15.17:0/1136626289 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7ff20812d110 con 0x7ff2201515f0 2024-09-10T21:25:31.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:31.026+0000 7ff21fe00640 1 --2- 172.21.15.17:0/1136626289 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7ff1fc072610 0x7ff1fc074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:31.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:31.026+0000 7ff21fe00640 1 --2- 172.21.15.17:0/1136626289 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7ff1fc072610 0x7ff1fc074ad0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7ff210009c40 tx=0x7ff210009260 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:31.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:31.030+0000 7ff21cc00640 1 -- 172.21.15.17:0/1136626289 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7ff2080c4060 con 0x7ff2201515f0 2024-09-10T21:25:31.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:31.198+0000 7ff226e00640 1 -- 172.21.15.17:0/1136626289 --> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff220157040 con 0x7ff1fc072610 2024-09-10T21:25:31.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:31.206+0000 7ff21cc00640 1 -- 172.21.15.17:0/1136626289 <== mgr.4100 v2:172.21.15.31:6808/3101308586 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+332620 (secure 0 0 0) 0x7ff220157040 con 0x7ff1fc072610 2024-09-10T21:25:31.213 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-10T21:25:31.215 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2024-09-10T21:25:31.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:31.210+0000 7ff226e00640 1 -- 172.21.15.17:0/1136626289 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7ff1fc072610 msgr2=0x7ff1fc074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:31.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:31.210+0000 7ff226e00640 1 --2- 172.21.15.17:0/1136626289 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7ff1fc072610 0x7ff1fc074ad0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7ff210009c40 tx=0x7ff210009260 comp rx=0 tx=0).stop 2024-09-10T21:25:31.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:31.210+0000 7ff226e00640 1 -- 172.21.15.17:0/1136626289 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff2201515f0 msgr2=0x7ff22013ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:31.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:31.210+0000 7ff226e00640 1 --2- 172.21.15.17:0/1136626289 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff2201515f0 0x7ff22013ee30 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7ff208019430 tx=0x7ff2080a5750 comp rx=0 tx=0).stop 2024-09-10T21:25:31.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:31.210+0000 7ff226e00640 1 -- 172.21.15.17:0/1136626289 shutdown_connections 2024-09-10T21:25:31.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:31.210+0000 7ff226e00640 1 --2- 172.21.15.17:0/1136626289 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7ff1fc072610 0x7ff1fc074ad0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:31.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:31.210+0000 7ff226e00640 1 --2- 172.21.15.17:0/1136626289 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff220157f80 0x7ff2201495d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:31.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:31.210+0000 7ff226e00640 1 --2- 172.21.15.17:0/1136626289 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff2201515f0 0x7ff22013ee30 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:31.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:31.210+0000 7ff226e00640 1 --2- 172.21.15.17:0/1136626289 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff2200c1580 0x7ff22014ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:31.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:31.210+0000 7ff226e00640 1 -- 172.21.15.17:0/1136626289 >> 172.21.15.17:0/1136626289 conn(0x7ff220070160 msgr2=0x7ff220153ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:31.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:31.210+0000 7ff226e00640 1 -- 172.21.15.17:0/1136626289 shutdown_connections 2024-09-10T21:25:31.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:31.210+0000 7ff226e00640 1 -- 172.21.15.17:0/1136626289 wait complete. 2024-09-10T21:25:31.234 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":48,"stamp":"2024-09-10T21:25:29.348856+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":149,"num_write_kb":1646,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":114,"ondisk_log_size":114,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":222616,"kb_used_data":2220,"kb_used_omap":12,"kb_used_meta":220211,"kb_avail":754752104,"statfs":{"total":773094113280,"available":772866154496,"internally_reserved":0,"allocated":2273280,"data_stored":2017130,"data_compressed":66932,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":12752,"internal_metadata":225496624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002308"},"pg_stats":[{"pgid":"2.1f","version":"24'4","reported_seq":27,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174456+0000","last_change":"2024-09-10T21:24:46.864556+0000","last_active":"2024-09-10T21:24:53.174456+0000","last_peered":"2024-09-10T21:24:53.174456+0000","last_clean":"2024-09-10T21:24:53.174456+0000","last_became_active":"2024-09-10T21:24:46.864288+0000","last_became_peered":"2024-09-10T21:24:46.864288+0000","last_unstale":"2024-09-10T21:24:53.174456+0000","last_undegraded":"2024-09-10T21:24:53.174456+0000","last_fullsized":"2024-09-10T21:24:53.174456+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:00:38.734807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.885137+0000","last_change":"2024-09-10T21:24:47.864452+0000","last_active":"2024-09-10T21:24:52.885137+0000","last_peered":"2024-09-10T21:24:52.885137+0000","last_clean":"2024-09-10T21:24:52.885137+0000","last_became_active":"2024-09-10T21:24:47.864168+0000","last_became_peered":"2024-09-10T21:24:47.864168+0000","last_unstale":"2024-09-10T21:24:52.885137+0000","last_undegraded":"2024-09-10T21:24:52.885137+0000","last_fullsized":"2024-09-10T21:24:52.885137+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:51:31.417845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.884507+0000","last_change":"2024-09-10T21:24:52.070054+0000","last_active":"2024-09-10T21:24:52.884507+0000","last_peered":"2024-09-10T21:24:52.884507+0000","last_clean":"2024-09-10T21:24:52.884507+0000","last_became_active":"2024-09-10T21:24:52.067258+0000","last_became_peered":"2024-09-10T21:24:52.067258+0000","last_unstale":"2024-09-10T21:24:52.884507+0000","last_undegraded":"2024-09-10T21:24:52.884507+0000","last_fullsized":"2024-09-10T21:24:52.884507+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:21:53.080013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"24'1","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.885320+0000","last_change":"2024-09-10T21:24:46.860648+0000","last_active":"2024-09-10T21:24:52.885320+0000","last_peered":"2024-09-10T21:24:52.885320+0000","last_clean":"2024-09-10T21:24:52.885320+0000","last_became_active":"2024-09-10T21:24:46.860341+0000","last_became_peered":"2024-09-10T21:24:46.860341+0000","last_unstale":"2024-09-10T21:24:52.885320+0000","last_undegraded":"2024-09-10T21:24:52.885320+0000","last_fullsized":"2024-09-10T21:24:52.885320+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:39:24.819979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174137+0000","last_change":"2024-09-10T21:24:47.865617+0000","last_active":"2024-09-10T21:24:53.174137+0000","last_peered":"2024-09-10T21:24:53.174137+0000","last_clean":"2024-09-10T21:24:53.174137+0000","last_became_active":"2024-09-10T21:24:47.865365+0000","last_became_peered":"2024-09-10T21:24:47.865365+0000","last_unstale":"2024-09-10T21:24:53.174137+0000","last_undegraded":"2024-09-10T21:24:53.174137+0000","last_fullsized":"2024-09-10T21:24:53.174137+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:10:05.971996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.883672+0000","last_change":"2024-09-10T21:24:51.900077+0000","last_active":"2024-09-10T21:24:52.883672+0000","last_peered":"2024-09-10T21:24:52.883672+0000","last_clean":"2024-09-10T21:24:52.883672+0000","last_became_active":"2024-09-10T21:24:51.899652+0000","last_became_peered":"2024-09-10T21:24:51.899652+0000","last_unstale":"2024-09-10T21:24:52.883672+0000","last_undegraded":"2024-09-10T21:24:52.883672+0000","last_fullsized":"2024-09-10T21:24:52.883672+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:40:49.585857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"24'2","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.175383+0000","last_change":"2024-09-10T21:24:46.861040+0000","last_active":"2024-09-10T21:24:53.175383+0000","last_peered":"2024-09-10T21:24:53.175383+0000","last_clean":"2024-09-10T21:24:53.175383+0000","last_became_active":"2024-09-10T21:24:46.860780+0000","last_became_peered":"2024-09-10T21:24:46.860780+0000","last_unstale":"2024-09-10T21:24:53.175383+0000","last_undegraded":"2024-09-10T21:24:53.175383+0000","last_fullsized":"2024-09-10T21:24:53.175383+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:53:31.454472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:25:28.798529+0000","last_change":"2024-09-10T21:24:47.877997+0000","last_active":"2024-09-10T21:25:28.798529+0000","last_peered":"2024-09-10T21:25:28.798529+0000","last_clean":"2024-09-10T21:25:28.798529+0000","last_became_active":"2024-09-10T21:24:47.877765+0000","last_became_peered":"2024-09-10T21:24:47.877765+0000","last_unstale":"2024-09-10T21:25:28.798529+0000","last_undegraded":"2024-09-10T21:25:28.798529+0000","last_fullsized":"2024-09-10T21:25:28.798529+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:33:20.590209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.886528+0000","last_change":"2024-09-10T21:24:51.908333+0000","last_active":"2024-09-10T21:24:52.886528+0000","last_peered":"2024-09-10T21:24:52.886528+0000","last_clean":"2024-09-10T21:24:52.886528+0000","last_became_active":"2024-09-10T21:24:51.907933+0000","last_became_peered":"2024-09-10T21:24:51.907933+0000","last_unstale":"2024-09-10T21:24:52.886528+0000","last_undegraded":"2024-09-10T21:24:52.886528+0000","last_fullsized":"2024-09-10T21:24:52.886528+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:07:33.193555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"24'2","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174867+0000","last_change":"2024-09-10T21:24:46.873044+0000","last_active":"2024-09-10T21:24:53.174867+0000","last_peered":"2024-09-10T21:24:53.174867+0000","last_clean":"2024-09-10T21:24:53.174867+0000","last_became_active":"2024-09-10T21:24:46.872075+0000","last_became_peered":"2024-09-10T21:24:46.872075+0000","last_unstale":"2024-09-10T21:24:53.174867+0000","last_undegraded":"2024-09-10T21:24:53.174867+0000","last_fullsized":"2024-09-10T21:24:53.174867+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:40:33.632846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:25:28.798704+0000","last_change":"2024-09-10T21:24:47.878094+0000","last_active":"2024-09-10T21:25:28.798704+0000","last_peered":"2024-09-10T21:25:28.798704+0000","last_clean":"2024-09-10T21:25:28.798704+0000","last_became_active":"2024-09-10T21:24:47.876688+0000","last_became_peered":"2024-09-10T21:24:47.876688+0000","last_unstale":"2024-09-10T21:25:28.798704+0000","last_undegraded":"2024-09-10T21:25:28.798704+0000","last_fullsized":"2024-09-10T21:25:28.798704+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:03:37.076080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.884328+0000","last_change":"2024-09-10T21:24:51.900208+0000","last_active":"2024-09-10T21:24:52.884328+0000","last_peered":"2024-09-10T21:24:52.884328+0000","last_clean":"2024-09-10T21:24:52.884328+0000","last_became_active":"2024-09-10T21:24:51.899576+0000","last_became_peered":"2024-09-10T21:24:51.899576+0000","last_unstale":"2024-09-10T21:24:52.884328+0000","last_undegraded":"2024-09-10T21:24:52.884328+0000","last_fullsized":"2024-09-10T21:24:52.884328+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:52:49.479932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.883746+0000","last_change":"2024-09-10T21:24:47.869830+0000","last_active":"2024-09-10T21:24:52.883746+0000","last_peered":"2024-09-10T21:24:52.883746+0000","last_clean":"2024-09-10T21:24:52.883746+0000","last_became_active":"2024-09-10T21:24:47.869569+0000","last_became_peered":"2024-09-10T21:24:47.869569+0000","last_unstale":"2024-09-10T21:24:52.883746+0000","last_undegraded":"2024-09-10T21:24:52.883746+0000","last_fullsized":"2024-09-10T21:24:52.883746+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:42:15.320518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"24'2","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174248+0000","last_change":"2024-09-10T21:24:46.867297+0000","last_active":"2024-09-10T21:24:53.174248+0000","last_peered":"2024-09-10T21:24:53.174248+0000","last_clean":"2024-09-10T21:24:53.174248+0000","last_became_active":"2024-09-10T21:24:46.866893+0000","last_became_peered":"2024-09-10T21:24:46.866893+0000","last_unstale":"2024-09-10T21:24:53.174248+0000","last_undegraded":"2024-09-10T21:24:53.174248+0000","last_fullsized":"2024-09-10T21:24:53.174248+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:53.197666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.885044+0000","last_change":"2024-09-10T21:24:51.908412+0000","last_active":"2024-09-10T21:24:52.885044+0000","last_peered":"2024-09-10T21:24:52.885044+0000","last_clean":"2024-09-10T21:24:52.885044+0000","last_became_active":"2024-09-10T21:24:51.907923+0000","last_became_peered":"2024-09-10T21:24:51.907923+0000","last_unstale":"2024-09-10T21:24:52.885044+0000","last_undegraded":"2024-09-10T21:24:52.885044+0000","last_fullsized":"2024-09-10T21:24:52.885044+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:26:46.479501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.884755+0000","last_change":"2024-09-10T21:24:47.862479+0000","last_active":"2024-09-10T21:24:52.884755+0000","last_peered":"2024-09-10T21:24:52.884755+0000","last_clean":"2024-09-10T21:24:52.884755+0000","last_became_active":"2024-09-10T21:24:47.861842+0000","last_became_peered":"2024-09-10T21:24:47.861842+0000","last_unstale":"2024-09-10T21:24:52.884755+0000","last_undegraded":"2024-09-10T21:24:52.884755+0000","last_fullsized":"2024-09-10T21:24:52.884755+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:53:09.082512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.878911+0000","last_change":"2024-09-10T21:24:46.852467+0000","last_active":"2024-09-10T21:24:52.878911+0000","last_peered":"2024-09-10T21:24:52.878911+0000","last_clean":"2024-09-10T21:24:52.878911+0000","last_became_active":"2024-09-10T21:24:46.851027+0000","last_became_peered":"2024-09-10T21:24:46.851027+0000","last_unstale":"2024-09-10T21:24:52.878911+0000","last_undegraded":"2024-09-10T21:24:52.878911+0000","last_fullsized":"2024-09-10T21:24:52.878911+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:39:27.441714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.886564+0000","last_change":"2024-09-10T21:24:51.903577+0000","last_active":"2024-09-10T21:24:52.886564+0000","last_peered":"2024-09-10T21:24:52.886564+0000","last_clean":"2024-09-10T21:24:52.886564+0000","last_became_active":"2024-09-10T21:24:51.903031+0000","last_became_peered":"2024-09-10T21:24:51.903031+0000","last_unstale":"2024-09-10T21:24:52.886564+0000","last_undegraded":"2024-09-10T21:24:52.886564+0000","last_fullsized":"2024-09-10T21:24:52.886564+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:08:52.787720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.884612+0000","last_change":"2024-09-10T21:24:47.856334+0000","last_active":"2024-09-10T21:24:52.884612+0000","last_peered":"2024-09-10T21:24:52.884612+0000","last_clean":"2024-09-10T21:24:52.884612+0000","last_became_active":"2024-09-10T21:24:47.856003+0000","last_became_peered":"2024-09-10T21:24:47.856003+0000","last_unstale":"2024-09-10T21:24:52.884612+0000","last_undegraded":"2024-09-10T21:24:52.884612+0000","last_fullsized":"2024-09-10T21:24:52.884612+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:58:53.698584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.878371+0000","last_change":"2024-09-10T21:24:46.853373+0000","last_active":"2024-09-10T21:24:52.878371+0000","last_peered":"2024-09-10T21:24:52.878371+0000","last_clean":"2024-09-10T21:24:52.878371+0000","last_became_active":"2024-09-10T21:24:46.852993+0000","last_became_peered":"2024-09-10T21:24:46.852993+0000","last_unstale":"2024-09-10T21:24:52.878371+0000","last_undegraded":"2024-09-10T21:24:52.878371+0000","last_fullsized":"2024-09-10T21:24:52.878371+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:53:31.454472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.879271+0000","last_change":"2024-09-10T21:24:52.065246+0000","last_active":"2024-09-10T21:24:52.879271+0000","last_peered":"2024-09-10T21:24:52.879271+0000","last_clean":"2024-09-10T21:24:52.879271+0000","last_became_active":"2024-09-10T21:24:52.064884+0000","last_became_peered":"2024-09-10T21:24:52.064884+0000","last_unstale":"2024-09-10T21:24:52.879271+0000","last_undegraded":"2024-09-10T21:24:52.879271+0000","last_fullsized":"2024-09-10T21:24:52.879271+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:54:54.393305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.884318+0000","last_change":"2024-09-10T21:24:47.856517+0000","last_active":"2024-09-10T21:24:52.884318+0000","last_peered":"2024-09-10T21:24:52.884318+0000","last_clean":"2024-09-10T21:24:52.884318+0000","last_became_active":"2024-09-10T21:24:47.856185+0000","last_became_peered":"2024-09-10T21:24:47.856185+0000","last_unstale":"2024-09-10T21:24:52.884318+0000","last_undegraded":"2024-09-10T21:24:52.884318+0000","last_fullsized":"2024-09-10T21:24:52.884318+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:26:00.852818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.173635+0000","last_change":"2024-09-10T21:24:46.867888+0000","last_active":"2024-09-10T21:24:53.173635+0000","last_peered":"2024-09-10T21:24:53.173635+0000","last_clean":"2024-09-10T21:24:53.173635+0000","last_became_active":"2024-09-10T21:24:46.866477+0000","last_became_peered":"2024-09-10T21:24:46.866477+0000","last_unstale":"2024-09-10T21:24:53.173635+0000","last_undegraded":"2024-09-10T21:24:53.173635+0000","last_fullsized":"2024-09-10T21:24:53.173635+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:39:27.441714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.885185+0000","last_change":"2024-09-10T21:24:51.908376+0000","last_active":"2024-09-10T21:24:52.885185+0000","last_peered":"2024-09-10T21:24:52.885185+0000","last_clean":"2024-09-10T21:24:52.885185+0000","last_became_active":"2024-09-10T21:24:51.907878+0000","last_became_peered":"2024-09-10T21:24:51.907878+0000","last_unstale":"2024-09-10T21:24:52.885185+0000","last_undegraded":"2024-09-10T21:24:52.885185+0000","last_fullsized":"2024-09-10T21:24:52.885185+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:39:02.492952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174342+0000","last_change":"2024-09-10T21:24:47.844823+0000","last_active":"2024-09-10T21:24:53.174342+0000","last_peered":"2024-09-10T21:24:53.174342+0000","last_clean":"2024-09-10T21:24:53.174342+0000","last_became_active":"2024-09-10T21:24:47.844505+0000","last_became_peered":"2024-09-10T21:24:47.844505+0000","last_unstale":"2024-09-10T21:24:53.174342+0000","last_undegraded":"2024-09-10T21:24:53.174342+0000","last_fullsized":"2024-09-10T21:24:53.174342+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:37:51.784258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"24'2","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.885358+0000","last_change":"2024-09-10T21:24:46.862172+0000","last_active":"2024-09-10T21:24:52.885358+0000","last_peered":"2024-09-10T21:24:52.885358+0000","last_clean":"2024-09-10T21:24:52.885358+0000","last_became_active":"2024-09-10T21:24:46.861473+0000","last_became_peered":"2024-09-10T21:24:46.861473+0000","last_unstale":"2024-09-10T21:24:52.885358+0000","last_undegraded":"2024-09-10T21:24:52.885358+0000","last_fullsized":"2024-09-10T21:24:52.885358+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:53:31.454472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.883945+0000","last_change":"2024-09-10T21:24:52.062841+0000","last_active":"2024-09-10T21:24:52.883945+0000","last_peered":"2024-09-10T21:24:52.883945+0000","last_clean":"2024-09-10T21:24:52.883945+0000","last_became_active":"2024-09-10T21:24:52.061581+0000","last_became_peered":"2024-09-10T21:24:52.061581+0000","last_unstale":"2024-09-10T21:24:52.883945+0000","last_undegraded":"2024-09-10T21:24:52.883945+0000","last_fullsized":"2024-09-10T21:24:52.883945+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:34:03.325781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.885543+0000","last_change":"2024-09-10T21:24:47.859801+0000","last_active":"2024-09-10T21:24:52.885543+0000","last_peered":"2024-09-10T21:24:52.885543+0000","last_clean":"2024-09-10T21:24:52.885543+0000","last_became_active":"2024-09-10T21:24:47.859483+0000","last_became_peered":"2024-09-10T21:24:47.859483+0000","last_unstale":"2024-09-10T21:24:52.885543+0000","last_undegraded":"2024-09-10T21:24:52.885543+0000","last_fullsized":"2024-09-10T21:24:52.885543+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:38:21.368499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"25'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.879175+0000","last_change":"2024-09-10T21:24:46.860624+0000","last_active":"2024-09-10T21:24:52.879175+0000","last_peered":"2024-09-10T21:24:52.879175+0000","last_clean":"2024-09-10T21:24:52.879175+0000","last_became_active":"2024-09-10T21:24:46.860309+0000","last_became_peered":"2024-09-10T21:24:46.860309+0000","last_unstale":"2024-09-10T21:24:52.879175+0000","last_undegraded":"2024-09-10T21:24:52.879175+0000","last_fullsized":"2024-09-10T21:24:52.879175+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:37:12.332013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.884414+0000","last_change":"2024-09-10T21:24:52.068581+0000","last_active":"2024-09-10T21:24:52.884414+0000","last_peered":"2024-09-10T21:24:52.884414+0000","last_clean":"2024-09-10T21:24:52.884414+0000","last_became_active":"2024-09-10T21:24:52.068250+0000","last_became_peered":"2024-09-10T21:24:52.068250+0000","last_unstale":"2024-09-10T21:24:52.884414+0000","last_undegraded":"2024-09-10T21:24:52.884414+0000","last_fullsized":"2024-09-10T21:24:52.884414+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:52:01.990009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.879000+0000","last_change":"2024-09-10T21:24:47.854042+0000","last_active":"2024-09-10T21:24:52.879000+0000","last_peered":"2024-09-10T21:24:52.879000+0000","last_clean":"2024-09-10T21:24:52.879000+0000","last_became_active":"2024-09-10T21:24:47.853750+0000","last_became_peered":"2024-09-10T21:24:47.853750+0000","last_unstale":"2024-09-10T21:24:52.879000+0000","last_undegraded":"2024-09-10T21:24:52.879000+0000","last_fullsized":"2024-09-10T21:24:52.879000+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:06:50.775132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.175510+0000","last_change":"2024-09-10T21:24:46.864617+0000","last_active":"2024-09-10T21:24:53.175510+0000","last_peered":"2024-09-10T21:24:53.175510+0000","last_clean":"2024-09-10T21:24:53.175510+0000","last_became_active":"2024-09-10T21:24:46.864149+0000","last_became_peered":"2024-09-10T21:24:46.864149+0000","last_unstale":"2024-09-10T21:24:53.175510+0000","last_undegraded":"2024-09-10T21:24:53.175510+0000","last_fullsized":"2024-09-10T21:24:53.175510+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:24:15.673107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174530+0000","last_change":"2024-09-10T21:24:52.061485+0000","last_active":"2024-09-10T21:24:53.174530+0000","last_peered":"2024-09-10T21:24:53.174530+0000","last_clean":"2024-09-10T21:24:53.174530+0000","last_became_active":"2024-09-10T21:24:52.060953+0000","last_became_peered":"2024-09-10T21:24:52.060953+0000","last_unstale":"2024-09-10T21:24:53.174530+0000","last_undegraded":"2024-09-10T21:24:53.174530+0000","last_fullsized":"2024-09-10T21:24:53.174530+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:49:30.147202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:25:28.798884+0000","last_change":"2024-09-10T21:24:47.867241+0000","last_active":"2024-09-10T21:25:28.798884+0000","last_peered":"2024-09-10T21:25:28.798884+0000","last_clean":"2024-09-10T21:25:28.798884+0000","last_became_active":"2024-09-10T21:24:47.866907+0000","last_became_peered":"2024-09-10T21:24:47.866907+0000","last_unstale":"2024-09-10T21:25:28.798884+0000","last_undegraded":"2024-09-10T21:25:28.798884+0000","last_fullsized":"2024-09-10T21:25:28.798884+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:07.620800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.879175+0000","last_change":"2024-09-10T21:24:46.856848+0000","last_active":"2024-09-10T21:24:52.879175+0000","last_peered":"2024-09-10T21:24:52.879175+0000","last_clean":"2024-09-10T21:24:52.879175+0000","last_became_active":"2024-09-10T21:24:46.856489+0000","last_became_peered":"2024-09-10T21:24:46.856489+0000","last_unstale":"2024-09-10T21:24:52.879175+0000","last_undegraded":"2024-09-10T21:24:52.879175+0000","last_fullsized":"2024-09-10T21:24:52.879175+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:46:55.286321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.878563+0000","last_change":"2024-09-10T21:24:51.901016+0000","last_active":"2024-09-10T21:24:52.878563+0000","last_peered":"2024-09-10T21:24:52.878563+0000","last_clean":"2024-09-10T21:24:52.878563+0000","last_became_active":"2024-09-10T21:24:51.900479+0000","last_became_peered":"2024-09-10T21:24:51.900479+0000","last_unstale":"2024-09-10T21:24:52.878563+0000","last_undegraded":"2024-09-10T21:24:52.878563+0000","last_fullsized":"2024-09-10T21:24:52.878563+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:36.741810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.886797+0000","last_change":"2024-09-10T21:24:47.858753+0000","last_active":"2024-09-10T21:24:52.886797+0000","last_peered":"2024-09-10T21:24:52.886797+0000","last_clean":"2024-09-10T21:24:52.886797+0000","last_became_active":"2024-09-10T21:24:47.858438+0000","last_became_peered":"2024-09-10T21:24:47.858438+0000","last_unstale":"2024-09-10T21:24:52.886797+0000","last_undegraded":"2024-09-10T21:24:52.886797+0000","last_fullsized":"2024-09-10T21:24:52.886797+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:12:09.198013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:25:28.798402+0000","last_change":"2024-09-10T21:24:46.863610+0000","last_active":"2024-09-10T21:25:28.798402+0000","last_peered":"2024-09-10T21:25:28.798402+0000","last_clean":"2024-09-10T21:25:28.798402+0000","last_became_active":"2024-09-10T21:24:46.863150+0000","last_became_peered":"2024-09-10T21:24:46.863150+0000","last_unstale":"2024-09-10T21:25:28.798402+0000","last_undegraded":"2024-09-10T21:25:28.798402+0000","last_fullsized":"2024-09-10T21:25:28.798402+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:27:35.473995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.878912+0000","last_change":"2024-09-10T21:24:51.907366+0000","last_active":"2024-09-10T21:24:52.878912+0000","last_peered":"2024-09-10T21:24:52.878912+0000","last_clean":"2024-09-10T21:24:52.878912+0000","last_became_active":"2024-09-10T21:24:51.906888+0000","last_became_peered":"2024-09-10T21:24:51.906888+0000","last_unstale":"2024-09-10T21:24:52.878912+0000","last_undegraded":"2024-09-10T21:24:52.878912+0000","last_fullsized":"2024-09-10T21:24:52.878912+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:24:54.278514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.878445+0000","last_change":"2024-09-10T21:24:47.850155+0000","last_active":"2024-09-10T21:24:52.878445+0000","last_peered":"2024-09-10T21:24:52.878445+0000","last_clean":"2024-09-10T21:24:52.878445+0000","last_became_active":"2024-09-10T21:24:47.849846+0000","last_became_peered":"2024-09-10T21:24:47.849846+0000","last_unstale":"2024-09-10T21:24:52.878445+0000","last_undegraded":"2024-09-10T21:24:52.878445+0000","last_fullsized":"2024-09-10T21:24:52.878445+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:13:40.032808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.173931+0000","last_change":"2024-09-10T21:24:46.851800+0000","last_active":"2024-09-10T21:24:53.173931+0000","last_peered":"2024-09-10T21:24:53.173931+0000","last_clean":"2024-09-10T21:24:53.173931+0000","last_became_active":"2024-09-10T21:24:46.851200+0000","last_became_peered":"2024-09-10T21:24:46.851200+0000","last_unstale":"2024-09-10T21:24:53.173931+0000","last_undegraded":"2024-09-10T21:24:53.173931+0000","last_fullsized":"2024-09-10T21:24:53.173931+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:00:22.839361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.879296+0000","last_change":"2024-09-10T21:24:51.902529+0000","last_active":"2024-09-10T21:24:52.879296+0000","last_peered":"2024-09-10T21:24:52.879296+0000","last_clean":"2024-09-10T21:24:52.879296+0000","last_became_active":"2024-09-10T21:24:51.901141+0000","last_became_peered":"2024-09-10T21:24:51.901141+0000","last_unstale":"2024-09-10T21:24:52.879296+0000","last_undegraded":"2024-09-10T21:24:52.879296+0000","last_fullsized":"2024-09-10T21:24:52.879296+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:08:40.203348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.173878+0000","last_change":"2024-09-10T21:24:47.846582+0000","last_active":"2024-09-10T21:24:53.173878+0000","last_peered":"2024-09-10T21:24:53.173878+0000","last_clean":"2024-09-10T21:24:53.173878+0000","last_became_active":"2024-09-10T21:24:47.846358+0000","last_became_peered":"2024-09-10T21:24:47.846358+0000","last_unstale":"2024-09-10T21:24:53.173878+0000","last_undegraded":"2024-09-10T21:24:53.173878+0000","last_fullsized":"2024-09-10T21:24:53.173878+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:58:53.698584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.173773+0000","last_change":"2024-09-10T21:24:46.870908+0000","last_active":"2024-09-10T21:24:53.173773+0000","last_peered":"2024-09-10T21:24:53.173773+0000","last_clean":"2024-09-10T21:24:53.173773+0000","last_became_active":"2024-09-10T21:24:46.868375+0000","last_became_peered":"2024-09-10T21:24:46.868375+0000","last_unstale":"2024-09-10T21:24:53.173773+0000","last_undegraded":"2024-09-10T21:24:53.173773+0000","last_fullsized":"2024-09-10T21:24:53.173773+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:37:12.332013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174362+0000","last_change":"2024-09-10T21:24:52.069018+0000","last_active":"2024-09-10T21:24:53.174362+0000","last_peered":"2024-09-10T21:24:53.174362+0000","last_clean":"2024-09-10T21:24:53.174362+0000","last_became_active":"2024-09-10T21:24:52.067308+0000","last_became_peered":"2024-09-10T21:24:52.067308+0000","last_unstale":"2024-09-10T21:24:53.174362+0000","last_undegraded":"2024-09-10T21:24:53.174362+0000","last_fullsized":"2024-09-10T21:24:53.174362+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:07:25.162168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.884767+0000","last_change":"2024-09-10T21:24:47.871542+0000","last_active":"2024-09-10T21:24:52.884767+0000","last_peered":"2024-09-10T21:24:52.884767+0000","last_clean":"2024-09-10T21:24:52.884767+0000","last_became_active":"2024-09-10T21:24:47.871128+0000","last_became_peered":"2024-09-10T21:24:47.871128+0000","last_unstale":"2024-09-10T21:24:52.884767+0000","last_undegraded":"2024-09-10T21:24:52.884767+0000","last_fullsized":"2024-09-10T21:24:52.884767+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:51:22.876126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:25:28.798353+0000","last_change":"2024-09-10T21:24:46.859218+0000","last_active":"2024-09-10T21:25:28.798353+0000","last_peered":"2024-09-10T21:25:28.798353+0000","last_clean":"2024-09-10T21:25:28.798353+0000","last_became_active":"2024-09-10T21:24:46.853190+0000","last_became_peered":"2024-09-10T21:24:46.853190+0000","last_unstale":"2024-09-10T21:25:28.798353+0000","last_undegraded":"2024-09-10T21:25:28.798353+0000","last_fullsized":"2024-09-10T21:25:28.798353+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:26:34.848759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":119,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.173768+0000","last_change":"2024-09-10T21:24:37.216542+0000","last_active":"2024-09-10T21:24:53.173768+0000","last_peered":"2024-09-10T21:24:53.173768+0000","last_clean":"2024-09-10T21:24:53.173768+0000","last_became_active":"2024-09-10T21:24:37.215443+0000","last_became_peered":"2024-09-10T21:24:37.215443+0000","last_unstale":"2024-09-10T21:24:53.173768+0000","last_undegraded":"2024-09-10T21:24:53.173768+0000","last_fullsized":"2024-09-10T21:24:53.173768+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:35.877801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:35.877801+0000","last_clean_scrub_stamp":"2024-09-10T21:24:35.877801+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:14:20.260329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.884231+0000","last_change":"2024-09-10T21:24:52.067645+0000","last_active":"2024-09-10T21:24:52.884231+0000","last_peered":"2024-09-10T21:24:52.884231+0000","last_clean":"2024-09-10T21:24:52.884231+0000","last_became_active":"2024-09-10T21:24:52.067148+0000","last_became_peered":"2024-09-10T21:24:52.067148+0000","last_unstale":"2024-09-10T21:24:52.884231+0000","last_undegraded":"2024-09-10T21:24:52.884231+0000","last_fullsized":"2024-09-10T21:24:52.884231+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:54:42.435011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.175128+0000","last_change":"2024-09-10T21:24:47.867282+0000","last_active":"2024-09-10T21:24:53.175128+0000","last_peered":"2024-09-10T21:24:53.175128+0000","last_clean":"2024-09-10T21:24:53.175128+0000","last_became_active":"2024-09-10T21:24:47.865582+0000","last_became_peered":"2024-09-10T21:24:47.865582+0000","last_unstale":"2024-09-10T21:24:53.175128+0000","last_undegraded":"2024-09-10T21:24:53.175128+0000","last_fullsized":"2024-09-10T21:24:53.175128+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:38:55.009670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174808+0000","last_change":"2024-09-10T21:24:46.859532+0000","last_active":"2024-09-10T21:24:53.174808+0000","last_peered":"2024-09-10T21:24:53.174808+0000","last_clean":"2024-09-10T21:24:53.174808+0000","last_became_active":"2024-09-10T21:24:46.859144+0000","last_became_peered":"2024-09-10T21:24:46.859144+0000","last_unstale":"2024-09-10T21:24:53.174808+0000","last_undegraded":"2024-09-10T21:24:53.174808+0000","last_fullsized":"2024-09-10T21:24:53.174808+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:48:41.992441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.883627+0000","last_change":"2024-09-10T21:24:52.061029+0000","last_active":"2024-09-10T21:24:52.883627+0000","last_peered":"2024-09-10T21:24:52.883627+0000","last_clean":"2024-09-10T21:24:52.883627+0000","last_became_active":"2024-09-10T21:24:52.060605+0000","last_became_peered":"2024-09-10T21:24:52.060605+0000","last_unstale":"2024-09-10T21:24:52.883627+0000","last_undegraded":"2024-09-10T21:24:52.883627+0000","last_fullsized":"2024-09-10T21:24:52.883627+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:09:48.077266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:25:28.798667+0000","last_change":"2024-09-10T21:24:47.866927+0000","last_active":"2024-09-10T21:25:28.798667+0000","last_peered":"2024-09-10T21:25:28.798667+0000","last_clean":"2024-09-10T21:25:28.798667+0000","last_became_active":"2024-09-10T21:24:47.866650+0000","last_became_peered":"2024-09-10T21:24:47.866650+0000","last_unstale":"2024-09-10T21:25:28.798667+0000","last_undegraded":"2024-09-10T21:25:28.798667+0000","last_fullsized":"2024-09-10T21:25:28.798667+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:47:37.304624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"24'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.879000+0000","last_change":"2024-09-10T21:24:46.857082+0000","last_active":"2024-09-10T21:24:52.879000+0000","last_peered":"2024-09-10T21:24:52.879000+0000","last_clean":"2024-09-10T21:24:52.879000+0000","last_became_active":"2024-09-10T21:24:46.856799+0000","last_became_peered":"2024-09-10T21:24:46.856799+0000","last_unstale":"2024-09-10T21:24:52.879000+0000","last_undegraded":"2024-09-10T21:24:52.879000+0000","last_fullsized":"2024-09-10T21:24:52.879000+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:40:33.632846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174743+0000","last_change":"2024-09-10T21:24:51.911880+0000","last_active":"2024-09-10T21:24:53.174743+0000","last_peered":"2024-09-10T21:24:53.174743+0000","last_clean":"2024-09-10T21:24:53.174743+0000","last_became_active":"2024-09-10T21:24:51.911088+0000","last_became_peered":"2024-09-10T21:24:51.911088+0000","last_unstale":"2024-09-10T21:24:53.174743+0000","last_undegraded":"2024-09-10T21:24:53.174743+0000","last_fullsized":"2024-09-10T21:24:53.174743+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:38:57.030655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.175370+0000","last_change":"2024-09-10T21:24:47.876946+0000","last_active":"2024-09-10T21:24:53.175370+0000","last_peered":"2024-09-10T21:24:53.175370+0000","last_clean":"2024-09-10T21:24:53.175370+0000","last_became_active":"2024-09-10T21:24:47.876690+0000","last_became_peered":"2024-09-10T21:24:47.876690+0000","last_unstale":"2024-09-10T21:24:53.175370+0000","last_undegraded":"2024-09-10T21:24:53.175370+0000","last_fullsized":"2024-09-10T21:24:53.175370+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:14:06.368275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"24'2","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.175105+0000","last_change":"2024-09-10T21:24:46.855248+0000","last_active":"2024-09-10T21:24:53.175105+0000","last_peered":"2024-09-10T21:24:53.175105+0000","last_clean":"2024-09-10T21:24:53.175105+0000","last_became_active":"2024-09-10T21:24:46.854930+0000","last_became_peered":"2024-09-10T21:24:46.854930+0000","last_unstale":"2024-09-10T21:24:53.175105+0000","last_undegraded":"2024-09-10T21:24:53.175105+0000","last_fullsized":"2024-09-10T21:24:53.175105+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:46:55.286321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:25:28.797831+0000","last_change":"2024-09-10T21:24:52.063112+0000","last_active":"2024-09-10T21:25:28.797831+0000","last_peered":"2024-09-10T21:25:28.797831+0000","last_clean":"2024-09-10T21:25:28.797831+0000","last_became_active":"2024-09-10T21:24:52.062395+0000","last_became_peered":"2024-09-10T21:24:52.062395+0000","last_unstale":"2024-09-10T21:25:28.797831+0000","last_undegraded":"2024-09-10T21:25:28.797831+0000","last_fullsized":"2024-09-10T21:25:28.797831+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:21:53.080013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.175254+0000","last_change":"2024-09-10T21:24:47.858593+0000","last_active":"2024-09-10T21:24:53.175254+0000","last_peered":"2024-09-10T21:24:53.175254+0000","last_clean":"2024-09-10T21:24:53.175254+0000","last_became_active":"2024-09-10T21:24:47.858216+0000","last_became_peered":"2024-09-10T21:24:47.858216+0000","last_unstale":"2024-09-10T21:24:53.175254+0000","last_undegraded":"2024-09-10T21:24:53.175254+0000","last_fullsized":"2024-09-10T21:24:53.175254+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:19:42.961291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"24'1","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.886950+0000","last_change":"2024-09-10T21:24:46.860330+0000","last_active":"2024-09-10T21:24:52.886950+0000","last_peered":"2024-09-10T21:24:52.886950+0000","last_clean":"2024-09-10T21:24:52.886950+0000","last_became_active":"2024-09-10T21:24:46.860053+0000","last_became_peered":"2024-09-10T21:24:46.860053+0000","last_unstale":"2024-09-10T21:24:52.886950+0000","last_undegraded":"2024-09-10T21:24:52.886950+0000","last_fullsized":"2024-09-10T21:24:52.886950+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:58:53.177740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.883701+0000","last_change":"2024-09-10T21:24:51.901673+0000","last_active":"2024-09-10T21:24:52.883701+0000","last_peered":"2024-09-10T21:24:52.883701+0000","last_clean":"2024-09-10T21:24:52.883701+0000","last_became_active":"2024-09-10T21:24:51.901222+0000","last_became_peered":"2024-09-10T21:24:51.901222+0000","last_unstale":"2024-09-10T21:24:52.883701+0000","last_undegraded":"2024-09-10T21:24:52.883701+0000","last_fullsized":"2024-09-10T21:24:52.883701+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:07:52.120299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.173790+0000","last_change":"2024-09-10T21:24:47.870589+0000","last_active":"2024-09-10T21:24:53.173790+0000","last_peered":"2024-09-10T21:24:53.173790+0000","last_clean":"2024-09-10T21:24:53.173790+0000","last_became_active":"2024-09-10T21:24:47.861834+0000","last_became_peered":"2024-09-10T21:24:47.861834+0000","last_unstale":"2024-09-10T21:24:53.173790+0000","last_undegraded":"2024-09-10T21:24:53.173790+0000","last_fullsized":"2024-09-10T21:24:53.173790+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:43.363013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"24'8","reported_seq":31,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.173490+0000","last_change":"2024-09-10T21:24:46.852064+0000","last_active":"2024-09-10T21:24:53.173490+0000","last_peered":"2024-09-10T21:24:53.173490+0000","last_clean":"2024-09-10T21:24:53.173490+0000","last_became_active":"2024-09-10T21:24:46.851776+0000","last_became_peered":"2024-09-10T21:24:46.851776+0000","last_unstale":"2024-09-10T21:24:53.173490+0000","last_undegraded":"2024-09-10T21:24:53.173490+0000","last_fullsized":"2024-09-10T21:24:53.173490+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:26:24.371844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174153+0000","last_change":"2024-09-10T21:24:51.904449+0000","last_active":"2024-09-10T21:24:53.174153+0000","last_peered":"2024-09-10T21:24:53.174153+0000","last_clean":"2024-09-10T21:24:53.174153+0000","last_became_active":"2024-09-10T21:24:51.903533+0000","last_became_peered":"2024-09-10T21:24:51.903533+0000","last_unstale":"2024-09-10T21:24:53.174153+0000","last_undegraded":"2024-09-10T21:24:53.174153+0000","last_fullsized":"2024-09-10T21:24:53.174153+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:11.654935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.883479+0000","last_change":"2024-09-10T21:24:47.857736+0000","last_active":"2024-09-10T21:24:52.883479+0000","last_peered":"2024-09-10T21:24:52.883479+0000","last_clean":"2024-09-10T21:24:52.883479+0000","last_became_active":"2024-09-10T21:24:47.857437+0000","last_became_peered":"2024-09-10T21:24:47.857437+0000","last_unstale":"2024-09-10T21:24:52.883479+0000","last_undegraded":"2024-09-10T21:24:52.883479+0000","last_fullsized":"2024-09-10T21:24:52.883479+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:13:40.032808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.883979+0000","last_change":"2024-09-10T21:24:46.860662+0000","last_active":"2024-09-10T21:24:52.883979+0000","last_peered":"2024-09-10T21:24:52.883979+0000","last_clean":"2024-09-10T21:24:52.883979+0000","last_became_active":"2024-09-10T21:24:46.859796+0000","last_became_peered":"2024-09-10T21:24:46.859796+0000","last_unstale":"2024-09-10T21:24:52.883979+0000","last_undegraded":"2024-09-10T21:24:52.883979+0000","last_fullsized":"2024-09-10T21:24:52.883979+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:48:27.758481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.885543+0000","last_change":"2024-09-10T21:24:52.069770+0000","last_active":"2024-09-10T21:24:52.885543+0000","last_peered":"2024-09-10T21:24:52.885543+0000","last_clean":"2024-09-10T21:24:52.885543+0000","last_became_active":"2024-09-10T21:24:52.067409+0000","last_became_peered":"2024-09-10T21:24:52.067409+0000","last_unstale":"2024-09-10T21:24:52.885543+0000","last_undegraded":"2024-09-10T21:24:52.885543+0000","last_fullsized":"2024-09-10T21:24:52.885543+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:23:39.622612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174136+0000","last_change":"2024-09-10T21:24:47.858692+0000","last_active":"2024-09-10T21:24:53.174136+0000","last_peered":"2024-09-10T21:24:53.174136+0000","last_clean":"2024-09-10T21:24:53.174136+0000","last_became_active":"2024-09-10T21:24:47.858343+0000","last_became_peered":"2024-09-10T21:24:47.858343+0000","last_unstale":"2024-09-10T21:24:53.174136+0000","last_undegraded":"2024-09-10T21:24:53.174136+0000","last_fullsized":"2024-09-10T21:24:53.174136+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:04:36.661791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:25:28.798320+0000","last_change":"2024-09-10T21:24:46.864592+0000","last_active":"2024-09-10T21:25:28.798320+0000","last_peered":"2024-09-10T21:25:28.798320+0000","last_clean":"2024-09-10T21:25:28.798320+0000","last_became_active":"2024-09-10T21:24:46.858696+0000","last_became_peered":"2024-09-10T21:24:46.858696+0000","last_unstale":"2024-09-10T21:25:28.798320+0000","last_undegraded":"2024-09-10T21:25:28.798320+0000","last_fullsized":"2024-09-10T21:25:28.798320+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:54:19.356867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.885074+0000","last_change":"2024-09-10T21:24:52.061572+0000","last_active":"2024-09-10T21:24:52.885074+0000","last_peered":"2024-09-10T21:24:52.885074+0000","last_clean":"2024-09-10T21:24:52.885074+0000","last_became_active":"2024-09-10T21:24:52.061061+0000","last_became_peered":"2024-09-10T21:24:52.061061+0000","last_unstale":"2024-09-10T21:24:52.885074+0000","last_undegraded":"2024-09-10T21:24:52.885074+0000","last_fullsized":"2024-09-10T21:24:52.885074+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:04:36.399811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174340+0000","last_change":"2024-09-10T21:24:47.857699+0000","last_active":"2024-09-10T21:24:53.174340+0000","last_peered":"2024-09-10T21:24:53.174340+0000","last_clean":"2024-09-10T21:24:53.174340+0000","last_became_active":"2024-09-10T21:24:47.857429+0000","last_became_peered":"2024-09-10T21:24:47.857429+0000","last_unstale":"2024-09-10T21:24:53.174340+0000","last_undegraded":"2024-09-10T21:24:53.174340+0000","last_fullsized":"2024-09-10T21:24:53.174340+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:07:11.880228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"24'2","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174839+0000","last_change":"2024-09-10T21:24:46.859753+0000","last_active":"2024-09-10T21:24:53.174839+0000","last_peered":"2024-09-10T21:24:53.174839+0000","last_clean":"2024-09-10T21:24:53.174839+0000","last_became_active":"2024-09-10T21:24:46.859299+0000","last_became_peered":"2024-09-10T21:24:46.859299+0000","last_unstale":"2024-09-10T21:24:53.174839+0000","last_undegraded":"2024-09-10T21:24:53.174839+0000","last_fullsized":"2024-09-10T21:24:53.174839+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:06:41.965052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:25:28.798655+0000","last_change":"2024-09-10T21:24:52.064718+0000","last_active":"2024-09-10T21:25:28.798655+0000","last_peered":"2024-09-10T21:25:28.798655+0000","last_clean":"2024-09-10T21:25:28.798655+0000","last_became_active":"2024-09-10T21:24:52.062327+0000","last_became_peered":"2024-09-10T21:24:52.062327+0000","last_unstale":"2024-09-10T21:25:28.798655+0000","last_undegraded":"2024-09-10T21:25:28.798655+0000","last_fullsized":"2024-09-10T21:25:28.798655+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:43:17.173095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.175009+0000","last_change":"2024-09-10T21:24:47.870766+0000","last_active":"2024-09-10T21:24:53.175009+0000","last_peered":"2024-09-10T21:24:53.175009+0000","last_clean":"2024-09-10T21:24:53.175009+0000","last_became_active":"2024-09-10T21:24:47.870276+0000","last_became_peered":"2024-09-10T21:24:47.870276+0000","last_unstale":"2024-09-10T21:24:53.175009+0000","last_undegraded":"2024-09-10T21:24:53.175009+0000","last_fullsized":"2024-09-10T21:24:53.175009+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:54:38.400877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"25'9","reported_seq":31,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:25:05.421191+0000","last_change":"2024-09-10T21:24:46.867790+0000","last_active":"2024-09-10T21:25:05.421191+0000","last_peered":"2024-09-10T21:25:05.421191+0000","last_clean":"2024-09-10T21:25:05.421191+0000","last_became_active":"2024-09-10T21:24:46.866899+0000","last_became_peered":"2024-09-10T21:24:46.866899+0000","last_unstale":"2024-09-10T21:25:05.421191+0000","last_undegraded":"2024-09-10T21:25:05.421191+0000","last_fullsized":"2024-09-10T21:25:05.421191+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:25:04.923730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.878572+0000","last_change":"2024-09-10T21:24:51.899803+0000","last_active":"2024-09-10T21:24:52.878572+0000","last_peered":"2024-09-10T21:24:52.878572+0000","last_clean":"2024-09-10T21:24:52.878572+0000","last_became_active":"2024-09-10T21:24:51.895398+0000","last_became_peered":"2024-09-10T21:24:51.895398+0000","last_unstale":"2024-09-10T21:24:52.878572+0000","last_undegraded":"2024-09-10T21:24:52.878572+0000","last_fullsized":"2024-09-10T21:24:52.878572+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:52:34.689860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.886661+0000","last_change":"2024-09-10T21:24:47.857544+0000","last_active":"2024-09-10T21:24:52.886661+0000","last_peered":"2024-09-10T21:24:52.886661+0000","last_clean":"2024-09-10T21:24:52.886661+0000","last_became_active":"2024-09-10T21:24:47.857284+0000","last_became_peered":"2024-09-10T21:24:47.857284+0000","last_unstale":"2024-09-10T21:24:52.886661+0000","last_undegraded":"2024-09-10T21:24:52.886661+0000","last_fullsized":"2024-09-10T21:24:52.886661+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:10:22.747606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"24'3","reported_seq":25,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.878942+0000","last_change":"2024-09-10T21:24:46.859633+0000","last_active":"2024-09-10T21:24:52.878942+0000","last_peered":"2024-09-10T21:24:52.878942+0000","last_clean":"2024-09-10T21:24:52.878942+0000","last_became_active":"2024-09-10T21:24:46.859332+0000","last_became_peered":"2024-09-10T21:24:46.859332+0000","last_unstale":"2024-09-10T21:24:52.878942+0000","last_undegraded":"2024-09-10T21:24:52.878942+0000","last_fullsized":"2024-09-10T21:24:52.878942+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:56:46.975225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.886734+0000","last_change":"2024-09-10T21:24:51.899100+0000","last_active":"2024-09-10T21:24:52.886734+0000","last_peered":"2024-09-10T21:24:52.886734+0000","last_clean":"2024-09-10T21:24:52.886734+0000","last_became_active":"2024-09-10T21:24:51.898642+0000","last_became_peered":"2024-09-10T21:24:51.898642+0000","last_unstale":"2024-09-10T21:24:52.886734+0000","last_undegraded":"2024-09-10T21:24:52.886734+0000","last_fullsized":"2024-09-10T21:24:52.886734+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:06.394081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174291+0000","last_change":"2024-09-10T21:24:47.867890+0000","last_active":"2024-09-10T21:24:53.174291+0000","last_peered":"2024-09-10T21:24:53.174291+0000","last_clean":"2024-09-10T21:24:53.174291+0000","last_became_active":"2024-09-10T21:24:47.865016+0000","last_became_peered":"2024-09-10T21:24:47.865016+0000","last_unstale":"2024-09-10T21:24:53.174291+0000","last_undegraded":"2024-09-10T21:24:53.174291+0000","last_fullsized":"2024-09-10T21:24:53.174291+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:53:09.082512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:25:28.798864+0000","last_change":"2024-09-10T21:24:46.861400+0000","last_active":"2024-09-10T21:25:28.798864+0000","last_peered":"2024-09-10T21:25:28.798864+0000","last_clean":"2024-09-10T21:25:28.798864+0000","last_became_active":"2024-09-10T21:24:46.861136+0000","last_became_peered":"2024-09-10T21:24:46.861136+0000","last_unstale":"2024-09-10T21:25:28.798864+0000","last_undegraded":"2024-09-10T21:25:28.798864+0000","last_fullsized":"2024-09-10T21:25:28.798864+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:35:31.682448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.884671+0000","last_change":"2024-09-10T21:24:51.907850+0000","last_active":"2024-09-10T21:24:52.884671+0000","last_peered":"2024-09-10T21:24:52.884671+0000","last_clean":"2024-09-10T21:24:52.884671+0000","last_became_active":"2024-09-10T21:24:51.907062+0000","last_became_peered":"2024-09-10T21:24:51.907062+0000","last_unstale":"2024-09-10T21:24:52.884671+0000","last_undegraded":"2024-09-10T21:24:52.884671+0000","last_fullsized":"2024-09-10T21:24:52.884671+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:32:52.519627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:25:28.798854+0000","last_change":"2024-09-10T21:24:47.861800+0000","last_active":"2024-09-10T21:25:28.798854+0000","last_peered":"2024-09-10T21:25:28.798854+0000","last_clean":"2024-09-10T21:25:28.798854+0000","last_became_active":"2024-09-10T21:24:47.861498+0000","last_became_peered":"2024-09-10T21:24:47.861498+0000","last_unstale":"2024-09-10T21:25:28.798854+0000","last_undegraded":"2024-09-10T21:25:28.798854+0000","last_fullsized":"2024-09-10T21:25:28.798854+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:07:21.128033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.885153+0000","last_change":"2024-09-10T21:24:46.871677+0000","last_active":"2024-09-10T21:24:52.885153+0000","last_peered":"2024-09-10T21:24:52.885153+0000","last_clean":"2024-09-10T21:24:52.885153+0000","last_became_active":"2024-09-10T21:24:46.871367+0000","last_became_peered":"2024-09-10T21:24:46.871367+0000","last_unstale":"2024-09-10T21:24:52.885153+0000","last_undegraded":"2024-09-10T21:24:52.885153+0000","last_fullsized":"2024-09-10T21:24:52.885153+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:40:33.632846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.884646+0000","last_change":"2024-09-10T21:24:52.068637+0000","last_active":"2024-09-10T21:24:52.884646+0000","last_peered":"2024-09-10T21:24:52.884646+0000","last_clean":"2024-09-10T21:24:52.884646+0000","last_became_active":"2024-09-10T21:24:52.068348+0000","last_became_peered":"2024-09-10T21:24:52.068348+0000","last_unstale":"2024-09-10T21:24:52.884646+0000","last_undegraded":"2024-09-10T21:24:52.884646+0000","last_fullsized":"2024-09-10T21:24:52.884646+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:43:17.173095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174107+0000","last_change":"2024-09-10T21:24:52.067732+0000","last_active":"2024-09-10T21:24:53.174107+0000","last_peered":"2024-09-10T21:24:53.174107+0000","last_clean":"2024-09-10T21:24:53.174107+0000","last_became_active":"2024-09-10T21:24:52.067330+0000","last_became_peered":"2024-09-10T21:24:52.067330+0000","last_unstale":"2024-09-10T21:24:53.174107+0000","last_undegraded":"2024-09-10T21:24:53.174107+0000","last_fullsized":"2024-09-10T21:24:53.174107+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:03:41.110214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.878246+0000","last_change":"2024-09-10T21:24:47.862774+0000","last_active":"2024-09-10T21:24:52.878246+0000","last_peered":"2024-09-10T21:24:52.878246+0000","last_clean":"2024-09-10T21:24:52.878246+0000","last_became_active":"2024-09-10T21:24:47.862556+0000","last_became_peered":"2024-09-10T21:24:47.862556+0000","last_unstale":"2024-09-10T21:24:52.878246+0000","last_undegraded":"2024-09-10T21:24:52.878246+0000","last_fullsized":"2024-09-10T21:24:52.878246+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:17:52.343553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:25:28.798072+0000","last_change":"2024-09-10T21:24:46.862847+0000","last_active":"2024-09-10T21:25:28.798072+0000","last_peered":"2024-09-10T21:25:28.798072+0000","last_clean":"2024-09-10T21:25:28.798072+0000","last_became_active":"2024-09-10T21:24:46.862540+0000","last_became_peered":"2024-09-10T21:24:46.862540+0000","last_unstale":"2024-09-10T21:25:28.798072+0000","last_undegraded":"2024-09-10T21:25:28.798072+0000","last_fullsized":"2024-09-10T21:25:28.798072+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:52:31.628568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.884678+0000","last_change":"2024-09-10T21:24:51.901224+0000","last_active":"2024-09-10T21:24:52.884678+0000","last_peered":"2024-09-10T21:24:52.884678+0000","last_clean":"2024-09-10T21:24:52.884678+0000","last_became_active":"2024-09-10T21:24:51.900610+0000","last_became_peered":"2024-09-10T21:24:51.900610+0000","last_unstale":"2024-09-10T21:24:52.884678+0000","last_undegraded":"2024-09-10T21:24:52.884678+0000","last_fullsized":"2024-09-10T21:24:52.884678+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:52:34.689860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.884489+0000","last_change":"2024-09-10T21:24:47.861255+0000","last_active":"2024-09-10T21:24:52.884489+0000","last_peered":"2024-09-10T21:24:52.884489+0000","last_clean":"2024-09-10T21:24:52.884489+0000","last_became_active":"2024-09-10T21:24:47.860930+0000","last_became_peered":"2024-09-10T21:24:47.860930+0000","last_unstale":"2024-09-10T21:24:52.884489+0000","last_undegraded":"2024-09-10T21:24:52.884489+0000","last_fullsized":"2024-09-10T21:24:52.884489+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:37:51.784258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.884796+0000","last_change":"2024-09-10T21:24:46.858561+0000","last_active":"2024-09-10T21:24:52.884796+0000","last_peered":"2024-09-10T21:24:52.884796+0000","last_clean":"2024-09-10T21:24:52.884796+0000","last_became_active":"2024-09-10T21:24:46.858265+0000","last_became_peered":"2024-09-10T21:24:46.858265+0000","last_unstale":"2024-09-10T21:24:52.884796+0000","last_undegraded":"2024-09-10T21:24:52.884796+0000","last_fullsized":"2024-09-10T21:24:52.884796+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:49:00.454609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"24'1","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.884271+0000","last_change":"2024-09-10T21:24:46.856886+0000","last_active":"2024-09-10T21:24:52.884271+0000","last_peered":"2024-09-10T21:24:52.884271+0000","last_clean":"2024-09-10T21:24:52.884271+0000","last_became_active":"2024-09-10T21:24:46.856506+0000","last_became_peered":"2024-09-10T21:24:46.856506+0000","last_unstale":"2024-09-10T21:24:52.884271+0000","last_undegraded":"2024-09-10T21:24:52.884271+0000","last_fullsized":"2024-09-10T21:24:52.884271+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:56:46.975225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174721+0000","last_change":"2024-09-10T21:24:47.848602+0000","last_active":"2024-09-10T21:24:53.174721+0000","last_peered":"2024-09-10T21:24:53.174721+0000","last_clean":"2024-09-10T21:24:53.174721+0000","last_became_active":"2024-09-10T21:24:47.848160+0000","last_became_peered":"2024-09-10T21:24:47.848160+0000","last_unstale":"2024-09-10T21:24:53.174721+0000","last_undegraded":"2024-09-10T21:24:53.174721+0000","last_fullsized":"2024-09-10T21:24:53.174721+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:32:50.027827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174649+0000","last_change":"2024-09-10T21:24:51.903553+0000","last_active":"2024-09-10T21:24:53.174649+0000","last_peered":"2024-09-10T21:24:53.174649+0000","last_clean":"2024-09-10T21:24:53.174649+0000","last_became_active":"2024-09-10T21:24:51.903120+0000","last_became_peered":"2024-09-10T21:24:51.903120+0000","last_unstale":"2024-09-10T21:24:53.174649+0000","last_undegraded":"2024-09-10T21:24:53.174649+0000","last_fullsized":"2024-09-10T21:24:53.174649+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:58.240380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"24'1","reported_seq":24,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.884865+0000","last_change":"2024-09-10T21:24:46.853550+0000","last_active":"2024-09-10T21:24:52.884865+0000","last_peered":"2024-09-10T21:24:52.884865+0000","last_clean":"2024-09-10T21:24:52.884865+0000","last_became_active":"2024-09-10T21:24:46.852204+0000","last_became_peered":"2024-09-10T21:24:46.852204+0000","last_unstale":"2024-09-10T21:24:52.884865+0000","last_undegraded":"2024-09-10T21:24:52.884865+0000","last_fullsized":"2024-09-10T21:24:52.884865+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:06:37.669444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.884581+0000","last_change":"2024-09-10T21:24:47.849151+0000","last_active":"2024-09-10T21:24:52.884581+0000","last_peered":"2024-09-10T21:24:52.884581+0000","last_clean":"2024-09-10T21:24:52.884581+0000","last_became_active":"2024-09-10T21:24:47.848851+0000","last_became_peered":"2024-09-10T21:24:47.848851+0000","last_unstale":"2024-09-10T21:24:52.884581+0000","last_undegraded":"2024-09-10T21:24:52.884581+0000","last_fullsized":"2024-09-10T21:24:52.884581+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:39:25.828558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":15,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.885276+0000","last_change":"2024-09-10T21:24:51.907549+0000","last_active":"2024-09-10T21:24:52.885276+0000","last_peered":"2024-09-10T21:24:52.885276+0000","last_clean":"2024-09-10T21:24:52.885276+0000","last_became_active":"2024-09-10T21:24:51.907082+0000","last_became_peered":"2024-09-10T21:24:51.907082+0000","last_unstale":"2024-09-10T21:24:52.885276+0000","last_undegraded":"2024-09-10T21:24:52.885276+0000","last_fullsized":"2024-09-10T21:24:52.885276+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:44:30.813754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.883360+0000","last_change":"2024-09-10T21:24:47.857804+0000","last_active":"2024-09-10T21:24:52.883360+0000","last_peered":"2024-09-10T21:24:52.883360+0000","last_clean":"2024-09-10T21:24:52.883360+0000","last_became_active":"2024-09-10T21:24:47.857501+0000","last_became_peered":"2024-09-10T21:24:47.857501+0000","last_unstale":"2024-09-10T21:24:52.883360+0000","last_undegraded":"2024-09-10T21:24:52.883360+0000","last_fullsized":"2024-09-10T21:24:52.883360+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:19:20.807422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.173632+0000","last_change":"2024-09-10T21:24:47.867803+0000","last_active":"2024-09-10T21:24:53.173632+0000","last_peered":"2024-09-10T21:24:53.173632+0000","last_clean":"2024-09-10T21:24:53.173632+0000","last_became_active":"2024-09-10T21:24:47.867504+0000","last_became_peered":"2024-09-10T21:24:47.867504+0000","last_unstale":"2024-09-10T21:24:53.173632+0000","last_undegraded":"2024-09-10T21:24:53.173632+0000","last_fullsized":"2024-09-10T21:24:53.173632+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:53:30.420454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:25:28.798278+0000","last_change":"2024-09-10T21:24:47.868901+0000","last_active":"2024-09-10T21:25:28.798278+0000","last_peered":"2024-09-10T21:25:28.798278+0000","last_clean":"2024-09-10T21:25:28.798278+0000","last_became_active":"2024-09-10T21:24:47.868168+0000","last_became_peered":"2024-09-10T21:24:47.868168+0000","last_unstale":"2024-09-10T21:25:28.798278+0000","last_undegraded":"2024-09-10T21:25:28.798278+0000","last_fullsized":"2024-09-10T21:25:28.798278+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:06:21.461089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174633+0000","last_change":"2024-09-10T21:24:47.864561+0000","last_active":"2024-09-10T21:24:53.174633+0000","last_peered":"2024-09-10T21:24:53.174633+0000","last_clean":"2024-09-10T21:24:53.174633+0000","last_became_active":"2024-09-10T21:24:47.864100+0000","last_became_peered":"2024-09-10T21:24:47.864100+0000","last_unstale":"2024-09-10T21:24:53.174633+0000","last_undegraded":"2024-09-10T21:24:53.174633+0000","last_fullsized":"2024-09-10T21:24:53.174633+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:34:13.053494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.885174+0000","last_change":"2024-09-10T21:24:47.864303+0000","last_active":"2024-09-10T21:24:52.885174+0000","last_peered":"2024-09-10T21:24:52.885174+0000","last_clean":"2024-09-10T21:24:52.885174+0000","last_became_active":"2024-09-10T21:24:47.863970+0000","last_became_peered":"2024-09-10T21:24:47.863970+0000","last_unstale":"2024-09-10T21:24:52.885174+0000","last_undegraded":"2024-09-10T21:24:52.885174+0000","last_fullsized":"2024-09-10T21:24:52.885174+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:46:36.918539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.878793+0000","last_change":"2024-09-10T21:24:47.860734+0000","last_active":"2024-09-10T21:24:52.878793+0000","last_peered":"2024-09-10T21:24:52.878793+0000","last_clean":"2024-09-10T21:24:52.878793+0000","last_became_active":"2024-09-10T21:24:47.860471+0000","last_became_peered":"2024-09-10T21:24:47.860471+0000","last_unstale":"2024-09-10T21:24:52.878793+0000","last_undegraded":"2024-09-10T21:24:52.878793+0000","last_fullsized":"2024-09-10T21:24:52.878793+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:54:13.200082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.885049+0000","last_change":"2024-09-10T21:24:47.872504+0000","last_active":"2024-09-10T21:24:52.885049+0000","last_peered":"2024-09-10T21:24:52.885049+0000","last_clean":"2024-09-10T21:24:52.885049+0000","last_became_active":"2024-09-10T21:24:47.872218+0000","last_became_peered":"2024-09-10T21:24:47.872218+0000","last_unstale":"2024-09-10T21:24:52.885049+0000","last_undegraded":"2024-09-10T21:24:52.885049+0000","last_fullsized":"2024-09-10T21:24:52.885049+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:21:32.400520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.885478+0000","last_change":"2024-09-10T21:24:47.862167+0000","last_active":"2024-09-10T21:24:52.885478+0000","last_peered":"2024-09-10T21:24:52.885478+0000","last_clean":"2024-09-10T21:24:52.885478+0000","last_became_active":"2024-09-10T21:24:47.859649+0000","last_became_peered":"2024-09-10T21:24:47.859649+0000","last_unstale":"2024-09-10T21:24:52.885478+0000","last_undegraded":"2024-09-10T21:24:52.885478+0000","last_fullsized":"2024-09-10T21:24:52.885478+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:42:59.204284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.173544+0000","last_change":"2024-09-10T21:24:47.865322+0000","last_active":"2024-09-10T21:24:53.173544+0000","last_peered":"2024-09-10T21:24:53.173544+0000","last_clean":"2024-09-10T21:24:53.173544+0000","last_became_active":"2024-09-10T21:24:47.864933+0000","last_became_peered":"2024-09-10T21:24:47.864933+0000","last_unstale":"2024-09-10T21:24:53.173544+0000","last_undegraded":"2024-09-10T21:24:53.173544+0000","last_fullsized":"2024-09-10T21:24:53.173544+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:40:45.551723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.884760+0000","last_change":"2024-09-10T21:24:47.860155+0000","last_active":"2024-09-10T21:24:52.884760+0000","last_peered":"2024-09-10T21:24:52.884760+0000","last_clean":"2024-09-10T21:24:52.884760+0000","last_became_active":"2024-09-10T21:24:47.859857+0000","last_became_peered":"2024-09-10T21:24:47.859857+0000","last_unstale":"2024-09-10T21:24:52.884760+0000","last_undegraded":"2024-09-10T21:24:52.884760+0000","last_fullsized":"2024-09-10T21:24:52.884760+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:32:50.027827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:25:28.798192+0000","last_change":"2024-09-10T21:24:47.872848+0000","last_active":"2024-09-10T21:25:28.798192+0000","last_peered":"2024-09-10T21:25:28.798192+0000","last_clean":"2024-09-10T21:25:28.798192+0000","last_became_active":"2024-09-10T21:24:47.868671+0000","last_became_peered":"2024-09-10T21:24:47.868671+0000","last_unstale":"2024-09-10T21:25:28.798192+0000","last_undegraded":"2024-09-10T21:25:28.798192+0000","last_fullsized":"2024-09-10T21:25:28.798192+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:33:59.291647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174519+0000","last_change":"2024-09-10T21:24:47.857464+0000","last_active":"2024-09-10T21:24:53.174519+0000","last_peered":"2024-09-10T21:24:53.174519+0000","last_clean":"2024-09-10T21:24:53.174519+0000","last_became_active":"2024-09-10T21:24:47.857116+0000","last_became_peered":"2024-09-10T21:24:47.857116+0000","last_unstale":"2024-09-10T21:24:53.174519+0000","last_undegraded":"2024-09-10T21:24:53.174519+0000","last_fullsized":"2024-09-10T21:24:53.174519+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:56:43.269026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.884005+0000","last_change":"2024-09-10T21:24:47.871619+0000","last_active":"2024-09-10T21:24:52.884005+0000","last_peered":"2024-09-10T21:24:52.884005+0000","last_clean":"2024-09-10T21:24:52.884005+0000","last_became_active":"2024-09-10T21:24:47.871350+0000","last_became_peered":"2024-09-10T21:24:47.871350+0000","last_unstale":"2024-09-10T21:24:52.884005+0000","last_undegraded":"2024-09-10T21:24:52.884005+0000","last_fullsized":"2024-09-10T21:24:52.884005+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:41.305116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.175386+0000","last_change":"2024-09-10T21:24:47.861352+0000","last_active":"2024-09-10T21:24:53.175386+0000","last_peered":"2024-09-10T21:24:53.175386+0000","last_clean":"2024-09-10T21:24:53.175386+0000","last_became_active":"2024-09-10T21:24:47.861007+0000","last_became_peered":"2024-09-10T21:24:47.861007+0000","last_unstale":"2024-09-10T21:24:53.175386+0000","last_undegraded":"2024-09-10T21:24:53.175386+0000","last_fullsized":"2024-09-10T21:24:53.175386+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:25.273458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.175077+0000","last_change":"2024-09-10T21:24:47.864205+0000","last_active":"2024-09-10T21:24:53.175077+0000","last_peered":"2024-09-10T21:24:53.175077+0000","last_clean":"2024-09-10T21:24:53.175077+0000","last_became_active":"2024-09-10T21:24:47.863932+0000","last_became_peered":"2024-09-10T21:24:47.863932+0000","last_unstale":"2024-09-10T21:24:53.175077+0000","last_undegraded":"2024-09-10T21:24:53.175077+0000","last_fullsized":"2024-09-10T21:24:53.175077+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:53:17.669629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:25:28.799060+0000","last_change":"2024-09-10T21:24:47.866225+0000","last_active":"2024-09-10T21:25:28.799060+0000","last_peered":"2024-09-10T21:25:28.799060+0000","last_clean":"2024-09-10T21:25:28.799060+0000","last_became_active":"2024-09-10T21:24:47.865587+0000","last_became_peered":"2024-09-10T21:24:47.865587+0000","last_unstale":"2024-09-10T21:25:28.799060+0000","last_undegraded":"2024-09-10T21:25:28.799060+0000","last_fullsized":"2024-09-10T21:25:28.799060+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:49:26.113068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:25:28.797907+0000","last_change":"2024-09-10T21:24:47.872669+0000","last_active":"2024-09-10T21:25:28.797907+0000","last_peered":"2024-09-10T21:25:28.797907+0000","last_clean":"2024-09-10T21:25:28.797907+0000","last_became_active":"2024-09-10T21:24:47.865837+0000","last_became_peered":"2024-09-10T21:24:47.865837+0000","last_unstale":"2024-09-10T21:25:28.797907+0000","last_undegraded":"2024-09-10T21:25:28.797907+0000","last_fullsized":"2024-09-10T21:25:28.797907+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:53:09.082512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.878332+0000","last_change":"2024-09-10T21:24:47.846567+0000","last_active":"2024-09-10T21:24:52.878332+0000","last_peered":"2024-09-10T21:24:52.878332+0000","last_clean":"2024-09-10T21:24:52.878332+0000","last_became_active":"2024-09-10T21:24:47.846283+0000","last_became_peered":"2024-09-10T21:24:47.846283+0000","last_unstale":"2024-09-10T21:24:52.878332+0000","last_undegraded":"2024-09-10T21:24:52.878332+0000","last_fullsized":"2024-09-10T21:24:52.878332+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:42:15.320518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174247+0000","last_change":"2024-09-10T21:24:47.844888+0000","last_active":"2024-09-10T21:24:53.174247+0000","last_peered":"2024-09-10T21:24:53.174247+0000","last_clean":"2024-09-10T21:24:53.174247+0000","last_became_active":"2024-09-10T21:24:47.844517+0000","last_became_peered":"2024-09-10T21:24:47.844517+0000","last_unstale":"2024-09-10T21:24:53.174247+0000","last_undegraded":"2024-09-10T21:24:53.174247+0000","last_fullsized":"2024-09-10T21:24:53.174247+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:00:14.988282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.884642+0000","last_change":"2024-09-10T21:24:47.862746+0000","last_active":"2024-09-10T21:24:52.884642+0000","last_peered":"2024-09-10T21:24:52.884642+0000","last_clean":"2024-09-10T21:24:52.884642+0000","last_became_active":"2024-09-10T21:24:47.862185+0000","last_became_peered":"2024-09-10T21:24:47.862185+0000","last_unstale":"2024-09-10T21:24:52.884642+0000","last_undegraded":"2024-09-10T21:24:52.884642+0000","last_fullsized":"2024-09-10T21:24:52.884642+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:06:21.461089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.886651+0000","last_change":"2024-09-10T21:24:47.867727+0000","last_active":"2024-09-10T21:24:52.886651+0000","last_peered":"2024-09-10T21:24:52.886651+0000","last_clean":"2024-09-10T21:24:52.886651+0000","last_became_active":"2024-09-10T21:24:47.861045+0000","last_became_peered":"2024-09-10T21:24:47.861045+0000","last_unstale":"2024-09-10T21:24:52.886651+0000","last_undegraded":"2024-09-10T21:24:52.886651+0000","last_fullsized":"2024-09-10T21:24:52.886651+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:08:30.065968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.884547+0000","last_change":"2024-09-10T21:24:47.867410+0000","last_active":"2024-09-10T21:24:52.884547+0000","last_peered":"2024-09-10T21:24:52.884547+0000","last_clean":"2024-09-10T21:24:52.884547+0000","last_became_active":"2024-09-10T21:24:47.867113+0000","last_became_peered":"2024-09-10T21:24:47.867113+0000","last_unstale":"2024-09-10T21:24:52.884547+0000","last_undegraded":"2024-09-10T21:24:52.884547+0000","last_fullsized":"2024-09-10T21:24:52.884547+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:49:53.502472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.884956+0000","last_change":"2024-09-10T21:24:47.867473+0000","last_active":"2024-09-10T21:24:52.884956+0000","last_peered":"2024-09-10T21:24:52.884956+0000","last_clean":"2024-09-10T21:24:52.884956+0000","last_became_active":"2024-09-10T21:24:47.867126+0000","last_became_peered":"2024-09-10T21:24:47.867126+0000","last_unstale":"2024-09-10T21:24:52.884956+0000","last_undegraded":"2024-09-10T21:24:52.884956+0000","last_fullsized":"2024-09-10T21:24:52.884956+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:54:32.307507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.175341+0000","last_change":"2024-09-10T21:24:47.865450+0000","last_active":"2024-09-10T21:24:53.175341+0000","last_peered":"2024-09-10T21:24:53.175341+0000","last_clean":"2024-09-10T21:24:53.175341+0000","last_became_active":"2024-09-10T21:24:47.865093+0000","last_became_peered":"2024-09-10T21:24:47.865093+0000","last_unstale":"2024-09-10T21:24:53.175341+0000","last_undegraded":"2024-09-10T21:24:53.175341+0000","last_fullsized":"2024-09-10T21:24:53.175341+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:12:18.854783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.884322+0000","last_change":"2024-09-10T21:24:47.868093+0000","last_active":"2024-09-10T21:24:52.884322+0000","last_peered":"2024-09-10T21:24:52.884322+0000","last_clean":"2024-09-10T21:24:52.884322+0000","last_became_active":"2024-09-10T21:24:47.867782+0000","last_became_peered":"2024-09-10T21:24:47.867782+0000","last_unstale":"2024-09-10T21:24:52.884322+0000","last_undegraded":"2024-09-10T21:24:52.884322+0000","last_fullsized":"2024-09-10T21:24:52.884322+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:34:13.053494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.173787+0000","last_change":"2024-09-10T21:24:47.862204+0000","last_active":"2024-09-10T21:24:53.173787+0000","last_peered":"2024-09-10T21:24:53.173787+0000","last_clean":"2024-09-10T21:24:53.173787+0000","last_became_active":"2024-09-10T21:24:47.861434+0000","last_became_peered":"2024-09-10T21:24:47.861434+0000","last_unstale":"2024-09-10T21:24:53.173787+0000","last_undegraded":"2024-09-10T21:24:53.173787+0000","last_fullsized":"2024-09-10T21:24:53.173787+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:50:03.245285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:25:28.798174+0000","last_change":"2024-09-10T21:24:47.869248+0000","last_active":"2024-09-10T21:25:28.798174+0000","last_peered":"2024-09-10T21:25:28.798174+0000","last_clean":"2024-09-10T21:25:28.798174+0000","last_became_active":"2024-09-10T21:24:47.868503+0000","last_became_peered":"2024-09-10T21:24:47.868503+0000","last_unstale":"2024-09-10T21:25:28.798174+0000","last_undegraded":"2024-09-10T21:25:28.798174+0000","last_fullsized":"2024-09-10T21:25:28.798174+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:51:31.417845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174007+0000","last_change":"2024-09-10T21:24:47.864802+0000","last_active":"2024-09-10T21:24:53.174007+0000","last_peered":"2024-09-10T21:24:53.174007+0000","last_clean":"2024-09-10T21:24:53.174007+0000","last_became_active":"2024-09-10T21:24:47.863976+0000","last_became_peered":"2024-09-10T21:24:47.863976+0000","last_unstale":"2024-09-10T21:24:53.174007+0000","last_undegraded":"2024-09-10T21:24:53.174007+0000","last_fullsized":"2024-09-10T21:24:53.174007+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:55:08.544075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:25:28.798522+0000","last_change":"2024-09-10T21:24:47.873364+0000","last_active":"2024-09-10T21:25:28.798522+0000","last_peered":"2024-09-10T21:25:28.798522+0000","last_clean":"2024-09-10T21:25:28.798522+0000","last_became_active":"2024-09-10T21:24:47.873069+0000","last_became_peered":"2024-09-10T21:24:47.873069+0000","last_unstale":"2024-09-10T21:25:28.798522+0000","last_undegraded":"2024-09-10T21:25:28.798522+0000","last_fullsized":"2024-09-10T21:25:28.798522+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:38:21.368499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.174196+0000","last_change":"2024-09-10T21:24:47.856936+0000","last_active":"2024-09-10T21:24:53.174196+0000","last_peered":"2024-09-10T21:24:53.174196+0000","last_clean":"2024-09-10T21:24:53.174196+0000","last_became_active":"2024-09-10T21:24:47.854713+0000","last_became_peered":"2024-09-10T21:24:47.854713+0000","last_unstale":"2024-09-10T21:24:53.174196+0000","last_undegraded":"2024-09-10T21:24:53.174196+0000","last_fullsized":"2024-09-10T21:24:53.174196+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:46:36.918539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:52.878924+0000","last_change":"2024-09-10T21:24:47.846492+0000","last_active":"2024-09-10T21:24:52.878924+0000","last_peered":"2024-09-10T21:24:52.878924+0000","last_clean":"2024-09-10T21:24:52.878924+0000","last_became_active":"2024-09-10T21:24:47.846227+0000","last_became_peered":"2024-09-10T21:24:47.846227+0000","last_unstale":"2024-09-10T21:24:52.878924+0000","last_undegraded":"2024-09-10T21:24:52.878924+0000","last_fullsized":"2024-09-10T21:24:52.878924+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:13:58.432819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":21,"reported_epoch":25,"state":"active+clean","last_fresh":"2024-09-10T21:24:53.175219+0000","last_change":"2024-09-10T21:24:47.871469+0000","last_active":"2024-09-10T21:24:53.175219+0000","last_peered":"2024-09-10T21:24:53.175219+0000","last_clean":"2024-09-10T21:24:53.175219+0000","last_became_active":"2024-09-10T21:24:47.871177+0000","last_became_peered":"2024-09-10T21:24:47.871177+0000","last_unstale":"2024-09-10T21:24:53.175219+0000","last_undegraded":"2024-09-10T21:24:53.175219+0000","last_fullsized":"2024-09-10T21:24:53.175219+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:19:24.226916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":66932,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509452,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":320,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":327680,"data_stored":693447,"data_compressed":33466,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509453,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":270336,"data_stored":106339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509452,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":258048,"data_stored":102497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509453,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31456,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340384,"statfs":{"total":96636764160,"available":96604553216,"internally_reserved":0,"allocated":266240,"data_stored":102657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542157,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27296,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":270336,"data_stored":102623,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542156,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":264,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":270336,"data_stored":106339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542159,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27364,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344476,"statfs":{"total":96636764160,"available":96608743424,"internally_reserved":0,"allocated":335872,"data_stored":693503,"data_compressed":33466,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574861,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27296,"kb_used_data":268,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":274432,"data_stored":109725,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33466,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33466,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T21:25:31.239 INFO:tasks.mon_thrash.ceph_manager:clean! 2024-09-10T21:25:31.239 DEBUG:tasks.mon_thrash:Yielding 2024-09-10T21:25:31.239 INFO:teuthology.run_tasks:Running task workunit... 2024-09-10T21:25:31.255 INFO:tasks.workunit:Pulling workunits from ref 8f219c476cb4f26fc5719c661c0323719a45cb30 2024-09-10T21:25:31.255 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-09-10T21:25:31.255 INFO:tasks.workunit:timeout=3h 2024-09-10T21:25:31.255 INFO:tasks.workunit:cleanup=True 2024-09-10T21:25:31.255 DEBUG:teuthology.orchestra.run.smithi017:> stat -- /home/ubuntu/cephtest/mnt.0 2024-09-10T21:25:31.256 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:25:31.285 INFO:teuthology.orchestra.run.smithi017.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-09-10T21:25:31.285 INFO:teuthology.orchestra.run.smithi017.stdout: Size: 0 Blocks: 1 IO Block: 4194304 directory 2024-09-10T21:25:31.285 INFO:teuthology.orchestra.run.smithi017.stdout:Device: 2ch/44d Inode: 1 Links: 2 2024-09-10T21:25:31.285 INFO:teuthology.orchestra.run.smithi017.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T21:25:31.285 INFO:teuthology.orchestra.run.smithi017.stdout:Access: 1970-01-01 00:00:00.000000000 +0000 2024-09-10T21:25:31.285 INFO:teuthology.orchestra.run.smithi017.stdout:Modify: 2024-09-10 21:24:52.860504725 +0000 2024-09-10T21:25:31.286 INFO:teuthology.orchestra.run.smithi017.stdout:Change: 2024-09-10 21:25:05.418930818 +0000 2024-09-10T21:25:31.286 INFO:teuthology.orchestra.run.smithi017.stdout: Birth: - 2024-09-10T21:25:31.286 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-09-10T21:25:31.286 DEBUG:teuthology.orchestra.run.smithi017:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-09-10T21:25:31.359 DEBUG:teuthology.orchestra.run.smithi017:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://github.com/ceph/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 8f219c476cb4f26fc5719c661c0323719a45cb30 2024-09-10T21:25:31.414 INFO:tasks.workunit.client.0.smithi017.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-09-10T21:25:31.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:31.422+0000 7ff2f4600640 1 -- 172.21.15.17:0/588119752 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff2d4004d50 msgr2=0x7ff2d40051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:31.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:31.422+0000 7ff2f4600640 1 --2- 172.21.15.17:0/588119752 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff2d4004d50 0x7ff2d40051d0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ff2e400c610 tx=0x7ff2e401d2a0 comp rx=0 tx=0).stop 2024-09-10T21:25:31.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:31.422+0000 7ff2f2600640 1 -- 172.21.15.17:0/588119752 reap_dead start 2024-09-10T21:25:31.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:31.422+0000 7ff2f4600640 1 -- 172.21.15.17:0/588119752 shutdown_connections 2024-09-10T21:25:31.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:31.422+0000 7ff2f4600640 1 -- 172.21.15.17:0/588119752 >> 172.21.15.17:0/588119752 conn(0x7ff2ec06cc20 msgr2=0x7ff2ec06d030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:31.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:31.422+0000 7ff2f4600640 1 -- 172.21.15.17:0/588119752 shutdown_connections 2024-09-10T21:25:31.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:31.422+0000 7ff2f4600640 1 -- 172.21.15.17:0/588119752 wait complete. 2024-09-10T21:25:31.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:31.426+0000 7ff2f4600640 1 Processor -- start 2024-09-10T21:25:31.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:31.426+0000 7ff2f4600640 1 -- start start 2024-09-10T21:25:31.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:31.426+0000 7ff2f4600640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff2d4004d50 0x7ff2ec155e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:31.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:31.426+0000 7ff2f4600640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff2ec0cbb10 0x7ff2ec1f3e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:31.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:31.426+0000 7ff2f4600640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff2ec146020 0x7ff2ec0c27e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:31.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:31.426+0000 7ff2f4600640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7ff2ec0c8640 con 0x7ff2d4004d50 2024-09-10T21:25:31.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:31.426+0000 7ff2f4600640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7ff2ec158210 con 0x7ff2ec146020 2024-09-10T21:25:31.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:31.426+0000 7ff2f4600640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7ff2ec0c87c0 con 0x7ff2ec0cbb10 2024-09-10T21:25:31.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:31.426+0000 7ff2f1c00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff2d4004d50 0x7ff2ec155e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:31.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:31.426+0000 7ff2f1c00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff2d4004d50 0x7ff2ec155e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43760/0 (socket says 172.21.15.17:43760) 2024-09-10T21:25:31.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:31.426+0000 7ff2f1c00640 1 -- 172.21.15.17:0/3027921412 learned_addr learned my addr 172.21.15.17:0/3027921412 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:25:31.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:31.426+0000 7ff2f1200640 1 --2- 172.21.15.17:0/3027921412 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff2ec0cbb10 0x7ff2ec1f3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:31.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:31.426+0000 7ff2f2600640 1 --2- 172.21.15.17:0/3027921412 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff2ec146020 0x7ff2ec0c27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:31.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:31.426+0000 7ff2f1c00640 1 -- 172.21.15.17:0/3027921412 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff2ec0cbb10 msgr2=0x7ff2ec1f3e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:31.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:31.426+0000 7ff2f1c00640 1 --2- 172.21.15.17:0/3027921412 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff2ec0cbb10 0x7ff2ec1f3e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:31.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:31.426+0000 7ff2f1c00640 1 -- 172.21.15.17:0/3027921412 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff2ec146020 msgr2=0x7ff2ec0c27e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:31.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:31.426+0000 7ff2f1c00640 1 --2- 172.21.15.17:0/3027921412 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff2ec146020 0x7ff2ec0c27e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:31.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:31.426+0000 7ff2f1c00640 1 -- 172.21.15.17:0/3027921412 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2ec156eb0 con 0x7ff2d4004d50 2024-09-10T21:25:31.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:31.426+0000 7ff2f2600640 1 --2- 172.21.15.17:0/3027921412 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff2ec146020 0x7ff2ec0c27e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:25:31.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:31.426+0000 7ff2f1c00640 1 --2- 172.21.15.17:0/3027921412 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff2d4004d50 0x7ff2ec155e00 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7ff2e0011b10 tx=0x7ff2e0009ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:31.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:31.426+0000 7ff2e9600640 1 -- 172.21.15.17:0/3027921412 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2e001e070 con 0x7ff2d4004d50 2024-09-10T21:25:31.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:31.426+0000 7ff2e9600640 1 -- 172.21.15.17:0/3027921412 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff2e000fbe0 con 0x7ff2d4004d50 2024-09-10T21:25:31.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:31.426+0000 7ff2e9600640 1 -- 172.21.15.17:0/3027921412 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2e0019410 con 0x7ff2d4004d50 2024-09-10T21:25:31.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:31.426+0000 7ff2f4600640 1 -- 172.21.15.17:0/3027921412 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2ec145a30 con 0x7ff2d4004d50 2024-09-10T21:25:31.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:31.426+0000 7ff2f4600640 1 -- 172.21.15.17:0/3027921412 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2ec155600 con 0x7ff2d4004d50 2024-09-10T21:25:31.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:31.430+0000 7ff2f4600640 1 -- 172.21.15.17:0/3027921412 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2ec072e40 con 0x7ff2d4004d50 2024-09-10T21:25:31.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:31.430+0000 7ff2e9600640 1 -- 172.21.15.17:0/3027921412 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7ff2e00195d0 con 0x7ff2d4004d50 2024-09-10T21:25:31.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:31.430+0000 7ff2e9600640 1 --2- 172.21.15.17:0/3027921412 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7ff2cc072610 0x7ff2cc074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:31.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:31.430+0000 7ff2e9600640 1 -- 172.21.15.17:0/3027921412 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7ff2e0094060 con 0x7ff2d4004d50 2024-09-10T21:25:31.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:31.430+0000 7ff2f1200640 1 --2- 172.21.15.17:0/3027921412 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7ff2cc072610 0x7ff2cc074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:31.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:31.434+0000 7ff2f1200640 1 --2- 172.21.15.17:0/3027921412 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7ff2cc072610 0x7ff2cc074ad0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ff2ec074790 tx=0x7ff2d80091e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:31.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:31.434+0000 7ff2e9600640 1 -- 172.21.15.17:0/3027921412 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7ff2e005ece0 con 0x7ff2d4004d50 2024-09-10T21:25:31.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:31.662+0000 7ff2f4600640 1 -- 172.21.15.17:0/3027921412 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff2ec14ebe0 con 0x7ff2d4004d50 2024-09-10T21:25:31.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:31.662+0000 7ff2e9600640 1 -- 172.21.15.17:0/3027921412 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+6870 (secure 0 0 0) 0x7ff2e0063710 con 0x7ff2d4004d50 2024-09-10T21:25:31.667 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-10T21:25:31.667 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":10,"btime":"2024-09-10T21:25:16:432823+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4350,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/1643071167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":1643071167},{"type":"v1","addr":"172.21.15.31:6835","nonce":1643071167}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4356,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6837/1666225712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6836","nonce":1666225712},{"type":"v1","addr":"172.21.15.17:6837","nonce":1666225712}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4384,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/3592734043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":3592734043},{"type":"v1","addr":"172.21.15.17:6833","nonce":3592734043}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4405,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/1293665856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":1293665856},{"type":"v1","addr":"172.21.15.17:6835","nonce":1293665856}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4411,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6838/1944852172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":1944852172},{"type":"v1","addr":"172.21.15.31:6838","nonce":1944852172}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":10,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:24:47.255397+0000","modified":"2024-09-10T21:25:15.961582+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4414},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4414":{"gid":4414,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.31:6839/3129062106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6837","nonce":3129062106},{"type":"v1","addr":"172.21.15.31:6839","nonce":3129062106}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4414,"qdb_cluster":[4414]},"id":1}]} 2024-09-10T21:25:31.667 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 10 2024-09-10T21:25:31.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:31.666+0000 7ff2f4600640 1 -- 172.21.15.17:0/3027921412 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7ff2cc072610 msgr2=0x7ff2cc074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:31.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:31.666+0000 7ff2f4600640 1 --2- 172.21.15.17:0/3027921412 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7ff2cc072610 0x7ff2cc074ad0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ff2ec074790 tx=0x7ff2d80091e0 comp rx=0 tx=0).stop 2024-09-10T21:25:31.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:31.666+0000 7ff2f4600640 1 -- 172.21.15.17:0/3027921412 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff2d4004d50 msgr2=0x7ff2ec155e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:31.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:31.666+0000 7ff2f4600640 1 --2- 172.21.15.17:0/3027921412 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff2d4004d50 0x7ff2ec155e00 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7ff2e0011b10 tx=0x7ff2e0009ec0 comp rx=0 tx=0).stop 2024-09-10T21:25:31.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:31.666+0000 7ff2f4600640 1 -- 172.21.15.17:0/3027921412 shutdown_connections 2024-09-10T21:25:31.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:31.666+0000 7ff2f4600640 1 --2- 172.21.15.17:0/3027921412 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7ff2cc072610 0x7ff2cc074ad0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:31.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:31.666+0000 7ff2f4600640 1 --2- 172.21.15.17:0/3027921412 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff2ec146020 0x7ff2ec0c27e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:31.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:31.666+0000 7ff2f4600640 1 --2- 172.21.15.17:0/3027921412 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff2ec0cbb10 0x7ff2ec1f3e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:31.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:31.666+0000 7ff2f4600640 1 --2- 172.21.15.17:0/3027921412 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff2d4004d50 0x7ff2ec155e00 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:31.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:31.666+0000 7ff2f4600640 1 -- 172.21.15.17:0/3027921412 >> 172.21.15.17:0/3027921412 conn(0x7ff2ec06cc20 msgr2=0x7ff2ec147c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:31.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:31.666+0000 7ff2f4600640 1 -- 172.21.15.17:0/3027921412 shutdown_connections 2024-09-10T21:25:31.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:31.666+0000 7ff2f4600640 1 -- 172.21.15.17:0/3027921412 wait complete. 2024-09-10T21:25:31.688 INFO:tasks.mon_thrash.mon_thrasher:start thrashing 2024-09-10T21:25:31.688 INFO:tasks.mon_thrash.mon_thrasher:seed: 1726003531, revive delay: 20.0, thrash delay: 10.0 thrash many: False, maintain quorum: True store thrash: False, probability: 50 freeze mon: prob 10.0 duration 15.0 2024-09-10T21:25:31.688 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-09-10T21:25:31.688 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-10T21:25:31.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:31.878+0000 7f9213c00640 1 -- 172.21.15.17:0/1137513436 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f920c150ea0 msgr2=0x7f920c073d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:31.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:31.878+0000 7f9213c00640 1 --2- 172.21.15.17:0/1137513436 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f920c150ea0 0x7f920c073d40 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f920c06db80 tx=0x7f920001f0b0 comp rx=0 tx=0).stop 2024-09-10T21:25:31.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:31.878+0000 7f9213c00640 1 -- 172.21.15.17:0/1137513436 shutdown_connections 2024-09-10T21:25:31.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:31.878+0000 7f9213c00640 1 --2- 172.21.15.17:0/1137513436 >> v2:172.21.15.31:3300/0 conn(0x7f920c0c8470 0x7f920c0c70e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:31.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:31.878+0000 7f9213c00640 1 --2- 172.21.15.17:0/1137513436 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f920c156af0 0x7f920c074280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:31.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:31.878+0000 7f9213c00640 1 --2- 172.21.15.17:0/1137513436 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f920c150ea0 0x7f920c073d40 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:31.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:31.878+0000 7f9213c00640 1 -- 172.21.15.17:0/1137513436 >> 172.21.15.17:0/1137513436 conn(0x7f920c070160 msgr2=0x7f920c070570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:31.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:31.878+0000 7f9213c00640 1 -- 172.21.15.17:0/1137513436 shutdown_connections 2024-09-10T21:25:31.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:31.878+0000 7f9213c00640 1 -- 172.21.15.17:0/1137513436 wait complete. 2024-09-10T21:25:31.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:31.878+0000 7f9213c00640 1 Processor -- start 2024-09-10T21:25:31.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:31.878+0000 7f9213c00640 1 -- start start 2024-09-10T21:25:31.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:31.878+0000 7f9213c00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f920c0c8470 0x7f920c0cb5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:31.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:31.878+0000 7f9213c00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f920c150ea0 0x7f920c154020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:31.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:31.878+0000 7f9213c00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f920c143bb0 0x7f920c0ca700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:31.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:31.878+0000 7f9213c00640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f920c06ee30 con 0x7f920c0c8470 2024-09-10T21:25:31.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:31.878+0000 7f9213c00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f920c157800 con 0x7f920c143bb0 2024-09-10T21:25:31.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:31.878+0000 7f9213c00640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f920c06efb0 con 0x7f920c150ea0 2024-09-10T21:25:31.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:31.878+0000 7f9211200640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f920c0c8470 0x7f920c0cb5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:31.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:31.878+0000 7f9211200640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f920c0c8470 0x7f920c0cb5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43772/0 (socket says 172.21.15.17:43772) 2024-09-10T21:25:31.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:31.878+0000 7f9211200640 1 -- 172.21.15.17:0/1190466129 learned_addr learned my addr 172.21.15.17:0/1190466129 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:25:31.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:31.878+0000 7f9211c00640 1 --2- 172.21.15.17:0/1190466129 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f920c143bb0 0x7f920c0ca700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:31.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:31.878+0000 7f920be00640 1 --2- 172.21.15.17:0/1190466129 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f920c150ea0 0x7f920c154020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:31.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:31.878+0000 7f9211200640 1 -- 172.21.15.17:0/1190466129 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f920c150ea0 msgr2=0x7f920c154020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:31.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:31.878+0000 7f9211200640 1 --2- 172.21.15.17:0/1190466129 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f920c150ea0 0x7f920c154020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:31.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:31.878+0000 7f9211200640 1 -- 172.21.15.17:0/1190466129 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f920c143bb0 msgr2=0x7f920c0ca700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:31.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:31.878+0000 7f9211200640 1 --2- 172.21.15.17:0/1190466129 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f920c143bb0 0x7f920c0ca700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:31.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:31.878+0000 7f9211200640 1 -- 172.21.15.17:0/1190466129 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f920c152710 con 0x7f920c0c8470 2024-09-10T21:25:31.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:31.878+0000 7f9211200640 1 --2- 172.21.15.17:0/1190466129 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f920c0c8470 0x7f920c0cb5c0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f92000b3040 tx=0x7f9200005d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:31.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:31.882+0000 7f9208c00640 1 -- 172.21.15.17:0/1190466129 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92000b5040 con 0x7f920c0c8470 2024-09-10T21:25:31.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:31.882+0000 7f9208c00640 1 -- 172.21.15.17:0/1190466129 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9200005340 con 0x7f920c0c8470 2024-09-10T21:25:31.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:31.882+0000 7f9208c00640 1 -- 172.21.15.17:0/1190466129 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92000046d0 con 0x7f920c0c8470 2024-09-10T21:25:31.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:31.882+0000 7f9213c00640 1 -- 172.21.15.17:0/1190466129 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f920c14cd50 con 0x7f920c0c8470 2024-09-10T21:25:31.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:31.882+0000 7f9213c00640 1 -- 172.21.15.17:0/1190466129 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f920c153820 con 0x7f920c0c8470 2024-09-10T21:25:31.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:31.882+0000 7f9213c00640 1 -- 172.21.15.17:0/1190466129 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f920c0c8250 con 0x7f920c0c8470 2024-09-10T21:25:31.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:31.882+0000 7f9208c00640 1 -- 172.21.15.17:0/1190466129 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f92000b8070 con 0x7f920c0c8470 2024-09-10T21:25:31.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:31.882+0000 7f9208c00640 1 --2- 172.21.15.17:0/1190466129 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f91ec0725d0 0x7f91ec074a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:31.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:31.882+0000 7f9208c00640 1 -- 172.21.15.17:0/1190466129 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f9200130490 con 0x7f920c0c8470 2024-09-10T21:25:31.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:31.882+0000 7f920be00640 1 --2- 172.21.15.17:0/1190466129 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f91ec0725d0 0x7f91ec074a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:31.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:31.886+0000 7f920be00640 1 --2- 172.21.15.17:0/1190466129 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f91ec0725d0 0x7f91ec074a90 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f920c074640 tx=0x7f91f80091e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:31.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:31.886+0000 7f9208c00640 1 -- 172.21.15.17:0/1190466129 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f9200018c10 con 0x7f920c0c8470 2024-09-10T21:25:32.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.130+0000 7f9213c00640 1 -- 172.21.15.17:0/1190466129 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f920c06e480 con 0x7f920c0c8470 2024-09-10T21:25:32.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.130+0000 7f9208c00640 1 -- 172.21.15.17:0/1190466129 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1556 (secure 0 0 0) 0x7f92000fb110 con 0x7f920c0c8470 2024-09-10T21:25:32.135 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":70,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5e2f023f-d6a6-49f4-a256-f4807dce3739","modified":"2024-09-10T21:23:46.892130Z","created":"2024-09-10T21:23:46.892130Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3301","nonce":0},{"type":"v1","addr":"172.21.15.31:6790","nonce":0}]},"addr":"172.21.15.31:6790/0","public_addr":"172.21.15.31:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-10T21:25:32.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.130+0000 7f9213c00640 1 -- 172.21.15.17:0/1190466129 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f91ec0725d0 msgr2=0x7f91ec074a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:32.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.130+0000 7f9213c00640 1 --2- 172.21.15.17:0/1190466129 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f91ec0725d0 0x7f91ec074a90 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f920c074640 tx=0x7f91f80091e0 comp rx=0 tx=0).stop 2024-09-10T21:25:32.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.134+0000 7f9213c00640 1 -- 172.21.15.17:0/1190466129 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f920c0c8470 msgr2=0x7f920c0cb5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:32.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.134+0000 7f9213c00640 1 --2- 172.21.15.17:0/1190466129 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f920c0c8470 0x7f920c0cb5c0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f92000b3040 tx=0x7f9200005d60 comp rx=0 tx=0).stop 2024-09-10T21:25:32.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.134+0000 7f9213c00640 1 -- 172.21.15.17:0/1190466129 shutdown_connections 2024-09-10T21:25:32.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.134+0000 7f9213c00640 1 --2- 172.21.15.17:0/1190466129 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f91ec0725d0 0x7f91ec074a90 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:32.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.134+0000 7f9213c00640 1 --2- 172.21.15.17:0/1190466129 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f920c143bb0 0x7f920c0ca700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:32.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.134+0000 7f9213c00640 1 --2- 172.21.15.17:0/1190466129 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f920c150ea0 0x7f920c154020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:32.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.134+0000 7f9213c00640 1 --2- 172.21.15.17:0/1190466129 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f920c0c8470 0x7f920c0cb5c0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:32.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.134+0000 7f9213c00640 1 -- 172.21.15.17:0/1190466129 >> 172.21.15.17:0/1190466129 conn(0x7f920c070160 msgr2=0x7f920c14f460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:32.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.134+0000 7f9213c00640 1 -- 172.21.15.17:0/1190466129 shutdown_connections 2024-09-10T21:25:32.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.134+0000 7f9213c00640 1 -- 172.21.15.17:0/1190466129 wait complete. 2024-09-10T21:25:32.156 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-09-10T21:25:32.156 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-09-10T21:25:32.156 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-09-10T21:25:32.156 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-10T21:25:32.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.302+0000 7f506d600640 1 -- 172.21.15.17:0/1911480915 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f504c005220 msgr2=0x7f504c00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:32.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.302+0000 7f506d600640 1 --2- 172.21.15.17:0/1911480915 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f504c005220 0x7f504c00b9d0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f506000c610 tx=0x7f506001d220 comp rx=0 tx=0).stop 2024-09-10T21:25:32.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.302+0000 7f506d600640 1 -- 172.21.15.17:0/1911480915 shutdown_connections 2024-09-10T21:25:32.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.302+0000 7f506d600640 1 --2- 172.21.15.17:0/1911480915 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f504c005220 0x7f504c00b9d0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:32.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.302+0000 7f506d600640 1 -- 172.21.15.17:0/1911480915 >> 172.21.15.17:0/1911480915 conn(0x7f5068068990 msgr2=0x7f5068068da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:32.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.302+0000 7f506d600640 1 -- 172.21.15.17:0/1911480915 shutdown_connections 2024-09-10T21:25:32.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.302+0000 7f506d600640 1 -- 172.21.15.17:0/1911480915 wait complete. 2024-09-10T21:25:32.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.302+0000 7f506d600640 1 Processor -- start 2024-09-10T21:25:32.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.302+0000 7f506d600640 1 -- start start 2024-09-10T21:25:32.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.302+0000 7f506d600640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f504c005220 0x7f50680cbf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:32.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.302+0000 7f506d600640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f50681f23e0 0x7f50680cb270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:32.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.302+0000 7f506d600640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f50680c9620 0x7f50680cc9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:32.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.302+0000 7f506d600640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f5068152a90 con 0x7f50680c9620 2024-09-10T21:25:32.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.302+0000 7f506d600640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f50680c8a40 con 0x7f50681f23e0 2024-09-10T21:25:32.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.302+0000 7f506d600640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f5068158f80 con 0x7f504c005220 2024-09-10T21:25:32.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.302+0000 7f5067400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f50680c9620 0x7f50680cc9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:32.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.302+0000 7f5067400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f50680c9620 0x7f50680cc9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43790/0 (socket says 172.21.15.17:43790) 2024-09-10T21:25:32.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.302+0000 7f5067400640 1 -- 172.21.15.17:0/2393585250 learned_addr learned my addr 172.21.15.17:0/2393585250 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:25:32.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.302+0000 7f505e000640 1 --2- 172.21.15.17:0/2393585250 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f50681f23e0 0x7f50680cb270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:32.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.302+0000 7f5066a00640 1 --2- 172.21.15.17:0/2393585250 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f504c005220 0x7f50680cbf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:32.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.302+0000 7f5067400640 1 -- 172.21.15.17:0/2393585250 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f504c005220 msgr2=0x7f50680cbf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:32.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.302+0000 7f5067400640 1 --2- 172.21.15.17:0/2393585250 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f504c005220 0x7f50680cbf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:32.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.302+0000 7f5067400640 1 -- 172.21.15.17:0/2393585250 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f50681f23e0 msgr2=0x7f50680cb270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:32.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.302+0000 7f5067400640 1 --2- 172.21.15.17:0/2393585250 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f50681f23e0 0x7f50680cb270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:32.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.302+0000 7f5067400640 1 -- 172.21.15.17:0/2393585250 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50680cd8a0 con 0x7f50680c9620 2024-09-10T21:25:32.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.302+0000 7f5067400640 1 --2- 172.21.15.17:0/2393585250 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f50680c9620 0x7f50680cc9b0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f506001b920 tx=0x7f5060006e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:32.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.306+0000 7f505f400640 1 -- 172.21.15.17:0/2393585250 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5060001a80 con 0x7f50680c9620 2024-09-10T21:25:32.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.306+0000 7f505f400640 1 -- 172.21.15.17:0/2393585250 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5060003a70 con 0x7f50680c9620 2024-09-10T21:25:32.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.306+0000 7f505f400640 1 -- 172.21.15.17:0/2393585250 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5060006070 con 0x7f50680c9620 2024-09-10T21:25:32.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.306+0000 7f506d600640 1 -- 172.21.15.17:0/2393585250 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50680cdb80 con 0x7f50680c9620 2024-09-10T21:25:32.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.306+0000 7f506d600640 1 -- 172.21.15.17:0/2393585250 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50681f1be0 con 0x7f50680c9620 2024-09-10T21:25:32.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.306+0000 7f506d600640 1 --2- 172.21.15.17:0/2393585250 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f50680ccef0 0x7f50681ee100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:32.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.306+0000 7f506d600640 1 -- 172.21.15.17:0/2393585250 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f50680cd2d0 con 0x7f50680ccef0 2024-09-10T21:25:32.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.306+0000 7f5066a00640 1 --2- 172.21.15.17:0/2393585250 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f50680ccef0 0x7f50681ee100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:32.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.306+0000 7f5066a00640 1 -- 172.21.15.17:0/2393585250 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5068159850 con 0x7f50680ccef0 2024-09-10T21:25:32.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.306+0000 7f5066a00640 1 --2- 172.21.15.17:0/2393585250 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f50680ccef0 0x7f50681ee100 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f5058013800 tx=0x7f5058013cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:32.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.306+0000 7f505f400640 1 -- 172.21.15.17:0/2393585250 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50580119d0 con 0x7f50680ccef0 2024-09-10T21:25:32.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.306+0000 7f505f400640 1 -- 172.21.15.17:0/2393585250 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f5058011b70 con 0x7f50680ccef0 2024-09-10T21:25:32.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.306+0000 7f505f400640 1 -- 172.21.15.17:0/2393585250 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f50680ccef0 msgr2=0x7f50681ee100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:32.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.306+0000 7f505f400640 1 --2- 172.21.15.17:0/2393585250 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f50680ccef0 0x7f50681ee100 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f5058013800 tx=0x7f5058013cc0 comp rx=0 tx=0).stop 2024-09-10T21:25:32.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.306+0000 7f505f400640 1 -- 172.21.15.17:0/2393585250 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f50680ccef0 msgr2=0x7f50681ee100 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:25:32.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.306+0000 7f505f400640 1 --2- 172.21.15.17:0/2393585250 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f50680ccef0 0x7f50681ee100 secure :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f5058013800 tx=0x7f5058013cc0 comp rx=0 tx=0).stop 2024-09-10T21:25:32.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.306+0000 7f505f400640 1 -- 172.21.15.17:0/2393585250 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f5060006210 con 0x7f50680c9620 2024-09-10T21:25:32.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.306+0000 7f505f400640 1 --2- 172.21.15.17:0/2393585250 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f50440720f0 0x7f50440745b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:32.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.306+0000 7f505f400640 1 -- 172.21.15.17:0/2393585250 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f506012d620 con 0x7f50680c9620 2024-09-10T21:25:32.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.306+0000 7f5066a00640 1 --2- 172.21.15.17:0/2393585250 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f50440720f0 0x7f50440745b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:32.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.310+0000 7f5066a00640 1 --2- 172.21.15.17:0/2393585250 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f50440720f0 0x7f50440745b0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f5058013d00 tx=0x7f5058014000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:32.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.314+0000 7f506d600640 1 --2- 172.21.15.17:0/2393585250 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f50680ce860 0x7f50681ee640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:32.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.314+0000 7f506d600640 1 -- 172.21.15.17:0/2393585250 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f50680cc440 con 0x7f50680ce860 2024-09-10T21:25:32.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.314+0000 7f505e000640 1 --2- 172.21.15.17:0/2393585250 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f50680ce860 0x7f50681ee640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:32.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.314+0000 7f505e000640 1 -- 172.21.15.17:0/2393585250 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f506814fc80 con 0x7f50680ce860 2024-09-10T21:25:32.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.314+0000 7f505e000640 1 --2- 172.21.15.17:0/2393585250 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f50680ce860 0x7f50681ee640 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f50680b3790 tx=0x7f5050001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:32.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.314+0000 7f505f400640 1 -- 172.21.15.17:0/2393585250 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5050016070 con 0x7f50680ce860 2024-09-10T21:25:32.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.314+0000 7f505f400640 1 -- 172.21.15.17:0/2393585250 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4876 (secure 0 0 0) 0x7f5050012850 con 0x7f50680ce860 2024-09-10T21:25:32.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.314+0000 7f505f400640 1 -- 172.21.15.17:0/2393585250 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f50680ce860 msgr2=0x7f50681ee640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:32.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.314+0000 7f505f400640 1 --2- 172.21.15.17:0/2393585250 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f50680ce860 0x7f50681ee640 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f50680b3790 tx=0x7f5050001640 comp rx=0 tx=0).stop 2024-09-10T21:25:32.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.314+0000 7f505f400640 1 -- 172.21.15.17:0/2393585250 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f50680ce860 msgr2=0x7f50681ee640 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:25:32.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.314+0000 7f505f400640 1 --2- 172.21.15.17:0/2393585250 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f50680ce860 0x7f50681ee640 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:32.319 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-10T21:25:32.319 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:25:32.319 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:25:32.319 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "leader", 2024-09-10T21:25:32.319 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 4, 2024-09-10T21:25:32.319 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 70799, 2024-09-10T21:25:32.320 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-10T21:25:32.320 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-10T21:25:32.320 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-10T21:25:32.320 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-10T21:25:32.320 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:25:32.320 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 70, 2024-09-10T21:25:32.320 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:25:32.320 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-10T21:25:32.320 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-10T21:25:32.320 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:25:32.320 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:25:32.320 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:25:32.320 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:25:32.321 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:25:32.321 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:25:32.321 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:25:32.321 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:25:32.321 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:25:32.321 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:25:32.321 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:25:32.321 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:25:32.321 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-10T21:25:32.321 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-10T21:25:32.321 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:25:32.321 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:25:32.321 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:25:32.322 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:25:32.322 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:25:32.322 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:25:32.322 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:25:32.322 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:25:32.322 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:25:32.322 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:25:32.322 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:25:32.322 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:25:32.322 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:25:32.322 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-10T21:25:32.322 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-10T21:25:32.322 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-10T21:25:32.323 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-10T21:25:32.323 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-10T21:25:32.323 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "5e2f023f-d6a6-49f4-a256-f4807dce3739", 2024-09-10T21:25:32.323 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:25:32.323 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:25:32.323 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-10T21:25:32.323 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-10T21:25:32.323 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-10T21:25:32.323 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-10T21:25:32.323 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-10T21:25:32.323 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-10T21:25:32.323 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-10T21:25:32.323 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:25:32.323 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-10T21:25:32.324 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:25:32.324 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:25:32.324 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:25:32.324 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:25:32.324 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:25:32.324 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:25:32.324 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:25:32.324 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:25:32.324 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:25:32.324 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:25:32.324 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:25:32.324 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:25:32.324 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-10T21:25:32.325 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:25:32.325 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-10T21:25:32.325 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:25:32.325 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:25:32.325 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:25:32.325 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:25:32.325 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:25:32.325 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:25:32.325 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:25:32.325 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-10T21:25:32.325 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:25:32.325 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:25:32.325 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:25:32.325 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:25:32.326 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-10T21:25:32.326 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:25:32.326 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:25:32.326 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:25:32.326 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:25:32.326 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-10T21:25:32.326 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-10T21:25:32.326 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:25:32.326 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:25:32.326 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:25:32.326 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:25:32.326 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:25:32.326 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:25:32.326 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:25:32.327 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:25:32.327 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:25:32.327 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:25:32.327 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:25:32.327 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3300", 2024-09-10T21:25:32.327 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:25:32.327 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:25:32.327 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:25:32.327 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:25:32.327 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789", 2024-09-10T21:25:32.327 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:25:32.327 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:25:32.327 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:25:32.327 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:25:32.328 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789/0", 2024-09-10T21:25:32.328 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6789/0", 2024-09-10T21:25:32.328 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:25:32.328 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:25:32.328 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:25:32.328 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:25:32.328 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:25:32.328 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:25:32.328 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:25:32.328 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:25:32.328 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:25:32.328 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:25:32.328 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:25:32.329 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3301", 2024-09-10T21:25:32.329 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:25:32.329 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:25:32.329 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:25:32.329 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:25:32.329 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790", 2024-09-10T21:25:32.329 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:25:32.329 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:25:32.329 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:25:32.329 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:25:32.329 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790/0", 2024-09-10T21:25:32.329 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6790/0", 2024-09-10T21:25:32.329 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:25:32.329 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:25:32.330 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:25:32.330 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:25:32.330 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:25:32.330 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:25:32.330 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-10T21:25:32.330 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-10T21:25:32.330 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:25:32.330 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:25:32.330 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:25:32.330 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:25:32.330 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:25:32.330 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:25:32.330 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": [ 2024-09-10T21:25:32.330 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:25:32.331 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:25:32.331 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:25:32.331 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 2 2024-09-10T21:25:32.331 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:25:32.331 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:25:32.331 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-10T21:25:32.331 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:25:32.331 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:25:32.331 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:25:32.331 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 3 2024-09-10T21:25:32.331 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:25:32.331 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:25:32.332 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-10T21:25:32.332 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:25:32.332 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:25:32.332 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:25:32.332 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 3 2024-09-10T21:25:32.332 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:25:32.332 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:25:32.332 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:25:32.332 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-10T21:25:32.332 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-10T21:25:32.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.314+0000 7f506d600640 1 -- 172.21.15.17:0/2393585250 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f50440720f0 msgr2=0x7f50440745b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:32.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.314+0000 7f506d600640 1 --2- 172.21.15.17:0/2393585250 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f50440720f0 0x7f50440745b0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f5058013d00 tx=0x7f5058014000 comp rx=0 tx=0).stop 2024-09-10T21:25:32.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.314+0000 7f506d600640 1 -- 172.21.15.17:0/2393585250 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f50680c9620 msgr2=0x7f50680cc9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:32.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.314+0000 7f506d600640 1 --2- 172.21.15.17:0/2393585250 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f50680c9620 0x7f50680cc9b0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f506001b920 tx=0x7f5060006e60 comp rx=0 tx=0).stop 2024-09-10T21:25:32.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.314+0000 7f5067400640 1 -- 172.21.15.17:0/2393585250 reap_dead start 2024-09-10T21:25:32.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.314+0000 7f5067400640 1 -- 172.21.15.17:0/2393585250 reap_dead start 2024-09-10T21:25:32.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.314+0000 7f506d600640 1 -- 172.21.15.17:0/2393585250 shutdown_connections 2024-09-10T21:25:32.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.314+0000 7f506d600640 1 -- 172.21.15.17:0/2393585250 >> 172.21.15.17:0/2393585250 conn(0x7f5068068990 msgr2=0x7f50681508b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:32.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.314+0000 7f506d600640 1 -- 172.21.15.17:0/2393585250 shutdown_connections 2024-09-10T21:25:32.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.314+0000 7f506d600640 1 -- 172.21.15.17:0/2393585250 wait complete. 2024-09-10T21:25:32.338 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-10T21:25:32.338 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-09-10T21:25:32.338 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-10T21:25:32.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.482+0000 7f4754000640 1 -- 172.21.15.17:0/2700779987 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f474c071760 msgr2=0x7f474c159850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:32.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.482+0000 7f4754000640 1 --2- 172.21.15.17:0/2700779987 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f474c071760 0x7f474c159850 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f474c0c3230 tx=0x7f473801f090 comp rx=0 tx=0).stop 2024-09-10T21:25:32.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.482+0000 7f4754000640 1 -- 172.21.15.17:0/2700779987 shutdown_connections 2024-09-10T21:25:32.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.482+0000 7f4754000640 1 --2- 172.21.15.17:0/2700779987 >> v2:172.21.15.31:3300/0 conn(0x7f474c0c4dd0 0x7f474c0ce130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:32.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.482+0000 7f4754000640 1 --2- 172.21.15.17:0/2700779987 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f474c071760 0x7f474c159850 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:32.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.482+0000 7f4754000640 1 -- 172.21.15.17:0/2700779987 >> 172.21.15.17:0/2700779987 conn(0x7f474c068990 msgr2=0x7f474c068da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:32.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.482+0000 7f4754000640 1 -- 172.21.15.17:0/2700779987 shutdown_connections 2024-09-10T21:25:32.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.482+0000 7f4754000640 1 -- 172.21.15.17:0/2700779987 wait complete. 2024-09-10T21:25:32.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.482+0000 7f4754000640 1 Processor -- start 2024-09-10T21:25:32.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.482+0000 7f4754000640 1 -- start start 2024-09-10T21:25:32.488 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.482+0000 7f4754000640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f474c071760 0x7f474c0cadc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:32.488 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.482+0000 7f4754000640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f474c0c4dd0 0x7f474c1e2d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:32.488 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.482+0000 7f4754000640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f474c1526b0 0x7f474c1e4220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:32.488 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.482+0000 7f4754000640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f474c152a90 con 0x7f474c071760 2024-09-10T21:25:32.488 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.482+0000 7f4754000640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f474c0c8a40 con 0x7f474c1526b0 2024-09-10T21:25:32.488 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.482+0000 7f4754000640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f474c0c8c90 con 0x7f474c0c4dd0 2024-09-10T21:25:32.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.482+0000 7f4751600640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f474c071760 0x7f474c0cadc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:32.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.482+0000 7f4751600640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f474c071760 0x7f474c0cadc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43820/0 (socket says 172.21.15.17:43820) 2024-09-10T21:25:32.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.482+0000 7f4751600640 1 -- 172.21.15.17:0/2074029832 learned_addr learned my addr 172.21.15.17:0/2074029832 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:25:32.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.482+0000 7f4750c00640 1 --2- 172.21.15.17:0/2074029832 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f474c0c4dd0 0x7f474c1e2d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:32.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.482+0000 7f4752000640 1 --2- 172.21.15.17:0/2074029832 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f474c1526b0 0x7f474c1e4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:32.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.482+0000 7f4751600640 1 -- 172.21.15.17:0/2074029832 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f474c0c4dd0 msgr2=0x7f474c1e2d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:32.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.482+0000 7f4751600640 1 --2- 172.21.15.17:0/2074029832 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f474c0c4dd0 0x7f474c1e2d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:32.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.482+0000 7f4751600640 1 -- 172.21.15.17:0/2074029832 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f474c1526b0 msgr2=0x7f474c1e4220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:32.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.482+0000 7f4751600640 1 --2- 172.21.15.17:0/2074029832 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f474c1526b0 0x7f474c1e4220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:32.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.482+0000 7f4751600640 1 -- 172.21.15.17:0/2074029832 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f474c0cb300 con 0x7f474c071760 2024-09-10T21:25:32.490 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.482+0000 7f4750c00640 1 --2- 172.21.15.17:0/2074029832 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f474c0c4dd0 0x7f474c1e2d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:25:32.490 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.486+0000 7f4751600640 1 --2- 172.21.15.17:0/2074029832 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f474c071760 0x7f474c0cadc0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f474000c3c0 tx=0x7f474000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:32.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.486+0000 7f4749600640 1 -- 172.21.15.17:0/2074029832 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4740017070 con 0x7f474c071760 2024-09-10T21:25:32.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.486+0000 7f4749600640 1 -- 172.21.15.17:0/2074029832 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f47400053c0 con 0x7f474c071760 2024-09-10T21:25:32.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.486+0000 7f4754000640 1 -- 172.21.15.17:0/2074029832 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f474c0ca500 con 0x7f474c071760 2024-09-10T21:25:32.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.486+0000 7f4749600640 1 -- 172.21.15.17:0/2074029832 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4740003860 con 0x7f474c071760 2024-09-10T21:25:32.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.486+0000 7f4754000640 1 -- 172.21.15.17:0/2074029832 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f474c0ca7a0 con 0x7f474c071760 2024-09-10T21:25:32.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.486+0000 7f4754000640 1 --2- 172.21.15.17:0/2074029832 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f474c1e3980 0x7f474c1f3c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:32.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.486+0000 7f4754000640 1 -- 172.21.15.17:0/2074029832 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f474c159260 con 0x7f474c1e3980 2024-09-10T21:25:32.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.486+0000 7f4750c00640 1 --2- 172.21.15.17:0/2074029832 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f474c1e3980 0x7f474c1f3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:32.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.486+0000 7f4750c00640 1 -- 172.21.15.17:0/2074029832 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f474c159f70 con 0x7f474c1e3980 2024-09-10T21:25:32.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.486+0000 7f4749600640 1 -- 172.21.15.17:0/2074029832 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f4740003a00 con 0x7f474c071760 2024-09-10T21:25:32.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.486+0000 7f4750c00640 1 --2- 172.21.15.17:0/2074029832 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f474c1e3980 0x7f474c1f3c60 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f474c070940 tx=0x7f47380050d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:32.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.486+0000 7f4749600640 1 --2- 172.21.15.17:0/2074029832 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f472c0725e0 0x7f472c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:32.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.486+0000 7f4749600640 1 -- 172.21.15.17:0/2074029832 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f4740092be0 con 0x7f474c071760 2024-09-10T21:25:32.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.486+0000 7f4752000640 1 --2- 172.21.15.17:0/2074029832 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f472c0725e0 0x7f472c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:32.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.486+0000 7f4749600640 1 -- 172.21.15.17:0/2074029832 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47380c0020 con 0x7f474c1e3980 2024-09-10T21:25:32.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.486+0000 7f4749600640 1 -- 172.21.15.17:0/2074029832 <== mon.1 v2:172.21.15.31:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f47380b6040 con 0x7f474c1e3980 2024-09-10T21:25:32.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.486+0000 7f4749600640 1 -- 172.21.15.17:0/2074029832 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f474c1e3980 msgr2=0x7f474c1f3c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:32.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.486+0000 7f4749600640 1 --2- 172.21.15.17:0/2074029832 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f474c1e3980 0x7f474c1f3c60 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f474c070940 tx=0x7f47380050d0 comp rx=0 tx=0).stop 2024-09-10T21:25:32.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.486+0000 7f4749600640 1 -- 172.21.15.17:0/2074029832 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f474c1e3980 msgr2=0x7f474c1f3c60 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:25:32.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.486+0000 7f4749600640 1 --2- 172.21.15.17:0/2074029832 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f474c1e3980 0x7f474c1f3c60 secure :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f474c070940 tx=0x7f47380050d0 comp rx=0 tx=0).stop 2024-09-10T21:25:32.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.490+0000 7f4752000640 1 --2- 172.21.15.17:0/2074029832 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f472c0725e0 0x7f472c074aa0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f47440092f0 tx=0x7f4744009650 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:32.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.494+0000 7f4754000640 1 --2- 172.21.15.17:0/2074029832 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f474c1e3d60 0x7f474c1f41a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:32.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.494+0000 7f4754000640 1 -- 172.21.15.17:0/2074029832 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f474c069050 con 0x7f474c1e3d60 2024-09-10T21:25:32.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.494+0000 7f4750c00640 1 --2- 172.21.15.17:0/2074029832 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f474c1e3d60 0x7f474c1f41a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:32.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.494+0000 7f4750c00640 1 -- 172.21.15.17:0/2074029832 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f474c068d70 con 0x7f474c1e3d60 2024-09-10T21:25:32.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.494+0000 7f4750c00640 1 --2- 172.21.15.17:0/2074029832 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f474c1e3d60 0x7f474c1f41a0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f47380b3040 tx=0x7f4738009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:32.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.494+0000 7f4749600640 1 -- 172.21.15.17:0/2074029832 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47380c0050 con 0x7f474c1e3d60 2024-09-10T21:25:32.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.494+0000 7f4749600640 1 -- 172.21.15.17:0/2074029832 <== mon.1 v2:172.21.15.31:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5041 (secure 0 0 0) 0x7f4738003c40 con 0x7f474c1e3d60 2024-09-10T21:25:32.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.494+0000 7f4749600640 1 -- 172.21.15.17:0/2074029832 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f474c1e3d60 msgr2=0x7f474c1f41a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:32.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.494+0000 7f4749600640 1 --2- 172.21.15.17:0/2074029832 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f474c1e3d60 0x7f474c1f41a0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f47380b3040 tx=0x7f4738009260 comp rx=0 tx=0).stop 2024-09-10T21:25:32.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.494+0000 7f4749600640 1 -- 172.21.15.17:0/2074029832 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f474c1e3d60 msgr2=0x7f474c1f41a0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:25:32.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.494+0000 7f4749600640 1 --2- 172.21.15.17:0/2074029832 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f474c1e3d60 0x7f474c1f41a0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:32.501 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-10T21:25:32.501 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:25:32.501 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:25:32.501 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-10T21:25:32.501 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 4, 2024-09-10T21:25:32.501 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 70941, 2024-09-10T21:25:32.501 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-10T21:25:32.502 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-10T21:25:32.502 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-10T21:25:32.502 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-10T21:25:32.502 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:25:32.502 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 70, 2024-09-10T21:25:32.502 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:25:32.502 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-10T21:25:32.502 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-10T21:25:32.502 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:25:32.502 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:25:32.502 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:25:32.502 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:25:32.503 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:25:32.503 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:25:32.503 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:25:32.503 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:25:32.503 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:25:32.503 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:25:32.503 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:25:32.503 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:25:32.503 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-10T21:25:32.503 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-10T21:25:32.503 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:25:32.503 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:25:32.504 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:25:32.504 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:25:32.504 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:25:32.504 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:25:32.504 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:25:32.504 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:25:32.504 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:25:32.504 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:25:32.504 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:25:32.504 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:25:32.505 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:25:32.505 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-10T21:25:32.505 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-10T21:25:32.505 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-10T21:25:32.505 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-10T21:25:32.505 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-10T21:25:32.505 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "5e2f023f-d6a6-49f4-a256-f4807dce3739", 2024-09-10T21:25:32.505 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:25:32.505 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:25:32.505 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-10T21:25:32.505 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-10T21:25:32.505 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-10T21:25:32.506 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-10T21:25:32.506 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-10T21:25:32.506 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-10T21:25:32.506 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-10T21:25:32.506 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:25:32.506 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-10T21:25:32.506 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:25:32.506 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:25:32.506 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:25:32.506 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:25:32.506 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:25:32.506 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:25:32.507 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:25:32.507 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:25:32.507 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:25:32.507 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:25:32.507 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:25:32.507 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:25:32.507 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-10T21:25:32.507 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:25:32.507 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-10T21:25:32.508 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:25:32.508 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:25:32.508 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:25:32.508 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:25:32.508 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:25:32.508 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:25:32.508 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:25:32.508 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-10T21:25:32.508 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:25:32.508 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:25:32.508 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:25:32.509 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:25:32.509 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-10T21:25:32.509 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:25:32.509 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:25:32.509 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:25:32.509 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:25:32.509 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-10T21:25:32.509 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-10T21:25:32.509 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:25:32.509 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:25:32.509 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:25:32.509 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:25:32.510 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:25:32.510 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:25:32.510 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:25:32.510 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:25:32.510 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:25:32.510 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:25:32.510 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:25:32.510 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3300", 2024-09-10T21:25:32.510 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:25:32.510 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:25:32.510 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:25:32.511 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:25:32.511 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789", 2024-09-10T21:25:32.511 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:25:32.511 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:25:32.511 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:25:32.511 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:25:32.511 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789/0", 2024-09-10T21:25:32.511 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6789/0", 2024-09-10T21:25:32.511 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:25:32.511 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:25:32.512 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:25:32.512 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:25:32.512 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:25:32.512 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:25:32.512 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:25:32.512 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:25:32.512 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:25:32.512 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:25:32.512 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:25:32.512 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3301", 2024-09-10T21:25:32.513 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:25:32.513 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:25:32.513 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:25:32.513 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:25:32.513 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790", 2024-09-10T21:25:32.513 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:25:32.513 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:25:32.513 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:25:32.513 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:25:32.513 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790/0", 2024-09-10T21:25:32.513 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6790/0", 2024-09-10T21:25:32.513 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:25:32.514 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:25:32.514 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:25:32.514 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:25:32.514 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:25:32.514 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:25:32.514 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-10T21:25:32.514 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-10T21:25:32.514 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:25:32.514 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:25:32.514 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:25:32.514 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:25:32.514 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:25:32.515 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:25:32.515 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": [ 2024-09-10T21:25:32.515 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:25:32.515 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:25:32.515 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:25:32.515 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 4 2024-09-10T21:25:32.515 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:25:32.515 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:25:32.515 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-10T21:25:32.515 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:25:32.515 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:25:32.516 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:25:32.516 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 2 2024-09-10T21:25:32.516 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:25:32.516 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:25:32.516 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-10T21:25:32.516 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:25:32.516 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:25:32.516 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:25:32.516 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 2 2024-09-10T21:25:32.516 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:25:32.516 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:25:32.516 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": [ 2024-09-10T21:25:32.517 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:25:32.517 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:25:32.517 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:25:32.517 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:25:32.517 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:25:32.517 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:25:32.517 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:25:32.517 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-10T21:25:32.517 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-10T21:25:32.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.498+0000 7f4754000640 1 -- 172.21.15.17:0/2074029832 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f472c0725e0 msgr2=0x7f472c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:32.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.498+0000 7f4754000640 1 --2- 172.21.15.17:0/2074029832 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f472c0725e0 0x7f472c074aa0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f47440092f0 tx=0x7f4744009650 comp rx=0 tx=0).stop 2024-09-10T21:25:32.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.498+0000 7f4754000640 1 -- 172.21.15.17:0/2074029832 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f474c071760 msgr2=0x7f474c0cadc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:32.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.498+0000 7f4754000640 1 --2- 172.21.15.17:0/2074029832 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f474c071760 0x7f474c0cadc0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f474000c3c0 tx=0x7f474000c880 comp rx=0 tx=0).stop 2024-09-10T21:25:32.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.498+0000 7f4752000640 1 -- 172.21.15.17:0/2074029832 reap_dead start 2024-09-10T21:25:32.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.498+0000 7f4752000640 1 -- 172.21.15.17:0/2074029832 reap_dead start 2024-09-10T21:25:32.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.498+0000 7f4754000640 1 -- 172.21.15.17:0/2074029832 shutdown_connections 2024-09-10T21:25:32.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.498+0000 7f4754000640 1 -- 172.21.15.17:0/2074029832 >> 172.21.15.17:0/2074029832 conn(0x7f474c068990 msgr2=0x7f474c15c020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:32.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.498+0000 7f4754000640 1 -- 172.21.15.17:0/2074029832 shutdown_connections 2024-09-10T21:25:32.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.498+0000 7f4754000640 1 -- 172.21.15.17:0/2074029832 wait complete. 2024-09-10T21:25:32.520 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-10T21:25:32.520 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-09-10T21:25:32.520 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-10T21:25:32.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.662+0000 7f7c0d800640 1 -- 172.21.15.17:0/1948210127 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7c08158f60 msgr2=0x7f7c08163bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:32.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.662+0000 7f7c0d800640 1 --2- 172.21.15.17:0/1948210127 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7c08158f60 0x7f7c08163bf0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f7c00009bc0 tx=0x7f7c0001f3b0 comp rx=0 tx=0).stop 2024-09-10T21:25:32.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.662+0000 7f7c0d800640 1 -- 172.21.15.17:0/1948210127 shutdown_connections 2024-09-10T21:25:32.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.662+0000 7f7c0d800640 1 --2- 172.21.15.17:0/1948210127 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7c08158f60 0x7f7c08163bf0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:32.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.662+0000 7f7c0d800640 1 -- 172.21.15.17:0/1948210127 >> 172.21.15.17:0/1948210127 conn(0x7f7c0806cc20 msgr2=0x7f7c0806d030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:32.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.662+0000 7f7c0d800640 1 -- 172.21.15.17:0/1948210127 shutdown_connections 2024-09-10T21:25:32.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.662+0000 7f7c0d800640 1 -- 172.21.15.17:0/1948210127 wait complete. 2024-09-10T21:25:32.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.666+0000 7f7c0d800640 1 Processor -- start 2024-09-10T21:25:32.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.666+0000 7f7c0d800640 1 -- start start 2024-09-10T21:25:32.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.666+0000 7f7c0d800640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f7c08152070 0x7f7c0814bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:32.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.666+0000 7f7c0d800640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7c08152450 0x7f7c081e2b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:32.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.666+0000 7f7c0d800640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c08158f60 0x7f7c080c9680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:32.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.666+0000 7f7c0d800640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f7c080cecf0 con 0x7f7c08152450 2024-09-10T21:25:32.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.666+0000 7f7c0d800640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f7c080c5090 con 0x7f7c08158f60 2024-09-10T21:25:32.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.666+0000 7f7c0d800640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f7c080cee70 con 0x7f7c08152070 2024-09-10T21:25:32.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.666+0000 7f7bffe00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7c08152450 0x7f7c081e2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:32.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.666+0000 7f7bffe00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7c08152450 0x7f7c081e2b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43846/0 (socket says 172.21.15.17:43846) 2024-09-10T21:25:32.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.666+0000 7f7bffe00640 1 -- 172.21.15.17:0/1421418650 learned_addr learned my addr 172.21.15.17:0/1421418650 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:25:32.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.666+0000 7f7c0ce00640 1 --2- 172.21.15.17:0/1421418650 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c08158f60 0x7f7c080c9680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:32.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.666+0000 7f7c07e00640 1 --2- 172.21.15.17:0/1421418650 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f7c08152070 0x7f7c0814bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:32.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.666+0000 7f7bffe00640 1 -- 172.21.15.17:0/1421418650 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f7c08152070 msgr2=0x7f7c0814bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:32.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.666+0000 7f7bffe00640 1 --2- 172.21.15.17:0/1421418650 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f7c08152070 0x7f7c0814bb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:32.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.666+0000 7f7bffe00640 1 -- 172.21.15.17:0/1421418650 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c08158f60 msgr2=0x7f7c080c9680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:32.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.666+0000 7f7bffe00640 1 --2- 172.21.15.17:0/1421418650 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c08158f60 0x7f7c080c9680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:32.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.666+0000 7f7bffe00640 1 -- 172.21.15.17:0/1421418650 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c0814cfd0 con 0x7f7c08152450 2024-09-10T21:25:32.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.666+0000 7f7bffe00640 1 --2- 172.21.15.17:0/1421418650 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7c08152450 0x7f7c081e2b80 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f7bf000c400 tx=0x7f7bf000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:32.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.666+0000 7f7c04c00640 1 -- 172.21.15.17:0/1421418650 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bf00068a0 con 0x7f7c08152450 2024-09-10T21:25:32.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.666+0000 7f7c04c00640 1 -- 172.21.15.17:0/1421418650 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7bf0006b70 con 0x7f7c08152450 2024-09-10T21:25:32.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.666+0000 7f7c0d800640 1 -- 172.21.15.17:0/1421418650 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c081f08b0 con 0x7f7c08152450 2024-09-10T21:25:32.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.666+0000 7f7c04c00640 1 -- 172.21.15.17:0/1421418650 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bf0007020 con 0x7f7c08152450 2024-09-10T21:25:32.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.666+0000 7f7c0d800640 1 -- 172.21.15.17:0/1421418650 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c081f0b50 con 0x7f7c08152450 2024-09-10T21:25:32.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.666+0000 7f7c0d800640 1 --2- 172.21.15.17:0/1421418650 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f7c081f11a0 0x7f7c080c9bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:32.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.666+0000 7f7c0d800640 1 -- 172.21.15.17:0/1421418650 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f7c0814cda0 con 0x7f7c081f11a0 2024-09-10T21:25:32.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.666+0000 7f7c07e00640 1 --2- 172.21.15.17:0/1421418650 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f7c081f11a0 0x7f7c080c9bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:32.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.670+0000 7f7c07e00640 1 -- 172.21.15.17:0/1421418650 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7c080cc710 con 0x7f7c081f11a0 2024-09-10T21:25:32.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.670+0000 7f7c07e00640 1 --2- 172.21.15.17:0/1421418650 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f7c081f11a0 0x7f7c080c9bc0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f7bf800f770 tx=0x7f7bf800fc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:32.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.670+0000 7f7c04c00640 1 -- 172.21.15.17:0/1421418650 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bf8003970 con 0x7f7c081f11a0 2024-09-10T21:25:32.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.670+0000 7f7c04c00640 1 -- 172.21.15.17:0/1421418650 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f7bf00071c0 con 0x7f7c08152450 2024-09-10T21:25:32.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.670+0000 7f7c04c00640 1 --2- 172.21.15.17:0/1421418650 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f7be4072510 0x7f7be40749d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:32.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.670+0000 7f7c04c00640 1 -- 172.21.15.17:0/1421418650 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f7bf0092fc0 con 0x7f7c08152450 2024-09-10T21:25:32.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.670+0000 7f7c04c00640 1 -- 172.21.15.17:0/1421418650 <== mon.2 v2:172.21.15.31:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f7bf8019070 con 0x7f7c081f11a0 2024-09-10T21:25:32.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.670+0000 7f7c04c00640 1 -- 172.21.15.17:0/1421418650 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f7c081f11a0 msgr2=0x7f7c080c9bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:32.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.670+0000 7f7c04c00640 1 --2- 172.21.15.17:0/1421418650 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f7c081f11a0 0x7f7c080c9bc0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f7bf800f770 tx=0x7f7bf800fc30 comp rx=0 tx=0).stop 2024-09-10T21:25:32.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.670+0000 7f7c04c00640 1 -- 172.21.15.17:0/1421418650 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f7c081f11a0 msgr2=0x7f7c080c9bc0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:25:32.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.670+0000 7f7c04c00640 1 --2- 172.21.15.17:0/1421418650 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f7c081f11a0 0x7f7c080c9bc0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:32.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.670+0000 7f7c0ce00640 1 --2- 172.21.15.17:0/1421418650 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f7be4072510 0x7f7be40749d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:32.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.670+0000 7f7c0ce00640 1 --2- 172.21.15.17:0/1421418650 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f7be4072510 0x7f7be40749d0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f7c000b1000 tx=0x7f7c0001dbf0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:32.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.678+0000 7f7c0d800640 1 --2- 172.21.15.17:0/1421418650 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f7c081f0d90 0x7f7c081f3d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:32.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.678+0000 7f7c0d800640 1 -- 172.21.15.17:0/1421418650 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f7c080cb7e0 con 0x7f7c081f0d90 2024-09-10T21:25:32.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.678+0000 7f7c07e00640 1 --2- 172.21.15.17:0/1421418650 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f7c081f0d90 0x7f7c081f3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:32.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.678+0000 7f7c07e00640 1 -- 172.21.15.17:0/1421418650 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f7c080cbcf0 con 0x7f7c081f0d90 2024-09-10T21:25:32.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.678+0000 7f7c07e00640 1 --2- 172.21.15.17:0/1421418650 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f7c081f0d90 0x7f7c081f3d90 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7bf800f770 tx=0x7f7bf8003770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:32.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.678+0000 7f7c04c00640 1 -- 172.21.15.17:0/1421418650 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bf801e7c0 con 0x7f7c081f0d90 2024-09-10T21:25:32.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.678+0000 7f7c04c00640 1 -- 172.21.15.17:0/1421418650 <== mon.2 v2:172.21.15.31:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4874 (secure 0 0 0) 0x7f7bf801e7c0 con 0x7f7c081f0d90 2024-09-10T21:25:32.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.678+0000 7f7c04c00640 1 -- 172.21.15.17:0/1421418650 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f7c081f0d90 msgr2=0x7f7c081f3d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:32.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.678+0000 7f7c04c00640 1 --2- 172.21.15.17:0/1421418650 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f7c081f0d90 0x7f7c081f3d90 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7bf800f770 tx=0x7f7bf8003770 comp rx=0 tx=0).stop 2024-09-10T21:25:32.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.678+0000 7f7c04c00640 1 -- 172.21.15.17:0/1421418650 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f7c081f0d90 msgr2=0x7f7c081f3d90 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:25:32.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.678+0000 7f7c04c00640 1 --2- 172.21.15.17:0/1421418650 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f7c081f0d90 0x7f7c081f3d90 secure :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f7bf800f770 tx=0x7f7bf8003770 comp rx=0 tx=0).stop 2024-09-10T21:25:32.684 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-10T21:25:32.684 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:25:32.684 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:25:32.684 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-10T21:25:32.684 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 4, 2024-09-10T21:25:32.685 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 71125, 2024-09-10T21:25:32.685 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-10T21:25:32.685 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-10T21:25:32.685 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-10T21:25:32.685 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-10T21:25:32.685 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:25:32.685 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 70, 2024-09-10T21:25:32.685 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:25:32.686 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-10T21:25:32.686 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-10T21:25:32.686 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:25:32.686 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:25:32.686 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:25:32.686 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:25:32.686 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:25:32.687 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:25:32.687 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:25:32.687 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:25:32.687 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:25:32.687 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:25:32.687 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:25:32.687 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:25:32.687 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-10T21:25:32.688 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-10T21:25:32.688 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:25:32.688 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:25:32.688 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:25:32.688 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:25:32.688 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:25:32.689 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:25:32.689 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:25:32.689 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:25:32.689 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:25:32.689 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:25:32.689 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:25:32.689 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:25:32.690 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:25:32.690 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-10T21:25:32.690 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-10T21:25:32.690 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-10T21:25:32.690 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-10T21:25:32.690 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-10T21:25:32.690 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "5e2f023f-d6a6-49f4-a256-f4807dce3739", 2024-09-10T21:25:32.690 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:25:32.691 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:25:32.691 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-10T21:25:32.691 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-10T21:25:32.691 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-10T21:25:32.691 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-10T21:25:32.691 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-10T21:25:32.691 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-10T21:25:32.691 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-10T21:25:32.692 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:25:32.692 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-10T21:25:32.692 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:25:32.692 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:25:32.692 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:25:32.692 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:25:32.692 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:25:32.692 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:25:32.693 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:25:32.693 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:25:32.693 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:25:32.693 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:25:32.693 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:25:32.693 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:25:32.693 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-10T21:25:32.693 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:25:32.694 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-10T21:25:32.694 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:25:32.694 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:25:32.694 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:25:32.694 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:25:32.694 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:25:32.694 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:25:32.694 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:25:32.695 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-10T21:25:32.695 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:25:32.695 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:25:32.695 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:25:32.695 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:25:32.695 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-10T21:25:32.695 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:25:32.695 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:25:32.696 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:25:32.696 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:25:32.696 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-10T21:25:32.696 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-10T21:25:32.696 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:25:32.696 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:25:32.696 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:25:32.697 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:25:32.697 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:25:32.697 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:25:32.697 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:25:32.697 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:25:32.697 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:25:32.697 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:25:32.697 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:25:32.698 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3300", 2024-09-10T21:25:32.698 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:25:32.698 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:25:32.698 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:25:32.698 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:25:32.698 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789", 2024-09-10T21:25:32.698 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:25:32.698 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:25:32.699 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:25:32.699 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:25:32.699 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789/0", 2024-09-10T21:25:32.699 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6789/0", 2024-09-10T21:25:32.699 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:25:32.699 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:25:32.699 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:25:32.699 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:25:32.700 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:25:32.700 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:25:32.700 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:25:32.700 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:25:32.700 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:25:32.700 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:25:32.700 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:25:32.700 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3301", 2024-09-10T21:25:32.701 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:25:32.701 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:25:32.701 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:25:32.701 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:25:32.701 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790", 2024-09-10T21:25:32.701 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:25:32.701 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:25:32.701 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:25:32.702 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:25:32.702 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790/0", 2024-09-10T21:25:32.702 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6790/0", 2024-09-10T21:25:32.702 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:25:32.702 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:25:32.702 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:25:32.702 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:25:32.703 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:25:32.703 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:25:32.703 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-10T21:25:32.703 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-10T21:25:32.703 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:25:32.703 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:25:32.703 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:25:32.703 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:25:32.703 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:25:32.704 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:25:32.704 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-10T21:25:32.704 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:25:32.704 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:25:32.704 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:25:32.704 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 3 2024-09-10T21:25:32.704 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:25:32.705 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:25:32.705 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-10T21:25:32.705 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:25:32.705 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:25:32.705 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:25:32.705 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 4 2024-09-10T21:25:32.705 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:25:32.705 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:25:32.706 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": [ 2024-09-10T21:25:32.706 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:25:32.706 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:25:32.706 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:25:32.706 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:25:32.706 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:25:32.706 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:25:32.706 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:25:32.707 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-10T21:25:32.707 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-10T21:25:32.709 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.678+0000 7f7c0d800640 1 -- 172.21.15.17:0/1421418650 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f7be4072510 msgr2=0x7f7be40749d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:32.709 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.678+0000 7f7c0d800640 1 --2- 172.21.15.17:0/1421418650 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f7be4072510 0x7f7be40749d0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f7c000b1000 tx=0x7f7c0001dbf0 comp rx=0 tx=0).stop 2024-09-10T21:25:32.709 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.678+0000 7f7c0d800640 1 -- 172.21.15.17:0/1421418650 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7c08152450 msgr2=0x7f7c081e2b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:32.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.678+0000 7f7c0d800640 1 --2- 172.21.15.17:0/1421418650 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7c08152450 0x7f7c081e2b80 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f7bf000c400 tx=0x7f7bf000c8c0 comp rx=0 tx=0).stop 2024-09-10T21:25:32.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.678+0000 7f7c0ce00640 1 -- 172.21.15.17:0/1421418650 reap_dead start 2024-09-10T21:25:32.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.678+0000 7f7c0ce00640 1 -- 172.21.15.17:0/1421418650 reap_dead start 2024-09-10T21:25:32.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.678+0000 7f7c0d800640 1 -- 172.21.15.17:0/1421418650 shutdown_connections 2024-09-10T21:25:32.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.678+0000 7f7c0d800640 1 -- 172.21.15.17:0/1421418650 >> 172.21.15.17:0/1421418650 conn(0x7f7c0806cc20 msgr2=0x7f7c081e4200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:32.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.678+0000 7f7c0d800640 1 -- 172.21.15.17:0/1421418650 shutdown_connections 2024-09-10T21:25:32.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.678+0000 7f7c0d800640 1 -- 172.21.15.17:0/1421418650 wait complete. 2024-09-10T21:25:32.711 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-10T21:25:32.711 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['a'] 2024-09-10T21:25:32.711 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2024-09-10T21:25:32.711 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.a 2024-09-10T21:25:32.711 INFO:tasks.mon_thrash.mon_thrasher:killing mon.a 2024-09-10T21:25:32.712 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-10T21:25:32.712 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:25:32.729 INFO:tasks.ceph.mon.a:Stopped 2024-09-10T21:25:32.729 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2024-09-10T21:25:32.730 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-10T21:25:32.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.914+0000 7f5f34000640 1 -- 172.21.15.17:0/1741297141 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f5f2c0b2070 msgr2=0x7f5f2c0725c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:32.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.914+0000 7f5f34000640 1 --2- 172.21.15.17:0/1741297141 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f5f2c0b2070 0x7f5f2c0725c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f5f24009c80 tx=0x7f5f2401f5a0 comp rx=0 tx=0).stop 2024-09-10T21:25:32.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.914+0000 7f5f34000640 1 -- 172.21.15.17:0/1741297141 shutdown_connections 2024-09-10T21:25:32.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.914+0000 7f5f34000640 1 --2- 172.21.15.17:0/1741297141 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f5f2c0b2070 0x7f5f2c0725c0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:32.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.914+0000 7f5f34000640 1 --2- 172.21.15.17:0/1741297141 >> v2:172.21.15.31:3300/0 conn(0x7f5f2c1516c0 0x7f5f2c0c25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:32.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.914+0000 7f5f34000640 1 -- 172.21.15.17:0/1741297141 >> 172.21.15.17:0/1741297141 conn(0x7f5f2c070160 msgr2=0x7f5f2c070570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:32.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.914+0000 7f5f34000640 1 -- 172.21.15.17:0/1741297141 shutdown_connections 2024-09-10T21:25:32.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.914+0000 7f5f34000640 1 -- 172.21.15.17:0/1741297141 wait complete. 2024-09-10T21:25:32.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.914+0000 7f5f34000640 1 Processor -- start 2024-09-10T21:25:32.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.914+0000 7f5f34000640 1 -- start start 2024-09-10T21:25:32.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.914+0000 7f5f34000640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f5f2c0b2070 0x7f5f2c1e9f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:32.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.914+0000 7f5f34000640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f2c1516c0 0x7f5f2c06dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:32.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.914+0000 7f5f34000640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5f2c151de0 0x7f5f2c06e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:32.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.914+0000 7f5f34000640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f5f2c0c2cd0 con 0x7f5f2c151de0 2024-09-10T21:25:32.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.914+0000 7f5f34000640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f5f2c15ad20 con 0x7f5f2c1516c0 2024-09-10T21:25:32.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.914+0000 7f5f34000640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f5f2c0c2e50 con 0x7f5f2c0b2070 2024-09-10T21:25:32.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.914+0000 7f5f32000640 1 -- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5f2c151de0 msgr2=0x7f5f2c06e270 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-10T21:25:32.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.914+0000 7f5f32000640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5f2c151de0 0x7f5f2c06e270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:25:32.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.914+0000 7f5f31600640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f5f2c0b2070 0x7f5f2c1e9f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:32.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.914+0000 7f5f31600640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f5f2c0b2070 0x7f5f2c1e9f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3301/0 says I am v2:172.21.15.17:47380/0 (socket says 172.21.15.17:47380) 2024-09-10T21:25:32.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.914+0000 7f5f31600640 1 -- 172.21.15.17:0/1613586062 learned_addr learned my addr 172.21.15.17:0/1613586062 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:25:32.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.914+0000 7f5f30c00640 1 --2- 172.21.15.17:0/1613586062 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f2c1516c0 0x7f5f2c06dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:32.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.918+0000 7f5f31600640 1 -- 172.21.15.17:0/1613586062 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f2c1516c0 msgr2=0x7f5f2c06dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:32.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.918+0000 7f5f31600640 1 --2- 172.21.15.17:0/1613586062 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f2c1516c0 0x7f5f2c06dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:32.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.918+0000 7f5f31600640 1 -- 172.21.15.17:0/1613586062 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5f2c151de0 msgr2=0x7f5f2c06e270 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:25:32.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.918+0000 7f5f31600640 1 --2- 172.21.15.17:0/1613586062 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5f2c151de0 0x7f5f2c06e270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:32.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.918+0000 7f5f31600640 1 -- 172.21.15.17:0/1613586062 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f2c1f0d20 con 0x7f5f2c0b2070 2024-09-10T21:25:32.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.918+0000 7f5f31600640 1 --2- 172.21.15.17:0/1613586062 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f5f2c0b2070 0x7f5f2c1e9f90 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f5f2000ea20 tx=0x7f5f2000eee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:32.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.918+0000 7f5f29600640 1 -- 172.21.15.17:0/1613586062 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f2000cd80 con 0x7f5f2c0b2070 2024-09-10T21:25:32.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.918+0000 7f5f29600640 1 -- 172.21.15.17:0/1613586062 <== mon.2 v2:172.21.15.31:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5f20010430 con 0x7f5f2c0b2070 2024-09-10T21:25:32.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.918+0000 7f5f29600640 1 -- 172.21.15.17:0/1613586062 <== mon.2 v2:172.21.15.31:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f20010860 con 0x7f5f2c0b2070 2024-09-10T21:25:32.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.918+0000 7f5f34000640 1 -- 172.21.15.17:0/1613586062 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f2c159bf0 con 0x7f5f2c0b2070 2024-09-10T21:25:32.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.918+0000 7f5f34000640 1 -- 172.21.15.17:0/1613586062 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f2c0ca410 con 0x7f5f2c0b2070 2024-09-10T21:25:32.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.918+0000 7f5f34000640 1 -- 172.21.15.17:0/1613586062 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f2c0c25d0 con 0x7f5f2c0b2070 2024-09-10T21:25:32.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.918+0000 7f5f29600640 1 -- 172.21.15.17:0/1613586062 <== mon.2 v2:172.21.15.31:3301/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f5f2003daa0 con 0x7f5f2c0b2070 2024-09-10T21:25:32.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.922+0000 7f5f29600640 1 --2- 172.21.15.17:0/1613586062 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f5f0c0725c0 0x7f5f0c074a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:32.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.922+0000 7f5f29600640 1 -- 172.21.15.17:0/1613586062 <== mon.2 v2:172.21.15.31:3301/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f5f200942c0 con 0x7f5f2c0b2070 2024-09-10T21:25:32.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.922+0000 7f5f30c00640 1 --2- 172.21.15.17:0/1613586062 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f5f0c0725c0 0x7f5f0c074a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:32.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.922+0000 7f5f30c00640 1 --2- 172.21.15.17:0/1613586062 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f5f0c0725c0 0x7f5f0c074a80 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f5f2c074d70 tx=0x7f5f18009110 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:32.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:32.922+0000 7f5f29600640 1 -- 172.21.15.17:0/1613586062 <== mon.2 v2:172.21.15.31:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f5f2005ef30 con 0x7f5f2c0b2070 2024-09-10T21:25:33.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:33.158+0000 7f5f34000640 1 -- 172.21.15.17:0/1613586062 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f5f2c0c10a0 con 0x7f5f2c0b2070 2024-09-10T21:25:33.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:33.158+0000 7f5f29600640 1 -- 172.21.15.17:0/1613586062 <== mon.2 v2:172.21.15.31:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1556 (secure 0 0 0) 0x7f5f20063960 con 0x7f5f2c0b2070 2024-09-10T21:25:33.162 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":71,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5e2f023f-d6a6-49f4-a256-f4807dce3739","modified":"2024-09-10T21:23:46.892130Z","created":"2024-09-10T21:23:46.892130Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3301","nonce":0},{"type":"v1","addr":"172.21.15.31:6790","nonce":0}]},"addr":"172.21.15.31:6790/0","public_addr":"172.21.15.31:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-10T21:25:33.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:33.162+0000 7f5f34000640 1 -- 172.21.15.17:0/1613586062 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f5f0c0725c0 msgr2=0x7f5f0c074a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:33.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:33.162+0000 7f5f34000640 1 --2- 172.21.15.17:0/1613586062 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f5f0c0725c0 0x7f5f0c074a80 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f5f2c074d70 tx=0x7f5f18009110 comp rx=0 tx=0).stop 2024-09-10T21:25:33.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:33.162+0000 7f5f34000640 1 -- 172.21.15.17:0/1613586062 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f5f2c0b2070 msgr2=0x7f5f2c1e9f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:33.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:33.162+0000 7f5f34000640 1 --2- 172.21.15.17:0/1613586062 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f5f2c0b2070 0x7f5f2c1e9f90 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f5f2000ea20 tx=0x7f5f2000eee0 comp rx=0 tx=0).stop 2024-09-10T21:25:33.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:33.162+0000 7f5f34000640 1 -- 172.21.15.17:0/1613586062 shutdown_connections 2024-09-10T21:25:33.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:33.162+0000 7f5f34000640 1 --2- 172.21.15.17:0/1613586062 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f5f0c0725c0 0x7f5f0c074a80 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:33.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:33.162+0000 7f5f34000640 1 --2- 172.21.15.17:0/1613586062 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5f2c151de0 0x7f5f2c06e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:33.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:33.162+0000 7f5f34000640 1 --2- 172.21.15.17:0/1613586062 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f2c1516c0 0x7f5f2c06dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:33.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:33.162+0000 7f5f34000640 1 --2- 172.21.15.17:0/1613586062 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f5f2c0b2070 0x7f5f2c1e9f90 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:33.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:33.162+0000 7f5f34000640 1 -- 172.21.15.17:0/1613586062 >> 172.21.15.17:0/1613586062 conn(0x7f5f2c070160 msgr2=0x7f5f2c0cb1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:33.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:33.162+0000 7f5f34000640 1 -- 172.21.15.17:0/1613586062 shutdown_connections 2024-09-10T21:25:33.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:33.162+0000 7f5f34000640 1 -- 172.21.15.17:0/1613586062 wait complete. 2024-09-10T21:25:33.735 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~0s 2024-09-10T21:25:36.186 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-10T21:25:36.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:36.338+0000 7fdd1fe00640 1 -- 172.21.15.17:0/1646460432 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd18152820 msgr2=0x7fdd1815c280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:36.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:36.338+0000 7fdd1fe00640 1 --2- 172.21.15.17:0/1646460432 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd18152820 0x7fdd1815c280 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fdd181594a0 tx=0x7fdd0401c850 comp rx=0 tx=0).stop 2024-09-10T21:25:36.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:36.338+0000 7fdd1fe00640 1 -- 172.21.15.17:0/1646460432 shutdown_connections 2024-09-10T21:25:36.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:36.338+0000 7fdd1fe00640 1 --2- 172.21.15.17:0/1646460432 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd18152820 0x7fdd1815c280 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:36.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:36.338+0000 7fdd1fe00640 1 --2- 172.21.15.17:0/1646460432 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fdd18152440 0x7fdd180b3eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:36.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:36.338+0000 7fdd1fe00640 1 -- 172.21.15.17:0/1646460432 >> 172.21.15.17:0/1646460432 conn(0x7fdd180702f0 msgr2=0x7fdd18070700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:36.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:36.338+0000 7fdd1fe00640 1 -- 172.21.15.17:0/1646460432 shutdown_connections 2024-09-10T21:25:36.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:36.338+0000 7fdd1fe00640 1 -- 172.21.15.17:0/1646460432 wait complete. 2024-09-10T21:25:36.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:36.342+0000 7fdd1fe00640 1 Processor -- start 2024-09-10T21:25:36.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:36.342+0000 7fdd1fe00640 1 -- start start 2024-09-10T21:25:36.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:36.342+0000 7fdd1fe00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdd180cdf20 0x7fdd181f5280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:36.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:36.342+0000 7fdd1fe00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd18152440 0x7fdd18154470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:36.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:36.342+0000 7fdd1fe00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fdd18152820 0x7fdd181534b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:36.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:36.342+0000 7fdd1fe00640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fdd1815c7c0 con 0x7fdd180cdf20 2024-09-10T21:25:36.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:36.342+0000 7fdd1fe00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fdd180ce760 con 0x7fdd18152440 2024-09-10T21:25:36.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:36.342+0000 7fdd1fe00640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7fdd180cb450 con 0x7fdd18152820 2024-09-10T21:25:36.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:36.342+0000 7fdd1d400640 1 -- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdd180cdf20 msgr2=0x7fdd181f5280 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-10T21:25:36.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:36.342+0000 7fdd1d400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdd180cdf20 0x7fdd181f5280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:25:36.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:36.342+0000 7fdd1de00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fdd18152820 0x7fdd181534b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:36.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:36.342+0000 7fdd1de00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fdd18152820 0x7fdd181534b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3301/0 says I am v2:172.21.15.17:33236/0 (socket says 172.21.15.17:33236) 2024-09-10T21:25:36.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:36.342+0000 7fdd1de00640 1 -- 172.21.15.17:0/291447980 learned_addr learned my addr 172.21.15.17:0/291447980 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:25:36.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:36.342+0000 7fdd1ca00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd18152440 0x7fdd18154470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:36.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:36.342+0000 7fdd1de00640 1 -- 172.21.15.17:0/291447980 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd18152440 msgr2=0x7fdd18154470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:36.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:36.342+0000 7fdd1de00640 1 --2- 172.21.15.17:0/291447980 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd18152440 0x7fdd18154470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:36.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:36.342+0000 7fdd1de00640 1 -- 172.21.15.17:0/291447980 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdd180cdf20 msgr2=0x7fdd181f5280 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:25:36.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:36.342+0000 7fdd1de00640 1 --2- 172.21.15.17:0/291447980 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdd180cdf20 0x7fdd181f5280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:36.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:36.342+0000 7fdd1de00640 1 -- 172.21.15.17:0/291447980 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd181549b0 con 0x7fdd18152820 2024-09-10T21:25:36.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:36.342+0000 7fdd1de00640 1 --2- 172.21.15.17:0/291447980 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fdd18152820 0x7fdd181534b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fdd1000b9e0 tx=0x7fdd1000bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:36.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:36.342+0000 7fdd15600640 1 -- 172.21.15.17:0/291447980 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd10019070 con 0x7fdd18152820 2024-09-10T21:25:36.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:36.342+0000 7fdd15600640 1 -- 172.21.15.17:0/291447980 <== mon.2 v2:172.21.15.31:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdd1000cc10 con 0x7fdd18152820 2024-09-10T21:25:36.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:36.342+0000 7fdd15600640 1 -- 172.21.15.17:0/291447980 <== mon.2 v2:172.21.15.31:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd100146a0 con 0x7fdd18152820 2024-09-10T21:25:36.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:36.342+0000 7fdd1fe00640 1 -- 172.21.15.17:0/291447980 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd18153e80 con 0x7fdd18152820 2024-09-10T21:25:36.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:36.346+0000 7fdd1fe00640 1 -- 172.21.15.17:0/291447980 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd181eacf0 con 0x7fdd18152820 2024-09-10T21:25:36.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:36.346+0000 7fdd1fe00640 1 -- 172.21.15.17:0/291447980 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd18154270 con 0x7fdd18152820 2024-09-10T21:25:36.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:36.346+0000 7fdd15600640 1 -- 172.21.15.17:0/291447980 <== mon.2 v2:172.21.15.31:3301/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fdd1001e440 con 0x7fdd18152820 2024-09-10T21:25:36.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:36.346+0000 7fdd15600640 1 --2- 172.21.15.17:0/291447980 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fdcf8072590 0x7fdcf8074a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:36.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:36.346+0000 7fdd15600640 1 -- 172.21.15.17:0/291447980 <== mon.2 v2:172.21.15.31:3301/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fdd10093af0 con 0x7fdd18152820 2024-09-10T21:25:36.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:36.346+0000 7fdd1d400640 1 --2- 172.21.15.17:0/291447980 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fdcf8072590 0x7fdcf8074a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:36.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:36.350+0000 7fdd1d400640 1 --2- 172.21.15.17:0/291447980 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fdcf8072590 0x7fdcf8074a50 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fdd0c009720 tx=0x7fdd0c0091e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:36.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:36.350+0000 7fdd15600640 1 -- 172.21.15.17:0/291447980 <== mon.2 v2:172.21.15.31:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fdd18154270 con 0x7fdd18152820 2024-09-10T21:25:36.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:36.590+0000 7fdd1fe00640 1 -- 172.21.15.17:0/291447980 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fdd180c2ff0 con 0x7fdd18152820 2024-09-10T21:25:36.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:36.594+0000 7fdd15600640 1 -- 172.21.15.17:0/291447980 <== mon.2 v2:172.21.15.31:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1556 (secure 0 0 0) 0x7fdd10023020 con 0x7fdd18152820 2024-09-10T21:25:36.596 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":74,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5e2f023f-d6a6-49f4-a256-f4807dce3739","modified":"2024-09-10T21:23:46.892130Z","created":"2024-09-10T21:23:46.892130Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3301","nonce":0},{"type":"v1","addr":"172.21.15.31:6790","nonce":0}]},"addr":"172.21.15.31:6790/0","public_addr":"172.21.15.31:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-10T21:25:36.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:36.594+0000 7fdd1fe00640 1 -- 172.21.15.17:0/291447980 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fdcf8072590 msgr2=0x7fdcf8074a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:36.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:36.594+0000 7fdd1fe00640 1 --2- 172.21.15.17:0/291447980 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fdcf8072590 0x7fdcf8074a50 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fdd0c009720 tx=0x7fdd0c0091e0 comp rx=0 tx=0).stop 2024-09-10T21:25:36.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:36.594+0000 7fdd1fe00640 1 -- 172.21.15.17:0/291447980 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fdd18152820 msgr2=0x7fdd181534b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:36.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:36.594+0000 7fdd1fe00640 1 --2- 172.21.15.17:0/291447980 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fdd18152820 0x7fdd181534b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fdd1000b9e0 tx=0x7fdd1000bea0 comp rx=0 tx=0).stop 2024-09-10T21:25:36.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:36.594+0000 7fdd1fe00640 1 -- 172.21.15.17:0/291447980 shutdown_connections 2024-09-10T21:25:36.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:36.594+0000 7fdd1fe00640 1 --2- 172.21.15.17:0/291447980 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fdcf8072590 0x7fdcf8074a50 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:36.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:36.594+0000 7fdd1fe00640 1 --2- 172.21.15.17:0/291447980 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fdd18152820 0x7fdd181534b0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:36.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:36.594+0000 7fdd1fe00640 1 --2- 172.21.15.17:0/291447980 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdd18152440 0x7fdd18154470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:36.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:36.594+0000 7fdd1fe00640 1 --2- 172.21.15.17:0/291447980 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdd180cdf20 0x7fdd181f5280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:36.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:36.594+0000 7fdd1fe00640 1 -- 172.21.15.17:0/291447980 >> 172.21.15.17:0/291447980 conn(0x7fdd180702f0 msgr2=0x7fdd18159520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:36.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:36.594+0000 7fdd1fe00640 1 -- 172.21.15.17:0/291447980 shutdown_connections 2024-09-10T21:25:36.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:36.594+0000 7fdd1fe00640 1 -- 172.21.15.17:0/291447980 wait complete. 2024-09-10T21:25:39.620 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-10T21:25:39.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:39.766+0000 7f2759400640 1 -- 172.21.15.17:0/3120239330 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2754159130 msgr2=0x7f2754152250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:39.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:39.766+0000 7f2759400640 1 --2- 172.21.15.17:0/3120239330 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2754159130 0x7f2754152250 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f2754072220 tx=0x7f274c01f390 comp rx=0 tx=0).stop 2024-09-10T21:25:39.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:39.766+0000 7f2759400640 1 -- 172.21.15.17:0/3120239330 shutdown_connections 2024-09-10T21:25:39.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:39.766+0000 7f2759400640 1 --2- 172.21.15.17:0/3120239330 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2754159130 0x7f2754152250 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:39.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:39.766+0000 7f2759400640 1 --2- 172.21.15.17:0/3120239330 >> v2:172.21.15.17:3300/0 conn(0x7f2754071750 0x7f27540ca0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:39.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:39.766+0000 7f2759400640 1 -- 172.21.15.17:0/3120239330 >> 172.21.15.17:0/3120239330 conn(0x7f275406fc60 msgr2=0x7f2754068990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:39.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:39.766+0000 7f2759400640 1 -- 172.21.15.17:0/3120239330 shutdown_connections 2024-09-10T21:25:39.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:39.766+0000 7f2759400640 1 -- 172.21.15.17:0/3120239330 wait complete. 2024-09-10T21:25:39.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:39.770+0000 7f2759400640 1 Processor -- start 2024-09-10T21:25:39.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:39.770+0000 7f2759400640 1 -- start start 2024-09-10T21:25:39.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:39.770+0000 7f2759400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2754071370 0x7f27541518d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:39.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:39.770+0000 7f2759400640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f2754071750 0x7f27540afa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:39.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:39.770+0000 7f2759400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2754159130 0x7f27540b08b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:39.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:39.770+0000 7f2759400640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f2754158c10 con 0x7f2754071370 2024-09-10T21:25:39.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:39.770+0000 7f2759400640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f2754158930 con 0x7f2754159130 2024-09-10T21:25:39.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:39.770+0000 7f2759400640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f275406cc20 con 0x7f2754071750 2024-09-10T21:25:39.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:39.770+0000 7f2752800640 1 -- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2754071370 msgr2=0x7f27541518d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-10T21:25:39.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:39.770+0000 7f2752800640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2754071370 0x7f27541518d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:25:39.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:39.770+0000 7f2751e00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f2754071750 0x7f27540afa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:39.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:39.770+0000 7f2753200640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2754159130 0x7f27540b08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:39.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:39.770+0000 7f2751e00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f2754071750 0x7f27540afa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3301/0 says I am v2:172.21.15.17:33252/0 (socket says 172.21.15.17:33252) 2024-09-10T21:25:39.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:39.770+0000 7f2751e00640 1 -- 172.21.15.17:0/4206074501 learned_addr learned my addr 172.21.15.17:0/4206074501 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:25:39.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:39.770+0000 7f2753200640 1 -- 172.21.15.17:0/4206074501 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f2754071750 msgr2=0x7f27540afa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:39.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:39.770+0000 7f2753200640 1 --2- 172.21.15.17:0/4206074501 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f2754071750 0x7f27540afa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:39.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:39.770+0000 7f2753200640 1 -- 172.21.15.17:0/4206074501 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2754071370 msgr2=0x7f27541518d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:25:39.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:39.770+0000 7f2753200640 1 --2- 172.21.15.17:0/4206074501 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2754071370 0x7f27541518d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:39.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:39.770+0000 7f2753200640 1 -- 172.21.15.17:0/4206074501 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27540aff80 con 0x7f2754159130 2024-09-10T21:25:39.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:39.770+0000 7f2751e00640 1 --2- 172.21.15.17:0/4206074501 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f2754071750 0x7f27540afa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:25:39.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:39.770+0000 7f2753200640 1 --2- 172.21.15.17:0/4206074501 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2754159130 0x7f27540b08b0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f274c0af000 tx=0x7f274c0af490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:39.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:39.770+0000 7f274aa00640 1 -- 172.21.15.17:0/4206074501 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f274c0b4040 con 0x7f2754159130 2024-09-10T21:25:39.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:39.770+0000 7f274aa00640 1 -- 172.21.15.17:0/4206074501 <== mon.1 v2:172.21.15.31:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f274c004b40 con 0x7f2754159130 2024-09-10T21:25:39.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:39.770+0000 7f274aa00640 1 -- 172.21.15.17:0/4206074501 <== mon.1 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f274c0b0930 con 0x7f2754159130 2024-09-10T21:25:39.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:39.770+0000 7f2759400640 1 -- 172.21.15.17:0/4206074501 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f275415bcc0 con 0x7f2754159130 2024-09-10T21:25:39.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:39.770+0000 7f2759400640 1 -- 172.21.15.17:0/4206074501 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f275415a7a0 con 0x7f2754159130 2024-09-10T21:25:39.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:39.774+0000 7f2759400640 1 -- 172.21.15.17:0/4206074501 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27540ca650 con 0x7f2754159130 2024-09-10T21:25:39.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:39.774+0000 7f274aa00640 1 -- 172.21.15.17:0/4206074501 <== mon.1 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f274c0e3b80 con 0x7f2754159130 2024-09-10T21:25:39.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:39.774+0000 7f274aa00640 1 --2- 172.21.15.17:0/4206074501 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f2734072590 0x7f2734074a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:39.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:39.774+0000 7f274aa00640 1 -- 172.21.15.17:0/4206074501 <== mon.1 v2:172.21.15.31:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f274c131d90 con 0x7f2754159130 2024-09-10T21:25:39.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:39.774+0000 7f2752800640 1 --2- 172.21.15.17:0/4206074501 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f2734072590 0x7f2734074a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:39.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:39.774+0000 7f2752800640 1 --2- 172.21.15.17:0/4206074501 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f2734072590 0x7f2734074a50 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f2744000c10 tx=0x7f2744007ae0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:39.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:39.778+0000 7f274aa00640 1 -- 172.21.15.17:0/4206074501 <== mon.1 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f274c018ef0 con 0x7f2754159130 2024-09-10T21:25:40.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:40.050+0000 7f2759400640 1 -- 172.21.15.17:0/4206074501 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f27540749b0 con 0x7f2754159130 2024-09-10T21:25:40.055 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:40.050+0000 7f274aa00640 1 -- 172.21.15.17:0/4206074501 <== mon.1 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1556 (secure 0 0 0) 0x7f274c0fd170 con 0x7f2754159130 2024-09-10T21:25:40.055 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":78,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5e2f023f-d6a6-49f4-a256-f4807dce3739","modified":"2024-09-10T21:23:46.892130Z","created":"2024-09-10T21:23:46.892130Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3301","nonce":0},{"type":"v1","addr":"172.21.15.31:6790","nonce":0}]},"addr":"172.21.15.31:6790/0","public_addr":"172.21.15.31:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-10T21:25:40.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:40.054+0000 7f2759400640 1 -- 172.21.15.17:0/4206074501 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f2734072590 msgr2=0x7f2734074a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:40.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:40.054+0000 7f2759400640 1 --2- 172.21.15.17:0/4206074501 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f2734072590 0x7f2734074a50 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f2744000c10 tx=0x7f2744007ae0 comp rx=0 tx=0).stop 2024-09-10T21:25:40.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:40.054+0000 7f2759400640 1 -- 172.21.15.17:0/4206074501 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2754159130 msgr2=0x7f27540b08b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:40.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:40.054+0000 7f2759400640 1 --2- 172.21.15.17:0/4206074501 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2754159130 0x7f27540b08b0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f274c0af000 tx=0x7f274c0af490 comp rx=0 tx=0).stop 2024-09-10T21:25:40.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:40.054+0000 7f2759400640 1 -- 172.21.15.17:0/4206074501 shutdown_connections 2024-09-10T21:25:40.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:40.054+0000 7f2759400640 1 --2- 172.21.15.17:0/4206074501 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f2734072590 0x7f2734074a50 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:40.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:40.054+0000 7f2759400640 1 --2- 172.21.15.17:0/4206074501 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2754159130 0x7f27540b08b0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:40.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:40.054+0000 7f2759400640 1 --2- 172.21.15.17:0/4206074501 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f2754071750 0x7f27540afa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:40.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:40.054+0000 7f2759400640 1 --2- 172.21.15.17:0/4206074501 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2754071370 0x7f27541518d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:40.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:40.054+0000 7f2759400640 1 -- 172.21.15.17:0/4206074501 >> 172.21.15.17:0/4206074501 conn(0x7f275406fc60 msgr2=0x7f275414ddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:40.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:40.054+0000 7f2759400640 1 -- 172.21.15.17:0/4206074501 shutdown_connections 2024-09-10T21:25:40.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:40.054+0000 7f2759400640 1 -- 172.21.15.17:0/4206074501 wait complete. 2024-09-10T21:25:40.549 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a has been restored 2024-09-10T21:25:43.082 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-10T21:25:45.538 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:25:45.541 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:25:49.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.258+0000 7f42d1000640 1 -- 172.21.15.17:0/3642394175 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f42ac00aa50 msgr2=0x7f42ac005b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:49.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.258+0000 7f42d1000640 1 --2- 172.21.15.17:0/3642394175 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f42ac00aa50 0x7f42ac005b30 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f42cc0b3630 tx=0x7f42b801dc30 comp rx=0 tx=0).stop 2024-09-10T21:25:49.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.258+0000 7f42d1000640 1 -- 172.21.15.17:0/3642394175 shutdown_connections 2024-09-10T21:25:49.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.258+0000 7f42d1000640 1 --2- 172.21.15.17:0/3642394175 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f42ac00aa50 0x7f42ac005b30 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:49.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.258+0000 7f42d1000640 1 -- 172.21.15.17:0/3642394175 >> 172.21.15.17:0/3642394175 conn(0x7f42cc0cdef0 msgr2=0x7f42cc0ce2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:49.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.258+0000 7f42d1000640 1 -- 172.21.15.17:0/3642394175 shutdown_connections 2024-09-10T21:25:49.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.258+0000 7f42d1000640 1 -- 172.21.15.17:0/3642394175 wait complete. 2024-09-10T21:25:49.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.258+0000 7f42d1000640 1 Processor -- start 2024-09-10T21:25:49.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.258+0000 7f42d1000640 1 -- start start 2024-09-10T21:25:49.265 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.262+0000 7f42d1000640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f42ac009810 0x7f42cc14ff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:49.266 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.262+0000 7f42d1000640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f42ac009f60 0x7f42cc1f3860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:49.266 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.262+0000 7f42d1000640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f42ac00aa50 0x7f42cc1f3da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:49.266 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.262+0000 7f42d1000640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f42cc0c9350 con 0x7f42ac009810 2024-09-10T21:25:49.266 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.262+0000 7f42d1000640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f42cc0c9070 con 0x7f42ac009f60 2024-09-10T21:25:49.266 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.262+0000 7f42d1000640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f42cc15b9e0 con 0x7f42ac00aa50 2024-09-10T21:25:49.266 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.262+0000 7f42cb400640 1 -- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f42ac009810 msgr2=0x7f42cc14ff60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-10T21:25:49.266 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.262+0000 7f42cb400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f42ac009810 0x7f42cc14ff60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:25:49.266 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.262+0000 7f42caa00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f42ac009f60 0x7f42cc1f3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:49.266 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.262+0000 7f42caa00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f42ac009f60 0x7f42cc1f3860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.17:42378/0 (socket says 172.21.15.17:42378) 2024-09-10T21:25:49.266 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.262+0000 7f42caa00640 1 -- 172.21.15.17:0/2351330055 learned_addr learned my addr 172.21.15.17:0/2351330055 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:25:49.266 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.262+0000 7f42cbe00640 1 --2- 172.21.15.17:0/2351330055 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f42ac00aa50 0x7f42cc1f3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:49.266 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.262+0000 7f42caa00640 1 -- 172.21.15.17:0/2351330055 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f42ac00aa50 msgr2=0x7f42cc1f3da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:49.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.262+0000 7f42caa00640 1 --2- 172.21.15.17:0/2351330055 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f42ac00aa50 0x7f42cc1f3da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:49.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.262+0000 7f42caa00640 1 -- 172.21.15.17:0/2351330055 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f42ac009810 msgr2=0x7f42cc14ff60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:25:49.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.262+0000 7f42caa00640 1 --2- 172.21.15.17:0/2351330055 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f42ac009810 0x7f42cc14ff60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:49.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.262+0000 7f42caa00640 1 -- 172.21.15.17:0/2351330055 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42cc156190 con 0x7f42ac009f60 2024-09-10T21:25:49.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.262+0000 7f42caa00640 1 --2- 172.21.15.17:0/2351330055 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f42ac009f60 0x7f42cc1f3860 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f42b801d6a0 tx=0x7f42b80040c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:49.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.262+0000 7f42c3400640 1 -- 172.21.15.17:0/2351330055 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42b8005220 con 0x7f42ac009f60 2024-09-10T21:25:49.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.262+0000 7f42c3400640 1 -- 172.21.15.17:0/2351330055 <== mon.1 v2:172.21.15.31:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f42b80054b0 con 0x7f42ac009f60 2024-09-10T21:25:49.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.262+0000 7f42c3400640 1 -- 172.21.15.17:0/2351330055 <== mon.1 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42b8004930 con 0x7f42ac009f60 2024-09-10T21:25:49.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.262+0000 7f42d1000640 1 -- 172.21.15.17:0/2351330055 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42cc1f0270 con 0x7f42ac009f60 2024-09-10T21:25:49.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.262+0000 7f42d1000640 1 -- 172.21.15.17:0/2351330055 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42cc1f0810 con 0x7f42ac009f60 2024-09-10T21:25:49.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.262+0000 7f42d1000640 1 -- 172.21.15.17:0/2351330055 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42cc15bf00 con 0x7f42ac009f60 2024-09-10T21:25:49.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.266+0000 7f42c3400640 1 -- 172.21.15.17:0/2351330055 <== mon.1 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f42b8004ad0 con 0x7f42ac009f60 2024-09-10T21:25:49.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.266+0000 7f42c3400640 1 --2- 172.21.15.17:0/2351330055 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f42ac076a90 0x7f42ac078f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:49.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.266+0000 7f42c3400640 1 -- 172.21.15.17:0/2351330055 <== mon.1 v2:172.21.15.31:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f42b812ef30 con 0x7f42ac009f60 2024-09-10T21:25:49.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.266+0000 7f42cb400640 1 --2- 172.21.15.17:0/2351330055 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f42ac076a90 0x7f42ac078f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:49.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.266+0000 7f42cb400640 1 --2- 172.21.15.17:0/2351330055 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f42ac076a90 0x7f42ac078f50 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f42bc001330 tx=0x7f42bc007910 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:49.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.270+0000 7f42c3400640 1 -- 172.21.15.17:0/2351330055 <== mon.1 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f42b8017940 con 0x7f42ac009f60 2024-09-10T21:25:49.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.522+0000 7f42d1000640 1 -- 172.21.15.17:0/2351330055 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f42cc0cb8c0 con 0x7f42ac009f60 2024-09-10T21:25:49.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.522+0000 7f42c3400640 1 -- 172.21.15.17:0/2351330055 <== mon.1 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1549 (secure 0 0 0) 0x7f42b80f9bb0 con 0x7f42ac009f60 2024-09-10T21:25:49.528 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":6,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":1,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5e2f023f-d6a6-49f4-a256-f4807dce3739","modified":"2024-09-10T21:23:46.892130Z","created":"2024-09-10T21:23:46.892130Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3301","nonce":0},{"type":"v1","addr":"172.21.15.31:6790","nonce":0}]},"addr":"172.21.15.31:6790/0","public_addr":"172.21.15.31:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-10T21:25:49.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.526+0000 7f42d1000640 1 -- 172.21.15.17:0/2351330055 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f42ac076a90 msgr2=0x7f42ac078f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:49.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.526+0000 7f42d1000640 1 --2- 172.21.15.17:0/2351330055 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f42ac076a90 0x7f42ac078f50 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f42bc001330 tx=0x7f42bc007910 comp rx=0 tx=0).stop 2024-09-10T21:25:49.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.526+0000 7f42d1000640 1 -- 172.21.15.17:0/2351330055 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f42ac009f60 msgr2=0x7f42cc1f3860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:49.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.526+0000 7f42d1000640 1 --2- 172.21.15.17:0/2351330055 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f42ac009f60 0x7f42cc1f3860 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f42b801d6a0 tx=0x7f42b80040c0 comp rx=0 tx=0).stop 2024-09-10T21:25:49.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.526+0000 7f42d1000640 1 -- 172.21.15.17:0/2351330055 shutdown_connections 2024-09-10T21:25:49.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.526+0000 7f42d1000640 1 --2- 172.21.15.17:0/2351330055 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f42ac076a90 0x7f42ac078f50 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:49.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.526+0000 7f42d1000640 1 --2- 172.21.15.17:0/2351330055 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f42ac00aa50 0x7f42cc1f3da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:49.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.526+0000 7f42d1000640 1 --2- 172.21.15.17:0/2351330055 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f42ac009f60 0x7f42cc1f3860 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:49.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.526+0000 7f42d1000640 1 --2- 172.21.15.17:0/2351330055 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f42ac009810 0x7f42cc14ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:49.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.526+0000 7f42d1000640 1 -- 172.21.15.17:0/2351330055 >> 172.21.15.17:0/2351330055 conn(0x7f42cc0cdef0 msgr2=0x7f42cc0ca6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:49.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.526+0000 7f42d1000640 1 -- 172.21.15.17:0/2351330055 shutdown_connections 2024-09-10T21:25:49.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.526+0000 7f42d1000640 1 -- 172.21.15.17:0/2351330055 wait complete. 2024-09-10T21:25:49.549 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2024-09-10T21:25:49.549 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon b 2024-09-10T21:25:49.550 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-10T21:25:49.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.702+0000 7f9625e00640 1 -- 172.21.15.17:0/200245606 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f96200c2f30 msgr2=0x7f96200ca3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:49.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.702+0000 7f9625e00640 1 --2- 172.21.15.17:0/200245606 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f96200c2f30 0x7f96200ca3f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f9618009bc0 tx=0x7f961801f3b0 comp rx=0 tx=0).stop 2024-09-10T21:25:49.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.702+0000 7f9625e00640 1 -- 172.21.15.17:0/200245606 shutdown_connections 2024-09-10T21:25:49.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.702+0000 7f9625e00640 1 --2- 172.21.15.17:0/200245606 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f96200c2f30 0x7f96200ca3f0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:49.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.702+0000 7f9625e00640 1 -- 172.21.15.17:0/200245606 >> 172.21.15.17:0/200245606 conn(0x7f96200cd690 msgr2=0x7f96200c98e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:49.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.702+0000 7f9625e00640 1 -- 172.21.15.17:0/200245606 shutdown_connections 2024-09-10T21:25:49.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.702+0000 7f9625e00640 1 -- 172.21.15.17:0/200245606 wait complete. 2024-09-10T21:25:49.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.702+0000 7f9625e00640 1 Processor -- start 2024-09-10T21:25:49.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.702+0000 7f9625e00640 1 -- start start 2024-09-10T21:25:49.709 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.706+0000 7f9625e00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f96200c2f30 0x7f96201f4190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:49.709 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.706+0000 7f9625e00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f96200cabe0 0x7f96201ddd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:49.709 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.706+0000 7f9625e00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f9620151450 0x7f96201de270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:49.709 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.706+0000 7f9625e00640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f96200c2710 con 0x7f96200cabe0 2024-09-10T21:25:49.709 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.706+0000 7f9625e00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f96200c2a10 con 0x7f96200c2f30 2024-09-10T21:25:49.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.706+0000 7f9625e00640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f9620151830 con 0x7f9620151450 2024-09-10T21:25:49.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.706+0000 7f961fe00640 1 -- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f96200cabe0 msgr2=0x7f96201ddd30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-10T21:25:49.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.706+0000 7f961fe00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f96200cabe0 0x7f96201ddd30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:25:49.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.706+0000 7f9624a00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f96200c2f30 0x7f96201f4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:49.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.706+0000 7f9624a00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f96200c2f30 0x7f96201f4190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.17:42396/0 (socket says 172.21.15.17:42396) 2024-09-10T21:25:49.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.706+0000 7f9624a00640 1 -- 172.21.15.17:0/1898728511 learned_addr learned my addr 172.21.15.17:0/1898728511 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:25:49.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.706+0000 7f9625400640 1 --2- 172.21.15.17:0/1898728511 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f9620151450 0x7f96201de270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:49.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.706+0000 7f9624a00640 1 -- 172.21.15.17:0/1898728511 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f9620151450 msgr2=0x7f96201de270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:49.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.706+0000 7f9624a00640 1 --2- 172.21.15.17:0/1898728511 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f9620151450 0x7f96201de270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:49.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.706+0000 7f9624a00640 1 -- 172.21.15.17:0/1898728511 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f96200cabe0 msgr2=0x7f96201ddd30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:25:49.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.706+0000 7f9624a00640 1 --2- 172.21.15.17:0/1898728511 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f96200cabe0 0x7f96201ddd30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:49.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.706+0000 7f9624a00640 1 -- 172.21.15.17:0/1898728511 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f962014f190 con 0x7f96200c2f30 2024-09-10T21:25:49.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.706+0000 7f9624a00640 1 --2- 172.21.15.17:0/1898728511 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f96200c2f30 0x7f96201f4190 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f961400b580 tx=0x7f961400ba40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:49.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.706+0000 7f961cc00640 1 -- 172.21.15.17:0/1898728511 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f961400f850 con 0x7f96200c2f30 2024-09-10T21:25:49.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.706+0000 7f961cc00640 1 -- 172.21.15.17:0/1898728511 <== mon.1 v2:172.21.15.31:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f961400fb00 con 0x7f96200c2f30 2024-09-10T21:25:49.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.706+0000 7f961cc00640 1 -- 172.21.15.17:0/1898728511 <== mon.1 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9614017400 con 0x7f96200c2f30 2024-09-10T21:25:49.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.706+0000 7f9625e00640 1 -- 172.21.15.17:0/1898728511 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96201f4bf0 con 0x7f96200c2f30 2024-09-10T21:25:49.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.706+0000 7f9625e00640 1 -- 172.21.15.17:0/1898728511 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f962014f400 con 0x7f96200c2f30 2024-09-10T21:25:49.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.706+0000 7f9625e00640 1 --2- 172.21.15.17:0/1898728511 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f96201f46d0 0x7f962014a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:49.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.706+0000 7f9625e00640 1 -- 172.21.15.17:0/1898728511 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f9620069ef0 con 0x7f96201f46d0 2024-09-10T21:25:49.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.706+0000 7f961fe00640 1 --2- 172.21.15.17:0/1898728511 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f96201f46d0 0x7f962014a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:49.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.706+0000 7f961fe00640 1 -- 172.21.15.17:0/1898728511 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f96200c89b0 con 0x7f96201f46d0 2024-09-10T21:25:49.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.710+0000 7f961fe00640 1 --2- 172.21.15.17:0/1898728511 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f96201f46d0 0x7f962014a4f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f962014d520 tx=0x7f960c00c8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:49.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.710+0000 7f961cc00640 1 -- 172.21.15.17:0/1898728511 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f960c017070 con 0x7f96201f46d0 2024-09-10T21:25:49.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.710+0000 7f961cc00640 1 -- 172.21.15.17:0/1898728511 <== mon.1 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f961400d3e0 con 0x7f96200c2f30 2024-09-10T21:25:49.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.710+0000 7f961cc00640 1 --2- 172.21.15.17:0/1898728511 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f9600072510 0x7f96000749d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:49.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.710+0000 7f961cc00640 1 -- 172.21.15.17:0/1898728511 <== mon.1 v2:172.21.15.31:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f96140936b0 con 0x7f96200c2f30 2024-09-10T21:25:49.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.710+0000 7f961cc00640 1 -- 172.21.15.17:0/1898728511 <== mon.1 v2:172.21.15.31:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f960c005ca0 con 0x7f96201f46d0 2024-09-10T21:25:49.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.710+0000 7f961cc00640 1 -- 172.21.15.17:0/1898728511 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f96201f46d0 msgr2=0x7f962014a4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:49.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.710+0000 7f961cc00640 1 --2- 172.21.15.17:0/1898728511 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f96201f46d0 0x7f962014a4f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f962014d520 tx=0x7f960c00c8e0 comp rx=0 tx=0).stop 2024-09-10T21:25:49.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.710+0000 7f961cc00640 1 -- 172.21.15.17:0/1898728511 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f96201f46d0 msgr2=0x7f962014a4f0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:25:49.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.710+0000 7f961cc00640 1 --2- 172.21.15.17:0/1898728511 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f96201f46d0 0x7f962014a4f0 secure :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f962014d520 tx=0x7f960c00c8e0 comp rx=0 tx=0).stop 2024-09-10T21:25:49.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.710+0000 7f9625400640 1 --2- 172.21.15.17:0/1898728511 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f9600072510 0x7f96000749d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:49.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.710+0000 7f9625400640 1 --2- 172.21.15.17:0/1898728511 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f9600072510 0x7f96000749d0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f96180b1000 tx=0x7f961801dbf0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:49.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.718+0000 7f9625e00640 1 --2- 172.21.15.17:0/1898728511 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f962014aa30 0x7f96201f09c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:49.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.718+0000 7f9625e00640 1 -- 172.21.15.17:0/1898728511 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f96200c8660 con 0x7f962014aa30 2024-09-10T21:25:49.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.718+0000 7f961fe00640 1 --2- 172.21.15.17:0/1898728511 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f962014aa30 0x7f96201f09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:49.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.718+0000 7f961fe00640 1 -- 172.21.15.17:0/1898728511 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f96200cb670 con 0x7f962014aa30 2024-09-10T21:25:49.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.718+0000 7f961fe00640 1 --2- 172.21.15.17:0/1898728511 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f962014aa30 0x7f96201f09c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f960c000f70 tx=0x7f960c000ee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:49.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.718+0000 7f961cc00640 1 -- 172.21.15.17:0/1898728511 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f960c012600 con 0x7f962014aa30 2024-09-10T21:25:49.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.718+0000 7f961cc00640 1 -- 172.21.15.17:0/1898728511 <== mon.1 v2:172.21.15.31:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5031 (secure 0 0 0) 0x7f960c005e50 con 0x7f962014aa30 2024-09-10T21:25:49.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.718+0000 7f961cc00640 1 -- 172.21.15.17:0/1898728511 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f962014aa30 msgr2=0x7f96201f09c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:49.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.718+0000 7f961cc00640 1 --2- 172.21.15.17:0/1898728511 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f962014aa30 0x7f96201f09c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f960c000f70 tx=0x7f960c000ee0 comp rx=0 tx=0).stop 2024-09-10T21:25:49.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.718+0000 7f961cc00640 1 -- 172.21.15.17:0/1898728511 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f962014aa30 msgr2=0x7f96201f09c0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:25:49.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.718+0000 7f961cc00640 1 --2- 172.21.15.17:0/1898728511 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f962014aa30 0x7f96201f09c0 secure :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f960c000f70 tx=0x7f960c000ee0 comp rx=0 tx=0).stop 2024-09-10T21:25:49.723 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-10T21:25:49.723 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:25:49.723 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:25:49.723 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "leader", 2024-09-10T21:25:49.724 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 6, 2024-09-10T21:25:49.724 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 88164, 2024-09-10T21:25:49.724 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-10T21:25:49.724 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-10T21:25:49.724 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-10T21:25:49.724 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:25:49.724 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 2, 2024-09-10T21:25:49.724 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:25:49.724 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-10T21:25:49.724 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-10T21:25:49.724 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:25:49.724 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:25:49.724 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:25:49.725 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:25:49.725 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:25:49.725 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:25:49.725 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:25:49.725 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:25:49.725 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:25:49.725 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:25:49.725 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:25:49.725 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:25:49.725 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-10T21:25:49.725 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-10T21:25:49.725 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:25:49.725 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:25:49.726 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:25:49.726 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:25:49.726 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:25:49.726 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:25:49.726 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:25:49.726 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:25:49.726 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:25:49.726 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:25:49.726 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:25:49.726 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:25:49.726 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:25:49.726 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-10T21:25:49.726 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-10T21:25:49.726 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-10T21:25:49.727 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-10T21:25:49.727 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-10T21:25:49.727 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "5e2f023f-d6a6-49f4-a256-f4807dce3739", 2024-09-10T21:25:49.727 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:25:49.727 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:25:49.727 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-10T21:25:49.727 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-10T21:25:49.727 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-10T21:25:49.727 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-10T21:25:49.727 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-10T21:25:49.727 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-10T21:25:49.727 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-10T21:25:49.727 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:25:49.727 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-10T21:25:49.728 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:25:49.728 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:25:49.728 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:25:49.728 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:25:49.728 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:25:49.728 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:25:49.728 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:25:49.728 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:25:49.728 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:25:49.728 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:25:49.728 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:25:49.728 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:25:49.728 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-10T21:25:49.729 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:25:49.729 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-10T21:25:49.729 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:25:49.729 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:25:49.729 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:25:49.729 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:25:49.729 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:25:49.729 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:25:49.729 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:25:49.729 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-10T21:25:49.729 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:25:49.729 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:25:49.729 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:25:49.729 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:25:49.730 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-10T21:25:49.730 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:25:49.730 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:25:49.730 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:25:49.730 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:25:49.730 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-10T21:25:49.730 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-10T21:25:49.730 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:25:49.730 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:25:49.730 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:25:49.730 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:25:49.730 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:25:49.730 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:25:49.730 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:25:49.731 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:25:49.731 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:25:49.731 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:25:49.731 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:25:49.731 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3300", 2024-09-10T21:25:49.731 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:25:49.731 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:25:49.731 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:25:49.731 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:25:49.731 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789", 2024-09-10T21:25:49.731 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:25:49.731 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:25:49.731 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:25:49.731 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:25:49.732 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789/0", 2024-09-10T21:25:49.732 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6789/0", 2024-09-10T21:25:49.732 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:25:49.732 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:25:49.732 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:25:49.732 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:25:49.732 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:25:49.732 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:25:49.732 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:25:49.732 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:25:49.733 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:25:49.733 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:25:49.733 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:25:49.733 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3301", 2024-09-10T21:25:49.733 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:25:49.733 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:25:49.733 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:25:49.733 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:25:49.733 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790", 2024-09-10T21:25:49.733 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:25:49.733 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:25:49.733 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:25:49.734 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:25:49.734 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790/0", 2024-09-10T21:25:49.734 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6790/0", 2024-09-10T21:25:49.734 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:25:49.734 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:25:49.734 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:25:49.734 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:25:49.734 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:25:49.734 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:25:49.734 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-10T21:25:49.735 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-10T21:25:49.735 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:25:49.735 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:25:49.735 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:25:49.735 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:25:49.735 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:25:49.735 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:25:49.735 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": [ 2024-09-10T21:25:49.735 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:25:49.735 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:25:49.735 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:25:49.736 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 5 2024-09-10T21:25:49.736 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:25:49.736 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:25:49.736 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-10T21:25:49.736 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:25:49.736 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:25:49.736 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:25:49.736 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 4 2024-09-10T21:25:49.736 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:25:49.736 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:25:49.736 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-10T21:25:49.737 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:25:49.737 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:25:49.737 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:25:49.737 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 3 2024-09-10T21:25:49.737 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:25:49.737 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:25:49.737 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": [ 2024-09-10T21:25:49.737 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:25:49.737 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:25:49.737 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:25:49.738 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:25:49.738 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:25:49.738 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:25:49.738 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:25:49.738 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-10T21:25:49.738 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-10T21:25:49.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.718+0000 7f9625e00640 1 -- 172.21.15.17:0/1898728511 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f9600072510 msgr2=0x7f96000749d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:49.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.718+0000 7f9625e00640 1 --2- 172.21.15.17:0/1898728511 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f9600072510 0x7f96000749d0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f96180b1000 tx=0x7f961801dbf0 comp rx=0 tx=0).stop 2024-09-10T21:25:49.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.718+0000 7f9625e00640 1 -- 172.21.15.17:0/1898728511 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f96200c2f30 msgr2=0x7f96201f4190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:49.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.718+0000 7f9625e00640 1 --2- 172.21.15.17:0/1898728511 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f96200c2f30 0x7f96201f4190 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f961400b580 tx=0x7f961400ba40 comp rx=0 tx=0).stop 2024-09-10T21:25:49.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.718+0000 7f9625400640 1 -- 172.21.15.17:0/1898728511 reap_dead start 2024-09-10T21:25:49.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.718+0000 7f9625400640 1 -- 172.21.15.17:0/1898728511 reap_dead start 2024-09-10T21:25:49.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.718+0000 7f9625e00640 1 -- 172.21.15.17:0/1898728511 shutdown_connections 2024-09-10T21:25:49.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.718+0000 7f9625e00640 1 -- 172.21.15.17:0/1898728511 >> 172.21.15.17:0/1898728511 conn(0x7f96200cd690 msgr2=0x7f96200cc280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:49.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.718+0000 7f9625e00640 1 -- 172.21.15.17:0/1898728511 shutdown_connections 2024-09-10T21:25:49.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.718+0000 7f9625e00640 1 -- 172.21.15.17:0/1898728511 wait complete. 2024-09-10T21:25:49.743 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-09-10T21:25:49.743 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon c 2024-09-10T21:25:49.743 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-10T21:25:49.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.898+0000 7f5168c00640 1 -- 172.21.15.17:0/3531609857 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5164150ad0 msgr2=0x7f5164070fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:49.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.898+0000 7f5168c00640 1 --2- 172.21.15.17:0/3531609857 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5164150ad0 0x7f5164070fd0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f515400aa10 tx=0x7f515401dae0 comp rx=0 tx=0).stop 2024-09-10T21:25:49.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.898+0000 7f5168c00640 1 -- 172.21.15.17:0/3531609857 shutdown_connections 2024-09-10T21:25:49.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.898+0000 7f5168c00640 1 --2- 172.21.15.17:0/3531609857 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f51641568a0 0x7f5164156d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:49.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.898+0000 7f5168c00640 1 --2- 172.21.15.17:0/3531609857 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5164150ad0 0x7f5164070fd0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:49.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.898+0000 7f5168c00640 1 -- 172.21.15.17:0/3531609857 >> 172.21.15.17:0/3531609857 conn(0x7f516406cc20 msgr2=0x7f516406d030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:49.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.898+0000 7f5168c00640 1 -- 172.21.15.17:0/3531609857 shutdown_connections 2024-09-10T21:25:49.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.898+0000 7f5168c00640 1 -- 172.21.15.17:0/3531609857 wait complete. 2024-09-10T21:25:49.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.898+0000 7f5168c00640 1 Processor -- start 2024-09-10T21:25:49.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.898+0000 7f5168c00640 1 -- start start 2024-09-10T21:25:49.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.902+0000 7f5168c00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f51640c78b0 0x7f51641f1260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:49.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.902+0000 7f5168c00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f5164150ad0 0x7f516414e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:49.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.902+0000 7f5168c00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f51641568a0 0x7f5164145c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:49.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.902+0000 7f5168c00640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f5164151030 con 0x7f51640c78b0 2024-09-10T21:25:49.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.902+0000 7f5168c00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f5164150eb0 con 0x7f51641568a0 2024-09-10T21:25:49.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.902+0000 7f5168c00640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f51640c8710 con 0x7f5164150ad0 2024-09-10T21:25:49.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.902+0000 7f5163400640 1 -- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f51640c78b0 msgr2=0x7f51641f1260 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-10T21:25:49.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.902+0000 7f5163400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f51640c78b0 0x7f51641f1260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:25:49.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.902+0000 7f5162a00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f5164150ad0 0x7f516414e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:49.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.902+0000 7f5163e00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f51641568a0 0x7f5164145c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:49.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.902+0000 7f5162a00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f5164150ad0 0x7f516414e0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3301/0 says I am v2:172.21.15.17:44980/0 (socket says 172.21.15.17:44980) 2024-09-10T21:25:49.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.902+0000 7f5162a00640 1 -- 172.21.15.17:0/1714500178 learned_addr learned my addr 172.21.15.17:0/1714500178 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:25:49.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.902+0000 7f5162a00640 1 -- 172.21.15.17:0/1714500178 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f51641568a0 msgr2=0x7f5164145c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:49.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.902+0000 7f5162a00640 1 --2- 172.21.15.17:0/1714500178 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f51641568a0 0x7f5164145c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:49.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.902+0000 7f5162a00640 1 -- 172.21.15.17:0/1714500178 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f51640c78b0 msgr2=0x7f51641f1260 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:25:49.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.902+0000 7f5162a00640 1 --2- 172.21.15.17:0/1714500178 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f51640c78b0 0x7f51641f1260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:25:49.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.902+0000 7f5162a00640 1 -- 172.21.15.17:0/1714500178 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f516414e790 con 0x7f5164150ad0 2024-09-10T21:25:49.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.902+0000 7f5163e00640 1 --2- 172.21.15.17:0/1714500178 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f51641568a0 0x7f5164145c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:25:49.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.902+0000 7f5162a00640 1 --2- 172.21.15.17:0/1714500178 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f5164150ad0 0x7f516414e0d0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f515000b4d0 tx=0x7f515000b990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:49.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.902+0000 7f515b400640 1 -- 172.21.15.17:0/1714500178 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5150014070 con 0x7f5164150ad0 2024-09-10T21:25:49.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.902+0000 7f515b400640 1 -- 172.21.15.17:0/1714500178 <== mon.2 v2:172.21.15.31:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5150010980 con 0x7f5164150ad0 2024-09-10T21:25:49.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.902+0000 7f515b400640 1 -- 172.21.15.17:0/1714500178 <== mon.2 v2:172.21.15.31:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5150010dd0 con 0x7f5164150ad0 2024-09-10T21:25:49.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.902+0000 7f5168c00640 1 -- 172.21.15.17:0/1714500178 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51640c64a0 con 0x7f5164150ad0 2024-09-10T21:25:49.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.902+0000 7f5168c00640 1 -- 172.21.15.17:0/1714500178 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f516414d430 con 0x7f5164150ad0 2024-09-10T21:25:49.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.902+0000 7f5168c00640 1 --2- 172.21.15.17:0/1714500178 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f516414fda0 0x7f5164143cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:49.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.902+0000 7f5168c00640 1 -- 172.21.15.17:0/1714500178 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f516414df20 con 0x7f516414fda0 2024-09-10T21:25:49.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.906+0000 7f5163400640 1 --2- 172.21.15.17:0/1714500178 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f516414fda0 0x7f5164143cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:49.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.906+0000 7f5163400640 1 -- 172.21.15.17:0/1714500178 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5164070fd0 con 0x7f516414fda0 2024-09-10T21:25:49.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.906+0000 7f5163400640 1 --2- 172.21.15.17:0/1714500178 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f516414fda0 0x7f5164143cf0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f516406db00 tx=0x7f5154005be0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:49.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.906+0000 7f515b400640 1 -- 172.21.15.17:0/1714500178 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51540bb020 con 0x7f516414fda0 2024-09-10T21:25:49.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.906+0000 7f515b400640 1 -- 172.21.15.17:0/1714500178 <== mon.2 v2:172.21.15.31:3301/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f515000f3c0 con 0x7f5164150ad0 2024-09-10T21:25:49.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.906+0000 7f515b400640 1 --2- 172.21.15.17:0/1714500178 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f5144072510 0x7f51440749d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:49.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.906+0000 7f515b400640 1 -- 172.21.15.17:0/1714500178 <== mon.2 v2:172.21.15.31:3301/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f51500936d0 con 0x7f5164150ad0 2024-09-10T21:25:49.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.906+0000 7f515b400640 1 -- 172.21.15.17:0/1714500178 <== mon.2 v2:172.21.15.31:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f515400a630 con 0x7f516414fda0 2024-09-10T21:25:49.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.906+0000 7f515b400640 1 -- 172.21.15.17:0/1714500178 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f516414fda0 msgr2=0x7f5164143cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:49.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.906+0000 7f515b400640 1 --2- 172.21.15.17:0/1714500178 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f516414fda0 0x7f5164143cf0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f516406db00 tx=0x7f5154005be0 comp rx=0 tx=0).stop 2024-09-10T21:25:49.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.906+0000 7f515b400640 1 -- 172.21.15.17:0/1714500178 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f516414fda0 msgr2=0x7f5164143cf0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:25:49.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.906+0000 7f515b400640 1 --2- 172.21.15.17:0/1714500178 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f516414fda0 0x7f5164143cf0 secure :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f516406db00 tx=0x7f5154005be0 comp rx=0 tx=0).stop 2024-09-10T21:25:49.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.906+0000 7f5163e00640 1 --2- 172.21.15.17:0/1714500178 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f5144072510 0x7f51440749d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:49.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.906+0000 7f5163e00640 1 --2- 172.21.15.17:0/1714500178 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f5144072510 0x7f51440749d0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f515c0116f0 tx=0x7f515c009500 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:49.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.914+0000 7f5168c00640 1 --2- 172.21.15.17:0/1714500178 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f51640c6670 0x7f5164144230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:25:49.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.914+0000 7f5168c00640 1 -- 172.21.15.17:0/1714500178 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f51640c3470 con 0x7f51640c6670 2024-09-10T21:25:49.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.914+0000 7f5163400640 1 --2- 172.21.15.17:0/1714500178 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f51640c6670 0x7f5164144230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:25:49.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.914+0000 7f5163400640 1 -- 172.21.15.17:0/1714500178 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f516406d1a0 con 0x7f51640c6670 2024-09-10T21:25:49.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.914+0000 7f5163400640 1 --2- 172.21.15.17:0/1714500178 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f51640c6670 0x7f5164144230 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f515400aa10 tx=0x7f515401dae0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:25:49.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.914+0000 7f515b400640 1 -- 172.21.15.17:0/1714500178 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51540b3040 con 0x7f51640c6670 2024-09-10T21:25:49.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.914+0000 7f515b400640 1 -- 172.21.15.17:0/1714500178 <== mon.2 v2:172.21.15.31:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5029 (secure 0 0 0) 0x7f5154003600 con 0x7f51640c6670 2024-09-10T21:25:49.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.914+0000 7f515b400640 1 -- 172.21.15.17:0/1714500178 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f51640c6670 msgr2=0x7f5164144230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:49.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.914+0000 7f515b400640 1 --2- 172.21.15.17:0/1714500178 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f51640c6670 0x7f5164144230 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f515400aa10 tx=0x7f515401dae0 comp rx=0 tx=0).stop 2024-09-10T21:25:49.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.914+0000 7f515b400640 1 -- 172.21.15.17:0/1714500178 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f51640c6670 msgr2=0x7f5164144230 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:25:49.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.914+0000 7f515b400640 1 --2- 172.21.15.17:0/1714500178 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f51640c6670 0x7f5164144230 secure :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f515400aa10 tx=0x7f515401dae0 comp rx=0 tx=0).stop 2024-09-10T21:25:49.920 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-10T21:25:49.920 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:25:49.920 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:25:49.920 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-10T21:25:49.920 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 6, 2024-09-10T21:25:49.920 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 88360, 2024-09-10T21:25:49.920 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-10T21:25:49.920 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-10T21:25:49.920 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-10T21:25:49.920 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:25:49.920 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 2, 2024-09-10T21:25:49.921 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:25:49.921 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-10T21:25:49.921 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-10T21:25:49.921 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:25:49.921 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:25:49.921 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:25:49.921 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:25:49.921 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:25:49.921 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:25:49.921 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:25:49.921 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:25:49.921 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:25:49.921 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:25:49.922 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:25:49.922 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:25:49.922 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-10T21:25:49.922 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-10T21:25:49.922 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:25:49.922 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:25:49.922 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:25:49.922 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:25:49.922 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:25:49.922 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:25:49.922 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:25:49.922 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:25:49.922 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:25:49.922 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:25:49.923 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:25:49.923 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:25:49.923 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:25:49.923 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-10T21:25:49.923 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-10T21:25:49.923 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-10T21:25:49.923 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-10T21:25:49.923 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-10T21:25:49.923 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "5e2f023f-d6a6-49f4-a256-f4807dce3739", 2024-09-10T21:25:49.923 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:25:49.923 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:25:49.923 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-10T21:25:49.923 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-10T21:25:49.923 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-10T21:25:49.924 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-10T21:25:49.924 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-10T21:25:49.924 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-10T21:25:49.924 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-10T21:25:49.924 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:25:49.924 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-10T21:25:49.924 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:25:49.924 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:25:49.924 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:25:49.924 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:25:49.924 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:25:49.924 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:25:49.924 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:25:49.925 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:25:49.925 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:25:49.925 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:25:49.925 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:25:49.925 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:25:49.925 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-10T21:25:49.925 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:25:49.925 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-10T21:25:49.925 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:25:49.925 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:25:49.925 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:25:49.925 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:25:49.925 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:25:49.925 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:25:49.926 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:25:49.926 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-10T21:25:49.926 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:25:49.926 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:25:49.926 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:25:49.926 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:25:49.926 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-10T21:25:49.926 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:25:49.926 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:25:49.926 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:25:49.926 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:25:49.926 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-10T21:25:49.926 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-10T21:25:49.927 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:25:49.927 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:25:49.927 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:25:49.927 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:25:49.927 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:25:49.927 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:25:49.927 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:25:49.927 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:25:49.927 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:25:49.927 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:25:49.927 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:25:49.927 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3300", 2024-09-10T21:25:49.927 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:25:49.928 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:25:49.928 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:25:49.928 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:25:49.928 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789", 2024-09-10T21:25:49.928 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:25:49.928 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:25:49.928 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:25:49.928 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:25:49.928 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789/0", 2024-09-10T21:25:49.928 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6789/0", 2024-09-10T21:25:49.928 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:25:49.928 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:25:49.928 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:25:49.928 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:25:49.928 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:25:49.929 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:25:49.929 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:25:49.929 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:25:49.929 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:25:49.929 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:25:49.929 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:25:49.929 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3301", 2024-09-10T21:25:49.929 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:25:49.929 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:25:49.929 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:25:49.929 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:25:49.929 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790", 2024-09-10T21:25:49.929 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:25:49.930 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:25:49.930 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:25:49.930 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:25:49.930 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790/0", 2024-09-10T21:25:49.930 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6790/0", 2024-09-10T21:25:49.930 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:25:49.930 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:25:49.930 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:25:49.930 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:25:49.930 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:25:49.930 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:25:49.930 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-10T21:25:49.930 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-10T21:25:49.930 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:25:49.931 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:25:49.931 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:25:49.931 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:25:49.931 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:25:49.931 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:25:49.931 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": [ 2024-09-10T21:25:49.931 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:25:49.931 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:25:49.931 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:25:49.931 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:25:49.931 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:25:49.931 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:25:49.931 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-10T21:25:49.932 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:25:49.932 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:25:49.932 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:25:49.932 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 4 2024-09-10T21:25:49.932 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:25:49.932 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:25:49.932 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-10T21:25:49.932 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:25:49.932 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:25:49.932 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:25:49.932 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 6 2024-09-10T21:25:49.932 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:25:49.932 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:25:49.933 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": [ 2024-09-10T21:25:49.933 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:25:49.933 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:25:49.933 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:25:49.933 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:25:49.933 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:25:49.933 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:25:49.933 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:25:49.933 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-10T21:25:49.933 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-10T21:25:49.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.914+0000 7f5168c00640 1 -- 172.21.15.17:0/1714500178 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f5144072510 msgr2=0x7f51440749d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:49.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.914+0000 7f5168c00640 1 --2- 172.21.15.17:0/1714500178 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f5144072510 0x7f51440749d0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f515c0116f0 tx=0x7f515c009500 comp rx=0 tx=0).stop 2024-09-10T21:25:49.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.914+0000 7f5168c00640 1 -- 172.21.15.17:0/1714500178 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f5164150ad0 msgr2=0x7f516414e0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:25:49.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.914+0000 7f5168c00640 1 --2- 172.21.15.17:0/1714500178 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f5164150ad0 0x7f516414e0d0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f515000b4d0 tx=0x7f515000b990 comp rx=0 tx=0).stop 2024-09-10T21:25:49.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.914+0000 7f5163e00640 1 -- 172.21.15.17:0/1714500178 reap_dead start 2024-09-10T21:25:49.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.914+0000 7f5163e00640 1 -- 172.21.15.17:0/1714500178 reap_dead start 2024-09-10T21:25:49.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.914+0000 7f5168c00640 1 -- 172.21.15.17:0/1714500178 shutdown_connections 2024-09-10T21:25:49.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.914+0000 7f5168c00640 1 -- 172.21.15.17:0/1714500178 >> 172.21.15.17:0/1714500178 conn(0x7f516406cc20 msgr2=0x7f5164073550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:25:49.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.914+0000 7f5168c00640 1 -- 172.21.15.17:0/1714500178 shutdown_connections 2024-09-10T21:25:49.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:25:49.914+0000 7f5168c00640 1 -- 172.21.15.17:0/1714500178 wait complete. 2024-09-10T21:25:49.939 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-09-10T21:25:49.939 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2024-09-10T21:26:09.940 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.a 2024-09-10T21:26:09.941 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-10T21:26:09.941 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-10T21:26:09.945 INFO:tasks.ceph.mon.a:Started 2024-09-10T21:26:09.945 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-09-10T21:26:09.945 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-10T21:26:10.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:10.170+0000 7f8123e00640 1 -- 172.21.15.17:0/2854404392 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f812406f860 msgr2=0x7f81240c7870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:10.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:10.170+0000 7f8123e00640 1 --2- 172.21.15.17:0/2854404392 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f812406f860 0x7f81240c7870 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f81240c3430 tx=0x7f811001c800 comp rx=0 tx=0).stop 2024-09-10T21:26:10.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:10.170+0000 7f8123e00640 1 -- 172.21.15.17:0/2854404392 shutdown_connections 2024-09-10T21:26:10.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:10.170+0000 7f8123e00640 1 --2- 172.21.15.17:0/2854404392 >> v2:172.21.15.17:3300/0 conn(0x7f81240c8070 0x7f81240b2c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:10.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:10.170+0000 7f8123e00640 1 --2- 172.21.15.17:0/2854404392 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f812406f860 0x7f81240c7870 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:10.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:10.170+0000 7f8123e00640 1 -- 172.21.15.17:0/2854404392 >> 172.21.15.17:0/2854404392 conn(0x7f8124070ca0 msgr2=0x7f81240710b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:10.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:10.170+0000 7f8123e00640 1 -- 172.21.15.17:0/2854404392 shutdown_connections 2024-09-10T21:26:10.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:10.174+0000 7f8123e00640 1 -- 172.21.15.17:0/2854404392 wait complete. 2024-09-10T21:26:10.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:10.174+0000 7f8123e00640 1 Processor -- start 2024-09-10T21:26:10.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:10.174+0000 7f8123e00640 1 -- start start 2024-09-10T21:26:10.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:10.174+0000 7f8123e00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f812406f480 0x7f8124140530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:10.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:10.174+0000 7f8123e00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f812406f860 0x7f8124140f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:10.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:10.174+0000 7f8123e00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f81240c8070 0x7f8124147290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:10.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:10.174+0000 7f8123e00640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f8124163e60 con 0x7f81240c8070 2024-09-10T21:26:10.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:10.174+0000 7f8123e00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f8124163ce0 con 0x7f812406f860 2024-09-10T21:26:10.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:10.174+0000 7f8123e00640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f8124163fe0 con 0x7f812406f480 2024-09-10T21:26:10.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:10.174+0000 7f8123400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f81240c8070 0x7f8124147290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:10.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:10.174+0000 7f8123400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f81240c8070 0x7f8124147290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:46670/0 (socket says 172.21.15.17:46670) 2024-09-10T21:26:10.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:10.174+0000 7f8123400640 1 -- 172.21.15.17:0/1532451676 learned_addr learned my addr 172.21.15.17:0/1532451676 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:26:10.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:10.174+0000 7f8122a00640 1 --2- 172.21.15.17:0/1532451676 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f812406f480 0x7f8124140530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:10.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:10.174+0000 7f8122000640 1 --2- 172.21.15.17:0/1532451676 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f812406f860 0x7f8124140f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:10.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:10.174+0000 7f8123400640 1 -- 172.21.15.17:0/1532451676 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f81240c8070 msgr2=0x7f8124147290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-09-10T21:26:10.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:10.174+0000 7f8123400640 1 -- 172.21.15.17:0/1532451676 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f81240c8070 msgr2=0x7f8124147290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-10T21:26:10.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:10.174+0000 7f8123400640 1 --2- 172.21.15.17:0/1532451676 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f81240c8070 0x7f8124147290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T21:26:10.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:10.174+0000 7f8123400640 1 --2- 172.21.15.17:0/1532451676 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f81240c8070 0x7f8124147290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:26:10.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:10.174+0000 7f8122000640 1 -- 172.21.15.17:0/1532451676 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f812406f480 msgr2=0x7f8124140530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:10.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:10.174+0000 7f8122000640 1 --2- 172.21.15.17:0/1532451676 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f812406f480 0x7f8124140530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:10.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:10.174+0000 7f8122000640 1 -- 172.21.15.17:0/1532451676 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f81240c8070 msgr2=0x7f8124147290 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:26:10.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:10.174+0000 7f8122000640 1 --2- 172.21.15.17:0/1532451676 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f81240c8070 0x7f8124147290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:10.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:10.174+0000 7f8122000640 1 -- 172.21.15.17:0/1532451676 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f812415a4d0 con 0x7f812406f860 2024-09-10T21:26:10.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:10.174+0000 7f8122a00640 1 --2- 172.21.15.17:0/1532451676 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f812406f480 0x7f8124140530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:26:10.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:10.178+0000 7f8122000640 1 --2- 172.21.15.17:0/1532451676 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f812406f860 0x7f8124140f20 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f81100ac000 tx=0x7f81100040c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:10.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:10.178+0000 7f811aa00640 1 -- 172.21.15.17:0/1532451676 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81100bb070 con 0x7f812406f860 2024-09-10T21:26:10.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:10.178+0000 7f811aa00640 1 -- 172.21.15.17:0/1532451676 <== mon.1 v2:172.21.15.31:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8110005030 con 0x7f812406f860 2024-09-10T21:26:10.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:10.178+0000 7f811aa00640 1 -- 172.21.15.17:0/1532451676 <== mon.1 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81100b2040 con 0x7f812406f860 2024-09-10T21:26:10.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:10.178+0000 7f8123e00640 1 -- 172.21.15.17:0/1532451676 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81241f7e10 con 0x7f812406f860 2024-09-10T21:26:10.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:10.178+0000 7f8123e00640 1 -- 172.21.15.17:0/1532451676 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8124159600 con 0x7f812406f860 2024-09-10T21:26:10.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:10.178+0000 7f810fe00640 1 -- 172.21.15.17:0/1532451676 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81240c7870 con 0x7f812406f860 2024-09-10T21:26:10.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:10.182+0000 7f811aa00640 1 -- 172.21.15.17:0/1532451676 <== mon.1 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f81100b5030 con 0x7f812406f860 2024-09-10T21:26:10.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:10.182+0000 7f811aa00640 1 --2- 172.21.15.17:0/1532451676 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f8104072610 0x7f8104074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:10.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:10.182+0000 7f811aa00640 1 -- 172.21.15.17:0/1532451676 <== mon.1 v2:172.21.15.31:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f811012cb00 con 0x7f812406f860 2024-09-10T21:26:10.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:10.182+0000 7f8122a00640 1 --2- 172.21.15.17:0/1532451676 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f8104072610 0x7f8104074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:10.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:10.182+0000 7f8122a00640 1 --2- 172.21.15.17:0/1532451676 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f8104072610 0x7f8104074ad0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f811400f290 tx=0x7f8114009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:10.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:10.186+0000 7f811aa00640 1 -- 172.21.15.17:0/1532451676 <== mon.1 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f81100f1de0 con 0x7f812406f860 2024-09-10T21:26:10.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:10.570+0000 7f810fe00640 1 -- 172.21.15.17:0/1532451676 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f8124071ce0 con 0x7f812406f860 2024-09-10T21:26:10.724 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:10.722+0000 7f811aa00640 1 -- 172.21.15.17:0/1532451676 <== mon.1 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1556 (secure 0 0 0) 0x7f811012ce20 con 0x7f812406f860 2024-09-10T21:26:10.725 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5e2f023f-d6a6-49f4-a256-f4807dce3739","modified":"2024-09-10T21:23:46.892130Z","created":"2024-09-10T21:23:46.892130Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3301","nonce":0},{"type":"v1","addr":"172.21.15.31:6790","nonce":0}]},"addr":"172.21.15.31:6790/0","public_addr":"172.21.15.31:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-10T21:26:11.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:10.722+0000 7f810fe00640 1 -- 172.21.15.17:0/1532451676 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f8104072610 msgr2=0x7f8104074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:11.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:10.722+0000 7f810fe00640 1 --2- 172.21.15.17:0/1532451676 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f8104072610 0x7f8104074ad0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f811400f290 tx=0x7f8114009040 comp rx=0 tx=0).stop 2024-09-10T21:26:11.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:10.722+0000 7f810fe00640 1 -- 172.21.15.17:0/1532451676 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f812406f860 msgr2=0x7f8124140f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:11.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:10.722+0000 7f810fe00640 1 --2- 172.21.15.17:0/1532451676 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f812406f860 0x7f8124140f20 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f81100ac000 tx=0x7f81100040c0 comp rx=0 tx=0).stop 2024-09-10T21:26:11.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:10.726+0000 7f810fe00640 1 -- 172.21.15.17:0/1532451676 shutdown_connections 2024-09-10T21:26:11.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:10.726+0000 7f810fe00640 1 --2- 172.21.15.17:0/1532451676 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f8104072610 0x7f8104074ad0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:11.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:10.726+0000 7f810fe00640 1 --2- 172.21.15.17:0/1532451676 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f81240c8070 0x7f8124147290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:11.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:10.726+0000 7f810fe00640 1 --2- 172.21.15.17:0/1532451676 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f812406f860 0x7f8124140f20 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:11.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:10.726+0000 7f810fe00640 1 --2- 172.21.15.17:0/1532451676 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f812406f480 0x7f8124140530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:11.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:10.726+0000 7f810fe00640 1 -- 172.21.15.17:0/1532451676 >> 172.21.15.17:0/1532451676 conn(0x7f8124070ca0 msgr2=0x7f812415a720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:11.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:10.726+0000 7f810fe00640 1 -- 172.21.15.17:0/1532451676 shutdown_connections 2024-09-10T21:26:11.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:10.726+0000 7f810fe00640 1 -- 172.21.15.17:0/1532451676 wait complete. 2024-09-10T21:26:11.045 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-09-10T21:26:11.045 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-09-10T21:26:11.046 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-10T21:26:11.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.234+0000 7fc21da00640 1 -- 172.21.15.17:0/3417732624 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc1fc004810 msgr2=0x7fc1fc004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:11.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.234+0000 7fc21da00640 1 --2- 172.21.15.17:0/3417732624 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc1fc004810 0x7fc1fc004c90 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fc204009d30 tx=0x7fc20401d3e0 comp rx=0 tx=0).stop 2024-09-10T21:26:11.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.234+0000 7fc217400640 1 -- 172.21.15.17:0/3417732624 reap_dead start 2024-09-10T21:26:11.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.234+0000 7fc21da00640 1 -- 172.21.15.17:0/3417732624 shutdown_connections 2024-09-10T21:26:11.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.234+0000 7fc21da00640 1 -- 172.21.15.17:0/3417732624 >> 172.21.15.17:0/3417732624 conn(0x7fc2180702f0 msgr2=0x7fc218070700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:11.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.234+0000 7fc21da00640 1 -- 172.21.15.17:0/3417732624 shutdown_connections 2024-09-10T21:26:11.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.234+0000 7fc21da00640 1 -- 172.21.15.17:0/3417732624 wait complete. 2024-09-10T21:26:11.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.234+0000 7fc21da00640 1 Processor -- start 2024-09-10T21:26:11.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.234+0000 7fc21da00640 1 -- start start 2024-09-10T21:26:11.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.234+0000 7fc21da00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc1fc004810 0x7fc21814f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:11.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.234+0000 7fc21da00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc2180cd2b0 0x7fc21815d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:11.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.234+0000 7fc21da00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc21815de70 0x7fc2180cc810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:11.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.234+0000 7fc21da00640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fc2180cb450 con 0x7fc2180cd2b0 2024-09-10T21:26:11.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.234+0000 7fc21da00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fc2180b42c0 con 0x7fc1fc004810 2024-09-10T21:26:11.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.234+0000 7fc21da00640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7fc2180cb5d0 con 0x7fc21815de70 2024-09-10T21:26:11.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.234+0000 7fc216000640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc2180cd2b0 0x7fc21815d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:11.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.234+0000 7fc216000640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc2180cd2b0 0x7fc21815d6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:46692/0 (socket says 172.21.15.17:46692) 2024-09-10T21:26:11.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.234+0000 7fc216000640 1 -- 172.21.15.17:0/2624950462 learned_addr learned my addr 172.21.15.17:0/2624950462 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:26:11.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.234+0000 7fc216a00640 1 --2- 172.21.15.17:0/2624950462 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc1fc004810 0x7fc21814f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:11.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.234+0000 7fc217400640 1 --2- 172.21.15.17:0/2624950462 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc21815de70 0x7fc2180cc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:11.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.234+0000 7fc216000640 1 -- 172.21.15.17:0/2624950462 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc21815de70 msgr2=0x7fc2180cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:11.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.234+0000 7fc216000640 1 --2- 172.21.15.17:0/2624950462 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc21815de70 0x7fc2180cc810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:11.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.234+0000 7fc216000640 1 -- 172.21.15.17:0/2624950462 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc1fc004810 msgr2=0x7fc21814f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:11.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.234+0000 7fc216000640 1 --2- 172.21.15.17:0/2624950462 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc1fc004810 0x7fc21814f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:11.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.238+0000 7fc216000640 1 -- 172.21.15.17:0/2624950462 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2180c7580 con 0x7fc2180cd2b0 2024-09-10T21:26:11.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.238+0000 7fc216000640 1 --2- 172.21.15.17:0/2624950462 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc2180cd2b0 0x7fc21815d6c0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fc20401d210 tx=0x7fc204005620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:11.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.238+0000 7fc20ea00640 1 -- 172.21.15.17:0/2624950462 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2040bc020 con 0x7fc2180cd2b0 2024-09-10T21:26:11.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.238+0000 7fc20ea00640 1 -- 172.21.15.17:0/2624950462 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc204009480 con 0x7fc2180cd2b0 2024-09-10T21:26:11.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.238+0000 7fc21da00640 1 -- 172.21.15.17:0/2624950462 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc21815e5c0 con 0x7fc2180cd2b0 2024-09-10T21:26:11.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.238+0000 7fc20ea00640 1 -- 172.21.15.17:0/2624950462 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc204009900 con 0x7fc2180cd2b0 2024-09-10T21:26:11.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.238+0000 7fc21da00640 1 -- 172.21.15.17:0/2624950462 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc21815cbe0 con 0x7fc2180cd2b0 2024-09-10T21:26:11.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.238+0000 7fc21da00640 1 --2- 172.21.15.17:0/2624950462 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc1f0002a80 0x7fc1f0004f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:11.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.238+0000 7fc21da00640 1 -- 172.21.15.17:0/2624950462 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fc1f0007ed0 con 0x7fc1f0002a80 2024-09-10T21:26:11.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.238+0000 7fc216a00640 1 --2- 172.21.15.17:0/2624950462 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc1f0002a80 0x7fc1f0004f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:11.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.238+0000 7fc216a00640 1 -- 172.21.15.17:0/2624950462 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc1f00080d0 con 0x7fc1f0002a80 2024-09-10T21:26:11.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.238+0000 7fc216a00640 1 --2- 172.21.15.17:0/2624950462 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc1f0002a80 0x7fc1f0004f40 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fc208006690 tx=0x7fc2080039d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:11.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.238+0000 7fc20ea00640 1 -- 172.21.15.17:0/2624950462 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fc2040049e0 con 0x7fc2180cd2b0 2024-09-10T21:26:11.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.242+0000 7fc20ea00640 1 --2- 172.21.15.17:0/2624950462 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fc1f8072610 0x7fc1f8074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:11.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.242+0000 7fc20ea00640 1 -- 172.21.15.17:0/2624950462 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fc20412ecf0 con 0x7fc2180cd2b0 2024-09-10T21:26:11.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.242+0000 7fc20ea00640 1 -- 172.21.15.17:0/2624950462 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc20800f950 con 0x7fc1f0002a80 2024-09-10T21:26:11.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.242+0000 7fc20ea00640 1 -- 172.21.15.17:0/2624950462 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fc20800faf0 con 0x7fc1f0002a80 2024-09-10T21:26:11.245 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.242+0000 7fc20ea00640 1 -- 172.21.15.17:0/2624950462 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc1f0002a80 msgr2=0x7fc1f0004f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:11.246 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.242+0000 7fc20ea00640 1 --2- 172.21.15.17:0/2624950462 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc1f0002a80 0x7fc1f0004f40 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fc208006690 tx=0x7fc2080039d0 comp rx=0 tx=0).stop 2024-09-10T21:26:11.246 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.242+0000 7fc217400640 1 --2- 172.21.15.17:0/2624950462 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fc1f8072610 0x7fc1f8074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:11.246 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.242+0000 7fc20ea00640 1 -- 172.21.15.17:0/2624950462 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc1f0002a80 msgr2=0x7fc1f0004f40 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:26:11.246 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.242+0000 7fc20ea00640 1 --2- 172.21.15.17:0/2624950462 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc1f0002a80 0x7fc1f0004f40 secure :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fc208006690 tx=0x7fc2080039d0 comp rx=0 tx=0).stop 2024-09-10T21:26:11.246 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.242+0000 7fc217400640 1 --2- 172.21.15.17:0/2624950462 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fc1f8072610 0x7fc1f8074ad0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fc2180712b0 tx=0x7fc210009500 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:11.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.246+0000 7fc21da00640 1 --2- 172.21.15.17:0/2624950462 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc1f0006f10 0x7fc1f00073a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:11.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.246+0000 7fc21da00640 1 -- 172.21.15.17:0/2624950462 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fc1f000a4b0 con 0x7fc1f0006f10 2024-09-10T21:26:11.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.250+0000 7fc216a00640 1 --2- 172.21.15.17:0/2624950462 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc1f0006f10 0x7fc1f00073a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:11.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.250+0000 7fc216a00640 1 -- 172.21.15.17:0/2624950462 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fc1f00078e0 con 0x7fc1f0006f10 2024-09-10T21:26:11.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.250+0000 7fc216a00640 1 --2- 172.21.15.17:0/2624950462 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc1f0006f10 0x7fc1f00073a0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fc208017000 tx=0x7fc2180c5200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:11.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.250+0000 7fc20ea00640 1 -- 172.21.15.17:0/2624950462 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc20801ee20 con 0x7fc1f0006f10 2024-09-10T21:26:11.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.250+0000 7fc20ea00640 1 -- 172.21.15.17:0/2624950462 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4541 (secure 0 0 0) 0x7fc208011430 con 0x7fc1f0006f10 2024-09-10T21:26:11.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.250+0000 7fc20ea00640 1 -- 172.21.15.17:0/2624950462 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc1f0006f10 msgr2=0x7fc1f00073a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:11.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.250+0000 7fc20ea00640 1 --2- 172.21.15.17:0/2624950462 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc1f0006f10 0x7fc1f00073a0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fc208017000 tx=0x7fc2180c5200 comp rx=0 tx=0).stop 2024-09-10T21:26:11.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.250+0000 7fc20ea00640 1 -- 172.21.15.17:0/2624950462 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc1f0006f10 msgr2=0x7fc1f00073a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:26:11.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.250+0000 7fc20ea00640 1 --2- 172.21.15.17:0/2624950462 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc1f0006f10 0x7fc1f00073a0 secure :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fc208017000 tx=0x7fc2180c5200 comp rx=0 tx=0).stop 2024-09-10T21:26:11.255 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-10T21:26:11.255 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:26:11.255 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:26:11.255 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "leader", 2024-09-10T21:26:11.255 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 10, 2024-09-10T21:26:11.255 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 1155, 2024-09-10T21:26:11.255 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-10T21:26:11.255 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-10T21:26:11.255 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-10T21:26:11.255 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-10T21:26:11.256 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:26:11.256 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 0, 2024-09-10T21:26:11.256 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:26:11.256 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-10T21:26:11.256 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-10T21:26:11.256 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:26:11.256 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:26:11.256 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:26:11.256 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:26:11.256 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:26:11.256 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:26:11.256 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:26:11.256 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:26:11.257 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:26:11.257 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:26:11.257 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:26:11.257 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:26:11.257 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-10T21:26:11.257 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-10T21:26:11.257 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:26:11.257 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:26:11.257 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:26:11.257 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:26:11.257 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:26:11.257 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:26:11.257 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:26:11.258 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:26:11.258 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:26:11.258 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:26:11.258 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:26:11.258 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:26:11.258 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:26:11.258 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-10T21:26:11.258 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-10T21:26:11.258 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-10T21:26:11.258 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-10T21:26:11.258 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-10T21:26:11.258 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "5e2f023f-d6a6-49f4-a256-f4807dce3739", 2024-09-10T21:26:11.258 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:26:11.258 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:26:11.259 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-10T21:26:11.259 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-10T21:26:11.259 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-10T21:26:11.259 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-10T21:26:11.259 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-10T21:26:11.259 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-10T21:26:11.259 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-10T21:26:11.259 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:26:11.259 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-10T21:26:11.259 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:26:11.259 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:26:11.259 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:26:11.259 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:26:11.260 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:26:11.260 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:26:11.260 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:26:11.260 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:26:11.260 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:26:11.260 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:26:11.260 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:26:11.260 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:26:11.260 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-10T21:26:11.260 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:26:11.260 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-10T21:26:11.260 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:26:11.260 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:26:11.261 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:26:11.261 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:26:11.261 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:26:11.261 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:26:11.261 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:26:11.261 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-10T21:26:11.261 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:26:11.261 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:26:11.261 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:26:11.261 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:26:11.261 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-10T21:26:11.261 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:26:11.261 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:26:11.262 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:26:11.262 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:26:11.262 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-10T21:26:11.262 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-10T21:26:11.262 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:26:11.262 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:26:11.262 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:26:11.262 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:26:11.262 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:26:11.262 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:26:11.262 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:26:11.262 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:26:11.262 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:26:11.262 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:26:11.263 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:26:11.263 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3300", 2024-09-10T21:26:11.263 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:26:11.263 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:26:11.263 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:26:11.263 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:26:11.263 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789", 2024-09-10T21:26:11.263 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:26:11.263 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:26:11.263 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:26:11.263 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:26:11.263 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789/0", 2024-09-10T21:26:11.263 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6789/0", 2024-09-10T21:26:11.263 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:26:11.264 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:26:11.264 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:26:11.264 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:26:11.264 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:26:11.264 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:26:11.264 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:26:11.264 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:26:11.264 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:26:11.264 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:26:11.264 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:26:11.265 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3301", 2024-09-10T21:26:11.265 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:26:11.265 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:26:11.265 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:26:11.265 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:26:11.265 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790", 2024-09-10T21:26:11.265 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:26:11.265 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:26:11.265 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:26:11.265 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:26:11.265 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790/0", 2024-09-10T21:26:11.265 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6790/0", 2024-09-10T21:26:11.265 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:26:11.266 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:26:11.266 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:26:11.266 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:26:11.266 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:26:11.266 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:26:11.266 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-10T21:26:11.266 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-10T21:26:11.266 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:26:11.266 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:26:11.266 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:26:11.266 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:26:11.266 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:26:11.266 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:26:11.266 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-10T21:26:11.267 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:26:11.267 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:26:11.267 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:26:11.267 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 2 2024-09-10T21:26:11.267 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:26:11.267 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:26:11.267 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:26:11.267 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-10T21:26:11.267 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-10T21:26:11.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.250+0000 7fc21da00640 1 -- 172.21.15.17:0/2624950462 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fc1f8072610 msgr2=0x7fc1f8074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:11.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.250+0000 7fc21da00640 1 --2- 172.21.15.17:0/2624950462 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fc1f8072610 0x7fc1f8074ad0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fc2180712b0 tx=0x7fc210009500 comp rx=0 tx=0).stop 2024-09-10T21:26:11.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.250+0000 7fc21da00640 1 -- 172.21.15.17:0/2624950462 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc2180cd2b0 msgr2=0x7fc21815d6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:11.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.250+0000 7fc21da00640 1 --2- 172.21.15.17:0/2624950462 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc2180cd2b0 0x7fc21815d6c0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fc20401d210 tx=0x7fc204005620 comp rx=0 tx=0).stop 2024-09-10T21:26:11.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.250+0000 7fc217400640 1 -- 172.21.15.17:0/2624950462 reap_dead start 2024-09-10T21:26:11.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.250+0000 7fc217400640 1 -- 172.21.15.17:0/2624950462 reap_dead start 2024-09-10T21:26:11.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.250+0000 7fc21da00640 1 -- 172.21.15.17:0/2624950462 shutdown_connections 2024-09-10T21:26:11.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.250+0000 7fc21da00640 1 -- 172.21.15.17:0/2624950462 >> 172.21.15.17:0/2624950462 conn(0x7fc2180702f0 msgr2=0x7fc2180c3200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:11.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.250+0000 7fc21da00640 1 -- 172.21.15.17:0/2624950462 shutdown_connections 2024-09-10T21:26:11.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.250+0000 7fc21da00640 1 -- 172.21.15.17:0/2624950462 wait complete. 2024-09-10T21:26:11.274 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-10T21:26:11.274 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-09-10T21:26:11.274 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-10T21:26:11.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.422+0000 7f4db6400640 1 -- 172.21.15.17:0/3124379142 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f4db006f730 msgr2=0x7f4db015c130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:11.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.422+0000 7f4db6400640 1 --2- 172.21.15.17:0/3124379142 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f4db006f730 0x7f4db015c130 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f4da8009840 tx=0x7f4da801c7f0 comp rx=0 tx=0).stop 2024-09-10T21:26:11.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.422+0000 7f4db6400640 1 -- 172.21.15.17:0/3124379142 shutdown_connections 2024-09-10T21:26:11.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.422+0000 7f4db6400640 1 --2- 172.21.15.17:0/3124379142 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f4db006f730 0x7f4db015c130 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:11.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.422+0000 7f4db6400640 1 --2- 172.21.15.17:0/3124379142 >> v2:172.21.15.17:3300/0 conn(0x7f4db006f350 0x7f4db0158430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:11.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.422+0000 7f4db6400640 1 -- 172.21.15.17:0/3124379142 >> 172.21.15.17:0/3124379142 conn(0x7f4db00cdbd0 msgr2=0x7f4db00c99f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:11.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.422+0000 7f4db6400640 1 -- 172.21.15.17:0/3124379142 shutdown_connections 2024-09-10T21:26:11.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.422+0000 7f4db6400640 1 -- 172.21.15.17:0/3124379142 wait complete. 2024-09-10T21:26:11.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.422+0000 7f4db6400640 1 Processor -- start 2024-09-10T21:26:11.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.422+0000 7f4db6400640 1 -- start start 2024-09-10T21:26:11.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.426+0000 7f4db6400640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f4db006f350 0x7f4db00c2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:11.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.426+0000 7f4db6400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4db006f730 0x7f4db00c2ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:11.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.426+0000 7f4db6400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4db00703f0 0x7f4db015a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:11.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.426+0000 7f4db6400640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f4db0158af0 con 0x7f4db00703f0 2024-09-10T21:26:11.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.426+0000 7f4db6400640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f4db0158970 con 0x7f4db006f730 2024-09-10T21:26:11.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.426+0000 7f4db6400640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f4db00cca50 con 0x7f4db006f350 2024-09-10T21:26:11.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.426+0000 7f4db5000640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f4db006f350 0x7f4db00c2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:11.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.426+0000 7f4db5000640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f4db006f350 0x7f4db00c2ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3301/0 says I am v2:172.21.15.17:43680/0 (socket says 172.21.15.17:43680) 2024-09-10T21:26:11.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.426+0000 7f4db5000640 1 -- 172.21.15.17:0/1102178498 learned_addr learned my addr 172.21.15.17:0/1102178498 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:26:11.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.426+0000 7f4db5a00640 1 --2- 172.21.15.17:0/1102178498 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4db00703f0 0x7f4db015a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:11.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.426+0000 7f4dafe00640 1 --2- 172.21.15.17:0/1102178498 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4db006f730 0x7f4db00c2ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:11.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.426+0000 7f4db5000640 1 -- 172.21.15.17:0/1102178498 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4db006f730 msgr2=0x7f4db00c2ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:11.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.426+0000 7f4db5000640 1 --2- 172.21.15.17:0/1102178498 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4db006f730 0x7f4db00c2ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:11.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.426+0000 7f4db5000640 1 -- 172.21.15.17:0/1102178498 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4db00703f0 msgr2=0x7f4db015a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:11.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.426+0000 7f4db5000640 1 --2- 172.21.15.17:0/1102178498 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4db00703f0 0x7f4db015a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:11.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.426+0000 7f4db5000640 1 -- 172.21.15.17:0/1102178498 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4db015b250 con 0x7f4db006f350 2024-09-10T21:26:11.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.426+0000 7f4db5a00640 1 --2- 172.21.15.17:0/1102178498 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4db00703f0 0x7f4db015a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:26:11.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.426+0000 7f4db5000640 1 --2- 172.21.15.17:0/1102178498 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f4db006f350 0x7f4db00c2ab0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f4da400ec00 tx=0x7f4da400c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:11.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.426+0000 7f4dacc00640 1 -- 172.21.15.17:0/1102178498 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4da4010630 con 0x7f4db006f350 2024-09-10T21:26:11.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.426+0000 7f4db6400640 1 -- 172.21.15.17:0/1102178498 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4db01f4800 con 0x7f4db006f350 2024-09-10T21:26:11.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.426+0000 7f4dacc00640 1 -- 172.21.15.17:0/1102178498 <== mon.2 v2:172.21.15.31:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4da40108c0 con 0x7f4db006f350 2024-09-10T21:26:11.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.426+0000 7f4db6400640 1 -- 172.21.15.17:0/1102178498 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f4db01f4ae0 con 0x7f4db006f350 2024-09-10T21:26:11.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.426+0000 7f4dacc00640 1 -- 172.21.15.17:0/1102178498 <== mon.2 v2:172.21.15.31:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4da4010cf0 con 0x7f4db006f350 2024-09-10T21:26:11.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.430+0000 7f4db6400640 1 --2- 172.21.15.17:0/1102178498 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d8c002a80 0x7f4d8c004f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:11.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.430+0000 7f4db6400640 1 -- 172.21.15.17:0/1102178498 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f4d8c007ed0 con 0x7f4d8c002a80 2024-09-10T21:26:11.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.430+0000 7f4dafe00640 1 --2- 172.21.15.17:0/1102178498 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d8c002a80 0x7f4d8c004f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:11.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.430+0000 7f4dafe00640 1 -- 172.21.15.17:0/1102178498 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4d8c0080d0 con 0x7f4d8c002a80 2024-09-10T21:26:11.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.430+0000 7f4dafe00640 1 --2- 172.21.15.17:0/1102178498 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d8c002a80 0x7f4d8c004f40 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f4db0163000 tx=0x7f4d9c00fc00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:11.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.430+0000 7f4dacc00640 1 -- 172.21.15.17:0/1102178498 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d9c018020 con 0x7f4d8c002a80 2024-09-10T21:26:11.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.430+0000 7f4dacc00640 1 -- 172.21.15.17:0/1102178498 <== mon.2 v2:172.21.15.31:3301/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f4da4018420 con 0x7f4db006f350 2024-09-10T21:26:11.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.430+0000 7f4dacc00640 1 --2- 172.21.15.17:0/1102178498 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f4d90072510 0x7f4d900749d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:11.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.430+0000 7f4dacc00640 1 -- 172.21.15.17:0/1102178498 <== mon.2 v2:172.21.15.31:3301/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f4da40942d0 con 0x7f4db006f350 2024-09-10T21:26:11.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.430+0000 7f4dacc00640 1 -- 172.21.15.17:0/1102178498 <== mon.1 v2:172.21.15.31:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f4d9c00d600 con 0x7f4d8c002a80 2024-09-10T21:26:11.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.430+0000 7f4dacc00640 1 -- 172.21.15.17:0/1102178498 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d8c002a80 msgr2=0x7f4d8c004f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:11.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.430+0000 7f4dacc00640 1 --2- 172.21.15.17:0/1102178498 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d8c002a80 0x7f4d8c004f40 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f4db0163000 tx=0x7f4d9c00fc00 comp rx=0 tx=0).stop 2024-09-10T21:26:11.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.430+0000 7f4dacc00640 1 -- 172.21.15.17:0/1102178498 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d8c002a80 msgr2=0x7f4d8c004f40 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:26:11.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.430+0000 7f4dacc00640 1 --2- 172.21.15.17:0/1102178498 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d8c002a80 0x7f4d8c004f40 secure :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f4db0163000 tx=0x7f4d9c00fc00 comp rx=0 tx=0).stop 2024-09-10T21:26:11.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.434+0000 7f4db5a00640 1 --2- 172.21.15.17:0/1102178498 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f4d90072510 0x7f4d900749d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:11.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.434+0000 7f4db5a00640 1 --2- 172.21.15.17:0/1102178498 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f4d90072510 0x7f4d900749d0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f4da80af000 tx=0x7f4da801b030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:11.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.438+0000 7f4db6400640 1 --2- 172.21.15.17:0/1102178498 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d8c006f60 0x7f4d8c0073f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:11.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.438+0000 7f4db6400640 1 -- 172.21.15.17:0/1102178498 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f4d8c00a4b0 con 0x7f4d8c006f60 2024-09-10T21:26:11.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.438+0000 7f4dafe00640 1 --2- 172.21.15.17:0/1102178498 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d8c006f60 0x7f4d8c0073f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:11.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.438+0000 7f4dafe00640 1 -- 172.21.15.17:0/1102178498 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f4d8c007930 con 0x7f4d8c006f60 2024-09-10T21:26:11.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.438+0000 7f4dafe00640 1 --2- 172.21.15.17:0/1102178498 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d8c006f60 0x7f4d8c0073f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f4d9c016000 tx=0x7f4d9c004630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:11.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.438+0000 7f4dacc00640 1 -- 172.21.15.17:0/1102178498 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d9c00dc40 con 0x7f4d8c006f60 2024-09-10T21:26:11.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.442+0000 7f4dacc00640 1 -- 172.21.15.17:0/1102178498 <== mon.1 v2:172.21.15.31:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5042 (secure 0 0 0) 0x7f4d9c00dc40 con 0x7f4d8c006f60 2024-09-10T21:26:11.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.442+0000 7f4dacc00640 1 -- 172.21.15.17:0/1102178498 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d8c006f60 msgr2=0x7f4d8c0073f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:11.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.442+0000 7f4dacc00640 1 --2- 172.21.15.17:0/1102178498 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d8c006f60 0x7f4d8c0073f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f4d9c016000 tx=0x7f4d9c004630 comp rx=0 tx=0).stop 2024-09-10T21:26:11.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.442+0000 7f4dacc00640 1 -- 172.21.15.17:0/1102178498 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d8c006f60 msgr2=0x7f4d8c0073f0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:26:11.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.442+0000 7f4dacc00640 1 --2- 172.21.15.17:0/1102178498 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d8c006f60 0x7f4d8c0073f0 secure :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f4d9c016000 tx=0x7f4d9c004630 comp rx=0 tx=0).stop 2024-09-10T21:26:11.445 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-10T21:26:11.445 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:26:11.445 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:26:11.445 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-10T21:26:11.445 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 10, 2024-09-10T21:26:11.445 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 109888, 2024-09-10T21:26:11.445 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-10T21:26:11.445 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-10T21:26:11.445 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-10T21:26:11.445 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-10T21:26:11.446 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:26:11.446 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 0, 2024-09-10T21:26:11.446 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:26:11.446 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-10T21:26:11.446 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-10T21:26:11.446 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:26:11.446 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:26:11.446 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:26:11.446 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:26:11.446 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:26:11.446 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:26:11.446 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:26:11.446 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:26:11.447 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:26:11.447 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:26:11.447 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:26:11.447 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:26:11.447 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-10T21:26:11.447 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-10T21:26:11.447 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:26:11.447 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:26:11.447 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:26:11.447 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:26:11.447 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:26:11.447 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:26:11.447 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:26:11.447 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:26:11.448 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:26:11.448 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:26:11.448 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:26:11.448 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:26:11.448 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:26:11.448 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-10T21:26:11.448 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-10T21:26:11.448 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-10T21:26:11.448 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-10T21:26:11.448 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-10T21:26:11.448 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "5e2f023f-d6a6-49f4-a256-f4807dce3739", 2024-09-10T21:26:11.448 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:26:11.448 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:26:11.449 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-10T21:26:11.449 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-10T21:26:11.449 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-10T21:26:11.449 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-10T21:26:11.449 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-10T21:26:11.449 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-10T21:26:11.449 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-10T21:26:11.449 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:26:11.449 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-10T21:26:11.449 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:26:11.449 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:26:11.449 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:26:11.449 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:26:11.450 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:26:11.450 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:26:11.450 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:26:11.450 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:26:11.450 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:26:11.450 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:26:11.450 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:26:11.450 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:26:11.450 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-10T21:26:11.450 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:26:11.450 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-10T21:26:11.450 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:26:11.450 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:26:11.450 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:26:11.451 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:26:11.451 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:26:11.451 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:26:11.451 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:26:11.451 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-10T21:26:11.451 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:26:11.451 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:26:11.451 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:26:11.451 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:26:11.451 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-10T21:26:11.451 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:26:11.451 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:26:11.451 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:26:11.451 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:26:11.451 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-10T21:26:11.452 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-10T21:26:11.452 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:26:11.452 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:26:11.452 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:26:11.452 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:26:11.452 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:26:11.452 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:26:11.452 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:26:11.452 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:26:11.452 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:26:11.452 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:26:11.452 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:26:11.452 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3300", 2024-09-10T21:26:11.453 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:26:11.453 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:26:11.453 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:26:11.453 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:26:11.453 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789", 2024-09-10T21:26:11.453 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:26:11.453 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:26:11.453 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:26:11.453 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:26:11.453 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789/0", 2024-09-10T21:26:11.453 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6789/0", 2024-09-10T21:26:11.453 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:26:11.453 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:26:11.453 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:26:11.454 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:26:11.454 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:26:11.454 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:26:11.454 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:26:11.454 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:26:11.454 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:26:11.454 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:26:11.454 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:26:11.454 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3301", 2024-09-10T21:26:11.454 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:26:11.454 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:26:11.454 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:26:11.454 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:26:11.454 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790", 2024-09-10T21:26:11.455 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:26:11.455 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:26:11.455 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:26:11.455 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:26:11.455 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790/0", 2024-09-10T21:26:11.455 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6790/0", 2024-09-10T21:26:11.455 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:26:11.455 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:26:11.455 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:26:11.455 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:26:11.455 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:26:11.455 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:26:11.455 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-10T21:26:11.456 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-10T21:26:11.456 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:26:11.456 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:26:11.456 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:26:11.456 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:26:11.456 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:26:11.456 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:26:11.456 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": [ 2024-09-10T21:26:11.456 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:26:11.456 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:26:11.456 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:26:11.456 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 5 2024-09-10T21:26:11.456 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:26:11.456 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:26:11.457 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-10T21:26:11.457 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:26:11.457 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:26:11.457 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:26:11.457 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 4 2024-09-10T21:26:11.457 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:26:11.457 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:26:11.457 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-10T21:26:11.457 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:26:11.457 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:26:11.457 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:26:11.457 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 2 2024-09-10T21:26:11.457 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:26:11.457 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:26:11.457 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": [ 2024-09-10T21:26:11.458 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:26:11.458 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:26:11.458 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:26:11.458 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:26:11.458 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:26:11.458 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:26:11.458 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:26:11.458 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-10T21:26:11.458 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-10T21:26:11.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.442+0000 7f4db6400640 1 -- 172.21.15.17:0/1102178498 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f4d90072510 msgr2=0x7f4d900749d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:11.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.442+0000 7f4db6400640 1 --2- 172.21.15.17:0/1102178498 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f4d90072510 0x7f4d900749d0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f4da80af000 tx=0x7f4da801b030 comp rx=0 tx=0).stop 2024-09-10T21:26:11.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.442+0000 7f4db6400640 1 -- 172.21.15.17:0/1102178498 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f4db006f350 msgr2=0x7f4db00c2ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:11.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.442+0000 7f4db6400640 1 --2- 172.21.15.17:0/1102178498 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f4db006f350 0x7f4db00c2ab0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f4da400ec00 tx=0x7f4da400c6a0 comp rx=0 tx=0).stop 2024-09-10T21:26:11.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.442+0000 7f4db5a00640 1 -- 172.21.15.17:0/1102178498 reap_dead start 2024-09-10T21:26:11.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.442+0000 7f4db5a00640 1 -- 172.21.15.17:0/1102178498 reap_dead start 2024-09-10T21:26:11.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.442+0000 7f4db6400640 1 -- 172.21.15.17:0/1102178498 shutdown_connections 2024-09-10T21:26:11.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.442+0000 7f4db6400640 1 -- 172.21.15.17:0/1102178498 >> 172.21.15.17:0/1102178498 conn(0x7f4db00cdbd0 msgr2=0x7f4db00cb810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:11.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.442+0000 7f4db6400640 1 -- 172.21.15.17:0/1102178498 shutdown_connections 2024-09-10T21:26:11.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.442+0000 7f4db6400640 1 -- 172.21.15.17:0/1102178498 wait complete. 2024-09-10T21:26:11.463 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-10T21:26:11.464 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-09-10T21:26:11.464 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-10T21:26:11.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.610+0000 7f93f6600640 1 -- 172.21.15.17:0/2087417800 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f93f00c9520 msgr2=0x7f93f0070c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:11.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.610+0000 7f93f6600640 1 --2- 172.21.15.17:0/2087417800 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f93f00c9520 0x7f93f0070c10 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f93dc009840 tx=0x7f93dc01c700 comp rx=0 tx=0).stop 2024-09-10T21:26:11.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.610+0000 7f93f6600640 1 -- 172.21.15.17:0/2087417800 shutdown_connections 2024-09-10T21:26:11.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.610+0000 7f93f6600640 1 --2- 172.21.15.17:0/2087417800 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f93f00c9520 0x7f93f0070c10 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:11.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.610+0000 7f93f6600640 1 -- 172.21.15.17:0/2087417800 >> 172.21.15.17:0/2087417800 conn(0x7f93f0068990 msgr2=0x7f93f0068da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:11.616 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.610+0000 7f93f6600640 1 -- 172.21.15.17:0/2087417800 shutdown_connections 2024-09-10T21:26:11.616 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.610+0000 7f93f6600640 1 -- 172.21.15.17:0/2087417800 wait complete. 2024-09-10T21:26:11.617 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.614+0000 7f93f6600640 1 Processor -- start 2024-09-10T21:26:11.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.614+0000 7f93f6600640 1 -- start start 2024-09-10T21:26:11.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.614+0000 7f93f6600640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f93f00c2f80 0x7f93f013aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:11.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.614+0000 7f93f6600640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f93f00c9520 0x7f93f006e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:11.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.614+0000 7f93f6600640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93f01523d0 0x7f93f006ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:11.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.614+0000 7f93f6600640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f93f01669e0 con 0x7f93f00c9520 2024-09-10T21:26:11.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.614+0000 7f93f6600640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f93f00cb3c0 con 0x7f93f01523d0 2024-09-10T21:26:11.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.614+0000 7f93f6600640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f93f0166b60 con 0x7f93f00c2f80 2024-09-10T21:26:11.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.614+0000 7f93efe00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f93f00c9520 0x7f93f006e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:11.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.614+0000 7f93efe00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f93f00c9520 0x7f93f006e7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:46766/0 (socket says 172.21.15.17:46766) 2024-09-10T21:26:11.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.614+0000 7f93efe00640 1 -- 172.21.15.17:0/3749877800 learned_addr learned my addr 172.21.15.17:0/3749877800 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:26:11.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.614+0000 7f93f5200640 1 --2- 172.21.15.17:0/3749877800 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f93f00c2f80 0x7f93f013aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:11.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.614+0000 7f93f5c00640 1 --2- 172.21.15.17:0/3749877800 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93f01523d0 0x7f93f006ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:11.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.618+0000 7f93efe00640 1 -- 172.21.15.17:0/3749877800 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f93f00c2f80 msgr2=0x7f93f013aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:11.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.618+0000 7f93efe00640 1 --2- 172.21.15.17:0/3749877800 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f93f00c2f80 0x7f93f013aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:11.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.618+0000 7f93efe00640 1 -- 172.21.15.17:0/3749877800 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93f01523d0 msgr2=0x7f93f006ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:11.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.618+0000 7f93efe00640 1 --2- 172.21.15.17:0/3749877800 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93f01523d0 0x7f93f006ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:11.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.618+0000 7f93efe00640 1 -- 172.21.15.17:0/3749877800 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93f0151560 con 0x7f93f00c9520 2024-09-10T21:26:11.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.618+0000 7f93efe00640 1 --2- 172.21.15.17:0/3749877800 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f93f00c9520 0x7f93f006e7f0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f93dc000c10 tx=0x7f93dc0a86e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:11.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.618+0000 7f93ecc00640 1 -- 172.21.15.17:0/3749877800 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93dc005140 con 0x7f93f00c9520 2024-09-10T21:26:11.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.618+0000 7f93ecc00640 1 -- 172.21.15.17:0/3749877800 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f93dc0052e0 con 0x7f93f00c9520 2024-09-10T21:26:11.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.618+0000 7f93f6600640 1 -- 172.21.15.17:0/3749877800 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93f0151800 con 0x7f93f00c9520 2024-09-10T21:26:11.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.618+0000 7f93ecc00640 1 -- 172.21.15.17:0/3749877800 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93dc0046d0 con 0x7f93f00c9520 2024-09-10T21:26:11.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.618+0000 7f93f6600640 1 -- 172.21.15.17:0/3749877800 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93f015b080 con 0x7f93f00c9520 2024-09-10T21:26:11.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.618+0000 7f93f6600640 1 --2- 172.21.15.17:0/3749877800 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f93f0146230 0x7f93f0146d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:11.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.618+0000 7f93f6600640 1 -- 172.21.15.17:0/3749877800 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f93f0069ef0 con 0x7f93f0146230 2024-09-10T21:26:11.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.618+0000 7f93f5200640 1 --2- 172.21.15.17:0/3749877800 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f93f0146230 0x7f93f0146d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:11.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.618+0000 7f93f5200640 1 -- 172.21.15.17:0/3749877800 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f93f0159ff0 con 0x7f93f0146230 2024-09-10T21:26:11.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.618+0000 7f93ecc00640 1 -- 172.21.15.17:0/3749877800 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f93dc004870 con 0x7f93f00c9520 2024-09-10T21:26:11.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.618+0000 7f93f5200640 1 --2- 172.21.15.17:0/3749877800 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f93f0146230 0x7f93f0146d50 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f93e400f770 tx=0x7f93e400fc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:11.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.622+0000 7f93ecc00640 1 --2- 172.21.15.17:0/3749877800 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f93d00725e0 0x7f93d0074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:11.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.622+0000 7f93ecc00640 1 -- 172.21.15.17:0/3749877800 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f93dc12df70 con 0x7f93f00c9520 2024-09-10T21:26:11.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.622+0000 7f93ecc00640 1 -- 172.21.15.17:0/3749877800 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93e40050d0 con 0x7f93f0146230 2024-09-10T21:26:11.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.622+0000 7f93ecc00640 1 -- 172.21.15.17:0/3749877800 <== mon.2 v2:172.21.15.31:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f93e4018070 con 0x7f93f0146230 2024-09-10T21:26:11.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.622+0000 7f93ecc00640 1 -- 172.21.15.17:0/3749877800 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f93f0146230 msgr2=0x7f93f0146d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:11.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.622+0000 7f93ecc00640 1 --2- 172.21.15.17:0/3749877800 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f93f0146230 0x7f93f0146d50 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f93e400f770 tx=0x7f93e400fc30 comp rx=0 tx=0).stop 2024-09-10T21:26:11.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.622+0000 7f93ecc00640 1 -- 172.21.15.17:0/3749877800 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f93f0146230 msgr2=0x7f93f0146d50 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:26:11.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.622+0000 7f93ecc00640 1 --2- 172.21.15.17:0/3749877800 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f93f0146230 0x7f93f0146d50 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:11.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.622+0000 7f93f5c00640 1 --2- 172.21.15.17:0/3749877800 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f93d00725e0 0x7f93d0074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:11.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.622+0000 7f93f5c00640 1 --2- 172.21.15.17:0/3749877800 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f93d00725e0 0x7f93d0074aa0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f93e80012f0 tx=0x7f93e8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:11.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.634+0000 7f93f6600640 1 --2- 172.21.15.17:0/3749877800 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f93f0159930 0x7f93f0147290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:11.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.634+0000 7f93f6600640 1 -- 172.21.15.17:0/3749877800 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f93f0155b40 con 0x7f93f0159930 2024-09-10T21:26:11.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.634+0000 7f93f5200640 1 --2- 172.21.15.17:0/3749877800 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f93f0159930 0x7f93f0147290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:11.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.634+0000 7f93f5200640 1 -- 172.21.15.17:0/3749877800 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f93f0144b80 con 0x7f93f0159930 2024-09-10T21:26:11.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.634+0000 7f93f5200640 1 --2- 172.21.15.17:0/3749877800 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f93f0159930 0x7f93f0147290 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f93e400f770 tx=0x7f93e4000ee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:11.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.634+0000 7f93ecc00640 1 -- 172.21.15.17:0/3749877800 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93e400d8e0 con 0x7f93f0159930 2024-09-10T21:26:11.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.634+0000 7f93ecc00640 1 -- 172.21.15.17:0/3749877800 <== mon.2 v2:172.21.15.31:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5042 (secure 0 0 0) 0x7f93e400d8e0 con 0x7f93f0159930 2024-09-10T21:26:11.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.634+0000 7f93ecc00640 1 -- 172.21.15.17:0/3749877800 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f93f0159930 msgr2=0x7f93f0147290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:11.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.634+0000 7f93ecc00640 1 --2- 172.21.15.17:0/3749877800 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f93f0159930 0x7f93f0147290 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f93e400f770 tx=0x7f93e4000ee0 comp rx=0 tx=0).stop 2024-09-10T21:26:11.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.634+0000 7f93ecc00640 1 -- 172.21.15.17:0/3749877800 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f93f0159930 msgr2=0x7f93f0147290 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:26:11.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.634+0000 7f93ecc00640 1 --2- 172.21.15.17:0/3749877800 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f93f0159930 0x7f93f0147290 secure :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f93e400f770 tx=0x7f93e4000ee0 comp rx=0 tx=0).stop 2024-09-10T21:26:11.640 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-10T21:26:11.640 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:26:11.641 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:26:11.641 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-10T21:26:11.641 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 10, 2024-09-10T21:26:11.641 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 110080, 2024-09-10T21:26:11.641 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-10T21:26:11.641 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-10T21:26:11.641 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-10T21:26:11.641 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-10T21:26:11.641 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:26:11.641 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 0, 2024-09-10T21:26:11.641 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:26:11.641 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-10T21:26:11.641 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-10T21:26:11.641 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:26:11.642 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:26:11.642 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:26:11.642 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:26:11.642 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:26:11.642 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:26:11.642 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:26:11.642 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:26:11.642 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:26:11.642 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:26:11.642 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:26:11.642 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:26:11.642 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-10T21:26:11.642 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-10T21:26:11.642 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:26:11.643 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:26:11.643 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:26:11.643 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:26:11.643 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:26:11.643 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:26:11.643 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:26:11.643 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:26:11.643 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:26:11.643 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:26:11.643 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:26:11.643 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:26:11.643 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:26:11.643 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-10T21:26:11.643 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-10T21:26:11.644 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-10T21:26:11.644 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-10T21:26:11.644 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-10T21:26:11.644 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "5e2f023f-d6a6-49f4-a256-f4807dce3739", 2024-09-10T21:26:11.644 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:26:11.644 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:26:11.644 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-10T21:26:11.644 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-10T21:26:11.644 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-10T21:26:11.644 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-10T21:26:11.644 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-10T21:26:11.644 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-10T21:26:11.644 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-10T21:26:11.645 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:26:11.645 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-10T21:26:11.645 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:26:11.645 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:26:11.645 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:26:11.645 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:26:11.645 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:26:11.645 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:26:11.645 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:26:11.645 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:26:11.645 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:26:11.645 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:26:11.645 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:26:11.645 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:26:11.646 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-10T21:26:11.646 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:26:11.646 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-10T21:26:11.646 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:26:11.646 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:26:11.646 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:26:11.646 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:26:11.646 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:26:11.646 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:26:11.646 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:26:11.646 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-10T21:26:11.646 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:26:11.646 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:26:11.646 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:26:11.647 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:26:11.647 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-10T21:26:11.647 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:26:11.647 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:26:11.647 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:26:11.647 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:26:11.647 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-10T21:26:11.647 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-10T21:26:11.647 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:26:11.647 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:26:11.647 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:26:11.647 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:26:11.647 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:26:11.647 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:26:11.648 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:26:11.648 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:26:11.648 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:26:11.648 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:26:11.648 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:26:11.648 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3300", 2024-09-10T21:26:11.648 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:26:11.648 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:26:11.648 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:26:11.648 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:26:11.648 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789", 2024-09-10T21:26:11.648 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:26:11.648 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:26:11.649 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:26:11.649 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:26:11.649 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789/0", 2024-09-10T21:26:11.649 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6789/0", 2024-09-10T21:26:11.649 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:26:11.649 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:26:11.649 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:26:11.649 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:26:11.649 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:26:11.649 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:26:11.649 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:26:11.649 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:26:11.649 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:26:11.649 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:26:11.650 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:26:11.650 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3301", 2024-09-10T21:26:11.650 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:26:11.650 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:26:11.650 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:26:11.650 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:26:11.650 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790", 2024-09-10T21:26:11.650 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:26:11.650 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:26:11.650 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:26:11.650 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:26:11.650 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790/0", 2024-09-10T21:26:11.650 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6790/0", 2024-09-10T21:26:11.651 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:26:11.651 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:26:11.651 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:26:11.651 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:26:11.651 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:26:11.651 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:26:11.651 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-10T21:26:11.651 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-10T21:26:11.651 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:26:11.651 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:26:11.651 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:26:11.651 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:26:11.651 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:26:11.652 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:26:11.652 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": [ 2024-09-10T21:26:11.652 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:26:11.652 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:26:11.652 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:26:11.652 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:26:11.652 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:26:11.652 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:26:11.652 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-10T21:26:11.652 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:26:11.652 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:26:11.652 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:26:11.652 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 4 2024-09-10T21:26:11.652 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:26:11.653 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:26:11.653 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-10T21:26:11.653 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:26:11.653 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:26:11.653 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:26:11.653 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 5 2024-09-10T21:26:11.653 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:26:11.653 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:26:11.653 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": [ 2024-09-10T21:26:11.653 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:26:11.653 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:26:11.653 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:26:11.653 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:26:11.654 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:26:11.654 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:26:11.654 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:26:11.654 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-10T21:26:11.654 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-10T21:26:11.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.638+0000 7f93f6600640 1 -- 172.21.15.17:0/3749877800 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f93d00725e0 msgr2=0x7f93d0074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:11.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.638+0000 7f93f6600640 1 --2- 172.21.15.17:0/3749877800 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f93d00725e0 0x7f93d0074aa0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f93e80012f0 tx=0x7f93e8009040 comp rx=0 tx=0).stop 2024-09-10T21:26:11.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.638+0000 7f93f6600640 1 -- 172.21.15.17:0/3749877800 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f93f00c9520 msgr2=0x7f93f006e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:11.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.638+0000 7f93f6600640 1 --2- 172.21.15.17:0/3749877800 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f93f00c9520 0x7f93f006e7f0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f93dc000c10 tx=0x7f93dc0a86e0 comp rx=0 tx=0).stop 2024-09-10T21:26:11.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.638+0000 7f93f5c00640 1 -- 172.21.15.17:0/3749877800 reap_dead start 2024-09-10T21:26:11.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.638+0000 7f93f6600640 1 -- 172.21.15.17:0/3749877800 shutdown_connections 2024-09-10T21:26:11.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.638+0000 7f93f6600640 1 --2- 172.21.15.17:0/3749877800 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f93f00c9520 0x7f93f006e7f0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:11.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.638+0000 7f93f6600640 1 -- 172.21.15.17:0/3749877800 >> 172.21.15.17:0/3749877800 conn(0x7f93f0068990 msgr2=0x7f93f0155d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:11.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.638+0000 7f93f6600640 1 -- 172.21.15.17:0/3749877800 shutdown_connections 2024-09-10T21:26:11.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.638+0000 7f93f6600640 1 -- 172.21.15.17:0/3749877800 wait complete. 2024-09-10T21:26:11.666 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-10T21:26:11.666 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-09-10T21:26:11.667 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-09-10T21:26:11.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.818+0000 7f2f5ce00640 1 -- 172.21.15.17:0/2813838276 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f2f580b2d80 msgr2=0x7f2f58158310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:11.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.818+0000 7f2f5ce00640 1 --2- 172.21.15.17:0/2813838276 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f2f580b2d80 0x7f2f58158310 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f2f44009870 tx=0x7f2f4401f070 comp rx=0 tx=0).stop 2024-09-10T21:26:11.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.818+0000 7f2f5ce00640 1 -- 172.21.15.17:0/2813838276 shutdown_connections 2024-09-10T21:26:11.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.818+0000 7f2f5ce00640 1 --2- 172.21.15.17:0/2813838276 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f2f580b2d80 0x7f2f58158310 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:11.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.818+0000 7f2f5ce00640 1 -- 172.21.15.17:0/2813838276 >> 172.21.15.17:0/2813838276 conn(0x7f2f58070ca0 msgr2=0x7f2f580710b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:11.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.818+0000 7f2f5ce00640 1 -- 172.21.15.17:0/2813838276 shutdown_connections 2024-09-10T21:26:11.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.822+0000 7f2f5ce00640 1 -- 172.21.15.17:0/2813838276 wait complete. 2024-09-10T21:26:11.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.822+0000 7f2f5ce00640 1 Processor -- start 2024-09-10T21:26:11.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.822+0000 7f2f5ce00640 1 -- start start 2024-09-10T21:26:11.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.826+0000 7f2f5ce00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2f5806fd00 0x7f2f581f2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:11.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.826+0000 7f2f57400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2f5806fd00 0x7f2f581f2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:11.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.826+0000 7f2f57400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2f5806fd00 0x7f2f581f2160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:46770/0 (socket says 172.21.15.17:46770) 2024-09-10T21:26:11.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.826+0000 7f2f57400640 1 -- 172.21.15.17:0/1878792357 learned_addr learned my addr 172.21.15.17:0/1878792357 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:26:11.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.826+0000 7f2f5ce00640 1 --2- 172.21.15.17:0/1878792357 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2f580b2d80 0x7f2f58156750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:11.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.826+0000 7f2f5ce00640 1 --2- 172.21.15.17:0/1878792357 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f2f581527e0 0x7f2f581f0380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:11.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.826+0000 7f2f5ce00640 1 -- 172.21.15.17:0/1878792357 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f2f5806f500 con 0x7f2f5806fd00 2024-09-10T21:26:11.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.826+0000 7f2f5ce00640 1 -- 172.21.15.17:0/1878792357 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f2f580701b0 con 0x7f2f580b2d80 2024-09-10T21:26:11.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.826+0000 7f2f5ce00640 1 -- 172.21.15.17:0/1878792357 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f2f5806ed40 con 0x7f2f581527e0 2024-09-10T21:26:11.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.826+0000 7f2f57400640 1 -- 172.21.15.17:0/1878792357 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f2f581527e0 msgr2=0x7f2f581f0380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:11.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.826+0000 7f2f57400640 1 --2- 172.21.15.17:0/1878792357 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f2f581527e0 0x7f2f581f0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:11.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.826+0000 7f2f56a00640 1 --2- 172.21.15.17:0/1878792357 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2f580b2d80 0x7f2f58156750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:11.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.826+0000 7f2f57400640 1 -- 172.21.15.17:0/1878792357 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2f580b2d80 msgr2=0x7f2f58156750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:11.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.826+0000 7f2f57400640 1 --2- 172.21.15.17:0/1878792357 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2f580b2d80 0x7f2f58156750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:11.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.826+0000 7f2f57400640 1 -- 172.21.15.17:0/1878792357 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f581f26a0 con 0x7f2f5806fd00 2024-09-10T21:26:11.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.826+0000 7f2f56a00640 1 --2- 172.21.15.17:0/1878792357 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2f580b2d80 0x7f2f58156750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:26:11.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.826+0000 7f2f57400640 1 --2- 172.21.15.17:0/1878792357 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2f5806fd00 0x7f2f581f2160 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f2f4800c410 tx=0x7f2f4800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:11.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.826+0000 7f2f4f400640 1 -- 172.21.15.17:0/1878792357 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f48011790 con 0x7f2f5806fd00 2024-09-10T21:26:11.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.826+0000 7f2f4f400640 1 -- 172.21.15.17:0/1878792357 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2f48017070 con 0x7f2f5806fd00 2024-09-10T21:26:11.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.826+0000 7f2f4f400640 1 -- 172.21.15.17:0/1878792357 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f48011d40 con 0x7f2f5806fd00 2024-09-10T21:26:11.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.826+0000 7f2f5ce00640 1 -- 172.21.15.17:0/1878792357 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f580b22b0 con 0x7f2f5806fd00 2024-09-10T21:26:11.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.826+0000 7f2f5ce00640 1 -- 172.21.15.17:0/1878792357 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f581ef8a0 con 0x7f2f5806fd00 2024-09-10T21:26:11.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.826+0000 7f2f5ce00640 1 -- 172.21.15.17:0/1878792357 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f58158310 con 0x7f2f5806fd00 2024-09-10T21:26:11.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.830+0000 7f2f4f400640 1 -- 172.21.15.17:0/1878792357 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f2f4801c460 con 0x7f2f5806fd00 2024-09-10T21:26:11.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.830+0000 7f2f4f400640 1 --2- 172.21.15.17:0/1878792357 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f2f38072610 0x7f2f38074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:11.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.830+0000 7f2f4f400640 1 -- 172.21.15.17:0/1878792357 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f2f480934a0 con 0x7f2f5806fd00 2024-09-10T21:26:11.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.834+0000 7f2f56a00640 1 --2- 172.21.15.17:0/1878792357 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f2f38072610 0x7f2f38074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:11.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.834+0000 7f2f56a00640 1 --2- 172.21.15.17:0/1878792357 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f2f38072610 0x7f2f38074ad0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f2f440b2000 tx=0x7f2f4401d8b0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:11.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:11.842+0000 7f2f4f400640 1 -- 172.21.15.17:0/1878792357 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f2f4805e770 con 0x7f2f5806fd00 2024-09-10T21:26:12.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:12.114+0000 7f2f5ce00640 1 -- 172.21.15.17:0/1878792357 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) -- 0x7f2f58155d30 con 0x7f2f5806fd00 2024-09-10T21:26:12.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:12.114+0000 7f2f4f400640 1 -- 172.21.15.17:0/1878792357 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) ==== 57+0+0 (secure 0 0 0) 0x7f2f4805e070 con 0x7f2f5806fd00 2024-09-10T21:26:12.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:12.118+0000 7f2f5ce00640 1 -- 172.21.15.17:0/1878792357 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f2f38072610 msgr2=0x7f2f38074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:12.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:12.118+0000 7f2f5ce00640 1 --2- 172.21.15.17:0/1878792357 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f2f38072610 0x7f2f38074ad0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f2f440b2000 tx=0x7f2f4401d8b0 comp rx=0 tx=0).stop 2024-09-10T21:26:12.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:12.118+0000 7f2f5ce00640 1 -- 172.21.15.17:0/1878792357 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2f5806fd00 msgr2=0x7f2f581f2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:12.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:12.118+0000 7f2f5ce00640 1 --2- 172.21.15.17:0/1878792357 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2f5806fd00 0x7f2f581f2160 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f2f4800c410 tx=0x7f2f4800c8d0 comp rx=0 tx=0).stop 2024-09-10T21:26:12.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:12.118+0000 7f2f5ce00640 1 -- 172.21.15.17:0/1878792357 shutdown_connections 2024-09-10T21:26:12.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:12.118+0000 7f2f5ce00640 1 --2- 172.21.15.17:0/1878792357 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f2f38072610 0x7f2f38074ad0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:12.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:12.118+0000 7f2f5ce00640 1 --2- 172.21.15.17:0/1878792357 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f2f581527e0 0x7f2f581f0380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:12.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:12.118+0000 7f2f5ce00640 1 --2- 172.21.15.17:0/1878792357 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2f580b2d80 0x7f2f58156750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:12.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:12.118+0000 7f2f5ce00640 1 --2- 172.21.15.17:0/1878792357 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2f5806fd00 0x7f2f581f2160 secure :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f2f4800c410 tx=0x7f2f4800c8d0 comp rx=0 tx=0).stop 2024-09-10T21:26:12.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:12.118+0000 7f2f5ce00640 1 -- 172.21.15.17:0/1878792357 >> 172.21.15.17:0/1878792357 conn(0x7f2f58070ca0 msgr2=0x7f2f580c7c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:12.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:12.118+0000 7f2f5ce00640 1 -- 172.21.15.17:0/1878792357 shutdown_connections 2024-09-10T21:26:12.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:12.118+0000 7f2f5ce00640 1 -- 172.21.15.17:0/1878792357 wait complete. 2024-09-10T21:26:12.152 INFO:tasks.mon_thrash.mon_thrasher:waiting for 10.0 secs before continuing thrashing 2024-09-10T21:26:15.591 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:26:15.596 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:26:22.153 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-09-10T21:26:22.153 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-10T21:26:22.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.297+0000 7fb9ae000640 1 -- 172.21.15.17:0/254398130 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb9a81524d0 msgr2=0x7fb9a815bf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:22.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.297+0000 7fb9ae000640 1 --2- 172.21.15.17:0/254398130 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb9a81524d0 0x7fb9a815bf00 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fb9a80718c0 tx=0x7fb99801f0b0 comp rx=0 tx=0).stop 2024-09-10T21:26:22.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.297+0000 7fb9ae000640 1 -- 172.21.15.17:0/254398130 shutdown_connections 2024-09-10T21:26:22.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.297+0000 7fb9ae000640 1 --2- 172.21.15.17:0/254398130 >> v2:172.21.15.31:3300/0 conn(0x7fb9a815c440 0x7fb9a8070d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:22.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.297+0000 7fb9ae000640 1 --2- 172.21.15.17:0/254398130 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb9a81524d0 0x7fb9a815bf00 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:22.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.297+0000 7fb9ae000640 1 -- 172.21.15.17:0/254398130 >> 172.21.15.17:0/254398130 conn(0x7fb9a80cdef0 msgr2=0x7fb9a80ce2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:22.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.297+0000 7fb9ae000640 1 -- 172.21.15.17:0/254398130 shutdown_connections 2024-09-10T21:26:22.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.297+0000 7fb9ae000640 1 -- 172.21.15.17:0/254398130 wait complete. 2024-09-10T21:26:22.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.301+0000 7fb9ae000640 1 Processor -- start 2024-09-10T21:26:22.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.301+0000 7fb9ae000640 1 -- start start 2024-09-10T21:26:22.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.301+0000 7fb9ae000640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb9a81524d0 0x7fb9a81ef030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:22.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.301+0000 7fb9ae000640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb9a8158c20 0x7fb9a814df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:22.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.301+0000 7fb9ae000640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fb9a815c440 0x7fb9a8151470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:22.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.301+0000 7fb9ae000640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fb9a80c9350 con 0x7fb9a8158c20 2024-09-10T21:26:22.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.301+0000 7fb9ae000640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fb9a80c9070 con 0x7fb9a81524d0 2024-09-10T21:26:22.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.301+0000 7fb9ae000640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7fb9a815b9e0 con 0x7fb9a815c440 2024-09-10T21:26:22.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.301+0000 7fb99fe00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb9a8158c20 0x7fb9a814df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:22.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.301+0000 7fb99fe00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb9a8158c20 0x7fb9a814df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:36668/0 (socket says 172.21.15.17:36668) 2024-09-10T21:26:22.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.301+0000 7fb99fe00640 1 -- 172.21.15.17:0/3425772256 learned_addr learned my addr 172.21.15.17:0/3425772256 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:26:22.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.301+0000 7fb9acc00640 1 --2- 172.21.15.17:0/3425772256 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb9a81524d0 0x7fb9a81ef030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:22.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.301+0000 7fb9ad600640 1 --2- 172.21.15.17:0/3425772256 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fb9a815c440 0x7fb9a8151470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:22.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.301+0000 7fb99fe00640 1 -- 172.21.15.17:0/3425772256 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fb9a815c440 msgr2=0x7fb9a8151470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:22.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.301+0000 7fb99fe00640 1 --2- 172.21.15.17:0/3425772256 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fb9a815c440 0x7fb9a8151470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:22.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.301+0000 7fb99fe00640 1 -- 172.21.15.17:0/3425772256 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb9a81524d0 msgr2=0x7fb9a81ef030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:22.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.301+0000 7fb99fe00640 1 --2- 172.21.15.17:0/3425772256 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb9a81524d0 0x7fb9a81ef030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:22.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.301+0000 7fb99fe00640 1 -- 172.21.15.17:0/3425772256 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9a80cd790 con 0x7fb9a8158c20 2024-09-10T21:26:22.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.301+0000 7fb99fe00640 1 --2- 172.21.15.17:0/3425772256 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb9a8158c20 0x7fb9a814df70 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fb99000c530 tx=0x7fb99000c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:22.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.305+0000 7fb9a5600640 1 -- 172.21.15.17:0/3425772256 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb990017070 con 0x7fb9a8158c20 2024-09-10T21:26:22.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.305+0000 7fb9a5600640 1 -- 172.21.15.17:0/3425772256 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb9900053c0 con 0x7fb9a8158c20 2024-09-10T21:26:22.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.305+0000 7fb9a5600640 1 -- 172.21.15.17:0/3425772256 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb990012890 con 0x7fb9a8158c20 2024-09-10T21:26:22.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.305+0000 7fb9ae000640 1 -- 172.21.15.17:0/3425772256 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9a81594e0 con 0x7fb9a8158c20 2024-09-10T21:26:22.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.305+0000 7fb9ae000640 1 -- 172.21.15.17:0/3425772256 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9a80cd0d0 con 0x7fb9a8158c20 2024-09-10T21:26:22.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.305+0000 7fb9ae000640 1 -- 172.21.15.17:0/3425772256 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9a80b3e00 con 0x7fb9a8158c20 2024-09-10T21:26:22.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.305+0000 7fb9a5600640 1 -- 172.21.15.17:0/3425772256 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fb990003600 con 0x7fb9a8158c20 2024-09-10T21:26:22.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.305+0000 7fb9a5600640 1 --2- 172.21.15.17:0/3425772256 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fb984076a20 0x7fb984078ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:22.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.309+0000 7fb9a5600640 1 -- 172.21.15.17:0/3425772256 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fb990092fe0 con 0x7fb9a8158c20 2024-09-10T21:26:22.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.309+0000 7fb9acc00640 1 --2- 172.21.15.17:0/3425772256 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fb984076a20 0x7fb984078ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:22.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.309+0000 7fb9acc00640 1 --2- 172.21.15.17:0/3425772256 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fb984076a20 0x7fb984078ee0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fb9980b2000 tx=0x7fb99801d8f0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:22.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.309+0000 7fb9a5600640 1 -- 172.21.15.17:0/3425772256 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fb990021020 con 0x7fb9a8158c20 2024-09-10T21:26:22.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.549+0000 7fb9ae000640 1 -- 172.21.15.17:0/3425772256 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fb9a81588a0 con 0x7fb9a8158c20 2024-09-10T21:26:22.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.549+0000 7fb9a5600640 1 -- 172.21.15.17:0/3425772256 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1557 (secure 0 0 0) 0x7fb9a81528b0 con 0x7fb9a8158c20 2024-09-10T21:26:22.554 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":11,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5e2f023f-d6a6-49f4-a256-f4807dce3739","modified":"2024-09-10T21:23:46.892130Z","created":"2024-09-10T21:23:46.892130Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3301","nonce":0},{"type":"v1","addr":"172.21.15.31:6790","nonce":0}]},"addr":"172.21.15.31:6790/0","public_addr":"172.21.15.31:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-10T21:26:22.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.553+0000 7fb9ae000640 1 -- 172.21.15.17:0/3425772256 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fb984076a20 msgr2=0x7fb984078ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:22.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.553+0000 7fb9ae000640 1 --2- 172.21.15.17:0/3425772256 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fb984076a20 0x7fb984078ee0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fb9980b2000 tx=0x7fb99801d8f0 comp rx=0 tx=0).stop 2024-09-10T21:26:22.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.553+0000 7fb9ae000640 1 -- 172.21.15.17:0/3425772256 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb9a8158c20 msgr2=0x7fb9a814df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:22.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.553+0000 7fb9ae000640 1 --2- 172.21.15.17:0/3425772256 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb9a8158c20 0x7fb9a814df70 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fb99000c530 tx=0x7fb99000c9f0 comp rx=0 tx=0).stop 2024-09-10T21:26:22.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.553+0000 7fb9ae000640 1 -- 172.21.15.17:0/3425772256 shutdown_connections 2024-09-10T21:26:22.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.553+0000 7fb9ae000640 1 --2- 172.21.15.17:0/3425772256 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fb984076a20 0x7fb984078ee0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:22.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.553+0000 7fb9ae000640 1 --2- 172.21.15.17:0/3425772256 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fb9a815c440 0x7fb9a8151470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:22.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.553+0000 7fb9ae000640 1 --2- 172.21.15.17:0/3425772256 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb9a8158c20 0x7fb9a814df70 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:22.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.553+0000 7fb9ae000640 1 --2- 172.21.15.17:0/3425772256 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb9a81524d0 0x7fb9a81ef030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:22.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.553+0000 7fb9ae000640 1 -- 172.21.15.17:0/3425772256 >> 172.21.15.17:0/3425772256 conn(0x7fb9a80cdef0 msgr2=0x7fb9a81567d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:22.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.553+0000 7fb9ae000640 1 -- 172.21.15.17:0/3425772256 shutdown_connections 2024-09-10T21:26:22.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.553+0000 7fb9ae000640 1 -- 172.21.15.17:0/3425772256 wait complete. 2024-09-10T21:26:22.577 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-09-10T21:26:22.578 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-09-10T21:26:22.578 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-09-10T21:26:22.578 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-10T21:26:22.598 INFO:tasks.workunit.client.0.smithi017.stderr:Updating files: 67% (8171/12088) Updating files: 68% (8220/12088) Updating files: 69% (8341/12088) Updating files: 70% (8462/12088) Updating files: 71% (8583/12088) Updating files: 72% (8704/12088) Updating files: 73% (8825/12088) Updating files: 74% (8946/12088) Updating files: 75% (9066/12088) Updating files: 76% (9187/12088) Updating files: 77% (9308/12088) Updating files: 78% (9429/12088) Updating files: 79% (9550/12088) Updating files: 80% (9671/12088) Updating files: 81% (9792/12088) Updating files: 82% (9913/12088) Updating files: 83% (10034/12088) Updating files: 84% (10154/12088) Updating files: 85% (10275/12088) Updating files: 86% (10396/12088) Updating files: 87% (10517/12088) Updating files: 88% (10638/12088) Updating files: 89% (10759/12088) Updating files: 90% (10880/12088) Updating files: 91% (11001/12088) Updating files: 92% (11121/12088) Updating files: 93% (11242/12088) Updating files: 94% (11363/12088) Updating files: 95% (11484/12088) Updating files: 96% (11605/12088) Updating files: 97% (11726/12088) Updating files: 98% (11847/12088) Updating files: 99% (11968/12088) Updating files: 100% (12088/12088) Updating files: 100% (12088/12088), done. 2024-09-10T21:26:22.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.725+0000 7f0e81200640 1 -- 172.21.15.17:0/167691453 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0e7c152820 msgr2=0x7f0e7c0b3eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:22.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.725+0000 7f0e81200640 1 --2- 172.21.15.17:0/167691453 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0e7c152820 0x7f0e7c0b3eb0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f0e68009900 tx=0x7f0e6801c850 comp rx=0 tx=0).stop 2024-09-10T21:26:22.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.725+0000 7f0e81200640 1 -- 172.21.15.17:0/167691453 shutdown_connections 2024-09-10T21:26:22.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.725+0000 7f0e81200640 1 --2- 172.21.15.17:0/167691453 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f0e7c0cdf20 0x7f0e7c0ce350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:22.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.725+0000 7f0e81200640 1 --2- 172.21.15.17:0/167691453 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0e7c152820 0x7f0e7c0b3eb0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:22.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.725+0000 7f0e81200640 1 -- 172.21.15.17:0/167691453 >> 172.21.15.17:0/167691453 conn(0x7f0e7c0702f0 msgr2=0x7f0e7c070700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:22.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.725+0000 7f0e81200640 1 -- 172.21.15.17:0/167691453 shutdown_connections 2024-09-10T21:26:22.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.725+0000 7f0e81200640 1 -- 172.21.15.17:0/167691453 wait complete. 2024-09-10T21:26:22.732 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.725+0000 7f0e81200640 1 Processor -- start 2024-09-10T21:26:22.732 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.729+0000 7f0e81200640 1 -- start start 2024-09-10T21:26:22.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.729+0000 7f0e81200640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f0e7c0cdf20 0x7f0e7c06f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:22.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.729+0000 7f0e81200640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0e7c152440 0x7f0e7c0c9c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:22.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.729+0000 7f0e81200640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0e7c152820 0x7f0e7c06ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:22.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.729+0000 7f0e81200640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f0e7c0cb5d0 con 0x7f0e7c152820 2024-09-10T21:26:22.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.729+0000 7f0e81200640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f0e7c0cb450 con 0x7f0e7c152440 2024-09-10T21:26:22.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.729+0000 7f0e81200640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f0e7c0c2ff0 con 0x7f0e7c0cdf20 2024-09-10T21:26:22.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.729+0000 7f0e7a800640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0e7c152820 0x7f0e7c06ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:22.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.729+0000 7f0e7a800640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0e7c152820 0x7f0e7c06ddc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:36682/0 (socket says 172.21.15.17:36682) 2024-09-10T21:26:22.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.729+0000 7f0e7a800640 1 -- 172.21.15.17:0/3950911082 learned_addr learned my addr 172.21.15.17:0/3950911082 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:26:22.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.729+0000 7f0e79e00640 1 --2- 172.21.15.17:0/3950911082 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f0e7c0cdf20 0x7f0e7c06f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:22.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.729+0000 7f0e79400640 1 --2- 172.21.15.17:0/3950911082 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0e7c152440 0x7f0e7c0c9c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:22.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.729+0000 7f0e7a800640 1 -- 172.21.15.17:0/3950911082 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f0e7c0cdf20 msgr2=0x7f0e7c06f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:22.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.729+0000 7f0e7a800640 1 --2- 172.21.15.17:0/3950911082 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f0e7c0cdf20 0x7f0e7c06f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:22.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.729+0000 7f0e7a800640 1 -- 172.21.15.17:0/3950911082 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0e7c152440 msgr2=0x7f0e7c0c9c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:22.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.729+0000 7f0e7a800640 1 --2- 172.21.15.17:0/3950911082 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0e7c152440 0x7f0e7c0c9c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:22.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.729+0000 7f0e7a800640 1 -- 172.21.15.17:0/3950911082 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e7c1466b0 con 0x7f0e7c152820 2024-09-10T21:26:22.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.729+0000 7f0e7a800640 1 --2- 172.21.15.17:0/3950911082 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0e7c152820 0x7f0e7c06ddc0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f0e7400e970 tx=0x7f0e7400ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:22.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.729+0000 7f0e72000640 1 -- 172.21.15.17:0/3950911082 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e74004d30 con 0x7f0e7c152820 2024-09-10T21:26:22.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.729+0000 7f0e72000640 1 -- 172.21.15.17:0/3950911082 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0e7400cac0 con 0x7f0e7c152820 2024-09-10T21:26:22.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.729+0000 7f0e81200640 1 -- 172.21.15.17:0/3950911082 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e7c161580 con 0x7f0e7c152820 2024-09-10T21:26:22.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.729+0000 7f0e72000640 1 -- 172.21.15.17:0/3950911082 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e74005020 con 0x7f0e7c152820 2024-09-10T21:26:22.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.729+0000 7f0e81200640 1 -- 172.21.15.17:0/3950911082 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e7c06ecc0 con 0x7f0e7c152820 2024-09-10T21:26:22.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.729+0000 7f0e81200640 1 --2- 172.21.15.17:0/3950911082 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0e7c06e300 0x7f0e7c1f6810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:22.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.729+0000 7f0e81200640 1 -- 172.21.15.17:0/3950911082 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f0e7c0cac50 con 0x7f0e7c06e300 2024-09-10T21:26:22.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.729+0000 7f0e79e00640 1 --2- 172.21.15.17:0/3950911082 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0e7c06e300 0x7f0e7c1f6810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:22.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.729+0000 7f0e79e00640 1 -- 172.21.15.17:0/3950911082 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0e7c1611b0 con 0x7f0e7c06e300 2024-09-10T21:26:22.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.733+0000 7f0e79e00640 1 --2- 172.21.15.17:0/3950911082 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0e7c06e300 0x7f0e7c1f6810 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f0e6c00f770 tx=0x7f0e6c00fc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:22.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.733+0000 7f0e72000640 1 -- 172.21.15.17:0/3950911082 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e6c0050d0 con 0x7f0e7c06e300 2024-09-10T21:26:22.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.733+0000 7f0e72000640 1 -- 172.21.15.17:0/3950911082 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f0e7400f040 con 0x7f0e7c152820 2024-09-10T21:26:22.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.733+0000 7f0e72000640 1 --2- 172.21.15.17:0/3950911082 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f0e5c072510 0x7f0e5c0749d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:22.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.733+0000 7f0e72000640 1 -- 172.21.15.17:0/3950911082 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f0e74093ad0 con 0x7f0e7c152820 2024-09-10T21:26:22.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.733+0000 7f0e72000640 1 -- 172.21.15.17:0/3950911082 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f0e6c005270 con 0x7f0e7c06e300 2024-09-10T21:26:22.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.733+0000 7f0e72000640 1 -- 172.21.15.17:0/3950911082 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0e7c06e300 msgr2=0x7f0e7c1f6810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:22.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.733+0000 7f0e72000640 1 --2- 172.21.15.17:0/3950911082 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0e7c06e300 0x7f0e7c1f6810 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f0e6c00f770 tx=0x7f0e6c00fc30 comp rx=0 tx=0).stop 2024-09-10T21:26:22.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.733+0000 7f0e72000640 1 -- 172.21.15.17:0/3950911082 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0e7c06e300 msgr2=0x7f0e7c1f6810 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:26:22.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.733+0000 7f0e72000640 1 --2- 172.21.15.17:0/3950911082 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0e7c06e300 0x7f0e7c1f6810 secure :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f0e6c00f770 tx=0x7f0e6c00fc30 comp rx=0 tx=0).stop 2024-09-10T21:26:22.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.733+0000 7f0e79400640 1 --2- 172.21.15.17:0/3950911082 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f0e5c072510 0x7f0e5c0749d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:22.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.733+0000 7f0e79400640 1 --2- 172.21.15.17:0/3950911082 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f0e5c072510 0x7f0e5c0749d0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f0e680af000 tx=0x7f0e6801b090 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:22.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.741+0000 7f0e81200640 1 --2- 172.21.15.17:0/3950911082 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0e7c06ef00 0x7f0e7c1f6d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:22.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.741+0000 7f0e81200640 1 -- 172.21.15.17:0/3950911082 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f0e7c158360 con 0x7f0e7c06ef00 2024-09-10T21:26:22.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.741+0000 7f0e79e00640 1 --2- 172.21.15.17:0/3950911082 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0e7c06ef00 0x7f0e7c1f6d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:22.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.741+0000 7f0e79e00640 1 -- 172.21.15.17:0/3950911082 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f0e7c14f690 con 0x7f0e7c06ef00 2024-09-10T21:26:22.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.741+0000 7f0e79e00640 1 --2- 172.21.15.17:0/3950911082 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0e7c06ef00 0x7f0e7c1f6d50 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f0e6c00f770 tx=0x7f0e6c001090 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:22.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.741+0000 7f0e72000640 1 -- 172.21.15.17:0/3950911082 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e6c00d960 con 0x7f0e7c06ef00 2024-09-10T21:26:22.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.741+0000 7f0e72000640 1 -- 172.21.15.17:0/3950911082 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4543 (secure 0 0 0) 0x7f0e6c00d960 con 0x7f0e7c06ef00 2024-09-10T21:26:22.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.741+0000 7f0e72000640 1 -- 172.21.15.17:0/3950911082 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0e7c06ef00 msgr2=0x7f0e7c1f6d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:22.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.741+0000 7f0e72000640 1 --2- 172.21.15.17:0/3950911082 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0e7c06ef00 0x7f0e7c1f6d50 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f0e6c00f770 tx=0x7f0e6c001090 comp rx=0 tx=0).stop 2024-09-10T21:26:22.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.741+0000 7f0e72000640 1 -- 172.21.15.17:0/3950911082 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0e7c06ef00 msgr2=0x7f0e7c1f6d50 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:26:22.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.741+0000 7f0e72000640 1 --2- 172.21.15.17:0/3950911082 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0e7c06ef00 0x7f0e7c1f6d50 secure :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f0e6c00f770 tx=0x7f0e6c001090 comp rx=0 tx=0).stop 2024-09-10T21:26:22.746 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-10T21:26:22.747 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:26:22.747 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:26:22.747 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "leader", 2024-09-10T21:26:22.747 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 10, 2024-09-10T21:26:22.747 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 12647, 2024-09-10T21:26:22.747 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-10T21:26:22.747 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-10T21:26:22.747 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-10T21:26:22.747 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-10T21:26:22.747 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:26:22.747 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 12, 2024-09-10T21:26:22.747 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:26:22.747 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-10T21:26:22.748 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-10T21:26:22.748 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:26:22.748 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:26:22.748 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:26:22.748 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:26:22.748 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:26:22.748 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:26:22.748 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:26:22.748 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:26:22.748 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:26:22.748 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:26:22.748 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:26:22.748 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:26:22.748 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-10T21:26:22.749 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-10T21:26:22.749 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:26:22.749 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:26:22.749 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:26:22.749 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:26:22.749 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:26:22.749 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:26:22.749 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:26:22.749 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:26:22.749 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:26:22.749 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:26:22.749 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:26:22.749 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:26:22.750 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:26:22.750 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-10T21:26:22.750 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-10T21:26:22.750 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-10T21:26:22.750 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-10T21:26:22.750 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-10T21:26:22.750 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "5e2f023f-d6a6-49f4-a256-f4807dce3739", 2024-09-10T21:26:22.750 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:26:22.750 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:26:22.750 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-10T21:26:22.750 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-10T21:26:22.750 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-10T21:26:22.750 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-10T21:26:22.751 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-10T21:26:22.751 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-10T21:26:22.751 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-10T21:26:22.751 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:26:22.751 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-10T21:26:22.751 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:26:22.751 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:26:22.751 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:26:22.751 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:26:22.751 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:26:22.751 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:26:22.751 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:26:22.751 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:26:22.751 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:26:22.752 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:26:22.752 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:26:22.752 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:26:22.752 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-10T21:26:22.752 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:26:22.752 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-10T21:26:22.752 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:26:22.752 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:26:22.752 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:26:22.752 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:26:22.752 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:26:22.752 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:26:22.752 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:26:22.753 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-10T21:26:22.753 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:26:22.753 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:26:22.753 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:26:22.753 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:26:22.753 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-10T21:26:22.753 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:26:22.753 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:26:22.753 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:26:22.753 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:26:22.753 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-10T21:26:22.753 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-10T21:26:22.753 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:26:22.754 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:26:22.754 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:26:22.754 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:26:22.754 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:26:22.754 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:26:22.754 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:26:22.754 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:26:22.754 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:26:22.754 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:26:22.754 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:26:22.754 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3300", 2024-09-10T21:26:22.754 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:26:22.754 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:26:22.755 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:26:22.755 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:26:22.755 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789", 2024-09-10T21:26:22.755 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:26:22.755 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:26:22.755 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:26:22.755 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:26:22.755 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789/0", 2024-09-10T21:26:22.755 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6789/0", 2024-09-10T21:26:22.755 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:26:22.755 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:26:22.755 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:26:22.755 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:26:22.755 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:26:22.756 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:26:22.756 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:26:22.756 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:26:22.756 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:26:22.756 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:26:22.756 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:26:22.756 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3301", 2024-09-10T21:26:22.756 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:26:22.756 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:26:22.756 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:26:22.756 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:26:22.756 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790", 2024-09-10T21:26:22.756 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:26:22.757 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:26:22.757 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:26:22.757 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:26:22.757 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790/0", 2024-09-10T21:26:22.757 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6790/0", 2024-09-10T21:26:22.757 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:26:22.757 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:26:22.757 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:26:22.757 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:26:22.757 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:26:22.757 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:26:22.757 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-10T21:26:22.757 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-10T21:26:22.758 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:26:22.758 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:26:22.758 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:26:22.758 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:26:22.758 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:26:22.758 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:26:22.758 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-10T21:26:22.758 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:26:22.758 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:26:22.758 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:26:22.758 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 2 2024-09-10T21:26:22.758 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:26:22.758 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:26:22.758 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:26:22.759 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-10T21:26:22.759 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-10T21:26:22.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.741+0000 7f0e81200640 1 -- 172.21.15.17:0/3950911082 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f0e5c072510 msgr2=0x7f0e5c0749d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:22.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.741+0000 7f0e81200640 1 --2- 172.21.15.17:0/3950911082 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f0e5c072510 0x7f0e5c0749d0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f0e680af000 tx=0x7f0e6801b090 comp rx=0 tx=0).stop 2024-09-10T21:26:22.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.741+0000 7f0e81200640 1 -- 172.21.15.17:0/3950911082 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0e7c152820 msgr2=0x7f0e7c06ddc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:22.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.741+0000 7f0e81200640 1 --2- 172.21.15.17:0/3950911082 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0e7c152820 0x7f0e7c06ddc0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f0e7400e970 tx=0x7f0e7400ee30 comp rx=0 tx=0).stop 2024-09-10T21:26:22.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.741+0000 7f0e7a800640 1 -- 172.21.15.17:0/3950911082 reap_dead start 2024-09-10T21:26:22.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.741+0000 7f0e81200640 1 -- 172.21.15.17:0/3950911082 shutdown_connections 2024-09-10T21:26:22.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.741+0000 7f0e81200640 1 --2- 172.21.15.17:0/3950911082 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0e7c152820 0x7f0e7c06ddc0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:22.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.741+0000 7f0e81200640 1 -- 172.21.15.17:0/3950911082 >> 172.21.15.17:0/3950911082 conn(0x7f0e7c0702f0 msgr2=0x7f0e7c0b3b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:22.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.741+0000 7f0e81200640 1 -- 172.21.15.17:0/3950911082 shutdown_connections 2024-09-10T21:26:22.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.741+0000 7f0e81200640 1 -- 172.21.15.17:0/3950911082 wait complete. 2024-09-10T21:26:22.765 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-10T21:26:22.765 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-09-10T21:26:22.765 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-10T21:26:22.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.953+0000 7ff273800640 1 -- 172.21.15.17:0/106670144 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff26c152440 msgr2=0x7ff254003a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:22.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.953+0000 7ff273800640 1 --2- 172.21.15.17:0/106670144 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff26c152440 0x7ff254003a30 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7ff260009b60 tx=0x7ff26001d280 comp rx=0 tx=0).stop 2024-09-10T21:26:22.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.953+0000 7ff271800640 1 -- 172.21.15.17:0/106670144 reap_dead start 2024-09-10T21:26:22.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.953+0000 7ff273800640 1 -- 172.21.15.17:0/106670144 shutdown_connections 2024-09-10T21:26:22.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.953+0000 7ff273800640 1 -- 172.21.15.17:0/106670144 >> 172.21.15.17:0/106670144 conn(0x7ff26c0702f0 msgr2=0x7ff26c070700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:22.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.953+0000 7ff273800640 1 -- 172.21.15.17:0/106670144 shutdown_connections 2024-09-10T21:26:22.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.953+0000 7ff273800640 1 -- 172.21.15.17:0/106670144 wait complete. 2024-09-10T21:26:22.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.957+0000 7ff273800640 1 Processor -- start 2024-09-10T21:26:22.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.957+0000 7ff273800640 1 -- start start 2024-09-10T21:26:22.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.957+0000 7ff273800640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff26c152440 0x7ff26c153660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:22.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.957+0000 7ff273800640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff26c151bc0 0x7ff26c1fb9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:22.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.957+0000 7ff273800640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff26c1fbf20 0x7ff26c155fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:22.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.957+0000 7ff273800640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7ff26c0ce740 con 0x7ff26c151bc0 2024-09-10T21:26:22.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.957+0000 7ff273800640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7ff26c0b42c0 con 0x7ff26c152440 2024-09-10T21:26:22.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.957+0000 7ff273800640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7ff26c15c7c0 con 0x7ff26c1fbf20 2024-09-10T21:26:22.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.957+0000 7ff26be00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff26c151bc0 0x7ff26c1fb9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:22.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.957+0000 7ff26be00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff26c151bc0 0x7ff26c1fb9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:36724/0 (socket says 172.21.15.17:36724) 2024-09-10T21:26:22.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.957+0000 7ff26be00640 1 -- 172.21.15.17:0/15599743 learned_addr learned my addr 172.21.15.17:0/15599743 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:26:22.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.957+0000 7ff271800640 1 --2- 172.21.15.17:0/15599743 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff26c1fbf20 0x7ff26c155fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:22.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.957+0000 7ff270e00640 1 --2- 172.21.15.17:0/15599743 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff26c152440 0x7ff26c153660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:22.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.957+0000 7ff26be00640 1 -- 172.21.15.17:0/15599743 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff26c1fbf20 msgr2=0x7ff26c155fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:22.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.957+0000 7ff26be00640 1 --2- 172.21.15.17:0/15599743 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff26c1fbf20 0x7ff26c155fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:22.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.957+0000 7ff26be00640 1 -- 172.21.15.17:0/15599743 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff26c152440 msgr2=0x7ff26c153660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:22.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.957+0000 7ff26be00640 1 --2- 172.21.15.17:0/15599743 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff26c152440 0x7ff26c153660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:22.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.957+0000 7ff26be00640 1 -- 172.21.15.17:0/15599743 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff26c153c70 con 0x7ff26c151bc0 2024-09-10T21:26:22.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.957+0000 7ff26be00640 1 --2- 172.21.15.17:0/15599743 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff26c151bc0 0x7ff26c1fb9e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7ff258011d50 tx=0x7ff25800f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:22.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.957+0000 7ff268c00640 1 -- 172.21.15.17:0/15599743 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff25801e070 con 0x7ff26c151bc0 2024-09-10T21:26:22.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.957+0000 7ff268c00640 1 -- 172.21.15.17:0/15599743 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff25800e7c0 con 0x7ff26c151bc0 2024-09-10T21:26:22.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.957+0000 7ff273800640 1 -- 172.21.15.17:0/15599743 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff26c155880 con 0x7ff26c151bc0 2024-09-10T21:26:22.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.957+0000 7ff268c00640 1 -- 172.21.15.17:0/15599743 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff25800ebb0 con 0x7ff26c151bc0 2024-09-10T21:26:22.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.957+0000 7ff273800640 1 -- 172.21.15.17:0/15599743 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff26c157450 con 0x7ff26c151bc0 2024-09-10T21:26:22.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.961+0000 7ff273800640 1 --2- 172.21.15.17:0/15599743 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff26c1fb310 0x7ff26c157730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:22.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.961+0000 7ff273800640 1 -- 172.21.15.17:0/15599743 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7ff26c1fb6f0 con 0x7ff26c1fb310 2024-09-10T21:26:22.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.961+0000 7ff270e00640 1 --2- 172.21.15.17:0/15599743 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff26c1fb310 0x7ff26c157730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:22.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.961+0000 7ff270e00640 1 -- 172.21.15.17:0/15599743 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff26c0cb450 con 0x7ff26c1fb310 2024-09-10T21:26:22.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.961+0000 7ff270e00640 1 --2- 172.21.15.17:0/15599743 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff26c1fb310 0x7ff26c157730 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ff260009b30 tx=0x7ff260009380 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:22.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.961+0000 7ff268c00640 1 -- 172.21.15.17:0/15599743 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7ff258019430 con 0x7ff26c151bc0 2024-09-10T21:26:22.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.961+0000 7ff268c00640 1 --2- 172.21.15.17:0/15599743 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7ff24c0725e0 0x7ff24c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:22.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.961+0000 7ff268c00640 1 -- 172.21.15.17:0/15599743 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7ff258094970 con 0x7ff26c151bc0 2024-09-10T21:26:22.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.961+0000 7ff268c00640 1 -- 172.21.15.17:0/15599743 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2600bc070 con 0x7ff26c1fb310 2024-09-10T21:26:22.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.961+0000 7ff268c00640 1 -- 172.21.15.17:0/15599743 <== mon.1 v2:172.21.15.31:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7ff260005040 con 0x7ff26c1fb310 2024-09-10T21:26:22.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.961+0000 7ff271800640 1 --2- 172.21.15.17:0/15599743 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7ff24c0725e0 0x7ff24c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:22.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.961+0000 7ff268c00640 1 -- 172.21.15.17:0/15599743 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff26c1fb310 msgr2=0x7ff26c157730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:22.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.961+0000 7ff268c00640 1 --2- 172.21.15.17:0/15599743 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff26c1fb310 0x7ff26c157730 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ff260009b30 tx=0x7ff260009380 comp rx=0 tx=0).stop 2024-09-10T21:26:22.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.961+0000 7ff268c00640 1 -- 172.21.15.17:0/15599743 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff26c1fb310 msgr2=0x7ff26c157730 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:26:22.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.961+0000 7ff268c00640 1 --2- 172.21.15.17:0/15599743 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff26c1fb310 0x7ff26c157730 secure :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ff260009b30 tx=0x7ff260009380 comp rx=0 tx=0).stop 2024-09-10T21:26:22.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.965+0000 7ff271800640 1 --2- 172.21.15.17:0/15599743 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7ff24c0725e0 0x7ff24c074aa0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7ff26c152c20 tx=0x7ff264006b30 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:22.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.969+0000 7ff273800640 1 --2- 172.21.15.17:0/15599743 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff26c1564e0 0x7ff26c1568c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:22.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.969+0000 7ff273800640 1 -- 172.21.15.17:0/15599743 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7ff26c156e00 con 0x7ff26c1564e0 2024-09-10T21:26:22.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.969+0000 7ff270e00640 1 --2- 172.21.15.17:0/15599743 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff26c1564e0 0x7ff26c1568c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:22.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.969+0000 7ff270e00640 1 -- 172.21.15.17:0/15599743 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7ff26c0ced80 con 0x7ff26c1564e0 2024-09-10T21:26:22.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.969+0000 7ff270e00640 1 --2- 172.21.15.17:0/15599743 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff26c1564e0 0x7ff26c1568c0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7ff260005e10 tx=0x7ff2600091e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:22.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.969+0000 7ff268c00640 1 -- 172.21.15.17:0/15599743 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2600b1040 con 0x7ff26c1564e0 2024-09-10T21:26:22.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.969+0000 7ff268c00640 1 -- 172.21.15.17:0/15599743 <== mon.1 v2:172.21.15.31:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5043 (secure 0 0 0) 0x7ff260004d80 con 0x7ff26c1564e0 2024-09-10T21:26:22.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.969+0000 7ff268c00640 1 -- 172.21.15.17:0/15599743 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff26c1564e0 msgr2=0x7ff26c1568c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:22.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.969+0000 7ff268c00640 1 --2- 172.21.15.17:0/15599743 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff26c1564e0 0x7ff26c1568c0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7ff260005e10 tx=0x7ff2600091e0 comp rx=0 tx=0).stop 2024-09-10T21:26:22.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.969+0000 7ff268c00640 1 -- 172.21.15.17:0/15599743 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff26c1564e0 msgr2=0x7ff26c1568c0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:26:22.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.969+0000 7ff268c00640 1 --2- 172.21.15.17:0/15599743 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff26c1564e0 0x7ff26c1568c0 secure :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7ff260005e10 tx=0x7ff2600091e0 comp rx=0 tx=0).stop 2024-09-10T21:26:22.976 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-10T21:26:22.976 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:26:22.976 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:26:22.976 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-10T21:26:22.976 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 10, 2024-09-10T21:26:22.976 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 121416, 2024-09-10T21:26:22.976 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-10T21:26:22.976 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-10T21:26:22.976 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-10T21:26:22.977 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-10T21:26:22.977 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:26:22.977 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 12, 2024-09-10T21:26:22.977 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:26:22.977 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-10T21:26:22.977 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-10T21:26:22.977 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:26:22.977 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:26:22.977 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:26:22.977 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:26:22.978 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:26:22.978 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:26:22.978 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:26:22.978 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:26:22.978 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:26:22.978 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:26:22.978 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:26:22.978 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:26:22.978 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-10T21:26:22.978 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-10T21:26:22.978 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:26:22.979 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:26:22.979 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:26:22.979 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:26:22.979 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:26:22.979 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:26:22.979 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:26:22.979 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:26:22.979 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:26:22.979 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:26:22.979 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:26:22.979 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:26:22.979 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:26:22.980 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-10T21:26:22.980 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-10T21:26:22.980 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-10T21:26:22.980 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-10T21:26:22.980 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-10T21:26:22.980 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "5e2f023f-d6a6-49f4-a256-f4807dce3739", 2024-09-10T21:26:22.980 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:26:22.980 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:26:22.980 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-10T21:26:22.980 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-10T21:26:22.980 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-10T21:26:22.980 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-10T21:26:22.981 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-10T21:26:22.981 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-10T21:26:22.981 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-10T21:26:22.981 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:26:22.981 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-10T21:26:22.981 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:26:22.981 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:26:22.981 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:26:22.981 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:26:22.981 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:26:22.981 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:26:22.981 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:26:22.981 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:26:22.981 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:26:22.982 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:26:22.982 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:26:22.982 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:26:22.982 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-10T21:26:22.982 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:26:22.982 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-10T21:26:22.982 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:26:22.982 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:26:22.982 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:26:22.982 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:26:22.982 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:26:22.982 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:26:22.982 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:26:22.982 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-10T21:26:22.983 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:26:22.983 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:26:22.983 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:26:22.983 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:26:22.983 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-10T21:26:22.983 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:26:22.983 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:26:22.983 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:26:22.983 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:26:22.983 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-10T21:26:22.983 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-10T21:26:22.984 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:26:22.984 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:26:22.984 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:26:22.984 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:26:22.984 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:26:22.984 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:26:22.984 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:26:22.984 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:26:22.984 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:26:22.984 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:26:22.984 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:26:22.984 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3300", 2024-09-10T21:26:22.984 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:26:22.985 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:26:22.985 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:26:22.985 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:26:22.985 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789", 2024-09-10T21:26:22.985 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:26:22.985 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:26:22.985 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:26:22.985 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:26:22.985 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789/0", 2024-09-10T21:26:22.985 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6789/0", 2024-09-10T21:26:22.985 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:26:22.986 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:26:22.986 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:26:22.986 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:26:22.986 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:26:22.986 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:26:22.986 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:26:22.986 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:26:22.986 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:26:22.986 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:26:22.986 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:26:22.986 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3301", 2024-09-10T21:26:22.986 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:26:22.986 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:26:22.986 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:26:22.987 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:26:22.987 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790", 2024-09-10T21:26:22.987 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:26:22.987 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:26:22.987 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:26:22.987 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:26:22.987 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790/0", 2024-09-10T21:26:22.987 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6790/0", 2024-09-10T21:26:22.987 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:26:22.987 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:26:22.987 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:26:22.987 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:26:22.987 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:26:22.987 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:26:22.988 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-10T21:26:22.988 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-10T21:26:22.988 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:26:22.988 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:26:22.988 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:26:22.988 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:26:22.988 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:26:22.988 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:26:22.988 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": [ 2024-09-10T21:26:22.989 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:26:22.989 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:26:22.989 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:26:22.989 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 5 2024-09-10T21:26:22.989 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:26:22.989 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:26:22.989 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-10T21:26:22.989 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:26:22.990 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:26:22.990 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:26:22.990 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 4 2024-09-10T21:26:22.990 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:26:22.990 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:26:22.990 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-10T21:26:22.990 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:26:22.990 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:26:22.990 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:26:22.990 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 2 2024-09-10T21:26:22.990 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:26:22.990 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:26:22.990 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": [ 2024-09-10T21:26:22.991 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:26:22.991 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:26:22.991 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:26:22.991 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:26:22.991 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:26:22.991 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:26:22.991 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:26:22.991 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-10T21:26:22.991 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-10T21:26:22.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.969+0000 7ff273800640 1 -- 172.21.15.17:0/15599743 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7ff24c0725e0 msgr2=0x7ff24c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:22.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.969+0000 7ff273800640 1 --2- 172.21.15.17:0/15599743 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7ff24c0725e0 0x7ff24c074aa0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7ff26c152c20 tx=0x7ff264006b30 comp rx=0 tx=0).stop 2024-09-10T21:26:22.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.969+0000 7ff273800640 1 -- 172.21.15.17:0/15599743 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff26c151bc0 msgr2=0x7ff26c1fb9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:22.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.969+0000 7ff273800640 1 --2- 172.21.15.17:0/15599743 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff26c151bc0 0x7ff26c1fb9e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7ff258011d50 tx=0x7ff25800f790 comp rx=0 tx=0).stop 2024-09-10T21:26:22.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.969+0000 7ff271800640 1 -- 172.21.15.17:0/15599743 reap_dead start 2024-09-10T21:26:22.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.969+0000 7ff271800640 1 -- 172.21.15.17:0/15599743 reap_dead start 2024-09-10T21:26:22.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.969+0000 7ff273800640 1 -- 172.21.15.17:0/15599743 shutdown_connections 2024-09-10T21:26:22.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.969+0000 7ff273800640 1 -- 172.21.15.17:0/15599743 >> 172.21.15.17:0/15599743 conn(0x7ff26c0702f0 msgr2=0x7ff26c0cc600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:22.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.973+0000 7ff273800640 1 -- 172.21.15.17:0/15599743 shutdown_connections 2024-09-10T21:26:22.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:22.973+0000 7ff273800640 1 -- 172.21.15.17:0/15599743 wait complete. 2024-09-10T21:26:23.002 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-10T21:26:23.002 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-09-10T21:26:23.002 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-10T21:26:23.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:23.145+0000 7f8a21800640 1 -- 172.21.15.17:0/3526585117 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8a1c070b70 msgr2=0x7f8a1c070fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:23.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:23.145+0000 7f8a21800640 1 --2- 172.21.15.17:0/3526585117 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8a1c070b70 0x7f8a1c070fc0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f8a1400acd0 tx=0x7f8a140206d0 comp rx=0 tx=0).stop 2024-09-10T21:26:23.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:23.145+0000 7f8a21800640 1 -- 172.21.15.17:0/3526585117 shutdown_connections 2024-09-10T21:26:23.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:23.145+0000 7f8a21800640 1 --2- 172.21.15.17:0/3526585117 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8a1c070b70 0x7f8a1c070fc0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:23.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:23.145+0000 7f8a21800640 1 -- 172.21.15.17:0/3526585117 >> 172.21.15.17:0/3526585117 conn(0x7f8a1c06cc20 msgr2=0x7f8a1c06d030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:23.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:23.145+0000 7f8a21800640 1 -- 172.21.15.17:0/3526585117 shutdown_connections 2024-09-10T21:26:23.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:23.145+0000 7f8a21800640 1 -- 172.21.15.17:0/3526585117 wait complete. 2024-09-10T21:26:23.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:23.149+0000 7f8a21800640 1 Processor -- start 2024-09-10T21:26:23.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:23.149+0000 7f8a21800640 1 -- start start 2024-09-10T21:26:23.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:23.149+0000 7f8a21800640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8a1c070b70 0x7f8a1c14a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:23.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:23.149+0000 7f8a21800640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8a1c0c78a0 0x7f8a1c1e45b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:23.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:23.149+0000 7f8a21800640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8a1c150aa0 0x7f8a1c14e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:23.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:23.149+0000 7f8a21800640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f8a1c070650 con 0x7f8a1c150aa0 2024-09-10T21:26:23.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:23.149+0000 7f8a21800640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f8a1c1579d0 con 0x7f8a1c070b70 2024-09-10T21:26:23.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:23.149+0000 7f8a21800640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f8a1c0c39d0 con 0x7f8a1c0c78a0 2024-09-10T21:26:23.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:23.149+0000 7f8a1b400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8a1c150aa0 0x7f8a1c14e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:23.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:23.149+0000 7f8a1b400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8a1c150aa0 0x7f8a1c14e270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:36730/0 (socket says 172.21.15.17:36730) 2024-09-10T21:26:23.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:23.149+0000 7f8a1b400640 1 -- 172.21.15.17:0/2721582071 learned_addr learned my addr 172.21.15.17:0/2721582071 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:26:23.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:23.149+0000 7f8a1a000640 1 --2- 172.21.15.17:0/2721582071 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8a1c0c78a0 0x7f8a1c1e45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:23.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:23.149+0000 7f8a1aa00640 1 --2- 172.21.15.17:0/2721582071 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8a1c070b70 0x7f8a1c14a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:23.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:23.149+0000 7f8a1b400640 1 -- 172.21.15.17:0/2721582071 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8a1c0c78a0 msgr2=0x7f8a1c1e45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:23.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:23.149+0000 7f8a1b400640 1 --2- 172.21.15.17:0/2721582071 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8a1c0c78a0 0x7f8a1c1e45b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:23.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:23.149+0000 7f8a1b400640 1 -- 172.21.15.17:0/2721582071 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8a1c070b70 msgr2=0x7f8a1c14a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:23.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:23.149+0000 7f8a1b400640 1 --2- 172.21.15.17:0/2721582071 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8a1c070b70 0x7f8a1c14a200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:23.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:23.149+0000 7f8a1b400640 1 -- 172.21.15.17:0/2721582071 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a1c149760 con 0x7f8a1c150aa0 2024-09-10T21:26:23.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:23.149+0000 7f8a1aa00640 1 --2- 172.21.15.17:0/2721582071 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8a1c070b70 0x7f8a1c14a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:26:23.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:23.149+0000 7f8a1a000640 1 --2- 172.21.15.17:0/2721582071 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8a1c0c78a0 0x7f8a1c1e45b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:26:23.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:23.149+0000 7f8a1b400640 1 --2- 172.21.15.17:0/2721582071 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8a1c150aa0 0x7f8a1c14e270 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f8a1401d2e0 tx=0x7f8a1400a310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:23.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:23.149+0000 7f8a12a00640 1 -- 172.21.15.17:0/2721582071 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a140b2070 con 0x7f8a1c150aa0 2024-09-10T21:26:23.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:23.149+0000 7f8a12a00640 1 -- 172.21.15.17:0/2721582071 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8a140af820 con 0x7f8a1c150aa0 2024-09-10T21:26:23.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:23.149+0000 7f8a12a00640 1 -- 172.21.15.17:0/2721582071 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a140afbd0 con 0x7f8a1c150aa0 2024-09-10T21:26:23.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:23.149+0000 7f8a21800640 1 -- 172.21.15.17:0/2721582071 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a1c14fb90 con 0x7f8a1c150aa0 2024-09-10T21:26:23.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:23.149+0000 7f8a21800640 1 -- 172.21.15.17:0/2721582071 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a1c154aa0 con 0x7f8a1c150aa0 2024-09-10T21:26:23.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:23.149+0000 7f8a21800640 1 --2- 172.21.15.17:0/2721582071 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8a1c14e7b0 0x7f8a1c13f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:23.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:23.149+0000 7f8a21800640 1 -- 172.21.15.17:0/2721582071 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f8a1c0cac60 con 0x7f8a1c14e7b0 2024-09-10T21:26:23.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:23.149+0000 7f8a1aa00640 1 --2- 172.21.15.17:0/2721582071 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8a1c14e7b0 0x7f8a1c13f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:23.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:23.153+0000 7f8a1aa00640 1 -- 172.21.15.17:0/2721582071 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8a1c147b60 con 0x7f8a1c14e7b0 2024-09-10T21:26:23.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:23.153+0000 7f8a1aa00640 1 --2- 172.21.15.17:0/2721582071 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8a1c14e7b0 0x7f8a1c13f070 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c00e4d0 tx=0x7f8a0c00e990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:23.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:23.153+0000 7f8a12a00640 1 -- 172.21.15.17:0/2721582071 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a0c0052e0 con 0x7f8a1c14e7b0 2024-09-10T21:26:23.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:23.153+0000 7f8a12a00640 1 -- 172.21.15.17:0/2721582071 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f8a140be070 con 0x7f8a1c150aa0 2024-09-10T21:26:23.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:23.153+0000 7f8a12a00640 1 --2- 172.21.15.17:0/2721582071 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f89fc0724e0 0x7f89fc0749a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:23.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:23.153+0000 7f8a12a00640 1 -- 172.21.15.17:0/2721582071 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f8a14131ac0 con 0x7f8a1c150aa0 2024-09-10T21:26:23.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:23.153+0000 7f8a12a00640 1 -- 172.21.15.17:0/2721582071 <== mon.2 v2:172.21.15.31:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f8a0c018070 con 0x7f8a1c14e7b0 2024-09-10T21:26:23.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:23.153+0000 7f8a1a000640 1 --2- 172.21.15.17:0/2721582071 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f89fc0724e0 0x7f89fc0749a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:23.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:23.153+0000 7f8a12a00640 1 -- 172.21.15.17:0/2721582071 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8a1c14e7b0 msgr2=0x7f8a1c13f070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:23.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:23.153+0000 7f8a12a00640 1 --2- 172.21.15.17:0/2721582071 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8a1c14e7b0 0x7f8a1c13f070 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c00e4d0 tx=0x7f8a0c00e990 comp rx=0 tx=0).stop 2024-09-10T21:26:23.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:23.153+0000 7f8a12a00640 1 -- 172.21.15.17:0/2721582071 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8a1c14e7b0 msgr2=0x7f8a1c13f070 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:26:23.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:23.153+0000 7f8a12a00640 1 --2- 172.21.15.17:0/2721582071 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8a1c14e7b0 0x7f8a1c13f070 secure :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c00e4d0 tx=0x7f8a0c00e990 comp rx=0 tx=0).stop 2024-09-10T21:26:23.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:23.153+0000 7f8a1a000640 1 --2- 172.21.15.17:0/2721582071 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f89fc0724e0 0x7f89fc0749a0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f8a1c154fc0 tx=0x7f8a08009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:23.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:23.161+0000 7f8a21800640 1 --2- 172.21.15.17:0/2721582071 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8a1c13f5b0 0x7f8a1c1f3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:23.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:23.161+0000 7f8a21800640 1 -- 172.21.15.17:0/2721582071 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f8a1c150e80 con 0x7f8a1c13f5b0 2024-09-10T21:26:23.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:23.161+0000 7f8a1aa00640 1 --2- 172.21.15.17:0/2721582071 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8a1c13f5b0 0x7f8a1c1f3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:23.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:23.161+0000 7f8a1aa00640 1 -- 172.21.15.17:0/2721582071 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f8a1c0c7e40 con 0x7f8a1c13f5b0 2024-09-10T21:26:23.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:23.161+0000 7f8a1aa00640 1 --2- 172.21.15.17:0/2721582071 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8a1c13f5b0 0x7f8a1c1f3650 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c017000 tx=0x7f8a0c000ff0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:23.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:23.161+0000 7f8a12a00640 1 -- 172.21.15.17:0/2721582071 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a0c003810 con 0x7f8a1c13f5b0 2024-09-10T21:26:23.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:23.161+0000 7f8a12a00640 1 -- 172.21.15.17:0/2721582071 <== mon.2 v2:172.21.15.31:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5043 (secure 0 0 0) 0x7f8a0c003810 con 0x7f8a1c13f5b0 2024-09-10T21:26:23.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:23.161+0000 7f8a12a00640 1 -- 172.21.15.17:0/2721582071 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8a1c13f5b0 msgr2=0x7f8a1c1f3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:23.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:23.161+0000 7f8a12a00640 1 --2- 172.21.15.17:0/2721582071 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8a1c13f5b0 0x7f8a1c1f3650 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c017000 tx=0x7f8a0c000ff0 comp rx=0 tx=0).stop 2024-09-10T21:26:23.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:23.161+0000 7f8a12a00640 1 -- 172.21.15.17:0/2721582071 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8a1c13f5b0 msgr2=0x7f8a1c1f3650 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:26:23.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:23.161+0000 7f8a12a00640 1 --2- 172.21.15.17:0/2721582071 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8a1c13f5b0 0x7f8a1c1f3650 secure :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c017000 tx=0x7f8a0c000ff0 comp rx=0 tx=0).stop 2024-09-10T21:26:23.170 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-10T21:26:23.170 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:26:23.171 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:26:23.171 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-10T21:26:23.171 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 10, 2024-09-10T21:26:23.171 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 121608, 2024-09-10T21:26:23.171 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-10T21:26:23.172 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-10T21:26:23.172 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-10T21:26:23.172 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-10T21:26:23.172 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:26:23.172 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 12, 2024-09-10T21:26:23.172 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:26:23.173 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-10T21:26:23.173 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-10T21:26:23.173 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:26:23.173 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:26:23.173 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:26:23.173 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:26:23.174 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:26:23.174 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:26:23.174 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:26:23.174 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:26:23.174 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:26:23.175 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:26:23.175 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:26:23.175 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:26:23.175 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-10T21:26:23.175 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-10T21:26:23.175 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:26:23.176 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:26:23.176 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:26:23.176 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:26:23.176 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:26:23.176 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:26:23.176 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:26:23.177 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:26:23.177 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:26:23.177 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:26:23.177 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:26:23.177 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:26:23.177 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:26:23.177 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-10T21:26:23.178 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-10T21:26:23.178 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-10T21:26:23.178 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-10T21:26:23.178 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-10T21:26:23.178 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "5e2f023f-d6a6-49f4-a256-f4807dce3739", 2024-09-10T21:26:23.178 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:26:23.179 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:26:23.179 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-10T21:26:23.179 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-10T21:26:23.179 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-10T21:26:23.179 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-10T21:26:23.179 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-10T21:26:23.179 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-10T21:26:23.179 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-10T21:26:23.179 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:26:23.180 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-10T21:26:23.180 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:26:23.180 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:26:23.180 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:26:23.180 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:26:23.180 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:26:23.180 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:26:23.180 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:26:23.180 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:26:23.180 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:26:23.180 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:26:23.181 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:26:23.181 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:26:23.181 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-10T21:26:23.181 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:26:23.181 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-10T21:26:23.181 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:26:23.181 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:26:23.181 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:26:23.182 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:26:23.182 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:26:23.182 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:26:23.182 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:26:23.182 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-10T21:26:23.182 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:26:23.182 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:26:23.182 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:26:23.182 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:26:23.182 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-10T21:26:23.182 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:26:23.182 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:26:23.182 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:26:23.182 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:26:23.182 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-10T21:26:23.183 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-10T21:26:23.183 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:26:23.183 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:26:23.183 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:26:23.183 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:26:23.183 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:26:23.183 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:26:23.183 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:26:23.183 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:26:23.183 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:26:23.183 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:26:23.183 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:26:23.183 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3300", 2024-09-10T21:26:23.183 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:26:23.184 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:26:23.184 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:26:23.184 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:26:23.184 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789", 2024-09-10T21:26:23.184 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:26:23.184 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:26:23.184 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:26:23.184 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:26:23.184 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789/0", 2024-09-10T21:26:23.184 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6789/0", 2024-09-10T21:26:23.185 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:26:23.185 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:26:23.185 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:26:23.185 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:26:23.185 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:26:23.185 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:26:23.185 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:26:23.185 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:26:23.185 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:26:23.185 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:26:23.185 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:26:23.185 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3301", 2024-09-10T21:26:23.185 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:26:23.186 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:26:23.186 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:26:23.186 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:26:23.186 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790", 2024-09-10T21:26:23.186 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:26:23.186 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:26:23.186 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:26:23.186 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:26:23.186 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790/0", 2024-09-10T21:26:23.186 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6790/0", 2024-09-10T21:26:23.186 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:26:23.187 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:26:23.187 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:26:23.187 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:26:23.187 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:26:23.187 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:26:23.187 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-10T21:26:23.187 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-10T21:26:23.187 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:26:23.187 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:26:23.187 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:26:23.187 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:26:23.187 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:26:23.187 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:26:23.187 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": [ 2024-09-10T21:26:23.188 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:26:23.188 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:26:23.188 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:26:23.188 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:26:23.188 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:26:23.188 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:26:23.188 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-10T21:26:23.188 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:26:23.188 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:26:23.188 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:26:23.189 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 4 2024-09-10T21:26:23.189 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:26:23.189 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:26:23.189 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-10T21:26:23.189 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:26:23.189 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:26:23.189 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:26:23.189 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 5 2024-09-10T21:26:23.189 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:26:23.190 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:26:23.190 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": [ 2024-09-10T21:26:23.190 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:26:23.190 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:26:23.190 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:26:23.190 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:26:23.190 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:26:23.190 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:26:23.190 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:26:23.190 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-10T21:26:23.190 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-10T21:26:23.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:23.161+0000 7f8a21800640 1 -- 172.21.15.17:0/2721582071 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f89fc0724e0 msgr2=0x7f89fc0749a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:23.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:23.161+0000 7f8a21800640 1 --2- 172.21.15.17:0/2721582071 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f89fc0724e0 0x7f89fc0749a0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f8a1c154fc0 tx=0x7f8a08009040 comp rx=0 tx=0).stop 2024-09-10T21:26:23.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:23.161+0000 7f8a21800640 1 -- 172.21.15.17:0/2721582071 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8a1c150aa0 msgr2=0x7f8a1c14e270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:23.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:23.161+0000 7f8a21800640 1 --2- 172.21.15.17:0/2721582071 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8a1c150aa0 0x7f8a1c14e270 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f8a1401d2e0 tx=0x7f8a1400a310 comp rx=0 tx=0).stop 2024-09-10T21:26:23.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:23.161+0000 7f8a1b400640 1 -- 172.21.15.17:0/2721582071 reap_dead start 2024-09-10T21:26:23.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:23.161+0000 7f8a1b400640 1 -- 172.21.15.17:0/2721582071 reap_dead start 2024-09-10T21:26:23.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:23.161+0000 7f8a21800640 1 -- 172.21.15.17:0/2721582071 shutdown_connections 2024-09-10T21:26:23.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:23.161+0000 7f8a21800640 1 -- 172.21.15.17:0/2721582071 >> 172.21.15.17:0/2721582071 conn(0x7f8a1c06cc20 msgr2=0x7f8a1c0c55e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:23.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:23.161+0000 7f8a21800640 1 -- 172.21.15.17:0/2721582071 shutdown_connections 2024-09-10T21:26:23.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:23.161+0000 7f8a21800640 1 -- 172.21.15.17:0/2721582071 wait complete. 2024-09-10T21:26:23.193 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-10T21:26:23.193 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['c'] 2024-09-10T21:26:23.193 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2024-09-10T21:26:23.193 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.c 2024-09-10T21:26:23.193 INFO:tasks.mon_thrash.mon_thrasher:killing mon.c 2024-09-10T21:26:23.194 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-10T21:26:23.194 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:26:23.217 INFO:tasks.ceph.mon.c:Stopped 2024-09-10T21:26:23.217 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2024-09-10T21:26:23.217 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-10T21:26:23.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:23.361+0000 7f7464200640 1 -- 172.21.15.17:0/2768611072 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f745c152440 msgr2=0x7f745c0b3eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:23.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:23.361+0000 7f7464200640 1 --2- 172.21.15.17:0/2768611072 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f745c152440 0x7f745c0b3eb0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f745c1594a0 tx=0x7f745001c790 comp rx=0 tx=0).stop 2024-09-10T21:26:23.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:23.361+0000 7f7464200640 1 -- 172.21.15.17:0/2768611072 shutdown_connections 2024-09-10T21:26:23.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:23.361+0000 7f7464200640 1 --2- 172.21.15.17:0/2768611072 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f745c152440 0x7f745c0b3eb0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:23.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:23.361+0000 7f7464200640 1 -- 172.21.15.17:0/2768611072 >> 172.21.15.17:0/2768611072 conn(0x7f745c0702f0 msgr2=0x7f745c070700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:23.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:23.361+0000 7f7464200640 1 -- 172.21.15.17:0/2768611072 shutdown_connections 2024-09-10T21:26:23.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:23.361+0000 7f7464200640 1 -- 172.21.15.17:0/2768611072 wait complete. 2024-09-10T21:26:23.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:23.361+0000 7f7464200640 1 Processor -- start 2024-09-10T21:26:23.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:23.361+0000 7f7464200640 1 -- start start 2024-09-10T21:26:23.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:23.361+0000 7f7464200640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f745c0cdf20 0x7f745c0c5be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:23.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:23.361+0000 7f7464200640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f745c152440 0x7f745c06dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:23.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:23.361+0000 7f7464200640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f745c152820 0x7f745c1f70a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:23.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:23.361+0000 7f7464200640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f745c0cb450 con 0x7f745c152820 2024-09-10T21:26:23.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:23.361+0000 7f7464200640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f745c0ce740 con 0x7f745c0cdf20 2024-09-10T21:26:23.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:23.361+0000 7f7464200640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f745c0cb5d0 con 0x7f745c152440 2024-09-10T21:26:23.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:23.361+0000 7f7460e00640 1 -- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f745c152440 msgr2=0x7f745c06dd30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:3301/0 2024-09-10T21:26:23.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:23.361+0000 7f7460e00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f745c152440 0x7f745c06dd30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:26:23.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:23.361+0000 7f7461800640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f745c0cdf20 0x7f745c0c5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:23.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:23.361+0000 7f7462200640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f745c152820 0x7f745c1f70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:23.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:23.361+0000 7f7461800640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f745c0cdf20 0x7f745c0c5be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.17:49534/0 (socket says 172.21.15.17:49534) 2024-09-10T21:26:23.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:23.361+0000 7f7461800640 1 -- 172.21.15.17:0/3773593717 learned_addr learned my addr 172.21.15.17:0/3773593717 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:26:23.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:23.365+0000 7f7461800640 1 -- 172.21.15.17:0/3773593717 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f745c152440 msgr2=0x7f745c06dd30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:26:23.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:23.365+0000 7f7461800640 1 --2- 172.21.15.17:0/3773593717 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f745c152440 0x7f745c06dd30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:23.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:23.365+0000 7f7461800640 1 -- 172.21.15.17:0/3773593717 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f745c152820 msgr2=0x7f745c1f70a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:23.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:23.365+0000 7f7461800640 1 --2- 172.21.15.17:0/3773593717 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f745c152820 0x7f745c1f70a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:23.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:23.365+0000 7f7461800640 1 -- 172.21.15.17:0/3773593717 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f745c15b410 con 0x7f745c0cdf20 2024-09-10T21:26:23.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:23.365+0000 7f7462200640 1 --2- 172.21.15.17:0/3773593717 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f745c152820 0x7f745c1f70a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:26:23.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:23.365+0000 7f7461800640 1 --2- 172.21.15.17:0/3773593717 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f745c0cdf20 0x7f745c0c5be0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f745001cc70 tx=0x7f7450005ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:23.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:23.365+0000 7f7459600640 1 -- 172.21.15.17:0/3773593717 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74500bb070 con 0x7f745c0cdf20 2024-09-10T21:26:23.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:23.365+0000 7f7459600640 1 -- 172.21.15.17:0/3773593717 <== mon.1 v2:172.21.15.31:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7450004330 con 0x7f745c0cdf20 2024-09-10T21:26:23.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:23.365+0000 7f7459600640 1 -- 172.21.15.17:0/3773593717 <== mon.1 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74500b2040 con 0x7f745c0cdf20 2024-09-10T21:26:23.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:23.365+0000 7f7464200640 1 -- 172.21.15.17:0/3773593717 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f745c0c6f80 con 0x7f745c0cdf20 2024-09-10T21:26:23.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:23.365+0000 7f7464200640 1 -- 172.21.15.17:0/3773593717 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f745c06e270 con 0x7f745c0cdf20 2024-09-10T21:26:23.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:23.365+0000 7f7464200640 1 -- 172.21.15.17:0/3773593717 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f745c0c3160 con 0x7f745c0cdf20 2024-09-10T21:26:23.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:23.365+0000 7f7459600640 1 -- 172.21.15.17:0/3773593717 <== mon.1 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f74500b5070 con 0x7f745c0cdf20 2024-09-10T21:26:23.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:23.369+0000 7f7459600640 1 --2- 172.21.15.17:0/3773593717 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f743c072590 0x7f743c074a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:23.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:23.369+0000 7f7459600640 1 -- 172.21.15.17:0/3773593717 <== mon.1 v2:172.21.15.31:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f745012d200 con 0x7f745c0cdf20 2024-09-10T21:26:23.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:23.369+0000 7f7460e00640 1 --2- 172.21.15.17:0/3773593717 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f743c072590 0x7f743c074a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:23.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:23.369+0000 7f7460e00640 1 --2- 172.21.15.17:0/3773593717 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f743c072590 0x7f743c074a50 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f745c14e170 tx=0x7f7448007a20 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:23.376 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:23.373+0000 7f7459600640 1 -- 172.21.15.17:0/3773593717 <== mon.1 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f74500c4070 con 0x7f745c0cdf20 2024-09-10T21:26:23.614 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:23.609+0000 7f7464200640 1 -- 172.21.15.17:0/3773593717 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f745c0c4870 con 0x7f745c0cdf20 2024-09-10T21:26:23.614 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:23.609+0000 7f7459600640 1 -- 172.21.15.17:0/3773593717 <== mon.1 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1557 (secure 0 0 0) 0x7f7450016410 con 0x7f745c0cdf20 2024-09-10T21:26:23.615 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":12,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5e2f023f-d6a6-49f4-a256-f4807dce3739","modified":"2024-09-10T21:23:46.892130Z","created":"2024-09-10T21:23:46.892130Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3301","nonce":0},{"type":"v1","addr":"172.21.15.31:6790","nonce":0}]},"addr":"172.21.15.31:6790/0","public_addr":"172.21.15.31:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-10T21:26:23.617 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:23.613+0000 7f7464200640 1 -- 172.21.15.17:0/3773593717 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f743c072590 msgr2=0x7f743c074a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:23.617 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:23.613+0000 7f7464200640 1 --2- 172.21.15.17:0/3773593717 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f743c072590 0x7f743c074a50 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f745c14e170 tx=0x7f7448007a20 comp rx=0 tx=0).stop 2024-09-10T21:26:23.617 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:23.613+0000 7f7464200640 1 -- 172.21.15.17:0/3773593717 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f745c0cdf20 msgr2=0x7f745c0c5be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:23.617 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:23.613+0000 7f7464200640 1 --2- 172.21.15.17:0/3773593717 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f745c0cdf20 0x7f745c0c5be0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f745001cc70 tx=0x7f7450005ca0 comp rx=0 tx=0).stop 2024-09-10T21:26:23.617 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:23.613+0000 7f7464200640 1 -- 172.21.15.17:0/3773593717 shutdown_connections 2024-09-10T21:26:23.617 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:23.613+0000 7f7464200640 1 --2- 172.21.15.17:0/3773593717 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f743c072590 0x7f743c074a50 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:23.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:23.613+0000 7f7464200640 1 --2- 172.21.15.17:0/3773593717 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f745c152820 0x7f745c1f70a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:23.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:23.613+0000 7f7464200640 1 --2- 172.21.15.17:0/3773593717 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f745c152440 0x7f745c06dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:23.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:23.613+0000 7f7464200640 1 --2- 172.21.15.17:0/3773593717 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f745c0cdf20 0x7f745c0c5be0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:23.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:23.613+0000 7f7464200640 1 -- 172.21.15.17:0/3773593717 >> 172.21.15.17:0/3773593717 conn(0x7f745c0702f0 msgr2=0x7f745c0b3b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:23.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:23.613+0000 7f7464200640 1 -- 172.21.15.17:0/3773593717 shutdown_connections 2024-09-10T21:26:23.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:23.613+0000 7f7464200640 1 -- 172.21.15.17:0/3773593717 wait complete. 2024-09-10T21:26:23.910 INFO:tasks.workunit.client.0.smithi017.stderr:Note: switching to '8f219c476cb4f26fc5719c661c0323719a45cb30'. 2024-09-10T21:26:23.910 INFO:tasks.workunit.client.0.smithi017.stderr: 2024-09-10T21:26:23.910 INFO:tasks.workunit.client.0.smithi017.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-09-10T21:26:23.910 INFO:tasks.workunit.client.0.smithi017.stderr:changes and commit them, and you can discard any commits you make in this 2024-09-10T21:26:23.910 INFO:tasks.workunit.client.0.smithi017.stderr:state without impacting any branches by switching back to a branch. 2024-09-10T21:26:23.911 INFO:tasks.workunit.client.0.smithi017.stderr: 2024-09-10T21:26:23.911 INFO:tasks.workunit.client.0.smithi017.stderr:If you want to create a new branch to retain commits you create, you may 2024-09-10T21:26:23.911 INFO:tasks.workunit.client.0.smithi017.stderr:do so (now or later) by using -c with the switch command. Example: 2024-09-10T21:26:23.911 INFO:tasks.workunit.client.0.smithi017.stderr: 2024-09-10T21:26:23.911 INFO:tasks.workunit.client.0.smithi017.stderr: git switch -c 2024-09-10T21:26:23.911 INFO:tasks.workunit.client.0.smithi017.stderr: 2024-09-10T21:26:23.911 INFO:tasks.workunit.client.0.smithi017.stderr:Or undo this operation with: 2024-09-10T21:26:23.911 INFO:tasks.workunit.client.0.smithi017.stderr: 2024-09-10T21:26:23.911 INFO:tasks.workunit.client.0.smithi017.stderr: git switch - 2024-09-10T21:26:23.911 INFO:tasks.workunit.client.0.smithi017.stderr: 2024-09-10T21:26:23.911 INFO:tasks.workunit.client.0.smithi017.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-09-10T21:26:23.911 INFO:tasks.workunit.client.0.smithi017.stderr: 2024-09-10T21:26:23.912 INFO:tasks.workunit.client.0.smithi017.stderr:HEAD is now at 8f219c476cb Merge branch 'mgr-vol-mod-disable' of https://github.com/rishabh-d-dave/ceph into wip-hemanth-testing-2024-09-10-0723 2024-09-10T21:26:23.922 DEBUG:teuthology.orchestra.run.smithi017:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-09-10T21:26:23.931 INFO:tasks.workunit.client.0.smithi017.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-09-10T21:26:23.934 INFO:tasks.workunit.client.0.smithi017.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-10T21:26:23.934 INFO:tasks.workunit.client.0.smithi017.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-09-10T21:26:24.725 INFO:tasks.workunit.client.0.smithi017.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-09-10T21:26:24.807 INFO:tasks.workunit.client.0.smithi017.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-09-10T21:26:24.887 INFO:tasks.workunit.client.0.smithi017.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-10T21:26:24.889 INFO:tasks.workunit.client.0.smithi017.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-10T21:26:24.890 INFO:tasks.workunit.client.0.smithi017.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-09-10T21:26:24.944 INFO:tasks.workunit.client.0.smithi017.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-10T21:26:24.950 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-09-10T21:26:24.950 DEBUG:teuthology.orchestra.run.smithi017:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-09-10T21:26:25.000 INFO:tasks.workunit:Running workunits matching suites/ffsb.sh on client.0... 2024-09-10T21:26:25.002 INFO:tasks.workunit:Running workunit suites/ffsb.sh... 2024-09-10T21:26:25.002 DEBUG:teuthology.orchestra.run.smithi017:workunit test suites/ffsb.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=8f219c476cb4f26fc5719c661c0323719a45cb30 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/ffsb.sh 2024-09-10T21:26:25.069 INFO:tasks.workunit.client.0.smithi017.stderr:++ dirname /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/ffsb.sh 2024-09-10T21:26:25.070 INFO:tasks.workunit.client.0.smithi017.stderr:+ mydir=/home/ubuntu/cephtest/clone.client.0/qa/workunits/suites 2024-09-10T21:26:25.070 INFO:tasks.workunit.client.0.smithi017.stderr:+ trap -- retry EXIT 2024-09-10T21:26:25.070 INFO:tasks.workunit.client.0.smithi017.stderr:+ rm -rf ffsb 2024-09-10T21:26:25.073 INFO:tasks.workunit.client.0.smithi017.stderr:+ timeout 1800 git clone https://git.ceph.com/ffsb.git --depth 1 2024-09-10T21:26:25.081 INFO:tasks.workunit.client.0.smithi017.stderr:Cloning into 'ffsb'... 2024-09-10T21:26:26.636 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-10T21:26:26.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:26.785+0000 7fcf97200640 1 -- 172.21.15.17:0/815260507 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf98154580 msgr2=0x7fcf98072a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:26.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:26.785+0000 7fcf97200640 1 --2- 172.21.15.17:0/815260507 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf98154580 0x7fcf98072a60 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fcf88009840 tx=0x7fcf8801f030 comp rx=0 tx=0).stop 2024-09-10T21:26:26.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:26.785+0000 7fcf97200640 1 -- 172.21.15.17:0/815260507 shutdown_connections 2024-09-10T21:26:26.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:26.785+0000 7fcf97200640 1 --2- 172.21.15.17:0/815260507 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fcf980c88f0 0x7fcf98070670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:26.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:26.785+0000 7fcf97200640 1 --2- 172.21.15.17:0/815260507 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf98154580 0x7fcf98072a60 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:26.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:26.785+0000 7fcf97200640 1 -- 172.21.15.17:0/815260507 >> 172.21.15.17:0/815260507 conn(0x7fcf980cdef0 msgr2=0x7fcf980ce2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:26.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:26.785+0000 7fcf97200640 1 -- 172.21.15.17:0/815260507 shutdown_connections 2024-09-10T21:26:26.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:26.785+0000 7fcf97200640 1 -- 172.21.15.17:0/815260507 wait complete. 2024-09-10T21:26:26.792 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:26.785+0000 7fcf97200640 1 Processor -- start 2024-09-10T21:26:26.792 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:26.785+0000 7fcf97200640 1 -- start start 2024-09-10T21:26:26.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:26.789+0000 7fcf97200640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcf980c88f0 0x7fcf981563d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:26.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:26.789+0000 7fcf97200640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf980c9550 0x7fcf98156910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:26.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:26.789+0000 7fcf97200640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fcf98154580 0x7fcf9814f200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:26.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:26.789+0000 7fcf97200640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fcf98072500 con 0x7fcf980c88f0 2024-09-10T21:26:26.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:26.789+0000 7fcf97200640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fcf980c99d0 con 0x7fcf980c9550 2024-09-10T21:26:26.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:26.789+0000 7fcf97200640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7fcf98072680 con 0x7fcf98154580 2024-09-10T21:26:26.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:26.789+0000 7fcf96800640 1 -- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fcf98154580 msgr2=0x7fcf9814f200 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:3301/0 2024-09-10T21:26:26.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:26.789+0000 7fcf95e00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcf980c88f0 0x7fcf981563d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:26.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:26.789+0000 7fcf96800640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fcf98154580 0x7fcf9814f200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:26:26.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:26.789+0000 7fcf95e00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcf980c88f0 0x7fcf981563d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45224/0 (socket says 172.21.15.17:45224) 2024-09-10T21:26:26.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:26.789+0000 7fcf95e00640 1 -- 172.21.15.17:0/483602232 learned_addr learned my addr 172.21.15.17:0/483602232 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:26:26.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:26.789+0000 7fcf95400640 1 --2- 172.21.15.17:0/483602232 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf980c9550 0x7fcf98156910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:26.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:26.789+0000 7fcf95e00640 1 -- 172.21.15.17:0/483602232 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fcf98154580 msgr2=0x7fcf9814f200 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:26:26.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:26.789+0000 7fcf95e00640 1 --2- 172.21.15.17:0/483602232 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fcf98154580 0x7fcf9814f200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:26.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:26.789+0000 7fcf95e00640 1 -- 172.21.15.17:0/483602232 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf980c9550 msgr2=0x7fcf98156910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:26.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:26.789+0000 7fcf95e00640 1 --2- 172.21.15.17:0/483602232 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf980c9550 0x7fcf98156910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:26.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:26.789+0000 7fcf95e00640 1 -- 172.21.15.17:0/483602232 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf981f0fb0 con 0x7fcf980c88f0 2024-09-10T21:26:26.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:26.789+0000 7fcf95e00640 1 --2- 172.21.15.17:0/483602232 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcf980c88f0 0x7fcf981563d0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fcf88009790 tx=0x7fcf880a8db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:26.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:26.789+0000 7fcf8e000640 1 -- 172.21.15.17:0/483602232 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf880bf070 con 0x7fcf980c88f0 2024-09-10T21:26:26.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:26.789+0000 7fcf8e000640 1 -- 172.21.15.17:0/483602232 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcf880053e0 con 0x7fcf980c88f0 2024-09-10T21:26:26.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:26.789+0000 7fcf8e000640 1 -- 172.21.15.17:0/483602232 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf880b5040 con 0x7fcf980c88f0 2024-09-10T21:26:26.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:26.789+0000 7fcf97200640 1 -- 172.21.15.17:0/483602232 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf981f1290 con 0x7fcf980c88f0 2024-09-10T21:26:26.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:26.789+0000 7fcf97200640 1 -- 172.21.15.17:0/483602232 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf9814ea00 con 0x7fcf980c88f0 2024-09-10T21:26:26.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:26.789+0000 7fcf97200640 1 -- 172.21.15.17:0/483602232 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf98154960 con 0x7fcf980c88f0 2024-09-10T21:26:26.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:26.793+0000 7fcf8e000640 1 -- 172.21.15.17:0/483602232 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fcf880b8070 con 0x7fcf980c88f0 2024-09-10T21:26:26.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:26.793+0000 7fcf8e000640 1 --2- 172.21.15.17:0/483602232 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fcf78072590 0x7fcf78074a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:26.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:26.793+0000 7fcf8e000640 1 -- 172.21.15.17:0/483602232 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fcf881304c0 con 0x7fcf980c88f0 2024-09-10T21:26:26.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:26.793+0000 7fcf95400640 1 --2- 172.21.15.17:0/483602232 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fcf78072590 0x7fcf78074a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:26.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:26.793+0000 7fcf95400640 1 --2- 172.21.15.17:0/483602232 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fcf78072590 0x7fcf78074a50 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fcf9814e5e0 tx=0x7fcf84007ea0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:26.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:26.797+0000 7fcf8e000640 1 -- 172.21.15.17:0/483602232 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fcf88018b90 con 0x7fcf980c88f0 2024-09-10T21:26:27.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:27.101+0000 7fcf97200640 1 -- 172.21.15.17:0/483602232 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fcf9806ea60 con 0x7fcf980c88f0 2024-09-10T21:26:27.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:27.101+0000 7fcf8e000640 1 -- 172.21.15.17:0/483602232 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1557 (secure 0 0 0) 0x7fcf880fb0c0 con 0x7fcf980c88f0 2024-09-10T21:26:27.108 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":16,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5e2f023f-d6a6-49f4-a256-f4807dce3739","modified":"2024-09-10T21:23:46.892130Z","created":"2024-09-10T21:23:46.892130Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3301","nonce":0},{"type":"v1","addr":"172.21.15.31:6790","nonce":0}]},"addr":"172.21.15.31:6790/0","public_addr":"172.21.15.31:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-10T21:26:27.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:27.105+0000 7fcf97200640 1 -- 172.21.15.17:0/483602232 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fcf78072590 msgr2=0x7fcf78074a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:27.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:27.105+0000 7fcf97200640 1 --2- 172.21.15.17:0/483602232 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fcf78072590 0x7fcf78074a50 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fcf9814e5e0 tx=0x7fcf84007ea0 comp rx=0 tx=0).stop 2024-09-10T21:26:27.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:27.105+0000 7fcf97200640 1 -- 172.21.15.17:0/483602232 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcf980c88f0 msgr2=0x7fcf981563d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:27.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:27.105+0000 7fcf97200640 1 --2- 172.21.15.17:0/483602232 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcf980c88f0 0x7fcf981563d0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fcf88009790 tx=0x7fcf880a8db0 comp rx=0 tx=0).stop 2024-09-10T21:26:27.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:27.105+0000 7fcf97200640 1 -- 172.21.15.17:0/483602232 shutdown_connections 2024-09-10T21:26:27.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:27.105+0000 7fcf97200640 1 --2- 172.21.15.17:0/483602232 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fcf78072590 0x7fcf78074a50 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:27.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:27.105+0000 7fcf97200640 1 --2- 172.21.15.17:0/483602232 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fcf98154580 0x7fcf9814f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:27.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:27.105+0000 7fcf97200640 1 --2- 172.21.15.17:0/483602232 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcf980c9550 0x7fcf98156910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:27.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:27.105+0000 7fcf97200640 1 --2- 172.21.15.17:0/483602232 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcf980c88f0 0x7fcf981563d0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:27.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:27.105+0000 7fcf97200640 1 -- 172.21.15.17:0/483602232 >> 172.21.15.17:0/483602232 conn(0x7fcf980cdef0 msgr2=0x7fcf980ce2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:27.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:27.105+0000 7fcf97200640 1 -- 172.21.15.17:0/483602232 shutdown_connections 2024-09-10T21:26:27.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:27.105+0000 7fcf97200640 1 -- 172.21.15.17:0/483602232 wait complete. 2024-09-10T21:26:27.551 INFO:tasks.workunit.client.0.smithi017.stderr:+ trap - EXIT 2024-09-10T21:26:27.551 INFO:tasks.workunit.client.0.smithi017.stderr:+ cd ffsb 2024-09-10T21:26:27.552 INFO:tasks.workunit.client.0.smithi017.stderr:+ ./configure 2024-09-10T21:26:27.860 INFO:tasks.workunit.client.0.smithi017.stdout:checking for a BSD-compatible install... /usr/bin/install -c 2024-09-10T21:26:28.878 INFO:tasks.workunit.client.0.smithi017.stdout:checking whether build environment is sane... yes 2024-09-10T21:26:28.911 INFO:tasks.workunit.client.0.smithi017.stdout:checking for a thread-safe mkdir -p... /usr/bin/mkdir -p 2024-09-10T21:26:28.912 INFO:tasks.workunit.client.0.smithi017.stdout:checking for gawk... gawk 2024-09-10T21:26:28.947 INFO:tasks.workunit.client.0.smithi017.stdout:checking whether make sets $(MAKE)... yes 2024-09-10T21:26:29.048 INFO:tasks.workunit.client.0.smithi017.stdout:checking build system type... x86_64-unknown-linux-gnu 2024-09-10T21:26:29.048 INFO:tasks.workunit.client.0.smithi017.stdout:checking host system type... x86_64-unknown-linux-gnu 2024-09-10T21:26:29.077 INFO:tasks.workunit.client.0.smithi017.stdout:checking for style of include used by make... GNU 2024-09-10T21:26:29.083 INFO:tasks.workunit.client.0.smithi017.stdout:checking for gcc... gcc 2024-09-10T21:26:29.221 INFO:tasks.workunit.client.0.smithi017.stdout:checking for C compiler default output file name... a.out 2024-09-10T21:26:29.228 INFO:tasks.workunit.client.0.smithi017.stdout:checking whether the C compiler works... yes 2024-09-10T21:26:29.233 INFO:tasks.workunit.client.0.smithi017.stdout:checking whether we are cross compiling... no 2024-09-10T21:26:29.314 INFO:tasks.workunit.client.0.smithi017.stdout:checking for suffix of executables... 2024-09-10T21:26:29.380 INFO:tasks.workunit.client.0.smithi017.stdout:checking for suffix of object files... o 2024-09-10T21:26:29.445 INFO:tasks.workunit.client.0.smithi017.stdout:checking whether we are using the GNU C compiler... yes 2024-09-10T21:26:29.512 INFO:tasks.workunit.client.0.smithi017.stdout:checking whether gcc accepts -g... yes 2024-09-10T21:26:29.639 INFO:tasks.workunit.client.0.smithi017.stdout:checking for gcc option to accept ISO C89... none needed 2024-09-10T21:26:30.134 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-10T21:26:30.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:30.277+0000 7f4ffa600640 1 -- 172.21.15.17:0/3717263203 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4ff4152440 msgr2=0x7f4ff40b3eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:30.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:30.277+0000 7f4ffa600640 1 --2- 172.21.15.17:0/3717263203 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4ff4152440 0x7f4ff40b3eb0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f4ff40cbe60 tx=0x7f4fe401c850 comp rx=0 tx=0).stop 2024-09-10T21:26:30.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:30.277+0000 7f4ffa600640 1 -- 172.21.15.17:0/3717263203 shutdown_connections 2024-09-10T21:26:30.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:30.277+0000 7f4ffa600640 1 --2- 172.21.15.17:0/3717263203 >> v2:172.21.15.31:3300/0 conn(0x7f4ff40cdf20 0x7f4ff40ce3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:30.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:30.277+0000 7f4ffa600640 1 --2- 172.21.15.17:0/3717263203 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f4ff4152820 0x7f4ff415c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:30.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:30.277+0000 7f4ffa600640 1 --2- 172.21.15.17:0/3717263203 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4ff4152440 0x7f4ff40b3eb0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:30.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:30.277+0000 7f4ffa600640 1 -- 172.21.15.17:0/3717263203 >> 172.21.15.17:0/3717263203 conn(0x7f4ff40702f0 msgr2=0x7f4ff4070700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:30.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:30.277+0000 7f4ffa600640 1 -- 172.21.15.17:0/3717263203 shutdown_connections 2024-09-10T21:26:30.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:30.277+0000 7f4ffa600640 1 -- 172.21.15.17:0/3717263203 wait complete. 2024-09-10T21:26:30.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:30.277+0000 7f4ffa600640 1 Processor -- start 2024-09-10T21:26:30.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:30.277+0000 7f4ffa600640 1 -- start start 2024-09-10T21:26:30.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:30.281+0000 7f4ffa600640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4ff40cdf20 0x7f4ff41501f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:30.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:30.281+0000 7f4ffa600640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4ff4152440 0x7f4ff406ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:30.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:30.281+0000 7f4ffa600640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f4ff4152820 0x7f4ff4145020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:30.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:30.281+0000 7f4ffa600640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f4ff40cb450 con 0x7f4ff40cdf20 2024-09-10T21:26:30.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:30.281+0000 7f4ffa600640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f4ff415c7c0 con 0x7f4ff4152440 2024-09-10T21:26:30.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:30.281+0000 7f4ffa600640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f4ff40cb5d0 con 0x7f4ff4152820 2024-09-10T21:26:30.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:30.281+0000 7f4ff3400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4ff40cdf20 0x7f4ff41501f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:30.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:30.281+0000 7f4ff3400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4ff40cdf20 0x7f4ff41501f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45246/0 (socket says 172.21.15.17:45246) 2024-09-10T21:26:30.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:30.281+0000 7f4ff3400640 1 -- 172.21.15.17:0/2499960697 learned_addr learned my addr 172.21.15.17:0/2499960697 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:26:30.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:30.281+0000 7f4ff3e00640 1 -- 172.21.15.17:0/2499960697 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f4ff4152820 msgr2=0x7f4ff4145020 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:3301/0 2024-09-10T21:26:30.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:30.281+0000 7f4ff3e00640 1 --2- 172.21.15.17:0/2499960697 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f4ff4152820 0x7f4ff4145020 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:26:30.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:30.281+0000 7f4ff2a00640 1 --2- 172.21.15.17:0/2499960697 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4ff4152440 0x7f4ff406ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:30.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:30.281+0000 7f4ff3400640 1 -- 172.21.15.17:0/2499960697 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f4ff4152820 msgr2=0x7f4ff4145020 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:26:30.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:30.281+0000 7f4ff3400640 1 --2- 172.21.15.17:0/2499960697 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f4ff4152820 0x7f4ff4145020 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:30.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:30.281+0000 7f4ff3400640 1 -- 172.21.15.17:0/2499960697 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4ff4152440 msgr2=0x7f4ff406ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:30.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:30.281+0000 7f4ff3400640 1 --2- 172.21.15.17:0/2499960697 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4ff4152440 0x7f4ff406ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:30.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:30.281+0000 7f4ff3400640 1 -- 172.21.15.17:0/2499960697 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ff4144400 con 0x7f4ff40cdf20 2024-09-10T21:26:30.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:30.281+0000 7f4ff2a00640 1 --2- 172.21.15.17:0/2499960697 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4ff4152440 0x7f4ff406ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:26:30.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:30.281+0000 7f4ff3400640 1 --2- 172.21.15.17:0/2499960697 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4ff40cdf20 0x7f4ff41501f0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f4fe401af00 tx=0x7f4fe40a56b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:30.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:30.281+0000 7f4feb400640 1 -- 172.21.15.17:0/2499960697 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fe40ba020 con 0x7f4ff40cdf20 2024-09-10T21:26:30.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:30.281+0000 7f4feb400640 1 -- 172.21.15.17:0/2499960697 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4fe4005070 con 0x7f4ff40cdf20 2024-09-10T21:26:30.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:30.281+0000 7f4feb400640 1 -- 172.21.15.17:0/2499960697 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fe40b2040 con 0x7f4ff40cdf20 2024-09-10T21:26:30.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:30.281+0000 7f4ffa600640 1 -- 172.21.15.17:0/2499960697 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ff406f280 con 0x7f4ff40cdf20 2024-09-10T21:26:30.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:30.281+0000 7f4ffa600640 1 -- 172.21.15.17:0/2499960697 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ff406e260 con 0x7f4ff40cdf20 2024-09-10T21:26:30.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:30.281+0000 7f4ffa600640 1 -- 172.21.15.17:0/2499960697 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ff40c30f0 con 0x7f4ff40cdf20 2024-09-10T21:26:30.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:30.281+0000 7f4feb400640 1 -- 172.21.15.17:0/2499960697 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f4fe40054a0 con 0x7f4ff40cdf20 2024-09-10T21:26:30.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:30.285+0000 7f4feb400640 1 --2- 172.21.15.17:0/2499960697 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f4fd4072590 0x7f4fd4074a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:30.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:30.285+0000 7f4feb400640 1 -- 172.21.15.17:0/2499960697 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f4fe412d610 con 0x7f4ff40cdf20 2024-09-10T21:26:30.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:30.285+0000 7f4ff2a00640 1 --2- 172.21.15.17:0/2499960697 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f4fd4072590 0x7f4fd4074a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:30.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:30.285+0000 7f4ff2a00640 1 --2- 172.21.15.17:0/2499960697 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f4fd4072590 0x7f4fd4074a50 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f4fe0006fd0 tx=0x7f4fe0008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:30.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:30.289+0000 7f4feb400640 1 -- 172.21.15.17:0/2499960697 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f4fe40c4060 con 0x7f4ff40cdf20 2024-09-10T21:26:30.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:30.525+0000 7f4ffa600640 1 -- 172.21.15.17:0/2499960697 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f4ff406c7e0 con 0x7f4ff40cdf20 2024-09-10T21:26:30.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:30.529+0000 7f4feb400640 1 -- 172.21.15.17:0/2499960697 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1557 (secure 0 0 0) 0x7f4fe40164d0 con 0x7f4ff40cdf20 2024-09-10T21:26:30.532 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":10,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":19,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5e2f023f-d6a6-49f4-a256-f4807dce3739","modified":"2024-09-10T21:23:46.892130Z","created":"2024-09-10T21:23:46.892130Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3301","nonce":0},{"type":"v1","addr":"172.21.15.31:6790","nonce":0}]},"addr":"172.21.15.31:6790/0","public_addr":"172.21.15.31:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-10T21:26:30.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:30.529+0000 7f4ffa600640 1 -- 172.21.15.17:0/2499960697 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f4fd4072590 msgr2=0x7f4fd4074a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:30.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:30.529+0000 7f4ffa600640 1 --2- 172.21.15.17:0/2499960697 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f4fd4072590 0x7f4fd4074a50 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f4fe0006fd0 tx=0x7f4fe0008040 comp rx=0 tx=0).stop 2024-09-10T21:26:30.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:30.529+0000 7f4ffa600640 1 -- 172.21.15.17:0/2499960697 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4ff40cdf20 msgr2=0x7f4ff41501f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:30.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:30.529+0000 7f4ffa600640 1 --2- 172.21.15.17:0/2499960697 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4ff40cdf20 0x7f4ff41501f0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f4fe401af00 tx=0x7f4fe40a56b0 comp rx=0 tx=0).stop 2024-09-10T21:26:30.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:30.529+0000 7f4ffa600640 1 -- 172.21.15.17:0/2499960697 shutdown_connections 2024-09-10T21:26:30.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:30.529+0000 7f4ffa600640 1 --2- 172.21.15.17:0/2499960697 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f4fd4072590 0x7f4fd4074a50 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:30.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:30.529+0000 7f4ffa600640 1 --2- 172.21.15.17:0/2499960697 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f4ff4152820 0x7f4ff4145020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:30.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:30.529+0000 7f4ffa600640 1 --2- 172.21.15.17:0/2499960697 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4ff4152440 0x7f4ff406ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:30.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:30.529+0000 7f4ffa600640 1 --2- 172.21.15.17:0/2499960697 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4ff40cdf20 0x7f4ff41501f0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:30.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:30.529+0000 7f4ffa600640 1 -- 172.21.15.17:0/2499960697 >> 172.21.15.17:0/2499960697 conn(0x7f4ff40702f0 msgr2=0x7f4ff4145b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:30.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:30.529+0000 7f4ffa600640 1 -- 172.21.15.17:0/2499960697 shutdown_connections 2024-09-10T21:26:30.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:30.529+0000 7f4ffa600640 1 -- 172.21.15.17:0/2499960697 wait complete. 2024-09-10T21:26:33.554 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-10T21:26:38.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.121+0000 7f6bc5600640 1 -- 172.21.15.17:0/2167082430 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6b9c006200 msgr2=0x7f6b9c0086a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:38.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.121+0000 7f6bc5600640 1 --2- 172.21.15.17:0/2167082430 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6b9c006200 0x7f6b9c0086a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f6ba8006b30 tx=0x7f6ba801d3f0 comp rx=0 tx=0).stop 2024-09-10T21:26:38.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.121+0000 7f6bbf400640 1 -- 172.21.15.17:0/2167082430 reap_dead start 2024-09-10T21:26:38.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.121+0000 7f6bc5600640 1 -- 172.21.15.17:0/2167082430 shutdown_connections 2024-09-10T21:26:38.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.121+0000 7f6bc5600640 1 --2- 172.21.15.17:0/2167082430 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6b9c006200 0x7f6b9c0086a0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:38.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.121+0000 7f6bc5600640 1 -- 172.21.15.17:0/2167082430 >> 172.21.15.17:0/2167082430 conn(0x7f6bc0070160 msgr2=0x7f6bc0070570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:38.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.121+0000 7f6bc5600640 1 -- 172.21.15.17:0/2167082430 shutdown_connections 2024-09-10T21:26:38.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.121+0000 7f6bc5600640 1 -- 172.21.15.17:0/2167082430 wait complete. 2024-09-10T21:26:38.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.125+0000 7f6bc5600640 1 Processor -- start 2024-09-10T21:26:38.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.125+0000 7f6bc5600640 1 -- start start 2024-09-10T21:26:38.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.125+0000 7f6bc5600640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6b9c006200 0x7f6bc00c21c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:38.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.125+0000 7f6bc5600640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6bc00c5cb0 0x7f6bc0073ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:38.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.125+0000 7f6bc5600640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6bc013a970 0x7f6bc013adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:38.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.125+0000 7f6bc5600640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f6bc006e4b0 con 0x7f6b9c006200 2024-09-10T21:26:38.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.125+0000 7f6bc5600640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f6bc006e1d0 con 0x7f6bc013a970 2024-09-10T21:26:38.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.125+0000 7f6bc5600640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f6bc00c7ed0 con 0x7f6bc00c5cb0 2024-09-10T21:26:38.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.125+0000 7f6bbea00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6b9c006200 0x7f6bc00c21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:38.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.125+0000 7f6bb7e00640 1 -- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6bc00c5cb0 msgr2=0x7f6bc0073ec0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:3301/0 2024-09-10T21:26:38.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.125+0000 7f6bbea00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6b9c006200 0x7f6bc00c21c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38096/0 (socket says 172.21.15.17:38096) 2024-09-10T21:26:38.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.125+0000 7f6bbea00640 1 -- 172.21.15.17:0/2549690852 learned_addr learned my addr 172.21.15.17:0/2549690852 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:26:38.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.125+0000 7f6bb7e00640 1 --2- 172.21.15.17:0/2549690852 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6bc00c5cb0 0x7f6bc0073ec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:26:38.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.125+0000 7f6bbf400640 1 --2- 172.21.15.17:0/2549690852 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6bc013a970 0x7f6bc013adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:38.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.125+0000 7f6bbea00640 1 -- 172.21.15.17:0/2549690852 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6bc00c5cb0 msgr2=0x7f6bc0073ec0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:26:38.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.125+0000 7f6bbea00640 1 --2- 172.21.15.17:0/2549690852 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6bc00c5cb0 0x7f6bc0073ec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:38.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.125+0000 7f6bbea00640 1 -- 172.21.15.17:0/2549690852 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6bc013a970 msgr2=0x7f6bc013adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:38.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.125+0000 7f6bbea00640 1 --2- 172.21.15.17:0/2549690852 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6bc013a970 0x7f6bc013adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:38.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.125+0000 7f6bbea00640 1 -- 172.21.15.17:0/2549690852 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bc014fe80 con 0x7f6b9c006200 2024-09-10T21:26:38.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.125+0000 7f6bbf400640 1 --2- 172.21.15.17:0/2549690852 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6bc013a970 0x7f6bc013adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:26:38.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.125+0000 7f6bbea00640 1 --2- 172.21.15.17:0/2549690852 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6b9c006200 0x7f6bc00c21c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f6bb0013880 tx=0x7f6bb0013d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:38.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.125+0000 7f6bb6a00640 1 -- 172.21.15.17:0/2549690852 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bb0020070 con 0x7f6b9c006200 2024-09-10T21:26:38.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.125+0000 7f6bb6a00640 1 -- 172.21.15.17:0/2549690852 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6bb001b3f0 con 0x7f6b9c006200 2024-09-10T21:26:38.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.125+0000 7f6bb6a00640 1 -- 172.21.15.17:0/2549690852 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bb001b840 con 0x7f6b9c006200 2024-09-10T21:26:38.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.125+0000 7f6bc5600640 1 -- 172.21.15.17:0/2549690852 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6bc01502b0 con 0x7f6b9c006200 2024-09-10T21:26:38.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.125+0000 7f6bc5600640 1 -- 172.21.15.17:0/2549690852 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6bc014fba0 con 0x7f6b9c006200 2024-09-10T21:26:38.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.129+0000 7f6bc5600640 1 -- 172.21.15.17:0/2549690852 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6bc014f990 con 0x7f6b9c006200 2024-09-10T21:26:38.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.129+0000 7f6bb6a00640 1 -- 172.21.15.17:0/2549690852 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f6bb001a030 con 0x7f6b9c006200 2024-09-10T21:26:38.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.129+0000 7f6bb6a00640 1 --2- 172.21.15.17:0/2549690852 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f6b9c075de0 0x7f6b9c0782a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:38.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.129+0000 7f6bb6a00640 1 -- 172.21.15.17:0/2549690852 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f6bb0094980 con 0x7f6b9c006200 2024-09-10T21:26:38.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.129+0000 7f6bb7e00640 1 --2- 172.21.15.17:0/2549690852 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f6b9c075de0 0x7f6b9c0782a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:38.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.133+0000 7f6bb7e00640 1 --2- 172.21.15.17:0/2549690852 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f6b9c075de0 0x7f6b9c0782a0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f6b9c0061d0 tx=0x7f6ba801bc30 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:38.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.133+0000 7f6bb6a00640 1 -- 172.21.15.17:0/2549690852 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f6bb0018070 con 0x7f6b9c006200 2024-09-10T21:26:38.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.373+0000 7f6bc5600640 1 -- 172.21.15.17:0/2549690852 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f6bc0157500 con 0x7f6b9c006200 2024-09-10T21:26:38.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.373+0000 7f6bb6a00640 1 -- 172.21.15.17:0/2549690852 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1550 (secure 0 0 0) 0x7f6bb005fc50 con 0x7f6b9c006200 2024-09-10T21:26:38.380 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":14,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5e2f023f-d6a6-49f4-a256-f4807dce3739","modified":"2024-09-10T21:23:46.892130Z","created":"2024-09-10T21:23:46.892130Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3301","nonce":0},{"type":"v1","addr":"172.21.15.31:6790","nonce":0}]},"addr":"172.21.15.31:6790/0","public_addr":"172.21.15.31:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-10T21:26:38.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.377+0000 7f6bc5600640 1 -- 172.21.15.17:0/2549690852 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f6b9c075de0 msgr2=0x7f6b9c0782a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:38.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.377+0000 7f6bc5600640 1 --2- 172.21.15.17:0/2549690852 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f6b9c075de0 0x7f6b9c0782a0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f6b9c0061d0 tx=0x7f6ba801bc30 comp rx=0 tx=0).stop 2024-09-10T21:26:38.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.377+0000 7f6bc5600640 1 -- 172.21.15.17:0/2549690852 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6b9c006200 msgr2=0x7f6bc00c21c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:38.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.377+0000 7f6bc5600640 1 --2- 172.21.15.17:0/2549690852 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6b9c006200 0x7f6bc00c21c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f6bb0013880 tx=0x7f6bb0013d40 comp rx=0 tx=0).stop 2024-09-10T21:26:38.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.377+0000 7f6bc5600640 1 -- 172.21.15.17:0/2549690852 shutdown_connections 2024-09-10T21:26:38.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.377+0000 7f6bc5600640 1 --2- 172.21.15.17:0/2549690852 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f6b9c075de0 0x7f6b9c0782a0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:38.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.377+0000 7f6bc5600640 1 --2- 172.21.15.17:0/2549690852 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6bc013a970 0x7f6bc013adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:38.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.377+0000 7f6bc5600640 1 --2- 172.21.15.17:0/2549690852 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6bc00c5cb0 0x7f6bc0073ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:38.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.377+0000 7f6bc5600640 1 --2- 172.21.15.17:0/2549690852 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6b9c006200 0x7f6bc00c21c0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:38.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.377+0000 7f6bc5600640 1 -- 172.21.15.17:0/2549690852 >> 172.21.15.17:0/2549690852 conn(0x7f6bc0070160 msgr2=0x7f6bc01546e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:38.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.377+0000 7f6bc5600640 1 -- 172.21.15.17:0/2549690852 shutdown_connections 2024-09-10T21:26:38.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.377+0000 7f6bc5600640 1 -- 172.21.15.17:0/2549690852 wait complete. 2024-09-10T21:26:38.401 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2024-09-10T21:26:38.402 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon a 2024-09-10T21:26:38.402 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-10T21:26:38.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.545+0000 7f0e06000640 1 -- 172.21.15.17:0/3477113953 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0e000b1b50 msgr2=0x7f0e00074d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:38.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.545+0000 7f0e06000640 1 --2- 172.21.15.17:0/3477113953 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0e000b1b50 0x7f0e00074d40 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f0df8009bc0 tx=0x7f0df801cbc0 comp rx=0 tx=0).stop 2024-09-10T21:26:38.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.545+0000 7f0e06000640 1 -- 172.21.15.17:0/3477113953 shutdown_connections 2024-09-10T21:26:38.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.545+0000 7f0e06000640 1 --2- 172.21.15.17:0/3477113953 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0e000b1b50 0x7f0e00074d40 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:38.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.545+0000 7f0e06000640 1 --2- 172.21.15.17:0/3477113953 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f0e000c8ca0 0x7f0e00157260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:38.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.545+0000 7f0e06000640 1 -- 172.21.15.17:0/3477113953 >> 172.21.15.17:0/3477113953 conn(0x7f0e000700d0 msgr2=0x7f0e000704e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:38.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.545+0000 7f0e06000640 1 -- 172.21.15.17:0/3477113953 shutdown_connections 2024-09-10T21:26:38.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.545+0000 7f0e06000640 1 -- 172.21.15.17:0/3477113953 wait complete. 2024-09-10T21:26:38.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.545+0000 7f0e06000640 1 Processor -- start 2024-09-10T21:26:38.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.545+0000 7f0e06000640 1 -- start start 2024-09-10T21:26:38.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.545+0000 7f0e06000640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0e000b1b50 0x7f0e00146bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:38.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.545+0000 7f0e06000640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0e000c8ca0 0x7f0e000cc630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:38.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.545+0000 7f0e06000640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f0e000c9080 0x7f0e00152390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:38.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.545+0000 7f0e06000640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f0e00163e00 con 0x7f0e000b1b50 2024-09-10T21:26:38.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.545+0000 7f0e06000640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f0e000c87c0 con 0x7f0e000c8ca0 2024-09-10T21:26:38.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.545+0000 7f0e06000640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f0e00163f80 con 0x7f0e000c9080 2024-09-10T21:26:38.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.545+0000 7f0dffe00640 1 -- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f0e000c9080 msgr2=0x7f0e00152390 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:3301/0 2024-09-10T21:26:38.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.545+0000 7f0dffe00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f0e000c9080 0x7f0e00152390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:26:38.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.545+0000 7f0dff400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0e000b1b50 0x7f0e00146bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:38.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.549+0000 7f0dff400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0e000b1b50 0x7f0e00146bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38098/0 (socket says 172.21.15.17:38098) 2024-09-10T21:26:38.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.549+0000 7f0dff400640 1 -- 172.21.15.17:0/2645224601 learned_addr learned my addr 172.21.15.17:0/2645224601 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:26:38.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.549+0000 7f0dfea00640 1 --2- 172.21.15.17:0/2645224601 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0e000c8ca0 0x7f0e000cc630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:38.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.549+0000 7f0dff400640 1 -- 172.21.15.17:0/2645224601 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f0e000c9080 msgr2=0x7f0e00152390 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:26:38.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.549+0000 7f0dff400640 1 --2- 172.21.15.17:0/2645224601 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f0e000c9080 0x7f0e00152390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:38.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.549+0000 7f0dff400640 1 -- 172.21.15.17:0/2645224601 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0e000c8ca0 msgr2=0x7f0e000cc630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:38.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.549+0000 7f0dff400640 1 --2- 172.21.15.17:0/2645224601 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0e000c8ca0 0x7f0e000cc630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:38.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.549+0000 7f0dff400640 1 -- 172.21.15.17:0/2645224601 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e000ca710 con 0x7f0e000b1b50 2024-09-10T21:26:38.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.549+0000 7f0dff400640 1 --2- 172.21.15.17:0/2645224601 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0e000b1b50 0x7f0e00146bb0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f0df000b580 tx=0x7f0df000ba40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:38.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.549+0000 7f0df7400640 1 -- 172.21.15.17:0/2645224601 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0df000f850 con 0x7f0e000b1b50 2024-09-10T21:26:38.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.549+0000 7f0df7400640 1 -- 172.21.15.17:0/2645224601 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0df000fb00 con 0x7f0e000b1b50 2024-09-10T21:26:38.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.549+0000 7f0e06000640 1 -- 172.21.15.17:0/2645224601 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e000cbe50 con 0x7f0e000b1b50 2024-09-10T21:26:38.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.549+0000 7f0df7400640 1 -- 172.21.15.17:0/2645224601 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0df0017400 con 0x7f0e000b1b50 2024-09-10T21:26:38.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.549+0000 7f0e06000640 1 -- 172.21.15.17:0/2645224601 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e001eaa30 con 0x7f0e000b1b50 2024-09-10T21:26:38.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.549+0000 7f0e06000640 1 --2- 172.21.15.17:0/2645224601 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0e000ccb70 0x7f0e001528d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:38.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.549+0000 7f0e06000640 1 -- 172.21.15.17:0/2645224601 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f0e00074d20 con 0x7f0e000ccb70 2024-09-10T21:26:38.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.549+0000 7f0dfea00640 1 --2- 172.21.15.17:0/2645224601 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0e000ccb70 0x7f0e001528d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:38.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.549+0000 7f0dfea00640 1 -- 172.21.15.17:0/2645224601 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0e00148dd0 con 0x7f0e000ccb70 2024-09-10T21:26:38.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.549+0000 7f0dfea00640 1 --2- 172.21.15.17:0/2645224601 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0e000ccb70 0x7f0e001528d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f0e001f7680 tx=0x7f0dec00fc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:38.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.549+0000 7f0df7400640 1 -- 172.21.15.17:0/2645224601 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dec018020 con 0x7f0e000ccb70 2024-09-10T21:26:38.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.549+0000 7f0df7400640 1 -- 172.21.15.17:0/2645224601 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f0dec00d600 con 0x7f0e000ccb70 2024-09-10T21:26:38.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.549+0000 7f0df7400640 1 -- 172.21.15.17:0/2645224601 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0e000ccb70 msgr2=0x7f0e001528d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:38.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.549+0000 7f0df7400640 1 --2- 172.21.15.17:0/2645224601 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0e000ccb70 0x7f0e001528d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f0e001f7680 tx=0x7f0dec00fc00 comp rx=0 tx=0).stop 2024-09-10T21:26:38.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.549+0000 7f0df7400640 1 -- 172.21.15.17:0/2645224601 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0e000ccb70 msgr2=0x7f0e001528d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:26:38.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.549+0000 7f0df7400640 1 --2- 172.21.15.17:0/2645224601 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0e000ccb70 0x7f0e001528d0 secure :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f0e001f7680 tx=0x7f0dec00fc00 comp rx=0 tx=0).stop 2024-09-10T21:26:38.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.549+0000 7f0df7400640 1 -- 172.21.15.17:0/2645224601 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f0df00203f0 con 0x7f0e000b1b50 2024-09-10T21:26:38.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.553+0000 7f0df7400640 1 --2- 172.21.15.17:0/2645224601 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f0de0072050 0x7f0de0074510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:38.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.553+0000 7f0df7400640 1 -- 172.21.15.17:0/2645224601 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f0df0093650 con 0x7f0e000b1b50 2024-09-10T21:26:38.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.553+0000 7f0dfea00640 1 --2- 172.21.15.17:0/2645224601 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f0de0072050 0x7f0de0074510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:38.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.553+0000 7f0dfea00640 1 --2- 172.21.15.17:0/2645224601 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f0de0072050 0x7f0de0074510 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f0dec000c10 tx=0x7f0dec009030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:38.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.557+0000 7f0e06000640 1 --2- 172.21.15.17:0/2645224601 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0e001e8f80 0x7f0e001e9360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:38.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.557+0000 7f0e06000640 1 -- 172.21.15.17:0/2645224601 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f0e000cc200 con 0x7f0e001e8f80 2024-09-10T21:26:38.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.557+0000 7f0dffe00640 1 --2- 172.21.15.17:0/2645224601 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0e001e8f80 0x7f0e001e9360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:38.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.557+0000 7f0dffe00640 1 -- 172.21.15.17:0/2645224601 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f0e0015b4f0 con 0x7f0e001e8f80 2024-09-10T21:26:38.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.557+0000 7f0dffe00640 1 --2- 172.21.15.17:0/2645224601 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0e001e8f80 0x7f0e001e9360 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f0e0006dce0 tx=0x7f0df80098b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:38.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.557+0000 7f0df7400640 1 -- 172.21.15.17:0/2645224601 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0df80bb070 con 0x7f0e001e8f80 2024-09-10T21:26:38.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.557+0000 7f0df7400640 1 -- 172.21.15.17:0/2645224601 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4865 (secure 0 0 0) 0x7f0df80b3040 con 0x7f0e001e8f80 2024-09-10T21:26:38.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.557+0000 7f0df7400640 1 -- 172.21.15.17:0/2645224601 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0e001e8f80 msgr2=0x7f0e001e9360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:38.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.557+0000 7f0df7400640 1 --2- 172.21.15.17:0/2645224601 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0e001e8f80 0x7f0e001e9360 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f0e0006dce0 tx=0x7f0df80098b0 comp rx=0 tx=0).stop 2024-09-10T21:26:38.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.557+0000 7f0df7400640 1 -- 172.21.15.17:0/2645224601 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0e001e8f80 msgr2=0x7f0e001e9360 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:26:38.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.557+0000 7f0df7400640 1 --2- 172.21.15.17:0/2645224601 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0e001e8f80 0x7f0e001e9360 secure :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f0e0006dce0 tx=0x7f0df80098b0 comp rx=0 tx=0).stop 2024-09-10T21:26:38.564 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-10T21:26:38.564 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:26:38.564 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:26:38.564 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "leader", 2024-09-10T21:26:38.564 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 14, 2024-09-10T21:26:38.564 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 28463, 2024-09-10T21:26:38.564 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-10T21:26:38.565 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-10T21:26:38.565 INFO:teuthology.orchestra.run.smithi017.stdout: 1 2024-09-10T21:26:38.565 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:26:38.565 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 0, 2024-09-10T21:26:38.565 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:26:38.565 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-10T21:26:38.565 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-10T21:26:38.565 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:26:38.565 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:26:38.565 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:26:38.565 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:26:38.565 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:26:38.565 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:26:38.566 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:26:38.566 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:26:38.566 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:26:38.566 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:26:38.566 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:26:38.566 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:26:38.566 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-10T21:26:38.566 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-10T21:26:38.566 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:26:38.566 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:26:38.566 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:26:38.566 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:26:38.566 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:26:38.566 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:26:38.567 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:26:38.567 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:26:38.567 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:26:38.567 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:26:38.567 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:26:38.567 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:26:38.567 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:26:38.567 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-10T21:26:38.567 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-10T21:26:38.568 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-10T21:26:38.568 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-10T21:26:38.568 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-10T21:26:38.568 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "5e2f023f-d6a6-49f4-a256-f4807dce3739", 2024-09-10T21:26:38.568 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:26:38.568 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:26:38.568 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-10T21:26:38.568 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-10T21:26:38.568 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-10T21:26:38.568 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-10T21:26:38.568 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-10T21:26:38.568 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-10T21:26:38.569 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-10T21:26:38.569 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:26:38.569 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-10T21:26:38.569 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:26:38.569 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:26:38.569 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:26:38.569 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:26:38.569 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:26:38.569 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:26:38.569 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:26:38.569 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:26:38.569 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:26:38.570 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:26:38.570 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:26:38.570 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:26:38.570 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-10T21:26:38.570 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:26:38.570 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-10T21:26:38.570 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:26:38.570 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:26:38.570 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:26:38.570 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:26:38.570 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:26:38.570 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:26:38.570 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:26:38.571 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-10T21:26:38.571 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:26:38.571 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:26:38.571 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:26:38.571 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:26:38.571 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-10T21:26:38.571 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:26:38.571 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:26:38.571 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:26:38.571 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:26:38.571 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-10T21:26:38.571 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-10T21:26:38.571 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:26:38.571 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:26:38.572 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:26:38.572 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:26:38.572 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:26:38.572 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:26:38.572 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:26:38.572 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:26:38.572 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:26:38.572 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:26:38.572 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:26:38.572 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3300", 2024-09-10T21:26:38.572 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:26:38.572 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:26:38.572 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:26:38.573 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:26:38.573 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789", 2024-09-10T21:26:38.573 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:26:38.573 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:26:38.573 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:26:38.573 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:26:38.573 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789/0", 2024-09-10T21:26:38.573 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6789/0", 2024-09-10T21:26:38.573 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:26:38.573 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:26:38.573 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:26:38.573 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:26:38.573 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:26:38.574 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:26:38.574 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:26:38.574 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:26:38.574 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:26:38.574 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:26:38.574 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:26:38.574 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3301", 2024-09-10T21:26:38.574 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:26:38.574 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:26:38.574 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:26:38.574 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:26:38.574 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790", 2024-09-10T21:26:38.574 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:26:38.575 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:26:38.575 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:26:38.575 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:26:38.575 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790/0", 2024-09-10T21:26:38.575 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6790/0", 2024-09-10T21:26:38.575 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:26:38.575 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:26:38.575 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:26:38.575 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:26:38.575 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:26:38.575 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:26:38.575 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-10T21:26:38.575 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-10T21:26:38.575 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:26:38.576 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:26:38.576 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:26:38.576 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:26:38.576 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:26:38.576 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:26:38.576 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": [ 2024-09-10T21:26:38.576 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:26:38.576 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:26:38.576 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:26:38.576 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:26:38.576 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:26:38.576 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:26:38.576 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-10T21:26:38.576 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:26:38.576 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:26:38.577 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:26:38.577 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 4 2024-09-10T21:26:38.577 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:26:38.577 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:26:38.577 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-10T21:26:38.577 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:26:38.577 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:26:38.577 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:26:38.577 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 5 2024-09-10T21:26:38.577 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:26:38.577 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:26:38.577 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:26:38.577 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-10T21:26:38.578 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-10T21:26:38.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.557+0000 7f0e06000640 1 -- 172.21.15.17:0/2645224601 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f0de0072050 msgr2=0x7f0de0074510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:38.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.557+0000 7f0e06000640 1 --2- 172.21.15.17:0/2645224601 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f0de0072050 0x7f0de0074510 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f0dec000c10 tx=0x7f0dec009030 comp rx=0 tx=0).stop 2024-09-10T21:26:38.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.557+0000 7f0e06000640 1 -- 172.21.15.17:0/2645224601 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0e000b1b50 msgr2=0x7f0e00146bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:38.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.557+0000 7f0e06000640 1 --2- 172.21.15.17:0/2645224601 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0e000b1b50 0x7f0e00146bb0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f0df000b580 tx=0x7f0df000ba40 comp rx=0 tx=0).stop 2024-09-10T21:26:38.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.557+0000 7f0dffe00640 1 -- 172.21.15.17:0/2645224601 reap_dead start 2024-09-10T21:26:38.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.557+0000 7f0dffe00640 1 -- 172.21.15.17:0/2645224601 reap_dead start 2024-09-10T21:26:38.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.557+0000 7f0e06000640 1 -- 172.21.15.17:0/2645224601 shutdown_connections 2024-09-10T21:26:38.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.557+0000 7f0e06000640 1 -- 172.21.15.17:0/2645224601 >> 172.21.15.17:0/2645224601 conn(0x7f0e000700d0 msgr2=0x7f0e001478d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:38.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.557+0000 7f0e06000640 1 -- 172.21.15.17:0/2645224601 shutdown_connections 2024-09-10T21:26:38.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.557+0000 7f0e06000640 1 -- 172.21.15.17:0/2645224601 wait complete. 2024-09-10T21:26:38.582 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-09-10T21:26:38.582 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon b 2024-09-10T21:26:38.582 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-10T21:26:38.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.725+0000 7ff585c00640 1 -- 172.21.15.17:0/3467203749 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff580152450 msgr2=0x7ff580158850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:38.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.725+0000 7ff585c00640 1 --2- 172.21.15.17:0/3467203749 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff580152450 0x7ff580158850 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff568009900 tx=0x7ff56801f0d0 comp rx=0 tx=0).stop 2024-09-10T21:26:38.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.725+0000 7ff585c00640 1 -- 172.21.15.17:0/3467203749 shutdown_connections 2024-09-10T21:26:38.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.725+0000 7ff585c00640 1 --2- 172.21.15.17:0/3467203749 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff580152450 0x7ff580158850 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:38.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.725+0000 7ff585c00640 1 -- 172.21.15.17:0/3467203749 >> 172.21.15.17:0/3467203749 conn(0x7ff58006cc20 msgr2=0x7ff58006d030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:38.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.725+0000 7ff585c00640 1 -- 172.21.15.17:0/3467203749 shutdown_connections 2024-09-10T21:26:38.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.725+0000 7ff585c00640 1 -- 172.21.15.17:0/3467203749 wait complete. 2024-09-10T21:26:38.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.725+0000 7ff585c00640 1 Processor -- start 2024-09-10T21:26:38.732 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.729+0000 7ff585c00640 1 -- start start 2024-09-10T21:26:38.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.729+0000 7ff585c00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff5800cbd10 0x7ff58006e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:38.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.729+0000 7ff585c00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff580152070 0x7ff58014e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:38.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.729+0000 7ff585c00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff580152450 0x7ff5800b2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:38.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.729+0000 7ff585c00640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7ff5800cecf0 con 0x7ff5800cbd10 2024-09-10T21:26:38.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.729+0000 7ff585c00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7ff5800c5090 con 0x7ff580152450 2024-09-10T21:26:38.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.729+0000 7ff585c00640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7ff5800cee70 con 0x7ff580152070 2024-09-10T21:26:38.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.729+0000 7ff577e00640 1 -- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff580152070 msgr2=0x7ff58014e7a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:3301/0 2024-09-10T21:26:38.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.729+0000 7ff577e00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff580152070 0x7ff58014e7a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:26:38.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.729+0000 7ff57fe00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff5800cbd10 0x7ff58006e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:38.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.729+0000 7ff57fe00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff5800cbd10 0x7ff58006e410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38120/0 (socket says 172.21.15.17:38120) 2024-09-10T21:26:38.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.729+0000 7ff57fe00640 1 -- 172.21.15.17:0/3365780266 learned_addr learned my addr 172.21.15.17:0/3365780266 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:26:38.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.729+0000 7ff585200640 1 --2- 172.21.15.17:0/3365780266 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff580152450 0x7ff5800b2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:38.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.729+0000 7ff57fe00640 1 -- 172.21.15.17:0/3365780266 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff580152070 msgr2=0x7ff58014e7a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:26:38.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.729+0000 7ff57fe00640 1 --2- 172.21.15.17:0/3365780266 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff580152070 0x7ff58014e7a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:38.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.729+0000 7ff57fe00640 1 -- 172.21.15.17:0/3365780266 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff580152450 msgr2=0x7ff5800b2b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:38.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.729+0000 7ff57fe00640 1 --2- 172.21.15.17:0/3365780266 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff580152450 0x7ff5800b2b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:38.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.729+0000 7ff57fe00640 1 -- 172.21.15.17:0/3365780266 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5801511e0 con 0x7ff5800cbd10 2024-09-10T21:26:38.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.729+0000 7ff57fe00640 1 --2- 172.21.15.17:0/3365780266 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff5800cbd10 0x7ff58006e410 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff57000c400 tx=0x7ff57000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:38.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.729+0000 7ff57cc00640 1 -- 172.21.15.17:0/3365780266 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff570017070 con 0x7ff5800cbd10 2024-09-10T21:26:38.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.729+0000 7ff57cc00640 1 -- 172.21.15.17:0/3365780266 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff5700053c0 con 0x7ff5800cbd10 2024-09-10T21:26:38.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.729+0000 7ff57cc00640 1 -- 172.21.15.17:0/3365780266 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff570003880 con 0x7ff5800cbd10 2024-09-10T21:26:38.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.729+0000 7ff585c00640 1 -- 172.21.15.17:0/3365780266 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5801e31a0 con 0x7ff5800cbd10 2024-09-10T21:26:38.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.729+0000 7ff585c00640 1 -- 172.21.15.17:0/3365780266 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff580151450 con 0x7ff5800cbd10 2024-09-10T21:26:38.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.729+0000 7ff585c00640 1 --2- 172.21.15.17:0/3365780266 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff5801f0760 0x7ff5800b3040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:38.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.729+0000 7ff585c00640 1 -- 172.21.15.17:0/3365780266 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7ff5801f1730 con 0x7ff5801f0760 2024-09-10T21:26:38.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.729+0000 7ff577e00640 1 --2- 172.21.15.17:0/3365780266 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff5801f0760 0x7ff5800b3040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:38.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.733+0000 7ff577e00640 1 -- 172.21.15.17:0/3365780266 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff580071e10 con 0x7ff5801f0760 2024-09-10T21:26:38.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.733+0000 7ff577e00640 1 --2- 172.21.15.17:0/3365780266 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff5801f0760 0x7ff5800b3040 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ff5800cec30 tx=0x7ff568005770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:38.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.733+0000 7ff57cc00640 1 -- 172.21.15.17:0/3365780266 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7ff570003a20 con 0x7ff5800cbd10 2024-09-10T21:26:38.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.733+0000 7ff57cc00640 1 --2- 172.21.15.17:0/3365780266 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7ff55c0725c0 0x7ff55c074a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:38.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.733+0000 7ff57cc00640 1 -- 172.21.15.17:0/3365780266 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7ff570092c00 con 0x7ff5800cbd10 2024-09-10T21:26:38.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.733+0000 7ff57cc00640 1 -- 172.21.15.17:0/3365780266 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5680bd070 con 0x7ff5801f0760 2024-09-10T21:26:38.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.733+0000 7ff57cc00640 1 -- 172.21.15.17:0/3365780266 <== mon.1 v2:172.21.15.31:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7ff5680b0440 con 0x7ff5801f0760 2024-09-10T21:26:38.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.733+0000 7ff57cc00640 1 -- 172.21.15.17:0/3365780266 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff5801f0760 msgr2=0x7ff5800b3040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:38.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.733+0000 7ff57cc00640 1 --2- 172.21.15.17:0/3365780266 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff5801f0760 0x7ff5800b3040 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ff5800cec30 tx=0x7ff568005770 comp rx=0 tx=0).stop 2024-09-10T21:26:38.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.733+0000 7ff57cc00640 1 -- 172.21.15.17:0/3365780266 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff5801f0760 msgr2=0x7ff5800b3040 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:26:38.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.733+0000 7ff57cc00640 1 --2- 172.21.15.17:0/3365780266 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff5801f0760 0x7ff5800b3040 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:38.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.733+0000 7ff585200640 1 --2- 172.21.15.17:0/3365780266 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7ff55c0725c0 0x7ff55c074a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:38.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.733+0000 7ff585200640 1 --2- 172.21.15.17:0/3365780266 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7ff55c0725c0 0x7ff55c074a80 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7ff580151080 tx=0x7ff578003d50 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:38.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.737+0000 7ff585c00640 1 --2- 172.21.15.17:0/3365780266 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff58014f8a0 0x7ff5801f3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:38.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.737+0000 7ff585c00640 1 -- 172.21.15.17:0/3365780266 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7ff580069ef0 con 0x7ff58014f8a0 2024-09-10T21:26:38.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.741+0000 7ff577e00640 1 --2- 172.21.15.17:0/3365780266 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff58014f8a0 0x7ff5801f3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:38.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.741+0000 7ff577e00640 1 -- 172.21.15.17:0/3365780266 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7ff5800cc710 con 0x7ff58014f8a0 2024-09-10T21:26:38.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.741+0000 7ff577e00640 1 --2- 172.21.15.17:0/3365780266 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff58014f8a0 0x7ff5801f3cd0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ff568009900 tx=0x7ff56801f0d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:38.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.741+0000 7ff57cc00640 1 -- 172.21.15.17:0/3365780266 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5680bd040 con 0x7ff58014f8a0 2024-09-10T21:26:38.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.741+0000 7ff57cc00640 1 -- 172.21.15.17:0/3365780266 <== mon.1 v2:172.21.15.31:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5031 (secure 0 0 0) 0x7ff5680c29b0 con 0x7ff58014f8a0 2024-09-10T21:26:38.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.741+0000 7ff57cc00640 1 -- 172.21.15.17:0/3365780266 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff58014f8a0 msgr2=0x7ff5801f3cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:38.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.741+0000 7ff57cc00640 1 --2- 172.21.15.17:0/3365780266 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff58014f8a0 0x7ff5801f3cd0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ff568009900 tx=0x7ff56801f0d0 comp rx=0 tx=0).stop 2024-09-10T21:26:38.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.741+0000 7ff57cc00640 1 -- 172.21.15.17:0/3365780266 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff58014f8a0 msgr2=0x7ff5801f3cd0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:26:38.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.741+0000 7ff57cc00640 1 --2- 172.21.15.17:0/3365780266 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff58014f8a0 0x7ff5801f3cd0 secure :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ff568009900 tx=0x7ff56801f0d0 comp rx=0 tx=0).stop 2024-09-10T21:26:38.747 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-10T21:26:38.747 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:26:38.747 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:26:38.747 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-10T21:26:38.747 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 14, 2024-09-10T21:26:38.747 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 137188, 2024-09-10T21:26:38.747 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-10T21:26:38.747 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-10T21:26:38.747 INFO:teuthology.orchestra.run.smithi017.stdout: 1 2024-09-10T21:26:38.748 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:26:38.748 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 0, 2024-09-10T21:26:38.748 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:26:38.748 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-10T21:26:38.748 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-10T21:26:38.748 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:26:38.748 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:26:38.748 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:26:38.748 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:26:38.748 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:26:38.748 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:26:38.748 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:26:38.749 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:26:38.749 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:26:38.749 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:26:38.749 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:26:38.749 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:26:38.749 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-10T21:26:38.749 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-10T21:26:38.749 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:26:38.749 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:26:38.749 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:26:38.749 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:26:38.749 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:26:38.749 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:26:38.749 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:26:38.750 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:26:38.750 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:26:38.750 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:26:38.750 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:26:38.750 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:26:38.750 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:26:38.750 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-10T21:26:38.750 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-10T21:26:38.750 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-10T21:26:38.750 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-10T21:26:38.750 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-10T21:26:38.750 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "5e2f023f-d6a6-49f4-a256-f4807dce3739", 2024-09-10T21:26:38.750 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:26:38.751 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:26:38.751 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-10T21:26:38.751 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-10T21:26:38.751 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-10T21:26:38.751 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-10T21:26:38.751 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-10T21:26:38.751 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-10T21:26:38.751 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-10T21:26:38.751 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:26:38.751 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-10T21:26:38.751 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:26:38.751 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:26:38.751 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:26:38.751 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:26:38.752 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:26:38.752 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:26:38.752 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:26:38.752 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:26:38.752 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:26:38.752 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:26:38.752 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:26:38.752 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:26:38.752 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-10T21:26:38.752 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:26:38.752 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-10T21:26:38.752 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:26:38.752 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:26:38.752 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:26:38.753 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:26:38.753 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:26:38.753 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:26:38.753 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:26:38.753 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-10T21:26:38.753 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:26:38.753 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:26:38.753 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:26:38.753 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:26:38.753 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-10T21:26:38.753 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:26:38.753 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:26:38.753 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:26:38.754 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:26:38.754 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-10T21:26:38.754 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-10T21:26:38.754 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:26:38.754 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:26:38.754 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:26:38.754 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:26:38.754 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:26:38.754 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:26:38.754 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:26:38.754 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:26:38.754 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:26:38.754 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:26:38.754 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:26:38.755 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3300", 2024-09-10T21:26:38.755 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:26:38.755 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:26:38.755 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:26:38.755 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:26:38.755 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789", 2024-09-10T21:26:38.755 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:26:38.755 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:26:38.755 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:26:38.755 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:26:38.755 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789/0", 2024-09-10T21:26:38.755 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6789/0", 2024-09-10T21:26:38.755 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:26:38.755 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:26:38.755 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:26:38.756 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:26:38.756 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:26:38.756 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:26:38.756 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:26:38.756 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:26:38.756 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:26:38.756 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:26:38.756 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:26:38.756 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3301", 2024-09-10T21:26:38.756 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:26:38.756 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:26:38.756 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:26:38.756 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:26:38.757 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790", 2024-09-10T21:26:38.757 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:26:38.757 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:26:38.757 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:26:38.757 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:26:38.757 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790/0", 2024-09-10T21:26:38.757 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6790/0", 2024-09-10T21:26:38.757 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:26:38.757 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:26:38.757 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:26:38.757 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:26:38.757 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:26:38.757 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:26:38.757 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-10T21:26:38.758 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-10T21:26:38.758 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:26:38.758 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:26:38.758 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:26:38.758 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:26:38.758 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:26:38.758 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:26:38.758 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": [ 2024-09-10T21:26:38.758 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:26:38.758 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:26:38.758 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:26:38.758 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 5 2024-09-10T21:26:38.758 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:26:38.758 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:26:38.758 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-10T21:26:38.759 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:26:38.759 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:26:38.759 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:26:38.759 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 4 2024-09-10T21:26:38.759 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:26:38.759 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:26:38.759 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-10T21:26:38.759 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:26:38.759 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:26:38.759 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:26:38.759 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 3 2024-09-10T21:26:38.759 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:26:38.759 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:26:38.759 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": [ 2024-09-10T21:26:38.760 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:26:38.760 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:26:38.760 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:26:38.760 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 2 2024-09-10T21:26:38.760 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:26:38.760 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:26:38.760 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:26:38.760 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-10T21:26:38.760 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-10T21:26:38.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.741+0000 7ff585c00640 1 -- 172.21.15.17:0/3365780266 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7ff55c0725c0 msgr2=0x7ff55c074a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:38.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.741+0000 7ff585c00640 1 --2- 172.21.15.17:0/3365780266 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7ff55c0725c0 0x7ff55c074a80 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7ff580151080 tx=0x7ff578003d50 comp rx=0 tx=0).stop 2024-09-10T21:26:38.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.741+0000 7ff585c00640 1 -- 172.21.15.17:0/3365780266 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff5800cbd10 msgr2=0x7ff58006e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:38.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.741+0000 7ff585c00640 1 --2- 172.21.15.17:0/3365780266 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff5800cbd10 0x7ff58006e410 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff57000c400 tx=0x7ff57000c8c0 comp rx=0 tx=0).stop 2024-09-10T21:26:38.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.741+0000 7ff585200640 1 -- 172.21.15.17:0/3365780266 reap_dead start 2024-09-10T21:26:38.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.741+0000 7ff585200640 1 -- 172.21.15.17:0/3365780266 reap_dead start 2024-09-10T21:26:38.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.741+0000 7ff585c00640 1 -- 172.21.15.17:0/3365780266 shutdown_connections 2024-09-10T21:26:38.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.741+0000 7ff585c00640 1 -- 172.21.15.17:0/3365780266 >> 172.21.15.17:0/3365780266 conn(0x7ff58006cc20 msgr2=0x7ff5800cd320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:38.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.741+0000 7ff585c00640 1 -- 172.21.15.17:0/3365780266 shutdown_connections 2024-09-10T21:26:38.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:38.741+0000 7ff585c00640 1 -- 172.21.15.17:0/3365780266 wait complete. 2024-09-10T21:26:38.765 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-09-10T21:26:38.765 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2024-09-10T21:26:41.966 INFO:tasks.workunit.client.0.smithi017.stdout:checking dependency style of gcc... gcc3 2024-09-10T21:26:42.112 INFO:tasks.workunit.client.0.smithi017.stdout:checking how to run the C preprocessor... gcc -E 2024-09-10T21:26:42.214 INFO:tasks.workunit.client.0.smithi017.stdout:checking for grep that handles long lines and -e... /usr/bin/grep 2024-09-10T21:26:42.218 INFO:tasks.workunit.client.0.smithi017.stdout:checking for egrep... /usr/bin/grep -E 2024-09-10T21:26:42.244 INFO:tasks.workunit.client.0.smithi017.stdout:checking for AIX... no 2024-09-10T21:26:42.268 INFO:tasks.workunit.client.0.smithi017.stdout:checking for gcc... (cached) gcc 2024-09-10T21:26:42.294 INFO:tasks.workunit.client.0.smithi017.stdout:checking whether we are using the GNU C compiler... (cached) yes 2024-09-10T21:26:42.296 INFO:tasks.workunit.client.0.smithi017.stdout:checking whether gcc accepts -g... (cached) yes 2024-09-10T21:26:42.297 INFO:tasks.workunit.client.0.smithi017.stdout:checking for gcc option to accept ISO C89... (cached) none needed 2024-09-10T21:26:42.298 INFO:tasks.workunit.client.0.smithi017.stdout:checking dependency style of gcc... (cached) gcc3 2024-09-10T21:26:42.430 INFO:tasks.workunit.client.0.smithi017.stdout:checking for main in -lm... yes 2024-09-10T21:26:42.558 INFO:tasks.workunit.client.0.smithi017.stdout:checking for main in -lpthread... yes 2024-09-10T21:26:42.890 INFO:tasks.workunit.client.0.smithi017.stdout:checking for ANSI C header files... yes 2024-09-10T21:26:43.000 INFO:tasks.workunit.client.0.smithi017.stdout:checking for sys/wait.h that is POSIX.1 compatible... yes 2024-09-10T21:26:43.094 INFO:tasks.workunit.client.0.smithi017.stdout:checking for sys/types.h... yes 2024-09-10T21:26:43.198 INFO:tasks.workunit.client.0.smithi017.stdout:checking for sys/stat.h... yes 2024-09-10T21:26:43.304 INFO:tasks.workunit.client.0.smithi017.stdout:checking for stdlib.h... yes 2024-09-10T21:26:43.410 INFO:tasks.workunit.client.0.smithi017.stdout:checking for string.h... yes 2024-09-10T21:26:43.524 INFO:tasks.workunit.client.0.smithi017.stdout:checking for memory.h... yes 2024-09-10T21:26:43.628 INFO:tasks.workunit.client.0.smithi017.stdout:checking for strings.h... yes 2024-09-10T21:26:43.731 INFO:tasks.workunit.client.0.smithi017.stdout:checking for inttypes.h... yes 2024-09-10T21:26:43.837 INFO:tasks.workunit.client.0.smithi017.stdout:checking for stdint.h... yes 2024-09-10T21:26:43.950 INFO:tasks.workunit.client.0.smithi017.stdout:checking for unistd.h... yes 2024-09-10T21:26:44.075 INFO:tasks.workunit.client.0.smithi017.stdout:checking pthread.h usability... yes 2024-09-10T21:26:44.124 INFO:tasks.workunit.client.0.smithi017.stdout:checking pthread.h presence... yes 2024-09-10T21:26:44.126 INFO:tasks.workunit.client.0.smithi017.stdout:checking for pthread.h... yes 2024-09-10T21:26:44.256 INFO:tasks.workunit.client.0.smithi017.stdout:checking fcntl.h usability... yes 2024-09-10T21:26:44.302 INFO:tasks.workunit.client.0.smithi017.stdout:checking fcntl.h presence... yes 2024-09-10T21:26:44.303 INFO:tasks.workunit.client.0.smithi017.stdout:checking for fcntl.h... yes 2024-09-10T21:26:44.418 INFO:tasks.workunit.client.0.smithi017.stdout:checking limits.h usability... yes 2024-09-10T21:26:44.466 INFO:tasks.workunit.client.0.smithi017.stdout:checking limits.h presence... yes 2024-09-10T21:26:44.467 INFO:tasks.workunit.client.0.smithi017.stdout:checking for limits.h... yes 2024-09-10T21:26:44.502 INFO:tasks.workunit.client.0.smithi017.stdout:checking for stdint.h... (cached) yes 2024-09-10T21:26:44.611 INFO:tasks.workunit.client.0.smithi017.stdout:checking sys/time.h usability... yes 2024-09-10T21:26:44.658 INFO:tasks.workunit.client.0.smithi017.stdout:checking sys/time.h presence... yes 2024-09-10T21:26:44.659 INFO:tasks.workunit.client.0.smithi017.stdout:checking for sys/time.h... yes 2024-09-10T21:26:44.692 INFO:tasks.workunit.client.0.smithi017.stdout:checking for unistd.h... (cached) yes 2024-09-10T21:26:44.798 INFO:tasks.workunit.client.0.smithi017.stdout:checking sys/vfs.h usability... yes 2024-09-10T21:26:44.847 INFO:tasks.workunit.client.0.smithi017.stdout:checking sys/vfs.h presence... yes 2024-09-10T21:26:44.848 INFO:tasks.workunit.client.0.smithi017.stdout:checking for sys/vfs.h... yes 2024-09-10T21:26:44.966 INFO:tasks.workunit.client.0.smithi017.stdout:checking sys/limits.h usability... no 2024-09-10T21:26:45.038 INFO:tasks.workunit.client.0.smithi017.stdout:checking sys/limits.h presence... no 2024-09-10T21:26:45.040 INFO:tasks.workunit.client.0.smithi017.stdout:checking for sys/limits.h... no 2024-09-10T21:26:45.111 INFO:tasks.workunit.client.0.smithi017.stdout:checking for an ANSI C-conforming const... yes 2024-09-10T21:26:45.184 INFO:tasks.workunit.client.0.smithi017.stdout:checking whether time.h and sys/time.h may both be included... yes 2024-09-10T21:26:45.189 INFO:tasks.workunit.client.0.smithi017.stdout:checking for function prototypes... yes 2024-09-10T21:26:45.268 INFO:tasks.workunit.client.0.smithi017.stdout:checking whether setvbuf arguments are reversed... no 2024-09-10T21:26:45.420 INFO:tasks.workunit.client.0.smithi017.stdout:checking for system... yes 2024-09-10T21:26:45.570 INFO:tasks.workunit.client.0.smithi017.stdout:checking for gettimeofday... yes 2024-09-10T21:26:45.662 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:26:45.665 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:26:45.717 INFO:tasks.workunit.client.0.smithi017.stdout:checking for mkdir... yes 2024-09-10T21:26:45.864 INFO:tasks.workunit.client.0.smithi017.stdout:checking for strerror... yes 2024-09-10T21:26:46.028 INFO:tasks.workunit.client.0.smithi017.stdout:checking for open64... yes 2024-09-10T21:26:46.171 INFO:tasks.workunit.client.0.smithi017.stdout:checking for stat64... yes 2024-09-10T21:26:46.314 INFO:tasks.workunit.client.0.smithi017.stdout:checking for fseeko64... yes 2024-09-10T21:26:46.460 INFO:tasks.workunit.client.0.smithi017.stdout:checking for lrand48_r... yes 2024-09-10T21:26:46.603 INFO:tasks.workunit.client.0.smithi017.stdout:checking for srand48_r... yes 2024-09-10T21:26:46.661 INFO:tasks.workunit.client.0.smithi017.stdout:configure: creating ./config.status 2024-09-10T21:26:46.926 INFO:tasks.workunit.client.0.smithi017.stdout:config.status: creating Makefile 2024-09-10T21:26:46.963 INFO:tasks.workunit.client.0.smithi017.stdout:config.status: creating config.h 2024-09-10T21:26:47.025 INFO:tasks.workunit.client.0.smithi017.stdout:config.status: executing depfiles commands 2024-09-10T21:26:47.171 INFO:tasks.workunit.client.0.smithi017.stdout:config.status: executing default commands 2024-09-10T21:26:47.261 INFO:tasks.workunit.client.0.smithi017.stderr:+ make 2024-09-10T21:26:47.307 INFO:tasks.workunit.client.0.smithi017.stdout:cd . && /bin/bash /home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing --run aclocal-1.10 2024-09-10T21:26:47.320 INFO:tasks.workunit.client.0.smithi017.stderr:/home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing: line 46: aclocal-1.10: command not found 2024-09-10T21:26:47.324 INFO:tasks.workunit.client.0.smithi017.stderr:WARNING: `aclocal-1.10' is missing on your system. You should only need it if 2024-09-10T21:26:47.324 INFO:tasks.workunit.client.0.smithi017.stderr: you modified `acinclude.m4' or `configure.in'. You might want 2024-09-10T21:26:47.324 INFO:tasks.workunit.client.0.smithi017.stderr: to install the `Automake' and `Perl' packages. Grab them from 2024-09-10T21:26:47.324 INFO:tasks.workunit.client.0.smithi017.stderr: any GNU archive site. 2024-09-10T21:26:47.333 INFO:tasks.workunit.client.0.smithi017.stdout: cd . && /bin/bash /home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing --run automake-1.10 --gnu 2024-09-10T21:26:47.343 INFO:tasks.workunit.client.0.smithi017.stderr:/home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing: line 46: automake-1.10: command not found 2024-09-10T21:26:47.345 INFO:tasks.workunit.client.0.smithi017.stderr:WARNING: `automake-1.10' is missing on your system. You should only need it if 2024-09-10T21:26:47.345 INFO:tasks.workunit.client.0.smithi017.stderr: you modified `Makefile.am', `acinclude.m4' or `configure.in'. 2024-09-10T21:26:47.345 INFO:tasks.workunit.client.0.smithi017.stderr: You might want to install the `Automake' and `Perl' packages. 2024-09-10T21:26:47.345 INFO:tasks.workunit.client.0.smithi017.stderr: Grab them from any GNU archive site. 2024-09-10T21:26:47.615 INFO:tasks.workunit.client.0.smithi017.stdout:cd . && /bin/bash /home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing --run autoconf 2024-09-10T21:26:47.811 INFO:tasks.workunit.client.0.smithi017.stderr:aclocal.m4:16: warning: this file was generated for autoconf 2.61. 2024-09-10T21:26:47.811 INFO:tasks.workunit.client.0.smithi017.stderr:You have another version of autoconf. It may work, but is not guaranteed to. 2024-09-10T21:26:47.812 INFO:tasks.workunit.client.0.smithi017.stderr:If you have problems, you may need to regenerate the build system entirely. 2024-09-10T21:26:47.812 INFO:tasks.workunit.client.0.smithi017.stderr:To do so, use the procedure documented by the package, typically `autoreconf'. 2024-09-10T21:26:47.920 INFO:tasks.workunit.client.0.smithi017.stderr:configure.in:4: warning: The macro `AC_FOREACH' is obsolete. 2024-09-10T21:26:47.920 INFO:tasks.workunit.client.0.smithi017.stderr:configure.in:4: You should run autoupdate. 2024-09-10T21:26:47.920 INFO:tasks.workunit.client.0.smithi017.stderr:./lib/autoconf/general.m4:191: AC_FOREACH is expanded from... 2024-09-10T21:26:47.920 INFO:tasks.workunit.client.0.smithi017.stderr:aclocal.m4:684: _AM_SET_OPTIONS is expanded from... 2024-09-10T21:26:47.920 INFO:tasks.workunit.client.0.smithi017.stderr:aclocal.m4:410: AM_INIT_AUTOMAKE is expanded from... 2024-09-10T21:26:47.921 INFO:tasks.workunit.client.0.smithi017.stderr:configure.in:4: the top level 2024-09-10T21:26:47.921 INFO:tasks.workunit.client.0.smithi017.stderr:configure.in:6: warning: The macro `AM_CONFIG_HEADER' is obsolete. 2024-09-10T21:26:47.921 INFO:tasks.workunit.client.0.smithi017.stderr:configure.in:6: You should run autoupdate. 2024-09-10T21:26:47.921 INFO:tasks.workunit.client.0.smithi017.stderr:aclocal.m4:383: AM_CONFIG_HEADER is expanded from... 2024-09-10T21:26:47.921 INFO:tasks.workunit.client.0.smithi017.stderr:configure.in:6: the top level 2024-09-10T21:26:47.921 INFO:tasks.workunit.client.0.smithi017.stderr:configure.in:10: warning: The macro `AC_AIX' is obsolete. 2024-09-10T21:26:47.921 INFO:tasks.workunit.client.0.smithi017.stderr:configure.in:10: You should run autoupdate. 2024-09-10T21:26:47.921 INFO:tasks.workunit.client.0.smithi017.stderr:./lib/autoconf/specific.m4:540: AC_AIX is expanded from... 2024-09-10T21:26:47.921 INFO:tasks.workunit.client.0.smithi017.stderr:configure.in:10: the top level 2024-09-10T21:26:47.921 INFO:tasks.workunit.client.0.smithi017.stderr:configure.in:36: warning: The macro `AC_HEADER_STDC' is obsolete. 2024-09-10T21:26:47.921 INFO:tasks.workunit.client.0.smithi017.stderr:configure.in:36: You should run autoupdate. 2024-09-10T21:26:47.921 INFO:tasks.workunit.client.0.smithi017.stderr:./lib/autoconf/headers.m4:704: AC_HEADER_STDC is expanded from... 2024-09-10T21:26:47.922 INFO:tasks.workunit.client.0.smithi017.stderr:configure.in:36: the top level 2024-09-10T21:26:47.922 INFO:tasks.workunit.client.0.smithi017.stderr:configure.in:42: warning: The macro `AC_HEADER_TIME' is obsolete. 2024-09-10T21:26:47.922 INFO:tasks.workunit.client.0.smithi017.stderr:configure.in:42: You should run autoupdate. 2024-09-10T21:26:47.922 INFO:tasks.workunit.client.0.smithi017.stderr:./lib/autoconf/headers.m4:743: AC_HEADER_TIME is expanded from... 2024-09-10T21:26:47.922 INFO:tasks.workunit.client.0.smithi017.stderr:configure.in:42: the top level 2024-09-10T21:26:47.922 INFO:tasks.workunit.client.0.smithi017.stderr:configure.in:45: warning: The macro `AC_FUNC_SETVBUF_REVERSED' is obsolete. Remove it and all references to SETVBUF_REVERSED. 2024-09-10T21:26:47.922 INFO:tasks.workunit.client.0.smithi017.stderr:./lib/autoconf/functions.m4:1766: AC_FUNC_SETVBUF_REVERSED is expanded from... 2024-09-10T21:26:47.922 INFO:tasks.workunit.client.0.smithi017.stderr:configure.in:45: the top level 2024-09-10T21:26:47.922 INFO:tasks.workunit.client.0.smithi017.stderr:configure.in:50: warning: AC_OUTPUT should be used without arguments. 2024-09-10T21:26:47.922 INFO:tasks.workunit.client.0.smithi017.stderr:configure.in:50: You should run autoupdate. 2024-09-10T21:26:48.017 INFO:tasks.workunit.client.0.smithi017.stdout:/bin/bash ./config.status --recheck 2024-09-10T21:26:48.155 INFO:tasks.workunit.client.0.smithi017.stdout:running CONFIG_SHELL=/bin/bash /bin/bash ./configure --no-create --no-recursion 2024-09-10T21:26:48.406 INFO:tasks.workunit.client.0.smithi017.stdout:checking for a BSD-compatible install... /usr/bin/install -c 2024-09-10T21:26:49.423 INFO:tasks.workunit.client.0.smithi017.stdout:checking whether build environment is sane... yes 2024-09-10T21:26:49.445 INFO:tasks.workunit.client.0.smithi017.stdout:checking for a race-free mkdir -p... /usr/bin/mkdir -p 2024-09-10T21:26:49.447 INFO:tasks.workunit.client.0.smithi017.stdout:checking for gawk... gawk 2024-09-10T21:26:49.482 INFO:tasks.workunit.client.0.smithi017.stdout:checking whether make sets $(MAKE)... yes 2024-09-10T21:26:49.573 INFO:tasks.workunit.client.0.smithi017.stdout:checking build system type... x86_64-unknown-linux-gnu 2024-09-10T21:26:49.573 INFO:tasks.workunit.client.0.smithi017.stdout:checking host system type... x86_64-unknown-linux-gnu 2024-09-10T21:26:49.601 INFO:tasks.workunit.client.0.smithi017.stdout:checking for style of include used by make... GNU 2024-09-10T21:26:49.610 INFO:tasks.workunit.client.0.smithi017.stdout:checking for gcc... gcc 2024-09-10T21:26:49.826 INFO:tasks.workunit.client.0.smithi017.stdout:checking whether the C compiler works... yes 2024-09-10T21:26:49.826 INFO:tasks.workunit.client.0.smithi017.stdout:checking for C compiler default output file name... a.out 2024-09-10T21:26:49.911 INFO:tasks.workunit.client.0.smithi017.stdout:checking for suffix of executables... 2024-09-10T21:26:50.004 INFO:tasks.workunit.client.0.smithi017.stdout:checking whether we are cross compiling... no 2024-09-10T21:26:50.068 INFO:tasks.workunit.client.0.smithi017.stdout:checking for suffix of object files... o 2024-09-10T21:26:50.150 INFO:tasks.workunit.client.0.smithi017.stdout:checking whether the compiler supports GNU C... yes 2024-09-10T21:26:50.198 INFO:tasks.workunit.client.0.smithi017.stdout:checking whether gcc accepts -g... yes 2024-09-10T21:26:50.282 INFO:tasks.workunit.client.0.smithi017.stdout:checking for gcc option to enable C11 features... none needed 2024-09-10T21:26:50.493 INFO:tasks.workunit.client.0.smithi017.stdout:checking dependency style of gcc... gcc3 2024-09-10T21:26:50.544 INFO:tasks.workunit.client.0.smithi017.stdout:checking for stdio.h... yes 2024-09-10T21:26:50.601 INFO:tasks.workunit.client.0.smithi017.stdout:checking for stdlib.h... yes 2024-09-10T21:26:50.661 INFO:tasks.workunit.client.0.smithi017.stdout:checking for string.h... yes 2024-09-10T21:26:50.720 INFO:tasks.workunit.client.0.smithi017.stdout:checking for inttypes.h... yes 2024-09-10T21:26:50.781 INFO:tasks.workunit.client.0.smithi017.stdout:checking for stdint.h... yes 2024-09-10T21:26:50.841 INFO:tasks.workunit.client.0.smithi017.stdout:checking for strings.h... yes 2024-09-10T21:26:50.901 INFO:tasks.workunit.client.0.smithi017.stdout:checking for sys/stat.h... yes 2024-09-10T21:26:50.961 INFO:tasks.workunit.client.0.smithi017.stdout:checking for sys/types.h... yes 2024-09-10T21:26:51.037 INFO:tasks.workunit.client.0.smithi017.stdout:checking for unistd.h... yes 2024-09-10T21:26:51.107 INFO:tasks.workunit.client.0.smithi017.stdout:checking for wchar.h... yes 2024-09-10T21:26:51.216 INFO:tasks.workunit.client.0.smithi017.stdout:checking for minix/config.h... no 2024-09-10T21:26:51.279 INFO:tasks.workunit.client.0.smithi017.stdout:checking for sys/time.h... yes 2024-09-10T21:26:51.350 INFO:tasks.workunit.client.0.smithi017.stdout:checking whether it is safe to define __EXTENSIONS__... yes 2024-09-10T21:26:51.406 INFO:tasks.workunit.client.0.smithi017.stdout:checking whether _XOPEN_SOURCE should be defined... no 2024-09-10T21:26:51.421 INFO:tasks.workunit.client.0.smithi017.stdout:checking for gcc... (cached) gcc 2024-09-10T21:26:51.536 INFO:tasks.workunit.client.0.smithi017.stdout:checking whether the compiler supports GNU C... (cached) yes 2024-09-10T21:26:51.537 INFO:tasks.workunit.client.0.smithi017.stdout:checking whether gcc accepts -g... (cached) yes 2024-09-10T21:26:51.538 INFO:tasks.workunit.client.0.smithi017.stdout:checking for gcc option to enable C11 features... (cached) none needed 2024-09-10T21:26:51.538 INFO:tasks.workunit.client.0.smithi017.stdout:checking dependency style of gcc... (cached) gcc3 2024-09-10T21:26:51.642 INFO:tasks.workunit.client.0.smithi017.stdout:checking for main in -lm... yes 2024-09-10T21:26:51.752 INFO:tasks.workunit.client.0.smithi017.stdout:checking for main in -lpthread... yes 2024-09-10T21:26:51.760 INFO:tasks.workunit.client.0.smithi017.stdout:checking for grep that handles long lines and -e... /usr/bin/grep 2024-09-10T21:26:51.764 INFO:tasks.workunit.client.0.smithi017.stdout:checking for egrep... /usr/bin/grep -E 2024-09-10T21:26:51.828 INFO:tasks.workunit.client.0.smithi017.stdout:checking for sys/wait.h that is POSIX.1 compatible... yes 2024-09-10T21:26:51.919 INFO:tasks.workunit.client.0.smithi017.stdout:checking for pthread.h... yes 2024-09-10T21:26:51.990 INFO:tasks.workunit.client.0.smithi017.stdout:checking for fcntl.h... yes 2024-09-10T21:26:52.059 INFO:tasks.workunit.client.0.smithi017.stdout:checking for limits.h... yes 2024-09-10T21:26:52.061 INFO:tasks.workunit.client.0.smithi017.stdout:checking for stdint.h... (cached) yes 2024-09-10T21:26:52.063 INFO:tasks.workunit.client.0.smithi017.stdout:checking for sys/time.h... (cached) yes 2024-09-10T21:26:52.065 INFO:tasks.workunit.client.0.smithi017.stdout:checking for unistd.h... (cached) yes 2024-09-10T21:26:52.141 INFO:tasks.workunit.client.0.smithi017.stdout:checking for sys/vfs.h... yes 2024-09-10T21:26:52.317 INFO:tasks.workunit.client.0.smithi017.stdout:checking for sys/limits.h... no 2024-09-10T21:26:53.254 INFO:tasks.workunit.client.0.smithi017.stdout:checking for an ANSI C-conforming const... yes 2024-09-10T21:26:53.375 INFO:tasks.workunit.client.0.smithi017.stdout:checking for system... yes 2024-09-10T21:26:53.481 INFO:tasks.workunit.client.0.smithi017.stdout:checking for gettimeofday... yes 2024-09-10T21:26:53.592 INFO:tasks.workunit.client.0.smithi017.stdout:checking for mkdir... yes 2024-09-10T21:26:53.700 INFO:tasks.workunit.client.0.smithi017.stdout:checking for strerror... yes 2024-09-10T21:26:53.807 INFO:tasks.workunit.client.0.smithi017.stdout:checking for open64... yes 2024-09-10T21:26:53.914 INFO:tasks.workunit.client.0.smithi017.stdout:checking for stat64... yes 2024-09-10T21:26:54.038 INFO:tasks.workunit.client.0.smithi017.stdout:checking for fseeko64... yes 2024-09-10T21:26:54.143 INFO:tasks.workunit.client.0.smithi017.stdout:checking for lrand48_r... yes 2024-09-10T21:26:54.255 INFO:tasks.workunit.client.0.smithi017.stdout:checking for srand48_r... yes 2024-09-10T21:26:54.290 INFO:tasks.workunit.client.0.smithi017.stdout:configure: creating ./config.status 2024-09-10T21:26:54.485 INFO:tasks.workunit.client.0.smithi017.stdout: /bin/bash ./config.status 2024-09-10T21:26:54.613 INFO:tasks.workunit.client.0.smithi017.stdout:config.status: creating Makefile 2024-09-10T21:26:54.646 INFO:tasks.workunit.client.0.smithi017.stdout:config.status: creating config.h 2024-09-10T21:26:54.692 INFO:tasks.workunit.client.0.smithi017.stdout:config.status: executing depfiles commands 2024-09-10T21:26:54.735 INFO:tasks.workunit.client.0.smithi017.stdout:config.status: executing default commands 2024-09-10T21:26:54.807 INFO:tasks.workunit.client.0.smithi017.stdout:cd . && /bin/bash /home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb/missing --run autoheader 2024-09-10T21:26:54.914 INFO:tasks.workunit.client.0.smithi017.stderr:autoheader: warning: autoconf input should be named 'configure.ac', not 'configure.in' 2024-09-10T21:26:55.016 INFO:tasks.workunit.client.0.smithi017.stderr:aclocal.m4:16: warning: this file was generated for autoconf 2.61. 2024-09-10T21:26:55.017 INFO:tasks.workunit.client.0.smithi017.stderr:You have another version of autoconf. It may work, but is not guaranteed to. 2024-09-10T21:26:55.017 INFO:tasks.workunit.client.0.smithi017.stderr:If you have problems, you may need to regenerate the build system entirely. 2024-09-10T21:26:55.017 INFO:tasks.workunit.client.0.smithi017.stderr:To do so, use the procedure documented by the package, typically `autoreconf'. 2024-09-10T21:26:55.169 INFO:tasks.workunit.client.0.smithi017.stdout:rm -f stamp-h1 2024-09-10T21:26:55.176 INFO:tasks.workunit.client.0.smithi017.stdout:touch config.h.in 2024-09-10T21:26:55.189 INFO:tasks.workunit.client.0.smithi017.stdout:cd . && /bin/bash ./config.status config.h 2024-09-10T21:26:55.292 INFO:tasks.workunit.client.0.smithi017.stdout:config.status: creating config.h 2024-09-10T21:26:55.352 INFO:tasks.workunit.client.0.smithi017.stdout:make all-am 2024-09-10T21:26:55.396 INFO:tasks.workunit.client.0.smithi017.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb' 2024-09-10T21:26:55.396 INFO:tasks.workunit.client.0.smithi017.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT fileops.o -MD -MP -MF .deps/fileops.Tpo -c -o fileops.o fileops.c 2024-09-10T21:26:55.642 INFO:tasks.workunit.client.0.smithi017.stdout:mv -f .deps/fileops.Tpo .deps/fileops.Po 2024-09-10T21:26:55.654 INFO:tasks.workunit.client.0.smithi017.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT rand.o -MD -MP -MF .deps/rand.Tpo -c -o rand.o rand.c 2024-09-10T21:26:55.805 INFO:tasks.workunit.client.0.smithi017.stdout:mv -f .deps/rand.Tpo .deps/rand.Po 2024-09-10T21:26:55.817 INFO:tasks.workunit.client.0.smithi017.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT main.o -MD -MP -MF .deps/main.Tpo -c -o main.o main.c 2024-09-10T21:26:55.974 INFO:tasks.workunit.client.0.smithi017.stdout:mv -f .deps/main.Tpo .deps/main.Po 2024-09-10T21:26:55.986 INFO:tasks.workunit.client.0.smithi017.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT fh.o -MD -MP -MF .deps/fh.Tpo -c -o fh.o fh.c 2024-09-10T21:26:56.172 INFO:tasks.workunit.client.0.smithi017.stdout:mv -f .deps/fh.Tpo .deps/fh.Po 2024-09-10T21:26:56.184 INFO:tasks.workunit.client.0.smithi017.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT filelist.o -MD -MP -MF .deps/filelist.Tpo -c -o filelist.o filelist.c 2024-09-10T21:26:56.377 INFO:tasks.workunit.client.0.smithi017.stdout:mv -f .deps/filelist.Tpo .deps/filelist.Po 2024-09-10T21:26:56.391 INFO:tasks.workunit.client.0.smithi017.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT metaops.o -MD -MP -MF .deps/metaops.Tpo -c -o metaops.o metaops.c 2024-09-10T21:26:56.501 INFO:tasks.workunit.client.0.smithi017.stdout:mv -f .deps/metaops.Tpo .deps/metaops.Po 2024-09-10T21:26:56.514 INFO:tasks.workunit.client.0.smithi017.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT rwlock.o -MD -MP -MF .deps/rwlock.Tpo -c -o rwlock.o rwlock.c 2024-09-10T21:26:56.592 INFO:tasks.workunit.client.0.smithi017.stdout:mv -f .deps/rwlock.Tpo .deps/rwlock.Po 2024-09-10T21:26:56.605 INFO:tasks.workunit.client.0.smithi017.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT cirlist.o -MD -MP -MF .deps/cirlist.Tpo -c -o cirlist.o cirlist.c 2024-09-10T21:26:56.698 INFO:tasks.workunit.client.0.smithi017.stdout:mv -f .deps/cirlist.Tpo .deps/cirlist.Po 2024-09-10T21:26:56.710 INFO:tasks.workunit.client.0.smithi017.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT rbt.o -MD -MP -MF .deps/rbt.Tpo -c -o rbt.o rbt.c 2024-09-10T21:26:56.966 INFO:tasks.workunit.client.0.smithi017.stdout:mv -f .deps/rbt.Tpo .deps/rbt.Po 2024-09-10T21:26:56.979 INFO:tasks.workunit.client.0.smithi017.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_tg.o -MD -MP -MF .deps/ffsb_tg.Tpo -c -o ffsb_tg.o ffsb_tg.c 2024-09-10T21:26:57.173 INFO:tasks.workunit.client.0.smithi017.stdout:mv -f .deps/ffsb_tg.Tpo .deps/ffsb_tg.Po 2024-09-10T21:26:57.185 INFO:tasks.workunit.client.0.smithi017.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_fs.o -MD -MP -MF .deps/ffsb_fs.Tpo -c -o ffsb_fs.o ffsb_fs.c 2024-09-10T21:26:57.421 INFO:tasks.workunit.client.0.smithi017.stdout:mv -f .deps/ffsb_fs.Tpo .deps/ffsb_fs.Po 2024-09-10T21:26:57.433 INFO:tasks.workunit.client.0.smithi017.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_thread.o -MD -MP -MF .deps/ffsb_thread.Tpo -c -o ffsb_thread.o ffsb_thread.c 2024-09-10T21:26:57.559 INFO:tasks.workunit.client.0.smithi017.stdout:mv -f .deps/ffsb_thread.Tpo .deps/ffsb_thread.Po 2024-09-10T21:26:57.572 INFO:tasks.workunit.client.0.smithi017.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_op.o -MD -MP -MF .deps/ffsb_op.Tpo -c -o ffsb_op.o ffsb_op.c 2024-09-10T21:26:57.644 INFO:tasks.workunit.client.0.smithi017.stderr:ffsb_op.c: In function 'print_op_results': 2024-09-10T21:26:57.644 INFO:tasks.workunit.client.0.smithi017.stderr:ffsb_op.c:83:37: warning: embedded '\0' in format [-Wformat-contains-nul] 2024-09-10T21:26:57.644 INFO:tasks.workunit.client.0.smithi017.stderr: 83 | sprintf(buf, "%s/sec\0", buf); 2024-09-10T21:26:57.644 INFO:tasks.workunit.client.0.smithi017.stderr: | ^~ 2024-09-10T21:26:57.644 INFO:tasks.workunit.client.0.smithi017.stderr:ffsb_op.c:86:33: warning: embedded '\0' in format [-Wformat-contains-nul] 2024-09-10T21:26:57.644 INFO:tasks.workunit.client.0.smithi017.stderr: 86 | sprintf(buf, "NA\0"); 2024-09-10T21:26:57.644 INFO:tasks.workunit.client.0.smithi017.stderr: | ^~ 2024-09-10T21:26:57.660 INFO:tasks.workunit.client.0.smithi017.stderr:ffsb_op.c: In function 'print_results': 2024-09-10T21:26:57.660 INFO:tasks.workunit.client.0.smithi017.stderr:ffsb_op.c:83:33: warning: '/sec' directive writing 4 bytes into a region of size between 1 and 256 [-Wformat-overflow=] 2024-09-10T21:26:57.660 INFO:tasks.workunit.client.0.smithi017.stderr: 83 | sprintf(buf, "%s/sec\0", buf); 2024-09-10T21:26:57.661 INFO:tasks.workunit.client.0.smithi017.stderr: | ^~~~ 2024-09-10T21:26:57.661 INFO:tasks.workunit.client.0.smithi017.stderr:In file included from /usr/include/stdio.h:894, 2024-09-10T21:26:57.661 INFO:tasks.workunit.client.0.smithi017.stderr: from ffsb_op.c:20: 2024-09-10T21:26:57.661 INFO:tasks.workunit.client.0.smithi017.stderr:/usr/include/x86_64-linux-gnu/bits/stdio2.h:38:10: note: '__builtin___sprintf_chk' output between 5 and 260 bytes into a destination of size 256 2024-09-10T21:26:57.661 INFO:tasks.workunit.client.0.smithi017.stderr: 38 | return __builtin___sprintf_chk (__s, __USE_FORTIFY_LEVEL - 1, 2024-09-10T21:26:57.661 INFO:tasks.workunit.client.0.smithi017.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-09-10T21:26:57.661 INFO:tasks.workunit.client.0.smithi017.stderr: 39 | __glibc_objsize (__s), __fmt, 2024-09-10T21:26:57.661 INFO:tasks.workunit.client.0.smithi017.stderr: | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-09-10T21:26:57.661 INFO:tasks.workunit.client.0.smithi017.stderr: 40 | __va_arg_pack ()); 2024-09-10T21:26:57.661 INFO:tasks.workunit.client.0.smithi017.stderr: | ~~~~~~~~~~~~~~~~~ 2024-09-10T21:26:57.706 INFO:tasks.workunit.client.0.smithi017.stdout:mv -f .deps/ffsb_op.Tpo .deps/ffsb_op.Po 2024-09-10T21:26:57.719 INFO:tasks.workunit.client.0.smithi017.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT util.o -MD -MP -MF .deps/util.Tpo -c -o util.o util.c 2024-09-10T21:26:57.901 INFO:tasks.workunit.client.0.smithi017.stdout:mv -f .deps/util.Tpo .deps/util.Po 2024-09-10T21:26:57.914 INFO:tasks.workunit.client.0.smithi017.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT parser.o -MD -MP -MF .deps/parser.Tpo -c -o parser.o parser.c 2024-09-10T21:26:57.990 INFO:tasks.workunit.client.0.smithi017.stderr:In file included from parser.h:22, 2024-09-10T21:26:57.990 INFO:tasks.workunit.client.0.smithi017.stderr: from parser.c:25: 2024-09-10T21:26:57.990 INFO:tasks.workunit.client.0.smithi017.stderr:parser.c: In function 'init_filesys': 2024-09-10T21:26:57.991 INFO:tasks.workunit.client.0.smithi017.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-09-10T21:26:57.992 INFO:tasks.workunit.client.0.smithi017.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-09-10T21:26:57.992 INFO:tasks.workunit.client.0.smithi017.stderr: | ^ 2024-09-10T21:26:57.992 INFO:tasks.workunit.client.0.smithi017.stderr:list.h:33:35: note: in expansion of macro 'offsetof' 2024-09-10T21:26:57.992 INFO:tasks.workunit.client.0.smithi017.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-09-10T21:26:57.992 INFO:tasks.workunit.client.0.smithi017.stderr: | ^~~~~~~~ 2024-09-10T21:26:57.992 INFO:tasks.workunit.client.0.smithi017.stderr:list.h:36:9: note: in expansion of macro 'container_of' 2024-09-10T21:26:57.992 INFO:tasks.workunit.client.0.smithi017.stderr: 36 | container_of(ptr, type, member) 2024-09-10T21:26:57.992 INFO:tasks.workunit.client.0.smithi017.stderr: | ^~~~~~~~~~~~ 2024-09-10T21:26:57.992 INFO:tasks.workunit.client.0.smithi017.stderr:list.h:39:20: note: in expansion of macro 'list_entry' 2024-09-10T21:26:57.993 INFO:tasks.workunit.client.0.smithi017.stderr: 39 | for (pos = list_entry((head)->next, typeof(*pos), member); \ 2024-09-10T21:26:57.993 INFO:tasks.workunit.client.0.smithi017.stderr: | ^~~~~~~~~~ 2024-09-10T21:26:57.993 INFO:tasks.workunit.client.0.smithi017.stderr:parser.c:934:17: note: in expansion of macro 'list_for_each_entry' 2024-09-10T21:26:57.993 INFO:tasks.workunit.client.0.smithi017.stderr: 934 | list_for_each_entry(tmp_list, &list_head->list, list) 2024-09-10T21:26:57.993 INFO:tasks.workunit.client.0.smithi017.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-09-10T21:26:57.994 INFO:tasks.workunit.client.0.smithi017.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-09-10T21:26:57.994 INFO:tasks.workunit.client.0.smithi017.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-09-10T21:26:57.994 INFO:tasks.workunit.client.0.smithi017.stderr: | ^ 2024-09-10T21:26:57.995 INFO:tasks.workunit.client.0.smithi017.stderr:list.h:33:35: note: in expansion of macro 'offsetof' 2024-09-10T21:26:57.995 INFO:tasks.workunit.client.0.smithi017.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-09-10T21:26:57.995 INFO:tasks.workunit.client.0.smithi017.stderr: | ^~~~~~~~ 2024-09-10T21:26:57.995 INFO:tasks.workunit.client.0.smithi017.stderr:list.h:36:9: note: in expansion of macro 'container_of' 2024-09-10T21:26:57.995 INFO:tasks.workunit.client.0.smithi017.stderr: 36 | container_of(ptr, type, member) 2024-09-10T21:26:57.995 INFO:tasks.workunit.client.0.smithi017.stderr: | ^~~~~~~~~~~~ 2024-09-10T21:26:57.995 INFO:tasks.workunit.client.0.smithi017.stderr:list.h:41:20: note: in expansion of macro 'list_entry' 2024-09-10T21:26:57.995 INFO:tasks.workunit.client.0.smithi017.stderr: 41 | pos = list_entry(pos->member.next, typeof(*pos), member)) 2024-09-10T21:26:57.995 INFO:tasks.workunit.client.0.smithi017.stderr: | ^~~~~~~~~~ 2024-09-10T21:26:57.995 INFO:tasks.workunit.client.0.smithi017.stderr:parser.c:934:17: note: in expansion of macro 'list_for_each_entry' 2024-09-10T21:26:57.995 INFO:tasks.workunit.client.0.smithi017.stderr: 934 | list_for_each_entry(tmp_list, &list_head->list, list) 2024-09-10T21:26:57.995 INFO:tasks.workunit.client.0.smithi017.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-09-10T21:26:57.996 INFO:tasks.workunit.client.0.smithi017.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-09-10T21:26:57.996 INFO:tasks.workunit.client.0.smithi017.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-09-10T21:26:57.996 INFO:tasks.workunit.client.0.smithi017.stderr: | ^ 2024-09-10T21:26:57.996 INFO:tasks.workunit.client.0.smithi017.stderr:list.h:33:35: note: in expansion of macro 'offsetof' 2024-09-10T21:26:57.996 INFO:tasks.workunit.client.0.smithi017.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-09-10T21:26:57.996 INFO:tasks.workunit.client.0.smithi017.stderr: | ^~~~~~~~ 2024-09-10T21:26:57.996 INFO:tasks.workunit.client.0.smithi017.stderr:list.h:36:9: note: in expansion of macro 'container_of' 2024-09-10T21:26:57.996 INFO:tasks.workunit.client.0.smithi017.stderr: 36 | container_of(ptr, type, member) 2024-09-10T21:26:57.996 INFO:tasks.workunit.client.0.smithi017.stderr: | ^~~~~~~~~~~~ 2024-09-10T21:26:57.996 INFO:tasks.workunit.client.0.smithi017.stderr:list.h:39:20: note: in expansion of macro 'list_entry' 2024-09-10T21:26:57.996 INFO:tasks.workunit.client.0.smithi017.stderr: 39 | for (pos = list_entry((head)->next, typeof(*pos), member); \ 2024-09-10T21:26:57.997 INFO:tasks.workunit.client.0.smithi017.stderr: | ^~~~~~~~~~ 2024-09-10T21:26:57.997 INFO:tasks.workunit.client.0.smithi017.stderr:parser.c:941:17: note: in expansion of macro 'list_for_each_entry' 2024-09-10T21:26:57.997 INFO:tasks.workunit.client.0.smithi017.stderr: 941 | list_for_each_entry(tmp_list, &list_head->list, list) { 2024-09-10T21:26:57.997 INFO:tasks.workunit.client.0.smithi017.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-09-10T21:26:57.997 INFO:tasks.workunit.client.0.smithi017.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-09-10T21:26:57.997 INFO:tasks.workunit.client.0.smithi017.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-09-10T21:26:57.997 INFO:tasks.workunit.client.0.smithi017.stderr: | ^ 2024-09-10T21:26:57.997 INFO:tasks.workunit.client.0.smithi017.stderr:list.h:33:35: note: in expansion of macro 'offsetof' 2024-09-10T21:26:57.997 INFO:tasks.workunit.client.0.smithi017.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-09-10T21:26:57.997 INFO:tasks.workunit.client.0.smithi017.stderr: | ^~~~~~~~ 2024-09-10T21:26:57.997 INFO:tasks.workunit.client.0.smithi017.stderr:list.h:36:9: note: in expansion of macro 'container_of' 2024-09-10T21:26:57.997 INFO:tasks.workunit.client.0.smithi017.stderr: 36 | container_of(ptr, type, member) 2024-09-10T21:26:57.997 INFO:tasks.workunit.client.0.smithi017.stderr: | ^~~~~~~~~~~~ 2024-09-10T21:26:57.998 INFO:tasks.workunit.client.0.smithi017.stderr:list.h:41:20: note: in expansion of macro 'list_entry' 2024-09-10T21:26:57.998 INFO:tasks.workunit.client.0.smithi017.stderr: 41 | pos = list_entry(pos->member.next, typeof(*pos), member)) 2024-09-10T21:26:57.998 INFO:tasks.workunit.client.0.smithi017.stderr: | ^~~~~~~~~~ 2024-09-10T21:26:57.998 INFO:tasks.workunit.client.0.smithi017.stderr:parser.c:941:17: note: in expansion of macro 'list_for_each_entry' 2024-09-10T21:26:57.998 INFO:tasks.workunit.client.0.smithi017.stderr: 941 | list_for_each_entry(tmp_list, &list_head->list, list) { 2024-09-10T21:26:57.998 INFO:tasks.workunit.client.0.smithi017.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-09-10T21:26:57.998 INFO:tasks.workunit.client.0.smithi017.stderr:parser.c: In function 'init_tg_stats': 2024-09-10T21:26:57.998 INFO:tasks.workunit.client.0.smithi017.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-09-10T21:26:57.998 INFO:tasks.workunit.client.0.smithi017.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-09-10T21:26:57.998 INFO:tasks.workunit.client.0.smithi017.stderr: | ^ 2024-09-10T21:26:57.998 INFO:tasks.workunit.client.0.smithi017.stderr:list.h:33:35: note: in expansion of macro 'offsetof' 2024-09-10T21:26:57.998 INFO:tasks.workunit.client.0.smithi017.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-09-10T21:26:57.999 INFO:tasks.workunit.client.0.smithi017.stderr: | ^~~~~~~~ 2024-09-10T21:26:57.999 INFO:tasks.workunit.client.0.smithi017.stderr:list.h:36:9: note: in expansion of macro 'container_of' 2024-09-10T21:26:57.999 INFO:tasks.workunit.client.0.smithi017.stderr: 36 | container_of(ptr, type, member) 2024-09-10T21:26:57.999 INFO:tasks.workunit.client.0.smithi017.stderr: | ^~~~~~~~~~~~ 2024-09-10T21:26:57.999 INFO:tasks.workunit.client.0.smithi017.stderr:list.h:39:20: note: in expansion of macro 'list_entry' 2024-09-10T21:26:57.999 INFO:tasks.workunit.client.0.smithi017.stderr: 39 | for (pos = list_entry((head)->next, typeof(*pos), member); \ 2024-09-10T21:26:57.999 INFO:tasks.workunit.client.0.smithi017.stderr: | ^~~~~~~~~~ 2024-09-10T21:26:57.999 INFO:tasks.workunit.client.0.smithi017.stderr:parser.c:971:41: note: in expansion of macro 'list_for_each_entry' 2024-09-10T21:26:57.999 INFO:tasks.workunit.client.0.smithi017.stderr: 971 | list_for_each_entry(tmp_list, 2024-09-10T21:26:57.999 INFO:tasks.workunit.client.0.smithi017.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-09-10T21:26:57.999 INFO:tasks.workunit.client.0.smithi017.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-09-10T21:26:58.000 INFO:tasks.workunit.client.0.smithi017.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-09-10T21:26:58.000 INFO:tasks.workunit.client.0.smithi017.stderr: | ^ 2024-09-10T21:26:58.000 INFO:tasks.workunit.client.0.smithi017.stderr:list.h:33:35: note: in expansion of macro 'offsetof' 2024-09-10T21:26:58.000 INFO:tasks.workunit.client.0.smithi017.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-09-10T21:26:58.000 INFO:tasks.workunit.client.0.smithi017.stderr: | ^~~~~~~~ 2024-09-10T21:26:58.000 INFO:tasks.workunit.client.0.smithi017.stderr:list.h:36:9: note: in expansion of macro 'container_of' 2024-09-10T21:26:58.000 INFO:tasks.workunit.client.0.smithi017.stderr: 36 | container_of(ptr, type, member) 2024-09-10T21:26:58.000 INFO:tasks.workunit.client.0.smithi017.stderr: | ^~~~~~~~~~~~ 2024-09-10T21:26:58.000 INFO:tasks.workunit.client.0.smithi017.stderr:list.h:41:20: note: in expansion of macro 'list_entry' 2024-09-10T21:26:58.000 INFO:tasks.workunit.client.0.smithi017.stderr: 41 | pos = list_entry(pos->member.next, typeof(*pos), member)) 2024-09-10T21:26:58.001 INFO:tasks.workunit.client.0.smithi017.stderr: | ^~~~~~~~~~ 2024-09-10T21:26:58.001 INFO:tasks.workunit.client.0.smithi017.stderr:parser.c:971:41: note: in expansion of macro 'list_for_each_entry' 2024-09-10T21:26:58.001 INFO:tasks.workunit.client.0.smithi017.stderr: 971 | list_for_each_entry(tmp_list, 2024-09-10T21:26:58.001 INFO:tasks.workunit.client.0.smithi017.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-09-10T21:26:58.001 INFO:tasks.workunit.client.0.smithi017.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-09-10T21:26:58.001 INFO:tasks.workunit.client.0.smithi017.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-09-10T21:26:58.001 INFO:tasks.workunit.client.0.smithi017.stderr: | ^ 2024-09-10T21:26:58.001 INFO:tasks.workunit.client.0.smithi017.stderr:list.h:33:35: note: in expansion of macro 'offsetof' 2024-09-10T21:26:58.001 INFO:tasks.workunit.client.0.smithi017.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-09-10T21:26:58.001 INFO:tasks.workunit.client.0.smithi017.stderr: | ^~~~~~~~ 2024-09-10T21:26:58.002 INFO:tasks.workunit.client.0.smithi017.stderr:list.h:36:9: note: in expansion of macro 'container_of' 2024-09-10T21:26:58.002 INFO:tasks.workunit.client.0.smithi017.stderr: 36 | container_of(ptr, type, member) 2024-09-10T21:26:58.002 INFO:tasks.workunit.client.0.smithi017.stderr: | ^~~~~~~~~~~~ 2024-09-10T21:26:58.002 INFO:tasks.workunit.client.0.smithi017.stderr:list.h:39:20: note: in expansion of macro 'list_entry' 2024-09-10T21:26:58.002 INFO:tasks.workunit.client.0.smithi017.stderr: 39 | for (pos = list_entry((head)->next, typeof(*pos), member); \ 2024-09-10T21:26:58.002 INFO:tasks.workunit.client.0.smithi017.stderr: | ^~~~~~~~~~ 2024-09-10T21:26:58.002 INFO:tasks.workunit.client.0.smithi017.stderr:parser.c:980:41: note: in expansion of macro 'list_for_each_entry' 2024-09-10T21:26:58.002 INFO:tasks.workunit.client.0.smithi017.stderr: 980 | list_for_each_entry(tmp_list, 2024-09-10T21:26:58.002 INFO:tasks.workunit.client.0.smithi017.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-09-10T21:26:58.003 INFO:tasks.workunit.client.0.smithi017.stderr:list.h:29:33: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast] 2024-09-10T21:26:58.003 INFO:tasks.workunit.client.0.smithi017.stderr: 29 | #define offsetof(type, member) ((int) &((type *)0)->member) 2024-09-10T21:26:58.003 INFO:tasks.workunit.client.0.smithi017.stderr: | ^ 2024-09-10T21:26:58.003 INFO:tasks.workunit.client.0.smithi017.stderr:list.h:33:35: note: in expansion of macro 'offsetof' 2024-09-10T21:26:58.003 INFO:tasks.workunit.client.0.smithi017.stderr: 33 | (type *)((char *)__mptr - offsetof(type, member)); }) 2024-09-10T21:26:58.003 INFO:tasks.workunit.client.0.smithi017.stderr: | ^~~~~~~~ 2024-09-10T21:26:58.003 INFO:tasks.workunit.client.0.smithi017.stderr:list.h:36:9: note: in expansion of macro 'container_of' 2024-09-10T21:26:58.003 INFO:tasks.workunit.client.0.smithi017.stderr: 36 | container_of(ptr, type, member) 2024-09-10T21:26:58.003 INFO:tasks.workunit.client.0.smithi017.stderr: | ^~~~~~~~~~~~ 2024-09-10T21:26:58.003 INFO:tasks.workunit.client.0.smithi017.stderr:list.h:41:20: note: in expansion of macro 'list_entry' 2024-09-10T21:26:58.004 INFO:tasks.workunit.client.0.smithi017.stderr: 41 | pos = list_entry(pos->member.next, typeof(*pos), member)) 2024-09-10T21:26:58.004 INFO:tasks.workunit.client.0.smithi017.stderr: | ^~~~~~~~~~ 2024-09-10T21:26:58.004 INFO:tasks.workunit.client.0.smithi017.stderr:parser.c:980:41: note: in expansion of macro 'list_for_each_entry' 2024-09-10T21:26:58.004 INFO:tasks.workunit.client.0.smithi017.stderr: 980 | list_for_each_entry(tmp_list, 2024-09-10T21:26:58.004 INFO:tasks.workunit.client.0.smithi017.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-09-10T21:26:58.047 INFO:tasks.workunit.client.0.smithi017.stderr:parser.c: In function 'get_optstr': 2024-09-10T21:26:58.047 INFO:tasks.workunit.client.0.smithi017.stderr:parser.c:206:23: warning: 'strnlen' specified bound 4095 exceeds source size 1024 [-Wstringop-overread] 2024-09-10T21:26:58.048 INFO:tasks.workunit.client.0.smithi017.stderr: 206 | len = strnlen(temp, 4095) + 1; 2024-09-10T21:26:58.048 INFO:tasks.workunit.client.0.smithi017.stderr: | ^~~~~~~~~~~~~~~~~~~ 2024-09-10T21:26:58.048 INFO:tasks.workunit.client.0.smithi017.stderr:parser.c:200:14: note: source object declared here 2024-09-10T21:26:58.048 INFO:tasks.workunit.client.0.smithi017.stderr: 200 | char temp[BUFSIZE]; 2024-09-10T21:26:58.048 INFO:tasks.workunit.client.0.smithi017.stderr: | ^~~~ 2024-09-10T21:26:58.064 INFO:tasks.workunit.client.0.smithi017.stderr:In file included from /usr/include/string.h:535, 2024-09-10T21:26:58.064 INFO:tasks.workunit.client.0.smithi017.stderr: from parser.c:19: 2024-09-10T21:26:58.064 INFO:tasks.workunit.client.0.smithi017.stderr:In function 'memcpy', 2024-09-10T21:26:58.064 INFO:tasks.workunit.client.0.smithi017.stderr: inlined from 'size64_convert' at parser.c:119:2: 2024-09-10T21:26:58.064 INFO:tasks.workunit.client.0.smithi017.stderr:/usr/include/x86_64-linux-gnu/bits/string_fortified.h:29:10: warning: '__builtin_memcpy' reading 3 bytes from a region of size 2 [-Wstringop-overread] 2024-09-10T21:26:58.064 INFO:tasks.workunit.client.0.smithi017.stderr: 29 | return __builtin___memcpy_chk (__dest, __src, __len, 2024-09-10T21:26:58.064 INFO:tasks.workunit.client.0.smithi017.stderr: | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-09-10T21:26:58.065 INFO:tasks.workunit.client.0.smithi017.stderr: 30 | __glibc_objsize0 (__dest)); 2024-09-10T21:26:58.065 INFO:tasks.workunit.client.0.smithi017.stderr: | ~~~~~~~~~~~~~~~~~~~~~~~~~~ 2024-09-10T21:26:58.382 INFO:tasks.workunit.client.0.smithi017.stdout:mv -f .deps/parser.Tpo .deps/parser.Po 2024-09-10T21:26:58.395 INFO:tasks.workunit.client.0.smithi017.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_fc.o -MD -MP -MF .deps/ffsb_fc.Tpo -c -o ffsb_fc.o ffsb_fc.c 2024-09-10T21:26:58.766 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.c 2024-09-10T21:26:58.766 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-10T21:26:58.767 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-10T21:26:58.770 INFO:tasks.ceph.mon.c:Started 2024-09-10T21:26:58.771 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-09-10T21:26:58.771 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-10T21:26:58.806 INFO:tasks.workunit.client.0.smithi017.stdout:mv -f .deps/ffsb_fc.Tpo .deps/ffsb_fc.Po 2024-09-10T21:26:58.818 INFO:tasks.workunit.client.0.smithi017.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT ffsb_stats.o -MD -MP -MF .deps/ffsb_stats.Tpo -c -o ffsb_stats.o ffsb_stats.c 2024-09-10T21:26:58.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:58.921+0000 7fc6dbc00640 1 -- 172.21.15.17:0/2019851356 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc6d4154aa0 msgr2=0x7fc6d4072080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:58.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:58.921+0000 7fc6dbc00640 1 --2- 172.21.15.17:0/2019851356 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc6d4154aa0 0x7fc6d4072080 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc6d4073220 tx=0x7fc6bc01c890 comp rx=0 tx=0).stop 2024-09-10T21:26:58.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:58.921+0000 7fc6dbc00640 1 -- 172.21.15.17:0/2019851356 shutdown_connections 2024-09-10T21:26:58.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:58.921+0000 7fc6dbc00640 1 --2- 172.21.15.17:0/2019851356 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc6d4154aa0 0x7fc6d4072080 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:58.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:58.921+0000 7fc6dbc00640 1 --2- 172.21.15.17:0/2019851356 >> v2:172.21.15.31:3300/0 conn(0x7fc6d41546c0 0x7fc6d40ca430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:58.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:58.921+0000 7fc6dbc00640 1 -- 172.21.15.17:0/2019851356 >> 172.21.15.17:0/2019851356 conn(0x7fc6d40702f0 msgr2=0x7fc6d4070700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:26:58.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:58.921+0000 7fc6dbc00640 1 -- 172.21.15.17:0/2019851356 shutdown_connections 2024-09-10T21:26:58.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:58.921+0000 7fc6dbc00640 1 -- 172.21.15.17:0/2019851356 wait complete. 2024-09-10T21:26:58.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:58.925+0000 7fc6dbc00640 1 Processor -- start 2024-09-10T21:26:58.930 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:58.925+0000 7fc6dbc00640 1 -- start start 2024-09-10T21:26:58.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:58.925+0000 7fc6dbc00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc6d40725c0 0x7fc6d4157ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:58.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:58.925+0000 7fc6dbc00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc6d41546c0 0x7fc6d4154ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:58.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:58.925+0000 7fc6dbc00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc6d4154aa0 0x7fc6d4156d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:58.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:58.925+0000 7fc6dbc00640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fc6d40ce7b0 con 0x7fc6d4154aa0 2024-09-10T21:26:58.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:58.925+0000 7fc6dbc00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fc6d40ce630 con 0x7fc6d40725c0 2024-09-10T21:26:58.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:58.925+0000 7fc6dbc00640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7fc6d40c9640 con 0x7fc6d41546c0 2024-09-10T21:26:58.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:58.925+0000 7fc6d9c00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc6d4154aa0 0x7fc6d4156d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:58.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:58.925+0000 7fc6d9200640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc6d40725c0 0x7fc6d4157ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:58.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:58.925+0000 7fc6d9200640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc6d40725c0 0x7fc6d4157ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.17:46912/0 (socket says 172.21.15.17:46912) 2024-09-10T21:26:58.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:58.925+0000 7fc6cbe00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc6d41546c0 0x7fc6d4154ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:58.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:58.925+0000 7fc6d9200640 1 -- 172.21.15.17:0/2836916578 learned_addr learned my addr 172.21.15.17:0/2836916578 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:26:58.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:58.925+0000 7fc6cbe00640 1 -- 172.21.15.17:0/2836916578 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc6d41546c0 msgr2=0x7fc6d4154ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-09-10T21:26:58.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:58.925+0000 7fc6cbe00640 1 -- 172.21.15.17:0/2836916578 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc6d41546c0 msgr2=0x7fc6d4154ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-10T21:26:58.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:58.925+0000 7fc6cbe00640 1 --2- 172.21.15.17:0/2836916578 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc6d41546c0 0x7fc6d4154ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T21:26:58.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:58.925+0000 7fc6cbe00640 1 --2- 172.21.15.17:0/2836916578 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc6d41546c0 0x7fc6d4154ea0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:26:58.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:58.929+0000 7fc6d9c00640 1 -- 172.21.15.17:0/2836916578 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc6d41546c0 msgr2=0x7fc6d4154ea0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:26:58.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:58.929+0000 7fc6d9c00640 1 --2- 172.21.15.17:0/2836916578 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc6d41546c0 0x7fc6d4154ea0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:58.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:58.929+0000 7fc6d9c00640 1 -- 172.21.15.17:0/2836916578 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc6d40725c0 msgr2=0x7fc6d4157ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:26:58.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:58.929+0000 7fc6d9c00640 1 --2- 172.21.15.17:0/2836916578 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc6d40725c0 0x7fc6d4157ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:26:58.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:58.929+0000 7fc6d9c00640 1 -- 172.21.15.17:0/2836916578 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6d41f4140 con 0x7fc6d4154aa0 2024-09-10T21:26:58.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:58.929+0000 7fc6d9200640 1 --2- 172.21.15.17:0/2836916578 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc6d40725c0 0x7fc6d4157ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:26:58.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:58.929+0000 7fc6d9c00640 1 --2- 172.21.15.17:0/2836916578 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc6d4154aa0 0x7fc6d4156d40 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fc6d40b41c0 tx=0x7fc6cc00cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:58.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:58.929+0000 7fc6d1600640 1 -- 172.21.15.17:0/2836916578 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6cc003b80 con 0x7fc6d4154aa0 2024-09-10T21:26:58.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:58.929+0000 7fc6d1600640 1 -- 172.21.15.17:0/2836916578 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc6cc003d20 con 0x7fc6d4154aa0 2024-09-10T21:26:58.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:58.929+0000 7fc6d1600640 1 -- 172.21.15.17:0/2836916578 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6cc005330 con 0x7fc6d4154aa0 2024-09-10T21:26:58.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:58.929+0000 7fc6dbc00640 1 -- 172.21.15.17:0/2836916578 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6d414af40 con 0x7fc6d4154aa0 2024-09-10T21:26:58.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:58.929+0000 7fc6dbc00640 1 -- 172.21.15.17:0/2836916578 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6d4157280 con 0x7fc6d4154aa0 2024-09-10T21:26:58.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:58.929+0000 7fc6dbc00640 1 -- 172.21.15.17:0/2836916578 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6d40ca430 con 0x7fc6d4154aa0 2024-09-10T21:26:58.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:58.929+0000 7fc6d1600640 1 -- 172.21.15.17:0/2836916578 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fc6cc00a640 con 0x7fc6d4154aa0 2024-09-10T21:26:58.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:58.933+0000 7fc6d1600640 1 --2- 172.21.15.17:0/2836916578 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fc6b00725e0 0x7fc6b0074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:26:58.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:58.933+0000 7fc6d1600640 1 -- 172.21.15.17:0/2836916578 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fc6cc0935c0 con 0x7fc6d4154aa0 2024-09-10T21:26:58.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:58.933+0000 7fc6d9200640 1 --2- 172.21.15.17:0/2836916578 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fc6b00725e0 0x7fc6b0074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:26:58.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:58.933+0000 7fc6d9200640 1 --2- 172.21.15.17:0/2836916578 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fc6b00725e0 0x7fc6b0074aa0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fc6c40058d0 tx=0x7fc6c400b480 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:26:58.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:58.937+0000 7fc6d1600640 1 -- 172.21.15.17:0/2836916578 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fc6cc05e800 con 0x7fc6d4154aa0 2024-09-10T21:26:59.106 INFO:tasks.workunit.client.0.smithi017.stdout:mv -f .deps/ffsb_stats.Tpo .deps/ffsb_stats.Po 2024-09-10T21:26:59.118 INFO:tasks.workunit.client.0.smithi017.stdout:gcc -DHAVE_CONFIG_H -I. -D_GNU_SOURCE -g -O2 -D_REENTRANT -MT list.o -MD -MP -MF .deps/list.Tpo -c -o list.o list.c 2024-09-10T21:26:59.181 INFO:tasks.workunit.client.0.smithi017.stdout:mv -f .deps/list.Tpo .deps/list.Po 2024-09-10T21:26:59.198 INFO:tasks.workunit.client.0.smithi017.stdout:gcc -D_GNU_SOURCE -g -O2 -D_REENTRANT -o ffsb fileops.o rand.o main.o fh.o filelist.o metaops.o rwlock.o cirlist.o rbt.o ffsb_tg.o ffsb_fs.o ffsb_thread.o ffsb_op.o util.o parser.o ffsb_fc.o ffsb_stats.o list.o -lpthread -lm 2024-09-10T21:26:59.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:26:59.193+0000 7fc6dbc00640 1 -- 172.21.15.17:0/2836916578 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fc6d4159650 con 0x7fc6d4154aa0 2024-09-10T21:26:59.397 INFO:tasks.workunit.client.0.smithi017.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/mnt.0/client.0/tmp/ffsb' 2024-09-10T21:26:59.398 INFO:tasks.workunit.client.0.smithi017.stderr:+ cd .. 2024-09-10T21:26:59.401 INFO:tasks.workunit.client.0.smithi017.stderr:+ mkdir tmp 2024-09-10T21:26:59.405 INFO:tasks.workunit.client.0.smithi017.stderr:+ cd tmp 2024-09-10T21:26:59.407 INFO:tasks.workunit.client.0.smithi017.stderr:+ for f in $mydir/*.ffsb 2024-09-10T21:26:59.407 INFO:tasks.workunit.client.0.smithi017.stderr:+ ../ffsb/ffsb /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/random_write.32.ffsb 2024-09-10T21:26:59.409 INFO:tasks.workunit.client.0.smithi017.stdout:FFSB version 6.0-RC2 started 2024-09-10T21:26:59.409 INFO:tasks.workunit.client.0.smithi017.stdout: 2024-09-10T21:26:59.415 INFO:tasks.workunit.client.0.smithi017.stdout:benchmark time = 300 2024-09-10T21:26:59.415 INFO:tasks.workunit.client.0.smithi017.stdout:ThreadGroup 0 2024-09-10T21:26:59.415 INFO:tasks.workunit.client.0.smithi017.stdout:================ 2024-09-10T21:26:59.416 INFO:tasks.workunit.client.0.smithi017.stdout: num_threads = 32 2024-09-10T21:26:59.416 INFO:tasks.workunit.client.0.smithi017.stdout: 2024-09-10T21:26:59.416 INFO:tasks.workunit.client.0.smithi017.stdout: read_random = off 2024-09-10T21:26:59.416 INFO:tasks.workunit.client.0.smithi017.stdout: read_size = 0 (0B) 2024-09-10T21:26:59.416 INFO:tasks.workunit.client.0.smithi017.stdout: read_blocksize = 0 (0B) 2024-09-10T21:26:59.416 INFO:tasks.workunit.client.0.smithi017.stdout: read_skip = off 2024-09-10T21:26:59.416 INFO:tasks.workunit.client.0.smithi017.stdout: read_skipsize = 0 (0B) 2024-09-10T21:26:59.416 INFO:tasks.workunit.client.0.smithi017.stdout: 2024-09-10T21:26:59.416 INFO:tasks.workunit.client.0.smithi017.stdout: write_random = on 2024-09-10T21:26:59.416 INFO:tasks.workunit.client.0.smithi017.stdout: write_size = 5242880 (5MB) 2024-09-10T21:26:59.416 INFO:tasks.workunit.client.0.smithi017.stdout: fsync_file = 0 2024-09-10T21:26:59.416 INFO:tasks.workunit.client.0.smithi017.stdout: write_blocksize = 4096 (4KB) 2024-09-10T21:26:59.416 INFO:tasks.workunit.client.0.smithi017.stdout: wait time = 0 2024-09-10T21:26:59.416 INFO:tasks.workunit.client.0.smithi017.stdout: 2024-09-10T21:26:59.416 INFO:tasks.workunit.client.0.smithi017.stdout: op weights 2024-09-10T21:26:59.417 INFO:tasks.workunit.client.0.smithi017.stdout: read = 0 (0.00%) 2024-09-10T21:26:59.417 INFO:tasks.workunit.client.0.smithi017.stdout: readall = 0 (0.00%) 2024-09-10T21:26:59.417 INFO:tasks.workunit.client.0.smithi017.stdout: write = 1 (100.00%) 2024-09-10T21:26:59.417 INFO:tasks.workunit.client.0.smithi017.stdout: create = 0 (0.00%) 2024-09-10T21:26:59.417 INFO:tasks.workunit.client.0.smithi017.stdout: append = 0 (0.00%) 2024-09-10T21:26:59.417 INFO:tasks.workunit.client.0.smithi017.stdout: delete = 0 (0.00%) 2024-09-10T21:26:59.417 INFO:tasks.workunit.client.0.smithi017.stdout: metaop = 0 (0.00%) 2024-09-10T21:26:59.417 INFO:tasks.workunit.client.0.smithi017.stdout: createdir = 0 (0.00%) 2024-09-10T21:26:59.417 INFO:tasks.workunit.client.0.smithi017.stdout: stat = 0 (0.00%) 2024-09-10T21:26:59.417 INFO:tasks.workunit.client.0.smithi017.stdout: writeall = 0 (0.00%) 2024-09-10T21:26:59.417 INFO:tasks.workunit.client.0.smithi017.stdout: writeall_fsync = 0 (0.00%) 2024-09-10T21:26:59.417 INFO:tasks.workunit.client.0.smithi017.stdout: open_close = 0 (0.00%) 2024-09-10T21:26:59.417 INFO:tasks.workunit.client.0.smithi017.stdout: write_fsync = 0 (0.00%) 2024-09-10T21:26:59.417 INFO:tasks.workunit.client.0.smithi017.stdout: create_fsync = 0 (0.00%) 2024-09-10T21:26:59.418 INFO:tasks.workunit.client.0.smithi017.stdout: append_fsync = 0 (0.00%) 2024-09-10T21:26:59.418 INFO:tasks.workunit.client.0.smithi017.stdout: 2024-09-10T21:26:59.418 INFO:tasks.workunit.client.0.smithi017.stdout:FileSystem . 2024-09-10T21:26:59.418 INFO:tasks.workunit.client.0.smithi017.stdout:========== 2024-09-10T21:26:59.418 INFO:tasks.workunit.client.0.smithi017.stdout: num_dirs = 0 2024-09-10T21:26:59.418 INFO:tasks.workunit.client.0.smithi017.stdout: starting files = 128 2024-09-10T21:26:59.418 INFO:tasks.workunit.client.0.smithi017.stdout: 2024-09-10T21:26:59.418 INFO:tasks.workunit.client.0.smithi017.stdout: min file size = 104857600 (100MB) 2024-09-10T21:26:59.418 INFO:tasks.workunit.client.0.smithi017.stdout: max file size = 104857600 (100MB) 2024-09-10T21:26:59.418 INFO:tasks.workunit.client.0.smithi017.stdout: directio = off 2024-09-10T21:26:59.418 INFO:tasks.workunit.client.0.smithi017.stdout: alignedio = on 2024-09-10T21:26:59.418 INFO:tasks.workunit.client.0.smithi017.stdout: bufferedio = off 2024-09-10T21:26:59.418 INFO:tasks.workunit.client.0.smithi017.stdout: 2024-09-10T21:26:59.418 INFO:tasks.workunit.client.0.smithi017.stdout: aging is off 2024-09-10T21:27:04.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:04.109+0000 7fc6d1600640 1 -- 172.21.15.17:0/2836916578 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1550 (secure 0 0 0) 0x7fc6cc065100 con 0x7fc6d4154aa0 2024-09-10T21:27:04.113 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":18,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5e2f023f-d6a6-49f4-a256-f4807dce3739","modified":"2024-09-10T21:23:46.892130Z","created":"2024-09-10T21:23:46.892130Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3301","nonce":0},{"type":"v1","addr":"172.21.15.31:6790","nonce":0}]},"addr":"172.21.15.31:6790/0","public_addr":"172.21.15.31:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-10T21:27:04.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:04.109+0000 7fc6dbc00640 1 -- 172.21.15.17:0/2836916578 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fc6b00725e0 msgr2=0x7fc6b0074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:04.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:04.109+0000 7fc6dbc00640 1 --2- 172.21.15.17:0/2836916578 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fc6b00725e0 0x7fc6b0074aa0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fc6c40058d0 tx=0x7fc6c400b480 comp rx=0 tx=0).stop 2024-09-10T21:27:04.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:04.109+0000 7fc6dbc00640 1 -- 172.21.15.17:0/2836916578 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc6d4154aa0 msgr2=0x7fc6d4156d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:04.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:04.109+0000 7fc6dbc00640 1 --2- 172.21.15.17:0/2836916578 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc6d4154aa0 0x7fc6d4156d40 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fc6d40b41c0 tx=0x7fc6cc00cb90 comp rx=0 tx=0).stop 2024-09-10T21:27:04.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:04.109+0000 7fc6dbc00640 1 -- 172.21.15.17:0/2836916578 shutdown_connections 2024-09-10T21:27:04.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:04.109+0000 7fc6dbc00640 1 --2- 172.21.15.17:0/2836916578 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fc6b00725e0 0x7fc6b0074aa0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:04.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:04.109+0000 7fc6dbc00640 1 --2- 172.21.15.17:0/2836916578 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc6d4154aa0 0x7fc6d4156d40 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:04.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:04.109+0000 7fc6dbc00640 1 --2- 172.21.15.17:0/2836916578 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc6d41546c0 0x7fc6d4154ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:04.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:04.109+0000 7fc6dbc00640 1 --2- 172.21.15.17:0/2836916578 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc6d40725c0 0x7fc6d4157ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:04.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:04.109+0000 7fc6dbc00640 1 -- 172.21.15.17:0/2836916578 >> 172.21.15.17:0/2836916578 conn(0x7fc6d40702f0 msgr2=0x7fc6d4159d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:27:04.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:04.109+0000 7fc6dbc00640 1 -- 172.21.15.17:0/2836916578 shutdown_connections 2024-09-10T21:27:04.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:04.113+0000 7fc6dbc00640 1 -- 172.21.15.17:0/2836916578 wait complete. 2024-09-10T21:27:04.118 INFO:tasks.workunit.client.0.smithi017.stdout: current utilization = 0.04% 2024-09-10T21:27:04.118 INFO:tasks.workunit.client.0.smithi017.stdout: 2024-09-10T21:27:04.118 INFO:tasks.workunit.client.0.smithi017.stdout:checking existing fs: . 2024-09-10T21:27:04.119 INFO:tasks.workunit.client.0.smithi017.stdout:recreating new fileset 2024-09-10T21:27:04.119 INFO:tasks.workunit.client.0.smithi017.stderr:opendir: No such file or directory 2024-09-10T21:27:07.144 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-10T21:27:07.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:07.305+0000 7f12eec00640 1 -- 172.21.15.17:0/3533854340 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f12e806f480 msgr2=0x7f12e80c7870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:07.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:07.305+0000 7f12eec00640 1 --2- 172.21.15.17:0/3533854340 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f12e806f480 0x7f12e80c7870 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f12dc009900 tx=0x7f12dc01c8e0 comp rx=0 tx=0).stop 2024-09-10T21:27:07.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:07.305+0000 7f12eec00640 1 -- 172.21.15.17:0/3533854340 shutdown_connections 2024-09-10T21:27:07.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:07.305+0000 7f12eec00640 1 --2- 172.21.15.17:0/3533854340 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f12e806f480 0x7f12e80c7870 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:07.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:07.305+0000 7f12eec00640 1 -- 172.21.15.17:0/3533854340 >> 172.21.15.17:0/3533854340 conn(0x7f12e8070ca0 msgr2=0x7f12e80710b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:27:07.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:07.305+0000 7f12eec00640 1 -- 172.21.15.17:0/3533854340 shutdown_connections 2024-09-10T21:27:07.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:07.305+0000 7f12eec00640 1 -- 172.21.15.17:0/3533854340 wait complete. 2024-09-10T21:27:07.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:07.305+0000 7f12eec00640 1 Processor -- start 2024-09-10T21:27:07.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:07.305+0000 7f12eec00640 1 -- start start 2024-09-10T21:27:07.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:07.305+0000 7f12eec00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f12e806f480 0x7f12e8155230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:07.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:07.305+0000 7f12eec00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f12e806f860 0x7f12e814a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:07.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:07.305+0000 7f12eec00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f12e80c8070 0x7f12e81578e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:07.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:07.305+0000 7f12eec00640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f12e80702a0 con 0x7f12e80c8070 2024-09-10T21:27:07.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:07.305+0000 7f12eec00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f12e8151b40 con 0x7f12e806f860 2024-09-10T21:27:07.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:07.305+0000 7f12eec00640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f12e8165500 con 0x7f12e806f480 2024-09-10T21:27:07.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:07.305+0000 7f12ee200640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f12e80c8070 0x7f12e81578e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:07.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:07.305+0000 7f12ee200640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f12e80c8070 0x7f12e81578e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44520/0 (socket says 172.21.15.17:44520) 2024-09-10T21:27:07.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:07.305+0000 7f12ee200640 1 -- 172.21.15.17:0/1309886163 learned_addr learned my addr 172.21.15.17:0/1309886163 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:27:07.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:07.305+0000 7f12ece00640 1 --2- 172.21.15.17:0/1309886163 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f12e806f860 0x7f12e814a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:07.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:07.305+0000 7f12ed800640 1 --2- 172.21.15.17:0/1309886163 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f12e806f480 0x7f12e8155230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:07.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:07.305+0000 7f12ed800640 1 -- 172.21.15.17:0/1309886163 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f12e806f480 msgr2=0x7f12e8155230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-09-10T21:27:07.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:07.305+0000 7f12ed800640 1 -- 172.21.15.17:0/1309886163 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f12e806f480 msgr2=0x7f12e8155230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-10T21:27:07.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:07.305+0000 7f12ed800640 1 --2- 172.21.15.17:0/1309886163 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f12e806f480 0x7f12e8155230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T21:27:07.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:07.305+0000 7f12ed800640 1 --2- 172.21.15.17:0/1309886163 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f12e806f480 0x7f12e8155230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:27:07.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:07.305+0000 7f12ee200640 1 -- 172.21.15.17:0/1309886163 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f12e806f480 msgr2=0x7f12e8155230 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:27:07.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:07.305+0000 7f12ee200640 1 --2- 172.21.15.17:0/1309886163 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f12e806f480 0x7f12e8155230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:07.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:07.305+0000 7f12ee200640 1 -- 172.21.15.17:0/1309886163 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f12e806f860 msgr2=0x7f12e814a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:07.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:07.305+0000 7f12ee200640 1 --2- 172.21.15.17:0/1309886163 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f12e806f860 0x7f12e814a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:07.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:07.305+0000 7f12ee200640 1 -- 172.21.15.17:0/1309886163 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12e80c4420 con 0x7f12e80c8070 2024-09-10T21:27:07.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:07.309+0000 7f12ee200640 1 --2- 172.21.15.17:0/1309886163 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f12e80c8070 0x7f12e81578e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f12e000c6e0 tx=0x7f12e000cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:07.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:07.309+0000 7f12e5600640 1 -- 172.21.15.17:0/1309886163 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12e0016020 con 0x7f12e80c8070 2024-09-10T21:27:07.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:07.309+0000 7f12e5600640 1 -- 172.21.15.17:0/1309886163 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f12e00039d0 con 0x7f12e80c8070 2024-09-10T21:27:07.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:07.309+0000 7f12e5600640 1 -- 172.21.15.17:0/1309886163 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12e0003e20 con 0x7f12e80c8070 2024-09-10T21:27:07.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:07.309+0000 7f12eec00640 1 -- 172.21.15.17:0/1309886163 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12e81f1610 con 0x7f12e80c8070 2024-09-10T21:27:07.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:07.309+0000 7f12eec00640 1 -- 172.21.15.17:0/1309886163 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12e8149c00 con 0x7f12e80c8070 2024-09-10T21:27:07.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:07.309+0000 7f12eec00640 1 -- 172.21.15.17:0/1309886163 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12e80c7870 con 0x7f12e80c8070 2024-09-10T21:27:07.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:07.309+0000 7f12e5600640 1 -- 172.21.15.17:0/1309886163 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f12e00050d0 con 0x7f12e80c8070 2024-09-10T21:27:07.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:07.313+0000 7f12e5600640 1 --2- 172.21.15.17:0/1309886163 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f12c80725e0 0x7f12c8074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:07.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:07.313+0000 7f12e5600640 1 -- 172.21.15.17:0/1309886163 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f12e0093180 con 0x7f12e80c8070 2024-09-10T21:27:07.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:07.313+0000 7f12ed800640 1 --2- 172.21.15.17:0/1309886163 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f12c80725e0 0x7f12c8074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:07.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:07.313+0000 7f12ed800640 1 --2- 172.21.15.17:0/1309886163 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f12c80725e0 0x7f12c8074aa0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f12dc0af000 tx=0x7f12dc01b120 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:07.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:07.317+0000 7f12e5600640 1 -- 172.21.15.17:0/1309886163 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f12e0020020 con 0x7f12e80c8070 2024-09-10T21:27:07.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:07.629+0000 7f12eec00640 1 -- 172.21.15.17:0/1309886163 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f12e8071ce0 con 0x7f12e80c8070 2024-09-10T21:27:07.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:07.629+0000 7f12e5600640 1 -- 172.21.15.17:0/1309886163 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1556 (secure 0 0 0) 0x7f12e005e4f0 con 0x7f12e80c8070 2024-09-10T21:27:07.635 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":20,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":2,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5e2f023f-d6a6-49f4-a256-f4807dce3739","modified":"2024-09-10T21:23:46.892130Z","created":"2024-09-10T21:23:46.892130Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3301","nonce":0},{"type":"v1","addr":"172.21.15.31:6790","nonce":0}]},"addr":"172.21.15.31:6790/0","public_addr":"172.21.15.31:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-10T21:27:07.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:07.633+0000 7f12eec00640 1 -- 172.21.15.17:0/1309886163 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f12c80725e0 msgr2=0x7f12c8074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:07.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:07.633+0000 7f12eec00640 1 --2- 172.21.15.17:0/1309886163 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f12c80725e0 0x7f12c8074aa0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f12dc0af000 tx=0x7f12dc01b120 comp rx=0 tx=0).stop 2024-09-10T21:27:07.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:07.633+0000 7f12eec00640 1 -- 172.21.15.17:0/1309886163 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f12e80c8070 msgr2=0x7f12e81578e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:07.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:07.633+0000 7f12eec00640 1 --2- 172.21.15.17:0/1309886163 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f12e80c8070 0x7f12e81578e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f12e000c6e0 tx=0x7f12e000cba0 comp rx=0 tx=0).stop 2024-09-10T21:27:07.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:07.633+0000 7f12eec00640 1 -- 172.21.15.17:0/1309886163 shutdown_connections 2024-09-10T21:27:07.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:07.633+0000 7f12eec00640 1 --2- 172.21.15.17:0/1309886163 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f12c80725e0 0x7f12c8074aa0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:07.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:07.633+0000 7f12eec00640 1 --2- 172.21.15.17:0/1309886163 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f12e80c8070 0x7f12e81578e0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:07.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:07.633+0000 7f12eec00640 1 --2- 172.21.15.17:0/1309886163 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f12e806f860 0x7f12e814a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:07.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:07.633+0000 7f12eec00640 1 --2- 172.21.15.17:0/1309886163 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f12e806f480 0x7f12e8155230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:07.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:07.633+0000 7f12eec00640 1 -- 172.21.15.17:0/1309886163 >> 172.21.15.17:0/1309886163 conn(0x7f12e8070ca0 msgr2=0x7f12e80cb6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:27:07.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:07.633+0000 7f12eec00640 1 -- 172.21.15.17:0/1309886163 shutdown_connections 2024-09-10T21:27:07.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:07.633+0000 7f12eec00640 1 -- 172.21.15.17:0/1309886163 wait complete. 2024-09-10T21:27:07.657 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-09-10T21:27:07.657 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-09-10T21:27:07.658 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-10T21:27:07.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:07.809+0000 7fb280a00640 1 -- 172.21.15.17:0/3569832478 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb2600039f0 msgr2=0x7fb260003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:07.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:07.809+0000 7fb280a00640 1 --2- 172.21.15.17:0/3569832478 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb2600039f0 0x7fb260003dd0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fb26c00c290 tx=0x7fb26c01cf70 comp rx=0 tx=0).stop 2024-09-10T21:27:07.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:07.809+0000 7fb27a800640 1 -- 172.21.15.17:0/3569832478 reap_dead start 2024-09-10T21:27:07.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:07.809+0000 7fb280a00640 1 -- 172.21.15.17:0/3569832478 shutdown_connections 2024-09-10T21:27:07.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:07.809+0000 7fb280a00640 1 -- 172.21.15.17:0/3569832478 >> 172.21.15.17:0/3569832478 conn(0x7fb27c0702f0 msgr2=0x7fb27c070700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:27:07.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:07.809+0000 7fb280a00640 1 -- 172.21.15.17:0/3569832478 shutdown_connections 2024-09-10T21:27:07.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:07.809+0000 7fb280a00640 1 -- 172.21.15.17:0/3569832478 wait complete. 2024-09-10T21:27:07.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:07.813+0000 7fb280a00640 1 Processor -- start 2024-09-10T21:27:07.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:07.813+0000 7fb280a00640 1 -- start start 2024-09-10T21:27:07.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:07.813+0000 7fb280a00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fb2600039f0 0x7fb27c155480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:07.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:07.813+0000 7fb280a00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb27c0b3720 0x7fb27c0c66f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:07.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:07.813+0000 7fb280a00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb27c15b350 0x7fb27c0c6c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:07.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:07.813+0000 7fb280a00640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fb27c0ce630 con 0x7fb27c15b350 2024-09-10T21:27:07.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:07.813+0000 7fb280a00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fb27c0b42c0 con 0x7fb27c0b3720 2024-09-10T21:27:07.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:07.813+0000 7fb280a00640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7fb27c0ce7b0 con 0x7fb2600039f0 2024-09-10T21:27:07.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:07.813+0000 7fb27a800640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb27c15b350 0x7fb27c0c6c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:07.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:07.813+0000 7fb27a800640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb27c15b350 0x7fb27c0c6c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44542/0 (socket says 172.21.15.17:44542) 2024-09-10T21:27:07.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:07.813+0000 7fb27a800640 1 -- 172.21.15.17:0/3504261046 learned_addr learned my addr 172.21.15.17:0/3504261046 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:27:07.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:07.813+0000 7fb279400640 1 --2- 172.21.15.17:0/3504261046 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb27c0b3720 0x7fb27c0c66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:07.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:07.813+0000 7fb279e00640 1 --2- 172.21.15.17:0/3504261046 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fb2600039f0 0x7fb27c155480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:07.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:07.813+0000 7fb27a800640 1 -- 172.21.15.17:0/3504261046 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fb2600039f0 msgr2=0x7fb27c155480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:07.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:07.813+0000 7fb27a800640 1 --2- 172.21.15.17:0/3504261046 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fb2600039f0 0x7fb27c155480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:07.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:07.813+0000 7fb27a800640 1 -- 172.21.15.17:0/3504261046 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb27c0b3720 msgr2=0x7fb27c0c66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:07.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:07.813+0000 7fb27a800640 1 --2- 172.21.15.17:0/3504261046 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb27c0b3720 0x7fb27c0c66f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:07.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:07.813+0000 7fb27a800640 1 -- 172.21.15.17:0/3504261046 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb27c1559c0 con 0x7fb27c15b350 2024-09-10T21:27:07.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:07.813+0000 7fb27a800640 1 --2- 172.21.15.17:0/3504261046 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb27c15b350 0x7fb27c0c6c30 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb274010a70 tx=0x7fb274010f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:07.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:07.813+0000 7fb272000640 1 -- 172.21.15.17:0/3504261046 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb27400eea0 con 0x7fb27c15b350 2024-09-10T21:27:07.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:07.813+0000 7fb272000640 1 -- 172.21.15.17:0/3504261046 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb274004b30 con 0x7fb27c15b350 2024-09-10T21:27:07.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:07.813+0000 7fb272000640 1 -- 172.21.15.17:0/3504261046 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb274005020 con 0x7fb27c15b350 2024-09-10T21:27:07.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:07.813+0000 7fb280a00640 1 -- 172.21.15.17:0/3504261046 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb27c1f3c40 con 0x7fb27c15b350 2024-09-10T21:27:07.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:07.813+0000 7fb280a00640 1 -- 172.21.15.17:0/3504261046 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb27c155fa0 con 0x7fb27c15b350 2024-09-10T21:27:07.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:07.817+0000 7fb280a00640 1 --2- 172.21.15.17:0/3504261046 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb27c1e93f0 0x7fb27c1e97d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:07.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:07.817+0000 7fb280a00640 1 -- 172.21.15.17:0/3504261046 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fb27c1e9210 con 0x7fb27c1e93f0 2024-09-10T21:27:07.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:07.817+0000 7fb279e00640 1 --2- 172.21.15.17:0/3504261046 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb27c1e93f0 0x7fb27c1e97d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:07.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:07.817+0000 7fb279e00640 1 -- 172.21.15.17:0/3504261046 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb27c0c2ff0 con 0x7fb27c1e93f0 2024-09-10T21:27:07.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:07.817+0000 7fb279e00640 1 --2- 172.21.15.17:0/3504261046 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb27c1e93f0 0x7fb27c1e97d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb26c00c260 tx=0x7fb26c001090 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:07.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:07.817+0000 7fb272000640 1 -- 172.21.15.17:0/3504261046 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb26c0bf070 con 0x7fb27c1e93f0 2024-09-10T21:27:07.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:07.817+0000 7fb272000640 1 -- 172.21.15.17:0/3504261046 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fb27400f040 con 0x7fb27c15b350 2024-09-10T21:27:07.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:07.817+0000 7fb272000640 1 --2- 172.21.15.17:0/3504261046 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fb25c072510 0x7fb25c0749d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:07.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:07.817+0000 7fb272000640 1 -- 172.21.15.17:0/3504261046 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fb274093e80 con 0x7fb27c15b350 2024-09-10T21:27:07.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:07.817+0000 7fb272000640 1 -- 172.21.15.17:0/3504261046 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fb26c006050 con 0x7fb27c1e93f0 2024-09-10T21:27:07.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:07.817+0000 7fb279400640 1 --2- 172.21.15.17:0/3504261046 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fb25c072510 0x7fb25c0749d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:07.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:07.817+0000 7fb272000640 1 -- 172.21.15.17:0/3504261046 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb27c1e93f0 msgr2=0x7fb27c1e97d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:07.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:07.817+0000 7fb272000640 1 --2- 172.21.15.17:0/3504261046 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb27c1e93f0 0x7fb27c1e97d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb26c00c260 tx=0x7fb26c001090 comp rx=0 tx=0).stop 2024-09-10T21:27:07.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:07.817+0000 7fb272000640 1 -- 172.21.15.17:0/3504261046 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb27c1e93f0 msgr2=0x7fb27c1e97d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:27:07.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:07.817+0000 7fb272000640 1 --2- 172.21.15.17:0/3504261046 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb27c1e93f0 0x7fb27c1e97d0 secure :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb26c00c260 tx=0x7fb26c001090 comp rx=0 tx=0).stop 2024-09-10T21:27:07.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:07.817+0000 7fb279400640 1 --2- 172.21.15.17:0/3504261046 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fb25c072510 0x7fb25c0749d0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fb27c1ed900 tx=0x7fb268006b20 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:07.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:07.825+0000 7fb280a00640 1 --2- 172.21.15.17:0/3504261046 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb27c156170 0x7fb27c1f6630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:07.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:07.825+0000 7fb280a00640 1 -- 172.21.15.17:0/3504261046 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fb27c0cc0e0 con 0x7fb27c156170 2024-09-10T21:27:07.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:07.825+0000 7fb279e00640 1 --2- 172.21.15.17:0/3504261046 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb27c156170 0x7fb27c1f6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:07.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:07.825+0000 7fb279e00640 1 -- 172.21.15.17:0/3504261046 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fb27c0708d0 con 0x7fb27c156170 2024-09-10T21:27:07.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:07.825+0000 7fb279e00640 1 --2- 172.21.15.17:0/3504261046 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb27c156170 0x7fb27c1f6630 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb26c01d810 tx=0x7fb26c0067b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:07.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:07.825+0000 7fb272000640 1 -- 172.21.15.17:0/3504261046 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb26c0c2990 con 0x7fb27c156170 2024-09-10T21:27:07.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:07.825+0000 7fb272000640 1 -- 172.21.15.17:0/3504261046 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4876 (secure 0 0 0) 0x7fb26c0c2990 con 0x7fb27c156170 2024-09-10T21:27:07.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:07.825+0000 7fb272000640 1 -- 172.21.15.17:0/3504261046 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb27c156170 msgr2=0x7fb27c1f6630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:07.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:07.825+0000 7fb272000640 1 --2- 172.21.15.17:0/3504261046 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb27c156170 0x7fb27c1f6630 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb26c01d810 tx=0x7fb26c0067b0 comp rx=0 tx=0).stop 2024-09-10T21:27:07.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:07.825+0000 7fb272000640 1 -- 172.21.15.17:0/3504261046 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb27c156170 msgr2=0x7fb27c1f6630 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:27:07.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:07.825+0000 7fb272000640 1 --2- 172.21.15.17:0/3504261046 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb27c156170 0x7fb27c1f6630 secure :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb26c01d810 tx=0x7fb26c0067b0 comp rx=0 tx=0).stop 2024-09-10T21:27:07.832 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-10T21:27:07.833 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:27:07.833 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:27:07.833 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "leader", 2024-09-10T21:27:07.833 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 20, 2024-09-10T21:27:07.833 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 57731, 2024-09-10T21:27:07.833 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-10T21:27:07.833 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-10T21:27:07.833 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-10T21:27:07.833 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-10T21:27:07.833 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:27:07.833 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 2, 2024-09-10T21:27:07.833 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:27:07.834 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-10T21:27:07.834 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-10T21:27:07.834 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:27:07.834 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:27:07.834 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:27:07.834 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:27:07.834 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:27:07.834 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:27:07.834 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:27:07.834 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:27:07.834 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:27:07.834 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:27:07.834 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:27:07.834 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:27:07.835 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-10T21:27:07.835 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-10T21:27:07.835 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:27:07.835 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:27:07.835 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:27:07.835 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:27:07.835 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:27:07.835 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:27:07.835 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:27:07.835 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:27:07.835 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:27:07.835 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:27:07.835 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:27:07.835 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:27:07.836 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:27:07.836 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-10T21:27:07.836 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-10T21:27:07.836 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-10T21:27:07.836 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-10T21:27:07.836 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-10T21:27:07.836 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "5e2f023f-d6a6-49f4-a256-f4807dce3739", 2024-09-10T21:27:07.836 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:27:07.836 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:27:07.836 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-10T21:27:07.836 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-10T21:27:07.836 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-10T21:27:07.836 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-10T21:27:07.837 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-10T21:27:07.837 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-10T21:27:07.837 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-10T21:27:07.837 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:27:07.837 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-10T21:27:07.837 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:27:07.837 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:27:07.837 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:27:07.837 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:27:07.837 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:27:07.837 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:27:07.837 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:27:07.837 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:27:07.837 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:27:07.838 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:27:07.838 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:27:07.838 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:27:07.838 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-10T21:27:07.838 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:27:07.838 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-10T21:27:07.838 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:27:07.838 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:27:07.838 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:27:07.838 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:27:07.838 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:27:07.838 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:27:07.838 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:27:07.839 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-10T21:27:07.839 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:27:07.839 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:27:07.839 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:27:07.839 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:27:07.839 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-10T21:27:07.839 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:27:07.839 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:27:07.839 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:27:07.839 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:27:07.839 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-10T21:27:07.839 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-10T21:27:07.839 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:27:07.839 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:27:07.840 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:27:07.840 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:27:07.840 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:27:07.840 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:27:07.840 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:27:07.840 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:27:07.840 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:27:07.840 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:27:07.840 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:27:07.840 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3300", 2024-09-10T21:27:07.840 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:27:07.840 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:27:07.840 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:27:07.841 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:27:07.841 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789", 2024-09-10T21:27:07.841 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:27:07.841 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:27:07.841 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:27:07.841 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:27:07.841 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789/0", 2024-09-10T21:27:07.841 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6789/0", 2024-09-10T21:27:07.841 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:27:07.841 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:27:07.841 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:27:07.841 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:27:07.841 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:27:07.842 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:27:07.842 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:27:07.842 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:27:07.842 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:27:07.842 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:27:07.842 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:27:07.842 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3301", 2024-09-10T21:27:07.842 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:27:07.842 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:27:07.842 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:27:07.842 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:27:07.842 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790", 2024-09-10T21:27:07.842 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:27:07.842 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:27:07.843 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:27:07.843 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:27:07.843 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790/0", 2024-09-10T21:27:07.843 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6790/0", 2024-09-10T21:27:07.843 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:27:07.843 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:27:07.843 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:27:07.843 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:27:07.843 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:27:07.843 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:27:07.844 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-10T21:27:07.844 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-10T21:27:07.844 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:27:07.844 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:27:07.844 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:27:07.844 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:27:07.844 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:27:07.844 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:27:07.844 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": [ 2024-09-10T21:27:07.844 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:27:07.844 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:27:07.844 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:27:07.844 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:27:07.845 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:27:07.845 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:27:07.845 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-10T21:27:07.845 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:27:07.845 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:27:07.845 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:27:07.845 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 4 2024-09-10T21:27:07.845 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:27:07.845 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:27:07.845 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-10T21:27:07.845 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:27:07.845 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:27:07.845 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:27:07.846 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 5 2024-09-10T21:27:07.846 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:27:07.846 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:27:07.846 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:27:07.846 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-10T21:27:07.846 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-10T21:27:07.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:07.829+0000 7fb280a00640 1 -- 172.21.15.17:0/3504261046 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fb25c072510 msgr2=0x7fb25c0749d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:07.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:07.829+0000 7fb280a00640 1 --2- 172.21.15.17:0/3504261046 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fb25c072510 0x7fb25c0749d0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fb27c1ed900 tx=0x7fb268006b20 comp rx=0 tx=0).stop 2024-09-10T21:27:07.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:07.829+0000 7fb280a00640 1 -- 172.21.15.17:0/3504261046 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb27c15b350 msgr2=0x7fb27c0c6c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:07.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:07.829+0000 7fb280a00640 1 --2- 172.21.15.17:0/3504261046 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb27c15b350 0x7fb27c0c6c30 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb274010a70 tx=0x7fb274010f30 comp rx=0 tx=0).stop 2024-09-10T21:27:07.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:07.829+0000 7fb27a800640 1 -- 172.21.15.17:0/3504261046 reap_dead start 2024-09-10T21:27:07.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:07.829+0000 7fb27a800640 1 -- 172.21.15.17:0/3504261046 reap_dead start 2024-09-10T21:27:07.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:07.829+0000 7fb280a00640 1 -- 172.21.15.17:0/3504261046 shutdown_connections 2024-09-10T21:27:07.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:07.829+0000 7fb280a00640 1 -- 172.21.15.17:0/3504261046 >> 172.21.15.17:0/3504261046 conn(0x7fb27c0702f0 msgr2=0x7fb27c1511c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:27:07.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:07.829+0000 7fb280a00640 1 -- 172.21.15.17:0/3504261046 shutdown_connections 2024-09-10T21:27:07.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:07.829+0000 7fb280a00640 1 -- 172.21.15.17:0/3504261046 wait complete. 2024-09-10T21:27:07.851 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-10T21:27:07.851 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-09-10T21:27:07.852 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-10T21:27:08.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:08.025+0000 7f743ce00640 1 -- 172.21.15.17:0/1784497386 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f741c004d50 msgr2=0x7f741c0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:08.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:08.025+0000 7f743ce00640 1 --2- 172.21.15.17:0/1784497386 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f741c004d50 0x7f741c0051d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f743000c610 tx=0x7f743001fad0 comp rx=0 tx=0).stop 2024-09-10T21:27:08.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:08.025+0000 7f7437e00640 1 -- 172.21.15.17:0/1784497386 reap_dead start 2024-09-10T21:27:08.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:08.025+0000 7f743ce00640 1 -- 172.21.15.17:0/1784497386 shutdown_connections 2024-09-10T21:27:08.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:08.025+0000 7f743ce00640 1 -- 172.21.15.17:0/1784497386 >> 172.21.15.17:0/1784497386 conn(0x7f743806cc20 msgr2=0x7f743806d030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:27:08.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:08.025+0000 7f743ce00640 1 -- 172.21.15.17:0/1784497386 shutdown_connections 2024-09-10T21:27:08.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:08.025+0000 7f743ce00640 1 -- 172.21.15.17:0/1784497386 wait complete. 2024-09-10T21:27:08.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:08.025+0000 7f743ce00640 1 Processor -- start 2024-09-10T21:27:08.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:08.025+0000 7f743ce00640 1 -- start start 2024-09-10T21:27:08.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:08.025+0000 7f743ce00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f741c004d50 0x7f74380c4120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:08.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:08.025+0000 7f743ce00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7438153140 0x7f74380cb5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:08.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:08.025+0000 7f743ce00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7438159080 0x7f7438159500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:08.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:08.025+0000 7f743ce00640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f743815a4f0 con 0x7f7438153140 2024-09-10T21:27:08.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:08.025+0000 7f743ce00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f743815ae00 con 0x7f7438159080 2024-09-10T21:27:08.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:08.025+0000 7f743ce00640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f7438154b20 con 0x7f741c004d50 2024-09-10T21:27:08.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:08.025+0000 7f7436a00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7438153140 0x7f74380cb5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:08.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:08.025+0000 7f7436a00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7438153140 0x7f74380cb5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44586/0 (socket says 172.21.15.17:44586) 2024-09-10T21:27:08.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:08.025+0000 7f7436a00640 1 -- 172.21.15.17:0/2180742418 learned_addr learned my addr 172.21.15.17:0/2180742418 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:27:08.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:08.025+0000 7f7437e00640 1 --2- 172.21.15.17:0/2180742418 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7438159080 0x7f7438159500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:08.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:08.025+0000 7f7437400640 1 --2- 172.21.15.17:0/2180742418 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f741c004d50 0x7f74380c4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:08.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:08.025+0000 7f7437400640 1 -- 172.21.15.17:0/2180742418 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f741c004d50 msgr2=0x7f74380c4120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-09-10T21:27:08.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:08.025+0000 7f7437400640 1 -- 172.21.15.17:0/2180742418 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f741c004d50 msgr2=0x7f74380c4120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-10T21:27:08.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:08.025+0000 7f7437400640 1 --2- 172.21.15.17:0/2180742418 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f741c004d50 0x7f74380c4120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T21:27:08.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:08.025+0000 7f7436a00640 1 -- 172.21.15.17:0/2180742418 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f741c004d50 msgr2=0x7f74380c4120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:08.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:08.025+0000 7f7437400640 1 --2- 172.21.15.17:0/2180742418 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f741c004d50 0x7f74380c4120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:27:08.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:08.025+0000 7f7436a00640 1 --2- 172.21.15.17:0/2180742418 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f741c004d50 0x7f74380c4120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:08.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:08.025+0000 7f7436a00640 1 -- 172.21.15.17:0/2180742418 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7438159080 msgr2=0x7f7438159500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:08.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:08.025+0000 7f7436a00640 1 --2- 172.21.15.17:0/2180742418 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7438159080 0x7f7438159500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:08.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:08.025+0000 7f7436a00640 1 -- 172.21.15.17:0/2180742418 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74380c4b60 con 0x7f7438153140 2024-09-10T21:27:08.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:08.025+0000 7f7437e00640 1 --2- 172.21.15.17:0/2180742418 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7438159080 0x7f7438159500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:27:08.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:08.029+0000 7f7436a00640 1 --2- 172.21.15.17:0/2180742418 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7438153140 0x7f74380cb5a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f7424003f80 tx=0x7f742400ba60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:08.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:08.029+0000 7f742f400640 1 -- 172.21.15.17:0/2180742418 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7424010a20 con 0x7f7438153140 2024-09-10T21:27:08.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:08.029+0000 7f742f400640 1 -- 172.21.15.17:0/2180742418 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7424015070 con 0x7f7438153140 2024-09-10T21:27:08.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:08.029+0000 7f742f400640 1 -- 172.21.15.17:0/2180742418 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7424010e40 con 0x7f7438153140 2024-09-10T21:27:08.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:08.029+0000 7f743ce00640 1 -- 172.21.15.17:0/2180742418 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7438152970 con 0x7f7438153140 2024-09-10T21:27:08.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:08.029+0000 7f743ce00640 1 -- 172.21.15.17:0/2180742418 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74381f37d0 con 0x7f7438153140 2024-09-10T21:27:08.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:08.029+0000 7f743ce00640 1 --2- 172.21.15.17:0/2180742418 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74381f39d0 0x7f74381f3db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:08.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:08.029+0000 7f743ce00640 1 -- 172.21.15.17:0/2180742418 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f74381f4590 con 0x7f74381f39d0 2024-09-10T21:27:08.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:08.029+0000 7f7437400640 1 --2- 172.21.15.17:0/2180742418 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74381f39d0 0x7f74381f3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:08.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:08.029+0000 7f7437400640 1 -- 172.21.15.17:0/2180742418 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f74380c6d60 con 0x7f74381f39d0 2024-09-10T21:27:08.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:08.029+0000 7f7437400640 1 --2- 172.21.15.17:0/2180742418 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74381f39d0 0x7f74381f3db0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f74280129f0 tx=0x7f7428012eb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:08.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:08.029+0000 7f742f400640 1 -- 172.21.15.17:0/2180742418 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f742800ea40 con 0x7f74381f39d0 2024-09-10T21:27:08.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:08.029+0000 7f742f400640 1 -- 172.21.15.17:0/2180742418 <== mon.1 v2:172.21.15.31:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f742800ebe0 con 0x7f74381f39d0 2024-09-10T21:27:08.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:08.029+0000 7f742f400640 1 -- 172.21.15.17:0/2180742418 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74381f39d0 msgr2=0x7f74381f3db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:08.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:08.029+0000 7f742f400640 1 --2- 172.21.15.17:0/2180742418 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74381f39d0 0x7f74381f3db0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f74280129f0 tx=0x7f7428012eb0 comp rx=0 tx=0).stop 2024-09-10T21:27:08.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:08.029+0000 7f742f400640 1 -- 172.21.15.17:0/2180742418 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74381f39d0 msgr2=0x7f74381f3db0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:27:08.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:08.029+0000 7f742f400640 1 --2- 172.21.15.17:0/2180742418 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74381f39d0 0x7f74381f3db0 secure :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f74280129f0 tx=0x7f7428012eb0 comp rx=0 tx=0).stop 2024-09-10T21:27:08.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:08.029+0000 7f742f400640 1 -- 172.21.15.17:0/2180742418 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f742401a470 con 0x7f7438153140 2024-09-10T21:27:08.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:08.033+0000 7f742f400640 1 --2- 172.21.15.17:0/2180742418 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f74180720c0 0x7f7418074580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:08.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:08.033+0000 7f742f400640 1 -- 172.21.15.17:0/2180742418 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f7424094180 con 0x7f7438153140 2024-09-10T21:27:08.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:08.033+0000 7f7437400640 1 --2- 172.21.15.17:0/2180742418 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f74180720c0 0x7f7418074580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:08.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:08.033+0000 7f7437400640 1 --2- 172.21.15.17:0/2180742418 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f74180720c0 0x7f7418074580 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f74280129f0 tx=0x7f7428011000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:08.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:08.037+0000 7f743ce00640 1 --2- 172.21.15.17:0/2180742418 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74381450e0 0x7f74381454c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:08.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:08.037+0000 7f743ce00640 1 -- 172.21.15.17:0/2180742418 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f7438145a00 con 0x7f74381450e0 2024-09-10T21:27:08.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:08.037+0000 7f7437e00640 1 --2- 172.21.15.17:0/2180742418 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74381450e0 0x7f74381454c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:08.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:08.037+0000 7f7437e00640 1 -- 172.21.15.17:0/2180742418 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f74381e5880 con 0x7f74381450e0 2024-09-10T21:27:08.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:08.037+0000 7f7437e00640 1 --2- 172.21.15.17:0/2180742418 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74381450e0 0x7f74381454c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f7438073060 tx=0x7f74300011d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:08.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:08.037+0000 7f742f400640 1 -- 172.21.15.17:0/2180742418 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7430001840 con 0x7f74381450e0 2024-09-10T21:27:08.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:08.037+0000 7f742f400640 1 -- 172.21.15.17:0/2180742418 <== mon.1 v2:172.21.15.31:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5042 (secure 0 0 0) 0x7f7430001840 con 0x7f74381450e0 2024-09-10T21:27:08.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:08.037+0000 7f742f400640 1 -- 172.21.15.17:0/2180742418 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74381450e0 msgr2=0x7f74381454c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:08.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:08.037+0000 7f742f400640 1 --2- 172.21.15.17:0/2180742418 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74381450e0 0x7f74381454c0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f7438073060 tx=0x7f74300011d0 comp rx=0 tx=0).stop 2024-09-10T21:27:08.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:08.037+0000 7f742f400640 1 -- 172.21.15.17:0/2180742418 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74381450e0 msgr2=0x7f74381454c0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:27:08.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:08.037+0000 7f742f400640 1 --2- 172.21.15.17:0/2180742418 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f74381450e0 0x7f74381454c0 secure :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f7438073060 tx=0x7f74300011d0 comp rx=0 tx=0).stop 2024-09-10T21:27:08.044 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-10T21:27:08.044 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:27:08.044 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:27:08.045 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-10T21:27:08.045 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 20, 2024-09-10T21:27:08.045 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 166487, 2024-09-10T21:27:08.045 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-10T21:27:08.045 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-10T21:27:08.045 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-10T21:27:08.045 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-10T21:27:08.045 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:27:08.045 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 2, 2024-09-10T21:27:08.045 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:27:08.045 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-10T21:27:08.045 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-10T21:27:08.045 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:27:08.046 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:27:08.046 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:27:08.046 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:27:08.046 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:27:08.046 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:27:08.046 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:27:08.046 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:27:08.046 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:27:08.046 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:27:08.046 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:27:08.046 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:27:08.046 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-10T21:27:08.046 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-10T21:27:08.046 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:27:08.046 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:27:08.047 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:27:08.047 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:27:08.047 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:27:08.047 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:27:08.047 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:27:08.047 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:27:08.047 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:27:08.047 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:27:08.047 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:27:08.047 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:27:08.047 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:27:08.047 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-10T21:27:08.047 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-10T21:27:08.047 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-10T21:27:08.047 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-10T21:27:08.048 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-10T21:27:08.048 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "5e2f023f-d6a6-49f4-a256-f4807dce3739", 2024-09-10T21:27:08.048 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:27:08.048 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:27:08.048 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-10T21:27:08.048 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-10T21:27:08.048 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-10T21:27:08.048 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-10T21:27:08.048 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-10T21:27:08.048 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-10T21:27:08.048 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-10T21:27:08.048 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:27:08.048 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-10T21:27:08.049 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:27:08.049 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:27:08.049 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:27:08.049 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:27:08.049 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:27:08.049 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:27:08.049 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:27:08.049 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:27:08.049 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:27:08.049 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:27:08.049 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:27:08.049 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:27:08.049 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-10T21:27:08.050 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:27:08.050 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-10T21:27:08.050 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:27:08.050 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:27:08.050 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:27:08.050 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:27:08.050 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:27:08.050 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:27:08.050 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:27:08.051 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-10T21:27:08.051 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:27:08.051 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:27:08.052 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:27:08.052 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:27:08.052 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-10T21:27:08.052 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:27:08.052 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:27:08.052 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:27:08.052 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:27:08.052 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-10T21:27:08.052 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-10T21:27:08.053 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:27:08.053 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:27:08.053 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:27:08.053 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:27:08.053 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:27:08.053 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:27:08.053 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:27:08.053 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:27:08.053 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:27:08.053 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:27:08.053 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:27:08.053 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3300", 2024-09-10T21:27:08.053 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:27:08.054 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:27:08.054 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:27:08.054 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:27:08.054 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789", 2024-09-10T21:27:08.054 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:27:08.054 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:27:08.054 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:27:08.054 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:27:08.054 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789/0", 2024-09-10T21:27:08.054 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6789/0", 2024-09-10T21:27:08.054 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:27:08.054 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:27:08.054 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:27:08.054 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:27:08.055 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:27:08.055 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:27:08.055 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:27:08.055 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:27:08.055 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:27:08.055 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:27:08.055 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:27:08.055 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3301", 2024-09-10T21:27:08.055 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:27:08.055 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:27:08.055 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:27:08.055 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:27:08.055 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790", 2024-09-10T21:27:08.056 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:27:08.056 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:27:08.056 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:27:08.056 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:27:08.056 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790/0", 2024-09-10T21:27:08.056 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6790/0", 2024-09-10T21:27:08.056 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:27:08.056 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:27:08.056 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:27:08.056 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:27:08.056 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:27:08.056 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:27:08.057 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-10T21:27:08.057 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-10T21:27:08.057 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:27:08.057 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:27:08.057 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:27:08.057 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:27:08.057 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:27:08.057 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:27:08.057 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": [ 2024-09-10T21:27:08.057 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:27:08.057 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:27:08.057 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:27:08.057 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 5 2024-09-10T21:27:08.058 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:27:08.058 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:27:08.058 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-10T21:27:08.058 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:27:08.058 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:27:08.058 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:27:08.058 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 4 2024-09-10T21:27:08.058 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:27:08.058 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:27:08.058 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-10T21:27:08.058 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:27:08.058 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:27:08.058 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:27:08.058 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 3 2024-09-10T21:27:08.059 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:27:08.059 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:27:08.059 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": [ 2024-09-10T21:27:08.059 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:27:08.059 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:27:08.059 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:27:08.059 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 2 2024-09-10T21:27:08.059 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:27:08.059 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:27:08.059 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:27:08.059 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-10T21:27:08.059 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-10T21:27:08.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:08.041+0000 7f743ce00640 1 -- 172.21.15.17:0/2180742418 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f74180720c0 msgr2=0x7f7418074580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:08.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:08.041+0000 7f743ce00640 1 --2- 172.21.15.17:0/2180742418 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f74180720c0 0x7f7418074580 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f74280129f0 tx=0x7f7428011000 comp rx=0 tx=0).stop 2024-09-10T21:27:08.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:08.041+0000 7f7437e00640 1 -- 172.21.15.17:0/2180742418 reap_dead start 2024-09-10T21:27:08.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:08.041+0000 7f743ce00640 1 -- 172.21.15.17:0/2180742418 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7438153140 msgr2=0x7f74380cb5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:08.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:08.041+0000 7f743ce00640 1 --2- 172.21.15.17:0/2180742418 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7438153140 0x7f74380cb5a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f7424003f80 tx=0x7f742400ba60 comp rx=0 tx=0).stop 2024-09-10T21:27:08.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:08.041+0000 7f743ce00640 1 -- 172.21.15.17:0/2180742418 shutdown_connections 2024-09-10T21:27:08.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:08.041+0000 7f743ce00640 1 --2- 172.21.15.17:0/2180742418 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7438153140 0x7f74380cb5a0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:08.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:08.041+0000 7f743ce00640 1 -- 172.21.15.17:0/2180742418 >> 172.21.15.17:0/2180742418 conn(0x7f743806cc20 msgr2=0x7f743814ea80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:27:08.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:08.041+0000 7f743ce00640 1 -- 172.21.15.17:0/2180742418 shutdown_connections 2024-09-10T21:27:08.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:08.041+0000 7f743ce00640 1 -- 172.21.15.17:0/2180742418 wait complete. 2024-09-10T21:27:08.063 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-10T21:27:08.063 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-09-10T21:27:08.063 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-10T21:27:08.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:08.221+0000 7f3947e00640 1 -- 172.21.15.17:0/1595440363 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f392c0037b0 msgr2=0x7f392c003c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:08.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:08.221+0000 7f3947e00640 1 --2- 172.21.15.17:0/1595440363 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f392c0037b0 0x7f392c003c30 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f3934006f00 tx=0x7f393401ce80 comp rx=0 tx=0).stop 2024-09-10T21:27:08.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:08.221+0000 7f3947400640 1 -- 172.21.15.17:0/1595440363 reap_dead start 2024-09-10T21:27:08.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:08.221+0000 7f3947e00640 1 -- 172.21.15.17:0/1595440363 shutdown_connections 2024-09-10T21:27:08.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:08.221+0000 7f3947e00640 1 -- 172.21.15.17:0/1595440363 >> 172.21.15.17:0/1595440363 conn(0x7f3948068990 msgr2=0x7f3948068da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:27:08.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:08.221+0000 7f3947e00640 1 -- 172.21.15.17:0/1595440363 shutdown_connections 2024-09-10T21:27:08.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:08.221+0000 7f3947e00640 1 -- 172.21.15.17:0/1595440363 wait complete. 2024-09-10T21:27:08.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:08.221+0000 7f3947e00640 1 Processor -- start 2024-09-10T21:27:08.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:08.221+0000 7f3947e00640 1 -- start start 2024-09-10T21:27:08.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:08.225+0000 7f3947e00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f392c0037b0 0x7f39481f0700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:08.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:08.225+0000 7f3947e00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f39481f48f0 0x7f394806fa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:08.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:08.225+0000 7f3947e00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3948070670 0x7f394806ff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:08.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:08.225+0000 7f3947e00640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f3948072500 con 0x7f3948070670 2024-09-10T21:27:08.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:08.225+0000 7f3947e00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f3948072220 con 0x7f392c0037b0 2024-09-10T21:27:08.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:08.225+0000 7f3947e00640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f39480727e0 con 0x7f39481f48f0 2024-09-10T21:27:08.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:08.225+0000 7f3947400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3948070670 0x7f394806ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:08.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:08.225+0000 7f3947400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3948070670 0x7f394806ff60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44606/0 (socket says 172.21.15.17:44606) 2024-09-10T21:27:08.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:08.225+0000 7f3947400640 1 -- 172.21.15.17:0/1609644565 learned_addr learned my addr 172.21.15.17:0/1609644565 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:27:08.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:08.225+0000 7f3946a00640 1 --2- 172.21.15.17:0/1609644565 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f392c0037b0 0x7f39481f0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:08.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:08.225+0000 7f3946000640 1 --2- 172.21.15.17:0/1609644565 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f39481f48f0 0x7f394806fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:08.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:08.225+0000 7f3947400640 1 -- 172.21.15.17:0/1609644565 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f39481f48f0 msgr2=0x7f394806fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:08.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:08.225+0000 7f3947400640 1 --2- 172.21.15.17:0/1609644565 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f39481f48f0 0x7f394806fa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:08.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:08.225+0000 7f3947400640 1 -- 172.21.15.17:0/1609644565 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f392c0037b0 msgr2=0x7f39481f0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:08.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:08.225+0000 7f3947400640 1 --2- 172.21.15.17:0/1609644565 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f392c0037b0 0x7f39481f0700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:08.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:08.225+0000 7f3947400640 1 -- 172.21.15.17:0/1609644565 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39480c3c00 con 0x7f3948070670 2024-09-10T21:27:08.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:08.225+0000 7f3947400640 1 --2- 172.21.15.17:0/1609644565 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3948070670 0x7f394806ff60 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f3948071b20 tx=0x7f3940010e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:08.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:08.225+0000 7f393ea00640 1 -- 172.21.15.17:0/1609644565 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3940003d90 con 0x7f3948070670 2024-09-10T21:27:08.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:08.225+0000 7f393ea00640 1 -- 172.21.15.17:0/1609644565 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f394001a070 con 0x7f3948070670 2024-09-10T21:27:08.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:08.225+0000 7f3947e00640 1 -- 172.21.15.17:0/1609644565 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39480c3ee0 con 0x7f3948070670 2024-09-10T21:27:08.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:08.225+0000 7f393ea00640 1 -- 172.21.15.17:0/1609644565 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f394000eae0 con 0x7f3948070670 2024-09-10T21:27:08.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:08.225+0000 7f3947e00640 1 -- 172.21.15.17:0/1609644565 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39481f40f0 con 0x7f3948070670 2024-09-10T21:27:08.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:08.225+0000 7f3947e00640 1 --2- 172.21.15.17:0/1609644565 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f39481e3b50 0x7f39481e3f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:08.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:08.225+0000 7f3947e00640 1 -- 172.21.15.17:0/1609644565 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f3948069ef0 con 0x7f39481e3b50 2024-09-10T21:27:08.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:08.225+0000 7f3946a00640 1 --2- 172.21.15.17:0/1609644565 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f39481e3b50 0x7f39481e3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:08.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:08.225+0000 7f3946a00640 1 -- 172.21.15.17:0/1609644565 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f39481e3b50 msgr2=0x7f39481e3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 20 2024-09-10T21:27:08.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:08.225+0000 7f3946a00640 1 -- 172.21.15.17:0/1609644565 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f39481e3b50 msgr2=0x7f39481e3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-10T21:27:08.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:08.225+0000 7f3946a00640 1 --2- 172.21.15.17:0/1609644565 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f39481e3b50 0x7f39481e3f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T21:27:08.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:08.225+0000 7f3946a00640 1 --2- 172.21.15.17:0/1609644565 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f39481e3b50 0x7f39481e3f30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:27:08.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:08.229+0000 7f393ea00640 1 -- 172.21.15.17:0/1609644565 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f394000f040 con 0x7f3948070670 2024-09-10T21:27:08.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:08.229+0000 7f393ea00640 1 --2- 172.21.15.17:0/1609644565 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f3928072610 0x7f3928074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:08.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:08.229+0000 7f393ea00640 1 -- 172.21.15.17:0/1609644565 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f3940093960 con 0x7f3948070670 2024-09-10T21:27:08.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:08.229+0000 7f3946000640 1 --2- 172.21.15.17:0/1609644565 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f3928072610 0x7f3928074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:08.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:08.229+0000 7f3946000640 1 --2- 172.21.15.17:0/1609644565 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f3928072610 0x7f3928074ad0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f39340ac000 tx=0x7f39340b0040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:08.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:08.429+0000 7f3946a00640 1 --2- 172.21.15.17:0/1609644565 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f39481e3b50 0x7f39481e3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:08.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:08.429+0000 7f3946a00640 1 -- 172.21.15.17:0/1609644565 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f39481e3b50 msgr2=0x7f39481e3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 21 2024-09-10T21:27:08.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:08.429+0000 7f3946a00640 1 -- 172.21.15.17:0/1609644565 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f39481e3b50 msgr2=0x7f39481e3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-10T21:27:08.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:08.429+0000 7f3946a00640 1 --2- 172.21.15.17:0/1609644565 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f39481e3b50 0x7f39481e3f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T21:27:08.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:08.429+0000 7f3946a00640 1 --2- 172.21.15.17:0/1609644565 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f39481e3b50 0x7f39481e3f30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-10T21:27:08.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:08.833+0000 7f3946a00640 1 --2- 172.21.15.17:0/1609644565 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f39481e3b50 0x7f39481e3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:08.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:08.833+0000 7f3946a00640 1 -- 172.21.15.17:0/1609644565 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f39481e3b50 msgr2=0x7f39481e3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 21 2024-09-10T21:27:08.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:08.833+0000 7f3946a00640 1 -- 172.21.15.17:0/1609644565 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f39481e3b50 msgr2=0x7f39481e3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-10T21:27:08.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:08.833+0000 7f3946a00640 1 --2- 172.21.15.17:0/1609644565 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f39481e3b50 0x7f39481e3f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T21:27:08.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:08.833+0000 7f3946a00640 1 --2- 172.21.15.17:0/1609644565 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f39481e3b50 0x7f39481e3f30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-10T21:27:09.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:09.637+0000 7f3946a00640 1 --2- 172.21.15.17:0/1609644565 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f39481e3b50 0x7f39481e3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:09.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:09.637+0000 7f3946a00640 1 -- 172.21.15.17:0/1609644565 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f39480704a0 con 0x7f39481e3b50 2024-09-10T21:27:09.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:09.637+0000 7f3946a00640 1 --2- 172.21.15.17:0/1609644565 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f39481e3b50 0x7f39481e3f30 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f3938004ec0 tx=0x7f3938013820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:09.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:09.637+0000 7f393ea00640 1 -- 172.21.15.17:0/1609644565 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39380115f0 con 0x7f39481e3b50 2024-09-10T21:27:09.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:09.637+0000 7f393ea00640 1 -- 172.21.15.17:0/1609644565 <== mon.2 v2:172.21.15.31:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f39380115f0 con 0x7f39481e3b50 2024-09-10T21:27:09.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:09.637+0000 7f393ea00640 1 -- 172.21.15.17:0/1609644565 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f39481e3b50 msgr2=0x7f39481e3f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:09.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:09.637+0000 7f393ea00640 1 --2- 172.21.15.17:0/1609644565 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f39481e3b50 0x7f39481e3f30 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f3938004ec0 tx=0x7f3938013820 comp rx=0 tx=0).stop 2024-09-10T21:27:09.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:09.637+0000 7f393ea00640 1 -- 172.21.15.17:0/1609644565 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f39481e3b50 msgr2=0x7f39481e3f30 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:27:09.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:09.637+0000 7f393ea00640 1 --2- 172.21.15.17:0/1609644565 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f39481e3b50 0x7f39481e3f30 secure :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f3938004ec0 tx=0x7f3938013820 comp rx=0 tx=0).stop 2024-09-10T21:27:09.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:09.645+0000 7f3947e00640 1 --2- 172.21.15.17:0/1609644565 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f39481e4470 0x7f394814d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:09.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:09.645+0000 7f3947e00640 1 -- 172.21.15.17:0/1609644565 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f39481f3c70 con 0x7f39481e4470 2024-09-10T21:27:09.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:09.645+0000 7f3946a00640 1 --2- 172.21.15.17:0/1609644565 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f39481e4470 0x7f394814d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:09.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:09.645+0000 7f3946a00640 1 -- 172.21.15.17:0/1609644565 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f39481580a0 con 0x7f39481e4470 2024-09-10T21:27:09.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:09.645+0000 7f3946a00640 1 --2- 172.21.15.17:0/1609644565 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f39481e4470 0x7f394814d480 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f3938004ec0 tx=0x7f3938001370 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:09.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:09.645+0000 7f393ea00640 1 -- 172.21.15.17:0/1609644565 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f393801ecd0 con 0x7f39481e4470 2024-09-10T21:27:09.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:09.645+0000 7f393ea00640 1 -- 172.21.15.17:0/1609644565 <== mon.2 v2:172.21.15.31:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4540 (secure 0 0 0) 0x7f3938029400 con 0x7f39481e4470 2024-09-10T21:27:09.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:09.645+0000 7f393ea00640 1 -- 172.21.15.17:0/1609644565 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f39481e4470 msgr2=0x7f394814d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:09.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:09.645+0000 7f393ea00640 1 --2- 172.21.15.17:0/1609644565 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f39481e4470 0x7f394814d480 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f3938004ec0 tx=0x7f3938001370 comp rx=0 tx=0).stop 2024-09-10T21:27:09.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:09.645+0000 7f393ea00640 1 -- 172.21.15.17:0/1609644565 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f39481e4470 msgr2=0x7f394814d480 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:27:09.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:09.645+0000 7f393ea00640 1 --2- 172.21.15.17:0/1609644565 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f39481e4470 0x7f394814d480 secure :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f3938004ec0 tx=0x7f3938001370 comp rx=0 tx=0).stop 2024-09-10T21:27:09.651 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-10T21:27:09.651 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:27:09.651 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:27:09.651 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-10T21:27:09.652 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 20, 2024-09-10T21:27:09.652 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 10759, 2024-09-10T21:27:09.652 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-10T21:27:09.652 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-10T21:27:09.652 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-10T21:27:09.652 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-10T21:27:09.652 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:27:09.652 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 4, 2024-09-10T21:27:09.652 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:27:09.652 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-10T21:27:09.652 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-10T21:27:09.652 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:27:09.652 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:27:09.653 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:27:09.653 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:27:09.653 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:27:09.653 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:27:09.653 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:27:09.653 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:27:09.653 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:27:09.653 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:27:09.653 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:27:09.653 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:27:09.653 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-10T21:27:09.654 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-10T21:27:09.654 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:27:09.654 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:27:09.654 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:27:09.654 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:27:09.654 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:27:09.654 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:27:09.654 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:27:09.654 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:27:09.654 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:27:09.654 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:27:09.654 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:27:09.655 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:27:09.655 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:27:09.655 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-10T21:27:09.655 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-10T21:27:09.655 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-10T21:27:09.655 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-10T21:27:09.655 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-10T21:27:09.655 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "5e2f023f-d6a6-49f4-a256-f4807dce3739", 2024-09-10T21:27:09.655 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:27:09.655 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:27:09.655 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-10T21:27:09.655 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-10T21:27:09.655 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-10T21:27:09.656 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-10T21:27:09.656 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-10T21:27:09.656 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-10T21:27:09.656 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-10T21:27:09.656 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:27:09.656 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-10T21:27:09.656 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:27:09.656 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:27:09.656 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:27:09.656 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:27:09.656 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:27:09.656 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:27:09.656 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:27:09.656 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:27:09.657 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:27:09.657 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:27:09.657 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:27:09.657 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:27:09.657 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-10T21:27:09.657 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:27:09.657 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-10T21:27:09.657 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:27:09.657 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:27:09.657 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:27:09.657 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:27:09.658 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:27:09.658 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:27:09.658 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:27:09.658 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-10T21:27:09.658 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:27:09.658 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:27:09.658 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:27:09.658 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:27:09.658 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-10T21:27:09.658 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:27:09.658 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:27:09.658 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:27:09.658 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:27:09.658 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-10T21:27:09.659 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-10T21:27:09.659 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:27:09.659 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:27:09.659 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:27:09.659 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:27:09.659 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:27:09.659 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:27:09.659 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:27:09.659 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:27:09.659 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:27:09.659 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:27:09.659 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:27:09.659 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3300", 2024-09-10T21:27:09.659 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:27:09.659 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:27:09.660 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:27:09.660 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:27:09.660 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789", 2024-09-10T21:27:09.660 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:27:09.660 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:27:09.660 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:27:09.660 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:27:09.660 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789/0", 2024-09-10T21:27:09.660 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6789/0", 2024-09-10T21:27:09.660 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:27:09.660 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:27:09.660 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:27:09.660 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:27:09.661 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:27:09.661 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:27:09.661 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:27:09.661 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:27:09.661 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:27:09.661 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:27:09.661 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:27:09.661 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3301", 2024-09-10T21:27:09.661 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:27:09.661 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:27:09.661 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:27:09.661 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:27:09.661 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790", 2024-09-10T21:27:09.661 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:27:09.662 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:27:09.662 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:27:09.662 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:27:09.662 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790/0", 2024-09-10T21:27:09.662 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6790/0", 2024-09-10T21:27:09.662 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:27:09.662 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:27:09.662 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:27:09.662 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:27:09.662 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:27:09.662 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:27:09.662 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-10T21:27:09.662 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-10T21:27:09.662 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:27:09.663 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:27:09.663 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:27:09.663 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:27:09.663 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:27:09.663 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:27:09.663 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-10T21:27:09.663 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:27:09.663 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:27:09.663 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:27:09.663 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:27:09.663 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:27:09.663 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:27:09.663 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:27:09.663 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-10T21:27:09.663 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-10T21:27:09.664 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:09.645+0000 7f3947e00640 1 -- 172.21.15.17:0/1609644565 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f3928072610 msgr2=0x7f3928074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:09.664 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:09.645+0000 7f3947e00640 1 --2- 172.21.15.17:0/1609644565 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f3928072610 0x7f3928074ad0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f39340ac000 tx=0x7f39340b0040 comp rx=0 tx=0).stop 2024-09-10T21:27:09.664 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:09.645+0000 7f3947e00640 1 -- 172.21.15.17:0/1609644565 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3948070670 msgr2=0x7f394806ff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:09.664 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:09.645+0000 7f3947e00640 1 --2- 172.21.15.17:0/1609644565 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3948070670 0x7f394806ff60 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f3948071b20 tx=0x7f3940010e20 comp rx=0 tx=0).stop 2024-09-10T21:27:09.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:09.645+0000 7f3947400640 1 -- 172.21.15.17:0/1609644565 reap_dead start 2024-09-10T21:27:09.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:09.645+0000 7f3947e00640 1 -- 172.21.15.17:0/1609644565 shutdown_connections 2024-09-10T21:27:09.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:09.645+0000 7f3947e00640 1 --2- 172.21.15.17:0/1609644565 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3948070670 0x7f394806ff60 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:09.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:09.645+0000 7f3947e00640 1 -- 172.21.15.17:0/1609644565 >> 172.21.15.17:0/1609644565 conn(0x7f3948068990 msgr2=0x7f39480ca9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:27:09.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:09.645+0000 7f3947e00640 1 -- 172.21.15.17:0/1609644565 shutdown_connections 2024-09-10T21:27:09.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:09.645+0000 7f3947e00640 1 -- 172.21.15.17:0/1609644565 wait complete. 2024-09-10T21:27:09.674 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-10T21:27:09.674 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-09-10T21:27:09.675 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-09-10T21:27:09.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:09.849+0000 7fc541800640 1 -- 172.21.15.17:0/1679855027 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc520003a10 msgr2=0x7fc520003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:09.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:09.849+0000 7fc541800640 1 --2- 172.21.15.17:0/1679855027 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc520003a10 0x7fc520003e30 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fc52c009b30 tx=0x7fc52c01d270 comp rx=0 tx=0).stop 2024-09-10T21:27:09.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:09.849+0000 7fc53b400640 1 -- 172.21.15.17:0/1679855027 reap_dead start 2024-09-10T21:27:09.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:09.849+0000 7fc541800640 1 -- 172.21.15.17:0/1679855027 shutdown_connections 2024-09-10T21:27:09.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:09.849+0000 7fc541800640 1 -- 172.21.15.17:0/1679855027 >> 172.21.15.17:0/1679855027 conn(0x7fc53c0702f0 msgr2=0x7fc53c070700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:27:09.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:09.849+0000 7fc541800640 1 -- 172.21.15.17:0/1679855027 shutdown_connections 2024-09-10T21:27:09.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:09.849+0000 7fc541800640 1 -- 172.21.15.17:0/1679855027 wait complete. 2024-09-10T21:27:09.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:09.849+0000 7fc541800640 1 Processor -- start 2024-09-10T21:27:09.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:09.849+0000 7fc541800640 1 -- start start 2024-09-10T21:27:09.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:09.853+0000 7fc541800640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc520003a10 0x7fc53c1f4900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:09.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:09.853+0000 7fc541800640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc53c15db20 0x7fc53c14ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:09.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:09.853+0000 7fc541800640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc53c1f3450 0x7fc53c1f38d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:09.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:09.853+0000 7fc541800640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fc53c0cb450 con 0x7fc520003a10 2024-09-10T21:27:09.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:09.853+0000 7fc541800640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fc53c0ce760 con 0x7fc53c15db20 2024-09-10T21:27:09.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:09.853+0000 7fc541800640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7fc53c0cb5d0 con 0x7fc53c1f3450 2024-09-10T21:27:09.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:09.853+0000 7fc53aa00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc520003a10 0x7fc53c1f4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:09.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:09.853+0000 7fc53aa00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc520003a10 0x7fc53c1f4900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44622/0 (socket says 172.21.15.17:44622) 2024-09-10T21:27:09.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:09.853+0000 7fc53aa00640 1 -- 172.21.15.17:0/3544347198 learned_addr learned my addr 172.21.15.17:0/3544347198 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:27:09.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:09.853+0000 7fc53a000640 1 --2- 172.21.15.17:0/3544347198 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc53c15db20 0x7fc53c14ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:09.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:09.853+0000 7fc53b400640 1 --2- 172.21.15.17:0/3544347198 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc53c1f3450 0x7fc53c1f38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:09.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:09.853+0000 7fc53aa00640 1 -- 172.21.15.17:0/3544347198 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc53c1f3450 msgr2=0x7fc53c1f38d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:09.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:09.853+0000 7fc53aa00640 1 --2- 172.21.15.17:0/3544347198 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc53c1f3450 0x7fc53c1f38d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:09.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:09.853+0000 7fc53aa00640 1 -- 172.21.15.17:0/3544347198 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc53c15db20 msgr2=0x7fc53c14ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:09.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:09.853+0000 7fc53aa00640 1 --2- 172.21.15.17:0/3544347198 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc53c15db20 0x7fc53c14ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:09.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:09.853+0000 7fc53aa00640 1 -- 172.21.15.17:0/3544347198 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc53c15df60 con 0x7fc520003a10 2024-09-10T21:27:09.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:09.853+0000 7fc53b400640 1 --2- 172.21.15.17:0/3544347198 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc53c1f3450 0x7fc53c1f38d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:27:09.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:09.853+0000 7fc53aa00640 1 --2- 172.21.15.17:0/3544347198 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc520003a10 0x7fc53c1f4900 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fc52c009a80 tx=0x7fc52c005640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:09.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:09.853+0000 7fc532a00640 1 -- 172.21.15.17:0/3544347198 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc52c0ba020 con 0x7fc520003a10 2024-09-10T21:27:09.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:09.853+0000 7fc532a00640 1 -- 172.21.15.17:0/3544347198 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc52c0ac8a0 con 0x7fc520003a10 2024-09-10T21:27:09.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:09.853+0000 7fc532a00640 1 -- 172.21.15.17:0/3544347198 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc52c0b1040 con 0x7fc520003a10 2024-09-10T21:27:09.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:09.853+0000 7fc541800640 1 -- 172.21.15.17:0/3544347198 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc53c1502c0 con 0x7fc520003a10 2024-09-10T21:27:09.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:09.853+0000 7fc541800640 1 -- 172.21.15.17:0/3544347198 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc53c1f76b0 con 0x7fc520003a10 2024-09-10T21:27:09.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:09.853+0000 7fc541800640 1 -- 172.21.15.17:0/3544347198 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc53c0c3110 con 0x7fc520003a10 2024-09-10T21:27:09.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:09.857+0000 7fc532a00640 1 -- 172.21.15.17:0/3544347198 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fc52c0accd0 con 0x7fc520003a10 2024-09-10T21:27:09.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:09.857+0000 7fc532a00640 1 --2- 172.21.15.17:0/3544347198 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fc51c072610 0x7fc51c074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:09.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:09.857+0000 7fc532a00640 1 -- 172.21.15.17:0/3544347198 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fc52c12e720 con 0x7fc520003a10 2024-09-10T21:27:09.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:09.857+0000 7fc53a000640 1 --2- 172.21.15.17:0/3544347198 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fc51c072610 0x7fc51c074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:09.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:09.857+0000 7fc53a000640 1 --2- 172.21.15.17:0/3544347198 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fc51c072610 0x7fc51c074ad0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fc53c151850 tx=0x7fc528005f20 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:09.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:09.861+0000 7fc532a00640 1 -- 172.21.15.17:0/3544347198 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fc52c0c4030 con 0x7fc520003a10 2024-09-10T21:27:10.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:10.181+0000 7fc541800640 1 -- 172.21.15.17:0/3544347198 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) -- 0x7fc53c14f190 con 0x7fc520003a10 2024-09-10T21:27:10.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:10.185+0000 7fc532a00640 1 -- 172.21.15.17:0/3544347198 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) ==== 57+0+0 (secure 0 0 0) 0x7fc53c15df60 con 0x7fc520003a10 2024-09-10T21:27:10.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:10.185+0000 7fc541800640 1 -- 172.21.15.17:0/3544347198 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fc51c072610 msgr2=0x7fc51c074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:10.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:10.185+0000 7fc541800640 1 --2- 172.21.15.17:0/3544347198 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fc51c072610 0x7fc51c074ad0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fc53c151850 tx=0x7fc528005f20 comp rx=0 tx=0).stop 2024-09-10T21:27:10.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:10.185+0000 7fc541800640 1 -- 172.21.15.17:0/3544347198 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc520003a10 msgr2=0x7fc53c1f4900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:10.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:10.185+0000 7fc541800640 1 --2- 172.21.15.17:0/3544347198 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc520003a10 0x7fc53c1f4900 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fc52c009a80 tx=0x7fc52c005640 comp rx=0 tx=0).stop 2024-09-10T21:27:10.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:10.185+0000 7fc541800640 1 -- 172.21.15.17:0/3544347198 shutdown_connections 2024-09-10T21:27:10.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:10.185+0000 7fc541800640 1 --2- 172.21.15.17:0/3544347198 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fc51c072610 0x7fc51c074ad0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:10.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:10.185+0000 7fc541800640 1 --2- 172.21.15.17:0/3544347198 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc53c1f3450 0x7fc53c1f38d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:10.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:10.185+0000 7fc541800640 1 --2- 172.21.15.17:0/3544347198 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc53c15db20 0x7fc53c14ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:10.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:10.185+0000 7fc541800640 1 --2- 172.21.15.17:0/3544347198 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc520003a10 0x7fc53c1f4900 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:10.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:10.185+0000 7fc541800640 1 -- 172.21.15.17:0/3544347198 >> 172.21.15.17:0/3544347198 conn(0x7fc53c0702f0 msgr2=0x7fc53c06ddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:27:10.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:10.185+0000 7fc541800640 1 -- 172.21.15.17:0/3544347198 shutdown_connections 2024-09-10T21:27:10.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:10.185+0000 7fc541800640 1 -- 172.21.15.17:0/3544347198 wait complete. 2024-09-10T21:27:10.212 INFO:tasks.mon_thrash.mon_thrasher:waiting for 10.0 secs before continuing thrashing 2024-09-10T21:27:15.719 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:27:15.723 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:27:20.213 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-09-10T21:27:20.214 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-10T21:27:20.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:20.385+0000 7fd15f400640 1 -- 172.21.15.17:0/899102055 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd1581572d0 msgr2=0x7fd15815b240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:20.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:20.385+0000 7fd15f400640 1 --2- 172.21.15.17:0/899102055 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd1581572d0 0x7fd15815b240 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd1580c7c70 tx=0x7fd14401f0b0 comp rx=0 tx=0).stop 2024-09-10T21:27:20.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:20.385+0000 7fd15f400640 1 -- 172.21.15.17:0/899102055 shutdown_connections 2024-09-10T21:27:20.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:20.385+0000 7fd15f400640 1 --2- 172.21.15.17:0/899102055 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd1580b2070 0x7fd1580725c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:20.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:20.385+0000 7fd15f400640 1 --2- 172.21.15.17:0/899102055 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd1581572d0 0x7fd15815b240 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:20.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:20.385+0000 7fd15f400640 1 --2- 172.21.15.17:0/899102055 >> v2:172.21.15.31:3300/0 conn(0x7fd1581516c0 0x7fd1580c25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:20.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:20.385+0000 7fd15f400640 1 -- 172.21.15.17:0/899102055 >> 172.21.15.17:0/899102055 conn(0x7fd158070160 msgr2=0x7fd158070570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:27:20.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:20.385+0000 7fd15f400640 1 -- 172.21.15.17:0/899102055 shutdown_connections 2024-09-10T21:27:20.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:20.385+0000 7fd15f400640 1 -- 172.21.15.17:0/899102055 wait complete. 2024-09-10T21:27:20.392 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:20.389+0000 7fd15f400640 1 Processor -- start 2024-09-10T21:27:20.392 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:20.389+0000 7fd15f400640 1 -- start start 2024-09-10T21:27:20.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:20.389+0000 7fd15f400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd1580b2070 0x7fd1580c8110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:20.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:20.389+0000 7fd15f400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd1581516c0 0x7fd1580c4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:20.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:20.389+0000 7fd15f400640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd15814c510 0x7fd1580c4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:20.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:20.389+0000 7fd15f400640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fd15815b780 con 0x7fd1580b2070 2024-09-10T21:27:20.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:20.389+0000 7fd15f400640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fd15815ad20 con 0x7fd1581516c0 2024-09-10T21:27:20.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:20.389+0000 7fd15f400640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7fd1580c2cd0 con 0x7fd15814c510 2024-09-10T21:27:20.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:20.389+0000 7fd15ca00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd1580b2070 0x7fd1580c8110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:20.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:20.389+0000 7fd15ca00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd1580b2070 0x7fd1580c8110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39950/0 (socket says 172.21.15.17:39950) 2024-09-10T21:27:20.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:20.389+0000 7fd15ca00640 1 -- 172.21.15.17:0/855639665 learned_addr learned my addr 172.21.15.17:0/855639665 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:27:20.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:20.389+0000 7fd15d400640 1 --2- 172.21.15.17:0/855639665 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd15814c510 0x7fd1580c4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:20.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:20.389+0000 7fd157e00640 1 --2- 172.21.15.17:0/855639665 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd1581516c0 0x7fd1580c4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:20.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:20.389+0000 7fd15ca00640 1 -- 172.21.15.17:0/855639665 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd15814c510 msgr2=0x7fd1580c4bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:20.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:20.389+0000 7fd15ca00640 1 --2- 172.21.15.17:0/855639665 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd15814c510 0x7fd1580c4bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:20.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:20.389+0000 7fd15ca00640 1 -- 172.21.15.17:0/855639665 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd1581516c0 msgr2=0x7fd1580c4690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:20.396 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:20.389+0000 7fd15ca00640 1 --2- 172.21.15.17:0/855639665 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd1581516c0 0x7fd1580c4690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:20.396 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:20.389+0000 7fd15ca00640 1 -- 172.21.15.17:0/855639665 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd158155bb0 con 0x7fd1580b2070 2024-09-10T21:27:20.396 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:20.389+0000 7fd15ca00640 1 --2- 172.21.15.17:0/855639665 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd1580b2070 0x7fd1580c8110 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd14c00db30 tx=0x7fd14c00b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:20.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:20.389+0000 7fd154c00640 1 -- 172.21.15.17:0/855639665 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd14c017070 con 0x7fd1580b2070 2024-09-10T21:27:20.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:20.389+0000 7fd154c00640 1 -- 172.21.15.17:0/855639665 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd14c004b30 con 0x7fd1580b2070 2024-09-10T21:27:20.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:20.389+0000 7fd15f400640 1 -- 172.21.15.17:0/855639665 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd158155e90 con 0x7fd1580b2070 2024-09-10T21:27:20.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:20.389+0000 7fd154c00640 1 -- 172.21.15.17:0/855639665 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd14c005020 con 0x7fd1580b2070 2024-09-10T21:27:20.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:20.389+0000 7fd15f400640 1 -- 172.21.15.17:0/855639665 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1580cbb90 con 0x7fd1580b2070 2024-09-10T21:27:20.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:20.393+0000 7fd15f400640 1 -- 172.21.15.17:0/855639665 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1580c25d0 con 0x7fd1580b2070 2024-09-10T21:27:20.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:20.393+0000 7fd154c00640 1 -- 172.21.15.17:0/855639665 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fd14c0051c0 con 0x7fd1580b2070 2024-09-10T21:27:20.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:20.393+0000 7fd154c00640 1 --2- 172.21.15.17:0/855639665 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fd1380725e0 0x7fd138074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:20.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:20.393+0000 7fd154c00640 1 -- 172.21.15.17:0/855639665 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fd14c093bd0 con 0x7fd1580b2070 2024-09-10T21:27:20.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:20.393+0000 7fd157e00640 1 --2- 172.21.15.17:0/855639665 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fd1380725e0 0x7fd138074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:20.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:20.397+0000 7fd157e00640 1 --2- 172.21.15.17:0/855639665 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fd1380725e0 0x7fd138074aa0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fd1440b2000 tx=0x7fd14401d8f0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:20.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:20.397+0000 7fd154c00640 1 -- 172.21.15.17:0/855639665 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fd14c020070 con 0x7fd1580b2070 2024-09-10T21:27:20.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:20.661+0000 7fd15f400640 1 -- 172.21.15.17:0/855639665 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fd1580c64f0 con 0x7fd1580b2070 2024-09-10T21:27:20.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:20.661+0000 7fd154c00640 1 -- 172.21.15.17:0/855639665 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1557 (secure 0 0 0) 0x7fd14c05eea0 con 0x7fd1580b2070 2024-09-10T21:27:20.667 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":20,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":15,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5e2f023f-d6a6-49f4-a256-f4807dce3739","modified":"2024-09-10T21:23:46.892130Z","created":"2024-09-10T21:23:46.892130Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3301","nonce":0},{"type":"v1","addr":"172.21.15.31:6790","nonce":0}]},"addr":"172.21.15.31:6790/0","public_addr":"172.21.15.31:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-10T21:27:20.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:20.665+0000 7fd15f400640 1 -- 172.21.15.17:0/855639665 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fd1380725e0 msgr2=0x7fd138074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:20.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:20.665+0000 7fd15f400640 1 --2- 172.21.15.17:0/855639665 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fd1380725e0 0x7fd138074aa0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fd1440b2000 tx=0x7fd14401d8f0 comp rx=0 tx=0).stop 2024-09-10T21:27:20.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:20.665+0000 7fd15f400640 1 -- 172.21.15.17:0/855639665 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd1580b2070 msgr2=0x7fd1580c8110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:20.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:20.665+0000 7fd15f400640 1 --2- 172.21.15.17:0/855639665 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd1580b2070 0x7fd1580c8110 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd14c00db30 tx=0x7fd14c00b260 comp rx=0 tx=0).stop 2024-09-10T21:27:20.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:20.665+0000 7fd15f400640 1 -- 172.21.15.17:0/855639665 shutdown_connections 2024-09-10T21:27:20.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:20.665+0000 7fd15f400640 1 --2- 172.21.15.17:0/855639665 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fd1380725e0 0x7fd138074aa0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:20.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:20.665+0000 7fd15f400640 1 --2- 172.21.15.17:0/855639665 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd15814c510 0x7fd1580c4bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:20.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:20.665+0000 7fd15f400640 1 --2- 172.21.15.17:0/855639665 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd1581516c0 0x7fd1580c4690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:20.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:20.665+0000 7fd15f400640 1 --2- 172.21.15.17:0/855639665 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd1580b2070 0x7fd1580c8110 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:20.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:20.665+0000 7fd15f400640 1 -- 172.21.15.17:0/855639665 >> 172.21.15.17:0/855639665 conn(0x7fd158070160 msgr2=0x7fd15806dd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:27:20.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:20.665+0000 7fd15f400640 1 -- 172.21.15.17:0/855639665 shutdown_connections 2024-09-10T21:27:20.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:20.665+0000 7fd15f400640 1 -- 172.21.15.17:0/855639665 wait complete. 2024-09-10T21:27:20.689 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-09-10T21:27:20.689 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-09-10T21:27:20.690 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-09-10T21:27:20.690 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-10T21:27:20.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:20.853+0000 7fcd25800640 1 -- 172.21.15.17:0/705733689 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fcd2015c540 msgr2=0x7fcd20151e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:20.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:20.853+0000 7fcd25800640 1 --2- 172.21.15.17:0/705733689 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fcd2015c540 0x7fcd20151e90 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fcd18009bc0 tx=0x7fcd1801f2d0 comp rx=0 tx=0).stop 2024-09-10T21:27:20.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:20.853+0000 7fcd25800640 1 -- 172.21.15.17:0/705733689 shutdown_connections 2024-09-10T21:27:20.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:20.853+0000 7fcd25800640 1 --2- 172.21.15.17:0/705733689 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fcd2015c540 0x7fcd20151e90 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:20.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:20.853+0000 7fcd25800640 1 --2- 172.21.15.17:0/705733689 >> v2:172.21.15.31:3300/0 conn(0x7fcd200c6180 0x7fcd200c37e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:20.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:20.853+0000 7fcd25800640 1 -- 172.21.15.17:0/705733689 >> 172.21.15.17:0/705733689 conn(0x7fcd2006cc20 msgr2=0x7fcd2006d030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:27:20.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:20.853+0000 7fcd25800640 1 -- 172.21.15.17:0/705733689 shutdown_connections 2024-09-10T21:27:20.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:20.853+0000 7fcd25800640 1 -- 172.21.15.17:0/705733689 wait complete. 2024-09-10T21:27:20.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:20.857+0000 7fcd25800640 1 Processor -- start 2024-09-10T21:27:20.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:20.857+0000 7fcd25800640 1 -- start start 2024-09-10T21:27:20.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:20.857+0000 7fcd25800640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fcd200c6180 0x7fcd20152e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:20.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:20.857+0000 7fcd25800640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcd200c6560 0x7fcd20154aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:20.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:20.857+0000 7fcd25800640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcd2015c540 0x7fcd201f3340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:20.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:20.857+0000 7fcd25800640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fcd201589e0 con 0x7fcd200c6560 2024-09-10T21:27:20.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:20.857+0000 7fcd25800640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fcd2015bd40 con 0x7fcd2015c540 2024-09-10T21:27:20.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:20.857+0000 7fcd25800640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7fcd20158b60 con 0x7fcd200c6180 2024-09-10T21:27:20.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:20.857+0000 7fcd1f400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcd200c6560 0x7fcd20154aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:20.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:20.857+0000 7fcd1f400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcd200c6560 0x7fcd20154aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39956/0 (socket says 172.21.15.17:39956) 2024-09-10T21:27:20.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:20.857+0000 7fcd1f400640 1 -- 172.21.15.17:0/268944336 learned_addr learned my addr 172.21.15.17:0/268944336 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:27:20.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:20.857+0000 7fcd1fe00640 1 --2- 172.21.15.17:0/268944336 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fcd200c6180 0x7fcd20152e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:20.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:20.857+0000 7fcd24e00640 1 --2- 172.21.15.17:0/268944336 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcd2015c540 0x7fcd201f3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:20.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:20.857+0000 7fcd1fe00640 1 -- 172.21.15.17:0/268944336 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcd2015c540 msgr2=0x7fcd201f3340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:20.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:20.857+0000 7fcd1fe00640 1 --2- 172.21.15.17:0/268944336 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcd2015c540 0x7fcd201f3340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:20.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:20.857+0000 7fcd1fe00640 1 -- 172.21.15.17:0/268944336 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcd200c6560 msgr2=0x7fcd20154aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:20.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:20.857+0000 7fcd1fe00640 1 --2- 172.21.15.17:0/268944336 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcd200c6560 0x7fcd20154aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:20.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:20.857+0000 7fcd1fe00640 1 -- 172.21.15.17:0/268944336 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd20074730 con 0x7fcd200c6180 2024-09-10T21:27:20.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:20.857+0000 7fcd1f400640 1 --2- 172.21.15.17:0/268944336 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcd200c6560 0x7fcd20154aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:27:20.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:20.857+0000 7fcd1fe00640 1 --2- 172.21.15.17:0/268944336 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fcd200c6180 0x7fcd20152e20 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fcd1000e9f0 tx=0x7fcd1000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:20.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:20.857+0000 7fcd17e00640 1 -- 172.21.15.17:0/268944336 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd1000ccc0 con 0x7fcd200c6180 2024-09-10T21:27:20.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:20.857+0000 7fcd17e00640 1 -- 172.21.15.17:0/268944336 <== mon.2 v2:172.21.15.31:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcd10010430 con 0x7fcd200c6180 2024-09-10T21:27:20.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:20.857+0000 7fcd17e00640 1 -- 172.21.15.17:0/268944336 <== mon.2 v2:172.21.15.31:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd10010860 con 0x7fcd200c6180 2024-09-10T21:27:20.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:20.857+0000 7fcd25800640 1 -- 172.21.15.17:0/268944336 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd20074a10 con 0x7fcd200c6180 2024-09-10T21:27:20.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:20.857+0000 7fcd25800640 1 -- 172.21.15.17:0/268944336 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd2014f310 con 0x7fcd200c6180 2024-09-10T21:27:20.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:20.861+0000 7fcd25800640 1 --2- 172.21.15.17:0/268944336 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcd20154130 0x7fcd20151c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:20.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:20.861+0000 7fcd25800640 1 -- 172.21.15.17:0/268944336 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fcd20068ab0 con 0x7fcd20154130 2024-09-10T21:27:20.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:20.861+0000 7fcd1f400640 1 --2- 172.21.15.17:0/268944336 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcd20154130 0x7fcd20151c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:20.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:20.861+0000 7fcd1f400640 1 -- 172.21.15.17:0/268944336 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fcd200cbcc0 con 0x7fcd20154130 2024-09-10T21:27:20.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:20.861+0000 7fcd1f400640 1 --2- 172.21.15.17:0/268944336 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcd20154130 0x7fcd20151c40 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fcd200c2fd0 tx=0x7fcd0c00d520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:20.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:20.861+0000 7fcd17e00640 1 -- 172.21.15.17:0/268944336 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd0c018020 con 0x7fcd20154130 2024-09-10T21:27:20.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:20.861+0000 7fcd17e00640 1 -- 172.21.15.17:0/268944336 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fcd0c00dd60 con 0x7fcd20154130 2024-09-10T21:27:20.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:20.861+0000 7fcd17e00640 1 -- 172.21.15.17:0/268944336 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcd20154130 msgr2=0x7fcd20151c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:20.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:20.861+0000 7fcd17e00640 1 --2- 172.21.15.17:0/268944336 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcd20154130 0x7fcd20151c40 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fcd200c2fd0 tx=0x7fcd0c00d520 comp rx=0 tx=0).stop 2024-09-10T21:27:20.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:20.861+0000 7fcd17e00640 1 -- 172.21.15.17:0/268944336 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcd20154130 msgr2=0x7fcd20151c40 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:27:20.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:20.861+0000 7fcd17e00640 1 --2- 172.21.15.17:0/268944336 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcd20154130 0x7fcd20151c40 secure :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fcd200c2fd0 tx=0x7fcd0c00d520 comp rx=0 tx=0).stop 2024-09-10T21:27:20.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:20.861+0000 7fcd17e00640 1 -- 172.21.15.17:0/268944336 <== mon.2 v2:172.21.15.31:3301/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fcd10010a00 con 0x7fcd200c6180 2024-09-10T21:27:20.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:20.861+0000 7fcd17e00640 1 --2- 172.21.15.17:0/268944336 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fcd000720f0 0x7fcd000745b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:20.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:20.861+0000 7fcd17e00640 1 -- 172.21.15.17:0/268944336 <== mon.2 v2:172.21.15.31:3301/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fcd10094160 con 0x7fcd200c6180 2024-09-10T21:27:20.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:20.861+0000 7fcd1f400640 1 --2- 172.21.15.17:0/268944336 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fcd000720f0 0x7fcd000745b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:20.868 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:20.865+0000 7fcd1f400640 1 --2- 172.21.15.17:0/268944336 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fcd000720f0 0x7fcd000745b0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fcd0c019000 tx=0x7fcd0c009030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:20.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:20.869+0000 7fcd25800640 1 --2- 172.21.15.17:0/268944336 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcd20073e80 0x7fcd20152180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:20.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:20.869+0000 7fcd25800640 1 -- 172.21.15.17:0/268944336 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fcd201526c0 con 0x7fcd20073e80 2024-09-10T21:27:20.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:20.869+0000 7fcd24e00640 1 --2- 172.21.15.17:0/268944336 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcd20073e80 0x7fcd20152180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:20.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:20.869+0000 7fcd24e00640 1 -- 172.21.15.17:0/268944336 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fcd2014fb10 con 0x7fcd20073e80 2024-09-10T21:27:20.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:20.869+0000 7fcd24e00640 1 --2- 172.21.15.17:0/268944336 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcd20073e80 0x7fcd20152180 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fcd200c81a0 tx=0x7fcd180098b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:20.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:20.869+0000 7fcd17e00640 1 -- 172.21.15.17:0/268944336 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd180bf070 con 0x7fcd20073e80 2024-09-10T21:27:20.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:20.869+0000 7fcd17e00640 1 -- 172.21.15.17:0/268944336 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4877 (secure 0 0 0) 0x7fcd18003b80 con 0x7fcd20073e80 2024-09-10T21:27:20.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:20.869+0000 7fcd17e00640 1 -- 172.21.15.17:0/268944336 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcd20073e80 msgr2=0x7fcd20152180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:20.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:20.869+0000 7fcd17e00640 1 --2- 172.21.15.17:0/268944336 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcd20073e80 0x7fcd20152180 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fcd200c81a0 tx=0x7fcd180098b0 comp rx=0 tx=0).stop 2024-09-10T21:27:20.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:20.869+0000 7fcd17e00640 1 -- 172.21.15.17:0/268944336 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcd20073e80 msgr2=0x7fcd20152180 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:27:20.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:20.869+0000 7fcd17e00640 1 --2- 172.21.15.17:0/268944336 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcd20073e80 0x7fcd20152180 secure :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fcd200c81a0 tx=0x7fcd180098b0 comp rx=0 tx=0).stop 2024-09-10T21:27:20.875 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-10T21:27:20.875 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:27:20.875 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:27:20.875 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "leader", 2024-09-10T21:27:20.876 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 20, 2024-09-10T21:27:20.876 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 70775, 2024-09-10T21:27:20.876 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-10T21:27:20.876 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-10T21:27:20.876 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-10T21:27:20.876 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-10T21:27:20.876 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:27:20.876 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 15, 2024-09-10T21:27:20.876 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:27:20.876 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-10T21:27:20.876 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-10T21:27:20.876 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:27:20.876 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:27:20.877 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:27:20.877 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:27:20.877 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:27:20.877 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:27:20.877 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:27:20.877 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:27:20.877 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:27:20.877 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:27:20.878 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:27:20.878 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:27:20.878 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-10T21:27:20.878 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-10T21:27:20.878 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:27:20.878 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:27:20.878 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:27:20.878 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:27:20.878 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:27:20.878 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:27:20.878 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:27:20.878 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:27:20.879 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:27:20.879 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:27:20.879 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:27:20.879 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:27:20.879 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:27:20.879 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-10T21:27:20.879 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-10T21:27:20.879 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-10T21:27:20.879 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-10T21:27:20.879 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-10T21:27:20.879 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "5e2f023f-d6a6-49f4-a256-f4807dce3739", 2024-09-10T21:27:20.879 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:27:20.879 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:27:20.879 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-10T21:27:20.880 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-10T21:27:20.880 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-10T21:27:20.880 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-10T21:27:20.880 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-10T21:27:20.880 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-10T21:27:20.880 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-10T21:27:20.880 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:27:20.880 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-10T21:27:20.881 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:27:20.881 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:27:20.881 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:27:20.881 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:27:20.881 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:27:20.881 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:27:20.881 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:27:20.881 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:27:20.881 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:27:20.881 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:27:20.881 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:27:20.881 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:27:20.882 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-10T21:27:20.882 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:27:20.882 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-10T21:27:20.882 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:27:20.882 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:27:20.882 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:27:20.882 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:27:20.882 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:27:20.882 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:27:20.882 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:27:20.882 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-10T21:27:20.882 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:27:20.882 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:27:20.883 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:27:20.883 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:27:20.883 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-10T21:27:20.883 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:27:20.883 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:27:20.883 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:27:20.883 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:27:20.883 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-10T21:27:20.883 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-10T21:27:20.883 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:27:20.883 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:27:20.883 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:27:20.883 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:27:20.884 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:27:20.884 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:27:20.884 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:27:20.884 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:27:20.884 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:27:20.884 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:27:20.884 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:27:20.884 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3300", 2024-09-10T21:27:20.884 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:27:20.884 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:27:20.884 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:27:20.884 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:27:20.884 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789", 2024-09-10T21:27:20.885 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:27:20.885 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:27:20.885 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:27:20.885 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:27:20.885 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789/0", 2024-09-10T21:27:20.885 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6789/0", 2024-09-10T21:27:20.885 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:27:20.885 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:27:20.885 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:27:20.885 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:27:20.885 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:27:20.885 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:27:20.885 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:27:20.885 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:27:20.886 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:27:20.886 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:27:20.886 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:27:20.886 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3301", 2024-09-10T21:27:20.886 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:27:20.886 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:27:20.886 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:27:20.886 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:27:20.886 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790", 2024-09-10T21:27:20.886 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:27:20.886 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:27:20.886 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:27:20.886 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:27:20.887 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790/0", 2024-09-10T21:27:20.887 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6790/0", 2024-09-10T21:27:20.887 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:27:20.887 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:27:20.887 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:27:20.887 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:27:20.887 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:27:20.887 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:27:20.887 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-10T21:27:20.887 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-10T21:27:20.887 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:27:20.887 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:27:20.887 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:27:20.888 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:27:20.888 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:27:20.888 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:27:20.888 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": [ 2024-09-10T21:27:20.888 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:27:20.888 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:27:20.888 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:27:20.888 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:27:20.888 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:27:20.888 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:27:20.888 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-10T21:27:20.888 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:27:20.889 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:27:20.889 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:27:20.889 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 4 2024-09-10T21:27:20.889 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:27:20.889 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:27:20.889 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-10T21:27:20.889 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:27:20.889 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:27:20.889 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:27:20.889 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 4 2024-09-10T21:27:20.889 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:27:20.889 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:27:20.889 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:27:20.890 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-10T21:27:20.890 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-10T21:27:20.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:20.869+0000 7fcd25800640 1 -- 172.21.15.17:0/268944336 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fcd000720f0 msgr2=0x7fcd000745b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:20.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:20.869+0000 7fcd25800640 1 --2- 172.21.15.17:0/268944336 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fcd000720f0 0x7fcd000745b0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fcd0c019000 tx=0x7fcd0c009030 comp rx=0 tx=0).stop 2024-09-10T21:27:20.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:20.869+0000 7fcd25800640 1 -- 172.21.15.17:0/268944336 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fcd200c6180 msgr2=0x7fcd20152e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:20.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:20.869+0000 7fcd25800640 1 --2- 172.21.15.17:0/268944336 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fcd200c6180 0x7fcd20152e20 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fcd1000e9f0 tx=0x7fcd1000eeb0 comp rx=0 tx=0).stop 2024-09-10T21:27:20.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:20.869+0000 7fcd24e00640 1 -- 172.21.15.17:0/268944336 reap_dead start 2024-09-10T21:27:20.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:20.869+0000 7fcd25800640 1 -- 172.21.15.17:0/268944336 shutdown_connections 2024-09-10T21:27:20.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:20.869+0000 7fcd25800640 1 --2- 172.21.15.17:0/268944336 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fcd200c6180 0x7fcd20152e20 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:20.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:20.869+0000 7fcd25800640 1 -- 172.21.15.17:0/268944336 >> 172.21.15.17:0/268944336 conn(0x7fcd2006cc20 msgr2=0x7fcd200c9190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:27:20.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:20.869+0000 7fcd25800640 1 -- 172.21.15.17:0/268944336 shutdown_connections 2024-09-10T21:27:20.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:20.869+0000 7fcd25800640 1 -- 172.21.15.17:0/268944336 wait complete. 2024-09-10T21:27:20.896 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-10T21:27:20.897 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-09-10T21:27:20.897 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-10T21:27:21.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:21.065+0000 7f37c7200640 1 -- 172.21.15.17:0/3619722848 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f37c80c2f30 msgr2=0x7f37c80c81f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:21.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:21.065+0000 7f37c7200640 1 --2- 172.21.15.17:0/3619722848 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f37c80c2f30 0x7f37c80c81f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f37c806edd0 tx=0x7f37c001f580 comp rx=0 tx=0).stop 2024-09-10T21:27:21.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:21.065+0000 7f37c7200640 1 -- 172.21.15.17:0/3619722848 shutdown_connections 2024-09-10T21:27:21.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:21.065+0000 7f37c7200640 1 --2- 172.21.15.17:0/3619722848 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f37c80c2f30 0x7f37c80c81f0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:21.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:21.065+0000 7f37c7200640 1 --2- 172.21.15.17:0/3619722848 >> v2:172.21.15.31:3300/0 conn(0x7f37c8151450 0x7f37c80ca3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:21.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:21.065+0000 7f37c7200640 1 -- 172.21.15.17:0/3619722848 >> 172.21.15.17:0/3619722848 conn(0x7f37c80cd690 msgr2=0x7f37c80c98e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:27:21.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:21.065+0000 7f37c7200640 1 -- 172.21.15.17:0/3619722848 shutdown_connections 2024-09-10T21:27:21.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:21.065+0000 7f37c7200640 1 -- 172.21.15.17:0/3619722848 wait complete. 2024-09-10T21:27:21.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:21.065+0000 7f37c7200640 1 Processor -- start 2024-09-10T21:27:21.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:21.065+0000 7f37c7200640 1 -- start start 2024-09-10T21:27:21.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:21.065+0000 7f37c7200640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f37c80c2f30 0x7f37c80c68e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:21.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:21.065+0000 7f37c7200640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f37c80cabe0 0x7f37c80c4560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:21.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:21.065+0000 7f37c7200640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f37c8151450 0x7f37c8146370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:21.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:21.065+0000 7f37c7200640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f37c80c2710 con 0x7f37c8151450 2024-09-10T21:27:21.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:21.065+0000 7f37c7200640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f37c80c2a10 con 0x7f37c80cabe0 2024-09-10T21:27:21.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:21.065+0000 7f37c7200640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f37c8151830 con 0x7f37c80c2f30 2024-09-10T21:27:21.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:21.065+0000 7f37c6800640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f37c8151450 0x7f37c8146370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:21.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:21.065+0000 7f37c6800640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f37c8151450 0x7f37c8146370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39974/0 (socket says 172.21.15.17:39974) 2024-09-10T21:27:21.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:21.065+0000 7f37c6800640 1 -- 172.21.15.17:0/1615980201 learned_addr learned my addr 172.21.15.17:0/1615980201 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:27:21.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:21.065+0000 7f37c5e00640 1 --2- 172.21.15.17:0/1615980201 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f37c80c2f30 0x7f37c80c68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:21.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:21.065+0000 7f37c5400640 1 --2- 172.21.15.17:0/1615980201 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f37c80cabe0 0x7f37c80c4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:21.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:21.069+0000 7f37c6800640 1 -- 172.21.15.17:0/1615980201 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f37c80c2f30 msgr2=0x7f37c80c68e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:21.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:21.069+0000 7f37c6800640 1 --2- 172.21.15.17:0/1615980201 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f37c80c2f30 0x7f37c80c68e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:21.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:21.069+0000 7f37c6800640 1 -- 172.21.15.17:0/1615980201 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f37c80cabe0 msgr2=0x7f37c80c4560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:21.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:21.069+0000 7f37c6800640 1 --2- 172.21.15.17:0/1615980201 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f37c80cabe0 0x7f37c80c4560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:21.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:21.069+0000 7f37c6800640 1 -- 172.21.15.17:0/1615980201 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37c814ed20 con 0x7f37c8151450 2024-09-10T21:27:21.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:21.069+0000 7f37c6800640 1 --2- 172.21.15.17:0/1615980201 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f37c8151450 0x7f37c8146370 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f37c0009bd0 tx=0x7f37c0003f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:21.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:21.069+0000 7f37be000640 1 -- 172.21.15.17:0/1615980201 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37c0004e60 con 0x7f37c8151450 2024-09-10T21:27:21.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:21.069+0000 7f37be000640 1 -- 172.21.15.17:0/1615980201 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f37c00af880 con 0x7f37c8151450 2024-09-10T21:27:21.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:21.069+0000 7f37be000640 1 -- 172.21.15.17:0/1615980201 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37c00afc50 con 0x7f37c8151450 2024-09-10T21:27:21.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:21.069+0000 7f37c7200640 1 -- 172.21.15.17:0/1615980201 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37c814efc0 con 0x7f37c8151450 2024-09-10T21:27:21.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:21.069+0000 7f37c7200640 1 -- 172.21.15.17:0/1615980201 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37c814f580 con 0x7f37c8151450 2024-09-10T21:27:21.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:21.069+0000 7f37c7200640 1 --2- 172.21.15.17:0/1615980201 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f37c814ffc0 0x7f37c81468b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:21.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:21.069+0000 7f37c7200640 1 -- 172.21.15.17:0/1615980201 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f37c8069ef0 con 0x7f37c814ffc0 2024-09-10T21:27:21.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:21.069+0000 7f37c5e00640 1 --2- 172.21.15.17:0/1615980201 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f37c814ffc0 0x7f37c81468b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:21.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:21.069+0000 7f37c5e00640 1 -- 172.21.15.17:0/1615980201 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f37c814fb70 con 0x7f37c814ffc0 2024-09-10T21:27:21.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:21.069+0000 7f37c5e00640 1 --2- 172.21.15.17:0/1615980201 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f37c814ffc0 0x7f37c81468b0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f37b80109a0 tx=0x7f37b8010e60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:21.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:21.069+0000 7f37be000640 1 -- 172.21.15.17:0/1615980201 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f37c0003fd0 con 0x7f37c8151450 2024-09-10T21:27:21.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:21.073+0000 7f37be000640 1 --2- 172.21.15.17:0/1615980201 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f37a8072610 0x7f37a8074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:21.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:21.073+0000 7f37be000640 1 -- 172.21.15.17:0/1615980201 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f37c0131e70 con 0x7f37c8151450 2024-09-10T21:27:21.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:21.073+0000 7f37be000640 1 -- 172.21.15.17:0/1615980201 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37b800edb0 con 0x7f37c814ffc0 2024-09-10T21:27:21.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:21.073+0000 7f37be000640 1 -- 172.21.15.17:0/1615980201 <== mon.1 v2:172.21.15.31:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f37b8005020 con 0x7f37c814ffc0 2024-09-10T21:27:21.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:21.073+0000 7f37be000640 1 -- 172.21.15.17:0/1615980201 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f37c814ffc0 msgr2=0x7f37c81468b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:21.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:21.073+0000 7f37be000640 1 --2- 172.21.15.17:0/1615980201 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f37c814ffc0 0x7f37c81468b0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f37b80109a0 tx=0x7f37b8010e60 comp rx=0 tx=0).stop 2024-09-10T21:27:21.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:21.073+0000 7f37c5400640 1 --2- 172.21.15.17:0/1615980201 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f37a8072610 0x7f37a8074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:21.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:21.073+0000 7f37be000640 1 -- 172.21.15.17:0/1615980201 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f37c814ffc0 msgr2=0x7f37c81468b0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:27:21.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:21.073+0000 7f37be000640 1 --2- 172.21.15.17:0/1615980201 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f37c814ffc0 0x7f37c81468b0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:21.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:21.073+0000 7f37c5400640 1 --2- 172.21.15.17:0/1615980201 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f37a8072610 0x7f37a8074ad0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f37c8157ca0 tx=0x7f37b4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:21.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:21.081+0000 7f37c7200640 1 --2- 172.21.15.17:0/1615980201 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f37c806d500 0x7f37c8155cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:21.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:21.081+0000 7f37c7200640 1 -- 172.21.15.17:0/1615980201 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f37c8153c40 con 0x7f37c806d500 2024-09-10T21:27:21.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:21.081+0000 7f37c5e00640 1 --2- 172.21.15.17:0/1615980201 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f37c806d500 0x7f37c8155cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:21.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:21.081+0000 7f37c5e00640 1 -- 172.21.15.17:0/1615980201 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f37c80c79a0 con 0x7f37c806d500 2024-09-10T21:27:21.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:21.081+0000 7f37c5e00640 1 --2- 172.21.15.17:0/1615980201 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f37c806d500 0x7f37c8155cb0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f37b800bfd0 tx=0x7f37b80046d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:21.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:21.081+0000 7f37be000640 1 -- 172.21.15.17:0/1615980201 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37b8020570 con 0x7f37c806d500 2024-09-10T21:27:21.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:21.081+0000 7f37be000640 1 -- 172.21.15.17:0/1615980201 <== mon.1 v2:172.21.15.31:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5043 (secure 0 0 0) 0x7f37b8020570 con 0x7f37c806d500 2024-09-10T21:27:21.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:21.081+0000 7f37be000640 1 -- 172.21.15.17:0/1615980201 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f37c806d500 msgr2=0x7f37c8155cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:21.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:21.081+0000 7f37be000640 1 --2- 172.21.15.17:0/1615980201 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f37c806d500 0x7f37c8155cb0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f37b800bfd0 tx=0x7f37b80046d0 comp rx=0 tx=0).stop 2024-09-10T21:27:21.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:21.081+0000 7f37be000640 1 -- 172.21.15.17:0/1615980201 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f37c806d500 msgr2=0x7f37c8155cb0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:27:21.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:21.081+0000 7f37be000640 1 --2- 172.21.15.17:0/1615980201 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f37c806d500 0x7f37c8155cb0 secure :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f37b800bfd0 tx=0x7f37b80046d0 comp rx=0 tx=0).stop 2024-09-10T21:27:21.088 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-10T21:27:21.088 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:27:21.088 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:27:21.089 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-10T21:27:21.089 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 20, 2024-09-10T21:27:21.089 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 179531, 2024-09-10T21:27:21.089 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-10T21:27:21.089 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-10T21:27:21.089 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-10T21:27:21.089 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-10T21:27:21.089 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:27:21.089 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 15, 2024-09-10T21:27:21.089 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:27:21.089 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-10T21:27:21.089 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-10T21:27:21.089 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:27:21.089 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:27:21.090 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:27:21.090 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:27:21.090 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:27:21.090 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:27:21.090 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:27:21.090 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:27:21.090 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:27:21.090 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:27:21.090 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:27:21.090 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:27:21.090 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-10T21:27:21.090 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-10T21:27:21.090 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:27:21.091 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:27:21.091 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:27:21.091 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:27:21.091 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:27:21.091 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:27:21.091 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:27:21.091 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:27:21.091 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:27:21.091 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:27:21.092 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:27:21.092 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:27:21.092 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:27:21.092 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-10T21:27:21.092 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-10T21:27:21.092 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-10T21:27:21.092 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-10T21:27:21.092 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-10T21:27:21.092 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "5e2f023f-d6a6-49f4-a256-f4807dce3739", 2024-09-10T21:27:21.092 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:27:21.092 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:27:21.092 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-10T21:27:21.092 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-10T21:27:21.093 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-10T21:27:21.093 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-10T21:27:21.093 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-10T21:27:21.093 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-10T21:27:21.093 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-10T21:27:21.093 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:27:21.093 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-10T21:27:21.093 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:27:21.093 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:27:21.093 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:27:21.093 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:27:21.093 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:27:21.093 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:27:21.094 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:27:21.094 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:27:21.094 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:27:21.094 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:27:21.094 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:27:21.094 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:27:21.094 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-10T21:27:21.094 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:27:21.094 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-10T21:27:21.094 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:27:21.094 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:27:21.094 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:27:21.094 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:27:21.094 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:27:21.094 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:27:21.095 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:27:21.095 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-10T21:27:21.095 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:27:21.095 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:27:21.095 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:27:21.095 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:27:21.095 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-10T21:27:21.095 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:27:21.095 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:27:21.095 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:27:21.095 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:27:21.095 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-10T21:27:21.095 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-10T21:27:21.095 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:27:21.096 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:27:21.096 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:27:21.096 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:27:21.096 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:27:21.096 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:27:21.096 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:27:21.096 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:27:21.096 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:27:21.096 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:27:21.096 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:27:21.096 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3300", 2024-09-10T21:27:21.096 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:27:21.096 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:27:21.097 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:27:21.097 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:27:21.097 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789", 2024-09-10T21:27:21.097 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:27:21.097 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:27:21.097 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:27:21.097 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:27:21.097 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789/0", 2024-09-10T21:27:21.097 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6789/0", 2024-09-10T21:27:21.097 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:27:21.097 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:27:21.097 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:27:21.097 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:27:21.097 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:27:21.098 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:27:21.098 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:27:21.098 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:27:21.098 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:27:21.098 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:27:21.098 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:27:21.098 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3301", 2024-09-10T21:27:21.098 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:27:21.098 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:27:21.098 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:27:21.098 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:27:21.098 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790", 2024-09-10T21:27:21.098 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:27:21.098 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:27:21.098 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:27:21.099 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:27:21.099 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790/0", 2024-09-10T21:27:21.099 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6790/0", 2024-09-10T21:27:21.099 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:27:21.099 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:27:21.099 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:27:21.099 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:27:21.099 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:27:21.099 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:27:21.099 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-10T21:27:21.099 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-10T21:27:21.099 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:27:21.099 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:27:21.099 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:27:21.099 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:27:21.099 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:27:21.100 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:27:21.100 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": [ 2024-09-10T21:27:21.100 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:27:21.100 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:27:21.100 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:27:21.100 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 5 2024-09-10T21:27:21.100 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:27:21.100 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:27:21.100 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-10T21:27:21.100 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:27:21.100 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:27:21.100 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:27:21.100 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 4 2024-09-10T21:27:21.100 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:27:21.100 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:27:21.101 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-10T21:27:21.101 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:27:21.101 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:27:21.101 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:27:21.101 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 3 2024-09-10T21:27:21.101 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:27:21.101 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:27:21.101 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": [ 2024-09-10T21:27:21.101 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:27:21.101 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:27:21.101 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:27:21.101 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 2 2024-09-10T21:27:21.101 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:27:21.102 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:27:21.102 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:27:21.102 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-10T21:27:21.102 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-10T21:27:21.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:21.085+0000 7f37c7200640 1 -- 172.21.15.17:0/1615980201 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f37a8072610 msgr2=0x7f37a8074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:21.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:21.085+0000 7f37c7200640 1 --2- 172.21.15.17:0/1615980201 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f37a8072610 0x7f37a8074ad0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f37c8157ca0 tx=0x7f37b4009040 comp rx=0 tx=0).stop 2024-09-10T21:27:21.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:21.085+0000 7f37c7200640 1 -- 172.21.15.17:0/1615980201 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f37c8151450 msgr2=0x7f37c8146370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:21.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:21.085+0000 7f37c7200640 1 --2- 172.21.15.17:0/1615980201 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f37c8151450 0x7f37c8146370 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f37c0009bd0 tx=0x7f37c0003f60 comp rx=0 tx=0).stop 2024-09-10T21:27:21.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:21.085+0000 7f37c6800640 1 -- 172.21.15.17:0/1615980201 reap_dead start 2024-09-10T21:27:21.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:21.085+0000 7f37c6800640 1 -- 172.21.15.17:0/1615980201 reap_dead start 2024-09-10T21:27:21.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:21.085+0000 7f37c7200640 1 -- 172.21.15.17:0/1615980201 shutdown_connections 2024-09-10T21:27:21.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:21.085+0000 7f37c7200640 1 -- 172.21.15.17:0/1615980201 >> 172.21.15.17:0/1615980201 conn(0x7f37c80cd690 msgr2=0x7f37c8153e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:27:21.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:21.085+0000 7f37c7200640 1 -- 172.21.15.17:0/1615980201 shutdown_connections 2024-09-10T21:27:21.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:21.085+0000 7f37c7200640 1 -- 172.21.15.17:0/1615980201 wait complete. 2024-09-10T21:27:21.115 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-10T21:27:21.115 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-09-10T21:27:21.115 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-10T21:27:21.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:21.269+0000 7f7588200640 1 -- 172.21.15.17:0/380862156 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7580152820 msgr2=0x7f75800b3eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:21.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:21.269+0000 7f7588200640 1 --2- 172.21.15.17:0/380862156 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7580152820 0x7f75800b3eb0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f75801594a0 tx=0x7f756c01c790 comp rx=0 tx=0).stop 2024-09-10T21:27:21.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:21.269+0000 7f7588200640 1 -- 172.21.15.17:0/380862156 shutdown_connections 2024-09-10T21:27:21.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:21.269+0000 7f7588200640 1 --2- 172.21.15.17:0/380862156 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7580152820 0x7f75800b3eb0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:21.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:21.269+0000 7f7588200640 1 -- 172.21.15.17:0/380862156 >> 172.21.15.17:0/380862156 conn(0x7f75800702f0 msgr2=0x7f7580070700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:27:21.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:21.269+0000 7f7588200640 1 -- 172.21.15.17:0/380862156 shutdown_connections 2024-09-10T21:27:21.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:21.269+0000 7f7588200640 1 -- 172.21.15.17:0/380862156 wait complete. 2024-09-10T21:27:21.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:21.269+0000 7f7588200640 1 Processor -- start 2024-09-10T21:27:21.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:21.269+0000 7f7588200640 1 -- start start 2024-09-10T21:27:21.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:21.273+0000 7f7588200640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f75800cdf20 0x7f758015b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:21.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:21.273+0000 7f7588200640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7580152440 0x7f758014bd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:21.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:21.273+0000 7f7588200640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7580152820 0x7f758014c240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:21.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:21.273+0000 7f7588200640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f75800cb450 con 0x7f7580152440 2024-09-10T21:27:21.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:21.273+0000 7f7588200640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f75800ce740 con 0x7f7580152820 2024-09-10T21:27:21.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:21.273+0000 7f7588200640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f75800cb5d0 con 0x7f75800cdf20 2024-09-10T21:27:21.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:21.273+0000 7f7584e00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7580152440 0x7f758014bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:21.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:21.273+0000 7f7584e00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7580152440 0x7f758014bd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:39992/0 (socket says 172.21.15.17:39992) 2024-09-10T21:27:21.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:21.273+0000 7f7584e00640 1 -- 172.21.15.17:0/683433230 learned_addr learned my addr 172.21.15.17:0/683433230 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:27:21.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:21.273+0000 7f7586200640 1 --2- 172.21.15.17:0/683433230 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7580152820 0x7f758014c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:21.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:21.273+0000 7f7585800640 1 --2- 172.21.15.17:0/683433230 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f75800cdf20 0x7f758015b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:21.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:21.273+0000 7f7584e00640 1 -- 172.21.15.17:0/683433230 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f75800cdf20 msgr2=0x7f758015b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:21.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:21.273+0000 7f7584e00640 1 --2- 172.21.15.17:0/683433230 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f75800cdf20 0x7f758015b0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:21.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:21.273+0000 7f7584e00640 1 -- 172.21.15.17:0/683433230 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7580152820 msgr2=0x7f758014c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:21.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:21.273+0000 7f7584e00640 1 --2- 172.21.15.17:0/683433230 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7580152820 0x7f758014c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:21.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:21.273+0000 7f7584e00640 1 -- 172.21.15.17:0/683433230 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f758015b5f0 con 0x7f7580152440 2024-09-10T21:27:21.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:21.273+0000 7f7584e00640 1 --2- 172.21.15.17:0/683433230 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7580152440 0x7f758014bd00 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f756c009790 tx=0x7f756c004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:21.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:21.273+0000 7f757d600640 1 -- 172.21.15.17:0/683433230 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f756c0bb070 con 0x7f7580152440 2024-09-10T21:27:21.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:21.273+0000 7f757d600640 1 -- 172.21.15.17:0/683433230 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f756c005030 con 0x7f7580152440 2024-09-10T21:27:21.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:21.273+0000 7f7588200640 1 -- 172.21.15.17:0/683433230 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7580151f20 con 0x7f7580152440 2024-09-10T21:27:21.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:21.273+0000 7f757d600640 1 -- 172.21.15.17:0/683433230 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f756c0b2040 con 0x7f7580152440 2024-09-10T21:27:21.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:21.273+0000 7f7588200640 1 -- 172.21.15.17:0/683433230 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75801599a0 con 0x7f7580152440 2024-09-10T21:27:21.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:21.273+0000 7f7588200640 1 --2- 172.21.15.17:0/683433230 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f758014d620 0x7f75800c59d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:21.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:21.273+0000 7f7588200640 1 -- 172.21.15.17:0/683433230 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f758014c780 con 0x7f758014d620 2024-09-10T21:27:21.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:21.273+0000 7f7585800640 1 --2- 172.21.15.17:0/683433230 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f758014d620 0x7f75800c59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:21.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:21.273+0000 7f7585800640 1 -- 172.21.15.17:0/683433230 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7580158360 con 0x7f758014d620 2024-09-10T21:27:21.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:21.277+0000 7f7585800640 1 --2- 172.21.15.17:0/683433230 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f758014d620 0x7f75800c59d0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f757400e500 tx=0x7f757400e9c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:21.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:21.277+0000 7f757d600640 1 -- 172.21.15.17:0/683433230 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75740052e0 con 0x7f758014d620 2024-09-10T21:27:21.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:21.277+0000 7f757d600640 1 -- 172.21.15.17:0/683433230 <== mon.2 v2:172.21.15.31:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f7574018070 con 0x7f758014d620 2024-09-10T21:27:21.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:21.277+0000 7f757d600640 1 -- 172.21.15.17:0/683433230 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f758014d620 msgr2=0x7f75800c59d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:21.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:21.277+0000 7f757d600640 1 --2- 172.21.15.17:0/683433230 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f758014d620 0x7f75800c59d0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f757400e500 tx=0x7f757400e9c0 comp rx=0 tx=0).stop 2024-09-10T21:27:21.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:21.277+0000 7f757d600640 1 -- 172.21.15.17:0/683433230 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f758014d620 msgr2=0x7f75800c59d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:27:21.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:21.277+0000 7f757d600640 1 --2- 172.21.15.17:0/683433230 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f758014d620 0x7f75800c59d0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:21.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:21.277+0000 7f757d600640 1 -- 172.21.15.17:0/683433230 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f756c0b5030 con 0x7f7580152440 2024-09-10T21:27:21.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:21.277+0000 7f757d600640 1 --2- 172.21.15.17:0/683433230 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f7560072080 0x7f7560074540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:21.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:21.277+0000 7f757d600640 1 -- 172.21.15.17:0/683433230 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f756c12d1c0 con 0x7f7580152440 2024-09-10T21:27:21.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:21.277+0000 7f7585800640 1 --2- 172.21.15.17:0/683433230 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f7560072080 0x7f7560074540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:21.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:21.277+0000 7f7585800640 1 --2- 172.21.15.17:0/683433230 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f7560072080 0x7f7560074540 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f757400e500 tx=0x7f7574005000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:21.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:21.281+0000 7f7588200640 1 --2- 172.21.15.17:0/683433230 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f75800c5f10 0x7f75800c6ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:21.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:21.281+0000 7f7588200640 1 -- 172.21.15.17:0/683433230 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f75800c9af0 con 0x7f75800c5f10 2024-09-10T21:27:21.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:21.285+0000 7f7586200640 1 --2- 172.21.15.17:0/683433230 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f75800c5f10 0x7f75800c6ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:21.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:21.285+0000 7f7586200640 1 -- 172.21.15.17:0/683433230 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f758014f9c0 con 0x7f75800c5f10 2024-09-10T21:27:21.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:21.285+0000 7f7586200640 1 --2- 172.21.15.17:0/683433230 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f75800c5f10 0x7f75800c6ba0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f7580151be0 tx=0x7f757800eba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:21.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:21.285+0000 7f757d600640 1 -- 172.21.15.17:0/683433230 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f757800c690 con 0x7f75800c5f10 2024-09-10T21:27:21.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:21.285+0000 7f757d600640 1 -- 172.21.15.17:0/683433230 <== mon.2 v2:172.21.15.31:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4541 (secure 0 0 0) 0x7f757800c690 con 0x7f75800c5f10 2024-09-10T21:27:21.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:21.285+0000 7f757d600640 1 -- 172.21.15.17:0/683433230 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f75800c5f10 msgr2=0x7f75800c6ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:21.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:21.285+0000 7f757d600640 1 --2- 172.21.15.17:0/683433230 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f75800c5f10 0x7f75800c6ba0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f7580151be0 tx=0x7f757800eba0 comp rx=0 tx=0).stop 2024-09-10T21:27:21.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:21.285+0000 7f757d600640 1 -- 172.21.15.17:0/683433230 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f75800c5f10 msgr2=0x7f75800c6ba0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:27:21.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:21.285+0000 7f757d600640 1 --2- 172.21.15.17:0/683433230 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f75800c5f10 0x7f75800c6ba0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:21.290 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-10T21:27:21.290 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:27:21.290 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:27:21.290 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-10T21:27:21.290 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 20, 2024-09-10T21:27:21.290 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 22399, 2024-09-10T21:27:21.290 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-10T21:27:21.291 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-10T21:27:21.291 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-10T21:27:21.291 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-10T21:27:21.291 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:27:21.291 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 16, 2024-09-10T21:27:21.291 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:27:21.291 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-10T21:27:21.291 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-10T21:27:21.291 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:27:21.291 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:27:21.291 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:27:21.291 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:27:21.291 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:27:21.291 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:27:21.292 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:27:21.292 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:27:21.292 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:27:21.292 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:27:21.292 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:27:21.292 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:27:21.292 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-10T21:27:21.292 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-10T21:27:21.292 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:27:21.293 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:27:21.293 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:27:21.293 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:27:21.293 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:27:21.293 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:27:21.293 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:27:21.293 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:27:21.293 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:27:21.293 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:27:21.293 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:27:21.293 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:27:21.293 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:27:21.293 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-10T21:27:21.294 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-10T21:27:21.294 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-10T21:27:21.294 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-10T21:27:21.294 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-10T21:27:21.294 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "5e2f023f-d6a6-49f4-a256-f4807dce3739", 2024-09-10T21:27:21.294 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:27:21.294 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:27:21.294 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-10T21:27:21.294 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-10T21:27:21.294 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-10T21:27:21.294 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-10T21:27:21.294 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-10T21:27:21.294 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-10T21:27:21.295 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-10T21:27:21.295 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:27:21.295 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-10T21:27:21.295 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:27:21.295 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:27:21.295 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:27:21.295 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:27:21.295 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:27:21.295 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:27:21.295 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:27:21.295 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:27:21.295 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:27:21.295 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:27:21.295 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:27:21.296 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:27:21.296 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-10T21:27:21.296 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:27:21.296 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-10T21:27:21.296 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:27:21.296 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:27:21.296 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:27:21.296 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:27:21.296 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:27:21.296 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:27:21.296 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:27:21.296 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-10T21:27:21.296 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:27:21.297 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:27:21.297 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:27:21.297 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:27:21.297 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-10T21:27:21.297 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:27:21.297 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:27:21.297 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:27:21.297 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:27:21.297 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-10T21:27:21.297 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-10T21:27:21.297 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:27:21.297 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:27:21.297 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:27:21.297 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:27:21.298 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:27:21.298 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:27:21.298 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:27:21.298 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:27:21.298 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:27:21.298 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:27:21.298 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:27:21.298 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3300", 2024-09-10T21:27:21.298 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:27:21.298 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:27:21.298 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:27:21.298 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:27:21.298 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789", 2024-09-10T21:27:21.298 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:27:21.299 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:27:21.299 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:27:21.299 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:27:21.299 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789/0", 2024-09-10T21:27:21.299 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6789/0", 2024-09-10T21:27:21.299 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:27:21.299 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:27:21.299 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:27:21.299 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:27:21.299 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:27:21.299 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:27:21.299 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:27:21.299 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:27:21.299 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:27:21.300 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:27:21.300 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:27:21.300 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3301", 2024-09-10T21:27:21.300 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:27:21.300 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:27:21.300 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:27:21.300 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:27:21.300 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790", 2024-09-10T21:27:21.300 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:27:21.300 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:27:21.300 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:27:21.300 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:27:21.300 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790/0", 2024-09-10T21:27:21.300 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6790/0", 2024-09-10T21:27:21.301 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:27:21.301 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:27:21.301 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:27:21.301 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:27:21.301 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:27:21.301 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:27:21.301 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-10T21:27:21.301 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-10T21:27:21.301 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:27:21.301 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:27:21.301 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:27:21.301 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:27:21.301 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:27:21.302 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:27:21.302 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-10T21:27:21.302 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:27:21.302 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:27:21.302 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:27:21.302 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:27:21.302 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:27:21.302 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:27:21.302 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:27:21.302 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-10T21:27:21.302 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-10T21:27:21.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:21.285+0000 7f7588200640 1 -- 172.21.15.17:0/683433230 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f7560072080 msgr2=0x7f7560074540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:21.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:21.285+0000 7f7588200640 1 --2- 172.21.15.17:0/683433230 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f7560072080 0x7f7560074540 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f757400e500 tx=0x7f7574005000 comp rx=0 tx=0).stop 2024-09-10T21:27:21.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:21.285+0000 7f7588200640 1 -- 172.21.15.17:0/683433230 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7580152440 msgr2=0x7f758014bd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:21.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:21.285+0000 7f7588200640 1 --2- 172.21.15.17:0/683433230 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7580152440 0x7f758014bd00 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f756c009790 tx=0x7f756c004040 comp rx=0 tx=0).stop 2024-09-10T21:27:21.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:21.285+0000 7f7586200640 1 -- 172.21.15.17:0/683433230 reap_dead start 2024-09-10T21:27:21.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:21.285+0000 7f7588200640 1 -- 172.21.15.17:0/683433230 shutdown_connections 2024-09-10T21:27:21.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:21.285+0000 7f7588200640 1 -- 172.21.15.17:0/683433230 >> 172.21.15.17:0/683433230 conn(0x7f75800702f0 msgr2=0x7f75800706d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:27:21.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:21.285+0000 7f7588200640 1 -- 172.21.15.17:0/683433230 shutdown_connections 2024-09-10T21:27:21.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:21.285+0000 7f7588200640 1 -- 172.21.15.17:0/683433230 wait complete. 2024-09-10T21:27:21.308 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-10T21:27:21.309 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['a'] 2024-09-10T21:27:21.309 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: ['c'] 2024-09-10T21:27:21.309 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.a 2024-09-10T21:27:21.309 INFO:tasks.mon_thrash.mon_thrasher:killing mon.a 2024-09-10T21:27:21.309 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-10T21:27:21.309 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:27:21.326 INFO:tasks.ceph.mon.a:Stopped 2024-09-10T21:27:21.327 INFO:tasks.mon_thrash:Sending STOP to mon c 2024-09-10T21:27:21.327 INFO:tasks.ceph.mon.c:Sent signal 19 2024-09-10T21:27:21.327 INFO:tasks.mon_thrash.mon_thrasher:waiting for 15.0 secs to unfreeze mons 2024-09-10T21:27:36.328 INFO:tasks.mon_thrash:Sending CONT to mon c 2024-09-10T21:27:36.329 INFO:tasks.ceph.mon.c:Sent signal 18 2024-09-10T21:27:36.329 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2024-09-10T21:27:36.330 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-10T21:27:41.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:41.357+0000 7f0373c00640 1 -- 172.21.15.17:0/1676815823 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f0350004810 msgr2=0x7f0350004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:41.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:41.357+0000 7f0373c00640 1 --2- 172.21.15.17:0/1676815823 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f0350004810 0x7f0350004c90 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f035800c2e0 tx=0x7f035801f720 comp rx=0 tx=0).stop 2024-09-10T21:27:41.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:41.357+0000 7f0373c00640 1 -- 172.21.15.17:0/1676815823 shutdown_connections 2024-09-10T21:27:41.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:41.357+0000 7f0373c00640 1 --2- 172.21.15.17:0/1676815823 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f03500052a0 0x7f035000b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:41.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:41.357+0000 7f0373c00640 1 --2- 172.21.15.17:0/1676815823 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f0350004810 0x7f0350004c90 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:41.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:41.357+0000 7f0373c00640 1 --2- 172.21.15.17:0/1676815823 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f03500039c0 0x7f0350003dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:41.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:41.357+0000 7f0373c00640 1 -- 172.21.15.17:0/1676815823 >> 172.21.15.17:0/1676815823 conn(0x7f036c0702f0 msgr2=0x7f036c070700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:27:41.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:41.357+0000 7f0373c00640 1 -- 172.21.15.17:0/1676815823 shutdown_connections 2024-09-10T21:27:41.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:41.357+0000 7f0373c00640 1 -- 172.21.15.17:0/1676815823 wait complete. 2024-09-10T21:27:41.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:41.361+0000 7f0373c00640 1 Processor -- start 2024-09-10T21:27:41.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:41.361+0000 7f0373c00640 1 -- start start 2024-09-10T21:27:41.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:41.361+0000 7f0373c00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f03500039c0 0x7f036c1f3490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:41.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:41.361+0000 7f0373c00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f0350004810 0x7f036c14efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:41.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:41.361+0000 7f0373c00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f03500052a0 0x7f036c14f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:41.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:41.361+0000 7f0373c00640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f036c15bd60 con 0x7f03500039c0 2024-09-10T21:27:41.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:41.361+0000 7f0373c00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f036c15ba80 con 0x7f03500052a0 2024-09-10T21:27:41.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:41.361+0000 7f0373c00640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f036c0cb450 con 0x7f0350004810 2024-09-10T21:27:41.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:41.361+0000 7f0371200640 1 -- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f03500039c0 msgr2=0x7f036c1f3490 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-10T21:27:41.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:41.361+0000 7f0371200640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f03500039c0 0x7f036c1f3490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:27:41.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:41.361+0000 7f036be00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f0350004810 0x7f036c14efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:41.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:41.361+0000 7f036be00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f0350004810 0x7f036c14efb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3301/0 says I am v2:172.21.15.17:54852/0 (socket says 172.21.15.17:54852) 2024-09-10T21:27:41.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:41.361+0000 7f036be00640 1 -- 172.21.15.17:0/1295337332 learned_addr learned my addr 172.21.15.17:0/1295337332 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:27:41.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:41.361+0000 7f0371c00640 1 --2- 172.21.15.17:0/1295337332 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f03500052a0 0x7f036c14f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:41.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:41.361+0000 7f036be00640 1 -- 172.21.15.17:0/1295337332 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f03500052a0 msgr2=0x7f036c14f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:41.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:41.361+0000 7f036be00640 1 --2- 172.21.15.17:0/1295337332 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f03500052a0 0x7f036c14f4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:41.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:41.361+0000 7f036be00640 1 -- 172.21.15.17:0/1295337332 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f03500039c0 msgr2=0x7f036c1f3490 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:27:41.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:41.361+0000 7f036be00640 1 --2- 172.21.15.17:0/1295337332 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f03500039c0 0x7f036c1f3490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:41.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:41.361+0000 7f036be00640 1 -- 172.21.15.17:0/1295337332 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f036c14fda0 con 0x7f0350004810 2024-09-10T21:27:41.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:41.361+0000 7f036be00640 1 --2- 172.21.15.17:0/1295337332 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f0350004810 0x7f036c14efb0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f035801de20 tx=0x7f0358000f80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:41.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:41.361+0000 7f0368c00640 1 -- 172.21.15.17:0/1295337332 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0358003b30 con 0x7f0350004810 2024-09-10T21:27:41.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:41.361+0000 7f0368c00640 1 -- 172.21.15.17:0/1295337332 <== mon.2 v2:172.21.15.31:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0358006220 con 0x7f0350004810 2024-09-10T21:27:41.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:41.361+0000 7f0373c00640 1 -- 172.21.15.17:0/1295337332 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f036c158260 con 0x7f0350004810 2024-09-10T21:27:41.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:41.361+0000 7f0368c00640 1 -- 172.21.15.17:0/1295337332 <== mon.2 v2:172.21.15.31:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0358006670 con 0x7f0350004810 2024-09-10T21:27:41.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:41.361+0000 7f0373c00640 1 -- 172.21.15.17:0/1295337332 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f036c1f2f70 con 0x7f0350004810 2024-09-10T21:27:41.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:41.365+0000 7f0373c00640 1 -- 172.21.15.17:0/1295337332 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f036c158cc0 con 0x7f0350004810 2024-09-10T21:27:41.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:41.365+0000 7f0368c00640 1 -- 172.21.15.17:0/1295337332 <== mon.2 v2:172.21.15.31:3301/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f0358006810 con 0x7f0350004810 2024-09-10T21:27:41.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:41.365+0000 7f0368c00640 1 --2- 172.21.15.17:0/1295337332 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f034c072d60 0x7f034c075220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:41.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:41.365+0000 7f0368c00640 1 -- 172.21.15.17:0/1295337332 <== mon.2 v2:172.21.15.31:3301/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f0358131430 con 0x7f0350004810 2024-09-10T21:27:41.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:41.365+0000 7f0371200640 1 --2- 172.21.15.17:0/1295337332 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f034c072d60 0x7f034c075220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:41.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:41.365+0000 7f0371200640 1 --2- 172.21.15.17:0/1295337332 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f034c072d60 0x7f034c075220 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f035c005c90 tx=0x7f035c012590 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:41.376 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:41.373+0000 7f0368c00640 1 -- 172.21.15.17:0/1295337332 <== mon.2 v2:172.21.15.31:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f03580c7020 con 0x7f0350004810 2024-09-10T21:27:41.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:41.617+0000 7f0373c00640 1 -- 172.21.15.17:0/1295337332 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f036c06f880 con 0x7f0350004810 2024-09-10T21:27:41.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:41.617+0000 7f0368c00640 1 -- 172.21.15.17:0/1295337332 <== mon.2 v2:172.21.15.31:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1550 (secure 0 0 0) 0x7f03580194d0 con 0x7f0350004810 2024-09-10T21:27:41.623 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":22,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":0,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5e2f023f-d6a6-49f4-a256-f4807dce3739","modified":"2024-09-10T21:23:46.892130Z","created":"2024-09-10T21:23:46.892130Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3301","nonce":0},{"type":"v1","addr":"172.21.15.31:6790","nonce":0}]},"addr":"172.21.15.31:6790/0","public_addr":"172.21.15.31:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-10T21:27:41.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:41.621+0000 7f0373c00640 1 -- 172.21.15.17:0/1295337332 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f034c072d60 msgr2=0x7f034c075220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:41.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:41.621+0000 7f0373c00640 1 --2- 172.21.15.17:0/1295337332 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f034c072d60 0x7f034c075220 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f035c005c90 tx=0x7f035c012590 comp rx=0 tx=0).stop 2024-09-10T21:27:41.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:41.621+0000 7f0373c00640 1 -- 172.21.15.17:0/1295337332 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f0350004810 msgr2=0x7f036c14efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:41.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:41.621+0000 7f0373c00640 1 --2- 172.21.15.17:0/1295337332 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f0350004810 0x7f036c14efb0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f035801de20 tx=0x7f0358000f80 comp rx=0 tx=0).stop 2024-09-10T21:27:41.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:41.621+0000 7f0373c00640 1 -- 172.21.15.17:0/1295337332 shutdown_connections 2024-09-10T21:27:41.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:41.621+0000 7f0373c00640 1 --2- 172.21.15.17:0/1295337332 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f034c072d60 0x7f034c075220 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:41.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:41.621+0000 7f0373c00640 1 --2- 172.21.15.17:0/1295337332 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f03500052a0 0x7f036c14f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:41.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:41.621+0000 7f0373c00640 1 --2- 172.21.15.17:0/1295337332 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f0350004810 0x7f036c14efb0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:41.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:41.621+0000 7f0373c00640 1 --2- 172.21.15.17:0/1295337332 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f03500039c0 0x7f036c1f3490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:41.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:41.621+0000 7f0373c00640 1 -- 172.21.15.17:0/1295337332 >> 172.21.15.17:0/1295337332 conn(0x7f036c0702f0 msgr2=0x7f036c0c3810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:27:41.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:41.621+0000 7f0373c00640 1 -- 172.21.15.17:0/1295337332 shutdown_connections 2024-09-10T21:27:41.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:41.621+0000 7f0373c00640 1 -- 172.21.15.17:0/1295337332 wait complete. 2024-09-10T21:27:41.645 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2024-09-10T21:27:41.645 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon b 2024-09-10T21:27:41.646 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-10T21:27:41.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:41.793+0000 7fc75fe00640 1 -- 172.21.15.17:0/1782229059 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc760158f60 msgr2=0x7fc760072960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:41.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:41.793+0000 7fc75fe00640 1 --2- 172.21.15.17:0/1782229059 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc760158f60 0x7fc760072960 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fc758009c80 tx=0x7fc75801f5a0 comp rx=0 tx=0).stop 2024-09-10T21:27:41.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:41.793+0000 7fc75fe00640 1 -- 172.21.15.17:0/1782229059 shutdown_connections 2024-09-10T21:27:41.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:41.793+0000 7fc75fe00640 1 --2- 172.21.15.17:0/1782229059 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc760158f60 0x7fc760072960 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:41.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:41.793+0000 7fc75fe00640 1 --2- 172.21.15.17:0/1782229059 >> v2:172.21.15.31:3300/0 conn(0x7fc760152070 0x7fc760072420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:41.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:41.793+0000 7fc75fe00640 1 -- 172.21.15.17:0/1782229059 >> 172.21.15.17:0/1782229059 conn(0x7fc76006cc20 msgr2=0x7fc76006d030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:27:41.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:41.793+0000 7fc75fe00640 1 -- 172.21.15.17:0/1782229059 shutdown_connections 2024-09-10T21:27:41.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:41.793+0000 7fc75fe00640 1 -- 172.21.15.17:0/1782229059 wait complete. 2024-09-10T21:27:41.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:41.797+0000 7fc75fe00640 1 Processor -- start 2024-09-10T21:27:41.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:41.797+0000 7fc75fe00640 1 -- start start 2024-09-10T21:27:41.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:41.797+0000 7fc75fe00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc760152070 0x7fc76006ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:41.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:41.797+0000 7fc75fe00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc760152450 0x7fc760071e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:41.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:41.797+0000 7fc75fe00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc760158f60 0x7fc7601f0740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:41.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:41.797+0000 7fc75fe00640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fc7600cecf0 con 0x7fc760158f60 2024-09-10T21:27:41.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:41.797+0000 7fc75fe00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fc7600c5090 con 0x7fc760152070 2024-09-10T21:27:41.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:41.797+0000 7fc75fe00640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7fc7600cee70 con 0x7fc760152450 2024-09-10T21:27:41.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:41.797+0000 7fc75f400640 1 -- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc760158f60 msgr2=0x7fc7601f0740 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-10T21:27:41.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:41.797+0000 7fc75f400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc760158f60 0x7fc7601f0740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:27:41.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:41.797+0000 7fc75ea00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc760152070 0x7fc76006ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:41.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:41.797+0000 7fc756000640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc760152450 0x7fc760071e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:41.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:41.797+0000 7fc75ea00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc760152070 0x7fc76006ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.17:53658/0 (socket says 172.21.15.17:53658) 2024-09-10T21:27:41.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:41.797+0000 7fc75ea00640 1 -- 172.21.15.17:0/3854717397 learned_addr learned my addr 172.21.15.17:0/3854717397 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:27:41.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:41.797+0000 7fc756000640 1 -- 172.21.15.17:0/3854717397 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc760152070 msgr2=0x7fc76006ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:41.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:41.797+0000 7fc756000640 1 --2- 172.21.15.17:0/3854717397 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc760152070 0x7fc76006ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:41.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:41.797+0000 7fc756000640 1 -- 172.21.15.17:0/3854717397 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc760158f60 msgr2=0x7fc7601f0740 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:27:41.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:41.797+0000 7fc756000640 1 --2- 172.21.15.17:0/3854717397 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc760158f60 0x7fc7601f0740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:41.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:41.797+0000 7fc756000640 1 -- 172.21.15.17:0/3854717397 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc760072e00 con 0x7fc760152450 2024-09-10T21:27:41.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:41.797+0000 7fc756000640 1 --2- 172.21.15.17:0/3854717397 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc760152450 0x7fc760071e40 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fc760152860 tx=0x7fc74800b900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:41.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:41.797+0000 7fc757400640 1 -- 172.21.15.17:0/3854717397 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc748015020 con 0x7fc760152450 2024-09-10T21:27:41.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:41.797+0000 7fc757400640 1 -- 172.21.15.17:0/3854717397 <== mon.2 v2:172.21.15.31:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc748011920 con 0x7fc760152450 2024-09-10T21:27:41.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:41.797+0000 7fc757400640 1 -- 172.21.15.17:0/3854717397 <== mon.2 v2:172.21.15.31:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc748011d50 con 0x7fc760152450 2024-09-10T21:27:41.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:41.797+0000 7fc75fe00640 1 -- 172.21.15.17:0/3854717397 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc76006f7c0 con 0x7fc760152450 2024-09-10T21:27:41.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:41.797+0000 7fc75fe00640 1 -- 172.21.15.17:0/3854717397 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fc76014fc50 con 0x7fc760152450 2024-09-10T21:27:41.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:41.797+0000 7fc75fe00640 1 --2- 172.21.15.17:0/3854717397 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc7601f0c80 0x7fc76014fe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:41.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:41.797+0000 7fc75fe00640 1 -- 172.21.15.17:0/3854717397 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fc760069ef0 con 0x7fc7601f0c80 2024-09-10T21:27:41.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:41.801+0000 7fc75ea00640 1 --2- 172.21.15.17:0/3854717397 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc7601f0c80 0x7fc76014fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:41.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:41.801+0000 7fc75ea00640 1 -- 172.21.15.17:0/3854717397 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc7600cbd10 con 0x7fc7601f0c80 2024-09-10T21:27:41.806 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:41.801+0000 7fc75ea00640 1 --2- 172.21.15.17:0/3854717397 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc7601f0c80 0x7fc76014fe20 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fc750011a80 tx=0x7fc750011f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:41.806 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:41.801+0000 7fc757400640 1 -- 172.21.15.17:0/3854717397 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc75000fdc0 con 0x7fc7601f0c80 2024-09-10T21:27:41.806 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:41.801+0000 7fc757400640 1 -- 172.21.15.17:0/3854717397 <== mon.2 v2:172.21.15.31:3301/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fc74801a400 con 0x7fc760152450 2024-09-10T21:27:41.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:41.801+0000 7fc757400640 1 --2- 172.21.15.17:0/3854717397 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fc73c0724e0 0x7fc73c0749a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:41.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:41.801+0000 7fc757400640 1 -- 172.21.15.17:0/3854717397 <== mon.2 v2:172.21.15.31:3301/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fc7480937a0 con 0x7fc760152450 2024-09-10T21:27:41.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:41.801+0000 7fc757400640 1 -- 172.21.15.17:0/3854717397 <== mon.1 v2:172.21.15.31:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fc750019410 con 0x7fc7601f0c80 2024-09-10T21:27:41.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:41.801+0000 7fc757400640 1 -- 172.21.15.17:0/3854717397 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc7601f0c80 msgr2=0x7fc76014fe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:41.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:41.801+0000 7fc757400640 1 --2- 172.21.15.17:0/3854717397 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc7601f0c80 0x7fc76014fe20 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fc750011a80 tx=0x7fc750011f40 comp rx=0 tx=0).stop 2024-09-10T21:27:41.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:41.801+0000 7fc757400640 1 -- 172.21.15.17:0/3854717397 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc7601f0c80 msgr2=0x7fc76014fe20 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:27:41.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:41.801+0000 7fc757400640 1 --2- 172.21.15.17:0/3854717397 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc7601f0c80 0x7fc76014fe20 secure :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fc750011a80 tx=0x7fc750011f40 comp rx=0 tx=0).stop 2024-09-10T21:27:41.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:41.801+0000 7fc75f400640 1 --2- 172.21.15.17:0/3854717397 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fc73c0724e0 0x7fc73c0749a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:41.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:41.801+0000 7fc75f400640 1 --2- 172.21.15.17:0/3854717397 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fc73c0724e0 0x7fc73c0749a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fc7580b3000 tx=0x7fc7580ab040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:41.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:41.809+0000 7fc75fe00640 1 --2- 172.21.15.17:0/3854717397 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc76006dd30 0x7fc76014b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:41.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:41.809+0000 7fc75fe00640 1 -- 172.21.15.17:0/3854717397 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fc760155b80 con 0x7fc76006dd30 2024-09-10T21:27:41.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:41.809+0000 7fc75ea00640 1 --2- 172.21.15.17:0/3854717397 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc76006dd30 0x7fc76014b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:41.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:41.809+0000 7fc75ea00640 1 -- 172.21.15.17:0/3854717397 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fc7600cc710 con 0x7fc76006dd30 2024-09-10T21:27:41.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:41.809+0000 7fc75ea00640 1 --2- 172.21.15.17:0/3854717397 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc76006dd30 0x7fc76014b880 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fc750011a80 tx=0x7fc7500046d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:41.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:41.809+0000 7fc757400640 1 -- 172.21.15.17:0/3854717397 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc750019cd0 con 0x7fc76006dd30 2024-09-10T21:27:41.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:41.809+0000 7fc757400640 1 -- 172.21.15.17:0/3854717397 <== mon.1 v2:172.21.15.31:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5033 (secure 0 0 0) 0x7fc7500202b0 con 0x7fc76006dd30 2024-09-10T21:27:41.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:41.809+0000 7fc757400640 1 -- 172.21.15.17:0/3854717397 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc76006dd30 msgr2=0x7fc76014b880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:41.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:41.809+0000 7fc757400640 1 --2- 172.21.15.17:0/3854717397 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc76006dd30 0x7fc76014b880 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fc750011a80 tx=0x7fc7500046d0 comp rx=0 tx=0).stop 2024-09-10T21:27:41.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:41.809+0000 7fc757400640 1 -- 172.21.15.17:0/3854717397 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc76006dd30 msgr2=0x7fc76014b880 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:27:41.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:41.809+0000 7fc757400640 1 --2- 172.21.15.17:0/3854717397 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc76006dd30 0x7fc76014b880 secure :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fc750011a80 tx=0x7fc7500046d0 comp rx=0 tx=0).stop 2024-09-10T21:27:41.815 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-10T21:27:41.815 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:27:41.815 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:27:41.815 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "leader", 2024-09-10T21:27:41.815 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 22, 2024-09-10T21:27:41.816 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 200255, 2024-09-10T21:27:41.816 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-10T21:27:41.816 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-10T21:27:41.816 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-10T21:27:41.816 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:27:41.816 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 0, 2024-09-10T21:27:41.816 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:27:41.816 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-10T21:27:41.816 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-10T21:27:41.816 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:27:41.816 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:27:41.816 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:27:41.816 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:27:41.817 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:27:41.817 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:27:41.817 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:27:41.817 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:27:41.817 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:27:41.817 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:27:41.817 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:27:41.817 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:27:41.817 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-10T21:27:41.817 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-10T21:27:41.817 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:27:41.817 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:27:41.817 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:27:41.818 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:27:41.818 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:27:41.818 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:27:41.818 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:27:41.818 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:27:41.818 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:27:41.818 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:27:41.818 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:27:41.818 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:27:41.818 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:27:41.818 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-10T21:27:41.818 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-10T21:27:41.818 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-10T21:27:41.819 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-10T21:27:41.819 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-10T21:27:41.819 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "5e2f023f-d6a6-49f4-a256-f4807dce3739", 2024-09-10T21:27:41.819 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:27:41.819 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:27:41.819 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-10T21:27:41.819 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-10T21:27:41.819 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-10T21:27:41.819 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-10T21:27:41.819 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-10T21:27:41.819 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-10T21:27:41.819 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-10T21:27:41.819 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:27:41.820 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-10T21:27:41.820 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:27:41.820 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:27:41.820 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:27:41.820 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:27:41.820 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:27:41.820 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:27:41.820 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:27:41.820 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:27:41.820 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:27:41.820 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:27:41.820 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:27:41.820 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:27:41.820 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-10T21:27:41.821 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:27:41.821 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-10T21:27:41.821 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:27:41.821 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:27:41.821 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:27:41.821 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:27:41.821 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:27:41.821 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:27:41.821 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:27:41.821 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-10T21:27:41.821 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:27:41.821 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:27:41.821 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:27:41.822 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:27:41.822 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-10T21:27:41.822 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:27:41.822 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:27:41.822 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:27:41.822 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:27:41.822 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-10T21:27:41.822 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-10T21:27:41.822 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:27:41.822 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:27:41.822 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:27:41.822 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:27:41.822 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:27:41.822 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:27:41.823 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:27:41.823 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:27:41.823 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:27:41.823 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:27:41.823 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:27:41.823 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3300", 2024-09-10T21:27:41.823 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:27:41.823 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:27:41.823 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:27:41.823 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:27:41.823 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789", 2024-09-10T21:27:41.823 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:27:41.823 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:27:41.823 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:27:41.824 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:27:41.824 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789/0", 2024-09-10T21:27:41.824 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6789/0", 2024-09-10T21:27:41.824 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:27:41.824 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:27:41.824 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:27:41.824 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:27:41.824 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:27:41.824 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:27:41.824 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:27:41.824 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:27:41.824 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:27:41.824 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:27:41.824 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:27:41.825 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3301", 2024-09-10T21:27:41.825 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:27:41.825 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:27:41.825 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:27:41.825 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:27:41.825 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790", 2024-09-10T21:27:41.825 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:27:41.825 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:27:41.825 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:27:41.825 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:27:41.825 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790/0", 2024-09-10T21:27:41.825 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6790/0", 2024-09-10T21:27:41.825 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:27:41.826 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:27:41.826 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:27:41.826 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:27:41.826 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:27:41.826 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:27:41.826 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-10T21:27:41.826 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-10T21:27:41.826 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:27:41.826 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:27:41.826 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:27:41.826 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:27:41.826 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:27:41.826 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:27:41.826 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": [ 2024-09-10T21:27:41.826 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:27:41.826 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:27:41.827 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:27:41.827 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 5 2024-09-10T21:27:41.827 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:27:41.827 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:27:41.827 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-10T21:27:41.827 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:27:41.827 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:27:41.827 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:27:41.827 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 4 2024-09-10T21:27:41.827 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:27:41.827 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:27:41.827 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-10T21:27:41.827 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:27:41.827 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:27:41.828 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:27:41.828 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 4 2024-09-10T21:27:41.828 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:27:41.828 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:27:41.828 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": [ 2024-09-10T21:27:41.828 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:27:41.828 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:27:41.828 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:27:41.828 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 2 2024-09-10T21:27:41.828 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:27:41.828 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:27:41.828 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:27:41.828 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-10T21:27:41.829 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-10T21:27:41.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:41.809+0000 7fc75fe00640 1 -- 172.21.15.17:0/3854717397 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fc73c0724e0 msgr2=0x7fc73c0749a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:41.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:41.809+0000 7fc75fe00640 1 --2- 172.21.15.17:0/3854717397 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fc73c0724e0 0x7fc73c0749a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fc7580b3000 tx=0x7fc7580ab040 comp rx=0 tx=0).stop 2024-09-10T21:27:41.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:41.809+0000 7fc75fe00640 1 -- 172.21.15.17:0/3854717397 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc760152450 msgr2=0x7fc760071e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:41.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:41.809+0000 7fc75fe00640 1 --2- 172.21.15.17:0/3854717397 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc760152450 0x7fc760071e40 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fc760152860 tx=0x7fc74800b900 comp rx=0 tx=0).stop 2024-09-10T21:27:41.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:41.809+0000 7fc75f400640 1 -- 172.21.15.17:0/3854717397 reap_dead start 2024-09-10T21:27:41.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:41.809+0000 7fc75f400640 1 -- 172.21.15.17:0/3854717397 reap_dead start 2024-09-10T21:27:41.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:41.809+0000 7fc75fe00640 1 -- 172.21.15.17:0/3854717397 shutdown_connections 2024-09-10T21:27:41.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:41.809+0000 7fc75fe00640 1 -- 172.21.15.17:0/3854717397 >> 172.21.15.17:0/3854717397 conn(0x7fc76006cc20 msgr2=0x7fc7600747e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:27:41.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:41.809+0000 7fc75fe00640 1 -- 172.21.15.17:0/3854717397 shutdown_connections 2024-09-10T21:27:41.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:41.809+0000 7fc75fe00640 1 -- 172.21.15.17:0/3854717397 wait complete. 2024-09-10T21:27:41.833 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-09-10T21:27:41.834 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon c 2024-09-10T21:27:41.834 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-10T21:27:42.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:42.009+0000 7fbd85400640 1 -- 172.21.15.17:0/3023476157 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbd80152070 msgr2=0x7fbd80072420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:42.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:42.009+0000 7fbd85400640 1 --2- 172.21.15.17:0/3023476157 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbd80152070 0x7fbd80072420 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fbd800cec30 tx=0x7fbd7001f0b0 comp rx=0 tx=0).stop 2024-09-10T21:27:42.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:42.009+0000 7fbd85400640 1 -- 172.21.15.17:0/3023476157 shutdown_connections 2024-09-10T21:27:42.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:42.009+0000 7fbd85400640 1 --2- 172.21.15.17:0/3023476157 >> v2:172.21.15.17:3300/0 conn(0x7fbd80158f60 0x7fbd80072960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:42.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:42.009+0000 7fbd85400640 1 --2- 172.21.15.17:0/3023476157 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbd80152070 0x7fbd80072420 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:42.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:42.009+0000 7fbd85400640 1 -- 172.21.15.17:0/3023476157 >> 172.21.15.17:0/3023476157 conn(0x7fbd8006cc20 msgr2=0x7fbd8006d030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:27:42.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:42.009+0000 7fbd85400640 1 -- 172.21.15.17:0/3023476157 shutdown_connections 2024-09-10T21:27:42.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:42.009+0000 7fbd85400640 1 -- 172.21.15.17:0/3023476157 wait complete. 2024-09-10T21:27:42.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:42.013+0000 7fbd85400640 1 Processor -- start 2024-09-10T21:27:42.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:42.013+0000 7fbd85400640 1 -- start start 2024-09-10T21:27:42.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:42.013+0000 7fbd85400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbd80152070 0x7fbd801f1c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:42.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:42.013+0000 7fbd85400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbd80152450 0x7fbd8014cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:42.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:42.013+0000 7fbd85400640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fbd80158f60 0x7fbd8014a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:42.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:42.013+0000 7fbd85400640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fbd800cecf0 con 0x7fbd80152070 2024-09-10T21:27:42.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:42.013+0000 7fbd85400640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fbd800c5090 con 0x7fbd80152450 2024-09-10T21:27:42.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:42.013+0000 7fbd85400640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7fbd800cee70 con 0x7fbd80158f60 2024-09-10T21:27:42.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:42.013+0000 7fbd7fe00640 1 -- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbd80152070 msgr2=0x7fbd801f1c60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-10T21:27:42.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:42.013+0000 7fbd7fe00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbd80152070 0x7fbd801f1c60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:27:42.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:42.013+0000 7fbd7f400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbd80152450 0x7fbd8014cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:42.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:42.013+0000 7fbd84a00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fbd80158f60 0x7fbd8014a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:42.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:42.013+0000 7fbd7f400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbd80152450 0x7fbd8014cb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.17:53694/0 (socket says 172.21.15.17:53694) 2024-09-10T21:27:42.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:42.013+0000 7fbd7f400640 1 -- 172.21.15.17:0/1533611036 learned_addr learned my addr 172.21.15.17:0/1533611036 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:27:42.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:42.013+0000 7fbd7f400640 1 -- 172.21.15.17:0/1533611036 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fbd80158f60 msgr2=0x7fbd8014a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:42.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:42.013+0000 7fbd7f400640 1 --2- 172.21.15.17:0/1533611036 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fbd80158f60 0x7fbd8014a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:42.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:42.013+0000 7fbd7f400640 1 -- 172.21.15.17:0/1533611036 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbd80152070 msgr2=0x7fbd801f1c60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:27:42.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:42.013+0000 7fbd7f400640 1 --2- 172.21.15.17:0/1533611036 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbd80152070 0x7fbd801f1c60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:42.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:42.013+0000 7fbd7f400640 1 -- 172.21.15.17:0/1533611036 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd801e4a10 con 0x7fbd80152450 2024-09-10T21:27:42.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:42.013+0000 7fbd7f400640 1 --2- 172.21.15.17:0/1533611036 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbd80152450 0x7fbd8014cb40 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fbd6c00b4d0 tx=0x7fbd6c00b990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:42.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:42.013+0000 7fbd77e00640 1 -- 172.21.15.17:0/1533611036 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd6c014070 con 0x7fbd80152450 2024-09-10T21:27:42.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:42.013+0000 7fbd77e00640 1 -- 172.21.15.17:0/1533611036 <== mon.1 v2:172.21.15.31:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbd6c010960 con 0x7fbd80152450 2024-09-10T21:27:42.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:42.013+0000 7fbd77e00640 1 -- 172.21.15.17:0/1533611036 <== mon.1 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd6c010db0 con 0x7fbd80152450 2024-09-10T21:27:42.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:42.013+0000 7fbd85400640 1 -- 172.21.15.17:0/1533611036 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd80152f70 con 0x7fbd80152450 2024-09-10T21:27:42.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:42.013+0000 7fbd85400640 1 -- 172.21.15.17:0/1533611036 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd801e4730 con 0x7fbd80152450 2024-09-10T21:27:42.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:42.017+0000 7fbd85400640 1 --2- 172.21.15.17:0/1533611036 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fbd801e3ed0 0x7fbd801e3340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:42.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:42.017+0000 7fbd85400640 1 -- 172.21.15.17:0/1533611036 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7fbd80069ef0 con 0x7fbd801e3ed0 2024-09-10T21:27:42.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:42.017+0000 7fbd7fe00640 1 --2- 172.21.15.17:0/1533611036 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fbd801e3ed0 0x7fbd801e3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:42.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:42.017+0000 7fbd7fe00640 1 -- 172.21.15.17:0/1533611036 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fbd80153250 con 0x7fbd801e3ed0 2024-09-10T21:27:42.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:42.017+0000 7fbd7fe00640 1 --2- 172.21.15.17:0/1533611036 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fbd801e3ed0 0x7fbd801e3340 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fbd800cc310 tx=0x7fbd70005d60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:42.023 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:42.017+0000 7fbd77e00640 1 -- 172.21.15.17:0/1533611036 <== mon.1 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fbd6c00f3c0 con 0x7fbd80152450 2024-09-10T21:27:42.023 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:42.017+0000 7fbd77e00640 1 --2- 172.21.15.17:0/1533611036 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fbd60072590 0x7fbd60074a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:42.023 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:42.017+0000 7fbd77e00640 1 -- 172.21.15.17:0/1533611036 <== mon.1 v2:172.21.15.31:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fbd6c0936b0 con 0x7fbd80152450 2024-09-10T21:27:42.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:42.017+0000 7fbd77e00640 1 -- 172.21.15.17:0/1533611036 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd700b5040 con 0x7fbd801e3ed0 2024-09-10T21:27:42.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:42.017+0000 7fbd77e00640 1 -- 172.21.15.17:0/1533611036 <== mon.2 v2:172.21.15.31:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fbd700b0440 con 0x7fbd801e3ed0 2024-09-10T21:27:42.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:42.017+0000 7fbd84a00640 1 --2- 172.21.15.17:0/1533611036 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fbd60072590 0x7fbd60074a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:42.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:42.017+0000 7fbd77e00640 1 -- 172.21.15.17:0/1533611036 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fbd801e3ed0 msgr2=0x7fbd801e3340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:42.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:42.017+0000 7fbd77e00640 1 --2- 172.21.15.17:0/1533611036 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fbd801e3ed0 0x7fbd801e3340 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fbd800cc310 tx=0x7fbd70005d60 comp rx=0 tx=0).stop 2024-09-10T21:27:42.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:42.017+0000 7fbd77e00640 1 -- 172.21.15.17:0/1533611036 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fbd801e3ed0 msgr2=0x7fbd801e3340 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:27:42.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:42.017+0000 7fbd77e00640 1 --2- 172.21.15.17:0/1533611036 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fbd801e3ed0 0x7fbd801e3340 secure :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fbd800cc310 tx=0x7fbd70005d60 comp rx=0 tx=0).stop 2024-09-10T21:27:42.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:42.021+0000 7fbd84a00640 1 --2- 172.21.15.17:0/1533611036 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fbd60072590 0x7fbd60074a50 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fbd78001430 tx=0x7fbd7800d040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:42.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:42.025+0000 7fbd85400640 1 --2- 172.21.15.17:0/1533611036 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fbd801e3910 0x7fbd8014ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:27:42.030 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:42.025+0000 7fbd85400640 1 -- 172.21.15.17:0/1533611036 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7fbd800cbd10 con 0x7fbd801e3910 2024-09-10T21:27:42.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:42.025+0000 7fbd7fe00640 1 --2- 172.21.15.17:0/1533611036 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fbd801e3910 0x7fbd8014ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:27:42.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:42.025+0000 7fbd7fe00640 1 -- 172.21.15.17:0/1533611036 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fbd800cc710 con 0x7fbd801e3910 2024-09-10T21:27:42.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:42.025+0000 7fbd7fe00640 1 --2- 172.21.15.17:0/1533611036 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fbd801e3910 0x7fbd8014ae70 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fbd70009230 tx=0x7fbd7001f0b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:27:42.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:42.029+0000 7fbd77e00640 1 -- 172.21.15.17:0/1533611036 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd700b5040 con 0x7fbd801e3910 2024-09-10T21:27:42.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:42.029+0000 7fbd77e00640 1 -- 172.21.15.17:0/1533611036 <== mon.2 v2:172.21.15.31:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4863 (secure 0 0 0) 0x7fbd700b0900 con 0x7fbd801e3910 2024-09-10T21:27:42.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:42.029+0000 7fbd77e00640 1 -- 172.21.15.17:0/1533611036 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fbd801e3910 msgr2=0x7fbd8014ae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:42.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:42.029+0000 7fbd77e00640 1 --2- 172.21.15.17:0/1533611036 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fbd801e3910 0x7fbd8014ae70 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fbd70009230 tx=0x7fbd7001f0b0 comp rx=0 tx=0).stop 2024-09-10T21:27:42.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:42.029+0000 7fbd77e00640 1 -- 172.21.15.17:0/1533611036 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fbd801e3910 msgr2=0x7fbd8014ae70 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:27:42.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:42.029+0000 7fbd77e00640 1 --2- 172.21.15.17:0/1533611036 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fbd801e3910 0x7fbd8014ae70 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:42.033 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-10T21:27:42.033 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:27:42.033 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:27:42.033 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-10T21:27:42.034 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 22, 2024-09-10T21:27:42.034 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 43143, 2024-09-10T21:27:42.034 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-10T21:27:42.034 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-10T21:27:42.034 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-10T21:27:42.034 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:27:42.034 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 0, 2024-09-10T21:27:42.034 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:27:42.034 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-10T21:27:42.034 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-10T21:27:42.034 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:27:42.034 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:27:42.035 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:27:42.035 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:27:42.035 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:27:42.035 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:27:42.035 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:27:42.035 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:27:42.035 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:27:42.035 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:27:42.035 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:27:42.035 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:27:42.035 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-10T21:27:42.035 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-10T21:27:42.035 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:27:42.035 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:27:42.036 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:27:42.036 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:27:42.036 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:27:42.036 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:27:42.036 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:27:42.036 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:27:42.036 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:27:42.036 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:27:42.036 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:27:42.037 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:27:42.037 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:27:42.037 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-10T21:27:42.037 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-10T21:27:42.037 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-10T21:27:42.037 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-10T21:27:42.037 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-10T21:27:42.037 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "5e2f023f-d6a6-49f4-a256-f4807dce3739", 2024-09-10T21:27:42.037 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:27:42.037 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:27:42.037 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-10T21:27:42.037 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-10T21:27:42.037 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-10T21:27:42.037 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-10T21:27:42.038 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-10T21:27:42.038 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-10T21:27:42.038 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-10T21:27:42.038 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:27:42.038 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-10T21:27:42.038 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:27:42.038 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:27:42.038 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:27:42.038 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:27:42.038 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:27:42.038 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:27:42.038 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:27:42.038 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:27:42.038 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:27:42.038 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:27:42.039 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:27:42.039 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:27:42.039 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-10T21:27:42.039 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:27:42.039 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-10T21:27:42.039 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:27:42.039 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:27:42.039 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:27:42.039 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:27:42.039 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:27:42.039 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:27:42.039 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:27:42.039 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-10T21:27:42.039 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:27:42.040 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:27:42.040 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:27:42.040 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:27:42.040 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-10T21:27:42.040 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:27:42.040 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:27:42.040 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:27:42.040 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:27:42.040 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-10T21:27:42.040 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-10T21:27:42.040 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:27:42.040 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:27:42.040 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:27:42.041 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:27:42.041 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:27:42.041 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:27:42.041 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:27:42.041 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:27:42.041 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:27:42.041 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:27:42.041 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:27:42.041 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3300", 2024-09-10T21:27:42.041 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:27:42.041 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:27:42.041 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:27:42.041 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:27:42.041 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789", 2024-09-10T21:27:42.042 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:27:42.042 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:27:42.042 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:27:42.042 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:27:42.042 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789/0", 2024-09-10T21:27:42.042 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6789/0", 2024-09-10T21:27:42.042 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:27:42.042 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:27:42.042 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:27:42.042 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:27:42.042 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:27:42.042 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:27:42.042 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:27:42.043 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:27:42.043 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:27:42.043 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:27:42.043 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:27:42.043 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3301", 2024-09-10T21:27:42.043 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:27:42.043 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:27:42.043 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:27:42.043 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:27:42.043 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790", 2024-09-10T21:27:42.043 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:27:42.043 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:27:42.043 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:27:42.043 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:27:42.044 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790/0", 2024-09-10T21:27:42.044 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6790/0", 2024-09-10T21:27:42.044 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:27:42.044 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:27:42.044 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:27:42.044 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:27:42.044 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:27:42.044 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:27:42.044 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-10T21:27:42.044 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-10T21:27:42.044 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:27:42.044 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:27:42.044 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:27:42.045 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:27:42.045 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:27:42.045 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:27:42.045 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": [ 2024-09-10T21:27:42.045 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:27:42.045 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:27:42.045 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:27:42.045 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:27:42.045 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:27:42.045 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:27:42.045 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-10T21:27:42.045 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:27:42.045 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:27:42.045 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:27:42.046 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 4 2024-09-10T21:27:42.046 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:27:42.046 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:27:42.046 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-10T21:27:42.046 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:27:42.046 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:27:42.046 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:27:42.046 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 3 2024-09-10T21:27:42.046 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:27:42.046 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:27:42.046 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:27:42.046 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-10T21:27:42.046 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-10T21:27:42.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:42.029+0000 7fbd85400640 1 -- 172.21.15.17:0/1533611036 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fbd60072590 msgr2=0x7fbd60074a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:42.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:42.029+0000 7fbd85400640 1 --2- 172.21.15.17:0/1533611036 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fbd60072590 0x7fbd60074a50 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fbd78001430 tx=0x7fbd7800d040 comp rx=0 tx=0).stop 2024-09-10T21:27:42.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:42.029+0000 7fbd85400640 1 -- 172.21.15.17:0/1533611036 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbd80152450 msgr2=0x7fbd8014cb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:27:42.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:42.029+0000 7fbd85400640 1 --2- 172.21.15.17:0/1533611036 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbd80152450 0x7fbd8014cb40 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fbd6c00b4d0 tx=0x7fbd6c00b990 comp rx=0 tx=0).stop 2024-09-10T21:27:42.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:42.029+0000 7fbd84a00640 1 -- 172.21.15.17:0/1533611036 reap_dead start 2024-09-10T21:27:42.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:42.029+0000 7fbd85400640 1 -- 172.21.15.17:0/1533611036 shutdown_connections 2024-09-10T21:27:42.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:42.029+0000 7fbd85400640 1 --2- 172.21.15.17:0/1533611036 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbd80152450 0x7fbd8014cb40 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:27:42.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:42.029+0000 7fbd85400640 1 -- 172.21.15.17:0/1533611036 >> 172.21.15.17:0/1533611036 conn(0x7fbd8006cc20 msgr2=0x7fbd800cc930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:27:42.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:42.029+0000 7fbd85400640 1 -- 172.21.15.17:0/1533611036 shutdown_connections 2024-09-10T21:27:42.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:27:42.029+0000 7fbd85400640 1 -- 172.21.15.17:0/1533611036 wait complete. 2024-09-10T21:27:42.053 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-09-10T21:27:42.054 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2024-09-10T21:27:45.783 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:27:45.786 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:28:02.054 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.a 2024-09-10T21:28:02.054 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-10T21:28:02.054 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-10T21:28:02.058 INFO:tasks.ceph.mon.a:Started 2024-09-10T21:28:02.058 INFO:tasks.mon_thrash:Sending STOP to mon c 2024-09-10T21:28:02.058 INFO:tasks.ceph.mon.c:Sent signal 19 2024-09-10T21:28:02.058 INFO:tasks.mon_thrash.mon_thrasher:waiting for 15.0 secs to unfreeze mons 2024-09-10T21:28:15.844 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:28:15.848 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:28:17.059 INFO:tasks.mon_thrash:Sending CONT to mon c 2024-09-10T21:28:17.060 INFO:tasks.ceph.mon.c:Sent signal 18 2024-09-10T21:28:17.060 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-09-10T21:28:17.060 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-10T21:28:17.483 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:17.476+0000 7f93e3200640 1 -- 172.21.15.17:0/2914768152 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f93c8004d50 msgr2=0x7f93c80051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:17.484 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:17.476+0000 7f93e3200640 1 --2- 172.21.15.17:0/2914768152 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f93c8004d50 0x7f93c80051d0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f93dc005fd0 tx=0x7f93dc01d750 comp rx=0 tx=0).stop 2024-09-10T21:28:17.484 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:17.476+0000 7f93e2800640 1 -- 172.21.15.17:0/2914768152 reap_dead start 2024-09-10T21:28:17.484 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:17.476+0000 7f93e3200640 1 -- 172.21.15.17:0/2914768152 shutdown_connections 2024-09-10T21:28:17.484 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:17.476+0000 7f93e3200640 1 -- 172.21.15.17:0/2914768152 >> 172.21.15.17:0/2914768152 conn(0x7f93e4070ca0 msgr2=0x7f93e40710b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:28:17.484 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:17.476+0000 7f93e3200640 1 -- 172.21.15.17:0/2914768152 shutdown_connections 2024-09-10T21:28:17.484 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:17.476+0000 7f93e3200640 1 -- 172.21.15.17:0/2914768152 wait complete. 2024-09-10T21:28:17.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:17.480+0000 7f93e3200640 1 Processor -- start 2024-09-10T21:28:17.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:17.480+0000 7f93e3200640 1 -- start start 2024-09-10T21:28:17.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:17.480+0000 7f93e3200640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f93c8004d50 0x7f93e414cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:17.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:17.480+0000 7f93e3200640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f93e406d620 0x7f93e40c5be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:17.488 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:17.480+0000 7f93e1e00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f93c8004d50 0x7f93e414cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:17.488 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:17.480+0000 7f93e1e00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f93c8004d50 0x7f93e414cfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:60468/0 (socket says 172.21.15.17:60468) 2024-09-10T21:28:17.488 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:17.480+0000 7f93e3200640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93e406ddb0 0x7f93e40c6120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:17.488 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:17.480+0000 7f93e3200640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f93e415b050 con 0x7f93c8004d50 2024-09-10T21:28:17.488 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:17.480+0000 7f93e3200640 1 -- 172.21.15.17:0/1243978055 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f93e406ff30 con 0x7f93e406ddb0 2024-09-10T21:28:17.488 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:17.480+0000 7f93e1e00640 1 -- 172.21.15.17:0/1243978055 learned_addr learned my addr 172.21.15.17:0/1243978055 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:28:17.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:17.480+0000 7f93e3200640 1 -- 172.21.15.17:0/1243978055 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f93e415b1d0 con 0x7f93e406d620 2024-09-10T21:28:17.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:17.480+0000 7f93e1400640 1 --2- 172.21.15.17:0/1243978055 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f93e406d620 0x7f93e40c5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:17.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:17.480+0000 7f93e1e00640 1 -- 172.21.15.17:0/1243978055 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f93e406d620 msgr2=0x7f93e40c5be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:17.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:17.480+0000 7f93e1e00640 1 --2- 172.21.15.17:0/1243978055 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f93e406d620 0x7f93e40c5be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:17.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:17.480+0000 7f93e1e00640 1 -- 172.21.15.17:0/1243978055 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93e406ddb0 msgr2=0x7f93e40c6120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:17.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:17.480+0000 7f93e1e00640 1 --2- 172.21.15.17:0/1243978055 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93e406ddb0 0x7f93e40c6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:17.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:17.480+0000 7f93e1e00640 1 -- 172.21.15.17:0/1243978055 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93e40c5450 con 0x7f93c8004d50 2024-09-10T21:28:17.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:17.480+0000 7f93e1e00640 1 --2- 172.21.15.17:0/1243978055 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f93c8004d50 0x7f93e414cfd0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f93d4003600 tx=0x7f93d4003ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:28:17.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:17.480+0000 7f93da000640 1 -- 172.21.15.17:0/1243978055 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93d4012ad0 con 0x7f93c8004d50 2024-09-10T21:28:17.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:17.484+0000 7f93da000640 1 -- 172.21.15.17:0/1243978055 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f93d4012c70 con 0x7f93c8004d50 2024-09-10T21:28:17.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:17.484+0000 7f93e3200640 1 -- 172.21.15.17:0/1243978055 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93e41f0020 con 0x7f93c8004d50 2024-09-10T21:28:17.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:17.484+0000 7f93da000640 1 -- 172.21.15.17:0/1243978055 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93d4012e60 con 0x7f93c8004d50 2024-09-10T21:28:17.490 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:17.484+0000 7f93e3200640 1 -- 172.21.15.17:0/1243978055 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93e406e730 con 0x7f93c8004d50 2024-09-10T21:28:17.490 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:17.484+0000 7f93e3200640 1 -- 172.21.15.17:0/1243978055 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93e40cc200 con 0x7f93c8004d50 2024-09-10T21:28:17.490 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:17.484+0000 7f93da000640 1 -- 172.21.15.17:0/1243978055 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f93d4019070 con 0x7f93c8004d50 2024-09-10T21:28:17.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:17.484+0000 7f93da000640 1 --2- 172.21.15.17:0/1243978055 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f93c4072610 0x7f93c4074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:17.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:17.488+0000 7f93da000640 1 -- 172.21.15.17:0/1243978055 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f93d40945c0 con 0x7f93c8004d50 2024-09-10T21:28:17.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:17.488+0000 7f93e1400640 1 --2- 172.21.15.17:0/1243978055 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f93c4072610 0x7f93c4074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:17.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:17.488+0000 7f93e1400640 1 --2- 172.21.15.17:0/1243978055 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f93c4072610 0x7f93c4074ad0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f93d0001000 tx=0x7f93d0006730 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:28:17.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:17.496+0000 7f93da000640 1 -- 172.21.15.17:0/1243978055 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f93d405f890 con 0x7f93c8004d50 2024-09-10T21:28:17.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:17.776+0000 7f93e3200640 1 -- 172.21.15.17:0/1243978055 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f93e4071ce0 con 0x7f93c8004d50 2024-09-10T21:28:17.784 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:17.776+0000 7f93da000640 1 -- 172.21.15.17:0/1243978055 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1556 (secure 0 0 0) 0x7f93d405f190 con 0x7f93c8004d50 2024-09-10T21:28:17.784 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":30,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5e2f023f-d6a6-49f4-a256-f4807dce3739","modified":"2024-09-10T21:23:46.892130Z","created":"2024-09-10T21:23:46.892130Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3301","nonce":0},{"type":"v1","addr":"172.21.15.31:6790","nonce":0}]},"addr":"172.21.15.31:6790/0","public_addr":"172.21.15.31:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-10T21:28:17.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:17.780+0000 7f93e3200640 1 -- 172.21.15.17:0/1243978055 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f93c4072610 msgr2=0x7f93c4074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:17.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:17.780+0000 7f93e3200640 1 --2- 172.21.15.17:0/1243978055 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f93c4072610 0x7f93c4074ad0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f93d0001000 tx=0x7f93d0006730 comp rx=0 tx=0).stop 2024-09-10T21:28:17.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:17.780+0000 7f93e3200640 1 -- 172.21.15.17:0/1243978055 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f93c8004d50 msgr2=0x7f93e414cfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:17.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:17.780+0000 7f93e3200640 1 --2- 172.21.15.17:0/1243978055 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f93c8004d50 0x7f93e414cfd0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f93d4003600 tx=0x7f93d4003ac0 comp rx=0 tx=0).stop 2024-09-10T21:28:17.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:17.780+0000 7f93e3200640 1 -- 172.21.15.17:0/1243978055 shutdown_connections 2024-09-10T21:28:17.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:17.780+0000 7f93e3200640 1 --2- 172.21.15.17:0/1243978055 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f93c4072610 0x7f93c4074ad0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:17.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:17.780+0000 7f93e3200640 1 --2- 172.21.15.17:0/1243978055 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f93e406ddb0 0x7f93e40c6120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:17.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:17.780+0000 7f93e3200640 1 --2- 172.21.15.17:0/1243978055 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f93e406d620 0x7f93e40c5be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:17.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:17.780+0000 7f93e3200640 1 --2- 172.21.15.17:0/1243978055 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f93c8004d50 0x7f93e414cfd0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:17.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:17.780+0000 7f93e3200640 1 -- 172.21.15.17:0/1243978055 >> 172.21.15.17:0/1243978055 conn(0x7f93e4070ca0 msgr2=0x7f93e4149f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:28:17.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:17.780+0000 7f93e3200640 1 -- 172.21.15.17:0/1243978055 shutdown_connections 2024-09-10T21:28:17.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:17.780+0000 7f93e3200640 1 -- 172.21.15.17:0/1243978055 wait complete. 2024-09-10T21:28:17.809 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-09-10T21:28:17.809 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-09-10T21:28:17.809 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-10T21:28:17.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:17.992+0000 7fdb6f200640 1 -- 172.21.15.17:0/1433913356 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdb700c9520 msgr2=0x7fdb70162aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:17.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:17.992+0000 7fdb6f200640 1 --2- 172.21.15.17:0/1433913356 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdb700c9520 0x7fdb70162aa0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fdb68009bc0 tx=0x7fdb6801ca80 comp rx=0 tx=0).stop 2024-09-10T21:28:17.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:17.992+0000 7fdb6f200640 1 -- 172.21.15.17:0/1433913356 shutdown_connections 2024-09-10T21:28:17.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:17.992+0000 7fdb6f200640 1 --2- 172.21.15.17:0/1433913356 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdb700c9520 0x7fdb70162aa0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:17.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:17.992+0000 7fdb6f200640 1 --2- 172.21.15.17:0/1433913356 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fdb701523d0 0x7fdb700c9f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:17.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:17.992+0000 7fdb6f200640 1 -- 172.21.15.17:0/1433913356 >> 172.21.15.17:0/1433913356 conn(0x7fdb70068990 msgr2=0x7fdb70068da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:28:18.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:17.992+0000 7fdb6f200640 1 -- 172.21.15.17:0/1433913356 shutdown_connections 2024-09-10T21:28:18.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:17.992+0000 7fdb6f200640 1 -- 172.21.15.17:0/1433913356 wait complete. 2024-09-10T21:28:18.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:17.996+0000 7fdb6f200640 1 Processor -- start 2024-09-10T21:28:18.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:17.996+0000 7fdb6f200640 1 -- start start 2024-09-10T21:28:18.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:17.996+0000 7fdb6f200640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fdb700c9520 0x7fdb70159710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:18.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:17.996+0000 7fdb6f200640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdb700ca4a0 0x7fdb7014b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:18.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:17.996+0000 7fdb6f200640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdb701523d0 0x7fdb70157500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:18.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:17.996+0000 7fdb6f200640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fdb70071320 con 0x7fdb700ca4a0 2024-09-10T21:28:18.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:17.996+0000 7fdb6f200640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fdb700cb3c0 con 0x7fdb701523d0 2024-09-10T21:28:18.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:17.996+0000 7fdb6f200640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7fdb700714a0 con 0x7fdb700c9520 2024-09-10T21:28:18.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:17.996+0000 7fdb6d400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdb700ca4a0 0x7fdb7014b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:18.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:17.996+0000 7fdb6d400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdb700ca4a0 0x7fdb7014b620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:60476/0 (socket says 172.21.15.17:60476) 2024-09-10T21:28:18.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:17.996+0000 7fdb6d400640 1 -- 172.21.15.17:0/2251599758 learned_addr learned my addr 172.21.15.17:0/2251599758 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:28:18.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:17.996+0000 7fdb6de00640 1 --2- 172.21.15.17:0/2251599758 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fdb700c9520 0x7fdb70159710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:18.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:17.996+0000 7fdb6e800640 1 --2- 172.21.15.17:0/2251599758 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdb701523d0 0x7fdb70157500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:18.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:17.996+0000 7fdb6d400640 1 -- 172.21.15.17:0/2251599758 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fdb700c9520 msgr2=0x7fdb70159710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:18.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:17.996+0000 7fdb6d400640 1 --2- 172.21.15.17:0/2251599758 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fdb700c9520 0x7fdb70159710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:18.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:17.996+0000 7fdb6d400640 1 -- 172.21.15.17:0/2251599758 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdb701523d0 msgr2=0x7fdb70157500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:18.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:17.996+0000 7fdb6d400640 1 --2- 172.21.15.17:0/2251599758 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdb701523d0 0x7fdb70157500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:18.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:17.996+0000 7fdb6d400640 1 -- 172.21.15.17:0/2251599758 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb701eefa0 con 0x7fdb700ca4a0 2024-09-10T21:28:18.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:17.996+0000 7fdb6d400640 1 --2- 172.21.15.17:0/2251599758 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdb700ca4a0 0x7fdb7014b620 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fdb5c00c400 tx=0x7fdb5c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:28:18.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:17.996+0000 7fdb66000640 1 -- 172.21.15.17:0/2251599758 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb5c011790 con 0x7fdb700ca4a0 2024-09-10T21:28:18.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:17.996+0000 7fdb66000640 1 -- 172.21.15.17:0/2251599758 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdb5c017070 con 0x7fdb700ca4a0 2024-09-10T21:28:18.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:17.996+0000 7fdb66000640 1 -- 172.21.15.17:0/2251599758 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb5c011d00 con 0x7fdb700ca4a0 2024-09-10T21:28:18.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:17.996+0000 7fdb6f200640 1 -- 172.21.15.17:0/2251599758 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb701eeb80 con 0x7fdb700ca4a0 2024-09-10T21:28:18.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:17.996+0000 7fdb6f200640 1 -- 172.21.15.17:0/2251599758 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb7015aa20 con 0x7fdb700ca4a0 2024-09-10T21:28:18.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.000+0000 7fdb6f200640 1 --2- 172.21.15.17:0/2251599758 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdb7015b220 0x7fdb70157a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:18.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.000+0000 7fdb6f200640 1 -- 172.21.15.17:0/2251599758 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fdb70069ef0 con 0x7fdb7015b220 2024-09-10T21:28:18.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.000+0000 7fdb6de00640 1 --2- 172.21.15.17:0/2251599758 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdb7015b220 0x7fdb70157a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:18.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.000+0000 7fdb6de00640 1 -- 172.21.15.17:0/2251599758 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fdb701ef4b0 con 0x7fdb7015b220 2024-09-10T21:28:18.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.000+0000 7fdb6de00640 1 --2- 172.21.15.17:0/2251599758 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdb7015b220 0x7fdb70157a40 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fdb600119e0 tx=0x7fdb60011ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:28:18.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.000+0000 7fdb66000640 1 -- 172.21.15.17:0/2251599758 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb60004300 con 0x7fdb7015b220 2024-09-10T21:28:18.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.000+0000 7fdb66000640 1 -- 172.21.15.17:0/2251599758 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fdb5c01c460 con 0x7fdb700ca4a0 2024-09-10T21:28:18.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.000+0000 7fdb66000640 1 --2- 172.21.15.17:0/2251599758 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fdb500724e0 0x7fdb500749a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:18.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.000+0000 7fdb66000640 1 -- 172.21.15.17:0/2251599758 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fdb600044a0 con 0x7fdb7015b220 2024-09-10T21:28:18.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.000+0000 7fdb66000640 1 -- 172.21.15.17:0/2251599758 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdb7015b220 msgr2=0x7fdb70157a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:18.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.000+0000 7fdb66000640 1 --2- 172.21.15.17:0/2251599758 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdb7015b220 0x7fdb70157a40 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fdb600119e0 tx=0x7fdb60011ea0 comp rx=0 tx=0).stop 2024-09-10T21:28:18.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.000+0000 7fdb66000640 1 -- 172.21.15.17:0/2251599758 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdb7015b220 msgr2=0x7fdb70157a40 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:28:18.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.000+0000 7fdb66000640 1 --2- 172.21.15.17:0/2251599758 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdb7015b220 0x7fdb70157a40 secure :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fdb600119e0 tx=0x7fdb60011ea0 comp rx=0 tx=0).stop 2024-09-10T21:28:18.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.000+0000 7fdb66000640 1 -- 172.21.15.17:0/2251599758 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fdb5c0933a0 con 0x7fdb700ca4a0 2024-09-10T21:28:18.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.000+0000 7fdb6e800640 1 --2- 172.21.15.17:0/2251599758 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fdb500724e0 0x7fdb500749a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:18.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.004+0000 7fdb6e800640 1 --2- 172.21.15.17:0/2251599758 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fdb500724e0 0x7fdb500749a0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fdb680af000 tx=0x7fdb6801b2c0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:28:18.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.008+0000 7fdb6f200640 1 --2- 172.21.15.17:0/2251599758 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdb7015ac60 0x7fdb70155b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:18.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.008+0000 7fdb6f200640 1 -- 172.21.15.17:0/2251599758 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fdb70157240 con 0x7fdb7015ac60 2024-09-10T21:28:18.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.012+0000 7fdb6de00640 1 --2- 172.21.15.17:0/2251599758 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdb7015ac60 0x7fdb70155b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:18.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.012+0000 7fdb6de00640 1 -- 172.21.15.17:0/2251599758 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fdb7014ecf0 con 0x7fdb7015ac60 2024-09-10T21:28:18.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.012+0000 7fdb6de00640 1 --2- 172.21.15.17:0/2251599758 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdb7015ac60 0x7fdb70155b40 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fdb600119e0 tx=0x7fdb600046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:28:18.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.012+0000 7fdb66000640 1 -- 172.21.15.17:0/2251599758 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb600054b0 con 0x7fdb7015ac60 2024-09-10T21:28:18.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.012+0000 7fdb66000640 1 -- 172.21.15.17:0/2251599758 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4542 (secure 0 0 0) 0x7fdb600054b0 con 0x7fdb7015ac60 2024-09-10T21:28:18.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.012+0000 7fdb66000640 1 -- 172.21.15.17:0/2251599758 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdb7015ac60 msgr2=0x7fdb70155b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:18.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.012+0000 7fdb66000640 1 --2- 172.21.15.17:0/2251599758 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdb7015ac60 0x7fdb70155b40 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fdb600119e0 tx=0x7fdb600046d0 comp rx=0 tx=0).stop 2024-09-10T21:28:18.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.012+0000 7fdb66000640 1 -- 172.21.15.17:0/2251599758 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdb7015ac60 msgr2=0x7fdb70155b40 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:28:18.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.012+0000 7fdb66000640 1 --2- 172.21.15.17:0/2251599758 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdb7015ac60 0x7fdb70155b40 secure :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fdb600119e0 tx=0x7fdb600046d0 comp rx=0 tx=0).stop 2024-09-10T21:28:18.018 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-10T21:28:18.018 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:28:18.018 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:28:18.018 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "leader", 2024-09-10T21:28:18.018 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 30, 2024-09-10T21:28:18.018 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 15851, 2024-09-10T21:28:18.018 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-10T21:28:18.019 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-10T21:28:18.019 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-10T21:28:18.019 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-10T21:28:18.019 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:28:18.019 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 0, 2024-09-10T21:28:18.019 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:28:18.019 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-10T21:28:18.019 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-10T21:28:18.019 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:28:18.019 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:28:18.019 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:28:18.019 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:28:18.020 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:28:18.020 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:28:18.020 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:28:18.020 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:28:18.020 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:28:18.020 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:28:18.020 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:28:18.020 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:28:18.020 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-10T21:28:18.020 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-10T21:28:18.020 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:28:18.020 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:28:18.021 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:28:18.021 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:28:18.021 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:28:18.021 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:28:18.021 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:28:18.021 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:28:18.021 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:28:18.021 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:28:18.021 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:28:18.021 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:28:18.021 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:28:18.021 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-10T21:28:18.022 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-10T21:28:18.022 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-10T21:28:18.022 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-10T21:28:18.022 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-10T21:28:18.022 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "5e2f023f-d6a6-49f4-a256-f4807dce3739", 2024-09-10T21:28:18.022 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:28:18.022 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:28:18.022 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-10T21:28:18.022 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-10T21:28:18.022 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-10T21:28:18.022 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-10T21:28:18.022 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-10T21:28:18.022 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-10T21:28:18.022 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-10T21:28:18.022 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:28:18.023 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-10T21:28:18.023 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:28:18.023 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:28:18.023 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:28:18.023 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:28:18.023 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:28:18.023 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:28:18.023 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:28:18.023 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:28:18.023 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:28:18.023 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:28:18.023 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:28:18.023 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:28:18.023 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-10T21:28:18.024 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:28:18.024 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-10T21:28:18.024 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:28:18.024 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:28:18.024 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:28:18.024 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:28:18.024 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:28:18.024 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:28:18.024 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:28:18.024 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-10T21:28:18.024 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:28:18.024 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:28:18.024 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:28:18.025 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:28:18.025 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-10T21:28:18.025 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:28:18.025 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:28:18.025 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:28:18.025 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:28:18.025 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-10T21:28:18.025 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-10T21:28:18.025 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:28:18.025 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:28:18.025 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:28:18.025 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:28:18.025 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:28:18.025 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:28:18.026 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:28:18.026 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:28:18.026 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:28:18.026 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:28:18.026 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:28:18.026 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3300", 2024-09-10T21:28:18.026 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:28:18.026 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:28:18.026 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:28:18.026 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:28:18.026 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789", 2024-09-10T21:28:18.026 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:28:18.026 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:28:18.026 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:28:18.027 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:28:18.027 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789/0", 2024-09-10T21:28:18.027 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6789/0", 2024-09-10T21:28:18.027 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:28:18.027 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:28:18.027 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:28:18.027 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:28:18.027 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:28:18.027 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:28:18.027 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:28:18.027 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:28:18.027 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:28:18.027 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:28:18.027 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:28:18.027 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3301", 2024-09-10T21:28:18.027 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:28:18.028 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:28:18.028 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:28:18.028 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:28:18.028 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790", 2024-09-10T21:28:18.028 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:28:18.028 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:28:18.028 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:28:18.028 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:28:18.028 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790/0", 2024-09-10T21:28:18.028 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6790/0", 2024-09-10T21:28:18.028 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:28:18.028 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:28:18.028 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:28:18.028 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:28:18.029 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:28:18.029 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:28:18.029 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-10T21:28:18.029 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-10T21:28:18.029 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:28:18.029 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:28:18.029 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:28:18.029 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:28:18.029 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:28:18.029 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:28:18.029 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-10T21:28:18.029 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:28:18.029 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:28:18.029 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:28:18.029 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 2 2024-09-10T21:28:18.030 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:28:18.030 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:28:18.030 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:28:18.030 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-10T21:28:18.030 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-10T21:28:18.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.012+0000 7fdb6f200640 1 -- 172.21.15.17:0/2251599758 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fdb500724e0 msgr2=0x7fdb500749a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:18.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.012+0000 7fdb6f200640 1 --2- 172.21.15.17:0/2251599758 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fdb500724e0 0x7fdb500749a0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fdb680af000 tx=0x7fdb6801b2c0 comp rx=0 tx=0).stop 2024-09-10T21:28:18.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.012+0000 7fdb6f200640 1 -- 172.21.15.17:0/2251599758 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdb700ca4a0 msgr2=0x7fdb7014b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:18.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.012+0000 7fdb6f200640 1 --2- 172.21.15.17:0/2251599758 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdb700ca4a0 0x7fdb7014b620 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fdb5c00c400 tx=0x7fdb5c00c8c0 comp rx=0 tx=0).stop 2024-09-10T21:28:18.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.012+0000 7fdb6e800640 1 -- 172.21.15.17:0/2251599758 reap_dead start 2024-09-10T21:28:18.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.012+0000 7fdb6e800640 1 -- 172.21.15.17:0/2251599758 reap_dead start 2024-09-10T21:28:18.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.012+0000 7fdb6f200640 1 -- 172.21.15.17:0/2251599758 shutdown_connections 2024-09-10T21:28:18.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.012+0000 7fdb6f200640 1 -- 172.21.15.17:0/2251599758 >> 172.21.15.17:0/2251599758 conn(0x7fdb70068990 msgr2=0x7fdb700b2c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:28:18.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.012+0000 7fdb6f200640 1 -- 172.21.15.17:0/2251599758 shutdown_connections 2024-09-10T21:28:18.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.012+0000 7fdb6f200640 1 -- 172.21.15.17:0/2251599758 wait complete. 2024-09-10T21:28:18.039 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-10T21:28:18.039 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-09-10T21:28:18.039 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-10T21:28:18.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.192+0000 7f37f8a00640 1 -- 172.21.15.17:0/4221536224 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f37f4158f60 msgr2=0x7f37f4163bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:18.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.192+0000 7f37f8a00640 1 --2- 172.21.15.17:0/4221536224 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f37f4158f60 0x7f37f4163bf0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f37ec009bf0 tx=0x7f37ec01f2f0 comp rx=0 tx=0).stop 2024-09-10T21:28:18.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.192+0000 7f37f8a00640 1 -- 172.21.15.17:0/4221536224 shutdown_connections 2024-09-10T21:28:18.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.192+0000 7f37f8a00640 1 --2- 172.21.15.17:0/4221536224 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f37f4158f60 0x7f37f4163bf0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:18.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.192+0000 7f37f8a00640 1 -- 172.21.15.17:0/4221536224 >> 172.21.15.17:0/4221536224 conn(0x7f37f406cc20 msgr2=0x7f37f406d030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:28:18.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.192+0000 7f37f8a00640 1 -- 172.21.15.17:0/4221536224 shutdown_connections 2024-09-10T21:28:18.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.192+0000 7f37f8a00640 1 -- 172.21.15.17:0/4221536224 wait complete. 2024-09-10T21:28:18.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.192+0000 7f37f8a00640 1 Processor -- start 2024-09-10T21:28:18.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.192+0000 7f37f8a00640 1 -- start start 2024-09-10T21:28:18.201 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.196+0000 7f37f8a00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f37f4152070 0x7f37f41de340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:18.201 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.196+0000 7f37f8a00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f37f4152450 0x7f37f41f06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:18.201 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.196+0000 7f37f8a00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f37f4158f60 0x7f37f40c9930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:18.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.196+0000 7f37f8a00640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f37f40cecf0 con 0x7f37f4158f60 2024-09-10T21:28:18.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.196+0000 7f37f8a00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f37f40c5090 con 0x7f37f4152070 2024-09-10T21:28:18.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.196+0000 7f37f8a00640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f37f40cee70 con 0x7f37f4152450 2024-09-10T21:28:18.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.196+0000 7f37f3200640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f37f4158f60 0x7f37f40c9930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:18.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.196+0000 7f37f3200640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f37f4158f60 0x7f37f40c9930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:60492/0 (socket says 172.21.15.17:60492) 2024-09-10T21:28:18.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.196+0000 7f37f3200640 1 -- 172.21.15.17:0/3492366975 learned_addr learned my addr 172.21.15.17:0/3492366975 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:28:18.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.196+0000 7f37f2800640 1 --2- 172.21.15.17:0/3492366975 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f37f4152070 0x7f37f41de340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:18.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.196+0000 7f37f1e00640 1 --2- 172.21.15.17:0/3492366975 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f37f4152450 0x7f37f41f06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:18.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.196+0000 7f37f3200640 1 -- 172.21.15.17:0/3492366975 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f37f4152450 msgr2=0x7f37f41f06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:18.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.196+0000 7f37f3200640 1 --2- 172.21.15.17:0/3492366975 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f37f4152450 0x7f37f41f06c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:18.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.196+0000 7f37f3200640 1 -- 172.21.15.17:0/3492366975 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f37f4152070 msgr2=0x7f37f41de340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:18.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.196+0000 7f37f3200640 1 --2- 172.21.15.17:0/3492366975 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f37f4152070 0x7f37f41de340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:18.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.196+0000 7f37f3200640 1 -- 172.21.15.17:0/3492366975 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37f4158180 con 0x7f37f4158f60 2024-09-10T21:28:18.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.196+0000 7f37f3200640 1 --2- 172.21.15.17:0/3492366975 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f37f4158f60 0x7f37f40c9930 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f37ec01d9a0 tx=0x7f37ec003f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:28:18.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.196+0000 7f37eaa00640 1 -- 172.21.15.17:0/3492366975 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37ec0b4040 con 0x7f37f4158f60 2024-09-10T21:28:18.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.196+0000 7f37f8a00640 1 -- 172.21.15.17:0/3492366975 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37f4157ee0 con 0x7f37f4158f60 2024-09-10T21:28:18.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.196+0000 7f37eaa00640 1 -- 172.21.15.17:0/3492366975 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f37ec0bf070 con 0x7f37f4158f60 2024-09-10T21:28:18.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.196+0000 7f37eaa00640 1 -- 172.21.15.17:0/3492366975 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37ec0afc10 con 0x7f37f4158f60 2024-09-10T21:28:18.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.196+0000 7f37f8a00640 1 -- 172.21.15.17:0/3492366975 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37f41de880 con 0x7f37f4158f60 2024-09-10T21:28:18.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.196+0000 7f37f8a00640 1 --2- 172.21.15.17:0/3492366975 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f37f406fab0 0x7f37f41f4700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:18.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.196+0000 7f37f8a00640 1 -- 172.21.15.17:0/3492366975 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f37f41563d0 con 0x7f37f406fab0 2024-09-10T21:28:18.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.196+0000 7f37f2800640 1 --2- 172.21.15.17:0/3492366975 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f37f406fab0 0x7f37f41f4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:18.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.200+0000 7f37f2800640 1 -- 172.21.15.17:0/3492366975 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f37f4072b80 con 0x7f37f406fab0 2024-09-10T21:28:18.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.200+0000 7f37eaa00640 1 -- 172.21.15.17:0/3492366975 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f37ec004040 con 0x7f37f4158f60 2024-09-10T21:28:18.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.200+0000 7f37f2800640 1 --2- 172.21.15.17:0/3492366975 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f37f406fab0 0x7f37f41f4700 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f37e4001230 tx=0x7f37e400feb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:28:18.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.200+0000 7f37eaa00640 1 --2- 172.21.15.17:0/3492366975 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f37d40725e0 0x7f37d4074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:18.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.200+0000 7f37eaa00640 1 -- 172.21.15.17:0/3492366975 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f37ec1314f0 con 0x7f37f4158f60 2024-09-10T21:28:18.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.200+0000 7f37f1e00640 1 --2- 172.21.15.17:0/3492366975 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f37d40725e0 0x7f37d4074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:18.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.200+0000 7f37eaa00640 1 -- 172.21.15.17:0/3492366975 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37e4018020 con 0x7f37f406fab0 2024-09-10T21:28:18.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.200+0000 7f37eaa00640 1 -- 172.21.15.17:0/3492366975 <== mon.1 v2:172.21.15.31:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f37e400d860 con 0x7f37f406fab0 2024-09-10T21:28:18.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.200+0000 7f37eaa00640 1 -- 172.21.15.17:0/3492366975 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f37f406fab0 msgr2=0x7f37f41f4700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:18.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.200+0000 7f37eaa00640 1 --2- 172.21.15.17:0/3492366975 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f37f406fab0 0x7f37f41f4700 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f37e4001230 tx=0x7f37e400feb0 comp rx=0 tx=0).stop 2024-09-10T21:28:18.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.200+0000 7f37eaa00640 1 -- 172.21.15.17:0/3492366975 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f37f406fab0 msgr2=0x7f37f41f4700 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:28:18.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.200+0000 7f37eaa00640 1 --2- 172.21.15.17:0/3492366975 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f37f406fab0 0x7f37f41f4700 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:18.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.200+0000 7f37f1e00640 1 --2- 172.21.15.17:0/3492366975 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f37d40725e0 0x7f37d4074aa0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f37f40ceab0 tx=0x7f37e0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:28:18.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.208+0000 7f37f8a00640 1 --2- 172.21.15.17:0/3492366975 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f37f41515b0 0x7f37f41497f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:18.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.208+0000 7f37f8a00640 1 -- 172.21.15.17:0/3492366975 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f37f406f810 con 0x7f37f41515b0 2024-09-10T21:28:18.214 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.208+0000 7f37f2800640 1 --2- 172.21.15.17:0/3492366975 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f37f41515b0 0x7f37f41497f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:18.214 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.208+0000 7f37f2800640 1 -- 172.21.15.17:0/3492366975 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f37f4075bd0 con 0x7f37f41515b0 2024-09-10T21:28:18.214 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.208+0000 7f37f2800640 1 --2- 172.21.15.17:0/3492366975 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f37f41515b0 0x7f37f41497f0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f37e4019000 tx=0x7f37e4004610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:28:18.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.208+0000 7f37eaa00640 1 -- 172.21.15.17:0/3492366975 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37e4018040 con 0x7f37f41515b0 2024-09-10T21:28:18.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.208+0000 7f37eaa00640 1 -- 172.21.15.17:0/3492366975 <== mon.1 v2:172.21.15.31:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5042 (secure 0 0 0) 0x7f37e401f220 con 0x7f37f41515b0 2024-09-10T21:28:18.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.208+0000 7f37eaa00640 1 -- 172.21.15.17:0/3492366975 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f37f41515b0 msgr2=0x7f37f41497f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:18.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.208+0000 7f37eaa00640 1 --2- 172.21.15.17:0/3492366975 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f37f41515b0 0x7f37f41497f0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f37e4019000 tx=0x7f37e4004610 comp rx=0 tx=0).stop 2024-09-10T21:28:18.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.208+0000 7f37eaa00640 1 -- 172.21.15.17:0/3492366975 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f37f41515b0 msgr2=0x7f37f41497f0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:28:18.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.208+0000 7f37eaa00640 1 --2- 172.21.15.17:0/3492366975 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f37f41515b0 0x7f37f41497f0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:18.216 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-10T21:28:18.216 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:28:18.216 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:28:18.216 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-10T21:28:18.216 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 30, 2024-09-10T21:28:18.216 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 236659, 2024-09-10T21:28:18.216 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-10T21:28:18.216 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-10T21:28:18.216 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-10T21:28:18.217 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-10T21:28:18.217 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:28:18.217 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 0, 2024-09-10T21:28:18.217 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:28:18.217 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-10T21:28:18.217 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-10T21:28:18.217 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:28:18.217 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:28:18.217 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:28:18.217 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:28:18.217 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:28:18.217 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:28:18.217 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:28:18.217 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:28:18.218 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:28:18.218 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:28:18.218 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:28:18.218 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:28:18.218 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-10T21:28:18.218 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-10T21:28:18.218 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:28:18.218 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:28:18.218 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:28:18.218 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:28:18.218 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:28:18.218 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:28:18.218 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:28:18.219 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:28:18.219 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:28:18.219 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:28:18.219 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:28:18.219 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:28:18.219 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:28:18.219 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-10T21:28:18.219 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-10T21:28:18.219 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-10T21:28:18.219 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-10T21:28:18.219 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-10T21:28:18.219 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "5e2f023f-d6a6-49f4-a256-f4807dce3739", 2024-09-10T21:28:18.219 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:28:18.220 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:28:18.220 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-10T21:28:18.220 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-10T21:28:18.220 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-10T21:28:18.220 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-10T21:28:18.220 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-10T21:28:18.220 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-10T21:28:18.220 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-10T21:28:18.220 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:28:18.220 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-10T21:28:18.220 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:28:18.220 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:28:18.220 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:28:18.221 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:28:18.221 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:28:18.221 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:28:18.221 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:28:18.221 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:28:18.221 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:28:18.221 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:28:18.221 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:28:18.221 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:28:18.221 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-10T21:28:18.221 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:28:18.221 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-10T21:28:18.222 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:28:18.222 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:28:18.222 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:28:18.222 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:28:18.222 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:28:18.222 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:28:18.222 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:28:18.222 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-10T21:28:18.222 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:28:18.222 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:28:18.222 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:28:18.222 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:28:18.222 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-10T21:28:18.223 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:28:18.223 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:28:18.223 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:28:18.223 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:28:18.223 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-10T21:28:18.223 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-10T21:28:18.223 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:28:18.223 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:28:18.223 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:28:18.223 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:28:18.223 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:28:18.223 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:28:18.223 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:28:18.224 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:28:18.224 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:28:18.224 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:28:18.224 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:28:18.224 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3300", 2024-09-10T21:28:18.224 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:28:18.224 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:28:18.224 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:28:18.224 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:28:18.224 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789", 2024-09-10T21:28:18.224 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:28:18.224 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:28:18.224 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:28:18.224 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:28:18.225 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789/0", 2024-09-10T21:28:18.225 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6789/0", 2024-09-10T21:28:18.225 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:28:18.225 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:28:18.225 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:28:18.225 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:28:18.225 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:28:18.225 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:28:18.225 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:28:18.225 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:28:18.225 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:28:18.225 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:28:18.225 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:28:18.226 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3301", 2024-09-10T21:28:18.226 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:28:18.226 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:28:18.226 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:28:18.226 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:28:18.226 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790", 2024-09-10T21:28:18.226 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:28:18.226 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:28:18.226 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:28:18.226 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:28:18.226 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790/0", 2024-09-10T21:28:18.226 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6790/0", 2024-09-10T21:28:18.226 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:28:18.226 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:28:18.227 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:28:18.227 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:28:18.227 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:28:18.227 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:28:18.227 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-10T21:28:18.227 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-10T21:28:18.227 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:28:18.227 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:28:18.227 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:28:18.227 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:28:18.227 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:28:18.227 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:28:18.227 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": [ 2024-09-10T21:28:18.228 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:28:18.228 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:28:18.228 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:28:18.228 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 5 2024-09-10T21:28:18.228 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:28:18.228 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:28:18.228 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-10T21:28:18.228 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:28:18.228 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:28:18.228 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:28:18.228 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 4 2024-09-10T21:28:18.228 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:28:18.228 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:28:18.228 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-10T21:28:18.229 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:28:18.229 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:28:18.229 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:28:18.229 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 4 2024-09-10T21:28:18.229 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:28:18.229 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:28:18.229 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": [ 2024-09-10T21:28:18.229 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:28:18.229 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:28:18.229 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:28:18.229 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 2 2024-09-10T21:28:18.229 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:28:18.229 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:28:18.229 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:28:18.230 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-10T21:28:18.230 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-10T21:28:18.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.208+0000 7f37f8a00640 1 -- 172.21.15.17:0/3492366975 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f37d40725e0 msgr2=0x7f37d4074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:18.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.208+0000 7f37f8a00640 1 --2- 172.21.15.17:0/3492366975 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f37d40725e0 0x7f37d4074aa0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f37f40ceab0 tx=0x7f37e0009040 comp rx=0 tx=0).stop 2024-09-10T21:28:18.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.208+0000 7f37f8a00640 1 -- 172.21.15.17:0/3492366975 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f37f4158f60 msgr2=0x7f37f40c9930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:18.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.208+0000 7f37f8a00640 1 --2- 172.21.15.17:0/3492366975 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f37f4158f60 0x7f37f40c9930 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f37ec01d9a0 tx=0x7f37ec003f60 comp rx=0 tx=0).stop 2024-09-10T21:28:18.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.208+0000 7f37f3200640 1 -- 172.21.15.17:0/3492366975 reap_dead start 2024-09-10T21:28:18.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.208+0000 7f37f3200640 1 -- 172.21.15.17:0/3492366975 reap_dead start 2024-09-10T21:28:18.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.208+0000 7f37f8a00640 1 -- 172.21.15.17:0/3492366975 shutdown_connections 2024-09-10T21:28:18.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.208+0000 7f37f8a00640 1 -- 172.21.15.17:0/3492366975 >> 172.21.15.17:0/3492366975 conn(0x7f37f406cc20 msgr2=0x7f37f40747e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:28:18.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.208+0000 7f37f8a00640 1 -- 172.21.15.17:0/3492366975 shutdown_connections 2024-09-10T21:28:18.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.212+0000 7f37f8a00640 1 -- 172.21.15.17:0/3492366975 wait complete. 2024-09-10T21:28:18.239 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-10T21:28:18.240 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-09-10T21:28:18.240 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-10T21:28:18.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.396+0000 7ff1d7200640 1 -- 172.21.15.17:0/1819384855 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff1d80c78b0 msgr2=0x7ff1d80c7d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:18.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.396+0000 7ff1d7200640 1 --2- 172.21.15.17:0/1819384855 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff1d80c78b0 0x7ff1d80c7d20 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7ff1c4009840 tx=0x7ff1c401c6d0 comp rx=0 tx=0).stop 2024-09-10T21:28:18.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.396+0000 7ff1d7200640 1 -- 172.21.15.17:0/1819384855 shutdown_connections 2024-09-10T21:28:18.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.396+0000 7ff1d7200640 1 --2- 172.21.15.17:0/1819384855 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff1d81568a0 0x7ff1d8156d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:18.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.396+0000 7ff1d7200640 1 --2- 172.21.15.17:0/1819384855 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff1d80c78b0 0x7ff1d80c7d20 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:18.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.396+0000 7ff1d7200640 1 -- 172.21.15.17:0/1819384855 >> 172.21.15.17:0/1819384855 conn(0x7ff1d806cc20 msgr2=0x7ff1d806d030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:28:18.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.396+0000 7ff1d7200640 1 -- 172.21.15.17:0/1819384855 shutdown_connections 2024-09-10T21:28:18.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.396+0000 7ff1d7200640 1 -- 172.21.15.17:0/1819384855 wait complete. 2024-09-10T21:28:18.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.400+0000 7ff1d7200640 1 Processor -- start 2024-09-10T21:28:18.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.400+0000 7ff1d7200640 1 -- start start 2024-09-10T21:28:18.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.400+0000 7ff1d7200640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff1d80c78b0 0x7ff1d80cb960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:18.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.400+0000 7ff1d7200640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff1d8150ad0 0x7ff1d80c4d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:18.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.400+0000 7ff1d7200640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff1d81568a0 0x7ff1d8144d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:18.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.400+0000 7ff1d7200640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7ff1d8150eb0 con 0x7ff1d8150ad0 2024-09-10T21:28:18.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.400+0000 7ff1d7200640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7ff1d80c7390 con 0x7ff1d80c78b0 2024-09-10T21:28:18.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.400+0000 7ff1d7200640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7ff1d8151030 con 0x7ff1d81568a0 2024-09-10T21:28:18.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.400+0000 7ff1d5400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff1d8150ad0 0x7ff1d80c4d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:18.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.400+0000 7ff1d5400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff1d8150ad0 0x7ff1d80c4d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:60506/0 (socket says 172.21.15.17:60506) 2024-09-10T21:28:18.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.400+0000 7ff1d5400640 1 -- 172.21.15.17:0/1725360279 learned_addr learned my addr 172.21.15.17:0/1725360279 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:28:18.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.400+0000 7ff1d6800640 1 --2- 172.21.15.17:0/1725360279 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff1d81568a0 0x7ff1d8144d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:18.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.400+0000 7ff1d5e00640 1 --2- 172.21.15.17:0/1725360279 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff1d80c78b0 0x7ff1d80cb960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:18.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.400+0000 7ff1d5400640 1 -- 172.21.15.17:0/1725360279 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff1d81568a0 msgr2=0x7ff1d8144d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:18.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.400+0000 7ff1d5400640 1 --2- 172.21.15.17:0/1725360279 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff1d81568a0 0x7ff1d8144d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:18.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.400+0000 7ff1d5400640 1 -- 172.21.15.17:0/1725360279 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff1d80c78b0 msgr2=0x7ff1d80cb960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:18.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.400+0000 7ff1d5400640 1 --2- 172.21.15.17:0/1725360279 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff1d80c78b0 0x7ff1d80cb960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:18.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.400+0000 7ff1d5400640 1 -- 172.21.15.17:0/1725360279 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1d81f6560 con 0x7ff1d8150ad0 2024-09-10T21:28:18.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.400+0000 7ff1d5e00640 1 --2- 172.21.15.17:0/1725360279 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff1d80c78b0 0x7ff1d80cb960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:28:18.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.400+0000 7ff1d6800640 1 --2- 172.21.15.17:0/1725360279 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff1d81568a0 0x7ff1d8144d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:28:18.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.400+0000 7ff1d5400640 1 --2- 172.21.15.17:0/1725360279 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff1d8150ad0 0x7ff1d80c4d10 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7ff1c401ad80 tx=0x7ff1c40a5630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:28:18.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.400+0000 7ff1ce000640 1 -- 172.21.15.17:0/1725360279 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1c40ba020 con 0x7ff1d8150ad0 2024-09-10T21:28:18.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.400+0000 7ff1d7200640 1 -- 172.21.15.17:0/1725360279 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1d81f6840 con 0x7ff1d8150ad0 2024-09-10T21:28:18.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.400+0000 7ff1ce000640 1 -- 172.21.15.17:0/1725360279 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff1c40043b0 con 0x7ff1d8150ad0 2024-09-10T21:28:18.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.400+0000 7ff1ce000640 1 -- 172.21.15.17:0/1725360279 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1c40b2040 con 0x7ff1d8150ad0 2024-09-10T21:28:18.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.404+0000 7ff1d7200640 1 -- 172.21.15.17:0/1725360279 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1d81f6e00 con 0x7ff1d8150ad0 2024-09-10T21:28:18.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.404+0000 7ff1d7200640 1 --2- 172.21.15.17:0/1725360279 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff1d8151b30 0x7ff1d8145240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:18.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.404+0000 7ff1d7200640 1 -- 172.21.15.17:0/1725360279 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7ff1d8156d10 con 0x7ff1d8151b30 2024-09-10T21:28:18.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.404+0000 7ff1d5e00640 1 --2- 172.21.15.17:0/1725360279 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff1d8151b30 0x7ff1d8145240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:18.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.404+0000 7ff1d5e00640 1 -- 172.21.15.17:0/1725360279 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff1d80c6620 con 0x7ff1d8151b30 2024-09-10T21:28:18.411 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.404+0000 7ff1d5e00640 1 --2- 172.21.15.17:0/1725360279 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff1d8151b30 0x7ff1d8145240 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7ff1c800d6f0 tx=0x7ff1c800dbb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:28:18.411 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.404+0000 7ff1ce000640 1 -- 172.21.15.17:0/1725360279 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7ff1c40b5070 con 0x7ff1d8150ad0 2024-09-10T21:28:18.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.404+0000 7ff1ce000640 1 --2- 172.21.15.17:0/1725360279 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7ff1b8072610 0x7ff1b8074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:18.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.404+0000 7ff1ce000640 1 -- 172.21.15.17:0/1725360279 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7ff1c412d530 con 0x7ff1d8150ad0 2024-09-10T21:28:18.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.408+0000 7ff1ce000640 1 -- 172.21.15.17:0/1725360279 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1c8005270 con 0x7ff1d8151b30 2024-09-10T21:28:18.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.408+0000 7ff1ce000640 1 -- 172.21.15.17:0/1725360279 <== mon.2 v2:172.21.15.31:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7ff1c8017070 con 0x7ff1d8151b30 2024-09-10T21:28:18.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.408+0000 7ff1ce000640 1 -- 172.21.15.17:0/1725360279 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff1d8151b30 msgr2=0x7ff1d8145240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:18.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.408+0000 7ff1ce000640 1 --2- 172.21.15.17:0/1725360279 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff1d8151b30 0x7ff1d8145240 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7ff1c800d6f0 tx=0x7ff1c800dbb0 comp rx=0 tx=0).stop 2024-09-10T21:28:18.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.408+0000 7ff1ce000640 1 -- 172.21.15.17:0/1725360279 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff1d8151b30 msgr2=0x7ff1d8145240 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:28:18.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.408+0000 7ff1ce000640 1 --2- 172.21.15.17:0/1725360279 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff1d8151b30 0x7ff1d8145240 secure :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7ff1c800d6f0 tx=0x7ff1c800dbb0 comp rx=0 tx=0).stop 2024-09-10T21:28:18.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.408+0000 7ff1d6800640 1 --2- 172.21.15.17:0/1725360279 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7ff1b8072610 0x7ff1b8074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:18.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.408+0000 7ff1d6800640 1 --2- 172.21.15.17:0/1725360279 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7ff1b8072610 0x7ff1b8074ad0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ff1d8068ca0 tx=0x7ff1d0009500 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:28:18.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.412+0000 7ff1d7200640 1 --2- 172.21.15.17:0/1725360279 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff1d80cc010 0x7ff1d81f2f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:18.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.412+0000 7ff1d7200640 1 -- 172.21.15.17:0/1725360279 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7ff1d8068dd0 con 0x7ff1d80cc010 2024-09-10T21:28:18.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.412+0000 7ff1d5e00640 1 --2- 172.21.15.17:0/1725360279 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff1d80cc010 0x7ff1d81f2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:18.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.412+0000 7ff1d5e00640 1 -- 172.21.15.17:0/1725360279 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7ff1d8076660 con 0x7ff1d80cc010 2024-09-10T21:28:18.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.412+0000 7ff1d5e00640 1 --2- 172.21.15.17:0/1725360279 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff1d80cc010 0x7ff1d81f2f40 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ff1c800d6f0 tx=0x7ff1c8001090 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:28:18.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.416+0000 7ff1ce000640 1 -- 172.21.15.17:0/1725360279 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1c800bae0 con 0x7ff1d80cc010 2024-09-10T21:28:18.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.416+0000 7ff1ce000640 1 -- 172.21.15.17:0/1725360279 <== mon.2 v2:172.21.15.31:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4874 (secure 0 0 0) 0x7ff1c800bae0 con 0x7ff1d80cc010 2024-09-10T21:28:18.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.416+0000 7ff1ce000640 1 -- 172.21.15.17:0/1725360279 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff1d80cc010 msgr2=0x7ff1d81f2f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:18.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.416+0000 7ff1ce000640 1 --2- 172.21.15.17:0/1725360279 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff1d80cc010 0x7ff1d81f2f40 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ff1c800d6f0 tx=0x7ff1c8001090 comp rx=0 tx=0).stop 2024-09-10T21:28:18.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.416+0000 7ff1ce000640 1 -- 172.21.15.17:0/1725360279 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff1d80cc010 msgr2=0x7ff1d81f2f40 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:28:18.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.416+0000 7ff1ce000640 1 --2- 172.21.15.17:0/1725360279 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff1d80cc010 0x7ff1d81f2f40 secure :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ff1c800d6f0 tx=0x7ff1c8001090 comp rx=0 tx=0).stop 2024-09-10T21:28:18.421 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-10T21:28:18.421 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:28:18.421 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:28:18.421 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-10T21:28:18.421 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 30, 2024-09-10T21:28:18.421 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 79531, 2024-09-10T21:28:18.421 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-10T21:28:18.422 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-10T21:28:18.422 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-10T21:28:18.422 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-10T21:28:18.422 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:28:18.422 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 0, 2024-09-10T21:28:18.422 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:28:18.422 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-10T21:28:18.422 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-10T21:28:18.422 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:28:18.422 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:28:18.422 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:28:18.422 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:28:18.422 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:28:18.423 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:28:18.423 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:28:18.423 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:28:18.423 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:28:18.423 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:28:18.423 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:28:18.423 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:28:18.423 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-10T21:28:18.423 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-10T21:28:18.423 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:28:18.423 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:28:18.423 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:28:18.423 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:28:18.424 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:28:18.424 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:28:18.424 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:28:18.424 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:28:18.424 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:28:18.424 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:28:18.424 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:28:18.424 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:28:18.424 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:28:18.424 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-10T21:28:18.424 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-10T21:28:18.424 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-10T21:28:18.425 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-10T21:28:18.425 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-10T21:28:18.425 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "5e2f023f-d6a6-49f4-a256-f4807dce3739", 2024-09-10T21:28:18.425 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:28:18.425 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:28:18.425 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-10T21:28:18.425 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-10T21:28:18.425 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-10T21:28:18.425 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-10T21:28:18.425 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-10T21:28:18.425 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-10T21:28:18.425 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-10T21:28:18.426 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:28:18.426 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-10T21:28:18.426 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:28:18.426 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:28:18.426 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:28:18.426 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:28:18.426 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:28:18.426 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:28:18.426 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:28:18.426 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:28:18.426 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:28:18.426 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:28:18.426 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:28:18.427 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:28:18.427 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-10T21:28:18.427 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:28:18.427 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-10T21:28:18.427 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:28:18.427 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:28:18.427 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:28:18.427 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:28:18.427 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:28:18.427 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:28:18.427 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:28:18.427 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-10T21:28:18.427 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:28:18.428 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:28:18.428 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:28:18.428 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:28:18.428 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-10T21:28:18.428 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:28:18.428 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:28:18.428 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:28:18.428 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:28:18.428 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-10T21:28:18.428 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-10T21:28:18.428 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:28:18.428 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:28:18.428 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:28:18.429 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:28:18.429 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:28:18.429 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:28:18.429 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:28:18.429 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:28:18.429 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:28:18.429 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:28:18.429 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:28:18.429 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3300", 2024-09-10T21:28:18.429 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:28:18.429 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:28:18.429 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:28:18.429 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:28:18.430 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789", 2024-09-10T21:28:18.430 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:28:18.430 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:28:18.430 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:28:18.430 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:28:18.430 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789/0", 2024-09-10T21:28:18.430 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6789/0", 2024-09-10T21:28:18.430 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:28:18.430 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:28:18.430 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:28:18.430 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:28:18.430 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:28:18.430 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:28:18.431 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:28:18.431 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:28:18.431 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:28:18.431 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:28:18.431 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:28:18.431 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3301", 2024-09-10T21:28:18.431 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:28:18.431 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:28:18.431 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:28:18.431 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:28:18.431 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790", 2024-09-10T21:28:18.431 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:28:18.431 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:28:18.432 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:28:18.432 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:28:18.432 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790/0", 2024-09-10T21:28:18.432 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6790/0", 2024-09-10T21:28:18.432 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:28:18.432 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:28:18.432 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:28:18.432 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:28:18.432 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:28:18.432 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:28:18.432 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-10T21:28:18.432 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-10T21:28:18.432 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:28:18.433 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:28:18.433 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:28:18.433 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:28:18.433 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:28:18.433 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:28:18.433 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": [ 2024-09-10T21:28:18.433 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:28:18.433 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:28:18.433 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:28:18.433 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:28:18.433 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:28:18.433 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:28:18.433 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-10T21:28:18.434 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:28:18.434 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:28:18.434 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:28:18.434 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 4 2024-09-10T21:28:18.434 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:28:18.434 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:28:18.434 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-10T21:28:18.434 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:28:18.434 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:28:18.434 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:28:18.434 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 3 2024-09-10T21:28:18.434 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:28:18.434 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:28:18.434 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:28:18.435 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-10T21:28:18.435 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-10T21:28:18.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.416+0000 7ff1d7200640 1 -- 172.21.15.17:0/1725360279 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7ff1b8072610 msgr2=0x7ff1b8074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:18.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.416+0000 7ff1d7200640 1 --2- 172.21.15.17:0/1725360279 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7ff1b8072610 0x7ff1b8074ad0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ff1d8068ca0 tx=0x7ff1d0009500 comp rx=0 tx=0).stop 2024-09-10T21:28:18.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.416+0000 7ff1d7200640 1 -- 172.21.15.17:0/1725360279 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff1d8150ad0 msgr2=0x7ff1d80c4d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:18.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.416+0000 7ff1d7200640 1 --2- 172.21.15.17:0/1725360279 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff1d8150ad0 0x7ff1d80c4d10 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7ff1c401ad80 tx=0x7ff1c40a5630 comp rx=0 tx=0).stop 2024-09-10T21:28:18.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.416+0000 7ff1d6800640 1 -- 172.21.15.17:0/1725360279 reap_dead start 2024-09-10T21:28:18.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.416+0000 7ff1d6800640 1 -- 172.21.15.17:0/1725360279 reap_dead start 2024-09-10T21:28:18.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.416+0000 7ff1d7200640 1 -- 172.21.15.17:0/1725360279 shutdown_connections 2024-09-10T21:28:18.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.416+0000 7ff1d7200640 1 -- 172.21.15.17:0/1725360279 >> 172.21.15.17:0/1725360279 conn(0x7ff1d806cc20 msgr2=0x7ff1d8160830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:28:18.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.416+0000 7ff1d7200640 1 -- 172.21.15.17:0/1725360279 shutdown_connections 2024-09-10T21:28:18.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.416+0000 7ff1d7200640 1 -- 172.21.15.17:0/1725360279 wait complete. 2024-09-10T21:28:18.439 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-10T21:28:18.439 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-09-10T21:28:18.440 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-09-10T21:28:18.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.604+0000 7fa392e00640 1 -- 172.21.15.17:0/2864499792 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa38c152500 msgr2=0x7fa38c1589b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:18.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.604+0000 7fa392e00640 1 --2- 172.21.15.17:0/2864499792 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa38c152500 0x7fa38c1589b0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fa38c0c92d0 tx=0x7fa37801efd0 comp rx=0 tx=0).stop 2024-09-10T21:28:18.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.604+0000 7fa392e00640 1 -- 172.21.15.17:0/2864499792 shutdown_connections 2024-09-10T21:28:18.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.604+0000 7fa392e00640 1 --2- 172.21.15.17:0/2864499792 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa38c152500 0x7fa38c1589b0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:18.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.604+0000 7fa392e00640 1 --2- 172.21.15.17:0/2864499792 >> v2:172.21.15.31:3300/0 conn(0x7fa38c152120 0x7fa38c068bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:18.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.604+0000 7fa392e00640 1 -- 172.21.15.17:0/2864499792 >> 172.21.15.17:0/2864499792 conn(0x7fa38c06cdf0 msgr2=0x7fa38c06d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:28:18.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.604+0000 7fa392e00640 1 -- 172.21.15.17:0/2864499792 shutdown_connections 2024-09-10T21:28:18.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.604+0000 7fa392e00640 1 -- 172.21.15.17:0/2864499792 wait complete. 2024-09-10T21:28:18.612 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.608+0000 7fa392e00640 1 Processor -- start 2024-09-10T21:28:18.612 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.608+0000 7fa392e00640 1 -- start start 2024-09-10T21:28:18.614 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.608+0000 7fa392e00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa38c152120 0x7fa38c0cb4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:18.614 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.608+0000 7fa392e00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa38c152500 0x7fa38c1f1530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:18.614 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.608+0000 7fa392e00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa38c158ef0 0x7fa38c1470b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:18.614 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.608+0000 7fa392e00640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fa38c0c74e0 con 0x7fa38c152500 2024-09-10T21:28:18.614 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.608+0000 7fa392e00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fa38c158490 con 0x7fa38c152120 2024-09-10T21:28:18.614 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.608+0000 7fa392e00640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7fa38c0c7660 con 0x7fa38c158ef0 2024-09-10T21:28:18.614 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.608+0000 7fa391000640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa38c152500 0x7fa38c1f1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:18.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.608+0000 7fa391000640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa38c152500 0x7fa38c1f1530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:60536/0 (socket says 172.21.15.17:60536) 2024-09-10T21:28:18.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.608+0000 7fa391000640 1 -- 172.21.15.17:0/1515505936 learned_addr learned my addr 172.21.15.17:0/1515505936 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:28:18.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.608+0000 7fa391a00640 1 --2- 172.21.15.17:0/1515505936 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa38c152120 0x7fa38c0cb4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:18.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.608+0000 7fa392400640 1 --2- 172.21.15.17:0/1515505936 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa38c158ef0 0x7fa38c1470b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:18.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.608+0000 7fa391000640 1 -- 172.21.15.17:0/1515505936 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa38c158ef0 msgr2=0x7fa38c1470b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:18.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.608+0000 7fa391000640 1 --2- 172.21.15.17:0/1515505936 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa38c158ef0 0x7fa38c1470b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:18.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.608+0000 7fa391000640 1 -- 172.21.15.17:0/1515505936 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa38c152120 msgr2=0x7fa38c0cb4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:18.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.608+0000 7fa391000640 1 --2- 172.21.15.17:0/1515505936 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa38c152120 0x7fa38c0cb4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:18.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.608+0000 7fa391000640 1 -- 172.21.15.17:0/1515505936 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa38c0cbf80 con 0x7fa38c152500 2024-09-10T21:28:18.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.608+0000 7fa392400640 1 --2- 172.21.15.17:0/1515505936 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa38c158ef0 0x7fa38c1470b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:28:18.616 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.608+0000 7fa391000640 1 --2- 172.21.15.17:0/1515505936 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa38c152500 0x7fa38c1f1530 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fa378000c10 tx=0x7fa3780a8c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:28:18.616 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.608+0000 7fa389600640 1 -- 172.21.15.17:0/1515505936 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3780b5040 con 0x7fa38c152500 2024-09-10T21:28:18.616 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.608+0000 7fa389600640 1 -- 172.21.15.17:0/1515505936 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa3780bf070 con 0x7fa38c152500 2024-09-10T21:28:18.616 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.608+0000 7fa389600640 1 -- 172.21.15.17:0/1515505936 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3780b1bb0 con 0x7fa38c152500 2024-09-10T21:28:18.616 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.608+0000 7fa392e00640 1 -- 172.21.15.17:0/1515505936 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa38c0cae40 con 0x7fa38c152500 2024-09-10T21:28:18.616 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.608+0000 7fa392e00640 1 -- 172.21.15.17:0/1515505936 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa38c1e2160 con 0x7fa38c152500 2024-09-10T21:28:18.617 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.612+0000 7fa392e00640 1 -- 172.21.15.17:0/1515505936 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa38c069e60 con 0x7fa38c152500 2024-09-10T21:28:18.617 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.612+0000 7fa389600640 1 -- 172.21.15.17:0/1515505936 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fa3780b8030 con 0x7fa38c152500 2024-09-10T21:28:18.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.612+0000 7fa389600640 1 --2- 172.21.15.17:0/1515505936 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fa36c0725e0 0x7fa36c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:18.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.612+0000 7fa389600640 1 -- 172.21.15.17:0/1515505936 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fa378130eb0 con 0x7fa38c152500 2024-09-10T21:28:18.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.612+0000 7fa391a00640 1 --2- 172.21.15.17:0/1515505936 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fa36c0725e0 0x7fa36c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:18.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.612+0000 7fa391a00640 1 --2- 172.21.15.17:0/1515505936 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fa36c0725e0 0x7fa36c074aa0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fa380009be0 tx=0x7fa3800091e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:28:18.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.616+0000 7fa389600640 1 -- 172.21.15.17:0/1515505936 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fa378018b30 con 0x7fa38c152500 2024-09-10T21:28:18.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.896+0000 7fa392e00640 1 -- 172.21.15.17:0/1515505936 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) -- 0x7fa38c1e6080 con 0x7fa38c152500 2024-09-10T21:28:18.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.896+0000 7fa389600640 1 -- 172.21.15.17:0/1515505936 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) ==== 57+0+0 (secure 0 0 0) 0x7fa3780fc180 con 0x7fa38c152500 2024-09-10T21:28:18.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.900+0000 7fa392e00640 1 -- 172.21.15.17:0/1515505936 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fa36c0725e0 msgr2=0x7fa36c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:18.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.900+0000 7fa392e00640 1 --2- 172.21.15.17:0/1515505936 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fa36c0725e0 0x7fa36c074aa0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fa380009be0 tx=0x7fa3800091e0 comp rx=0 tx=0).stop 2024-09-10T21:28:18.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.900+0000 7fa392e00640 1 -- 172.21.15.17:0/1515505936 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa38c152500 msgr2=0x7fa38c1f1530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:18.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.900+0000 7fa392e00640 1 --2- 172.21.15.17:0/1515505936 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa38c152500 0x7fa38c1f1530 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fa378000c10 tx=0x7fa3780a8c60 comp rx=0 tx=0).stop 2024-09-10T21:28:18.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.900+0000 7fa392e00640 1 -- 172.21.15.17:0/1515505936 shutdown_connections 2024-09-10T21:28:18.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.900+0000 7fa392e00640 1 --2- 172.21.15.17:0/1515505936 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fa36c0725e0 0x7fa36c074aa0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:18.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.900+0000 7fa392e00640 1 --2- 172.21.15.17:0/1515505936 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa38c158ef0 0x7fa38c1470b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:18.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.900+0000 7fa392e00640 1 --2- 172.21.15.17:0/1515505936 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa38c152500 0x7fa38c1f1530 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:18.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.900+0000 7fa392e00640 1 --2- 172.21.15.17:0/1515505936 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa38c152120 0x7fa38c0cb4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:18.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.900+0000 7fa392e00640 1 -- 172.21.15.17:0/1515505936 >> 172.21.15.17:0/1515505936 conn(0x7fa38c06cdf0 msgr2=0x7fa38c147ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:28:18.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.900+0000 7fa392e00640 1 -- 172.21.15.17:0/1515505936 shutdown_connections 2024-09-10T21:28:18.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:18.900+0000 7fa392e00640 1 -- 172.21.15.17:0/1515505936 wait complete. 2024-09-10T21:28:18.930 INFO:tasks.mon_thrash.mon_thrasher:waiting for 10.0 secs before continuing thrashing 2024-09-10T21:28:28.930 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-09-10T21:28:28.931 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-10T21:28:29.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.096+0000 7fe0e7200640 1 -- 172.21.15.17:0/4075385066 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe0e806fd00 msgr2=0x7fe0e80c7870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:29.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.096+0000 7fe0e7200640 1 --2- 172.21.15.17:0/4075385066 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe0e806fd00 0x7fe0e80c7870 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fe0e0009830 tx=0x7fe0e001f010 comp rx=0 tx=0).stop 2024-09-10T21:28:29.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.096+0000 7fe0e7200640 1 -- 172.21.15.17:0/4075385066 shutdown_connections 2024-09-10T21:28:29.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.096+0000 7fe0e7200640 1 --2- 172.21.15.17:0/4075385066 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe0e806fd00 0x7fe0e80c7870 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:29.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.096+0000 7fe0e7200640 1 --2- 172.21.15.17:0/4075385066 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe0e80b2d80 0x7fe0e8158310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:29.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.096+0000 7fe0e7200640 1 -- 172.21.15.17:0/4075385066 >> 172.21.15.17:0/4075385066 conn(0x7fe0e8070ca0 msgr2=0x7fe0e80710b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:28:29.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.096+0000 7fe0e7200640 1 -- 172.21.15.17:0/4075385066 shutdown_connections 2024-09-10T21:28:29.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.096+0000 7fe0e7200640 1 -- 172.21.15.17:0/4075385066 wait complete. 2024-09-10T21:28:29.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.100+0000 7fe0e7200640 1 Processor -- start 2024-09-10T21:28:29.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.100+0000 7fe0e7200640 1 -- start start 2024-09-10T21:28:29.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.100+0000 7fe0e7200640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe0e806fd00 0x7fe0e814d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:29.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.100+0000 7fe0e7200640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe0e80b2d80 0x7fe0e814d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:29.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.100+0000 7fe0e7200640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe0e81527e0 0x7fe0e81ee780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:29.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.100+0000 7fe0e7200640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fe0e806f500 con 0x7fe0e81527e0 2024-09-10T21:28:29.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.100+0000 7fe0e7200640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fe0e806f220 con 0x7fe0e80b2d80 2024-09-10T21:28:29.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.100+0000 7fe0e7200640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7fe0e806ed40 con 0x7fe0e806fd00 2024-09-10T21:28:29.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.100+0000 7fe0e6800640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe0e81527e0 0x7fe0e81ee780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:29.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.100+0000 7fe0e6800640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe0e81527e0 0x7fe0e81ee780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42562/0 (socket says 172.21.15.17:42562) 2024-09-10T21:28:29.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.100+0000 7fe0e6800640 1 -- 172.21.15.17:0/57779979 learned_addr learned my addr 172.21.15.17:0/57779979 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:28:29.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.100+0000 7fe0dd400640 1 --2- 172.21.15.17:0/57779979 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe0e80b2d80 0x7fe0e814d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:29.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.100+0000 7fe0e5e00640 1 --2- 172.21.15.17:0/57779979 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe0e806fd00 0x7fe0e814d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:29.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.100+0000 7fe0e6800640 1 -- 172.21.15.17:0/57779979 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe0e806fd00 msgr2=0x7fe0e814d1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:29.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.100+0000 7fe0e6800640 1 --2- 172.21.15.17:0/57779979 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe0e806fd00 0x7fe0e814d1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:29.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.100+0000 7fe0e6800640 1 -- 172.21.15.17:0/57779979 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe0e80b2d80 msgr2=0x7fe0e814d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:29.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.100+0000 7fe0e6800640 1 --2- 172.21.15.17:0/57779979 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe0e80b2d80 0x7fe0e814d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:29.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.100+0000 7fe0e6800640 1 -- 172.21.15.17:0/57779979 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0e814c460 con 0x7fe0e81527e0 2024-09-10T21:28:29.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.100+0000 7fe0dd400640 1 --2- 172.21.15.17:0/57779979 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe0e80b2d80 0x7fe0e814d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:28:29.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.100+0000 7fe0e6800640 1 --2- 172.21.15.17:0/57779979 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe0e81527e0 0x7fe0e81ee780 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fe0e001cc40 tx=0x7fe0e0005da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:28:29.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.100+0000 7fe0dea00640 1 -- 172.21.15.17:0/57779979 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0e00bf070 con 0x7fe0e81527e0 2024-09-10T21:28:29.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.100+0000 7fe0dea00640 1 -- 172.21.15.17:0/57779979 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe0e0003850 con 0x7fe0e81527e0 2024-09-10T21:28:29.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.100+0000 7fe0dea00640 1 -- 172.21.15.17:0/57779979 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0e00b5040 con 0x7fe0e81527e0 2024-09-10T21:28:29.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.100+0000 7fe0e7200640 1 -- 172.21.15.17:0/57779979 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0e814eb20 con 0x7fe0e81527e0 2024-09-10T21:28:29.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.100+0000 7fe0e7200640 1 -- 172.21.15.17:0/57779979 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0e814edc0 con 0x7fe0e81527e0 2024-09-10T21:28:29.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.104+0000 7fe0e7200640 1 -- 172.21.15.17:0/57779979 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0e814f000 con 0x7fe0e81527e0 2024-09-10T21:28:29.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.104+0000 7fe0dea00640 1 -- 172.21.15.17:0/57779979 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fe0e00b8070 con 0x7fe0e81527e0 2024-09-10T21:28:29.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.104+0000 7fe0dea00640 1 --2- 172.21.15.17:0/57779979 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fe0c40725e0 0x7fe0c4074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:29.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.104+0000 7fe0dea00640 1 -- 172.21.15.17:0/57779979 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fe0e0130570 con 0x7fe0e81527e0 2024-09-10T21:28:29.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.108+0000 7fe0e5e00640 1 --2- 172.21.15.17:0/57779979 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fe0c40725e0 0x7fe0c4074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:29.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.108+0000 7fe0e5e00640 1 --2- 172.21.15.17:0/57779979 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fe0c40725e0 0x7fe0c4074aa0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fe0d80012f0 tx=0x7fe0d8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:28:29.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.112+0000 7fe0dea00640 1 -- 172.21.15.17:0/57779979 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fe0e814f000 con 0x7fe0e81527e0 2024-09-10T21:28:29.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.432+0000 7fe0e7200640 1 -- 172.21.15.17:0/57779979 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fe0e8158310 con 0x7fe0e81527e0 2024-09-10T21:28:29.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.432+0000 7fe0dea00640 1 -- 172.21.15.17:0/57779979 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1557 (secure 0 0 0) 0x7fe0e0100e20 con 0x7fe0e81527e0 2024-09-10T21:28:29.439 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":30,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":11,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5e2f023f-d6a6-49f4-a256-f4807dce3739","modified":"2024-09-10T21:23:46.892130Z","created":"2024-09-10T21:23:46.892130Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3301","nonce":0},{"type":"v1","addr":"172.21.15.31:6790","nonce":0}]},"addr":"172.21.15.31:6790/0","public_addr":"172.21.15.31:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-10T21:28:29.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.436+0000 7fe0e7200640 1 -- 172.21.15.17:0/57779979 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fe0c40725e0 msgr2=0x7fe0c4074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:29.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.436+0000 7fe0e7200640 1 --2- 172.21.15.17:0/57779979 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fe0c40725e0 0x7fe0c4074aa0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fe0d80012f0 tx=0x7fe0d8009040 comp rx=0 tx=0).stop 2024-09-10T21:28:29.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.436+0000 7fe0e7200640 1 -- 172.21.15.17:0/57779979 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe0e81527e0 msgr2=0x7fe0e81ee780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:29.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.436+0000 7fe0e7200640 1 --2- 172.21.15.17:0/57779979 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe0e81527e0 0x7fe0e81ee780 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fe0e001cc40 tx=0x7fe0e0005da0 comp rx=0 tx=0).stop 2024-09-10T21:28:29.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.436+0000 7fe0e7200640 1 -- 172.21.15.17:0/57779979 shutdown_connections 2024-09-10T21:28:29.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.436+0000 7fe0e7200640 1 --2- 172.21.15.17:0/57779979 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fe0c40725e0 0x7fe0c4074aa0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:29.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.436+0000 7fe0e7200640 1 --2- 172.21.15.17:0/57779979 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe0e81527e0 0x7fe0e81ee780 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:29.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.436+0000 7fe0e7200640 1 --2- 172.21.15.17:0/57779979 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe0e80b2d80 0x7fe0e814d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:29.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.436+0000 7fe0e7200640 1 --2- 172.21.15.17:0/57779979 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe0e806fd00 0x7fe0e814d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:29.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.436+0000 7fe0e7200640 1 -- 172.21.15.17:0/57779979 >> 172.21.15.17:0/57779979 conn(0x7fe0e8070ca0 msgr2=0x7fe0e80cc640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:28:29.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.436+0000 7fe0e7200640 1 -- 172.21.15.17:0/57779979 shutdown_connections 2024-09-10T21:28:29.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.436+0000 7fe0e7200640 1 -- 172.21.15.17:0/57779979 wait complete. 2024-09-10T21:28:29.461 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-09-10T21:28:29.461 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-09-10T21:28:29.461 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-09-10T21:28:29.461 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-10T21:28:29.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.620+0000 7f0424800640 1 -- 172.21.15.17:0/1140706757 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0404004810 msgr2=0x7f0404004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:29.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.620+0000 7f0424800640 1 --2- 172.21.15.17:0/1140706757 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0404004810 0x7f0404004c90 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f040c001860 tx=0x7f040c01f050 comp rx=0 tx=0).stop 2024-09-10T21:28:29.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.620+0000 7f0422800640 1 -- 172.21.15.17:0/1140706757 reap_dead start 2024-09-10T21:28:29.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.620+0000 7f0424800640 1 -- 172.21.15.17:0/1140706757 shutdown_connections 2024-09-10T21:28:29.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.620+0000 7f0424800640 1 -- 172.21.15.17:0/1140706757 >> 172.21.15.17:0/1140706757 conn(0x7f041c0702f0 msgr2=0x7f041c070700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:28:29.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.620+0000 7f0424800640 1 -- 172.21.15.17:0/1140706757 shutdown_connections 2024-09-10T21:28:29.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.620+0000 7f0424800640 1 -- 172.21.15.17:0/1140706757 wait complete. 2024-09-10T21:28:29.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.620+0000 7f0424800640 1 Processor -- start 2024-09-10T21:28:29.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.620+0000 7f0424800640 1 -- start start 2024-09-10T21:28:29.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.620+0000 7f0424800640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f0404004810 0x7f041c15af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:29.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.620+0000 7f0424800640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f041c072a90 0x7f041c154a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:29.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.620+0000 7f0424800640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f041c155570 0x7f041c1f04c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:29.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.620+0000 7f0424800640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f041c15bd60 con 0x7f041c155570 2024-09-10T21:28:29.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.620+0000 7f0424800640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f041c15ba80 con 0x7f041c072a90 2024-09-10T21:28:29.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.620+0000 7f0424800640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f041c0b42e0 con 0x7f0404004810 2024-09-10T21:28:29.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.620+0000 7f0422800640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f041c155570 0x7f041c1f04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:29.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.620+0000 7f0421400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f041c072a90 0x7f041c154a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:29.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.620+0000 7f0421e00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f0404004810 0x7f041c15af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:29.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.620+0000 7f0422800640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f041c155570 0x7f041c1f04c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42586/0 (socket says 172.21.15.17:42586) 2024-09-10T21:28:29.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.620+0000 7f0422800640 1 -- 172.21.15.17:0/256419312 learned_addr learned my addr 172.21.15.17:0/256419312 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:28:29.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.624+0000 7f0421e00640 1 -- 172.21.15.17:0/256419312 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f041c072a90 msgr2=0x7f041c154a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:29.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.624+0000 7f0421e00640 1 --2- 172.21.15.17:0/256419312 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f041c072a90 0x7f041c154a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:29.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.624+0000 7f0421e00640 1 -- 172.21.15.17:0/256419312 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f041c155570 msgr2=0x7f041c1f04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:29.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.624+0000 7f0421e00640 1 --2- 172.21.15.17:0/256419312 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f041c155570 0x7f041c1f04c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:29.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.624+0000 7f0421e00640 1 -- 172.21.15.17:0/256419312 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f041c1f3290 con 0x7f0404004810 2024-09-10T21:28:29.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.624+0000 7f0421e00640 1 --2- 172.21.15.17:0/256419312 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f0404004810 0x7f041c15af80 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f0410006360 tx=0x7f04100039d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:28:29.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.624+0000 7f0416000640 1 -- 172.21.15.17:0/256419312 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0410015070 con 0x7f0404004810 2024-09-10T21:28:29.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.624+0000 7f0416000640 1 -- 172.21.15.17:0/256419312 <== mon.2 v2:172.21.15.31:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0410011860 con 0x7f0404004810 2024-09-10T21:28:29.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.624+0000 7f0416000640 1 -- 172.21.15.17:0/256419312 <== mon.2 v2:172.21.15.31:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0410011cb0 con 0x7f0404004810 2024-09-10T21:28:29.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.624+0000 7f0424800640 1 -- 172.21.15.17:0/256419312 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f041c1efd20 con 0x7f0404004810 2024-09-10T21:28:29.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.624+0000 7f0424800640 1 -- 172.21.15.17:0/256419312 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f041c15b570 con 0x7f0404004810 2024-09-10T21:28:29.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.624+0000 7f0424800640 1 --2- 172.21.15.17:0/256419312 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f041c155950 0x7f041c14eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:29.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.624+0000 7f0424800640 1 -- 172.21.15.17:0/256419312 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f041c0cb450 con 0x7f041c155950 2024-09-10T21:28:29.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.624+0000 7f0421400640 1 --2- 172.21.15.17:0/256419312 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f041c155950 0x7f041c14eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:29.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.624+0000 7f0421400640 1 -- 172.21.15.17:0/256419312 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f041c0cc1d0 con 0x7f041c155950 2024-09-10T21:28:29.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.624+0000 7f0421400640 1 --2- 172.21.15.17:0/256419312 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f041c155950 0x7f041c14eda0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f040c004000 tx=0x7f040c001510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:28:29.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.624+0000 7f0416000640 1 -- 172.21.15.17:0/256419312 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f040c006110 con 0x7f041c155950 2024-09-10T21:28:29.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.624+0000 7f0416000640 1 -- 172.21.15.17:0/256419312 <== mon.2 v2:172.21.15.31:3301/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f04100103c0 con 0x7f0404004810 2024-09-10T21:28:29.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.628+0000 7f0416000640 1 --2- 172.21.15.17:0/256419312 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f0400072510 0x7f04000749d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:29.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.628+0000 7f0416000640 1 -- 172.21.15.17:0/256419312 <== mon.2 v2:172.21.15.31:3301/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f04100941a0 con 0x7f0404004810 2024-09-10T21:28:29.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.628+0000 7f0416000640 1 -- 172.21.15.17:0/256419312 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f040c0062b0 con 0x7f041c155950 2024-09-10T21:28:29.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.628+0000 7f0416000640 1 -- 172.21.15.17:0/256419312 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f041c155950 msgr2=0x7f041c14eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:29.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.628+0000 7f0416000640 1 --2- 172.21.15.17:0/256419312 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f041c155950 0x7f041c14eda0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f040c004000 tx=0x7f040c001510 comp rx=0 tx=0).stop 2024-09-10T21:28:29.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.628+0000 7f0416000640 1 -- 172.21.15.17:0/256419312 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f041c155950 msgr2=0x7f041c14eda0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:28:29.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.628+0000 7f0416000640 1 --2- 172.21.15.17:0/256419312 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f041c155950 0x7f041c14eda0 secure :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f040c004000 tx=0x7f040c001510 comp rx=0 tx=0).stop 2024-09-10T21:28:29.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.628+0000 7f0422800640 1 --2- 172.21.15.17:0/256419312 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f0400072510 0x7f04000749d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:29.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.628+0000 7f0422800640 1 --2- 172.21.15.17:0/256419312 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f0400072510 0x7f04000749d0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f041c076910 tx=0x7f0418009500 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:28:29.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.636+0000 7f0424800640 1 --2- 172.21.15.17:0/256419312 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f041c1eff00 0x7f041c14f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:29.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.636+0000 7f0424800640 1 -- 172.21.15.17:0/256419312 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f041c15b740 con 0x7f041c1eff00 2024-09-10T21:28:29.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.636+0000 7f0421400640 1 --2- 172.21.15.17:0/256419312 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f041c1eff00 0x7f041c14f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:29.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.636+0000 7f0421400640 1 -- 172.21.15.17:0/256419312 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f041c06c7e0 con 0x7f041c1eff00 2024-09-10T21:28:29.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.636+0000 7f0421400640 1 --2- 172.21.15.17:0/256419312 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f041c1eff00 0x7f041c14f2e0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f040c01d750 tx=0x7f040c006b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:28:29.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.636+0000 7f0416000640 1 -- 172.21.15.17:0/256419312 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f040c0b3620 con 0x7f041c1eff00 2024-09-10T21:28:29.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.636+0000 7f0416000640 1 -- 172.21.15.17:0/256419312 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4543 (secure 0 0 0) 0x7f040c0b3620 con 0x7f041c1eff00 2024-09-10T21:28:29.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.636+0000 7f0416000640 1 -- 172.21.15.17:0/256419312 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f041c1eff00 msgr2=0x7f041c14f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:29.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.636+0000 7f0416000640 1 --2- 172.21.15.17:0/256419312 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f041c1eff00 0x7f041c14f2e0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f040c01d750 tx=0x7f040c006b20 comp rx=0 tx=0).stop 2024-09-10T21:28:29.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.636+0000 7f0416000640 1 -- 172.21.15.17:0/256419312 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f041c1eff00 msgr2=0x7f041c14f2e0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:28:29.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.636+0000 7f0416000640 1 --2- 172.21.15.17:0/256419312 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f041c1eff00 0x7f041c14f2e0 secure :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f040c01d750 tx=0x7f040c006b20 comp rx=0 tx=0).stop 2024-09-10T21:28:29.642 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-10T21:28:29.642 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:28:29.643 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:28:29.643 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "leader", 2024-09-10T21:28:29.643 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 30, 2024-09-10T21:28:29.643 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 27475, 2024-09-10T21:28:29.643 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-10T21:28:29.643 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-10T21:28:29.643 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-10T21:28:29.643 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-10T21:28:29.643 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:28:29.643 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 12, 2024-09-10T21:28:29.643 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:28:29.643 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-10T21:28:29.644 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-10T21:28:29.644 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:28:29.644 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:28:29.644 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:28:29.644 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:28:29.644 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:28:29.644 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:28:29.644 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:28:29.644 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:28:29.644 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:28:29.644 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:28:29.644 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:28:29.644 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:28:29.645 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-10T21:28:29.645 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-10T21:28:29.645 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:28:29.645 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:28:29.645 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:28:29.645 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:28:29.645 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:28:29.645 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:28:29.645 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:28:29.645 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:28:29.645 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:28:29.645 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:28:29.646 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:28:29.646 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:28:29.646 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:28:29.646 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-10T21:28:29.646 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-10T21:28:29.646 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-10T21:28:29.646 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-10T21:28:29.646 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-10T21:28:29.646 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "5e2f023f-d6a6-49f4-a256-f4807dce3739", 2024-09-10T21:28:29.646 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:28:29.646 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:28:29.646 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-10T21:28:29.646 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-10T21:28:29.646 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-10T21:28:29.647 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-10T21:28:29.647 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-10T21:28:29.647 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-10T21:28:29.647 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-10T21:28:29.647 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:28:29.647 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-10T21:28:29.647 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:28:29.647 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:28:29.647 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:28:29.647 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:28:29.647 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:28:29.647 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:28:29.647 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:28:29.647 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:28:29.648 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:28:29.648 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:28:29.648 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:28:29.648 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:28:29.648 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-10T21:28:29.648 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:28:29.648 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-10T21:28:29.648 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:28:29.648 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:28:29.648 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:28:29.648 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:28:29.648 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:28:29.649 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:28:29.649 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:28:29.649 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-10T21:28:29.649 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:28:29.649 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:28:29.649 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:28:29.649 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:28:29.649 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-10T21:28:29.649 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:28:29.649 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:28:29.650 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:28:29.650 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:28:29.650 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-10T21:28:29.650 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-10T21:28:29.650 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:28:29.650 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:28:29.650 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:28:29.650 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:28:29.650 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:28:29.650 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:28:29.651 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:28:29.651 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:28:29.651 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:28:29.651 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:28:29.651 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:28:29.651 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3300", 2024-09-10T21:28:29.651 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:28:29.651 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:28:29.651 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:28:29.651 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:28:29.651 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789", 2024-09-10T21:28:29.651 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:28:29.651 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:28:29.652 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:28:29.652 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:28:29.652 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789/0", 2024-09-10T21:28:29.652 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6789/0", 2024-09-10T21:28:29.652 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:28:29.652 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:28:29.652 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:28:29.652 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:28:29.652 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:28:29.652 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:28:29.652 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:28:29.652 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:28:29.652 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:28:29.653 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:28:29.653 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:28:29.653 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3301", 2024-09-10T21:28:29.653 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:28:29.653 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:28:29.653 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:28:29.653 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:28:29.653 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790", 2024-09-10T21:28:29.653 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:28:29.653 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:28:29.653 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:28:29.654 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:28:29.654 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790/0", 2024-09-10T21:28:29.654 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6790/0", 2024-09-10T21:28:29.654 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:28:29.654 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:28:29.654 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:28:29.654 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:28:29.654 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:28:29.654 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:28:29.654 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-10T21:28:29.654 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-10T21:28:29.654 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:28:29.654 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:28:29.654 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:28:29.655 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:28:29.655 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:28:29.655 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:28:29.655 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-10T21:28:29.655 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:28:29.655 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:28:29.655 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:28:29.655 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:28:29.655 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:28:29.655 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:28:29.655 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:28:29.655 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-10T21:28:29.656 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-10T21:28:29.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.636+0000 7f0424800640 1 -- 172.21.15.17:0/256419312 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f0400072510 msgr2=0x7f04000749d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:29.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.636+0000 7f0424800640 1 --2- 172.21.15.17:0/256419312 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f0400072510 0x7f04000749d0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f041c076910 tx=0x7f0418009500 comp rx=0 tx=0).stop 2024-09-10T21:28:29.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.636+0000 7f0424800640 1 -- 172.21.15.17:0/256419312 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f0404004810 msgr2=0x7f041c15af80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:29.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.636+0000 7f0424800640 1 --2- 172.21.15.17:0/256419312 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f0404004810 0x7f041c15af80 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f0410006360 tx=0x7f04100039d0 comp rx=0 tx=0).stop 2024-09-10T21:28:29.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.636+0000 7f0422800640 1 -- 172.21.15.17:0/256419312 reap_dead start 2024-09-10T21:28:29.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.636+0000 7f0422800640 1 -- 172.21.15.17:0/256419312 reap_dead start 2024-09-10T21:28:29.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.636+0000 7f0424800640 1 -- 172.21.15.17:0/256419312 shutdown_connections 2024-09-10T21:28:29.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.636+0000 7f0424800640 1 -- 172.21.15.17:0/256419312 >> 172.21.15.17:0/256419312 conn(0x7f041c0702f0 msgr2=0x7f041c151960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:28:29.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.636+0000 7f0424800640 1 -- 172.21.15.17:0/256419312 shutdown_connections 2024-09-10T21:28:29.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.636+0000 7f0424800640 1 -- 172.21.15.17:0/256419312 wait complete. 2024-09-10T21:28:29.661 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-10T21:28:29.661 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-09-10T21:28:29.661 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-10T21:28:29.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.828+0000 7f3217600640 1 -- 172.21.15.17:0/704313861 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f32101546c0 msgr2=0x7f32100ca430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:29.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.828+0000 7f3217600640 1 --2- 172.21.15.17:0/704313861 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f32101546c0 0x7f32100ca430 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f32100cbac0 tx=0x7f320401c890 comp rx=0 tx=0).stop 2024-09-10T21:28:29.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.828+0000 7f3217600640 1 -- 172.21.15.17:0/704313861 shutdown_connections 2024-09-10T21:28:29.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.828+0000 7f3217600640 1 --2- 172.21.15.17:0/704313861 >> v2:172.21.15.31:3300/0 conn(0x7f32100725c0 0x7f32100cdf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:29.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.828+0000 7f3217600640 1 --2- 172.21.15.17:0/704313861 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3210154aa0 0x7f3210072080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:29.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.828+0000 7f3217600640 1 --2- 172.21.15.17:0/704313861 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f32101546c0 0x7f32100ca430 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:29.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.828+0000 7f3217600640 1 -- 172.21.15.17:0/704313861 >> 172.21.15.17:0/704313861 conn(0x7f32100702f0 msgr2=0x7f3210070700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:28:29.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.828+0000 7f3217600640 1 -- 172.21.15.17:0/704313861 shutdown_connections 2024-09-10T21:28:29.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.828+0000 7f3217600640 1 -- 172.21.15.17:0/704313861 wait complete. 2024-09-10T21:28:29.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.828+0000 7f3217600640 1 Processor -- start 2024-09-10T21:28:29.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.832+0000 7f3217600640 1 -- start start 2024-09-10T21:28:29.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.832+0000 7f3217600640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f32100725c0 0x7f3210154e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:29.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.832+0000 7f3217600640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f32101546c0 0x7f32101553c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:29.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.832+0000 7f3217600640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3210154aa0 0x7f32101566c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:29.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.832+0000 7f3217600640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f32100c97c0 con 0x7f32101546c0 2024-09-10T21:28:29.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.832+0000 7f3217600640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f32100c9640 con 0x7f32100725c0 2024-09-10T21:28:29.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.832+0000 7f3217600640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f321015e350 con 0x7f3210154aa0 2024-09-10T21:28:29.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.832+0000 7f320fe00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f32101546c0 0x7f32101553c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:29.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.832+0000 7f320fe00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f32101546c0 0x7f32101553c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42628/0 (socket says 172.21.15.17:42628) 2024-09-10T21:28:29.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.832+0000 7f320fe00640 1 -- 172.21.15.17:0/1241450845 learned_addr learned my addr 172.21.15.17:0/1241450845 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:28:29.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.832+0000 7f3214c00640 1 --2- 172.21.15.17:0/1241450845 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f32100725c0 0x7f3210154e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:29.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.832+0000 7f3215600640 1 --2- 172.21.15.17:0/1241450845 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3210154aa0 0x7f32101566c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:29.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.832+0000 7f320fe00640 1 -- 172.21.15.17:0/1241450845 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3210154aa0 msgr2=0x7f32101566c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:29.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.832+0000 7f320fe00640 1 --2- 172.21.15.17:0/1241450845 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3210154aa0 0x7f32101566c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:29.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.832+0000 7f320fe00640 1 -- 172.21.15.17:0/1241450845 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f32100725c0 msgr2=0x7f3210154e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:29.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.832+0000 7f320fe00640 1 --2- 172.21.15.17:0/1241450845 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f32100725c0 0x7f3210154e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:29.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.832+0000 7f320fe00640 1 -- 172.21.15.17:0/1241450845 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3210158200 con 0x7f32101546c0 2024-09-10T21:28:29.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.832+0000 7f3215600640 1 --2- 172.21.15.17:0/1241450845 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3210154aa0 0x7f32101566c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:28:29.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.832+0000 7f320fe00640 1 --2- 172.21.15.17:0/1241450845 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f32101546c0 0x7f32101553c0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f31fc00d750 tx=0x7f31fc00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:28:29.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.832+0000 7f320cc00640 1 -- 172.21.15.17:0/1241450845 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31fc017070 con 0x7f32101546c0 2024-09-10T21:28:29.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.832+0000 7f320cc00640 1 -- 172.21.15.17:0/1241450845 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f31fc009d20 con 0x7f32101546c0 2024-09-10T21:28:29.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.832+0000 7f320cc00640 1 -- 172.21.15.17:0/1241450845 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31fc00de60 con 0x7f32101546c0 2024-09-10T21:28:29.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.832+0000 7f3217600640 1 -- 172.21.15.17:0/1241450845 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3210156480 con 0x7f32101546c0 2024-09-10T21:28:29.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.832+0000 7f3217600640 1 -- 172.21.15.17:0/1241450845 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3210155c20 con 0x7f32101546c0 2024-09-10T21:28:29.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.832+0000 7f3217600640 1 --2- 172.21.15.17:0/1241450845 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f32101f4040 0x7f3210156c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:29.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.832+0000 7f3217600640 1 -- 172.21.15.17:0/1241450845 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f32101f4420 con 0x7f32101f4040 2024-09-10T21:28:29.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.832+0000 7f3214c00640 1 --2- 172.21.15.17:0/1241450845 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f32101f4040 0x7f3210156c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:29.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.836+0000 7f3214c00640 1 -- 172.21.15.17:0/1241450845 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3210151540 con 0x7f32101f4040 2024-09-10T21:28:29.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.836+0000 7f3214c00640 1 --2- 172.21.15.17:0/1241450845 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f32101f4040 0x7f3210156c00 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f3210073220 tx=0x7f32040095f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:28:29.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.836+0000 7f320cc00640 1 -- 172.21.15.17:0/1241450845 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f31fc004280 con 0x7f32101546c0 2024-09-10T21:28:29.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.836+0000 7f320cc00640 1 --2- 172.21.15.17:0/1241450845 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f31f00725e0 0x7f31f0074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:29.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.836+0000 7f320cc00640 1 -- 172.21.15.17:0/1241450845 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32040b2040 con 0x7f32101f4040 2024-09-10T21:28:29.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.836+0000 7f320cc00640 1 -- 172.21.15.17:0/1241450845 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f31fc093a90 con 0x7f32101546c0 2024-09-10T21:28:29.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.836+0000 7f3215600640 1 --2- 172.21.15.17:0/1241450845 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f31f00725e0 0x7f31f0074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:29.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.836+0000 7f320cc00640 1 -- 172.21.15.17:0/1241450845 <== mon.1 v2:172.21.15.31:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f3204005220 con 0x7f32101f4040 2024-09-10T21:28:29.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.836+0000 7f320cc00640 1 -- 172.21.15.17:0/1241450845 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f32101f4040 msgr2=0x7f3210156c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:29.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.836+0000 7f320cc00640 1 --2- 172.21.15.17:0/1241450845 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f32101f4040 0x7f3210156c00 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f3210073220 tx=0x7f32040095f0 comp rx=0 tx=0).stop 2024-09-10T21:28:29.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.836+0000 7f320cc00640 1 -- 172.21.15.17:0/1241450845 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f32101f4040 msgr2=0x7f3210156c00 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:28:29.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.836+0000 7f320cc00640 1 --2- 172.21.15.17:0/1241450845 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f32101f4040 0x7f3210156c00 secure :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f3210073220 tx=0x7f32040095f0 comp rx=0 tx=0).stop 2024-09-10T21:28:29.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.836+0000 7f3215600640 1 --2- 172.21.15.17:0/1241450845 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f31f00725e0 0x7f31f0074aa0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f32100729c0 tx=0x7f3208009500 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:28:29.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.844+0000 7f3217600640 1 --2- 172.21.15.17:0/1241450845 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f32100c7520 0x7f32100c7900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:29.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.844+0000 7f3217600640 1 -- 172.21.15.17:0/1241450845 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f321015e7b0 con 0x7f32100c7520 2024-09-10T21:28:29.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.848+0000 7f3214c00640 1 --2- 172.21.15.17:0/1241450845 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f32100c7520 0x7f32100c7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:29.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.848+0000 7f3214c00640 1 -- 172.21.15.17:0/1241450845 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f3210152690 con 0x7f32100c7520 2024-09-10T21:28:29.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.848+0000 7f3214c00640 1 --2- 172.21.15.17:0/1241450845 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f32100c7520 0x7f32100c7900 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f320401d130 tx=0x7f3204009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:28:29.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.848+0000 7f320cc00640 1 -- 172.21.15.17:0/1241450845 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32040b2040 con 0x7f32100c7520 2024-09-10T21:28:29.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.848+0000 7f320cc00640 1 -- 172.21.15.17:0/1241450845 <== mon.1 v2:172.21.15.31:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5043 (secure 0 0 0) 0x7f32040046d0 con 0x7f32100c7520 2024-09-10T21:28:29.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.848+0000 7f320cc00640 1 -- 172.21.15.17:0/1241450845 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f32100c7520 msgr2=0x7f32100c7900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:29.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.848+0000 7f320cc00640 1 --2- 172.21.15.17:0/1241450845 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f32100c7520 0x7f32100c7900 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f320401d130 tx=0x7f3204009260 comp rx=0 tx=0).stop 2024-09-10T21:28:29.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.848+0000 7f320cc00640 1 -- 172.21.15.17:0/1241450845 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f32100c7520 msgr2=0x7f32100c7900 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:28:29.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.848+0000 7f320cc00640 1 --2- 172.21.15.17:0/1241450845 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f32100c7520 0x7f32100c7900 secure :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f320401d130 tx=0x7f3204009260 comp rx=0 tx=0).stop 2024-09-10T21:28:29.856 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-10T21:28:29.856 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:28:29.856 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:28:29.856 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-10T21:28:29.856 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 30, 2024-09-10T21:28:29.856 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 248295, 2024-09-10T21:28:29.856 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-10T21:28:29.856 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-10T21:28:29.856 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-10T21:28:29.856 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-10T21:28:29.857 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:28:29.857 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 12, 2024-09-10T21:28:29.857 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:28:29.857 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-10T21:28:29.857 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-10T21:28:29.857 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:28:29.857 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:28:29.857 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:28:29.857 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:28:29.857 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:28:29.857 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:28:29.857 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:28:29.857 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:28:29.857 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:28:29.858 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:28:29.858 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:28:29.858 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:28:29.858 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-10T21:28:29.858 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-10T21:28:29.858 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:28:29.858 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:28:29.858 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:28:29.858 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:28:29.858 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:28:29.858 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:28:29.858 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:28:29.858 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:28:29.858 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:28:29.858 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:28:29.859 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:28:29.859 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:28:29.859 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:28:29.859 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-10T21:28:29.859 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-10T21:28:29.859 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-10T21:28:29.859 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-10T21:28:29.859 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-10T21:28:29.859 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "5e2f023f-d6a6-49f4-a256-f4807dce3739", 2024-09-10T21:28:29.859 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:28:29.859 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:28:29.859 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-10T21:28:29.859 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-10T21:28:29.860 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-10T21:28:29.860 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-10T21:28:29.860 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-10T21:28:29.860 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-10T21:28:29.860 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-10T21:28:29.860 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:28:29.860 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-10T21:28:29.860 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:28:29.860 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:28:29.860 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:28:29.860 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:28:29.860 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:28:29.860 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:28:29.860 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:28:29.861 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:28:29.861 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:28:29.861 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:28:29.861 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:28:29.861 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:28:29.861 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-10T21:28:29.861 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:28:29.861 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-10T21:28:29.861 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:28:29.862 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:28:29.862 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:28:29.862 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:28:29.862 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:28:29.862 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:28:29.862 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:28:29.862 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-10T21:28:29.862 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:28:29.862 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:28:29.862 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:28:29.862 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:28:29.862 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-10T21:28:29.862 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:28:29.862 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:28:29.863 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:28:29.863 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:28:29.863 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-10T21:28:29.863 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-10T21:28:29.863 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:28:29.863 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:28:29.863 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:28:29.863 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:28:29.863 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:28:29.863 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:28:29.863 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:28:29.863 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:28:29.863 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:28:29.864 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:28:29.864 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:28:29.864 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3300", 2024-09-10T21:28:29.864 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:28:29.864 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:28:29.864 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:28:29.864 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:28:29.864 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789", 2024-09-10T21:28:29.864 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:28:29.864 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:28:29.864 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:28:29.864 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:28:29.864 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789/0", 2024-09-10T21:28:29.864 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6789/0", 2024-09-10T21:28:29.865 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:28:29.865 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:28:29.865 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:28:29.865 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:28:29.865 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:28:29.865 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:28:29.865 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:28:29.865 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:28:29.865 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:28:29.865 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:28:29.865 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:28:29.865 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3301", 2024-09-10T21:28:29.865 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:28:29.866 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:28:29.866 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:28:29.866 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:28:29.866 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790", 2024-09-10T21:28:29.866 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:28:29.866 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:28:29.866 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:28:29.866 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:28:29.866 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790/0", 2024-09-10T21:28:29.867 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6790/0", 2024-09-10T21:28:29.867 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:28:29.867 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:28:29.867 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:28:29.867 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:28:29.867 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:28:29.867 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:28:29.867 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-10T21:28:29.867 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-10T21:28:29.867 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:28:29.867 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:28:29.867 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:28:29.867 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:28:29.867 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:28:29.868 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:28:29.868 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": [ 2024-09-10T21:28:29.868 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:28:29.868 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:28:29.868 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:28:29.868 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 5 2024-09-10T21:28:29.868 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:28:29.868 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:28:29.868 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-10T21:28:29.868 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:28:29.868 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:28:29.869 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:28:29.869 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 4 2024-09-10T21:28:29.869 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:28:29.869 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:28:29.869 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-10T21:28:29.869 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:28:29.869 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:28:29.869 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:28:29.869 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 4 2024-09-10T21:28:29.869 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:28:29.869 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:28:29.869 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": [ 2024-09-10T21:28:29.869 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:28:29.869 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:28:29.870 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:28:29.870 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 2 2024-09-10T21:28:29.870 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:28:29.870 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:28:29.870 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:28:29.870 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-10T21:28:29.870 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-10T21:28:29.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.848+0000 7f3217600640 1 -- 172.21.15.17:0/1241450845 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f31f00725e0 msgr2=0x7f31f0074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:29.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.848+0000 7f3217600640 1 --2- 172.21.15.17:0/1241450845 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f31f00725e0 0x7f31f0074aa0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f32100729c0 tx=0x7f3208009500 comp rx=0 tx=0).stop 2024-09-10T21:28:29.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.848+0000 7f3217600640 1 -- 172.21.15.17:0/1241450845 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f32101546c0 msgr2=0x7f32101553c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:29.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.848+0000 7f3217600640 1 --2- 172.21.15.17:0/1241450845 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f32101546c0 0x7f32101553c0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f31fc00d750 tx=0x7f31fc00dc10 comp rx=0 tx=0).stop 2024-09-10T21:28:29.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.848+0000 7f3215600640 1 -- 172.21.15.17:0/1241450845 reap_dead start 2024-09-10T21:28:29.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.848+0000 7f3217600640 1 -- 172.21.15.17:0/1241450845 shutdown_connections 2024-09-10T21:28:29.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.848+0000 7f3217600640 1 --2- 172.21.15.17:0/1241450845 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f32101546c0 0x7f32101553c0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:29.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.848+0000 7f3217600640 1 -- 172.21.15.17:0/1241450845 >> 172.21.15.17:0/1241450845 conn(0x7f32100702f0 msgr2=0x7f32100706d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:28:29.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.848+0000 7f3217600640 1 -- 172.21.15.17:0/1241450845 shutdown_connections 2024-09-10T21:28:29.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:29.848+0000 7f3217600640 1 -- 172.21.15.17:0/1241450845 wait complete. 2024-09-10T21:28:29.873 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-10T21:28:29.873 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-09-10T21:28:29.873 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-10T21:28:30.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:30.052+0000 7fdfadc00640 1 -- 172.21.15.17:0/2439722347 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdfa80c0fd0 msgr2=0x7fdfa815a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:30.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:30.052+0000 7fdfadc00640 1 --2- 172.21.15.17:0/2439722347 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdfa80c0fd0 0x7fdfa815a4a0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fdfa8074860 tx=0x7fdf9001c8c0 comp rx=0 tx=0).stop 2024-09-10T21:28:30.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:30.052+0000 7fdfadc00640 1 -- 172.21.15.17:0/2439722347 shutdown_connections 2024-09-10T21:28:30.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:30.052+0000 7fdfadc00640 1 --2- 172.21.15.17:0/2439722347 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdfa80c0fd0 0x7fdfa815a4a0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:30.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:30.052+0000 7fdfadc00640 1 --2- 172.21.15.17:0/2439722347 >> v2:172.21.15.31:3300/0 conn(0x7fdfa8073cf0 0x7fdfa80740d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:30.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:30.052+0000 7fdfadc00640 1 -- 172.21.15.17:0/2439722347 >> 172.21.15.17:0/2439722347 conn(0x7fdfa80700d0 msgr2=0x7fdfa80704e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:28:30.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:30.052+0000 7fdfadc00640 1 -- 172.21.15.17:0/2439722347 shutdown_connections 2024-09-10T21:28:30.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:30.052+0000 7fdfadc00640 1 -- 172.21.15.17:0/2439722347 wait complete. 2024-09-10T21:28:30.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:30.052+0000 7fdfadc00640 1 Processor -- start 2024-09-10T21:28:30.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:30.052+0000 7fdfadc00640 1 -- start start 2024-09-10T21:28:30.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:30.056+0000 7fdfadc00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdfa8073cf0 0x7fdfa815f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:30.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:30.056+0000 7fdfadc00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdfa80c0fd0 0x7fdfa813e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:30.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:30.056+0000 7fdfadc00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fdfa815a9e0 0x7fdfa813eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:30.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:30.056+0000 7fdfadc00640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fdfa80c8370 con 0x7fdfa8073cf0 2024-09-10T21:28:30.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:30.056+0000 7fdfadc00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fdfa806e640 con 0x7fdfa80c0fd0 2024-09-10T21:28:30.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:30.056+0000 7fdfadc00640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7fdfa80c7dd0 con 0x7fdfa815a9e0 2024-09-10T21:28:30.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:30.056+0000 7fdfa6a00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdfa8073cf0 0x7fdfa815f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:30.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:30.056+0000 7fdf9fe00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdfa80c0fd0 0x7fdfa813e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:30.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:30.056+0000 7fdfa6a00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdfa8073cf0 0x7fdfa815f660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42644/0 (socket says 172.21.15.17:42644) 2024-09-10T21:28:30.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:30.056+0000 7fdfa6a00640 1 -- 172.21.15.17:0/3778925083 learned_addr learned my addr 172.21.15.17:0/3778925083 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:28:30.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:30.056+0000 7fdfa7400640 1 --2- 172.21.15.17:0/3778925083 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fdfa815a9e0 0x7fdfa813eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:30.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:30.056+0000 7fdfa6a00640 1 -- 172.21.15.17:0/3778925083 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fdfa815a9e0 msgr2=0x7fdfa813eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:30.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:30.056+0000 7fdfa6a00640 1 --2- 172.21.15.17:0/3778925083 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fdfa815a9e0 0x7fdfa813eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:30.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:30.056+0000 7fdfa6a00640 1 -- 172.21.15.17:0/3778925083 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdfa80c0fd0 msgr2=0x7fdfa813e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:30.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:30.056+0000 7fdfa6a00640 1 --2- 172.21.15.17:0/3778925083 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdfa80c0fd0 0x7fdfa813e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:30.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:30.056+0000 7fdfa6a00640 1 -- 172.21.15.17:0/3778925083 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfa814ad40 con 0x7fdfa8073cf0 2024-09-10T21:28:30.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:30.056+0000 7fdf9fe00640 1 --2- 172.21.15.17:0/3778925083 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdfa80c0fd0 0x7fdfa813e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:28:30.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:30.056+0000 7fdfa6a00640 1 --2- 172.21.15.17:0/3778925083 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdfa8073cf0 0x7fdfa815f660 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fdf9800b550 tx=0x7fdf9800ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:28:30.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:30.056+0000 7fdf9ea00640 1 -- 172.21.15.17:0/3778925083 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf98014070 con 0x7fdfa8073cf0 2024-09-10T21:28:30.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:30.056+0000 7fdf9ea00640 1 -- 172.21.15.17:0/3778925083 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdf98010980 con 0x7fdfa8073cf0 2024-09-10T21:28:30.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:30.056+0000 7fdfadc00640 1 -- 172.21.15.17:0/3778925083 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfa8156890 con 0x7fdfa8073cf0 2024-09-10T21:28:30.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:30.056+0000 7fdf9ea00640 1 -- 172.21.15.17:0/3778925083 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf98010dd0 con 0x7fdfa8073cf0 2024-09-10T21:28:30.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:30.056+0000 7fdfadc00640 1 -- 172.21.15.17:0/3778925083 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfa8157c80 con 0x7fdfa8073cf0 2024-09-10T21:28:30.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:30.056+0000 7fdfadc00640 1 --2- 172.21.15.17:0/3778925083 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fdfa8157ec0 0x7fdfa813c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:30.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:30.056+0000 7fdfadc00640 1 -- 172.21.15.17:0/3778925083 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7fdfa80752a0 con 0x7fdfa8157ec0 2024-09-10T21:28:30.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:30.056+0000 7fdf9fe00640 1 --2- 172.21.15.17:0/3778925083 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fdfa8157ec0 0x7fdfa813c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:30.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:30.060+0000 7fdf9fe00640 1 -- 172.21.15.17:0/3778925083 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fdfa8151050 con 0x7fdfa8157ec0 2024-09-10T21:28:30.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:30.060+0000 7fdf9ea00640 1 -- 172.21.15.17:0/3778925083 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fdf9800f3c0 con 0x7fdfa8073cf0 2024-09-10T21:28:30.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:30.060+0000 7fdf9fe00640 1 --2- 172.21.15.17:0/3778925083 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fdfa8157ec0 0x7fdfa813c820 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fdfa806fc30 tx=0x7fdf900a7f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:28:30.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:30.060+0000 7fdf9ea00640 1 --2- 172.21.15.17:0/3778925083 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fdf840725e0 0x7fdf84074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:30.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:30.060+0000 7fdf9ea00640 1 -- 172.21.15.17:0/3778925083 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fdf980936b0 con 0x7fdfa8073cf0 2024-09-10T21:28:30.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:30.060+0000 7fdf9ea00640 1 -- 172.21.15.17:0/3778925083 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf900bb030 con 0x7fdfa8157ec0 2024-09-10T21:28:30.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:30.060+0000 7fdfa7400640 1 --2- 172.21.15.17:0/3778925083 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fdf840725e0 0x7fdf84074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:30.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:30.060+0000 7fdf9ea00640 1 -- 172.21.15.17:0/3778925083 <== mon.2 v2:172.21.15.31:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fdf900b3040 con 0x7fdfa8157ec0 2024-09-10T21:28:30.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:30.060+0000 7fdf9ea00640 1 -- 172.21.15.17:0/3778925083 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fdfa8157ec0 msgr2=0x7fdfa813c820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:30.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:30.060+0000 7fdf9ea00640 1 --2- 172.21.15.17:0/3778925083 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fdfa8157ec0 0x7fdfa813c820 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fdfa806fc30 tx=0x7fdf900a7f90 comp rx=0 tx=0).stop 2024-09-10T21:28:30.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:30.060+0000 7fdf9ea00640 1 -- 172.21.15.17:0/3778925083 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fdfa8157ec0 msgr2=0x7fdfa813c820 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:28:30.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:30.060+0000 7fdf9ea00640 1 --2- 172.21.15.17:0/3778925083 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fdfa8157ec0 0x7fdfa813c820 secure :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fdfa806fc30 tx=0x7fdf900a7f90 comp rx=0 tx=0).stop 2024-09-10T21:28:30.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:30.060+0000 7fdfa7400640 1 --2- 172.21.15.17:0/3778925083 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fdf840725e0 0x7fdf84074aa0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fdfa0009f60 tx=0x7fdfa000d040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:28:30.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:30.064+0000 7fdfadc00640 1 --2- 172.21.15.17:0/3778925083 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fdfa813cd60 0x7fdfa81f6db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:30.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:30.064+0000 7fdfadc00640 1 -- 172.21.15.17:0/3778925083 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7fdfa8074e50 con 0x7fdfa813cd60 2024-09-10T21:28:30.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:30.068+0000 7fdf9fe00640 1 --2- 172.21.15.17:0/3778925083 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fdfa813cd60 0x7fdfa81f6db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:30.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:30.068+0000 7fdf9fe00640 1 -- 172.21.15.17:0/3778925083 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fdfa80c8150 con 0x7fdfa813cd60 2024-09-10T21:28:30.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:30.068+0000 7fdf9fe00640 1 --2- 172.21.15.17:0/3778925083 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fdfa813cd60 0x7fdfa81f6db0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fdf9001af70 tx=0x7fdf90009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:28:30.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:30.068+0000 7fdf9ea00640 1 -- 172.21.15.17:0/3778925083 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf900bb040 con 0x7fdfa813cd60 2024-09-10T21:28:30.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:30.068+0000 7fdf9ea00640 1 -- 172.21.15.17:0/3778925083 <== mon.2 v2:172.21.15.31:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4875 (secure 0 0 0) 0x7fdf90004ba0 con 0x7fdfa813cd60 2024-09-10T21:28:30.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:30.068+0000 7fdf9ea00640 1 -- 172.21.15.17:0/3778925083 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fdfa813cd60 msgr2=0x7fdfa81f6db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:30.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:30.068+0000 7fdf9ea00640 1 --2- 172.21.15.17:0/3778925083 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fdfa813cd60 0x7fdfa81f6db0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fdf9001af70 tx=0x7fdf90009260 comp rx=0 tx=0).stop 2024-09-10T21:28:30.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:30.068+0000 7fdf9ea00640 1 -- 172.21.15.17:0/3778925083 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fdfa813cd60 msgr2=0x7fdfa81f6db0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:28:30.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:30.068+0000 7fdf9ea00640 1 --2- 172.21.15.17:0/3778925083 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fdfa813cd60 0x7fdfa81f6db0 secure :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fdf9001af70 tx=0x7fdf90009260 comp rx=0 tx=0).stop 2024-09-10T21:28:30.074 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-10T21:28:30.074 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:28:30.074 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:28:30.074 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-10T21:28:30.074 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 30, 2024-09-10T21:28:30.074 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 91183, 2024-09-10T21:28:30.075 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-10T21:28:30.075 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-10T21:28:30.075 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-10T21:28:30.075 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-10T21:28:30.075 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:28:30.075 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 12, 2024-09-10T21:28:30.075 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:28:30.075 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-10T21:28:30.075 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-10T21:28:30.075 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:28:30.075 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:28:30.075 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:28:30.075 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:28:30.075 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:28:30.076 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:28:30.076 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:28:30.076 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:28:30.076 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:28:30.076 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:28:30.076 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:28:30.076 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:28:30.076 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-10T21:28:30.076 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-10T21:28:30.076 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:28:30.076 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:28:30.076 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:28:30.076 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:28:30.077 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:28:30.077 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:28:30.077 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:28:30.077 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:28:30.077 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:28:30.077 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:28:30.077 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:28:30.077 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:28:30.077 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:28:30.077 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-10T21:28:30.077 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-10T21:28:30.078 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-10T21:28:30.078 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-10T21:28:30.078 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-10T21:28:30.078 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "5e2f023f-d6a6-49f4-a256-f4807dce3739", 2024-09-10T21:28:30.078 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:28:30.078 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:28:30.078 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-10T21:28:30.078 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-10T21:28:30.078 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-10T21:28:30.078 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-10T21:28:30.078 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-10T21:28:30.079 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-10T21:28:30.079 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-10T21:28:30.079 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:28:30.079 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-10T21:28:30.079 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:28:30.079 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:28:30.079 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:28:30.079 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:28:30.079 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:28:30.079 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:28:30.079 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:28:30.079 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:28:30.079 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:28:30.079 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:28:30.079 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:28:30.080 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:28:30.080 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-10T21:28:30.080 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:28:30.080 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-10T21:28:30.080 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:28:30.080 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:28:30.080 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:28:30.080 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:28:30.080 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:28:30.080 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:28:30.080 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:28:30.080 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-10T21:28:30.080 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:28:30.080 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:28:30.081 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:28:30.081 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:28:30.081 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-10T21:28:30.081 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:28:30.081 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:28:30.081 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:28:30.081 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:28:30.081 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-10T21:28:30.081 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-10T21:28:30.081 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:28:30.081 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:28:30.081 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:28:30.081 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:28:30.082 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:28:30.082 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:28:30.082 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:28:30.082 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:28:30.082 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:28:30.082 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:28:30.082 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:28:30.082 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3300", 2024-09-10T21:28:30.082 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:28:30.082 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:28:30.082 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:28:30.082 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:28:30.082 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789", 2024-09-10T21:28:30.082 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:28:30.083 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:28:30.083 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:28:30.083 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:28:30.083 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789/0", 2024-09-10T21:28:30.083 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6789/0", 2024-09-10T21:28:30.083 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:28:30.083 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:28:30.083 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:28:30.083 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:28:30.083 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:28:30.083 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:28:30.083 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:28:30.083 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:28:30.083 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:28:30.084 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:28:30.084 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:28:30.084 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3301", 2024-09-10T21:28:30.084 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:28:30.084 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:28:30.084 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:28:30.084 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:28:30.084 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790", 2024-09-10T21:28:30.084 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:28:30.084 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:28:30.084 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:28:30.084 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:28:30.084 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790/0", 2024-09-10T21:28:30.085 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6790/0", 2024-09-10T21:28:30.085 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:28:30.085 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:28:30.085 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:28:30.085 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:28:30.085 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:28:30.085 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:28:30.085 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-10T21:28:30.085 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-10T21:28:30.085 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:28:30.085 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:28:30.085 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:28:30.086 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:28:30.086 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:28:30.086 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:28:30.086 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": [ 2024-09-10T21:28:30.086 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:28:30.086 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:28:30.086 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:28:30.086 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:28:30.086 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:28:30.086 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:28:30.086 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-10T21:28:30.086 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:28:30.086 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:28:30.086 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:28:30.086 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 4 2024-09-10T21:28:30.087 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:28:30.087 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:28:30.087 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-10T21:28:30.087 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:28:30.087 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:28:30.087 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:28:30.087 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 3 2024-09-10T21:28:30.087 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:28:30.087 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:28:30.087 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:28:30.087 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-10T21:28:30.087 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-10T21:28:30.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:30.068+0000 7fdfadc00640 1 -- 172.21.15.17:0/3778925083 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fdf840725e0 msgr2=0x7fdf84074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:30.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:30.068+0000 7fdfadc00640 1 --2- 172.21.15.17:0/3778925083 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fdf840725e0 0x7fdf84074aa0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fdfa0009f60 tx=0x7fdfa000d040 comp rx=0 tx=0).stop 2024-09-10T21:28:30.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:30.068+0000 7fdfadc00640 1 -- 172.21.15.17:0/3778925083 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdfa8073cf0 msgr2=0x7fdfa815f660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:30.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:30.068+0000 7fdfadc00640 1 --2- 172.21.15.17:0/3778925083 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdfa8073cf0 0x7fdfa815f660 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fdf9800b550 tx=0x7fdf9800ba10 comp rx=0 tx=0).stop 2024-09-10T21:28:30.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:30.068+0000 7fdfa7400640 1 -- 172.21.15.17:0/3778925083 reap_dead start 2024-09-10T21:28:30.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:30.068+0000 7fdfadc00640 1 -- 172.21.15.17:0/3778925083 shutdown_connections 2024-09-10T21:28:30.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:30.068+0000 7fdfadc00640 1 --2- 172.21.15.17:0/3778925083 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdfa8073cf0 0x7fdfa815f660 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:30.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:30.068+0000 7fdfadc00640 1 -- 172.21.15.17:0/3778925083 >> 172.21.15.17:0/3778925083 conn(0x7fdfa80700d0 msgr2=0x7fdfa806dc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:28:30.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:30.068+0000 7fdfadc00640 1 -- 172.21.15.17:0/3778925083 shutdown_connections 2024-09-10T21:28:30.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:30.068+0000 7fdfadc00640 1 -- 172.21.15.17:0/3778925083 wait complete. 2024-09-10T21:28:30.097 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-10T21:28:30.097 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['c'] 2024-09-10T21:28:30.097 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2024-09-10T21:28:30.097 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.c 2024-09-10T21:28:30.097 INFO:tasks.mon_thrash.mon_thrasher:killing mon.c 2024-09-10T21:28:30.098 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-10T21:28:30.098 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:28:30.117 INFO:tasks.ceph.mon.c:Stopped 2024-09-10T21:28:30.117 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2024-09-10T21:28:30.117 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-10T21:28:30.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:30.280+0000 7fcc4ec00640 1 -- 172.21.15.17:0/1713483874 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcc48158310 msgr2=0x7fcc4806fd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:30.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:30.280+0000 7fcc4ec00640 1 --2- 172.21.15.17:0/1713483874 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcc48158310 0x7fcc4806fd00 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fcc40005fd0 tx=0x7fcc4001f390 comp rx=0 tx=0).stop 2024-09-10T21:28:30.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:30.280+0000 7fcc4ec00640 1 -- 172.21.15.17:0/1713483874 shutdown_connections 2024-09-10T21:28:30.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:30.280+0000 7fcc4ec00640 1 --2- 172.21.15.17:0/1713483874 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcc48158310 0x7fcc4806fd00 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:30.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:30.280+0000 7fcc4ec00640 1 -- 172.21.15.17:0/1713483874 >> 172.21.15.17:0/1713483874 conn(0x7fcc48070ca0 msgr2=0x7fcc480710b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:28:30.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:30.280+0000 7fcc4ec00640 1 -- 172.21.15.17:0/1713483874 shutdown_connections 2024-09-10T21:28:30.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:30.280+0000 7fcc4ec00640 1 -- 172.21.15.17:0/1713483874 wait complete. 2024-09-10T21:28:30.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:30.284+0000 7fcc4ec00640 1 Processor -- start 2024-09-10T21:28:30.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:30.284+0000 7fcc4ec00640 1 -- start start 2024-09-10T21:28:30.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:30.284+0000 7fcc4ec00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcc480b2d80 0x7fcc48150060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:30.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:30.284+0000 7fcc4ec00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fcc481527e0 0x7fcc4814ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:30.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:30.284+0000 7fcc4ec00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcc48158310 0x7fcc480c3140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:30.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:30.284+0000 7fcc4ec00640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fcc4806ed40 con 0x7fcc48158310 2024-09-10T21:28:30.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:30.284+0000 7fcc4ec00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fcc4806f500 con 0x7fcc480b2d80 2024-09-10T21:28:30.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:30.284+0000 7fcc4ec00640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7fcc4806eec0 con 0x7fcc481527e0 2024-09-10T21:28:30.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:30.284+0000 7fcc4ce00640 1 -- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fcc481527e0 msgr2=0x7fcc4814ee60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:3301/0 2024-09-10T21:28:30.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:30.284+0000 7fcc4ce00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fcc481527e0 0x7fcc4814ee60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:28:30.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:30.284+0000 7fcc4e200640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcc48158310 0x7fcc480c3140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:30.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:30.284+0000 7fcc4e200640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcc48158310 0x7fcc480c3140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42686/0 (socket says 172.21.15.17:42686) 2024-09-10T21:28:30.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:30.284+0000 7fcc4e200640 1 -- 172.21.15.17:0/4105424177 learned_addr learned my addr 172.21.15.17:0/4105424177 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:28:30.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:30.284+0000 7fcc4d800640 1 --2- 172.21.15.17:0/4105424177 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcc480b2d80 0x7fcc48150060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:30.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:30.284+0000 7fcc4e200640 1 -- 172.21.15.17:0/4105424177 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fcc481527e0 msgr2=0x7fcc4814ee60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:28:30.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:30.284+0000 7fcc4e200640 1 --2- 172.21.15.17:0/4105424177 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fcc481527e0 0x7fcc4814ee60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:30.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:30.284+0000 7fcc4e200640 1 -- 172.21.15.17:0/4105424177 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcc480b2d80 msgr2=0x7fcc48150060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:30.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:30.284+0000 7fcc4e200640 1 --2- 172.21.15.17:0/4105424177 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcc480b2d80 0x7fcc48150060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:30.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:30.284+0000 7fcc4e200640 1 -- 172.21.15.17:0/4105424177 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc4814faa0 con 0x7fcc48158310 2024-09-10T21:28:30.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:30.284+0000 7fcc4e200640 1 --2- 172.21.15.17:0/4105424177 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcc48158310 0x7fcc480c3140 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fcc40009b30 tx=0x7fcc400092a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:28:30.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:30.284+0000 7fcc45600640 1 -- 172.21.15.17:0/4105424177 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc400b4040 con 0x7fcc48158310 2024-09-10T21:28:30.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:30.284+0000 7fcc4ec00640 1 -- 172.21.15.17:0/4105424177 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc4814fd80 con 0x7fcc48158310 2024-09-10T21:28:30.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:30.284+0000 7fcc45600640 1 -- 172.21.15.17:0/4105424177 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcc40004c90 con 0x7fcc48158310 2024-09-10T21:28:30.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:30.284+0000 7fcc45600640 1 -- 172.21.15.17:0/4105424177 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc400afb00 con 0x7fcc48158310 2024-09-10T21:28:30.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:30.284+0000 7fcc4ec00640 1 -- 172.21.15.17:0/4105424177 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc4814f3a0 con 0x7fcc48158310 2024-09-10T21:28:30.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:30.288+0000 7fcc4ec00640 1 -- 172.21.15.17:0/4105424177 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc4806fd00 con 0x7fcc48158310 2024-09-10T21:28:30.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:30.288+0000 7fcc45600640 1 -- 172.21.15.17:0/4105424177 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fcc400afca0 con 0x7fcc48158310 2024-09-10T21:28:30.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:30.288+0000 7fcc45600640 1 --2- 172.21.15.17:0/4105424177 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fcc28072590 0x7fcc28074a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:30.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:30.288+0000 7fcc45600640 1 -- 172.21.15.17:0/4105424177 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fcc401314e0 con 0x7fcc48158310 2024-09-10T21:28:30.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:30.288+0000 7fcc4d800640 1 --2- 172.21.15.17:0/4105424177 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fcc28072590 0x7fcc28074a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:30.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:30.292+0000 7fcc4d800640 1 --2- 172.21.15.17:0/4105424177 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fcc28072590 0x7fcc28074a50 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fcc3c007fd0 tx=0x7fcc3c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:28:30.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:30.296+0000 7fcc45600640 1 -- 172.21.15.17:0/4105424177 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fcc40018ef0 con 0x7fcc48158310 2024-09-10T21:28:30.576 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:30.568+0000 7fcc4ec00640 1 -- 172.21.15.17:0/4105424177 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fcc480700e0 con 0x7fcc48158310 2024-09-10T21:28:30.576 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:30.572+0000 7fcc45600640 1 -- 172.21.15.17:0/4105424177 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1557 (secure 0 0 0) 0x7fcc400fc7d0 con 0x7fcc48158310 2024-09-10T21:28:30.576 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":30,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":13,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5e2f023f-d6a6-49f4-a256-f4807dce3739","modified":"2024-09-10T21:23:46.892130Z","created":"2024-09-10T21:23:46.892130Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3301","nonce":0},{"type":"v1","addr":"172.21.15.31:6790","nonce":0}]},"addr":"172.21.15.31:6790/0","public_addr":"172.21.15.31:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-10T21:28:30.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:30.572+0000 7fcc4ec00640 1 -- 172.21.15.17:0/4105424177 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fcc28072590 msgr2=0x7fcc28074a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:30.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:30.572+0000 7fcc4ec00640 1 --2- 172.21.15.17:0/4105424177 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fcc28072590 0x7fcc28074a50 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fcc3c007fd0 tx=0x7fcc3c009040 comp rx=0 tx=0).stop 2024-09-10T21:28:30.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:30.572+0000 7fcc4ec00640 1 -- 172.21.15.17:0/4105424177 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcc48158310 msgr2=0x7fcc480c3140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:30.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:30.572+0000 7fcc4ec00640 1 --2- 172.21.15.17:0/4105424177 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcc48158310 0x7fcc480c3140 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fcc40009b30 tx=0x7fcc400092a0 comp rx=0 tx=0).stop 2024-09-10T21:28:30.580 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:30.572+0000 7fcc4ec00640 1 -- 172.21.15.17:0/4105424177 shutdown_connections 2024-09-10T21:28:30.580 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:30.572+0000 7fcc4ec00640 1 --2- 172.21.15.17:0/4105424177 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fcc28072590 0x7fcc28074a50 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:30.580 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:30.572+0000 7fcc4ec00640 1 --2- 172.21.15.17:0/4105424177 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcc48158310 0x7fcc480c3140 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:30.580 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:30.572+0000 7fcc4ec00640 1 --2- 172.21.15.17:0/4105424177 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fcc481527e0 0x7fcc4814ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:30.580 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:30.572+0000 7fcc4ec00640 1 --2- 172.21.15.17:0/4105424177 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcc480b2d80 0x7fcc48150060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:30.580 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:30.572+0000 7fcc4ec00640 1 -- 172.21.15.17:0/4105424177 >> 172.21.15.17:0/4105424177 conn(0x7fcc48070ca0 msgr2=0x7fcc480c7560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:28:30.580 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:30.572+0000 7fcc4ec00640 1 -- 172.21.15.17:0/4105424177 shutdown_connections 2024-09-10T21:28:30.580 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:30.572+0000 7fcc4ec00640 1 -- 172.21.15.17:0/4105424177 wait complete. 2024-09-10T21:28:33.603 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-10T21:28:33.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:33.748+0000 7fa32a000640 1 -- 172.21.15.17:0/2856924111 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa3240c80d0 msgr2=0x7fa32406ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:33.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:33.748+0000 7fa32a000640 1 --2- 172.21.15.17:0/2856924111 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa3240c80d0 0x7fa32406ed90 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fa31c009bf0 tx=0x7fa31c01ca50 comp rx=0 tx=0).stop 2024-09-10T21:28:33.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:33.748+0000 7fa32a000640 1 -- 172.21.15.17:0/2856924111 shutdown_connections 2024-09-10T21:28:33.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:33.748+0000 7fa32a000640 1 --2- 172.21.15.17:0/2856924111 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa3240c80d0 0x7fa32406ed90 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:33.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:33.748+0000 7fa32a000640 1 --2- 172.21.15.17:0/2856924111 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa3240c7080 0x7fa32406e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:33.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:33.748+0000 7fa32a000640 1 -- 172.21.15.17:0/2856924111 >> 172.21.15.17:0/2856924111 conn(0x7fa32406dd30 msgr2=0x7fa32406e140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:28:33.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:33.748+0000 7fa32a000640 1 -- 172.21.15.17:0/2856924111 shutdown_connections 2024-09-10T21:28:33.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:33.748+0000 7fa32a000640 1 -- 172.21.15.17:0/2856924111 wait complete. 2024-09-10T21:28:33.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:33.748+0000 7fa32a000640 1 Processor -- start 2024-09-10T21:28:33.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:33.748+0000 7fa32a000640 1 -- start start 2024-09-10T21:28:33.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:33.748+0000 7fa32a000640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa324070670 0x7fa324150180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:33.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:33.748+0000 7fa32a000640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa3240c7080 0x7fa324159b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:33.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:33.748+0000 7fa32a000640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa3240c80d0 0x7fa32415cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:33.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:33.748+0000 7fa32a000640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fa324167b00 con 0x7fa3240c7080 2024-09-10T21:28:33.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:33.748+0000 7fa32a000640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fa324167980 con 0x7fa324070670 2024-09-10T21:28:33.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:33.748+0000 7fa32a000640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7fa324167c80 con 0x7fa3240c80d0 2024-09-10T21:28:33.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:33.748+0000 7fa322a00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa3240c7080 0x7fa324159b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:33.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:33.748+0000 7fa323e00640 1 -- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa3240c80d0 msgr2=0x7fa32415cde0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:3301/0 2024-09-10T21:28:33.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:33.748+0000 7fa322a00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa3240c7080 0x7fa324159b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42700/0 (socket says 172.21.15.17:42700) 2024-09-10T21:28:33.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:33.748+0000 7fa322a00640 1 -- 172.21.15.17:0/192579080 learned_addr learned my addr 172.21.15.17:0/192579080 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:28:33.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:33.748+0000 7fa323e00640 1 --2- 172.21.15.17:0/192579080 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa3240c80d0 0x7fa32415cde0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:28:33.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:33.752+0000 7fa323400640 1 --2- 172.21.15.17:0/192579080 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa324070670 0x7fa324150180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:33.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:33.752+0000 7fa322a00640 1 -- 172.21.15.17:0/192579080 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa3240c80d0 msgr2=0x7fa32415cde0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:28:33.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:33.752+0000 7fa322a00640 1 --2- 172.21.15.17:0/192579080 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa3240c80d0 0x7fa32415cde0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:33.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:33.752+0000 7fa322a00640 1 -- 172.21.15.17:0/192579080 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa324070670 msgr2=0x7fa324150180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:33.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:33.752+0000 7fa322a00640 1 --2- 172.21.15.17:0/192579080 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa324070670 0x7fa324150180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:33.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:33.752+0000 7fa322a00640 1 -- 172.21.15.17:0/192579080 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa324149910 con 0x7fa3240c7080 2024-09-10T21:28:33.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:33.752+0000 7fa323400640 1 --2- 172.21.15.17:0/192579080 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa324070670 0x7fa324150180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:28:33.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:33.752+0000 7fa322a00640 1 --2- 172.21.15.17:0/192579080 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa3240c7080 0x7fa324159b90 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa31000c730 tx=0x7fa31000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:28:33.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:33.752+0000 7fa31b400640 1 -- 172.21.15.17:0/192579080 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa310016020 con 0x7fa3240c7080 2024-09-10T21:28:33.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:33.752+0000 7fa31b400640 1 -- 172.21.15.17:0/192579080 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa31000a930 con 0x7fa3240c7080 2024-09-10T21:28:33.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:33.752+0000 7fa32a000640 1 -- 172.21.15.17:0/192579080 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa32414f6e0 con 0x7fa3240c7080 2024-09-10T21:28:33.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:33.752+0000 7fa31b400640 1 -- 172.21.15.17:0/192579080 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa31000ad60 con 0x7fa3240c7080 2024-09-10T21:28:33.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:33.752+0000 7fa32a000640 1 -- 172.21.15.17:0/192579080 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa324149e90 con 0x7fa3240c7080 2024-09-10T21:28:33.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:33.752+0000 7fa32a000640 1 -- 172.21.15.17:0/192579080 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa324069e60 con 0x7fa3240c7080 2024-09-10T21:28:33.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:33.752+0000 7fa31b400640 1 -- 172.21.15.17:0/192579080 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fa310012420 con 0x7fa3240c7080 2024-09-10T21:28:33.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:33.756+0000 7fa31b400640 1 --2- 172.21.15.17:0/192579080 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fa304072590 0x7fa304074a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:33.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:33.756+0000 7fa31b400640 1 -- 172.21.15.17:0/192579080 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fa3100935a0 con 0x7fa3240c7080 2024-09-10T21:28:33.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:33.756+0000 7fa323400640 1 --2- 172.21.15.17:0/192579080 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fa304072590 0x7fa304074a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:33.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:33.756+0000 7fa323400640 1 --2- 172.21.15.17:0/192579080 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fa304072590 0x7fa304074a50 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fa314006fd0 tx=0x7fa314008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:28:33.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:33.760+0000 7fa31b400640 1 -- 172.21.15.17:0/192579080 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fa310020070 con 0x7fa3240c7080 2024-09-10T21:28:34.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:34.068+0000 7fa32a000640 1 -- 172.21.15.17:0/192579080 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fa3240c45d0 con 0x7fa3240c7080 2024-09-10T21:28:34.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:34.072+0000 7fa31b400640 1 -- 172.21.15.17:0/192579080 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1557 (secure 0 0 0) 0x7fa31005e7e0 con 0x7fa3240c7080 2024-09-10T21:28:34.076 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":30,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":16,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5e2f023f-d6a6-49f4-a256-f4807dce3739","modified":"2024-09-10T21:23:46.892130Z","created":"2024-09-10T21:23:46.892130Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3301","nonce":0},{"type":"v1","addr":"172.21.15.31:6790","nonce":0}]},"addr":"172.21.15.31:6790/0","public_addr":"172.21.15.31:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-10T21:28:34.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:34.072+0000 7fa32a000640 1 -- 172.21.15.17:0/192579080 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fa304072590 msgr2=0x7fa304074a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:34.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:34.072+0000 7fa32a000640 1 --2- 172.21.15.17:0/192579080 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fa304072590 0x7fa304074a50 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fa314006fd0 tx=0x7fa314008040 comp rx=0 tx=0).stop 2024-09-10T21:28:34.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:34.072+0000 7fa32a000640 1 -- 172.21.15.17:0/192579080 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa3240c7080 msgr2=0x7fa324159b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:34.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:34.072+0000 7fa32a000640 1 --2- 172.21.15.17:0/192579080 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa3240c7080 0x7fa324159b90 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fa31000c730 tx=0x7fa31000cbf0 comp rx=0 tx=0).stop 2024-09-10T21:28:34.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:34.072+0000 7fa32a000640 1 -- 172.21.15.17:0/192579080 shutdown_connections 2024-09-10T21:28:34.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:34.072+0000 7fa32a000640 1 --2- 172.21.15.17:0/192579080 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fa304072590 0x7fa304074a50 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:34.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:34.072+0000 7fa32a000640 1 --2- 172.21.15.17:0/192579080 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa3240c80d0 0x7fa32415cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:34.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:34.072+0000 7fa32a000640 1 --2- 172.21.15.17:0/192579080 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa3240c7080 0x7fa324159b90 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:34.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:34.072+0000 7fa32a000640 1 --2- 172.21.15.17:0/192579080 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa324070670 0x7fa324150180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:34.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:34.072+0000 7fa32a000640 1 -- 172.21.15.17:0/192579080 >> 172.21.15.17:0/192579080 conn(0x7fa32406dd30 msgr2=0x7fa32406f340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:28:34.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:34.072+0000 7fa32a000640 1 -- 172.21.15.17:0/192579080 shutdown_connections 2024-09-10T21:28:34.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:34.072+0000 7fa32a000640 1 -- 172.21.15.17:0/192579080 wait complete. 2024-09-10T21:28:37.104 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-10T21:28:44.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:44.584+0000 7eff93800640 1 -- 172.21.15.17:0/2266827306 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7eff740052f0 msgr2=0x7eff7400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:44.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:44.584+0000 7eff93800640 1 --2- 172.21.15.17:0/2266827306 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7eff740052f0 0x7eff7400b9d0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7eff8c0cf1b0 tx=0x7eff8401d3c0 comp rx=0 tx=0).stop 2024-09-10T21:28:44.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:44.584+0000 7eff93800640 1 -- 172.21.15.17:0/2266827306 shutdown_connections 2024-09-10T21:28:44.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:44.584+0000 7eff93800640 1 --2- 172.21.15.17:0/2266827306 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7eff740052f0 0x7eff7400b9d0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:44.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:44.584+0000 7eff93800640 1 -- 172.21.15.17:0/2266827306 >> 172.21.15.17:0/2266827306 conn(0x7eff8c070610 msgr2=0x7eff8c070a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:28:44.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:44.584+0000 7eff93800640 1 -- 172.21.15.17:0/2266827306 shutdown_connections 2024-09-10T21:28:44.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:44.584+0000 7eff93800640 1 -- 172.21.15.17:0/2266827306 wait complete. 2024-09-10T21:28:44.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:44.588+0000 7eff93800640 1 Processor -- start 2024-09-10T21:28:44.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:44.588+0000 7eff93800640 1 -- start start 2024-09-10T21:28:44.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:44.588+0000 7eff93800640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7eff740052f0 0x7eff8c149dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:44.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:44.588+0000 7eff93800640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7eff8c0b33c0 0x7eff8c148190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:44.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:44.588+0000 7eff93800640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7eff8c1486d0 0x7eff8c0c5760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:44.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:44.588+0000 7eff93800640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7eff8c14f5a0 con 0x7eff740052f0 2024-09-10T21:28:44.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:44.588+0000 7eff93800640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7eff8c0c95e0 con 0x7eff8c0b33c0 2024-09-10T21:28:44.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:44.588+0000 7eff93800640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7eff8c14f720 con 0x7eff8c1486d0 2024-09-10T21:28:44.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:44.588+0000 7eff90e00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7eff740052f0 0x7eff8c149dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:44.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:44.588+0000 7eff90e00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7eff740052f0 0x7eff8c149dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:50820/0 (socket says 172.21.15.17:50820) 2024-09-10T21:28:44.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:44.588+0000 7eff90e00640 1 -- 172.21.15.17:0/3385613942 learned_addr learned my addr 172.21.15.17:0/3385613942 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:28:44.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:44.588+0000 7eff91800640 1 -- 172.21.15.17:0/3385613942 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7eff8c1486d0 msgr2=0x7eff8c0c5760 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:3301/0 2024-09-10T21:28:44.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:44.588+0000 7eff91800640 1 --2- 172.21.15.17:0/3385613942 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7eff8c1486d0 0x7eff8c0c5760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:28:44.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:44.588+0000 7eff8be00640 1 --2- 172.21.15.17:0/3385613942 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7eff8c0b33c0 0x7eff8c148190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:44.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:44.588+0000 7eff90e00640 1 -- 172.21.15.17:0/3385613942 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7eff8c1486d0 msgr2=0x7eff8c0c5760 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:28:44.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:44.588+0000 7eff90e00640 1 --2- 172.21.15.17:0/3385613942 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7eff8c1486d0 0x7eff8c0c5760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:44.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:44.588+0000 7eff90e00640 1 -- 172.21.15.17:0/3385613942 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7eff8c0b33c0 msgr2=0x7eff8c148190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:44.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:44.588+0000 7eff90e00640 1 --2- 172.21.15.17:0/3385613942 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7eff8c0b33c0 0x7eff8c148190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:44.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:44.588+0000 7eff90e00640 1 -- 172.21.15.17:0/3385613942 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff8c1f4da0 con 0x7eff740052f0 2024-09-10T21:28:44.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:44.588+0000 7eff90e00640 1 --2- 172.21.15.17:0/3385613942 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7eff740052f0 0x7eff8c149dc0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7eff80001630 tx=0x7eff8000afd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:28:44.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:44.588+0000 7eff88c00640 1 -- 172.21.15.17:0/3385613942 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff8001a070 con 0x7eff740052f0 2024-09-10T21:28:44.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:44.588+0000 7eff88c00640 1 -- 172.21.15.17:0/3385613942 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7eff800078b0 con 0x7eff740052f0 2024-09-10T21:28:44.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:44.588+0000 7eff88c00640 1 -- 172.21.15.17:0/3385613942 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff80007d00 con 0x7eff740052f0 2024-09-10T21:28:44.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:44.588+0000 7eff93800640 1 -- 172.21.15.17:0/3385613942 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff8c0b2720 con 0x7eff740052f0 2024-09-10T21:28:44.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:44.592+0000 7eff93800640 1 -- 172.21.15.17:0/3385613942 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff8c0b21c0 con 0x7eff740052f0 2024-09-10T21:28:44.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:44.592+0000 7eff93800640 1 -- 172.21.15.17:0/3385613942 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff8c1f4bb0 con 0x7eff740052f0 2024-09-10T21:28:44.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:44.592+0000 7eff88c00640 1 -- 172.21.15.17:0/3385613942 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7eff80027060 con 0x7eff740052f0 2024-09-10T21:28:44.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:44.592+0000 7eff88c00640 1 --2- 172.21.15.17:0/3385613942 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7eff6c075b40 0x7eff6c078000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:44.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:44.592+0000 7eff88c00640 1 -- 172.21.15.17:0/3385613942 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7eff80094160 con 0x7eff740052f0 2024-09-10T21:28:44.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:44.596+0000 7eff8be00640 1 --2- 172.21.15.17:0/3385613942 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7eff6c075b40 0x7eff6c078000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:44.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:44.596+0000 7eff8be00640 1 --2- 172.21.15.17:0/3385613942 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7eff6c075b40 0x7eff6c078000 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7eff8c0b1f00 tx=0x7eff78016040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:28:44.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:44.600+0000 7eff88c00640 1 -- 172.21.15.17:0/3385613942 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7eff8005f4f0 con 0x7eff740052f0 2024-09-10T21:28:44.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:44.904+0000 7eff93800640 1 -- 172.21.15.17:0/3385613942 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7eff8c148ad0 con 0x7eff740052f0 2024-09-10T21:28:44.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:44.904+0000 7eff88c00640 1 -- 172.21.15.17:0/3385613942 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1550 (secure 0 0 0) 0x7eff8005edf0 con 0x7eff740052f0 2024-09-10T21:28:44.910 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":34,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5e2f023f-d6a6-49f4-a256-f4807dce3739","modified":"2024-09-10T21:23:46.892130Z","created":"2024-09-10T21:23:46.892130Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3301","nonce":0},{"type":"v1","addr":"172.21.15.31:6790","nonce":0}]},"addr":"172.21.15.31:6790/0","public_addr":"172.21.15.31:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-10T21:28:44.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:44.908+0000 7eff93800640 1 -- 172.21.15.17:0/3385613942 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7eff6c075b40 msgr2=0x7eff6c078000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:44.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:44.908+0000 7eff93800640 1 --2- 172.21.15.17:0/3385613942 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7eff6c075b40 0x7eff6c078000 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7eff8c0b1f00 tx=0x7eff78016040 comp rx=0 tx=0).stop 2024-09-10T21:28:44.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:44.908+0000 7eff93800640 1 -- 172.21.15.17:0/3385613942 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7eff740052f0 msgr2=0x7eff8c149dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:44.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:44.908+0000 7eff93800640 1 --2- 172.21.15.17:0/3385613942 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7eff740052f0 0x7eff8c149dc0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7eff80001630 tx=0x7eff8000afd0 comp rx=0 tx=0).stop 2024-09-10T21:28:44.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:44.908+0000 7eff93800640 1 -- 172.21.15.17:0/3385613942 shutdown_connections 2024-09-10T21:28:44.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:44.908+0000 7eff93800640 1 --2- 172.21.15.17:0/3385613942 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7eff6c075b40 0x7eff6c078000 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:44.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:44.908+0000 7eff93800640 1 --2- 172.21.15.17:0/3385613942 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7eff8c1486d0 0x7eff8c0c5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:44.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:44.908+0000 7eff93800640 1 --2- 172.21.15.17:0/3385613942 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7eff8c0b33c0 0x7eff8c148190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:44.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:44.908+0000 7eff93800640 1 --2- 172.21.15.17:0/3385613942 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7eff740052f0 0x7eff8c149dc0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:44.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:44.908+0000 7eff93800640 1 -- 172.21.15.17:0/3385613942 >> 172.21.15.17:0/3385613942 conn(0x7eff8c070610 msgr2=0x7eff8c0b0020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:28:44.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:44.908+0000 7eff93800640 1 -- 172.21.15.17:0/3385613942 shutdown_connections 2024-09-10T21:28:44.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:44.908+0000 7eff93800640 1 -- 172.21.15.17:0/3385613942 wait complete. 2024-09-10T21:28:44.934 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2024-09-10T21:28:44.935 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon a 2024-09-10T21:28:44.935 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-10T21:28:45.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:45.088+0000 7f98f5c00640 1 -- 172.21.15.17:0/1178721104 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f98d4004810 msgr2=0x7f98d4004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:45.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:45.088+0000 7f98f5c00640 1 --2- 172.21.15.17:0/1178721104 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f98d4004810 0x7f98d4004c90 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f98dc009b60 tx=0x7f98dc01d280 comp rx=0 tx=0).stop 2024-09-10T21:28:45.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:45.088+0000 7f98ef400640 1 -- 172.21.15.17:0/1178721104 reap_dead start 2024-09-10T21:28:45.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:45.088+0000 7f98f5c00640 1 -- 172.21.15.17:0/1178721104 shutdown_connections 2024-09-10T21:28:45.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:45.088+0000 7f98f5c00640 1 -- 172.21.15.17:0/1178721104 >> 172.21.15.17:0/1178721104 conn(0x7f98f00702f0 msgr2=0x7f98f0070700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:28:45.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:45.088+0000 7f98f5c00640 1 -- 172.21.15.17:0/1178721104 shutdown_connections 2024-09-10T21:28:45.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:45.088+0000 7f98f5c00640 1 -- 172.21.15.17:0/1178721104 wait complete. 2024-09-10T21:28:45.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:45.088+0000 7f98f5c00640 1 Processor -- start 2024-09-10T21:28:45.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:45.088+0000 7f98f5c00640 1 -- start start 2024-09-10T21:28:45.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:45.088+0000 7f98f5c00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f98d4004810 0x7f98f0157d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:45.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:45.088+0000 7f98f5c00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f98f00ab0a0 0x7f98f0154990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:45.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:45.088+0000 7f98f5c00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f98f0072360 0x7f98f0154ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:45.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:45.088+0000 7f98f5c00640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f98f00ce740 con 0x7f98d4004810 2024-09-10T21:28:45.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:45.088+0000 7f98f5c00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f98f00b42c0 con 0x7f98f0072360 2024-09-10T21:28:45.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:45.088+0000 7f98f5c00640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f98f015c7c0 con 0x7f98f00ab0a0 2024-09-10T21:28:45.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:45.088+0000 7f98eea00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f98d4004810 0x7f98f0157d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:45.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:45.088+0000 7f98eea00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f98d4004810 0x7f98f0157d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:46654/0 (socket says 172.21.15.17:46654) 2024-09-10T21:28:45.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:45.088+0000 7f98ee000640 1 -- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f98f00ab0a0 msgr2=0x7f98f0154990 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:3301/0 2024-09-10T21:28:45.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:45.088+0000 7f98eea00640 1 -- 172.21.15.17:0/546205441 learned_addr learned my addr 172.21.15.17:0/546205441 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:28:45.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:45.088+0000 7f98ee000640 1 --2- 172.21.15.17:0/546205441 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f98f00ab0a0 0x7f98f0154990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:28:45.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:45.092+0000 7f98ef400640 1 --2- 172.21.15.17:0/546205441 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f98f0072360 0x7f98f0154ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:45.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:45.092+0000 7f98eea00640 1 -- 172.21.15.17:0/546205441 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f98f00ab0a0 msgr2=0x7f98f0154990 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:28:45.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:45.092+0000 7f98eea00640 1 --2- 172.21.15.17:0/546205441 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f98f00ab0a0 0x7f98f0154990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:45.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:45.092+0000 7f98eea00640 1 -- 172.21.15.17:0/546205441 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f98f0072360 msgr2=0x7f98f0154ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:45.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:45.092+0000 7f98eea00640 1 --2- 172.21.15.17:0/546205441 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f98f0072360 0x7f98f0154ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:45.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:45.092+0000 7f98eea00640 1 -- 172.21.15.17:0/546205441 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98f00c9020 con 0x7f98d4004810 2024-09-10T21:28:45.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:45.092+0000 7f98eea00640 1 --2- 172.21.15.17:0/546205441 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f98d4004810 0x7f98f0157d00 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f98e000e6c0 tx=0x7f98e000eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:28:45.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:45.092+0000 7f98e6a00640 1 -- 172.21.15.17:0/546205441 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98e000cab0 con 0x7f98d4004810 2024-09-10T21:28:45.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:45.092+0000 7f98e6a00640 1 -- 172.21.15.17:0/546205441 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f98e000cc70 con 0x7f98d4004810 2024-09-10T21:28:45.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:45.092+0000 7f98e6a00640 1 -- 172.21.15.17:0/546205441 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98e0010690 con 0x7f98d4004810 2024-09-10T21:28:45.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:45.092+0000 7f98f5c00640 1 -- 172.21.15.17:0/546205441 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98f0152da0 con 0x7f98d4004810 2024-09-10T21:28:45.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:45.092+0000 7f98f5c00640 1 -- 172.21.15.17:0/546205441 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98f0150350 con 0x7f98d4004810 2024-09-10T21:28:45.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:45.092+0000 7f98f5c00640 1 --2- 172.21.15.17:0/546205441 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f98f014e750 0x7f98f014eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:45.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:45.092+0000 7f98f5c00640 1 -- 172.21.15.17:0/546205441 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f98f00c92e0 con 0x7f98f014e750 2024-09-10T21:28:45.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:45.092+0000 7f98ee000640 1 --2- 172.21.15.17:0/546205441 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f98f014e750 0x7f98f014eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:45.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:45.092+0000 7f98ee000640 1 -- 172.21.15.17:0/546205441 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f98f0153fb0 con 0x7f98f014e750 2024-09-10T21:28:45.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:45.092+0000 7f98ee000640 1 --2- 172.21.15.17:0/546205441 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f98f014e750 0x7f98f014eb30 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f98dc0ae000 tx=0x7f98dc005620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:28:45.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:45.092+0000 7f98e6a00640 1 -- 172.21.15.17:0/546205441 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98dc0bb020 con 0x7f98f014e750 2024-09-10T21:28:45.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:45.092+0000 7f98e6a00640 1 -- 172.21.15.17:0/546205441 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f98dc0046d0 con 0x7f98f014e750 2024-09-10T21:28:45.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:45.092+0000 7f98e6a00640 1 -- 172.21.15.17:0/546205441 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f98f014e750 msgr2=0x7f98f014eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:45.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:45.092+0000 7f98e6a00640 1 --2- 172.21.15.17:0/546205441 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f98f014e750 0x7f98f014eb30 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f98dc0ae000 tx=0x7f98dc005620 comp rx=0 tx=0).stop 2024-09-10T21:28:45.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:45.092+0000 7f98e6a00640 1 -- 172.21.15.17:0/546205441 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f98f014e750 msgr2=0x7f98f014eb30 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:28:45.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:45.092+0000 7f98e6a00640 1 --2- 172.21.15.17:0/546205441 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f98f014e750 0x7f98f014eb30 secure :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f98dc0ae000 tx=0x7f98dc005620 comp rx=0 tx=0).stop 2024-09-10T21:28:45.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:45.092+0000 7f98e6a00640 1 -- 172.21.15.17:0/546205441 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f98e0004930 con 0x7f98d4004810 2024-09-10T21:28:45.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:45.096+0000 7f98e6a00640 1 --2- 172.21.15.17:0/546205441 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f98d00720c0 0x7f98d0074580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:45.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:45.096+0000 7f98e6a00640 1 -- 172.21.15.17:0/546205441 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f98e0093e20 con 0x7f98d4004810 2024-09-10T21:28:45.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:45.096+0000 7f98ee000640 1 --2- 172.21.15.17:0/546205441 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f98d00720c0 0x7f98d0074580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:45.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:45.096+0000 7f98ee000640 1 --2- 172.21.15.17:0/546205441 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f98d00720c0 0x7f98d0074580 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f98f00c9460 tx=0x7f98dc0b7000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:28:45.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:45.100+0000 7f98f5c00640 1 --2- 172.21.15.17:0/546205441 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f98f01f4780 0x7f98f01f4b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:45.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:45.100+0000 7f98f5c00640 1 -- 172.21.15.17:0/546205441 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f98f0154360 con 0x7f98f01f4780 2024-09-10T21:28:45.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:45.100+0000 7f98ef400640 1 --2- 172.21.15.17:0/546205441 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f98f01f4780 0x7f98f01f4b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:45.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:45.100+0000 7f98ef400640 1 -- 172.21.15.17:0/546205441 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f98f00cb530 con 0x7f98f01f4780 2024-09-10T21:28:45.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:45.100+0000 7f98ef400640 1 --2- 172.21.15.17:0/546205441 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f98f01f4780 0x7f98f01f4b60 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f98f0073220 tx=0x7f98e800ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:28:45.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:45.100+0000 7f98e6a00640 1 -- 172.21.15.17:0/546205441 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98e8017020 con 0x7f98f01f4780 2024-09-10T21:28:45.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:45.100+0000 7f98e6a00640 1 -- 172.21.15.17:0/546205441 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4865 (secure 0 0 0) 0x7f98e800c7c0 con 0x7f98f01f4780 2024-09-10T21:28:45.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:45.100+0000 7f98e6a00640 1 -- 172.21.15.17:0/546205441 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f98f01f4780 msgr2=0x7f98f01f4b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:45.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:45.100+0000 7f98e6a00640 1 --2- 172.21.15.17:0/546205441 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f98f01f4780 0x7f98f01f4b60 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f98f0073220 tx=0x7f98e800ec70 comp rx=0 tx=0).stop 2024-09-10T21:28:45.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:45.104+0000 7f98e6a00640 1 -- 172.21.15.17:0/546205441 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f98f01f4780 msgr2=0x7f98f01f4b60 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:28:45.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:45.104+0000 7f98e6a00640 1 --2- 172.21.15.17:0/546205441 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f98f01f4780 0x7f98f01f4b60 secure :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f98f0073220 tx=0x7f98e800ec70 comp rx=0 tx=0).stop 2024-09-10T21:28:45.108 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-10T21:28:45.109 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:28:45.109 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:28:45.109 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "leader", 2024-09-10T21:28:45.109 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 34, 2024-09-10T21:28:45.109 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 42939, 2024-09-10T21:28:45.109 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-10T21:28:45.109 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-10T21:28:45.109 INFO:teuthology.orchestra.run.smithi017.stdout: 1 2024-09-10T21:28:45.109 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:28:45.109 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 0, 2024-09-10T21:28:45.109 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:28:45.109 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-10T21:28:45.110 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-10T21:28:45.110 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:28:45.110 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:28:45.110 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:28:45.110 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:28:45.110 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:28:45.110 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:28:45.110 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:28:45.110 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:28:45.110 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:28:45.110 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:28:45.110 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:28:45.110 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:28:45.110 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-10T21:28:45.111 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-10T21:28:45.111 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:28:45.111 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:28:45.111 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:28:45.111 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:28:45.111 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:28:45.111 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:28:45.111 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:28:45.111 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:28:45.111 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:28:45.111 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:28:45.111 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:28:45.111 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:28:45.112 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:28:45.112 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-10T21:28:45.112 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-10T21:28:45.112 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-10T21:28:45.112 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-10T21:28:45.112 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-10T21:28:45.112 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "5e2f023f-d6a6-49f4-a256-f4807dce3739", 2024-09-10T21:28:45.112 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:28:45.112 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:28:45.112 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-10T21:28:45.112 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-10T21:28:45.112 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-10T21:28:45.112 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-10T21:28:45.113 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-10T21:28:45.113 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-10T21:28:45.113 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-10T21:28:45.113 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:28:45.113 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-10T21:28:45.113 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:28:45.113 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:28:45.113 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:28:45.113 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:28:45.113 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:28:45.113 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:28:45.113 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:28:45.113 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:28:45.114 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:28:45.114 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:28:45.114 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:28:45.114 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:28:45.114 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-10T21:28:45.114 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:28:45.114 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-10T21:28:45.114 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:28:45.114 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:28:45.114 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:28:45.114 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:28:45.114 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:28:45.114 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:28:45.115 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:28:45.115 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-10T21:28:45.115 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:28:45.115 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:28:45.115 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:28:45.115 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:28:45.115 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-10T21:28:45.115 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:28:45.115 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:28:45.115 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:28:45.115 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:28:45.115 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-10T21:28:45.115 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-10T21:28:45.116 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:28:45.116 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:28:45.116 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:28:45.116 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:28:45.116 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:28:45.116 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:28:45.116 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:28:45.116 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:28:45.116 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:28:45.116 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:28:45.116 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:28:45.116 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3300", 2024-09-10T21:28:45.117 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:28:45.117 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:28:45.117 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:28:45.117 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:28:45.117 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789", 2024-09-10T21:28:45.117 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:28:45.117 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:28:45.117 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:28:45.117 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:28:45.117 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789/0", 2024-09-10T21:28:45.117 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6789/0", 2024-09-10T21:28:45.117 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:28:45.117 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:28:45.118 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:28:45.118 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:28:45.118 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:28:45.118 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:28:45.118 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:28:45.118 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:28:45.118 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:28:45.118 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:28:45.118 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:28:45.118 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3301", 2024-09-10T21:28:45.118 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:28:45.118 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:28:45.118 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:28:45.118 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:28:45.119 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790", 2024-09-10T21:28:45.119 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:28:45.119 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:28:45.119 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:28:45.119 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:28:45.119 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790/0", 2024-09-10T21:28:45.119 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6790/0", 2024-09-10T21:28:45.119 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:28:45.119 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:28:45.119 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:28:45.119 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:28:45.119 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:28:45.120 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:28:45.120 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-10T21:28:45.120 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-10T21:28:45.120 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:28:45.120 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:28:45.120 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:28:45.120 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:28:45.120 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:28:45.120 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:28:45.120 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": [ 2024-09-10T21:28:45.120 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:28:45.120 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:28:45.120 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:28:45.121 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:28:45.121 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:28:45.121 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:28:45.121 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-10T21:28:45.121 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:28:45.121 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:28:45.121 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:28:45.121 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:28:45.121 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:28:45.121 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:28:45.121 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-10T21:28:45.121 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:28:45.121 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:28:45.122 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:28:45.122 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 4 2024-09-10T21:28:45.122 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:28:45.122 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:28:45.122 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:28:45.122 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-10T21:28:45.122 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-10T21:28:45.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:45.104+0000 7f98f5c00640 1 -- 172.21.15.17:0/546205441 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f98d00720c0 msgr2=0x7f98d0074580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:45.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:45.104+0000 7f98f5c00640 1 --2- 172.21.15.17:0/546205441 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f98d00720c0 0x7f98d0074580 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f98f00c9460 tx=0x7f98dc0b7000 comp rx=0 tx=0).stop 2024-09-10T21:28:45.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:45.104+0000 7f98f5c00640 1 -- 172.21.15.17:0/546205441 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f98d4004810 msgr2=0x7f98f0157d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:45.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:45.104+0000 7f98f5c00640 1 --2- 172.21.15.17:0/546205441 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f98d4004810 0x7f98f0157d00 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f98e000e6c0 tx=0x7f98e000eb80 comp rx=0 tx=0).stop 2024-09-10T21:28:45.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:45.104+0000 7f98ef400640 1 -- 172.21.15.17:0/546205441 reap_dead start 2024-09-10T21:28:45.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:45.104+0000 7f98f5c00640 1 -- 172.21.15.17:0/546205441 shutdown_connections 2024-09-10T21:28:45.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:45.104+0000 7f98f5c00640 1 --2- 172.21.15.17:0/546205441 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f98d4004810 0x7f98f0157d00 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:45.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:45.104+0000 7f98f5c00640 1 -- 172.21.15.17:0/546205441 >> 172.21.15.17:0/546205441 conn(0x7f98f00702f0 msgr2=0x7f98f015ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:28:45.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:45.104+0000 7f98f5c00640 1 -- 172.21.15.17:0/546205441 shutdown_connections 2024-09-10T21:28:45.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:45.104+0000 7f98f5c00640 1 -- 172.21.15.17:0/546205441 wait complete. 2024-09-10T21:28:45.128 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-09-10T21:28:45.128 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon b 2024-09-10T21:28:45.129 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-10T21:28:45.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:45.296+0000 7f4b57400640 1 -- 172.21.15.17:0/2771459572 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4b50151800 msgr2=0x7f4b500c8fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:45.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:45.296+0000 7f4b57400640 1 --2- 172.21.15.17:0/2771459572 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4b50151800 0x7f4b500c8fc0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f4b44009900 tx=0x7f4b4401c8e0 comp rx=0 tx=0).stop 2024-09-10T21:28:45.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:45.296+0000 7f4b57400640 1 -- 172.21.15.17:0/2771459572 shutdown_connections 2024-09-10T21:28:45.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:45.296+0000 7f4b57400640 1 --2- 172.21.15.17:0/2771459572 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4b50151800 0x7f4b500c8fc0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:45.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:45.296+0000 7f4b57400640 1 -- 172.21.15.17:0/2771459572 >> 172.21.15.17:0/2771459572 conn(0x7f4b5006cc20 msgr2=0x7f4b5006d030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:28:45.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:45.296+0000 7f4b57400640 1 -- 172.21.15.17:0/2771459572 shutdown_connections 2024-09-10T21:28:45.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:45.296+0000 7f4b57400640 1 -- 172.21.15.17:0/2771459572 wait complete. 2024-09-10T21:28:45.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:45.300+0000 7f4b57400640 1 Processor -- start 2024-09-10T21:28:45.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:45.300+0000 7f4b57400640 1 -- start start 2024-09-10T21:28:45.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:45.300+0000 7f4b57400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4b500c3a30 0x7f4b50154900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:45.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:45.300+0000 7f4b57400640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f4b500c9500 0x7f4b5014c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:45.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:45.300+0000 7f4b57400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4b50151800 0x7f4b501eb100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:45.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:45.300+0000 7f4b57400640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f4b500c8d60 con 0x7f4b50151800 2024-09-10T21:28:45.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:45.300+0000 7f4b57400640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f4b500c8be0 con 0x7f4b500c3a30 2024-09-10T21:28:45.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:45.300+0000 7f4b57400640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f4b50072e40 con 0x7f4b500c9500 2024-09-10T21:28:45.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:45.300+0000 7f4b4fe00640 1 -- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f4b500c9500 msgr2=0x7f4b5014c0e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:3301/0 2024-09-10T21:28:45.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:45.300+0000 7f4b4fe00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f4b500c9500 0x7f4b5014c0e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:28:45.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:45.300+0000 7f4b54a00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4b500c3a30 0x7f4b50154900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:45.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:45.300+0000 7f4b55400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4b50151800 0x7f4b501eb100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:45.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:45.300+0000 7f4b54a00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4b500c3a30 0x7f4b50154900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.17:42090/0 (socket says 172.21.15.17:42090) 2024-09-10T21:28:45.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:45.300+0000 7f4b54a00640 1 -- 172.21.15.17:0/3588285844 learned_addr learned my addr 172.21.15.17:0/3588285844 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:28:45.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:45.300+0000 7f4b55400640 1 -- 172.21.15.17:0/3588285844 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f4b500c9500 msgr2=0x7f4b5014c0e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:28:45.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:45.300+0000 7f4b55400640 1 --2- 172.21.15.17:0/3588285844 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f4b500c9500 0x7f4b5014c0e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:45.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:45.300+0000 7f4b55400640 1 -- 172.21.15.17:0/3588285844 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4b500c3a30 msgr2=0x7f4b50154900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:45.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:45.300+0000 7f4b55400640 1 --2- 172.21.15.17:0/3588285844 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4b500c3a30 0x7f4b50154900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:28:45.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:45.300+0000 7f4b55400640 1 -- 172.21.15.17:0/3588285844 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b501f20b0 con 0x7f4b50151800 2024-09-10T21:28:45.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:45.300+0000 7f4b54a00640 1 --2- 172.21.15.17:0/3588285844 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4b500c3a30 0x7f4b50154900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:28:45.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:45.300+0000 7f4b55400640 1 --2- 172.21.15.17:0/3588285844 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4b50151800 0x7f4b501eb100 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f4b4800c700 tx=0x7f4b4800cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:28:45.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:45.300+0000 7f4b4cc00640 1 -- 172.21.15.17:0/3588285844 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b48003b80 con 0x7f4b50151800 2024-09-10T21:28:45.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:45.300+0000 7f4b4cc00640 1 -- 172.21.15.17:0/3588285844 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4b48003d20 con 0x7f4b50151800 2024-09-10T21:28:45.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:45.300+0000 7f4b4cc00640 1 -- 172.21.15.17:0/3588285844 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b4800aab0 con 0x7f4b50151800 2024-09-10T21:28:45.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:45.300+0000 7f4b57400640 1 -- 172.21.15.17:0/3588285844 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b5014a550 con 0x7f4b50151800 2024-09-10T21:28:45.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:45.300+0000 7f4b57400640 1 -- 172.21.15.17:0/3588285844 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b5014a7f0 con 0x7f4b50151800 2024-09-10T21:28:45.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:45.304+0000 7f4b57400640 1 --2- 172.21.15.17:0/3588285844 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4b501eb640 0x7f4b501ea900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:45.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:45.304+0000 7f4b57400640 1 -- 172.21.15.17:0/3588285844 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f4b501eae40 con 0x7f4b501eb640 2024-09-10T21:28:45.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:45.304+0000 7f4b54a00640 1 --2- 172.21.15.17:0/3588285844 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4b501eb640 0x7f4b501ea900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:45.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:45.304+0000 7f4b54a00640 1 -- 172.21.15.17:0/3588285844 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4b500c89c0 con 0x7f4b501eb640 2024-09-10T21:28:45.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:45.304+0000 7f4b54a00640 1 --2- 172.21.15.17:0/3588285844 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4b501eb640 0x7f4b501ea900 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f4b440194f0 tx=0x7f4b440095f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:28:45.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:45.304+0000 7f4b4cc00640 1 -- 172.21.15.17:0/3588285844 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b440bb020 con 0x7f4b501eb640 2024-09-10T21:28:45.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:45.304+0000 7f4b4cc00640 1 -- 172.21.15.17:0/3588285844 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f4b4800ac50 con 0x7f4b50151800 2024-09-10T21:28:45.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:45.304+0000 7f4b4cc00640 1 --2- 172.21.15.17:0/3588285844 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f4b300724e0 0x7f4b300749a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:45.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:45.304+0000 7f4b4cc00640 1 -- 172.21.15.17:0/3588285844 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f4b480933a0 con 0x7f4b50151800 2024-09-10T21:28:45.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:45.304+0000 7f4b4cc00640 1 -- 172.21.15.17:0/3588285844 <== mon.1 v2:172.21.15.31:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f4b440b3040 con 0x7f4b501eb640 2024-09-10T21:28:45.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:45.304+0000 7f4b4fe00640 1 --2- 172.21.15.17:0/3588285844 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f4b300724e0 0x7f4b300749a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:45.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:45.304+0000 7f4b4cc00640 1 -- 172.21.15.17:0/3588285844 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4b501eb640 msgr2=0x7f4b501ea900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:45.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:45.304+0000 7f4b4cc00640 1 --2- 172.21.15.17:0/3588285844 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4b501eb640 0x7f4b501ea900 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f4b440194f0 tx=0x7f4b440095f0 comp rx=0 tx=0).stop 2024-09-10T21:28:45.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:45.304+0000 7f4b4cc00640 1 -- 172.21.15.17:0/3588285844 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4b501eb640 msgr2=0x7f4b501ea900 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:28:45.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:45.304+0000 7f4b4cc00640 1 --2- 172.21.15.17:0/3588285844 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4b501eb640 0x7f4b501ea900 secure :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f4b440194f0 tx=0x7f4b440095f0 comp rx=0 tx=0).stop 2024-09-10T21:28:45.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:45.308+0000 7f4b4fe00640 1 --2- 172.21.15.17:0/3588285844 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f4b300724e0 0x7f4b300749a0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f4b501eed70 tx=0x7f4b3c007a20 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:28:45.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:45.316+0000 7f4b57400640 1 --2- 172.21.15.17:0/3588285844 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4b50149e00 0x7f4b50149240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:28:45.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:45.316+0000 7f4b57400640 1 -- 172.21.15.17:0/3588285844 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f4b50151e70 con 0x7f4b50149e00 2024-09-10T21:28:45.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:45.316+0000 7f4b54a00640 1 --2- 172.21.15.17:0/3588285844 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4b50149e00 0x7f4b50149240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:28:45.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:45.316+0000 7f4b54a00640 1 -- 172.21.15.17:0/3588285844 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f4b50074990 con 0x7f4b50149e00 2024-09-10T21:28:45.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:45.316+0000 7f4b54a00640 1 --2- 172.21.15.17:0/3588285844 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4b50149e00 0x7f4b50149240 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f4b44000c10 tx=0x7f4b4401c8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:28:45.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:45.316+0000 7f4b4cc00640 1 -- 172.21.15.17:0/3588285844 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b440c8070 con 0x7f4b50149e00 2024-09-10T21:28:45.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:45.316+0000 7f4b4cc00640 1 -- 172.21.15.17:0/3588285844 <== mon.1 v2:172.21.15.31:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5031 (secure 0 0 0) 0x7f4b44003dd0 con 0x7f4b50149e00 2024-09-10T21:28:45.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:45.316+0000 7f4b4cc00640 1 -- 172.21.15.17:0/3588285844 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4b50149e00 msgr2=0x7f4b50149240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:45.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:45.316+0000 7f4b4cc00640 1 --2- 172.21.15.17:0/3588285844 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4b50149e00 0x7f4b50149240 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f4b44000c10 tx=0x7f4b4401c8e0 comp rx=0 tx=0).stop 2024-09-10T21:28:45.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:45.316+0000 7f4b4cc00640 1 -- 172.21.15.17:0/3588285844 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4b50149e00 msgr2=0x7f4b50149240 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:28:45.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:45.316+0000 7f4b4cc00640 1 --2- 172.21.15.17:0/3588285844 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4b50149e00 0x7f4b50149240 secure :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f4b44000c10 tx=0x7f4b4401c8e0 comp rx=0 tx=0).stop 2024-09-10T21:28:45.323 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-10T21:28:45.323 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:28:45.323 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:28:45.323 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-10T21:28:45.323 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 34, 2024-09-10T21:28:45.323 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 263762, 2024-09-10T21:28:45.323 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-10T21:28:45.323 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-10T21:28:45.323 INFO:teuthology.orchestra.run.smithi017.stdout: 1 2024-09-10T21:28:45.323 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:28:45.323 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 0, 2024-09-10T21:28:45.323 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:28:45.323 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-10T21:28:45.324 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-10T21:28:45.324 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:28:45.324 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:28:45.324 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:28:45.324 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:28:45.324 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:28:45.324 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:28:45.324 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:28:45.324 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:28:45.324 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:28:45.324 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:28:45.325 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:28:45.325 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:28:45.325 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-10T21:28:45.325 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-10T21:28:45.325 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:28:45.325 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:28:45.325 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:28:45.325 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:28:45.325 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:28:45.325 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:28:45.325 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:28:45.325 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:28:45.326 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:28:45.326 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:28:45.326 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:28:45.326 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:28:45.326 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:28:45.326 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-10T21:28:45.326 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-10T21:28:45.326 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-10T21:28:45.326 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-10T21:28:45.326 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-10T21:28:45.326 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "5e2f023f-d6a6-49f4-a256-f4807dce3739", 2024-09-10T21:28:45.326 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:28:45.326 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:28:45.326 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-10T21:28:45.327 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-10T21:28:45.327 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-10T21:28:45.327 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-10T21:28:45.327 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-10T21:28:45.327 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-10T21:28:45.327 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-10T21:28:45.327 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:28:45.327 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-10T21:28:45.327 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:28:45.327 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:28:45.327 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:28:45.327 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:28:45.327 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:28:45.327 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:28:45.328 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:28:45.328 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:28:45.328 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:28:45.328 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:28:45.328 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:28:45.328 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:28:45.328 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-10T21:28:45.328 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:28:45.328 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-10T21:28:45.328 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:28:45.328 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:28:45.328 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:28:45.328 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:28:45.329 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:28:45.329 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:28:45.329 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:28:45.329 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-10T21:28:45.329 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:28:45.329 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:28:45.329 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:28:45.329 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:28:45.329 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-10T21:28:45.329 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:28:45.329 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:28:45.330 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:28:45.330 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:28:45.330 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-10T21:28:45.330 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-10T21:28:45.330 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:28:45.330 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:28:45.330 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:28:45.330 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:28:45.330 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:28:45.330 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:28:45.330 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:28:45.330 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:28:45.330 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:28:45.331 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:28:45.331 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:28:45.331 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3300", 2024-09-10T21:28:45.331 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:28:45.331 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:28:45.331 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:28:45.331 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:28:45.331 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789", 2024-09-10T21:28:45.331 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:28:45.332 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:28:45.332 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:28:45.332 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:28:45.332 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789/0", 2024-09-10T21:28:45.332 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6789/0", 2024-09-10T21:28:45.332 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:28:45.332 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:28:45.332 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:28:45.332 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:28:45.332 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:28:45.332 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:28:45.332 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:28:45.333 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:28:45.333 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:28:45.333 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:28:45.333 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:28:45.333 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3301", 2024-09-10T21:28:45.333 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:28:45.333 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:28:45.333 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:28:45.333 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:28:45.333 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790", 2024-09-10T21:28:45.333 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:28:45.333 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:28:45.333 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:28:45.334 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:28:45.334 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790/0", 2024-09-10T21:28:45.334 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6790/0", 2024-09-10T21:28:45.334 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:28:45.334 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:28:45.334 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:28:45.334 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:28:45.334 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:28:45.334 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:28:45.334 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-10T21:28:45.334 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-10T21:28:45.334 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:28:45.334 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:28:45.335 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:28:45.335 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:28:45.335 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:28:45.335 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:28:45.335 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": [ 2024-09-10T21:28:45.335 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:28:45.335 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:28:45.335 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:28:45.335 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 5 2024-09-10T21:28:45.335 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:28:45.335 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:28:45.335 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-10T21:28:45.335 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:28:45.335 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:28:45.336 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:28:45.336 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 7 2024-09-10T21:28:45.336 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:28:45.336 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:28:45.336 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-10T21:28:45.336 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:28:45.336 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:28:45.336 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:28:45.336 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 4 2024-09-10T21:28:45.336 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:28:45.336 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:28:45.336 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": [ 2024-09-10T21:28:45.337 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:28:45.337 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:28:45.337 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:28:45.337 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 2 2024-09-10T21:28:45.337 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:28:45.337 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:28:45.337 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:28:45.337 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-10T21:28:45.337 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-10T21:28:45.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:45.316+0000 7f4b57400640 1 -- 172.21.15.17:0/3588285844 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f4b300724e0 msgr2=0x7f4b300749a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:45.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:45.316+0000 7f4b57400640 1 --2- 172.21.15.17:0/3588285844 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f4b300724e0 0x7f4b300749a0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f4b501eed70 tx=0x7f4b3c007a20 comp rx=0 tx=0).stop 2024-09-10T21:28:45.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:45.316+0000 7f4b57400640 1 -- 172.21.15.17:0/3588285844 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4b50151800 msgr2=0x7f4b501eb100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:28:45.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:45.316+0000 7f4b57400640 1 --2- 172.21.15.17:0/3588285844 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4b50151800 0x7f4b501eb100 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f4b4800c700 tx=0x7f4b4800cbc0 comp rx=0 tx=0).stop 2024-09-10T21:28:45.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:45.316+0000 7f4b55400640 1 -- 172.21.15.17:0/3588285844 reap_dead start 2024-09-10T21:28:45.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:45.316+0000 7f4b55400640 1 -- 172.21.15.17:0/3588285844 reap_dead start 2024-09-10T21:28:45.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:45.316+0000 7f4b57400640 1 -- 172.21.15.17:0/3588285844 shutdown_connections 2024-09-10T21:28:45.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:45.316+0000 7f4b57400640 1 -- 172.21.15.17:0/3588285844 >> 172.21.15.17:0/3588285844 conn(0x7f4b5006cc20 msgr2=0x7f4b500c5220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:28:45.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:45.316+0000 7f4b57400640 1 -- 172.21.15.17:0/3588285844 shutdown_connections 2024-09-10T21:28:45.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:28:45.316+0000 7f4b57400640 1 -- 172.21.15.17:0/3588285844 wait complete. 2024-09-10T21:28:45.343 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-09-10T21:28:45.344 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2024-09-10T21:28:46.985 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:28:46.989 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:29:05.344 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.c 2024-09-10T21:29:05.345 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-10T21:29:05.345 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-10T21:29:05.349 INFO:tasks.ceph.mon.c:Started 2024-09-10T21:29:05.349 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-09-10T21:29:05.349 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-10T21:29:05.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:05.504+0000 7f998ec00640 1 -- 172.21.15.17:0/901575612 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f99881527e0 msgr2=0x7f9988158310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:05.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:05.504+0000 7f998ec00640 1 --2- 172.21.15.17:0/901575612 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f99881527e0 0x7f9988158310 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f997c009900 tx=0x7f997c01f0d0 comp rx=0 tx=0).stop 2024-09-10T21:29:05.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:05.504+0000 7f998ec00640 1 -- 172.21.15.17:0/901575612 shutdown_connections 2024-09-10T21:29:05.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:05.504+0000 7f998ec00640 1 --2- 172.21.15.17:0/901575612 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f998806fd00 0x7f99880c7870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:05.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:05.504+0000 7f998ec00640 1 --2- 172.21.15.17:0/901575612 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f99881527e0 0x7f9988158310 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:05.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:05.504+0000 7f998ec00640 1 -- 172.21.15.17:0/901575612 >> 172.21.15.17:0/901575612 conn(0x7f9988070ca0 msgr2=0x7f99880710b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:05.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:05.508+0000 7f998ec00640 1 -- 172.21.15.17:0/901575612 shutdown_connections 2024-09-10T21:29:05.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:05.508+0000 7f998ec00640 1 -- 172.21.15.17:0/901575612 wait complete. 2024-09-10T21:29:05.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:05.508+0000 7f998ec00640 1 Processor -- start 2024-09-10T21:29:05.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:05.508+0000 7f998ec00640 1 -- start start 2024-09-10T21:29:05.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:05.508+0000 7f998ec00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f998806fd00 0x7f99881f08c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:05.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:05.508+0000 7f998ec00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f99880b2d80 0x7f99881f4990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:05.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:05.508+0000 7f998ec00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f99881527e0 0x7f99880741a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:05.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:05.508+0000 7f998ec00640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f998806f500 con 0x7f99881527e0 2024-09-10T21:29:05.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:05.508+0000 7f998ec00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f998806f220 con 0x7f998806fd00 2024-09-10T21:29:05.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:05.508+0000 7f998ec00640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f998806ed40 con 0x7f99880b2d80 2024-09-10T21:29:05.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:05.508+0000 7f998d800640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f998806fd00 0x7f99881f08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:05.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:05.508+0000 7f998e200640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f99881527e0 0x7f99880741a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:05.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:05.508+0000 7f998d800640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f998806fd00 0x7f99881f08c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.17:34550/0 (socket says 172.21.15.17:34550) 2024-09-10T21:29:05.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:05.508+0000 7f998d800640 1 -- 172.21.15.17:0/450906627 learned_addr learned my addr 172.21.15.17:0/450906627 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:29:05.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:05.508+0000 7f998ce00640 1 --2- 172.21.15.17:0/450906627 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f99880b2d80 0x7f99881f4990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:05.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:05.508+0000 7f998ce00640 1 -- 172.21.15.17:0/450906627 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f99880b2d80 msgr2=0x7f99881f4990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-09-10T21:29:05.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:05.508+0000 7f998ce00640 1 -- 172.21.15.17:0/450906627 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f99880b2d80 msgr2=0x7f99881f4990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-10T21:29:05.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:05.508+0000 7f998ce00640 1 --2- 172.21.15.17:0/450906627 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f99880b2d80 0x7f99881f4990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T21:29:05.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:05.508+0000 7f998ce00640 1 --2- 172.21.15.17:0/450906627 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f99880b2d80 0x7f99881f4990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:29:05.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:05.508+0000 7f998e200640 1 -- 172.21.15.17:0/450906627 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f99880b2d80 msgr2=0x7f99881f4990 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:29:05.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:05.508+0000 7f998e200640 1 --2- 172.21.15.17:0/450906627 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f99880b2d80 0x7f99881f4990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:05.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:05.508+0000 7f998e200640 1 -- 172.21.15.17:0/450906627 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f998806fd00 msgr2=0x7f99881f08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:05.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:05.508+0000 7f998e200640 1 --2- 172.21.15.17:0/450906627 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f998806fd00 0x7f99881f08c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:05.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:05.508+0000 7f998e200640 1 -- 172.21.15.17:0/450906627 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99880c6420 con 0x7f99881527e0 2024-09-10T21:29:05.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:05.508+0000 7f998d800640 1 --2- 172.21.15.17:0/450906627 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f998806fd00 0x7f99881f08c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:29:05.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:05.512+0000 7f998e200640 1 --2- 172.21.15.17:0/450906627 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f99881527e0 0x7f99880741a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f998000e990 tx=0x7f998000ed90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:05.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:05.512+0000 7f9985600640 1 -- 172.21.15.17:0/450906627 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9980016070 con 0x7f99881527e0 2024-09-10T21:29:05.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:05.512+0000 7f9985600640 1 -- 172.21.15.17:0/450906627 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f998000c850 con 0x7f99881527e0 2024-09-10T21:29:05.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:05.512+0000 7f998ec00640 1 -- 172.21.15.17:0/450906627 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99880c6700 con 0x7f99881527e0 2024-09-10T21:29:05.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:05.512+0000 7f9985600640 1 -- 172.21.15.17:0/450906627 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f998000cca0 con 0x7f99881527e0 2024-09-10T21:29:05.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:05.512+0000 7f998ec00640 1 -- 172.21.15.17:0/450906627 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99881f3eb0 con 0x7f99881527e0 2024-09-10T21:29:05.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:05.512+0000 7f998ec00640 1 -- 172.21.15.17:0/450906627 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9988158310 con 0x7f99881527e0 2024-09-10T21:29:05.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:05.512+0000 7f9985600640 1 -- 172.21.15.17:0/450906627 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f998000f040 con 0x7f99881527e0 2024-09-10T21:29:05.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:05.516+0000 7f9985600640 1 --2- 172.21.15.17:0/450906627 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f99680725e0 0x7f9968074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:05.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:05.516+0000 7f9985600640 1 -- 172.21.15.17:0/450906627 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f9980093c40 con 0x7f99881527e0 2024-09-10T21:29:05.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:05.516+0000 7f998d800640 1 --2- 172.21.15.17:0/450906627 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f99680725e0 0x7f9968074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:05.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:05.516+0000 7f998d800640 1 --2- 172.21.15.17:0/450906627 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f99680725e0 0x7f9968074aa0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f997c01d780 tx=0x7f997c01d910 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:05.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:05.520+0000 7f9985600640 1 -- 172.21.15.17:0/450906627 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f998005e830 con 0x7f99881527e0 2024-09-10T21:29:05.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:05.788+0000 7f998ec00640 1 -- 172.21.15.17:0/450906627 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f9988155d30 con 0x7f99881527e0 2024-09-10T21:29:10.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:10.672+0000 7f9985600640 1 -- 172.21.15.17:0/450906627 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1550 (secure 0 0 0) 0x7f9980063260 con 0x7f99881527e0 2024-09-10T21:29:10.679 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":38,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5e2f023f-d6a6-49f4-a256-f4807dce3739","modified":"2024-09-10T21:23:46.892130Z","created":"2024-09-10T21:23:46.892130Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3301","nonce":0},{"type":"v1","addr":"172.21.15.31:6790","nonce":0}]},"addr":"172.21.15.31:6790/0","public_addr":"172.21.15.31:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-10T21:29:10.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:10.676+0000 7f998ec00640 1 -- 172.21.15.17:0/450906627 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f99680725e0 msgr2=0x7f9968074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:10.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:10.676+0000 7f998ec00640 1 --2- 172.21.15.17:0/450906627 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f99680725e0 0x7f9968074aa0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f997c01d780 tx=0x7f997c01d910 comp rx=0 tx=0).stop 2024-09-10T21:29:10.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:10.676+0000 7f998ec00640 1 -- 172.21.15.17:0/450906627 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f99881527e0 msgr2=0x7f99880741a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:10.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:10.676+0000 7f998ec00640 1 --2- 172.21.15.17:0/450906627 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f99881527e0 0x7f99880741a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f998000e990 tx=0x7f998000ed90 comp rx=0 tx=0).stop 2024-09-10T21:29:10.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:10.676+0000 7f998ec00640 1 -- 172.21.15.17:0/450906627 shutdown_connections 2024-09-10T21:29:10.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:10.676+0000 7f998ec00640 1 --2- 172.21.15.17:0/450906627 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f99680725e0 0x7f9968074aa0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:10.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:10.676+0000 7f998ec00640 1 --2- 172.21.15.17:0/450906627 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f99881527e0 0x7f99880741a0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:10.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:10.676+0000 7f998ec00640 1 --2- 172.21.15.17:0/450906627 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f99880b2d80 0x7f99881f4990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:10.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:10.676+0000 7f998ec00640 1 --2- 172.21.15.17:0/450906627 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f998806fd00 0x7f99881f08c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:10.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:10.676+0000 7f998ec00640 1 -- 172.21.15.17:0/450906627 >> 172.21.15.17:0/450906627 conn(0x7f9988070ca0 msgr2=0x7f9988151d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:10.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:10.676+0000 7f998ec00640 1 -- 172.21.15.17:0/450906627 shutdown_connections 2024-09-10T21:29:10.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:10.676+0000 7f998ec00640 1 -- 172.21.15.17:0/450906627 wait complete. 2024-09-10T21:29:13.708 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-10T21:29:13.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:13.872+0000 7fbfd6e00640 1 -- 172.21.15.17:0/2945713748 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbfd0152820 msgr2=0x7fbfd00b3eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:13.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:13.872+0000 7fbfd6e00640 1 --2- 172.21.15.17:0/2945713748 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbfd0152820 0x7fbfd00b3eb0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fbfd01594a0 tx=0x7fbfbc01c850 comp rx=0 tx=0).stop 2024-09-10T21:29:13.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:13.872+0000 7fbfd6e00640 1 -- 172.21.15.17:0/2945713748 shutdown_connections 2024-09-10T21:29:13.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:13.872+0000 7fbfd6e00640 1 --2- 172.21.15.17:0/2945713748 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbfd0152820 0x7fbfd00b3eb0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:13.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:13.872+0000 7fbfd6e00640 1 -- 172.21.15.17:0/2945713748 >> 172.21.15.17:0/2945713748 conn(0x7fbfd00702f0 msgr2=0x7fbfd0070700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:13.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:13.872+0000 7fbfd6e00640 1 -- 172.21.15.17:0/2945713748 shutdown_connections 2024-09-10T21:29:13.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:13.872+0000 7fbfd6e00640 1 -- 172.21.15.17:0/2945713748 wait complete. 2024-09-10T21:29:13.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:13.876+0000 7fbfd6e00640 1 Processor -- start 2024-09-10T21:29:13.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:13.876+0000 7fbfd6e00640 1 -- start start 2024-09-10T21:29:13.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:13.876+0000 7fbfd6e00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbfd00cdf20 0x7fbfd01f4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:13.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:13.876+0000 7fbfd6e00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbfd0152440 0x7fbfd0157480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:13.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:13.876+0000 7fbfd6e00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fbfd0152820 0x7fbfd015a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:13.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:13.876+0000 7fbfd6e00640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fbfd00cb450 con 0x7fbfd00cdf20 2024-09-10T21:29:13.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:13.876+0000 7fbfd6e00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fbfd00ce740 con 0x7fbfd0152440 2024-09-10T21:29:13.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:13.876+0000 7fbfd6e00640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7fbfd00cb5d0 con 0x7fbfd0152820 2024-09-10T21:29:13.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:13.876+0000 7fbfcfe00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbfd00cdf20 0x7fbfd01f4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:13.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:13.876+0000 7fbfcfe00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbfd00cdf20 0x7fbfd01f4bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:52522/0 (socket says 172.21.15.17:52522) 2024-09-10T21:29:13.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:13.876+0000 7fbfcfe00640 1 -- 172.21.15.17:0/3451084269 learned_addr learned my addr 172.21.15.17:0/3451084269 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:29:13.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:13.876+0000 7fbfcf400640 1 --2- 172.21.15.17:0/3451084269 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbfd0152440 0x7fbfd0157480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:13.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:13.876+0000 7fbfd4e00640 1 --2- 172.21.15.17:0/3451084269 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fbfd0152820 0x7fbfd015a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:13.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:13.876+0000 7fbfd4e00640 1 -- 172.21.15.17:0/3451084269 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fbfd0152820 msgr2=0x7fbfd015a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-09-10T21:29:13.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:13.876+0000 7fbfd4e00640 1 -- 172.21.15.17:0/3451084269 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fbfd0152820 msgr2=0x7fbfd015a850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-10T21:29:13.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:13.876+0000 7fbfd4e00640 1 --2- 172.21.15.17:0/3451084269 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fbfd0152820 0x7fbfd015a850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T21:29:13.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:13.876+0000 7fbfd4e00640 1 --2- 172.21.15.17:0/3451084269 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fbfd0152820 0x7fbfd015a850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:29:13.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:13.876+0000 7fbfcfe00640 1 -- 172.21.15.17:0/3451084269 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fbfd0152820 msgr2=0x7fbfd015a850 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:29:13.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:13.876+0000 7fbfcfe00640 1 --2- 172.21.15.17:0/3451084269 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fbfd0152820 0x7fbfd015a850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:13.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:13.876+0000 7fbfcfe00640 1 -- 172.21.15.17:0/3451084269 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbfd0152440 msgr2=0x7fbfd0157480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:13.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:13.876+0000 7fbfcfe00640 1 --2- 172.21.15.17:0/3451084269 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbfd0152440 0x7fbfd0157480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:13.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:13.876+0000 7fbfcfe00640 1 -- 172.21.15.17:0/3451084269 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfd00b4050 con 0x7fbfd00cdf20 2024-09-10T21:29:13.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:13.876+0000 7fbfcf400640 1 --2- 172.21.15.17:0/3451084269 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbfd0152440 0x7fbfd0157480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:29:13.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:13.876+0000 7fbfcfe00640 1 --2- 172.21.15.17:0/3451084269 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbfd00cdf20 0x7fbfd01f4bd0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fbfc000c450 tx=0x7fbfc000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:13.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:13.876+0000 7fbfc7e00640 1 -- 172.21.15.17:0/3451084269 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfc00052d0 con 0x7fbfd00cdf20 2024-09-10T21:29:13.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:13.876+0000 7fbfc7e00640 1 -- 172.21.15.17:0/3451084269 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbfc0005470 con 0x7fbfd00cdf20 2024-09-10T21:29:13.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:13.876+0000 7fbfd6e00640 1 -- 172.21.15.17:0/3451084269 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfd015a060 con 0x7fbfd00cdf20 2024-09-10T21:29:13.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:13.876+0000 7fbfc7e00640 1 -- 172.21.15.17:0/3451084269 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfc0012870 con 0x7fbfd00cdf20 2024-09-10T21:29:13.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:13.876+0000 7fbfd6e00640 1 -- 172.21.15.17:0/3451084269 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfd01579c0 con 0x7fbfd00cdf20 2024-09-10T21:29:13.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:13.880+0000 7fbfd6e00640 1 -- 172.21.15.17:0/3451084269 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbfd015a390 con 0x7fbfd00cdf20 2024-09-10T21:29:13.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:13.880+0000 7fbfc7e00640 1 -- 172.21.15.17:0/3451084269 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fbfc0003600 con 0x7fbfd00cdf20 2024-09-10T21:29:13.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:13.880+0000 7fbfc7e00640 1 --2- 172.21.15.17:0/3451084269 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fbfb00725e0 0x7fbfb0074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:13.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:13.880+0000 7fbfc7e00640 1 -- 172.21.15.17:0/3451084269 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fbfc0093080 con 0x7fbfd00cdf20 2024-09-10T21:29:13.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:13.884+0000 7fbfcf400640 1 --2- 172.21.15.17:0/3451084269 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fbfb00725e0 0x7fbfb0074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:13.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:13.884+0000 7fbfcf400640 1 --2- 172.21.15.17:0/3451084269 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fbfb00725e0 0x7fbfb0074aa0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc0af000 tx=0x7fbfbc01b090 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:13.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:13.888+0000 7fbfc7e00640 1 -- 172.21.15.17:0/3451084269 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fbfc005e350 con 0x7fbfd00cdf20 2024-09-10T21:29:14.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.232+0000 7fbfd6e00640 1 -- 172.21.15.17:0/3451084269 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fbfd00c3160 con 0x7fbfd00cdf20 2024-09-10T21:29:14.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.232+0000 7fbfc7e00640 1 -- 172.21.15.17:0/3451084269 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1556 (secure 0 0 0) 0x7fbfc005dc50 con 0x7fbfd00cdf20 2024-09-10T21:29:14.239 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":40,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":3,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5e2f023f-d6a6-49f4-a256-f4807dce3739","modified":"2024-09-10T21:23:46.892130Z","created":"2024-09-10T21:23:46.892130Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3301","nonce":0},{"type":"v1","addr":"172.21.15.31:6790","nonce":0}]},"addr":"172.21.15.31:6790/0","public_addr":"172.21.15.31:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-10T21:29:14.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.236+0000 7fbfd6e00640 1 -- 172.21.15.17:0/3451084269 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fbfb00725e0 msgr2=0x7fbfb0074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:14.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.236+0000 7fbfd6e00640 1 --2- 172.21.15.17:0/3451084269 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fbfb00725e0 0x7fbfb0074aa0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fbfbc0af000 tx=0x7fbfbc01b090 comp rx=0 tx=0).stop 2024-09-10T21:29:14.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.236+0000 7fbfd6e00640 1 -- 172.21.15.17:0/3451084269 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbfd00cdf20 msgr2=0x7fbfd01f4bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:14.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.236+0000 7fbfd6e00640 1 --2- 172.21.15.17:0/3451084269 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbfd00cdf20 0x7fbfd01f4bd0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fbfc000c450 tx=0x7fbfc000c910 comp rx=0 tx=0).stop 2024-09-10T21:29:14.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.236+0000 7fbfd6e00640 1 -- 172.21.15.17:0/3451084269 shutdown_connections 2024-09-10T21:29:14.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.236+0000 7fbfd6e00640 1 --2- 172.21.15.17:0/3451084269 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fbfb00725e0 0x7fbfb0074aa0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:14.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.236+0000 7fbfd6e00640 1 --2- 172.21.15.17:0/3451084269 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fbfd0152820 0x7fbfd015a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:14.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.236+0000 7fbfd6e00640 1 --2- 172.21.15.17:0/3451084269 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbfd0152440 0x7fbfd0157480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:14.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.236+0000 7fbfd6e00640 1 --2- 172.21.15.17:0/3451084269 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbfd00cdf20 0x7fbfd01f4bd0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:14.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.236+0000 7fbfd6e00640 1 -- 172.21.15.17:0/3451084269 >> 172.21.15.17:0/3451084269 conn(0x7fbfd00702f0 msgr2=0x7fbfd00c3370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:14.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.236+0000 7fbfd6e00640 1 -- 172.21.15.17:0/3451084269 shutdown_connections 2024-09-10T21:29:14.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.236+0000 7fbfd6e00640 1 -- 172.21.15.17:0/3451084269 wait complete. 2024-09-10T21:29:14.261 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-09-10T21:29:14.261 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-09-10T21:29:14.261 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-10T21:29:14.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.412+0000 7f9c26200640 1 -- 172.21.15.17:0/978896622 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9c20159850 msgr2=0x7f9c200c4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:14.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.412+0000 7f9c26200640 1 --2- 172.21.15.17:0/978896622 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9c20159850 0x7f9c200c4dd0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f9c18009c80 tx=0x7f9c1801f5a0 comp rx=0 tx=0).stop 2024-09-10T21:29:14.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.412+0000 7f9c26200640 1 -- 172.21.15.17:0/978896622 shutdown_connections 2024-09-10T21:29:14.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.412+0000 7f9c26200640 1 --2- 172.21.15.17:0/978896622 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9c20159850 0x7f9c200c4dd0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:14.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.412+0000 7f9c26200640 1 -- 172.21.15.17:0/978896622 >> 172.21.15.17:0/978896622 conn(0x7f9c20068990 msgr2=0x7f9c20068da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:14.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.412+0000 7f9c26200640 1 -- 172.21.15.17:0/978896622 shutdown_connections 2024-09-10T21:29:14.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.412+0000 7f9c26200640 1 -- 172.21.15.17:0/978896622 wait complete. 2024-09-10T21:29:14.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.416+0000 7f9c26200640 1 Processor -- start 2024-09-10T21:29:14.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.416+0000 7f9c26200640 1 -- start start 2024-09-10T21:29:14.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.416+0000 7f9c26200640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f9c20071760 0x7f9c2013fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:14.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.416+0000 7f9c26200640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9c201526b0 0x7f9c20143de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:14.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.416+0000 7f9c26200640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c20159850 0x7f9c20144320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:14.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.416+0000 7f9c26200640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f9c200c8a40 con 0x7f9c201526b0 2024-09-10T21:29:14.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.416+0000 7f9c26200640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f9c20152a90 con 0x7f9c20159850 2024-09-10T21:29:14.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.416+0000 7f9c26200640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f9c20159260 con 0x7f9c20071760 2024-09-10T21:29:14.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.416+0000 7f9c17e00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9c201526b0 0x7f9c20143de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:14.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.416+0000 7f9c17e00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9c201526b0 0x7f9c20143de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:52546/0 (socket says 172.21.15.17:52546) 2024-09-10T21:29:14.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.416+0000 7f9c17e00640 1 -- 172.21.15.17:0/2566508576 learned_addr learned my addr 172.21.15.17:0/2566508576 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:29:14.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.416+0000 7f9c1fe00640 1 --2- 172.21.15.17:0/2566508576 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c20159850 0x7f9c20144320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:14.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.416+0000 7f9c1f400640 1 --2- 172.21.15.17:0/2566508576 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f9c20071760 0x7f9c2013fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:14.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.416+0000 7f9c17e00640 1 -- 172.21.15.17:0/2566508576 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f9c20071760 msgr2=0x7f9c2013fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:14.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.416+0000 7f9c17e00640 1 --2- 172.21.15.17:0/2566508576 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f9c20071760 0x7f9c2013fbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:14.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.416+0000 7f9c17e00640 1 -- 172.21.15.17:0/2566508576 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c20159850 msgr2=0x7f9c20144320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:14.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.416+0000 7f9c17e00640 1 --2- 172.21.15.17:0/2566508576 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c20159850 0x7f9c20144320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:14.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.416+0000 7f9c17e00640 1 -- 172.21.15.17:0/2566508576 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c20148f60 con 0x7f9c201526b0 2024-09-10T21:29:14.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.416+0000 7f9c1fe00640 1 --2- 172.21.15.17:0/2566508576 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9c20159850 0x7f9c20144320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:29:14.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.416+0000 7f9c17e00640 1 --2- 172.21.15.17:0/2566508576 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9c201526b0 0x7f9c20143de0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f9c0800c430 tx=0x7f9c0800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:14.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.416+0000 7f9c17400640 1 -- 172.21.15.17:0/2566508576 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c08017070 con 0x7f9c201526b0 2024-09-10T21:29:14.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.416+0000 7f9c17400640 1 -- 172.21.15.17:0/2566508576 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9c080053c0 con 0x7f9c201526b0 2024-09-10T21:29:14.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.416+0000 7f9c17400640 1 -- 172.21.15.17:0/2566508576 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c08003880 con 0x7f9c201526b0 2024-09-10T21:29:14.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.416+0000 7f9c26200640 1 -- 172.21.15.17:0/2566508576 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c201489e0 con 0x7f9c201526b0 2024-09-10T21:29:14.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.416+0000 7f9c26200640 1 -- 172.21.15.17:0/2566508576 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c20148c80 con 0x7f9c201526b0 2024-09-10T21:29:14.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.420+0000 7f9c26200640 1 --2- 172.21.15.17:0/2566508576 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9c200cdb30 0x7f9c2006e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:14.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.420+0000 7f9c26200640 1 -- 172.21.15.17:0/2566508576 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f9c200714e0 con 0x7f9c200cdb30 2024-09-10T21:29:14.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.420+0000 7f9c1f400640 1 --2- 172.21.15.17:0/2566508576 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9c200cdb30 0x7f9c2006e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:14.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.420+0000 7f9c1f400640 1 -- 172.21.15.17:0/2566508576 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9c200c4dd0 con 0x7f9c200cdb30 2024-09-10T21:29:14.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.420+0000 7f9c1f400640 1 --2- 172.21.15.17:0/2566508576 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9c200cdb30 0x7f9c2006e510 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f9c1000e4b0 tx=0x7f9c1000e970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:14.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.420+0000 7f9c17400640 1 -- 172.21.15.17:0/2566508576 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c100052e0 con 0x7f9c200cdb30 2024-09-10T21:29:14.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.420+0000 7f9c17400640 1 -- 172.21.15.17:0/2566508576 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f9c10018070 con 0x7f9c200cdb30 2024-09-10T21:29:14.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.420+0000 7f9c17400640 1 -- 172.21.15.17:0/2566508576 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9c200cdb30 msgr2=0x7f9c2006e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:14.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.420+0000 7f9c17400640 1 --2- 172.21.15.17:0/2566508576 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9c200cdb30 0x7f9c2006e510 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f9c1000e4b0 tx=0x7f9c1000e970 comp rx=0 tx=0).stop 2024-09-10T21:29:14.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.420+0000 7f9c17400640 1 -- 172.21.15.17:0/2566508576 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9c200cdb30 msgr2=0x7f9c2006e510 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:29:14.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.420+0000 7f9c17400640 1 --2- 172.21.15.17:0/2566508576 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9c200cdb30 0x7f9c2006e510 secure :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f9c1000e4b0 tx=0x7f9c1000e970 comp rx=0 tx=0).stop 2024-09-10T21:29:14.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.420+0000 7f9c17400640 1 -- 172.21.15.17:0/2566508576 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f9c08003a20 con 0x7f9c201526b0 2024-09-10T21:29:14.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.420+0000 7f9c17400640 1 --2- 172.21.15.17:0/2566508576 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f9bfc0720f0 0x7f9bfc0745b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:14.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.420+0000 7f9c17400640 1 -- 172.21.15.17:0/2566508576 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f9c08092d40 con 0x7f9c201526b0 2024-09-10T21:29:14.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.420+0000 7f9c1f400640 1 --2- 172.21.15.17:0/2566508576 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f9bfc0720f0 0x7f9bfc0745b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:14.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.424+0000 7f9c1f400640 1 --2- 172.21.15.17:0/2566508576 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f9bfc0720f0 0x7f9bfc0745b0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f9c10001020 tx=0x7f9c10005000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:14.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.428+0000 7f9c26200640 1 --2- 172.21.15.17:0/2566508576 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9c2006de80 0x7f9c2006ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:14.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.428+0000 7f9c26200640 1 -- 172.21.15.17:0/2566508576 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f9c2014e8f0 con 0x7f9c2006de80 2024-09-10T21:29:14.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.428+0000 7f9c1fe00640 1 --2- 172.21.15.17:0/2566508576 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9c2006de80 0x7f9c2006ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:14.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.428+0000 7f9c1fe00640 1 -- 172.21.15.17:0/2566508576 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f9c200cb490 con 0x7f9c2006de80 2024-09-10T21:29:14.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.428+0000 7f9c1fe00640 1 --2- 172.21.15.17:0/2566508576 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9c2006de80 0x7f9c2006ea50 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9c200cbed0 tx=0x7f9c18003a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:14.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.428+0000 7f9c17400640 1 -- 172.21.15.17:0/2566508576 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c18004040 con 0x7f9c2006de80 2024-09-10T21:29:14.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.428+0000 7f9c17400640 1 -- 172.21.15.17:0/2566508576 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4876 (secure 0 0 0) 0x7f9c180b4070 con 0x7f9c2006de80 2024-09-10T21:29:14.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.428+0000 7f9c17400640 1 -- 172.21.15.17:0/2566508576 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9c2006de80 msgr2=0x7f9c2006ea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:14.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.428+0000 7f9c17400640 1 --2- 172.21.15.17:0/2566508576 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9c2006de80 0x7f9c2006ea50 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9c200cbed0 tx=0x7f9c18003a10 comp rx=0 tx=0).stop 2024-09-10T21:29:14.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.428+0000 7f9c17400640 1 -- 172.21.15.17:0/2566508576 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9c2006de80 msgr2=0x7f9c2006ea50 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:29:14.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.428+0000 7f9c17400640 1 --2- 172.21.15.17:0/2566508576 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9c2006de80 0x7f9c2006ea50 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:14.435 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-10T21:29:14.435 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:29:14.435 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:29:14.435 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "leader", 2024-09-10T21:29:14.435 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 40, 2024-09-10T21:29:14.435 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 72267, 2024-09-10T21:29:14.435 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-10T21:29:14.435 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-10T21:29:14.435 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-10T21:29:14.435 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-10T21:29:14.435 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:29:14.436 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 3, 2024-09-10T21:29:14.436 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:29:14.436 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-10T21:29:14.436 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-10T21:29:14.436 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:29:14.436 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:29:14.436 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:29:14.436 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:29:14.436 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:29:14.436 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:29:14.436 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:29:14.436 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:29:14.436 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:29:14.436 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:29:14.437 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:29:14.437 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:29:14.437 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-10T21:29:14.437 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-10T21:29:14.437 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:29:14.437 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:29:14.437 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:29:14.437 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:29:14.437 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:29:14.437 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:29:14.437 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:29:14.437 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:29:14.437 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:29:14.437 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:29:14.438 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:29:14.438 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:29:14.438 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:29:14.438 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-10T21:29:14.438 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-10T21:29:14.438 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-10T21:29:14.438 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-10T21:29:14.438 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-10T21:29:14.438 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "5e2f023f-d6a6-49f4-a256-f4807dce3739", 2024-09-10T21:29:14.438 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:29:14.438 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:29:14.438 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-10T21:29:14.438 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-10T21:29:14.438 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-10T21:29:14.439 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-10T21:29:14.439 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-10T21:29:14.439 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-10T21:29:14.439 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-10T21:29:14.439 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:29:14.439 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-10T21:29:14.439 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:29:14.439 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:29:14.439 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:29:14.439 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:29:14.439 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:29:14.439 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:29:14.440 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:29:14.440 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:29:14.440 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:29:14.440 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:29:14.440 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:29:14.440 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:29:14.440 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-10T21:29:14.440 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:29:14.440 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-10T21:29:14.440 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:29:14.440 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:29:14.440 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:29:14.440 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:29:14.440 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:29:14.441 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:29:14.441 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:29:14.441 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-10T21:29:14.441 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:29:14.441 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:29:14.441 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:29:14.441 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:29:14.441 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-10T21:29:14.441 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:29:14.441 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:29:14.441 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:29:14.441 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:29:14.441 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-10T21:29:14.442 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-10T21:29:14.442 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:29:14.442 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:29:14.442 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:29:14.442 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:29:14.442 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:29:14.442 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:29:14.442 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:29:14.442 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:29:14.442 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:29:14.442 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:29:14.442 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:29:14.442 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3300", 2024-09-10T21:29:14.442 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:29:14.443 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:29:14.443 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:29:14.443 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:29:14.443 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789", 2024-09-10T21:29:14.443 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:29:14.443 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:29:14.443 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:29:14.443 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:29:14.443 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789/0", 2024-09-10T21:29:14.443 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6789/0", 2024-09-10T21:29:14.443 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:29:14.443 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:29:14.443 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:29:14.444 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:29:14.444 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:29:14.444 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:29:14.444 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:29:14.444 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:29:14.444 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:29:14.444 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:29:14.444 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:29:14.444 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3301", 2024-09-10T21:29:14.444 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:29:14.444 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:29:14.444 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:29:14.444 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:29:14.444 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790", 2024-09-10T21:29:14.445 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:29:14.445 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:29:14.445 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:29:14.445 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:29:14.445 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790/0", 2024-09-10T21:29:14.445 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6790/0", 2024-09-10T21:29:14.445 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:29:14.445 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:29:14.445 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:29:14.445 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:29:14.445 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:29:14.445 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:29:14.445 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-10T21:29:14.446 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-10T21:29:14.446 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:29:14.446 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:29:14.446 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:29:14.446 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:29:14.446 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:29:14.446 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:29:14.446 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": [ 2024-09-10T21:29:14.446 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:29:14.446 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:29:14.446 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:29:14.446 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:29:14.446 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:29:14.447 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:29:14.447 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-10T21:29:14.447 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:29:14.447 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:29:14.447 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:29:14.447 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:29:14.447 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:29:14.447 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:29:14.447 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-10T21:29:14.447 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:29:14.447 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:29:14.447 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:29:14.447 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 4 2024-09-10T21:29:14.448 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:29:14.448 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:29:14.448 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:29:14.448 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-10T21:29:14.448 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-10T21:29:14.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.428+0000 7f9c26200640 1 -- 172.21.15.17:0/2566508576 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f9bfc0720f0 msgr2=0x7f9bfc0745b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:14.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.428+0000 7f9c26200640 1 --2- 172.21.15.17:0/2566508576 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f9bfc0720f0 0x7f9bfc0745b0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f9c10001020 tx=0x7f9c10005000 comp rx=0 tx=0).stop 2024-09-10T21:29:14.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.428+0000 7f9c26200640 1 -- 172.21.15.17:0/2566508576 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9c201526b0 msgr2=0x7f9c20143de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:14.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.428+0000 7f9c26200640 1 --2- 172.21.15.17:0/2566508576 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9c201526b0 0x7f9c20143de0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f9c0800c430 tx=0x7f9c0800c8f0 comp rx=0 tx=0).stop 2024-09-10T21:29:14.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.428+0000 7f9c1fe00640 1 -- 172.21.15.17:0/2566508576 reap_dead start 2024-09-10T21:29:14.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.428+0000 7f9c26200640 1 -- 172.21.15.17:0/2566508576 shutdown_connections 2024-09-10T21:29:14.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.428+0000 7f9c26200640 1 -- 172.21.15.17:0/2566508576 >> 172.21.15.17:0/2566508576 conn(0x7f9c20068990 msgr2=0x7f9c20155fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:14.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.428+0000 7f9c26200640 1 -- 172.21.15.17:0/2566508576 shutdown_connections 2024-09-10T21:29:14.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.428+0000 7f9c26200640 1 -- 172.21.15.17:0/2566508576 wait complete. 2024-09-10T21:29:14.454 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-10T21:29:14.455 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-09-10T21:29:14.455 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-10T21:29:14.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.628+0000 7f202fe00640 1 -- 172.21.15.17:0/1734008753 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2030152450 msgr2=0x7f2030158850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:14.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.628+0000 7f202fe00640 1 --2- 172.21.15.17:0/1734008753 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2030152450 0x7f2030158850 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f201c009840 tx=0x7f201c01c820 comp rx=0 tx=0).stop 2024-09-10T21:29:14.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.628+0000 7f202fe00640 1 -- 172.21.15.17:0/1734008753 shutdown_connections 2024-09-10T21:29:14.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.628+0000 7f202fe00640 1 --2- 172.21.15.17:0/1734008753 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f20300cbd10 0x7f2030158d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:14.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.628+0000 7f202fe00640 1 --2- 172.21.15.17:0/1734008753 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2030152450 0x7f2030158850 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:14.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.628+0000 7f202fe00640 1 -- 172.21.15.17:0/1734008753 >> 172.21.15.17:0/1734008753 conn(0x7f203006cc20 msgr2=0x7f203006d030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:14.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.628+0000 7f202fe00640 1 -- 172.21.15.17:0/1734008753 shutdown_connections 2024-09-10T21:29:14.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.628+0000 7f202fe00640 1 -- 172.21.15.17:0/1734008753 wait complete. 2024-09-10T21:29:14.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.628+0000 7f202fe00640 1 Processor -- start 2024-09-10T21:29:14.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.628+0000 7f202fe00640 1 -- start start 2024-09-10T21:29:14.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.632+0000 7f202fe00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f20300cbd10 0x7f2030150c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:14.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.632+0000 7f202fe00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2030152070 0x7f20301473e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:14.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.632+0000 7f202fe00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f2030152450 0x7f20300c9630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:14.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.632+0000 7f202fe00640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f2030072cb0 con 0x7f2030152070 2024-09-10T21:29:14.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.632+0000 7f202fe00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f2030072b30 con 0x7f20300cbd10 2024-09-10T21:29:14.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.632+0000 7f202fe00640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f2030165400 con 0x7f2030152450 2024-09-10T21:29:14.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.632+0000 7f202e000640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2030152070 0x7f20301473e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:14.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.632+0000 7f202e000640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2030152070 0x7f20301473e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:52578/0 (socket says 172.21.15.17:52578) 2024-09-10T21:29:14.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.632+0000 7f202e000640 1 -- 172.21.15.17:0/2026192847 learned_addr learned my addr 172.21.15.17:0/2026192847 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:29:14.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.632+0000 7f202ea00640 1 --2- 172.21.15.17:0/2026192847 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f20300cbd10 0x7f2030150c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:14.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.632+0000 7f202f400640 1 --2- 172.21.15.17:0/2026192847 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f2030152450 0x7f20300c9630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:14.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.632+0000 7f202f400640 1 -- 172.21.15.17:0/2026192847 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f2030152450 msgr2=0x7f20300c9630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-09-10T21:29:14.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.632+0000 7f202f400640 1 -- 172.21.15.17:0/2026192847 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f2030152450 msgr2=0x7f20300c9630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-10T21:29:14.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.632+0000 7f202f400640 1 --2- 172.21.15.17:0/2026192847 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f2030152450 0x7f20300c9630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T21:29:14.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.632+0000 7f202e000640 1 -- 172.21.15.17:0/2026192847 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f2030152450 msgr2=0x7f20300c9630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:14.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.632+0000 7f202f400640 1 --2- 172.21.15.17:0/2026192847 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f2030152450 0x7f20300c9630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:29:14.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.632+0000 7f202e000640 1 --2- 172.21.15.17:0/2026192847 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f2030152450 0x7f20300c9630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:14.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.632+0000 7f202e000640 1 -- 172.21.15.17:0/2026192847 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f20300cbd10 msgr2=0x7f2030150c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:14.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.632+0000 7f202e000640 1 --2- 172.21.15.17:0/2026192847 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f20300cbd10 0x7f2030150c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:14.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.632+0000 7f202e000640 1 -- 172.21.15.17:0/2026192847 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2030148270 con 0x7f2030152070 2024-09-10T21:29:14.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.632+0000 7f202e000640 1 --2- 172.21.15.17:0/2026192847 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2030152070 0x7f20301473e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f201c009790 tx=0x7f201c0a6600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:14.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.632+0000 7f2026a00640 1 -- 172.21.15.17:0/2026192847 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f201c004240 con 0x7f2030152070 2024-09-10T21:29:14.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.632+0000 7f202fe00640 1 -- 172.21.15.17:0/2026192847 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20300c7be0 con 0x7f2030152070 2024-09-10T21:29:14.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.632+0000 7f2026a00640 1 -- 172.21.15.17:0/2026192847 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f201c0bf070 con 0x7f2030152070 2024-09-10T21:29:14.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.632+0000 7f2026a00640 1 -- 172.21.15.17:0/2026192847 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f201c003820 con 0x7f2030152070 2024-09-10T21:29:14.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.632+0000 7f202fe00640 1 -- 172.21.15.17:0/2026192847 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20300c7ea0 con 0x7f2030152070 2024-09-10T21:29:14.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.632+0000 7f202fe00640 1 --2- 172.21.15.17:0/2026192847 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f20301485f0 0x7f2030148b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:14.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.632+0000 7f202fe00640 1 -- 172.21.15.17:0/2026192847 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f2030069ef0 con 0x7f20301485f0 2024-09-10T21:29:14.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.632+0000 7f202ea00640 1 --2- 172.21.15.17:0/2026192847 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f20301485f0 0x7f2030148b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:14.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.636+0000 7f202ea00640 1 -- 172.21.15.17:0/2026192847 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2030072420 con 0x7f20301485f0 2024-09-10T21:29:14.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.636+0000 7f202ea00640 1 --2- 172.21.15.17:0/2026192847 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f20301485f0 0x7f2030148b90 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f202000f7a0 tx=0x7f202000fc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:14.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.636+0000 7f2026a00640 1 -- 172.21.15.17:0/2026192847 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2020018020 con 0x7f20301485f0 2024-09-10T21:29:14.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.636+0000 7f2026a00640 1 -- 172.21.15.17:0/2026192847 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f201c0039c0 con 0x7f2030152070 2024-09-10T21:29:14.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.636+0000 7f2026a00640 1 --2- 172.21.15.17:0/2026192847 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f20100724e0 0x7f20100749a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:14.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.636+0000 7f2026a00640 1 -- 172.21.15.17:0/2026192847 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f201c12ea70 con 0x7f2030152070 2024-09-10T21:29:14.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.636+0000 7f2026a00640 1 -- 172.21.15.17:0/2026192847 <== mon.1 v2:172.21.15.31:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f202000fe50 con 0x7f20301485f0 2024-09-10T21:29:14.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.636+0000 7f2026a00640 1 -- 172.21.15.17:0/2026192847 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f20301485f0 msgr2=0x7f2030148b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:14.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.636+0000 7f2026a00640 1 --2- 172.21.15.17:0/2026192847 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f20301485f0 0x7f2030148b90 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f202000f7a0 tx=0x7f202000fc60 comp rx=0 tx=0).stop 2024-09-10T21:29:14.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.636+0000 7f2026a00640 1 -- 172.21.15.17:0/2026192847 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f20301485f0 msgr2=0x7f2030148b90 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:29:14.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.636+0000 7f2026a00640 1 --2- 172.21.15.17:0/2026192847 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f20301485f0 0x7f2030148b90 secure :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f202000f7a0 tx=0x7f202000fc60 comp rx=0 tx=0).stop 2024-09-10T21:29:14.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.636+0000 7f202f400640 1 --2- 172.21.15.17:0/2026192847 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f20100724e0 0x7f20100749a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:14.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.636+0000 7f202f400640 1 --2- 172.21.15.17:0/2026192847 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f20100724e0 0x7f20100749a0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f2028005c50 tx=0x7f2028005be0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:14.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.644+0000 7f202fe00640 1 --2- 172.21.15.17:0/2026192847 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2030147d80 0x7f20300c9b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:14.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.644+0000 7f202fe00640 1 -- 172.21.15.17:0/2026192847 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f20300cc710 con 0x7f2030147d80 2024-09-10T21:29:14.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.644+0000 7f202ea00640 1 --2- 172.21.15.17:0/2026192847 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2030147d80 0x7f20300c9b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:14.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.644+0000 7f202ea00640 1 -- 172.21.15.17:0/2026192847 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f20300cb7e0 con 0x7f2030147d80 2024-09-10T21:29:14.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.644+0000 7f202ea00640 1 --2- 172.21.15.17:0/2026192847 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2030147d80 0x7f20300c9b70 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f202000f7a0 tx=0x7f2020004630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:14.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.644+0000 7f2026a00640 1 -- 172.21.15.17:0/2026192847 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2020014470 con 0x7f2030147d80 2024-09-10T21:29:14.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.644+0000 7f2026a00640 1 -- 172.21.15.17:0/2026192847 <== mon.1 v2:172.21.15.31:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5042 (secure 0 0 0) 0x7f2020014470 con 0x7f2030147d80 2024-09-10T21:29:14.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.644+0000 7f2026a00640 1 -- 172.21.15.17:0/2026192847 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2030147d80 msgr2=0x7f20300c9b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:14.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.644+0000 7f2026a00640 1 --2- 172.21.15.17:0/2026192847 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2030147d80 0x7f20300c9b70 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f202000f7a0 tx=0x7f2020004630 comp rx=0 tx=0).stop 2024-09-10T21:29:14.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.644+0000 7f2026a00640 1 -- 172.21.15.17:0/2026192847 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2030147d80 msgr2=0x7f20300c9b70 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:29:14.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.644+0000 7f2026a00640 1 --2- 172.21.15.17:0/2026192847 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2030147d80 0x7f20300c9b70 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:14.652 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-10T21:29:14.652 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:29:14.652 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:29:14.652 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-10T21:29:14.652 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 40, 2024-09-10T21:29:14.652 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 293094, 2024-09-10T21:29:14.652 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-10T21:29:14.652 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-10T21:29:14.652 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-10T21:29:14.653 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-10T21:29:14.653 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:29:14.653 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 3, 2024-09-10T21:29:14.653 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:29:14.653 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-10T21:29:14.653 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-10T21:29:14.653 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:29:14.653 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:29:14.653 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:29:14.653 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:29:14.653 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:29:14.653 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:29:14.653 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:29:14.653 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:29:14.653 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:29:14.654 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:29:14.654 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:29:14.654 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:29:14.654 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-10T21:29:14.654 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-10T21:29:14.654 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:29:14.654 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:29:14.654 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:29:14.654 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:29:14.654 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:29:14.655 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:29:14.655 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:29:14.655 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:29:14.655 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:29:14.655 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:29:14.655 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:29:14.655 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:29:14.655 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:29:14.655 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-10T21:29:14.655 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-10T21:29:14.655 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-10T21:29:14.655 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-10T21:29:14.655 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-10T21:29:14.655 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "5e2f023f-d6a6-49f4-a256-f4807dce3739", 2024-09-10T21:29:14.656 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:29:14.656 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:29:14.656 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-10T21:29:14.656 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-10T21:29:14.656 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-10T21:29:14.656 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-10T21:29:14.656 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-10T21:29:14.656 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-10T21:29:14.656 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-10T21:29:14.656 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:29:14.656 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-10T21:29:14.656 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:29:14.656 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:29:14.657 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:29:14.657 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:29:14.657 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:29:14.657 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:29:14.657 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:29:14.657 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:29:14.657 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:29:14.657 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:29:14.657 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:29:14.657 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:29:14.657 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-10T21:29:14.657 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:29:14.657 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-10T21:29:14.658 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:29:14.658 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:29:14.658 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:29:14.658 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:29:14.658 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:29:14.658 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:29:14.658 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:29:14.658 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-10T21:29:14.658 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:29:14.658 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:29:14.658 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:29:14.659 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:29:14.659 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-10T21:29:14.659 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:29:14.659 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:29:14.659 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:29:14.659 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:29:14.659 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-10T21:29:14.659 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-10T21:29:14.659 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:29:14.659 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:29:14.659 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:29:14.659 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:29:14.659 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:29:14.660 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:29:14.660 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:29:14.660 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:29:14.660 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:29:14.660 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:29:14.660 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:29:14.660 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3300", 2024-09-10T21:29:14.660 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:29:14.660 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:29:14.660 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:29:14.660 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:29:14.660 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789", 2024-09-10T21:29:14.660 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:29:14.661 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:29:14.661 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:29:14.661 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:29:14.661 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789/0", 2024-09-10T21:29:14.661 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6789/0", 2024-09-10T21:29:14.661 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:29:14.661 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:29:14.661 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:29:14.661 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:29:14.661 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:29:14.661 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:29:14.661 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:29:14.661 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:29:14.662 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:29:14.662 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:29:14.662 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:29:14.662 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3301", 2024-09-10T21:29:14.662 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:29:14.662 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:29:14.662 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:29:14.662 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:29:14.662 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790", 2024-09-10T21:29:14.662 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:29:14.662 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:29:14.662 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:29:14.662 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:29:14.662 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790/0", 2024-09-10T21:29:14.662 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6790/0", 2024-09-10T21:29:14.663 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:29:14.663 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:29:14.663 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:29:14.663 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:29:14.663 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:29:14.663 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:29:14.663 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-10T21:29:14.663 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-10T21:29:14.663 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:29:14.663 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:29:14.663 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:29:14.663 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:29:14.663 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:29:14.663 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:29:14.664 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": [ 2024-09-10T21:29:14.664 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:29:14.664 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:29:14.664 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:29:14.664 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 5 2024-09-10T21:29:14.664 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:29:14.664 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:29:14.664 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-10T21:29:14.664 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:29:14.664 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:29:14.664 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:29:14.664 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 7 2024-09-10T21:29:14.664 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:29:14.664 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:29:14.665 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-10T21:29:14.665 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:29:14.665 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:29:14.665 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:29:14.665 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 4 2024-09-10T21:29:14.665 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:29:14.665 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:29:14.665 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": [ 2024-09-10T21:29:14.665 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:29:14.665 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:29:14.665 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:29:14.665 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 2 2024-09-10T21:29:14.665 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:29:14.665 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:29:14.666 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:29:14.666 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-10T21:29:14.666 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-10T21:29:14.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.644+0000 7f202fe00640 1 -- 172.21.15.17:0/2026192847 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f20100724e0 msgr2=0x7f20100749a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:14.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.644+0000 7f202fe00640 1 --2- 172.21.15.17:0/2026192847 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f20100724e0 0x7f20100749a0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f2028005c50 tx=0x7f2028005be0 comp rx=0 tx=0).stop 2024-09-10T21:29:14.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.644+0000 7f202fe00640 1 -- 172.21.15.17:0/2026192847 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2030152070 msgr2=0x7f20301473e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:14.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.644+0000 7f202fe00640 1 --2- 172.21.15.17:0/2026192847 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2030152070 0x7f20301473e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f201c009790 tx=0x7f201c0a6600 comp rx=0 tx=0).stop 2024-09-10T21:29:14.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.644+0000 7f202f400640 1 -- 172.21.15.17:0/2026192847 reap_dead start 2024-09-10T21:29:14.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.644+0000 7f202f400640 1 -- 172.21.15.17:0/2026192847 reap_dead start 2024-09-10T21:29:14.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.644+0000 7f202fe00640 1 -- 172.21.15.17:0/2026192847 shutdown_connections 2024-09-10T21:29:14.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.644+0000 7f202fe00640 1 -- 172.21.15.17:0/2026192847 >> 172.21.15.17:0/2026192847 conn(0x7f203006cc20 msgr2=0x7f20301571a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:14.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.644+0000 7f202fe00640 1 -- 172.21.15.17:0/2026192847 shutdown_connections 2024-09-10T21:29:14.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.644+0000 7f202fe00640 1 -- 172.21.15.17:0/2026192847 wait complete. 2024-09-10T21:29:14.671 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-10T21:29:14.671 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-09-10T21:29:14.672 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-10T21:29:14.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.824+0000 7fba3b200640 1 -- 172.21.15.17:0/2668121913 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fba3c1523d0 msgr2=0x7fba3c0c9f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:14.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.824+0000 7fba3b200640 1 --2- 172.21.15.17:0/2668121913 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fba3c1523d0 0x7fba3c0c9f60 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fba3c0c8d70 tx=0x7fba2c01c780 comp rx=0 tx=0).stop 2024-09-10T21:29:14.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.824+0000 7fba3b200640 1 -- 172.21.15.17:0/2668121913 shutdown_connections 2024-09-10T21:29:14.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.824+0000 7fba3b200640 1 --2- 172.21.15.17:0/2668121913 >> v2:172.21.15.31:3300/0 conn(0x7fba3c0ca4a0 0x7fba3c070c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:14.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.824+0000 7fba3b200640 1 --2- 172.21.15.17:0/2668121913 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fba3c1523d0 0x7fba3c0c9f60 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:14.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.824+0000 7fba3b200640 1 -- 172.21.15.17:0/2668121913 >> 172.21.15.17:0/2668121913 conn(0x7fba3c068990 msgr2=0x7fba3c068da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:14.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.824+0000 7fba3b200640 1 -- 172.21.15.17:0/2668121913 shutdown_connections 2024-09-10T21:29:14.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.824+0000 7fba3b200640 1 -- 172.21.15.17:0/2668121913 wait complete. 2024-09-10T21:29:14.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.828+0000 7fba3b200640 1 Processor -- start 2024-09-10T21:29:14.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.828+0000 7fba3b200640 1 -- start start 2024-09-10T21:29:14.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.828+0000 7fba3b200640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba3c0c9520 0x7fba3c152b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:14.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.828+0000 7fba3b200640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fba3c0ca4a0 0x7fba3c14bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:14.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.828+0000 7fba3b200640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fba3c1523d0 0x7fba3c153b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:14.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.828+0000 7fba3b200640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fba3c071150 con 0x7fba3c1523d0 2024-09-10T21:29:14.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.828+0000 7fba3b200640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fba3c0cb3c0 con 0x7fba3c0c9520 2024-09-10T21:29:14.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.828+0000 7fba3b200640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7fba3c0712d0 con 0x7fba3c0ca4a0 2024-09-10T21:29:14.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.828+0000 7fba39e00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba3c0c9520 0x7fba3c152b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:14.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.828+0000 7fba39e00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba3c0c9520 0x7fba3c152b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.17:38540/0 (socket says 172.21.15.17:38540) 2024-09-10T21:29:14.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.828+0000 7fba39e00640 1 -- 172.21.15.17:0/3089705323 learned_addr learned my addr 172.21.15.17:0/3089705323 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:29:14.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.828+0000 7fba3a800640 1 --2- 172.21.15.17:0/3089705323 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fba3c1523d0 0x7fba3c153b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:14.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.828+0000 7fba33e00640 1 --2- 172.21.15.17:0/3089705323 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fba3c0ca4a0 0x7fba3c14bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:14.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.828+0000 7fba33e00640 1 -- 172.21.15.17:0/3089705323 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fba3c0ca4a0 msgr2=0x7fba3c14bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-09-10T21:29:14.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.828+0000 7fba33e00640 1 -- 172.21.15.17:0/3089705323 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fba3c0ca4a0 msgr2=0x7fba3c14bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-10T21:29:14.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.828+0000 7fba33e00640 1 --2- 172.21.15.17:0/3089705323 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fba3c0ca4a0 0x7fba3c14bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T21:29:14.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.828+0000 7fba33e00640 1 --2- 172.21.15.17:0/3089705323 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fba3c0ca4a0 0x7fba3c14bbe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:29:14.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.828+0000 7fba39e00640 1 -- 172.21.15.17:0/3089705323 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fba3c0ca4a0 msgr2=0x7fba3c14bbe0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:29:14.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.828+0000 7fba39e00640 1 --2- 172.21.15.17:0/3089705323 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fba3c0ca4a0 0x7fba3c14bbe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:14.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.828+0000 7fba39e00640 1 -- 172.21.15.17:0/3089705323 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fba3c1523d0 msgr2=0x7fba3c153b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:14.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.828+0000 7fba39e00640 1 --2- 172.21.15.17:0/3089705323 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fba3c1523d0 0x7fba3c153b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:14.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.828+0000 7fba39e00640 1 -- 172.21.15.17:0/3089705323 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba3c0c76e0 con 0x7fba3c0c9520 2024-09-10T21:29:14.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.828+0000 7fba39e00640 1 --2- 172.21.15.17:0/3089705323 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba3c0c9520 0x7fba3c152b50 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fba2c01cc60 tx=0x7fba2c005e90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:14.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.828+0000 7fba32000640 1 -- 172.21.15.17:0/3089705323 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba2c0bb070 con 0x7fba3c0c9520 2024-09-10T21:29:14.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.832+0000 7fba32000640 1 -- 172.21.15.17:0/3089705323 <== mon.1 v2:172.21.15.31:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fba2c004400 con 0x7fba3c0c9520 2024-09-10T21:29:14.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.832+0000 7fba3b200640 1 -- 172.21.15.17:0/3089705323 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba3c1f5570 con 0x7fba3c0c9520 2024-09-10T21:29:14.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.832+0000 7fba32000640 1 -- 172.21.15.17:0/3089705323 <== mon.1 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba2c0b2040 con 0x7fba3c0c9520 2024-09-10T21:29:14.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.832+0000 7fba3b200640 1 -- 172.21.15.17:0/3089705323 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba3c15b0a0 con 0x7fba3c0c9520 2024-09-10T21:29:14.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.832+0000 7fba3b200640 1 --2- 172.21.15.17:0/3089705323 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fba3c15bd40 0x7fba3c154060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:14.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.832+0000 7fba3b200640 1 -- 172.21.15.17:0/3089705323 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7fba3c069ef0 con 0x7fba3c15bd40 2024-09-10T21:29:14.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.832+0000 7fba33e00640 1 --2- 172.21.15.17:0/3089705323 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fba3c15bd40 0x7fba3c154060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:14.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.832+0000 7fba33e00640 1 -- 172.21.15.17:0/3089705323 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fba3c15bd40 msgr2=0x7fba3c154060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 20 2024-09-10T21:29:14.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.832+0000 7fba33e00640 1 -- 172.21.15.17:0/3089705323 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fba3c15bd40 msgr2=0x7fba3c154060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-10T21:29:14.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.832+0000 7fba33e00640 1 --2- 172.21.15.17:0/3089705323 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fba3c15bd40 0x7fba3c154060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T21:29:14.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.832+0000 7fba33e00640 1 --2- 172.21.15.17:0/3089705323 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fba3c15bd40 0x7fba3c154060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:29:14.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.832+0000 7fba32000640 1 -- 172.21.15.17:0/3089705323 <== mon.1 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fba2c0052d0 con 0x7fba3c0c9520 2024-09-10T21:29:14.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.832+0000 7fba32000640 1 --2- 172.21.15.17:0/3089705323 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fba180725e0 0x7fba18074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:14.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.836+0000 7fba32000640 1 -- 172.21.15.17:0/3089705323 <== mon.1 v2:172.21.15.31:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fba2c12cb10 con 0x7fba3c0c9520 2024-09-10T21:29:14.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.836+0000 7fba3a800640 1 --2- 172.21.15.17:0/3089705323 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fba180725e0 0x7fba18074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:14.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:14.836+0000 7fba3a800640 1 --2- 172.21.15.17:0/3089705323 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fba180725e0 0x7fba18074aa0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fba340012f0 tx=0x7fba34009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:15.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:15.036+0000 7fba33e00640 1 --2- 172.21.15.17:0/3089705323 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fba3c15bd40 0x7fba3c154060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:15.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:15.036+0000 7fba33e00640 1 -- 172.21.15.17:0/3089705323 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fba3c15bd40 msgr2=0x7fba3c154060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 21 2024-09-10T21:29:15.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:15.036+0000 7fba33e00640 1 -- 172.21.15.17:0/3089705323 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fba3c15bd40 msgr2=0x7fba3c154060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-10T21:29:15.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:15.036+0000 7fba33e00640 1 --2- 172.21.15.17:0/3089705323 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fba3c15bd40 0x7fba3c154060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T21:29:15.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:15.036+0000 7fba33e00640 1 --2- 172.21.15.17:0/3089705323 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fba3c15bd40 0x7fba3c154060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-10T21:29:15.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:15.440+0000 7fba33e00640 1 --2- 172.21.15.17:0/3089705323 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fba3c15bd40 0x7fba3c154060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:15.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:15.440+0000 7fba33e00640 1 -- 172.21.15.17:0/3089705323 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fba3c15bd40 msgr2=0x7fba3c154060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 21 2024-09-10T21:29:15.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:15.440+0000 7fba33e00640 1 -- 172.21.15.17:0/3089705323 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fba3c15bd40 msgr2=0x7fba3c154060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-10T21:29:15.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:15.440+0000 7fba33e00640 1 --2- 172.21.15.17:0/3089705323 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fba3c15bd40 0x7fba3c154060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T21:29:15.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:15.440+0000 7fba33e00640 1 --2- 172.21.15.17:0/3089705323 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fba3c15bd40 0x7fba3c154060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-10T21:29:16.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:16.244+0000 7fba33e00640 1 --2- 172.21.15.17:0/3089705323 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fba3c15bd40 0x7fba3c154060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:16.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:16.244+0000 7fba33e00640 1 -- 172.21.15.17:0/3089705323 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fba3c156150 con 0x7fba3c15bd40 2024-09-10T21:29:16.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:16.244+0000 7fba33e00640 1 --2- 172.21.15.17:0/3089705323 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fba3c15bd40 0x7fba3c154060 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fba3c14dde0 tx=0x7fba24014ea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:16.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:16.244+0000 7fba32000640 1 -- 172.21.15.17:0/3089705323 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba24004030 con 0x7fba3c15bd40 2024-09-10T21:29:16.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:16.244+0000 7fba32000640 1 -- 172.21.15.17:0/3089705323 <== mon.2 v2:172.21.15.31:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fba24004030 con 0x7fba3c15bd40 2024-09-10T21:29:16.251 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:16.244+0000 7fba32000640 1 -- 172.21.15.17:0/3089705323 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fba3c15bd40 msgr2=0x7fba3c154060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:16.251 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:16.244+0000 7fba32000640 1 --2- 172.21.15.17:0/3089705323 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fba3c15bd40 0x7fba3c154060 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fba3c14dde0 tx=0x7fba24014ea0 comp rx=0 tx=0).stop 2024-09-10T21:29:16.251 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:16.244+0000 7fba32000640 1 -- 172.21.15.17:0/3089705323 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fba3c15bd40 msgr2=0x7fba3c154060 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:29:16.251 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:16.244+0000 7fba32000640 1 --2- 172.21.15.17:0/3089705323 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fba3c15bd40 0x7fba3c154060 secure :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fba3c14dde0 tx=0x7fba24014ea0 comp rx=0 tx=0).stop 2024-09-10T21:29:16.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:16.252+0000 7fba3b200640 1 --2- 172.21.15.17:0/3089705323 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fba3c15a6c0 0x7fba3c146a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:16.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:16.252+0000 7fba3b200640 1 -- 172.21.15.17:0/3089705323 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7fba3c1533f0 con 0x7fba3c15a6c0 2024-09-10T21:29:16.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:16.252+0000 7fba33e00640 1 --2- 172.21.15.17:0/3089705323 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fba3c15a6c0 0x7fba3c146a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:16.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:16.252+0000 7fba33e00640 1 -- 172.21.15.17:0/3089705323 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fba3c155e70 con 0x7fba3c15a6c0 2024-09-10T21:29:16.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:16.252+0000 7fba33e00640 1 --2- 172.21.15.17:0/3089705323 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fba3c15a6c0 0x7fba3c146a50 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fba3c147820 tx=0x7fba24004740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:16.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:16.252+0000 7fba32000640 1 -- 172.21.15.17:0/3089705323 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba24011e70 con 0x7fba3c15a6c0 2024-09-10T21:29:16.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:16.252+0000 7fba32000640 1 -- 172.21.15.17:0/3089705323 <== mon.2 v2:172.21.15.31:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4540 (secure 0 0 0) 0x7fba24000e50 con 0x7fba3c15a6c0 2024-09-10T21:29:16.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:16.252+0000 7fba32000640 1 -- 172.21.15.17:0/3089705323 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fba3c15a6c0 msgr2=0x7fba3c146a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:16.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:16.252+0000 7fba32000640 1 --2- 172.21.15.17:0/3089705323 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fba3c15a6c0 0x7fba3c146a50 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fba3c147820 tx=0x7fba24004740 comp rx=0 tx=0).stop 2024-09-10T21:29:16.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:16.252+0000 7fba32000640 1 -- 172.21.15.17:0/3089705323 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fba3c15a6c0 msgr2=0x7fba3c146a50 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:29:16.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:16.252+0000 7fba32000640 1 --2- 172.21.15.17:0/3089705323 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fba3c15a6c0 0x7fba3c146a50 secure :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fba3c147820 tx=0x7fba24004740 comp rx=0 tx=0).stop 2024-09-10T21:29:16.260 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-10T21:29:16.260 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:29:16.260 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:29:16.260 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-10T21:29:16.260 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 40, 2024-09-10T21:29:16.260 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 10807, 2024-09-10T21:29:16.260 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-10T21:29:16.260 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-10T21:29:16.260 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-10T21:29:16.260 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-10T21:29:16.260 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:29:16.260 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 5, 2024-09-10T21:29:16.261 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:29:16.261 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-10T21:29:16.261 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-10T21:29:16.261 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:29:16.261 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:29:16.261 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:29:16.261 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:29:16.261 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:29:16.261 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:29:16.261 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:29:16.261 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:29:16.261 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:29:16.261 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:29:16.261 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:29:16.261 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:29:16.262 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-10T21:29:16.262 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-10T21:29:16.262 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:29:16.262 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:29:16.262 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:29:16.262 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:29:16.262 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:29:16.262 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:29:16.262 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:29:16.262 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:29:16.262 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:29:16.262 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:29:16.262 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:29:16.262 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:29:16.263 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:29:16.263 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-10T21:29:16.263 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-10T21:29:16.263 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-10T21:29:16.263 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-10T21:29:16.263 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-10T21:29:16.263 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "5e2f023f-d6a6-49f4-a256-f4807dce3739", 2024-09-10T21:29:16.263 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:29:16.263 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:29:16.263 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-10T21:29:16.263 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-10T21:29:16.263 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-10T21:29:16.263 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-10T21:29:16.264 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-10T21:29:16.264 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-10T21:29:16.264 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-10T21:29:16.264 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:29:16.264 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-10T21:29:16.264 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:29:16.265 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:29:16.265 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:29:16.265 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:29:16.265 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:29:16.265 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:29:16.266 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:29:16.266 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:29:16.266 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:29:16.266 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:29:16.266 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:29:16.266 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:29:16.266 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-10T21:29:16.266 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:29:16.266 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-10T21:29:16.266 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:29:16.266 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:29:16.266 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:29:16.267 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:29:16.267 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:29:16.267 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:29:16.267 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:29:16.267 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-10T21:29:16.267 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:29:16.267 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:29:16.267 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:29:16.267 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:29:16.267 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-10T21:29:16.268 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:29:16.268 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:29:16.268 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:29:16.268 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:29:16.268 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-10T21:29:16.268 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-10T21:29:16.268 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:29:16.268 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:29:16.268 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:29:16.268 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:29:16.268 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:29:16.268 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:29:16.268 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:29:16.269 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:29:16.269 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:29:16.269 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:29:16.269 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:29:16.269 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3300", 2024-09-10T21:29:16.269 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:29:16.269 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:29:16.269 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:29:16.269 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:29:16.269 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789", 2024-09-10T21:29:16.269 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:29:16.269 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:29:16.269 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:29:16.270 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:29:16.270 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789/0", 2024-09-10T21:29:16.270 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6789/0", 2024-09-10T21:29:16.270 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:29:16.270 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:29:16.270 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:29:16.270 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:29:16.270 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:29:16.270 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:29:16.270 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:29:16.270 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:29:16.270 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:29:16.270 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:29:16.270 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:29:16.270 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3301", 2024-09-10T21:29:16.271 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:29:16.271 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:29:16.271 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:29:16.271 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:29:16.271 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790", 2024-09-10T21:29:16.271 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:29:16.271 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:29:16.271 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:29:16.271 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:29:16.271 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790/0", 2024-09-10T21:29:16.271 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6790/0", 2024-09-10T21:29:16.271 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:29:16.271 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:29:16.271 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:29:16.271 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:29:16.272 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:29:16.272 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:29:16.272 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-10T21:29:16.272 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-10T21:29:16.272 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:29:16.272 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:29:16.272 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:29:16.272 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:29:16.272 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:29:16.272 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:29:16.272 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-10T21:29:16.273 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:29:16.273 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:29:16.273 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:29:16.273 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:29:16.273 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:29:16.273 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:29:16.273 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:29:16.273 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-10T21:29:16.273 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-10T21:29:16.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:16.252+0000 7fba3b200640 1 -- 172.21.15.17:0/3089705323 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fba180725e0 msgr2=0x7fba18074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:16.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:16.252+0000 7fba3b200640 1 --2- 172.21.15.17:0/3089705323 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fba180725e0 0x7fba18074aa0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fba340012f0 tx=0x7fba34009040 comp rx=0 tx=0).stop 2024-09-10T21:29:16.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:16.252+0000 7fba3b200640 1 -- 172.21.15.17:0/3089705323 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba3c0c9520 msgr2=0x7fba3c152b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:16.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:16.252+0000 7fba3b200640 1 --2- 172.21.15.17:0/3089705323 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba3c0c9520 0x7fba3c152b50 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fba2c01cc60 tx=0x7fba2c005e90 comp rx=0 tx=0).stop 2024-09-10T21:29:16.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:16.252+0000 7fba3a800640 1 -- 172.21.15.17:0/3089705323 reap_dead start 2024-09-10T21:29:16.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:16.252+0000 7fba3a800640 1 -- 172.21.15.17:0/3089705323 reap_dead start 2024-09-10T21:29:16.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:16.252+0000 7fba3b200640 1 -- 172.21.15.17:0/3089705323 shutdown_connections 2024-09-10T21:29:16.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:16.252+0000 7fba3b200640 1 -- 172.21.15.17:0/3089705323 >> 172.21.15.17:0/3089705323 conn(0x7fba3c068990 msgr2=0x7fba3c157a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:16.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:16.252+0000 7fba3b200640 1 -- 172.21.15.17:0/3089705323 shutdown_connections 2024-09-10T21:29:16.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:16.256+0000 7fba3b200640 1 -- 172.21.15.17:0/3089705323 wait complete. 2024-09-10T21:29:16.280 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-10T21:29:16.280 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-09-10T21:29:16.281 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-09-10T21:29:16.460 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:16.456+0000 7fd4df200640 1 -- 172.21.15.17:0/491568811 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd4e01513e0 msgr2=0x7fd4e00c6510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:16.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:16.456+0000 7fd4df200640 1 --2- 172.21.15.17:0/491568811 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd4e01513e0 0x7fd4e00c6510 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fd4d8009bc0 tx=0x7fd4d801f3b0 comp rx=0 tx=0).stop 2024-09-10T21:29:16.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:16.456+0000 7fd4df200640 1 -- 172.21.15.17:0/491568811 shutdown_connections 2024-09-10T21:29:16.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:16.456+0000 7fd4df200640 1 --2- 172.21.15.17:0/491568811 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd4e01513e0 0x7fd4e00c6510 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:16.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:16.456+0000 7fd4df200640 1 --2- 172.21.15.17:0/491568811 >> v2:172.21.15.31:3300/0 conn(0x7fd4e0151000 0x7fd4e00c5fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:16.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:16.456+0000 7fd4df200640 1 -- 172.21.15.17:0/491568811 >> 172.21.15.17:0/491568811 conn(0x7fd4e006cdf0 msgr2=0x7fd4e006d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:16.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:16.456+0000 7fd4df200640 1 -- 172.21.15.17:0/491568811 shutdown_connections 2024-09-10T21:29:16.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:16.456+0000 7fd4df200640 1 -- 172.21.15.17:0/491568811 wait complete. 2024-09-10T21:29:16.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:16.456+0000 7fd4df200640 1 Processor -- start 2024-09-10T21:29:16.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:16.456+0000 7fd4df200640 1 -- start start 2024-09-10T21:29:16.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:16.456+0000 7fd4df200640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd4e0068e00 0x7fd4e0143fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:16.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:16.456+0000 7fd4df200640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd4e0151000 0x7fd4e01e4a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:16.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:16.456+0000 7fd4df200640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd4e01513e0 0x7fd4e01f1680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:16.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:16.456+0000 7fd4df200640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fd4e00c98d0 con 0x7fd4e0068e00 2024-09-10T21:29:16.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:16.456+0000 7fd4df200640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fd4e00c34c0 con 0x7fd4e0151000 2024-09-10T21:29:16.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:16.456+0000 7fd4df200640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7fd4e00c9a50 con 0x7fd4e01513e0 2024-09-10T21:29:16.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:16.456+0000 7fd4dde00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd4e0068e00 0x7fd4e0143fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:16.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:16.456+0000 7fd4dde00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd4e0068e00 0x7fd4e0143fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55904/0 (socket says 172.21.15.17:55904) 2024-09-10T21:29:16.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:16.456+0000 7fd4dde00640 1 -- 172.21.15.17:0/1132684114 learned_addr learned my addr 172.21.15.17:0/1132684114 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:29:16.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:16.460+0000 7fd4dd400640 1 --2- 172.21.15.17:0/1132684114 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd4e0151000 0x7fd4e01e4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:16.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:16.460+0000 7fd4de800640 1 --2- 172.21.15.17:0/1132684114 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd4e01513e0 0x7fd4e01f1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:16.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:16.460+0000 7fd4dde00640 1 -- 172.21.15.17:0/1132684114 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd4e01513e0 msgr2=0x7fd4e01f1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:16.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:16.460+0000 7fd4dde00640 1 --2- 172.21.15.17:0/1132684114 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd4e01513e0 0x7fd4e01f1680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:16.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:16.460+0000 7fd4dde00640 1 -- 172.21.15.17:0/1132684114 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd4e0151000 msgr2=0x7fd4e01e4a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:16.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:16.460+0000 7fd4dde00640 1 --2- 172.21.15.17:0/1132684114 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd4e0151000 0x7fd4e01e4a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:16.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:16.460+0000 7fd4dde00640 1 -- 172.21.15.17:0/1132684114 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4e0144510 con 0x7fd4e0068e00 2024-09-10T21:29:16.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:16.460+0000 7fd4dde00640 1 --2- 172.21.15.17:0/1132684114 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd4e0068e00 0x7fd4e0143fd0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd4d000ea20 tx=0x7fd4d000eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:16.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:16.460+0000 7fd4d6000640 1 -- 172.21.15.17:0/1132684114 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4d000cd80 con 0x7fd4e0068e00 2024-09-10T21:29:16.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:16.460+0000 7fd4d6000640 1 -- 172.21.15.17:0/1132684114 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd4d0010430 con 0x7fd4e0068e00 2024-09-10T21:29:16.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:16.460+0000 7fd4d6000640 1 -- 172.21.15.17:0/1132684114 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4d0010860 con 0x7fd4e0068e00 2024-09-10T21:29:16.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:16.460+0000 7fd4df200640 1 -- 172.21.15.17:0/1132684114 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4e0144930 con 0x7fd4e0068e00 2024-09-10T21:29:16.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:16.460+0000 7fd4df200640 1 -- 172.21.15.17:0/1132684114 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4e01e5430 con 0x7fd4e0068e00 2024-09-10T21:29:16.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:16.460+0000 7fd4df200640 1 -- 172.21.15.17:0/1132684114 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4e0145440 con 0x7fd4e0068e00 2024-09-10T21:29:16.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:16.460+0000 7fd4d6000640 1 -- 172.21.15.17:0/1132684114 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fd4d0010a00 con 0x7fd4e0068e00 2024-09-10T21:29:16.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:16.464+0000 7fd4d6000640 1 --2- 172.21.15.17:0/1132684114 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fd4c00725e0 0x7fd4c0074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:16.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:16.464+0000 7fd4d6000640 1 -- 172.21.15.17:0/1132684114 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fd4d00941f0 con 0x7fd4e0068e00 2024-09-10T21:29:16.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:16.464+0000 7fd4dd400640 1 --2- 172.21.15.17:0/1132684114 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fd4c00725e0 0x7fd4c0074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:16.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:16.464+0000 7fd4dd400640 1 --2- 172.21.15.17:0/1132684114 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fd4c00725e0 0x7fd4c0074aa0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fd4cc007fd0 tx=0x7fd4cc007f60 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:16.473 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:16.468+0000 7fd4d6000640 1 -- 172.21.15.17:0/1132684114 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fd4d005ede0 con 0x7fd4e0068e00 2024-09-10T21:29:16.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:16.760+0000 7fd4df200640 1 -- 172.21.15.17:0/1132684114 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) -- 0x7fd4e0157b90 con 0x7fd4e0068e00 2024-09-10T21:29:16.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:16.760+0000 7fd4d6000640 1 -- 172.21.15.17:0/1132684114 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) ==== 57+0+0 (secure 0 0 0) 0x7fd4d0063810 con 0x7fd4e0068e00 2024-09-10T21:29:16.769 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:16.764+0000 7fd4df200640 1 -- 172.21.15.17:0/1132684114 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fd4c00725e0 msgr2=0x7fd4c0074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:16.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:16.764+0000 7fd4df200640 1 --2- 172.21.15.17:0/1132684114 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fd4c00725e0 0x7fd4c0074aa0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fd4cc007fd0 tx=0x7fd4cc007f60 comp rx=0 tx=0).stop 2024-09-10T21:29:16.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:16.764+0000 7fd4df200640 1 -- 172.21.15.17:0/1132684114 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd4e0068e00 msgr2=0x7fd4e0143fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:16.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:16.764+0000 7fd4df200640 1 --2- 172.21.15.17:0/1132684114 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd4e0068e00 0x7fd4e0143fd0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd4d000ea20 tx=0x7fd4d000eee0 comp rx=0 tx=0).stop 2024-09-10T21:29:16.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:16.764+0000 7fd4df200640 1 -- 172.21.15.17:0/1132684114 shutdown_connections 2024-09-10T21:29:16.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:16.764+0000 7fd4df200640 1 --2- 172.21.15.17:0/1132684114 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fd4c00725e0 0x7fd4c0074aa0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:16.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:16.764+0000 7fd4df200640 1 --2- 172.21.15.17:0/1132684114 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd4e01513e0 0x7fd4e01f1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:16.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:16.764+0000 7fd4df200640 1 --2- 172.21.15.17:0/1132684114 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd4e0151000 0x7fd4e01e4a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:16.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:16.764+0000 7fd4df200640 1 --2- 172.21.15.17:0/1132684114 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd4e0068e00 0x7fd4e0143fd0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:16.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:16.764+0000 7fd4df200640 1 -- 172.21.15.17:0/1132684114 >> 172.21.15.17:0/1132684114 conn(0x7fd4e006cdf0 msgr2=0x7fd4e0153d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:16.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:16.764+0000 7fd4df200640 1 -- 172.21.15.17:0/1132684114 shutdown_connections 2024-09-10T21:29:16.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:16.764+0000 7fd4df200640 1 -- 172.21.15.17:0/1132684114 wait complete. 2024-09-10T21:29:16.798 INFO:tasks.mon_thrash.mon_thrasher:waiting for 10.0 secs before continuing thrashing 2024-09-10T21:29:17.047 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:29:17.051 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:29:26.799 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-09-10T21:29:26.799 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-10T21:29:26.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:26.963+0000 7fd987000640 1 -- 172.21.15.17:0/2674387755 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd9800b2d80 msgr2=0x7fd980158310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:26.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:26.963+0000 7fd987000640 1 --2- 172.21.15.17:0/2674387755 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd9800b2d80 0x7fd980158310 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fd96c009900 tx=0x7fd96c01f0d0 comp rx=0 tx=0).stop 2024-09-10T21:29:26.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:26.963+0000 7fd987000640 1 -- 172.21.15.17:0/2674387755 shutdown_connections 2024-09-10T21:29:26.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:26.963+0000 7fd987000640 1 --2- 172.21.15.17:0/2674387755 >> v2:172.21.15.31:3300/0 conn(0x7fd98006fd00 0x7fd9800c7870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:26.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:26.963+0000 7fd987000640 1 --2- 172.21.15.17:0/2674387755 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd9800b2d80 0x7fd980158310 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:26.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:26.963+0000 7fd987000640 1 -- 172.21.15.17:0/2674387755 >> 172.21.15.17:0/2674387755 conn(0x7fd980070ca0 msgr2=0x7fd9800710b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:26.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:26.963+0000 7fd987000640 1 -- 172.21.15.17:0/2674387755 shutdown_connections 2024-09-10T21:29:26.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:26.963+0000 7fd987000640 1 -- 172.21.15.17:0/2674387755 wait complete. 2024-09-10T21:29:26.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:26.963+0000 7fd987000640 1 Processor -- start 2024-09-10T21:29:26.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:26.963+0000 7fd987000640 1 -- start start 2024-09-10T21:29:26.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:26.967+0000 7fd987000640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd98006fd00 0x7fd9800c3440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:26.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:26.967+0000 7fd987000640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd9800b2d80 0x7fd980154ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:26.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:26.967+0000 7fd987000640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd9801527e0 0x7fd9800c2220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:26.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:26.967+0000 7fd987000640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fd98006f500 con 0x7fd9801527e0 2024-09-10T21:29:26.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:26.967+0000 7fd987000640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fd98006f220 con 0x7fd9800b2d80 2024-09-10T21:29:26.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:26.967+0000 7fd987000640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7fd98006ed40 con 0x7fd98006fd00 2024-09-10T21:29:26.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:26.967+0000 7fd986600640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd9801527e0 0x7fd9800c2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:26.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:26.967+0000 7fd986600640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd9801527e0 0x7fd9800c2220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40588/0 (socket says 172.21.15.17:40588) 2024-09-10T21:29:26.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:26.967+0000 7fd986600640 1 -- 172.21.15.17:0/3774405784 learned_addr learned my addr 172.21.15.17:0/3774405784 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:29:26.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:26.967+0000 7fd985200640 1 --2- 172.21.15.17:0/3774405784 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd9800b2d80 0x7fd980154ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:26.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:26.967+0000 7fd985c00640 1 --2- 172.21.15.17:0/3774405784 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd98006fd00 0x7fd9800c3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:26.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:26.967+0000 7fd986600640 1 -- 172.21.15.17:0/3774405784 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd98006fd00 msgr2=0x7fd9800c3440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:26.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:26.967+0000 7fd986600640 1 --2- 172.21.15.17:0/3774405784 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd98006fd00 0x7fd9800c3440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:26.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:26.967+0000 7fd986600640 1 -- 172.21.15.17:0/3774405784 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd9800b2d80 msgr2=0x7fd980154ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:26.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:26.967+0000 7fd986600640 1 --2- 172.21.15.17:0/3774405784 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd9800b2d80 0x7fd980154ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:26.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:26.967+0000 7fd986600640 1 -- 172.21.15.17:0/3774405784 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9801f44f0 con 0x7fd9801527e0 2024-09-10T21:29:26.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:26.967+0000 7fd985200640 1 --2- 172.21.15.17:0/3774405784 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd9800b2d80 0x7fd980154ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:29:26.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:26.967+0000 7fd986600640 1 --2- 172.21.15.17:0/3774405784 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd9801527e0 0x7fd9800c2220 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd97800eb70 tx=0x7fd97800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:26.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:26.967+0000 7fd97d600640 1 -- 172.21.15.17:0/3774405784 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd978010630 con 0x7fd9801527e0 2024-09-10T21:29:26.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:26.967+0000 7fd97d600640 1 -- 172.21.15.17:0/3774405784 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd9780107d0 con 0x7fd9801527e0 2024-09-10T21:29:26.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:26.967+0000 7fd97d600640 1 -- 172.21.15.17:0/3774405784 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd978010c00 con 0x7fd9801527e0 2024-09-10T21:29:26.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:26.967+0000 7fd987000640 1 -- 172.21.15.17:0/3774405784 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd980154210 con 0x7fd9801527e0 2024-09-10T21:29:26.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:26.967+0000 7fd987000640 1 -- 172.21.15.17:0/3774405784 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd980152f40 con 0x7fd9801527e0 2024-09-10T21:29:26.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:26.967+0000 7fd987000640 1 -- 172.21.15.17:0/3774405784 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd980158310 con 0x7fd9801527e0 2024-09-10T21:29:26.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:26.971+0000 7fd97d600640 1 -- 172.21.15.17:0/3774405784 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fd978021440 con 0x7fd9801527e0 2024-09-10T21:29:26.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:26.971+0000 7fd97d600640 1 --2- 172.21.15.17:0/3774405784 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fd960072610 0x7fd960074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:26.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:26.971+0000 7fd97d600640 1 -- 172.21.15.17:0/3774405784 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fd9780945c0 con 0x7fd9801527e0 2024-09-10T21:29:26.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:26.971+0000 7fd985c00640 1 --2- 172.21.15.17:0/3774405784 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fd960072610 0x7fd960074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:26.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:26.971+0000 7fd985c00640 1 --2- 172.21.15.17:0/3774405784 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fd960072610 0x7fd960074ad0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fd974001650 tx=0x7fd974009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:26.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:26.975+0000 7fd97d600640 1 -- 172.21.15.17:0/3774405784 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fd97805f890 con 0x7fd9801527e0 2024-09-10T21:29:27.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.267+0000 7fd987000640 1 -- 172.21.15.17:0/3774405784 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fd980155d30 con 0x7fd9801527e0 2024-09-10T21:29:27.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.267+0000 7fd97d600640 1 -- 172.21.15.17:0/3774405784 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1557 (secure 0 0 0) 0x7fd97805f190 con 0x7fd9801527e0 2024-09-10T21:29:27.274 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":40,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":16,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5e2f023f-d6a6-49f4-a256-f4807dce3739","modified":"2024-09-10T21:23:46.892130Z","created":"2024-09-10T21:23:46.892130Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3301","nonce":0},{"type":"v1","addr":"172.21.15.31:6790","nonce":0}]},"addr":"172.21.15.31:6790/0","public_addr":"172.21.15.31:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-10T21:29:27.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.267+0000 7fd987000640 1 -- 172.21.15.17:0/3774405784 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fd960072610 msgr2=0x7fd960074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:27.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.267+0000 7fd987000640 1 --2- 172.21.15.17:0/3774405784 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fd960072610 0x7fd960074ad0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fd974001650 tx=0x7fd974009040 comp rx=0 tx=0).stop 2024-09-10T21:29:27.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.267+0000 7fd987000640 1 -- 172.21.15.17:0/3774405784 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd9801527e0 msgr2=0x7fd9800c2220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:27.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.267+0000 7fd987000640 1 --2- 172.21.15.17:0/3774405784 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd9801527e0 0x7fd9800c2220 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd97800eb70 tx=0x7fd97800c6a0 comp rx=0 tx=0).stop 2024-09-10T21:29:27.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.271+0000 7fd987000640 1 -- 172.21.15.17:0/3774405784 shutdown_connections 2024-09-10T21:29:27.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.271+0000 7fd987000640 1 --2- 172.21.15.17:0/3774405784 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fd960072610 0x7fd960074ad0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:27.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.271+0000 7fd987000640 1 --2- 172.21.15.17:0/3774405784 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd9801527e0 0x7fd9800c2220 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:27.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.271+0000 7fd987000640 1 --2- 172.21.15.17:0/3774405784 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd9800b2d80 0x7fd980154ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:27.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.271+0000 7fd987000640 1 --2- 172.21.15.17:0/3774405784 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd98006fd00 0x7fd9800c3440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:27.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.271+0000 7fd987000640 1 -- 172.21.15.17:0/3774405784 >> 172.21.15.17:0/3774405784 conn(0x7fd980070ca0 msgr2=0x7fd980155f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:27.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.271+0000 7fd987000640 1 -- 172.21.15.17:0/3774405784 shutdown_connections 2024-09-10T21:29:27.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.271+0000 7fd987000640 1 -- 172.21.15.17:0/3774405784 wait complete. 2024-09-10T21:29:27.298 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-09-10T21:29:27.298 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-09-10T21:29:27.298 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-09-10T21:29:27.298 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-10T21:29:27.475 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.467+0000 7f114be00640 1 -- 172.21.15.17:0/1501854475 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f114c151250 msgr2=0x7f114c0705c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:27.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.467+0000 7f114be00640 1 --2- 172.21.15.17:0/1501854475 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f114c151250 0x7f114c0705c0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f113c009900 tx=0x7f113c01f0d0 comp rx=0 tx=0).stop 2024-09-10T21:29:27.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.467+0000 7f114be00640 1 -- 172.21.15.17:0/1501854475 shutdown_connections 2024-09-10T21:29:27.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.467+0000 7f114be00640 1 --2- 172.21.15.17:0/1501854475 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f114c068d90 0x7f114c070b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:27.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.467+0000 7f114be00640 1 --2- 172.21.15.17:0/1501854475 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f114c151250 0x7f114c0705c0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:27.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.467+0000 7f114be00640 1 -- 172.21.15.17:0/1501854475 >> 172.21.15.17:0/1501854475 conn(0x7f114c06cc20 msgr2=0x7f114c06d030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:27.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.467+0000 7f114be00640 1 -- 172.21.15.17:0/1501854475 shutdown_connections 2024-09-10T21:29:27.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.467+0000 7f114be00640 1 -- 172.21.15.17:0/1501854475 wait complete. 2024-09-10T21:29:27.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.471+0000 7f114be00640 1 Processor -- start 2024-09-10T21:29:27.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.471+0000 7f114be00640 1 -- start start 2024-09-10T21:29:27.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.471+0000 7f114be00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f114c068d90 0x7f114c1f4380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:27.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.471+0000 7f114be00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f114c072bb0 0x7f114c14eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:27.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.471+0000 7f114be00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f114c151250 0x7f114c14d240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:27.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.471+0000 7f114be00640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f114c0c8840 con 0x7f114c072bb0 2024-09-10T21:29:27.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.471+0000 7f114be00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f114c0c86c0 con 0x7f114c068d90 2024-09-10T21:29:27.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.471+0000 7f114be00640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f114c0c7ff0 con 0x7f114c151250 2024-09-10T21:29:27.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.471+0000 7f114a000640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f114c072bb0 0x7f114c14eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:27.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.471+0000 7f114a000640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f114c072bb0 0x7f114c14eeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40602/0 (socket says 172.21.15.17:40602) 2024-09-10T21:29:27.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.471+0000 7f114a000640 1 -- 172.21.15.17:0/345248659 learned_addr learned my addr 172.21.15.17:0/345248659 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:29:27.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.471+0000 7f114b400640 1 --2- 172.21.15.17:0/345248659 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f114c151250 0x7f114c14d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:27.481 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.471+0000 7f114aa00640 1 --2- 172.21.15.17:0/345248659 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f114c068d90 0x7f114c1f4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:27.481 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.471+0000 7f114a000640 1 -- 172.21.15.17:0/345248659 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f114c151250 msgr2=0x7f114c14d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:27.481 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.471+0000 7f114a000640 1 --2- 172.21.15.17:0/345248659 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f114c151250 0x7f114c14d240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:27.481 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.471+0000 7f114a000640 1 -- 172.21.15.17:0/345248659 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f114c068d90 msgr2=0x7f114c1f4380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:27.481 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.471+0000 7f114a000640 1 --2- 172.21.15.17:0/345248659 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f114c068d90 0x7f114c1f4380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:27.481 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.471+0000 7f114a000640 1 -- 172.21.15.17:0/345248659 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f114c1f3910 con 0x7f114c072bb0 2024-09-10T21:29:27.481 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.471+0000 7f114aa00640 1 --2- 172.21.15.17:0/345248659 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f114c068d90 0x7f114c1f4380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:29:27.481 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.471+0000 7f114a000640 1 --2- 172.21.15.17:0/345248659 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f114c072bb0 0x7f114c14eeb0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f113800d980 tx=0x7f113800de40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:27.482 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.471+0000 7f1143400640 1 -- 172.21.15.17:0/345248659 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1138017070 con 0x7f114c072bb0 2024-09-10T21:29:27.482 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.471+0000 7f1143400640 1 -- 172.21.15.17:0/345248659 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1138009d20 con 0x7f114c072bb0 2024-09-10T21:29:27.482 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.471+0000 7f114be00640 1 -- 172.21.15.17:0/345248659 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f114c1e21f0 con 0x7f114c072bb0 2024-09-10T21:29:27.482 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.471+0000 7f1143400640 1 -- 172.21.15.17:0/345248659 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11380106c0 con 0x7f114c072bb0 2024-09-10T21:29:27.482 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.471+0000 7f114be00640 1 -- 172.21.15.17:0/345248659 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f114c1f3e60 con 0x7f114c072bb0 2024-09-10T21:29:27.484 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.475+0000 7f114be00640 1 --2- 172.21.15.17:0/345248659 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f114c1e03a0 0x7f114c14b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:27.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.475+0000 7f114be00640 1 -- 172.21.15.17:0/345248659 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f114c1f4030 con 0x7f114c1e03a0 2024-09-10T21:29:27.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.475+0000 7f114aa00640 1 --2- 172.21.15.17:0/345248659 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f114c1e03a0 0x7f114c14b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:27.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.475+0000 7f114aa00640 1 -- 172.21.15.17:0/345248659 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f114c1e1bc0 con 0x7f114c1e03a0 2024-09-10T21:29:27.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.475+0000 7f114aa00640 1 --2- 172.21.15.17:0/345248659 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f114c1e03a0 0x7f114c14b820 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f114c06e2d0 tx=0x7f113c005040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:27.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.475+0000 7f1143400640 1 -- 172.21.15.17:0/345248659 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f113c0bf070 con 0x7f114c1e03a0 2024-09-10T21:29:27.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.475+0000 7f1143400640 1 -- 172.21.15.17:0/345248659 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f113c0053b0 con 0x7f114c1e03a0 2024-09-10T21:29:27.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.475+0000 7f1143400640 1 -- 172.21.15.17:0/345248659 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f114c1e03a0 msgr2=0x7f114c14b820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:27.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.475+0000 7f1143400640 1 --2- 172.21.15.17:0/345248659 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f114c1e03a0 0x7f114c14b820 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f114c06e2d0 tx=0x7f113c005040 comp rx=0 tx=0).stop 2024-09-10T21:29:27.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.475+0000 7f1143400640 1 -- 172.21.15.17:0/345248659 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f114c1e03a0 msgr2=0x7f114c14b820 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:29:27.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.475+0000 7f1143400640 1 --2- 172.21.15.17:0/345248659 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f114c1e03a0 0x7f114c14b820 secure :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f114c06e2d0 tx=0x7f113c005040 comp rx=0 tx=0).stop 2024-09-10T21:29:27.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.475+0000 7f1143400640 1 -- 172.21.15.17:0/345248659 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f1138010860 con 0x7f114c072bb0 2024-09-10T21:29:27.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.479+0000 7f1143400640 1 --2- 172.21.15.17:0/345248659 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f112c0720c0 0x7f112c074580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:27.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.479+0000 7f1143400640 1 -- 172.21.15.17:0/345248659 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f11380939f0 con 0x7f114c072bb0 2024-09-10T21:29:27.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.479+0000 7f114aa00640 1 --2- 172.21.15.17:0/345248659 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f112c0720c0 0x7f112c074580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:27.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.479+0000 7f114aa00640 1 --2- 172.21.15.17:0/345248659 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f112c0720c0 0x7f112c074580 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f114c06f850 tx=0x7f113c0b9000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:27.490 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.483+0000 7f114be00640 1 --2- 172.21.15.17:0/345248659 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f114c14bd60 0x7f114c155560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:27.490 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.483+0000 7f114be00640 1 -- 172.21.15.17:0/345248659 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f114c0c37e0 con 0x7f114c14bd60 2024-09-10T21:29:27.490 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.483+0000 7f114b400640 1 --2- 172.21.15.17:0/345248659 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f114c14bd60 0x7f114c155560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:27.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.483+0000 7f114b400640 1 -- 172.21.15.17:0/345248659 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f114c0c84a0 con 0x7f114c14bd60 2024-09-10T21:29:27.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.483+0000 7f114b400640 1 --2- 172.21.15.17:0/345248659 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f114c14bd60 0x7f114c155560 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f11440012f0 tx=0x7f114400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:27.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.483+0000 7f1143400640 1 -- 172.21.15.17:0/345248659 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1144003b90 con 0x7f114c14bd60 2024-09-10T21:29:27.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.483+0000 7f1143400640 1 -- 172.21.15.17:0/345248659 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4877 (secure 0 0 0) 0x7f1144003b90 con 0x7f114c14bd60 2024-09-10T21:29:27.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.483+0000 7f1143400640 1 -- 172.21.15.17:0/345248659 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f114c14bd60 msgr2=0x7f114c155560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:27.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.483+0000 7f1143400640 1 --2- 172.21.15.17:0/345248659 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f114c14bd60 0x7f114c155560 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f11440012f0 tx=0x7f114400ecd0 comp rx=0 tx=0).stop 2024-09-10T21:29:27.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.483+0000 7f1143400640 1 -- 172.21.15.17:0/345248659 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f114c14bd60 msgr2=0x7f114c155560 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:29:27.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.483+0000 7f1143400640 1 --2- 172.21.15.17:0/345248659 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f114c14bd60 0x7f114c155560 secure :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f11440012f0 tx=0x7f114400ecd0 comp rx=0 tx=0).stop 2024-09-10T21:29:27.492 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-10T21:29:27.492 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:29:27.493 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:29:27.493 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "leader", 2024-09-10T21:29:27.493 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 40, 2024-09-10T21:29:27.493 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 85323, 2024-09-10T21:29:27.493 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-10T21:29:27.493 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-10T21:29:27.496 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-10T21:29:27.496 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-10T21:29:27.496 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:29:27.496 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 16, 2024-09-10T21:29:27.496 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:29:27.497 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-10T21:29:27.497 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-10T21:29:27.497 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:29:27.497 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:29:27.497 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:29:27.497 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:29:27.497 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:29:27.497 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:29:27.497 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:29:27.497 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:29:27.497 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:29:27.498 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:29:27.498 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:29:27.498 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:29:27.498 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-10T21:29:27.498 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-10T21:29:27.498 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:29:27.498 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:29:27.498 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:29:27.498 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:29:27.498 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:29:27.498 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:29:27.499 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:29:27.499 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:29:27.499 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:29:27.499 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:29:27.499 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:29:27.499 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:29:27.499 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:29:27.499 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-10T21:29:27.499 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-10T21:29:27.499 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-10T21:29:27.499 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-10T21:29:27.499 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-10T21:29:27.500 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "5e2f023f-d6a6-49f4-a256-f4807dce3739", 2024-09-10T21:29:27.500 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:29:27.500 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:29:27.500 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-10T21:29:27.500 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-10T21:29:27.500 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-10T21:29:27.500 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-10T21:29:27.500 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-10T21:29:27.500 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-10T21:29:27.500 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-10T21:29:27.500 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:29:27.500 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-10T21:29:27.501 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:29:27.501 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:29:27.501 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:29:27.501 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:29:27.501 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:29:27.501 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:29:27.501 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:29:27.501 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:29:27.501 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:29:27.501 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:29:27.501 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:29:27.501 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:29:27.502 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-10T21:29:27.502 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:29:27.502 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-10T21:29:27.502 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:29:27.502 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:29:27.502 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:29:27.502 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:29:27.502 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:29:27.502 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:29:27.502 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:29:27.502 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-10T21:29:27.503 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:29:27.503 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:29:27.503 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:29:27.503 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:29:27.503 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-10T21:29:27.503 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:29:27.503 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:29:27.503 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:29:27.503 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:29:27.503 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-10T21:29:27.503 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-10T21:29:27.504 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:29:27.504 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:29:27.504 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:29:27.504 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:29:27.504 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:29:27.504 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:29:27.504 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:29:27.504 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:29:27.504 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:29:27.504 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:29:27.504 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:29:27.505 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3300", 2024-09-10T21:29:27.505 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:29:27.505 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:29:27.505 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:29:27.505 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:29:27.505 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789", 2024-09-10T21:29:27.505 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:29:27.505 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:29:27.505 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:29:27.505 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:29:27.505 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789/0", 2024-09-10T21:29:27.506 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6789/0", 2024-09-10T21:29:27.506 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:29:27.506 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:29:27.506 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:29:27.506 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:29:27.506 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:29:27.506 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:29:27.506 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:29:27.506 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:29:27.506 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:29:27.506 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:29:27.507 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:29:27.507 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3301", 2024-09-10T21:29:27.507 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:29:27.507 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:29:27.507 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:29:27.507 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:29:27.507 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790", 2024-09-10T21:29:27.507 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:29:27.507 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:29:27.507 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:29:27.507 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:29:27.508 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790/0", 2024-09-10T21:29:27.508 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6790/0", 2024-09-10T21:29:27.508 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:29:27.508 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:29:27.508 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:29:27.508 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:29:27.508 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:29:27.509 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:29:27.509 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-10T21:29:27.509 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-10T21:29:27.509 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:29:27.509 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:29:27.509 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:29:27.509 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:29:27.509 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:29:27.509 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:29:27.509 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": [ 2024-09-10T21:29:27.509 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:29:27.509 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:29:27.510 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:29:27.510 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:29:27.510 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:29:27.510 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:29:27.510 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-10T21:29:27.510 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:29:27.510 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:29:27.511 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:29:27.511 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:29:27.511 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:29:27.511 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:29:27.511 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-10T21:29:27.511 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:29:27.511 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:29:27.511 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:29:27.511 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 4 2024-09-10T21:29:27.511 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:29:27.511 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:29:27.511 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:29:27.512 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-10T21:29:27.512 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-10T21:29:27.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.487+0000 7f114be00640 1 -- 172.21.15.17:0/345248659 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f112c0720c0 msgr2=0x7f112c074580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:27.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.487+0000 7f114be00640 1 --2- 172.21.15.17:0/345248659 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f112c0720c0 0x7f112c074580 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f114c06f850 tx=0x7f113c0b9000 comp rx=0 tx=0).stop 2024-09-10T21:29:27.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.487+0000 7f114be00640 1 -- 172.21.15.17:0/345248659 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f114c072bb0 msgr2=0x7f114c14eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:27.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.487+0000 7f114be00640 1 --2- 172.21.15.17:0/345248659 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f114c072bb0 0x7f114c14eeb0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f113800d980 tx=0x7f113800de40 comp rx=0 tx=0).stop 2024-09-10T21:29:27.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.487+0000 7f114b400640 1 -- 172.21.15.17:0/345248659 reap_dead start 2024-09-10T21:29:27.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.487+0000 7f114be00640 1 -- 172.21.15.17:0/345248659 shutdown_connections 2024-09-10T21:29:27.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.487+0000 7f114be00640 1 -- 172.21.15.17:0/345248659 >> 172.21.15.17:0/345248659 conn(0x7f114c06cc20 msgr2=0x7f114c14de30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:27.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.487+0000 7f114be00640 1 -- 172.21.15.17:0/345248659 shutdown_connections 2024-09-10T21:29:27.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.487+0000 7f114be00640 1 -- 172.21.15.17:0/345248659 wait complete. 2024-09-10T21:29:27.515 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-10T21:29:27.515 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-09-10T21:29:27.515 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-10T21:29:27.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.663+0000 7f43e1c00640 1 -- 172.21.15.17:0/1362920493 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f43dc0cdf20 msgr2=0x7f43dc0ce350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:27.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.663+0000 7f43e1c00640 1 --2- 172.21.15.17:0/1362920493 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f43dc0cdf20 0x7f43dc0ce350 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f43dc073220 tx=0x7f43d401ccf0 comp rx=0 tx=0).stop 2024-09-10T21:29:27.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.663+0000 7f43e1c00640 1 -- 172.21.15.17:0/1362920493 shutdown_connections 2024-09-10T21:29:27.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.663+0000 7f43e1c00640 1 --2- 172.21.15.17:0/1362920493 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f43dc0cdf20 0x7f43dc0ce350 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:27.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.663+0000 7f43e1c00640 1 --2- 172.21.15.17:0/1362920493 >> v2:172.21.15.31:3300/0 conn(0x7f43dc152440 0x7f43dc0b3eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:27.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.663+0000 7f43e1c00640 1 -- 172.21.15.17:0/1362920493 >> 172.21.15.17:0/1362920493 conn(0x7f43dc0702f0 msgr2=0x7f43dc070700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:27.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.663+0000 7f43e1c00640 1 -- 172.21.15.17:0/1362920493 shutdown_connections 2024-09-10T21:29:27.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.663+0000 7f43e1c00640 1 -- 172.21.15.17:0/1362920493 wait complete. 2024-09-10T21:29:27.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.663+0000 7f43e1c00640 1 Processor -- start 2024-09-10T21:29:27.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.663+0000 7f43e1c00640 1 -- start start 2024-09-10T21:29:27.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.667+0000 7f43e1c00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f43dc0cdf20 0x7f43dc14c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:27.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.667+0000 7f43e1c00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f43dc152440 0x7f43dc162100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:27.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.667+0000 7f43e1c00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f43dc152820 0x7f43dc1f2e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:27.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.667+0000 7f43e1c00640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f43dc0cb5d0 con 0x7f43dc152820 2024-09-10T21:29:27.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.667+0000 7f43e1c00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f43dc0cb450 con 0x7f43dc0cdf20 2024-09-10T21:29:27.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.667+0000 7f43e1c00640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f43dc0c2ff0 con 0x7f43dc152440 2024-09-10T21:29:27.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.667+0000 7f43daa00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f43dc0cdf20 0x7f43dc14c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:27.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.667+0000 7f43daa00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f43dc0cdf20 0x7f43dc14c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.17:59248/0 (socket says 172.21.15.17:59248) 2024-09-10T21:29:27.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.667+0000 7f43daa00640 1 -- 172.21.15.17:0/3663277683 learned_addr learned my addr 172.21.15.17:0/3663277683 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:29:27.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.667+0000 7f43db400640 1 --2- 172.21.15.17:0/3663277683 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f43dc152820 0x7f43dc1f2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:27.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.667+0000 7f43da000640 1 --2- 172.21.15.17:0/3663277683 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f43dc152440 0x7f43dc162100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:27.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.667+0000 7f43db400640 1 -- 172.21.15.17:0/3663277683 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f43dc152440 msgr2=0x7f43dc162100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:27.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.667+0000 7f43db400640 1 --2- 172.21.15.17:0/3663277683 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f43dc152440 0x7f43dc162100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:27.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.667+0000 7f43db400640 1 -- 172.21.15.17:0/3663277683 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f43dc0cdf20 msgr2=0x7f43dc14c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:27.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.667+0000 7f43db400640 1 --2- 172.21.15.17:0/3663277683 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f43dc0cdf20 0x7f43dc14c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:27.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.667+0000 7f43db400640 1 -- 172.21.15.17:0/3663277683 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43dc15b200 con 0x7f43dc152820 2024-09-10T21:29:27.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.667+0000 7f43daa00640 1 --2- 172.21.15.17:0/3663277683 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f43dc0cdf20 0x7f43dc14c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:29:27.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.667+0000 7f43db400640 1 --2- 172.21.15.17:0/3663277683 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f43dc152820 0x7f43dc1f2e40 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f43d40ac000 tx=0x7f43d4005aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:27.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.667+0000 7f43d2a00640 1 -- 172.21.15.17:0/3663277683 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43d40b2040 con 0x7f43dc152820 2024-09-10T21:29:27.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.667+0000 7f43d2a00640 1 -- 172.21.15.17:0/3663277683 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f43d4004b40 con 0x7f43dc152820 2024-09-10T21:29:27.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.667+0000 7f43e1c00640 1 -- 172.21.15.17:0/3663277683 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43dc15b4e0 con 0x7f43dc152820 2024-09-10T21:29:27.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.667+0000 7f43d2a00640 1 -- 172.21.15.17:0/3663277683 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43d4003fe0 con 0x7f43dc152820 2024-09-10T21:29:27.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.667+0000 7f43e1c00640 1 -- 172.21.15.17:0/3663277683 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43dc06f5c0 con 0x7f43dc152820 2024-09-10T21:29:27.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.667+0000 7f43e1c00640 1 --2- 172.21.15.17:0/3663277683 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f43dc06f800 0x7f43dc1f3380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:27.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.667+0000 7f43e1c00640 1 -- 172.21.15.17:0/3663277683 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f43dc06f140 con 0x7f43dc06f800 2024-09-10T21:29:27.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.667+0000 7f43daa00640 1 --2- 172.21.15.17:0/3663277683 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f43dc06f800 0x7f43dc1f3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:27.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.667+0000 7f43daa00640 1 -- 172.21.15.17:0/3663277683 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f43dc158360 con 0x7f43dc06f800 2024-09-10T21:29:27.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.667+0000 7f43daa00640 1 --2- 172.21.15.17:0/3663277683 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f43dc06f800 0x7f43dc1f3380 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f43cc009f40 tx=0x7f43cc00f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:27.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.667+0000 7f43d2a00640 1 -- 172.21.15.17:0/3663277683 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43cc01e070 con 0x7f43dc06f800 2024-09-10T21:29:27.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.671+0000 7f43d2a00640 1 -- 172.21.15.17:0/3663277683 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f43d40b5070 con 0x7f43dc152820 2024-09-10T21:29:27.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.671+0000 7f43d2a00640 1 --2- 172.21.15.17:0/3663277683 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f43bc072510 0x7f43bc0749d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:27.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.671+0000 7f43d2a00640 1 -- 172.21.15.17:0/3663277683 <== mon.1 v2:172.21.15.31:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f43cc004030 con 0x7f43dc06f800 2024-09-10T21:29:27.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.671+0000 7f43d2a00640 1 -- 172.21.15.17:0/3663277683 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f43dc06f800 msgr2=0x7f43dc1f3380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:27.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.671+0000 7f43d2a00640 1 --2- 172.21.15.17:0/3663277683 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f43dc06f800 0x7f43dc1f3380 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f43cc009f40 tx=0x7f43cc00f790 comp rx=0 tx=0).stop 2024-09-10T21:29:27.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.671+0000 7f43d2a00640 1 -- 172.21.15.17:0/3663277683 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f43dc06f800 msgr2=0x7f43dc1f3380 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:29:27.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.671+0000 7f43d2a00640 1 --2- 172.21.15.17:0/3663277683 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f43dc06f800 0x7f43dc1f3380 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:27.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.671+0000 7f43d2a00640 1 -- 172.21.15.17:0/3663277683 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f43d412cf30 con 0x7f43dc152820 2024-09-10T21:29:27.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.671+0000 7f43da000640 1 --2- 172.21.15.17:0/3663277683 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f43bc072510 0x7f43bc0749d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:27.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.671+0000 7f43da000640 1 --2- 172.21.15.17:0/3663277683 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f43bc072510 0x7f43bc0749d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f43dc1f0a70 tx=0x7f43c8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:27.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.675+0000 7f43e1c00640 1 --2- 172.21.15.17:0/3663277683 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f43dc14b7b0 0x7f43dc14bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:27.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.675+0000 7f43e1c00640 1 -- 172.21.15.17:0/3663277683 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f43dc14cb70 con 0x7f43dc14b7b0 2024-09-10T21:29:27.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.675+0000 7f43daa00640 1 --2- 172.21.15.17:0/3663277683 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f43dc14b7b0 0x7f43dc14bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:27.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.679+0000 7f43daa00640 1 -- 172.21.15.17:0/3663277683 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f43dc14f5e0 con 0x7f43dc14b7b0 2024-09-10T21:29:27.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.679+0000 7f43daa00640 1 --2- 172.21.15.17:0/3663277683 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f43dc14b7b0 0x7f43dc14bb90 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f43cc01b000 tx=0x7f43cc0046d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:27.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.679+0000 7f43d2a00640 1 -- 172.21.15.17:0/3663277683 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43cc003800 con 0x7f43dc14b7b0 2024-09-10T21:29:27.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.679+0000 7f43d2a00640 1 -- 172.21.15.17:0/3663277683 <== mon.1 v2:172.21.15.31:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5043 (secure 0 0 0) 0x7f43cc003800 con 0x7f43dc14b7b0 2024-09-10T21:29:27.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.679+0000 7f43d2a00640 1 -- 172.21.15.17:0/3663277683 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f43dc14b7b0 msgr2=0x7f43dc14bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:27.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.679+0000 7f43d2a00640 1 --2- 172.21.15.17:0/3663277683 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f43dc14b7b0 0x7f43dc14bb90 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f43cc01b000 tx=0x7f43cc0046d0 comp rx=0 tx=0).stop 2024-09-10T21:29:27.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.679+0000 7f43d2a00640 1 -- 172.21.15.17:0/3663277683 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f43dc14b7b0 msgr2=0x7f43dc14bb90 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:29:27.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.679+0000 7f43d2a00640 1 --2- 172.21.15.17:0/3663277683 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f43dc14b7b0 0x7f43dc14bb90 secure :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f43cc01b000 tx=0x7f43cc0046d0 comp rx=0 tx=0).stop 2024-09-10T21:29:27.686 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-10T21:29:27.686 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:29:27.686 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:29:27.686 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-10T21:29:27.686 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 40, 2024-09-10T21:29:27.686 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 306126, 2024-09-10T21:29:27.686 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-10T21:29:27.686 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-10T21:29:27.686 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-10T21:29:27.687 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-10T21:29:27.687 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:29:27.687 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 16, 2024-09-10T21:29:27.687 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:29:27.687 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-10T21:29:27.687 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-10T21:29:27.687 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:29:27.687 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:29:27.687 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:29:27.687 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:29:27.687 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:29:27.687 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:29:27.687 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:29:27.688 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:29:27.688 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:29:27.688 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:29:27.688 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:29:27.688 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:29:27.688 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-10T21:29:27.688 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-10T21:29:27.688 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:29:27.688 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:29:27.688 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:29:27.688 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:29:27.688 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:29:27.688 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:29:27.689 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:29:27.689 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:29:27.689 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:29:27.689 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:29:27.689 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:29:27.689 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:29:27.689 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:29:27.689 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-10T21:29:27.689 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-10T21:29:27.689 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-10T21:29:27.689 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-10T21:29:27.689 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-10T21:29:27.689 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "5e2f023f-d6a6-49f4-a256-f4807dce3739", 2024-09-10T21:29:27.690 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:29:27.690 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:29:27.690 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-10T21:29:27.690 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-10T21:29:27.690 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-10T21:29:27.690 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-10T21:29:27.690 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-10T21:29:27.690 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-10T21:29:27.690 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-10T21:29:27.690 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:29:27.690 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-10T21:29:27.690 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:29:27.690 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:29:27.690 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:29:27.691 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:29:27.691 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:29:27.691 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:29:27.691 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:29:27.691 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:29:27.691 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:29:27.691 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:29:27.691 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:29:27.691 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:29:27.691 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-10T21:29:27.691 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:29:27.691 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-10T21:29:27.691 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:29:27.692 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:29:27.692 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:29:27.692 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:29:27.692 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:29:27.692 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:29:27.692 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:29:27.692 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-10T21:29:27.692 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:29:27.692 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:29:27.692 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:29:27.692 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:29:27.692 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-10T21:29:27.692 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:29:27.692 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:29:27.693 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:29:27.693 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:29:27.693 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-10T21:29:27.693 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-10T21:29:27.693 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:29:27.693 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:29:27.693 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:29:27.693 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:29:27.693 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:29:27.693 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:29:27.693 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:29:27.693 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:29:27.693 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:29:27.693 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:29:27.694 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:29:27.694 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3300", 2024-09-10T21:29:27.694 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:29:27.694 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:29:27.694 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:29:27.694 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:29:27.694 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789", 2024-09-10T21:29:27.694 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:29:27.694 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:29:27.694 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:29:27.694 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:29:27.694 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789/0", 2024-09-10T21:29:27.694 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6789/0", 2024-09-10T21:29:27.694 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:29:27.695 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:29:27.695 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:29:27.695 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:29:27.695 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:29:27.695 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:29:27.695 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:29:27.695 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:29:27.695 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:29:27.695 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:29:27.695 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:29:27.695 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3301", 2024-09-10T21:29:27.695 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:29:27.695 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:29:27.695 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:29:27.695 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:29:27.696 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790", 2024-09-10T21:29:27.696 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:29:27.696 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:29:27.696 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:29:27.696 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:29:27.696 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790/0", 2024-09-10T21:29:27.696 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6790/0", 2024-09-10T21:29:27.696 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:29:27.696 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:29:27.696 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:29:27.696 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:29:27.696 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:29:27.696 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:29:27.697 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-10T21:29:27.697 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-10T21:29:27.697 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:29:27.697 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:29:27.697 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:29:27.697 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:29:27.697 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:29:27.697 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:29:27.697 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": [ 2024-09-10T21:29:27.697 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:29:27.697 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:29:27.697 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:29:27.697 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 5 2024-09-10T21:29:27.697 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:29:27.698 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:29:27.698 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-10T21:29:27.698 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:29:27.698 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:29:27.698 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:29:27.698 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 7 2024-09-10T21:29:27.698 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:29:27.698 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:29:27.698 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-10T21:29:27.698 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:29:27.698 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:29:27.698 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:29:27.698 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 4 2024-09-10T21:29:27.699 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:29:27.699 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:29:27.699 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": [ 2024-09-10T21:29:27.699 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:29:27.699 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:29:27.699 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:29:27.699 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 2 2024-09-10T21:29:27.699 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:29:27.699 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:29:27.699 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:29:27.699 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-10T21:29:27.699 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-10T21:29:27.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.679+0000 7f43e1c00640 1 -- 172.21.15.17:0/3663277683 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f43bc072510 msgr2=0x7f43bc0749d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:27.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.679+0000 7f43e1c00640 1 --2- 172.21.15.17:0/3663277683 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f43bc072510 0x7f43bc0749d0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f43dc1f0a70 tx=0x7f43c8009040 comp rx=0 tx=0).stop 2024-09-10T21:29:27.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.679+0000 7f43e1c00640 1 -- 172.21.15.17:0/3663277683 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f43dc152820 msgr2=0x7f43dc1f2e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:27.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.679+0000 7f43e1c00640 1 --2- 172.21.15.17:0/3663277683 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f43dc152820 0x7f43dc1f2e40 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f43d40ac000 tx=0x7f43d4005aa0 comp rx=0 tx=0).stop 2024-09-10T21:29:27.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.679+0000 7f43db400640 1 -- 172.21.15.17:0/3663277683 reap_dead start 2024-09-10T21:29:27.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.679+0000 7f43db400640 1 -- 172.21.15.17:0/3663277683 reap_dead start 2024-09-10T21:29:27.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.679+0000 7f43e1c00640 1 -- 172.21.15.17:0/3663277683 shutdown_connections 2024-09-10T21:29:27.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.679+0000 7f43e1c00640 1 -- 172.21.15.17:0/3663277683 >> 172.21.15.17:0/3663277683 conn(0x7f43dc0702f0 msgr2=0x7f43dc156e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:27.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.679+0000 7f43e1c00640 1 -- 172.21.15.17:0/3663277683 shutdown_connections 2024-09-10T21:29:27.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.679+0000 7f43e1c00640 1 -- 172.21.15.17:0/3663277683 wait complete. 2024-09-10T21:29:27.706 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-10T21:29:27.706 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-09-10T21:29:27.706 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-10T21:29:27.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.867+0000 7f08a1400640 1 -- 172.21.15.17:0/3037574470 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f089c14f260 msgr2=0x7f089c06eae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:27.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.867+0000 7f08a1400640 1 --2- 172.21.15.17:0/3037574470 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f089c14f260 0x7f089c06eae0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f088c009900 tx=0x7f088c01c8e0 comp rx=0 tx=0).stop 2024-09-10T21:29:27.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.867+0000 7f08a1400640 1 -- 172.21.15.17:0/3037574470 shutdown_connections 2024-09-10T21:29:27.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.867+0000 7f08a1400640 1 --2- 172.21.15.17:0/3037574470 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f089c14f260 0x7f089c06eae0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:27.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.867+0000 7f08a1400640 1 -- 172.21.15.17:0/3037574470 >> 172.21.15.17:0/3037574470 conn(0x7f089c0714a0 msgr2=0x7f089c0718b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:27.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.867+0000 7f08a1400640 1 -- 172.21.15.17:0/3037574470 shutdown_connections 2024-09-10T21:29:27.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.867+0000 7f08a1400640 1 -- 172.21.15.17:0/3037574470 wait complete. 2024-09-10T21:29:27.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.867+0000 7f08a1400640 1 Processor -- start 2024-09-10T21:29:27.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.867+0000 7f08a1400640 1 -- start start 2024-09-10T21:29:27.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.867+0000 7f08a1400640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f089c0caf00 0x7f089c1f4aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:27.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.867+0000 7f08a1400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f089c0cb350 0x7f089c0c4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:27.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.867+0000 7f08a1400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f089c14f260 0x7f089c150190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:27.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.867+0000 7f08a1400640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f089c166950 con 0x7f089c14f260 2024-09-10T21:29:27.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.867+0000 7f08a1400640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f089c1667d0 con 0x7f089c0cb350 2024-09-10T21:29:27.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.867+0000 7f08a1400640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f089c166ad0 con 0x7f089c0caf00 2024-09-10T21:29:27.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.871+0000 7f089b200640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f089c14f260 0x7f089c150190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:27.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.871+0000 7f089a800640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f089c0caf00 0x7f089c1f4aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:27.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.871+0000 7f0899e00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f089c0cb350 0x7f089c0c4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:27.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.871+0000 7f089a800640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f089c0caf00 0x7f089c1f4aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3301/0 says I am v2:172.21.15.17:50206/0 (socket says 172.21.15.17:50206) 2024-09-10T21:29:27.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.871+0000 7f089a800640 1 -- 172.21.15.17:0/3879472252 learned_addr learned my addr 172.21.15.17:0/3879472252 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:29:27.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.871+0000 7f089a800640 1 -- 172.21.15.17:0/3879472252 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f089c0cb350 msgr2=0x7f089c0c4e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:27.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.871+0000 7f089a800640 1 --2- 172.21.15.17:0/3879472252 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f089c0cb350 0x7f089c0c4e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:27.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.871+0000 7f089a800640 1 -- 172.21.15.17:0/3879472252 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f089c14f260 msgr2=0x7f089c150190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:27.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.871+0000 7f089a800640 1 --2- 172.21.15.17:0/3879472252 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f089c14f260 0x7f089c150190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:27.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.871+0000 7f089a800640 1 -- 172.21.15.17:0/3879472252 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f089c14fdb0 con 0x7f089c0caf00 2024-09-10T21:29:27.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.871+0000 7f089b200640 1 --2- 172.21.15.17:0/3879472252 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f089c14f260 0x7f089c150190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:29:27.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.871+0000 7f089a800640 1 --2- 172.21.15.17:0/3879472252 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f089c0caf00 0x7f089c1f4aa0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f088c009850 tx=0x7f088c004040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:27.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.871+0000 7f0892a00640 1 -- 172.21.15.17:0/3879472252 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f088c0b2040 con 0x7f089c0caf00 2024-09-10T21:29:27.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.871+0000 7f0892a00640 1 -- 172.21.15.17:0/3879472252 <== mon.2 v2:172.21.15.31:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f088c004430 con 0x7f089c0caf00 2024-09-10T21:29:27.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.871+0000 7f0892a00640 1 -- 172.21.15.17:0/3879472252 <== mon.2 v2:172.21.15.31:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f088c003860 con 0x7f089c0caf00 2024-09-10T21:29:27.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.871+0000 7f08a1400640 1 -- 172.21.15.17:0/3879472252 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f089c153c40 con 0x7f089c0caf00 2024-09-10T21:29:27.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.871+0000 7f08a1400640 1 -- 172.21.15.17:0/3879472252 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f089c153160 con 0x7f089c0caf00 2024-09-10T21:29:27.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.871+0000 7f08a1400640 1 --2- 172.21.15.17:0/3879472252 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f089c1506d0 0x7f089c148950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:27.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.871+0000 7f08a1400640 1 -- 172.21.15.17:0/3879472252 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f089c1f4fe0 con 0x7f089c1506d0 2024-09-10T21:29:27.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.871+0000 7f0899e00640 1 --2- 172.21.15.17:0/3879472252 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f089c1506d0 0x7f089c148950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:27.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.871+0000 7f0899e00640 1 -- 172.21.15.17:0/3879472252 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f089c06eae0 con 0x7f089c1506d0 2024-09-10T21:29:27.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.871+0000 7f0899e00640 1 --2- 172.21.15.17:0/3879472252 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f089c1506d0 0x7f089c148950 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f089c0ccaf0 tx=0x7f088800fc00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:27.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.871+0000 7f0892a00640 1 -- 172.21.15.17:0/3879472252 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08880050d0 con 0x7f089c1506d0 2024-09-10T21:29:27.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.871+0000 7f0892a00640 1 -- 172.21.15.17:0/3879472252 <== mon.2 v2:172.21.15.31:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f0888018070 con 0x7f089c1506d0 2024-09-10T21:29:27.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.871+0000 7f0892a00640 1 -- 172.21.15.17:0/3879472252 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f089c1506d0 msgr2=0x7f089c148950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:27.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.871+0000 7f0892a00640 1 --2- 172.21.15.17:0/3879472252 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f089c1506d0 0x7f089c148950 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f089c0ccaf0 tx=0x7f088800fc00 comp rx=0 tx=0).stop 2024-09-10T21:29:27.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.875+0000 7f0892a00640 1 -- 172.21.15.17:0/3879472252 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f089c1506d0 msgr2=0x7f089c148950 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:29:27.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.875+0000 7f0892a00640 1 --2- 172.21.15.17:0/3879472252 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f089c1506d0 0x7f089c148950 secure :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f089c0ccaf0 tx=0x7f088800fc00 comp rx=0 tx=0).stop 2024-09-10T21:29:27.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.875+0000 7f0892a00640 1 -- 172.21.15.17:0/3879472252 <== mon.2 v2:172.21.15.31:3301/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f088c003a00 con 0x7f089c0caf00 2024-09-10T21:29:27.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.875+0000 7f0892a00640 1 --2- 172.21.15.17:0/3879472252 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f087c072210 0x7f087c0746d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:27.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.875+0000 7f0892a00640 1 -- 172.21.15.17:0/3879472252 <== mon.2 v2:172.21.15.31:3301/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f088c12d230 con 0x7f089c0caf00 2024-09-10T21:29:27.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.875+0000 7f0899e00640 1 --2- 172.21.15.17:0/3879472252 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f087c072210 0x7f087c0746d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:27.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.875+0000 7f0899e00640 1 --2- 172.21.15.17:0/3879472252 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f087c072210 0x7f087c0746d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f0888017000 tx=0x7f0888005000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:27.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.879+0000 7f08a1400640 1 --2- 172.21.15.17:0/3879472252 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f089c148e90 0x7f089c1f7590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:27.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.879+0000 7f08a1400640 1 -- 172.21.15.17:0/3879472252 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f089c0c71c0 con 0x7f089c148e90 2024-09-10T21:29:27.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.879+0000 7f089b200640 1 --2- 172.21.15.17:0/3879472252 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f089c148e90 0x7f089c1f7590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:27.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.879+0000 7f089b200640 1 -- 172.21.15.17:0/3879472252 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f089c06e300 con 0x7f089c148e90 2024-09-10T21:29:27.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.879+0000 7f089b200640 1 --2- 172.21.15.17:0/3879472252 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f089c148e90 0x7f089c1f7590 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f0894001ee0 tx=0x7f089400d520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:27.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.883+0000 7f0892a00640 1 -- 172.21.15.17:0/3879472252 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f089400de10 con 0x7f089c148e90 2024-09-10T21:29:27.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.883+0000 7f0892a00640 1 -- 172.21.15.17:0/3879472252 <== mon.2 v2:172.21.15.31:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4541 (secure 0 0 0) 0x7f089400de10 con 0x7f089c148e90 2024-09-10T21:29:27.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.883+0000 7f0892a00640 1 -- 172.21.15.17:0/3879472252 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f089c148e90 msgr2=0x7f089c1f7590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:27.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.883+0000 7f0892a00640 1 --2- 172.21.15.17:0/3879472252 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f089c148e90 0x7f089c1f7590 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f0894001ee0 tx=0x7f089400d520 comp rx=0 tx=0).stop 2024-09-10T21:29:27.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.883+0000 7f0892a00640 1 -- 172.21.15.17:0/3879472252 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f089c148e90 msgr2=0x7f089c1f7590 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:29:27.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.883+0000 7f0892a00640 1 --2- 172.21.15.17:0/3879472252 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f089c148e90 0x7f089c1f7590 secure :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f0894001ee0 tx=0x7f089400d520 comp rx=0 tx=0).stop 2024-09-10T21:29:27.889 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-10T21:29:27.889 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:29:27.889 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:29:27.889 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-10T21:29:27.889 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 40, 2024-09-10T21:29:27.889 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 22435, 2024-09-10T21:29:27.889 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-10T21:29:27.889 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-10T21:29:27.890 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-10T21:29:27.890 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-10T21:29:27.890 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:29:27.890 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 16, 2024-09-10T21:29:27.890 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:29:27.890 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-10T21:29:27.890 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-10T21:29:27.890 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:29:27.890 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:29:27.890 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:29:27.890 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:29:27.890 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:29:27.890 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:29:27.890 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:29:27.891 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:29:27.891 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:29:27.891 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:29:27.891 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:29:27.891 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:29:27.891 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-10T21:29:27.891 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-10T21:29:27.891 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:29:27.891 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:29:27.891 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:29:27.891 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:29:27.891 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:29:27.891 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:29:27.891 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:29:27.892 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:29:27.892 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:29:27.892 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:29:27.892 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:29:27.892 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:29:27.892 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:29:27.892 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-10T21:29:27.892 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-10T21:29:27.892 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-10T21:29:27.892 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-10T21:29:27.892 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-10T21:29:27.892 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "5e2f023f-d6a6-49f4-a256-f4807dce3739", 2024-09-10T21:29:27.892 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:29:27.892 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:29:27.893 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-10T21:29:27.893 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-10T21:29:27.893 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-10T21:29:27.893 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-10T21:29:27.893 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-10T21:29:27.893 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-10T21:29:27.893 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-10T21:29:27.893 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:29:27.893 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-10T21:29:27.893 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:29:27.893 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:29:27.893 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:29:27.893 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:29:27.894 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:29:27.894 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:29:27.894 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:29:27.894 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:29:27.894 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:29:27.894 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:29:27.894 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:29:27.894 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:29:27.894 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-10T21:29:27.894 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:29:27.894 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-10T21:29:27.894 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:29:27.894 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:29:27.894 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:29:27.895 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:29:27.895 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:29:27.895 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:29:27.895 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:29:27.895 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-10T21:29:27.895 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:29:27.895 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:29:27.895 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:29:27.895 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:29:27.895 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-10T21:29:27.895 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:29:27.895 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:29:27.895 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:29:27.895 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:29:27.896 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-10T21:29:27.896 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-10T21:29:27.896 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:29:27.896 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:29:27.896 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:29:27.896 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:29:27.896 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:29:27.896 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:29:27.896 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:29:27.896 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:29:27.896 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:29:27.896 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:29:27.896 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:29:27.897 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3300", 2024-09-10T21:29:27.897 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:29:27.897 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:29:27.897 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:29:27.897 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:29:27.897 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789", 2024-09-10T21:29:27.897 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:29:27.897 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:29:27.897 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:29:27.897 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:29:27.897 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789/0", 2024-09-10T21:29:27.897 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6789/0", 2024-09-10T21:29:27.897 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:29:27.898 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:29:27.898 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:29:27.898 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:29:27.898 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:29:27.898 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:29:27.898 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:29:27.898 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:29:27.898 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:29:27.898 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:29:27.898 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:29:27.898 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3301", 2024-09-10T21:29:27.898 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:29:27.898 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:29:27.899 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:29:27.899 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:29:27.899 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790", 2024-09-10T21:29:27.899 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:29:27.899 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:29:27.899 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:29:27.899 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:29:27.899 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790/0", 2024-09-10T21:29:27.899 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6790/0", 2024-09-10T21:29:27.899 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:29:27.899 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:29:27.899 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:29:27.899 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:29:27.899 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:29:27.900 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:29:27.900 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-10T21:29:27.900 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-10T21:29:27.900 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:29:27.900 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:29:27.900 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:29:27.900 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:29:27.900 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:29:27.900 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:29:27.900 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-10T21:29:27.900 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:29:27.900 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:29:27.900 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:29:27.900 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 2 2024-09-10T21:29:27.901 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:29:27.901 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:29:27.901 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:29:27.901 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-10T21:29:27.901 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-10T21:29:27.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.883+0000 7f08a1400640 1 -- 172.21.15.17:0/3879472252 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f087c072210 msgr2=0x7f087c0746d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:27.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.883+0000 7f08a1400640 1 --2- 172.21.15.17:0/3879472252 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f087c072210 0x7f087c0746d0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f0888017000 tx=0x7f0888005000 comp rx=0 tx=0).stop 2024-09-10T21:29:27.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.883+0000 7f08a1400640 1 -- 172.21.15.17:0/3879472252 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f089c0caf00 msgr2=0x7f089c1f4aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:27.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.883+0000 7f08a1400640 1 --2- 172.21.15.17:0/3879472252 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f089c0caf00 0x7f089c1f4aa0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f088c009850 tx=0x7f088c004040 comp rx=0 tx=0).stop 2024-09-10T21:29:27.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.883+0000 7f089b200640 1 -- 172.21.15.17:0/3879472252 reap_dead start 2024-09-10T21:29:27.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.883+0000 7f08a1400640 1 -- 172.21.15.17:0/3879472252 shutdown_connections 2024-09-10T21:29:27.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.883+0000 7f08a1400640 1 --2- 172.21.15.17:0/3879472252 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f089c0caf00 0x7f089c1f4aa0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:27.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.883+0000 7f08a1400640 1 -- 172.21.15.17:0/3879472252 >> 172.21.15.17:0/3879472252 conn(0x7f089c0714a0 msgr2=0x7f089c0cd180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:27.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.883+0000 7f08a1400640 1 -- 172.21.15.17:0/3879472252 shutdown_connections 2024-09-10T21:29:27.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:27.883+0000 7f08a1400640 1 -- 172.21.15.17:0/3879472252 wait complete. 2024-09-10T21:29:27.908 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-10T21:29:27.909 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['c'] 2024-09-10T21:29:27.909 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2024-09-10T21:29:27.909 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.c 2024-09-10T21:29:27.909 INFO:tasks.mon_thrash.mon_thrasher:killing mon.c 2024-09-10T21:29:27.909 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-10T21:29:27.909 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:29:27.925 INFO:tasks.ceph.mon.c:Stopped 2024-09-10T21:29:27.925 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2024-09-10T21:29:27.926 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-10T21:29:28.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:28.079+0000 7f373da00640 1 -- 172.21.15.17:0/2778042586 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f371c005d40 msgr2=0x7f371c006140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:28.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:28.079+0000 7f373da00640 1 --2- 172.21.15.17:0/2778042586 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f371c005d40 0x7f371c006140 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f3728001840 tx=0x7f372801c870 comp rx=0 tx=0).stop 2024-09-10T21:29:28.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:28.079+0000 7f3737400640 1 -- 172.21.15.17:0/2778042586 reap_dead start 2024-09-10T21:29:28.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:28.079+0000 7f373da00640 1 -- 172.21.15.17:0/2778042586 shutdown_connections 2024-09-10T21:29:28.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:28.079+0000 7f373da00640 1 -- 172.21.15.17:0/2778042586 >> 172.21.15.17:0/2778042586 conn(0x7f3738070160 msgr2=0x7f3738070570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:28.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:28.079+0000 7f373da00640 1 -- 172.21.15.17:0/2778042586 shutdown_connections 2024-09-10T21:29:28.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:28.079+0000 7f373da00640 1 -- 172.21.15.17:0/2778042586 wait complete. 2024-09-10T21:29:28.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:28.079+0000 7f373da00640 1 Processor -- start 2024-09-10T21:29:28.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:28.079+0000 7f373da00640 1 -- start start 2024-09-10T21:29:28.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:28.083+0000 7f373da00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f371c005d40 0x7f3738144c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:28.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:28.083+0000 7f373da00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f37380c3420 0x7f37380aeb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:28.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:28.083+0000 7f373da00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f37380c4c10 0x7f37380c5090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:28.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:28.083+0000 7f373da00640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f37380c8050 con 0x7f37380c3420 2024-09-10T21:29:28.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:28.083+0000 7f373da00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f37380c7ed0 con 0x7f371c005d40 2024-09-10T21:29:28.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:28.083+0000 7f373da00640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f3738074450 con 0x7f37380c4c10 2024-09-10T21:29:28.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:28.083+0000 7f3736000640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f37380c3420 0x7f37380aeb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:28.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:28.083+0000 7f3737400640 1 -- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f37380c4c10 msgr2=0x7f37380c5090 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:3301/0 2024-09-10T21:29:28.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:28.083+0000 7f3737400640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f37380c4c10 0x7f37380c5090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:29:28.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:28.083+0000 7f3736000640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f37380c3420 0x7f37380aeb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40662/0 (socket says 172.21.15.17:40662) 2024-09-10T21:29:28.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:28.083+0000 7f3736000640 1 -- 172.21.15.17:0/4148881444 learned_addr learned my addr 172.21.15.17:0/4148881444 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:29:28.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:28.083+0000 7f3736a00640 1 --2- 172.21.15.17:0/4148881444 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f371c005d40 0x7f3738144c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:28.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:28.083+0000 7f3736000640 1 -- 172.21.15.17:0/4148881444 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f37380c4c10 msgr2=0x7f37380c5090 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:29:28.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:28.083+0000 7f3736000640 1 --2- 172.21.15.17:0/4148881444 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f37380c4c10 0x7f37380c5090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:28.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:28.083+0000 7f3736000640 1 -- 172.21.15.17:0/4148881444 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f371c005d40 msgr2=0x7f3738144c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:28.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:28.083+0000 7f3736000640 1 --2- 172.21.15.17:0/4148881444 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f371c005d40 0x7f3738144c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:28.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:28.083+0000 7f3736000640 1 -- 172.21.15.17:0/4148881444 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37381451b0 con 0x7f37380c3420 2024-09-10T21:29:28.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:28.083+0000 7f3736000640 1 --2- 172.21.15.17:0/4148881444 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f37380c3420 0x7f37380aeb50 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f3724003830 tx=0x7f3724003cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:28.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:28.083+0000 7f372ea00640 1 -- 172.21.15.17:0/4148881444 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f372400fb40 con 0x7f37380c3420 2024-09-10T21:29:28.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:28.083+0000 7f372ea00640 1 -- 172.21.15.17:0/4148881444 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3724016020 con 0x7f37380c3420 2024-09-10T21:29:28.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:28.083+0000 7f372ea00640 1 -- 172.21.15.17:0/4148881444 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3724011430 con 0x7f37380c3420 2024-09-10T21:29:28.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:28.083+0000 7f373da00640 1 -- 172.21.15.17:0/4148881444 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37380c3910 con 0x7f37380c3420 2024-09-10T21:29:28.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:28.083+0000 7f373da00640 1 -- 172.21.15.17:0/4148881444 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37381f7470 con 0x7f37380c3420 2024-09-10T21:29:28.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:28.083+0000 7f373da00640 1 -- 172.21.15.17:0/4148881444 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37380c7030 con 0x7f37380c3420 2024-09-10T21:29:28.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:28.087+0000 7f372ea00640 1 -- 172.21.15.17:0/4148881444 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f37240115d0 con 0x7f37380c3420 2024-09-10T21:29:28.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:28.087+0000 7f372ea00640 1 --2- 172.21.15.17:0/4148881444 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f37180725c0 0x7f3718074a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:28.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:28.087+0000 7f372ea00640 1 -- 172.21.15.17:0/4148881444 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f3724094550 con 0x7f37380c3420 2024-09-10T21:29:28.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:28.087+0000 7f3736a00640 1 --2- 172.21.15.17:0/4148881444 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f37180725c0 0x7f3718074a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:28.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:28.087+0000 7f3736a00640 1 --2- 172.21.15.17:0/4148881444 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f37180725c0 0x7f3718074a80 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f3728001810 tx=0x7f37280196b0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:28.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:28.091+0000 7f372ea00640 1 -- 172.21.15.17:0/4148881444 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f3724020020 con 0x7f37380c3420 2024-09-10T21:29:28.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:28.351+0000 7f373da00640 1 -- 172.21.15.17:0/4148881444 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f37380c31d0 con 0x7f37380c3420 2024-09-10T21:29:28.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:28.351+0000 7f372ea00640 1 -- 172.21.15.17:0/4148881444 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1557 (secure 0 0 0) 0x7f3724003e20 con 0x7f37380c3420 2024-09-10T21:29:28.359 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":40,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":17,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5e2f023f-d6a6-49f4-a256-f4807dce3739","modified":"2024-09-10T21:23:46.892130Z","created":"2024-09-10T21:23:46.892130Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3301","nonce":0},{"type":"v1","addr":"172.21.15.31:6790","nonce":0}]},"addr":"172.21.15.31:6790/0","public_addr":"172.21.15.31:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-10T21:29:28.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:28.355+0000 7f373da00640 1 -- 172.21.15.17:0/4148881444 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f37180725c0 msgr2=0x7f3718074a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:28.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:28.355+0000 7f373da00640 1 --2- 172.21.15.17:0/4148881444 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f37180725c0 0x7f3718074a80 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f3728001810 tx=0x7f37280196b0 comp rx=0 tx=0).stop 2024-09-10T21:29:28.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:28.355+0000 7f373da00640 1 -- 172.21.15.17:0/4148881444 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f37380c3420 msgr2=0x7f37380aeb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:28.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:28.355+0000 7f373da00640 1 --2- 172.21.15.17:0/4148881444 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f37380c3420 0x7f37380aeb50 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f3724003830 tx=0x7f3724003cf0 comp rx=0 tx=0).stop 2024-09-10T21:29:28.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:28.355+0000 7f373da00640 1 -- 172.21.15.17:0/4148881444 shutdown_connections 2024-09-10T21:29:28.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:28.355+0000 7f373da00640 1 --2- 172.21.15.17:0/4148881444 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f37180725c0 0x7f3718074a80 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:28.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:28.355+0000 7f373da00640 1 --2- 172.21.15.17:0/4148881444 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f37380c4c10 0x7f37380c5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:28.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:28.355+0000 7f373da00640 1 --2- 172.21.15.17:0/4148881444 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f37380c3420 0x7f37380aeb50 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:28.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:28.355+0000 7f373da00640 1 --2- 172.21.15.17:0/4148881444 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f371c005d40 0x7f3738144c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:28.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:28.355+0000 7f373da00640 1 -- 172.21.15.17:0/4148881444 >> 172.21.15.17:0/4148881444 conn(0x7f3738070160 msgr2=0x7f37380cc6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:28.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:28.355+0000 7f373da00640 1 -- 172.21.15.17:0/4148881444 shutdown_connections 2024-09-10T21:29:28.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:28.355+0000 7f373da00640 1 -- 172.21.15.17:0/4148881444 wait complete. 2024-09-10T21:29:31.385 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-10T21:29:31.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:31.563+0000 7f932ce00640 1 -- 172.21.15.17:0/425674921 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f932806f480 msgr2=0x7f93280c7870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:31.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:31.563+0000 7f932ce00640 1 --2- 172.21.15.17:0/425674921 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f932806f480 0x7f93280c7870 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f9318009900 tx=0x7f931801c8e0 comp rx=0 tx=0).stop 2024-09-10T21:29:31.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:31.563+0000 7f932ce00640 1 -- 172.21.15.17:0/425674921 shutdown_connections 2024-09-10T21:29:31.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:31.563+0000 7f932ce00640 1 --2- 172.21.15.17:0/425674921 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f932806f480 0x7f93280c7870 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:31.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:31.563+0000 7f932ce00640 1 -- 172.21.15.17:0/425674921 >> 172.21.15.17:0/425674921 conn(0x7f9328070ca0 msgr2=0x7f93280710b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:31.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:31.563+0000 7f932ce00640 1 -- 172.21.15.17:0/425674921 shutdown_connections 2024-09-10T21:29:31.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:31.563+0000 7f932ce00640 1 -- 172.21.15.17:0/425674921 wait complete. 2024-09-10T21:29:31.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:31.563+0000 7f932ce00640 1 Processor -- start 2024-09-10T21:29:31.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:31.563+0000 7f932ce00640 1 -- start start 2024-09-10T21:29:31.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:31.567+0000 7f932ce00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f932806f480 0x7f9328074600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:31.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:31.567+0000 7f932ce00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f932806f860 0x7f93281f7240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:31.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:31.567+0000 7f932ce00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f93280c8070 0x7f93281f7780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:31.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:31.567+0000 7f932ce00640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f93280702a0 con 0x7f93280c8070 2024-09-10T21:29:31.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:31.567+0000 7f932ce00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f9328151b40 con 0x7f932806f860 2024-09-10T21:29:31.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:31.567+0000 7f932ce00640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f9328165500 con 0x7f932806f480 2024-09-10T21:29:31.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:31.567+0000 7f9327400640 1 -- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f932806f480 msgr2=0x7f9328074600 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:3301/0 2024-09-10T21:29:31.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:31.567+0000 7f9327e00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f93280c8070 0x7f93281f7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:31.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:31.567+0000 7f9327400640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f932806f480 0x7f9328074600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:29:31.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:31.567+0000 7f9327e00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f93280c8070 0x7f93281f7780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40686/0 (socket says 172.21.15.17:40686) 2024-09-10T21:29:31.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:31.567+0000 7f9327e00640 1 -- 172.21.15.17:0/1074192703 learned_addr learned my addr 172.21.15.17:0/1074192703 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:29:31.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:31.567+0000 7f9326a00640 1 --2- 172.21.15.17:0/1074192703 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f932806f860 0x7f93281f7240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:31.574 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:31.567+0000 7f9327e00640 1 -- 172.21.15.17:0/1074192703 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f932806f480 msgr2=0x7f9328074600 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:29:31.575 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:31.567+0000 7f9327e00640 1 --2- 172.21.15.17:0/1074192703 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f932806f480 0x7f9328074600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:31.575 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:31.567+0000 7f9327e00640 1 -- 172.21.15.17:0/1074192703 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f932806f860 msgr2=0x7f93281f7240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:31.575 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:31.567+0000 7f9327e00640 1 --2- 172.21.15.17:0/1074192703 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f932806f860 0x7f93281f7240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:31.575 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:31.567+0000 7f9327e00640 1 -- 172.21.15.17:0/1074192703 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93280c4fa0 con 0x7f93280c8070 2024-09-10T21:29:31.575 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:31.567+0000 7f9327e00640 1 --2- 172.21.15.17:0/1074192703 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f93280c8070 0x7f93281f7780 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f932000b970 tx=0x7f932000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:31.575 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:31.567+0000 7f931f400640 1 -- 172.21.15.17:0/1074192703 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9320019070 con 0x7f93280c8070 2024-09-10T21:29:31.575 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:31.567+0000 7f931f400640 1 -- 172.21.15.17:0/1074192703 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f932000cc30 con 0x7f93280c8070 2024-09-10T21:29:31.575 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:31.567+0000 7f931f400640 1 -- 172.21.15.17:0/1074192703 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93200146a0 con 0x7f93280c8070 2024-09-10T21:29:31.575 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:31.567+0000 7f932ce00640 1 -- 172.21.15.17:0/1074192703 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93280c4cc0 con 0x7f93280c8070 2024-09-10T21:29:31.575 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:31.567+0000 7f932ce00640 1 -- 172.21.15.17:0/1074192703 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93280c4590 con 0x7f93280c8070 2024-09-10T21:29:31.576 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:31.567+0000 7f932ce00640 1 -- 172.21.15.17:0/1074192703 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93280c7870 con 0x7f93280c8070 2024-09-10T21:29:31.576 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:31.571+0000 7f931f400640 1 -- 172.21.15.17:0/1074192703 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f932001e440 con 0x7f93280c8070 2024-09-10T21:29:31.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:31.571+0000 7f931f400640 1 --2- 172.21.15.17:0/1074192703 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f93080725c0 0x7f9308074a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:31.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:31.571+0000 7f931f400640 1 -- 172.21.15.17:0/1074192703 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f93200934a0 con 0x7f93280c8070 2024-09-10T21:29:31.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:31.571+0000 7f9327400640 1 --2- 172.21.15.17:0/1074192703 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f93080725c0 0x7f9308074a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:31.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:31.571+0000 7f9327400640 1 --2- 172.21.15.17:0/1074192703 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f93080725c0 0x7f9308074a80 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f93180af000 tx=0x7f931801b120 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:31.581 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:31.575+0000 7f931f400640 1 -- 172.21.15.17:0/1074192703 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f932005e0a0 con 0x7f93280c8070 2024-09-10T21:29:31.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:31.891+0000 7f932ce00640 1 -- 172.21.15.17:0/1074192703 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f932814d6c0 con 0x7f93280c8070 2024-09-10T21:29:31.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:31.891+0000 7f931f400640 1 -- 172.21.15.17:0/1074192703 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1557 (secure 0 0 0) 0x7f9320062ad0 con 0x7f93280c8070 2024-09-10T21:29:31.897 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":40,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":20,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5e2f023f-d6a6-49f4-a256-f4807dce3739","modified":"2024-09-10T21:23:46.892130Z","created":"2024-09-10T21:23:46.892130Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3301","nonce":0},{"type":"v1","addr":"172.21.15.31:6790","nonce":0}]},"addr":"172.21.15.31:6790/0","public_addr":"172.21.15.31:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-10T21:29:31.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:31.891+0000 7f932ce00640 1 -- 172.21.15.17:0/1074192703 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f93080725c0 msgr2=0x7f9308074a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:31.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:31.891+0000 7f932ce00640 1 --2- 172.21.15.17:0/1074192703 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f93080725c0 0x7f9308074a80 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f93180af000 tx=0x7f931801b120 comp rx=0 tx=0).stop 2024-09-10T21:29:31.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:31.891+0000 7f932ce00640 1 -- 172.21.15.17:0/1074192703 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f93280c8070 msgr2=0x7f93281f7780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:31.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:31.891+0000 7f932ce00640 1 --2- 172.21.15.17:0/1074192703 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f93280c8070 0x7f93281f7780 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f932000b970 tx=0x7f932000be30 comp rx=0 tx=0).stop 2024-09-10T21:29:31.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:31.891+0000 7f932ce00640 1 -- 172.21.15.17:0/1074192703 shutdown_connections 2024-09-10T21:29:31.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:31.891+0000 7f932ce00640 1 --2- 172.21.15.17:0/1074192703 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f93080725c0 0x7f9308074a80 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:31.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:31.891+0000 7f932ce00640 1 --2- 172.21.15.17:0/1074192703 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f93280c8070 0x7f93281f7780 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:31.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:31.891+0000 7f932ce00640 1 --2- 172.21.15.17:0/1074192703 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f932806f860 0x7f93281f7240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:31.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:31.891+0000 7f932ce00640 1 --2- 172.21.15.17:0/1074192703 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f932806f480 0x7f9328074600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:31.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:31.891+0000 7f932ce00640 1 -- 172.21.15.17:0/1074192703 >> 172.21.15.17:0/1074192703 conn(0x7f9328070ca0 msgr2=0x7f932814fc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:31.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:31.891+0000 7f932ce00640 1 -- 172.21.15.17:0/1074192703 shutdown_connections 2024-09-10T21:29:31.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:31.891+0000 7f932ce00640 1 -- 172.21.15.17:0/1074192703 wait complete. 2024-09-10T21:29:34.924 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-10T21:29:42.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:42.823+0000 7f8a91400640 1 -- 172.21.15.17:0/1046697686 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8a700039c0 msgr2=0x7f8a70003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:42.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:42.823+0000 7f8a91400640 1 --2- 172.21.15.17:0/1046697686 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8a700039c0 0x7f8a70003de0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f8a7c0048e0 tx=0x7f8a7c01dbc0 comp rx=0 tx=0).stop 2024-09-10T21:29:42.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:42.823+0000 7f8a8b200640 1 -- 172.21.15.17:0/1046697686 reap_dead start 2024-09-10T21:29:42.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:42.823+0000 7f8a91400640 1 -- 172.21.15.17:0/1046697686 shutdown_connections 2024-09-10T21:29:42.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:42.823+0000 7f8a91400640 1 -- 172.21.15.17:0/1046697686 >> 172.21.15.17:0/1046697686 conn(0x7f8a8c0702f0 msgr2=0x7f8a8c070700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:42.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:42.823+0000 7f8a91400640 1 -- 172.21.15.17:0/1046697686 shutdown_connections 2024-09-10T21:29:42.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:42.823+0000 7f8a91400640 1 -- 172.21.15.17:0/1046697686 wait complete. 2024-09-10T21:29:42.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:42.827+0000 7f8a91400640 1 Processor -- start 2024-09-10T21:29:42.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:42.827+0000 7f8a91400640 1 -- start start 2024-09-10T21:29:42.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:42.827+0000 7f8a91400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8a8c0c6220 0x7f8a8c0c6600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:42.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:42.827+0000 7f8a91400640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8a8c0c57b0 0x7f8a8c1553d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:42.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:42.827+0000 7f8a91400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8a8c1517f0 0x7f8a8c155910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:42.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:42.827+0000 7f8a91400640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f8a8c0ce710 con 0x7f8a8c0c6220 2024-09-10T21:29:42.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:42.827+0000 7f8a91400640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f8a8c0b42c0 con 0x7f8a8c1517f0 2024-09-10T21:29:42.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:42.827+0000 7f8a91400640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f8a8c0cb450 con 0x7f8a8c0c57b0 2024-09-10T21:29:42.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:42.827+0000 7f8a89e00640 1 -- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8a8c0c57b0 msgr2=0x7f8a8c1553d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:3301/0 2024-09-10T21:29:42.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:42.827+0000 7f8a89e00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8a8c0c57b0 0x7f8a8c1553d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:29:42.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:42.827+0000 7f8a8a800640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8a8c0c6220 0x7f8a8c0c6600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:42.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:42.827+0000 7f8a8a800640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8a8c0c6220 0x7f8a8c0c6600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:48140/0 (socket says 172.21.15.17:48140) 2024-09-10T21:29:42.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:42.827+0000 7f8a8a800640 1 -- 172.21.15.17:0/3778744607 learned_addr learned my addr 172.21.15.17:0/3778744607 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:29:42.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:42.827+0000 7f8a8b200640 1 --2- 172.21.15.17:0/3778744607 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8a8c1517f0 0x7f8a8c155910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:42.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:42.827+0000 7f8a8a800640 1 -- 172.21.15.17:0/3778744607 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8a8c0c57b0 msgr2=0x7f8a8c1553d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:29:42.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:42.827+0000 7f8a8a800640 1 --2- 172.21.15.17:0/3778744607 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8a8c0c57b0 0x7f8a8c1553d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:42.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:42.827+0000 7f8a8a800640 1 -- 172.21.15.17:0/3778744607 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8a8c1517f0 msgr2=0x7f8a8c155910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:42.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:42.827+0000 7f8a8a800640 1 --2- 172.21.15.17:0/3778744607 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8a8c1517f0 0x7f8a8c155910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:42.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:42.827+0000 7f8a8a800640 1 -- 172.21.15.17:0/3778744607 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a8c0c73e0 con 0x7f8a8c0c6220 2024-09-10T21:29:42.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:42.827+0000 7f8a8a800640 1 --2- 172.21.15.17:0/3778744607 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8a8c0c6220 0x7f8a8c0c6600 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f8a7c0b2040 tx=0x7f8a7c004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:42.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:42.827+0000 7f8a82a00640 1 -- 172.21.15.17:0/3778744607 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a7c0bf070 con 0x7f8a8c0c6220 2024-09-10T21:29:42.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:42.827+0000 7f8a82a00640 1 -- 172.21.15.17:0/3778744607 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8a7c0a6a40 con 0x7f8a8c0c6220 2024-09-10T21:29:42.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:42.827+0000 7f8a91400640 1 -- 172.21.15.17:0/3778744607 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a8c0c6c80 con 0x7f8a8c0c6220 2024-09-10T21:29:42.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:42.827+0000 7f8a82a00640 1 -- 172.21.15.17:0/3778744607 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a7c0ac050 con 0x7f8a8c0c6220 2024-09-10T21:29:42.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:42.827+0000 7f8a91400640 1 -- 172.21.15.17:0/3778744607 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a8c1e4030 con 0x7f8a8c0c6220 2024-09-10T21:29:42.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:42.831+0000 7f8a91400640 1 -- 172.21.15.17:0/3778744607 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a8c0c2ff0 con 0x7f8a8c0c6220 2024-09-10T21:29:42.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:42.831+0000 7f8a82a00640 1 -- 172.21.15.17:0/3778744607 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f8a7c003600 con 0x7f8a8c0c6220 2024-09-10T21:29:42.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:42.831+0000 7f8a82a00640 1 --2- 172.21.15.17:0/3778744607 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f8a6c073490 0x7f8a6c075950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:42.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:42.831+0000 7f8a82a00640 1 -- 172.21.15.17:0/3778744607 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f8a7c12e660 con 0x7f8a8c0c6220 2024-09-10T21:29:42.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:42.831+0000 7f8a89e00640 1 --2- 172.21.15.17:0/3778744607 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f8a6c073490 0x7f8a6c075950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:42.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:42.835+0000 7f8a89e00640 1 --2- 172.21.15.17:0/3778744607 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f8a6c073490 0x7f8a6c075950 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f8a8c1f7d40 tx=0x7f8a7800b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:42.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:42.839+0000 7f8a82a00640 1 -- 172.21.15.17:0/3778744607 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f8a7c017a00 con 0x7f8a8c0c6220 2024-09-10T21:29:43.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:43.127+0000 7f8a91400640 1 -- 172.21.15.17:0/3778744607 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f8a8c15a8a0 con 0x7f8a8c0c6220 2024-09-10T21:29:43.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:43.127+0000 7f8a82a00640 1 -- 172.21.15.17:0/3778744607 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1550 (secure 0 0 0) 0x7f8a7c0f9270 con 0x7f8a8c0c6220 2024-09-10T21:29:43.133 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":44,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5e2f023f-d6a6-49f4-a256-f4807dce3739","modified":"2024-09-10T21:23:46.892130Z","created":"2024-09-10T21:23:46.892130Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3301","nonce":0},{"type":"v1","addr":"172.21.15.31:6790","nonce":0}]},"addr":"172.21.15.31:6790/0","public_addr":"172.21.15.31:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-10T21:29:43.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:43.127+0000 7f8a91400640 1 -- 172.21.15.17:0/3778744607 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f8a6c073490 msgr2=0x7f8a6c075950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:43.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:43.127+0000 7f8a91400640 1 --2- 172.21.15.17:0/3778744607 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f8a6c073490 0x7f8a6c075950 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f8a8c1f7d40 tx=0x7f8a7800b040 comp rx=0 tx=0).stop 2024-09-10T21:29:43.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:43.127+0000 7f8a91400640 1 -- 172.21.15.17:0/3778744607 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8a8c0c6220 msgr2=0x7f8a8c0c6600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:43.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:43.127+0000 7f8a91400640 1 --2- 172.21.15.17:0/3778744607 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8a8c0c6220 0x7f8a8c0c6600 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f8a7c0b2040 tx=0x7f8a7c004610 comp rx=0 tx=0).stop 2024-09-10T21:29:43.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:43.127+0000 7f8a91400640 1 -- 172.21.15.17:0/3778744607 shutdown_connections 2024-09-10T21:29:43.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:43.127+0000 7f8a91400640 1 --2- 172.21.15.17:0/3778744607 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f8a6c073490 0x7f8a6c075950 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:43.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:43.127+0000 7f8a91400640 1 --2- 172.21.15.17:0/3778744607 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8a8c1517f0 0x7f8a8c155910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:43.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:43.127+0000 7f8a91400640 1 --2- 172.21.15.17:0/3778744607 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8a8c0c57b0 0x7f8a8c1553d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:43.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:43.127+0000 7f8a91400640 1 --2- 172.21.15.17:0/3778744607 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8a8c0c6220 0x7f8a8c0c6600 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:43.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:43.127+0000 7f8a91400640 1 -- 172.21.15.17:0/3778744607 >> 172.21.15.17:0/3778744607 conn(0x7f8a8c0702f0 msgr2=0x7f8a8c0ca230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:43.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:43.131+0000 7f8a91400640 1 -- 172.21.15.17:0/3778744607 shutdown_connections 2024-09-10T21:29:43.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:43.131+0000 7f8a91400640 1 -- 172.21.15.17:0/3778744607 wait complete. 2024-09-10T21:29:43.157 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2024-09-10T21:29:43.157 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon a 2024-09-10T21:29:43.157 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-10T21:29:43.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:43.327+0000 7fadc1800640 1 -- 172.21.15.17:0/4104882185 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fadbc0cad50 msgr2=0x7fadbc15b1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:43.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:43.327+0000 7fadc1800640 1 --2- 172.21.15.17:0/4104882185 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fadbc0cad50 0x7fadbc15b1b0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fadbc0c7570 tx=0x7fadb401ccf0 comp rx=0 tx=0).stop 2024-09-10T21:29:43.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:43.327+0000 7fadc1800640 1 -- 172.21.15.17:0/4104882185 shutdown_connections 2024-09-10T21:29:43.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:43.327+0000 7fadc1800640 1 --2- 172.21.15.17:0/4104882185 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fadbc0cad50 0x7fadbc15b1b0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:43.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:43.331+0000 7fadc1800640 1 --2- 172.21.15.17:0/4104882185 >> v2:172.21.15.31:3300/0 conn(0x7fadbc0c78d0 0x7fadbc15ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:43.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:43.331+0000 7fadc1800640 1 -- 172.21.15.17:0/4104882185 >> 172.21.15.17:0/4104882185 conn(0x7fadbc0cdbd0 msgr2=0x7fadbc0c99f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:43.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:43.331+0000 7fadc1800640 1 -- 172.21.15.17:0/4104882185 shutdown_connections 2024-09-10T21:29:43.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:43.331+0000 7fadc1800640 1 -- 172.21.15.17:0/4104882185 wait complete. 2024-09-10T21:29:43.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:43.331+0000 7fadc1800640 1 Processor -- start 2024-09-10T21:29:43.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:43.331+0000 7fadc1800640 1 -- start start 2024-09-10T21:29:43.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:43.331+0000 7fadc1800640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fadbc0c78d0 0x7fadbc159ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:43.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:43.331+0000 7fadc1800640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fadbc0cad50 0x7fadbc1f3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:43.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:43.331+0000 7fadc1800640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fadbc151590 0x7fadbc14d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:43.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:43.331+0000 7fadc1800640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fadbc151970 con 0x7fadbc0c78d0 2024-09-10T21:29:43.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:43.331+0000 7fadc1800640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fadbc0c8d80 con 0x7fadbc151590 2024-09-10T21:29:43.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:43.331+0000 7fadc1800640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7fadbc147a10 con 0x7fadbc0cad50 2024-09-10T21:29:43.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:43.331+0000 7fadbb400640 1 -- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fadbc0cad50 msgr2=0x7fadbc1f3650 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:3301/0 2024-09-10T21:29:43.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:43.331+0000 7fadbbe00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fadbc0c78d0 0x7fadbc159ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:43.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:43.331+0000 7fadbb400640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fadbc0cad50 0x7fadbc1f3650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:29:43.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:43.331+0000 7fadbbe00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fadbc0c78d0 0x7fadbc159ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:48154/0 (socket says 172.21.15.17:48154) 2024-09-10T21:29:43.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:43.331+0000 7fadbbe00640 1 -- 172.21.15.17:0/984334237 learned_addr learned my addr 172.21.15.17:0/984334237 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:29:43.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:43.331+0000 7fadc0e00640 1 --2- 172.21.15.17:0/984334237 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fadbc151590 0x7fadbc14d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:43.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:43.331+0000 7fadbbe00640 1 -- 172.21.15.17:0/984334237 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fadbc0cad50 msgr2=0x7fadbc1f3650 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:29:43.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:43.331+0000 7fadbbe00640 1 --2- 172.21.15.17:0/984334237 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fadbc0cad50 0x7fadbc1f3650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:43.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:43.331+0000 7fadbbe00640 1 -- 172.21.15.17:0/984334237 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fadbc151590 msgr2=0x7fadbc14d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:43.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:43.331+0000 7fadbbe00640 1 --2- 172.21.15.17:0/984334237 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fadbc151590 0x7fadbc14d100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:43.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:43.331+0000 7fadbbe00640 1 -- 172.21.15.17:0/984334237 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadbc145e00 con 0x7fadbc0c78d0 2024-09-10T21:29:43.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:43.331+0000 7fadbbe00640 1 --2- 172.21.15.17:0/984334237 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fadbc0c78d0 0x7fadbc159ec0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fadac00c3c0 tx=0x7fadac00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:43.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:43.335+0000 7fadb3e00640 1 -- 172.21.15.17:0/984334237 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadac0052d0 con 0x7fadbc0c78d0 2024-09-10T21:29:43.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:43.335+0000 7fadb3e00640 1 -- 172.21.15.17:0/984334237 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fadac007020 con 0x7fadbc0c78d0 2024-09-10T21:29:43.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:43.335+0000 7fadc1800640 1 -- 172.21.15.17:0/984334237 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fadbc14b840 con 0x7fadbc0c78d0 2024-09-10T21:29:43.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:43.335+0000 7fadb3e00640 1 -- 172.21.15.17:0/984334237 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadac007470 con 0x7fadbc0c78d0 2024-09-10T21:29:43.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:43.335+0000 7fadc1800640 1 -- 172.21.15.17:0/984334237 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fadbc1476a0 con 0x7fadbc0c78d0 2024-09-10T21:29:43.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:43.335+0000 7fadc1800640 1 --2- 172.21.15.17:0/984334237 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fadbc146eb0 0x7fadbc1f6bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:43.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:43.335+0000 7fadc1800640 1 -- 172.21.15.17:0/984334237 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fadbc151af0 con 0x7fadbc146eb0 2024-09-10T21:29:43.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:43.335+0000 7fadbb400640 1 --2- 172.21.15.17:0/984334237 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fadbc146eb0 0x7fadbc1f6bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:43.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:43.335+0000 7fadbb400640 1 -- 172.21.15.17:0/984334237 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fadbc14bd60 con 0x7fadbc146eb0 2024-09-10T21:29:43.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:43.335+0000 7fadbb400640 1 --2- 172.21.15.17:0/984334237 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fadbc146eb0 0x7fadbc1f6bc0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fadbc157010 tx=0x7fada800dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:43.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:43.335+0000 7fadb3e00640 1 -- 172.21.15.17:0/984334237 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fada8017070 con 0x7fadbc146eb0 2024-09-10T21:29:43.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:43.335+0000 7fadb3e00640 1 -- 172.21.15.17:0/984334237 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fada8005220 con 0x7fadbc146eb0 2024-09-10T21:29:43.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:43.335+0000 7fadb3e00640 1 -- 172.21.15.17:0/984334237 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fadbc146eb0 msgr2=0x7fadbc1f6bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:43.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:43.335+0000 7fadb3e00640 1 --2- 172.21.15.17:0/984334237 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fadbc146eb0 0x7fadbc1f6bc0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fadbc157010 tx=0x7fada800dae0 comp rx=0 tx=0).stop 2024-09-10T21:29:43.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:43.335+0000 7fadb3e00640 1 -- 172.21.15.17:0/984334237 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fadbc146eb0 msgr2=0x7fadbc1f6bc0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:29:43.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:43.335+0000 7fadb3e00640 1 --2- 172.21.15.17:0/984334237 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fadbc146eb0 0x7fadbc1f6bc0 secure :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fadbc157010 tx=0x7fada800dae0 comp rx=0 tx=0).stop 2024-09-10T21:29:43.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:43.335+0000 7fadb3e00640 1 -- 172.21.15.17:0/984334237 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fadac0067e0 con 0x7fadbc0c78d0 2024-09-10T21:29:43.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:43.339+0000 7fadb3e00640 1 --2- 172.21.15.17:0/984334237 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fad9c072090 0x7fad9c074550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:43.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:43.339+0000 7fadb3e00640 1 -- 172.21.15.17:0/984334237 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fadac092ab0 con 0x7fadbc0c78d0 2024-09-10T21:29:43.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:43.339+0000 7fadbb400640 1 --2- 172.21.15.17:0/984334237 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fad9c072090 0x7fad9c074550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:43.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:43.339+0000 7fadbb400640 1 --2- 172.21.15.17:0/984334237 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fad9c072090 0x7fad9c074550 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fadbc147290 tx=0x7fada801d040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:43.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:43.343+0000 7fadc1800640 1 --2- 172.21.15.17:0/984334237 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fadbc14c1b0 0x7fadbc1f7100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:43.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:43.343+0000 7fadc1800640 1 -- 172.21.15.17:0/984334237 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fadbc069ef0 con 0x7fadbc14c1b0 2024-09-10T21:29:43.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:43.347+0000 7fadc0e00640 1 --2- 172.21.15.17:0/984334237 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fadbc14c1b0 0x7fadbc1f7100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:43.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:43.347+0000 7fadc0e00640 1 -- 172.21.15.17:0/984334237 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fadbc0cb270 con 0x7fadbc14c1b0 2024-09-10T21:29:43.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:43.347+0000 7fadc0e00640 1 --2- 172.21.15.17:0/984334237 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fadbc14c1b0 0x7fadbc1f7100 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fadbc0ca5e0 tx=0x7fadb4004bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:43.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:43.347+0000 7fadb3e00640 1 -- 172.21.15.17:0/984334237 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadb40bc020 con 0x7fadbc14c1b0 2024-09-10T21:29:43.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:43.347+0000 7fadb3e00640 1 -- 172.21.15.17:0/984334237 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4866 (secure 0 0 0) 0x7fadb4003b80 con 0x7fadbc14c1b0 2024-09-10T21:29:43.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:43.347+0000 7fadb3e00640 1 -- 172.21.15.17:0/984334237 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fadbc14c1b0 msgr2=0x7fadbc1f7100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:43.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:43.347+0000 7fadb3e00640 1 --2- 172.21.15.17:0/984334237 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fadbc14c1b0 0x7fadbc1f7100 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fadbc0ca5e0 tx=0x7fadb4004bb0 comp rx=0 tx=0).stop 2024-09-10T21:29:43.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:43.347+0000 7fadb3e00640 1 -- 172.21.15.17:0/984334237 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fadbc14c1b0 msgr2=0x7fadbc1f7100 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:29:43.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:43.347+0000 7fadb3e00640 1 --2- 172.21.15.17:0/984334237 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fadbc14c1b0 0x7fadbc1f7100 secure :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fadbc0ca5e0 tx=0x7fadb4004bb0 comp rx=0 tx=0).stop 2024-09-10T21:29:43.354 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-10T21:29:43.354 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:29:43.354 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:29:43.354 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "leader", 2024-09-10T21:29:43.354 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 44, 2024-09-10T21:29:43.354 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 101186, 2024-09-10T21:29:43.354 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-10T21:29:43.355 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-10T21:29:43.355 INFO:teuthology.orchestra.run.smithi017.stdout: 1 2024-09-10T21:29:43.355 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:29:43.355 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 0, 2024-09-10T21:29:43.355 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:29:43.355 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-10T21:29:43.355 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-10T21:29:43.355 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:29:43.356 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:29:43.356 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:29:43.356 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:29:43.356 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:29:43.356 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:29:43.357 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:29:43.357 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:29:43.357 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:29:43.357 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:29:43.357 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:29:43.357 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:29:43.357 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-10T21:29:43.357 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-10T21:29:43.357 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:29:43.358 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:29:43.358 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:29:43.358 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:29:43.358 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:29:43.358 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:29:43.358 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:29:43.358 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:29:43.358 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:29:43.358 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:29:43.359 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:29:43.359 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:29:43.359 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:29:43.359 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-10T21:29:43.359 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-10T21:29:43.359 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-10T21:29:43.359 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-10T21:29:43.359 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-10T21:29:43.359 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "5e2f023f-d6a6-49f4-a256-f4807dce3739", 2024-09-10T21:29:43.359 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:29:43.359 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:29:43.359 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-10T21:29:43.359 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-10T21:29:43.359 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-10T21:29:43.360 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-10T21:29:43.360 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-10T21:29:43.360 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-10T21:29:43.360 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-10T21:29:43.360 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:29:43.360 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-10T21:29:43.360 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:29:43.360 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:29:43.360 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:29:43.360 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:29:43.361 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:29:43.361 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:29:43.361 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:29:43.361 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:29:43.361 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:29:43.361 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:29:43.361 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:29:43.361 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:29:43.361 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-10T21:29:43.361 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:29:43.361 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-10T21:29:43.362 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:29:43.362 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:29:43.362 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:29:43.362 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:29:43.362 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:29:43.362 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:29:43.362 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:29:43.362 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-10T21:29:43.362 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:29:43.362 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:29:43.362 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:29:43.362 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:29:43.362 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-10T21:29:43.363 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:29:43.363 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:29:43.363 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:29:43.363 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:29:43.363 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-10T21:29:43.363 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-10T21:29:43.363 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:29:43.363 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:29:43.364 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:29:43.364 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:29:43.364 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:29:43.364 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:29:43.364 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:29:43.364 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:29:43.364 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:29:43.364 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:29:43.364 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:29:43.365 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3300", 2024-09-10T21:29:43.365 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:29:43.365 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:29:43.365 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:29:43.365 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:29:43.365 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789", 2024-09-10T21:29:43.365 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:29:43.365 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:29:43.365 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:29:43.365 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:29:43.366 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789/0", 2024-09-10T21:29:43.366 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6789/0", 2024-09-10T21:29:43.366 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:29:43.366 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:29:43.366 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:29:43.366 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:29:43.367 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:29:43.367 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:29:43.367 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:29:43.367 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:29:43.367 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:29:43.367 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:29:43.368 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:29:43.368 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3301", 2024-09-10T21:29:43.368 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:29:43.368 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:29:43.368 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:29:43.368 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:29:43.369 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790", 2024-09-10T21:29:43.369 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:29:43.369 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:29:43.369 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:29:43.369 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:29:43.369 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790/0", 2024-09-10T21:29:43.370 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6790/0", 2024-09-10T21:29:43.370 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:29:43.370 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:29:43.370 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:29:43.370 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:29:43.370 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:29:43.370 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:29:43.370 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-10T21:29:43.370 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-10T21:29:43.371 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:29:43.371 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:29:43.371 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:29:43.371 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:29:43.371 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:29:43.371 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:29:43.371 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": [ 2024-09-10T21:29:43.372 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:29:43.372 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:29:43.372 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:29:43.372 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:29:43.372 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:29:43.372 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:29:43.372 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-10T21:29:43.372 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:29:43.372 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:29:43.372 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:29:43.372 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:29:43.372 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:29:43.373 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:29:43.373 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-10T21:29:43.373 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:29:43.373 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:29:43.373 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:29:43.373 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 4 2024-09-10T21:29:43.373 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:29:43.373 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:29:43.373 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:29:43.373 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-10T21:29:43.374 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-10T21:29:43.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:43.347+0000 7fadc1800640 1 -- 172.21.15.17:0/984334237 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fad9c072090 msgr2=0x7fad9c074550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:43.376 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:43.347+0000 7fadc1800640 1 --2- 172.21.15.17:0/984334237 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fad9c072090 0x7fad9c074550 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fadbc147290 tx=0x7fada801d040 comp rx=0 tx=0).stop 2024-09-10T21:29:43.376 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:43.347+0000 7fadc1800640 1 -- 172.21.15.17:0/984334237 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fadbc0c78d0 msgr2=0x7fadbc159ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:43.376 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:43.347+0000 7fadc1800640 1 --2- 172.21.15.17:0/984334237 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fadbc0c78d0 0x7fadbc159ec0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fadac00c3c0 tx=0x7fadac00c880 comp rx=0 tx=0).stop 2024-09-10T21:29:43.376 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:43.347+0000 7fadc0e00640 1 -- 172.21.15.17:0/984334237 reap_dead start 2024-09-10T21:29:43.376 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:43.347+0000 7fadc1800640 1 -- 172.21.15.17:0/984334237 shutdown_connections 2024-09-10T21:29:43.376 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:43.347+0000 7fadc1800640 1 --2- 172.21.15.17:0/984334237 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fadbc0c78d0 0x7fadbc159ec0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:43.376 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:43.347+0000 7fadc1800640 1 -- 172.21.15.17:0/984334237 >> 172.21.15.17:0/984334237 conn(0x7fadbc0cdbd0 msgr2=0x7fadbc06d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:43.376 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:43.347+0000 7fadc1800640 1 -- 172.21.15.17:0/984334237 shutdown_connections 2024-09-10T21:29:43.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:43.347+0000 7fadc1800640 1 -- 172.21.15.17:0/984334237 wait complete. 2024-09-10T21:29:43.377 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-09-10T21:29:43.377 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon b 2024-09-10T21:29:43.377 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-10T21:29:43.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:43.527+0000 7fbe14a00640 1 -- 172.21.15.17:0/3425570449 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbe10151070 msgr2=0x7fbe1006ed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:43.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:43.527+0000 7fbe14a00640 1 --2- 172.21.15.17:0/3425570449 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbe10151070 0x7fbe1006ed80 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fbe100cd0b0 tx=0x7fbdf801dbf0 comp rx=0 tx=0).stop 2024-09-10T21:29:43.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:43.527+0000 7fbe14a00640 1 -- 172.21.15.17:0/3425570449 shutdown_connections 2024-09-10T21:29:43.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:43.527+0000 7fbe14a00640 1 --2- 172.21.15.17:0/3425570449 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbe10151070 0x7fbe1006ed80 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:43.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:43.527+0000 7fbe14a00640 1 --2- 172.21.15.17:0/3425570449 >> v2:172.21.15.31:3300/0 conn(0x7fbe10150c90 0x7fbe1006e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:43.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:43.527+0000 7fbe14a00640 1 -- 172.21.15.17:0/3425570449 >> 172.21.15.17:0/3425570449 conn(0x7fbe100c8ab0 msgr2=0x7fbe100c8ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:43.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:43.527+0000 7fbe14a00640 1 -- 172.21.15.17:0/3425570449 shutdown_connections 2024-09-10T21:29:43.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:43.527+0000 7fbe14a00640 1 -- 172.21.15.17:0/3425570449 wait complete. 2024-09-10T21:29:43.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:43.531+0000 7fbe14a00640 1 Processor -- start 2024-09-10T21:29:43.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:43.531+0000 7fbe14a00640 1 -- start start 2024-09-10T21:29:43.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:43.531+0000 7fbe14a00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fbe10150c90 0x7fbe1013b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:43.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:43.531+0000 7fbe14a00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbe10151070 0x7fbe101546d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:43.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:43.531+0000 7fbe14a00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbe10151910 0x7fbe101603e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:43.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:43.531+0000 7fbe14a00640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fbe100735f0 con 0x7fbe10151070 2024-09-10T21:29:43.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:43.531+0000 7fbe14a00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fbe10075140 con 0x7fbe10151910 2024-09-10T21:29:43.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:43.531+0000 7fbe14a00640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7fbe10073770 con 0x7fbe10150c90 2024-09-10T21:29:43.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:43.531+0000 7fbe0e800640 1 -- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fbe10150c90 msgr2=0x7fbe1013b110 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:3301/0 2024-09-10T21:29:43.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:43.531+0000 7fbe0e800640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fbe10150c90 0x7fbe1013b110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:29:43.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:43.531+0000 7fbe07e00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbe10151070 0x7fbe101546d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:43.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:43.531+0000 7fbe07e00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbe10151070 0x7fbe101546d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:48190/0 (socket says 172.21.15.17:48190) 2024-09-10T21:29:43.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:43.531+0000 7fbe07e00640 1 -- 172.21.15.17:0/659747775 learned_addr learned my addr 172.21.15.17:0/659747775 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:29:43.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:43.531+0000 7fbe0f200640 1 --2- 172.21.15.17:0/659747775 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbe10151910 0x7fbe101603e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:43.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:43.531+0000 7fbe07e00640 1 -- 172.21.15.17:0/659747775 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fbe10150c90 msgr2=0x7fbe1013b110 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:29:43.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:43.531+0000 7fbe07e00640 1 --2- 172.21.15.17:0/659747775 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fbe10150c90 0x7fbe1013b110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:43.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:43.531+0000 7fbe07e00640 1 -- 172.21.15.17:0/659747775 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbe10151910 msgr2=0x7fbe101603e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:43.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:43.531+0000 7fbe07e00640 1 --2- 172.21.15.17:0/659747775 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbe10151910 0x7fbe101603e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:43.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:43.531+0000 7fbe07e00640 1 -- 172.21.15.17:0/659747775 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe1013b650 con 0x7fbe10151070 2024-09-10T21:29:43.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:43.531+0000 7fbe07e00640 1 --2- 172.21.15.17:0/659747775 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbe10151070 0x7fbe101546d0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fbdf801a800 tx=0x7fbdf8005de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:43.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:43.531+0000 7fbe06a00640 1 -- 172.21.15.17:0/659747775 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdf80b3070 con 0x7fbe10151070 2024-09-10T21:29:43.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:43.531+0000 7fbe06a00640 1 -- 172.21.15.17:0/659747775 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbdf8005530 con 0x7fbe10151070 2024-09-10T21:29:43.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:43.531+0000 7fbe14a00640 1 -- 172.21.15.17:0/659747775 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe10152310 con 0x7fbe10151070 2024-09-10T21:29:43.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:43.531+0000 7fbe06a00640 1 -- 172.21.15.17:0/659747775 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdf8004930 con 0x7fbe10151070 2024-09-10T21:29:43.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:43.531+0000 7fbe14a00640 1 -- 172.21.15.17:0/659747775 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe101f8530 con 0x7fbe10151070 2024-09-10T21:29:43.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:43.535+0000 7fbe14a00640 1 --2- 172.21.15.17:0/659747775 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbe100c1930 0x7fbe10160920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:43.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:43.535+0000 7fbe14a00640 1 -- 172.21.15.17:0/659747775 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fbe101525f0 con 0x7fbe100c1930 2024-09-10T21:29:43.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:43.535+0000 7fbe0e800640 1 --2- 172.21.15.17:0/659747775 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbe100c1930 0x7fbe10160920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:43.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:43.535+0000 7fbe0e800640 1 -- 172.21.15.17:0/659747775 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fbe101f8320 con 0x7fbe100c1930 2024-09-10T21:29:43.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:43.535+0000 7fbe0e800640 1 --2- 172.21.15.17:0/659747775 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbe100c1930 0x7fbe10160920 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fbe0000e8f0 tx=0x7fbe0000edb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:43.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:43.535+0000 7fbe06a00640 1 -- 172.21.15.17:0/659747775 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fbdf800b040 con 0x7fbe10151070 2024-09-10T21:29:43.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:43.535+0000 7fbe06a00640 1 --2- 172.21.15.17:0/659747775 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fbdec072590 0x7fbdec074a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:43.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:43.535+0000 7fbe06a00640 1 -- 172.21.15.17:0/659747775 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe0000cd30 con 0x7fbe100c1930 2024-09-10T21:29:43.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:43.535+0000 7fbe06a00640 1 -- 172.21.15.17:0/659747775 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fbdf812d670 con 0x7fbe10151070 2024-09-10T21:29:43.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:43.535+0000 7fbe06a00640 1 -- 172.21.15.17:0/659747775 <== mon.1 v2:172.21.15.31:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fbe00004030 con 0x7fbe100c1930 2024-09-10T21:29:43.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:43.535+0000 7fbe06a00640 1 -- 172.21.15.17:0/659747775 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbe100c1930 msgr2=0x7fbe10160920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:43.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:43.535+0000 7fbe06a00640 1 --2- 172.21.15.17:0/659747775 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbe100c1930 0x7fbe10160920 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fbe0000e8f0 tx=0x7fbe0000edb0 comp rx=0 tx=0).stop 2024-09-10T21:29:43.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:43.535+0000 7fbe06a00640 1 -- 172.21.15.17:0/659747775 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbe100c1930 msgr2=0x7fbe10160920 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:29:43.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:43.535+0000 7fbe06a00640 1 --2- 172.21.15.17:0/659747775 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbe100c1930 0x7fbe10160920 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:29:43.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:43.535+0000 7fbe0f200640 1 --2- 172.21.15.17:0/659747775 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fbdec072590 0x7fbdec074a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:43.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:43.539+0000 7fbe0f200640 1 --2- 172.21.15.17:0/659747775 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fbdec072590 0x7fbdec074a50 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fbe080012f0 tx=0x7fbe08009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:43.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:43.543+0000 7fbe14a00640 1 --2- 172.21.15.17:0/659747775 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbe10153e70 0x7fbe10146ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:29:43.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:43.543+0000 7fbe14a00640 1 -- 172.21.15.17:0/659747775 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fbe10156e10 con 0x7fbe10153e70 2024-09-10T21:29:43.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:43.543+0000 7fbe0e800640 1 --2- 172.21.15.17:0/659747775 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbe10153e70 0x7fbe10146ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:29:43.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:43.543+0000 7fbe0e800640 1 -- 172.21.15.17:0/659747775 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fbe10074d10 con 0x7fbe10153e70 2024-09-10T21:29:43.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:43.547+0000 7fbe0e800640 1 --2- 172.21.15.17:0/659747775 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbe10153e70 0x7fbe10146ff0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fbe0000e8f0 tx=0x7fbe00004740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:29:43.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:43.547+0000 7fbe06a00640 1 -- 172.21.15.17:0/659747775 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe000047b0 con 0x7fbe10153e70 2024-09-10T21:29:43.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:43.547+0000 7fbe06a00640 1 -- 172.21.15.17:0/659747775 <== mon.1 v2:172.21.15.31:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5031 (secure 0 0 0) 0x7fbe000047b0 con 0x7fbe10153e70 2024-09-10T21:29:43.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:43.547+0000 7fbe06a00640 1 -- 172.21.15.17:0/659747775 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbe10153e70 msgr2=0x7fbe10146ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:43.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:43.547+0000 7fbe06a00640 1 --2- 172.21.15.17:0/659747775 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbe10153e70 0x7fbe10146ff0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fbe0000e8f0 tx=0x7fbe00004740 comp rx=0 tx=0).stop 2024-09-10T21:29:43.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:43.547+0000 7fbe06a00640 1 -- 172.21.15.17:0/659747775 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbe10153e70 msgr2=0x7fbe10146ff0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:29:43.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:43.547+0000 7fbe06a00640 1 --2- 172.21.15.17:0/659747775 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbe10153e70 0x7fbe10146ff0 secure :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fbe0000e8f0 tx=0x7fbe00004740 comp rx=0 tx=0).stop 2024-09-10T21:29:43.554 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-10T21:29:43.554 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:29:43.554 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:29:43.554 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-10T21:29:43.554 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 44, 2024-09-10T21:29:43.554 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 321994, 2024-09-10T21:29:43.554 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-10T21:29:43.554 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-10T21:29:43.554 INFO:teuthology.orchestra.run.smithi017.stdout: 1 2024-09-10T21:29:43.554 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:29:43.554 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 0, 2024-09-10T21:29:43.554 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:29:43.555 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-10T21:29:43.555 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-10T21:29:43.555 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:29:43.555 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:29:43.555 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:29:43.555 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:29:43.555 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:29:43.555 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:29:43.555 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:29:43.555 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:29:43.555 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:29:43.555 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:29:43.555 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:29:43.556 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:29:43.556 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-10T21:29:43.556 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-10T21:29:43.556 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:29:43.556 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:29:43.556 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:29:43.556 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:29:43.556 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:29:43.556 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:29:43.556 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:29:43.556 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:29:43.556 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:29:43.556 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:29:43.557 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:29:43.557 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:29:43.557 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:29:43.557 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-10T21:29:43.557 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-10T21:29:43.557 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-10T21:29:43.557 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-10T21:29:43.557 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-10T21:29:43.557 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "5e2f023f-d6a6-49f4-a256-f4807dce3739", 2024-09-10T21:29:43.557 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:29:43.557 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:29:43.557 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-10T21:29:43.558 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-10T21:29:43.558 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-10T21:29:43.558 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-10T21:29:43.558 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-10T21:29:43.558 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-10T21:29:43.558 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-10T21:29:43.558 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:29:43.558 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-10T21:29:43.558 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:29:43.558 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:29:43.558 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:29:43.558 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:29:43.558 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:29:43.558 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:29:43.559 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:29:43.559 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:29:43.559 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:29:43.559 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:29:43.559 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:29:43.559 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:29:43.559 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-10T21:29:43.559 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:29:43.559 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-10T21:29:43.559 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:29:43.559 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:29:43.559 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:29:43.560 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:29:43.560 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:29:43.560 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:29:43.560 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:29:43.560 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-10T21:29:43.560 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:29:43.560 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:29:43.560 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:29:43.560 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:29:43.560 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-10T21:29:43.560 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:29:43.560 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:29:43.560 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:29:43.560 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:29:43.561 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-10T21:29:43.561 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-10T21:29:43.561 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:29:43.561 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:29:43.561 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:29:43.561 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:29:43.561 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:29:43.561 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:29:43.561 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:29:43.561 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:29:43.561 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:29:43.561 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:29:43.562 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:29:43.562 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3300", 2024-09-10T21:29:43.562 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:29:43.562 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:29:43.562 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:29:43.562 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:29:43.562 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789", 2024-09-10T21:29:43.562 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:29:43.562 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:29:43.562 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:29:43.562 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:29:43.562 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789/0", 2024-09-10T21:29:43.562 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6789/0", 2024-09-10T21:29:43.563 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:29:43.563 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:29:43.563 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:29:43.563 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:29:43.563 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:29:43.563 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:29:43.563 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:29:43.563 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:29:43.563 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:29:43.563 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:29:43.563 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:29:43.563 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3301", 2024-09-10T21:29:43.563 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:29:43.564 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:29:43.564 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:29:43.564 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:29:43.564 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790", 2024-09-10T21:29:43.564 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:29:43.564 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:29:43.564 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:29:43.564 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:29:43.564 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790/0", 2024-09-10T21:29:43.564 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6790/0", 2024-09-10T21:29:43.564 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:29:43.564 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:29:43.564 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:29:43.564 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:29:43.565 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:29:43.565 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:29:43.565 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-10T21:29:43.565 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-10T21:29:43.565 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:29:43.565 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:29:43.565 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:29:43.565 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:29:43.565 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:29:43.565 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:29:43.565 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": [ 2024-09-10T21:29:43.565 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:29:43.566 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:29:43.566 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:29:43.566 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 5 2024-09-10T21:29:43.566 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:29:43.566 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:29:43.566 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-10T21:29:43.566 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:29:43.566 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:29:43.566 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:29:43.566 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 7 2024-09-10T21:29:43.566 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:29:43.566 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:29:43.566 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-10T21:29:43.566 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:29:43.566 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:29:43.567 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:29:43.567 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 4 2024-09-10T21:29:43.567 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:29:43.567 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:29:43.567 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": [ 2024-09-10T21:29:43.567 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:29:43.567 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:29:43.567 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:29:43.567 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 2 2024-09-10T21:29:43.567 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:29:43.567 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:29:43.567 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:29:43.567 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-10T21:29:43.568 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-10T21:29:43.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:43.547+0000 7fbe14a00640 1 -- 172.21.15.17:0/659747775 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fbdec072590 msgr2=0x7fbdec074a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:43.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:43.547+0000 7fbe14a00640 1 --2- 172.21.15.17:0/659747775 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fbdec072590 0x7fbdec074a50 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fbe080012f0 tx=0x7fbe08009040 comp rx=0 tx=0).stop 2024-09-10T21:29:43.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:43.547+0000 7fbe14a00640 1 -- 172.21.15.17:0/659747775 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbe10151070 msgr2=0x7fbe101546d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:29:43.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:43.547+0000 7fbe14a00640 1 --2- 172.21.15.17:0/659747775 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbe10151070 0x7fbe101546d0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fbdf801a800 tx=0x7fbdf8005de0 comp rx=0 tx=0).stop 2024-09-10T21:29:43.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:43.547+0000 7fbe0f200640 1 -- 172.21.15.17:0/659747775 reap_dead start 2024-09-10T21:29:43.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:43.547+0000 7fbe0f200640 1 -- 172.21.15.17:0/659747775 reap_dead start 2024-09-10T21:29:43.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:43.547+0000 7fbe14a00640 1 -- 172.21.15.17:0/659747775 shutdown_connections 2024-09-10T21:29:43.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:43.547+0000 7fbe14a00640 1 -- 172.21.15.17:0/659747775 >> 172.21.15.17:0/659747775 conn(0x7fbe100c8ab0 msgr2=0x7fbe10153760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:29:43.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:43.547+0000 7fbe14a00640 1 -- 172.21.15.17:0/659747775 shutdown_connections 2024-09-10T21:29:43.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:29:43.547+0000 7fbe14a00640 1 -- 172.21.15.17:0/659747775 wait complete. 2024-09-10T21:29:43.572 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-09-10T21:29:43.572 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2024-09-10T21:29:47.111 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:29:47.114 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:30:03.572 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.c 2024-09-10T21:30:03.573 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-10T21:30:03.573 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-10T21:30:03.577 INFO:tasks.ceph.mon.c:Started 2024-09-10T21:30:03.577 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-09-10T21:30:03.577 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-10T21:30:03.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:03.743+0000 7fda2ea00640 1 -- 172.21.15.17:0/3584096445 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fda10003e10 msgr2=0x7fda1000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:03.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:03.743+0000 7fda2ea00640 1 --2- 172.21.15.17:0/3584096445 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fda10003e10 0x7fda1000b9d0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fda200017b0 tx=0x7fda2001d130 comp rx=0 tx=0).stop 2024-09-10T21:30:03.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:03.743+0000 7fda2e000640 1 -- 172.21.15.17:0/3584096445 reap_dead start 2024-09-10T21:30:03.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:03.743+0000 7fda2ea00640 1 -- 172.21.15.17:0/3584096445 shutdown_connections 2024-09-10T21:30:03.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:03.743+0000 7fda2ea00640 1 -- 172.21.15.17:0/3584096445 >> 172.21.15.17:0/3584096445 conn(0x7fda2806cdf0 msgr2=0x7fda2806d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:03.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:03.743+0000 7fda2ea00640 1 -- 172.21.15.17:0/3584096445 shutdown_connections 2024-09-10T21:30:03.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:03.743+0000 7fda2ea00640 1 -- 172.21.15.17:0/3584096445 wait complete. 2024-09-10T21:30:03.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:03.743+0000 7fda2ea00640 1 Processor -- start 2024-09-10T21:30:03.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:03.743+0000 7fda2ea00640 1 -- start start 2024-09-10T21:30:03.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:03.743+0000 7fda2ea00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fda10003e10 0x7fda28150c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:03.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:03.743+0000 7fda2ea00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fda281ee550 0x7fda2815b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:03.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:03.743+0000 7fda2ea00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fda2815b9e0 0x7fda28149780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:03.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:03.743+0000 7fda2ea00640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fda281590c0 con 0x7fda281ee550 2024-09-10T21:30:03.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:03.743+0000 7fda2ea00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fda28158490 con 0x7fda2815b9e0 2024-09-10T21:30:03.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:03.743+0000 7fda2ea00640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7fda28159240 con 0x7fda10003e10 2024-09-10T21:30:03.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:03.743+0000 7fda2cc00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fda281ee550 0x7fda2815b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:03.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:03.743+0000 7fda2cc00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fda281ee550 0x7fda2815b4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41558/0 (socket says 172.21.15.17:41558) 2024-09-10T21:30:03.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:03.743+0000 7fda2cc00640 1 -- 172.21.15.17:0/1649736365 learned_addr learned my addr 172.21.15.17:0/1649736365 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:30:03.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:03.743+0000 7fda2d600640 1 --2- 172.21.15.17:0/1649736365 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fda10003e10 0x7fda28150c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:03.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:03.743+0000 7fda2e000640 1 --2- 172.21.15.17:0/1649736365 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fda2815b9e0 0x7fda28149780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:03.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:03.743+0000 7fda2d600640 1 -- 172.21.15.17:0/1649736365 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fda10003e10 msgr2=0x7fda28150c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-09-10T21:30:03.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:03.743+0000 7fda2d600640 1 -- 172.21.15.17:0/1649736365 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fda10003e10 msgr2=0x7fda28150c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-10T21:30:03.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:03.743+0000 7fda2d600640 1 --2- 172.21.15.17:0/1649736365 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fda10003e10 0x7fda28150c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T21:30:03.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:03.743+0000 7fda2cc00640 1 -- 172.21.15.17:0/1649736365 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fda10003e10 msgr2=0x7fda28150c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:03.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:03.743+0000 7fda2d600640 1 --2- 172.21.15.17:0/1649736365 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fda10003e10 0x7fda28150c00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:30:03.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:03.743+0000 7fda2cc00640 1 --2- 172.21.15.17:0/1649736365 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fda10003e10 0x7fda28150c00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:03.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:03.743+0000 7fda2cc00640 1 -- 172.21.15.17:0/1649736365 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fda2815b9e0 msgr2=0x7fda28149780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:03.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:03.743+0000 7fda2cc00640 1 --2- 172.21.15.17:0/1649736365 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fda2815b9e0 0x7fda28149780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:03.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:03.743+0000 7fda2cc00640 1 -- 172.21.15.17:0/1649736365 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda2815c1c0 con 0x7fda281ee550 2024-09-10T21:30:03.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:03.743+0000 7fda2cc00640 1 --2- 172.21.15.17:0/1649736365 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fda281ee550 0x7fda2815b4a0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fda14010aa0 tx=0x7fda14010f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:03.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:03.747+0000 7fda25600640 1 -- 172.21.15.17:0/1649736365 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda1400edb0 con 0x7fda281ee550 2024-09-10T21:30:03.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:03.747+0000 7fda25600640 1 -- 172.21.15.17:0/1649736365 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fda1401c480 con 0x7fda281ee550 2024-09-10T21:30:03.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:03.747+0000 7fda25600640 1 -- 172.21.15.17:0/1649736365 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda1401c8d0 con 0x7fda281ee550 2024-09-10T21:30:03.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:03.747+0000 7fda2ea00640 1 -- 172.21.15.17:0/1649736365 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda2815c480 con 0x7fda281ee550 2024-09-10T21:30:03.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:03.747+0000 7fda2ea00640 1 -- 172.21.15.17:0/1649736365 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda28149260 con 0x7fda281ee550 2024-09-10T21:30:03.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:03.747+0000 7fda2ea00640 1 -- 172.21.15.17:0/1649736365 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda280c8bf0 con 0x7fda281ee550 2024-09-10T21:30:03.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:03.747+0000 7fda25600640 1 -- 172.21.15.17:0/1649736365 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fda14024030 con 0x7fda281ee550 2024-09-10T21:30:03.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:03.751+0000 7fda25600640 1 --2- 172.21.15.17:0/1649736365 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fda08076a40 0x7fda08078f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:03.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:03.751+0000 7fda25600640 1 -- 172.21.15.17:0/1649736365 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fda140941b0 con 0x7fda281ee550 2024-09-10T21:30:03.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:03.751+0000 7fda2d600640 1 --2- 172.21.15.17:0/1649736365 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fda08076a40 0x7fda08078f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:03.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:03.751+0000 7fda2d600640 1 --2- 172.21.15.17:0/1649736365 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fda08076a40 0x7fda08078f00 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fda1c001230 tx=0x7fda1c001120 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:03.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:03.751+0000 7fda25600640 1 -- 172.21.15.17:0/1649736365 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fda1405f500 con 0x7fda281ee550 2024-09-10T21:30:04.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:04.051+0000 7fda2ea00640 1 -- 172.21.15.17:0/1649736365 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fda28069e60 con 0x7fda281ee550 2024-09-10T21:30:04.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:04.051+0000 7fda25600640 1 -- 172.21.15.17:0/1649736365 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1551 (secure 0 0 0) 0x7fda1405ee00 con 0x7fda281ee550 2024-09-10T21:30:04.053 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":44,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":21,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5e2f023f-d6a6-49f4-a256-f4807dce3739","modified":"2024-09-10T21:23:46.892130Z","created":"2024-09-10T21:23:46.892130Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3301","nonce":0},{"type":"v1","addr":"172.21.15.31:6790","nonce":0}]},"addr":"172.21.15.31:6790/0","public_addr":"172.21.15.31:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-10T21:30:04.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:04.051+0000 7fda2ea00640 1 -- 172.21.15.17:0/1649736365 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fda08076a40 msgr2=0x7fda08078f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:04.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:04.051+0000 7fda2ea00640 1 --2- 172.21.15.17:0/1649736365 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fda08076a40 0x7fda08078f00 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fda1c001230 tx=0x7fda1c001120 comp rx=0 tx=0).stop 2024-09-10T21:30:04.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:04.051+0000 7fda2ea00640 1 -- 172.21.15.17:0/1649736365 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fda281ee550 msgr2=0x7fda2815b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:04.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:04.051+0000 7fda2ea00640 1 --2- 172.21.15.17:0/1649736365 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fda281ee550 0x7fda2815b4a0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fda14010aa0 tx=0x7fda14010f60 comp rx=0 tx=0).stop 2024-09-10T21:30:04.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:04.051+0000 7fda2ea00640 1 -- 172.21.15.17:0/1649736365 shutdown_connections 2024-09-10T21:30:04.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:04.051+0000 7fda2ea00640 1 --2- 172.21.15.17:0/1649736365 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fda08076a40 0x7fda08078f00 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:04.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:04.051+0000 7fda2ea00640 1 --2- 172.21.15.17:0/1649736365 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fda2815b9e0 0x7fda28149780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:04.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:04.051+0000 7fda2ea00640 1 --2- 172.21.15.17:0/1649736365 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fda281ee550 0x7fda2815b4a0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:04.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:04.051+0000 7fda2ea00640 1 --2- 172.21.15.17:0/1649736365 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fda10003e10 0x7fda28150c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:04.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:04.051+0000 7fda2ea00640 1 -- 172.21.15.17:0/1649736365 >> 172.21.15.17:0/1649736365 conn(0x7fda2806cdf0 msgr2=0x7fda2814ef90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:04.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:04.055+0000 7fda2ea00640 1 -- 172.21.15.17:0/1649736365 shutdown_connections 2024-09-10T21:30:04.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:04.055+0000 7fda2ea00640 1 -- 172.21.15.17:0/1649736365 wait complete. 2024-09-10T21:30:07.080 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-10T21:30:09.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:09.327+0000 7fc9d3600640 1 -- 172.21.15.17:0/2069121284 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc9cc0b3eb0 msgr2=0x7fc9b4004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:09.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:09.327+0000 7fc9d3600640 1 --2- 172.21.15.17:0/2069121284 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc9cc0b3eb0 0x7fc9b4004810 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fc9b8006eb0 tx=0x7fc9b801cdf0 comp rx=0 tx=0).stop 2024-09-10T21:30:09.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:09.327+0000 7fc9d1600640 1 -- 172.21.15.17:0/2069121284 reap_dead start 2024-09-10T21:30:09.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:09.327+0000 7fc9d3600640 1 -- 172.21.15.17:0/2069121284 shutdown_connections 2024-09-10T21:30:09.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:09.327+0000 7fc9d3600640 1 -- 172.21.15.17:0/2069121284 >> 172.21.15.17:0/2069121284 conn(0x7fc9cc0702f0 msgr2=0x7fc9cc070700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:09.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:09.327+0000 7fc9d3600640 1 -- 172.21.15.17:0/2069121284 shutdown_connections 2024-09-10T21:30:09.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:09.327+0000 7fc9d3600640 1 -- 172.21.15.17:0/2069121284 wait complete. 2024-09-10T21:30:09.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:09.331+0000 7fc9d3600640 1 Processor -- start 2024-09-10T21:30:09.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:09.331+0000 7fc9d3600640 1 -- start start 2024-09-10T21:30:09.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:09.331+0000 7fc9d3600640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc9cc0b3eb0 0x7fc9cc06f490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:09.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:09.331+0000 7fc9d3600640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc9cc14e420 0x7fc9cc1f77c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:09.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:09.331+0000 7fc9d3600640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc9cc0c6d10 0x7fc9cc0b2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:09.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:09.331+0000 7fc9d3600640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fc9cc0ce630 con 0x7fc9cc14e420 2024-09-10T21:30:09.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:09.331+0000 7fc9d3600640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fc9cc0b42c0 con 0x7fc9cc0b3eb0 2024-09-10T21:30:09.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:09.331+0000 7fc9d3600640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7fc9cc0ce7b0 con 0x7fc9cc0c6d10 2024-09-10T21:30:09.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:09.331+0000 7fc9cbe00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc9cc14e420 0x7fc9cc1f77c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:09.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:09.331+0000 7fc9d0c00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc9cc0b3eb0 0x7fc9cc06f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:09.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:09.331+0000 7fc9d0c00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc9cc0b3eb0 0x7fc9cc06f490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.17:57946/0 (socket says 172.21.15.17:57946) 2024-09-10T21:30:09.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:09.331+0000 7fc9cbe00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc9cc14e420 0x7fc9cc1f77c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:36950/0 (socket says 172.21.15.17:36950) 2024-09-10T21:30:09.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:09.331+0000 7fc9d0c00640 1 -- 172.21.15.17:0/3474692439 learned_addr learned my addr 172.21.15.17:0/3474692439 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:30:09.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:09.331+0000 7fc9d1600640 1 --2- 172.21.15.17:0/3474692439 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc9cc0c6d10 0x7fc9cc0b2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:09.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:09.331+0000 7fc9d1600640 1 -- 172.21.15.17:0/3474692439 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc9cc0c6d10 msgr2=0x7fc9cc0b2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-09-10T21:30:09.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:09.331+0000 7fc9d1600640 1 -- 172.21.15.17:0/3474692439 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc9cc0c6d10 msgr2=0x7fc9cc0b2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-10T21:30:09.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:09.331+0000 7fc9d1600640 1 --2- 172.21.15.17:0/3474692439 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc9cc0c6d10 0x7fc9cc0b2ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T21:30:09.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:09.331+0000 7fc9d1600640 1 --2- 172.21.15.17:0/3474692439 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc9cc0c6d10 0x7fc9cc0b2ef0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:30:09.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:09.331+0000 7fc9d0c00640 1 -- 172.21.15.17:0/3474692439 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc9cc0c6d10 msgr2=0x7fc9cc0b2ef0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:30:09.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:09.331+0000 7fc9d0c00640 1 --2- 172.21.15.17:0/3474692439 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc9cc0c6d10 0x7fc9cc0b2ef0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:09.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:09.331+0000 7fc9d0c00640 1 -- 172.21.15.17:0/3474692439 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc9cc14e420 msgr2=0x7fc9cc1f77c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:09.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:09.331+0000 7fc9d0c00640 1 --2- 172.21.15.17:0/3474692439 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc9cc14e420 0x7fc9cc1f77c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:09.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:09.331+0000 7fc9d0c00640 1 -- 172.21.15.17:0/3474692439 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9cc0c74c0 con 0x7fc9cc0b3eb0 2024-09-10T21:30:09.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:09.331+0000 7fc9cbe00640 1 --2- 172.21.15.17:0/3474692439 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc9cc14e420 0x7fc9cc1f77c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:30:09.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:09.331+0000 7fc9d0c00640 1 --2- 172.21.15.17:0/3474692439 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc9cc0b3eb0 0x7fc9cc06f490 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fc9c000c460 tx=0x7fc9c000c920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:09.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:09.331+0000 7fc9c8c00640 1 -- 172.21.15.17:0/3474692439 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9c0016020 con 0x7fc9cc0b3eb0 2024-09-10T21:30:09.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:09.331+0000 7fc9c8c00640 1 -- 172.21.15.17:0/3474692439 <== mon.1 v2:172.21.15.31:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc9c0012920 con 0x7fc9cc0b3eb0 2024-09-10T21:30:09.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:09.331+0000 7fc9c8c00640 1 -- 172.21.15.17:0/3474692439 <== mon.1 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9c0012d50 con 0x7fc9cc0b3eb0 2024-09-10T21:30:09.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:09.331+0000 7fc9d3600640 1 -- 172.21.15.17:0/3474692439 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9cc14df40 con 0x7fc9cc0b3eb0 2024-09-10T21:30:09.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:09.335+0000 7fc9d3600640 1 -- 172.21.15.17:0/3474692439 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9cc14f240 con 0x7fc9cc0b3eb0 2024-09-10T21:30:09.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:09.335+0000 7fc9d3600640 1 -- 172.21.15.17:0/3474692439 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9cc0c2ff0 con 0x7fc9cc0b3eb0 2024-09-10T21:30:09.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:09.335+0000 7fc9c8c00640 1 -- 172.21.15.17:0/3474692439 <== mon.1 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fc9c001a3b0 con 0x7fc9cc0b3eb0 2024-09-10T21:30:09.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:09.335+0000 7fc9c8c00640 1 --2- 172.21.15.17:0/3474692439 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fc9ac0725e0 0x7fc9ac074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:09.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:09.335+0000 7fc9c8c00640 1 -- 172.21.15.17:0/3474692439 <== mon.1 v2:172.21.15.31:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fc9c00935a0 con 0x7fc9cc0b3eb0 2024-09-10T21:30:09.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:09.335+0000 7fc9cbe00640 1 --2- 172.21.15.17:0/3474692439 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fc9ac0725e0 0x7fc9ac074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:09.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:09.339+0000 7fc9cbe00640 1 --2- 172.21.15.17:0/3474692439 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fc9ac0725e0 0x7fc9ac074aa0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc9b80ab000 tx=0x7fc9b8019c30 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:09.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:09.339+0000 7fc9c8c00640 1 -- 172.21.15.17:0/3474692439 <== mon.1 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fc9c0020030 con 0x7fc9cc0b3eb0 2024-09-10T21:30:09.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:09.611+0000 7fc9d3600640 1 -- 172.21.15.17:0/3474692439 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fc9cc0ca020 con 0x7fc9cc0b3eb0 2024-09-10T21:30:09.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:09.611+0000 7fc9c8c00640 1 -- 172.21.15.17:0/3474692439 <== mon.1 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1550 (secure 0 0 0) 0x7fc9c005e880 con 0x7fc9cc0b3eb0 2024-09-10T21:30:09.615 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":48,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5e2f023f-d6a6-49f4-a256-f4807dce3739","modified":"2024-09-10T21:23:46.892130Z","created":"2024-09-10T21:23:46.892130Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3301","nonce":0},{"type":"v1","addr":"172.21.15.31:6790","nonce":0}]},"addr":"172.21.15.31:6790/0","public_addr":"172.21.15.31:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-10T21:30:09.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:09.615+0000 7fc9d3600640 1 -- 172.21.15.17:0/3474692439 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fc9ac0725e0 msgr2=0x7fc9ac074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:09.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:09.615+0000 7fc9d3600640 1 --2- 172.21.15.17:0/3474692439 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fc9ac0725e0 0x7fc9ac074aa0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc9b80ab000 tx=0x7fc9b8019c30 comp rx=0 tx=0).stop 2024-09-10T21:30:09.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:09.615+0000 7fc9d3600640 1 -- 172.21.15.17:0/3474692439 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc9cc0b3eb0 msgr2=0x7fc9cc06f490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:09.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:09.615+0000 7fc9d3600640 1 --2- 172.21.15.17:0/3474692439 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc9cc0b3eb0 0x7fc9cc06f490 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fc9c000c460 tx=0x7fc9c000c920 comp rx=0 tx=0).stop 2024-09-10T21:30:09.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:09.615+0000 7fc9d3600640 1 -- 172.21.15.17:0/3474692439 shutdown_connections 2024-09-10T21:30:09.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:09.615+0000 7fc9d3600640 1 --2- 172.21.15.17:0/3474692439 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fc9ac0725e0 0x7fc9ac074aa0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:09.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:09.615+0000 7fc9d3600640 1 --2- 172.21.15.17:0/3474692439 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc9cc0c6d10 0x7fc9cc0b2ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:09.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:09.615+0000 7fc9d3600640 1 --2- 172.21.15.17:0/3474692439 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc9cc14e420 0x7fc9cc1f77c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:09.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:09.615+0000 7fc9d3600640 1 --2- 172.21.15.17:0/3474692439 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc9cc0b3eb0 0x7fc9cc06f490 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:09.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:09.615+0000 7fc9d3600640 1 -- 172.21.15.17:0/3474692439 >> 172.21.15.17:0/3474692439 conn(0x7fc9cc0702f0 msgr2=0x7fc9cc0ca540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:09.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:09.615+0000 7fc9d3600640 1 -- 172.21.15.17:0/3474692439 shutdown_connections 2024-09-10T21:30:09.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:09.615+0000 7fc9d3600640 1 -- 172.21.15.17:0/3474692439 wait complete. 2024-09-10T21:30:12.644 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-10T21:30:12.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:12.791+0000 7fc4f7200640 1 -- 172.21.15.17:0/2425801911 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc4f80cc5d0 msgr2=0x7fc4f815a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:12.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:12.791+0000 7fc4f7200640 1 --2- 172.21.15.17:0/2425801911 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc4f80cc5d0 0x7fc4f815a5a0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fc4e4009840 tx=0x7fc4e401f010 comp rx=0 tx=0).stop 2024-09-10T21:30:12.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:12.791+0000 7fc4f7200640 1 -- 172.21.15.17:0/2425801911 shutdown_connections 2024-09-10T21:30:12.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:12.791+0000 7fc4f7200640 1 --2- 172.21.15.17:0/2425801911 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc4f80cc5d0 0x7fc4f815a5a0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:12.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:12.791+0000 7fc4f7200640 1 --2- 172.21.15.17:0/2425801911 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc4f80b4580 0x7fc4f80b4960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:12.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:12.791+0000 7fc4f7200640 1 -- 172.21.15.17:0/2425801911 >> 172.21.15.17:0/2425801911 conn(0x7fc4f806d700 msgr2=0x7fc4f8068990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:12.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:12.791+0000 7fc4f7200640 1 -- 172.21.15.17:0/2425801911 shutdown_connections 2024-09-10T21:30:12.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:12.791+0000 7fc4f7200640 1 -- 172.21.15.17:0/2425801911 wait complete. 2024-09-10T21:30:12.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:12.791+0000 7fc4f7200640 1 Processor -- start 2024-09-10T21:30:12.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:12.791+0000 7fc4f7200640 1 -- start start 2024-09-10T21:30:12.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:12.791+0000 7fc4f7200640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc4f80b4580 0x7fc4f81565e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:12.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:12.791+0000 7fc4f7200640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc4f80cb870 0x7fc4f8156b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:12.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:12.791+0000 7fc4f7200640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc4f80cc5d0 0x7fc4f81f02a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:12.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:12.791+0000 7fc4f7200640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fc4f815e3c0 con 0x7fc4f80cc5d0 2024-09-10T21:30:12.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:12.791+0000 7fc4f7200640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fc4f815e0e0 con 0x7fc4f80cb870 2024-09-10T21:30:12.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:12.791+0000 7fc4f7200640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7fc4f815e6a0 con 0x7fc4f80b4580 2024-09-10T21:30:12.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:12.795+0000 7fc4f5e00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc4f80b4580 0x7fc4f81565e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:12.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:12.795+0000 7fc4f6800640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc4f80cc5d0 0x7fc4f81f02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:12.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:12.795+0000 7fc4f5e00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc4f80b4580 0x7fc4f81565e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3301/0 says I am v2:172.21.15.17:52410/0 (socket says 172.21.15.17:52410) 2024-09-10T21:30:12.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:12.795+0000 7fc4f5e00640 1 -- 172.21.15.17:0/2595024576 learned_addr learned my addr 172.21.15.17:0/2595024576 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:30:12.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:12.795+0000 7fc4f5400640 1 --2- 172.21.15.17:0/2595024576 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc4f80cb870 0x7fc4f8156b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:12.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:12.795+0000 7fc4f5e00640 1 -- 172.21.15.17:0/2595024576 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc4f80b4580 msgr2=0x7fc4f81565e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-09-10T21:30:12.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:12.795+0000 7fc4f5e00640 1 -- 172.21.15.17:0/2595024576 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc4f80b4580 msgr2=0x7fc4f81565e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-10T21:30:12.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:12.795+0000 7fc4f5e00640 1 --2- 172.21.15.17:0/2595024576 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc4f80b4580 0x7fc4f81565e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T21:30:12.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:12.795+0000 7fc4f5e00640 1 --2- 172.21.15.17:0/2595024576 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc4f80b4580 0x7fc4f81565e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:30:12.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:12.795+0000 7fc4f5400640 1 -- 172.21.15.17:0/2595024576 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc4f80b4580 msgr2=0x7fc4f81565e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:30:12.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:12.795+0000 7fc4f5400640 1 --2- 172.21.15.17:0/2595024576 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc4f80b4580 0x7fc4f81565e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:12.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:12.795+0000 7fc4f5400640 1 -- 172.21.15.17:0/2595024576 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc4f80cc5d0 msgr2=0x7fc4f81f02a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:12.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:12.795+0000 7fc4f5400640 1 --2- 172.21.15.17:0/2595024576 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc4f80cc5d0 0x7fc4f81f02a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:12.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:12.795+0000 7fc4f5400640 1 -- 172.21.15.17:0/2595024576 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4f80c4450 con 0x7fc4f80cb870 2024-09-10T21:30:12.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:12.795+0000 7fc4f6800640 1 --2- 172.21.15.17:0/2595024576 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc4f80cc5d0 0x7fc4f81f02a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:30:12.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:12.795+0000 7fc4f5400640 1 --2- 172.21.15.17:0/2595024576 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc4f80cb870 0x7fc4f8156b20 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fc4e40af000 tx=0x7fc4e4005790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:12.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:12.795+0000 7fc4ee000640 1 -- 172.21.15.17:0/2595024576 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4e40b5040 con 0x7fc4f80cb870 2024-09-10T21:30:12.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:12.795+0000 7fc4ee000640 1 -- 172.21.15.17:0/2595024576 <== mon.1 v2:172.21.15.31:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc4e4003800 con 0x7fc4f80cb870 2024-09-10T21:30:12.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:12.795+0000 7fc4f7200640 1 -- 172.21.15.17:0/2595024576 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4f8157280 con 0x7fc4f80cb870 2024-09-10T21:30:12.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:12.795+0000 7fc4ee000640 1 -- 172.21.15.17:0/2595024576 <== mon.1 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4e4003c10 con 0x7fc4f80cb870 2024-09-10T21:30:12.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:12.795+0000 7fc4f7200640 1 -- 172.21.15.17:0/2595024576 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4f80ce4e0 con 0x7fc4f80cb870 2024-09-10T21:30:12.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:12.795+0000 7fc4f7200640 1 -- 172.21.15.17:0/2595024576 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4f80cb350 con 0x7fc4f80cb870 2024-09-10T21:30:12.799 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:12.795+0000 7fc4ee000640 1 -- 172.21.15.17:0/2595024576 <== mon.1 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fc4e40b8070 con 0x7fc4f80cb870 2024-09-10T21:30:12.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:12.799+0000 7fc4ee000640 1 --2- 172.21.15.17:0/2595024576 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fc4d80725e0 0x7fc4d8074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:12.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:12.799+0000 7fc4ee000640 1 -- 172.21.15.17:0/2595024576 <== mon.1 v2:172.21.15.31:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fc4e41303e0 con 0x7fc4f80cb870 2024-09-10T21:30:12.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:12.799+0000 7fc4f5e00640 1 --2- 172.21.15.17:0/2595024576 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fc4d80725e0 0x7fc4d8074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:12.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:12.799+0000 7fc4f5e00640 1 --2- 172.21.15.17:0/2595024576 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fc4d80725e0 0x7fc4d8074aa0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fc4e8005fd0 tx=0x7fc4e800b690 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:12.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:12.803+0000 7fc4ee000640 1 -- 172.21.15.17:0/2595024576 <== mon.1 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fc4e4018b70 con 0x7fc4f80cb870 2024-09-10T21:30:13.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.063+0000 7fc4f7200640 1 -- 172.21.15.17:0/2595024576 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fc4f80cd980 con 0x7fc4f80cb870 2024-09-10T21:30:13.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.063+0000 7fc4ee000640 1 -- 172.21.15.17:0/2595024576 <== mon.1 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1556 (secure 0 0 0) 0x7fc4e40fb640 con 0x7fc4f80cb870 2024-09-10T21:30:13.067 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":50,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":2,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5e2f023f-d6a6-49f4-a256-f4807dce3739","modified":"2024-09-10T21:23:46.892130Z","created":"2024-09-10T21:23:46.892130Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3301","nonce":0},{"type":"v1","addr":"172.21.15.31:6790","nonce":0}]},"addr":"172.21.15.31:6790/0","public_addr":"172.21.15.31:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-10T21:30:13.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.067+0000 7fc4f7200640 1 -- 172.21.15.17:0/2595024576 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fc4d80725e0 msgr2=0x7fc4d8074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:13.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.067+0000 7fc4f7200640 1 --2- 172.21.15.17:0/2595024576 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fc4d80725e0 0x7fc4d8074aa0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fc4e8005fd0 tx=0x7fc4e800b690 comp rx=0 tx=0).stop 2024-09-10T21:30:13.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.067+0000 7fc4f7200640 1 -- 172.21.15.17:0/2595024576 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc4f80cb870 msgr2=0x7fc4f8156b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:13.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.067+0000 7fc4f7200640 1 --2- 172.21.15.17:0/2595024576 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc4f80cb870 0x7fc4f8156b20 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fc4e40af000 tx=0x7fc4e4005790 comp rx=0 tx=0).stop 2024-09-10T21:30:13.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.067+0000 7fc4f7200640 1 -- 172.21.15.17:0/2595024576 shutdown_connections 2024-09-10T21:30:13.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.067+0000 7fc4f7200640 1 --2- 172.21.15.17:0/2595024576 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fc4d80725e0 0x7fc4d8074aa0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:13.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.067+0000 7fc4f7200640 1 --2- 172.21.15.17:0/2595024576 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc4f80cc5d0 0x7fc4f81f02a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:13.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.067+0000 7fc4f7200640 1 --2- 172.21.15.17:0/2595024576 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc4f80cb870 0x7fc4f8156b20 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:13.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.067+0000 7fc4f7200640 1 --2- 172.21.15.17:0/2595024576 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc4f80b4580 0x7fc4f81565e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:13.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.067+0000 7fc4f7200640 1 -- 172.21.15.17:0/2595024576 >> 172.21.15.17:0/2595024576 conn(0x7fc4f806d700 msgr2=0x7fc4f80cca70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:13.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.067+0000 7fc4f7200640 1 -- 172.21.15.17:0/2595024576 shutdown_connections 2024-09-10T21:30:13.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.067+0000 7fc4f7200640 1 -- 172.21.15.17:0/2595024576 wait complete. 2024-09-10T21:30:13.097 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-09-10T21:30:13.097 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-09-10T21:30:13.097 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-10T21:30:13.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.271+0000 7fe4bf200640 1 -- 172.21.15.17:0/45971742 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe4a0004d50 msgr2=0x7fe4a00051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:13.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.271+0000 7fe4bf200640 1 --2- 172.21.15.17:0/45971742 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe4a0004d50 0x7fe4a00051d0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fe4b8073540 tx=0x7fe4b401d1c0 comp rx=0 tx=0).stop 2024-09-10T21:30:13.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.271+0000 7fe4be800640 1 -- 172.21.15.17:0/45971742 reap_dead start 2024-09-10T21:30:13.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.271+0000 7fe4bf200640 1 -- 172.21.15.17:0/45971742 shutdown_connections 2024-09-10T21:30:13.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.271+0000 7fe4bf200640 1 -- 172.21.15.17:0/45971742 >> 172.21.15.17:0/45971742 conn(0x7fe4b806fe60 msgr2=0x7fe4b8070270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:13.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.271+0000 7fe4bf200640 1 -- 172.21.15.17:0/45971742 shutdown_connections 2024-09-10T21:30:13.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.271+0000 7fe4bf200640 1 -- 172.21.15.17:0/45971742 wait complete. 2024-09-10T21:30:13.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.275+0000 7fe4bf200640 1 Processor -- start 2024-09-10T21:30:13.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.275+0000 7fe4bf200640 1 -- start start 2024-09-10T21:30:13.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.275+0000 7fe4bf200640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe4a0004d50 0x7fe4b8152b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:13.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.275+0000 7fe4bf200640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe4b8154110 0x7fe4b814fa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:13.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.275+0000 7fe4bf200640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe4b8074c80 0x7fe4b81492c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:13.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.275+0000 7fe4bf200640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fe4b806dc30 con 0x7fe4b8154110 2024-09-10T21:30:13.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.275+0000 7fe4bf200640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fe4b8151070 con 0x7fe4a0004d50 2024-09-10T21:30:13.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.275+0000 7fe4bf200640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7fe4b80c8470 con 0x7fe4b8074c80 2024-09-10T21:30:13.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.275+0000 7fe4bd400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe4b8154110 0x7fe4b814fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:13.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.275+0000 7fe4bd400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe4b8154110 0x7fe4b814fa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:36964/0 (socket says 172.21.15.17:36964) 2024-09-10T21:30:13.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.275+0000 7fe4bd400640 1 -- 172.21.15.17:0/842124997 learned_addr learned my addr 172.21.15.17:0/842124997 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:30:13.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.275+0000 7fe4bde00640 1 --2- 172.21.15.17:0/842124997 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe4a0004d50 0x7fe4b8152b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:13.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.275+0000 7fe4be800640 1 --2- 172.21.15.17:0/842124997 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe4b8074c80 0x7fe4b81492c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:13.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.275+0000 7fe4be800640 1 -- 172.21.15.17:0/842124997 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe4b8074c80 msgr2=0x7fe4b81492c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-09-10T21:30:13.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.275+0000 7fe4be800640 1 -- 172.21.15.17:0/842124997 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe4b8074c80 msgr2=0x7fe4b81492c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-10T21:30:13.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.275+0000 7fe4be800640 1 --2- 172.21.15.17:0/842124997 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe4b8074c80 0x7fe4b81492c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T21:30:13.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.275+0000 7fe4be800640 1 --2- 172.21.15.17:0/842124997 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe4b8074c80 0x7fe4b81492c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:30:13.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.275+0000 7fe4bd400640 1 -- 172.21.15.17:0/842124997 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe4b8074c80 msgr2=0x7fe4b81492c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:30:13.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.275+0000 7fe4bd400640 1 --2- 172.21.15.17:0/842124997 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe4b8074c80 0x7fe4b81492c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:13.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.275+0000 7fe4bd400640 1 -- 172.21.15.17:0/842124997 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe4a0004d50 msgr2=0x7fe4b8152b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:13.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.275+0000 7fe4bd400640 1 --2- 172.21.15.17:0/842124997 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe4a0004d50 0x7fe4b8152b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:13.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.275+0000 7fe4bd400640 1 -- 172.21.15.17:0/842124997 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4b8076d80 con 0x7fe4b8154110 2024-09-10T21:30:13.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.275+0000 7fe4bd400640 1 --2- 172.21.15.17:0/842124997 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe4b8154110 0x7fe4b814fa70 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fe4a8000fe0 tx=0x7fe4a800ea90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:13.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.275+0000 7fe4b2000640 1 -- 172.21.15.17:0/842124997 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4a8015070 con 0x7fe4b8154110 2024-09-10T21:30:13.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.275+0000 7fe4b2000640 1 -- 172.21.15.17:0/842124997 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe4a800c9a0 con 0x7fe4b8154110 2024-09-10T21:30:13.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.275+0000 7fe4bf200640 1 -- 172.21.15.17:0/842124997 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4b81513f0 con 0x7fe4b8154110 2024-09-10T21:30:13.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.275+0000 7fe4b2000640 1 -- 172.21.15.17:0/842124997 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4a800cdb0 con 0x7fe4b8154110 2024-09-10T21:30:13.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.275+0000 7fe4bf200640 1 -- 172.21.15.17:0/842124997 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4b81519b0 con 0x7fe4b8154110 2024-09-10T21:30:13.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.279+0000 7fe4bf200640 1 --2- 172.21.15.17:0/842124997 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe4b8149800 0x7fe4b8151bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:13.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.279+0000 7fe4bf200640 1 -- 172.21.15.17:0/842124997 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fe4b8071d20 con 0x7fe4b8149800 2024-09-10T21:30:13.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.279+0000 7fe4bde00640 1 --2- 172.21.15.17:0/842124997 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe4b8149800 0x7fe4b8151bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:13.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.279+0000 7fe4bde00640 1 -- 172.21.15.17:0/842124997 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe4b8149be0 con 0x7fe4b8149800 2024-09-10T21:30:13.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.279+0000 7fe4bde00640 1 --2- 172.21.15.17:0/842124997 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe4b8149800 0x7fe4b8151bb0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fe4ac00bf60 tx=0x7fe4ac00bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:13.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.279+0000 7fe4b2000640 1 -- 172.21.15.17:0/842124997 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4ac011950 con 0x7fe4b8149800 2024-09-10T21:30:13.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.279+0000 7fe4b2000640 1 -- 172.21.15.17:0/842124997 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fe4a8011460 con 0x7fe4b8154110 2024-09-10T21:30:13.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.279+0000 7fe4b2000640 1 --2- 172.21.15.17:0/842124997 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fe49c0724e0 0x7fe49c0749a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:13.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.279+0000 7fe4b2000640 1 -- 172.21.15.17:0/842124997 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fe4a8093d60 con 0x7fe4b8154110 2024-09-10T21:30:13.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.279+0000 7fe4b2000640 1 -- 172.21.15.17:0/842124997 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fe4ac011af0 con 0x7fe4b8149800 2024-09-10T21:30:13.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.279+0000 7fe4b2000640 1 -- 172.21.15.17:0/842124997 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe4b8149800 msgr2=0x7fe4b8151bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:13.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.279+0000 7fe4b2000640 1 --2- 172.21.15.17:0/842124997 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe4b8149800 0x7fe4b8151bb0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fe4ac00bf60 tx=0x7fe4ac00bf90 comp rx=0 tx=0).stop 2024-09-10T21:30:13.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.279+0000 7fe4b2000640 1 -- 172.21.15.17:0/842124997 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe4b8149800 msgr2=0x7fe4b8151bb0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:30:13.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.279+0000 7fe4b2000640 1 --2- 172.21.15.17:0/842124997 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe4b8149800 0x7fe4b8151bb0 secure :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fe4ac00bf60 tx=0x7fe4ac00bf90 comp rx=0 tx=0).stop 2024-09-10T21:30:13.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.279+0000 7fe4be800640 1 --2- 172.21.15.17:0/842124997 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fe49c0724e0 0x7fe49c0749a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:13.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.279+0000 7fe4be800640 1 --2- 172.21.15.17:0/842124997 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fe49c0724e0 0x7fe49c0749a0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fe4b40ae000 tx=0x7fe4b4019f60 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:13.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.287+0000 7fe4bf200640 1 --2- 172.21.15.17:0/842124997 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe4b81ec1f0 0x7fe4b81ec600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:13.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.287+0000 7fe4bf200640 1 -- 172.21.15.17:0/842124997 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fe4b8153070 con 0x7fe4b81ec1f0 2024-09-10T21:30:13.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.287+0000 7fe4bde00640 1 --2- 172.21.15.17:0/842124997 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe4b81ec1f0 0x7fe4b81ec600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:13.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.287+0000 7fe4bde00640 1 -- 172.21.15.17:0/842124997 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fe4b8070d70 con 0x7fe4b81ec1f0 2024-09-10T21:30:13.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.287+0000 7fe4bde00640 1 --2- 172.21.15.17:0/842124997 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe4b81ec1f0 0x7fe4b81ec600 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fe4ac01b000 tx=0x7fe4ac004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:13.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.291+0000 7fe4b2000640 1 -- 172.21.15.17:0/842124997 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4ac0212d0 con 0x7fe4b81ec1f0 2024-09-10T21:30:13.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.291+0000 7fe4b2000640 1 -- 172.21.15.17:0/842124997 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4877 (secure 0 0 0) 0x7fe4ac01b630 con 0x7fe4b81ec1f0 2024-09-10T21:30:13.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.291+0000 7fe4b2000640 1 -- 172.21.15.17:0/842124997 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe4b81ec1f0 msgr2=0x7fe4b81ec600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:13.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.291+0000 7fe4b2000640 1 --2- 172.21.15.17:0/842124997 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe4b81ec1f0 0x7fe4b81ec600 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fe4ac01b000 tx=0x7fe4ac004740 comp rx=0 tx=0).stop 2024-09-10T21:30:13.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.291+0000 7fe4b2000640 1 -- 172.21.15.17:0/842124997 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe4b81ec1f0 msgr2=0x7fe4b81ec600 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:30:13.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.291+0000 7fe4b2000640 1 --2- 172.21.15.17:0/842124997 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe4b81ec1f0 0x7fe4b81ec600 secure :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fe4ac01b000 tx=0x7fe4ac004740 comp rx=0 tx=0).stop 2024-09-10T21:30:13.293 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-10T21:30:13.293 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:30:13.293 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:30:13.293 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "leader", 2024-09-10T21:30:13.293 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 50, 2024-09-10T21:30:13.293 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 131130, 2024-09-10T21:30:13.293 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-10T21:30:13.294 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-10T21:30:13.294 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-10T21:30:13.294 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-10T21:30:13.294 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:30:13.294 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 2, 2024-09-10T21:30:13.294 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:30:13.294 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-10T21:30:13.294 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-10T21:30:13.294 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:30:13.294 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:30:13.294 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:30:13.294 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:30:13.294 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:30:13.294 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:30:13.295 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:30:13.295 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:30:13.295 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:30:13.295 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:30:13.295 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:30:13.295 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:30:13.295 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-10T21:30:13.295 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-10T21:30:13.295 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:30:13.295 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:30:13.295 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:30:13.295 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:30:13.295 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:30:13.295 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:30:13.296 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:30:13.296 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:30:13.296 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:30:13.296 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:30:13.296 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:30:13.296 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:30:13.296 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:30:13.296 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-10T21:30:13.296 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-10T21:30:13.296 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-10T21:30:13.296 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-10T21:30:13.296 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-10T21:30:13.297 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "5e2f023f-d6a6-49f4-a256-f4807dce3739", 2024-09-10T21:30:13.297 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:30:13.297 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:30:13.297 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-10T21:30:13.297 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-10T21:30:13.297 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-10T21:30:13.297 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-10T21:30:13.297 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-10T21:30:13.297 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-10T21:30:13.297 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-10T21:30:13.297 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:30:13.297 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-10T21:30:13.297 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:30:13.297 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:30:13.298 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:30:13.298 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:30:13.298 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:30:13.298 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:30:13.298 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:30:13.298 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:30:13.298 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:30:13.298 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:30:13.298 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:30:13.298 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:30:13.298 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-10T21:30:13.298 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:30:13.298 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-10T21:30:13.298 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:30:13.299 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:30:13.299 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:30:13.299 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:30:13.299 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:30:13.299 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:30:13.299 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:30:13.299 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-10T21:30:13.299 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:30:13.299 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:30:13.299 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:30:13.299 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:30:13.299 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-10T21:30:13.299 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:30:13.299 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:30:13.300 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:30:13.300 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:30:13.300 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-10T21:30:13.300 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-10T21:30:13.300 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:30:13.300 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:30:13.300 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:30:13.300 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:30:13.300 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:30:13.300 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:30:13.300 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:30:13.300 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:30:13.300 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:30:13.301 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:30:13.301 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:30:13.301 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3300", 2024-09-10T21:30:13.301 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:30:13.301 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:30:13.301 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:30:13.301 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:30:13.301 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789", 2024-09-10T21:30:13.301 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:30:13.301 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:30:13.301 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:30:13.301 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:30:13.301 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789/0", 2024-09-10T21:30:13.301 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6789/0", 2024-09-10T21:30:13.302 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:30:13.302 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:30:13.302 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:30:13.302 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:30:13.302 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:30:13.302 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:30:13.302 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:30:13.302 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:30:13.302 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:30:13.302 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:30:13.302 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:30:13.302 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3301", 2024-09-10T21:30:13.302 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:30:13.303 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:30:13.303 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:30:13.303 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:30:13.303 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790", 2024-09-10T21:30:13.303 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:30:13.303 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:30:13.303 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:30:13.303 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:30:13.303 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790/0", 2024-09-10T21:30:13.303 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6790/0", 2024-09-10T21:30:13.303 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:30:13.303 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:30:13.303 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:30:13.304 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:30:13.304 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:30:13.304 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:30:13.304 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-10T21:30:13.304 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-10T21:30:13.304 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:30:13.304 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:30:13.304 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:30:13.304 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:30:13.304 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:30:13.304 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:30:13.304 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": [ 2024-09-10T21:30:13.304 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:30:13.305 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:30:13.305 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:30:13.305 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:30:13.305 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:30:13.305 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:30:13.305 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-10T21:30:13.305 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:30:13.305 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:30:13.305 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:30:13.305 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:30:13.305 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:30:13.305 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:30:13.305 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-10T21:30:13.306 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:30:13.306 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:30:13.306 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:30:13.306 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 4 2024-09-10T21:30:13.306 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:30:13.306 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:30:13.306 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:30:13.306 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-10T21:30:13.306 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-10T21:30:13.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.291+0000 7fe4bf200640 1 -- 172.21.15.17:0/842124997 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fe49c0724e0 msgr2=0x7fe49c0749a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:13.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.291+0000 7fe4bf200640 1 --2- 172.21.15.17:0/842124997 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fe49c0724e0 0x7fe49c0749a0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fe4b40ae000 tx=0x7fe4b4019f60 comp rx=0 tx=0).stop 2024-09-10T21:30:13.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.291+0000 7fe4bf200640 1 -- 172.21.15.17:0/842124997 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe4b8154110 msgr2=0x7fe4b814fa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:13.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.291+0000 7fe4bf200640 1 --2- 172.21.15.17:0/842124997 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe4b8154110 0x7fe4b814fa70 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fe4a8000fe0 tx=0x7fe4a800ea90 comp rx=0 tx=0).stop 2024-09-10T21:30:13.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.291+0000 7fe4be800640 1 -- 172.21.15.17:0/842124997 reap_dead start 2024-09-10T21:30:13.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.291+0000 7fe4be800640 1 -- 172.21.15.17:0/842124997 reap_dead start 2024-09-10T21:30:13.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.291+0000 7fe4bf200640 1 -- 172.21.15.17:0/842124997 shutdown_connections 2024-09-10T21:30:13.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.291+0000 7fe4bf200640 1 -- 172.21.15.17:0/842124997 >> 172.21.15.17:0/842124997 conn(0x7fe4b806fe60 msgr2=0x7fe4b806ed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:13.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.291+0000 7fe4bf200640 1 -- 172.21.15.17:0/842124997 shutdown_connections 2024-09-10T21:30:13.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.291+0000 7fe4bf200640 1 -- 172.21.15.17:0/842124997 wait complete. 2024-09-10T21:30:13.315 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-10T21:30:13.315 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-09-10T21:30:13.315 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-10T21:30:13.488 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.487+0000 7f205ca00640 1 -- 172.21.15.17:0/603066266 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f20580c6530 msgr2=0x7f2058068a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:13.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.487+0000 7f205ca00640 1 --2- 172.21.15.17:0/603066266 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f20580c6530 0x7f2058068a40 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f2050009c80 tx=0x7f205001cda0 comp rx=0 tx=0).stop 2024-09-10T21:30:13.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.487+0000 7f205ca00640 1 -- 172.21.15.17:0/603066266 shutdown_connections 2024-09-10T21:30:13.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.487+0000 7f205ca00640 1 --2- 172.21.15.17:0/603066266 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f20580c6530 0x7f2058068a40 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:13.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.487+0000 7f205ca00640 1 --2- 172.21.15.17:0/603066266 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f20580c6150 0x7f205815b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:13.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.487+0000 7f205ca00640 1 -- 172.21.15.17:0/603066266 >> 172.21.15.17:0/603066266 conn(0x7f205806cc20 msgr2=0x7f205806d030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:13.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.487+0000 7f205ca00640 1 -- 172.21.15.17:0/603066266 shutdown_connections 2024-09-10T21:30:13.490 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.487+0000 7f205ca00640 1 -- 172.21.15.17:0/603066266 wait complete. 2024-09-10T21:30:13.490 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.487+0000 7f205ca00640 1 Processor -- start 2024-09-10T21:30:13.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.487+0000 7f205ca00640 1 -- start start 2024-09-10T21:30:13.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.487+0000 7f205ca00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f20580c37e0 0x7f20581f3c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:13.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.487+0000 7f205ca00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f20580c6150 0x7f2058072bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:13.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.487+0000 7f205ca00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f20580c6530 0x7f205814e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:13.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.487+0000 7f205ca00640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f2058164d40 con 0x7f20580c6530 2024-09-10T21:30:13.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.487+0000 7f205ca00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f2058164bc0 con 0x7f20580c6150 2024-09-10T21:30:13.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.487+0000 7f205ca00640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f2058164ec0 con 0x7f20580c37e0 2024-09-10T21:30:13.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.491+0000 7f2057200640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f20580c6530 0x7f205814e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:13.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.491+0000 7f2055e00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f20580c6150 0x7f2058072bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:13.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.491+0000 7f2056800640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f20580c37e0 0x7f20581f3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:13.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.491+0000 7f2057200640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f20580c6530 0x7f205814e550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:36986/0 (socket says 172.21.15.17:36986) 2024-09-10T21:30:13.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.491+0000 7f2057200640 1 -- 172.21.15.17:0/4110771329 learned_addr learned my addr 172.21.15.17:0/4110771329 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:30:13.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.491+0000 7f2056800640 1 -- 172.21.15.17:0/4110771329 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f20580c37e0 msgr2=0x7f20581f3c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-09-10T21:30:13.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.491+0000 7f2056800640 1 -- 172.21.15.17:0/4110771329 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f20580c37e0 msgr2=0x7f20581f3c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-10T21:30:13.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.491+0000 7f2056800640 1 --2- 172.21.15.17:0/4110771329 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f20580c37e0 0x7f20581f3c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T21:30:13.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.491+0000 7f2057200640 1 -- 172.21.15.17:0/4110771329 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f20580c37e0 msgr2=0x7f20581f3c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:13.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.491+0000 7f2056800640 1 --2- 172.21.15.17:0/4110771329 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f20580c37e0 0x7f20581f3c60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:30:13.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.491+0000 7f2057200640 1 --2- 172.21.15.17:0/4110771329 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f20580c37e0 0x7f20581f3c60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:13.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.491+0000 7f2057200640 1 -- 172.21.15.17:0/4110771329 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f20580c6150 msgr2=0x7f2058072bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:13.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.491+0000 7f2057200640 1 --2- 172.21.15.17:0/4110771329 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f20580c6150 0x7f2058072bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:13.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.491+0000 7f2057200640 1 -- 172.21.15.17:0/4110771329 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2058074340 con 0x7f20580c6530 2024-09-10T21:30:13.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.491+0000 7f2057200640 1 --2- 172.21.15.17:0/4110771329 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f20580c6530 0x7f205814e550 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f205001b450 tx=0x7f2050009350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:13.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.491+0000 7f204ea00640 1 -- 172.21.15.17:0/4110771329 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20500bb070 con 0x7f20580c6530 2024-09-10T21:30:13.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.491+0000 7f204ea00640 1 -- 172.21.15.17:0/4110771329 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2050004c50 con 0x7f20580c6530 2024-09-10T21:30:13.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.491+0000 7f204ea00640 1 -- 172.21.15.17:0/4110771329 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20500b2040 con 0x7f20580c6530 2024-09-10T21:30:13.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.491+0000 7f205ca00640 1 -- 172.21.15.17:0/4110771329 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f205814d590 con 0x7f20580c6530 2024-09-10T21:30:13.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.491+0000 7f205ca00640 1 -- 172.21.15.17:0/4110771329 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20581f3460 con 0x7f20580c6530 2024-09-10T21:30:13.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.491+0000 7f205ca00640 1 --2- 172.21.15.17:0/4110771329 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f20580747e0 0x7f205813e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:13.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.491+0000 7f205ca00640 1 -- 172.21.15.17:0/4110771329 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f205814d340 con 0x7f20580747e0 2024-09-10T21:30:13.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.495+0000 7f2056800640 1 --2- 172.21.15.17:0/4110771329 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f20580747e0 0x7f205813e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:13.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.495+0000 7f2056800640 1 -- 172.21.15.17:0/4110771329 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f205815b820 con 0x7f20580747e0 2024-09-10T21:30:13.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.495+0000 7f2056800640 1 --2- 172.21.15.17:0/4110771329 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f20580747e0 0x7f205813e750 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f204800f900 tx=0x7f204800fdc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:13.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.495+0000 7f204ea00640 1 -- 172.21.15.17:0/4110771329 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f20500b5030 con 0x7f20580c6530 2024-09-10T21:30:13.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.495+0000 7f204ea00640 1 --2- 172.21.15.17:0/4110771329 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f20380725e0 0x7f2038074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:13.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.495+0000 7f204ea00640 1 -- 172.21.15.17:0/4110771329 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f204800dd40 con 0x7f20580747e0 2024-09-10T21:30:13.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.495+0000 7f204ea00640 1 -- 172.21.15.17:0/4110771329 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f205012cee0 con 0x7f20580c6530 2024-09-10T21:30:13.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.495+0000 7f204ea00640 1 -- 172.21.15.17:0/4110771329 <== mon.1 v2:172.21.15.31:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f2048011440 con 0x7f20580747e0 2024-09-10T21:30:13.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.495+0000 7f204ea00640 1 -- 172.21.15.17:0/4110771329 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f20580747e0 msgr2=0x7f205813e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:13.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.495+0000 7f204ea00640 1 --2- 172.21.15.17:0/4110771329 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f20580747e0 0x7f205813e750 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f204800f900 tx=0x7f204800fdc0 comp rx=0 tx=0).stop 2024-09-10T21:30:13.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.495+0000 7f204ea00640 1 -- 172.21.15.17:0/4110771329 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f20580747e0 msgr2=0x7f205813e750 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:30:13.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.495+0000 7f204ea00640 1 --2- 172.21.15.17:0/4110771329 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f20580747e0 0x7f205813e750 secure :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f204800f900 tx=0x7f204800fdc0 comp rx=0 tx=0).stop 2024-09-10T21:30:13.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.495+0000 7f2055e00640 1 --2- 172.21.15.17:0/4110771329 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f20380725e0 0x7f2038074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:13.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.495+0000 7f2055e00640 1 --2- 172.21.15.17:0/4110771329 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f20380725e0 0x7f2038074aa0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f2058068cb0 tx=0x7f2044009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:13.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.503+0000 7f205ca00640 1 --2- 172.21.15.17:0/4110771329 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f205813ec90 0x7f205814ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:13.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.503+0000 7f205ca00640 1 -- 172.21.15.17:0/4110771329 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f20580c8c40 con 0x7f205813ec90 2024-09-10T21:30:13.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.503+0000 7f2056800640 1 --2- 172.21.15.17:0/4110771329 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f205813ec90 0x7f205814ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:13.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.503+0000 7f2056800640 1 -- 172.21.15.17:0/4110771329 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f2058159030 con 0x7f205813ec90 2024-09-10T21:30:13.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.503+0000 7f2056800640 1 --2- 172.21.15.17:0/4110771329 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f205813ec90 0x7f205814ef80 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f204801a000 tx=0x7f20480046d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:13.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.503+0000 7f204ea00640 1 -- 172.21.15.17:0/4110771329 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2048026070 con 0x7f205813ec90 2024-09-10T21:30:13.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.503+0000 7f204ea00640 1 -- 172.21.15.17:0/4110771329 <== mon.1 v2:172.21.15.31:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5042 (secure 0 0 0) 0x7f2048004230 con 0x7f205813ec90 2024-09-10T21:30:13.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.503+0000 7f204ea00640 1 -- 172.21.15.17:0/4110771329 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f205813ec90 msgr2=0x7f205814ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:13.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.503+0000 7f204ea00640 1 --2- 172.21.15.17:0/4110771329 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f205813ec90 0x7f205814ef80 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f204801a000 tx=0x7f20480046d0 comp rx=0 tx=0).stop 2024-09-10T21:30:13.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.503+0000 7f204ea00640 1 -- 172.21.15.17:0/4110771329 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f205813ec90 msgr2=0x7f205814ef80 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:30:13.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.507+0000 7f204ea00640 1 --2- 172.21.15.17:0/4110771329 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f205813ec90 0x7f205814ef80 secure :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f204801a000 tx=0x7f20480046d0 comp rx=0 tx=0).stop 2024-09-10T21:30:13.512 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-10T21:30:13.512 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:30:13.512 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:30:13.512 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-10T21:30:13.513 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 50, 2024-09-10T21:30:13.513 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 351949, 2024-09-10T21:30:13.513 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-10T21:30:13.513 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-10T21:30:13.513 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-10T21:30:13.513 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-10T21:30:13.514 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:30:13.514 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 3, 2024-09-10T21:30:13.514 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:30:13.514 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-10T21:30:13.514 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-10T21:30:13.514 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:30:13.514 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:30:13.514 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:30:13.514 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:30:13.514 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:30:13.514 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:30:13.514 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:30:13.514 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:30:13.515 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:30:13.515 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:30:13.515 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:30:13.515 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:30:13.515 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-10T21:30:13.515 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-10T21:30:13.515 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:30:13.515 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:30:13.515 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:30:13.515 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:30:13.515 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:30:13.515 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:30:13.515 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:30:13.515 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:30:13.516 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:30:13.516 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:30:13.516 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:30:13.516 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:30:13.516 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:30:13.516 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-10T21:30:13.516 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-10T21:30:13.516 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-10T21:30:13.516 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-10T21:30:13.516 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-10T21:30:13.516 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "5e2f023f-d6a6-49f4-a256-f4807dce3739", 2024-09-10T21:30:13.516 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:30:13.516 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:30:13.517 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-10T21:30:13.517 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-10T21:30:13.517 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-10T21:30:13.517 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-10T21:30:13.517 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-10T21:30:13.517 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-10T21:30:13.517 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-10T21:30:13.517 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:30:13.517 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-10T21:30:13.517 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:30:13.517 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:30:13.517 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:30:13.517 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:30:13.517 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:30:13.518 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:30:13.518 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:30:13.518 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:30:13.518 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:30:13.518 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:30:13.518 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:30:13.518 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:30:13.518 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-10T21:30:13.518 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:30:13.518 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-10T21:30:13.518 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:30:13.518 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:30:13.518 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:30:13.519 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:30:13.519 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:30:13.519 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:30:13.519 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:30:13.519 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-10T21:30:13.519 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:30:13.519 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:30:13.519 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:30:13.519 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:30:13.519 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-10T21:30:13.519 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:30:13.519 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:30:13.519 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:30:13.520 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:30:13.520 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-10T21:30:13.520 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-10T21:30:13.520 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:30:13.520 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:30:13.520 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:30:13.520 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:30:13.520 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:30:13.520 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:30:13.520 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:30:13.520 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:30:13.521 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:30:13.521 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:30:13.521 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:30:13.521 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3300", 2024-09-10T21:30:13.521 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:30:13.521 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:30:13.521 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:30:13.521 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:30:13.521 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789", 2024-09-10T21:30:13.522 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:30:13.522 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:30:13.522 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:30:13.522 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:30:13.522 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789/0", 2024-09-10T21:30:13.522 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6789/0", 2024-09-10T21:30:13.522 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:30:13.522 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:30:13.522 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:30:13.522 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:30:13.522 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:30:13.522 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:30:13.522 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:30:13.523 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:30:13.523 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:30:13.523 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:30:13.523 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:30:13.523 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3301", 2024-09-10T21:30:13.523 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:30:13.523 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:30:13.523 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:30:13.523 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:30:13.523 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790", 2024-09-10T21:30:13.523 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:30:13.523 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:30:13.523 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:30:13.524 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:30:13.524 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790/0", 2024-09-10T21:30:13.524 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6790/0", 2024-09-10T21:30:13.524 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:30:13.524 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:30:13.524 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:30:13.524 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:30:13.524 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:30:13.524 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:30:13.524 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-10T21:30:13.524 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-10T21:30:13.524 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:30:13.524 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:30:13.524 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:30:13.525 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:30:13.525 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:30:13.525 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:30:13.525 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": [ 2024-09-10T21:30:13.525 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:30:13.525 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:30:13.525 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:30:13.525 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 5 2024-09-10T21:30:13.525 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:30:13.525 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:30:13.525 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-10T21:30:13.525 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:30:13.525 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:30:13.526 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:30:13.526 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 7 2024-09-10T21:30:13.526 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:30:13.526 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:30:13.526 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-10T21:30:13.526 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:30:13.526 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:30:13.526 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:30:13.526 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 4 2024-09-10T21:30:13.526 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:30:13.526 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:30:13.526 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": [ 2024-09-10T21:30:13.526 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:30:13.526 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:30:13.527 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:30:13.527 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 2 2024-09-10T21:30:13.527 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:30:13.527 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:30:13.527 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:30:13.527 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-10T21:30:13.527 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-10T21:30:13.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.507+0000 7f205ca00640 1 -- 172.21.15.17:0/4110771329 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f20380725e0 msgr2=0x7f2038074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:13.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.507+0000 7f205ca00640 1 --2- 172.21.15.17:0/4110771329 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f20380725e0 0x7f2038074aa0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f2058068cb0 tx=0x7f2044009040 comp rx=0 tx=0).stop 2024-09-10T21:30:13.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.507+0000 7f205ca00640 1 -- 172.21.15.17:0/4110771329 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f20580c6530 msgr2=0x7f205814e550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:13.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.507+0000 7f205ca00640 1 --2- 172.21.15.17:0/4110771329 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f20580c6530 0x7f205814e550 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f205001b450 tx=0x7f2050009350 comp rx=0 tx=0).stop 2024-09-10T21:30:13.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.507+0000 7f2057200640 1 -- 172.21.15.17:0/4110771329 reap_dead start 2024-09-10T21:30:13.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.507+0000 7f2057200640 1 -- 172.21.15.17:0/4110771329 reap_dead start 2024-09-10T21:30:13.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.507+0000 7f205ca00640 1 -- 172.21.15.17:0/4110771329 shutdown_connections 2024-09-10T21:30:13.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.507+0000 7f205ca00640 1 -- 172.21.15.17:0/4110771329 >> 172.21.15.17:0/4110771329 conn(0x7f205806cc20 msgr2=0x7f20580c9440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:13.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.507+0000 7f205ca00640 1 -- 172.21.15.17:0/4110771329 shutdown_connections 2024-09-10T21:30:13.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.507+0000 7f205ca00640 1 -- 172.21.15.17:0/4110771329 wait complete. 2024-09-10T21:30:13.530 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-10T21:30:13.530 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-09-10T21:30:13.531 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-10T21:30:13.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.687+0000 7fa1f7800640 1 -- 172.21.15.17:0/1674294093 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa1d8004810 msgr2=0x7fa1d8004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:13.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.687+0000 7fa1f7800640 1 --2- 172.21.15.17:0/1674294093 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa1d8004810 0x7fa1d8004c90 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa1e400baf0 tx=0x7fa1e401cd80 comp rx=0 tx=0).stop 2024-09-10T21:30:13.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.687+0000 7fa1f5800640 1 -- 172.21.15.17:0/1674294093 reap_dead start 2024-09-10T21:30:13.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.687+0000 7fa1f7800640 1 -- 172.21.15.17:0/1674294093 shutdown_connections 2024-09-10T21:30:13.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.687+0000 7fa1f7800640 1 -- 172.21.15.17:0/1674294093 >> 172.21.15.17:0/1674294093 conn(0x7fa1f00702f0 msgr2=0x7fa1f0070700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:13.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.687+0000 7fa1f7800640 1 -- 172.21.15.17:0/1674294093 shutdown_connections 2024-09-10T21:30:13.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.687+0000 7fa1f7800640 1 -- 172.21.15.17:0/1674294093 wait complete. 2024-09-10T21:30:13.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.687+0000 7fa1f7800640 1 Processor -- start 2024-09-10T21:30:13.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.687+0000 7fa1f7800640 1 -- start start 2024-09-10T21:30:13.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.691+0000 7fa1f7800640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa1d8004810 0x7fa1f01e7140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:13.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.691+0000 7fa1f7800640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa1f00c3570 0x7fa1f006e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:13.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.691+0000 7fa1f7800640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa1f0155fa0 0x7fa1f006e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:13.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.691+0000 7fa1f7800640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fa1f00cb450 con 0x7fa1f0155fa0 2024-09-10T21:30:13.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.691+0000 7fa1f7800640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fa1f00ce740 con 0x7fa1d8004810 2024-09-10T21:30:13.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.691+0000 7fa1f7800640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7fa1f00cb5d0 con 0x7fa1f00c3570 2024-09-10T21:30:13.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.691+0000 7fa1f5800640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa1f0155fa0 0x7fa1f006e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:13.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.691+0000 7fa1efe00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa1f00c3570 0x7fa1f006e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:13.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.691+0000 7fa1f4e00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa1d8004810 0x7fa1f01e7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:13.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.691+0000 7fa1f5800640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa1f0155fa0 0x7fa1f006e700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37006/0 (socket says 172.21.15.17:37006) 2024-09-10T21:30:13.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.691+0000 7fa1f5800640 1 -- 172.21.15.17:0/170285323 learned_addr learned my addr 172.21.15.17:0/170285323 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:30:13.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.691+0000 7fa1efe00640 1 -- 172.21.15.17:0/170285323 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa1f00c3570 msgr2=0x7fa1f006e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-09-10T21:30:13.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.691+0000 7fa1efe00640 1 -- 172.21.15.17:0/170285323 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa1f00c3570 msgr2=0x7fa1f006e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-10T21:30:13.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.691+0000 7fa1efe00640 1 --2- 172.21.15.17:0/170285323 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa1f00c3570 0x7fa1f006e1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T21:30:13.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.691+0000 7fa1efe00640 1 --2- 172.21.15.17:0/170285323 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa1f00c3570 0x7fa1f006e1c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:30:13.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.691+0000 7fa1f5800640 1 -- 172.21.15.17:0/170285323 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa1f00c3570 msgr2=0x7fa1f006e1c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:30:13.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.691+0000 7fa1f5800640 1 --2- 172.21.15.17:0/170285323 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa1f00c3570 0x7fa1f006e1c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:13.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.691+0000 7fa1f5800640 1 -- 172.21.15.17:0/170285323 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa1d8004810 msgr2=0x7fa1f01e7140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:13.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.691+0000 7fa1f5800640 1 --2- 172.21.15.17:0/170285323 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa1d8004810 0x7fa1f01e7140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:13.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.691+0000 7fa1f5800640 1 -- 172.21.15.17:0/170285323 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1f006f4a0 con 0x7fa1f0155fa0 2024-09-10T21:30:13.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.691+0000 7fa1f5800640 1 --2- 172.21.15.17:0/170285323 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa1f0155fa0 0x7fa1f006e700 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fa1e8001340 tx=0x7fa1e8006650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:13.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.691+0000 7fa1ecc00640 1 -- 172.21.15.17:0/170285323 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1e800e990 con 0x7fa1f0155fa0 2024-09-10T21:30:13.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.691+0000 7fa1ecc00640 1 -- 172.21.15.17:0/170285323 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa1e8018070 con 0x7fa1f0155fa0 2024-09-10T21:30:13.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.691+0000 7fa1f7800640 1 -- 172.21.15.17:0/170285323 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1f006f1c0 con 0x7fa1f0155fa0 2024-09-10T21:30:13.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.691+0000 7fa1ecc00640 1 -- 172.21.15.17:0/170285323 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1e8011460 con 0x7fa1f0155fa0 2024-09-10T21:30:13.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.691+0000 7fa1f7800640 1 -- 172.21.15.17:0/170285323 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1f006fbf0 con 0x7fa1f0155fa0 2024-09-10T21:30:13.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.691+0000 7fa1f7800640 1 --2- 172.21.15.17:0/170285323 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa1f01f6d50 0x7fa1f01f7130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:13.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.691+0000 7fa1f7800640 1 -- 172.21.15.17:0/170285323 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7fa1f0157610 con 0x7fa1f01f6d50 2024-09-10T21:30:13.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.695+0000 7fa1f4e00640 1 --2- 172.21.15.17:0/170285323 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa1f01f6d50 0x7fa1f01f7130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:13.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.695+0000 7fa1f4e00640 1 -- 172.21.15.17:0/170285323 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa1f01f6d50 msgr2=0x7fa1f01f7130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 20 2024-09-10T21:30:13.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.695+0000 7fa1f4e00640 1 -- 172.21.15.17:0/170285323 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa1f01f6d50 msgr2=0x7fa1f01f7130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-10T21:30:13.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.695+0000 7fa1f4e00640 1 --2- 172.21.15.17:0/170285323 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa1f01f6d50 0x7fa1f01f7130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T21:30:13.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.695+0000 7fa1f4e00640 1 --2- 172.21.15.17:0/170285323 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa1f01f6d50 0x7fa1f01f7130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:30:13.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.695+0000 7fa1ecc00640 1 -- 172.21.15.17:0/170285323 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fa1e8011600 con 0x7fa1f0155fa0 2024-09-10T21:30:13.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.695+0000 7fa1ecc00640 1 --2- 172.21.15.17:0/170285323 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fa1d00725e0 0x7fa1d0074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:13.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.695+0000 7fa1ecc00640 1 -- 172.21.15.17:0/170285323 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fa1e8093de0 con 0x7fa1f0155fa0 2024-09-10T21:30:13.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.695+0000 7fa1efe00640 1 --2- 172.21.15.17:0/170285323 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fa1d00725e0 0x7fa1d0074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:13.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.695+0000 7fa1efe00640 1 --2- 172.21.15.17:0/170285323 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fa1d00725e0 0x7fa1d0074aa0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fa1e40af000 tx=0x7fa1e4019b20 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:13.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.899+0000 7fa1f4e00640 1 --2- 172.21.15.17:0/170285323 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa1f01f6d50 0x7fa1f01f7130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:13.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.899+0000 7fa1f4e00640 1 -- 172.21.15.17:0/170285323 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa1f00c3160 con 0x7fa1f01f6d50 2024-09-10T21:30:13.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.899+0000 7fa1f4e00640 1 --2- 172.21.15.17:0/170285323 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa1f01f6d50 0x7fa1f01f7130 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fa1dc009690 tx=0x7fa1dc010e60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:13.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.899+0000 7fa1ecc00640 1 -- 172.21.15.17:0/170285323 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1dc020070 con 0x7fa1f01f6d50 2024-09-10T21:30:13.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.899+0000 7fa1ecc00640 1 -- 172.21.15.17:0/170285323 <== mon.2 v2:172.21.15.31:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fa1dc004030 con 0x7fa1f01f6d50 2024-09-10T21:30:13.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.899+0000 7fa1ecc00640 1 -- 172.21.15.17:0/170285323 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa1f01f6d50 msgr2=0x7fa1f01f7130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:13.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.899+0000 7fa1ecc00640 1 --2- 172.21.15.17:0/170285323 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa1f01f6d50 0x7fa1f01f7130 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fa1dc009690 tx=0x7fa1dc010e60 comp rx=0 tx=0).stop 2024-09-10T21:30:13.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.899+0000 7fa1ecc00640 1 -- 172.21.15.17:0/170285323 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa1f01f6d50 msgr2=0x7fa1f01f7130 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:30:13.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.899+0000 7fa1ecc00640 1 --2- 172.21.15.17:0/170285323 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa1f01f6d50 0x7fa1f01f7130 secure :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fa1dc009690 tx=0x7fa1dc010e60 comp rx=0 tx=0).stop 2024-09-10T21:30:13.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.907+0000 7fa1f7800640 1 --2- 172.21.15.17:0/170285323 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa1f01f7670 0x7fa1f0150620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:13.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.907+0000 7fa1f7800640 1 -- 172.21.15.17:0/170285323 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7fa1f006dee0 con 0x7fa1f01f7670 2024-09-10T21:30:13.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.907+0000 7fa1f4e00640 1 --2- 172.21.15.17:0/170285323 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa1f01f7670 0x7fa1f0150620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:13.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.907+0000 7fa1f4e00640 1 -- 172.21.15.17:0/170285323 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fa1f00cceb0 con 0x7fa1f01f7670 2024-09-10T21:30:13.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.907+0000 7fa1f4e00640 1 --2- 172.21.15.17:0/170285323 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa1f01f7670 0x7fa1f0150620 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fa1dc009690 tx=0x7fa1dc0046d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:13.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.907+0000 7fa1ecc00640 1 -- 172.21.15.17:0/170285323 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1dc020050 con 0x7fa1f01f7670 2024-09-10T21:30:13.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.907+0000 7fa1ecc00640 1 -- 172.21.15.17:0/170285323 <== mon.2 v2:172.21.15.31:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4540 (secure 0 0 0) 0x7fa1dc021040 con 0x7fa1f01f7670 2024-09-10T21:30:13.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.907+0000 7fa1ecc00640 1 -- 172.21.15.17:0/170285323 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa1f01f7670 msgr2=0x7fa1f0150620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:13.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.907+0000 7fa1ecc00640 1 --2- 172.21.15.17:0/170285323 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa1f01f7670 0x7fa1f0150620 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fa1dc009690 tx=0x7fa1dc0046d0 comp rx=0 tx=0).stop 2024-09-10T21:30:13.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.907+0000 7fa1ecc00640 1 -- 172.21.15.17:0/170285323 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa1f01f7670 msgr2=0x7fa1f0150620 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:30:13.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.907+0000 7fa1ecc00640 1 --2- 172.21.15.17:0/170285323 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa1f01f7670 0x7fa1f0150620 secure :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fa1dc009690 tx=0x7fa1dc0046d0 comp rx=0 tx=0).stop 2024-09-10T21:30:13.912 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-10T21:30:13.912 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:30:13.912 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:30:13.912 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-10T21:30:13.912 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 50, 2024-09-10T21:30:13.912 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 10219, 2024-09-10T21:30:13.912 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-10T21:30:13.912 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-10T21:30:13.912 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-10T21:30:13.912 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-10T21:30:13.912 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:30:13.913 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 3, 2024-09-10T21:30:13.913 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:30:13.913 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-10T21:30:13.913 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-10T21:30:13.913 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:30:13.913 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:30:13.913 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:30:13.913 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:30:13.913 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:30:13.913 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:30:13.913 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:30:13.913 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:30:13.913 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:30:13.914 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:30:13.914 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:30:13.914 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:30:13.914 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-10T21:30:13.914 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-10T21:30:13.914 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:30:13.914 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:30:13.914 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:30:13.914 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:30:13.914 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:30:13.914 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:30:13.914 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:30:13.914 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:30:13.914 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:30:13.915 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:30:13.915 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:30:13.915 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:30:13.915 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:30:13.915 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-10T21:30:13.915 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-10T21:30:13.915 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-10T21:30:13.915 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-10T21:30:13.915 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-10T21:30:13.915 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "5e2f023f-d6a6-49f4-a256-f4807dce3739", 2024-09-10T21:30:13.915 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:30:13.915 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:30:13.915 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-10T21:30:13.915 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-10T21:30:13.916 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-10T21:30:13.916 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-10T21:30:13.916 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-10T21:30:13.916 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-10T21:30:13.916 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-10T21:30:13.916 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:30:13.916 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-10T21:30:13.916 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:30:13.916 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:30:13.916 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:30:13.916 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:30:13.916 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:30:13.916 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:30:13.916 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:30:13.917 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:30:13.917 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:30:13.917 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:30:13.917 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:30:13.917 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:30:13.917 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-10T21:30:13.917 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:30:13.917 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-10T21:30:13.917 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:30:13.917 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:30:13.917 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:30:13.917 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:30:13.917 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:30:13.917 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:30:13.918 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:30:13.918 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-10T21:30:13.918 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:30:13.918 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:30:13.918 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:30:13.918 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:30:13.918 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-10T21:30:13.918 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:30:13.918 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:30:13.918 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:30:13.918 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:30:13.918 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-10T21:30:13.918 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-10T21:30:13.918 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:30:13.919 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:30:13.919 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:30:13.919 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:30:13.919 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:30:13.919 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:30:13.919 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:30:13.919 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:30:13.919 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:30:13.919 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:30:13.919 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:30:13.919 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3300", 2024-09-10T21:30:13.919 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:30:13.919 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:30:13.919 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:30:13.920 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:30:13.920 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789", 2024-09-10T21:30:13.920 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:30:13.920 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:30:13.920 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:30:13.920 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:30:13.920 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789/0", 2024-09-10T21:30:13.920 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6789/0", 2024-09-10T21:30:13.920 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:30:13.920 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:30:13.920 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:30:13.920 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:30:13.920 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:30:13.920 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:30:13.921 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:30:13.921 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:30:13.921 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:30:13.921 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:30:13.921 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:30:13.921 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3301", 2024-09-10T21:30:13.921 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:30:13.921 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:30:13.921 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:30:13.921 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:30:13.921 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790", 2024-09-10T21:30:13.921 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:30:13.921 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:30:13.922 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:30:13.922 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:30:13.922 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790/0", 2024-09-10T21:30:13.922 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6790/0", 2024-09-10T21:30:13.922 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:30:13.922 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:30:13.922 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:30:13.922 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:30:13.922 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:30:13.922 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:30:13.922 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-10T21:30:13.922 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-10T21:30:13.922 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:30:13.922 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:30:13.923 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:30:13.923 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:30:13.923 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:30:13.923 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:30:13.923 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-10T21:30:13.923 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:30:13.923 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:30:13.923 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:30:13.923 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:30:13.923 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:30:13.923 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:30:13.923 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:30:13.923 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-10T21:30:13.923 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-10T21:30:13.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.907+0000 7fa1f7800640 1 -- 172.21.15.17:0/170285323 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fa1d00725e0 msgr2=0x7fa1d0074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:13.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.907+0000 7fa1f7800640 1 --2- 172.21.15.17:0/170285323 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fa1d00725e0 0x7fa1d0074aa0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fa1e40af000 tx=0x7fa1e4019b20 comp rx=0 tx=0).stop 2024-09-10T21:30:13.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.907+0000 7fa1f7800640 1 -- 172.21.15.17:0/170285323 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa1f0155fa0 msgr2=0x7fa1f006e700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:13.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.907+0000 7fa1f7800640 1 --2- 172.21.15.17:0/170285323 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa1f0155fa0 0x7fa1f006e700 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fa1e8001340 tx=0x7fa1e8006650 comp rx=0 tx=0).stop 2024-09-10T21:30:13.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.907+0000 7fa1f5800640 1 -- 172.21.15.17:0/170285323 reap_dead start 2024-09-10T21:30:13.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.907+0000 7fa1f5800640 1 -- 172.21.15.17:0/170285323 reap_dead start 2024-09-10T21:30:13.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.907+0000 7fa1f7800640 1 -- 172.21.15.17:0/170285323 shutdown_connections 2024-09-10T21:30:13.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.907+0000 7fa1f7800640 1 -- 172.21.15.17:0/170285323 >> 172.21.15.17:0/170285323 conn(0x7fa1f00702f0 msgr2=0x7fa1f00cd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:13.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.907+0000 7fa1f7800640 1 -- 172.21.15.17:0/170285323 shutdown_connections 2024-09-10T21:30:13.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:13.911+0000 7fa1f7800640 1 -- 172.21.15.17:0/170285323 wait complete. 2024-09-10T21:30:13.933 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-10T21:30:13.933 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-09-10T21:30:13.933 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-09-10T21:30:14.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:14.091+0000 7f0cb5800640 1 -- 172.21.15.17:0/2308730426 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0cb0152820 msgr2=0x7f0cb015c280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:14.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:14.091+0000 7f0cb5800640 1 --2- 172.21.15.17:0/2308730426 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0cb0152820 0x7f0cb015c280 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f0cb0073220 tx=0x7f0c9c01c830 comp rx=0 tx=0).stop 2024-09-10T21:30:14.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:14.091+0000 7f0cb5800640 1 -- 172.21.15.17:0/2308730426 shutdown_connections 2024-09-10T21:30:14.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:14.091+0000 7f0cb5800640 1 --2- 172.21.15.17:0/2308730426 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f0cb00cdf20 0x7f0cb00ce350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:14.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:14.091+0000 7f0cb5800640 1 --2- 172.21.15.17:0/2308730426 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0cb0152820 0x7f0cb015c280 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:14.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:14.091+0000 7f0cb5800640 1 --2- 172.21.15.17:0/2308730426 >> v2:172.21.15.31:3300/0 conn(0x7f0cb0152440 0x7f0cb00b3eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:14.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:14.091+0000 7f0cb5800640 1 -- 172.21.15.17:0/2308730426 >> 172.21.15.17:0/2308730426 conn(0x7f0cb00702f0 msgr2=0x7f0cb0070700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:14.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:14.091+0000 7f0cb5800640 1 -- 172.21.15.17:0/2308730426 shutdown_connections 2024-09-10T21:30:14.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:14.091+0000 7f0cb5800640 1 -- 172.21.15.17:0/2308730426 wait complete. 2024-09-10T21:30:14.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:14.091+0000 7f0cb5800640 1 Processor -- start 2024-09-10T21:30:14.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:14.091+0000 7f0cb5800640 1 -- start start 2024-09-10T21:30:14.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:14.095+0000 7f0cb5800640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f0cb00cdf20 0x7f0cb01f37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:14.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:14.095+0000 7f0cb5800640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0cb0152440 0x7f0cb0156af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:14.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:14.095+0000 7f0cb5800640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0cb0152820 0x7f0cb014f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:14.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:14.095+0000 7f0cb5800640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f0cb00cb450 con 0x7f0cb0152440 2024-09-10T21:30:14.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:14.095+0000 7f0cb5800640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f0cb015c7c0 con 0x7f0cb0152820 2024-09-10T21:30:14.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:14.095+0000 7f0cb5800640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f0cb00cb5d0 con 0x7f0cb00cdf20 2024-09-10T21:30:14.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:14.095+0000 7f0cae000640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0cb0152440 0x7f0cb0156af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:14.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:14.095+0000 7f0cae000640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0cb0152440 0x7f0cb0156af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37014/0 (socket says 172.21.15.17:37014) 2024-09-10T21:30:14.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:14.095+0000 7f0cae000640 1 -- 172.21.15.17:0/139858066 learned_addr learned my addr 172.21.15.17:0/139858066 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:30:14.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:14.095+0000 7f0caf400640 1 --2- 172.21.15.17:0/139858066 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0cb0152820 0x7f0cb014f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:14.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:14.095+0000 7f0caea00640 1 --2- 172.21.15.17:0/139858066 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f0cb00cdf20 0x7f0cb01f37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:14.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:14.095+0000 7f0cae000640 1 -- 172.21.15.17:0/139858066 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f0cb00cdf20 msgr2=0x7f0cb01f37d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:14.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:14.095+0000 7f0cae000640 1 --2- 172.21.15.17:0/139858066 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f0cb00cdf20 0x7f0cb01f37d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:14.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:14.095+0000 7f0cae000640 1 -- 172.21.15.17:0/139858066 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0cb0152820 msgr2=0x7f0cb014f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:14.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:14.095+0000 7f0cae000640 1 --2- 172.21.15.17:0/139858066 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0cb0152820 0x7f0cb014f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:14.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:14.095+0000 7f0cae000640 1 -- 172.21.15.17:0/139858066 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cb00cd4d0 con 0x7f0cb0152440 2024-09-10T21:30:14.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:14.095+0000 7f0cae000640 1 --2- 172.21.15.17:0/139858066 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0cb0152440 0x7f0cb0156af0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f0c9c01aee0 tx=0x7f0c9c005e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:14.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:14.095+0000 7f0ca6a00640 1 -- 172.21.15.17:0/139858066 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c9c0b2040 con 0x7f0cb0152440 2024-09-10T21:30:14.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:14.095+0000 7f0ca6a00640 1 -- 172.21.15.17:0/139858066 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0c9c004250 con 0x7f0cb0152440 2024-09-10T21:30:14.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:14.095+0000 7f0cb5800640 1 -- 172.21.15.17:0/139858066 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0cb01621d0 con 0x7f0cb0152440 2024-09-10T21:30:14.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:14.095+0000 7f0ca6a00640 1 -- 172.21.15.17:0/139858066 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c9c005070 con 0x7f0cb0152440 2024-09-10T21:30:14.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:14.095+0000 7f0cb5800640 1 -- 172.21.15.17:0/139858066 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0cb01f2fd0 con 0x7f0cb0152440 2024-09-10T21:30:14.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:14.095+0000 7f0cb5800640 1 -- 172.21.15.17:0/139858066 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0cb0158360 con 0x7f0cb0152440 2024-09-10T21:30:14.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:14.099+0000 7f0ca6a00640 1 -- 172.21.15.17:0/139858066 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f0c9c005210 con 0x7f0cb0152440 2024-09-10T21:30:14.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:14.099+0000 7f0ca6a00640 1 --2- 172.21.15.17:0/139858066 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f0c900725e0 0x7f0c90074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:14.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:14.099+0000 7f0ca6a00640 1 -- 172.21.15.17:0/139858066 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f0c9c12d260 con 0x7f0cb0152440 2024-09-10T21:30:14.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:14.099+0000 7f0caea00640 1 --2- 172.21.15.17:0/139858066 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f0c900725e0 0x7f0c90074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:14.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:14.099+0000 7f0caea00640 1 --2- 172.21.15.17:0/139858066 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f0c900725e0 0x7f0c90074aa0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f0ca0009be0 tx=0x7f0ca00091e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:14.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:14.103+0000 7f0ca6a00640 1 -- 172.21.15.17:0/139858066 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f0c9c0164b0 con 0x7f0cb0152440 2024-09-10T21:30:14.396 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:14.391+0000 7f0cb5800640 1 -- 172.21.15.17:0/139858066 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) -- 0x7f0cb0072f70 con 0x7f0cb0152440 2024-09-10T21:30:14.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:14.395+0000 7f0ca6a00640 1 -- 172.21.15.17:0/139858066 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) ==== 57+0+0 (secure 0 0 0) 0x7f0c9c0f84a0 con 0x7f0cb0152440 2024-09-10T21:30:14.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:14.395+0000 7f0cb5800640 1 -- 172.21.15.17:0/139858066 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f0c900725e0 msgr2=0x7f0c90074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:14.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:14.395+0000 7f0cb5800640 1 --2- 172.21.15.17:0/139858066 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f0c900725e0 0x7f0c90074aa0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f0ca0009be0 tx=0x7f0ca00091e0 comp rx=0 tx=0).stop 2024-09-10T21:30:14.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:14.395+0000 7f0cb5800640 1 -- 172.21.15.17:0/139858066 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0cb0152440 msgr2=0x7f0cb0156af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:14.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:14.395+0000 7f0cb5800640 1 --2- 172.21.15.17:0/139858066 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0cb0152440 0x7f0cb0156af0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f0c9c01aee0 tx=0x7f0c9c005e90 comp rx=0 tx=0).stop 2024-09-10T21:30:14.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:14.395+0000 7f0cb5800640 1 -- 172.21.15.17:0/139858066 shutdown_connections 2024-09-10T21:30:14.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:14.395+0000 7f0cb5800640 1 --2- 172.21.15.17:0/139858066 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f0c900725e0 0x7f0c90074aa0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:14.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:14.395+0000 7f0cb5800640 1 --2- 172.21.15.17:0/139858066 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0cb0152820 0x7f0cb014f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:14.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:14.395+0000 7f0cb5800640 1 --2- 172.21.15.17:0/139858066 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0cb0152440 0x7f0cb0156af0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:14.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:14.395+0000 7f0cb5800640 1 --2- 172.21.15.17:0/139858066 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f0cb00cdf20 0x7f0cb01f37d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:14.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:14.395+0000 7f0cb5800640 1 -- 172.21.15.17:0/139858066 >> 172.21.15.17:0/139858066 conn(0x7f0cb00702f0 msgr2=0x7f0cb0151b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:14.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:14.399+0000 7f0cb5800640 1 -- 172.21.15.17:0/139858066 shutdown_connections 2024-09-10T21:30:14.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:14.399+0000 7f0cb5800640 1 -- 172.21.15.17:0/139858066 wait complete. 2024-09-10T21:30:14.423 INFO:tasks.mon_thrash.mon_thrasher:waiting for 10.0 secs before continuing thrashing 2024-09-10T21:30:17.165 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:30:17.169 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:30:24.425 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-09-10T21:30:24.425 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-10T21:30:24.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:24.579+0000 7fabcd800640 1 -- 172.21.15.17:0/3919583435 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fabc8152440 msgr2=0x7fabc80b3eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:24.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:24.579+0000 7fabcd800640 1 --2- 172.21.15.17:0/3919583435 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fabc8152440 0x7fabc80b3eb0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fabc81594a0 tx=0x7fabb801c850 comp rx=0 tx=0).stop 2024-09-10T21:30:24.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:24.579+0000 7fabcd800640 1 -- 172.21.15.17:0/3919583435 shutdown_connections 2024-09-10T21:30:24.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:24.579+0000 7fabcd800640 1 --2- 172.21.15.17:0/3919583435 >> v2:172.21.15.31:3300/0 conn(0x7fabc80cdf20 0x7fabc80ce350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:24.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:24.579+0000 7fabcd800640 1 --2- 172.21.15.17:0/3919583435 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fabc8152440 0x7fabc80b3eb0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:24.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:24.579+0000 7fabcd800640 1 -- 172.21.15.17:0/3919583435 >> 172.21.15.17:0/3919583435 conn(0x7fabc80702f0 msgr2=0x7fabc8070700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:24.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:24.579+0000 7fabcd800640 1 -- 172.21.15.17:0/3919583435 shutdown_connections 2024-09-10T21:30:24.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:24.579+0000 7fabcd800640 1 -- 172.21.15.17:0/3919583435 wait complete. 2024-09-10T21:30:24.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:24.583+0000 7fabcd800640 1 Processor -- start 2024-09-10T21:30:24.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:24.583+0000 7fabcd800640 1 -- start start 2024-09-10T21:30:24.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:24.583+0000 7fabcd800640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fabc80cdf20 0x7fabc814b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:24.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:24.583+0000 7fabcd800640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fabc8152440 0x7fabc8159c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:24.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:24.583+0000 7fabcd800640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fabc8152820 0x7fabc814d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:24.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:24.583+0000 7fabcd800640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fabc80cb5d0 con 0x7fabc8152440 2024-09-10T21:30:24.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:24.583+0000 7fabcd800640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fabc80cb450 con 0x7fabc80cdf20 2024-09-10T21:30:24.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:24.583+0000 7fabcd800640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7fabc80c2ff0 con 0x7fabc8152820 2024-09-10T21:30:24.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:24.583+0000 7fabc6000640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fabc8152440 0x7fabc8159c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:24.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:24.583+0000 7fabc6000640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fabc8152440 0x7fabc8159c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49192/0 (socket says 172.21.15.17:49192) 2024-09-10T21:30:24.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:24.583+0000 7fabc6000640 1 -- 172.21.15.17:0/4213001145 learned_addr learned my addr 172.21.15.17:0/4213001145 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:30:24.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:24.583+0000 7fabc7400640 1 --2- 172.21.15.17:0/4213001145 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fabc8152820 0x7fabc814d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:24.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:24.583+0000 7fabc6a00640 1 --2- 172.21.15.17:0/4213001145 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fabc80cdf20 0x7fabc814b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:24.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:24.583+0000 7fabc6000640 1 -- 172.21.15.17:0/4213001145 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fabc8152820 msgr2=0x7fabc814d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:24.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:24.583+0000 7fabc6000640 1 --2- 172.21.15.17:0/4213001145 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fabc8152820 0x7fabc814d880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:24.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:24.583+0000 7fabc6000640 1 -- 172.21.15.17:0/4213001145 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fabc80cdf20 msgr2=0x7fabc814b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:24.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:24.583+0000 7fabc6000640 1 --2- 172.21.15.17:0/4213001145 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fabc80cdf20 0x7fabc814b1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:24.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:24.583+0000 7fabc6000640 1 -- 172.21.15.17:0/4213001145 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabc81f6910 con 0x7fabc8152440 2024-09-10T21:30:24.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:24.583+0000 7fabc6a00640 1 --2- 172.21.15.17:0/4213001145 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fabc80cdf20 0x7fabc814b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:30:24.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:24.583+0000 7fabc6000640 1 --2- 172.21.15.17:0/4213001145 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fabc8152440 0x7fabc8159c70 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fabb400c530 tx=0x7fabb400c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:24.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:24.583+0000 7fabbea00640 1 -- 172.21.15.17:0/4213001145 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabb4017070 con 0x7fabc8152440 2024-09-10T21:30:24.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:24.583+0000 7fabbea00640 1 -- 172.21.15.17:0/4213001145 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fabb40053c0 con 0x7fabc8152440 2024-09-10T21:30:24.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:24.583+0000 7fabbea00640 1 -- 172.21.15.17:0/4213001145 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabb4012890 con 0x7fabc8152440 2024-09-10T21:30:24.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:24.583+0000 7fabcd800640 1 -- 172.21.15.17:0/4213001145 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabc806f640 con 0x7fabc8152440 2024-09-10T21:30:24.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:24.583+0000 7fabcd800640 1 -- 172.21.15.17:0/4213001145 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabc806f360 con 0x7fabc8152440 2024-09-10T21:30:24.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:24.587+0000 7fabcd800640 1 -- 172.21.15.17:0/4213001145 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fabc8158360 con 0x7fabc8152440 2024-09-10T21:30:24.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:24.587+0000 7fabbea00640 1 -- 172.21.15.17:0/4213001145 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fabb4003600 con 0x7fabc8152440 2024-09-10T21:30:24.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:24.587+0000 7fabbea00640 1 --2- 172.21.15.17:0/4213001145 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7faba80725e0 0x7faba8074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:24.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:24.587+0000 7fabbea00640 1 -- 172.21.15.17:0/4213001145 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fabb4092c70 con 0x7fabc8152440 2024-09-10T21:30:24.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:24.587+0000 7fabc6a00640 1 --2- 172.21.15.17:0/4213001145 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7faba80725e0 0x7faba8074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:24.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:24.587+0000 7fabc6a00640 1 --2- 172.21.15.17:0/4213001145 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7faba80725e0 0x7faba8074aa0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fabb80af000 tx=0x7fabb801b090 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:24.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:24.591+0000 7fabbea00640 1 -- 172.21.15.17:0/4213001145 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fabb405d860 con 0x7fabc8152440 2024-09-10T21:30:24.853 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:24.851+0000 7fabcd800640 1 -- 172.21.15.17:0/4213001145 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fabc8155d30 con 0x7fabc8152440 2024-09-10T21:30:24.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:24.851+0000 7fabbea00640 1 -- 172.21.15.17:0/4213001145 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1557 (secure 0 0 0) 0x7fabb4062290 con 0x7fabc8152440 2024-09-10T21:30:24.854 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":50,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":14,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5e2f023f-d6a6-49f4-a256-f4807dce3739","modified":"2024-09-10T21:23:46.892130Z","created":"2024-09-10T21:23:46.892130Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3301","nonce":0},{"type":"v1","addr":"172.21.15.31:6790","nonce":0}]},"addr":"172.21.15.31:6790/0","public_addr":"172.21.15.31:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-10T21:30:24.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:24.855+0000 7fabcd800640 1 -- 172.21.15.17:0/4213001145 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7faba80725e0 msgr2=0x7faba8074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:24.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:24.855+0000 7fabcd800640 1 --2- 172.21.15.17:0/4213001145 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7faba80725e0 0x7faba8074aa0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fabb80af000 tx=0x7fabb801b090 comp rx=0 tx=0).stop 2024-09-10T21:30:24.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:24.855+0000 7fabcd800640 1 -- 172.21.15.17:0/4213001145 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fabc8152440 msgr2=0x7fabc8159c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:24.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:24.855+0000 7fabcd800640 1 --2- 172.21.15.17:0/4213001145 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fabc8152440 0x7fabc8159c70 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fabb400c530 tx=0x7fabb400c9f0 comp rx=0 tx=0).stop 2024-09-10T21:30:24.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:24.855+0000 7fabcd800640 1 -- 172.21.15.17:0/4213001145 shutdown_connections 2024-09-10T21:30:24.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:24.855+0000 7fabcd800640 1 --2- 172.21.15.17:0/4213001145 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7faba80725e0 0x7faba8074aa0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:24.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:24.855+0000 7fabcd800640 1 --2- 172.21.15.17:0/4213001145 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fabc8152820 0x7fabc814d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:24.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:24.855+0000 7fabcd800640 1 --2- 172.21.15.17:0/4213001145 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fabc8152440 0x7fabc8159c70 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:24.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:24.855+0000 7fabcd800640 1 --2- 172.21.15.17:0/4213001145 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fabc80cdf20 0x7fabc814b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:24.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:24.855+0000 7fabcd800640 1 -- 172.21.15.17:0/4213001145 >> 172.21.15.17:0/4213001145 conn(0x7fabc80702f0 msgr2=0x7fabc80727f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:24.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:24.855+0000 7fabcd800640 1 -- 172.21.15.17:0/4213001145 shutdown_connections 2024-09-10T21:30:24.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:24.855+0000 7fabcd800640 1 -- 172.21.15.17:0/4213001145 wait complete. 2024-09-10T21:30:24.878 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-09-10T21:30:24.879 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-09-10T21:30:24.879 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-09-10T21:30:24.879 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-10T21:30:25.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.035+0000 7f94c1200640 1 -- 172.21.15.17:0/930558964 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f94bc0725c0 msgr2=0x7f94bc0cdf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:25.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.035+0000 7f94c1200640 1 --2- 172.21.15.17:0/930558964 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f94bc0725c0 0x7f94bc0cdf20 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f94bc073220 tx=0x7f94b401cd50 comp rx=0 tx=0).stop 2024-09-10T21:30:25.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.035+0000 7f94c1200640 1 -- 172.21.15.17:0/930558964 shutdown_connections 2024-09-10T21:30:25.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.035+0000 7f94c1200640 1 --2- 172.21.15.17:0/930558964 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f94bc0725c0 0x7f94bc0cdf20 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:25.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.035+0000 7f94c1200640 1 --2- 172.21.15.17:0/930558964 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f94bc154aa0 0x7f94bc072080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:25.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.035+0000 7f94c1200640 1 --2- 172.21.15.17:0/930558964 >> v2:172.21.15.31:3300/0 conn(0x7f94bc1546c0 0x7f94bc0ca430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:25.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.035+0000 7f94c1200640 1 -- 172.21.15.17:0/930558964 >> 172.21.15.17:0/930558964 conn(0x7f94bc0702f0 msgr2=0x7f94bc070700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:25.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.035+0000 7f94c1200640 1 -- 172.21.15.17:0/930558964 shutdown_connections 2024-09-10T21:30:25.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.035+0000 7f94c1200640 1 -- 172.21.15.17:0/930558964 wait complete. 2024-09-10T21:30:25.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.035+0000 7f94c1200640 1 Processor -- start 2024-09-10T21:30:25.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.035+0000 7f94c1200640 1 -- start start 2024-09-10T21:30:25.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.035+0000 7f94c1200640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f94bc0725c0 0x7f94bc1f4100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:25.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.035+0000 7f94c1200640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f94bc1546c0 0x7f94bc0c6d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:25.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.035+0000 7f94c1200640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f94bc154aa0 0x7f94bc068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:25.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.035+0000 7f94c1200640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f94bc0c97c0 con 0x7f94bc0725c0 2024-09-10T21:30:25.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.035+0000 7f94c1200640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f94bc0c9640 con 0x7f94bc1546c0 2024-09-10T21:30:25.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.035+0000 7f94c1200640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f94bc15e350 con 0x7f94bc154aa0 2024-09-10T21:30:25.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.035+0000 7f94b9e00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f94bc0725c0 0x7f94bc1f4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:25.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.035+0000 7f94b9e00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f94bc0725c0 0x7f94bc1f4100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:48302/0 (socket says 172.21.15.17:48302) 2024-09-10T21:30:25.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.035+0000 7f94b9e00640 1 -- 172.21.15.17:0/651730548 learned_addr learned my addr 172.21.15.17:0/651730548 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:30:25.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.035+0000 7f94b9400640 1 --2- 172.21.15.17:0/651730548 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f94bc1546c0 0x7f94bc0c6d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:25.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.035+0000 7f94ba800640 1 --2- 172.21.15.17:0/651730548 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f94bc154aa0 0x7f94bc068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:25.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.039+0000 7f94b9e00640 1 -- 172.21.15.17:0/651730548 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f94bc154aa0 msgr2=0x7f94bc068bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:25.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.039+0000 7f94b9e00640 1 --2- 172.21.15.17:0/651730548 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f94bc154aa0 0x7f94bc068bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:25.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.039+0000 7f94b9e00640 1 -- 172.21.15.17:0/651730548 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f94bc1546c0 msgr2=0x7f94bc0c6d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:25.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.039+0000 7f94b9e00640 1 --2- 172.21.15.17:0/651730548 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f94bc1546c0 0x7f94bc0c6d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:25.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.039+0000 7f94b9e00640 1 -- 172.21.15.17:0/651730548 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94bc1574e0 con 0x7f94bc0725c0 2024-09-10T21:30:25.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.039+0000 7f94b9e00640 1 --2- 172.21.15.17:0/651730548 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f94bc0725c0 0x7f94bc1f4100 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f94ac00c940 tx=0x7f94ac00ce00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:25.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.039+0000 7f94b2000640 1 -- 172.21.15.17:0/651730548 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94ac007bb0 con 0x7f94bc0725c0 2024-09-10T21:30:25.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.039+0000 7f94b2000640 1 -- 172.21.15.17:0/651730548 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f94ac00f450 con 0x7f94bc0725c0 2024-09-10T21:30:25.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.039+0000 7f94c1200640 1 -- 172.21.15.17:0/651730548 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94bc0c7270 con 0x7f94bc0725c0 2024-09-10T21:30:25.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.039+0000 7f94b2000640 1 -- 172.21.15.17:0/651730548 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94ac007e60 con 0x7f94bc0725c0 2024-09-10T21:30:25.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.039+0000 7f94c1200640 1 -- 172.21.15.17:0/651730548 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94bc1f6df0 con 0x7f94bc0725c0 2024-09-10T21:30:25.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.039+0000 7f94c1200640 1 --2- 172.21.15.17:0/651730548 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f94bc1f7890 0x7f94bc151640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:25.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.039+0000 7f94c1200640 1 -- 172.21.15.17:0/651730548 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f94bc1f4640 con 0x7f94bc1f7890 2024-09-10T21:30:25.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.039+0000 7f94b9400640 1 --2- 172.21.15.17:0/651730548 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f94bc1f7890 0x7f94bc151640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:25.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.039+0000 7f94b9400640 1 -- 172.21.15.17:0/651730548 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f94bc15e7b0 con 0x7f94bc1f7890 2024-09-10T21:30:25.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.039+0000 7f94b9400640 1 --2- 172.21.15.17:0/651730548 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f94bc1f7890 0x7f94bc151640 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f94bc1523b0 tx=0x7f94a800fe60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:25.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.039+0000 7f94b2000640 1 -- 172.21.15.17:0/651730548 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94a800de60 con 0x7f94bc1f7890 2024-09-10T21:30:25.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.039+0000 7f94b2000640 1 -- 172.21.15.17:0/651730548 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f94a8005020 con 0x7f94bc1f7890 2024-09-10T21:30:25.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.039+0000 7f94b2000640 1 -- 172.21.15.17:0/651730548 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f94bc1f7890 msgr2=0x7f94bc151640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:25.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.039+0000 7f94b2000640 1 --2- 172.21.15.17:0/651730548 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f94bc1f7890 0x7f94bc151640 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f94bc1523b0 tx=0x7f94a800fe60 comp rx=0 tx=0).stop 2024-09-10T21:30:25.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.039+0000 7f94b2000640 1 -- 172.21.15.17:0/651730548 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f94bc1f7890 msgr2=0x7f94bc151640 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:30:25.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.039+0000 7f94b2000640 1 --2- 172.21.15.17:0/651730548 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f94bc1f7890 0x7f94bc151640 secure :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f94bc1523b0 tx=0x7f94a800fe60 comp rx=0 tx=0).stop 2024-09-10T21:30:25.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.039+0000 7f94b2000640 1 -- 172.21.15.17:0/651730548 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f94ac020030 con 0x7f94bc0725c0 2024-09-10T21:30:25.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.043+0000 7f94b2000640 1 --2- 172.21.15.17:0/651730548 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f949c0720c0 0x7f949c074580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:25.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.043+0000 7f94b2000640 1 -- 172.21.15.17:0/651730548 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f94ac093bf0 con 0x7f94bc0725c0 2024-09-10T21:30:25.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.043+0000 7f94b9400640 1 --2- 172.21.15.17:0/651730548 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f949c0720c0 0x7f949c074580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:25.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.043+0000 7f94b9400640 1 --2- 172.21.15.17:0/651730548 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f949c0720c0 0x7f949c074580 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f94a800d7b0 tx=0x7f94a800ff90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:25.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.047+0000 7f94c1200640 1 --2- 172.21.15.17:0/651730548 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f94bc151b80 0x7f94bc155ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:25.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.047+0000 7f94c1200640 1 -- 172.21.15.17:0/651730548 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f94bc164c30 con 0x7f94bc151b80 2024-09-10T21:30:25.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.047+0000 7f94ba800640 1 --2- 172.21.15.17:0/651730548 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f94bc151b80 0x7f94bc155ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:25.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.047+0000 7f94ba800640 1 -- 172.21.15.17:0/651730548 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f94bc1579a0 con 0x7f94bc151b80 2024-09-10T21:30:25.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.047+0000 7f94ba800640 1 --2- 172.21.15.17:0/651730548 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f94bc151b80 0x7f94bc155ed0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f94bc0cbac0 tx=0x7f94b40a5d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:25.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.051+0000 7f94b2000640 1 -- 172.21.15.17:0/651730548 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94b40bb070 con 0x7f94bc151b80 2024-09-10T21:30:25.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.051+0000 7f94b2000640 1 -- 172.21.15.17:0/651730548 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4878 (secure 0 0 0) 0x7f94b4003b80 con 0x7f94bc151b80 2024-09-10T21:30:25.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.051+0000 7f94b2000640 1 -- 172.21.15.17:0/651730548 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f94bc151b80 msgr2=0x7f94bc155ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:25.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.051+0000 7f94b2000640 1 --2- 172.21.15.17:0/651730548 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f94bc151b80 0x7f94bc155ed0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f94bc0cbac0 tx=0x7f94b40a5d40 comp rx=0 tx=0).stop 2024-09-10T21:30:25.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.051+0000 7f94b2000640 1 -- 172.21.15.17:0/651730548 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f94bc151b80 msgr2=0x7f94bc155ed0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:30:25.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.051+0000 7f94b2000640 1 --2- 172.21.15.17:0/651730548 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f94bc151b80 0x7f94bc155ed0 secure :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f94bc0cbac0 tx=0x7f94b40a5d40 comp rx=0 tx=0).stop 2024-09-10T21:30:25.053 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-10T21:30:25.053 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:30:25.053 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:30:25.053 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "leader", 2024-09-10T21:30:25.053 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 50, 2024-09-10T21:30:25.053 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 142890, 2024-09-10T21:30:25.054 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-10T21:30:25.054 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-10T21:30:25.054 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-10T21:30:25.054 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-10T21:30:25.054 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:30:25.054 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 14, 2024-09-10T21:30:25.054 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:30:25.054 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-10T21:30:25.054 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-10T21:30:25.054 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:30:25.054 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:30:25.054 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:30:25.055 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:30:25.055 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:30:25.055 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:30:25.055 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:30:25.055 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:30:25.055 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:30:25.055 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:30:25.055 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:30:25.055 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:30:25.055 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-10T21:30:25.055 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-10T21:30:25.055 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:30:25.055 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:30:25.056 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:30:25.056 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:30:25.056 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:30:25.056 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:30:25.056 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:30:25.056 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:30:25.056 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:30:25.056 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:30:25.056 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:30:25.056 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:30:25.056 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:30:25.056 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-10T21:30:25.056 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-10T21:30:25.057 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-10T21:30:25.057 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-10T21:30:25.057 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-10T21:30:25.057 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "5e2f023f-d6a6-49f4-a256-f4807dce3739", 2024-09-10T21:30:25.057 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:30:25.057 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:30:25.057 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-10T21:30:25.057 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-10T21:30:25.057 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-10T21:30:25.057 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-10T21:30:25.057 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-10T21:30:25.057 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-10T21:30:25.058 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-10T21:30:25.058 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:30:25.058 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-10T21:30:25.058 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:30:25.058 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:30:25.058 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:30:25.058 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:30:25.058 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:30:25.058 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:30:25.058 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:30:25.058 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:30:25.058 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:30:25.058 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:30:25.058 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:30:25.059 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:30:25.059 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-10T21:30:25.059 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:30:25.059 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-10T21:30:25.059 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:30:25.059 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:30:25.059 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:30:25.059 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:30:25.059 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:30:25.059 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:30:25.059 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:30:25.059 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-10T21:30:25.060 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:30:25.060 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:30:25.060 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:30:25.060 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:30:25.060 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-10T21:30:25.060 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:30:25.060 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:30:25.060 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:30:25.060 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:30:25.060 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-10T21:30:25.060 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-10T21:30:25.060 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:30:25.060 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:30:25.061 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:30:25.061 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:30:25.061 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:30:25.061 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:30:25.061 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:30:25.061 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:30:25.061 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:30:25.061 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:30:25.061 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:30:25.061 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3300", 2024-09-10T21:30:25.061 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:30:25.061 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:30:25.061 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:30:25.062 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:30:25.062 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789", 2024-09-10T21:30:25.062 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:30:25.062 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:30:25.062 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:30:25.062 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:30:25.062 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789/0", 2024-09-10T21:30:25.062 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6789/0", 2024-09-10T21:30:25.062 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:30:25.062 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:30:25.062 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:30:25.062 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:30:25.062 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:30:25.062 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:30:25.063 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:30:25.063 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:30:25.063 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:30:25.063 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:30:25.063 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:30:25.063 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3301", 2024-09-10T21:30:25.063 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:30:25.063 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:30:25.063 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:30:25.063 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:30:25.063 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790", 2024-09-10T21:30:25.063 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:30:25.063 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:30:25.064 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:30:25.064 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:30:25.064 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790/0", 2024-09-10T21:30:25.064 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6790/0", 2024-09-10T21:30:25.064 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:30:25.064 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:30:25.064 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:30:25.064 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:30:25.064 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:30:25.064 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:30:25.064 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-10T21:30:25.064 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-10T21:30:25.064 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:30:25.064 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:30:25.064 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:30:25.065 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:30:25.065 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:30:25.065 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:30:25.065 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": [ 2024-09-10T21:30:25.065 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:30:25.065 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:30:25.065 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:30:25.065 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:30:25.065 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:30:25.065 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:30:25.065 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-10T21:30:25.065 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:30:25.065 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:30:25.066 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:30:25.066 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:30:25.066 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:30:25.066 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:30:25.066 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-10T21:30:25.066 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:30:25.066 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:30:25.066 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:30:25.066 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 4 2024-09-10T21:30:25.066 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:30:25.066 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:30:25.066 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:30:25.066 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-10T21:30:25.067 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-10T21:30:25.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.051+0000 7f94c1200640 1 -- 172.21.15.17:0/651730548 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f949c0720c0 msgr2=0x7f949c074580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:25.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.051+0000 7f94c1200640 1 --2- 172.21.15.17:0/651730548 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f949c0720c0 0x7f949c074580 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f94a800d7b0 tx=0x7f94a800ff90 comp rx=0 tx=0).stop 2024-09-10T21:30:25.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.051+0000 7f94c1200640 1 -- 172.21.15.17:0/651730548 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f94bc0725c0 msgr2=0x7f94bc1f4100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:25.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.051+0000 7f94c1200640 1 --2- 172.21.15.17:0/651730548 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f94bc0725c0 0x7f94bc1f4100 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f94ac00c940 tx=0x7f94ac00ce00 comp rx=0 tx=0).stop 2024-09-10T21:30:25.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.051+0000 7f94ba800640 1 -- 172.21.15.17:0/651730548 reap_dead start 2024-09-10T21:30:25.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.051+0000 7f94c1200640 1 -- 172.21.15.17:0/651730548 shutdown_connections 2024-09-10T21:30:25.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.051+0000 7f94c1200640 1 -- 172.21.15.17:0/651730548 >> 172.21.15.17:0/651730548 conn(0x7f94bc0702f0 msgr2=0x7f94bc159dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:25.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.051+0000 7f94c1200640 1 -- 172.21.15.17:0/651730548 shutdown_connections 2024-09-10T21:30:25.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.051+0000 7f94c1200640 1 -- 172.21.15.17:0/651730548 wait complete. 2024-09-10T21:30:25.072 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-10T21:30:25.073 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-09-10T21:30:25.073 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-10T21:30:25.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.255+0000 7f0ed3e00640 1 -- 172.21.15.17:0/439108750 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f0ed40c77e0 msgr2=0x7f0eb800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:25.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.255+0000 7f0ed3e00640 1 --2- 172.21.15.17:0/439108750 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f0ed40c77e0 0x7f0eb800b9d0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f0ecc009eb0 tx=0x7f0ecc01d5f0 comp rx=0 tx=0).stop 2024-09-10T21:30:25.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.255+0000 7f0ed3400640 1 -- 172.21.15.17:0/439108750 reap_dead start 2024-09-10T21:30:25.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.255+0000 7f0ed3e00640 1 -- 172.21.15.17:0/439108750 shutdown_connections 2024-09-10T21:30:25.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.255+0000 7f0ed3e00640 1 -- 172.21.15.17:0/439108750 >> 172.21.15.17:0/439108750 conn(0x7f0ed40cd690 msgr2=0x7f0ed40c98e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:25.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.255+0000 7f0ed3e00640 1 -- 172.21.15.17:0/439108750 shutdown_connections 2024-09-10T21:30:25.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.255+0000 7f0ed3e00640 1 -- 172.21.15.17:0/439108750 wait complete. 2024-09-10T21:30:25.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.255+0000 7f0ed3e00640 1 Processor -- start 2024-09-10T21:30:25.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.255+0000 7f0ed3e00640 1 -- start start 2024-09-10T21:30:25.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.255+0000 7f0ed3e00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f0ed40c77e0 0x7f0ed41465c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:25.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.255+0000 7f0ed3e00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ed40c2490 0x7f0ed414e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:25.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.255+0000 7f0ed3e00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0ed40c2e70 0x7f0ed40c32f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:25.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.255+0000 7f0ed3e00640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f0ed406e7a0 con 0x7f0ed40c2e70 2024-09-10T21:30:25.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.255+0000 7f0ed3e00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f0ed40ca800 con 0x7f0ed40c2490 2024-09-10T21:30:25.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.255+0000 7f0ed3e00640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f0ed40cc730 con 0x7f0ed40c77e0 2024-09-10T21:30:25.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.259+0000 7f0ed3400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0ed40c2e70 0x7f0ed40c32f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:25.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.259+0000 7f0ed3400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0ed40c2e70 0x7f0ed40c32f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:48330/0 (socket says 172.21.15.17:48330) 2024-09-10T21:30:25.261 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.259+0000 7f0ed3400640 1 -- 172.21.15.17:0/2581854177 learned_addr learned my addr 172.21.15.17:0/2581854177 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:30:25.261 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.259+0000 7f0ed2a00640 1 --2- 172.21.15.17:0/2581854177 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f0ed40c77e0 0x7f0ed41465c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:25.261 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.259+0000 7f0ed2000640 1 --2- 172.21.15.17:0/2581854177 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ed40c2490 0x7f0ed414e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:25.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.259+0000 7f0ed3400640 1 -- 172.21.15.17:0/2581854177 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f0ed40c77e0 msgr2=0x7f0ed41465c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:25.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.259+0000 7f0ed3400640 1 --2- 172.21.15.17:0/2581854177 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f0ed40c77e0 0x7f0ed41465c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:25.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.259+0000 7f0ed3400640 1 -- 172.21.15.17:0/2581854177 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ed40c2490 msgr2=0x7f0ed414e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:25.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.259+0000 7f0ed3400640 1 --2- 172.21.15.17:0/2581854177 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ed40c2490 0x7f0ed414e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:25.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.259+0000 7f0ed3400640 1 -- 172.21.15.17:0/2581854177 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ed4146b00 con 0x7f0ed40c2e70 2024-09-10T21:30:25.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.259+0000 7f0ed3400640 1 --2- 172.21.15.17:0/2581854177 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0ed40c2e70 0x7f0ed40c32f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f0ecc009eb0 tx=0x7f0ecc0aca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:25.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.259+0000 7f0ecaa00640 1 -- 172.21.15.17:0/2581854177 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ec4001420 con 0x7f0ed40c2e70 2024-09-10T21:30:25.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.259+0000 7f0ecaa00640 1 -- 172.21.15.17:0/2581854177 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0ecc0ba070 con 0x7f0ed40c2e70 2024-09-10T21:30:25.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.259+0000 7f0ecaa00640 1 -- 172.21.15.17:0/2581854177 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ecc009ab0 con 0x7f0ed40c2e70 2024-09-10T21:30:25.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.259+0000 7f0ed3e00640 1 -- 172.21.15.17:0/2581854177 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ed40c3830 con 0x7f0ed40c2e70 2024-09-10T21:30:25.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.259+0000 7f0ed3e00640 1 -- 172.21.15.17:0/2581854177 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ed4145550 con 0x7f0ed40c2e70 2024-09-10T21:30:25.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.259+0000 7f0ed3e00640 1 --2- 172.21.15.17:0/2581854177 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ed4145810 0x7f0ed4150ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:25.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.259+0000 7f0ed3e00640 1 -- 172.21.15.17:0/2581854177 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f0ed4069ef0 con 0x7f0ed4145810 2024-09-10T21:30:25.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.259+0000 7f0ed2a00640 1 --2- 172.21.15.17:0/2581854177 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ed4145810 0x7f0ed4150ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:25.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.259+0000 7f0ed2a00640 1 -- 172.21.15.17:0/2581854177 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0ed4146260 con 0x7f0ed4145810 2024-09-10T21:30:25.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.259+0000 7f0ed2a00640 1 --2- 172.21.15.17:0/2581854177 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ed4145810 0x7f0ed4150ce0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f0ec4001de0 tx=0x7f0ec40039d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:25.264 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.259+0000 7f0ecaa00640 1 -- 172.21.15.17:0/2581854177 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ec4012a30 con 0x7f0ed4145810 2024-09-10T21:30:25.264 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.263+0000 7f0ecaa00640 1 -- 172.21.15.17:0/2581854177 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f0ecc0ab070 con 0x7f0ed40c2e70 2024-09-10T21:30:25.265 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.263+0000 7f0ecaa00640 1 --2- 172.21.15.17:0/2581854177 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f0eb4072510 0x7f0eb40749d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:25.265 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.263+0000 7f0ecaa00640 1 -- 172.21.15.17:0/2581854177 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f0ecc12dfc0 con 0x7f0ed40c2e70 2024-09-10T21:30:25.266 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.263+0000 7f0ecaa00640 1 -- 172.21.15.17:0/2581854177 <== mon.1 v2:172.21.15.31:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f0ec4012bd0 con 0x7f0ed4145810 2024-09-10T21:30:25.266 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.263+0000 7f0ecaa00640 1 -- 172.21.15.17:0/2581854177 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ed4145810 msgr2=0x7f0ed4150ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:25.266 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.263+0000 7f0ecaa00640 1 --2- 172.21.15.17:0/2581854177 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ed4145810 0x7f0ed4150ce0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f0ec4001de0 tx=0x7f0ec40039d0 comp rx=0 tx=0).stop 2024-09-10T21:30:25.266 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.263+0000 7f0ecaa00640 1 -- 172.21.15.17:0/2581854177 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ed4145810 msgr2=0x7f0ed4150ce0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:30:25.266 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.263+0000 7f0ecaa00640 1 --2- 172.21.15.17:0/2581854177 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ed4145810 0x7f0ed4150ce0 secure :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f0ec4001de0 tx=0x7f0ec40039d0 comp rx=0 tx=0).stop 2024-09-10T21:30:25.266 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.263+0000 7f0ed2000640 1 --2- 172.21.15.17:0/2581854177 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f0eb4072510 0x7f0eb40749d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:25.266 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.263+0000 7f0ed2000640 1 --2- 172.21.15.17:0/2581854177 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f0eb4072510 0x7f0eb40749d0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f0ec0000c10 tx=0x7f0ec0000fd0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:25.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.271+0000 7f0ed3e00640 1 --2- 172.21.15.17:0/2581854177 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ed4145bf0 0x7f0ed4151220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:25.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.271+0000 7f0ed3e00640 1 -- 172.21.15.17:0/2581854177 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f0ed406f350 con 0x7f0ed4145bf0 2024-09-10T21:30:25.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.271+0000 7f0ed2a00640 1 --2- 172.21.15.17:0/2581854177 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ed4145bf0 0x7f0ed4151220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:25.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.271+0000 7f0ed2a00640 1 -- 172.21.15.17:0/2581854177 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f0ed40cc510 con 0x7f0ed4145bf0 2024-09-10T21:30:25.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.271+0000 7f0ed2a00640 1 --2- 172.21.15.17:0/2581854177 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ed4145bf0 0x7f0ed4151220 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f0ec4001de0 tx=0x7f0ec4001300 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:25.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.271+0000 7f0ecaa00640 1 -- 172.21.15.17:0/2581854177 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ec400f7f0 con 0x7f0ed4145bf0 2024-09-10T21:30:25.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.271+0000 7f0ecaa00640 1 -- 172.21.15.17:0/2581854177 <== mon.1 v2:172.21.15.31:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5043 (secure 0 0 0) 0x7f0ec400f7f0 con 0x7f0ed4145bf0 2024-09-10T21:30:25.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.271+0000 7f0ecaa00640 1 -- 172.21.15.17:0/2581854177 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ed4145bf0 msgr2=0x7f0ed4151220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:25.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.271+0000 7f0ecaa00640 1 --2- 172.21.15.17:0/2581854177 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ed4145bf0 0x7f0ed4151220 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f0ec4001de0 tx=0x7f0ec4001300 comp rx=0 tx=0).stop 2024-09-10T21:30:25.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.271+0000 7f0ecaa00640 1 -- 172.21.15.17:0/2581854177 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ed4145bf0 msgr2=0x7f0ed4151220 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:30:25.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.271+0000 7f0ecaa00640 1 --2- 172.21.15.17:0/2581854177 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0ed4145bf0 0x7f0ed4151220 secure :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f0ec4001de0 tx=0x7f0ec4001300 comp rx=0 tx=0).stop 2024-09-10T21:30:25.275 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-10T21:30:25.275 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:30:25.275 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:30:25.275 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-10T21:30:25.275 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 50, 2024-09-10T21:30:25.275 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 363717, 2024-09-10T21:30:25.275 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-10T21:30:25.275 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-10T21:30:25.275 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-10T21:30:25.276 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-10T21:30:25.276 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:30:25.276 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 14, 2024-09-10T21:30:25.276 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:30:25.276 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-10T21:30:25.276 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-10T21:30:25.276 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:30:25.276 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:30:25.276 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:30:25.276 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:30:25.276 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:30:25.276 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:30:25.276 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:30:25.277 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:30:25.277 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:30:25.277 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:30:25.277 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:30:25.277 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:30:25.277 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-10T21:30:25.277 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-10T21:30:25.277 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:30:25.277 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:30:25.277 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:30:25.277 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:30:25.277 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:30:25.277 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:30:25.278 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:30:25.278 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:30:25.278 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:30:25.278 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:30:25.278 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:30:25.278 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:30:25.278 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:30:25.278 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-10T21:30:25.278 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-10T21:30:25.278 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-10T21:30:25.278 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-10T21:30:25.278 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-10T21:30:25.278 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "5e2f023f-d6a6-49f4-a256-f4807dce3739", 2024-09-10T21:30:25.279 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:30:25.279 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:30:25.279 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-10T21:30:25.279 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-10T21:30:25.279 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-10T21:30:25.279 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-10T21:30:25.279 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-10T21:30:25.279 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-10T21:30:25.279 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-10T21:30:25.279 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:30:25.279 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-10T21:30:25.279 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:30:25.279 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:30:25.280 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:30:25.280 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:30:25.280 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:30:25.280 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:30:25.280 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:30:25.280 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:30:25.280 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:30:25.280 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:30:25.280 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:30:25.280 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:30:25.280 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-10T21:30:25.280 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:30:25.280 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-10T21:30:25.280 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:30:25.281 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:30:25.281 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:30:25.281 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:30:25.281 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:30:25.281 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:30:25.281 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:30:25.281 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-10T21:30:25.281 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:30:25.281 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:30:25.281 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:30:25.281 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:30:25.281 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-10T21:30:25.281 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:30:25.282 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:30:25.282 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:30:25.282 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:30:25.282 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-10T21:30:25.282 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-10T21:30:25.282 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:30:25.282 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:30:25.282 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:30:25.282 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:30:25.282 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:30:25.282 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:30:25.282 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:30:25.282 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:30:25.282 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:30:25.283 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:30:25.283 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:30:25.283 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3300", 2024-09-10T21:30:25.283 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:30:25.283 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:30:25.283 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:30:25.283 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:30:25.283 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789", 2024-09-10T21:30:25.283 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:30:25.283 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:30:25.283 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:30:25.283 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:30:25.283 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789/0", 2024-09-10T21:30:25.283 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6789/0", 2024-09-10T21:30:25.284 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:30:25.284 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:30:25.284 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:30:25.284 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:30:25.284 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:30:25.284 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:30:25.284 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:30:25.284 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:30:25.284 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:30:25.284 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:30:25.284 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:30:25.284 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3301", 2024-09-10T21:30:25.284 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:30:25.285 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:30:25.285 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:30:25.285 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:30:25.285 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790", 2024-09-10T21:30:25.285 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:30:25.285 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:30:25.285 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:30:25.285 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:30:25.285 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790/0", 2024-09-10T21:30:25.285 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6790/0", 2024-09-10T21:30:25.285 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:30:25.285 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:30:25.285 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:30:25.285 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:30:25.286 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:30:25.286 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:30:25.286 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-10T21:30:25.286 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-10T21:30:25.286 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:30:25.286 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:30:25.286 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:30:25.286 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:30:25.286 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:30:25.286 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:30:25.286 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": [ 2024-09-10T21:30:25.286 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:30:25.286 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:30:25.286 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:30:25.287 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 5 2024-09-10T21:30:25.287 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:30:25.287 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:30:25.287 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-10T21:30:25.287 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:30:25.287 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:30:25.287 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:30:25.287 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 7 2024-09-10T21:30:25.287 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:30:25.287 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:30:25.287 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-10T21:30:25.287 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:30:25.287 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:30:25.287 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:30:25.288 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 4 2024-09-10T21:30:25.288 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:30:25.288 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:30:25.288 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": [ 2024-09-10T21:30:25.288 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:30:25.288 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:30:25.288 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:30:25.288 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 2 2024-09-10T21:30:25.288 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:30:25.288 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:30:25.288 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:30:25.288 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-10T21:30:25.289 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-10T21:30:25.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.271+0000 7f0ed3e00640 1 -- 172.21.15.17:0/2581854177 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f0eb4072510 msgr2=0x7f0eb40749d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:25.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.271+0000 7f0ed3e00640 1 --2- 172.21.15.17:0/2581854177 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f0eb4072510 0x7f0eb40749d0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f0ec0000c10 tx=0x7f0ec0000fd0 comp rx=0 tx=0).stop 2024-09-10T21:30:25.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.271+0000 7f0ed3e00640 1 -- 172.21.15.17:0/2581854177 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0ed40c2e70 msgr2=0x7f0ed40c32f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:25.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.271+0000 7f0ed3400640 1 -- 172.21.15.17:0/2581854177 reap_dead start 2024-09-10T21:30:25.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.271+0000 7f0ed3e00640 1 --2- 172.21.15.17:0/2581854177 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0ed40c2e70 0x7f0ed40c32f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f0ecc009eb0 tx=0x7f0ecc0aca30 comp rx=0 tx=0).stop 2024-09-10T21:30:25.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.271+0000 7f0ed3e00640 1 -- 172.21.15.17:0/2581854177 shutdown_connections 2024-09-10T21:30:25.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.271+0000 7f0ed3e00640 1 --2- 172.21.15.17:0/2581854177 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0ed40c2e70 0x7f0ed40c32f0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:25.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.271+0000 7f0ed3e00640 1 -- 172.21.15.17:0/2581854177 >> 172.21.15.17:0/2581854177 conn(0x7f0ed40cd690 msgr2=0x7f0ed4073900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:25.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.271+0000 7f0ed3e00640 1 -- 172.21.15.17:0/2581854177 shutdown_connections 2024-09-10T21:30:25.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.271+0000 7f0ed3e00640 1 -- 172.21.15.17:0/2581854177 wait complete. 2024-09-10T21:30:25.296 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-10T21:30:25.296 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-09-10T21:30:25.296 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-10T21:30:25.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.463+0000 7ff307000640 1 -- 172.21.15.17:0/2116474879 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff300152460 msgr2=0x7ff3000c8770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:25.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.463+0000 7ff307000640 1 --2- 172.21.15.17:0/2116474879 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff300152460 0x7ff3000c8770 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7ff2f4009840 tx=0x7ff2f401c6a0 comp rx=0 tx=0).stop 2024-09-10T21:30:25.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.463+0000 7ff307000640 1 -- 172.21.15.17:0/2116474879 shutdown_connections 2024-09-10T21:30:25.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.463+0000 7ff307000640 1 --2- 172.21.15.17:0/2116474879 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff30006ec30 0x7ff30006f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:25.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.463+0000 7ff307000640 1 --2- 172.21.15.17:0/2116474879 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff300152460 0x7ff3000c8770 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:25.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.463+0000 7ff307000640 1 -- 172.21.15.17:0/2116474879 >> 172.21.15.17:0/2116474879 conn(0x7ff3000cd690 msgr2=0x7ff3000c98e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:25.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.463+0000 7ff307000640 1 -- 172.21.15.17:0/2116474879 shutdown_connections 2024-09-10T21:30:25.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.463+0000 7ff307000640 1 -- 172.21.15.17:0/2116474879 wait complete. 2024-09-10T21:30:25.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.467+0000 7ff307000640 1 Processor -- start 2024-09-10T21:30:25.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.467+0000 7ff307000640 1 -- start start 2024-09-10T21:30:25.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.467+0000 7ff307000640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff30006ec30 0x7ff300157610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:25.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.467+0000 7ff307000640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff3000c77e0 0x7ff3001f4b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:25.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.467+0000 7ff307000640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff300152460 0x7ff3001f5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:25.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.467+0000 7ff307000640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7ff3000ca770 con 0x7ff3000c77e0 2024-09-10T21:30:25.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.467+0000 7ff307000640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7ff3000ca5f0 con 0x7ff300152460 2024-09-10T21:30:25.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.467+0000 7ff307000640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7ff3000cc730 con 0x7ff30006ec30 2024-09-10T21:30:25.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.467+0000 7ff305200640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff3000c77e0 0x7ff3001f4b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:25.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.467+0000 7ff305200640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff3000c77e0 0x7ff3001f4b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:48342/0 (socket says 172.21.15.17:48342) 2024-09-10T21:30:25.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.467+0000 7ff305200640 1 -- 172.21.15.17:0/583150589 learned_addr learned my addr 172.21.15.17:0/583150589 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:30:25.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.467+0000 7ff306600640 1 --2- 172.21.15.17:0/583150589 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff300152460 0x7ff3001f5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:25.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.467+0000 7ff305c00640 1 --2- 172.21.15.17:0/583150589 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff30006ec30 0x7ff300157610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:25.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.467+0000 7ff305200640 1 -- 172.21.15.17:0/583150589 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff30006ec30 msgr2=0x7ff300157610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:25.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.467+0000 7ff305200640 1 --2- 172.21.15.17:0/583150589 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff30006ec30 0x7ff300157610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:25.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.467+0000 7ff305200640 1 -- 172.21.15.17:0/583150589 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff300152460 msgr2=0x7ff3001f5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:25.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.467+0000 7ff305200640 1 --2- 172.21.15.17:0/583150589 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff300152460 0x7ff3001f5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:25.473 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.467+0000 7ff305200640 1 -- 172.21.15.17:0/583150589 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff30015b730 con 0x7ff3000c77e0 2024-09-10T21:30:25.473 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.467+0000 7ff305200640 1 --2- 172.21.15.17:0/583150589 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff3000c77e0 0x7ff3001f4b10 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7ff2ec00d980 tx=0x7ff2ec00de40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:25.473 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.467+0000 7ff2fd600640 1 -- 172.21.15.17:0/583150589 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2ec017070 con 0x7ff3000c77e0 2024-09-10T21:30:25.473 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.467+0000 7ff2fd600640 1 -- 172.21.15.17:0/583150589 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff2ec009d20 con 0x7ff3000c77e0 2024-09-10T21:30:25.473 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.467+0000 7ff2fd600640 1 -- 172.21.15.17:0/583150589 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2ec004280 con 0x7ff3000c77e0 2024-09-10T21:30:25.473 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.467+0000 7ff307000640 1 -- 172.21.15.17:0/583150589 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff30014d0b0 con 0x7ff3000c77e0 2024-09-10T21:30:25.473 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.467+0000 7ff307000640 1 -- 172.21.15.17:0/583150589 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff30014dfb0 con 0x7ff3000c77e0 2024-09-10T21:30:25.474 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.471+0000 7ff307000640 1 --2- 172.21.15.17:0/583150589 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff300159620 0x7ff300159a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:25.474 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.471+0000 7ff307000640 1 -- 172.21.15.17:0/583150589 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7ff300069ef0 con 0x7ff300159620 2024-09-10T21:30:25.474 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.471+0000 7ff305c00640 1 --2- 172.21.15.17:0/583150589 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff300159620 0x7ff300159a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:25.474 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.471+0000 7ff305c00640 1 -- 172.21.15.17:0/583150589 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff300159f40 con 0x7ff300159620 2024-09-10T21:30:25.474 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.471+0000 7ff2fd600640 1 -- 172.21.15.17:0/583150589 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7ff2ec003600 con 0x7ff3000c77e0 2024-09-10T21:30:25.474 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.471+0000 7ff305c00640 1 --2- 172.21.15.17:0/583150589 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff300159620 0x7ff300159a00 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7ff30006fe70 tx=0x7ff2f4009530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:25.475 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.471+0000 7ff2fd600640 1 --2- 172.21.15.17:0/583150589 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7ff2e00725e0 0x7ff2e0074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:25.475 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.471+0000 7ff2fd600640 1 -- 172.21.15.17:0/583150589 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7ff2ec093790 con 0x7ff3000c77e0 2024-09-10T21:30:25.475 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.471+0000 7ff2fd600640 1 -- 172.21.15.17:0/583150589 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2f40ad040 con 0x7ff300159620 2024-09-10T21:30:25.475 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.471+0000 7ff2fd600640 1 -- 172.21.15.17:0/583150589 <== mon.2 v2:172.21.15.31:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7ff2f4004020 con 0x7ff300159620 2024-09-10T21:30:25.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.471+0000 7ff2fd600640 1 -- 172.21.15.17:0/583150589 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff300159620 msgr2=0x7ff300159a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:25.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.471+0000 7ff2fd600640 1 --2- 172.21.15.17:0/583150589 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff300159620 0x7ff300159a00 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7ff30006fe70 tx=0x7ff2f4009530 comp rx=0 tx=0).stop 2024-09-10T21:30:25.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.471+0000 7ff2fd600640 1 -- 172.21.15.17:0/583150589 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff300159620 msgr2=0x7ff300159a00 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:30:25.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.471+0000 7ff2fd600640 1 --2- 172.21.15.17:0/583150589 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff300159620 0x7ff300159a00 secure :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7ff30006fe70 tx=0x7ff2f4009530 comp rx=0 tx=0).stop 2024-09-10T21:30:25.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.471+0000 7ff306600640 1 --2- 172.21.15.17:0/583150589 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7ff2e00725e0 0x7ff2e0074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:25.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.475+0000 7ff306600640 1 --2- 172.21.15.17:0/583150589 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7ff2e00725e0 0x7ff2e0074aa0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7ff2f80012f0 tx=0x7ff2f8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:25.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.483+0000 7ff307000640 1 --2- 172.21.15.17:0/583150589 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff30013b590 0x7ff30013b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:25.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.483+0000 7ff307000640 1 -- 172.21.15.17:0/583150589 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7ff30014e600 con 0x7ff30013b590 2024-09-10T21:30:25.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.483+0000 7ff305c00640 1 --2- 172.21.15.17:0/583150589 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff30013b590 0x7ff30013b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:25.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.483+0000 7ff305c00640 1 -- 172.21.15.17:0/583150589 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7ff30006f350 con 0x7ff30013b590 2024-09-10T21:30:25.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.483+0000 7ff305c00640 1 --2- 172.21.15.17:0/583150589 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff30013b590 0x7ff30013b970 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7ff2f401a330 tx=0x7ff2f40091e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:25.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.483+0000 7ff2fd600640 1 -- 172.21.15.17:0/583150589 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2f40ad050 con 0x7ff30013b590 2024-09-10T21:30:25.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.483+0000 7ff2fd600640 1 -- 172.21.15.17:0/583150589 <== mon.2 v2:172.21.15.31:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4541 (secure 0 0 0) 0x7ff2f4003a10 con 0x7ff30013b590 2024-09-10T21:30:25.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.483+0000 7ff2fd600640 1 -- 172.21.15.17:0/583150589 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff30013b590 msgr2=0x7ff30013b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:25.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.483+0000 7ff2fd600640 1 --2- 172.21.15.17:0/583150589 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff30013b590 0x7ff30013b970 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7ff2f401a330 tx=0x7ff2f40091e0 comp rx=0 tx=0).stop 2024-09-10T21:30:25.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.483+0000 7ff2fd600640 1 -- 172.21.15.17:0/583150589 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff30013b590 msgr2=0x7ff30013b970 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:30:25.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.483+0000 7ff2fd600640 1 --2- 172.21.15.17:0/583150589 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff30013b590 0x7ff30013b970 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:25.487 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-10T21:30:25.488 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:30:25.488 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:30:25.488 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-10T21:30:25.488 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 50, 2024-09-10T21:30:25.488 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 21795, 2024-09-10T21:30:25.488 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-10T21:30:25.488 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-10T21:30:25.488 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-10T21:30:25.488 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-10T21:30:25.488 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:30:25.489 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 15, 2024-09-10T21:30:25.489 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:30:25.489 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-10T21:30:25.489 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-10T21:30:25.489 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:30:25.489 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:30:25.489 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:30:25.489 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:30:25.489 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:30:25.489 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:30:25.489 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:30:25.489 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:30:25.490 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:30:25.490 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:30:25.490 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:30:25.490 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:30:25.490 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-10T21:30:25.490 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-10T21:30:25.490 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:30:25.490 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:30:25.490 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:30:25.490 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:30:25.490 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:30:25.490 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:30:25.490 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:30:25.491 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:30:25.491 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:30:25.491 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:30:25.491 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:30:25.491 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:30:25.491 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:30:25.491 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-10T21:30:25.491 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-10T21:30:25.491 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-10T21:30:25.491 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-10T21:30:25.491 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-10T21:30:25.491 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "5e2f023f-d6a6-49f4-a256-f4807dce3739", 2024-09-10T21:30:25.491 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:30:25.492 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:30:25.492 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-10T21:30:25.492 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-10T21:30:25.492 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-10T21:30:25.492 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-10T21:30:25.492 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-10T21:30:25.492 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-10T21:30:25.492 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-10T21:30:25.492 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:30:25.492 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-10T21:30:25.492 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:30:25.493 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:30:25.493 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:30:25.493 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:30:25.493 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:30:25.493 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:30:25.493 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:30:25.493 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:30:25.493 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:30:25.493 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:30:25.493 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:30:25.493 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:30:25.493 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-10T21:30:25.493 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:30:25.493 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-10T21:30:25.494 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:30:25.494 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:30:25.494 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:30:25.494 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:30:25.494 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:30:25.494 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:30:25.494 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:30:25.494 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-10T21:30:25.494 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:30:25.494 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:30:25.494 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:30:25.494 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:30:25.495 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-10T21:30:25.495 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:30:25.495 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:30:25.495 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:30:25.495 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:30:25.495 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-10T21:30:25.495 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-10T21:30:25.495 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:30:25.495 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:30:25.495 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:30:25.495 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:30:25.495 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:30:25.495 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:30:25.496 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:30:25.496 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:30:25.496 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:30:25.496 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:30:25.496 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:30:25.496 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3300", 2024-09-10T21:30:25.496 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:30:25.496 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:30:25.496 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:30:25.496 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:30:25.496 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789", 2024-09-10T21:30:25.497 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:30:25.497 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:30:25.497 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:30:25.497 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:30:25.497 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789/0", 2024-09-10T21:30:25.497 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6789/0", 2024-09-10T21:30:25.497 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:30:25.497 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:30:25.497 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:30:25.497 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:30:25.497 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:30:25.497 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:30:25.497 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:30:25.498 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:30:25.498 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:30:25.498 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:30:25.498 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:30:25.498 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3301", 2024-09-10T21:30:25.498 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:30:25.498 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:30:25.498 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:30:25.498 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:30:25.498 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790", 2024-09-10T21:30:25.498 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:30:25.498 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:30:25.498 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:30:25.498 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:30:25.499 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790/0", 2024-09-10T21:30:25.499 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6790/0", 2024-09-10T21:30:25.499 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:30:25.499 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:30:25.499 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:30:25.499 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:30:25.499 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:30:25.499 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:30:25.499 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-10T21:30:25.499 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-10T21:30:25.499 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:30:25.499 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:30:25.499 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:30:25.500 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:30:25.500 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:30:25.500 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:30:25.500 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-10T21:30:25.500 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:30:25.500 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:30:25.500 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:30:25.500 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:30:25.500 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:30:25.500 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:30:25.500 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:30:25.500 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-10T21:30:25.500 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-10T21:30:25.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.483+0000 7ff307000640 1 -- 172.21.15.17:0/583150589 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7ff2e00725e0 msgr2=0x7ff2e0074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:25.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.483+0000 7ff307000640 1 --2- 172.21.15.17:0/583150589 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7ff2e00725e0 0x7ff2e0074aa0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7ff2f80012f0 tx=0x7ff2f8009040 comp rx=0 tx=0).stop 2024-09-10T21:30:25.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.483+0000 7ff307000640 1 -- 172.21.15.17:0/583150589 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff3000c77e0 msgr2=0x7ff3001f4b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:25.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.483+0000 7ff307000640 1 --2- 172.21.15.17:0/583150589 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff3000c77e0 0x7ff3001f4b10 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7ff2ec00d980 tx=0x7ff2ec00de40 comp rx=0 tx=0).stop 2024-09-10T21:30:25.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.483+0000 7ff306600640 1 -- 172.21.15.17:0/583150589 reap_dead start 2024-09-10T21:30:25.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.483+0000 7ff306600640 1 -- 172.21.15.17:0/583150589 reap_dead start 2024-09-10T21:30:25.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.483+0000 7ff307000640 1 -- 172.21.15.17:0/583150589 shutdown_connections 2024-09-10T21:30:25.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.483+0000 7ff307000640 1 -- 172.21.15.17:0/583150589 >> 172.21.15.17:0/583150589 conn(0x7ff3000cd690 msgr2=0x7ff3000733e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:25.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.483+0000 7ff307000640 1 -- 172.21.15.17:0/583150589 shutdown_connections 2024-09-10T21:30:25.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:25.483+0000 7ff307000640 1 -- 172.21.15.17:0/583150589 wait complete. 2024-09-10T21:30:25.510 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-10T21:30:25.510 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['c'] 2024-09-10T21:30:25.510 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: ['b'] 2024-09-10T21:30:25.510 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.c 2024-09-10T21:30:25.510 INFO:tasks.mon_thrash.mon_thrasher:killing mon.c 2024-09-10T21:30:25.510 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-10T21:30:25.510 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:30:25.527 INFO:tasks.ceph.mon.c:Stopped 2024-09-10T21:30:25.527 INFO:tasks.mon_thrash:Sending STOP to mon b 2024-09-10T21:30:25.527 INFO:tasks.ceph.mon.b:Sent signal 19 2024-09-10T21:30:25.527 INFO:tasks.mon_thrash.mon_thrasher:waiting for 15.0 secs to unfreeze mons 2024-09-10T21:30:40.529 INFO:tasks.mon_thrash:Sending CONT to mon b 2024-09-10T21:30:40.639 INFO:tasks.ceph.mon.b:Sent signal 18 2024-09-10T21:30:40.640 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2024-09-10T21:30:40.640 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-10T21:30:46.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:46.795+0000 7f02fd600640 1 -- 172.21.15.17:0/3946767949 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f02d4003b60 msgr2=0x7f02d40065e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:46.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:46.795+0000 7f02fd600640 1 --2- 172.21.15.17:0/3946767949 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f02d4003b60 0x7f02d40065e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f02f80cb970 tx=0x7f02e001e470 comp rx=0 tx=0).stop 2024-09-10T21:30:46.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:46.795+0000 7f02fd600640 1 -- 172.21.15.17:0/3946767949 shutdown_connections 2024-09-10T21:30:46.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:46.795+0000 7f02fd600640 1 --2- 172.21.15.17:0/3946767949 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f02d4006b20 0x7f02d4006fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:46.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:46.795+0000 7f02fd600640 1 --2- 172.21.15.17:0/3946767949 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f02d4003b60 0x7f02d40065e0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:46.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:46.795+0000 7f02fd600640 1 --2- 172.21.15.17:0/3946767949 >> v2:172.21.15.31:3300/0 conn(0x7f02f8152820 0x7f02d400b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:46.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:46.795+0000 7f02fd600640 1 -- 172.21.15.17:0/3946767949 >> 172.21.15.17:0/3946767949 conn(0x7f02f80702f0 msgr2=0x7f02f8070700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:46.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:46.795+0000 7f02fd600640 1 -- 172.21.15.17:0/3946767949 shutdown_connections 2024-09-10T21:30:46.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:46.795+0000 7f02fd600640 1 -- 172.21.15.17:0/3946767949 wait complete. 2024-09-10T21:30:46.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:46.799+0000 7f02fd600640 1 Processor -- start 2024-09-10T21:30:46.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:46.799+0000 7f02fd600640 1 -- start start 2024-09-10T21:30:46.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:46.799+0000 7f02fd600640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f02d4003b60 0x7f02f815c450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:46.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:46.799+0000 7f02fd600640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f02d4006b20 0x7f02f81547d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:46.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:46.799+0000 7f02fd600640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f02f8152820 0x7f02f8154d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:46.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:46.799+0000 7f02fd600640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f02f80ce740 con 0x7f02f8152820 2024-09-10T21:30:46.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:46.799+0000 7f02fd600640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f02f815bdb0 con 0x7f02d4006b20 2024-09-10T21:30:46.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:46.799+0000 7f02fd600640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f02f80cb450 con 0x7f02d4003b60 2024-09-10T21:30:46.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:46.799+0000 7f02f6a00640 1 -- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f02d4003b60 msgr2=0x7f02f815c450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:3301/0 2024-09-10T21:30:46.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:46.799+0000 7f02f6a00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f02d4003b60 0x7f02f815c450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:30:46.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:46.799+0000 7f02f7400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f02f8152820 0x7f02f8154d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:46.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:46.799+0000 7f02ee000640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f02d4006b20 0x7f02f81547d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:46.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:46.799+0000 7f02f7400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f02f8152820 0x7f02f8154d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:52612/0 (socket says 172.21.15.17:52612) 2024-09-10T21:30:46.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:46.799+0000 7f02f7400640 1 -- 172.21.15.17:0/1526332991 learned_addr learned my addr 172.21.15.17:0/1526332991 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:30:46.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:46.799+0000 7f02f7400640 1 -- 172.21.15.17:0/1526332991 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f02d4003b60 msgr2=0x7f02f815c450 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:30:46.806 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:46.799+0000 7f02f7400640 1 --2- 172.21.15.17:0/1526332991 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f02d4003b60 0x7f02f815c450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:46.806 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:46.799+0000 7f02f7400640 1 -- 172.21.15.17:0/1526332991 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f02d4006b20 msgr2=0x7f02f81547d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:46.806 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:46.799+0000 7f02f7400640 1 --2- 172.21.15.17:0/1526332991 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f02d4006b20 0x7f02f81547d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:46.806 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:46.799+0000 7f02f7400640 1 -- 172.21.15.17:0/1526332991 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02f8157b00 con 0x7f02f8152820 2024-09-10T21:30:46.806 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:46.799+0000 7f02f7400640 1 --2- 172.21.15.17:0/1526332991 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f02f8152820 0x7f02f8154d10 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f02f80731f0 tx=0x7f02f000ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:46.806 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:46.799+0000 7f02ef400640 1 -- 172.21.15.17:0/1526332991 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02f0012830 con 0x7f02f8152820 2024-09-10T21:30:46.806 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:46.799+0000 7f02ef400640 1 -- 172.21.15.17:0/1526332991 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f02f0018070 con 0x7f02f8152820 2024-09-10T21:30:46.806 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:46.799+0000 7f02ef400640 1 -- 172.21.15.17:0/1526332991 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02f0012d40 con 0x7f02f8152820 2024-09-10T21:30:46.806 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:46.803+0000 7f02fd600640 1 -- 172.21.15.17:0/1526332991 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02f81f58e0 con 0x7f02f8152820 2024-09-10T21:30:46.806 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:46.803+0000 7f02fd600640 1 -- 172.21.15.17:0/1526332991 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02f8158080 con 0x7f02f8152820 2024-09-10T21:30:46.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:46.803+0000 7f02fd600640 1 -- 172.21.15.17:0/1526332991 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02f80c30d0 con 0x7f02f8152820 2024-09-10T21:30:46.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:46.803+0000 7f02ef400640 1 -- 172.21.15.17:0/1526332991 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f02f001d460 con 0x7f02f8152820 2024-09-10T21:30:46.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:46.803+0000 7f02ef400640 1 --2- 172.21.15.17:0/1526332991 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f02d4073fc0 0x7f02d4076480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:46.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:46.803+0000 7f02ef400640 1 -- 172.21.15.17:0/1526332991 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f02f00933a0 con 0x7f02f8152820 2024-09-10T21:30:46.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:46.807+0000 7f02f6a00640 1 --2- 172.21.15.17:0/1526332991 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f02d4073fc0 0x7f02d4076480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:46.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:46.807+0000 7f02f6a00640 1 --2- 172.21.15.17:0/1526332991 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f02d4073fc0 0x7f02d4076480 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f02f815afe0 tx=0x7f02e8001c60 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:46.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:46.807+0000 7f02ef400640 1 -- 172.21.15.17:0/1526332991 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f02f005e5f0 con 0x7f02f8152820 2024-09-10T21:30:47.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:47.111+0000 7f02fd600640 1 -- 172.21.15.17:0/1526332991 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f02f8159590 con 0x7f02f8152820 2024-09-10T21:30:47.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:47.111+0000 7f02ef400640 1 -- 172.21.15.17:0/1526332991 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1550 (secure 0 0 0) 0x7f02f005def0 con 0x7f02f8152820 2024-09-10T21:30:47.114 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":52,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":1,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5e2f023f-d6a6-49f4-a256-f4807dce3739","modified":"2024-09-10T21:23:46.892130Z","created":"2024-09-10T21:23:46.892130Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3301","nonce":0},{"type":"v1","addr":"172.21.15.31:6790","nonce":0}]},"addr":"172.21.15.31:6790/0","public_addr":"172.21.15.31:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-10T21:30:47.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:47.111+0000 7f02fd600640 1 -- 172.21.15.17:0/1526332991 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f02d4073fc0 msgr2=0x7f02d4076480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:47.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:47.111+0000 7f02fd600640 1 --2- 172.21.15.17:0/1526332991 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f02d4073fc0 0x7f02d4076480 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f02f815afe0 tx=0x7f02e8001c60 comp rx=0 tx=0).stop 2024-09-10T21:30:47.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:47.111+0000 7f02fd600640 1 -- 172.21.15.17:0/1526332991 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f02f8152820 msgr2=0x7f02f8154d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:47.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:47.111+0000 7f02fd600640 1 --2- 172.21.15.17:0/1526332991 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f02f8152820 0x7f02f8154d10 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f02f80731f0 tx=0x7f02f000ca70 comp rx=0 tx=0).stop 2024-09-10T21:30:47.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:47.111+0000 7f02fd600640 1 -- 172.21.15.17:0/1526332991 shutdown_connections 2024-09-10T21:30:47.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:47.111+0000 7f02fd600640 1 --2- 172.21.15.17:0/1526332991 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f02d4073fc0 0x7f02d4076480 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:47.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:47.111+0000 7f02fd600640 1 --2- 172.21.15.17:0/1526332991 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f02f8152820 0x7f02f8154d10 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:47.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:47.111+0000 7f02fd600640 1 --2- 172.21.15.17:0/1526332991 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f02d4006b20 0x7f02f81547d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:47.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:47.111+0000 7f02fd600640 1 --2- 172.21.15.17:0/1526332991 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f02d4003b60 0x7f02f815c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:47.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:47.111+0000 7f02fd600640 1 -- 172.21.15.17:0/1526332991 >> 172.21.15.17:0/1526332991 conn(0x7f02f80702f0 msgr2=0x7f02f80ca500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:47.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:47.111+0000 7f02fd600640 1 -- 172.21.15.17:0/1526332991 shutdown_connections 2024-09-10T21:30:47.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:47.111+0000 7f02fd600640 1 -- 172.21.15.17:0/1526332991 wait complete. 2024-09-10T21:30:47.137 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2024-09-10T21:30:47.137 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon a 2024-09-10T21:30:47.137 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-10T21:30:47.230 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:30:47.233 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:30:47.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:47.299+0000 7f48dd200640 1 -- 172.21.15.17:0/2036582676 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f48d8152820 msgr2=0x7f48d80b3eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:47.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:47.299+0000 7f48dd200640 1 --2- 172.21.15.17:0/2036582676 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f48d8152820 0x7f48d80b3eb0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f48c4009900 tx=0x7f48c401c870 comp rx=0 tx=0).stop 2024-09-10T21:30:47.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:47.299+0000 7f48dd200640 1 -- 172.21.15.17:0/2036582676 shutdown_connections 2024-09-10T21:30:47.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:47.299+0000 7f48dd200640 1 --2- 172.21.15.17:0/2036582676 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f48d80cdf20 0x7f48d80ce350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:47.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:47.299+0000 7f48dd200640 1 --2- 172.21.15.17:0/2036582676 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f48d8152820 0x7f48d80b3eb0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:47.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:47.299+0000 7f48dd200640 1 -- 172.21.15.17:0/2036582676 >> 172.21.15.17:0/2036582676 conn(0x7f48d80702f0 msgr2=0x7f48d8070700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:47.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:47.299+0000 7f48dd200640 1 -- 172.21.15.17:0/2036582676 shutdown_connections 2024-09-10T21:30:47.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:47.299+0000 7f48dd200640 1 -- 172.21.15.17:0/2036582676 wait complete. 2024-09-10T21:30:47.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:47.303+0000 7f48dd200640 1 Processor -- start 2024-09-10T21:30:47.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:47.303+0000 7f48dd200640 1 -- start start 2024-09-10T21:30:47.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:47.303+0000 7f48dd200640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f48d80cdf20 0x7f48d806f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:47.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:47.303+0000 7f48dd200640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f48d8152440 0x7f48d8141730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:47.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:47.303+0000 7f48dd200640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48d8152820 0x7f48d814aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:47.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:47.303+0000 7f48dd200640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f48d80cb5d0 con 0x7f48d8152440 2024-09-10T21:30:47.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:47.303+0000 7f48dd200640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f48d80cb450 con 0x7f48d8152820 2024-09-10T21:30:47.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:47.303+0000 7f48dd200640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f48d80c2ff0 con 0x7f48d80cdf20 2024-09-10T21:30:47.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:47.303+0000 7f48d5e00640 1 -- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f48d80cdf20 msgr2=0x7f48d806f630 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:3301/0 2024-09-10T21:30:47.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:47.303+0000 7f48d5400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f48d8152440 0x7f48d8141730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:47.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:47.303+0000 7f48d5e00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f48d80cdf20 0x7f48d806f630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:30:47.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:47.303+0000 7f48d5400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f48d8152440 0x7f48d8141730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:52634/0 (socket says 172.21.15.17:52634) 2024-09-10T21:30:47.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:47.303+0000 7f48d5400640 1 -- 172.21.15.17:0/3707266598 learned_addr learned my addr 172.21.15.17:0/3707266598 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:30:47.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:47.303+0000 7f48d6800640 1 --2- 172.21.15.17:0/3707266598 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48d8152820 0x7f48d814aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:47.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:47.303+0000 7f48d5400640 1 -- 172.21.15.17:0/3707266598 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f48d80cdf20 msgr2=0x7f48d806f630 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:30:47.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:47.303+0000 7f48d5400640 1 --2- 172.21.15.17:0/3707266598 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f48d80cdf20 0x7f48d806f630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:47.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:47.303+0000 7f48d5400640 1 -- 172.21.15.17:0/3707266598 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48d8152820 msgr2=0x7f48d814aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:47.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:47.303+0000 7f48d5400640 1 --2- 172.21.15.17:0/3707266598 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48d8152820 0x7f48d814aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:47.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:47.303+0000 7f48d5400640 1 -- 172.21.15.17:0/3707266598 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48d814af70 con 0x7f48d8152440 2024-09-10T21:30:47.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:47.303+0000 7f48d6800640 1 --2- 172.21.15.17:0/3707266598 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48d8152820 0x7f48d814aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:30:47.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:47.303+0000 7f48d5400640 1 --2- 172.21.15.17:0/3707266598 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f48d8152440 0x7f48d8141730 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f48c40b0040 tx=0x7f48c4004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:47.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:47.303+0000 7f48ce000640 1 -- 172.21.15.17:0/3707266598 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48c40b2040 con 0x7f48d8152440 2024-09-10T21:30:47.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:47.303+0000 7f48ce000640 1 -- 172.21.15.17:0/3707266598 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f48c4005020 con 0x7f48d8152440 2024-09-10T21:30:47.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:47.303+0000 7f48ce000640 1 -- 172.21.15.17:0/3707266598 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48c4005470 con 0x7f48d8152440 2024-09-10T21:30:47.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:47.303+0000 7f48dd200640 1 -- 172.21.15.17:0/3707266598 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48d81f3880 con 0x7f48d8152440 2024-09-10T21:30:47.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:47.303+0000 7f48dd200640 1 -- 172.21.15.17:0/3707266598 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48d81f3e40 con 0x7f48d8152440 2024-09-10T21:30:47.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:47.307+0000 7f48dd200640 1 --2- 172.21.15.17:0/3707266598 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f48d806e740 0x7f48d814ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:47.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:47.307+0000 7f48dd200640 1 -- 172.21.15.17:0/3707266598 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f48d8158360 con 0x7f48d806e740 2024-09-10T21:30:47.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:47.307+0000 7f48d5e00640 1 --2- 172.21.15.17:0/3707266598 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f48d806e740 0x7f48d814ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:47.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:47.307+0000 7f48d5e00640 1 -- 172.21.15.17:0/3707266598 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f48d8142220 con 0x7f48d806e740 2024-09-10T21:30:47.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:47.307+0000 7f48d5e00640 1 --2- 172.21.15.17:0/3707266598 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f48d806e740 0x7f48d814ccf0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f48c8000bd0 tx=0x7f48c800c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:47.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:47.307+0000 7f48ce000640 1 -- 172.21.15.17:0/3707266598 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f48c40046d0 con 0x7f48d8152440 2024-09-10T21:30:47.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:47.307+0000 7f48ce000640 1 --2- 172.21.15.17:0/3707266598 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f48b8072590 0x7f48b8074a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:47.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:47.307+0000 7f48ce000640 1 -- 172.21.15.17:0/3707266598 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48c80074e0 con 0x7f48d806e740 2024-09-10T21:30:47.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:47.307+0000 7f48ce000640 1 -- 172.21.15.17:0/3707266598 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f48c412cce0 con 0x7f48d8152440 2024-09-10T21:30:47.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:47.307+0000 7f48ce000640 1 -- 172.21.15.17:0/3707266598 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f48c8004020 con 0x7f48d806e740 2024-09-10T21:30:47.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:47.307+0000 7f48ce000640 1 -- 172.21.15.17:0/3707266598 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f48d806e740 msgr2=0x7f48d814ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:47.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:47.307+0000 7f48ce000640 1 --2- 172.21.15.17:0/3707266598 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f48d806e740 0x7f48d814ccf0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f48c8000bd0 tx=0x7f48c800c830 comp rx=0 tx=0).stop 2024-09-10T21:30:47.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:47.307+0000 7f48ce000640 1 -- 172.21.15.17:0/3707266598 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f48d806e740 msgr2=0x7f48d814ccf0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:30:47.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:47.307+0000 7f48ce000640 1 --2- 172.21.15.17:0/3707266598 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f48d806e740 0x7f48d814ccf0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:47.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:47.307+0000 7f48d6800640 1 --2- 172.21.15.17:0/3707266598 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f48b8072590 0x7f48b8074a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:47.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:47.311+0000 7f48d6800640 1 --2- 172.21.15.17:0/3707266598 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f48b8072590 0x7f48b8074a50 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f48d80c3d30 tx=0x7f48d0009510 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:47.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:47.315+0000 7f48dd200640 1 --2- 172.21.15.17:0/3707266598 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f48d814d230 0x7f48d8149190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:47.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:47.315+0000 7f48dd200640 1 -- 172.21.15.17:0/3707266598 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f48d814d640 con 0x7f48d814d230 2024-09-10T21:30:47.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:47.315+0000 7f48d5e00640 1 --2- 172.21.15.17:0/3707266598 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f48d814d230 0x7f48d8149190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:47.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:47.315+0000 7f48d5e00640 1 -- 172.21.15.17:0/3707266598 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f48d80ca9e0 con 0x7f48d814d230 2024-09-10T21:30:47.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:47.315+0000 7f48d5e00640 1 --2- 172.21.15.17:0/3707266598 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f48d814d230 0x7f48d8149190 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f48c8013000 tx=0x7f48c8000ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:47.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:47.315+0000 7f48ce000640 1 -- 172.21.15.17:0/3707266598 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48c8019670 con 0x7f48d814d230 2024-09-10T21:30:47.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:47.319+0000 7f48ce000640 1 -- 172.21.15.17:0/3707266598 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4866 (secure 0 0 0) 0x7f48c8019670 con 0x7f48d814d230 2024-09-10T21:30:47.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:47.319+0000 7f48ce000640 1 -- 172.21.15.17:0/3707266598 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f48d814d230 msgr2=0x7f48d8149190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:47.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:47.319+0000 7f48ce000640 1 --2- 172.21.15.17:0/3707266598 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f48d814d230 0x7f48d8149190 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f48c8013000 tx=0x7f48c8000ee0 comp rx=0 tx=0).stop 2024-09-10T21:30:47.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:47.319+0000 7f48ce000640 1 -- 172.21.15.17:0/3707266598 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f48d814d230 msgr2=0x7f48d8149190 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:30:47.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:47.319+0000 7f48ce000640 1 --2- 172.21.15.17:0/3707266598 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f48d814d230 0x7f48d8149190 secure :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f48c8013000 tx=0x7f48c8000ee0 comp rx=0 tx=0).stop 2024-09-10T21:30:47.321 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-10T21:30:47.321 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:30:47.321 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:30:47.321 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "leader", 2024-09-10T21:30:47.321 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 52, 2024-09-10T21:30:47.321 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 165154, 2024-09-10T21:30:47.321 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-10T21:30:47.321 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-10T21:30:47.322 INFO:teuthology.orchestra.run.smithi017.stdout: 1 2024-09-10T21:30:47.322 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:30:47.322 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 1, 2024-09-10T21:30:47.322 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:30:47.322 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-10T21:30:47.322 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-10T21:30:47.322 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:30:47.322 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:30:47.322 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:30:47.322 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:30:47.322 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:30:47.322 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:30:47.322 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:30:47.322 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:30:47.323 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:30:47.323 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:30:47.323 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:30:47.323 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:30:47.323 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-10T21:30:47.323 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-10T21:30:47.323 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:30:47.323 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:30:47.323 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:30:47.323 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:30:47.323 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:30:47.323 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:30:47.323 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:30:47.323 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:30:47.324 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:30:47.324 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:30:47.324 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:30:47.324 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:30:47.324 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:30:47.324 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-10T21:30:47.324 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-10T21:30:47.324 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-10T21:30:47.324 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-10T21:30:47.324 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-10T21:30:47.324 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "5e2f023f-d6a6-49f4-a256-f4807dce3739", 2024-09-10T21:30:47.324 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:30:47.324 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:30:47.324 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-10T21:30:47.325 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-10T21:30:47.325 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-10T21:30:47.325 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-10T21:30:47.325 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-10T21:30:47.325 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-10T21:30:47.325 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-10T21:30:47.325 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:30:47.325 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-10T21:30:47.325 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:30:47.325 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:30:47.325 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:30:47.325 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:30:47.325 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:30:47.326 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:30:47.326 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:30:47.326 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:30:47.326 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:30:47.326 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:30:47.326 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:30:47.326 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:30:47.326 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-10T21:30:47.326 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:30:47.326 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-10T21:30:47.326 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:30:47.326 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:30:47.326 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:30:47.326 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:30:47.327 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:30:47.327 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:30:47.327 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:30:47.327 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-10T21:30:47.327 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:30:47.327 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:30:47.327 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:30:47.327 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:30:47.327 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-10T21:30:47.327 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:30:47.327 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:30:47.328 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:30:47.328 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:30:47.328 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-10T21:30:47.328 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-10T21:30:47.328 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:30:47.328 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:30:47.328 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:30:47.328 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:30:47.328 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:30:47.328 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:30:47.328 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:30:47.328 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:30:47.328 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:30:47.328 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:30:47.329 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:30:47.329 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3300", 2024-09-10T21:30:47.329 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:30:47.329 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:30:47.329 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:30:47.329 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:30:47.329 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789", 2024-09-10T21:30:47.329 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:30:47.329 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:30:47.329 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:30:47.330 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:30:47.330 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789/0", 2024-09-10T21:30:47.330 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6789/0", 2024-09-10T21:30:47.330 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:30:47.330 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:30:47.330 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:30:47.330 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:30:47.330 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:30:47.330 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:30:47.330 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:30:47.330 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:30:47.330 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:30:47.330 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:30:47.330 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:30:47.331 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3301", 2024-09-10T21:30:47.331 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:30:47.331 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:30:47.331 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:30:47.331 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:30:47.331 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790", 2024-09-10T21:30:47.331 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:30:47.331 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:30:47.331 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:30:47.331 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:30:47.331 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790/0", 2024-09-10T21:30:47.332 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6790/0", 2024-09-10T21:30:47.332 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:30:47.332 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:30:47.332 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:30:47.333 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:30:47.333 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:30:47.333 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:30:47.333 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-10T21:30:47.333 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-10T21:30:47.333 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:30:47.333 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:30:47.333 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:30:47.334 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:30:47.334 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:30:47.334 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:30:47.334 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": [ 2024-09-10T21:30:47.334 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:30:47.334 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:30:47.334 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:30:47.334 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:30:47.334 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:30:47.334 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:30:47.334 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-10T21:30:47.334 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:30:47.334 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:30:47.335 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:30:47.335 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:30:47.335 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:30:47.335 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:30:47.335 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-10T21:30:47.335 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:30:47.335 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:30:47.335 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:30:47.335 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 4 2024-09-10T21:30:47.335 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:30:47.335 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:30:47.335 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:30:47.335 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-10T21:30:47.335 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-10T21:30:47.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:47.319+0000 7f48dd200640 1 -- 172.21.15.17:0/3707266598 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f48b8072590 msgr2=0x7f48b8074a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:47.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:47.319+0000 7f48dd200640 1 --2- 172.21.15.17:0/3707266598 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f48b8072590 0x7f48b8074a50 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f48d80c3d30 tx=0x7f48d0009510 comp rx=0 tx=0).stop 2024-09-10T21:30:47.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:47.319+0000 7f48dd200640 1 -- 172.21.15.17:0/3707266598 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f48d8152440 msgr2=0x7f48d8141730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:47.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:47.319+0000 7f48dd200640 1 --2- 172.21.15.17:0/3707266598 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f48d8152440 0x7f48d8141730 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f48c40b0040 tx=0x7f48c4004040 comp rx=0 tx=0).stop 2024-09-10T21:30:47.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:47.319+0000 7f48d6800640 1 -- 172.21.15.17:0/3707266598 reap_dead start 2024-09-10T21:30:47.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:47.319+0000 7f48d6800640 1 -- 172.21.15.17:0/3707266598 reap_dead start 2024-09-10T21:30:47.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:47.319+0000 7f48dd200640 1 -- 172.21.15.17:0/3707266598 shutdown_connections 2024-09-10T21:30:47.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:47.319+0000 7f48dd200640 1 -- 172.21.15.17:0/3707266598 >> 172.21.15.17:0/3707266598 conn(0x7f48d80702f0 msgr2=0x7f48d814e020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:47.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:47.319+0000 7f48dd200640 1 -- 172.21.15.17:0/3707266598 shutdown_connections 2024-09-10T21:30:47.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:47.319+0000 7f48dd200640 1 -- 172.21.15.17:0/3707266598 wait complete. 2024-09-10T21:30:47.339 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-09-10T21:30:47.339 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon b 2024-09-10T21:30:47.339 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-10T21:30:47.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:47.539+0000 7f7c14e00640 1 -- 172.21.15.17:0/3388244410 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7c10152070 msgr2=0x7f7c10072420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:47.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:47.539+0000 7f7c14e00640 1 --2- 172.21.15.17:0/3388244410 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7c10152070 0x7f7c10072420 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f7c100cc310 tx=0x7f7c0001f0b0 comp rx=0 tx=0).stop 2024-09-10T21:30:47.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:47.539+0000 7f7c14e00640 1 -- 172.21.15.17:0/3388244410 shutdown_connections 2024-09-10T21:30:47.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:47.539+0000 7f7c14e00640 1 --2- 172.21.15.17:0/3388244410 >> v2:172.21.15.31:3300/0 conn(0x7f7c10152450 0x7f7c10072960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:47.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:47.539+0000 7f7c14e00640 1 --2- 172.21.15.17:0/3388244410 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7c10152070 0x7f7c10072420 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:47.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:47.539+0000 7f7c14e00640 1 -- 172.21.15.17:0/3388244410 >> 172.21.15.17:0/3388244410 conn(0x7f7c1006cc20 msgr2=0x7f7c1006d030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:47.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:47.539+0000 7f7c14e00640 1 -- 172.21.15.17:0/3388244410 shutdown_connections 2024-09-10T21:30:47.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:47.539+0000 7f7c14e00640 1 -- 172.21.15.17:0/3388244410 wait complete. 2024-09-10T21:30:47.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:47.539+0000 7f7c14e00640 1 Processor -- start 2024-09-10T21:30:47.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:47.539+0000 7f7c14e00640 1 -- start start 2024-09-10T21:30:47.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:47.539+0000 7f7c14e00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c100cbd10 0x7f7c1014caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:47.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:47.539+0000 7f7c14e00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7c10152070 0x7f7c1014cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:47.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:47.539+0000 7f7c14e00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f7c10152450 0x7f7c100cc930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:47.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:47.539+0000 7f7c14e00640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f7c100cecf0 con 0x7f7c10152070 2024-09-10T21:30:47.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:47.539+0000 7f7c14e00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f7c100c5090 con 0x7f7c100cbd10 2024-09-10T21:30:47.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:47.539+0000 7f7c14e00640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f7c100cee70 con 0x7f7c10152450 2024-09-10T21:30:47.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:47.539+0000 7f7c0ea00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7c10152070 0x7f7c1014cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:47.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:47.543+0000 7f7c0fe00640 1 -- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f7c10152450 msgr2=0x7f7c100cc930 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:3301/0 2024-09-10T21:30:47.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:47.543+0000 7f7c0ea00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7c10152070 0x7f7c1014cfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:52650/0 (socket says 172.21.15.17:52650) 2024-09-10T21:30:47.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:47.543+0000 7f7c0ea00640 1 -- 172.21.15.17:0/1250607756 learned_addr learned my addr 172.21.15.17:0/1250607756 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:30:47.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:47.543+0000 7f7c0fe00640 1 --2- 172.21.15.17:0/1250607756 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f7c10152450 0x7f7c100cc930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:30:47.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:47.543+0000 7f7c0f400640 1 --2- 172.21.15.17:0/1250607756 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c100cbd10 0x7f7c1014caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:47.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:47.543+0000 7f7c0ea00640 1 -- 172.21.15.17:0/1250607756 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f7c10152450 msgr2=0x7f7c100cc930 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:30:47.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:47.543+0000 7f7c0ea00640 1 --2- 172.21.15.17:0/1250607756 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f7c10152450 0x7f7c100cc930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:47.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:47.543+0000 7f7c0ea00640 1 -- 172.21.15.17:0/1250607756 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c100cbd10 msgr2=0x7f7c1014caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:47.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:47.543+0000 7f7c0ea00640 1 --2- 172.21.15.17:0/1250607756 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c100cbd10 0x7f7c1014caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:30:47.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:47.543+0000 7f7c0ea00640 1 -- 172.21.15.17:0/1250607756 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c100ca750 con 0x7f7c10152070 2024-09-10T21:30:47.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:47.543+0000 7f7c0ea00640 1 --2- 172.21.15.17:0/1250607756 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7c10152070 0x7f7c1014cfe0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f7bfc00e770 tx=0x7f7bfc00ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:47.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:47.543+0000 7f7c07400640 1 -- 172.21.15.17:0/1250607756 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bfc00ccc0 con 0x7f7c10152070 2024-09-10T21:30:47.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:47.543+0000 7f7c07400640 1 -- 172.21.15.17:0/1250607756 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7bfc00ee80 con 0x7f7c10152070 2024-09-10T21:30:47.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:47.543+0000 7f7c07400640 1 -- 172.21.15.17:0/1250607756 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bfc00ce60 con 0x7f7c10152070 2024-09-10T21:30:47.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:47.543+0000 7f7c14e00640 1 -- 172.21.15.17:0/1250607756 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c100ca190 con 0x7f7c10152070 2024-09-10T21:30:47.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:47.543+0000 7f7c14e00640 1 -- 172.21.15.17:0/1250607756 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c101f43a0 con 0x7f7c10152070 2024-09-10T21:30:47.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:47.543+0000 7f7c14e00640 1 --2- 172.21.15.17:0/1250607756 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c10149eb0 0x7f7c100cce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:47.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:47.543+0000 7f7c14e00640 1 -- 172.21.15.17:0/1250607756 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f7c10069ef0 con 0x7f7c10149eb0 2024-09-10T21:30:47.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:47.543+0000 7f7c0f400640 1 --2- 172.21.15.17:0/1250607756 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c10149eb0 0x7f7c100cce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:47.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:47.543+0000 7f7c0f400640 1 -- 172.21.15.17:0/1250607756 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7c10158e30 con 0x7f7c10149eb0 2024-09-10T21:30:47.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:47.543+0000 7f7c0f400640 1 --2- 172.21.15.17:0/1250607756 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c10149eb0 0x7f7c100cce70 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f7c100cec30 tx=0x7f7c00005760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:47.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:47.543+0000 7f7c07400640 1 -- 172.21.15.17:0/1250607756 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c000b5040 con 0x7f7c10149eb0 2024-09-10T21:30:47.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:47.543+0000 7f7c07400640 1 -- 172.21.15.17:0/1250607756 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f7bfc005280 con 0x7f7c10152070 2024-09-10T21:30:47.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:47.547+0000 7f7c07400640 1 --2- 172.21.15.17:0/1250607756 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f7bf00724e0 0x7f7bf00749a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:47.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:47.547+0000 7f7c07400640 1 -- 172.21.15.17:0/1250607756 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f7bfc093ed0 con 0x7f7c10152070 2024-09-10T21:30:47.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:47.547+0000 7f7c07400640 1 -- 172.21.15.17:0/1250607756 <== mon.1 v2:172.21.15.31:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f7c00005220 con 0x7f7c10149eb0 2024-09-10T21:30:47.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:47.547+0000 7f7c07400640 1 -- 172.21.15.17:0/1250607756 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c10149eb0 msgr2=0x7f7c100cce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:47.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:47.547+0000 7f7c07400640 1 --2- 172.21.15.17:0/1250607756 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c10149eb0 0x7f7c100cce70 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f7c100cec30 tx=0x7f7c00005760 comp rx=0 tx=0).stop 2024-09-10T21:30:47.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:47.547+0000 7f7c07400640 1 -- 172.21.15.17:0/1250607756 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c10149eb0 msgr2=0x7f7c100cce70 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:30:47.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:47.547+0000 7f7c07400640 1 --2- 172.21.15.17:0/1250607756 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c10149eb0 0x7f7c100cce70 secure :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f7c100cec30 tx=0x7f7c00005760 comp rx=0 tx=0).stop 2024-09-10T21:30:47.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:47.547+0000 7f7c0fe00640 1 --2- 172.21.15.17:0/1250607756 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f7bf00724e0 0x7f7bf00749a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:47.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:47.547+0000 7f7c0fe00640 1 --2- 172.21.15.17:0/1250607756 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f7bf00724e0 0x7f7bf00749a0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f7c08007fd0 tx=0x7f7c08007d00 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:47.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:47.551+0000 7f7c14e00640 1 --2- 172.21.15.17:0/1250607756 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c101f45a0 0x7f7c100cd3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:30:47.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:47.551+0000 7f7c14e00640 1 -- 172.21.15.17:0/1250607756 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f7c101f4060 con 0x7f7c101f45a0 2024-09-10T21:30:47.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:47.551+0000 7f7c0f400640 1 --2- 172.21.15.17:0/1250607756 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c101f45a0 0x7f7c100cd3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:30:47.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:47.555+0000 7f7c0f400640 1 -- 172.21.15.17:0/1250607756 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f7c10071e10 con 0x7f7c101f45a0 2024-09-10T21:30:47.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:47.555+0000 7f7c0f400640 1 --2- 172.21.15.17:0/1250607756 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c101f45a0 0x7f7c100cd3b0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f7c0001cce0 tx=0x7f7c00009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:30:47.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:47.555+0000 7f7c07400640 1 -- 172.21.15.17:0/1250607756 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c00004a10 con 0x7f7c101f45a0 2024-09-10T21:30:47.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:47.555+0000 7f7c07400640 1 -- 172.21.15.17:0/1250607756 <== mon.1 v2:172.21.15.31:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5031 (secure 0 0 0) 0x7f7c000049f0 con 0x7f7c101f45a0 2024-09-10T21:30:47.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:47.555+0000 7f7c07400640 1 -- 172.21.15.17:0/1250607756 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c101f45a0 msgr2=0x7f7c100cd3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:47.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:47.555+0000 7f7c07400640 1 --2- 172.21.15.17:0/1250607756 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c101f45a0 0x7f7c100cd3b0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f7c0001cce0 tx=0x7f7c00009260 comp rx=0 tx=0).stop 2024-09-10T21:30:47.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:47.555+0000 7f7c07400640 1 -- 172.21.15.17:0/1250607756 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c101f45a0 msgr2=0x7f7c100cd3b0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:30:47.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:47.555+0000 7f7c07400640 1 --2- 172.21.15.17:0/1250607756 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7c101f45a0 0x7f7c100cd3b0 secure :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f7c0001cce0 tx=0x7f7c00009260 comp rx=0 tx=0).stop 2024-09-10T21:30:47.558 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-10T21:30:47.558 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:30:47.558 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:30:47.558 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-10T21:30:47.558 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 52, 2024-09-10T21:30:47.559 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 386001, 2024-09-10T21:30:47.559 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-10T21:30:47.559 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-10T21:30:47.559 INFO:teuthology.orchestra.run.smithi017.stdout: 1 2024-09-10T21:30:47.559 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:30:47.559 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 1, 2024-09-10T21:30:47.559 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:30:47.559 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-10T21:30:47.559 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-10T21:30:47.559 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:30:47.559 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:30:47.559 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:30:47.559 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:30:47.560 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:30:47.560 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:30:47.560 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:30:47.560 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:30:47.560 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:30:47.560 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:30:47.560 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:30:47.560 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:30:47.560 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-10T21:30:47.560 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-10T21:30:47.560 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:30:47.560 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:30:47.560 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:30:47.561 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:30:47.561 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:30:47.561 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:30:47.561 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:30:47.561 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:30:47.561 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:30:47.561 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:30:47.561 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:30:47.562 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:30:47.562 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:30:47.562 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-10T21:30:47.562 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-10T21:30:47.562 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-10T21:30:47.562 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-10T21:30:47.562 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-10T21:30:47.562 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "5e2f023f-d6a6-49f4-a256-f4807dce3739", 2024-09-10T21:30:47.562 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:30:47.562 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:30:47.562 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-10T21:30:47.562 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-10T21:30:47.563 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-10T21:30:47.563 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-10T21:30:47.563 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-10T21:30:47.563 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-10T21:30:47.563 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-10T21:30:47.563 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:30:47.563 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-10T21:30:47.563 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:30:47.563 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:30:47.563 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:30:47.563 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:30:47.563 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:30:47.563 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:30:47.564 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:30:47.564 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:30:47.564 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:30:47.564 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:30:47.564 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:30:47.564 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:30:47.564 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-10T21:30:47.564 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:30:47.564 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-10T21:30:47.564 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:30:47.564 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:30:47.564 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:30:47.564 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:30:47.565 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:30:47.565 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:30:47.565 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:30:47.565 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-10T21:30:47.565 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:30:47.565 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:30:47.565 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:30:47.565 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:30:47.565 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-10T21:30:47.565 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:30:47.566 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:30:47.566 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:30:47.566 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:30:47.566 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-10T21:30:47.566 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-10T21:30:47.566 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:30:47.566 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:30:47.566 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:30:47.566 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:30:47.567 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:30:47.567 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:30:47.567 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:30:47.567 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:30:47.567 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:30:47.567 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:30:47.567 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:30:47.567 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3300", 2024-09-10T21:30:47.567 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:30:47.567 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:30:47.567 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:30:47.567 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:30:47.567 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789", 2024-09-10T21:30:47.568 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:30:47.568 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:30:47.568 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:30:47.568 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:30:47.568 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789/0", 2024-09-10T21:30:47.568 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6789/0", 2024-09-10T21:30:47.568 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:30:47.568 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:30:47.568 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:30:47.568 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:30:47.568 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:30:47.568 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:30:47.569 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:30:47.569 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:30:47.569 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:30:47.569 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:30:47.569 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:30:47.569 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3301", 2024-09-10T21:30:47.569 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:30:47.569 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:30:47.569 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:30:47.569 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:30:47.569 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790", 2024-09-10T21:30:47.570 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:30:47.570 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:30:47.570 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:30:47.570 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:30:47.570 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790/0", 2024-09-10T21:30:47.570 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6790/0", 2024-09-10T21:30:47.570 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:30:47.570 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:30:47.570 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:30:47.570 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:30:47.570 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:30:47.570 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:30:47.570 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-10T21:30:47.571 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-10T21:30:47.571 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:30:47.571 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:30:47.571 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:30:47.571 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:30:47.571 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:30:47.571 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:30:47.571 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": [ 2024-09-10T21:30:47.571 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:30:47.571 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:30:47.571 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:30:47.571 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 5 2024-09-10T21:30:47.571 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:30:47.571 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:30:47.572 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-10T21:30:47.572 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:30:47.572 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:30:47.572 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:30:47.572 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 7 2024-09-10T21:30:47.572 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:30:47.572 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:30:47.572 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-10T21:30:47.572 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:30:47.572 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:30:47.572 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:30:47.572 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 4 2024-09-10T21:30:47.572 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:30:47.573 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:30:47.573 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": [ 2024-09-10T21:30:47.573 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:30:47.573 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:30:47.573 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:30:47.573 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 2 2024-09-10T21:30:47.573 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:30:47.573 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:30:47.573 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:30:47.573 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-10T21:30:47.573 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-10T21:30:47.574 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:47.555+0000 7f7c14e00640 1 -- 172.21.15.17:0/1250607756 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f7bf00724e0 msgr2=0x7f7bf00749a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:47.574 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:47.555+0000 7f7c14e00640 1 --2- 172.21.15.17:0/1250607756 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f7bf00724e0 0x7f7bf00749a0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f7c08007fd0 tx=0x7f7c08007d00 comp rx=0 tx=0).stop 2024-09-10T21:30:47.574 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:47.555+0000 7f7c14e00640 1 -- 172.21.15.17:0/1250607756 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7c10152070 msgr2=0x7f7c1014cfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:30:47.574 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:47.555+0000 7f7c14e00640 1 --2- 172.21.15.17:0/1250607756 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7c10152070 0x7f7c1014cfe0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f7bfc00e770 tx=0x7f7bfc00ec30 comp rx=0 tx=0).stop 2024-09-10T21:30:47.574 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:47.555+0000 7f7c0fe00640 1 -- 172.21.15.17:0/1250607756 reap_dead start 2024-09-10T21:30:47.574 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:47.555+0000 7f7c0fe00640 1 -- 172.21.15.17:0/1250607756 reap_dead start 2024-09-10T21:30:47.575 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:47.555+0000 7f7c14e00640 1 -- 172.21.15.17:0/1250607756 shutdown_connections 2024-09-10T21:30:47.575 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:47.555+0000 7f7c14e00640 1 -- 172.21.15.17:0/1250607756 >> 172.21.15.17:0/1250607756 conn(0x7f7c1006cc20 msgr2=0x7f7c101e05e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:30:47.575 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:47.555+0000 7f7c14e00640 1 -- 172.21.15.17:0/1250607756 shutdown_connections 2024-09-10T21:30:47.575 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:30:47.555+0000 7f7c14e00640 1 -- 172.21.15.17:0/1250607756 wait complete. 2024-09-10T21:30:47.578 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-09-10T21:30:47.578 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2024-09-10T21:31:07.579 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.c 2024-09-10T21:31:07.580 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-10T21:31:07.580 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-10T21:31:07.584 INFO:tasks.ceph.mon.c:Started 2024-09-10T21:31:07.585 INFO:tasks.mon_thrash:Sending STOP to mon b 2024-09-10T21:31:07.585 INFO:tasks.ceph.mon.b:Sent signal 19 2024-09-10T21:31:07.585 INFO:tasks.mon_thrash.mon_thrasher:waiting for 15.0 secs to unfreeze mons 2024-09-10T21:31:17.289 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:31:17.293 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:31:22.585 INFO:tasks.mon_thrash:Sending CONT to mon b 2024-09-10T21:31:22.586 INFO:tasks.ceph.mon.b:Sent signal 18 2024-09-10T21:31:22.586 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-09-10T21:31:22.587 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-10T21:31:28.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:28.742+0000 7f0680200640 1 -- 172.21.15.17:0/3200632375 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f0658006220 msgr2=0x7f0658008660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:28.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:28.742+0000 7f0680200640 1 --2- 172.21.15.17:0/3200632375 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f0658006220 0x7f0658008660 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f066c0065c0 tx=0x7f066c01d6f0 comp rx=0 tx=0).stop 2024-09-10T21:31:28.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:28.742+0000 7f0680200640 1 -- 172.21.15.17:0/3200632375 shutdown_connections 2024-09-10T21:31:28.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:28.742+0000 7f0680200640 1 --2- 172.21.15.17:0/3200632375 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f0658006220 0x7f0658008660 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:28.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:28.742+0000 7f0680200640 1 -- 172.21.15.17:0/3200632375 >> 172.21.15.17:0/3200632375 conn(0x7f067806fc60 msgr2=0x7f0678068990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:28.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:28.742+0000 7f0680200640 1 -- 172.21.15.17:0/3200632375 shutdown_connections 2024-09-10T21:31:28.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:28.742+0000 7f0680200640 1 -- 172.21.15.17:0/3200632375 wait complete. 2024-09-10T21:31:28.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:28.742+0000 7f0680200640 1 Processor -- start 2024-09-10T21:31:28.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:28.742+0000 7f0680200640 1 -- start start 2024-09-10T21:31:28.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:28.746+0000 7f0680200640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0658003b40 0x7f067815aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:28.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:28.746+0000 7f0680200640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0658006220 0x7f06780af400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:28.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:28.746+0000 7f0680200640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f065800aa20 0x7f06780af940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:28.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:28.746+0000 7f0680200640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f0678158c10 con 0x7f0658006220 2024-09-10T21:31:28.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:28.746+0000 7f0680200640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f0678158930 con 0x7f0658003b40 2024-09-10T21:31:28.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:28.746+0000 7f0680200640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f067806cc20 con 0x7f065800aa20 2024-09-10T21:31:28.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:28.746+0000 7f067d800640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0658003b40 0x7f067815aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:28.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:28.746+0000 7f067ce00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0658006220 0x7f06780af400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:28.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:28.746+0000 7f067d800640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0658003b40 0x7f067815aae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.17:37112/0 (socket says 172.21.15.17:37112) 2024-09-10T21:31:28.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:28.746+0000 7f067d800640 1 -- 172.21.15.17:0/823938343 learned_addr learned my addr 172.21.15.17:0/823938343 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:31:28.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:28.746+0000 7f067e200640 1 --2- 172.21.15.17:0/823938343 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f065800aa20 0x7f06780af940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:28.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:28.746+0000 7f067d800640 1 -- 172.21.15.17:0/823938343 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f065800aa20 msgr2=0x7f06780af940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:28.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:28.746+0000 7f067d800640 1 --2- 172.21.15.17:0/823938343 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f065800aa20 0x7f06780af940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:28.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:28.746+0000 7f067d800640 1 -- 172.21.15.17:0/823938343 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0658006220 msgr2=0x7f06780af400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:28.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:28.746+0000 7f067d800640 1 --2- 172.21.15.17:0/823938343 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0658006220 0x7f06780af400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:28.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:28.746+0000 7f067d800640 1 -- 172.21.15.17:0/823938343 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06780b18f0 con 0x7f0658003b40 2024-09-10T21:31:28.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:28.746+0000 7f067ce00640 1 --2- 172.21.15.17:0/823938343 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0658006220 0x7f06780af400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:31:28.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:28.746+0000 7f067d800640 1 --2- 172.21.15.17:0/823938343 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0658003b40 0x7f067815aae0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f066c000c10 tx=0x7f066c0a7f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:28.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:28.746+0000 7f0675600640 1 -- 172.21.15.17:0/823938343 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f066c0c1070 con 0x7f0658003b40 2024-09-10T21:31:28.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:28.746+0000 7f0675600640 1 -- 172.21.15.17:0/823938343 <== mon.1 v2:172.21.15.31:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f066c0b9630 con 0x7f0658003b40 2024-09-10T21:31:28.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:28.746+0000 7f0680200640 1 -- 172.21.15.17:0/823938343 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06780aec40 con 0x7f0658003b40 2024-09-10T21:31:28.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:28.746+0000 7f0675600640 1 -- 172.21.15.17:0/823938343 <== mon.1 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f066c0b9a60 con 0x7f0658003b40 2024-09-10T21:31:28.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:28.746+0000 7f0680200640 1 -- 172.21.15.17:0/823938343 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06781f15a0 con 0x7f0658003b40 2024-09-10T21:31:28.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:28.750+0000 7f0680200640 1 -- 172.21.15.17:0/823938343 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06780c7ae0 con 0x7f0658003b40 2024-09-10T21:31:28.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:28.750+0000 7f0675600640 1 -- 172.21.15.17:0/823938343 <== mon.1 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f066c0ba070 con 0x7f0658003b40 2024-09-10T21:31:28.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:28.750+0000 7f0675600640 1 --2- 172.21.15.17:0/823938343 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f0658076550 0x7f0658078a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:28.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:28.750+0000 7f0675600640 1 -- 172.21.15.17:0/823938343 <== mon.1 v2:172.21.15.31:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f066c12ec70 con 0x7f0658003b40 2024-09-10T21:31:28.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:28.750+0000 7f067ce00640 1 --2- 172.21.15.17:0/823938343 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f0658076550 0x7f0658078a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:28.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:28.750+0000 7f067ce00640 1 --2- 172.21.15.17:0/823938343 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f0658076550 0x7f0658078a10 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f0678073af0 tx=0x7f0664005860 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:28.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:28.754+0000 7f0675600640 1 -- 172.21.15.17:0/823938343 <== mon.1 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f06780b18f0 con 0x7f0658003b40 2024-09-10T21:31:29.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:28.998+0000 7f0680200640 1 -- 172.21.15.17:0/823938343 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f0678075160 con 0x7f0658003b40 2024-09-10T21:31:29.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:28.998+0000 7f0675600640 1 -- 172.21.15.17:0/823938343 <== mon.1 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1556 (secure 0 0 0) 0x7f066c0174c0 con 0x7f0658003b40 2024-09-10T21:31:29.004 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":58,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":1,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5e2f023f-d6a6-49f4-a256-f4807dce3739","modified":"2024-09-10T21:23:46.892130Z","created":"2024-09-10T21:23:46.892130Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3301","nonce":0},{"type":"v1","addr":"172.21.15.31:6790","nonce":0}]},"addr":"172.21.15.31:6790/0","public_addr":"172.21.15.31:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-10T21:31:29.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.002+0000 7f0680200640 1 -- 172.21.15.17:0/823938343 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f0658076550 msgr2=0x7f0658078a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:29.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.002+0000 7f0680200640 1 --2- 172.21.15.17:0/823938343 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f0658076550 0x7f0658078a10 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f0678073af0 tx=0x7f0664005860 comp rx=0 tx=0).stop 2024-09-10T21:31:29.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.002+0000 7f0680200640 1 -- 172.21.15.17:0/823938343 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0658003b40 msgr2=0x7f067815aae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:29.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.002+0000 7f0680200640 1 --2- 172.21.15.17:0/823938343 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0658003b40 0x7f067815aae0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f066c000c10 tx=0x7f066c0a7f10 comp rx=0 tx=0).stop 2024-09-10T21:31:29.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.002+0000 7f0680200640 1 -- 172.21.15.17:0/823938343 shutdown_connections 2024-09-10T21:31:29.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.002+0000 7f0680200640 1 --2- 172.21.15.17:0/823938343 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f0658076550 0x7f0658078a10 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:29.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.002+0000 7f0680200640 1 --2- 172.21.15.17:0/823938343 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f065800aa20 0x7f06780af940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:29.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.002+0000 7f0680200640 1 --2- 172.21.15.17:0/823938343 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0658006220 0x7f06780af400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:29.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.002+0000 7f0680200640 1 --2- 172.21.15.17:0/823938343 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0658003b40 0x7f067815aae0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:29.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.002+0000 7f0680200640 1 -- 172.21.15.17:0/823938343 >> 172.21.15.17:0/823938343 conn(0x7f067806fc60 msgr2=0x7f06781562f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:29.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.002+0000 7f0680200640 1 -- 172.21.15.17:0/823938343 shutdown_connections 2024-09-10T21:31:29.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.002+0000 7f0680200640 1 -- 172.21.15.17:0/823938343 wait complete. 2024-09-10T21:31:29.025 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-09-10T21:31:29.026 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-09-10T21:31:29.026 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-10T21:31:29.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.170+0000 7fbd36a00640 1 -- 172.21.15.17:0/3011479370 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbd300c0fd0 msgr2=0x7fbd300740d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:29.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.170+0000 7fbd36a00640 1 --2- 172.21.15.17:0/3011479370 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbd300c0fd0 0x7fbd300740d0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fbd1800aa10 tx=0x7fbd1801dc30 comp rx=0 tx=0).stop 2024-09-10T21:31:29.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.170+0000 7fbd36a00640 1 -- 172.21.15.17:0/3011479370 shutdown_connections 2024-09-10T21:31:29.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.170+0000 7fbd36a00640 1 --2- 172.21.15.17:0/3011479370 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fbd3015abb0 0x7fbd301613a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:29.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.170+0000 7fbd36a00640 1 --2- 172.21.15.17:0/3011479370 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbd300c0fd0 0x7fbd300740d0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:29.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.170+0000 7fbd36a00640 1 -- 172.21.15.17:0/3011479370 >> 172.21.15.17:0/3011479370 conn(0x7fbd300700d0 msgr2=0x7fbd300704e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:29.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.170+0000 7fbd36a00640 1 -- 172.21.15.17:0/3011479370 shutdown_connections 2024-09-10T21:31:29.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.170+0000 7fbd36a00640 1 -- 172.21.15.17:0/3011479370 wait complete. 2024-09-10T21:31:29.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.170+0000 7fbd36a00640 1 Processor -- start 2024-09-10T21:31:29.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.170+0000 7fbd36a00640 1 -- start start 2024-09-10T21:31:29.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.174+0000 7fbd36a00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fbd30073cf0 0x7fbd300cc700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:29.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.174+0000 7fbd36a00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbd300c0fd0 0x7fbd301f7190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:29.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.174+0000 7fbd36a00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbd3015abb0 0x7fbd301f52c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:29.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.174+0000 7fbd36a00640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fbd300c7dd0 con 0x7fbd300c0fd0 2024-09-10T21:31:29.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.174+0000 7fbd36a00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fbd300c8370 con 0x7fbd3015abb0 2024-09-10T21:31:29.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.174+0000 7fbd36a00640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7fbd300c7f50 con 0x7fbd30073cf0 2024-09-10T21:31:29.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.174+0000 7fbd27e00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbd300c0fd0 0x7fbd301f7190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:29.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.174+0000 7fbd27e00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbd300c0fd0 0x7fbd301f7190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:46820/0 (socket says 172.21.15.17:46820) 2024-09-10T21:31:29.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.174+0000 7fbd27e00640 1 -- 172.21.15.17:0/207215982 learned_addr learned my addr 172.21.15.17:0/207215982 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:31:29.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.174+0000 7fbd2fe00640 1 --2- 172.21.15.17:0/207215982 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fbd30073cf0 0x7fbd300cc700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:29.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.174+0000 7fbd34a00640 1 --2- 172.21.15.17:0/207215982 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbd3015abb0 0x7fbd301f52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:29.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.174+0000 7fbd27e00640 1 -- 172.21.15.17:0/207215982 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fbd30073cf0 msgr2=0x7fbd300cc700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:29.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.174+0000 7fbd27e00640 1 --2- 172.21.15.17:0/207215982 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fbd30073cf0 0x7fbd300cc700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:29.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.174+0000 7fbd27e00640 1 -- 172.21.15.17:0/207215982 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbd3015abb0 msgr2=0x7fbd301f52c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:29.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.174+0000 7fbd27e00640 1 --2- 172.21.15.17:0/207215982 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbd3015abb0 0x7fbd301f52c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:29.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.174+0000 7fbd27e00640 1 -- 172.21.15.17:0/207215982 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd301eb3a0 con 0x7fbd300c0fd0 2024-09-10T21:31:29.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.174+0000 7fbd2fe00640 1 --2- 172.21.15.17:0/207215982 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fbd30073cf0 0x7fbd300cc700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:31:29.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.174+0000 7fbd27e00640 1 --2- 172.21.15.17:0/207215982 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbd300c0fd0 0x7fbd301f7190 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fbd1800a960 tx=0x7fbd180a6900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:29.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.174+0000 7fbd2cc00640 1 -- 172.21.15.17:0/207215982 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd180b3070 con 0x7fbd300c0fd0 2024-09-10T21:31:29.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.174+0000 7fbd2cc00640 1 -- 172.21.15.17:0/207215982 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbd180055f0 con 0x7fbd300c0fd0 2024-09-10T21:31:29.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.174+0000 7fbd2cc00640 1 -- 172.21.15.17:0/207215982 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd18004930 con 0x7fbd300c0fd0 2024-09-10T21:31:29.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.174+0000 7fbd36a00640 1 -- 172.21.15.17:0/207215982 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd300ccc40 con 0x7fbd300c0fd0 2024-09-10T21:31:29.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.174+0000 7fbd36a00640 1 -- 172.21.15.17:0/207215982 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd301f7db0 con 0x7fbd300c0fd0 2024-09-10T21:31:29.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.174+0000 7fbd36a00640 1 --2- 172.21.15.17:0/207215982 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbd301ea4f0 0x7fbd301f5800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:29.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.174+0000 7fbd36a00640 1 -- 172.21.15.17:0/207215982 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fbd301f4e30 con 0x7fbd301ea4f0 2024-09-10T21:31:29.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.174+0000 7fbd2fe00640 1 --2- 172.21.15.17:0/207215982 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbd301ea4f0 0x7fbd301f5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:29.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.174+0000 7fbd2fe00640 1 -- 172.21.15.17:0/207215982 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fbd300725c0 con 0x7fbd301ea4f0 2024-09-10T21:31:29.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.174+0000 7fbd2fe00640 1 --2- 172.21.15.17:0/207215982 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbd301ea4f0 0x7fbd301f5800 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fbd2000e500 tx=0x7fbd2000e9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:29.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.174+0000 7fbd2cc00640 1 -- 172.21.15.17:0/207215982 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd20007030 con 0x7fbd301ea4f0 2024-09-10T21:31:29.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.178+0000 7fbd2cc00640 1 -- 172.21.15.17:0/207215982 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fbd1800b040 con 0x7fbd300c0fd0 2024-09-10T21:31:29.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.178+0000 7fbd2cc00640 1 --2- 172.21.15.17:0/207215982 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fbd0c0724e0 0x7fbd0c0749a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:29.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.178+0000 7fbd2cc00640 1 -- 172.21.15.17:0/207215982 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fbd1812d930 con 0x7fbd300c0fd0 2024-09-10T21:31:29.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.178+0000 7fbd2cc00640 1 -- 172.21.15.17:0/207215982 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fbd200071d0 con 0x7fbd301ea4f0 2024-09-10T21:31:29.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.178+0000 7fbd2cc00640 1 -- 172.21.15.17:0/207215982 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbd301ea4f0 msgr2=0x7fbd301f5800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:29.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.178+0000 7fbd2cc00640 1 --2- 172.21.15.17:0/207215982 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbd301ea4f0 0x7fbd301f5800 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fbd2000e500 tx=0x7fbd2000e9c0 comp rx=0 tx=0).stop 2024-09-10T21:31:29.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.178+0000 7fbd2cc00640 1 -- 172.21.15.17:0/207215982 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbd301ea4f0 msgr2=0x7fbd301f5800 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:31:29.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.178+0000 7fbd2cc00640 1 --2- 172.21.15.17:0/207215982 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbd301ea4f0 0x7fbd301f5800 secure :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fbd2000e500 tx=0x7fbd2000e9c0 comp rx=0 tx=0).stop 2024-09-10T21:31:29.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.178+0000 7fbd34a00640 1 --2- 172.21.15.17:0/207215982 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fbd0c0724e0 0x7fbd0c0749a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:29.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.178+0000 7fbd34a00640 1 --2- 172.21.15.17:0/207215982 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fbd0c0724e0 0x7fbd0c0749a0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fbd28005c50 tx=0x7fbd28005be0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:29.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.182+0000 7fbd36a00640 1 --2- 172.21.15.17:0/207215982 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbd30143db0 0x7fbd301441c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:29.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.182+0000 7fbd36a00640 1 -- 172.21.15.17:0/207215982 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fbd300752a0 con 0x7fbd30143db0 2024-09-10T21:31:29.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.182+0000 7fbd2fe00640 1 --2- 172.21.15.17:0/207215982 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbd30143db0 0x7fbd301441c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:29.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.182+0000 7fbd2fe00640 1 -- 172.21.15.17:0/207215982 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fbd30074e50 con 0x7fbd30143db0 2024-09-10T21:31:29.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.186+0000 7fbd2fe00640 1 --2- 172.21.15.17:0/207215982 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbd30143db0 0x7fbd301441c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fbd2000e500 tx=0x7fbd20003770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:29.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.186+0000 7fbd2cc00640 1 -- 172.21.15.17:0/207215982 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd2001e960 con 0x7fbd30143db0 2024-09-10T21:31:29.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.186+0000 7fbd2cc00640 1 -- 172.21.15.17:0/207215982 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4877 (secure 0 0 0) 0x7fbd2001e960 con 0x7fbd30143db0 2024-09-10T21:31:29.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.186+0000 7fbd2cc00640 1 -- 172.21.15.17:0/207215982 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbd30143db0 msgr2=0x7fbd301441c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:29.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.186+0000 7fbd2cc00640 1 --2- 172.21.15.17:0/207215982 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbd30143db0 0x7fbd301441c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fbd2000e500 tx=0x7fbd20003770 comp rx=0 tx=0).stop 2024-09-10T21:31:29.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.186+0000 7fbd2cc00640 1 -- 172.21.15.17:0/207215982 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbd30143db0 msgr2=0x7fbd301441c0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:31:29.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.186+0000 7fbd2cc00640 1 --2- 172.21.15.17:0/207215982 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbd30143db0 0x7fbd301441c0 secure :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fbd2000e500 tx=0x7fbd20003770 comp rx=0 tx=0).stop 2024-09-10T21:31:29.189 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-10T21:31:29.189 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:31:29.190 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:31:29.190 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "leader", 2024-09-10T21:31:29.190 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 58, 2024-09-10T21:31:29.190 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 207025, 2024-09-10T21:31:29.190 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-10T21:31:29.190 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-10T21:31:29.190 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-10T21:31:29.190 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-10T21:31:29.190 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:31:29.190 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 1, 2024-09-10T21:31:29.190 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:31:29.190 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-10T21:31:29.190 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-10T21:31:29.191 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:31:29.191 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:31:29.191 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:31:29.191 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:31:29.191 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:31:29.191 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:31:29.191 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:31:29.191 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:31:29.191 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:31:29.191 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:31:29.191 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:31:29.191 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:31:29.191 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-10T21:31:29.191 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-10T21:31:29.192 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:31:29.192 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:31:29.192 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:31:29.192 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:31:29.192 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:31:29.192 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:31:29.192 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:31:29.192 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:31:29.192 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:31:29.192 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:31:29.192 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:31:29.192 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:31:29.193 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:31:29.193 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-10T21:31:29.193 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-10T21:31:29.193 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-10T21:31:29.193 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-10T21:31:29.193 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-10T21:31:29.193 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "5e2f023f-d6a6-49f4-a256-f4807dce3739", 2024-09-10T21:31:29.193 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:31:29.193 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:31:29.193 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-10T21:31:29.193 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-10T21:31:29.193 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-10T21:31:29.193 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-10T21:31:29.194 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-10T21:31:29.194 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-10T21:31:29.194 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-10T21:31:29.194 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:31:29.194 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-10T21:31:29.194 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:31:29.194 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:31:29.194 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:31:29.194 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:31:29.194 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:31:29.194 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:31:29.194 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:31:29.194 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:31:29.194 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:31:29.195 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:31:29.195 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:31:29.195 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:31:29.195 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-10T21:31:29.195 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:31:29.195 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-10T21:31:29.195 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:31:29.195 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:31:29.195 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:31:29.195 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:31:29.195 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:31:29.195 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:31:29.195 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:31:29.196 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-10T21:31:29.196 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:31:29.196 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:31:29.196 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:31:29.196 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:31:29.196 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-10T21:31:29.196 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:31:29.196 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:31:29.196 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:31:29.196 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:31:29.196 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-10T21:31:29.196 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-10T21:31:29.196 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:31:29.196 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:31:29.197 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:31:29.197 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:31:29.197 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:31:29.197 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:31:29.197 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:31:29.197 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:31:29.197 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:31:29.197 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:31:29.197 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:31:29.197 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3300", 2024-09-10T21:31:29.197 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:31:29.197 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:31:29.198 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:31:29.198 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:31:29.198 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789", 2024-09-10T21:31:29.198 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:31:29.198 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:31:29.198 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:31:29.198 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:31:29.198 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789/0", 2024-09-10T21:31:29.198 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6789/0", 2024-09-10T21:31:29.198 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:31:29.198 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:31:29.198 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:31:29.198 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:31:29.198 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:31:29.199 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:31:29.199 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:31:29.199 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:31:29.199 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:31:29.199 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:31:29.199 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:31:29.199 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3301", 2024-09-10T21:31:29.199 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:31:29.199 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:31:29.199 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:31:29.199 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:31:29.199 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790", 2024-09-10T21:31:29.199 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:31:29.200 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:31:29.200 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:31:29.200 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:31:29.200 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790/0", 2024-09-10T21:31:29.200 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6790/0", 2024-09-10T21:31:29.200 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:31:29.200 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:31:29.200 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:31:29.200 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:31:29.200 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:31:29.201 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:31:29.201 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-10T21:31:29.201 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-10T21:31:29.201 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:31:29.201 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:31:29.201 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:31:29.201 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:31:29.201 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:31:29.201 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:31:29.201 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": [ 2024-09-10T21:31:29.201 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:31:29.201 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:31:29.201 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:31:29.201 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:31:29.202 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:31:29.202 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:31:29.202 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-10T21:31:29.202 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:31:29.202 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:31:29.202 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:31:29.202 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:31:29.202 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:31:29.202 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:31:29.202 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-10T21:31:29.202 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:31:29.202 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:31:29.202 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:31:29.203 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 4 2024-09-10T21:31:29.203 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:31:29.203 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:31:29.203 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:31:29.203 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-10T21:31:29.203 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-10T21:31:29.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.186+0000 7fbd36a00640 1 -- 172.21.15.17:0/207215982 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fbd0c0724e0 msgr2=0x7fbd0c0749a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:29.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.186+0000 7fbd36a00640 1 --2- 172.21.15.17:0/207215982 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fbd0c0724e0 0x7fbd0c0749a0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fbd28005c50 tx=0x7fbd28005be0 comp rx=0 tx=0).stop 2024-09-10T21:31:29.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.186+0000 7fbd36a00640 1 -- 172.21.15.17:0/207215982 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbd300c0fd0 msgr2=0x7fbd301f7190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:29.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.186+0000 7fbd36a00640 1 --2- 172.21.15.17:0/207215982 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbd300c0fd0 0x7fbd301f7190 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fbd1800a960 tx=0x7fbd180a6900 comp rx=0 tx=0).stop 2024-09-10T21:31:29.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.186+0000 7fbd34a00640 1 -- 172.21.15.17:0/207215982 reap_dead start 2024-09-10T21:31:29.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.186+0000 7fbd34a00640 1 -- 172.21.15.17:0/207215982 reap_dead start 2024-09-10T21:31:29.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.186+0000 7fbd36a00640 1 -- 172.21.15.17:0/207215982 shutdown_connections 2024-09-10T21:31:29.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.186+0000 7fbd36a00640 1 -- 172.21.15.17:0/207215982 >> 172.21.15.17:0/207215982 conn(0x7fbd300700d0 msgr2=0x7fbd30146720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:29.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.186+0000 7fbd36a00640 1 -- 172.21.15.17:0/207215982 shutdown_connections 2024-09-10T21:31:29.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.186+0000 7fbd36a00640 1 -- 172.21.15.17:0/207215982 wait complete. 2024-09-10T21:31:29.207 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-10T21:31:29.207 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-09-10T21:31:29.207 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-10T21:31:29.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.350+0000 7fcaeec00640 1 -- 172.21.15.17:0/1434179158 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcae80c89d0 msgr2=0x7fcae80cdf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:29.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.350+0000 7fcaeec00640 1 --2- 172.21.15.17:0/1434179158 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcae80c89d0 0x7fcae80cdf20 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fcae8076930 tx=0x7fcae001cd50 comp rx=0 tx=0).stop 2024-09-10T21:31:29.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.350+0000 7fcaeec00640 1 -- 172.21.15.17:0/1434179158 shutdown_connections 2024-09-10T21:31:29.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.350+0000 7fcaeec00640 1 --2- 172.21.15.17:0/1434179158 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcae80c89d0 0x7fcae80cdf20 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:29.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.350+0000 7fcaeec00640 1 --2- 172.21.15.17:0/1434179158 >> v2:172.21.15.31:3300/0 conn(0x7fcae8154aa0 0x7fcae80ca430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:29.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.350+0000 7fcaeec00640 1 -- 172.21.15.17:0/1434179158 >> 172.21.15.17:0/1434179158 conn(0x7fcae80702f0 msgr2=0x7fcae8070700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:29.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.350+0000 7fcaeec00640 1 -- 172.21.15.17:0/1434179158 shutdown_connections 2024-09-10T21:31:29.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.350+0000 7fcaeec00640 1 -- 172.21.15.17:0/1434179158 wait complete. 2024-09-10T21:31:29.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.354+0000 7fcaeec00640 1 Processor -- start 2024-09-10T21:31:29.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.354+0000 7fcaeec00640 1 -- start start 2024-09-10T21:31:29.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.354+0000 7fcaeec00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcae80c89d0 0x7fcae81550b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:29.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.354+0000 7fcaeec00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fcae81546c0 0x7fcae8069100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:29.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.354+0000 7fcaeec00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcae8154aa0 0x7fcae80c7580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:29.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.354+0000 7fcaeec00640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fcae80c9640 con 0x7fcae80c89d0 2024-09-10T21:31:29.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.354+0000 7fcaeec00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fcae8072790 con 0x7fcae8154aa0 2024-09-10T21:31:29.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.354+0000 7fcaeec00640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7fcae80c97c0 con 0x7fcae81546c0 2024-09-10T21:31:29.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.354+0000 7fcae7e00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcae80c89d0 0x7fcae81550b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:29.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.354+0000 7fcae7e00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcae80c89d0 0x7fcae81550b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:46844/0 (socket says 172.21.15.17:46844) 2024-09-10T21:31:29.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.354+0000 7fcae7e00640 1 -- 172.21.15.17:0/1996711002 learned_addr learned my addr 172.21.15.17:0/1996711002 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:31:29.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.354+0000 7fcae7400640 1 --2- 172.21.15.17:0/1996711002 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fcae81546c0 0x7fcae8069100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:29.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.354+0000 7fcaecc00640 1 --2- 172.21.15.17:0/1996711002 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcae8154aa0 0x7fcae80c7580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:29.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.354+0000 7fcae7e00640 1 -- 172.21.15.17:0/1996711002 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fcae81546c0 msgr2=0x7fcae8069100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:29.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.354+0000 7fcae7e00640 1 --2- 172.21.15.17:0/1996711002 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fcae81546c0 0x7fcae8069100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:29.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.354+0000 7fcae7e00640 1 -- 172.21.15.17:0/1996711002 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcae8154aa0 msgr2=0x7fcae80c7580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:29.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.354+0000 7fcae7e00640 1 --2- 172.21.15.17:0/1996711002 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcae8154aa0 0x7fcae80c7580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:29.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.354+0000 7fcae7e00640 1 -- 172.21.15.17:0/1996711002 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcae81f4090 con 0x7fcae80c89d0 2024-09-10T21:31:29.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.354+0000 7fcae7e00640 1 --2- 172.21.15.17:0/1996711002 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcae80c89d0 0x7fcae81550b0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fcad800c530 tx=0x7fcad800c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:29.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.354+0000 7fcadfe00640 1 -- 172.21.15.17:0/1996711002 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcad8017070 con 0x7fcae80c89d0 2024-09-10T21:31:29.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.354+0000 7fcadfe00640 1 -- 172.21.15.17:0/1996711002 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcad80053c0 con 0x7fcae80c89d0 2024-09-10T21:31:29.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.354+0000 7fcadfe00640 1 -- 172.21.15.17:0/1996711002 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcad8012890 con 0x7fcae80c89d0 2024-09-10T21:31:29.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.354+0000 7fcaeec00640 1 -- 172.21.15.17:0/1996711002 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcae81f4370 con 0x7fcae80c89d0 2024-09-10T21:31:29.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.354+0000 7fcaeec00640 1 -- 172.21.15.17:0/1996711002 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcae81f7050 con 0x7fcae80c89d0 2024-09-10T21:31:29.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.354+0000 7fcaeec00640 1 --2- 172.21.15.17:0/1996711002 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcae814f0c0 0x7fcae80c7ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:29.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.354+0000 7fcaeec00640 1 -- 172.21.15.17:0/1996711002 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fcae8154090 con 0x7fcae814f0c0 2024-09-10T21:31:29.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.358+0000 7fcae7400640 1 --2- 172.21.15.17:0/1996711002 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcae814f0c0 0x7fcae80c7ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:29.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.358+0000 7fcae7400640 1 -- 172.21.15.17:0/1996711002 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fcae80cdf20 con 0x7fcae814f0c0 2024-09-10T21:31:29.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.358+0000 7fcae7400640 1 --2- 172.21.15.17:0/1996711002 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcae814f0c0 0x7fcae80c7ac0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fcae81f3fc0 tx=0x7fcad4011e70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:29.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.358+0000 7fcadfe00640 1 -- 172.21.15.17:0/1996711002 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fcad8003600 con 0x7fcae80c89d0 2024-09-10T21:31:29.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.358+0000 7fcadfe00640 1 --2- 172.21.15.17:0/1996711002 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fcac80725e0 0x7fcac8074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:29.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.358+0000 7fcadfe00640 1 -- 172.21.15.17:0/1996711002 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fcad8092ba0 con 0x7fcae80c89d0 2024-09-10T21:31:29.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.358+0000 7fcadfe00640 1 -- 172.21.15.17:0/1996711002 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcad4004460 con 0x7fcae814f0c0 2024-09-10T21:31:29.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.358+0000 7fcadfe00640 1 -- 172.21.15.17:0/1996711002 <== mon.1 v2:172.21.15.31:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fcad400f980 con 0x7fcae814f0c0 2024-09-10T21:31:29.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.358+0000 7fcadfe00640 1 -- 172.21.15.17:0/1996711002 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcae814f0c0 msgr2=0x7fcae80c7ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:29.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.358+0000 7fcadfe00640 1 --2- 172.21.15.17:0/1996711002 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcae814f0c0 0x7fcae80c7ac0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fcae81f3fc0 tx=0x7fcad4011e70 comp rx=0 tx=0).stop 2024-09-10T21:31:29.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.358+0000 7fcadfe00640 1 -- 172.21.15.17:0/1996711002 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcae814f0c0 msgr2=0x7fcae80c7ac0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:31:29.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.358+0000 7fcadfe00640 1 --2- 172.21.15.17:0/1996711002 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcae814f0c0 0x7fcae80c7ac0 secure :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fcae81f3fc0 tx=0x7fcad4011e70 comp rx=0 tx=0).stop 2024-09-10T21:31:29.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.358+0000 7fcaecc00640 1 --2- 172.21.15.17:0/1996711002 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fcac80725e0 0x7fcac8074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:29.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.358+0000 7fcaecc00640 1 --2- 172.21.15.17:0/1996711002 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fcac80725e0 0x7fcac8074aa0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fcae00af000 tx=0x7fcae001b590 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:29.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.366+0000 7fcaeec00640 1 --2- 172.21.15.17:0/1996711002 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcae81f7850 0x7fcae81ffec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:29.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.366+0000 7fcaeec00640 1 -- 172.21.15.17:0/1996711002 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fcae81f7220 con 0x7fcae81f7850 2024-09-10T21:31:29.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.366+0000 7fcae7400640 1 --2- 172.21.15.17:0/1996711002 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcae81f7850 0x7fcae81ffec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:29.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.366+0000 7fcae7400640 1 -- 172.21.15.17:0/1996711002 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fcae80cdaf0 con 0x7fcae81f7850 2024-09-10T21:31:29.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.366+0000 7fcae7400640 1 --2- 172.21.15.17:0/1996711002 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcae81f7850 0x7fcae81ffec0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fcad401a000 tx=0x7fcad40046d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:29.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.366+0000 7fcadfe00640 1 -- 172.21.15.17:0/1996711002 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcad4005570 con 0x7fcae81f7850 2024-09-10T21:31:29.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.366+0000 7fcadfe00640 1 -- 172.21.15.17:0/1996711002 <== mon.1 v2:172.21.15.31:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5042 (secure 0 0 0) 0x7fcad4005570 con 0x7fcae81f7850 2024-09-10T21:31:29.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.366+0000 7fcadfe00640 1 -- 172.21.15.17:0/1996711002 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcae81f7850 msgr2=0x7fcae81ffec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:29.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.366+0000 7fcadfe00640 1 --2- 172.21.15.17:0/1996711002 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcae81f7850 0x7fcae81ffec0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fcad401a000 tx=0x7fcad40046d0 comp rx=0 tx=0).stop 2024-09-10T21:31:29.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.366+0000 7fcadfe00640 1 -- 172.21.15.17:0/1996711002 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcae81f7850 msgr2=0x7fcae81ffec0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:31:29.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.366+0000 7fcadfe00640 1 --2- 172.21.15.17:0/1996711002 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcae81f7850 0x7fcae81ffec0 secure :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fcad401a000 tx=0x7fcad40046d0 comp rx=0 tx=0).stop 2024-09-10T21:31:29.371 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-10T21:31:29.371 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:31:29.371 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:31:29.371 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-10T21:31:29.371 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 58, 2024-09-10T21:31:29.371 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 427816, 2024-09-10T21:31:29.371 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-10T21:31:29.371 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-10T21:31:29.371 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-10T21:31:29.371 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-10T21:31:29.371 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:31:29.372 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 1, 2024-09-10T21:31:29.372 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:31:29.372 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-10T21:31:29.372 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-10T21:31:29.372 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:31:29.372 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:31:29.372 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:31:29.372 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:31:29.372 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:31:29.372 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:31:29.372 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:31:29.372 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:31:29.372 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:31:29.373 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:31:29.373 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:31:29.373 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:31:29.373 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-10T21:31:29.373 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-10T21:31:29.373 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:31:29.373 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:31:29.373 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:31:29.373 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:31:29.373 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:31:29.373 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:31:29.373 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:31:29.373 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:31:29.374 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:31:29.374 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:31:29.374 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:31:29.374 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:31:29.374 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:31:29.374 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-10T21:31:29.374 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-10T21:31:29.374 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-10T21:31:29.374 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-10T21:31:29.374 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-10T21:31:29.374 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "5e2f023f-d6a6-49f4-a256-f4807dce3739", 2024-09-10T21:31:29.374 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:31:29.374 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:31:29.374 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-10T21:31:29.374 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-10T21:31:29.375 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-10T21:31:29.375 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-10T21:31:29.375 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-10T21:31:29.375 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-10T21:31:29.375 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-10T21:31:29.375 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:31:29.375 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-10T21:31:29.375 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:31:29.375 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:31:29.375 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:31:29.375 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:31:29.375 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:31:29.375 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:31:29.375 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:31:29.376 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:31:29.376 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:31:29.376 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:31:29.376 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:31:29.376 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:31:29.376 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-10T21:31:29.376 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:31:29.376 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-10T21:31:29.376 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:31:29.376 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:31:29.376 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:31:29.376 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:31:29.376 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:31:29.377 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:31:29.377 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:31:29.377 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-10T21:31:29.377 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:31:29.377 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:31:29.377 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:31:29.377 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:31:29.377 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-10T21:31:29.377 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:31:29.377 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:31:29.377 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:31:29.377 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:31:29.377 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-10T21:31:29.377 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-10T21:31:29.378 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:31:29.378 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:31:29.378 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:31:29.378 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:31:29.378 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:31:29.378 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:31:29.378 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:31:29.378 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:31:29.378 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:31:29.378 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:31:29.378 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:31:29.378 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3300", 2024-09-10T21:31:29.379 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:31:29.379 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:31:29.379 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:31:29.379 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:31:29.379 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789", 2024-09-10T21:31:29.379 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:31:29.379 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:31:29.379 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:31:29.379 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:31:29.379 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789/0", 2024-09-10T21:31:29.379 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6789/0", 2024-09-10T21:31:29.379 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:31:29.379 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:31:29.379 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:31:29.380 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:31:29.380 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:31:29.380 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:31:29.380 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:31:29.380 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:31:29.380 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:31:29.380 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:31:29.380 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:31:29.380 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3301", 2024-09-10T21:31:29.380 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:31:29.380 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:31:29.380 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:31:29.380 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:31:29.381 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790", 2024-09-10T21:31:29.381 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:31:29.381 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:31:29.381 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:31:29.381 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:31:29.381 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790/0", 2024-09-10T21:31:29.381 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6790/0", 2024-09-10T21:31:29.381 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:31:29.381 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:31:29.381 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:31:29.381 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:31:29.381 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:31:29.381 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:31:29.382 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-10T21:31:29.382 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-10T21:31:29.382 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:31:29.382 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:31:29.382 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:31:29.382 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:31:29.382 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:31:29.382 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:31:29.382 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": [ 2024-09-10T21:31:29.382 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:31:29.382 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:31:29.382 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:31:29.382 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 5 2024-09-10T21:31:29.382 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:31:29.383 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:31:29.383 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-10T21:31:29.383 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:31:29.383 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:31:29.383 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:31:29.383 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 7 2024-09-10T21:31:29.383 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:31:29.383 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:31:29.383 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-10T21:31:29.383 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:31:29.383 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:31:29.383 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:31:29.383 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 4 2024-09-10T21:31:29.383 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:31:29.384 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:31:29.384 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": [ 2024-09-10T21:31:29.384 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:31:29.384 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:31:29.384 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:31:29.384 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 2 2024-09-10T21:31:29.384 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:31:29.384 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:31:29.384 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:31:29.384 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-10T21:31:29.384 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-10T21:31:29.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.366+0000 7fcaeec00640 1 -- 172.21.15.17:0/1996711002 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fcac80725e0 msgr2=0x7fcac8074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:29.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.366+0000 7fcaeec00640 1 --2- 172.21.15.17:0/1996711002 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fcac80725e0 0x7fcac8074aa0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fcae00af000 tx=0x7fcae001b590 comp rx=0 tx=0).stop 2024-09-10T21:31:29.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.366+0000 7fcaeec00640 1 -- 172.21.15.17:0/1996711002 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcae80c89d0 msgr2=0x7fcae81550b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:29.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.366+0000 7fcaeec00640 1 --2- 172.21.15.17:0/1996711002 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcae80c89d0 0x7fcae81550b0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fcad800c530 tx=0x7fcad800c9f0 comp rx=0 tx=0).stop 2024-09-10T21:31:29.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.366+0000 7fcaecc00640 1 -- 172.21.15.17:0/1996711002 reap_dead start 2024-09-10T21:31:29.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.366+0000 7fcaeec00640 1 -- 172.21.15.17:0/1996711002 shutdown_connections 2024-09-10T21:31:29.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.366+0000 7fcaeec00640 1 --2- 172.21.15.17:0/1996711002 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcae80c89d0 0x7fcae81550b0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:29.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.366+0000 7fcaeec00640 1 -- 172.21.15.17:0/1996711002 >> 172.21.15.17:0/1996711002 conn(0x7fcae80702f0 msgr2=0x7fcae80706d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:29.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.366+0000 7fcaeec00640 1 -- 172.21.15.17:0/1996711002 shutdown_connections 2024-09-10T21:31:29.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.366+0000 7fcaeec00640 1 -- 172.21.15.17:0/1996711002 wait complete. 2024-09-10T21:31:29.389 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-10T21:31:29.389 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-09-10T21:31:29.389 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-10T21:31:29.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.534+0000 7fa73e400640 1 -- 172.21.15.17:0/1829847021 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa7380c8ca0 msgr2=0x7fa738157260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:29.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.534+0000 7fa73e400640 1 --2- 172.21.15.17:0/1829847021 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa7380c8ca0 0x7fa738157260 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fa728009840 tx=0x7fa72801c820 comp rx=0 tx=0).stop 2024-09-10T21:31:29.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.534+0000 7fa73e400640 1 -- 172.21.15.17:0/1829847021 shutdown_connections 2024-09-10T21:31:29.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.534+0000 7fa73e400640 1 --2- 172.21.15.17:0/1829847021 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa7380c8ca0 0x7fa738157260 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:29.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.534+0000 7fa73e400640 1 -- 172.21.15.17:0/1829847021 >> 172.21.15.17:0/1829847021 conn(0x7fa7380700d0 msgr2=0x7fa7380704e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:29.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.534+0000 7fa73e400640 1 -- 172.21.15.17:0/1829847021 shutdown_connections 2024-09-10T21:31:29.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.534+0000 7fa73e400640 1 -- 172.21.15.17:0/1829847021 wait complete. 2024-09-10T21:31:29.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.534+0000 7fa73e400640 1 Processor -- start 2024-09-10T21:31:29.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.534+0000 7fa73e400640 1 -- start start 2024-09-10T21:31:29.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.534+0000 7fa73e400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa7380b1b50 0x7fa73814ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:29.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.534+0000 7fa73e400640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa7380c8ca0 0x7fa738153400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:29.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.534+0000 7fa73e400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa7380c9080 0x7fa73814ff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:29.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.534+0000 7fa73e400640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fa7380c87c0 con 0x7fa7380c9080 2024-09-10T21:31:29.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.534+0000 7fa73e400640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fa738151710 con 0x7fa7380b1b50 2024-09-10T21:31:29.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.534+0000 7fa73e400640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7fa738074d40 con 0x7fa7380c8ca0 2024-09-10T21:31:29.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.538+0000 7fa737e00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa7380c9080 0x7fa73814ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:29.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.538+0000 7fa737e00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa7380c9080 0x7fa73814ff00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:46850/0 (socket says 172.21.15.17:46850) 2024-09-10T21:31:29.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.538+0000 7fa737e00640 1 -- 172.21.15.17:0/3299065105 learned_addr learned my addr 172.21.15.17:0/3299065105 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:31:29.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.538+0000 7fa737400640 1 --2- 172.21.15.17:0/3299065105 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa7380b1b50 0x7fa73814ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:29.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.538+0000 7fa72fe00640 1 --2- 172.21.15.17:0/3299065105 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa7380c8ca0 0x7fa738153400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:29.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.538+0000 7fa737e00640 1 -- 172.21.15.17:0/3299065105 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa7380c8ca0 msgr2=0x7fa738153400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:29.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.538+0000 7fa737e00640 1 --2- 172.21.15.17:0/3299065105 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa7380c8ca0 0x7fa738153400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:29.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.538+0000 7fa737e00640 1 -- 172.21.15.17:0/3299065105 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa7380b1b50 msgr2=0x7fa73814ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:29.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.538+0000 7fa737e00640 1 --2- 172.21.15.17:0/3299065105 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa7380b1b50 0x7fa73814ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:29.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.538+0000 7fa737e00640 1 -- 172.21.15.17:0/3299065105 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa738153e50 con 0x7fa7380c9080 2024-09-10T21:31:29.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.538+0000 7fa72fe00640 1 --2- 172.21.15.17:0/3299065105 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa7380c8ca0 0x7fa738153400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:31:29.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.538+0000 7fa737e00640 1 --2- 172.21.15.17:0/3299065105 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa7380c9080 0x7fa73814ff00 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fa73000c7b0 tx=0x7fa73000cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:29.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.538+0000 7fa72f400640 1 -- 172.21.15.17:0/3299065105 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa730004b70 con 0x7fa7380c9080 2024-09-10T21:31:29.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.538+0000 7fa72f400640 1 -- 172.21.15.17:0/3299065105 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa730004e00 con 0x7fa7380c9080 2024-09-10T21:31:29.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.538+0000 7fa72f400640 1 -- 172.21.15.17:0/3299065105 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa73000aab0 con 0x7fa7380c9080 2024-09-10T21:31:29.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.538+0000 7fa73e400640 1 -- 172.21.15.17:0/3299065105 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7381540c0 con 0x7fa7380c9080 2024-09-10T21:31:29.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.538+0000 7fa73e400640 1 -- 172.21.15.17:0/3299065105 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa738154680 con 0x7fa7380c9080 2024-09-10T21:31:29.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.538+0000 7fa73e400640 1 --2- 172.21.15.17:0/3299065105 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa7381f1300 0x7fa738152260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:29.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.538+0000 7fa73e400640 1 -- 172.21.15.17:0/3299065105 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7fa738154850 con 0x7fa7381f1300 2024-09-10T21:31:29.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.538+0000 7fa737400640 1 --2- 172.21.15.17:0/3299065105 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa7381f1300 0x7fa738152260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:29.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.538+0000 7fa737400640 1 -- 172.21.15.17:0/3299065105 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa7380c66a0 con 0x7fa7381f1300 2024-09-10T21:31:29.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.538+0000 7fa737400640 1 --2- 172.21.15.17:0/3299065105 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa7381f1300 0x7fa738152260 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fa738154e30 tx=0x7fa728005cc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:29.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.538+0000 7fa72f400640 1 -- 172.21.15.17:0/3299065105 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fa73000ac50 con 0x7fa7380c9080 2024-09-10T21:31:29.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.542+0000 7fa72f400640 1 --2- 172.21.15.17:0/3299065105 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fa7140725e0 0x7fa714074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:29.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.542+0000 7fa72f400640 1 -- 172.21.15.17:0/3299065105 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fa730093950 con 0x7fa7380c9080 2024-09-10T21:31:29.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.542+0000 7fa72f400640 1 -- 172.21.15.17:0/3299065105 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7280bd070 con 0x7fa7381f1300 2024-09-10T21:31:29.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.542+0000 7fa72f400640 1 -- 172.21.15.17:0/3299065105 <== mon.2 v2:172.21.15.31:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fa7280052c0 con 0x7fa7381f1300 2024-09-10T21:31:29.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.542+0000 7fa72f400640 1 -- 172.21.15.17:0/3299065105 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa7381f1300 msgr2=0x7fa738152260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:29.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.542+0000 7fa72f400640 1 --2- 172.21.15.17:0/3299065105 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa7381f1300 0x7fa738152260 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fa738154e30 tx=0x7fa728005cc0 comp rx=0 tx=0).stop 2024-09-10T21:31:29.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.542+0000 7fa72f400640 1 -- 172.21.15.17:0/3299065105 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa7381f1300 msgr2=0x7fa738152260 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:31:29.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.542+0000 7fa72f400640 1 --2- 172.21.15.17:0/3299065105 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa7381f1300 0x7fa738152260 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:29.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.542+0000 7fa72fe00640 1 --2- 172.21.15.17:0/3299065105 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fa7140725e0 0x7fa714074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:29.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.542+0000 7fa72fe00640 1 --2- 172.21.15.17:0/3299065105 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fa7140725e0 0x7fa714074aa0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fa720001230 tx=0x7fa720009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:29.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.546+0000 7fa73e400640 1 --2- 172.21.15.17:0/3299065105 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa73813ea40 0x7fa73813ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:29.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.546+0000 7fa73e400640 1 -- 172.21.15.17:0/3299065105 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7fa73813f390 con 0x7fa73813ea40 2024-09-10T21:31:29.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.546+0000 7fa737400640 1 --2- 172.21.15.17:0/3299065105 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa73813ea40 0x7fa73813ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:29.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.550+0000 7fa737400640 1 -- 172.21.15.17:0/3299065105 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fa738143e50 con 0x7fa73813ea40 2024-09-10T21:31:29.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.550+0000 7fa737400640 1 --2- 172.21.15.17:0/3299065105 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa73813ea40 0x7fa73813ee50 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fa72801a450 tx=0x7fa7280091e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:29.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.550+0000 7fa72f400640 1 -- 172.21.15.17:0/3299065105 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7280049d0 con 0x7fa73813ea40 2024-09-10T21:31:29.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.550+0000 7fa72f400640 1 -- 172.21.15.17:0/3299065105 <== mon.2 v2:172.21.15.31:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4540 (secure 0 0 0) 0x7fa7280049d0 con 0x7fa73813ea40 2024-09-10T21:31:29.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.550+0000 7fa72f400640 1 -- 172.21.15.17:0/3299065105 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa73813ea40 msgr2=0x7fa73813ee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:29.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.550+0000 7fa72f400640 1 --2- 172.21.15.17:0/3299065105 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa73813ea40 0x7fa73813ee50 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fa72801a450 tx=0x7fa7280091e0 comp rx=0 tx=0).stop 2024-09-10T21:31:29.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.550+0000 7fa72f400640 1 -- 172.21.15.17:0/3299065105 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa73813ea40 msgr2=0x7fa73813ee50 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:31:29.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.550+0000 7fa72f400640 1 --2- 172.21.15.17:0/3299065105 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa73813ea40 0x7fa73813ee50 secure :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fa72801a450 tx=0x7fa7280091e0 comp rx=0 tx=0).stop 2024-09-10T21:31:29.554 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-10T21:31:29.554 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:31:29.554 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:31:29.554 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-10T21:31:29.554 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 58, 2024-09-10T21:31:29.554 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 21855, 2024-09-10T21:31:29.554 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-10T21:31:29.554 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-10T21:31:29.555 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-10T21:31:29.555 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-10T21:31:29.555 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:31:29.555 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 1, 2024-09-10T21:31:29.555 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:31:29.555 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-10T21:31:29.555 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-10T21:31:29.555 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:31:29.555 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:31:29.555 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:31:29.555 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:31:29.555 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:31:29.555 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:31:29.556 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:31:29.556 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:31:29.556 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:31:29.556 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:31:29.556 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:31:29.556 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:31:29.556 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-10T21:31:29.556 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-10T21:31:29.556 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:31:29.556 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:31:29.556 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:31:29.556 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:31:29.556 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:31:29.556 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:31:29.557 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:31:29.557 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:31:29.557 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:31:29.557 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:31:29.557 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:31:29.557 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:31:29.557 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:31:29.557 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-10T21:31:29.557 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-10T21:31:29.557 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-10T21:31:29.557 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-10T21:31:29.557 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-10T21:31:29.558 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "5e2f023f-d6a6-49f4-a256-f4807dce3739", 2024-09-10T21:31:29.558 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:31:29.558 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:31:29.558 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-10T21:31:29.558 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-10T21:31:29.558 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-10T21:31:29.558 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-10T21:31:29.558 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-10T21:31:29.558 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-10T21:31:29.558 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-10T21:31:29.558 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:31:29.558 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-10T21:31:29.558 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:31:29.558 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:31:29.559 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:31:29.559 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:31:29.559 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:31:29.559 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:31:29.559 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:31:29.559 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:31:29.559 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:31:29.559 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:31:29.559 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:31:29.559 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:31:29.559 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-10T21:31:29.559 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:31:29.559 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-10T21:31:29.559 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:31:29.560 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:31:29.560 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:31:29.560 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:31:29.560 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:31:29.560 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:31:29.560 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:31:29.560 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-10T21:31:29.560 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:31:29.560 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:31:29.560 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:31:29.560 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:31:29.560 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-10T21:31:29.560 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:31:29.561 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:31:29.561 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:31:29.561 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:31:29.561 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-10T21:31:29.561 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-10T21:31:29.561 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:31:29.561 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:31:29.561 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:31:29.561 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:31:29.561 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:31:29.561 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:31:29.561 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:31:29.561 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:31:29.561 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:31:29.562 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:31:29.562 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:31:29.562 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3300", 2024-09-10T21:31:29.562 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:31:29.562 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:31:29.562 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:31:29.562 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:31:29.562 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789", 2024-09-10T21:31:29.562 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:31:29.562 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:31:29.562 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:31:29.562 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:31:29.562 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789/0", 2024-09-10T21:31:29.562 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6789/0", 2024-09-10T21:31:29.563 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:31:29.563 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:31:29.563 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:31:29.563 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:31:29.563 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:31:29.563 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:31:29.563 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:31:29.563 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:31:29.563 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:31:29.563 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:31:29.563 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:31:29.563 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3301", 2024-09-10T21:31:29.563 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:31:29.564 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:31:29.564 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:31:29.564 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:31:29.564 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790", 2024-09-10T21:31:29.564 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:31:29.564 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:31:29.564 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:31:29.564 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:31:29.564 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790/0", 2024-09-10T21:31:29.564 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6790/0", 2024-09-10T21:31:29.565 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:31:29.565 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:31:29.565 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:31:29.565 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:31:29.565 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:31:29.565 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:31:29.565 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-10T21:31:29.565 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-10T21:31:29.565 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:31:29.565 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:31:29.565 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:31:29.565 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:31:29.565 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:31:29.566 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:31:29.566 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-10T21:31:29.566 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:31:29.566 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:31:29.566 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:31:29.566 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:31:29.566 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:31:29.566 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:31:29.566 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:31:29.566 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-10T21:31:29.566 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-10T21:31:29.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.550+0000 7fa73e400640 1 -- 172.21.15.17:0/3299065105 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fa7140725e0 msgr2=0x7fa714074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:29.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.550+0000 7fa73e400640 1 --2- 172.21.15.17:0/3299065105 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fa7140725e0 0x7fa714074aa0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fa720001230 tx=0x7fa720009040 comp rx=0 tx=0).stop 2024-09-10T21:31:29.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.550+0000 7fa73e400640 1 -- 172.21.15.17:0/3299065105 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa7380c9080 msgr2=0x7fa73814ff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:29.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.550+0000 7fa73e400640 1 --2- 172.21.15.17:0/3299065105 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa7380c9080 0x7fa73814ff00 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fa73000c7b0 tx=0x7fa73000cc70 comp rx=0 tx=0).stop 2024-09-10T21:31:29.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.550+0000 7fa737e00640 1 -- 172.21.15.17:0/3299065105 reap_dead start 2024-09-10T21:31:29.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.550+0000 7fa73e400640 1 -- 172.21.15.17:0/3299065105 shutdown_connections 2024-09-10T21:31:29.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.550+0000 7fa73e400640 1 --2- 172.21.15.17:0/3299065105 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa7380c9080 0x7fa73814ff00 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:29.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.550+0000 7fa73e400640 1 -- 172.21.15.17:0/3299065105 >> 172.21.15.17:0/3299065105 conn(0x7fa7380700d0 msgr2=0x7fa7380c0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:29.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.550+0000 7fa73e400640 1 -- 172.21.15.17:0/3299065105 shutdown_connections 2024-09-10T21:31:29.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.550+0000 7fa73e400640 1 -- 172.21.15.17:0/3299065105 wait complete. 2024-09-10T21:31:29.571 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-10T21:31:29.572 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-09-10T21:31:29.572 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-09-10T21:31:29.719 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.714+0000 7f773ea00640 1 -- 172.21.15.17:0/2353684277 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f77380cdf20 msgr2=0x7f77380ce3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:29.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.714+0000 7f773ea00640 1 --2- 172.21.15.17:0/2353684277 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f77380cdf20 0x7f77380ce3b0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f7730009bc0 tx=0x7f773001cb10 comp rx=0 tx=0).stop 2024-09-10T21:31:29.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.714+0000 7f773ea00640 1 -- 172.21.15.17:0/2353684277 shutdown_connections 2024-09-10T21:31:29.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.714+0000 7f773ea00640 1 --2- 172.21.15.17:0/2353684277 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f77380cdf20 0x7f77380ce3b0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:29.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.714+0000 7f773ea00640 1 --2- 172.21.15.17:0/2353684277 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f7738152820 0x7f77380b3eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:29.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.714+0000 7f773ea00640 1 -- 172.21.15.17:0/2353684277 >> 172.21.15.17:0/2353684277 conn(0x7f77380702f0 msgr2=0x7f7738070700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:29.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.714+0000 7f773ea00640 1 -- 172.21.15.17:0/2353684277 shutdown_connections 2024-09-10T21:31:29.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.714+0000 7f773ea00640 1 -- 172.21.15.17:0/2353684277 wait complete. 2024-09-10T21:31:29.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.718+0000 7f773ea00640 1 Processor -- start 2024-09-10T21:31:29.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.718+0000 7f773ea00640 1 -- start start 2024-09-10T21:31:29.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.718+0000 7f773ea00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f77380cdf20 0x7f7738149940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:29.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.718+0000 7f773ea00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7738152440 0x7f77380ccbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:29.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.718+0000 7f773ea00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7738152820 0x7f77380cd0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:29.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.718+0000 7f773ea00640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f77380cb5d0 con 0x7f7738152820 2024-09-10T21:31:29.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.718+0000 7f773ea00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f77380cb450 con 0x7f7738152440 2024-09-10T21:31:29.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.718+0000 7f773ea00640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f77380c2ff0 con 0x7f77380cdf20 2024-09-10T21:31:29.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.718+0000 7f773ca00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7738152820 0x7f77380cd0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:29.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.718+0000 7f773ca00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7738152820 0x7f77380cd0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:46866/0 (socket says 172.21.15.17:46866) 2024-09-10T21:31:29.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.718+0000 7f773ca00640 1 -- 172.21.15.17:0/1958323875 learned_addr learned my addr 172.21.15.17:0/1958323875 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:31:29.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.718+0000 7f7737e00640 1 --2- 172.21.15.17:0/1958323875 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f77380cdf20 0x7f7738149940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:29.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.718+0000 7f772fe00640 1 --2- 172.21.15.17:0/1958323875 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7738152440 0x7f77380ccbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:29.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.718+0000 7f773ca00640 1 -- 172.21.15.17:0/1958323875 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f77380cdf20 msgr2=0x7f7738149940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:29.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.718+0000 7f773ca00640 1 --2- 172.21.15.17:0/1958323875 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f77380cdf20 0x7f7738149940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:29.724 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.718+0000 7f773ca00640 1 -- 172.21.15.17:0/1958323875 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7738152440 msgr2=0x7f77380ccbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:29.724 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.718+0000 7f773ca00640 1 --2- 172.21.15.17:0/1958323875 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7738152440 0x7f77380ccbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:29.724 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.718+0000 7f773ca00640 1 -- 172.21.15.17:0/1958323875 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77380cb000 con 0x7f7738152820 2024-09-10T21:31:29.724 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.718+0000 7f7737e00640 1 --2- 172.21.15.17:0/1958323875 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f77380cdf20 0x7f7738149940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:31:29.724 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.718+0000 7f772fe00640 1 --2- 172.21.15.17:0/1958323875 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7738152440 0x7f77380ccbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:31:29.724 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.718+0000 7f773ca00640 1 --2- 172.21.15.17:0/1958323875 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7738152820 0x7f77380cd0f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f77300b0040 tx=0x7f7730003f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:29.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.718+0000 7f7734c00640 1 -- 172.21.15.17:0/1958323875 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77300b2040 con 0x7f7738152820 2024-09-10T21:31:29.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.718+0000 7f7734c00640 1 -- 172.21.15.17:0/1958323875 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7730004040 con 0x7f7738152820 2024-09-10T21:31:29.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.718+0000 7f7734c00640 1 -- 172.21.15.17:0/1958323875 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7730004430 con 0x7f7738152820 2024-09-10T21:31:29.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.718+0000 7f773ea00640 1 -- 172.21.15.17:0/1958323875 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7738149e80 con 0x7f7738152820 2024-09-10T21:31:29.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.718+0000 7f773ea00640 1 -- 172.21.15.17:0/1958323875 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77381572c0 con 0x7f7738152820 2024-09-10T21:31:29.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.722+0000 7f773ea00640 1 -- 172.21.15.17:0/1958323875 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7738158360 con 0x7f7738152820 2024-09-10T21:31:29.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.722+0000 7f7734c00640 1 -- 172.21.15.17:0/1958323875 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f77300b5070 con 0x7f7738152820 2024-09-10T21:31:29.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.722+0000 7f7734c00640 1 --2- 172.21.15.17:0/1958323875 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f77140725e0 0x7f7714074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:29.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.722+0000 7f7734c00640 1 -- 172.21.15.17:0/1958323875 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f773012d5a0 con 0x7f7738152820 2024-09-10T21:31:29.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.722+0000 7f7737e00640 1 --2- 172.21.15.17:0/1958323875 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f77140725e0 0x7f7714074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:29.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.722+0000 7f7737e00640 1 --2- 172.21.15.17:0/1958323875 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f77140725e0 0x7f7714074aa0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f7728007a60 tx=0x7f7728009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:29.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.726+0000 7f7734c00640 1 -- 172.21.15.17:0/1958323875 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f77300c4060 con 0x7f7738152820 2024-09-10T21:31:29.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.962+0000 7f773ea00640 1 -- 172.21.15.17:0/1958323875 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) -- 0x7f77380b3f90 con 0x7f7738152820 2024-09-10T21:31:29.968 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.966+0000 7f7734c00640 1 -- 172.21.15.17:0/1958323875 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) ==== 57+0+0 (secure 0 0 0) 0x7f7738149e80 con 0x7f7738152820 2024-09-10T21:31:29.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.966+0000 7f773ea00640 1 -- 172.21.15.17:0/1958323875 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f77140725e0 msgr2=0x7f7714074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:29.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.966+0000 7f773ea00640 1 --2- 172.21.15.17:0/1958323875 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f77140725e0 0x7f7714074aa0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f7728007a60 tx=0x7f7728009040 comp rx=0 tx=0).stop 2024-09-10T21:31:29.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.966+0000 7f773ea00640 1 -- 172.21.15.17:0/1958323875 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7738152820 msgr2=0x7f77380cd0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:29.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.966+0000 7f773ea00640 1 --2- 172.21.15.17:0/1958323875 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7738152820 0x7f77380cd0f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f77300b0040 tx=0x7f7730003f40 comp rx=0 tx=0).stop 2024-09-10T21:31:29.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.966+0000 7f773ea00640 1 -- 172.21.15.17:0/1958323875 shutdown_connections 2024-09-10T21:31:29.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.966+0000 7f773ea00640 1 --2- 172.21.15.17:0/1958323875 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f77140725e0 0x7f7714074aa0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:29.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.966+0000 7f773ea00640 1 --2- 172.21.15.17:0/1958323875 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7738152820 0x7f77380cd0f0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:29.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.966+0000 7f773ea00640 1 --2- 172.21.15.17:0/1958323875 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7738152440 0x7f77380ccbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:29.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.966+0000 7f773ea00640 1 --2- 172.21.15.17:0/1958323875 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f77380cdf20 0x7f7738149940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:29.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.966+0000 7f773ea00640 1 -- 172.21.15.17:0/1958323875 >> 172.21.15.17:0/1958323875 conn(0x7f77380702f0 msgr2=0x7f7738151aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:29.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.966+0000 7f773ea00640 1 -- 172.21.15.17:0/1958323875 shutdown_connections 2024-09-10T21:31:29.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:29.966+0000 7f773ea00640 1 -- 172.21.15.17:0/1958323875 wait complete. 2024-09-10T21:31:29.991 INFO:tasks.mon_thrash.mon_thrasher:waiting for 10.0 secs before continuing thrashing 2024-09-10T21:31:39.992 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-09-10T21:31:39.993 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-10T21:31:40.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.162+0000 7feaf4a00640 1 -- 172.21.15.17:0/1396386401 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feaf0158c20 msgr2=0x7feaf0070670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:40.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.162+0000 7feaf4a00640 1 --2- 172.21.15.17:0/1396386401 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feaf0158c20 0x7feaf0070670 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7feaf00718c0 tx=0x7feadc01f0b0 comp rx=0 tx=0).stop 2024-09-10T21:31:40.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.162+0000 7feaf4a00640 1 -- 172.21.15.17:0/1396386401 shutdown_connections 2024-09-10T21:31:40.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.162+0000 7feaf4a00640 1 --2- 172.21.15.17:0/1396386401 >> v2:172.21.15.31:3300/0 conn(0x7feaf00b3e00 0x7feaf0070bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:40.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.162+0000 7feaf4a00640 1 --2- 172.21.15.17:0/1396386401 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feaf0158c20 0x7feaf0070670 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:40.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.162+0000 7feaf4a00640 1 -- 172.21.15.17:0/1396386401 >> 172.21.15.17:0/1396386401 conn(0x7feaf00cdef0 msgr2=0x7feaf00ce2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:40.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.162+0000 7feaf4a00640 1 -- 172.21.15.17:0/1396386401 shutdown_connections 2024-09-10T21:31:40.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.162+0000 7feaf4a00640 1 -- 172.21.15.17:0/1396386401 wait complete. 2024-09-10T21:31:40.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.162+0000 7feaf4a00640 1 Processor -- start 2024-09-10T21:31:40.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.166+0000 7feaf4a00640 1 -- start start 2024-09-10T21:31:40.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.166+0000 7feaf4a00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7feaf00b3e00 0x7feaf015b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:40.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.166+0000 7feaf4a00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feaf01524d0 0x7feaf014fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:40.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.166+0000 7feaf4a00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feaf0158c20 0x7feaf0150a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:40.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.166+0000 7feaf4a00640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7feaf00c9350 con 0x7feaf01524d0 2024-09-10T21:31:40.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.166+0000 7feaf4a00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7feaf00c9070 con 0x7feaf0158c20 2024-09-10T21:31:40.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.166+0000 7feaf4a00640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7feaf015b9e0 con 0x7feaf00b3e00 2024-09-10T21:31:40.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.166+0000 7feaede00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feaf01524d0 0x7feaf014fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:40.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.166+0000 7feaede00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feaf01524d0 0x7feaf014fd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42662/0 (socket says 172.21.15.17:42662) 2024-09-10T21:31:40.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.166+0000 7feaede00640 1 -- 172.21.15.17:0/2511593598 learned_addr learned my addr 172.21.15.17:0/2511593598 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:31:40.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.166+0000 7feaef200640 1 --2- 172.21.15.17:0/2511593598 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feaf0158c20 0x7feaf0150a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:40.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.166+0000 7feaee800640 1 --2- 172.21.15.17:0/2511593598 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7feaf00b3e00 0x7feaf015b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:40.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.166+0000 7feaede00640 1 -- 172.21.15.17:0/2511593598 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7feaf00b3e00 msgr2=0x7feaf015b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:40.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.166+0000 7feaede00640 1 --2- 172.21.15.17:0/2511593598 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7feaf00b3e00 0x7feaf015b190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:40.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.166+0000 7feaede00640 1 -- 172.21.15.17:0/2511593598 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feaf0158c20 msgr2=0x7feaf0150a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:40.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.166+0000 7feaede00640 1 --2- 172.21.15.17:0/2511593598 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feaf0158c20 0x7feaf0150a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:40.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.166+0000 7feaede00640 1 -- 172.21.15.17:0/2511593598 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feaf0150400 con 0x7feaf01524d0 2024-09-10T21:31:40.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.166+0000 7feaede00640 1 --2- 172.21.15.17:0/2511593598 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feaf01524d0 0x7feaf014fd00 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7feadc0b3040 tx=0x7feadc004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:40.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.166+0000 7feae6a00640 1 -- 172.21.15.17:0/2511593598 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feadc0be070 con 0x7feaf01524d0 2024-09-10T21:31:40.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.166+0000 7feae6a00640 1 -- 172.21.15.17:0/2511593598 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7feadc0053e0 con 0x7feaf01524d0 2024-09-10T21:31:40.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.166+0000 7feae6a00640 1 -- 172.21.15.17:0/2511593598 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feadc0b5040 con 0x7feaf01524d0 2024-09-10T21:31:40.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.166+0000 7feaf4a00640 1 -- 172.21.15.17:0/2511593598 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feaf01f0d40 con 0x7feaf01524d0 2024-09-10T21:31:40.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.166+0000 7feaf4a00640 1 -- 172.21.15.17:0/2511593598 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feaf01f0fe0 con 0x7feaf01524d0 2024-09-10T21:31:40.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.166+0000 7feaf4a00640 1 -- 172.21.15.17:0/2511593598 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feaf015bf00 con 0x7feaf01524d0 2024-09-10T21:31:40.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.170+0000 7feae6a00640 1 -- 172.21.15.17:0/2511593598 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7feadc0b8030 con 0x7feaf01524d0 2024-09-10T21:31:40.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.170+0000 7feae6a00640 1 --2- 172.21.15.17:0/2511593598 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fead00725e0 0x7fead0074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:40.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.170+0000 7feae6a00640 1 -- 172.21.15.17:0/2511593598 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7feadc1303f0 con 0x7feaf01524d0 2024-09-10T21:31:40.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.170+0000 7feaee800640 1 --2- 172.21.15.17:0/2511593598 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fead00725e0 0x7fead0074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:40.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.170+0000 7feaee800640 1 --2- 172.21.15.17:0/2511593598 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fead00725e0 0x7fead0074aa0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7feae0007fd0 tx=0x7feae0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:40.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.174+0000 7feae6a00640 1 -- 172.21.15.17:0/2511593598 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7feadc018c10 con 0x7feaf01524d0 2024-09-10T21:31:40.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.442+0000 7feaf4a00640 1 -- 172.21.15.17:0/2511593598 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7feaf00c8ad0 con 0x7feaf01524d0 2024-09-10T21:31:40.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.442+0000 7feae6a00640 1 -- 172.21.15.17:0/2511593598 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1557 (secure 0 0 0) 0x7feadc0fafe0 con 0x7feaf01524d0 2024-09-10T21:31:40.446 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":58,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":12,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5e2f023f-d6a6-49f4-a256-f4807dce3739","modified":"2024-09-10T21:23:46.892130Z","created":"2024-09-10T21:23:46.892130Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3301","nonce":0},{"type":"v1","addr":"172.21.15.31:6790","nonce":0}]},"addr":"172.21.15.31:6790/0","public_addr":"172.21.15.31:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-10T21:31:40.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.446+0000 7feaf4a00640 1 -- 172.21.15.17:0/2511593598 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fead00725e0 msgr2=0x7fead0074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:40.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.446+0000 7feaf4a00640 1 --2- 172.21.15.17:0/2511593598 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fead00725e0 0x7fead0074aa0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7feae0007fd0 tx=0x7feae0009040 comp rx=0 tx=0).stop 2024-09-10T21:31:40.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.446+0000 7feaf4a00640 1 -- 172.21.15.17:0/2511593598 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feaf01524d0 msgr2=0x7feaf014fd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:40.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.446+0000 7feaf4a00640 1 --2- 172.21.15.17:0/2511593598 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feaf01524d0 0x7feaf014fd00 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7feadc0b3040 tx=0x7feadc004600 comp rx=0 tx=0).stop 2024-09-10T21:31:40.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.446+0000 7feaf4a00640 1 -- 172.21.15.17:0/2511593598 shutdown_connections 2024-09-10T21:31:40.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.446+0000 7feaf4a00640 1 --2- 172.21.15.17:0/2511593598 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fead00725e0 0x7fead0074aa0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:40.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.446+0000 7feaf4a00640 1 --2- 172.21.15.17:0/2511593598 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feaf0158c20 0x7feaf0150a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:40.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.446+0000 7feaf4a00640 1 --2- 172.21.15.17:0/2511593598 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feaf01524d0 0x7feaf014fd00 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:40.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.446+0000 7feaf4a00640 1 --2- 172.21.15.17:0/2511593598 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7feaf00b3e00 0x7feaf015b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:40.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.446+0000 7feaf4a00640 1 -- 172.21.15.17:0/2511593598 >> 172.21.15.17:0/2511593598 conn(0x7feaf00cdef0 msgr2=0x7feaf014f270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:40.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.446+0000 7feaf4a00640 1 -- 172.21.15.17:0/2511593598 shutdown_connections 2024-09-10T21:31:40.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.446+0000 7feaf4a00640 1 -- 172.21.15.17:0/2511593598 wait complete. 2024-09-10T21:31:40.471 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-09-10T21:31:40.471 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-09-10T21:31:40.472 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-09-10T21:31:40.472 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-10T21:31:40.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.650+0000 7f9d38a00640 1 -- 172.21.15.17:0/1767181798 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9d340c1e90 msgr2=0x7f9d340c2300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:40.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.650+0000 7f9d38a00640 1 --2- 172.21.15.17:0/1767181798 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9d340c1e90 0x7f9d340c2300 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f9d2c009bc0 tx=0x7f9d2c01f3b0 comp rx=0 tx=0).stop 2024-09-10T21:31:40.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.650+0000 7f9d38a00640 1 -- 172.21.15.17:0/1767181798 shutdown_connections 2024-09-10T21:31:40.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.650+0000 7f9d38a00640 1 --2- 172.21.15.17:0/1767181798 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9d340c1e90 0x7f9d340c2300 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:40.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.650+0000 7f9d38a00640 1 --2- 172.21.15.17:0/1767181798 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f9d34151400 0x7f9d340b1be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:40.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.650+0000 7f9d38a00640 1 -- 172.21.15.17:0/1767181798 >> 172.21.15.17:0/1767181798 conn(0x7f9d340c8ab0 msgr2=0x7f9d340c8ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:40.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.650+0000 7f9d38a00640 1 -- 172.21.15.17:0/1767181798 shutdown_connections 2024-09-10T21:31:40.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.650+0000 7f9d38a00640 1 -- 172.21.15.17:0/1767181798 wait complete. 2024-09-10T21:31:40.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.650+0000 7f9d38a00640 1 Processor -- start 2024-09-10T21:31:40.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.650+0000 7f9d38a00640 1 -- start start 2024-09-10T21:31:40.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.650+0000 7f9d38a00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f9d340726b0 0x7f9d341f42f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:40.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.650+0000 7f9d38a00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9d340c1e90 0x7f9d341f0d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:40.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.650+0000 7f9d38a00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9d34151400 0x7f9d340c68b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:40.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.650+0000 7f9d38a00640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f9d340c8450 con 0x7f9d34151400 2024-09-10T21:31:40.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.650+0000 7f9d38a00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f9d340c82d0 con 0x7f9d340c1e90 2024-09-10T21:31:40.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.650+0000 7f9d38a00640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f9d34072ad0 con 0x7f9d340726b0 2024-09-10T21:31:40.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.650+0000 7f9d33e00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9d34151400 0x7f9d340c68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:40.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.650+0000 7f9d33e00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9d34151400 0x7f9d340c68b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42672/0 (socket says 172.21.15.17:42672) 2024-09-10T21:31:40.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.650+0000 7f9d33e00640 1 -- 172.21.15.17:0/143427746 learned_addr learned my addr 172.21.15.17:0/143427746 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:31:40.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.650+0000 7f9d33400640 1 --2- 172.21.15.17:0/143427746 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f9d340726b0 0x7f9d341f42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:40.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.650+0000 7f9d32a00640 1 --2- 172.21.15.17:0/143427746 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9d340c1e90 0x7f9d341f0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:40.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.650+0000 7f9d33e00640 1 -- 172.21.15.17:0/143427746 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f9d340726b0 msgr2=0x7f9d341f42f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:40.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.650+0000 7f9d33e00640 1 --2- 172.21.15.17:0/143427746 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f9d340726b0 0x7f9d341f42f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:40.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.650+0000 7f9d33e00640 1 -- 172.21.15.17:0/143427746 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9d340c1e90 msgr2=0x7f9d341f0d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:40.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.650+0000 7f9d33e00640 1 --2- 172.21.15.17:0/143427746 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9d340c1e90 0x7f9d341f0d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:40.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.650+0000 7f9d33e00640 1 -- 172.21.15.17:0/143427746 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d341e3cd0 con 0x7f9d34151400 2024-09-10T21:31:40.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.650+0000 7f9d33400640 1 --2- 172.21.15.17:0/143427746 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f9d340726b0 0x7f9d341f42f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:31:40.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.654+0000 7f9d33e00640 1 --2- 172.21.15.17:0/143427746 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9d34151400 0x7f9d340c68b0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f9d2c009b10 tx=0x7f9d2c003f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:40.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.654+0000 7f9d2b400640 1 -- 172.21.15.17:0/143427746 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d2c0bd030 con 0x7f9d34151400 2024-09-10T21:31:40.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.654+0000 7f9d2b400640 1 -- 172.21.15.17:0/143427746 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9d2c004a50 con 0x7f9d34151400 2024-09-10T21:31:40.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.654+0000 7f9d2b400640 1 -- 172.21.15.17:0/143427746 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d2c0b4040 con 0x7f9d34151400 2024-09-10T21:31:40.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.654+0000 7f9d38a00640 1 -- 172.21.15.17:0/143427746 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d3406ee10 con 0x7f9d34151400 2024-09-10T21:31:40.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.654+0000 7f9d38a00640 1 -- 172.21.15.17:0/143427746 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d3406f3d0 con 0x7f9d34151400 2024-09-10T21:31:40.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.654+0000 7f9d38a00640 1 --2- 172.21.15.17:0/143427746 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9d341f03a0 0x7f9d340c6df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:40.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.654+0000 7f9d38a00640 1 -- 172.21.15.17:0/143427746 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f9d340c2300 con 0x7f9d341f03a0 2024-09-10T21:31:40.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.654+0000 7f9d33400640 1 --2- 172.21.15.17:0/143427746 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9d341f03a0 0x7f9d340c6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:40.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.654+0000 7f9d33400640 1 -- 172.21.15.17:0/143427746 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9d340c8790 con 0x7f9d341f03a0 2024-09-10T21:31:40.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.654+0000 7f9d33400640 1 --2- 172.21.15.17:0/143427746 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9d341f03a0 0x7f9d340c6df0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f9d240119b0 tx=0x7f9d24011e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:40.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.654+0000 7f9d2b400640 1 -- 172.21.15.17:0/143427746 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d2401f070 con 0x7f9d341f03a0 2024-09-10T21:31:40.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.654+0000 7f9d2b400640 1 -- 172.21.15.17:0/143427746 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f9d2400fd60 con 0x7f9d341f03a0 2024-09-10T21:31:40.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.654+0000 7f9d2b400640 1 -- 172.21.15.17:0/143427746 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9d341f03a0 msgr2=0x7f9d340c6df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:40.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.654+0000 7f9d2b400640 1 --2- 172.21.15.17:0/143427746 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9d341f03a0 0x7f9d340c6df0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f9d240119b0 tx=0x7f9d24011e70 comp rx=0 tx=0).stop 2024-09-10T21:31:40.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.654+0000 7f9d2b400640 1 -- 172.21.15.17:0/143427746 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9d341f03a0 msgr2=0x7f9d340c6df0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:31:40.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.654+0000 7f9d2b400640 1 --2- 172.21.15.17:0/143427746 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9d341f03a0 0x7f9d340c6df0 secure :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f9d240119b0 tx=0x7f9d24011e70 comp rx=0 tx=0).stop 2024-09-10T21:31:40.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.654+0000 7f9d2b400640 1 -- 172.21.15.17:0/143427746 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f9d2c0af600 con 0x7f9d34151400 2024-09-10T21:31:40.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.658+0000 7f9d2b400640 1 --2- 172.21.15.17:0/143427746 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f9d140720c0 0x7f9d14074580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:40.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.658+0000 7f9d2b400640 1 -- 172.21.15.17:0/143427746 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f9d2c1312b0 con 0x7f9d34151400 2024-09-10T21:31:40.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.658+0000 7f9d33400640 1 --2- 172.21.15.17:0/143427746 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f9d140720c0 0x7f9d14074580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:40.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.658+0000 7f9d33400640 1 --2- 172.21.15.17:0/143427746 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f9d140720c0 0x7f9d14074580 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f9d24011eb0 tx=0x7f9d24016000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:40.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.662+0000 7f9d38a00640 1 --2- 172.21.15.17:0/143427746 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9d34148a30 0x7f9d341f3a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:40.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.662+0000 7f9d38a00640 1 -- 172.21.15.17:0/143427746 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f9d341f3f90 con 0x7f9d34148a30 2024-09-10T21:31:40.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.662+0000 7f9d32a00640 1 --2- 172.21.15.17:0/143427746 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9d34148a30 0x7f9d341f3a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:40.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.662+0000 7f9d32a00640 1 -- 172.21.15.17:0/143427746 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f9d341e4710 con 0x7f9d34148a30 2024-09-10T21:31:40.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.662+0000 7f9d32a00640 1 --2- 172.21.15.17:0/143427746 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9d34148a30 0x7f9d341f3a50 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f9d341e1880 tx=0x7f9d2000e960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:40.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.662+0000 7f9d2b400640 1 -- 172.21.15.17:0/143427746 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d20005020 con 0x7f9d34148a30 2024-09-10T21:31:40.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.662+0000 7f9d2b400640 1 -- 172.21.15.17:0/143427746 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4878 (secure 0 0 0) 0x7f9d20005020 con 0x7f9d34148a30 2024-09-10T21:31:40.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.662+0000 7f9d2b400640 1 -- 172.21.15.17:0/143427746 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9d34148a30 msgr2=0x7f9d341f3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:40.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.662+0000 7f9d2b400640 1 --2- 172.21.15.17:0/143427746 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9d34148a30 0x7f9d341f3a50 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f9d341e1880 tx=0x7f9d2000e960 comp rx=0 tx=0).stop 2024-09-10T21:31:40.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.662+0000 7f9d2b400640 1 -- 172.21.15.17:0/143427746 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9d34148a30 msgr2=0x7f9d341f3a50 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:31:40.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.662+0000 7f9d2b400640 1 --2- 172.21.15.17:0/143427746 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9d34148a30 0x7f9d341f3a50 secure :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f9d341e1880 tx=0x7f9d2000e960 comp rx=0 tx=0).stop 2024-09-10T21:31:40.668 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-10T21:31:40.668 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:31:40.668 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:31:40.668 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "leader", 2024-09-10T21:31:40.668 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 58, 2024-09-10T21:31:40.668 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 218501, 2024-09-10T21:31:40.668 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-10T21:31:40.668 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-10T21:31:40.668 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-10T21:31:40.669 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-10T21:31:40.669 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:31:40.669 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 13, 2024-09-10T21:31:40.669 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:31:40.669 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-10T21:31:40.669 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-10T21:31:40.669 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:31:40.669 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:31:40.669 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:31:40.669 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:31:40.669 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:31:40.669 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:31:40.669 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:31:40.669 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:31:40.669 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:31:40.670 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:31:40.670 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:31:40.670 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:31:40.670 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-10T21:31:40.670 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-10T21:31:40.670 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:31:40.670 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:31:40.670 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:31:40.670 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:31:40.670 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:31:40.670 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:31:40.670 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:31:40.670 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:31:40.670 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:31:40.671 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:31:40.671 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:31:40.671 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:31:40.671 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:31:40.671 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-10T21:31:40.671 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-10T21:31:40.671 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-10T21:31:40.671 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-10T21:31:40.671 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-10T21:31:40.671 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "5e2f023f-d6a6-49f4-a256-f4807dce3739", 2024-09-10T21:31:40.671 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:31:40.671 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:31:40.671 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-10T21:31:40.671 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-10T21:31:40.672 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-10T21:31:40.672 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-10T21:31:40.672 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-10T21:31:40.672 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-10T21:31:40.672 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-10T21:31:40.672 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:31:40.672 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-10T21:31:40.672 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:31:40.672 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:31:40.672 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:31:40.672 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:31:40.672 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:31:40.672 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:31:40.672 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:31:40.673 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:31:40.673 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:31:40.673 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:31:40.673 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:31:40.673 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:31:40.673 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-10T21:31:40.673 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:31:40.673 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-10T21:31:40.673 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:31:40.673 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:31:40.673 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:31:40.673 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:31:40.673 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:31:40.673 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:31:40.674 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:31:40.674 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-10T21:31:40.674 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:31:40.674 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:31:40.674 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:31:40.674 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:31:40.674 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-10T21:31:40.674 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:31:40.674 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:31:40.674 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:31:40.674 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:31:40.674 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-10T21:31:40.674 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-10T21:31:40.674 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:31:40.675 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:31:40.675 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:31:40.675 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:31:40.675 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:31:40.675 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:31:40.675 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:31:40.675 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:31:40.675 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:31:40.675 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:31:40.675 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:31:40.675 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3300", 2024-09-10T21:31:40.675 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:31:40.675 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:31:40.675 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:31:40.676 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:31:40.676 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789", 2024-09-10T21:31:40.676 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:31:40.676 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:31:40.676 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:31:40.676 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:31:40.676 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789/0", 2024-09-10T21:31:40.676 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6789/0", 2024-09-10T21:31:40.676 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:31:40.676 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:31:40.676 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:31:40.676 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:31:40.676 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:31:40.676 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:31:40.677 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:31:40.677 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:31:40.677 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:31:40.677 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:31:40.677 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:31:40.677 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3301", 2024-09-10T21:31:40.677 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:31:40.677 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:31:40.677 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:31:40.677 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:31:40.677 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790", 2024-09-10T21:31:40.677 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:31:40.677 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:31:40.677 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:31:40.677 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:31:40.678 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790/0", 2024-09-10T21:31:40.678 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6790/0", 2024-09-10T21:31:40.678 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:31:40.678 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:31:40.678 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:31:40.678 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:31:40.678 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:31:40.678 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:31:40.678 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-10T21:31:40.678 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-10T21:31:40.678 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:31:40.678 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:31:40.678 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:31:40.678 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:31:40.679 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:31:40.679 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:31:40.679 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": [ 2024-09-10T21:31:40.679 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:31:40.679 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:31:40.679 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:31:40.679 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:31:40.679 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:31:40.679 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:31:40.679 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-10T21:31:40.679 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:31:40.679 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:31:40.679 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:31:40.679 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:31:40.680 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:31:40.680 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:31:40.680 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-10T21:31:40.680 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:31:40.680 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:31:40.680 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:31:40.680 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 4 2024-09-10T21:31:40.680 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:31:40.680 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:31:40.680 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:31:40.680 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-10T21:31:40.680 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-10T21:31:40.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.662+0000 7f9d38a00640 1 -- 172.21.15.17:0/143427746 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f9d140720c0 msgr2=0x7f9d14074580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:40.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.662+0000 7f9d38a00640 1 --2- 172.21.15.17:0/143427746 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f9d140720c0 0x7f9d14074580 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f9d24011eb0 tx=0x7f9d24016000 comp rx=0 tx=0).stop 2024-09-10T21:31:40.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.662+0000 7f9d38a00640 1 -- 172.21.15.17:0/143427746 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9d34151400 msgr2=0x7f9d340c68b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:40.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.662+0000 7f9d38a00640 1 --2- 172.21.15.17:0/143427746 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9d34151400 0x7f9d340c68b0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f9d2c009b10 tx=0x7f9d2c003f60 comp rx=0 tx=0).stop 2024-09-10T21:31:40.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.662+0000 7f9d33e00640 1 -- 172.21.15.17:0/143427746 reap_dead start 2024-09-10T21:31:40.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.662+0000 7f9d33e00640 1 -- 172.21.15.17:0/143427746 reap_dead start 2024-09-10T21:31:40.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.662+0000 7f9d38a00640 1 -- 172.21.15.17:0/143427746 shutdown_connections 2024-09-10T21:31:40.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.662+0000 7f9d38a00640 1 -- 172.21.15.17:0/143427746 >> 172.21.15.17:0/143427746 conn(0x7f9d340c8ab0 msgr2=0x7f9d340c5d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:40.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.662+0000 7f9d38a00640 1 -- 172.21.15.17:0/143427746 shutdown_connections 2024-09-10T21:31:40.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.662+0000 7f9d38a00640 1 -- 172.21.15.17:0/143427746 wait complete. 2024-09-10T21:31:40.690 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-10T21:31:40.690 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-09-10T21:31:40.690 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-10T21:31:40.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.850+0000 7f0a36400640 1 -- 172.21.15.17:0/2688266347 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0a30158510 msgr2=0x7f0a30158980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:40.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.850+0000 7f0a36400640 1 --2- 172.21.15.17:0/2688266347 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0a30158510 0x7f0a30158980 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f0a28009c80 tx=0x7f0a2801f5a0 comp rx=0 tx=0).stop 2024-09-10T21:31:40.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.850+0000 7f0a36400640 1 -- 172.21.15.17:0/2688266347 shutdown_connections 2024-09-10T21:31:40.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.850+0000 7f0a36400640 1 --2- 172.21.15.17:0/2688266347 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0a30158510 0x7f0a30158980 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:40.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.850+0000 7f0a36400640 1 --2- 172.21.15.17:0/2688266347 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f0a300ce7b0 0x7f0a300cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:40.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.850+0000 7f0a36400640 1 -- 172.21.15.17:0/2688266347 >> 172.21.15.17:0/2688266347 conn(0x7f0a30068d90 msgr2=0x7f0a300691a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:40.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.850+0000 7f0a36400640 1 -- 172.21.15.17:0/2688266347 shutdown_connections 2024-09-10T21:31:40.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.850+0000 7f0a36400640 1 -- 172.21.15.17:0/2688266347 wait complete. 2024-09-10T21:31:40.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.854+0000 7f0a36400640 1 Processor -- start 2024-09-10T21:31:40.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.854+0000 7f0a36400640 1 -- start start 2024-09-10T21:31:40.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.854+0000 7f0a36400640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f0a300ce7b0 0x7f0a300caf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:40.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.854+0000 7f0a36400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0a30152650 0x7f0a30154f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:40.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.854+0000 7f0a36400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0a30158510 0x7f0a3014d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:40.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.854+0000 7f0a36400640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f0a300b16c0 con 0x7f0a30158510 2024-09-10T21:31:40.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.854+0000 7f0a36400640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f0a300b13e0 con 0x7f0a30152650 2024-09-10T21:31:40.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.854+0000 7f0a36400640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f0a30152a30 con 0x7f0a300ce7b0 2024-09-10T21:31:40.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.854+0000 7f0a2fe00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0a30158510 0x7f0a3014d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:40.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.854+0000 7f0a2fe00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0a30158510 0x7f0a3014d4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42682/0 (socket says 172.21.15.17:42682) 2024-09-10T21:31:40.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.854+0000 7f0a2fe00640 1 -- 172.21.15.17:0/4207653740 learned_addr learned my addr 172.21.15.17:0/4207653740 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:31:40.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.854+0000 7f0a2ea00640 1 --2- 172.21.15.17:0/4207653740 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0a30152650 0x7f0a30154f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:40.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.854+0000 7f0a2f400640 1 --2- 172.21.15.17:0/4207653740 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f0a300ce7b0 0x7f0a300caf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:40.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.854+0000 7f0a2fe00640 1 -- 172.21.15.17:0/4207653740 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f0a300ce7b0 msgr2=0x7f0a300caf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:40.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.854+0000 7f0a2fe00640 1 --2- 172.21.15.17:0/4207653740 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f0a300ce7b0 0x7f0a300caf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:40.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.854+0000 7f0a2fe00640 1 -- 172.21.15.17:0/4207653740 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0a30152650 msgr2=0x7f0a30154f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:40.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.854+0000 7f0a2fe00640 1 --2- 172.21.15.17:0/4207653740 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0a30152650 0x7f0a30154f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:40.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.854+0000 7f0a2fe00640 1 -- 172.21.15.17:0/4207653740 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a301ec9d0 con 0x7f0a30158510 2024-09-10T21:31:40.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.854+0000 7f0a2ea00640 1 --2- 172.21.15.17:0/4207653740 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0a30152650 0x7f0a30154f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:31:40.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.854+0000 7f0a2fe00640 1 --2- 172.21.15.17:0/4207653740 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0a30158510 0x7f0a3014d4a0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f0a28009bd0 tx=0x7f0a28009350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:40.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.854+0000 7f0a27400640 1 -- 172.21.15.17:0/4207653740 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a28004c50 con 0x7f0a30158510 2024-09-10T21:31:40.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.854+0000 7f0a36400640 1 -- 172.21.15.17:0/4207653740 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a3006fa00 con 0x7f0a30158510 2024-09-10T21:31:40.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.854+0000 7f0a27400640 1 -- 172.21.15.17:0/4207653740 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0a280be030 con 0x7f0a30158510 2024-09-10T21:31:40.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.854+0000 7f0a27400640 1 -- 172.21.15.17:0/4207653740 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a280afb00 con 0x7f0a30158510 2024-09-10T21:31:40.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.854+0000 7f0a36400640 1 -- 172.21.15.17:0/4207653740 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a300cdae0 con 0x7f0a30158510 2024-09-10T21:31:40.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.858+0000 7f0a36400640 1 --2- 172.21.15.17:0/4207653740 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0a301554b0 0x7f0a3014d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:40.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.858+0000 7f0a36400640 1 -- 172.21.15.17:0/4207653740 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f0a300cdce0 con 0x7f0a301554b0 2024-09-10T21:31:40.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.858+0000 7f0a2f400640 1 --2- 172.21.15.17:0/4207653740 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0a301554b0 0x7f0a3014d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:40.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.858+0000 7f0a2f400640 1 -- 172.21.15.17:0/4207653740 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0a300ceb90 con 0x7f0a301554b0 2024-09-10T21:31:40.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.858+0000 7f0a2f400640 1 --2- 172.21.15.17:0/4207653740 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0a301554b0 0x7f0a3014d9e0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f0a2000e500 tx=0x7f0a2000e9c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:40.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.858+0000 7f0a27400640 1 -- 172.21.15.17:0/4207653740 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a200052e0 con 0x7f0a301554b0 2024-09-10T21:31:40.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.858+0000 7f0a27400640 1 -- 172.21.15.17:0/4207653740 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f0a280afca0 con 0x7f0a30158510 2024-09-10T21:31:40.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.858+0000 7f0a27400640 1 --2- 172.21.15.17:0/4207653740 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f0a100724e0 0x7f0a100749a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:40.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.858+0000 7f0a27400640 1 -- 172.21.15.17:0/4207653740 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f0a28132050 con 0x7f0a30158510 2024-09-10T21:31:40.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.858+0000 7f0a27400640 1 -- 172.21.15.17:0/4207653740 <== mon.1 v2:172.21.15.31:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f0a20018070 con 0x7f0a301554b0 2024-09-10T21:31:40.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.858+0000 7f0a27400640 1 -- 172.21.15.17:0/4207653740 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0a301554b0 msgr2=0x7f0a3014d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:40.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.858+0000 7f0a27400640 1 --2- 172.21.15.17:0/4207653740 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0a301554b0 0x7f0a3014d9e0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f0a2000e500 tx=0x7f0a2000e9c0 comp rx=0 tx=0).stop 2024-09-10T21:31:40.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.858+0000 7f0a27400640 1 -- 172.21.15.17:0/4207653740 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0a301554b0 msgr2=0x7f0a3014d9e0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:31:40.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.858+0000 7f0a27400640 1 --2- 172.21.15.17:0/4207653740 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0a301554b0 0x7f0a3014d9e0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:40.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.858+0000 7f0a2ea00640 1 --2- 172.21.15.17:0/4207653740 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f0a100724e0 0x7f0a100749a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:40.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.862+0000 7f0a2ea00640 1 --2- 172.21.15.17:0/4207653740 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f0a100724e0 0x7f0a100749a0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f0a300cdec0 tx=0x7f0a1c0091e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:40.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.866+0000 7f0a36400640 1 --2- 172.21.15.17:0/4207653740 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0a30155de0 0x7f0a301530a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:40.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.866+0000 7f0a36400640 1 -- 172.21.15.17:0/4207653740 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f0a300b20a0 con 0x7f0a30155de0 2024-09-10T21:31:40.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.866+0000 7f0a2f400640 1 --2- 172.21.15.17:0/4207653740 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0a30155de0 0x7f0a301530a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:40.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.866+0000 7f0a2f400640 1 -- 172.21.15.17:0/4207653740 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f0a30156500 con 0x7f0a30155de0 2024-09-10T21:31:40.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.866+0000 7f0a2f400640 1 --2- 172.21.15.17:0/4207653740 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0a30155de0 0x7f0a301530a0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f0a2000e500 tx=0x7f0a20000ee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:40.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.866+0000 7f0a27400640 1 -- 172.21.15.17:0/4207653740 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a20007240 con 0x7f0a30155de0 2024-09-10T21:31:40.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.870+0000 7f0a27400640 1 -- 172.21.15.17:0/4207653740 <== mon.1 v2:172.21.15.31:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5043 (secure 0 0 0) 0x7f0a20007240 con 0x7f0a30155de0 2024-09-10T21:31:40.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.870+0000 7f0a27400640 1 -- 172.21.15.17:0/4207653740 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0a30155de0 msgr2=0x7f0a301530a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:40.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.870+0000 7f0a27400640 1 --2- 172.21.15.17:0/4207653740 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0a30155de0 0x7f0a301530a0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f0a2000e500 tx=0x7f0a20000ee0 comp rx=0 tx=0).stop 2024-09-10T21:31:40.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.870+0000 7f0a27400640 1 -- 172.21.15.17:0/4207653740 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0a30155de0 msgr2=0x7f0a301530a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:31:40.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.870+0000 7f0a27400640 1 --2- 172.21.15.17:0/4207653740 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0a30155de0 0x7f0a301530a0 secure :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f0a2000e500 tx=0x7f0a20000ee0 comp rx=0 tx=0).stop 2024-09-10T21:31:40.873 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-10T21:31:40.873 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:31:40.873 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:31:40.873 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-10T21:31:40.873 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 58, 2024-09-10T21:31:40.873 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 439316, 2024-09-10T21:31:40.873 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-10T21:31:40.873 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-10T21:31:40.873 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-10T21:31:40.874 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-10T21:31:40.874 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:31:40.874 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 13, 2024-09-10T21:31:40.874 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:31:40.874 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-10T21:31:40.874 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-10T21:31:40.874 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:31:40.874 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:31:40.874 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:31:40.874 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:31:40.874 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:31:40.874 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:31:40.874 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:31:40.874 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:31:40.875 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:31:40.875 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:31:40.875 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:31:40.875 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:31:40.875 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-10T21:31:40.875 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-10T21:31:40.875 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:31:40.875 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:31:40.875 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:31:40.875 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:31:40.875 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:31:40.875 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:31:40.875 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:31:40.875 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:31:40.876 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:31:40.876 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:31:40.876 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:31:40.876 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:31:40.876 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:31:40.876 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-10T21:31:40.876 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-10T21:31:40.876 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-10T21:31:40.876 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-10T21:31:40.876 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-10T21:31:40.876 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "5e2f023f-d6a6-49f4-a256-f4807dce3739", 2024-09-10T21:31:40.876 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:31:40.876 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:31:40.877 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-10T21:31:40.877 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-10T21:31:40.877 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-10T21:31:40.877 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-10T21:31:40.877 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-10T21:31:40.877 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-10T21:31:40.877 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-10T21:31:40.877 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:31:40.877 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-10T21:31:40.877 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:31:40.877 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:31:40.877 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:31:40.877 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:31:40.877 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:31:40.878 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:31:40.878 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:31:40.878 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:31:40.878 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:31:40.878 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:31:40.878 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:31:40.878 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:31:40.878 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-10T21:31:40.878 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:31:40.878 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-10T21:31:40.878 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:31:40.878 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:31:40.878 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:31:40.879 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:31:40.879 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:31:40.879 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:31:40.879 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:31:40.879 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-10T21:31:40.879 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:31:40.879 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:31:40.879 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:31:40.879 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:31:40.879 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-10T21:31:40.879 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:31:40.879 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:31:40.879 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:31:40.880 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:31:40.880 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-10T21:31:40.880 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-10T21:31:40.880 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:31:40.880 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:31:40.880 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:31:40.880 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:31:40.880 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:31:40.880 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:31:40.880 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:31:40.880 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:31:40.880 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:31:40.880 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:31:40.881 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:31:40.881 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3300", 2024-09-10T21:31:40.881 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:31:40.881 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:31:40.881 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:31:40.881 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:31:40.881 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789", 2024-09-10T21:31:40.881 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:31:40.881 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:31:40.881 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:31:40.881 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:31:40.881 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789/0", 2024-09-10T21:31:40.881 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6789/0", 2024-09-10T21:31:40.881 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:31:40.882 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:31:40.882 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:31:40.882 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:31:40.882 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:31:40.882 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:31:40.882 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:31:40.882 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:31:40.882 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:31:40.882 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:31:40.882 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:31:40.882 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3301", 2024-09-10T21:31:40.882 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:31:40.882 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:31:40.883 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:31:40.883 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:31:40.883 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790", 2024-09-10T21:31:40.883 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:31:40.883 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:31:40.883 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:31:40.883 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:31:40.883 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790/0", 2024-09-10T21:31:40.883 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6790/0", 2024-09-10T21:31:40.883 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:31:40.883 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:31:40.883 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:31:40.883 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:31:40.883 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:31:40.884 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:31:40.884 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-10T21:31:40.884 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-10T21:31:40.884 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:31:40.884 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:31:40.884 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:31:40.884 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:31:40.884 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:31:40.884 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:31:40.884 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": [ 2024-09-10T21:31:40.884 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:31:40.884 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:31:40.884 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:31:40.885 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 5 2024-09-10T21:31:40.885 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:31:40.885 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:31:40.885 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-10T21:31:40.885 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:31:40.885 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:31:40.885 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:31:40.885 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 7 2024-09-10T21:31:40.885 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:31:40.885 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:31:40.885 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-10T21:31:40.885 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:31:40.885 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:31:40.886 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:31:40.886 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 4 2024-09-10T21:31:40.886 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:31:40.886 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:31:40.886 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": [ 2024-09-10T21:31:40.886 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:31:40.886 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:31:40.886 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:31:40.886 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 2 2024-09-10T21:31:40.886 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:31:40.886 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:31:40.886 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:31:40.886 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-10T21:31:40.886 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-10T21:31:40.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.870+0000 7f0a36400640 1 -- 172.21.15.17:0/4207653740 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f0a100724e0 msgr2=0x7f0a100749a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:40.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.870+0000 7f0a36400640 1 --2- 172.21.15.17:0/4207653740 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f0a100724e0 0x7f0a100749a0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f0a300cdec0 tx=0x7f0a1c0091e0 comp rx=0 tx=0).stop 2024-09-10T21:31:40.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.870+0000 7f0a36400640 1 -- 172.21.15.17:0/4207653740 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0a30158510 msgr2=0x7f0a3014d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:40.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.870+0000 7f0a36400640 1 --2- 172.21.15.17:0/4207653740 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0a30158510 0x7f0a3014d4a0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f0a28009bd0 tx=0x7f0a28009350 comp rx=0 tx=0).stop 2024-09-10T21:31:40.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.870+0000 7f0a2fe00640 1 -- 172.21.15.17:0/4207653740 reap_dead start 2024-09-10T21:31:40.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.870+0000 7f0a2fe00640 1 -- 172.21.15.17:0/4207653740 reap_dead start 2024-09-10T21:31:40.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.870+0000 7f0a36400640 1 -- 172.21.15.17:0/4207653740 shutdown_connections 2024-09-10T21:31:40.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.870+0000 7f0a36400640 1 -- 172.21.15.17:0/4207653740 >> 172.21.15.17:0/4207653740 conn(0x7f0a30068d90 msgr2=0x7f0a30151680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:40.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.870+0000 7f0a36400640 1 -- 172.21.15.17:0/4207653740 shutdown_connections 2024-09-10T21:31:40.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:40.870+0000 7f0a36400640 1 -- 172.21.15.17:0/4207653740 wait complete. 2024-09-10T21:31:40.901 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-10T21:31:40.901 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-09-10T21:31:40.901 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-10T21:31:41.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:41.058+0000 7f6b1be00640 1 -- 172.21.15.17:0/3727517612 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b1c151450 msgr2=0x7f6b1c0ca3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:41.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:41.058+0000 7f6b1be00640 1 --2- 172.21.15.17:0/3727517612 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b1c151450 0x7f6b1c0ca3f0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f6b0c009840 tx=0x7f6b0c01f010 comp rx=0 tx=0).stop 2024-09-10T21:31:41.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:41.058+0000 7f6b1be00640 1 -- 172.21.15.17:0/3727517612 shutdown_connections 2024-09-10T21:31:41.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:41.058+0000 7f6b1be00640 1 --2- 172.21.15.17:0/3727517612 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6b1c0cabe0 0x7f6b1c15a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:41.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:41.058+0000 7f6b1be00640 1 --2- 172.21.15.17:0/3727517612 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b1c151450 0x7f6b1c0ca3f0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:41.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:41.058+0000 7f6b1be00640 1 -- 172.21.15.17:0/3727517612 >> 172.21.15.17:0/3727517612 conn(0x7f6b1c0cd690 msgr2=0x7f6b1c0c98e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:41.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:41.058+0000 7f6b1be00640 1 -- 172.21.15.17:0/3727517612 shutdown_connections 2024-09-10T21:31:41.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:41.058+0000 7f6b1be00640 1 -- 172.21.15.17:0/3727517612 wait complete. 2024-09-10T21:31:41.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:41.058+0000 7f6b1be00640 1 Processor -- start 2024-09-10T21:31:41.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:41.058+0000 7f6b1be00640 1 -- start start 2024-09-10T21:31:41.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:41.062+0000 7f6b1be00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6b1c0c2f30 0x7f6b1c154e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:41.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:41.062+0000 7f6b1be00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b1c0cabe0 0x7f6b1c14c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:41.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:41.062+0000 7f6b1be00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6b1c151450 0x7f6b1c14cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:41.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:41.062+0000 7f6b1be00640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f6b1c0c2a10 con 0x7f6b1c151450 2024-09-10T21:31:41.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:41.062+0000 7f6b1be00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f6b1c06f990 con 0x7f6b1c0cabe0 2024-09-10T21:31:41.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:41.062+0000 7f6b1be00640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f6b1c0c2710 con 0x7f6b1c0c2f30 2024-09-10T21:31:41.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:41.062+0000 7f6b1b400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6b1c151450 0x7f6b1c14cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:41.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:41.062+0000 7f6b1b400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6b1c151450 0x7f6b1c14cac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:42698/0 (socket says 172.21.15.17:42698) 2024-09-10T21:31:41.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:41.062+0000 7f6b1b400640 1 -- 172.21.15.17:0/4109604247 learned_addr learned my addr 172.21.15.17:0/4109604247 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:31:41.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:41.062+0000 7f6b1aa00640 1 --2- 172.21.15.17:0/4109604247 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6b1c0c2f30 0x7f6b1c154e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:41.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:41.062+0000 7f6b1a000640 1 --2- 172.21.15.17:0/4109604247 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b1c0cabe0 0x7f6b1c14c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:41.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:41.062+0000 7f6b1b400640 1 -- 172.21.15.17:0/4109604247 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6b1c0c2f30 msgr2=0x7f6b1c154e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:41.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:41.062+0000 7f6b1b400640 1 --2- 172.21.15.17:0/4109604247 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6b1c0c2f30 0x7f6b1c154e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:41.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:41.062+0000 7f6b1b400640 1 -- 172.21.15.17:0/4109604247 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b1c0cabe0 msgr2=0x7f6b1c14c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:41.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:41.062+0000 7f6b1b400640 1 --2- 172.21.15.17:0/4109604247 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6b1c0cabe0 0x7f6b1c14c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:41.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:41.062+0000 7f6b1b400640 1 -- 172.21.15.17:0/4109604247 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b1c1569d0 con 0x7f6b1c151450 2024-09-10T21:31:41.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:41.062+0000 7f6b1b400640 1 --2- 172.21.15.17:0/4109604247 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6b1c151450 0x7f6b1c14cac0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f6b1400cb60 tx=0x7f6b1400cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:41.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:41.062+0000 7f6b12a00640 1 -- 172.21.15.17:0/4109604247 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b14016020 con 0x7f6b1c151450 2024-09-10T21:31:41.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:41.062+0000 7f6b12a00640 1 -- 172.21.15.17:0/4109604247 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6b1400ab40 con 0x7f6b1c151450 2024-09-10T21:31:41.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:41.062+0000 7f6b1be00640 1 -- 172.21.15.17:0/4109604247 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b1c15b5b0 con 0x7f6b1c151450 2024-09-10T21:31:41.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:41.062+0000 7f6b12a00640 1 -- 172.21.15.17:0/4109604247 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b14012420 con 0x7f6b1c151450 2024-09-10T21:31:41.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:41.062+0000 7f6b1be00640 1 -- 172.21.15.17:0/4109604247 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b1c156f70 con 0x7f6b1c151450 2024-09-10T21:31:41.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:41.062+0000 7f6b1be00640 1 --2- 172.21.15.17:0/4109604247 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6b1c14d220 0x7f6b1c146390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:41.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:41.062+0000 7f6b1be00640 1 -- 172.21.15.17:0/4109604247 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f6b1c069ef0 con 0x7f6b1c14d220 2024-09-10T21:31:41.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:41.066+0000 7f6b1aa00640 1 --2- 172.21.15.17:0/4109604247 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6b1c14d220 0x7f6b1c146390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:41.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:41.066+0000 7f6b1aa00640 1 -- 172.21.15.17:0/4109604247 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6b1c0c79a0 con 0x7f6b1c14d220 2024-09-10T21:31:41.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:41.066+0000 7f6b1aa00640 1 --2- 172.21.15.17:0/4109604247 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6b1c14d220 0x7f6b1c146390 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f6b1c0c3e20 tx=0x7f6b0c005b10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:41.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:41.066+0000 7f6b12a00640 1 -- 172.21.15.17:0/4109604247 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f6b140125c0 con 0x7f6b1c151450 2024-09-10T21:31:41.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:41.066+0000 7f6b12a00640 1 --2- 172.21.15.17:0/4109604247 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f6afc072610 0x7f6afc074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:41.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:41.066+0000 7f6b12a00640 1 -- 172.21.15.17:0/4109604247 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f6b14093ad0 con 0x7f6b1c151450 2024-09-10T21:31:41.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:41.066+0000 7f6b12a00640 1 -- 172.21.15.17:0/4109604247 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b0c0b5040 con 0x7f6b1c14d220 2024-09-10T21:31:41.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:41.066+0000 7f6b12a00640 1 -- 172.21.15.17:0/4109604247 <== mon.2 v2:172.21.15.31:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f6b0c005020 con 0x7f6b1c14d220 2024-09-10T21:31:41.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:41.066+0000 7f6b12a00640 1 -- 172.21.15.17:0/4109604247 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6b1c14d220 msgr2=0x7f6b1c146390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:41.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:41.066+0000 7f6b12a00640 1 --2- 172.21.15.17:0/4109604247 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6b1c14d220 0x7f6b1c146390 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f6b1c0c3e20 tx=0x7f6b0c005b10 comp rx=0 tx=0).stop 2024-09-10T21:31:41.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:41.066+0000 7f6b12a00640 1 -- 172.21.15.17:0/4109604247 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6b1c14d220 msgr2=0x7f6b1c146390 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:31:41.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:41.066+0000 7f6b12a00640 1 --2- 172.21.15.17:0/4109604247 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6b1c14d220 0x7f6b1c146390 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:31:41.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:41.066+0000 7f6b1a000640 1 --2- 172.21.15.17:0/4109604247 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f6afc072610 0x7f6afc074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:41.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:41.066+0000 7f6b1a000640 1 --2- 172.21.15.17:0/4109604247 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f6afc072610 0x7f6afc074ad0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f6b1c15a9e0 tx=0x7f6b080091e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:41.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:41.074+0000 7f6b1be00640 1 --2- 172.21.15.17:0/4109604247 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6b1c146c30 0x7f6b1c14b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:31:41.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:41.074+0000 7f6b1be00640 1 -- 172.21.15.17:0/4109604247 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f6b1c0cb670 con 0x7f6b1c146c30 2024-09-10T21:31:41.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:41.074+0000 7f6b1aa00640 1 --2- 172.21.15.17:0/4109604247 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6b1c146c30 0x7f6b1c14b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:31:41.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:41.074+0000 7f6b1aa00640 1 -- 172.21.15.17:0/4109604247 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f6b1c157ac0 con 0x7f6b1c146c30 2024-09-10T21:31:41.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:41.074+0000 7f6b1aa00640 1 --2- 172.21.15.17:0/4109604247 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6b1c146c30 0x7f6b1c14b340 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f6b0c000c10 tx=0x7f6b0c0091e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:31:41.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:41.074+0000 7f6b12a00640 1 -- 172.21.15.17:0/4109604247 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b0c0b5050 con 0x7f6b1c146c30 2024-09-10T21:31:41.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:41.078+0000 7f6b12a00640 1 -- 172.21.15.17:0/4109604247 <== mon.2 v2:172.21.15.31:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4541 (secure 0 0 0) 0x7f6b0c0046d0 con 0x7f6b1c146c30 2024-09-10T21:31:41.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:41.078+0000 7f6b12a00640 1 -- 172.21.15.17:0/4109604247 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6b1c146c30 msgr2=0x7f6b1c14b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:41.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:41.078+0000 7f6b12a00640 1 --2- 172.21.15.17:0/4109604247 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6b1c146c30 0x7f6b1c14b340 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f6b0c000c10 tx=0x7f6b0c0091e0 comp rx=0 tx=0).stop 2024-09-10T21:31:41.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:41.078+0000 7f6b12a00640 1 -- 172.21.15.17:0/4109604247 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6b1c146c30 msgr2=0x7f6b1c14b340 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:31:41.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:41.078+0000 7f6b12a00640 1 --2- 172.21.15.17:0/4109604247 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6b1c146c30 0x7f6b1c14b340 secure :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f6b0c000c10 tx=0x7f6b0c0091e0 comp rx=0 tx=0).stop 2024-09-10T21:31:41.081 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-10T21:31:41.081 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:31:41.081 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:31:41.081 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-10T21:31:41.081 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 58, 2024-09-10T21:31:41.081 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 33383, 2024-09-10T21:31:41.081 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-10T21:31:41.082 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-10T21:31:41.082 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-10T21:31:41.082 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-10T21:31:41.082 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:31:41.082 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 13, 2024-09-10T21:31:41.082 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:31:41.082 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-10T21:31:41.082 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-10T21:31:41.082 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:31:41.082 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:31:41.082 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:31:41.082 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:31:41.083 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:31:41.083 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:31:41.083 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:31:41.083 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:31:41.083 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:31:41.083 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:31:41.083 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:31:41.083 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:31:41.083 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-10T21:31:41.083 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-10T21:31:41.083 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:31:41.084 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:31:41.084 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:31:41.084 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:31:41.084 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:31:41.084 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:31:41.084 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:31:41.084 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:31:41.084 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:31:41.084 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:31:41.084 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:31:41.084 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:31:41.084 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:31:41.085 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-10T21:31:41.085 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-10T21:31:41.085 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-10T21:31:41.085 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-10T21:31:41.085 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-10T21:31:41.085 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "5e2f023f-d6a6-49f4-a256-f4807dce3739", 2024-09-10T21:31:41.085 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:31:41.085 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:31:41.085 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-10T21:31:41.085 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-10T21:31:41.085 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-10T21:31:41.086 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-10T21:31:41.086 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-10T21:31:41.086 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-10T21:31:41.086 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-10T21:31:41.086 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:31:41.086 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-10T21:31:41.086 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:31:41.086 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:31:41.086 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:31:41.086 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:31:41.086 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:31:41.087 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:31:41.087 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:31:41.087 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:31:41.087 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:31:41.087 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:31:41.087 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:31:41.087 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:31:41.087 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-10T21:31:41.087 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:31:41.087 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-10T21:31:41.087 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:31:41.088 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:31:41.088 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:31:41.088 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:31:41.088 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:31:41.088 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:31:41.088 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:31:41.088 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-10T21:31:41.088 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:31:41.088 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:31:41.088 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:31:41.088 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:31:41.088 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-10T21:31:41.089 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:31:41.089 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:31:41.089 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:31:41.089 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:31:41.089 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-10T21:31:41.089 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-10T21:31:41.089 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:31:41.089 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:31:41.089 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:31:41.089 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:31:41.089 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:31:41.090 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:31:41.090 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:31:41.090 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:31:41.090 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:31:41.090 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:31:41.090 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:31:41.090 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3300", 2024-09-10T21:31:41.090 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:31:41.090 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:31:41.090 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:31:41.090 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:31:41.091 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789", 2024-09-10T21:31:41.091 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:31:41.091 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:31:41.091 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:31:41.091 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:31:41.091 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789/0", 2024-09-10T21:31:41.091 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6789/0", 2024-09-10T21:31:41.091 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:31:41.091 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:31:41.091 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:31:41.091 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:31:41.091 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:31:41.092 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:31:41.092 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:31:41.092 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:31:41.092 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:31:41.092 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:31:41.092 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:31:41.092 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3301", 2024-09-10T21:31:41.092 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:31:41.092 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:31:41.092 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:31:41.092 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:31:41.092 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790", 2024-09-10T21:31:41.093 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:31:41.093 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:31:41.093 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:31:41.093 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:31:41.093 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790/0", 2024-09-10T21:31:41.093 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6790/0", 2024-09-10T21:31:41.093 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:31:41.093 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:31:41.093 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:31:41.093 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:31:41.093 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:31:41.094 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:31:41.094 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-10T21:31:41.094 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-10T21:31:41.094 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:31:41.094 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:31:41.094 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:31:41.094 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:31:41.094 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:31:41.094 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:31:41.094 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-10T21:31:41.094 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:31:41.094 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:31:41.095 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:31:41.095 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:31:41.095 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:31:41.095 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:31:41.095 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:31:41.095 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-10T21:31:41.095 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-10T21:31:41.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:41.078+0000 7f6b1be00640 1 -- 172.21.15.17:0/4109604247 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f6afc072610 msgr2=0x7f6afc074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:41.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:41.078+0000 7f6b1be00640 1 --2- 172.21.15.17:0/4109604247 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f6afc072610 0x7f6afc074ad0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f6b1c15a9e0 tx=0x7f6b080091e0 comp rx=0 tx=0).stop 2024-09-10T21:31:41.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:41.078+0000 7f6b1be00640 1 -- 172.21.15.17:0/4109604247 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6b1c151450 msgr2=0x7f6b1c14cac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:31:41.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:41.078+0000 7f6b1be00640 1 --2- 172.21.15.17:0/4109604247 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6b1c151450 0x7f6b1c14cac0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f6b1400cb60 tx=0x7f6b1400cc50 comp rx=0 tx=0).stop 2024-09-10T21:31:41.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:41.078+0000 7f6b1b400640 1 -- 172.21.15.17:0/4109604247 reap_dead start 2024-09-10T21:31:41.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:41.078+0000 7f6b1b400640 1 -- 172.21.15.17:0/4109604247 reap_dead start 2024-09-10T21:31:41.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:41.078+0000 7f6b1be00640 1 -- 172.21.15.17:0/4109604247 shutdown_connections 2024-09-10T21:31:41.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:41.078+0000 7f6b1be00640 1 -- 172.21.15.17:0/4109604247 >> 172.21.15.17:0/4109604247 conn(0x7f6b1c0cd690 msgr2=0x7f6b1c0c9870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:31:41.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:41.078+0000 7f6b1be00640 1 -- 172.21.15.17:0/4109604247 shutdown_connections 2024-09-10T21:31:41.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:31:41.078+0000 7f6b1be00640 1 -- 172.21.15.17:0/4109604247 wait complete. 2024-09-10T21:31:41.103 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-10T21:31:41.103 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['a'] 2024-09-10T21:31:41.103 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: ['b'] 2024-09-10T21:31:41.103 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.a 2024-09-10T21:31:41.103 INFO:tasks.mon_thrash.mon_thrasher:killing mon.a 2024-09-10T21:31:41.103 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-10T21:31:41.104 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:31:41.125 INFO:tasks.ceph.mon.a:Stopped 2024-09-10T21:31:41.126 INFO:tasks.mon_thrash:Sending STOP to mon b 2024-09-10T21:31:41.126 INFO:tasks.ceph.mon.b:Sent signal 19 2024-09-10T21:31:41.126 INFO:tasks.mon_thrash.mon_thrasher:waiting for 15.0 secs to unfreeze mons 2024-09-10T21:31:47.348 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:31:47.352 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:31:56.127 INFO:tasks.mon_thrash:Sending CONT to mon b 2024-09-10T21:31:56.127 INFO:tasks.ceph.mon.b:Sent signal 18 2024-09-10T21:31:56.127 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2024-09-10T21:31:56.128 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-10T21:32:02.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.282+0000 7f67d9000640 1 -- 172.21.15.17:0/873926407 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67b40091e0 msgr2=0x7f67b400b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:02.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.282+0000 7f67d9000640 1 --2- 172.21.15.17:0/873926407 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67b40091e0 0x7f67b400b630 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f67c40094b0 tx=0x7f67c401d370 comp rx=0 tx=0).stop 2024-09-10T21:32:02.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.282+0000 7f67d9000640 1 -- 172.21.15.17:0/873926407 shutdown_connections 2024-09-10T21:32:02.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.282+0000 7f67d9000640 1 --2- 172.21.15.17:0/873926407 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67b40091e0 0x7f67b400b630 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:02.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.282+0000 7f67d9000640 1 -- 172.21.15.17:0/873926407 >> 172.21.15.17:0/873926407 conn(0x7f67d4070ca0 msgr2=0x7f67d40710b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:32:02.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.282+0000 7f67d9000640 1 -- 172.21.15.17:0/873926407 shutdown_connections 2024-09-10T21:32:02.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.282+0000 7f67d9000640 1 -- 172.21.15.17:0/873926407 wait complete. 2024-09-10T21:32:02.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.286+0000 7f67d9000640 1 Processor -- start 2024-09-10T21:32:02.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.286+0000 7f67d9000640 1 -- start start 2024-09-10T21:32:02.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.286+0000 7f67d9000640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67b40091e0 0x7f67d4145520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:02.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.286+0000 7f67d9000640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f67d406f480 0x7f67d40c87b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:02.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.286+0000 7f67d9000640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f67d406f860 0x7f67d4146090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:02.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.286+0000 7f67d9000640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f67d40c7870 con 0x7f67d406f860 2024-09-10T21:32:02.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.286+0000 7f67d9000640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f67d40702a0 con 0x7f67b40091e0 2024-09-10T21:32:02.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.286+0000 7f67d9000640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f67d40c79f0 con 0x7f67d406f480 2024-09-10T21:32:02.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.286+0000 7f67d3e00640 1 -- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f67d406f860 msgr2=0x7f67d4146090 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-10T21:32:02.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.286+0000 7f67d3e00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f67d406f860 0x7f67d4146090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:32:02.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.286+0000 7f67d2a00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f67d406f480 0x7f67d40c87b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:02.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.286+0000 7f67d3400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67b40091e0 0x7f67d4145520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:02.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.286+0000 7f67d2a00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f67d406f480 0x7f67d40c87b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3301/0 says I am v2:172.21.15.17:55398/0 (socket says 172.21.15.17:55398) 2024-09-10T21:32:02.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.286+0000 7f67d2a00640 1 -- 172.21.15.17:0/1198201669 learned_addr learned my addr 172.21.15.17:0/1198201669 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:32:02.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.286+0000 7f67d3400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67b40091e0 0x7f67d4145520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.17:56956/0 (socket says 172.21.15.17:56956) 2024-09-10T21:32:02.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.286+0000 7f67d3400640 1 -- 172.21.15.17:0/1198201669 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f67d406f480 msgr2=0x7f67d40c87b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:02.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.286+0000 7f67d3400640 1 --2- 172.21.15.17:0/1198201669 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f67d406f480 0x7f67d40c87b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:02.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.286+0000 7f67d3400640 1 -- 172.21.15.17:0/1198201669 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f67d406f860 msgr2=0x7f67d4146090 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:32:02.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.286+0000 7f67d3400640 1 --2- 172.21.15.17:0/1198201669 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f67d406f860 0x7f67d4146090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:02.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.286+0000 7f67d3400640 1 -- 172.21.15.17:0/1198201669 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67d4145a60 con 0x7f67b40091e0 2024-09-10T21:32:02.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.286+0000 7f67d3400640 1 --2- 172.21.15.17:0/1198201669 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67b40091e0 0x7f67d4145520 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f67c40ad000 tx=0x7f67c4001310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:02.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.286+0000 7f67cb400640 1 -- 172.21.15.17:0/1198201669 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67c40bf070 con 0x7f67b40091e0 2024-09-10T21:32:02.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.286+0000 7f67cb400640 1 -- 172.21.15.17:0/1198201669 <== mon.1 v2:172.21.15.31:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f67c40ae7f0 con 0x7f67b40091e0 2024-09-10T21:32:02.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.286+0000 7f67d9000640 1 -- 172.21.15.17:0/1198201669 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67d414b8d0 con 0x7f67b40091e0 2024-09-10T21:32:02.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.286+0000 7f67cb400640 1 -- 172.21.15.17:0/1198201669 <== mon.1 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67c40b3040 con 0x7f67b40091e0 2024-09-10T21:32:02.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.286+0000 7f67d9000640 1 -- 172.21.15.17:0/1198201669 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67d41498f0 con 0x7f67b40091e0 2024-09-10T21:32:02.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.290+0000 7f67d9000640 1 -- 172.21.15.17:0/1198201669 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67d40b2c60 con 0x7f67b40091e0 2024-09-10T21:32:02.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.290+0000 7f67cb400640 1 -- 172.21.15.17:0/1198201669 <== mon.1 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f67c40aec00 con 0x7f67b40091e0 2024-09-10T21:32:02.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.290+0000 7f67cb400640 1 --2- 172.21.15.17:0/1198201669 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f67b4076710 0x7f67b4078bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:02.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.290+0000 7f67cb400640 1 -- 172.21.15.17:0/1198201669 <== mon.1 v2:172.21.15.31:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f67c412df30 con 0x7f67b40091e0 2024-09-10T21:32:02.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.290+0000 7f67d2a00640 1 --2- 172.21.15.17:0/1198201669 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f67b4076710 0x7f67b4078bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:02.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.294+0000 7f67d2a00640 1 --2- 172.21.15.17:0/1198201669 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f67b4076710 0x7f67b4078bd0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f67d41574c0 tx=0x7f67c0016040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:02.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.294+0000 7f67cb400640 1 -- 172.21.15.17:0/1198201669 <== mon.1 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f67c415cde0 con 0x7f67b40091e0 2024-09-10T21:32:02.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.550+0000 7f67d9000640 1 -- 172.21.15.17:0/1198201669 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f67d4071ce0 con 0x7f67b40091e0 2024-09-10T21:32:02.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.550+0000 7f67cb400640 1 -- 172.21.15.17:0/1198201669 <== mon.1 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1550 (secure 0 0 0) 0x7f67c40170c0 con 0x7f67b40091e0 2024-09-10T21:32:02.555 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":60,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":1,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5e2f023f-d6a6-49f4-a256-f4807dce3739","modified":"2024-09-10T21:23:46.892130Z","created":"2024-09-10T21:23:46.892130Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3301","nonce":0},{"type":"v1","addr":"172.21.15.31:6790","nonce":0}]},"addr":"172.21.15.31:6790/0","public_addr":"172.21.15.31:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-10T21:32:02.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.554+0000 7f67d9000640 1 -- 172.21.15.17:0/1198201669 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f67b4076710 msgr2=0x7f67b4078bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:02.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.554+0000 7f67d9000640 1 --2- 172.21.15.17:0/1198201669 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f67b4076710 0x7f67b4078bd0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f67d41574c0 tx=0x7f67c0016040 comp rx=0 tx=0).stop 2024-09-10T21:32:02.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.554+0000 7f67d9000640 1 -- 172.21.15.17:0/1198201669 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67b40091e0 msgr2=0x7f67d4145520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:02.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.554+0000 7f67d9000640 1 --2- 172.21.15.17:0/1198201669 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67b40091e0 0x7f67d4145520 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f67c40ad000 tx=0x7f67c4001310 comp rx=0 tx=0).stop 2024-09-10T21:32:02.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.554+0000 7f67d9000640 1 -- 172.21.15.17:0/1198201669 shutdown_connections 2024-09-10T21:32:02.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.554+0000 7f67d9000640 1 --2- 172.21.15.17:0/1198201669 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f67b4076710 0x7f67b4078bd0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:02.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.554+0000 7f67d9000640 1 --2- 172.21.15.17:0/1198201669 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f67d406f860 0x7f67d4146090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:02.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.554+0000 7f67d9000640 1 --2- 172.21.15.17:0/1198201669 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f67d406f480 0x7f67d40c87b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:02.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.554+0000 7f67d9000640 1 --2- 172.21.15.17:0/1198201669 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67b40091e0 0x7f67d4145520 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:02.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.554+0000 7f67d9000640 1 -- 172.21.15.17:0/1198201669 >> 172.21.15.17:0/1198201669 conn(0x7f67d4070ca0 msgr2=0x7f67d4148410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:32:02.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.554+0000 7f67d9000640 1 -- 172.21.15.17:0/1198201669 shutdown_connections 2024-09-10T21:32:02.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.554+0000 7f67d9000640 1 -- 172.21.15.17:0/1198201669 wait complete. 2024-09-10T21:32:02.577 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2024-09-10T21:32:02.577 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon b 2024-09-10T21:32:02.578 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-10T21:32:02.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.734+0000 7f8abee00640 1 -- 172.21.15.17:0/2863777639 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8ab8068d20 msgr2=0x7f8ab8073a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:02.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.734+0000 7f8abee00640 1 --2- 172.21.15.17:0/2863777639 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8ab8068d20 0x7f8ab8073a70 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f8aa4009900 tx=0x7f8aa401f0b0 comp rx=0 tx=0).stop 2024-09-10T21:32:02.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.734+0000 7f8abee00640 1 -- 172.21.15.17:0/2863777639 shutdown_connections 2024-09-10T21:32:02.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.734+0000 7f8abee00640 1 --2- 172.21.15.17:0/2863777639 >> v2:172.21.15.31:3300/0 conn(0x7f8ab8072bb0 0x7f8ab815dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:02.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.734+0000 7f8abee00640 1 --2- 172.21.15.17:0/2863777639 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8ab8068d20 0x7f8ab8073a70 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:02.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.734+0000 7f8abee00640 1 --2- 172.21.15.17:0/2863777639 >> v2:172.21.15.17:3300/0 conn(0x7f8ab8154740 0x7f8ab8073530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:02.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.734+0000 7f8abee00640 1 -- 172.21.15.17:0/2863777639 >> 172.21.15.17:0/2863777639 conn(0x7f8ab806cc20 msgr2=0x7f8ab806d030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:32:02.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.734+0000 7f8abee00640 1 -- 172.21.15.17:0/2863777639 shutdown_connections 2024-09-10T21:32:02.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.734+0000 7f8abee00640 1 -- 172.21.15.17:0/2863777639 wait complete. 2024-09-10T21:32:02.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.734+0000 7f8abee00640 1 Processor -- start 2024-09-10T21:32:02.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.734+0000 7f8abee00640 1 -- start start 2024-09-10T21:32:02.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.738+0000 7f8abee00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8ab8068d20 0x7f8ab80c3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:02.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.738+0000 7f8abee00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8ab8072bb0 0x7f8ab8159a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:02.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.738+0000 7f8abee00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8ab8154740 0x7f8ab8147b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:02.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.738+0000 7f8abee00640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f8ab815a4f0 con 0x7f8ab8068d20 2024-09-10T21:32:02.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.738+0000 7f8abee00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f8ab815ae00 con 0x7f8ab8154740 2024-09-10T21:32:02.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.738+0000 7f8abee00640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f8ab8154b20 con 0x7f8ab8072bb0 2024-09-10T21:32:02.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.738+0000 7f8abda00640 1 -- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8ab8068d20 msgr2=0x7f8ab80c3ca0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-10T21:32:02.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.738+0000 7f8abda00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8ab8068d20 0x7f8ab80c3ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:32:02.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.738+0000 7f8abd000640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8ab8072bb0 0x7f8ab8159a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:02.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.738+0000 7f8abd000640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8ab8072bb0 0x7f8ab8159a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3301/0 says I am v2:172.21.15.17:55428/0 (socket says 172.21.15.17:55428) 2024-09-10T21:32:02.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.738+0000 7f8abe400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8ab8154740 0x7f8ab8147b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:02.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.738+0000 7f8abd000640 1 -- 172.21.15.17:0/70978730 learned_addr learned my addr 172.21.15.17:0/70978730 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:32:02.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.738+0000 7f8abd000640 1 -- 172.21.15.17:0/70978730 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8ab8154740 msgr2=0x7f8ab8147b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:02.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.738+0000 7f8abd000640 1 --2- 172.21.15.17:0/70978730 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8ab8154740 0x7f8ab8147b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:02.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.738+0000 7f8abd000640 1 -- 172.21.15.17:0/70978730 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8ab8068d20 msgr2=0x7f8ab80c3ca0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:32:02.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.738+0000 7f8abd000640 1 --2- 172.21.15.17:0/70978730 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8ab8068d20 0x7f8ab80c3ca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:02.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.738+0000 7f8abd000640 1 -- 172.21.15.17:0/70978730 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ab81484f0 con 0x7f8ab8072bb0 2024-09-10T21:32:02.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.738+0000 7f8abd000640 1 --2- 172.21.15.17:0/70978730 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8ab8072bb0 0x7f8ab8159a40 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f8aa4009850 tx=0x7f8aa4005df0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:02.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.738+0000 7f8ab5600640 1 -- 172.21.15.17:0/70978730 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aa40b5040 con 0x7f8ab8072bb0 2024-09-10T21:32:02.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.738+0000 7f8ab5600640 1 -- 172.21.15.17:0/70978730 <== mon.2 v2:172.21.15.31:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8aa4005320 con 0x7f8ab8072bb0 2024-09-10T21:32:02.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.738+0000 7f8ab5600640 1 -- 172.21.15.17:0/70978730 <== mon.2 v2:172.21.15.31:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aa40046d0 con 0x7f8ab8072bb0 2024-09-10T21:32:02.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.738+0000 7f8abee00640 1 -- 172.21.15.17:0/70978730 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ab8153d30 con 0x7f8ab8072bb0 2024-09-10T21:32:02.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.738+0000 7f8abee00640 1 -- 172.21.15.17:0/70978730 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ab80cb8f0 con 0x7f8ab8072bb0 2024-09-10T21:32:02.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.738+0000 7f8abee00640 1 --2- 172.21.15.17:0/70978730 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8ab8153440 0x7f8ab80cbb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:02.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.738+0000 7f8abee00640 1 -- 172.21.15.17:0/70978730 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f8ab81488f0 con 0x7f8ab8153440 2024-09-10T21:32:02.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.738+0000 7f8abda00640 1 --2- 172.21.15.17:0/70978730 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8ab8153440 0x7f8ab80cbb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:02.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.738+0000 7f8abda00640 1 -- 172.21.15.17:0/70978730 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8ab80c6d60 con 0x7f8ab8153440 2024-09-10T21:32:02.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.738+0000 7f8abda00640 1 --2- 172.21.15.17:0/70978730 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8ab8153440 0x7f8ab80cbb30 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f8aac00b450 tx=0x7f8aac00b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:02.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.742+0000 7f8ab5600640 1 -- 172.21.15.17:0/70978730 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aac016070 con 0x7f8ab8153440 2024-09-10T21:32:02.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.742+0000 7f8ab5600640 1 -- 172.21.15.17:0/70978730 <== mon.1 v2:172.21.15.31:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f8aac003d80 con 0x7f8ab8153440 2024-09-10T21:32:02.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.742+0000 7f8ab5600640 1 -- 172.21.15.17:0/70978730 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8ab8153440 msgr2=0x7f8ab80cbb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:02.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.742+0000 7f8ab5600640 1 --2- 172.21.15.17:0/70978730 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8ab8153440 0x7f8ab80cbb30 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f8aac00b450 tx=0x7f8aac00b910 comp rx=0 tx=0).stop 2024-09-10T21:32:02.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.742+0000 7f8ab5600640 1 -- 172.21.15.17:0/70978730 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8ab8153440 msgr2=0x7f8ab80cbb30 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:32:02.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.742+0000 7f8ab5600640 1 --2- 172.21.15.17:0/70978730 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8ab8153440 0x7f8ab80cbb30 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:02.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.742+0000 7f8ab5600640 1 -- 172.21.15.17:0/70978730 <== mon.2 v2:172.21.15.31:3301/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f8aa40c2060 con 0x7f8ab8072bb0 2024-09-10T21:32:02.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.742+0000 7f8ab5600640 1 --2- 172.21.15.17:0/70978730 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f8a98072050 0x7f8a98074510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:02.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.742+0000 7f8ab5600640 1 -- 172.21.15.17:0/70978730 <== mon.2 v2:172.21.15.31:3301/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f8aa412fc30 con 0x7f8ab8072bb0 2024-09-10T21:32:02.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.742+0000 7f8abda00640 1 --2- 172.21.15.17:0/70978730 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f8a98072050 0x7f8a98074510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:02.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.742+0000 7f8abda00640 1 --2- 172.21.15.17:0/70978730 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f8a98072050 0x7f8a98074510 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f8aac000c10 tx=0x7f8aac008030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:02.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.750+0000 7f8abee00640 1 --2- 172.21.15.17:0/70978730 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8ab8144be0 0x7f8ab8144fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:02.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.750+0000 7f8abee00640 1 -- 172.21.15.17:0/70978730 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f8ab80cc070 con 0x7f8ab8144be0 2024-09-10T21:32:02.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.750+0000 7f8abe400640 1 --2- 172.21.15.17:0/70978730 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8ab8144be0 0x7f8ab8144fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:02.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.750+0000 7f8abe400640 1 -- 172.21.15.17:0/70978730 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f8ab80737a0 con 0x7f8ab8144be0 2024-09-10T21:32:02.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.750+0000 7f8abe400640 1 --2- 172.21.15.17:0/70978730 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8ab8144be0 0x7f8ab8144fc0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f8ab0011c40 tx=0x7f8ab000f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:02.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.750+0000 7f8ab5600640 1 -- 172.21.15.17:0/70978730 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ab0003e50 con 0x7f8ab8144be0 2024-09-10T21:32:02.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.750+0000 7f8ab5600640 1 -- 172.21.15.17:0/70978730 <== mon.1 v2:172.21.15.31:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5033 (secure 0 0 0) 0x7f8ab0004050 con 0x7f8ab8144be0 2024-09-10T21:32:02.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.750+0000 7f8ab5600640 1 -- 172.21.15.17:0/70978730 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8ab8144be0 msgr2=0x7f8ab8144fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:02.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.750+0000 7f8ab5600640 1 --2- 172.21.15.17:0/70978730 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8ab8144be0 0x7f8ab8144fc0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f8ab0011c40 tx=0x7f8ab000f790 comp rx=0 tx=0).stop 2024-09-10T21:32:02.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.750+0000 7f8ab5600640 1 -- 172.21.15.17:0/70978730 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8ab8144be0 msgr2=0x7f8ab8144fc0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:32:02.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.750+0000 7f8ab5600640 1 --2- 172.21.15.17:0/70978730 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8ab8144be0 0x7f8ab8144fc0 secure :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f8ab0011c40 tx=0x7f8ab000f790 comp rx=0 tx=0).stop 2024-09-10T21:32:02.755 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-10T21:32:02.755 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:32:02.755 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:32:02.756 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "leader", 2024-09-10T21:32:02.756 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 60, 2024-09-10T21:32:02.756 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 461200, 2024-09-10T21:32:02.756 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-10T21:32:02.756 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-10T21:32:02.756 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-10T21:32:02.756 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:32:02.756 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 1, 2024-09-10T21:32:02.756 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:32:02.756 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-10T21:32:02.756 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-10T21:32:02.756 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:32:02.756 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:32:02.757 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:32:02.757 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:32:02.757 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:32:02.757 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:32:02.757 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:32:02.757 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:32:02.757 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:32:02.757 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:32:02.757 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:32:02.757 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:32:02.757 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-10T21:32:02.757 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-10T21:32:02.757 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:32:02.758 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:32:02.758 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:32:02.758 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:32:02.758 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:32:02.758 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:32:02.758 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:32:02.758 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:32:02.758 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:32:02.758 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:32:02.758 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:32:02.759 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:32:02.759 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:32:02.759 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-10T21:32:02.759 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-10T21:32:02.759 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-10T21:32:02.759 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-10T21:32:02.759 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-10T21:32:02.759 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "5e2f023f-d6a6-49f4-a256-f4807dce3739", 2024-09-10T21:32:02.759 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:32:02.759 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:32:02.759 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-10T21:32:02.760 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-10T21:32:02.760 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-10T21:32:02.760 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-10T21:32:02.760 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-10T21:32:02.760 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-10T21:32:02.760 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-10T21:32:02.760 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:32:02.760 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-10T21:32:02.760 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:32:02.760 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:32:02.760 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:32:02.760 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:32:02.761 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:32:02.761 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:32:02.761 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:32:02.761 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:32:02.761 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:32:02.761 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:32:02.761 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:32:02.761 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:32:02.761 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-10T21:32:02.761 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:32:02.761 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-10T21:32:02.761 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:32:02.761 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:32:02.761 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:32:02.762 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:32:02.762 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:32:02.762 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:32:02.762 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:32:02.762 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-10T21:32:02.762 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:32:02.762 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:32:02.762 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:32:02.762 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:32:02.762 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-10T21:32:02.762 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:32:02.762 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:32:02.762 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:32:02.763 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:32:02.763 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-10T21:32:02.763 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-10T21:32:02.763 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:32:02.763 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:32:02.763 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:32:02.763 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:32:02.763 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:32:02.763 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:32:02.763 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:32:02.764 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:32:02.764 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:32:02.764 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:32:02.764 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:32:02.764 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3300", 2024-09-10T21:32:02.764 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:32:02.764 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:32:02.764 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:32:02.764 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:32:02.765 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789", 2024-09-10T21:32:02.765 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:32:02.765 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:32:02.765 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:32:02.765 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:32:02.765 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789/0", 2024-09-10T21:32:02.765 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6789/0", 2024-09-10T21:32:02.765 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:32:02.765 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:32:02.765 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:32:02.765 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:32:02.765 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:32:02.765 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:32:02.766 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:32:02.766 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:32:02.766 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:32:02.766 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:32:02.766 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:32:02.766 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3301", 2024-09-10T21:32:02.766 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:32:02.766 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:32:02.766 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:32:02.766 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:32:02.766 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790", 2024-09-10T21:32:02.766 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:32:02.766 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:32:02.766 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:32:02.767 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:32:02.767 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790/0", 2024-09-10T21:32:02.767 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6790/0", 2024-09-10T21:32:02.767 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:32:02.767 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:32:02.767 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:32:02.767 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:32:02.767 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:32:02.767 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:32:02.767 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-10T21:32:02.768 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-10T21:32:02.768 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:32:02.768 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:32:02.768 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:32:02.768 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:32:02.768 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:32:02.768 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:32:02.768 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": [ 2024-09-10T21:32:02.768 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:32:02.769 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:32:02.769 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:32:02.769 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 5 2024-09-10T21:32:02.769 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:32:02.769 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:32:02.769 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-10T21:32:02.769 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:32:02.770 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:32:02.770 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:32:02.770 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 7 2024-09-10T21:32:02.770 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:32:02.770 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:32:02.770 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-10T21:32:02.770 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:32:02.770 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:32:02.770 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:32:02.771 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 5 2024-09-10T21:32:02.771 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:32:02.771 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:32:02.771 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": [ 2024-09-10T21:32:02.771 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:32:02.771 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:32:02.771 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:32:02.771 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 2 2024-09-10T21:32:02.772 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:32:02.772 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:32:02.772 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:32:02.772 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-10T21:32:02.772 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-10T21:32:02.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.750+0000 7f8abee00640 1 -- 172.21.15.17:0/70978730 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f8a98072050 msgr2=0x7f8a98074510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:02.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.750+0000 7f8abee00640 1 --2- 172.21.15.17:0/70978730 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f8a98072050 0x7f8a98074510 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f8aac000c10 tx=0x7f8aac008030 comp rx=0 tx=0).stop 2024-09-10T21:32:02.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.750+0000 7f8abee00640 1 -- 172.21.15.17:0/70978730 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8ab8072bb0 msgr2=0x7f8ab8159a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:02.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.750+0000 7f8abee00640 1 --2- 172.21.15.17:0/70978730 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8ab8072bb0 0x7f8ab8159a40 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f8aa4009850 tx=0x7f8aa4005df0 comp rx=0 tx=0).stop 2024-09-10T21:32:02.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.750+0000 7f8abe400640 1 -- 172.21.15.17:0/70978730 reap_dead start 2024-09-10T21:32:02.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.750+0000 7f8abee00640 1 -- 172.21.15.17:0/70978730 shutdown_connections 2024-09-10T21:32:02.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.750+0000 7f8abee00640 1 --2- 172.21.15.17:0/70978730 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8ab8072bb0 0x7f8ab8159a40 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:02.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.750+0000 7f8abee00640 1 -- 172.21.15.17:0/70978730 >> 172.21.15.17:0/70978730 conn(0x7f8ab806cc20 msgr2=0x7f8ab806d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:32:02.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.750+0000 7f8abee00640 1 -- 172.21.15.17:0/70978730 shutdown_connections 2024-09-10T21:32:02.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.750+0000 7f8abee00640 1 -- 172.21.15.17:0/70978730 wait complete. 2024-09-10T21:32:02.776 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-09-10T21:32:02.776 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon c 2024-09-10T21:32:02.776 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-10T21:32:02.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.942+0000 7f9641e00640 1 -- 172.21.15.17:0/621863646 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f963c0c77e0 msgr2=0x7f963c06f890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:02.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.942+0000 7f9641e00640 1 --2- 172.21.15.17:0/621863646 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f963c0c77e0 0x7f963c06f890 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f963c0cde20 tx=0x7f963401cc80 comp rx=0 tx=0).stop 2024-09-10T21:32:02.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.942+0000 7f9641e00640 1 -- 172.21.15.17:0/621863646 shutdown_connections 2024-09-10T21:32:02.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.942+0000 7f9641e00640 1 --2- 172.21.15.17:0/621863646 >> v2:172.21.15.17:3300/0 conn(0x7f963c06ec30 0x7f963c06f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:02.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.942+0000 7f9641e00640 1 --2- 172.21.15.17:0/621863646 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f963c0c77e0 0x7f963c06f890 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:02.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.942+0000 7f9641e00640 1 -- 172.21.15.17:0/621863646 >> 172.21.15.17:0/621863646 conn(0x7f963c0cd690 msgr2=0x7f963c0c98e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:32:02.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.942+0000 7f9641e00640 1 -- 172.21.15.17:0/621863646 shutdown_connections 2024-09-10T21:32:02.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.942+0000 7f9641e00640 1 -- 172.21.15.17:0/621863646 wait complete. 2024-09-10T21:32:02.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.942+0000 7f9641e00640 1 Processor -- start 2024-09-10T21:32:02.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.942+0000 7f9641e00640 1 -- start start 2024-09-10T21:32:02.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.946+0000 7f9641e00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f963c06ec30 0x7f963c146da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:02.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.946+0000 7f9641e00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f963c0c77e0 0x7f963c147f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:02.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.946+0000 7f9641e00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f963c152460 0x7f963c150890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:02.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.946+0000 7f9641e00640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f963c0cc730 con 0x7f963c0c77e0 2024-09-10T21:32:02.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.946+0000 7f9641e00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f963c0ca800 con 0x7f963c06ec30 2024-09-10T21:32:02.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.946+0000 7f9641e00640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f963c0cc8b0 con 0x7f963c152460 2024-09-10T21:32:02.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.946+0000 7f963be00640 1 -- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f963c0c77e0 msgr2=0x7f963c147f40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-10T21:32:02.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.946+0000 7f963be00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f963c0c77e0 0x7f963c147f40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:32:02.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.946+0000 7f9640a00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f963c06ec30 0x7f963c146da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:02.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.946+0000 7f9641400640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f963c152460 0x7f963c150890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:02.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.946+0000 7f9640a00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f963c06ec30 0x7f963c146da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.17:57010/0 (socket says 172.21.15.17:57010) 2024-09-10T21:32:02.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.946+0000 7f9640a00640 1 -- 172.21.15.17:0/1807328182 learned_addr learned my addr 172.21.15.17:0/1807328182 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:32:02.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.946+0000 7f9641400640 1 -- 172.21.15.17:0/1807328182 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f963c06ec30 msgr2=0x7f963c146da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:02.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.946+0000 7f9641400640 1 --2- 172.21.15.17:0/1807328182 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f963c06ec30 0x7f963c146da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:02.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.946+0000 7f9641400640 1 -- 172.21.15.17:0/1807328182 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f963c0c77e0 msgr2=0x7f963c147f40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:32:02.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.946+0000 7f9641400640 1 --2- 172.21.15.17:0/1807328182 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f963c0c77e0 0x7f963c147f40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:02.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.946+0000 7f9641400640 1 -- 172.21.15.17:0/1807328182 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f963c0b1f90 con 0x7f963c152460 2024-09-10T21:32:02.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.946+0000 7f9640a00640 1 --2- 172.21.15.17:0/1807328182 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f963c06ec30 0x7f963c146da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:32:02.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.946+0000 7f9641400640 1 --2- 172.21.15.17:0/1807328182 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f963c152460 0x7f963c150890 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f963c06d290 tx=0x7f96340a5a00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:02.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.946+0000 7f9638c00640 1 -- 172.21.15.17:0/1807328182 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96340bc070 con 0x7f963c152460 2024-09-10T21:32:02.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.946+0000 7f9638c00640 1 -- 172.21.15.17:0/1807328182 <== mon.2 v2:172.21.15.31:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9634004430 con 0x7f963c152460 2024-09-10T21:32:02.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.946+0000 7f9638c00640 1 -- 172.21.15.17:0/1807328182 <== mon.2 v2:172.21.15.31:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96340b2040 con 0x7f963c152460 2024-09-10T21:32:02.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.946+0000 7f9641e00640 1 -- 172.21.15.17:0/1807328182 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f963c0b2230 con 0x7f963c152460 2024-09-10T21:32:02.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.946+0000 7f9641e00640 1 -- 172.21.15.17:0/1807328182 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f963c0c3ad0 con 0x7f963c152460 2024-09-10T21:32:02.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.950+0000 7f9641e00640 1 --2- 172.21.15.17:0/1807328182 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f963c0c3ca0 0x7f963c150dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:02.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.950+0000 7f9641e00640 1 -- 172.21.15.17:0/1807328182 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f963c069ef0 con 0x7f963c0c3ca0 2024-09-10T21:32:02.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.950+0000 7f9640a00640 1 --2- 172.21.15.17:0/1807328182 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f963c0c3ca0 0x7f963c150dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:02.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.950+0000 7f9640a00640 1 -- 172.21.15.17:0/1807328182 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f963c150470 con 0x7f963c0c3ca0 2024-09-10T21:32:02.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.950+0000 7f9640a00640 1 --2- 172.21.15.17:0/1807328182 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f963c0c3ca0 0x7f963c150dd0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f963000eae0 tx=0x7f963000c150 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:02.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.950+0000 7f9638c00640 1 -- 172.21.15.17:0/1807328182 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9630017070 con 0x7f963c0c3ca0 2024-09-10T21:32:02.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.950+0000 7f9638c00640 1 -- 172.21.15.17:0/1807328182 <== mon.2 v2:172.21.15.31:3301/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f96340b5070 con 0x7f963c152460 2024-09-10T21:32:02.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.950+0000 7f9638c00640 1 --2- 172.21.15.17:0/1807328182 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f961c0724e0 0x7f961c0749a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:02.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.950+0000 7f9638c00640 1 -- 172.21.15.17:0/1807328182 <== mon.2 v2:172.21.15.31:3301/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f963412d7e0 con 0x7f963c152460 2024-09-10T21:32:02.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.950+0000 7f9638c00640 1 -- 172.21.15.17:0/1807328182 <== mon.2 v2:172.21.15.31:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f963000cba0 con 0x7f963c0c3ca0 2024-09-10T21:32:02.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.950+0000 7f9638c00640 1 -- 172.21.15.17:0/1807328182 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f963c0c3ca0 msgr2=0x7f963c150dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:02.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.950+0000 7f9638c00640 1 --2- 172.21.15.17:0/1807328182 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f963c0c3ca0 0x7f963c150dd0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f963000eae0 tx=0x7f963000c150 comp rx=0 tx=0).stop 2024-09-10T21:32:02.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.950+0000 7f9638c00640 1 -- 172.21.15.17:0/1807328182 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f963c0c3ca0 msgr2=0x7f963c150dd0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:32:02.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.950+0000 7f9638c00640 1 --2- 172.21.15.17:0/1807328182 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f963c0c3ca0 0x7f963c150dd0 secure :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f963000eae0 tx=0x7f963000c150 comp rx=0 tx=0).stop 2024-09-10T21:32:02.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.950+0000 7f963be00640 1 --2- 172.21.15.17:0/1807328182 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f961c0724e0 0x7f961c0749a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:02.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.954+0000 7f963be00640 1 --2- 172.21.15.17:0/1807328182 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f961c0724e0 0x7f961c0749a0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f963c06d800 tx=0x7f9628006d30 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:02.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.958+0000 7f9641e00640 1 --2- 172.21.15.17:0/1807328182 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f963c148480 0x7f963c0c1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:02.963 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.958+0000 7f9641e00640 1 -- 172.21.15.17:0/1807328182 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f963c0c8770 con 0x7f963c148480 2024-09-10T21:32:02.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.962+0000 7f9640a00640 1 --2- 172.21.15.17:0/1807328182 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f963c148480 0x7f963c0c1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:02.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.962+0000 7f9640a00640 1 -- 172.21.15.17:0/1807328182 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f963c14e030 con 0x7f963c148480 2024-09-10T21:32:02.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.962+0000 7f9640a00640 1 --2- 172.21.15.17:0/1807328182 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f963c148480 0x7f963c0c1fb0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f9630019000 tx=0x7f9630004810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:02.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.962+0000 7f9638c00640 1 -- 172.21.15.17:0/1807328182 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9630006710 con 0x7f963c148480 2024-09-10T21:32:02.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.962+0000 7f9638c00640 1 -- 172.21.15.17:0/1807328182 <== mon.2 v2:172.21.15.31:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4863 (secure 0 0 0) 0x7f9630006710 con 0x7f963c148480 2024-09-10T21:32:02.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.962+0000 7f9638c00640 1 -- 172.21.15.17:0/1807328182 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f963c148480 msgr2=0x7f963c0c1fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:02.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.962+0000 7f9638c00640 1 --2- 172.21.15.17:0/1807328182 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f963c148480 0x7f963c0c1fb0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f9630019000 tx=0x7f9630004810 comp rx=0 tx=0).stop 2024-09-10T21:32:02.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.962+0000 7f9638c00640 1 -- 172.21.15.17:0/1807328182 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f963c148480 msgr2=0x7f963c0c1fb0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:32:02.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.962+0000 7f9638c00640 1 --2- 172.21.15.17:0/1807328182 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f963c148480 0x7f963c0c1fb0 secure :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f9630019000 tx=0x7f9630004810 comp rx=0 tx=0).stop 2024-09-10T21:32:02.966 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-10T21:32:02.966 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:32:02.966 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:32:02.966 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-10T21:32:02.966 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 60, 2024-09-10T21:32:02.966 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 55267, 2024-09-10T21:32:02.967 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-10T21:32:02.967 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-10T21:32:02.967 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-10T21:32:02.967 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:32:02.967 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 1, 2024-09-10T21:32:02.967 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:32:02.967 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-10T21:32:02.967 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-10T21:32:02.967 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:32:02.967 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:32:02.967 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:32:02.967 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:32:02.967 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:32:02.967 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:32:02.968 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:32:02.968 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:32:02.968 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:32:02.968 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:32:02.968 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:32:02.968 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:32:02.968 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-10T21:32:02.968 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-10T21:32:02.968 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:32:02.968 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:32:02.968 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:32:02.968 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:32:02.968 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:32:02.968 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:32:02.969 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:32:02.969 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:32:02.969 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:32:02.969 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:32:02.969 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:32:02.969 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:32:02.969 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:32:02.969 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-10T21:32:02.969 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-10T21:32:02.969 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-10T21:32:02.969 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-10T21:32:02.969 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-10T21:32:02.969 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "5e2f023f-d6a6-49f4-a256-f4807dce3739", 2024-09-10T21:32:02.969 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:32:02.970 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:32:02.970 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-10T21:32:02.970 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-10T21:32:02.970 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-10T21:32:02.970 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-10T21:32:02.970 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-10T21:32:02.970 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-10T21:32:02.970 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-10T21:32:02.970 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:32:02.970 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-10T21:32:02.970 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:32:02.970 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:32:02.971 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:32:02.971 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:32:02.971 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:32:02.971 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:32:02.971 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:32:02.971 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:32:02.971 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:32:02.971 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:32:02.971 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:32:02.971 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:32:02.971 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-10T21:32:02.972 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:32:02.972 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-10T21:32:02.972 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:32:02.972 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:32:02.972 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:32:02.972 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:32:02.972 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:32:02.972 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:32:02.972 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:32:02.972 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-10T21:32:02.973 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:32:02.973 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:32:02.973 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:32:02.973 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:32:02.973 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-10T21:32:02.973 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:32:02.973 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:32:02.973 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:32:02.973 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:32:02.973 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-10T21:32:02.973 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-10T21:32:02.973 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:32:02.974 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:32:02.974 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:32:02.974 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:32:02.974 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:32:02.974 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:32:02.974 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:32:02.974 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:32:02.974 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:32:02.974 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:32:02.974 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:32:02.975 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3300", 2024-09-10T21:32:02.975 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:32:02.975 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:32:02.975 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:32:02.975 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:32:02.975 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789", 2024-09-10T21:32:02.975 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:32:02.975 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:32:02.975 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:32:02.975 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:32:02.975 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789/0", 2024-09-10T21:32:02.975 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6789/0", 2024-09-10T21:32:02.975 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:32:02.976 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:32:02.976 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:32:02.976 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:32:02.976 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:32:02.976 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:32:02.976 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:32:02.976 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:32:02.976 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:32:02.976 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:32:02.976 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:32:02.976 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3301", 2024-09-10T21:32:02.976 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:32:02.977 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:32:02.977 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:32:02.977 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:32:02.977 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790", 2024-09-10T21:32:02.977 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:32:02.977 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:32:02.977 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:32:02.977 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:32:02.977 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790/0", 2024-09-10T21:32:02.977 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6790/0", 2024-09-10T21:32:02.977 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:32:02.977 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:32:02.977 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:32:02.978 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:32:02.978 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:32:02.978 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:32:02.978 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-10T21:32:02.978 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-10T21:32:02.978 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:32:02.978 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:32:02.978 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:32:02.978 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:32:02.978 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:32:02.978 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:32:02.978 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": [ 2024-09-10T21:32:02.979 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:32:02.979 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:32:02.979 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:32:02.979 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:32:02.979 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:32:02.979 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:32:02.979 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-10T21:32:02.979 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:32:02.979 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:32:02.979 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:32:02.979 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:32:02.979 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:32:02.979 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:32:02.980 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-10T21:32:02.980 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:32:02.980 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:32:02.980 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:32:02.980 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 3 2024-09-10T21:32:02.980 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:32:02.980 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:32:02.980 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:32:02.980 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-10T21:32:02.980 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-10T21:32:02.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.962+0000 7f9641e00640 1 -- 172.21.15.17:0/1807328182 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f961c0724e0 msgr2=0x7f961c0749a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:02.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.962+0000 7f9641e00640 1 --2- 172.21.15.17:0/1807328182 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f961c0724e0 0x7f961c0749a0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f963c06d800 tx=0x7f9628006d30 comp rx=0 tx=0).stop 2024-09-10T21:32:02.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.962+0000 7f9641e00640 1 -- 172.21.15.17:0/1807328182 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f963c152460 msgr2=0x7f963c150890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:02.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.962+0000 7f9641e00640 1 --2- 172.21.15.17:0/1807328182 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f963c152460 0x7f963c150890 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f963c06d290 tx=0x7f96340a5a00 comp rx=0 tx=0).stop 2024-09-10T21:32:02.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.962+0000 7f9641400640 1 -- 172.21.15.17:0/1807328182 reap_dead start 2024-09-10T21:32:02.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.962+0000 7f9641400640 1 -- 172.21.15.17:0/1807328182 reap_dead start 2024-09-10T21:32:02.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.962+0000 7f9641e00640 1 -- 172.21.15.17:0/1807328182 shutdown_connections 2024-09-10T21:32:02.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.962+0000 7f9641e00640 1 -- 172.21.15.17:0/1807328182 >> 172.21.15.17:0/1807328182 conn(0x7f963c0cd690 msgr2=0x7f963c142be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:32:02.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.962+0000 7f9641e00640 1 -- 172.21.15.17:0/1807328182 shutdown_connections 2024-09-10T21:32:02.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:02.962+0000 7f9641e00640 1 -- 172.21.15.17:0/1807328182 wait complete. 2024-09-10T21:32:02.987 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-09-10T21:32:02.987 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2024-09-10T21:32:17.436 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:32:17.440 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:32:22.988 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.a 2024-09-10T21:32:22.988 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-10T21:32:22.988 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-10T21:32:22.992 INFO:tasks.ceph.mon.a:Started 2024-09-10T21:32:22.992 INFO:tasks.mon_thrash:Sending STOP to mon b 2024-09-10T21:32:22.992 INFO:tasks.ceph.mon.b:Sent signal 19 2024-09-10T21:32:22.993 INFO:tasks.mon_thrash.mon_thrasher:waiting for 15.0 secs to unfreeze mons 2024-09-10T21:32:37.993 INFO:tasks.mon_thrash:Sending CONT to mon b 2024-09-10T21:32:37.994 INFO:tasks.ceph.mon.b:Sent signal 18 2024-09-10T21:32:37.995 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-09-10T21:32:37.995 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-10T21:32:47.508 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:32:47.512 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:32:48.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:48.045+0000 7f50dca00640 1 -- 172.21.15.17:0/2050393591 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f50bc0052a0 msgr2=0x7f50bc00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:48.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:48.045+0000 7f50dca00640 1 --2- 172.21.15.17:0/2050393591 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f50bc0052a0 0x7f50bc00b9d0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f50d8152c00 tx=0x7f50d001cdf0 comp rx=0 tx=0).stop 2024-09-10T21:32:48.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:48.045+0000 7f50dca00640 1 -- 172.21.15.17:0/2050393591 shutdown_connections 2024-09-10T21:32:48.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:48.045+0000 7f50dca00640 1 --2- 172.21.15.17:0/2050393591 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f50bc0052a0 0x7f50bc00b9d0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:48.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:48.045+0000 7f50dca00640 1 --2- 172.21.15.17:0/2050393591 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f50bc004810 0x7f50bc004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:48.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:48.045+0000 7f50dca00640 1 --2- 172.21.15.17:0/2050393591 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f50bc0039c0 0x7f50bc003dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:48.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:48.045+0000 7f50dca00640 1 -- 172.21.15.17:0/2050393591 >> 172.21.15.17:0/2050393591 conn(0x7f50d80702f0 msgr2=0x7f50d8070700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:32:48.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:48.045+0000 7f50dca00640 1 -- 172.21.15.17:0/2050393591 shutdown_connections 2024-09-10T21:32:48.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:48.045+0000 7f50dca00640 1 -- 172.21.15.17:0/2050393591 wait complete. 2024-09-10T21:32:48.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:48.049+0000 7f50dca00640 1 Processor -- start 2024-09-10T21:32:48.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:48.049+0000 7f50dca00640 1 -- start start 2024-09-10T21:32:48.055 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:48.049+0000 7f50dca00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f50bc0039c0 0x7f50d8149140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:48.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:48.049+0000 7f50dca00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f50bc004810 0x7f50d815aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:48.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:48.049+0000 7f50dca00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f50bc0052a0 0x7f50d815afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:48.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:48.049+0000 7f50dca00640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f50d815bd60 con 0x7f50bc004810 2024-09-10T21:32:48.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:48.049+0000 7f50dca00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f50d815ba80 con 0x7f50bc0039c0 2024-09-10T21:32:48.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:48.049+0000 7f50dca00640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f50d80cb450 con 0x7f50bc0052a0 2024-09-10T21:32:48.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:48.049+0000 7f50cfe00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f50bc004810 0x7f50d815aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:48.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:48.049+0000 7f50cfe00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f50bc004810 0x7f50d815aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37740/0 (socket says 172.21.15.17:37740) 2024-09-10T21:32:48.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:48.049+0000 7f50cfe00640 1 -- 172.21.15.17:0/3002467672 learned_addr learned my addr 172.21.15.17:0/3002467672 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:32:48.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:48.049+0000 7f50d5e00640 1 --2- 172.21.15.17:0/3002467672 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f50bc0039c0 0x7f50d8149140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:48.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:48.049+0000 7f50d6800640 1 --2- 172.21.15.17:0/3002467672 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f50bc0052a0 0x7f50d815afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:48.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:48.049+0000 7f50d6800640 1 -- 172.21.15.17:0/3002467672 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f50bc0052a0 msgr2=0x7f50d815afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-09-10T21:32:48.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:48.049+0000 7f50d6800640 1 -- 172.21.15.17:0/3002467672 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f50bc0052a0 msgr2=0x7f50d815afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-10T21:32:48.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:48.049+0000 7f50d6800640 1 --2- 172.21.15.17:0/3002467672 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f50bc0052a0 0x7f50d815afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T21:32:48.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:48.049+0000 7f50d6800640 1 --2- 172.21.15.17:0/3002467672 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f50bc0052a0 0x7f50d815afc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:32:48.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:48.049+0000 7f50cfe00640 1 -- 172.21.15.17:0/3002467672 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f50bc0052a0 msgr2=0x7f50d815afc0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:32:48.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:48.049+0000 7f50cfe00640 1 --2- 172.21.15.17:0/3002467672 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f50bc0052a0 0x7f50d815afc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:48.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:48.049+0000 7f50cfe00640 1 -- 172.21.15.17:0/3002467672 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f50bc0039c0 msgr2=0x7f50d8149140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:48.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:48.049+0000 7f50cfe00640 1 --2- 172.21.15.17:0/3002467672 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f50bc0039c0 0x7f50d8149140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:48.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:48.049+0000 7f50cfe00640 1 -- 172.21.15.17:0/3002467672 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50d8149680 con 0x7f50bc004810 2024-09-10T21:32:48.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:48.049+0000 7f50cfe00640 1 --2- 172.21.15.17:0/3002467672 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f50bc004810 0x7f50d815aa80 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f50c0010ea0 tx=0x7f50c000f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:48.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:48.053+0000 7f50ce000640 1 -- 172.21.15.17:0/3002467672 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50c0004050 con 0x7f50bc004810 2024-09-10T21:32:48.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:48.053+0000 7f50ce000640 1 -- 172.21.15.17:0/3002467672 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f50c0026070 con 0x7f50bc004810 2024-09-10T21:32:48.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:48.053+0000 7f50dca00640 1 -- 172.21.15.17:0/3002467672 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50d80c60d0 con 0x7f50bc004810 2024-09-10T21:32:48.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:48.053+0000 7f50ce000640 1 -- 172.21.15.17:0/3002467672 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50c0004470 con 0x7f50bc004810 2024-09-10T21:32:48.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:48.053+0000 7f50dca00640 1 -- 172.21.15.17:0/3002467672 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50d80c6390 con 0x7f50bc004810 2024-09-10T21:32:48.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:48.053+0000 7f50dca00640 1 -- 172.21.15.17:0/3002467672 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50d80c2ff0 con 0x7f50bc004810 2024-09-10T21:32:48.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:48.053+0000 7f50ce000640 1 -- 172.21.15.17:0/3002467672 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f50c0003600 con 0x7f50bc004810 2024-09-10T21:32:48.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:48.057+0000 7f50ce000640 1 --2- 172.21.15.17:0/3002467672 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f50b4073c80 0x7f50b4076140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:48.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:48.057+0000 7f50ce000640 1 -- 172.21.15.17:0/3002467672 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f50c0099790 con 0x7f50bc004810 2024-09-10T21:32:48.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:48.057+0000 7f50d5e00640 1 --2- 172.21.15.17:0/3002467672 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f50b4073c80 0x7f50b4076140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:48.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:48.057+0000 7f50d5e00640 1 --2- 172.21.15.17:0/3002467672 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f50b4073c80 0x7f50b4076140 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f50c8000c10 tx=0x7f50c8006050 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:48.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:48.061+0000 7f50ce000640 1 -- 172.21.15.17:0/3002467672 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f50c0064a50 con 0x7f50bc004810 2024-09-10T21:32:48.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:48.337+0000 7f50dca00640 1 -- 172.21.15.17:0/3002467672 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f50d80ca450 con 0x7f50bc004810 2024-09-10T21:32:48.341 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":68,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5e2f023f-d6a6-49f4-a256-f4807dce3739","modified":"2024-09-10T21:23:46.892130Z","created":"2024-09-10T21:23:46.892130Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3301","nonce":0},{"type":"v1","addr":"172.21.15.31:6790","nonce":0}]},"addr":"172.21.15.31:6790/0","public_addr":"172.21.15.31:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-10T21:32:48.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:48.337+0000 7f50ce000640 1 -- 172.21.15.17:0/3002467672 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1550 (secure 0 0 0) 0x7f50c006d170 con 0x7f50bc004810 2024-09-10T21:32:48.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:48.337+0000 7f50dca00640 1 -- 172.21.15.17:0/3002467672 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f50b4073c80 msgr2=0x7f50b4076140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:48.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:48.337+0000 7f50dca00640 1 --2- 172.21.15.17:0/3002467672 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f50b4073c80 0x7f50b4076140 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f50c8000c10 tx=0x7f50c8006050 comp rx=0 tx=0).stop 2024-09-10T21:32:48.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:48.337+0000 7f50dca00640 1 -- 172.21.15.17:0/3002467672 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f50bc004810 msgr2=0x7f50d815aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:48.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:48.337+0000 7f50dca00640 1 --2- 172.21.15.17:0/3002467672 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f50bc004810 0x7f50d815aa80 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f50c0010ea0 tx=0x7f50c000f790 comp rx=0 tx=0).stop 2024-09-10T21:32:48.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:48.337+0000 7f50dca00640 1 -- 172.21.15.17:0/3002467672 shutdown_connections 2024-09-10T21:32:48.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:48.337+0000 7f50dca00640 1 --2- 172.21.15.17:0/3002467672 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f50b4073c80 0x7f50b4076140 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:48.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:48.337+0000 7f50dca00640 1 --2- 172.21.15.17:0/3002467672 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f50bc0052a0 0x7f50d815afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:48.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:48.337+0000 7f50dca00640 1 --2- 172.21.15.17:0/3002467672 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f50bc004810 0x7f50d815aa80 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:48.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:48.337+0000 7f50dca00640 1 --2- 172.21.15.17:0/3002467672 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f50bc0039c0 0x7f50d8149140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:48.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:48.337+0000 7f50dca00640 1 -- 172.21.15.17:0/3002467672 >> 172.21.15.17:0/3002467672 conn(0x7f50d80702f0 msgr2=0x7f50d80c3200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:32:48.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:48.337+0000 7f50dca00640 1 -- 172.21.15.17:0/3002467672 shutdown_connections 2024-09-10T21:32:48.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:48.337+0000 7f50dca00640 1 -- 172.21.15.17:0/3002467672 wait complete. 2024-09-10T21:32:51.450 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-10T21:32:51.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:51.605+0000 7f221be00640 1 -- 172.21.15.17:0/139322274 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f221c06f860 msgr2=0x7f221c0b2c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:51.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:51.605+0000 7f221be00640 1 --2- 172.21.15.17:0/139322274 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f221c06f860 0x7f221c0b2c60 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f221c072990 tx=0x7f220801c8c0 comp rx=0 tx=0).stop 2024-09-10T21:32:51.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:51.605+0000 7f221be00640 1 -- 172.21.15.17:0/139322274 shutdown_connections 2024-09-10T21:32:51.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:51.605+0000 7f221be00640 1 --2- 172.21.15.17:0/139322274 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f221c0c8070 0x7f221c15a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:51.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:51.605+0000 7f221be00640 1 --2- 172.21.15.17:0/139322274 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f221c06f860 0x7f221c0b2c60 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:51.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:51.605+0000 7f221be00640 1 --2- 172.21.15.17:0/139322274 >> v2:172.21.15.31:3300/0 conn(0x7f221c06f480 0x7f221c0c7870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:51.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:51.605+0000 7f221be00640 1 -- 172.21.15.17:0/139322274 >> 172.21.15.17:0/139322274 conn(0x7f221c070ca0 msgr2=0x7f221c0710b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:32:51.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:51.605+0000 7f221be00640 1 -- 172.21.15.17:0/139322274 shutdown_connections 2024-09-10T21:32:51.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:51.605+0000 7f221be00640 1 -- 172.21.15.17:0/139322274 wait complete. 2024-09-10T21:32:51.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:51.605+0000 7f221be00640 1 Processor -- start 2024-09-10T21:32:51.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:51.605+0000 7f221be00640 1 -- start start 2024-09-10T21:32:51.613 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:51.609+0000 7f221be00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f221c06f480 0x7f221c155c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:51.613 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:51.609+0000 7f221be00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f221c06f860 0x7f221c1f7300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:51.613 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:51.609+0000 7f221be00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f221c0c8070 0x7f221c1f7840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:51.613 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:51.609+0000 7f221be00640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f221c157700 con 0x7f221c0c8070 2024-09-10T21:32:51.613 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:51.609+0000 7f221be00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f221c157580 con 0x7f221c06f860 2024-09-10T21:32:51.613 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:51.609+0000 7f221be00640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f221c157880 con 0x7f221c06f480 2024-09-10T21:32:51.613 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:51.609+0000 7f221b400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f221c0c8070 0x7f221c1f7840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:51.613 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:51.609+0000 7f221a000640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f221c06f860 0x7f221c1f7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:51.614 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:51.609+0000 7f221b400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f221c0c8070 0x7f221c1f7840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37754/0 (socket says 172.21.15.17:37754) 2024-09-10T21:32:51.614 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:51.609+0000 7f221b400640 1 -- 172.21.15.17:0/2054472030 learned_addr learned my addr 172.21.15.17:0/2054472030 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:32:51.614 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:51.609+0000 7f221aa00640 1 --2- 172.21.15.17:0/2054472030 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f221c06f480 0x7f221c155c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:51.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:51.609+0000 7f221b400640 1 -- 172.21.15.17:0/2054472030 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f221c06f480 msgr2=0x7f221c155c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:51.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:51.609+0000 7f221b400640 1 --2- 172.21.15.17:0/2054472030 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f221c06f480 0x7f221c155c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:51.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:51.609+0000 7f221b400640 1 -- 172.21.15.17:0/2054472030 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f221c06f860 msgr2=0x7f221c1f7300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:51.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:51.609+0000 7f221b400640 1 --2- 172.21.15.17:0/2054472030 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f221c06f860 0x7f221c1f7300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:51.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:51.609+0000 7f221b400640 1 -- 172.21.15.17:0/2054472030 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f221c14efd0 con 0x7f221c0c8070 2024-09-10T21:32:51.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:51.609+0000 7f221a000640 1 --2- 172.21.15.17:0/2054472030 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f221c06f860 0x7f221c1f7300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:32:51.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:51.609+0000 7f221b400640 1 --2- 172.21.15.17:0/2054472030 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f221c0c8070 0x7f221c1f7840 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f221400e970 tx=0x7f221400ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:51.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:51.609+0000 7f2212a00640 1 -- 172.21.15.17:0/2054472030 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2214016070 con 0x7f221c0c8070 2024-09-10T21:32:51.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:51.609+0000 7f2212a00640 1 -- 172.21.15.17:0/2054472030 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2214004b80 con 0x7f221c0c8070 2024-09-10T21:32:51.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:51.609+0000 7f221be00640 1 -- 172.21.15.17:0/2054472030 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f221c14fb50 con 0x7f221c0c8070 2024-09-10T21:32:51.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:51.609+0000 7f2212a00640 1 -- 172.21.15.17:0/2054472030 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f221400c910 con 0x7f221c0c8070 2024-09-10T21:32:51.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:51.609+0000 7f221be00640 1 -- 172.21.15.17:0/2054472030 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f221c14f810 con 0x7f221c0c8070 2024-09-10T21:32:51.616 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:51.609+0000 7f221be00640 1 -- 172.21.15.17:0/2054472030 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f221c0b2c60 con 0x7f221c0c8070 2024-09-10T21:32:51.616 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:51.613+0000 7f2212a00640 1 -- 172.21.15.17:0/2054472030 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f221400f040 con 0x7f221c0c8070 2024-09-10T21:32:51.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:51.613+0000 7f2212a00640 1 --2- 172.21.15.17:0/2054472030 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f21fc0725e0 0x7f21fc074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:51.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:51.613+0000 7f2212a00640 1 -- 172.21.15.17:0/2054472030 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f22140939f0 con 0x7f221c0c8070 2024-09-10T21:32:51.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:51.613+0000 7f221aa00640 1 --2- 172.21.15.17:0/2054472030 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f21fc0725e0 0x7f21fc074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:51.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:51.613+0000 7f221aa00640 1 --2- 172.21.15.17:0/2054472030 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f21fc0725e0 0x7f21fc074aa0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f221c149970 tx=0x7f220c00b2e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:51.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:51.617+0000 7f2212a00640 1 -- 172.21.15.17:0/2054472030 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f221405e5f0 con 0x7f221c0c8070 2024-09-10T21:32:51.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:51.893+0000 7f221be00640 1 -- 172.21.15.17:0/2054472030 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f221c1465a0 con 0x7f221c0c8070 2024-09-10T21:32:51.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:51.893+0000 7f2212a00640 1 -- 172.21.15.17:0/2054472030 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1556 (secure 0 0 0) 0x7f2214065030 con 0x7f221c0c8070 2024-09-10T21:32:51.898 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":70,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":3,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5e2f023f-d6a6-49f4-a256-f4807dce3739","modified":"2024-09-10T21:23:46.892130Z","created":"2024-09-10T21:23:46.892130Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3301","nonce":0},{"type":"v1","addr":"172.21.15.31:6790","nonce":0}]},"addr":"172.21.15.31:6790/0","public_addr":"172.21.15.31:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-10T21:32:51.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:51.893+0000 7f221be00640 1 -- 172.21.15.17:0/2054472030 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f21fc0725e0 msgr2=0x7f21fc074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:51.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:51.893+0000 7f221be00640 1 --2- 172.21.15.17:0/2054472030 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f21fc0725e0 0x7f21fc074aa0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f221c149970 tx=0x7f220c00b2e0 comp rx=0 tx=0).stop 2024-09-10T21:32:51.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:51.893+0000 7f221be00640 1 -- 172.21.15.17:0/2054472030 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f221c0c8070 msgr2=0x7f221c1f7840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:51.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:51.893+0000 7f221be00640 1 --2- 172.21.15.17:0/2054472030 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f221c0c8070 0x7f221c1f7840 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f221400e970 tx=0x7f221400ee30 comp rx=0 tx=0).stop 2024-09-10T21:32:51.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:51.897+0000 7f221be00640 1 -- 172.21.15.17:0/2054472030 shutdown_connections 2024-09-10T21:32:51.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:51.897+0000 7f221be00640 1 --2- 172.21.15.17:0/2054472030 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f21fc0725e0 0x7f21fc074aa0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:51.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:51.897+0000 7f221be00640 1 --2- 172.21.15.17:0/2054472030 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f221c0c8070 0x7f221c1f7840 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:51.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:51.897+0000 7f221be00640 1 --2- 172.21.15.17:0/2054472030 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f221c06f860 0x7f221c1f7300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:51.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:51.897+0000 7f221be00640 1 --2- 172.21.15.17:0/2054472030 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f221c06f480 0x7f221c155c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:51.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:51.897+0000 7f221be00640 1 -- 172.21.15.17:0/2054472030 >> 172.21.15.17:0/2054472030 conn(0x7f221c070ca0 msgr2=0x7f221c0c34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:32:51.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:51.897+0000 7f221be00640 1 -- 172.21.15.17:0/2054472030 shutdown_connections 2024-09-10T21:32:51.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:51.897+0000 7f221be00640 1 -- 172.21.15.17:0/2054472030 wait complete. 2024-09-10T21:32:51.924 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-09-10T21:32:51.924 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-09-10T21:32:51.924 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-10T21:32:52.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.105+0000 7fdc4e800640 1 -- 172.21.15.17:0/2749089209 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdc300053d0 msgr2=0x7fdc3000b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:52.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.105+0000 7fdc4e800640 1 --2- 172.21.15.17:0/2749089209 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdc300053d0 0x7fdc3000b9d0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fdc4000c640 tx=0x7fdc4001d240 comp rx=0 tx=0).stop 2024-09-10T21:32:52.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.105+0000 7fdc4e800640 1 -- 172.21.15.17:0/2749089209 shutdown_connections 2024-09-10T21:32:52.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.105+0000 7fdc4e800640 1 --2- 172.21.15.17:0/2749089209 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdc300053d0 0x7fdc3000b9d0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:52.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.105+0000 7fdc4e800640 1 -- 172.21.15.17:0/2749089209 >> 172.21.15.17:0/2749089209 conn(0x7fdc48068990 msgr2=0x7fdc48068da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:32:52.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.105+0000 7fdc4e800640 1 -- 172.21.15.17:0/2749089209 shutdown_connections 2024-09-10T21:32:52.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.105+0000 7fdc4e800640 1 -- 172.21.15.17:0/2749089209 wait complete. 2024-09-10T21:32:52.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.109+0000 7fdc4e800640 1 Processor -- start 2024-09-10T21:32:52.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.109+0000 7fdc4e800640 1 -- start start 2024-09-10T21:32:52.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.109+0000 7fdc4e800640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdc300053d0 0x7fdc48072490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:52.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.109+0000 7fdc4e800640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdc48156d90 0x7fdc481f6fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:52.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.109+0000 7fdc4e800640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fdc4813ebf0 0x7fdc481f7520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:52.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.109+0000 7fdc4e800640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fdc4815c120 con 0x7fdc48156d90 2024-09-10T21:32:52.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.109+0000 7fdc4e800640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fdc480c8be0 con 0x7fdc300053d0 2024-09-10T21:32:52.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.109+0000 7fdc4e800640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7fdc4815c2a0 con 0x7fdc4813ebf0 2024-09-10T21:32:52.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.109+0000 7fdc4ca00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdc48156d90 0x7fdc481f6fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:52.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.109+0000 7fdc4ca00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdc48156d90 0x7fdc481f6fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37772/0 (socket says 172.21.15.17:37772) 2024-09-10T21:32:52.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.109+0000 7fdc4ca00640 1 -- 172.21.15.17:0/4194638623 learned_addr learned my addr 172.21.15.17:0/4194638623 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:32:52.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.109+0000 7fdc4d400640 1 --2- 172.21.15.17:0/4194638623 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdc300053d0 0x7fdc48072490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:52.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.109+0000 7fdc4de00640 1 --2- 172.21.15.17:0/4194638623 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fdc4813ebf0 0x7fdc481f7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:52.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.109+0000 7fdc4ca00640 1 -- 172.21.15.17:0/4194638623 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fdc4813ebf0 msgr2=0x7fdc481f7520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:52.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.109+0000 7fdc4ca00640 1 --2- 172.21.15.17:0/4194638623 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fdc4813ebf0 0x7fdc481f7520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:52.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.109+0000 7fdc4ca00640 1 -- 172.21.15.17:0/4194638623 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdc300053d0 msgr2=0x7fdc48072490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:52.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.109+0000 7fdc4ca00640 1 --2- 172.21.15.17:0/4194638623 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdc300053d0 0x7fdc48072490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:52.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.109+0000 7fdc4ca00640 1 -- 172.21.15.17:0/4194638623 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc481f3b30 con 0x7fdc48156d90 2024-09-10T21:32:52.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.109+0000 7fdc4ca00640 1 --2- 172.21.15.17:0/4194638623 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdc48156d90 0x7fdc481f6fe0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fdc34003e10 tx=0x7fdc34001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:52.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.109+0000 7fdc45600640 1 -- 172.21.15.17:0/4194638623 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc34016030 con 0x7fdc48156d90 2024-09-10T21:32:52.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.109+0000 7fdc45600640 1 -- 172.21.15.17:0/4194638623 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdc34004e50 con 0x7fdc48156d90 2024-09-10T21:32:52.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.109+0000 7fdc45600640 1 -- 172.21.15.17:0/4194638623 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc34011940 con 0x7fdc48156d90 2024-09-10T21:32:52.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.109+0000 7fdc4e800640 1 -- 172.21.15.17:0/4194638623 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc481f3e10 con 0x7fdc48156d90 2024-09-10T21:32:52.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.109+0000 7fdc4e800640 1 -- 172.21.15.17:0/4194638623 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc481f43d0 con 0x7fdc48156d90 2024-09-10T21:32:52.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.113+0000 7fdc4e800640 1 --2- 172.21.15.17:0/4194638623 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdc4813d720 0x7fdc4813db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:52.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.113+0000 7fdc4e800640 1 -- 172.21.15.17:0/4194638623 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fdc481f45a0 con 0x7fdc4813d720 2024-09-10T21:32:52.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.113+0000 7fdc4d400640 1 --2- 172.21.15.17:0/4194638623 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdc4813d720 0x7fdc4813db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:52.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.113+0000 7fdc4d400640 1 -- 172.21.15.17:0/4194638623 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fdc48157900 con 0x7fdc4813d720 2024-09-10T21:32:52.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.113+0000 7fdc4d400640 1 --2- 172.21.15.17:0/4194638623 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdc4813d720 0x7fdc4813db00 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fdc3c012a20 tx=0x7fdc3c012ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:52.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.113+0000 7fdc45600640 1 -- 172.21.15.17:0/4194638623 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc3c00fa00 con 0x7fdc4813d720 2024-09-10T21:32:52.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.113+0000 7fdc45600640 1 -- 172.21.15.17:0/4194638623 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fdc3c00fba0 con 0x7fdc4813d720 2024-09-10T21:32:52.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.113+0000 7fdc45600640 1 -- 172.21.15.17:0/4194638623 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdc4813d720 msgr2=0x7fdc4813db00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:52.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.113+0000 7fdc45600640 1 --2- 172.21.15.17:0/4194638623 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdc4813d720 0x7fdc4813db00 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fdc3c012a20 tx=0x7fdc3c012ee0 comp rx=0 tx=0).stop 2024-09-10T21:32:52.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.113+0000 7fdc45600640 1 -- 172.21.15.17:0/4194638623 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdc4813d720 msgr2=0x7fdc4813db00 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:32:52.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.113+0000 7fdc45600640 1 --2- 172.21.15.17:0/4194638623 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdc4813d720 0x7fdc4813db00 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:52.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.113+0000 7fdc45600640 1 -- 172.21.15.17:0/4194638623 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fdc34023060 con 0x7fdc48156d90 2024-09-10T21:32:52.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.113+0000 7fdc45600640 1 --2- 172.21.15.17:0/4194638623 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fdc28071fc0 0x7fdc28074480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:52.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.113+0000 7fdc45600640 1 -- 172.21.15.17:0/4194638623 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fdc34093c40 con 0x7fdc48156d90 2024-09-10T21:32:52.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.113+0000 7fdc4d400640 1 --2- 172.21.15.17:0/4194638623 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fdc28071fc0 0x7fdc28074480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:52.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.117+0000 7fdc4d400640 1 --2- 172.21.15.17:0/4194638623 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fdc28071fc0 0x7fdc28074480 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fdc3c000c10 tx=0x7fdc3c012000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:52.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.121+0000 7fdc4e800640 1 --2- 172.21.15.17:0/4194638623 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdc4802d450 0x7fdc4802d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:52.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.121+0000 7fdc4e800640 1 -- 172.21.15.17:0/4194638623 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fdc4806fd40 con 0x7fdc4802d450 2024-09-10T21:32:52.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.121+0000 7fdc4de00640 1 --2- 172.21.15.17:0/4194638623 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdc4802d450 0x7fdc4802d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:52.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.121+0000 7fdc4de00640 1 -- 172.21.15.17:0/4194638623 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fdc48158940 con 0x7fdc4802d450 2024-09-10T21:32:52.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.121+0000 7fdc4de00640 1 --2- 172.21.15.17:0/4194638623 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdc4802d450 0x7fdc4802d830 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fdc480753b0 tx=0x7fdc40006d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:52.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.121+0000 7fdc45600640 1 -- 172.21.15.17:0/4194638623 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc400017c0 con 0x7fdc4802d450 2024-09-10T21:32:52.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.121+0000 7fdc45600640 1 -- 172.21.15.17:0/4194638623 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4709 (secure 0 0 0) 0x7fdc400017c0 con 0x7fdc4802d450 2024-09-10T21:32:52.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.121+0000 7fdc45600640 1 -- 172.21.15.17:0/4194638623 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdc4802d450 msgr2=0x7fdc4802d830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:52.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.121+0000 7fdc45600640 1 --2- 172.21.15.17:0/4194638623 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdc4802d450 0x7fdc4802d830 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fdc480753b0 tx=0x7fdc40006d70 comp rx=0 tx=0).stop 2024-09-10T21:32:52.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.121+0000 7fdc45600640 1 -- 172.21.15.17:0/4194638623 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdc4802d450 msgr2=0x7fdc4802d830 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:32:52.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.121+0000 7fdc45600640 1 --2- 172.21.15.17:0/4194638623 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdc4802d450 0x7fdc4802d830 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:52.130 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-10T21:32:52.130 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:32:52.131 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:32:52.131 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "leader", 2024-09-10T21:32:52.131 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 70, 2024-09-10T21:32:52.131 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 29007, 2024-09-10T21:32:52.131 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-10T21:32:52.132 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-10T21:32:52.132 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-10T21:32:52.132 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-10T21:32:52.132 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:32:52.132 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 3, 2024-09-10T21:32:52.133 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:32:52.133 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-10T21:32:52.133 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-10T21:32:52.133 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:32:52.133 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:32:52.134 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:32:52.134 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:32:52.134 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:32:52.134 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:32:52.134 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:32:52.135 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:32:52.135 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:32:52.135 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:32:52.135 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:32:52.135 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:32:52.135 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-10T21:32:52.136 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-10T21:32:52.136 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:32:52.136 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:32:52.136 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:32:52.136 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:32:52.136 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:32:52.137 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:32:52.137 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:32:52.137 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:32:52.137 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:32:52.137 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:32:52.137 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:32:52.138 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:32:52.138 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:32:52.138 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-10T21:32:52.138 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-10T21:32:52.138 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-10T21:32:52.138 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-10T21:32:52.139 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-10T21:32:52.139 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "5e2f023f-d6a6-49f4-a256-f4807dce3739", 2024-09-10T21:32:52.139 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:32:52.139 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:32:52.139 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-10T21:32:52.140 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-10T21:32:52.140 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-10T21:32:52.140 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-10T21:32:52.140 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-10T21:32:52.140 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-10T21:32:52.140 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-10T21:32:52.141 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:32:52.141 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-10T21:32:52.141 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:32:52.141 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:32:52.141 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:32:52.142 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:32:52.142 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:32:52.142 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:32:52.142 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:32:52.142 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:32:52.142 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:32:52.143 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:32:52.143 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:32:52.143 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:32:52.143 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-10T21:32:52.144 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:32:52.144 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-10T21:32:52.144 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:32:52.145 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:32:52.145 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:32:52.145 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:32:52.145 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:32:52.145 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:32:52.146 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:32:52.146 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-10T21:32:52.146 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:32:52.146 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:32:52.146 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:32:52.146 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:32:52.146 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-10T21:32:52.147 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:32:52.147 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:32:52.147 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:32:52.147 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:32:52.147 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-10T21:32:52.147 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-10T21:32:52.148 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:32:52.148 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:32:52.148 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:32:52.148 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:32:52.149 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:32:52.149 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:32:52.149 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:32:52.149 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:32:52.150 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:32:52.150 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:32:52.150 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:32:52.150 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3300", 2024-09-10T21:32:52.150 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:32:52.151 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:32:52.151 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:32:52.151 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:32:52.151 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789", 2024-09-10T21:32:52.151 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:32:52.151 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:32:52.152 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:32:52.152 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:32:52.152 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789/0", 2024-09-10T21:32:52.152 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6789/0", 2024-09-10T21:32:52.152 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:32:52.152 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:32:52.153 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:32:52.153 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:32:52.153 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:32:52.153 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:32:52.153 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:32:52.153 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:32:52.153 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:32:52.153 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:32:52.154 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:32:52.154 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3301", 2024-09-10T21:32:52.154 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:32:52.154 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:32:52.154 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:32:52.154 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:32:52.154 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790", 2024-09-10T21:32:52.154 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:32:52.154 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:32:52.154 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:32:52.154 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:32:52.154 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790/0", 2024-09-10T21:32:52.154 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6790/0", 2024-09-10T21:32:52.155 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:32:52.155 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:32:52.155 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:32:52.155 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:32:52.155 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:32:52.155 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:32:52.155 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-10T21:32:52.155 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-10T21:32:52.155 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:32:52.155 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:32:52.155 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:32:52.155 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:32:52.155 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:32:52.156 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:32:52.156 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": [ 2024-09-10T21:32:52.156 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:32:52.156 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:32:52.156 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:32:52.156 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:32:52.156 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:32:52.156 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:32:52.156 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-10T21:32:52.156 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:32:52.156 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:32:52.157 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:32:52.157 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 2 2024-09-10T21:32:52.157 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:32:52.157 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:32:52.157 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:32:52.157 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-10T21:32:52.157 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-10T21:32:52.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.125+0000 7fdc4e800640 1 -- 172.21.15.17:0/4194638623 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fdc28071fc0 msgr2=0x7fdc28074480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:52.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.125+0000 7fdc4e800640 1 --2- 172.21.15.17:0/4194638623 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fdc28071fc0 0x7fdc28074480 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fdc3c000c10 tx=0x7fdc3c012000 comp rx=0 tx=0).stop 2024-09-10T21:32:52.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.125+0000 7fdc4e800640 1 -- 172.21.15.17:0/4194638623 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdc48156d90 msgr2=0x7fdc481f6fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:52.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.125+0000 7fdc4e800640 1 --2- 172.21.15.17:0/4194638623 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdc48156d90 0x7fdc481f6fe0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fdc34003e10 tx=0x7fdc34001640 comp rx=0 tx=0).stop 2024-09-10T21:32:52.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.125+0000 7fdc4de00640 1 -- 172.21.15.17:0/4194638623 reap_dead start 2024-09-10T21:32:52.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.125+0000 7fdc4de00640 1 -- 172.21.15.17:0/4194638623 reap_dead start 2024-09-10T21:32:52.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.125+0000 7fdc4e800640 1 -- 172.21.15.17:0/4194638623 shutdown_connections 2024-09-10T21:32:52.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.125+0000 7fdc4e800640 1 -- 172.21.15.17:0/4194638623 >> 172.21.15.17:0/4194638623 conn(0x7fdc48068990 msgr2=0x7fdc48070d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:32:52.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.125+0000 7fdc4e800640 1 -- 172.21.15.17:0/4194638623 shutdown_connections 2024-09-10T21:32:52.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.125+0000 7fdc4e800640 1 -- 172.21.15.17:0/4194638623 wait complete. 2024-09-10T21:32:52.160 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-10T21:32:52.160 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-09-10T21:32:52.160 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-10T21:32:52.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.317+0000 7fc20ae00640 1 -- 172.21.15.17:0/3266630478 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc2040c6180 msgr2=0x7fc2040c37e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:52.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.317+0000 7fc20ae00640 1 --2- 172.21.15.17:0/3266630478 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc2040c6180 0x7fc2040c37e0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fc1f8009840 tx=0x7fc1f801f010 comp rx=0 tx=0).stop 2024-09-10T21:32:52.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.317+0000 7fc20ae00640 1 -- 172.21.15.17:0/3266630478 shutdown_connections 2024-09-10T21:32:52.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.317+0000 7fc20ae00640 1 --2- 172.21.15.17:0/3266630478 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc20415c540 0x7fc204151e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:52.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.317+0000 7fc20ae00640 1 --2- 172.21.15.17:0/3266630478 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc2040c6180 0x7fc2040c37e0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:52.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.317+0000 7fc20ae00640 1 -- 172.21.15.17:0/3266630478 >> 172.21.15.17:0/3266630478 conn(0x7fc20406cc20 msgr2=0x7fc20406d030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:32:52.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.317+0000 7fc20ae00640 1 -- 172.21.15.17:0/3266630478 shutdown_connections 2024-09-10T21:32:52.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.317+0000 7fc20ae00640 1 -- 172.21.15.17:0/3266630478 wait complete. 2024-09-10T21:32:52.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.317+0000 7fc20ae00640 1 Processor -- start 2024-09-10T21:32:52.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.317+0000 7fc20ae00640 1 -- start start 2024-09-10T21:32:52.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.321+0000 7fc20ae00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc2040c6180 0x7fc2040caae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:52.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.321+0000 7fc20ae00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc2040c6560 0x7fc2041f1330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:52.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.321+0000 7fc20ae00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc20415c540 0x7fc2041f1870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:52.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.321+0000 7fc20ae00640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fc2041589e0 con 0x7fc20415c540 2024-09-10T21:32:52.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.321+0000 7fc20ae00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fc20415bd40 con 0x7fc2040c6180 2024-09-10T21:32:52.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.321+0000 7fc20ae00640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7fc204158b60 con 0x7fc2040c6560 2024-09-10T21:32:52.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.321+0000 7fc20a400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc20415c540 0x7fc2041f1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:52.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.321+0000 7fc20a400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc20415c540 0x7fc2041f1870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37810/0 (socket says 172.21.15.17:37810) 2024-09-10T21:32:52.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.321+0000 7fc20a400640 1 -- 172.21.15.17:0/617774080 learned_addr learned my addr 172.21.15.17:0/617774080 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:32:52.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.321+0000 7fc209a00640 1 --2- 172.21.15.17:0/617774080 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc2040c6180 0x7fc2040caae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:52.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.321+0000 7fc209000640 1 --2- 172.21.15.17:0/617774080 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc2040c6560 0x7fc2041f1330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:52.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.321+0000 7fc209a00640 1 -- 172.21.15.17:0/617774080 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc2040c6560 msgr2=0x7fc2041f1330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:52.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.321+0000 7fc209a00640 1 --2- 172.21.15.17:0/617774080 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc2040c6560 0x7fc2041f1330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:52.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.321+0000 7fc209a00640 1 -- 172.21.15.17:0/617774080 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc20415c540 msgr2=0x7fc2041f1870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:52.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.321+0000 7fc209a00640 1 --2- 172.21.15.17:0/617774080 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc20415c540 0x7fc2041f1870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:52.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.321+0000 7fc209a00640 1 -- 172.21.15.17:0/617774080 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc20414ef60 con 0x7fc2040c6180 2024-09-10T21:32:52.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.321+0000 7fc209a00640 1 --2- 172.21.15.17:0/617774080 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc2040c6180 0x7fc2040caae0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fc1f80af000 tx=0x7fc1f8005780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:52.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.321+0000 7fc201600640 1 -- 172.21.15.17:0/617774080 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1f80b5040 con 0x7fc2040c6180 2024-09-10T21:32:52.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.321+0000 7fc201600640 1 -- 172.21.15.17:0/617774080 <== mon.1 v2:172.21.15.31:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc1f8003600 con 0x7fc2040c6180 2024-09-10T21:32:52.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.321+0000 7fc201600640 1 -- 172.21.15.17:0/617774080 <== mon.1 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1f8003a10 con 0x7fc2040c6180 2024-09-10T21:32:52.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.321+0000 7fc20ae00640 1 -- 172.21.15.17:0/617774080 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2041e4d20 con 0x7fc2040c6180 2024-09-10T21:32:52.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.321+0000 7fc20ae00640 1 -- 172.21.15.17:0/617774080 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2040c7590 con 0x7fc2040c6180 2024-09-10T21:32:52.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.325+0000 7fc20ae00640 1 --2- 172.21.15.17:0/617774080 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc2041e3dc0 0x7fc2041e41a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:52.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.325+0000 7fc20ae00640 1 -- 172.21.15.17:0/617774080 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fc2041e5650 con 0x7fc2041e3dc0 2024-09-10T21:32:52.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.325+0000 7fc209000640 1 --2- 172.21.15.17:0/617774080 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc2041e3dc0 0x7fc2041e41a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:52.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.325+0000 7fc209000640 1 -- 172.21.15.17:0/617774080 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc2041e46e0 con 0x7fc2041e3dc0 2024-09-10T21:32:52.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.325+0000 7fc209000640 1 --2- 172.21.15.17:0/617774080 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc2041e3dc0 0x7fc2041e41a0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fc2041501d0 tx=0x7fc1f000fc00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:52.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.325+0000 7fc201600640 1 -- 172.21.15.17:0/617774080 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1f0018020 con 0x7fc2041e3dc0 2024-09-10T21:32:52.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.325+0000 7fc201600640 1 -- 172.21.15.17:0/617774080 <== mon.1 v2:172.21.15.31:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fc1f000d600 con 0x7fc2041e3dc0 2024-09-10T21:32:52.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.325+0000 7fc201600640 1 -- 172.21.15.17:0/617774080 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc2041e3dc0 msgr2=0x7fc2041e41a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:52.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.325+0000 7fc201600640 1 --2- 172.21.15.17:0/617774080 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc2041e3dc0 0x7fc2041e41a0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fc2041501d0 tx=0x7fc1f000fc00 comp rx=0 tx=0).stop 2024-09-10T21:32:52.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.325+0000 7fc201600640 1 -- 172.21.15.17:0/617774080 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc2041e3dc0 msgr2=0x7fc2041e41a0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:32:52.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.325+0000 7fc201600640 1 --2- 172.21.15.17:0/617774080 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc2041e3dc0 0x7fc2041e41a0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:52.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.325+0000 7fc201600640 1 -- 172.21.15.17:0/617774080 <== mon.1 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fc1f80b8070 con 0x7fc2040c6180 2024-09-10T21:32:52.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.329+0000 7fc201600640 1 --2- 172.21.15.17:0/617774080 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fc1e4072170 0x7fc1e4074630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:52.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.329+0000 7fc201600640 1 -- 172.21.15.17:0/617774080 <== mon.1 v2:172.21.15.31:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fc1f8130150 con 0x7fc2040c6180 2024-09-10T21:32:52.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.329+0000 7fc209000640 1 --2- 172.21.15.17:0/617774080 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fc1e4072170 0x7fc1e4074630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:52.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.329+0000 7fc209000640 1 --2- 172.21.15.17:0/617774080 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fc1e4072170 0x7fc1e4074630 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fc1f0019000 tx=0x7fc1f0009030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:52.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.333+0000 7fc20ae00640 1 --2- 172.21.15.17:0/617774080 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc20414e0e0 0x7fc20414e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:52.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.333+0000 7fc20ae00640 1 -- 172.21.15.17:0/617774080 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fc204068ab0 con 0x7fc20414e0e0 2024-09-10T21:32:52.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.337+0000 7fc20a400640 1 --2- 172.21.15.17:0/617774080 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc20414e0e0 0x7fc20414e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:52.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.337+0000 7fc20a400640 1 -- 172.21.15.17:0/617774080 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fc204151460 con 0x7fc20414e0e0 2024-09-10T21:32:52.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.337+0000 7fc20a400640 1 --2- 172.21.15.17:0/617774080 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc20414e0e0 0x7fc20414e4c0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fc1fc011d20 tx=0x7fc1fc00f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:52.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.337+0000 7fc201600640 1 -- 172.21.15.17:0/617774080 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1fc004d30 con 0x7fc20414e0e0 2024-09-10T21:32:52.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.337+0000 7fc201600640 1 -- 172.21.15.17:0/617774080 <== mon.1 v2:172.21.15.31:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5042 (secure 0 0 0) 0x7fc1fc004d30 con 0x7fc20414e0e0 2024-09-10T21:32:52.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.337+0000 7fc201600640 1 -- 172.21.15.17:0/617774080 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc20414e0e0 msgr2=0x7fc20414e4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:52.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.337+0000 7fc201600640 1 --2- 172.21.15.17:0/617774080 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc20414e0e0 0x7fc20414e4c0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fc1fc011d20 tx=0x7fc1fc00f790 comp rx=0 tx=0).stop 2024-09-10T21:32:52.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.337+0000 7fc201600640 1 -- 172.21.15.17:0/617774080 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc20414e0e0 msgr2=0x7fc20414e4c0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:32:52.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.337+0000 7fc201600640 1 --2- 172.21.15.17:0/617774080 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc20414e0e0 0x7fc20414e4c0 secure :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fc1fc011d20 tx=0x7fc1fc00f790 comp rx=0 tx=0).stop 2024-09-10T21:32:52.343 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-10T21:32:52.343 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:32:52.343 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:32:52.343 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-10T21:32:52.343 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 70, 2024-09-10T21:32:52.343 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 510788, 2024-09-10T21:32:52.343 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-10T21:32:52.343 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-10T21:32:52.343 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-10T21:32:52.343 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-10T21:32:52.343 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:32:52.343 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 3, 2024-09-10T21:32:52.343 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:32:52.344 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-10T21:32:52.344 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-10T21:32:52.344 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:32:52.344 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:32:52.344 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:32:52.344 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:32:52.344 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:32:52.344 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:32:52.344 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:32:52.344 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:32:52.344 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:32:52.344 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:32:52.345 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:32:52.345 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:32:52.345 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-10T21:32:52.345 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-10T21:32:52.345 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:32:52.345 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:32:52.345 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:32:52.345 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:32:52.346 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:32:52.346 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:32:52.346 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:32:52.346 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:32:52.346 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:32:52.346 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:32:52.346 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:32:52.346 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:32:52.346 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:32:52.346 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-10T21:32:52.347 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-10T21:32:52.347 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-10T21:32:52.347 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-10T21:32:52.347 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-10T21:32:52.347 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "5e2f023f-d6a6-49f4-a256-f4807dce3739", 2024-09-10T21:32:52.347 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:32:52.347 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:32:52.347 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-10T21:32:52.347 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-10T21:32:52.347 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-10T21:32:52.347 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-10T21:32:52.348 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-10T21:32:52.348 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-10T21:32:52.348 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-10T21:32:52.348 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:32:52.348 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-10T21:32:52.348 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:32:52.348 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:32:52.348 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:32:52.348 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:32:52.348 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:32:52.348 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:32:52.348 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:32:52.349 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:32:52.349 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:32:52.349 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:32:52.349 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:32:52.349 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:32:52.349 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-10T21:32:52.349 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:32:52.349 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-10T21:32:52.349 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:32:52.349 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:32:52.349 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:32:52.349 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:32:52.350 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:32:52.350 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:32:52.350 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:32:52.350 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-10T21:32:52.350 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:32:52.350 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:32:52.350 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:32:52.350 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:32:52.350 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-10T21:32:52.350 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:32:52.351 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:32:52.351 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:32:52.351 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:32:52.351 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-10T21:32:52.351 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-10T21:32:52.351 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:32:52.351 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:32:52.351 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:32:52.351 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:32:52.351 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:32:52.351 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:32:52.351 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:32:52.351 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:32:52.352 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:32:52.352 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:32:52.352 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:32:52.352 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3300", 2024-09-10T21:32:52.352 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:32:52.352 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:32:52.352 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:32:52.352 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:32:52.352 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789", 2024-09-10T21:32:52.352 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:32:52.352 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:32:52.352 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:32:52.352 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:32:52.353 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789/0", 2024-09-10T21:32:52.353 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6789/0", 2024-09-10T21:32:52.353 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:32:52.353 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:32:52.353 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:32:52.353 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:32:52.353 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:32:52.353 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:32:52.353 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:32:52.353 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:32:52.353 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:32:52.353 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:32:52.353 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:32:52.354 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3301", 2024-09-10T21:32:52.354 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:32:52.354 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:32:52.354 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:32:52.354 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:32:52.354 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790", 2024-09-10T21:32:52.354 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:32:52.354 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:32:52.354 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:32:52.354 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:32:52.354 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790/0", 2024-09-10T21:32:52.354 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6790/0", 2024-09-10T21:32:52.355 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:32:52.355 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:32:52.355 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:32:52.355 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:32:52.355 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:32:52.355 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:32:52.355 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-10T21:32:52.355 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-10T21:32:52.355 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:32:52.355 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:32:52.355 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:32:52.356 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:32:52.356 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:32:52.356 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:32:52.356 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": [ 2024-09-10T21:32:52.356 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:32:52.356 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:32:52.356 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:32:52.356 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 5 2024-09-10T21:32:52.356 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:32:52.356 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:32:52.356 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-10T21:32:52.356 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:32:52.357 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:32:52.357 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:32:52.357 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 7 2024-09-10T21:32:52.357 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:32:52.357 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:32:52.357 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-10T21:32:52.357 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:32:52.357 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:32:52.357 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:32:52.357 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 7 2024-09-10T21:32:52.357 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:32:52.357 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:32:52.357 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": [ 2024-09-10T21:32:52.358 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:32:52.358 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:32:52.358 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:32:52.358 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 2 2024-09-10T21:32:52.358 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:32:52.358 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:32:52.358 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:32:52.358 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-10T21:32:52.358 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-10T21:32:52.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.337+0000 7fc20ae00640 1 -- 172.21.15.17:0/617774080 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fc1e4072170 msgr2=0x7fc1e4074630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:52.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.337+0000 7fc20ae00640 1 --2- 172.21.15.17:0/617774080 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fc1e4072170 0x7fc1e4074630 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fc1f0019000 tx=0x7fc1f0009030 comp rx=0 tx=0).stop 2024-09-10T21:32:52.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.337+0000 7fc20ae00640 1 -- 172.21.15.17:0/617774080 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc2040c6180 msgr2=0x7fc2040caae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:52.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.337+0000 7fc20ae00640 1 --2- 172.21.15.17:0/617774080 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc2040c6180 0x7fc2040caae0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fc1f80af000 tx=0x7fc1f8005780 comp rx=0 tx=0).stop 2024-09-10T21:32:52.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.337+0000 7fc20a400640 1 -- 172.21.15.17:0/617774080 reap_dead start 2024-09-10T21:32:52.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.337+0000 7fc20a400640 1 -- 172.21.15.17:0/617774080 reap_dead start 2024-09-10T21:32:52.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.337+0000 7fc20ae00640 1 -- 172.21.15.17:0/617774080 shutdown_connections 2024-09-10T21:32:52.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.337+0000 7fc20ae00640 1 -- 172.21.15.17:0/617774080 >> 172.21.15.17:0/617774080 conn(0x7fc20406cc20 msgr2=0x7fc20406d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:32:52.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.337+0000 7fc20ae00640 1 -- 172.21.15.17:0/617774080 shutdown_connections 2024-09-10T21:32:52.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.337+0000 7fc20ae00640 1 -- 172.21.15.17:0/617774080 wait complete. 2024-09-10T21:32:52.365 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-10T21:32:52.365 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-09-10T21:32:52.365 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-10T21:32:52.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.525+0000 7ff137600640 1 -- 172.21.15.17:0/643288113 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff130152440 msgr2=0x7ff1300b3eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:52.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.525+0000 7ff137600640 1 --2- 172.21.15.17:0/643288113 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff130152440 0x7ff1300b3eb0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7ff124009900 tx=0x7ff12401c850 comp rx=0 tx=0).stop 2024-09-10T21:32:52.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.525+0000 7ff137600640 1 -- 172.21.15.17:0/643288113 shutdown_connections 2024-09-10T21:32:52.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.525+0000 7ff137600640 1 --2- 172.21.15.17:0/643288113 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff130152820 0x7ff13015c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:52.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.525+0000 7ff137600640 1 --2- 172.21.15.17:0/643288113 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff130152440 0x7ff1300b3eb0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:52.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.525+0000 7ff137600640 1 -- 172.21.15.17:0/643288113 >> 172.21.15.17:0/643288113 conn(0x7ff1300702f0 msgr2=0x7ff130070700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:32:52.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.525+0000 7ff137600640 1 -- 172.21.15.17:0/643288113 shutdown_connections 2024-09-10T21:32:52.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.525+0000 7ff137600640 1 -- 172.21.15.17:0/643288113 wait complete. 2024-09-10T21:32:52.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.525+0000 7ff137600640 1 Processor -- start 2024-09-10T21:32:52.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.525+0000 7ff137600640 1 -- start start 2024-09-10T21:32:52.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.529+0000 7ff137600640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff1300cdf20 0x7ff1301575b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:52.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.529+0000 7ff137600640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff130152440 0x7ff130159520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:52.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.529+0000 7ff137600640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff130152820 0x7ff1300c7300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:52.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.529+0000 7ff137600640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7ff13015c7c0 con 0x7ff130152440 2024-09-10T21:32:52.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.529+0000 7ff137600640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7ff1300ce740 con 0x7ff1300cdf20 2024-09-10T21:32:52.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.529+0000 7ff137600640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7ff1300cb450 con 0x7ff130152820 2024-09-10T21:32:52.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.529+0000 7ff12fe00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff130152440 0x7ff130159520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:52.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.529+0000 7ff12fe00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff130152440 0x7ff130159520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37840/0 (socket says 172.21.15.17:37840) 2024-09-10T21:32:52.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.529+0000 7ff12fe00640 1 -- 172.21.15.17:0/711991175 learned_addr learned my addr 172.21.15.17:0/711991175 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:32:52.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.529+0000 7ff134c00640 1 --2- 172.21.15.17:0/711991175 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff1300cdf20 0x7ff1301575b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:52.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.529+0000 7ff135600640 1 --2- 172.21.15.17:0/711991175 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff130152820 0x7ff1300c7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:52.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.529+0000 7ff12fe00640 1 -- 172.21.15.17:0/711991175 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff130152820 msgr2=0x7ff1300c7300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:52.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.529+0000 7ff12fe00640 1 --2- 172.21.15.17:0/711991175 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff130152820 0x7ff1300c7300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:52.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.529+0000 7ff12fe00640 1 -- 172.21.15.17:0/711991175 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff1300cdf20 msgr2=0x7ff1301575b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:52.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.529+0000 7ff12fe00640 1 --2- 172.21.15.17:0/711991175 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff1300cdf20 0x7ff1301575b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:52.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.529+0000 7ff12fe00640 1 -- 172.21.15.17:0/711991175 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff130157f90 con 0x7ff130152440 2024-09-10T21:32:52.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.529+0000 7ff12fe00640 1 --2- 172.21.15.17:0/711991175 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff130152440 0x7ff130159520 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7ff11c00ea10 tx=0x7ff11c00eed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:52.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.529+0000 7ff12cc00640 1 -- 172.21.15.17:0/711991175 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff11c00cde0 con 0x7ff130152440 2024-09-10T21:32:52.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.529+0000 7ff12cc00640 1 -- 172.21.15.17:0/711991175 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff11c004020 con 0x7ff130152440 2024-09-10T21:32:52.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.529+0000 7ff12cc00640 1 -- 172.21.15.17:0/711991175 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff11c004430 con 0x7ff130152440 2024-09-10T21:32:52.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.529+0000 7ff137600640 1 -- 172.21.15.17:0/711991175 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff130158260 con 0x7ff130152440 2024-09-10T21:32:52.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.529+0000 7ff137600640 1 -- 172.21.15.17:0/711991175 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1300cc210 con 0x7ff130152440 2024-09-10T21:32:52.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.529+0000 7ff137600640 1 --2- 172.21.15.17:0/711991175 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff1300cc3e0 0x7ff1301f6bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:52.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.529+0000 7ff137600640 1 -- 172.21.15.17:0/711991175 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7ff130162df0 con 0x7ff1300cc3e0 2024-09-10T21:32:52.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.529+0000 7ff134c00640 1 --2- 172.21.15.17:0/711991175 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff1300cc3e0 0x7ff1301f6bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:52.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.533+0000 7ff134c00640 1 -- 172.21.15.17:0/711991175 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff1301f69b0 con 0x7ff1300cc3e0 2024-09-10T21:32:52.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.533+0000 7ff134c00640 1 --2- 172.21.15.17:0/711991175 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff1300cc3e0 0x7ff1301f6bf0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7ff1300cbe60 tx=0x7ff124009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:52.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.533+0000 7ff12cc00640 1 -- 172.21.15.17:0/711991175 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7ff11c003600 con 0x7ff130152440 2024-09-10T21:32:52.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.533+0000 7ff12cc00640 1 --2- 172.21.15.17:0/711991175 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7ff110072610 0x7ff110074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:52.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.533+0000 7ff12cc00640 1 -- 172.21.15.17:0/711991175 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7ff11c093e50 con 0x7ff130152440 2024-09-10T21:32:52.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.533+0000 7ff135600640 1 --2- 172.21.15.17:0/711991175 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7ff110072610 0x7ff110074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:52.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.533+0000 7ff12cc00640 1 -- 172.21.15.17:0/711991175 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1240b2040 con 0x7ff1300cc3e0 2024-09-10T21:32:52.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.533+0000 7ff12cc00640 1 -- 172.21.15.17:0/711991175 <== mon.2 v2:172.21.15.31:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7ff124005220 con 0x7ff1300cc3e0 2024-09-10T21:32:52.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.533+0000 7ff12cc00640 1 -- 172.21.15.17:0/711991175 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff1300cc3e0 msgr2=0x7ff1301f6bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:52.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.533+0000 7ff12cc00640 1 --2- 172.21.15.17:0/711991175 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff1300cc3e0 0x7ff1301f6bf0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7ff1300cbe60 tx=0x7ff124009580 comp rx=0 tx=0).stop 2024-09-10T21:32:52.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.533+0000 7ff12cc00640 1 -- 172.21.15.17:0/711991175 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff1300cc3e0 msgr2=0x7ff1301f6bf0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:32:52.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.533+0000 7ff12cc00640 1 --2- 172.21.15.17:0/711991175 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff1300cc3e0 0x7ff1301f6bf0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:52.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.533+0000 7ff135600640 1 --2- 172.21.15.17:0/711991175 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7ff110072610 0x7ff110074ad0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7ff1301574a0 tx=0x7ff128009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:52.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.541+0000 7ff137600640 1 --2- 172.21.15.17:0/711991175 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff1301f7130 0x7ff13014e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:52.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.541+0000 7ff137600640 1 -- 172.21.15.17:0/711991175 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7ff13006ec00 con 0x7ff1301f7130 2024-09-10T21:32:52.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.541+0000 7ff134c00640 1 --2- 172.21.15.17:0/711991175 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff1301f7130 0x7ff13014e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:52.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.541+0000 7ff134c00640 1 -- 172.21.15.17:0/711991175 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7ff13006e920 con 0x7ff1301f7130 2024-09-10T21:32:52.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.541+0000 7ff134c00640 1 --2- 172.21.15.17:0/711991175 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff1301f7130 0x7ff13014e8c0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7ff12401af00 tx=0x7ff124009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:52.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.541+0000 7ff12cc00640 1 -- 172.21.15.17:0/711991175 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1240b2040 con 0x7ff1301f7130 2024-09-10T21:32:52.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.541+0000 7ff12cc00640 1 -- 172.21.15.17:0/711991175 <== mon.2 v2:172.21.15.31:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4708 (secure 0 0 0) 0x7ff1240bd8b0 con 0x7ff1301f7130 2024-09-10T21:32:52.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.541+0000 7ff12cc00640 1 -- 172.21.15.17:0/711991175 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff1301f7130 msgr2=0x7ff13014e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:52.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.541+0000 7ff12cc00640 1 --2- 172.21.15.17:0/711991175 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff1301f7130 0x7ff13014e8c0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7ff12401af00 tx=0x7ff124009260 comp rx=0 tx=0).stop 2024-09-10T21:32:52.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.541+0000 7ff12cc00640 1 -- 172.21.15.17:0/711991175 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff1301f7130 msgr2=0x7ff13014e8c0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:32:52.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.541+0000 7ff12cc00640 1 --2- 172.21.15.17:0/711991175 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff1301f7130 0x7ff13014e8c0 secure :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7ff12401af00 tx=0x7ff124009260 comp rx=0 tx=0).stop 2024-09-10T21:32:52.548 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-10T21:32:52.549 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:32:52.549 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:32:52.549 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-10T21:32:52.549 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 70, 2024-09-10T21:32:52.549 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 104850, 2024-09-10T21:32:52.549 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-10T21:32:52.549 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-10T21:32:52.549 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-10T21:32:52.549 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-10T21:32:52.549 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:32:52.549 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 3, 2024-09-10T21:32:52.549 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:32:52.550 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-10T21:32:52.550 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-10T21:32:52.550 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:32:52.550 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:32:52.550 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:32:52.550 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:32:52.550 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:32:52.550 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:32:52.550 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:32:52.550 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:32:52.550 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:32:52.550 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:32:52.551 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:32:52.551 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:32:52.551 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-10T21:32:52.551 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-10T21:32:52.551 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:32:52.551 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:32:52.551 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:32:52.551 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:32:52.551 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:32:52.551 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:32:52.551 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:32:52.551 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:32:52.552 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:32:52.552 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:32:52.552 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:32:52.552 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:32:52.552 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:32:52.552 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-10T21:32:52.552 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-10T21:32:52.552 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-10T21:32:52.552 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-10T21:32:52.552 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-10T21:32:52.552 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "5e2f023f-d6a6-49f4-a256-f4807dce3739", 2024-09-10T21:32:52.552 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:32:52.553 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:32:52.553 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-10T21:32:52.553 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-10T21:32:52.553 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-10T21:32:52.553 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-10T21:32:52.553 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-10T21:32:52.553 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-10T21:32:52.553 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-10T21:32:52.553 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:32:52.553 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-10T21:32:52.553 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:32:52.553 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:32:52.553 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:32:52.553 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:32:52.554 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:32:52.554 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:32:52.554 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:32:52.554 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:32:52.554 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:32:52.554 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:32:52.554 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:32:52.554 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:32:52.554 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-10T21:32:52.554 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:32:52.554 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-10T21:32:52.554 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:32:52.555 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:32:52.555 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:32:52.555 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:32:52.555 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:32:52.555 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:32:52.555 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:32:52.555 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-10T21:32:52.555 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:32:52.555 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:32:52.555 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:32:52.555 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:32:52.555 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-10T21:32:52.555 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:32:52.556 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:32:52.556 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:32:52.556 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:32:52.556 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-10T21:32:52.556 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-10T21:32:52.556 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:32:52.556 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:32:52.556 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:32:52.556 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:32:52.556 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:32:52.556 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:32:52.556 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:32:52.557 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:32:52.557 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:32:52.557 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:32:52.557 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:32:52.557 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3300", 2024-09-10T21:32:52.557 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:32:52.557 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:32:52.557 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:32:52.557 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:32:52.557 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789", 2024-09-10T21:32:52.557 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:32:52.557 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:32:52.557 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:32:52.557 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:32:52.558 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789/0", 2024-09-10T21:32:52.558 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6789/0", 2024-09-10T21:32:52.558 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:32:52.558 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:32:52.558 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:32:52.558 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:32:52.558 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:32:52.558 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:32:52.558 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:32:52.558 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:32:52.558 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:32:52.558 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:32:52.558 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:32:52.559 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3301", 2024-09-10T21:32:52.559 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:32:52.559 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:32:52.559 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:32:52.559 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:32:52.559 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790", 2024-09-10T21:32:52.559 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:32:52.559 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:32:52.559 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:32:52.559 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:32:52.559 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790/0", 2024-09-10T21:32:52.559 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6790/0", 2024-09-10T21:32:52.559 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:32:52.560 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:32:52.560 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:32:52.560 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:32:52.560 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:32:52.560 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:32:52.560 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-10T21:32:52.560 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-10T21:32:52.560 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:32:52.560 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:32:52.560 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:32:52.560 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:32:52.560 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:32:52.560 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:32:52.561 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-10T21:32:52.561 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:32:52.561 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:32:52.561 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:32:52.561 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:32:52.561 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:32:52.561 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:32:52.561 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-10T21:32:52.561 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:32:52.561 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:32:52.561 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:32:52.561 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:32:52.561 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:32:52.561 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:32:52.562 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:32:52.562 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-10T21:32:52.562 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-10T21:32:52.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.545+0000 7ff137600640 1 -- 172.21.15.17:0/711991175 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7ff110072610 msgr2=0x7ff110074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:52.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.545+0000 7ff137600640 1 --2- 172.21.15.17:0/711991175 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7ff110072610 0x7ff110074ad0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7ff1301574a0 tx=0x7ff128009040 comp rx=0 tx=0).stop 2024-09-10T21:32:52.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.545+0000 7ff137600640 1 -- 172.21.15.17:0/711991175 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff130152440 msgr2=0x7ff130159520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:52.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.545+0000 7ff137600640 1 --2- 172.21.15.17:0/711991175 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff130152440 0x7ff130159520 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7ff11c00ea10 tx=0x7ff11c00eed0 comp rx=0 tx=0).stop 2024-09-10T21:32:52.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.545+0000 7ff135600640 1 -- 172.21.15.17:0/711991175 reap_dead start 2024-09-10T21:32:52.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.545+0000 7ff137600640 1 -- 172.21.15.17:0/711991175 shutdown_connections 2024-09-10T21:32:52.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.545+0000 7ff137600640 1 --2- 172.21.15.17:0/711991175 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff130152440 0x7ff130159520 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:52.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.545+0000 7ff137600640 1 -- 172.21.15.17:0/711991175 >> 172.21.15.17:0/711991175 conn(0x7ff1300702f0 msgr2=0x7ff130151c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:32:52.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.545+0000 7ff137600640 1 -- 172.21.15.17:0/711991175 shutdown_connections 2024-09-10T21:32:52.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.545+0000 7ff137600640 1 -- 172.21.15.17:0/711991175 wait complete. 2024-09-10T21:32:52.568 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-10T21:32:52.568 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-09-10T21:32:52.568 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-09-10T21:32:52.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.737+0000 7f6c4a800640 1 -- 172.21.15.17:0/3233700300 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6c440c9990 msgr2=0x7f6c44162df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:52.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.737+0000 7f6c4a800640 1 --2- 172.21.15.17:0/3233700300 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6c440c9990 0x7f6c44162df0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c009bf0 tx=0x7f6c3c01cbe0 comp rx=0 tx=0).stop 2024-09-10T21:32:52.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.737+0000 7f6c4a800640 1 -- 172.21.15.17:0/3233700300 shutdown_connections 2024-09-10T21:32:52.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.737+0000 7f6c4a800640 1 --2- 172.21.15.17:0/3233700300 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6c440c9990 0x7f6c44162df0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:52.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.737+0000 7f6c4a800640 1 -- 172.21.15.17:0/3233700300 >> 172.21.15.17:0/3233700300 conn(0x7f6c4406ce20 msgr2=0x7f6c4406d230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:32:52.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.737+0000 7f6c4a800640 1 -- 172.21.15.17:0/3233700300 shutdown_connections 2024-09-10T21:32:52.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.737+0000 7f6c4a800640 1 -- 172.21.15.17:0/3233700300 wait complete. 2024-09-10T21:32:52.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.741+0000 7f6c4a800640 1 Processor -- start 2024-09-10T21:32:52.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.741+0000 7f6c4a800640 1 -- start start 2024-09-10T21:32:52.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.741+0000 7f6c4a800640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6c440c3aa0 0x7f6c441385c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:52.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.741+0000 7f6c4a800640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c440c95b0 0x7f6c44138b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:52.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.741+0000 7f6c4a800640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6c440c9990 0x7f6c4414fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:52.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.741+0000 7f6c4a800640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f6c440c8420 con 0x7f6c440c9990 2024-09-10T21:32:52.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.741+0000 7f6c4a800640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f6c44072d00 con 0x7f6c440c95b0 2024-09-10T21:32:52.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.741+0000 7f6c4a800640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f6c4414ed00 con 0x7f6c440c3aa0 2024-09-10T21:32:52.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.741+0000 7f6c43e00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6c440c9990 0x7f6c4414fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:52.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.741+0000 7f6c42a00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c440c95b0 0x7f6c44138b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:52.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.741+0000 7f6c43e00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6c440c9990 0x7f6c4414fb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37856/0 (socket says 172.21.15.17:37856) 2024-09-10T21:32:52.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.741+0000 7f6c43e00640 1 -- 172.21.15.17:0/88592972 learned_addr learned my addr 172.21.15.17:0/88592972 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:32:52.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.741+0000 7f6c43400640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6c440c3aa0 0x7f6c441385c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:52.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.741+0000 7f6c43e00640 1 -- 172.21.15.17:0/88592972 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6c440c3aa0 msgr2=0x7f6c441385c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:52.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.741+0000 7f6c43e00640 1 --2- 172.21.15.17:0/88592972 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6c440c3aa0 0x7f6c441385c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:52.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.741+0000 7f6c43e00640 1 -- 172.21.15.17:0/88592972 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c440c95b0 msgr2=0x7f6c44138b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:52.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.741+0000 7f6c43e00640 1 --2- 172.21.15.17:0/88592972 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c440c95b0 0x7f6c44138b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:52.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.741+0000 7f6c43e00640 1 -- 172.21.15.17:0/88592972 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c440ca880 con 0x7f6c440c9990 2024-09-10T21:32:52.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.741+0000 7f6c42a00640 1 --2- 172.21.15.17:0/88592972 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c440c95b0 0x7f6c44138b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:32:52.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.741+0000 7f6c43e00640 1 --2- 172.21.15.17:0/88592972 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6c440c9990 0x7f6c4414fb20 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c0b0040 tx=0x7f6c3c005a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:52.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.741+0000 7f6c3b400640 1 -- 172.21.15.17:0/88592972 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c3c0b2040 con 0x7f6c440c9990 2024-09-10T21:32:52.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.741+0000 7f6c3b400640 1 -- 172.21.15.17:0/88592972 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6c3c0bb070 con 0x7f6c440c9990 2024-09-10T21:32:52.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.741+0000 7f6c3b400640 1 -- 172.21.15.17:0/88592972 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c3c0045f0 con 0x7f6c440c9990 2024-09-10T21:32:52.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.741+0000 7f6c4a800640 1 -- 172.21.15.17:0/88592972 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c44150aa0 con 0x7f6c440c9990 2024-09-10T21:32:52.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.741+0000 7f6c4a800640 1 -- 172.21.15.17:0/88592972 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c44137800 con 0x7f6c440c9990 2024-09-10T21:32:52.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.745+0000 7f6c4a800640 1 -- 172.21.15.17:0/88592972 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c44073cf0 con 0x7f6c440c9990 2024-09-10T21:32:52.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.745+0000 7f6c3b400640 1 -- 172.21.15.17:0/88592972 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f6c3c0b5070 con 0x7f6c440c9990 2024-09-10T21:32:52.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.745+0000 7f6c3b400640 1 --2- 172.21.15.17:0/88592972 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f6c240725e0 0x7f6c24074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:32:52.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.745+0000 7f6c3b400640 1 -- 172.21.15.17:0/88592972 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f6c3c12dc00 con 0x7f6c440c9990 2024-09-10T21:32:52.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.745+0000 7f6c43400640 1 --2- 172.21.15.17:0/88592972 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f6c240725e0 0x7f6c24074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:32:52.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.745+0000 7f6c43400640 1 --2- 172.21.15.17:0/88592972 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f6c240725e0 0x7f6c24074aa0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f6c34001910 tx=0x7f6c34001420 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:32:52.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:52.749+0000 7f6c3b400640 1 -- 172.21.15.17:0/88592972 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f6c3c0167d0 con 0x7f6c440c9990 2024-09-10T21:32:53.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:53.013+0000 7f6c4a800640 1 -- 172.21.15.17:0/88592972 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) -- 0x7f6c44154d80 con 0x7f6c440c9990 2024-09-10T21:32:53.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:53.013+0000 7f6c3b400640 1 -- 172.21.15.17:0/88592972 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) ==== 57+0+0 (secure 0 0 0) 0x7f6c3c0f87f0 con 0x7f6c440c9990 2024-09-10T21:32:53.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:53.017+0000 7f6c4a800640 1 -- 172.21.15.17:0/88592972 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f6c240725e0 msgr2=0x7f6c24074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:53.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:53.017+0000 7f6c4a800640 1 --2- 172.21.15.17:0/88592972 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f6c240725e0 0x7f6c24074aa0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f6c34001910 tx=0x7f6c34001420 comp rx=0 tx=0).stop 2024-09-10T21:32:53.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:53.017+0000 7f6c4a800640 1 -- 172.21.15.17:0/88592972 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6c440c9990 msgr2=0x7f6c4414fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:32:53.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:53.017+0000 7f6c4a800640 1 --2- 172.21.15.17:0/88592972 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6c440c9990 0x7f6c4414fb20 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c0b0040 tx=0x7f6c3c005a60 comp rx=0 tx=0).stop 2024-09-10T21:32:53.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:53.017+0000 7f6c4a800640 1 -- 172.21.15.17:0/88592972 shutdown_connections 2024-09-10T21:32:53.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:53.017+0000 7f6c4a800640 1 --2- 172.21.15.17:0/88592972 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f6c240725e0 0x7f6c24074aa0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:53.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:53.017+0000 7f6c4a800640 1 --2- 172.21.15.17:0/88592972 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6c440c9990 0x7f6c4414fb20 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:53.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:53.017+0000 7f6c4a800640 1 --2- 172.21.15.17:0/88592972 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c440c95b0 0x7f6c44138b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:53.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:53.017+0000 7f6c4a800640 1 --2- 172.21.15.17:0/88592972 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6c440c3aa0 0x7f6c441385c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:32:53.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:53.017+0000 7f6c4a800640 1 -- 172.21.15.17:0/88592972 >> 172.21.15.17:0/88592972 conn(0x7f6c4406ce20 msgr2=0x7f6c440c7540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:32:53.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:53.017+0000 7f6c4a800640 1 -- 172.21.15.17:0/88592972 shutdown_connections 2024-09-10T21:32:53.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:32:53.017+0000 7f6c4a800640 1 -- 172.21.15.17:0/88592972 wait complete. 2024-09-10T21:32:53.043 INFO:tasks.mon_thrash.mon_thrasher:waiting for 10.0 secs before continuing thrashing 2024-09-10T21:33:03.044 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-09-10T21:33:03.045 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-10T21:33:03.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.193+0000 7f6c7a400640 1 -- 172.21.15.17:0/723659492 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6c740c7030 msgr2=0x7f6c740c74e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:03.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.193+0000 7f6c7a400640 1 --2- 172.21.15.17:0/723659492 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6c740c7030 0x7f6c740c74e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f6c6c00acd0 tx=0x7f6c6c01dec0 comp rx=0 tx=0).stop 2024-09-10T21:33:03.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.193+0000 7f6c7a400640 1 -- 172.21.15.17:0/723659492 shutdown_connections 2024-09-10T21:33:03.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.193+0000 7f6c7a400640 1 --2- 172.21.15.17:0/723659492 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6c740c7030 0x7f6c740c74e0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:03.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.193+0000 7f6c7a400640 1 --2- 172.21.15.17:0/723659492 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6c74150fc0 0x7f6c74157500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:03.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.193+0000 7f6c7a400640 1 -- 172.21.15.17:0/723659492 >> 172.21.15.17:0/723659492 conn(0x7f6c74070160 msgr2=0x7f6c74070570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:03.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.193+0000 7f6c7a400640 1 -- 172.21.15.17:0/723659492 shutdown_connections 2024-09-10T21:33:03.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.193+0000 7f6c7a400640 1 -- 172.21.15.17:0/723659492 wait complete. 2024-09-10T21:33:03.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.193+0000 7f6c7a400640 1 Processor -- start 2024-09-10T21:33:03.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.193+0000 7f6c7a400640 1 -- start start 2024-09-10T21:33:03.201 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.193+0000 7f6c7a400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c740c7030 0x7f6c740c2750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:03.201 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.193+0000 7f6c7a400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6c740c8470 0x7f6c741f7290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:03.201 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.193+0000 7f6c7a400640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6c74150fc0 0x7f6c7413d320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:03.201 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.193+0000 7f6c7a400640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f6c740c7ed0 con 0x7f6c740c8470 2024-09-10T21:33:03.201 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.193+0000 7f6c7a400640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f6c7406e4b0 con 0x7f6c740c7030 2024-09-10T21:33:03.201 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.193+0000 7f6c7a400640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f6c740c8050 con 0x7f6c74150fc0 2024-09-10T21:33:03.201 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.193+0000 7f6c72a00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6c740c8470 0x7f6c741f7290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:03.201 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.193+0000 7f6c72a00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6c740c8470 0x7f6c741f7290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44012/0 (socket says 172.21.15.17:44012) 2024-09-10T21:33:03.201 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.193+0000 7f6c72a00640 1 -- 172.21.15.17:0/120475664 learned_addr learned my addr 172.21.15.17:0/120475664 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:33:03.201 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.193+0000 7f6c73400640 1 --2- 172.21.15.17:0/120475664 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c740c7030 0x7f6c740c2750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:03.201 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.193+0000 7f6c73e00640 1 --2- 172.21.15.17:0/120475664 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6c74150fc0 0x7f6c7413d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:03.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.197+0000 7f6c72a00640 1 -- 172.21.15.17:0/120475664 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6c74150fc0 msgr2=0x7f6c7413d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:03.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.197+0000 7f6c72a00640 1 --2- 172.21.15.17:0/120475664 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6c74150fc0 0x7f6c7413d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:03.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.197+0000 7f6c72a00640 1 -- 172.21.15.17:0/120475664 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c740c7030 msgr2=0x7f6c740c2750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:03.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.197+0000 7f6c72a00640 1 --2- 172.21.15.17:0/120475664 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c740c7030 0x7f6c740c2750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:03.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.197+0000 7f6c72a00640 1 -- 172.21.15.17:0/120475664 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c7414aa70 con 0x7f6c740c8470 2024-09-10T21:33:03.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.197+0000 7f6c72a00640 1 --2- 172.21.15.17:0/120475664 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6c740c8470 0x7f6c741f7290 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f6c6000c480 tx=0x7f6c6000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:03.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.197+0000 7f6c6b400640 1 -- 172.21.15.17:0/120475664 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c60017070 con 0x7f6c740c8470 2024-09-10T21:33:03.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.197+0000 7f6c6b400640 1 -- 172.21.15.17:0/120475664 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6c60003600 con 0x7f6c740c8470 2024-09-10T21:33:03.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.197+0000 7f6c6b400640 1 -- 172.21.15.17:0/120475664 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c60003a50 con 0x7f6c740c8470 2024-09-10T21:33:03.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.197+0000 7f6c7a400640 1 -- 172.21.15.17:0/120475664 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c740cb580 con 0x7f6c740c8470 2024-09-10T21:33:03.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.197+0000 7f6c7a400640 1 -- 172.21.15.17:0/120475664 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c740cbb00 con 0x7f6c740c8470 2024-09-10T21:33:03.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.197+0000 7f6c7a400640 1 -- 172.21.15.17:0/120475664 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c740cbd40 con 0x7f6c740c8470 2024-09-10T21:33:03.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.197+0000 7f6c6b400640 1 -- 172.21.15.17:0/120475664 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f6c60003bf0 con 0x7f6c740c8470 2024-09-10T21:33:03.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.201+0000 7f6c6b400640 1 --2- 172.21.15.17:0/120475664 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f6c540725e0 0x7f6c54074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:03.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.201+0000 7f6c6b400640 1 -- 172.21.15.17:0/120475664 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f6c60092cc0 con 0x7f6c740c8470 2024-09-10T21:33:03.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.201+0000 7f6c73400640 1 --2- 172.21.15.17:0/120475664 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f6c540725e0 0x7f6c54074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:03.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.201+0000 7f6c73400640 1 --2- 172.21.15.17:0/120475664 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f6c540725e0 0x7f6c54074aa0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f6c6400bb50 tx=0x7f6c6400b2e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:03.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.205+0000 7f6c6b400640 1 -- 172.21.15.17:0/120475664 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f6c740cbd40 con 0x7f6c740c8470 2024-09-10T21:33:03.482 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.477+0000 7f6c7a400640 1 -- 172.21.15.17:0/120475664 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f6c740c8250 con 0x7f6c740c8470 2024-09-10T21:33:03.482 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.477+0000 7f6c6b400640 1 -- 172.21.15.17:0/120475664 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1557 (secure 0 0 0) 0x7f6c60099080 con 0x7f6c740c8470 2024-09-10T21:33:03.482 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":70,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":14,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5e2f023f-d6a6-49f4-a256-f4807dce3739","modified":"2024-09-10T21:23:46.892130Z","created":"2024-09-10T21:23:46.892130Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3301","nonce":0},{"type":"v1","addr":"172.21.15.31:6790","nonce":0}]},"addr":"172.21.15.31:6790/0","public_addr":"172.21.15.31:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-10T21:33:03.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.481+0000 7f6c7a400640 1 -- 172.21.15.17:0/120475664 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f6c540725e0 msgr2=0x7f6c54074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:03.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.481+0000 7f6c7a400640 1 --2- 172.21.15.17:0/120475664 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f6c540725e0 0x7f6c54074aa0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f6c6400bb50 tx=0x7f6c6400b2e0 comp rx=0 tx=0).stop 2024-09-10T21:33:03.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.481+0000 7f6c7a400640 1 -- 172.21.15.17:0/120475664 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6c740c8470 msgr2=0x7f6c741f7290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:03.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.481+0000 7f6c7a400640 1 --2- 172.21.15.17:0/120475664 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6c740c8470 0x7f6c741f7290 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f6c6000c480 tx=0x7f6c6000c940 comp rx=0 tx=0).stop 2024-09-10T21:33:03.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.481+0000 7f6c7a400640 1 -- 172.21.15.17:0/120475664 shutdown_connections 2024-09-10T21:33:03.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.481+0000 7f6c7a400640 1 --2- 172.21.15.17:0/120475664 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f6c540725e0 0x7f6c54074aa0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:03.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.481+0000 7f6c7a400640 1 --2- 172.21.15.17:0/120475664 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6c74150fc0 0x7f6c7413d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:03.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.481+0000 7f6c7a400640 1 --2- 172.21.15.17:0/120475664 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6c740c8470 0x7f6c741f7290 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:03.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.481+0000 7f6c7a400640 1 --2- 172.21.15.17:0/120475664 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c740c7030 0x7f6c740c2750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:03.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.481+0000 7f6c7a400640 1 -- 172.21.15.17:0/120475664 >> 172.21.15.17:0/120475664 conn(0x7f6c74070160 msgr2=0x7f6c7414e9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:03.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.481+0000 7f6c7a400640 1 -- 172.21.15.17:0/120475664 shutdown_connections 2024-09-10T21:33:03.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.481+0000 7f6c7a400640 1 -- 172.21.15.17:0/120475664 wait complete. 2024-09-10T21:33:03.506 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-09-10T21:33:03.506 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-09-10T21:33:03.506 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-09-10T21:33:03.507 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-10T21:33:03.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.673+0000 7ff134800640 1 -- 172.21.15.17:0/3612243487 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff12c071760 msgr2=0x7ff12c159850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:03.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.673+0000 7ff134800640 1 --2- 172.21.15.17:0/3612243487 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff12c071760 0x7ff12c159850 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ff12c0c3230 tx=0x7ff11c01f090 comp rx=0 tx=0).stop 2024-09-10T21:33:03.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.673+0000 7ff134800640 1 -- 172.21.15.17:0/3612243487 shutdown_connections 2024-09-10T21:33:03.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.673+0000 7ff134800640 1 --2- 172.21.15.17:0/3612243487 >> v2:172.21.15.31:3300/0 conn(0x7ff12c0c4dd0 0x7ff12c0ce130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:03.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.673+0000 7ff134800640 1 --2- 172.21.15.17:0/3612243487 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff12c071760 0x7ff12c159850 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:03.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.673+0000 7ff134800640 1 -- 172.21.15.17:0/3612243487 >> 172.21.15.17:0/3612243487 conn(0x7ff12c068990 msgr2=0x7ff12c068da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:03.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.673+0000 7ff134800640 1 -- 172.21.15.17:0/3612243487 shutdown_connections 2024-09-10T21:33:03.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.673+0000 7ff134800640 1 -- 172.21.15.17:0/3612243487 wait complete. 2024-09-10T21:33:03.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.673+0000 7ff134800640 1 Processor -- start 2024-09-10T21:33:03.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.673+0000 7ff134800640 1 -- start start 2024-09-10T21:33:03.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.673+0000 7ff134800640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff12c071760 0x7ff12c15a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:03.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.673+0000 7ff134800640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff12c0c4dd0 0x7ff12c151360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:03.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.673+0000 7ff134800640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff12c1526b0 0x7ff12c1518a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:03.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.673+0000 7ff134800640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7ff12c152a90 con 0x7ff12c1526b0 2024-09-10T21:33:03.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.673+0000 7ff134800640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7ff12c0c8a40 con 0x7ff12c071760 2024-09-10T21:33:03.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.673+0000 7ff134800640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7ff12c0c8c90 con 0x7ff12c0c4dd0 2024-09-10T21:33:03.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.673+0000 7ff132800640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff12c1526b0 0x7ff12c1518a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:03.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.673+0000 7ff132800640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff12c1526b0 0x7ff12c1518a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44024/0 (socket says 172.21.15.17:44024) 2024-09-10T21:33:03.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.673+0000 7ff132800640 1 -- 172.21.15.17:0/1759055473 learned_addr learned my addr 172.21.15.17:0/1759055473 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:33:03.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.677+0000 7ff131e00640 1 --2- 172.21.15.17:0/1759055473 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff12c071760 0x7ff12c15a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:03.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.677+0000 7ff131400640 1 --2- 172.21.15.17:0/1759055473 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff12c0c4dd0 0x7ff12c151360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:03.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.677+0000 7ff132800640 1 -- 172.21.15.17:0/1759055473 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff12c0c4dd0 msgr2=0x7ff12c151360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:03.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.677+0000 7ff132800640 1 --2- 172.21.15.17:0/1759055473 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff12c0c4dd0 0x7ff12c151360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:03.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.677+0000 7ff132800640 1 -- 172.21.15.17:0/1759055473 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff12c071760 msgr2=0x7ff12c15a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:03.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.677+0000 7ff132800640 1 --2- 172.21.15.17:0/1759055473 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff12c071760 0x7ff12c15a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:03.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.677+0000 7ff132800640 1 -- 172.21.15.17:0/1759055473 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff12c150990 con 0x7ff12c1526b0 2024-09-10T21:33:03.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.677+0000 7ff131400640 1 --2- 172.21.15.17:0/1759055473 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff12c0c4dd0 0x7ff12c151360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:33:03.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.677+0000 7ff131e00640 1 --2- 172.21.15.17:0/1759055473 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff12c071760 0x7ff12c15a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:33:03.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.677+0000 7ff132800640 1 --2- 172.21.15.17:0/1759055473 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff12c1526b0 0x7ff12c1518a0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7ff12800eb20 tx=0x7ff12800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:03.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.677+0000 7ff126000640 1 -- 172.21.15.17:0/1759055473 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff128004b30 con 0x7ff12c1526b0 2024-09-10T21:33:03.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.677+0000 7ff126000640 1 -- 172.21.15.17:0/1759055473 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff128004cf0 con 0x7ff12c1526b0 2024-09-10T21:33:03.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.677+0000 7ff126000640 1 -- 172.21.15.17:0/1759055473 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff128005280 con 0x7ff12c1526b0 2024-09-10T21:33:03.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.677+0000 7ff134800640 1 -- 172.21.15.17:0/1759055473 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff12c14c1c0 con 0x7ff12c1526b0 2024-09-10T21:33:03.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.677+0000 7ff134800640 1 -- 172.21.15.17:0/1759055473 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff12c06dd30 con 0x7ff12c1526b0 2024-09-10T21:33:03.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.677+0000 7ff134800640 1 --2- 172.21.15.17:0/1759055473 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff12c0cda50 0x7ff12c06e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:03.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.677+0000 7ff134800640 1 -- 172.21.15.17:0/1759055473 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7ff12c06e250 con 0x7ff12c0cda50 2024-09-10T21:33:03.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.677+0000 7ff131e00640 1 --2- 172.21.15.17:0/1759055473 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff12c0cda50 0x7ff12c06e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:03.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.677+0000 7ff131e00640 1 -- 172.21.15.17:0/1759055473 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff12c14c560 con 0x7ff12c0cda50 2024-09-10T21:33:03.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.677+0000 7ff131e00640 1 --2- 172.21.15.17:0/1759055473 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff12c0cda50 0x7ff12c06e510 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ff12000d3f0 tx=0x7ff12000d8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:03.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.677+0000 7ff126000640 1 -- 172.21.15.17:0/1759055473 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff120017070 con 0x7ff12c0cda50 2024-09-10T21:33:03.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.677+0000 7ff126000640 1 -- 172.21.15.17:0/1759055473 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7ff120007030 con 0x7ff12c0cda50 2024-09-10T21:33:03.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.677+0000 7ff126000640 1 -- 172.21.15.17:0/1759055473 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff12c0cda50 msgr2=0x7ff12c06e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:03.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.677+0000 7ff126000640 1 --2- 172.21.15.17:0/1759055473 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff12c0cda50 0x7ff12c06e510 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ff12000d3f0 tx=0x7ff12000d8b0 comp rx=0 tx=0).stop 2024-09-10T21:33:03.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.677+0000 7ff126000640 1 -- 172.21.15.17:0/1759055473 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff12c0cda50 msgr2=0x7ff12c06e510 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:33:03.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.677+0000 7ff126000640 1 --2- 172.21.15.17:0/1759055473 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff12c0cda50 0x7ff12c06e510 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:03.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.677+0000 7ff126000640 1 -- 172.21.15.17:0/1759055473 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7ff128005420 con 0x7ff12c1526b0 2024-09-10T21:33:03.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.681+0000 7ff126000640 1 --2- 172.21.15.17:0/1759055473 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7ff110072170 0x7ff110074630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:03.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.681+0000 7ff126000640 1 -- 172.21.15.17:0/1759055473 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7ff128094370 con 0x7ff12c1526b0 2024-09-10T21:33:03.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.681+0000 7ff131e00640 1 --2- 172.21.15.17:0/1759055473 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7ff110072170 0x7ff110074630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:03.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.681+0000 7ff131e00640 1 --2- 172.21.15.17:0/1759055473 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7ff110072170 0x7ff110074630 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7ff120009010 tx=0x7ff12000b000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:03.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.689+0000 7ff134800640 1 --2- 172.21.15.17:0/1759055473 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff12c06ea50 0x7ff12c1464b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:03.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.689+0000 7ff134800640 1 -- 172.21.15.17:0/1759055473 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7ff12c159260 con 0x7ff12c06ea50 2024-09-10T21:33:03.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.689+0000 7ff131400640 1 --2- 172.21.15.17:0/1759055473 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff12c06ea50 0x7ff12c1464b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:03.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.689+0000 7ff131400640 1 -- 172.21.15.17:0/1759055473 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7ff12c1e3e20 con 0x7ff12c06ea50 2024-09-10T21:33:03.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.689+0000 7ff131400640 1 --2- 172.21.15.17:0/1759055473 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff12c06ea50 0x7ff12c1464b0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff12c070940 tx=0x7ff11c005d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:03.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.689+0000 7ff126000640 1 -- 172.21.15.17:0/1759055473 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff11c0b5040 con 0x7ff12c06ea50 2024-09-10T21:33:03.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.689+0000 7ff126000640 1 -- 172.21.15.17:0/1759055473 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4710 (secure 0 0 0) 0x7ff11c005230 con 0x7ff12c06ea50 2024-09-10T21:33:03.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.689+0000 7ff126000640 1 -- 172.21.15.17:0/1759055473 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff12c06ea50 msgr2=0x7ff12c1464b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:03.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.689+0000 7ff126000640 1 --2- 172.21.15.17:0/1759055473 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff12c06ea50 0x7ff12c1464b0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff12c070940 tx=0x7ff11c005d60 comp rx=0 tx=0).stop 2024-09-10T21:33:03.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.689+0000 7ff126000640 1 -- 172.21.15.17:0/1759055473 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff12c06ea50 msgr2=0x7ff12c1464b0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:33:03.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.689+0000 7ff126000640 1 --2- 172.21.15.17:0/1759055473 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff12c06ea50 0x7ff12c1464b0 secure :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff12c070940 tx=0x7ff11c005d60 comp rx=0 tx=0).stop 2024-09-10T21:33:03.696 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-10T21:33:03.696 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:33:03.696 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:33:03.696 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "leader", 2024-09-10T21:33:03.697 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 70, 2024-09-10T21:33:03.697 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 40575, 2024-09-10T21:33:03.697 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-10T21:33:03.697 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-10T21:33:03.697 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-10T21:33:03.697 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-10T21:33:03.697 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:33:03.698 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 15, 2024-09-10T21:33:03.698 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:33:03.698 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-10T21:33:03.698 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-10T21:33:03.698 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:33:03.698 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:33:03.698 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:33:03.699 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:33:03.699 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:33:03.699 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:33:03.699 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:33:03.699 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:33:03.699 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:33:03.699 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:33:03.699 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:33:03.700 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:33:03.700 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-10T21:33:03.700 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-10T21:33:03.700 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:33:03.700 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:33:03.700 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:33:03.701 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:33:03.701 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:33:03.701 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:33:03.701 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:33:03.701 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:33:03.701 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:33:03.702 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:33:03.702 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:33:03.702 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:33:03.702 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:33:03.702 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-10T21:33:03.702 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-10T21:33:03.703 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-10T21:33:03.703 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-10T21:33:03.703 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-10T21:33:03.703 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "5e2f023f-d6a6-49f4-a256-f4807dce3739", 2024-09-10T21:33:03.703 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:33:03.703 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:33:03.703 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-10T21:33:03.704 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-10T21:33:03.704 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-10T21:33:03.704 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-10T21:33:03.704 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-10T21:33:03.704 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-10T21:33:03.704 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-10T21:33:03.704 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:33:03.704 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-10T21:33:03.704 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:33:03.704 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:33:03.704 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:33:03.705 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:33:03.705 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:33:03.705 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:33:03.705 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:33:03.705 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:33:03.705 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:33:03.705 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:33:03.706 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:33:03.706 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:33:03.706 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-10T21:33:03.706 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:33:03.706 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-10T21:33:03.706 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:33:03.707 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:33:03.707 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:33:03.707 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:33:03.707 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:33:03.707 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:33:03.707 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:33:03.707 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-10T21:33:03.708 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:33:03.708 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:33:03.708 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:33:03.708 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:33:03.708 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-10T21:33:03.708 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:33:03.708 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:33:03.708 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:33:03.708 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:33:03.708 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-10T21:33:03.708 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-10T21:33:03.708 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:33:03.709 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:33:03.709 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:33:03.709 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:33:03.709 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:33:03.709 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:33:03.709 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:33:03.709 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:33:03.709 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:33:03.709 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:33:03.709 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:33:03.709 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3300", 2024-09-10T21:33:03.709 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:33:03.709 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:33:03.710 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:33:03.710 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:33:03.710 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789", 2024-09-10T21:33:03.710 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:33:03.710 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:33:03.710 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:33:03.710 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:33:03.710 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789/0", 2024-09-10T21:33:03.710 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6789/0", 2024-09-10T21:33:03.710 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:33:03.710 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:33:03.710 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:33:03.710 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:33:03.711 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:33:03.711 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:33:03.711 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:33:03.711 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:33:03.711 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:33:03.711 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:33:03.711 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:33:03.711 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3301", 2024-09-10T21:33:03.711 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:33:03.711 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:33:03.711 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:33:03.711 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:33:03.711 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790", 2024-09-10T21:33:03.711 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:33:03.712 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:33:03.712 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:33:03.712 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:33:03.712 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790/0", 2024-09-10T21:33:03.712 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6790/0", 2024-09-10T21:33:03.712 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:33:03.712 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:33:03.712 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:33:03.712 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:33:03.712 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:33:03.712 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:33:03.712 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-10T21:33:03.712 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-10T21:33:03.713 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:33:03.713 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:33:03.713 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:33:03.713 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:33:03.713 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:33:03.713 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:33:03.713 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": [ 2024-09-10T21:33:03.713 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:33:03.713 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:33:03.713 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:33:03.713 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:33:03.713 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:33:03.713 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:33:03.713 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-10T21:33:03.714 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:33:03.714 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:33:03.714 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:33:03.714 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 2 2024-09-10T21:33:03.714 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:33:03.714 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:33:03.714 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:33:03.714 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-10T21:33:03.714 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-10T21:33:03.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.689+0000 7ff134800640 1 -- 172.21.15.17:0/1759055473 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7ff110072170 msgr2=0x7ff110074630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:03.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.689+0000 7ff134800640 1 --2- 172.21.15.17:0/1759055473 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7ff110072170 0x7ff110074630 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7ff120009010 tx=0x7ff12000b000 comp rx=0 tx=0).stop 2024-09-10T21:33:03.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.689+0000 7ff134800640 1 -- 172.21.15.17:0/1759055473 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff12c1526b0 msgr2=0x7ff12c1518a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:03.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.689+0000 7ff134800640 1 --2- 172.21.15.17:0/1759055473 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff12c1526b0 0x7ff12c1518a0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7ff12800eb20 tx=0x7ff12800c2d0 comp rx=0 tx=0).stop 2024-09-10T21:33:03.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.689+0000 7ff132800640 1 -- 172.21.15.17:0/1759055473 reap_dead start 2024-09-10T21:33:03.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.689+0000 7ff132800640 1 -- 172.21.15.17:0/1759055473 reap_dead start 2024-09-10T21:33:03.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.689+0000 7ff134800640 1 -- 172.21.15.17:0/1759055473 shutdown_connections 2024-09-10T21:33:03.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.689+0000 7ff134800640 1 -- 172.21.15.17:0/1759055473 >> 172.21.15.17:0/1759055473 conn(0x7ff12c068990 msgr2=0x7ff12c15af60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:03.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.689+0000 7ff134800640 1 -- 172.21.15.17:0/1759055473 shutdown_connections 2024-09-10T21:33:03.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.689+0000 7ff134800640 1 -- 172.21.15.17:0/1759055473 wait complete. 2024-09-10T21:33:03.716 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-10T21:33:03.717 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-09-10T21:33:03.717 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-10T21:33:03.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.885+0000 7f8e45e00640 1 -- 172.21.15.17:0/2075035433 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8e400c37e0 msgr2=0x7f8e40158c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:03.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.885+0000 7f8e45e00640 1 --2- 172.21.15.17:0/2075035433 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8e400c37e0 0x7f8e40158c20 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f8e2c009840 tx=0x7f8e2c01c820 comp rx=0 tx=0).stop 2024-09-10T21:33:03.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.885+0000 7f8e45e00640 1 -- 172.21.15.17:0/2075035433 shutdown_connections 2024-09-10T21:33:03.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.885+0000 7f8e45e00640 1 --2- 172.21.15.17:0/2075035433 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8e400c37e0 0x7f8e40158c20 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:03.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.885+0000 7f8e45e00640 1 -- 172.21.15.17:0/2075035433 >> 172.21.15.17:0/2075035433 conn(0x7f8e4006cc20 msgr2=0x7f8e4006d030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:03.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.885+0000 7f8e45e00640 1 -- 172.21.15.17:0/2075035433 shutdown_connections 2024-09-10T21:33:03.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.885+0000 7f8e45e00640 1 -- 172.21.15.17:0/2075035433 wait complete. 2024-09-10T21:33:03.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.885+0000 7f8e45e00640 1 Processor -- start 2024-09-10T21:33:03.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.889+0000 7f8e45e00640 1 -- start start 2024-09-10T21:33:03.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.889+0000 7f8e45e00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e400c37e0 0x7f8e400cb6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:03.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.889+0000 7f8e45e00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8e400c6150 0x7f8e401f6660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:03.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.889+0000 7f8e45e00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8e400c6530 0x7f8e40153b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:03.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.889+0000 7f8e45e00640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f8e40165f60 con 0x7f8e400c6530 2024-09-10T21:33:03.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.889+0000 7f8e45e00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f8e40165de0 con 0x7f8e400c37e0 2024-09-10T21:33:03.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.889+0000 7f8e45e00640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f8e401660e0 con 0x7f8e400c6150 2024-09-10T21:33:03.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.889+0000 7f8e45400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8e400c6530 0x7f8e40153b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:03.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.889+0000 7f8e45400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8e400c6530 0x7f8e40153b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44058/0 (socket says 172.21.15.17:44058) 2024-09-10T21:33:03.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.889+0000 7f8e45400640 1 -- 172.21.15.17:0/729287107 learned_addr learned my addr 172.21.15.17:0/729287107 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:33:03.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.889+0000 7f8e3fe00640 1 --2- 172.21.15.17:0/729287107 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8e400c6150 0x7f8e401f6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:03.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.889+0000 7f8e44a00640 1 --2- 172.21.15.17:0/729287107 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e400c37e0 0x7f8e400cb6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:03.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.889+0000 7f8e45400640 1 -- 172.21.15.17:0/729287107 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8e400c6150 msgr2=0x7f8e401f6660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:03.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.889+0000 7f8e45400640 1 --2- 172.21.15.17:0/729287107 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8e400c6150 0x7f8e401f6660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:03.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.889+0000 7f8e45400640 1 -- 172.21.15.17:0/729287107 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e400c37e0 msgr2=0x7f8e400cb6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:03.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.889+0000 7f8e45400640 1 --2- 172.21.15.17:0/729287107 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e400c37e0 0x7f8e400cb6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:03.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.889+0000 7f8e45400640 1 -- 172.21.15.17:0/729287107 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e40163610 con 0x7f8e400c6530 2024-09-10T21:33:03.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.889+0000 7f8e44a00640 1 --2- 172.21.15.17:0/729287107 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e400c37e0 0x7f8e400cb6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:33:03.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.889+0000 7f8e45400640 1 --2- 172.21.15.17:0/729287107 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8e400c6530 0x7f8e40153b40 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f8e38002240 tx=0x7f8e3800cb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:03.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.889+0000 7f8e3cc00640 1 -- 172.21.15.17:0/729287107 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e38016020 con 0x7f8e400c6530 2024-09-10T21:33:03.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.889+0000 7f8e3cc00640 1 -- 172.21.15.17:0/729287107 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8e3800a930 con 0x7f8e400c6530 2024-09-10T21:33:03.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.889+0000 7f8e45e00640 1 -- 172.21.15.17:0/729287107 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e401557e0 con 0x7f8e400c6530 2024-09-10T21:33:03.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.889+0000 7f8e3cc00640 1 -- 172.21.15.17:0/729287107 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e3800adc0 con 0x7f8e400c6530 2024-09-10T21:33:03.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.889+0000 7f8e45e00640 1 -- 172.21.15.17:0/729287107 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e40154fe0 con 0x7f8e400c6530 2024-09-10T21:33:03.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.889+0000 7f8e45e00640 1 --2- 172.21.15.17:0/729287107 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e401f6ba0 0x7f8e40154080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:03.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.889+0000 7f8e45e00640 1 -- 172.21.15.17:0/729287107 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f8e40158c20 con 0x7f8e401f6ba0 2024-09-10T21:33:03.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.889+0000 7f8e44a00640 1 --2- 172.21.15.17:0/729287107 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e401f6ba0 0x7f8e40154080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:03.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.893+0000 7f8e44a00640 1 -- 172.21.15.17:0/729287107 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8e401545c0 con 0x7f8e401f6ba0 2024-09-10T21:33:03.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.893+0000 7f8e44a00640 1 --2- 172.21.15.17:0/729287107 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e401f6ba0 0x7f8e40154080 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f8e3400e700 tx=0x7f8e3400ebc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:03.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.893+0000 7f8e3cc00640 1 -- 172.21.15.17:0/729287107 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f8e38012420 con 0x7f8e400c6530 2024-09-10T21:33:03.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.893+0000 7f8e3cc00640 1 --2- 172.21.15.17:0/729287107 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f8e200725e0 0x7f8e20074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:03.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.893+0000 7f8e3cc00640 1 -- 172.21.15.17:0/729287107 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f8e38093ae0 con 0x7f8e400c6530 2024-09-10T21:33:03.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.893+0000 7f8e3fe00640 1 --2- 172.21.15.17:0/729287107 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f8e200725e0 0x7f8e20074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:03.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.893+0000 7f8e3cc00640 1 -- 172.21.15.17:0/729287107 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e340050d0 con 0x7f8e401f6ba0 2024-09-10T21:33:03.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.893+0000 7f8e3cc00640 1 -- 172.21.15.17:0/729287107 <== mon.1 v2:172.21.15.31:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f8e34018070 con 0x7f8e401f6ba0 2024-09-10T21:33:03.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.893+0000 7f8e3cc00640 1 -- 172.21.15.17:0/729287107 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e401f6ba0 msgr2=0x7f8e40154080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:03.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.893+0000 7f8e3cc00640 1 --2- 172.21.15.17:0/729287107 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e401f6ba0 0x7f8e40154080 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f8e3400e700 tx=0x7f8e3400ebc0 comp rx=0 tx=0).stop 2024-09-10T21:33:03.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.893+0000 7f8e3cc00640 1 -- 172.21.15.17:0/729287107 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e401f6ba0 msgr2=0x7f8e40154080 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:33:03.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.893+0000 7f8e3cc00640 1 --2- 172.21.15.17:0/729287107 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e401f6ba0 0x7f8e40154080 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:03.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.893+0000 7f8e3fe00640 1 --2- 172.21.15.17:0/729287107 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f8e200725e0 0x7f8e20074aa0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f8e2c0af000 tx=0x7f8e2c01b060 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:03.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.901+0000 7f8e45e00640 1 --2- 172.21.15.17:0/729287107 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e40155220 0x7f8e4014e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:03.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.901+0000 7f8e45e00640 1 -- 172.21.15.17:0/729287107 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f8e401505e0 con 0x7f8e40155220 2024-09-10T21:33:03.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.901+0000 7f8e44a00640 1 --2- 172.21.15.17:0/729287107 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e40155220 0x7f8e4014e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:03.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.901+0000 7f8e44a00640 1 -- 172.21.15.17:0/729287107 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f8e40150300 con 0x7f8e40155220 2024-09-10T21:33:03.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.901+0000 7f8e44a00640 1 --2- 172.21.15.17:0/729287107 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e40155220 0x7f8e4014e2e0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f8e34017000 tx=0x7f8e34001180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:03.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.901+0000 7f8e3cc00640 1 -- 172.21.15.17:0/729287107 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e3401d230 con 0x7f8e40155220 2024-09-10T21:33:03.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.901+0000 7f8e3cc00640 1 -- 172.21.15.17:0/729287107 <== mon.1 v2:172.21.15.31:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5043 (secure 0 0 0) 0x7f8e340054a0 con 0x7f8e40155220 2024-09-10T21:33:03.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.901+0000 7f8e3cc00640 1 -- 172.21.15.17:0/729287107 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e40155220 msgr2=0x7f8e4014e2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:03.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.901+0000 7f8e3cc00640 1 --2- 172.21.15.17:0/729287107 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e40155220 0x7f8e4014e2e0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f8e34017000 tx=0x7f8e34001180 comp rx=0 tx=0).stop 2024-09-10T21:33:03.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.901+0000 7f8e3cc00640 1 -- 172.21.15.17:0/729287107 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e40155220 msgr2=0x7f8e4014e2e0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:33:03.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.901+0000 7f8e3cc00640 1 --2- 172.21.15.17:0/729287107 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e40155220 0x7f8e4014e2e0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:03.908 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-10T21:33:03.908 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:33:03.908 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:33:03.908 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-10T21:33:03.908 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 70, 2024-09-10T21:33:03.909 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 522351, 2024-09-10T21:33:03.909 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-10T21:33:03.909 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-10T21:33:03.909 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-10T21:33:03.909 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-10T21:33:03.909 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:33:03.909 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 15, 2024-09-10T21:33:03.909 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:33:03.909 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-10T21:33:03.909 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-10T21:33:03.909 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:33:03.909 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:33:03.909 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:33:03.909 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:33:03.910 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:33:03.910 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:33:03.910 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:33:03.910 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:33:03.910 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:33:03.910 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:33:03.910 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:33:03.910 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:33:03.910 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-10T21:33:03.910 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-10T21:33:03.910 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:33:03.910 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:33:03.911 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:33:03.911 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:33:03.911 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:33:03.911 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:33:03.911 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:33:03.911 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:33:03.911 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:33:03.911 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:33:03.911 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:33:03.911 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:33:03.911 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:33:03.912 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-10T21:33:03.912 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-10T21:33:03.912 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-10T21:33:03.912 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-10T21:33:03.912 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-10T21:33:03.912 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "5e2f023f-d6a6-49f4-a256-f4807dce3739", 2024-09-10T21:33:03.912 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:33:03.912 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:33:03.913 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-10T21:33:03.913 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-10T21:33:03.913 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-10T21:33:03.913 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-10T21:33:03.913 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-10T21:33:03.913 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-10T21:33:03.913 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-10T21:33:03.913 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:33:03.913 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-10T21:33:03.914 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:33:03.914 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:33:03.914 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:33:03.914 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:33:03.914 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:33:03.914 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:33:03.915 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:33:03.915 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:33:03.915 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:33:03.915 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:33:03.915 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:33:03.915 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:33:03.916 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-10T21:33:03.916 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:33:03.916 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-10T21:33:03.916 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:33:03.916 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:33:03.916 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:33:03.916 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:33:03.916 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:33:03.916 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:33:03.916 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:33:03.916 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-10T21:33:03.917 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:33:03.917 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:33:03.917 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:33:03.917 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:33:03.917 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-10T21:33:03.917 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:33:03.917 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:33:03.917 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:33:03.917 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:33:03.917 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-10T21:33:03.917 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-10T21:33:03.917 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:33:03.917 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:33:03.918 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:33:03.918 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:33:03.918 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:33:03.918 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:33:03.918 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:33:03.918 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:33:03.918 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:33:03.918 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:33:03.918 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:33:03.918 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3300", 2024-09-10T21:33:03.919 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:33:03.919 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:33:03.919 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:33:03.919 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:33:03.919 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789", 2024-09-10T21:33:03.919 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:33:03.919 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:33:03.919 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:33:03.919 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:33:03.919 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789/0", 2024-09-10T21:33:03.919 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6789/0", 2024-09-10T21:33:03.919 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:33:03.919 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:33:03.919 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:33:03.920 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:33:03.920 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:33:03.920 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:33:03.920 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:33:03.920 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:33:03.920 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:33:03.920 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:33:03.920 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:33:03.920 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3301", 2024-09-10T21:33:03.920 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:33:03.920 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:33:03.921 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:33:03.921 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:33:03.921 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790", 2024-09-10T21:33:03.921 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:33:03.921 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:33:03.921 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:33:03.921 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:33:03.921 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790/0", 2024-09-10T21:33:03.921 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6790/0", 2024-09-10T21:33:03.922 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:33:03.922 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:33:03.922 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:33:03.922 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:33:03.922 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:33:03.922 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:33:03.922 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-10T21:33:03.922 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-10T21:33:03.922 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:33:03.922 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:33:03.922 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:33:03.922 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:33:03.922 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:33:03.923 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:33:03.923 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": [ 2024-09-10T21:33:03.923 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:33:03.923 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:33:03.923 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:33:03.923 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 5 2024-09-10T21:33:03.923 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:33:03.923 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:33:03.923 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-10T21:33:03.923 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:33:03.923 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:33:03.923 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:33:03.923 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 7 2024-09-10T21:33:03.924 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:33:03.924 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:33:03.924 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-10T21:33:03.924 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:33:03.924 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:33:03.924 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:33:03.924 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 6 2024-09-10T21:33:03.924 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:33:03.924 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:33:03.924 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": [ 2024-09-10T21:33:03.924 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:33:03.924 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:33:03.925 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:33:03.925 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 2 2024-09-10T21:33:03.925 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:33:03.925 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:33:03.925 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:33:03.925 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-10T21:33:03.925 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-10T21:33:03.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.901+0000 7f8e45e00640 1 -- 172.21.15.17:0/729287107 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f8e200725e0 msgr2=0x7f8e20074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:03.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.901+0000 7f8e45e00640 1 --2- 172.21.15.17:0/729287107 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f8e200725e0 0x7f8e20074aa0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f8e2c0af000 tx=0x7f8e2c01b060 comp rx=0 tx=0).stop 2024-09-10T21:33:03.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.901+0000 7f8e45e00640 1 -- 172.21.15.17:0/729287107 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8e400c6530 msgr2=0x7f8e40153b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:03.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.901+0000 7f8e45e00640 1 --2- 172.21.15.17:0/729287107 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8e400c6530 0x7f8e40153b40 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f8e38002240 tx=0x7f8e3800cb80 comp rx=0 tx=0).stop 2024-09-10T21:33:03.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.901+0000 7f8e45400640 1 -- 172.21.15.17:0/729287107 reap_dead start 2024-09-10T21:33:03.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.901+0000 7f8e45400640 1 -- 172.21.15.17:0/729287107 reap_dead start 2024-09-10T21:33:03.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.901+0000 7f8e45e00640 1 -- 172.21.15.17:0/729287107 shutdown_connections 2024-09-10T21:33:03.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.901+0000 7f8e45e00640 1 -- 172.21.15.17:0/729287107 >> 172.21.15.17:0/729287107 conn(0x7f8e4006cc20 msgr2=0x7f8e400c87a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:03.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.901+0000 7f8e45e00640 1 -- 172.21.15.17:0/729287107 shutdown_connections 2024-09-10T21:33:03.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:03.901+0000 7f8e45e00640 1 -- 172.21.15.17:0/729287107 wait complete. 2024-09-10T21:33:03.927 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-10T21:33:03.928 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-09-10T21:33:03.928 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-10T21:33:04.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:04.093+0000 7f30da200640 1 -- 172.21.15.17:0/3814973653 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f30d40c0fd0 msgr2=0x7f30d40740d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:04.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:04.093+0000 7f30da200640 1 --2- 172.21.15.17:0/3814973653 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f30d40c0fd0 0x7f30d40740d0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f30c0009840 tx=0x7f30c001c820 comp rx=0 tx=0).stop 2024-09-10T21:33:04.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:04.093+0000 7f30da200640 1 -- 172.21.15.17:0/3814973653 shutdown_connections 2024-09-10T21:33:04.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:04.093+0000 7f30da200640 1 --2- 172.21.15.17:0/3814973653 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f30d40c0fd0 0x7f30d40740d0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:04.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:04.093+0000 7f30da200640 1 -- 172.21.15.17:0/3814973653 >> 172.21.15.17:0/3814973653 conn(0x7f30d40700d0 msgr2=0x7f30d40704e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:04.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:04.093+0000 7f30da200640 1 -- 172.21.15.17:0/3814973653 shutdown_connections 2024-09-10T21:33:04.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:04.093+0000 7f30da200640 1 -- 172.21.15.17:0/3814973653 wait complete. 2024-09-10T21:33:04.100 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:04.097+0000 7f30da200640 1 Processor -- start 2024-09-10T21:33:04.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:04.097+0000 7f30da200640 1 -- start start 2024-09-10T21:33:04.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:04.097+0000 7f30da200640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f30d4073cf0 0x7f30d41f5390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:04.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:04.097+0000 7f30da200640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f30d40c0fd0 0x7f30d4141df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:04.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:04.097+0000 7f30da200640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f30d415abb0 0x7f30d4142330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:04.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:04.097+0000 7f30da200640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f30d40c7dd0 con 0x7f30d415abb0 2024-09-10T21:33:04.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:04.097+0000 7f30da200640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f30d40c8370 con 0x7f30d40c0fd0 2024-09-10T21:33:04.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:04.097+0000 7f30da200640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f30d40c7f50 con 0x7f30d4073cf0 2024-09-10T21:33:04.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:04.097+0000 7f30d3e00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f30d415abb0 0x7f30d4142330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:04.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:04.097+0000 7f30d3e00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f30d415abb0 0x7f30d4142330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:44078/0 (socket says 172.21.15.17:44078) 2024-09-10T21:33:04.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:04.097+0000 7f30d3e00640 1 -- 172.21.15.17:0/4094280781 learned_addr learned my addr 172.21.15.17:0/4094280781 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:33:04.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:04.097+0000 7f30d2a00640 1 --2- 172.21.15.17:0/4094280781 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f30d40c0fd0 0x7f30d4141df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:04.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:04.097+0000 7f30d3400640 1 --2- 172.21.15.17:0/4094280781 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f30d4073cf0 0x7f30d41f5390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:04.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:04.097+0000 7f30d3e00640 1 -- 172.21.15.17:0/4094280781 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f30d4073cf0 msgr2=0x7f30d41f5390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:04.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:04.097+0000 7f30d3e00640 1 --2- 172.21.15.17:0/4094280781 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f30d4073cf0 0x7f30d41f5390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:04.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:04.097+0000 7f30d3e00640 1 -- 172.21.15.17:0/4094280781 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f30d40c0fd0 msgr2=0x7f30d4141df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:04.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:04.097+0000 7f30d3e00640 1 --2- 172.21.15.17:0/4094280781 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f30d40c0fd0 0x7f30d4141df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:04.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:04.097+0000 7f30d3e00640 1 -- 172.21.15.17:0/4094280781 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30d41f5df0 con 0x7f30d415abb0 2024-09-10T21:33:04.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:04.097+0000 7f30d3e00640 1 --2- 172.21.15.17:0/4094280781 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f30d415abb0 0x7f30d4142330 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f30cc00cb10 tx=0x7f30cc00cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:04.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:04.097+0000 7f30cb400640 1 -- 172.21.15.17:0/4094280781 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30cc016020 con 0x7f30d415abb0 2024-09-10T21:33:04.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:04.097+0000 7f30cb400640 1 -- 172.21.15.17:0/4094280781 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f30cc00ce50 con 0x7f30d415abb0 2024-09-10T21:33:04.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:04.097+0000 7f30cb400640 1 -- 172.21.15.17:0/4094280781 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30cc00ad80 con 0x7f30d415abb0 2024-09-10T21:33:04.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:04.097+0000 7f30da200640 1 -- 172.21.15.17:0/4094280781 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30d41f6090 con 0x7f30d415abb0 2024-09-10T21:33:04.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:04.101+0000 7f30da200640 1 -- 172.21.15.17:0/4094280781 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30d40cbc80 con 0x7f30d415abb0 2024-09-10T21:33:04.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:04.101+0000 7f30da200640 1 --2- 172.21.15.17:0/4094280781 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f30d41f58d0 0x7f30d41f35e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:04.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:04.101+0000 7f30da200640 1 -- 172.21.15.17:0/4094280781 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f30d40752a0 con 0x7f30d41f58d0 2024-09-10T21:33:04.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:04.101+0000 7f30d3400640 1 --2- 172.21.15.17:0/4094280781 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f30d41f58d0 0x7f30d41f35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:04.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:04.101+0000 7f30d3400640 1 -- 172.21.15.17:0/4094280781 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f30d4151050 con 0x7f30d41f58d0 2024-09-10T21:33:04.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:04.101+0000 7f30d3400640 1 --2- 172.21.15.17:0/4094280781 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f30d41f58d0 0x7f30d41f35e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f30c400f770 tx=0x7f30c400fc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:04.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:04.101+0000 7f30cb400640 1 -- 172.21.15.17:0/4094280781 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f30cc012420 con 0x7f30d415abb0 2024-09-10T21:33:04.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:04.101+0000 7f30cb400640 1 --2- 172.21.15.17:0/4094280781 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f30b40725e0 0x7f30b4074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:04.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:04.101+0000 7f30cb400640 1 -- 172.21.15.17:0/4094280781 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f30cc093790 con 0x7f30d415abb0 2024-09-10T21:33:04.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:04.101+0000 7f30cb400640 1 -- 172.21.15.17:0/4094280781 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30c40050d0 con 0x7f30d41f58d0 2024-09-10T21:33:04.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:04.101+0000 7f30cb400640 1 -- 172.21.15.17:0/4094280781 <== mon.2 v2:172.21.15.31:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f30c4005270 con 0x7f30d41f58d0 2024-09-10T21:33:04.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:04.101+0000 7f30cb400640 1 -- 172.21.15.17:0/4094280781 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f30d41f58d0 msgr2=0x7f30d41f35e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:04.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:04.101+0000 7f30cb400640 1 --2- 172.21.15.17:0/4094280781 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f30d41f58d0 0x7f30d41f35e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f30c400f770 tx=0x7f30c400fc30 comp rx=0 tx=0).stop 2024-09-10T21:33:04.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:04.101+0000 7f30cb400640 1 -- 172.21.15.17:0/4094280781 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f30d41f58d0 msgr2=0x7f30d41f35e0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:33:04.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:04.101+0000 7f30cb400640 1 --2- 172.21.15.17:0/4094280781 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f30d41f58d0 0x7f30d41f35e0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:04.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:04.101+0000 7f30d2a00640 1 --2- 172.21.15.17:0/4094280781 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f30b40725e0 0x7f30b4074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:04.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:04.105+0000 7f30d2a00640 1 --2- 172.21.15.17:0/4094280781 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f30b40725e0 0x7f30b4074aa0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f30c00af000 tx=0x7f30c001b060 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:04.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:04.109+0000 7f30da200640 1 --2- 172.21.15.17:0/4094280781 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f30d41f3b20 0x7f30d41f4540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:04.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:04.109+0000 7f30da200640 1 -- 172.21.15.17:0/4094280781 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f30d414b0e0 con 0x7f30d41f3b20 2024-09-10T21:33:04.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:04.109+0000 7f30d3400640 1 --2- 172.21.15.17:0/4094280781 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f30d41f3b20 0x7f30d41f4540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:04.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:04.109+0000 7f30d3400640 1 -- 172.21.15.17:0/4094280781 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f30d4074e50 con 0x7f30d41f3b20 2024-09-10T21:33:04.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:04.109+0000 7f30d3400640 1 --2- 172.21.15.17:0/4094280781 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f30d41f3b20 0x7f30d41f4540 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f30c400f770 tx=0x7f30c4000ee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:04.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:04.113+0000 7f30cb400640 1 -- 172.21.15.17:0/4094280781 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30c4007460 con 0x7f30d41f3b20 2024-09-10T21:33:04.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:04.113+0000 7f30cb400640 1 -- 172.21.15.17:0/4094280781 <== mon.2 v2:172.21.15.31:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4709 (secure 0 0 0) 0x7f30c4007460 con 0x7f30d41f3b20 2024-09-10T21:33:04.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:04.113+0000 7f30cb400640 1 -- 172.21.15.17:0/4094280781 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f30d41f3b20 msgr2=0x7f30d41f4540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:04.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:04.113+0000 7f30cb400640 1 --2- 172.21.15.17:0/4094280781 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f30d41f3b20 0x7f30d41f4540 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f30c400f770 tx=0x7f30c4000ee0 comp rx=0 tx=0).stop 2024-09-10T21:33:04.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:04.113+0000 7f30cb400640 1 -- 172.21.15.17:0/4094280781 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f30d41f3b20 msgr2=0x7f30d41f4540 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:33:04.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:04.113+0000 7f30cb400640 1 --2- 172.21.15.17:0/4094280781 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f30d41f3b20 0x7f30d41f4540 secure :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f30c400f770 tx=0x7f30c4000ee0 comp rx=0 tx=0).stop 2024-09-10T21:33:04.117 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-10T21:33:04.117 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:33:04.117 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:33:04.117 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-10T21:33:04.117 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 70, 2024-09-10T21:33:04.117 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 116418, 2024-09-10T21:33:04.118 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-10T21:33:04.118 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-10T21:33:04.118 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-10T21:33:04.118 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-10T21:33:04.118 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:33:04.118 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 15, 2024-09-10T21:33:04.118 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:33:04.118 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-10T21:33:04.118 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-10T21:33:04.118 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:33:04.119 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:33:04.119 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:33:04.119 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:33:04.119 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:33:04.119 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:33:04.119 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:33:04.119 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:33:04.119 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:33:04.119 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:33:04.119 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:33:04.120 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:33:04.120 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-10T21:33:04.120 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-10T21:33:04.120 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:33:04.120 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:33:04.120 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:33:04.120 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:33:04.120 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:33:04.121 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:33:04.121 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:33:04.121 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:33:04.121 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:33:04.121 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:33:04.121 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:33:04.122 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:33:04.122 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:33:04.122 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-10T21:33:04.122 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-10T21:33:04.122 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-10T21:33:04.122 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-10T21:33:04.122 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-10T21:33:04.122 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "5e2f023f-d6a6-49f4-a256-f4807dce3739", 2024-09-10T21:33:04.122 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:33:04.122 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:33:04.123 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-10T21:33:04.123 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-10T21:33:04.123 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-10T21:33:04.123 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-10T21:33:04.123 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-10T21:33:04.123 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-10T21:33:04.123 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-10T21:33:04.123 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:33:04.123 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-10T21:33:04.123 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:33:04.123 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:33:04.123 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:33:04.124 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:33:04.124 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:33:04.124 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:33:04.124 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:33:04.124 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:33:04.125 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:33:04.125 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:33:04.125 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:33:04.125 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:33:04.126 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-10T21:33:04.126 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:33:04.126 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-10T21:33:04.126 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:33:04.126 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:33:04.127 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:33:04.127 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:33:04.127 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:33:04.127 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:33:04.127 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:33:04.127 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-10T21:33:04.127 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:33:04.128 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:33:04.128 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:33:04.128 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:33:04.128 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-10T21:33:04.128 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:33:04.128 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:33:04.129 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:33:04.129 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:33:04.129 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-10T21:33:04.129 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-10T21:33:04.129 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:33:04.130 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:33:04.130 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:33:04.130 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:33:04.130 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:33:04.130 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:33:04.130 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:33:04.131 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:33:04.131 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:33:04.131 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:33:04.131 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:33:04.131 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3300", 2024-09-10T21:33:04.131 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:33:04.131 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:33:04.131 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:33:04.131 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:33:04.131 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789", 2024-09-10T21:33:04.131 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:33:04.132 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:33:04.132 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:33:04.132 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:33:04.132 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789/0", 2024-09-10T21:33:04.132 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6789/0", 2024-09-10T21:33:04.132 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:33:04.132 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:33:04.132 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:33:04.132 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:33:04.132 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:33:04.133 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:33:04.133 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:33:04.133 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:33:04.133 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:33:04.133 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:33:04.133 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:33:04.133 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3301", 2024-09-10T21:33:04.133 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:33:04.133 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:33:04.133 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:33:04.133 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:33:04.134 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790", 2024-09-10T21:33:04.134 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:33:04.134 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:33:04.134 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:33:04.134 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:33:04.134 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790/0", 2024-09-10T21:33:04.134 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6790/0", 2024-09-10T21:33:04.134 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:33:04.134 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:33:04.134 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:33:04.135 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:33:04.135 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:33:04.135 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:33:04.135 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-10T21:33:04.135 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-10T21:33:04.135 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:33:04.135 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:33:04.135 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:33:04.135 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:33:04.135 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:33:04.135 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:33:04.135 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-10T21:33:04.135 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:33:04.136 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:33:04.136 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:33:04.136 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:33:04.136 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:33:04.136 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:33:04.136 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-10T21:33:04.136 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:33:04.136 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:33:04.136 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:33:04.136 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:33:04.137 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:33:04.137 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:33:04.137 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:33:04.137 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-10T21:33:04.137 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-10T21:33:04.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:04.113+0000 7f30da200640 1 -- 172.21.15.17:0/4094280781 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f30b40725e0 msgr2=0x7f30b4074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:04.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:04.113+0000 7f30da200640 1 --2- 172.21.15.17:0/4094280781 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f30b40725e0 0x7f30b4074aa0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f30c00af000 tx=0x7f30c001b060 comp rx=0 tx=0).stop 2024-09-10T21:33:04.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:04.113+0000 7f30da200640 1 -- 172.21.15.17:0/4094280781 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f30d415abb0 msgr2=0x7f30d4142330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:04.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:04.113+0000 7f30da200640 1 --2- 172.21.15.17:0/4094280781 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f30d415abb0 0x7f30d4142330 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f30cc00cb10 tx=0x7f30cc00cc00 comp rx=0 tx=0).stop 2024-09-10T21:33:04.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:04.113+0000 7f30d3e00640 1 -- 172.21.15.17:0/4094280781 reap_dead start 2024-09-10T21:33:04.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:04.113+0000 7f30d3e00640 1 -- 172.21.15.17:0/4094280781 reap_dead start 2024-09-10T21:33:04.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:04.113+0000 7f30da200640 1 -- 172.21.15.17:0/4094280781 shutdown_connections 2024-09-10T21:33:04.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:04.113+0000 7f30da200640 1 -- 172.21.15.17:0/4094280781 >> 172.21.15.17:0/4094280781 conn(0x7f30d40700d0 msgr2=0x7f30d40c7390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:04.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:04.113+0000 7f30da200640 1 -- 172.21.15.17:0/4094280781 shutdown_connections 2024-09-10T21:33:04.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:04.113+0000 7f30da200640 1 -- 172.21.15.17:0/4094280781 wait complete. 2024-09-10T21:33:04.140 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-10T21:33:04.140 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['a'] 2024-09-10T21:33:04.140 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2024-09-10T21:33:04.140 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.a 2024-09-10T21:33:04.140 INFO:tasks.mon_thrash.mon_thrasher:killing mon.a 2024-09-10T21:33:04.140 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-10T21:33:04.140 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:33:04.159 INFO:tasks.ceph.mon.a:Stopped 2024-09-10T21:33:04.159 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2024-09-10T21:33:04.160 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-10T21:33:04.376 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:04.369+0000 7fa842a00640 1 -- 172.21.15.17:0/687179899 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa83c0c76a0 msgr2=0x7fa83c0cc6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:04.376 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:04.369+0000 7fa842a00640 1 --2- 172.21.15.17:0/687179899 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa83c0c76a0 0x7fa83c0cc6d0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fa83c0701b0 tx=0x7fa83001f0b0 comp rx=0 tx=0).stop 2024-09-10T21:33:04.376 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:04.369+0000 7fa842a00640 1 -- 172.21.15.17:0/687179899 shutdown_connections 2024-09-10T21:33:04.376 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:04.369+0000 7fa842a00640 1 --2- 172.21.15.17:0/687179899 >> v2:172.21.15.17:3300/0 conn(0x7fa83c15b2a0 0x7fa83c0c7a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:04.376 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:04.369+0000 7fa842a00640 1 --2- 172.21.15.17:0/687179899 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa83c0c76a0 0x7fa83c0cc6d0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:04.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:04.369+0000 7fa842a00640 1 -- 172.21.15.17:0/687179899 >> 172.21.15.17:0/687179899 conn(0x7fa83c0cbbb0 msgr2=0x7fa83c0cbfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:04.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:04.369+0000 7fa842a00640 1 -- 172.21.15.17:0/687179899 shutdown_connections 2024-09-10T21:33:04.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:04.369+0000 7fa842a00640 1 -- 172.21.15.17:0/687179899 wait complete. 2024-09-10T21:33:04.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:04.373+0000 7fa842a00640 1 Processor -- start 2024-09-10T21:33:04.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:04.373+0000 7fa842a00640 1 -- start start 2024-09-10T21:33:04.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:04.373+0000 7fa842a00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa83c0704c0 0x7fa83c14e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:04.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:04.373+0000 7fa842a00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa83c0c76a0 0x7fa83c154f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:04.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:04.373+0000 7fa842a00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa83c15b2a0 0x7fa83c155480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:04.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:04.373+0000 7fa842a00640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fa83c0b1de0 con 0x7fa83c15b2a0 2024-09-10T21:33:04.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:04.373+0000 7fa842a00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fa83c15b6a0 con 0x7fa83c0704c0 2024-09-10T21:33:04.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:04.373+0000 7fa842a00640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7fa83c157e00 con 0x7fa83c0c76a0 2024-09-10T21:33:04.381 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:04.373+0000 7fa842000640 1 -- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa83c15b2a0 msgr2=0x7fa83c155480 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-10T21:33:04.381 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:04.373+0000 7fa842000640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa83c15b2a0 0x7fa83c155480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:33:04.381 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:04.373+0000 7fa841600640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa83c0704c0 0x7fa83c14e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:04.381 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:04.373+0000 7fa840c00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa83c0c76a0 0x7fa83c154f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:04.381 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:04.373+0000 7fa841600640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa83c0704c0 0x7fa83c14e270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.17:33632/0 (socket says 172.21.15.17:33632) 2024-09-10T21:33:04.381 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:04.373+0000 7fa841600640 1 -- 172.21.15.17:0/931561600 learned_addr learned my addr 172.21.15.17:0/931561600 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:33:04.381 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:04.377+0000 7fa841600640 1 -- 172.21.15.17:0/931561600 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa83c0c76a0 msgr2=0x7fa83c154f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:04.381 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:04.377+0000 7fa841600640 1 --2- 172.21.15.17:0/931561600 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa83c0c76a0 0x7fa83c154f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:04.381 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:04.377+0000 7fa841600640 1 -- 172.21.15.17:0/931561600 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa83c15b2a0 msgr2=0x7fa83c155480 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:33:04.381 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:04.377+0000 7fa841600640 1 --2- 172.21.15.17:0/931561600 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa83c15b2a0 0x7fa83c155480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:04.381 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:04.377+0000 7fa841600640 1 -- 172.21.15.17:0/931561600 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa83c150670 con 0x7fa83c0704c0 2024-09-10T21:33:04.381 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:04.377+0000 7fa841600640 1 --2- 172.21.15.17:0/931561600 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa83c0704c0 0x7fa83c14e270 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fa8300b3040 tx=0x7fa8300a8de0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:04.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:04.377+0000 7fa839600640 1 -- 172.21.15.17:0/931561600 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8300b5040 con 0x7fa83c0704c0 2024-09-10T21:33:04.383 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:04.377+0000 7fa839600640 1 -- 172.21.15.17:0/931561600 <== mon.1 v2:172.21.15.31:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa830005320 con 0x7fa83c0704c0 2024-09-10T21:33:04.383 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:04.377+0000 7fa842a00640 1 -- 172.21.15.17:0/931561600 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa83c14e7b0 con 0x7fa83c0704c0 2024-09-10T21:33:04.383 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:04.377+0000 7fa839600640 1 -- 172.21.15.17:0/931561600 <== mon.1 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8300046d0 con 0x7fa83c0704c0 2024-09-10T21:33:04.383 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:04.377+0000 7fa842a00640 1 -- 172.21.15.17:0/931561600 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa83c0c21d0 con 0x7fa83c0704c0 2024-09-10T21:33:04.383 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:04.377+0000 7fa842a00640 1 -- 172.21.15.17:0/931561600 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa83c074dc0 con 0x7fa83c0704c0 2024-09-10T21:33:04.383 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:04.377+0000 7fa839600640 1 -- 172.21.15.17:0/931561600 <== mon.1 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fa830004890 con 0x7fa83c0704c0 2024-09-10T21:33:04.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:04.381+0000 7fa839600640 1 --2- 172.21.15.17:0/931561600 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fa81c072610 0x7fa81c074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:04.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:04.381+0000 7fa839600640 1 -- 172.21.15.17:0/931561600 <== mon.1 v2:172.21.15.31:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fa830130820 con 0x7fa83c0704c0 2024-09-10T21:33:04.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:04.381+0000 7fa840c00640 1 --2- 172.21.15.17:0/931561600 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fa81c072610 0x7fa81c074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:04.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:04.381+0000 7fa840c00640 1 --2- 172.21.15.17:0/931561600 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fa81c072610 0x7fa81c074ad0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fa828006b10 tx=0x7fa828006aa0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:04.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:04.385+0000 7fa839600640 1 -- 172.21.15.17:0/931561600 <== mon.1 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fa830018c10 con 0x7fa83c0704c0 2024-09-10T21:33:04.732 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:04.725+0000 7fa842a00640 1 -- 172.21.15.17:0/931561600 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fa83c0c6060 con 0x7fa83c0704c0 2024-09-10T21:33:04.732 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:04.729+0000 7fa839600640 1 -- 172.21.15.17:0/931561600 <== mon.1 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1557 (secure 0 0 0) 0x7fa8300fbaf0 con 0x7fa83c0704c0 2024-09-10T21:33:04.733 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":70,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":16,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5e2f023f-d6a6-49f4-a256-f4807dce3739","modified":"2024-09-10T21:23:46.892130Z","created":"2024-09-10T21:23:46.892130Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3301","nonce":0},{"type":"v1","addr":"172.21.15.31:6790","nonce":0}]},"addr":"172.21.15.31:6790/0","public_addr":"172.21.15.31:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-10T21:33:04.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:04.729+0000 7fa842a00640 1 -- 172.21.15.17:0/931561600 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fa81c072610 msgr2=0x7fa81c074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:04.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:04.729+0000 7fa842a00640 1 --2- 172.21.15.17:0/931561600 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fa81c072610 0x7fa81c074ad0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fa828006b10 tx=0x7fa828006aa0 comp rx=0 tx=0).stop 2024-09-10T21:33:04.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:04.729+0000 7fa842a00640 1 -- 172.21.15.17:0/931561600 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa83c0704c0 msgr2=0x7fa83c14e270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:04.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:04.729+0000 7fa842a00640 1 --2- 172.21.15.17:0/931561600 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa83c0704c0 0x7fa83c14e270 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fa8300b3040 tx=0x7fa8300a8de0 comp rx=0 tx=0).stop 2024-09-10T21:33:04.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:04.729+0000 7fa842a00640 1 -- 172.21.15.17:0/931561600 shutdown_connections 2024-09-10T21:33:04.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:04.729+0000 7fa842a00640 1 --2- 172.21.15.17:0/931561600 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fa81c072610 0x7fa81c074ad0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:04.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:04.729+0000 7fa842a00640 1 --2- 172.21.15.17:0/931561600 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa83c15b2a0 0x7fa83c155480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:04.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:04.729+0000 7fa842a00640 1 --2- 172.21.15.17:0/931561600 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa83c0c76a0 0x7fa83c154f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:04.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:04.729+0000 7fa842a00640 1 --2- 172.21.15.17:0/931561600 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa83c0704c0 0x7fa83c14e270 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:04.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:04.729+0000 7fa842a00640 1 -- 172.21.15.17:0/931561600 >> 172.21.15.17:0/931561600 conn(0x7fa83c0cbbb0 msgr2=0x7fa83c158a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:04.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:04.729+0000 7fa842a00640 1 -- 172.21.15.17:0/931561600 shutdown_connections 2024-09-10T21:33:04.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:04.729+0000 7fa842a00640 1 -- 172.21.15.17:0/931561600 wait complete. 2024-09-10T21:33:07.762 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-10T21:33:07.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:07.921+0000 7f2379e00640 1 -- 172.21.15.17:0/493627674 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f23741546c0 msgr2=0x7f23740ca430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:07.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:07.921+0000 7f2379e00640 1 --2- 172.21.15.17:0/493627674 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f23741546c0 0x7f23740ca430 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f2364009900 tx=0x7f236401c8d0 comp rx=0 tx=0).stop 2024-09-10T21:33:07.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:07.921+0000 7f2379e00640 1 -- 172.21.15.17:0/493627674 shutdown_connections 2024-09-10T21:33:07.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:07.921+0000 7f2379e00640 1 --2- 172.21.15.17:0/493627674 >> v2:172.21.15.17:3300/0 conn(0x7f23740c89d0 0x7f23740cdf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:07.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:07.921+0000 7f2379e00640 1 --2- 172.21.15.17:0/493627674 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f23741546c0 0x7f23740ca430 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:07.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:07.921+0000 7f2379e00640 1 -- 172.21.15.17:0/493627674 >> 172.21.15.17:0/493627674 conn(0x7f23740702f0 msgr2=0x7f2374070700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:07.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:07.921+0000 7f2379e00640 1 -- 172.21.15.17:0/493627674 shutdown_connections 2024-09-10T21:33:07.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:07.921+0000 7f2379e00640 1 -- 172.21.15.17:0/493627674 wait complete. 2024-09-10T21:33:07.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:07.925+0000 7f2379e00640 1 Processor -- start 2024-09-10T21:33:07.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:07.925+0000 7f2379e00640 1 -- start start 2024-09-10T21:33:07.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:07.925+0000 7f2379e00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f23740c89d0 0x7f2374068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:07.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:07.925+0000 7f2379e00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f23741546c0 0x7f2374154ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:07.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:07.925+0000 7f2379e00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2374154aa0 0x7f23741553e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:07.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:07.925+0000 7f2379e00640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f23740c9640 con 0x7f23740c89d0 2024-09-10T21:33:07.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:07.925+0000 7f2379e00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f2374072790 con 0x7f2374154aa0 2024-09-10T21:33:07.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:07.925+0000 7f2379e00640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f23740c97c0 con 0x7f23741546c0 2024-09-10T21:33:07.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:07.925+0000 7f2372a00640 1 -- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f23740c89d0 msgr2=0x7f2374068bf0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-10T21:33:07.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:07.925+0000 7f2372a00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f23740c89d0 0x7f2374068bf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:33:07.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:07.925+0000 7f2372000640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f23741546c0 0x7f2374154ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:07.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:07.925+0000 7f2372000640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f23741546c0 0x7f2374154ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3301/0 says I am v2:172.21.15.17:49492/0 (socket says 172.21.15.17:49492) 2024-09-10T21:33:07.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:07.925+0000 7f2372000640 1 -- 172.21.15.17:0/2412747043 learned_addr learned my addr 172.21.15.17:0/2412747043 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:33:07.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:07.925+0000 7f2373400640 1 --2- 172.21.15.17:0/2412747043 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2374154aa0 0x7f23741553e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:07.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:07.925+0000 7f2372000640 1 -- 172.21.15.17:0/2412747043 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2374154aa0 msgr2=0x7f23741553e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:07.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:07.925+0000 7f2372000640 1 --2- 172.21.15.17:0/2412747043 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2374154aa0 0x7f23741553e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:07.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:07.925+0000 7f2372000640 1 -- 172.21.15.17:0/2412747043 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f23740c89d0 msgr2=0x7f2374068bf0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:33:07.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:07.925+0000 7f2372000640 1 --2- 172.21.15.17:0/2412747043 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f23740c89d0 0x7f2374068bf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:07.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:07.925+0000 7f2372000640 1 -- 172.21.15.17:0/2412747043 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23741f37f0 con 0x7f23741546c0 2024-09-10T21:33:07.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:07.925+0000 7f2372000640 1 --2- 172.21.15.17:0/2412747043 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f23741546c0 0x7f2374154ea0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f236000b4a0 tx=0x7f236000b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:07.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:07.929+0000 7f236b400640 1 -- 172.21.15.17:0/2412747043 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2360014070 con 0x7f23741546c0 2024-09-10T21:33:07.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:07.929+0000 7f236b400640 1 -- 172.21.15.17:0/2412747043 <== mon.2 v2:172.21.15.31:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2360010700 con 0x7f23741546c0 2024-09-10T21:33:07.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:07.929+0000 7f236b400640 1 -- 172.21.15.17:0/2412747043 <== mon.2 v2:172.21.15.31:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2360010b50 con 0x7f23741546c0 2024-09-10T21:33:07.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:07.929+0000 7f2379e00640 1 -- 172.21.15.17:0/2412747043 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2374159820 con 0x7f23741546c0 2024-09-10T21:33:07.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:07.929+0000 7f2379e00640 1 -- 172.21.15.17:0/2412747043 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f23741f3ab0 con 0x7f23741546c0 2024-09-10T21:33:07.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:07.929+0000 7f2379e00640 1 -- 172.21.15.17:0/2412747043 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23740cdf20 con 0x7f23741546c0 2024-09-10T21:33:07.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:07.929+0000 7f236b400640 1 -- 172.21.15.17:0/2412747043 <== mon.2 v2:172.21.15.31:3301/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f2360017360 con 0x7f23741546c0 2024-09-10T21:33:07.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:07.929+0000 7f236b400640 1 --2- 172.21.15.17:0/2412747043 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f2354072590 0x7f2354074a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:07.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:07.929+0000 7f236b400640 1 -- 172.21.15.17:0/2412747043 <== mon.2 v2:172.21.15.31:3301/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f2360093330 con 0x7f23741546c0 2024-09-10T21:33:07.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:07.933+0000 7f2372a00640 1 --2- 172.21.15.17:0/2412747043 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f2354072590 0x7f2354074a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:07.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:07.933+0000 7f2372a00640 1 --2- 172.21.15.17:0/2412747043 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f2354072590 0x7f2354074a50 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f23640af000 tx=0x7f236401b110 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:07.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:07.937+0000 7f236b400640 1 -- 172.21.15.17:0/2412747043 <== mon.2 v2:172.21.15.31:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f236005dfc0 con 0x7f23741546c0 2024-09-10T21:33:08.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:08.209+0000 7f2379e00640 1 -- 172.21.15.17:0/2412747043 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f2374072080 con 0x7f23741546c0 2024-09-10T21:33:08.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:08.209+0000 7f236b400640 1 -- 172.21.15.17:0/2412747043 <== mon.2 v2:172.21.15.31:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1557 (secure 0 0 0) 0x7f23600629f0 con 0x7f23741546c0 2024-09-10T21:33:08.213 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":70,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":19,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5e2f023f-d6a6-49f4-a256-f4807dce3739","modified":"2024-09-10T21:23:46.892130Z","created":"2024-09-10T21:23:46.892130Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3301","nonce":0},{"type":"v1","addr":"172.21.15.31:6790","nonce":0}]},"addr":"172.21.15.31:6790/0","public_addr":"172.21.15.31:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-10T21:33:08.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:08.209+0000 7f2379e00640 1 -- 172.21.15.17:0/2412747043 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f2354072590 msgr2=0x7f2354074a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:08.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:08.209+0000 7f2379e00640 1 --2- 172.21.15.17:0/2412747043 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f2354072590 0x7f2354074a50 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f23640af000 tx=0x7f236401b110 comp rx=0 tx=0).stop 2024-09-10T21:33:08.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:08.209+0000 7f2379e00640 1 -- 172.21.15.17:0/2412747043 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f23741546c0 msgr2=0x7f2374154ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:08.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:08.209+0000 7f2379e00640 1 --2- 172.21.15.17:0/2412747043 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f23741546c0 0x7f2374154ea0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f236000b4a0 tx=0x7f236000b960 comp rx=0 tx=0).stop 2024-09-10T21:33:08.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:08.209+0000 7f2379e00640 1 -- 172.21.15.17:0/2412747043 shutdown_connections 2024-09-10T21:33:08.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:08.209+0000 7f2379e00640 1 --2- 172.21.15.17:0/2412747043 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f2354072590 0x7f2354074a50 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:08.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:08.209+0000 7f2379e00640 1 --2- 172.21.15.17:0/2412747043 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2374154aa0 0x7f23741553e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:08.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:08.209+0000 7f2379e00640 1 --2- 172.21.15.17:0/2412747043 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f23741546c0 0x7f2374154ea0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:08.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:08.209+0000 7f2379e00640 1 --2- 172.21.15.17:0/2412747043 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f23740c89d0 0x7f2374068bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:08.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:08.209+0000 7f2379e00640 1 -- 172.21.15.17:0/2412747043 >> 172.21.15.17:0/2412747043 conn(0x7f23740702f0 msgr2=0x7f23740c9a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:08.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:08.213+0000 7f2379e00640 1 -- 172.21.15.17:0/2412747043 shutdown_connections 2024-09-10T21:33:08.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:08.213+0000 7f2379e00640 1 -- 172.21.15.17:0/2412747043 wait complete. 2024-09-10T21:33:11.237 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-10T21:33:11.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:11.393+0000 7f3101600640 1 -- 172.21.15.17:0/25986548 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f30fc1524d0 msgr2=0x7f30fc15bf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:11.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:11.393+0000 7f3101600640 1 --2- 172.21.15.17:0/25986548 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f30fc1524d0 0x7f30fc15bf00 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f30fc15b910 tx=0x7f30ec01eef0 comp rx=0 tx=0).stop 2024-09-10T21:33:11.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:11.393+0000 7f3101600640 1 -- 172.21.15.17:0/25986548 shutdown_connections 2024-09-10T21:33:11.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:11.393+0000 7f3101600640 1 --2- 172.21.15.17:0/25986548 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f30fc1524d0 0x7f30fc15bf00 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:11.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:11.393+0000 7f3101600640 1 -- 172.21.15.17:0/25986548 >> 172.21.15.17:0/25986548 conn(0x7f30fc0cdef0 msgr2=0x7f30fc0ce2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:11.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:11.393+0000 7f3101600640 1 -- 172.21.15.17:0/25986548 shutdown_connections 2024-09-10T21:33:11.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:11.393+0000 7f3101600640 1 -- 172.21.15.17:0/25986548 wait complete. 2024-09-10T21:33:11.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:11.397+0000 7f3101600640 1 Processor -- start 2024-09-10T21:33:11.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:11.397+0000 7f3101600640 1 -- start start 2024-09-10T21:33:11.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:11.397+0000 7f3101600640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f30fc1524d0 0x7f30fc148160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:11.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:11.397+0000 7f3101600640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f30fc158c20 0x7f30fc15af10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:11.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:11.397+0000 7f3101600640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f30fc15c440 0x7f30fc070b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:11.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:11.397+0000 7f3101600640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f30fc0c9350 con 0x7f30fc15c440 2024-09-10T21:33:11.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:11.397+0000 7f3101600640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f30fc0c9070 con 0x7f30fc158c20 2024-09-10T21:33:11.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:11.397+0000 7f3101600640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f30fc15b9e0 con 0x7f30fc1524d0 2024-09-10T21:33:11.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:11.397+0000 7f3100c00640 1 -- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f30fc15c440 msgr2=0x7f30fc070b20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-10T21:33:11.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:11.397+0000 7f3100c00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f30fc15c440 0x7f30fc070b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:33:11.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:11.397+0000 7f30fb400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f30fc158c20 0x7f30fc15af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:11.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:11.397+0000 7f30fbe00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f30fc1524d0 0x7f30fc148160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:11.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:11.397+0000 7f30fb400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f30fc158c20 0x7f30fc15af10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.17:38468/0 (socket says 172.21.15.17:38468) 2024-09-10T21:33:11.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:11.397+0000 7f30fb400640 1 -- 172.21.15.17:0/3141910432 learned_addr learned my addr 172.21.15.17:0/3141910432 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:33:11.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:11.397+0000 7f30fb400640 1 -- 172.21.15.17:0/3141910432 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f30fc1524d0 msgr2=0x7f30fc148160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:11.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:11.397+0000 7f30fb400640 1 --2- 172.21.15.17:0/3141910432 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f30fc1524d0 0x7f30fc148160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:11.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:11.397+0000 7f30fb400640 1 -- 172.21.15.17:0/3141910432 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f30fc15c440 msgr2=0x7f30fc070b20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:33:11.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:11.397+0000 7f30fb400640 1 --2- 172.21.15.17:0/3141910432 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f30fc15c440 0x7f30fc070b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:11.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:11.397+0000 7f30fb400640 1 -- 172.21.15.17:0/3141910432 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30fc0c40b0 con 0x7f30fc158c20 2024-09-10T21:33:11.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:11.397+0000 7f30fb400640 1 --2- 172.21.15.17:0/3141910432 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f30fc158c20 0x7f30fc15af10 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f30e800b5b0 tx=0x7f30e800ba70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:11.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:11.397+0000 7f30f3e00640 1 -- 172.21.15.17:0/3141910432 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30e800f870 con 0x7f30fc158c20 2024-09-10T21:33:11.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:11.397+0000 7f30f3e00640 1 -- 172.21.15.17:0/3141910432 <== mon.1 v2:172.21.15.31:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f30e800fb20 con 0x7f30fc158c20 2024-09-10T21:33:11.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:11.397+0000 7f3101600640 1 -- 172.21.15.17:0/3141910432 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30fc1f0cb0 con 0x7f30fc158c20 2024-09-10T21:33:11.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:11.397+0000 7f30f3e00640 1 -- 172.21.15.17:0/3141910432 <== mon.1 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30e8017400 con 0x7f30fc158c20 2024-09-10T21:33:11.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:11.397+0000 7f3101600640 1 -- 172.21.15.17:0/3141910432 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30fc1f0f50 con 0x7f30fc158c20 2024-09-10T21:33:11.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:11.397+0000 7f3101600640 1 -- 172.21.15.17:0/3141910432 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30fc0b3e00 con 0x7f30fc158c20 2024-09-10T21:33:11.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:11.401+0000 7f30f3e00640 1 -- 172.21.15.17:0/3141910432 <== mon.1 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f30e800d400 con 0x7f30fc158c20 2024-09-10T21:33:11.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:11.401+0000 7f30f3e00640 1 --2- 172.21.15.17:0/3141910432 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f30dc0725c0 0x7f30dc074a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:11.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:11.401+0000 7f30f3e00640 1 -- 172.21.15.17:0/3141910432 <== mon.1 v2:172.21.15.31:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f30e80937a0 con 0x7f30fc158c20 2024-09-10T21:33:11.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:11.401+0000 7f30fbe00640 1 --2- 172.21.15.17:0/3141910432 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f30dc0725c0 0x7f30dc074a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:11.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:11.401+0000 7f30fbe00640 1 --2- 172.21.15.17:0/3141910432 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f30dc0725c0 0x7f30dc074a80 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f30ec0b2000 tx=0x7f30ec01d730 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:11.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:11.405+0000 7f30f3e00640 1 -- 172.21.15.17:0/3141910432 <== mon.1 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f30e8013030 con 0x7f30fc158c20 2024-09-10T21:33:11.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:11.685+0000 7f3101600640 1 -- 172.21.15.17:0/3141910432 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f30fc050fd0 con 0x7f30fc158c20 2024-09-10T21:33:11.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:11.685+0000 7f30f3e00640 1 -- 172.21.15.17:0/3141910432 <== mon.1 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1557 (secure 0 0 0) 0x7f30e805e9e0 con 0x7f30fc158c20 2024-09-10T21:33:11.692 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":70,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":23,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5e2f023f-d6a6-49f4-a256-f4807dce3739","modified":"2024-09-10T21:23:46.892130Z","created":"2024-09-10T21:23:46.892130Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3301","nonce":0},{"type":"v1","addr":"172.21.15.31:6790","nonce":0}]},"addr":"172.21.15.31:6790/0","public_addr":"172.21.15.31:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-10T21:33:11.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:11.689+0000 7f3101600640 1 -- 172.21.15.17:0/3141910432 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f30dc0725c0 msgr2=0x7f30dc074a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:11.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:11.689+0000 7f3101600640 1 --2- 172.21.15.17:0/3141910432 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f30dc0725c0 0x7f30dc074a80 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f30ec0b2000 tx=0x7f30ec01d730 comp rx=0 tx=0).stop 2024-09-10T21:33:11.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:11.689+0000 7f3101600640 1 -- 172.21.15.17:0/3141910432 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f30fc158c20 msgr2=0x7f30fc15af10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:11.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:11.689+0000 7f3101600640 1 --2- 172.21.15.17:0/3141910432 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f30fc158c20 0x7f30fc15af10 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f30e800b5b0 tx=0x7f30e800ba70 comp rx=0 tx=0).stop 2024-09-10T21:33:11.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:11.689+0000 7f3101600640 1 -- 172.21.15.17:0/3141910432 shutdown_connections 2024-09-10T21:33:11.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:11.689+0000 7f3101600640 1 --2- 172.21.15.17:0/3141910432 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f30dc0725c0 0x7f30dc074a80 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:11.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:11.689+0000 7f3101600640 1 --2- 172.21.15.17:0/3141910432 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f30fc15c440 0x7f30fc070b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:11.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:11.689+0000 7f3101600640 1 --2- 172.21.15.17:0/3141910432 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f30fc158c20 0x7f30fc15af10 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:11.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:11.689+0000 7f3101600640 1 --2- 172.21.15.17:0/3141910432 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f30fc1524d0 0x7f30fc148160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:11.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:11.689+0000 7f3101600640 1 -- 172.21.15.17:0/3141910432 >> 172.21.15.17:0/3141910432 conn(0x7f30fc0cdef0 msgr2=0x7f30fc14f570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:11.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:11.689+0000 7f3101600640 1 -- 172.21.15.17:0/3141910432 shutdown_connections 2024-09-10T21:33:11.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:11.689+0000 7f3101600640 1 -- 172.21.15.17:0/3141910432 wait complete. 2024-09-10T21:33:14.718 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-10T21:33:18.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:18.585+0000 7fde93000640 1 -- 172.21.15.17:0/4199185749 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fde740039c0 msgr2=0x7fde74003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:18.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:18.585+0000 7fde93000640 1 --2- 172.21.15.17:0/4199185749 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fde740039c0 0x7fde74003dc0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fde800010b0 tx=0x7fde8001f700 comp rx=0 tx=0).stop 2024-09-10T21:33:18.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:18.585+0000 7fde93000640 1 -- 172.21.15.17:0/4199185749 shutdown_connections 2024-09-10T21:33:18.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:18.585+0000 7fde93000640 1 --2- 172.21.15.17:0/4199185749 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fde740052f0 0x7fde7400b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:18.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:18.585+0000 7fde93000640 1 --2- 172.21.15.17:0/4199185749 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde74004810 0x7fde74004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:18.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:18.585+0000 7fde93000640 1 --2- 172.21.15.17:0/4199185749 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fde740039c0 0x7fde74003dc0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:18.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:18.585+0000 7fde93000640 1 -- 172.21.15.17:0/4199185749 >> 172.21.15.17:0/4199185749 conn(0x7fde8c070ca0 msgr2=0x7fde8c0710b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:18.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:18.589+0000 7fde93000640 1 -- 172.21.15.17:0/4199185749 shutdown_connections 2024-09-10T21:33:18.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:18.589+0000 7fde93000640 1 -- 172.21.15.17:0/4199185749 wait complete. 2024-09-10T21:33:18.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:18.589+0000 7fde93000640 1 Processor -- start 2024-09-10T21:33:18.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:18.589+0000 7fde93000640 1 -- start start 2024-09-10T21:33:18.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:18.589+0000 7fde93000640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fde740039c0 0x7fde8c0c5020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:18.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:18.589+0000 7fde93000640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde74004810 0x7fde8c0c5a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:18.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:18.589+0000 7fde93000640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fde740052f0 0x7fde8c1dd710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:18.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:18.589+0000 7fde93000640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fde8c0c7870 con 0x7fde740052f0 2024-09-10T21:33:18.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:18.589+0000 7fde93000640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fde8c06ff30 con 0x7fde74004810 2024-09-10T21:33:18.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:18.589+0000 7fde93000640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7fde8c0c79f0 con 0x7fde740039c0 2024-09-10T21:33:18.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:18.589+0000 7fde92600640 1 -- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fde740052f0 msgr2=0x7fde8c1dd710 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-10T21:33:18.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:18.589+0000 7fde92600640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fde740052f0 0x7fde8c1dd710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:33:18.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:18.589+0000 7fde91c00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fde740039c0 0x7fde8c0c5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:18.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:18.589+0000 7fde91c00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fde740039c0 0x7fde8c0c5020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3301/0 says I am v2:172.21.15.17:51206/0 (socket says 172.21.15.17:51206) 2024-09-10T21:33:18.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:18.589+0000 7fde91c00640 1 -- 172.21.15.17:0/3737162231 learned_addr learned my addr 172.21.15.17:0/3737162231 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:33:18.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:18.589+0000 7fde91200640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde74004810 0x7fde8c0c5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:18.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:18.593+0000 7fde91c00640 1 -- 172.21.15.17:0/3737162231 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde74004810 msgr2=0x7fde8c0c5a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:18.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:18.593+0000 7fde91c00640 1 --2- 172.21.15.17:0/3737162231 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde74004810 0x7fde8c0c5a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:18.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:18.593+0000 7fde91c00640 1 -- 172.21.15.17:0/3737162231 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fde740052f0 msgr2=0x7fde8c1dd710 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:33:18.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:18.593+0000 7fde91c00640 1 --2- 172.21.15.17:0/3737162231 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fde740052f0 0x7fde8c1dd710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:18.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:18.593+0000 7fde91c00640 1 -- 172.21.15.17:0/3737162231 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde8c0c6d50 con 0x7fde740039c0 2024-09-10T21:33:18.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:18.593+0000 7fde91c00640 1 --2- 172.21.15.17:0/3737162231 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fde740039c0 0x7fde8c0c5020 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fde80026410 tx=0x7fde80000f80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:18.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:18.593+0000 7fde89600640 1 -- 172.21.15.17:0/3737162231 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde80006110 con 0x7fde740039c0 2024-09-10T21:33:18.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:18.593+0000 7fde89600640 1 -- 172.21.15.17:0/3737162231 <== mon.2 v2:172.21.15.31:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fde800be070 con 0x7fde740039c0 2024-09-10T21:33:18.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:18.593+0000 7fde93000640 1 -- 172.21.15.17:0/3737162231 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde8c0c6ab0 con 0x7fde740039c0 2024-09-10T21:33:18.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:18.593+0000 7fde89600640 1 -- 172.21.15.17:0/3737162231 <== mon.2 v2:172.21.15.31:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde80006600 con 0x7fde740039c0 2024-09-10T21:33:18.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:18.593+0000 7fde93000640 1 -- 172.21.15.17:0/3737162231 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fde8c1de100 con 0x7fde740039c0 2024-09-10T21:33:18.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:18.593+0000 7fde93000640 1 -- 172.21.15.17:0/3737162231 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde8c0b2c60 con 0x7fde740039c0 2024-09-10T21:33:18.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:18.593+0000 7fde89600640 1 -- 172.21.15.17:0/3737162231 <== mon.2 v2:172.21.15.31:3301/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fde800067a0 con 0x7fde740039c0 2024-09-10T21:33:18.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:18.597+0000 7fde89600640 1 --2- 172.21.15.17:0/3737162231 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fde6c072cb0 0x7fde6c075170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:18.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:18.597+0000 7fde89600640 1 -- 172.21.15.17:0/3737162231 <== mon.2 v2:172.21.15.31:3301/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fde80131920 con 0x7fde740039c0 2024-09-10T21:33:18.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:18.597+0000 7fde91200640 1 --2- 172.21.15.17:0/3737162231 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fde6c072cb0 0x7fde6c075170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:18.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:18.597+0000 7fde91200640 1 --2- 172.21.15.17:0/3737162231 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fde6c072cb0 0x7fde6c075170 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fde8c154470 tx=0x7fde78017040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:18.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:18.601+0000 7fde89600640 1 -- 172.21.15.17:0/3737162231 <== mon.2 v2:172.21.15.31:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fde800194b0 con 0x7fde740039c0 2024-09-10T21:33:18.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:18.841+0000 7fde93000640 1 -- 172.21.15.17:0/3737162231 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fde8c071ce0 con 0x7fde740039c0 2024-09-10T21:33:18.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:18.841+0000 7fde89600640 1 -- 172.21.15.17:0/3737162231 <== mon.2 v2:172.21.15.31:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1550 (secure 0 0 0) 0x7fde800fcc70 con 0x7fde740039c0 2024-09-10T21:33:18.846 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":72,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":0,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5e2f023f-d6a6-49f4-a256-f4807dce3739","modified":"2024-09-10T21:23:46.892130Z","created":"2024-09-10T21:23:46.892130Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3301","nonce":0},{"type":"v1","addr":"172.21.15.31:6790","nonce":0}]},"addr":"172.21.15.31:6790/0","public_addr":"172.21.15.31:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-10T21:33:18.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:18.845+0000 7fde93000640 1 -- 172.21.15.17:0/3737162231 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fde6c072cb0 msgr2=0x7fde6c075170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:18.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:18.845+0000 7fde93000640 1 --2- 172.21.15.17:0/3737162231 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fde6c072cb0 0x7fde6c075170 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fde8c154470 tx=0x7fde78017040 comp rx=0 tx=0).stop 2024-09-10T21:33:18.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:18.845+0000 7fde93000640 1 -- 172.21.15.17:0/3737162231 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fde740039c0 msgr2=0x7fde8c0c5020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:18.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:18.845+0000 7fde93000640 1 --2- 172.21.15.17:0/3737162231 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fde740039c0 0x7fde8c0c5020 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fde80026410 tx=0x7fde80000f80 comp rx=0 tx=0).stop 2024-09-10T21:33:18.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:18.845+0000 7fde93000640 1 -- 172.21.15.17:0/3737162231 shutdown_connections 2024-09-10T21:33:18.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:18.845+0000 7fde93000640 1 --2- 172.21.15.17:0/3737162231 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fde6c072cb0 0x7fde6c075170 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:18.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:18.845+0000 7fde93000640 1 --2- 172.21.15.17:0/3737162231 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fde740052f0 0x7fde8c1dd710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:18.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:18.845+0000 7fde93000640 1 --2- 172.21.15.17:0/3737162231 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fde74004810 0x7fde8c0c5a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:18.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:18.845+0000 7fde93000640 1 --2- 172.21.15.17:0/3737162231 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fde740039c0 0x7fde8c0c5020 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:18.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:18.845+0000 7fde93000640 1 -- 172.21.15.17:0/3737162231 >> 172.21.15.17:0/3737162231 conn(0x7fde8c070ca0 msgr2=0x7fde8c0cacb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:18.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:18.845+0000 7fde93000640 1 -- 172.21.15.17:0/3737162231 shutdown_connections 2024-09-10T21:33:18.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:18.845+0000 7fde93000640 1 -- 172.21.15.17:0/3737162231 wait complete. 2024-09-10T21:33:18.868 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2024-09-10T21:33:18.868 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon b 2024-09-10T21:33:18.868 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-10T21:33:18.960 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:33:18.963 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:33:19.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:19.013+0000 7f8e14000640 1 -- 172.21.15.17:0/319366242 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8e0c152440 msgr2=0x7f8e0c0b3eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:19.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:19.013+0000 7f8e14000640 1 --2- 172.21.15.17:0/319366242 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8e0c152440 0x7f8e0c0b3eb0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f8e0c1594a0 tx=0x7f8e0001c870 comp rx=0 tx=0).stop 2024-09-10T21:33:19.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:19.013+0000 7f8e14000640 1 -- 172.21.15.17:0/319366242 shutdown_connections 2024-09-10T21:33:19.019 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:19.013+0000 7f8e14000640 1 --2- 172.21.15.17:0/319366242 >> v2:172.21.15.17:3300/0 conn(0x7f8e0c0cdf20 0x7f8e0c0ce350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:19.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:19.013+0000 7f8e14000640 1 --2- 172.21.15.17:0/319366242 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8e0c152440 0x7f8e0c0b3eb0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:19.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:19.013+0000 7f8e14000640 1 -- 172.21.15.17:0/319366242 >> 172.21.15.17:0/319366242 conn(0x7f8e0c0702f0 msgr2=0x7f8e0c070700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:19.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:19.013+0000 7f8e14000640 1 -- 172.21.15.17:0/319366242 shutdown_connections 2024-09-10T21:33:19.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:19.013+0000 7f8e14000640 1 -- 172.21.15.17:0/319366242 wait complete. 2024-09-10T21:33:19.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:19.017+0000 7f8e14000640 1 Processor -- start 2024-09-10T21:33:19.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:19.017+0000 7f8e14000640 1 -- start start 2024-09-10T21:33:19.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:19.017+0000 7f8e14000640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e0c0cdf20 0x7f8e0c159ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:19.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:19.017+0000 7f8e14000640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8e0c152440 0x7f8e0c06e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:19.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:19.017+0000 7f8e14000640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8e0c152820 0x7f8e0c0c7300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:19.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:19.017+0000 7f8e14000640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f8e0c0cb5d0 con 0x7f8e0c152820 2024-09-10T21:33:19.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:19.017+0000 7f8e14000640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f8e0c0cb450 con 0x7f8e0c0cdf20 2024-09-10T21:33:19.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:19.017+0000 7f8e14000640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f8e0c0c2ff0 con 0x7f8e0c152440 2024-09-10T21:33:19.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:19.017+0000 7f8e12000640 1 -- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8e0c152820 msgr2=0x7f8e0c0c7300 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-10T21:33:19.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:19.017+0000 7f8e12000640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8e0c152820 0x7f8e0c0c7300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:33:19.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:19.017+0000 7f8e11600640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e0c0cdf20 0x7f8e0c159ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:19.023 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:19.017+0000 7f8e10c00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8e0c152440 0x7f8e0c06e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:19.023 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:19.017+0000 7f8e11600640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e0c0cdf20 0x7f8e0c159ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.17:33980/0 (socket says 172.21.15.17:33980) 2024-09-10T21:33:19.023 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:19.017+0000 7f8e11600640 1 -- 172.21.15.17:0/3020651238 learned_addr learned my addr 172.21.15.17:0/3020651238 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:33:19.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:19.017+0000 7f8e11600640 1 -- 172.21.15.17:0/3020651238 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8e0c152440 msgr2=0x7f8e0c06e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:19.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:19.017+0000 7f8e11600640 1 --2- 172.21.15.17:0/3020651238 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8e0c152440 0x7f8e0c06e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:19.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:19.017+0000 7f8e11600640 1 -- 172.21.15.17:0/3020651238 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8e0c152820 msgr2=0x7f8e0c0c7300 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:33:19.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:19.017+0000 7f8e11600640 1 --2- 172.21.15.17:0/3020651238 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8e0c152820 0x7f8e0c0c7300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:19.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:19.017+0000 7f8e11600640 1 -- 172.21.15.17:0/3020651238 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e0c0ccf10 con 0x7f8e0c0cdf20 2024-09-10T21:33:19.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:19.017+0000 7f8e11600640 1 --2- 172.21.15.17:0/3020651238 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e0c0cdf20 0x7f8e0c159ce0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f8e000b0040 tx=0x7f8e000040c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:19.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:19.017+0000 7f8e09600640 1 -- 172.21.15.17:0/3020651238 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e000bb070 con 0x7f8e0c0cdf20 2024-09-10T21:33:19.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:19.017+0000 7f8e09600640 1 -- 172.21.15.17:0/3020651238 <== mon.1 v2:172.21.15.31:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8e00005070 con 0x7f8e0c0cdf20 2024-09-10T21:33:19.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:19.017+0000 7f8e14000640 1 -- 172.21.15.17:0/3020651238 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e0c15af70 con 0x7f8e0c0cdf20 2024-09-10T21:33:19.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:19.017+0000 7f8e09600640 1 -- 172.21.15.17:0/3020651238 <== mon.1 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e000b2040 con 0x7f8e0c0cdf20 2024-09-10T21:33:19.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:19.017+0000 7f8e14000640 1 -- 172.21.15.17:0/3020651238 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e0c1f78c0 con 0x7f8e0c0cdf20 2024-09-10T21:33:19.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:19.021+0000 7f8e14000640 1 --2- 172.21.15.17:0/3020651238 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e0c149890 0x7f8e0c149c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:19.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:19.021+0000 7f8e14000640 1 -- 172.21.15.17:0/3020651238 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f8e0c0c7840 con 0x7f8e0c149890 2024-09-10T21:33:19.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:19.021+0000 7f8e10c00640 1 --2- 172.21.15.17:0/3020651238 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e0c149890 0x7f8e0c149c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:19.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:19.021+0000 7f8e10c00640 1 -- 172.21.15.17:0/3020651238 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8e0c06f990 con 0x7f8e0c149890 2024-09-10T21:33:19.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:19.021+0000 7f8e10c00640 1 --2- 172.21.15.17:0/3020651238 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e0c149890 0x7f8e0c149c70 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f8df8000fc0 tx=0x7f8df800ec00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:19.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:19.021+0000 7f8e09600640 1 -- 172.21.15.17:0/3020651238 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8df8017070 con 0x7f8e0c149890 2024-09-10T21:33:19.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:19.021+0000 7f8e09600640 1 -- 172.21.15.17:0/3020651238 <== mon.1 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f8e00005480 con 0x7f8e0c0cdf20 2024-09-10T21:33:19.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:19.021+0000 7f8e09600640 1 --2- 172.21.15.17:0/3020651238 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f8dec0724e0 0x7f8dec0749a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:19.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:19.021+0000 7f8e09600640 1 -- 172.21.15.17:0/3020651238 <== mon.1 v2:172.21.15.31:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f8e0012ce00 con 0x7f8e0c0cdf20 2024-09-10T21:33:19.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:19.021+0000 7f8e12000640 1 --2- 172.21.15.17:0/3020651238 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f8dec0724e0 0x7f8dec0749a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:19.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:19.021+0000 7f8e09600640 1 -- 172.21.15.17:0/3020651238 <== mon.1 v2:172.21.15.31:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f8df800c770 con 0x7f8e0c149890 2024-09-10T21:33:19.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:19.021+0000 7f8e09600640 1 -- 172.21.15.17:0/3020651238 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e0c149890 msgr2=0x7f8e0c149c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:19.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:19.021+0000 7f8e09600640 1 --2- 172.21.15.17:0/3020651238 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e0c149890 0x7f8e0c149c70 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f8df8000fc0 tx=0x7f8df800ec00 comp rx=0 tx=0).stop 2024-09-10T21:33:19.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:19.021+0000 7f8e09600640 1 -- 172.21.15.17:0/3020651238 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e0c149890 msgr2=0x7f8e0c149c70 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:33:19.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:19.021+0000 7f8e09600640 1 --2- 172.21.15.17:0/3020651238 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e0c149890 0x7f8e0c149c70 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:19.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:19.021+0000 7f8e12000640 1 --2- 172.21.15.17:0/3020651238 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f8dec0724e0 0x7f8dec0749a0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f8e040013e0 tx=0x7f8e04009270 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:19.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:19.029+0000 7f8e14000640 1 --2- 172.21.15.17:0/3020651238 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e0c06e980 0x7f8e0c06ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:19.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:19.029+0000 7f8e14000640 1 -- 172.21.15.17:0/3020651238 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f8e0c14a1b0 con 0x7f8e0c06e980 2024-09-10T21:33:19.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:19.029+0000 7f8e10c00640 1 --2- 172.21.15.17:0/3020651238 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e0c06e980 0x7f8e0c06ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:19.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:19.029+0000 7f8e10c00640 1 -- 172.21.15.17:0/3020651238 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f8e0c1f7b00 con 0x7f8e0c06e980 2024-09-10T21:33:19.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:19.029+0000 7f8e10c00640 1 --2- 172.21.15.17:0/3020651238 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e0c06e980 0x7f8e0c06ed90 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f8df8000fc0 tx=0x7f8df8004810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:19.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:19.029+0000 7f8e09600640 1 -- 172.21.15.17:0/3020651238 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8df8006520 con 0x7f8e0c06e980 2024-09-10T21:33:19.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:19.029+0000 7f8e09600640 1 -- 172.21.15.17:0/3020651238 <== mon.1 v2:172.21.15.31:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5033 (secure 0 0 0) 0x7f8df8006520 con 0x7f8e0c06e980 2024-09-10T21:33:19.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:19.029+0000 7f8e09600640 1 -- 172.21.15.17:0/3020651238 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e0c06e980 msgr2=0x7f8e0c06ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:19.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:19.029+0000 7f8e09600640 1 --2- 172.21.15.17:0/3020651238 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e0c06e980 0x7f8e0c06ed90 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f8df8000fc0 tx=0x7f8df8004810 comp rx=0 tx=0).stop 2024-09-10T21:33:19.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:19.029+0000 7f8e09600640 1 -- 172.21.15.17:0/3020651238 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e0c06e980 msgr2=0x7f8e0c06ed90 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:33:19.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:19.029+0000 7f8e09600640 1 --2- 172.21.15.17:0/3020651238 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e0c06e980 0x7f8e0c06ed90 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:19.036 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-10T21:33:19.036 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:33:19.036 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:33:19.036 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "leader", 2024-09-10T21:33:19.036 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 72, 2024-09-10T21:33:19.036 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 537479, 2024-09-10T21:33:19.036 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-10T21:33:19.036 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-10T21:33:19.036 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-10T21:33:19.036 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:33:19.036 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 0, 2024-09-10T21:33:19.036 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:33:19.037 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-10T21:33:19.037 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-10T21:33:19.037 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:33:19.037 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:33:19.037 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:33:19.037 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:33:19.037 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:33:19.037 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:33:19.037 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:33:19.037 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:33:19.037 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:33:19.037 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:33:19.037 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:33:19.037 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:33:19.038 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-10T21:33:19.038 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-10T21:33:19.038 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:33:19.038 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:33:19.038 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:33:19.038 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:33:19.038 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:33:19.038 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:33:19.038 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:33:19.038 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:33:19.038 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:33:19.038 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:33:19.038 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:33:19.038 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:33:19.039 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:33:19.039 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-10T21:33:19.039 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-10T21:33:19.039 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-10T21:33:19.039 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-10T21:33:19.039 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-10T21:33:19.039 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "5e2f023f-d6a6-49f4-a256-f4807dce3739", 2024-09-10T21:33:19.039 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:33:19.039 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:33:19.039 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-10T21:33:19.039 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-10T21:33:19.039 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-10T21:33:19.039 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-10T21:33:19.039 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-10T21:33:19.040 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-10T21:33:19.040 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-10T21:33:19.040 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:33:19.040 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-10T21:33:19.040 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:33:19.040 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:33:19.040 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:33:19.040 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:33:19.040 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:33:19.040 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:33:19.040 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:33:19.040 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:33:19.040 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:33:19.041 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:33:19.041 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:33:19.041 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:33:19.041 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-10T21:33:19.041 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:33:19.041 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-10T21:33:19.041 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:33:19.041 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:33:19.041 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:33:19.041 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:33:19.041 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:33:19.041 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:33:19.041 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:33:19.041 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-10T21:33:19.042 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:33:19.042 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:33:19.042 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:33:19.042 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:33:19.042 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-10T21:33:19.042 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:33:19.042 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:33:19.042 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:33:19.042 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:33:19.042 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-10T21:33:19.042 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-10T21:33:19.042 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:33:19.042 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:33:19.043 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:33:19.043 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:33:19.043 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:33:19.043 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:33:19.043 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:33:19.043 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:33:19.043 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:33:19.043 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:33:19.043 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:33:19.043 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3300", 2024-09-10T21:33:19.043 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:33:19.043 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:33:19.043 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:33:19.043 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:33:19.044 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789", 2024-09-10T21:33:19.044 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:33:19.044 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:33:19.044 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:33:19.044 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:33:19.044 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789/0", 2024-09-10T21:33:19.044 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6789/0", 2024-09-10T21:33:19.044 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:33:19.044 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:33:19.044 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:33:19.044 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:33:19.044 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:33:19.044 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:33:19.045 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:33:19.045 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:33:19.045 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:33:19.045 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:33:19.045 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:33:19.045 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3301", 2024-09-10T21:33:19.045 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:33:19.045 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:33:19.045 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:33:19.045 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:33:19.045 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790", 2024-09-10T21:33:19.045 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:33:19.045 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:33:19.045 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:33:19.046 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:33:19.046 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790/0", 2024-09-10T21:33:19.046 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6790/0", 2024-09-10T21:33:19.046 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:33:19.046 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:33:19.046 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:33:19.046 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:33:19.046 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:33:19.046 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:33:19.046 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-10T21:33:19.046 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-10T21:33:19.046 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:33:19.046 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:33:19.046 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:33:19.047 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:33:19.047 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:33:19.047 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:33:19.047 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": [ 2024-09-10T21:33:19.047 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:33:19.047 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:33:19.047 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:33:19.047 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 5 2024-09-10T21:33:19.047 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:33:19.047 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:33:19.047 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-10T21:33:19.047 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:33:19.047 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:33:19.047 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:33:19.048 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 7 2024-09-10T21:33:19.048 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:33:19.048 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:33:19.048 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-10T21:33:19.048 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:33:19.048 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:33:19.048 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:33:19.048 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 7 2024-09-10T21:33:19.048 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:33:19.048 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:33:19.048 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": [ 2024-09-10T21:33:19.048 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:33:19.048 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:33:19.048 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:33:19.049 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 2 2024-09-10T21:33:19.049 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:33:19.049 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:33:19.049 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:33:19.049 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-10T21:33:19.049 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-10T21:33:19.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:19.029+0000 7f8e14000640 1 -- 172.21.15.17:0/3020651238 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f8dec0724e0 msgr2=0x7f8dec0749a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:19.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:19.029+0000 7f8e14000640 1 --2- 172.21.15.17:0/3020651238 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f8dec0724e0 0x7f8dec0749a0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f8e040013e0 tx=0x7f8e04009270 comp rx=0 tx=0).stop 2024-09-10T21:33:19.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:19.029+0000 7f8e14000640 1 -- 172.21.15.17:0/3020651238 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e0c0cdf20 msgr2=0x7f8e0c159ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:19.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:19.029+0000 7f8e14000640 1 --2- 172.21.15.17:0/3020651238 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8e0c0cdf20 0x7f8e0c159ce0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f8e000b0040 tx=0x7f8e000040c0 comp rx=0 tx=0).stop 2024-09-10T21:33:19.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:19.029+0000 7f8e12000640 1 -- 172.21.15.17:0/3020651238 reap_dead start 2024-09-10T21:33:19.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:19.029+0000 7f8e12000640 1 -- 172.21.15.17:0/3020651238 reap_dead start 2024-09-10T21:33:19.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:19.029+0000 7f8e14000640 1 -- 172.21.15.17:0/3020651238 shutdown_connections 2024-09-10T21:33:19.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:19.029+0000 7f8e14000640 1 -- 172.21.15.17:0/3020651238 >> 172.21.15.17:0/3020651238 conn(0x7f8e0c0702f0 msgr2=0x7f8e0c0c9ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:19.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:19.029+0000 7f8e14000640 1 -- 172.21.15.17:0/3020651238 shutdown_connections 2024-09-10T21:33:19.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:19.029+0000 7f8e14000640 1 -- 172.21.15.17:0/3020651238 wait complete. 2024-09-10T21:33:19.053 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-09-10T21:33:19.053 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon c 2024-09-10T21:33:19.054 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-10T21:33:19.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:19.241+0000 7fb3fda00640 1 -- 172.21.15.17:0/3106675793 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fb3f815a4a0 msgr2=0x7fb3f815a900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:19.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:19.241+0000 7fb3fda00640 1 --2- 172.21.15.17:0/3106675793 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fb3f815a4a0 0x7fb3f815a900 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fb3f0009c80 tx=0x7fb3f001cda0 comp rx=0 tx=0).stop 2024-09-10T21:33:19.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:19.241+0000 7fb3fda00640 1 -- 172.21.15.17:0/3106675793 shutdown_connections 2024-09-10T21:33:19.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:19.241+0000 7fb3fda00640 1 --2- 172.21.15.17:0/3106675793 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fb3f815a4a0 0x7fb3f815a900 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:19.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:19.241+0000 7fb3fda00640 1 --2- 172.21.15.17:0/3106675793 >> v2:172.21.15.31:3300/0 conn(0x7fb3f8073cd0 0x7fb3f80740f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:19.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:19.241+0000 7fb3fda00640 1 -- 172.21.15.17:0/3106675793 >> 172.21.15.17:0/3106675793 conn(0x7fb3f80700d0 msgr2=0x7fb3f80704e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:19.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:19.241+0000 7fb3fda00640 1 -- 172.21.15.17:0/3106675793 shutdown_connections 2024-09-10T21:33:19.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:19.241+0000 7fb3fda00640 1 -- 172.21.15.17:0/3106675793 wait complete. 2024-09-10T21:33:19.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:19.245+0000 7fb3fda00640 1 Processor -- start 2024-09-10T21:33:19.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:19.245+0000 7fb3fda00640 1 -- start start 2024-09-10T21:33:19.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:19.245+0000 7fb3fda00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fb3f8073cd0 0x7fb3f8153db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:19.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:19.245+0000 7fb3fda00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb3f80c0fd0 0x7fb3f8152fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:19.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:19.245+0000 7fb3fda00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3f815a4a0 0x7fb3f8141900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:19.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:19.245+0000 7fb3fda00640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fb3f806e640 con 0x7fb3f80c0fd0 2024-09-10T21:33:19.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:19.245+0000 7fb3fda00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fb3f815ae40 con 0x7fb3f815a4a0 2024-09-10T21:33:19.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:19.245+0000 7fb3fda00640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7fb3f80c8370 con 0x7fb3f8073cd0 2024-09-10T21:33:19.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:19.245+0000 7fb3efe00640 1 -- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb3f80c0fd0 msgr2=0x7fb3f8152fd0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-10T21:33:19.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:19.245+0000 7fb3efe00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb3f80c0fd0 0x7fb3f8152fd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:33:19.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:19.245+0000 7fb3f7400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3f815a4a0 0x7fb3f8141900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:19.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:19.245+0000 7fb3f7400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3f815a4a0 0x7fb3f8141900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.17:34028/0 (socket says 172.21.15.17:34028) 2024-09-10T21:33:19.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:19.245+0000 7fb3f7400640 1 -- 172.21.15.17:0/2558072912 learned_addr learned my addr 172.21.15.17:0/2558072912 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:33:19.251 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:19.245+0000 7fb3f6a00640 1 --2- 172.21.15.17:0/2558072912 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fb3f8073cd0 0x7fb3f8153db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:19.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:19.245+0000 7fb3f7400640 1 -- 172.21.15.17:0/2558072912 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fb3f8073cd0 msgr2=0x7fb3f8153db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:19.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:19.245+0000 7fb3f7400640 1 --2- 172.21.15.17:0/2558072912 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fb3f8073cd0 0x7fb3f8153db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:19.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:19.245+0000 7fb3f7400640 1 -- 172.21.15.17:0/2558072912 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb3f80c0fd0 msgr2=0x7fb3f8152fd0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:33:19.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:19.245+0000 7fb3f7400640 1 --2- 172.21.15.17:0/2558072912 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb3f80c0fd0 0x7fb3f8152fd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:19.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:19.245+0000 7fb3f7400640 1 -- 172.21.15.17:0/2558072912 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3f8159c80 con 0x7fb3f815a4a0 2024-09-10T21:33:19.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:19.245+0000 7fb3f7400640 1 --2- 172.21.15.17:0/2558072912 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3f815a4a0 0x7fb3f8141900 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fb3f0009bd0 tx=0x7fb3f0009300 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:19.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:19.245+0000 7fb3eea00640 1 -- 172.21.15.17:0/2558072912 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3f00bb070 con 0x7fb3f815a4a0 2024-09-10T21:33:19.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:19.245+0000 7fb3eea00640 1 -- 172.21.15.17:0/2558072912 <== mon.1 v2:172.21.15.31:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb3f0004d40 con 0x7fb3f815a4a0 2024-09-10T21:33:19.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:19.245+0000 7fb3eea00640 1 -- 172.21.15.17:0/2558072912 <== mon.1 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3f00b2040 con 0x7fb3f815a4a0 2024-09-10T21:33:19.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:19.245+0000 7fb3fda00640 1 -- 172.21.15.17:0/2558072912 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3f8158120 con 0x7fb3f815a4a0 2024-09-10T21:33:19.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:19.245+0000 7fb3fda00640 1 -- 172.21.15.17:0/2558072912 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3f8156890 con 0x7fb3f815a4a0 2024-09-10T21:33:19.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:19.249+0000 7fb3fda00640 1 --2- 172.21.15.17:0/2558072912 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fb3f8153510 0x7fb3f8141e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:19.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:19.249+0000 7fb3fda00640 1 -- 172.21.15.17:0/2558072912 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7fb3f80752a0 con 0x7fb3f8153510 2024-09-10T21:33:19.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:19.249+0000 7fb3f6a00640 1 --2- 172.21.15.17:0/2558072912 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fb3f8153510 0x7fb3f8141e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:19.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:19.249+0000 7fb3f6a00640 1 -- 172.21.15.17:0/2558072912 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb3f814b240 con 0x7fb3f8153510 2024-09-10T21:33:19.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:19.249+0000 7fb3f6a00640 1 --2- 172.21.15.17:0/2558072912 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fb3f8153510 0x7fb3f8141e40 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fb3e800f770 tx=0x7fb3e800fc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:19.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:19.249+0000 7fb3eea00640 1 -- 172.21.15.17:0/2558072912 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3e80050d0 con 0x7fb3f8153510 2024-09-10T21:33:19.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:19.249+0000 7fb3eea00640 1 -- 172.21.15.17:0/2558072912 <== mon.2 v2:172.21.15.31:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fb3e8005270 con 0x7fb3f8153510 2024-09-10T21:33:19.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:19.249+0000 7fb3eea00640 1 -- 172.21.15.17:0/2558072912 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fb3f8153510 msgr2=0x7fb3f8141e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:19.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:19.249+0000 7fb3eea00640 1 --2- 172.21.15.17:0/2558072912 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fb3f8153510 0x7fb3f8141e40 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fb3e800f770 tx=0x7fb3e800fc30 comp rx=0 tx=0).stop 2024-09-10T21:33:19.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:19.249+0000 7fb3eea00640 1 -- 172.21.15.17:0/2558072912 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fb3f8153510 msgr2=0x7fb3f8141e40 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:33:19.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:19.249+0000 7fb3eea00640 1 --2- 172.21.15.17:0/2558072912 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fb3f8153510 0x7fb3f8141e40 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:19.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:19.249+0000 7fb3eea00640 1 -- 172.21.15.17:0/2558072912 <== mon.1 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fb3f00b5030 con 0x7fb3f815a4a0 2024-09-10T21:33:19.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:19.249+0000 7fb3eea00640 1 --2- 172.21.15.17:0/2558072912 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fb3d4072050 0x7fb3d4074510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:19.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:19.249+0000 7fb3eea00640 1 -- 172.21.15.17:0/2558072912 <== mon.1 v2:172.21.15.31:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fb3f012cff0 con 0x7fb3f815a4a0 2024-09-10T21:33:19.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:19.249+0000 7fb3f6a00640 1 --2- 172.21.15.17:0/2558072912 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fb3d4072050 0x7fb3d4074510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:19.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:19.249+0000 7fb3f6a00640 1 --2- 172.21.15.17:0/2558072912 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fb3d4072050 0x7fb3d4074510 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fb3e8001060 tx=0x7fb3e8005000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:19.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:19.253+0000 7fb3fda00640 1 --2- 172.21.15.17:0/2558072912 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fb3f8154cd0 0x7fb3f81550b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:19.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:19.253+0000 7fb3fda00640 1 -- 172.21.15.17:0/2558072912 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7fb3f8151050 con 0x7fb3f8154cd0 2024-09-10T21:33:19.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:19.257+0000 7fb3efe00640 1 --2- 172.21.15.17:0/2558072912 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fb3f8154cd0 0x7fb3f81550b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:19.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:19.257+0000 7fb3efe00640 1 -- 172.21.15.17:0/2558072912 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fb3f8074e50 con 0x7fb3f8154cd0 2024-09-10T21:33:19.261 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:19.257+0000 7fb3efe00640 1 --2- 172.21.15.17:0/2558072912 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fb3f8154cd0 0x7fb3f81550b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fb3e00094f0 tx=0x7fb3e000ea80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:19.261 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:19.257+0000 7fb3eea00640 1 -- 172.21.15.17:0/2558072912 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3e0004230 con 0x7fb3f8154cd0 2024-09-10T21:33:19.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:19.257+0000 7fb3eea00640 1 -- 172.21.15.17:0/2558072912 <== mon.2 v2:172.21.15.31:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4864 (secure 0 0 0) 0x7fb3e0004230 con 0x7fb3f8154cd0 2024-09-10T21:33:19.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:19.257+0000 7fb3eea00640 1 -- 172.21.15.17:0/2558072912 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fb3f8154cd0 msgr2=0x7fb3f81550b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:19.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:19.257+0000 7fb3eea00640 1 --2- 172.21.15.17:0/2558072912 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fb3f8154cd0 0x7fb3f81550b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fb3e00094f0 tx=0x7fb3e000ea80 comp rx=0 tx=0).stop 2024-09-10T21:33:19.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:19.257+0000 7fb3eea00640 1 -- 172.21.15.17:0/2558072912 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fb3f8154cd0 msgr2=0x7fb3f81550b0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:33:19.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:19.257+0000 7fb3eea00640 1 --2- 172.21.15.17:0/2558072912 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fb3f8154cd0 0x7fb3f81550b0 secure :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fb3e00094f0 tx=0x7fb3e000ea80 comp rx=0 tx=0).stop 2024-09-10T21:33:19.263 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-10T21:33:19.263 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:33:19.263 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:33:19.263 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-10T21:33:19.263 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 72, 2024-09-10T21:33:19.263 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 131566, 2024-09-10T21:33:19.263 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-10T21:33:19.263 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-10T21:33:19.263 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-10T21:33:19.263 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:33:19.263 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 0, 2024-09-10T21:33:19.264 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:33:19.264 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-10T21:33:19.264 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-10T21:33:19.264 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:33:19.264 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:33:19.264 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:33:19.264 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:33:19.264 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:33:19.264 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:33:19.264 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:33:19.264 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:33:19.265 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:33:19.265 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:33:19.265 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:33:19.265 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:33:19.265 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-10T21:33:19.265 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-10T21:33:19.265 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:33:19.265 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:33:19.265 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:33:19.265 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:33:19.265 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:33:19.265 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:33:19.266 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:33:19.266 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:33:19.266 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:33:19.266 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:33:19.266 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:33:19.266 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:33:19.266 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:33:19.266 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-10T21:33:19.266 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-10T21:33:19.266 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-10T21:33:19.266 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-10T21:33:19.266 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-10T21:33:19.267 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "5e2f023f-d6a6-49f4-a256-f4807dce3739", 2024-09-10T21:33:19.267 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:33:19.267 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:33:19.267 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-10T21:33:19.267 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-10T21:33:19.267 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-10T21:33:19.267 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-10T21:33:19.267 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-10T21:33:19.267 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-10T21:33:19.267 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-10T21:33:19.268 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:33:19.268 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-10T21:33:19.268 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:33:19.268 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:33:19.268 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:33:19.268 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:33:19.268 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:33:19.268 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:33:19.268 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:33:19.268 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:33:19.268 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:33:19.269 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:33:19.269 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:33:19.269 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:33:19.269 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-10T21:33:19.269 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:33:19.269 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-10T21:33:19.269 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:33:19.269 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:33:19.269 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:33:19.269 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:33:19.269 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:33:19.269 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:33:19.270 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:33:19.270 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-10T21:33:19.270 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:33:19.270 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:33:19.270 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:33:19.270 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:33:19.270 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-10T21:33:19.270 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:33:19.270 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:33:19.270 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:33:19.271 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:33:19.271 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-10T21:33:19.271 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-10T21:33:19.271 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:33:19.271 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:33:19.271 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:33:19.271 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:33:19.271 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:33:19.271 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:33:19.271 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:33:19.271 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:33:19.272 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:33:19.272 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:33:19.272 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:33:19.272 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3300", 2024-09-10T21:33:19.272 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:33:19.272 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:33:19.272 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:33:19.272 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:33:19.272 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789", 2024-09-10T21:33:19.272 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:33:19.272 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:33:19.272 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:33:19.273 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:33:19.273 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789/0", 2024-09-10T21:33:19.273 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6789/0", 2024-09-10T21:33:19.273 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:33:19.273 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:33:19.273 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:33:19.273 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:33:19.273 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:33:19.273 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:33:19.273 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:33:19.273 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:33:19.273 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:33:19.274 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:33:19.274 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:33:19.274 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3301", 2024-09-10T21:33:19.274 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:33:19.274 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:33:19.274 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:33:19.274 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:33:19.274 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790", 2024-09-10T21:33:19.274 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:33:19.274 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:33:19.274 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:33:19.275 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:33:19.275 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790/0", 2024-09-10T21:33:19.275 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6790/0", 2024-09-10T21:33:19.275 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:33:19.275 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:33:19.275 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:33:19.275 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:33:19.275 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:33:19.275 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:33:19.275 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-10T21:33:19.276 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-10T21:33:19.276 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:33:19.276 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:33:19.276 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:33:19.276 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:33:19.276 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:33:19.276 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:33:19.276 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": [ 2024-09-10T21:33:19.276 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:33:19.276 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:33:19.276 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:33:19.276 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:33:19.277 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:33:19.277 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:33:19.277 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-10T21:33:19.277 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:33:19.277 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:33:19.277 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:33:19.277 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:33:19.277 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:33:19.277 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:33:19.277 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-10T21:33:19.277 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:33:19.278 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:33:19.278 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:33:19.278 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:33:19.278 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:33:19.278 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:33:19.278 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:33:19.278 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-10T21:33:19.278 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-10T21:33:19.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:19.257+0000 7fb3fda00640 1 -- 172.21.15.17:0/2558072912 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fb3d4072050 msgr2=0x7fb3d4074510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:19.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:19.257+0000 7fb3fda00640 1 --2- 172.21.15.17:0/2558072912 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fb3d4072050 0x7fb3d4074510 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fb3e8001060 tx=0x7fb3e8005000 comp rx=0 tx=0).stop 2024-09-10T21:33:19.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:19.257+0000 7fb3fda00640 1 -- 172.21.15.17:0/2558072912 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3f815a4a0 msgr2=0x7fb3f8141900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:19.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:19.257+0000 7fb3fda00640 1 --2- 172.21.15.17:0/2558072912 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb3f815a4a0 0x7fb3f8141900 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fb3f0009bd0 tx=0x7fb3f0009300 comp rx=0 tx=0).stop 2024-09-10T21:33:19.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:19.257+0000 7fb3f7400640 1 -- 172.21.15.17:0/2558072912 reap_dead start 2024-09-10T21:33:19.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:19.257+0000 7fb3f7400640 1 -- 172.21.15.17:0/2558072912 reap_dead start 2024-09-10T21:33:19.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:19.257+0000 7fb3fda00640 1 -- 172.21.15.17:0/2558072912 shutdown_connections 2024-09-10T21:33:19.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:19.257+0000 7fb3fda00640 1 -- 172.21.15.17:0/2558072912 >> 172.21.15.17:0/2558072912 conn(0x7fb3f80700d0 msgr2=0x7fb3f814f160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:19.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:19.257+0000 7fb3fda00640 1 -- 172.21.15.17:0/2558072912 shutdown_connections 2024-09-10T21:33:19.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:19.257+0000 7fb3fda00640 1 -- 172.21.15.17:0/2558072912 wait complete. 2024-09-10T21:33:19.281 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-09-10T21:33:19.281 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2024-09-10T21:33:39.282 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.a 2024-09-10T21:33:39.283 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-10T21:33:39.283 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-10T21:33:39.288 INFO:tasks.ceph.mon.a:Started 2024-09-10T21:33:39.288 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-09-10T21:33:39.288 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-10T21:33:39.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:39.465+0000 7f1069400640 1 -- 172.21.15.17:0/1708460154 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1064152120 msgr2=0x7f1064068bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:39.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:39.465+0000 7f1069400640 1 --2- 172.21.15.17:0/1708460154 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1064152120 0x7f1064068bb0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f10640c92d0 tx=0x7f105401efd0 comp rx=0 tx=0).stop 2024-09-10T21:33:39.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:39.465+0000 7f1069400640 1 -- 172.21.15.17:0/1708460154 shutdown_connections 2024-09-10T21:33:39.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:39.465+0000 7f1069400640 1 --2- 172.21.15.17:0/1708460154 >> v2:172.21.15.17:3300/0 conn(0x7f10640c6dd0 0x7f10640c7240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:39.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:39.465+0000 7f1069400640 1 --2- 172.21.15.17:0/1708460154 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1064152120 0x7f1064068bb0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:39.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:39.465+0000 7f1069400640 1 -- 172.21.15.17:0/1708460154 >> 172.21.15.17:0/1708460154 conn(0x7f106406cdf0 msgr2=0x7f106406d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:39.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:39.465+0000 7f1069400640 1 -- 172.21.15.17:0/1708460154 shutdown_connections 2024-09-10T21:33:39.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:39.465+0000 7f1069400640 1 -- 172.21.15.17:0/1708460154 wait complete. 2024-09-10T21:33:39.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:39.465+0000 7f1069400640 1 Processor -- start 2024-09-10T21:33:39.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:39.465+0000 7f1069400640 1 -- start start 2024-09-10T21:33:39.474 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:39.469+0000 7f1069400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f10640c6dd0 0x7f106414d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:39.474 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:39.469+0000 7f1069400640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f1064152120 0x7f1064138ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:39.474 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:39.469+0000 7f1069400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1064152500 0x7f10641376d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:39.474 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:39.469+0000 7f1069400640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f10641590c0 con 0x7f1064152500 2024-09-10T21:33:39.474 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:39.469+0000 7f1069400640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f1064158490 con 0x7f10640c6dd0 2024-09-10T21:33:39.474 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:39.469+0000 7f1069400640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f1064159240 con 0x7f1064152120 2024-09-10T21:33:39.474 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:39.469+0000 7f1068a00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1064152500 0x7f10641376d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:39.475 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:39.469+0000 7f1068a00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1064152500 0x7f10641376d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:53970/0 (socket says 172.21.15.17:53970) 2024-09-10T21:33:39.475 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:39.469+0000 7f1063e00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f10640c6dd0 0x7f106414d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:39.475 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:39.469+0000 7f1068a00640 1 -- 172.21.15.17:0/3407872198 learned_addr learned my addr 172.21.15.17:0/3407872198 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:33:39.475 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:39.469+0000 7f1063400640 1 --2- 172.21.15.17:0/3407872198 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f1064152120 0x7f1064138ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:39.475 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:39.469+0000 7f1068a00640 1 -- 172.21.15.17:0/3407872198 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1064152500 msgr2=0x7f10641376d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-09-10T21:33:39.475 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:39.469+0000 7f1068a00640 1 -- 172.21.15.17:0/3407872198 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1064152500 msgr2=0x7f10641376d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-10T21:33:39.475 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:39.469+0000 7f1068a00640 1 --2- 172.21.15.17:0/3407872198 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1064152500 0x7f10641376d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T21:33:39.475 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:39.469+0000 7f1068a00640 1 --2- 172.21.15.17:0/3407872198 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1064152500 0x7f10641376d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:33:39.475 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:39.469+0000 7f1063400640 1 -- 172.21.15.17:0/3407872198 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f10640c6dd0 msgr2=0x7f106414d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:39.475 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:39.469+0000 7f1063400640 1 --2- 172.21.15.17:0/3407872198 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f10640c6dd0 0x7f106414d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:39.475 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:39.469+0000 7f1063400640 1 -- 172.21.15.17:0/3407872198 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1064152500 msgr2=0x7f10641376d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:33:39.475 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:39.469+0000 7f1063400640 1 --2- 172.21.15.17:0/3407872198 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1064152500 0x7f10641376d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:39.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:39.469+0000 7f1063400640 1 -- 172.21.15.17:0/3407872198 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1064141b00 con 0x7f1064152120 2024-09-10T21:33:39.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:39.469+0000 7f1063e00640 1 --2- 172.21.15.17:0/3407872198 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f10640c6dd0 0x7f106414d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:33:39.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:39.469+0000 7f1063400640 1 --2- 172.21.15.17:0/3407872198 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f1064152120 0x7f1064138ec0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f105000c390 tx=0x7f105000c850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:39.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:39.469+0000 7f105be00640 1 -- 172.21.15.17:0/3407872198 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1050017070 con 0x7f1064152120 2024-09-10T21:33:39.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:39.469+0000 7f105be00640 1 -- 172.21.15.17:0/3407872198 <== mon.2 v2:172.21.15.31:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f10500053c0 con 0x7f1064152120 2024-09-10T21:33:39.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:39.469+0000 7f105be00640 1 -- 172.21.15.17:0/3407872198 <== mon.2 v2:172.21.15.31:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1050003840 con 0x7f1064152120 2024-09-10T21:33:39.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:39.469+0000 7f1069400640 1 -- 172.21.15.17:0/3407872198 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1064141de0 con 0x7f1064152120 2024-09-10T21:33:39.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:39.469+0000 7f1069400640 1 -- 172.21.15.17:0/3407872198 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f10641423a0 con 0x7f1064152120 2024-09-10T21:33:39.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:39.469+0000 7f1069400640 1 -- 172.21.15.17:0/3407872198 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10640c7240 con 0x7f1064152120 2024-09-10T21:33:39.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:39.473+0000 7f105be00640 1 -- 172.21.15.17:0/3407872198 <== mon.2 v2:172.21.15.31:3301/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f10500039e0 con 0x7f1064152120 2024-09-10T21:33:39.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:39.473+0000 7f105be00640 1 --2- 172.21.15.17:0/3407872198 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f10440725e0 0x7f1044074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:39.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:39.473+0000 7f105be00640 1 -- 172.21.15.17:0/3407872198 <== mon.2 v2:172.21.15.31:3301/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f1050092c50 con 0x7f1064152120 2024-09-10T21:33:39.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:39.473+0000 7f1063e00640 1 --2- 172.21.15.17:0/3407872198 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f10440725e0 0x7f1044074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:39.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:39.473+0000 7f1063e00640 1 --2- 172.21.15.17:0/3407872198 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f10440725e0 0x7f1044074aa0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f10540b2000 tx=0x7f105401d810 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:39.484 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:39.481+0000 7f105be00640 1 -- 172.21.15.17:0/3407872198 <== mon.2 v2:172.21.15.31:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f105005d7b0 con 0x7f1064152120 2024-09-10T21:33:39.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:39.773+0000 7f1069400640 1 -- 172.21.15.17:0/3407872198 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f1064137500 con 0x7f1064152120 2024-09-10T21:33:39.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:39.773+0000 7f105be00640 1 -- 172.21.15.17:0/3407872198 <== mon.2 v2:172.21.15.31:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1551 (secure 0 0 0) 0x7f10500621e0 con 0x7f1064152120 2024-09-10T21:33:39.779 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":72,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":21,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5e2f023f-d6a6-49f4-a256-f4807dce3739","modified":"2024-09-10T21:23:46.892130Z","created":"2024-09-10T21:23:46.892130Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3301","nonce":0},{"type":"v1","addr":"172.21.15.31:6790","nonce":0}]},"addr":"172.21.15.31:6790/0","public_addr":"172.21.15.31:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-10T21:33:39.781 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:39.777+0000 7f1069400640 1 -- 172.21.15.17:0/3407872198 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f10440725e0 msgr2=0x7f1044074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:39.781 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:39.777+0000 7f1069400640 1 --2- 172.21.15.17:0/3407872198 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f10440725e0 0x7f1044074aa0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f10540b2000 tx=0x7f105401d810 comp rx=0 tx=0).stop 2024-09-10T21:33:39.781 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:39.777+0000 7f1069400640 1 -- 172.21.15.17:0/3407872198 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f1064152120 msgr2=0x7f1064138ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:39.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:39.777+0000 7f1069400640 1 --2- 172.21.15.17:0/3407872198 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f1064152120 0x7f1064138ec0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f105000c390 tx=0x7f105000c850 comp rx=0 tx=0).stop 2024-09-10T21:33:39.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:39.777+0000 7f1069400640 1 -- 172.21.15.17:0/3407872198 shutdown_connections 2024-09-10T21:33:39.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:39.777+0000 7f1069400640 1 --2- 172.21.15.17:0/3407872198 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f10440725e0 0x7f1044074aa0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:39.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:39.777+0000 7f1069400640 1 --2- 172.21.15.17:0/3407872198 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1064152500 0x7f10641376d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:39.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:39.777+0000 7f1069400640 1 --2- 172.21.15.17:0/3407872198 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f1064152120 0x7f1064138ec0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:39.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:39.777+0000 7f1069400640 1 --2- 172.21.15.17:0/3407872198 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f10640c6dd0 0x7f106414d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:39.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:39.777+0000 7f1069400640 1 -- 172.21.15.17:0/3407872198 >> 172.21.15.17:0/3407872198 conn(0x7f106406cdf0 msgr2=0x7f1064150e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:39.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:39.777+0000 7f1069400640 1 -- 172.21.15.17:0/3407872198 shutdown_connections 2024-09-10T21:33:39.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:39.777+0000 7f1069400640 1 -- 172.21.15.17:0/3407872198 wait complete. 2024-09-10T21:33:42.804 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-10T21:33:47.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:47.457+0000 7f28ea000640 1 -- 172.21.15.17:0/2518795757 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f28c8003a10 msgr2=0x7f28c8003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:47.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:47.457+0000 7f28ea000640 1 --2- 172.21.15.17:0/2518795757 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f28c8003a10 0x7f28c8003e10 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f28d40066c0 tx=0x7f28d401d720 comp rx=0 tx=0).stop 2024-09-10T21:33:47.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:47.457+0000 7f28ea000640 1 -- 172.21.15.17:0/2518795757 shutdown_connections 2024-09-10T21:33:47.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:47.457+0000 7f28ea000640 1 --2- 172.21.15.17:0/2518795757 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f28c4003680 0x7f28c800b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:47.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:47.457+0000 7f28ea000640 1 --2- 172.21.15.17:0/2518795757 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f28c8003a10 0x7f28c8003e10 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:47.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:47.457+0000 7f28ea000640 1 --2- 172.21.15.17:0/2518795757 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f28c8004810 0x7f28c8004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:47.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:47.457+0000 7f28ea000640 1 -- 172.21.15.17:0/2518795757 >> 172.21.15.17:0/2518795757 conn(0x7f28e40702f0 msgr2=0x7f28e4070700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:47.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:47.457+0000 7f28ea000640 1 -- 172.21.15.17:0/2518795757 shutdown_connections 2024-09-10T21:33:47.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:47.457+0000 7f28ea000640 1 -- 172.21.15.17:0/2518795757 wait complete. 2024-09-10T21:33:47.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:47.461+0000 7f28ea000640 1 Processor -- start 2024-09-10T21:33:47.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:47.461+0000 7f28ea000640 1 -- start start 2024-09-10T21:33:47.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:47.461+0000 7f28ea000640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f28c4003680 0x7f28e4069050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:47.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:47.461+0000 7f28ea000640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f28c8003a10 0x7f28e41edb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:47.467 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:47.461+0000 7f28ea000640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f28c8004810 0x7f28e414bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:47.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:47.461+0000 7f28ea000640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f28e415af50 con 0x7f28c8003a10 2024-09-10T21:33:47.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:47.461+0000 7f28ea000640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f28e415ac70 con 0x7f28c8004810 2024-09-10T21:33:47.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:47.461+0000 7f28ea000640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f28e40c9640 con 0x7f28c4003680 2024-09-10T21:33:47.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:47.461+0000 7f28e2a00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f28c8003a10 0x7f28e41edb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:47.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:47.461+0000 7f28e2a00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f28c8003a10 0x7f28e41edb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:60524/0 (socket says 172.21.15.17:60524) 2024-09-10T21:33:47.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:47.461+0000 7f28e2a00640 1 -- 172.21.15.17:0/2562485461 learned_addr learned my addr 172.21.15.17:0/2562485461 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:33:47.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:47.461+0000 7f28e3e00640 1 --2- 172.21.15.17:0/2562485461 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f28c8004810 0x7f28e414bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:47.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:47.461+0000 7f28e3400640 1 --2- 172.21.15.17:0/2562485461 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f28c4003680 0x7f28e4069050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:47.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:47.461+0000 7f28e2a00640 1 -- 172.21.15.17:0/2562485461 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f28c4003680 msgr2=0x7f28e4069050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:47.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:47.461+0000 7f28e2a00640 1 --2- 172.21.15.17:0/2562485461 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f28c4003680 0x7f28e4069050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:47.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:47.461+0000 7f28e2a00640 1 -- 172.21.15.17:0/2562485461 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f28c8004810 msgr2=0x7f28e414bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:47.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:47.461+0000 7f28e2a00640 1 --2- 172.21.15.17:0/2562485461 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f28c8004810 0x7f28e414bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:47.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:47.461+0000 7f28e2a00640 1 -- 172.21.15.17:0/2562485461 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28e414c2f0 con 0x7f28c8003a10 2024-09-10T21:33:47.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:47.461+0000 7f28e3e00640 1 --2- 172.21.15.17:0/2562485461 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f28c8004810 0x7f28e414bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:33:47.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:47.461+0000 7f28e3400640 1 --2- 172.21.15.17:0/2562485461 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f28c4003680 0x7f28e4069050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:33:47.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:47.461+0000 7f28e2a00640 1 --2- 172.21.15.17:0/2562485461 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f28c8003a10 0x7f28e41edb60 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f28d0013a00 tx=0x7f28d0013ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:47.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:47.461+0000 7f28db400640 1 -- 172.21.15.17:0/2562485461 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28d0021070 con 0x7f28c8003a10 2024-09-10T21:33:47.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:47.461+0000 7f28db400640 1 -- 172.21.15.17:0/2562485461 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f28d0011bd0 con 0x7f28c8003a10 2024-09-10T21:33:47.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:47.461+0000 7f28ea000640 1 -- 172.21.15.17:0/2562485461 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28e40c8110 con 0x7f28c8003a10 2024-09-10T21:33:47.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:47.461+0000 7f28db400640 1 -- 172.21.15.17:0/2562485461 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28d0004030 con 0x7f28c8003a10 2024-09-10T21:33:47.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:47.461+0000 7f28ea000640 1 -- 172.21.15.17:0/2562485461 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28e40c5530 con 0x7f28c8003a10 2024-09-10T21:33:47.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:47.465+0000 7f28ea000640 1 -- 172.21.15.17:0/2562485461 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28e40ca430 con 0x7f28c8003a10 2024-09-10T21:33:47.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:47.465+0000 7f28db400640 1 -- 172.21.15.17:0/2562485461 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f28d00041d0 con 0x7f28c8003a10 2024-09-10T21:33:47.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:47.469+0000 7f28db400640 1 --2- 172.21.15.17:0/2562485461 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f28c4075b90 0x7f28c4078050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:47.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:47.469+0000 7f28db400640 1 -- 172.21.15.17:0/2562485461 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f28d0094c70 con 0x7f28c8003a10 2024-09-10T21:33:47.473 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:47.469+0000 7f28e3400640 1 --2- 172.21.15.17:0/2562485461 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f28c4075b90 0x7f28c4078050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:47.473 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:47.469+0000 7f28e3400640 1 --2- 172.21.15.17:0/2562485461 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f28c4075b90 0x7f28c4078050 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f28d4006690 tx=0x7f28d4006040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:47.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:47.473+0000 7f28db400640 1 -- 172.21.15.17:0/2562485461 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f28d002a020 con 0x7f28c8003a10 2024-09-10T21:33:47.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:47.765+0000 7f28ea000640 1 -- 172.21.15.17:0/2562485461 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f28e406e8c0 con 0x7f28c8003a10 2024-09-10T21:33:47.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:47.765+0000 7f28db400640 1 -- 172.21.15.17:0/2562485461 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1550 (secure 0 0 0) 0x7f28d005ffc0 con 0x7f28c8003a10 2024-09-10T21:33:47.771 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":76,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5e2f023f-d6a6-49f4-a256-f4807dce3739","modified":"2024-09-10T21:23:46.892130Z","created":"2024-09-10T21:23:46.892130Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3301","nonce":0},{"type":"v1","addr":"172.21.15.31:6790","nonce":0}]},"addr":"172.21.15.31:6790/0","public_addr":"172.21.15.31:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-10T21:33:47.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:47.769+0000 7f28ea000640 1 -- 172.21.15.17:0/2562485461 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f28c4075b90 msgr2=0x7f28c4078050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:47.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:47.769+0000 7f28ea000640 1 --2- 172.21.15.17:0/2562485461 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f28c4075b90 0x7f28c4078050 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f28d4006690 tx=0x7f28d4006040 comp rx=0 tx=0).stop 2024-09-10T21:33:47.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:47.769+0000 7f28ea000640 1 -- 172.21.15.17:0/2562485461 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f28c8003a10 msgr2=0x7f28e41edb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:47.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:47.769+0000 7f28ea000640 1 --2- 172.21.15.17:0/2562485461 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f28c8003a10 0x7f28e41edb60 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f28d0013a00 tx=0x7f28d0013ec0 comp rx=0 tx=0).stop 2024-09-10T21:33:47.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:47.769+0000 7f28ea000640 1 -- 172.21.15.17:0/2562485461 shutdown_connections 2024-09-10T21:33:47.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:47.769+0000 7f28ea000640 1 --2- 172.21.15.17:0/2562485461 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f28c4075b90 0x7f28c4078050 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:47.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:47.769+0000 7f28ea000640 1 --2- 172.21.15.17:0/2562485461 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f28c8004810 0x7f28e414bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:47.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:47.769+0000 7f28ea000640 1 --2- 172.21.15.17:0/2562485461 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f28c8003a10 0x7f28e41edb60 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:47.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:47.769+0000 7f28ea000640 1 --2- 172.21.15.17:0/2562485461 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f28c4003680 0x7f28e4069050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:47.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:47.769+0000 7f28ea000640 1 -- 172.21.15.17:0/2562485461 >> 172.21.15.17:0/2562485461 conn(0x7f28e40702f0 msgr2=0x7f28e414c710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:47.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:47.769+0000 7f28ea000640 1 -- 172.21.15.17:0/2562485461 shutdown_connections 2024-09-10T21:33:47.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:47.769+0000 7f28ea000640 1 -- 172.21.15.17:0/2562485461 wait complete. 2024-09-10T21:33:49.052 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:33:49.056 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:33:50.795 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-10T21:33:52.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.045+0000 7f0a93e00640 1 -- 172.21.15.17:0/861524431 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0a780040e0 msgr2=0x7f0a78004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:52.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.045+0000 7f0a93e00640 1 --2- 172.21.15.17:0/861524431 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0a780040e0 0x7f0a78004810 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f0a941510e0 tx=0x7f0a8001d170 comp rx=0 tx=0).stop 2024-09-10T21:33:52.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.045+0000 7f0a93400640 1 -- 172.21.15.17:0/861524431 reap_dead start 2024-09-10T21:33:52.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.045+0000 7f0a93e00640 1 -- 172.21.15.17:0/861524431 shutdown_connections 2024-09-10T21:33:52.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.045+0000 7f0a93e00640 1 -- 172.21.15.17:0/861524431 >> 172.21.15.17:0/861524431 conn(0x7f0a9406cdf0 msgr2=0x7f0a9406d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:52.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.045+0000 7f0a93e00640 1 -- 172.21.15.17:0/861524431 shutdown_connections 2024-09-10T21:33:52.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.045+0000 7f0a93e00640 1 -- 172.21.15.17:0/861524431 wait complete. 2024-09-10T21:33:52.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.049+0000 7f0a93e00640 1 Processor -- start 2024-09-10T21:33:52.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.049+0000 7f0a93e00640 1 -- start start 2024-09-10T21:33:52.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.049+0000 7f0a93e00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f0a780040e0 0x7f0a94151690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:52.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.049+0000 7f0a93e00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0a940cb550 0x7f0a940cab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:52.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.049+0000 7f0a93e00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0a941550f0 0x7f0a940cc010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:52.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.049+0000 7f0a93e00640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f0a9415a690 con 0x7f0a941550f0 2024-09-10T21:33:52.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.049+0000 7f0a93e00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f0a9415a3b0 con 0x7f0a940cb550 2024-09-10T21:33:52.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.049+0000 7f0a93e00640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f0a940c6820 con 0x7f0a780040e0 2024-09-10T21:33:52.055 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.049+0000 7f0a93400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0a941550f0 0x7f0a940cc010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:52.055 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.049+0000 7f0a93400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0a941550f0 0x7f0a940cc010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:60538/0 (socket says 172.21.15.17:60538) 2024-09-10T21:33:52.055 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.049+0000 7f0a93400640 1 -- 172.21.15.17:0/3717328003 learned_addr learned my addr 172.21.15.17:0/3717328003 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:33:52.055 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.049+0000 7f0a92000640 1 --2- 172.21.15.17:0/3717328003 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0a940cb550 0x7f0a940cab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:52.055 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.049+0000 7f0a92a00640 1 --2- 172.21.15.17:0/3717328003 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f0a780040e0 0x7f0a94151690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:52.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.049+0000 7f0a93400640 1 -- 172.21.15.17:0/3717328003 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f0a780040e0 msgr2=0x7f0a94151690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:52.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.049+0000 7f0a93400640 1 --2- 172.21.15.17:0/3717328003 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f0a780040e0 0x7f0a94151690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:52.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.049+0000 7f0a93400640 1 -- 172.21.15.17:0/3717328003 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0a940cb550 msgr2=0x7f0a940cab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:52.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.049+0000 7f0a93400640 1 --2- 172.21.15.17:0/3717328003 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0a940cb550 0x7f0a940cab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:52.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.049+0000 7f0a93400640 1 -- 172.21.15.17:0/3717328003 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a941efbd0 con 0x7f0a941550f0 2024-09-10T21:33:52.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.049+0000 7f0a93400640 1 --2- 172.21.15.17:0/3717328003 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0a941550f0 0x7f0a940cc010 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f0a9406e2a0 tx=0x7f0a8c00cd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:52.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.049+0000 7f0a8aa00640 1 -- 172.21.15.17:0/3717328003 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a8c003b90 con 0x7f0a941550f0 2024-09-10T21:33:52.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.049+0000 7f0a8aa00640 1 -- 172.21.15.17:0/3717328003 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0a8c003d30 con 0x7f0a941550f0 2024-09-10T21:33:52.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.049+0000 7f0a93e00640 1 -- 172.21.15.17:0/3717328003 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a941efeb0 con 0x7f0a941550f0 2024-09-10T21:33:52.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.049+0000 7f0a8aa00640 1 -- 172.21.15.17:0/3717328003 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a8c006410 con 0x7f0a941550f0 2024-09-10T21:33:52.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.049+0000 7f0a93e00640 1 -- 172.21.15.17:0/3717328003 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a941f0470 con 0x7f0a941550f0 2024-09-10T21:33:52.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.053+0000 7f0a93e00640 1 -- 172.21.15.17:0/3717328003 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a940c9a20 con 0x7f0a941550f0 2024-09-10T21:33:52.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.053+0000 7f0a8aa00640 1 -- 172.21.15.17:0/3717328003 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f0a8c01f030 con 0x7f0a941550f0 2024-09-10T21:33:52.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.053+0000 7f0a8aa00640 1 --2- 172.21.15.17:0/3717328003 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f0a74072610 0x7f0a74074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:52.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.053+0000 7f0a8aa00640 1 -- 172.21.15.17:0/3717328003 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f0a8c0933f0 con 0x7f0a941550f0 2024-09-10T21:33:52.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.053+0000 7f0a92a00640 1 --2- 172.21.15.17:0/3717328003 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f0a74072610 0x7f0a74074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:52.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.057+0000 7f0a92a00640 1 --2- 172.21.15.17:0/3717328003 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f0a74072610 0x7f0a74074ad0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f0a84005fd0 tx=0x7f0a8400f410 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:52.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.061+0000 7f0a8aa00640 1 -- 172.21.15.17:0/3717328003 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f0a8c05e070 con 0x7f0a941550f0 2024-09-10T21:33:52.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.333+0000 7f0a93e00640 1 -- 172.21.15.17:0/3717328003 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f0a94069e60 con 0x7f0a941550f0 2024-09-10T21:33:52.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.333+0000 7f0a8aa00640 1 -- 172.21.15.17:0/3717328003 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1556 (secure 0 0 0) 0x7f0a8c062aa0 con 0x7f0a941550f0 2024-09-10T21:33:52.338 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":80,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5e2f023f-d6a6-49f4-a256-f4807dce3739","modified":"2024-09-10T21:23:46.892130Z","created":"2024-09-10T21:23:46.892130Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3301","nonce":0},{"type":"v1","addr":"172.21.15.31:6790","nonce":0}]},"addr":"172.21.15.31:6790/0","public_addr":"172.21.15.31:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-10T21:33:52.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.337+0000 7f0a93e00640 1 -- 172.21.15.17:0/3717328003 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f0a74072610 msgr2=0x7f0a74074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:52.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.337+0000 7f0a93e00640 1 --2- 172.21.15.17:0/3717328003 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f0a74072610 0x7f0a74074ad0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f0a84005fd0 tx=0x7f0a8400f410 comp rx=0 tx=0).stop 2024-09-10T21:33:52.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.337+0000 7f0a93e00640 1 -- 172.21.15.17:0/3717328003 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0a941550f0 msgr2=0x7f0a940cc010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:52.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.337+0000 7f0a93e00640 1 --2- 172.21.15.17:0/3717328003 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0a941550f0 0x7f0a940cc010 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f0a9406e2a0 tx=0x7f0a8c00cd40 comp rx=0 tx=0).stop 2024-09-10T21:33:52.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.337+0000 7f0a93e00640 1 -- 172.21.15.17:0/3717328003 shutdown_connections 2024-09-10T21:33:52.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.337+0000 7f0a93e00640 1 --2- 172.21.15.17:0/3717328003 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f0a74072610 0x7f0a74074ad0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:52.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.337+0000 7f0a93e00640 1 --2- 172.21.15.17:0/3717328003 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0a941550f0 0x7f0a940cc010 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:52.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.337+0000 7f0a93e00640 1 --2- 172.21.15.17:0/3717328003 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0a940cb550 0x7f0a940cab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:52.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.337+0000 7f0a93e00640 1 --2- 172.21.15.17:0/3717328003 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f0a780040e0 0x7f0a94151690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:52.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.337+0000 7f0a93e00640 1 -- 172.21.15.17:0/3717328003 >> 172.21.15.17:0/3717328003 conn(0x7f0a9406cdf0 msgr2=0x7f0a9406fd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:52.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.337+0000 7f0a93e00640 1 -- 172.21.15.17:0/3717328003 shutdown_connections 2024-09-10T21:33:52.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.337+0000 7f0a93e00640 1 -- 172.21.15.17:0/3717328003 wait complete. 2024-09-10T21:33:52.363 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-09-10T21:33:52.364 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-09-10T21:33:52.364 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-10T21:33:52.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.517+0000 7f063be00640 1 -- 172.21.15.17:0/3884340207 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f0634070450 msgr2=0x7f063406e850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:52.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.517+0000 7f063be00640 1 --2- 172.21.15.17:0/3884340207 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f0634070450 0x7f063406e850 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f0620009840 tx=0x7f062001f010 comp rx=0 tx=0).stop 2024-09-10T21:33:52.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.517+0000 7f063be00640 1 -- 172.21.15.17:0/3884340207 shutdown_connections 2024-09-10T21:33:52.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.517+0000 7f063be00640 1 --2- 172.21.15.17:0/3884340207 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f0634070450 0x7f063406e850 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:52.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.517+0000 7f063be00640 1 --2- 172.21.15.17:0/3884340207 >> v2:172.21.15.31:3300/0 conn(0x7f0634074f70 0x7f06340c7130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:52.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.517+0000 7f063be00640 1 -- 172.21.15.17:0/3884340207 >> 172.21.15.17:0/3884340207 conn(0x7f063406dd30 msgr2=0x7f063406e140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:52.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.517+0000 7f063be00640 1 -- 172.21.15.17:0/3884340207 shutdown_connections 2024-09-10T21:33:52.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.517+0000 7f063be00640 1 -- 172.21.15.17:0/3884340207 wait complete. 2024-09-10T21:33:52.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.517+0000 7f063be00640 1 Processor -- start 2024-09-10T21:33:52.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.517+0000 7f063be00640 1 -- start start 2024-09-10T21:33:52.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.521+0000 7f063be00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f063406ed90 0x7f06341f39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:52.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.521+0000 7f063be00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0634070450 0x7f06341e3010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:52.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.521+0000 7f063be00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f0634074f70 0x7f06340ae8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:52.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.521+0000 7f063be00640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f06341592e0 con 0x7f063406ed90 2024-09-10T21:33:52.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.521+0000 7f063be00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f0634159000 con 0x7f0634070450 2024-09-10T21:33:52.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.521+0000 7f063be00640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f0634069b90 con 0x7f0634074f70 2024-09-10T21:33:52.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.521+0000 7f0639400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f063406ed90 0x7f06341f39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:52.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.521+0000 7f0639400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f063406ed90 0x7f06341f39d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:60554/0 (socket says 172.21.15.17:60554) 2024-09-10T21:33:52.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.521+0000 7f0639400640 1 -- 172.21.15.17:0/769961136 learned_addr learned my addr 172.21.15.17:0/769961136 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:33:52.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.521+0000 7f0638a00640 1 --2- 172.21.15.17:0/769961136 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0634070450 0x7f06341e3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:52.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.521+0000 7f0639e00640 1 --2- 172.21.15.17:0/769961136 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f0634074f70 0x7f06340ae8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:52.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.521+0000 7f0639400640 1 -- 172.21.15.17:0/769961136 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f0634074f70 msgr2=0x7f06340ae8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:52.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.521+0000 7f0639400640 1 --2- 172.21.15.17:0/769961136 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f0634074f70 0x7f06340ae8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:52.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.521+0000 7f0639400640 1 -- 172.21.15.17:0/769961136 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0634070450 msgr2=0x7f06341e3010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:52.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.521+0000 7f0639400640 1 --2- 172.21.15.17:0/769961136 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0634070450 0x7f06341e3010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:52.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.521+0000 7f0639400640 1 -- 172.21.15.17:0/769961136 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06341e1f70 con 0x7f063406ed90 2024-09-10T21:33:52.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.521+0000 7f0639400640 1 --2- 172.21.15.17:0/769961136 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f063406ed90 0x7f06341f39d0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f062800e7f0 tx=0x7f062800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:52.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.521+0000 7f0631600640 1 -- 172.21.15.17:0/769961136 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f062800ccc0 con 0x7f063406ed90 2024-09-10T21:33:52.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.521+0000 7f0631600640 1 -- 172.21.15.17:0/769961136 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f062800ce80 con 0x7f063406ed90 2024-09-10T21:33:52.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.521+0000 7f063be00640 1 -- 172.21.15.17:0/769961136 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06341e2250 con 0x7f063406ed90 2024-09-10T21:33:52.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.521+0000 7f0631600640 1 -- 172.21.15.17:0/769961136 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0628005280 con 0x7f063406ed90 2024-09-10T21:33:52.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.521+0000 7f063be00640 1 -- 172.21.15.17:0/769961136 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06341e2810 con 0x7f063406ed90 2024-09-10T21:33:52.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.521+0000 7f063be00640 1 --2- 172.21.15.17:0/769961136 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f06341f0ac0 0x7f06340aede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:52.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.521+0000 7f063be00640 1 -- 172.21.15.17:0/769961136 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f0634069e60 con 0x7f06341f0ac0 2024-09-10T21:33:52.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.521+0000 7f0638a00640 1 --2- 172.21.15.17:0/769961136 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f06341f0ac0 0x7f06340aede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:52.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.525+0000 7f0638a00640 1 -- 172.21.15.17:0/769961136 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f06340c6520 con 0x7f06341f0ac0 2024-09-10T21:33:52.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.525+0000 7f0638a00640 1 --2- 172.21.15.17:0/769961136 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f06341f0ac0 0x7f06340aede0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f0634071120 tx=0x7f0620005b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:52.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.525+0000 7f0631600640 1 -- 172.21.15.17:0/769961136 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06200b5040 con 0x7f06341f0ac0 2024-09-10T21:33:52.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.525+0000 7f0631600640 1 -- 172.21.15.17:0/769961136 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f0628005420 con 0x7f063406ed90 2024-09-10T21:33:52.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.525+0000 7f0631600640 1 --2- 172.21.15.17:0/769961136 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f06140724e0 0x7f06140749a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:52.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.525+0000 7f0631600640 1 -- 172.21.15.17:0/769961136 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f0628093f00 con 0x7f063406ed90 2024-09-10T21:33:52.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.525+0000 7f0631600640 1 -- 172.21.15.17:0/769961136 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f0620003600 con 0x7f06341f0ac0 2024-09-10T21:33:52.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.525+0000 7f0631600640 1 -- 172.21.15.17:0/769961136 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f06341f0ac0 msgr2=0x7f06340aede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:52.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.525+0000 7f0631600640 1 --2- 172.21.15.17:0/769961136 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f06341f0ac0 0x7f06340aede0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f0634071120 tx=0x7f0620005b30 comp rx=0 tx=0).stop 2024-09-10T21:33:52.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.525+0000 7f0639e00640 1 --2- 172.21.15.17:0/769961136 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f06140724e0 0x7f06140749a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:52.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.525+0000 7f0631600640 1 -- 172.21.15.17:0/769961136 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f06341f0ac0 msgr2=0x7f06340aede0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:33:52.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.525+0000 7f0631600640 1 --2- 172.21.15.17:0/769961136 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f06341f0ac0 0x7f06340aede0 secure :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f0634071120 tx=0x7f0620005b30 comp rx=0 tx=0).stop 2024-09-10T21:33:52.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.525+0000 7f0639e00640 1 --2- 172.21.15.17:0/769961136 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f06140724e0 0x7f06140749a0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f062c001850 tx=0x7f062c0017e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:52.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.533+0000 7f063be00640 1 --2- 172.21.15.17:0/769961136 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f06340c11e0 0x7f06341f40b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:52.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.533+0000 7f063be00640 1 -- 172.21.15.17:0/769961136 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f06340c8a80 con 0x7f06340c11e0 2024-09-10T21:33:52.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.533+0000 7f0638a00640 1 --2- 172.21.15.17:0/769961136 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f06340c11e0 0x7f06341f40b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:52.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.533+0000 7f0638a00640 1 -- 172.21.15.17:0/769961136 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f0634070960 con 0x7f06340c11e0 2024-09-10T21:33:52.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.533+0000 7f0638a00640 1 --2- 172.21.15.17:0/769961136 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f06340c11e0 0x7f06341f40b0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f0620000c10 tx=0x7f06200091e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:52.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.537+0000 7f0631600640 1 -- 172.21.15.17:0/769961136 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0620003bf0 con 0x7f06340c11e0 2024-09-10T21:33:52.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.537+0000 7f0631600640 1 -- 172.21.15.17:0/769961136 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4542 (secure 0 0 0) 0x7f0620003bf0 con 0x7f06340c11e0 2024-09-10T21:33:52.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.537+0000 7f0631600640 1 -- 172.21.15.17:0/769961136 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f06340c11e0 msgr2=0x7f06341f40b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:52.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.537+0000 7f0631600640 1 --2- 172.21.15.17:0/769961136 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f06340c11e0 0x7f06341f40b0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f0620000c10 tx=0x7f06200091e0 comp rx=0 tx=0).stop 2024-09-10T21:33:52.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.537+0000 7f0631600640 1 -- 172.21.15.17:0/769961136 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f06340c11e0 msgr2=0x7f06341f40b0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:33:52.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.537+0000 7f0631600640 1 --2- 172.21.15.17:0/769961136 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f06340c11e0 0x7f06341f40b0 secure :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f0620000c10 tx=0x7f06200091e0 comp rx=0 tx=0).stop 2024-09-10T21:33:52.541 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-10T21:33:52.544 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:33:52.544 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:33:52.544 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "leader", 2024-09-10T21:33:52.544 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 80, 2024-09-10T21:33:52.544 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 13143, 2024-09-10T21:33:52.544 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-10T21:33:52.544 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-10T21:33:52.544 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-10T21:33:52.544 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-10T21:33:52.544 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:33:52.544 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 0, 2024-09-10T21:33:52.545 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:33:52.545 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-10T21:33:52.545 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-10T21:33:52.545 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:33:52.545 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:33:52.545 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:33:52.545 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:33:52.545 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:33:52.545 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:33:52.545 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:33:52.545 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:33:52.545 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:33:52.546 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:33:52.546 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:33:52.546 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:33:52.546 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-10T21:33:52.546 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-10T21:33:52.546 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:33:52.546 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:33:52.546 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:33:52.546 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:33:52.546 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:33:52.546 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:33:52.546 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:33:52.546 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:33:52.547 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:33:52.547 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:33:52.547 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:33:52.547 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:33:52.547 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:33:52.547 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-10T21:33:52.547 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-10T21:33:52.547 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-10T21:33:52.547 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-10T21:33:52.547 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-10T21:33:52.547 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "5e2f023f-d6a6-49f4-a256-f4807dce3739", 2024-09-10T21:33:52.548 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:33:52.548 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:33:52.548 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-10T21:33:52.548 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-10T21:33:52.548 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-10T21:33:52.548 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-10T21:33:52.548 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-10T21:33:52.548 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-10T21:33:52.548 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-10T21:33:52.548 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:33:52.548 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-10T21:33:52.548 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:33:52.548 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:33:52.549 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:33:52.549 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:33:52.549 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:33:52.549 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:33:52.549 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:33:52.549 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:33:52.549 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:33:52.549 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:33:52.549 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:33:52.549 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:33:52.549 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-10T21:33:52.549 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:33:52.549 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-10T21:33:52.550 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:33:52.550 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:33:52.550 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:33:52.550 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:33:52.550 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:33:52.550 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:33:52.550 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:33:52.550 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-10T21:33:52.550 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:33:52.550 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:33:52.550 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:33:52.550 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:33:52.550 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-10T21:33:52.551 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:33:52.551 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:33:52.551 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:33:52.551 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:33:52.551 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-10T21:33:52.551 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-10T21:33:52.551 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:33:52.551 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:33:52.551 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:33:52.551 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:33:52.551 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:33:52.551 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:33:52.551 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:33:52.552 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:33:52.552 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:33:52.552 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:33:52.552 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:33:52.552 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3300", 2024-09-10T21:33:52.552 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:33:52.552 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:33:52.552 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:33:52.552 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:33:52.552 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789", 2024-09-10T21:33:52.552 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:33:52.552 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:33:52.552 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:33:52.552 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:33:52.553 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789/0", 2024-09-10T21:33:52.553 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6789/0", 2024-09-10T21:33:52.553 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:33:52.553 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:33:52.553 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:33:52.553 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:33:52.553 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:33:52.553 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:33:52.553 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:33:52.553 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:33:52.553 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:33:52.553 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:33:52.553 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:33:52.554 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3301", 2024-09-10T21:33:52.554 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:33:52.554 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:33:52.554 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:33:52.554 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:33:52.554 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790", 2024-09-10T21:33:52.554 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:33:52.554 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:33:52.554 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:33:52.554 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:33:52.554 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790/0", 2024-09-10T21:33:52.554 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6790/0", 2024-09-10T21:33:52.554 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:33:52.555 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:33:52.555 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:33:52.555 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:33:52.555 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:33:52.555 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:33:52.555 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-10T21:33:52.555 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-10T21:33:52.555 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:33:52.555 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:33:52.555 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:33:52.555 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:33:52.555 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:33:52.555 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:33:52.555 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-10T21:33:52.556 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:33:52.556 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:33:52.556 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:33:52.556 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 2 2024-09-10T21:33:52.556 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:33:52.556 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:33:52.556 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:33:52.556 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-10T21:33:52.556 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-10T21:33:52.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.537+0000 7f063be00640 1 -- 172.21.15.17:0/769961136 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f06140724e0 msgr2=0x7f06140749a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:52.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.537+0000 7f063be00640 1 --2- 172.21.15.17:0/769961136 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f06140724e0 0x7f06140749a0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f062c001850 tx=0x7f062c0017e0 comp rx=0 tx=0).stop 2024-09-10T21:33:52.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.537+0000 7f063be00640 1 -- 172.21.15.17:0/769961136 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f063406ed90 msgr2=0x7f06341f39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:52.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.537+0000 7f063be00640 1 --2- 172.21.15.17:0/769961136 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f063406ed90 0x7f06341f39d0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f062800e7f0 tx=0x7f062800ecb0 comp rx=0 tx=0).stop 2024-09-10T21:33:52.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.537+0000 7f0639e00640 1 -- 172.21.15.17:0/769961136 reap_dead start 2024-09-10T21:33:52.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.537+0000 7f0639e00640 1 -- 172.21.15.17:0/769961136 reap_dead start 2024-09-10T21:33:52.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.537+0000 7f063be00640 1 -- 172.21.15.17:0/769961136 shutdown_connections 2024-09-10T21:33:52.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.537+0000 7f063be00640 1 -- 172.21.15.17:0/769961136 >> 172.21.15.17:0/769961136 conn(0x7f063406dd30 msgr2=0x7f0634157d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:52.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.537+0000 7f063be00640 1 -- 172.21.15.17:0/769961136 shutdown_connections 2024-09-10T21:33:52.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.537+0000 7f063be00640 1 -- 172.21.15.17:0/769961136 wait complete. 2024-09-10T21:33:52.560 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-10T21:33:52.561 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-09-10T21:33:52.561 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-10T21:33:52.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.733+0000 7f434b200640 1 -- 172.21.15.17:0/4009314845 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f434c152070 msgr2=0x7f434c072420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:52.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.733+0000 7f434b200640 1 --2- 172.21.15.17:0/4009314845 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f434c152070 0x7f434c072420 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f433c009900 tx=0x7f433c01f0d0 comp rx=0 tx=0).stop 2024-09-10T21:33:52.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.733+0000 7f434b200640 1 -- 172.21.15.17:0/4009314845 shutdown_connections 2024-09-10T21:33:52.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.733+0000 7f434b200640 1 --2- 172.21.15.17:0/4009314845 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f434c152450 0x7f434c072960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:52.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.733+0000 7f434b200640 1 --2- 172.21.15.17:0/4009314845 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f434c152070 0x7f434c072420 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:52.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.733+0000 7f434b200640 1 -- 172.21.15.17:0/4009314845 >> 172.21.15.17:0/4009314845 conn(0x7f434c06cc20 msgr2=0x7f434c06d030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:52.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.733+0000 7f434b200640 1 -- 172.21.15.17:0/4009314845 shutdown_connections 2024-09-10T21:33:52.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.733+0000 7f434b200640 1 -- 172.21.15.17:0/4009314845 wait complete. 2024-09-10T21:33:52.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.733+0000 7f434b200640 1 Processor -- start 2024-09-10T21:33:52.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.733+0000 7f434b200640 1 -- start start 2024-09-10T21:33:52.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.733+0000 7f434b200640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f434c0cbd10 0x7f434c14c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:52.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.733+0000 7f434b200640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f434c152070 0x7f434c0ca120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:52.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.733+0000 7f434b200640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f434c152450 0x7f434c1f1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:52.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.733+0000 7f434b200640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f434c0cecf0 con 0x7f434c0cbd10 2024-09-10T21:33:52.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.733+0000 7f434b200640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f434c0c5090 con 0x7f434c152070 2024-09-10T21:33:52.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.733+0000 7f434b200640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f434c0cee70 con 0x7f434c152450 2024-09-10T21:33:52.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.737+0000 7f4349e00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f434c0cbd10 0x7f434c14c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:52.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.737+0000 7f4349e00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f434c0cbd10 0x7f434c14c7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:60594/0 (socket says 172.21.15.17:60594) 2024-09-10T21:33:52.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.737+0000 7f4349e00640 1 -- 172.21.15.17:0/2924816125 learned_addr learned my addr 172.21.15.17:0/2924816125 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:33:52.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.737+0000 7f4349400640 1 --2- 172.21.15.17:0/2924816125 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f434c152070 0x7f434c0ca120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:52.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.737+0000 7f434a800640 1 --2- 172.21.15.17:0/2924816125 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f434c152450 0x7f434c1f1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:52.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.737+0000 7f4349e00640 1 -- 172.21.15.17:0/2924816125 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f434c152450 msgr2=0x7f434c1f1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:52.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.737+0000 7f4349e00640 1 --2- 172.21.15.17:0/2924816125 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f434c152450 0x7f434c1f1ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:52.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.737+0000 7f4349e00640 1 -- 172.21.15.17:0/2924816125 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f434c152070 msgr2=0x7f434c0ca120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:52.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.737+0000 7f4349e00640 1 --2- 172.21.15.17:0/2924816125 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f434c152070 0x7f434c0ca120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:52.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.737+0000 7f4349e00640 1 -- 172.21.15.17:0/2924816125 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f434c151580 con 0x7f434c0cbd10 2024-09-10T21:33:52.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.737+0000 7f4349e00640 1 --2- 172.21.15.17:0/2924816125 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f434c0cbd10 0x7f434c14c7d0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f433c009850 tx=0x7f433c0a8ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:52.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.737+0000 7f4342000640 1 -- 172.21.15.17:0/2924816125 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f433c0be070 con 0x7f434c0cbd10 2024-09-10T21:33:52.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.737+0000 7f4342000640 1 -- 172.21.15.17:0/2924816125 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f433c005400 con 0x7f434c0cbd10 2024-09-10T21:33:52.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.737+0000 7f4342000640 1 -- 172.21.15.17:0/2924816125 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f433c0b5040 con 0x7f434c0cbd10 2024-09-10T21:33:52.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.737+0000 7f434b200640 1 -- 172.21.15.17:0/2924816125 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f434c0c85e0 con 0x7f434c0cbd10 2024-09-10T21:33:52.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.737+0000 7f434b200640 1 -- 172.21.15.17:0/2924816125 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f434c1eedb0 con 0x7f434c0cbd10 2024-09-10T21:33:52.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.737+0000 7f434b200640 1 --2- 172.21.15.17:0/2924816125 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f434c1f2530 0x7f434c1f1880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:52.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.737+0000 7f434b200640 1 -- 172.21.15.17:0/2924816125 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f434c0c9660 con 0x7f434c1f2530 2024-09-10T21:33:52.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.737+0000 7f4349400640 1 --2- 172.21.15.17:0/2924816125 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f434c1f2530 0x7f434c1f1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:52.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.737+0000 7f4349400640 1 -- 172.21.15.17:0/2924816125 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f434c071e10 con 0x7f434c1f2530 2024-09-10T21:33:52.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.737+0000 7f4349400640 1 --2- 172.21.15.17:0/2924816125 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f434c1f2530 0x7f434c1f1880 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f434c155d50 tx=0x7f433800b9b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:52.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.741+0000 7f4342000640 1 -- 172.21.15.17:0/2924816125 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f433c0b8030 con 0x7f434c0cbd10 2024-09-10T21:33:52.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.741+0000 7f4342000640 1 --2- 172.21.15.17:0/2924816125 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f432c072600 0x7f432c074ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:52.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.741+0000 7f4342000640 1 -- 172.21.15.17:0/2924816125 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f433c1303a0 con 0x7f434c0cbd10 2024-09-10T21:33:52.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.741+0000 7f4342000640 1 -- 172.21.15.17:0/2924816125 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4338005220 con 0x7f434c1f2530 2024-09-10T21:33:52.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.741+0000 7f4342000640 1 -- 172.21.15.17:0/2924816125 <== mon.1 v2:172.21.15.31:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f4338015070 con 0x7f434c1f2530 2024-09-10T21:33:52.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.741+0000 7f434a800640 1 --2- 172.21.15.17:0/2924816125 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f432c072600 0x7f432c074ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:52.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.741+0000 7f4342000640 1 -- 172.21.15.17:0/2924816125 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f434c1f2530 msgr2=0x7f434c1f1880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:52.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.741+0000 7f4342000640 1 --2- 172.21.15.17:0/2924816125 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f434c1f2530 0x7f434c1f1880 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f434c155d50 tx=0x7f433800b9b0 comp rx=0 tx=0).stop 2024-09-10T21:33:52.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.741+0000 7f4342000640 1 -- 172.21.15.17:0/2924816125 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f434c1f2530 msgr2=0x7f434c1f1880 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:33:52.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.741+0000 7f4342000640 1 --2- 172.21.15.17:0/2924816125 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f434c1f2530 0x7f434c1f1880 secure :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f434c155d50 tx=0x7f433800b9b0 comp rx=0 tx=0).stop 2024-09-10T21:33:52.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.741+0000 7f434a800640 1 --2- 172.21.15.17:0/2924816125 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f432c072600 0x7f432c074ac0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f43440012f0 tx=0x7f4344009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:52.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.749+0000 7f434b200640 1 --2- 172.21.15.17:0/2924816125 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f434c1e16d0 0x7f434c1e1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:52.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.749+0000 7f434b200640 1 -- 172.21.15.17:0/2924816125 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f434c069ef0 con 0x7f434c1e16d0 2024-09-10T21:33:52.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.749+0000 7f4349400640 1 --2- 172.21.15.17:0/2924816125 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f434c1e16d0 0x7f434c1e1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:52.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.749+0000 7f4349400640 1 -- 172.21.15.17:0/2924816125 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f434c0cc710 con 0x7f434c1e16d0 2024-09-10T21:33:52.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.749+0000 7f4349400640 1 --2- 172.21.15.17:0/2924816125 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f434c1e16d0 0x7f434c1e1ab0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f434c156900 tx=0x7f43380046d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:52.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.749+0000 7f4342000640 1 -- 172.21.15.17:0/2924816125 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4338005660 con 0x7f434c1e16d0 2024-09-10T21:33:52.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.749+0000 7f4342000640 1 -- 172.21.15.17:0/2924816125 <== mon.1 v2:172.21.15.31:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5042 (secure 0 0 0) 0x7f4338004bf0 con 0x7f434c1e16d0 2024-09-10T21:33:52.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.749+0000 7f4342000640 1 -- 172.21.15.17:0/2924816125 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f434c1e16d0 msgr2=0x7f434c1e1ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:52.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.749+0000 7f4342000640 1 --2- 172.21.15.17:0/2924816125 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f434c1e16d0 0x7f434c1e1ab0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f434c156900 tx=0x7f43380046d0 comp rx=0 tx=0).stop 2024-09-10T21:33:52.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.749+0000 7f4342000640 1 -- 172.21.15.17:0/2924816125 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f434c1e16d0 msgr2=0x7f434c1e1ab0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:33:52.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.749+0000 7f4342000640 1 --2- 172.21.15.17:0/2924816125 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f434c1e16d0 0x7f434c1e1ab0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:52.755 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-10T21:33:52.755 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:33:52.755 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:33:52.755 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-10T21:33:52.755 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 80, 2024-09-10T21:33:52.755 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 571199, 2024-09-10T21:33:52.755 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-10T21:33:52.755 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-10T21:33:52.756 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-10T21:33:52.756 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-10T21:33:52.756 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:33:52.756 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 0, 2024-09-10T21:33:52.756 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:33:52.756 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-10T21:33:52.756 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-10T21:33:52.756 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:33:52.756 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:33:52.756 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:33:52.756 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:33:52.757 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:33:52.757 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:33:52.757 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:33:52.757 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:33:52.757 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:33:52.757 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:33:52.757 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:33:52.757 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:33:52.757 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-10T21:33:52.757 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-10T21:33:52.757 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:33:52.757 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:33:52.757 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:33:52.757 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:33:52.758 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:33:52.758 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:33:52.758 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:33:52.758 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:33:52.758 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:33:52.758 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:33:52.758 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:33:52.758 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:33:52.758 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:33:52.758 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-10T21:33:52.758 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-10T21:33:52.758 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-10T21:33:52.758 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-10T21:33:52.759 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-10T21:33:52.759 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "5e2f023f-d6a6-49f4-a256-f4807dce3739", 2024-09-10T21:33:52.759 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:33:52.759 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:33:52.759 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-10T21:33:52.759 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-10T21:33:52.759 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-10T21:33:52.759 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-10T21:33:52.759 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-10T21:33:52.759 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-10T21:33:52.759 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-10T21:33:52.759 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:33:52.759 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-10T21:33:52.760 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:33:52.760 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:33:52.760 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:33:52.760 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:33:52.760 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:33:52.760 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:33:52.760 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:33:52.760 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:33:52.760 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:33:52.760 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:33:52.760 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:33:52.760 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:33:52.760 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-10T21:33:52.760 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:33:52.761 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-10T21:33:52.761 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:33:52.761 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:33:52.761 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:33:52.761 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:33:52.761 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:33:52.761 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:33:52.761 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:33:52.761 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-10T21:33:52.761 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:33:52.761 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:33:52.761 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:33:52.761 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:33:52.762 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-10T21:33:52.762 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:33:52.762 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:33:52.762 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:33:52.762 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:33:52.762 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-10T21:33:52.762 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-10T21:33:52.762 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:33:52.762 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:33:52.762 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:33:52.762 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:33:52.762 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:33:52.762 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:33:52.763 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:33:52.763 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:33:52.763 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:33:52.763 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:33:52.763 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:33:52.763 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3300", 2024-09-10T21:33:52.763 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:33:52.763 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:33:52.763 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:33:52.763 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:33:52.763 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789", 2024-09-10T21:33:52.764 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:33:52.764 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:33:52.764 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:33:52.764 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:33:52.764 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789/0", 2024-09-10T21:33:52.764 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6789/0", 2024-09-10T21:33:52.764 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:33:52.764 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:33:52.764 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:33:52.764 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:33:52.764 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:33:52.764 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:33:52.764 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:33:52.765 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:33:52.765 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:33:52.765 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:33:52.765 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:33:52.765 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3301", 2024-09-10T21:33:52.765 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:33:52.765 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:33:52.765 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:33:52.765 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:33:52.765 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790", 2024-09-10T21:33:52.765 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:33:52.765 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:33:52.765 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:33:52.766 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:33:52.766 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790/0", 2024-09-10T21:33:52.766 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6790/0", 2024-09-10T21:33:52.766 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:33:52.766 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:33:52.766 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:33:52.766 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:33:52.766 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:33:52.766 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:33:52.766 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-10T21:33:52.766 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-10T21:33:52.766 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:33:52.766 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:33:52.767 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:33:52.767 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:33:52.767 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:33:52.767 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:33:52.767 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": [ 2024-09-10T21:33:52.767 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:33:52.767 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:33:52.767 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:33:52.767 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 5 2024-09-10T21:33:52.767 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:33:52.767 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:33:52.767 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-10T21:33:52.767 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:33:52.768 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:33:52.768 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:33:52.768 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 7 2024-09-10T21:33:52.768 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:33:52.768 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:33:52.768 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-10T21:33:52.768 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:33:52.768 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:33:52.768 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:33:52.768 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 6 2024-09-10T21:33:52.768 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:33:52.768 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:33:52.768 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": [ 2024-09-10T21:33:52.769 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:33:52.769 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:33:52.769 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:33:52.769 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 2 2024-09-10T21:33:52.769 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:33:52.769 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:33:52.769 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:33:52.769 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-10T21:33:52.769 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-10T21:33:52.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.749+0000 7f434b200640 1 -- 172.21.15.17:0/2924816125 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f432c072600 msgr2=0x7f432c074ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:52.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.749+0000 7f434b200640 1 --2- 172.21.15.17:0/2924816125 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f432c072600 0x7f432c074ac0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f43440012f0 tx=0x7f4344009040 comp rx=0 tx=0).stop 2024-09-10T21:33:52.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.749+0000 7f434b200640 1 -- 172.21.15.17:0/2924816125 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f434c0cbd10 msgr2=0x7f434c14c7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:52.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.749+0000 7f434b200640 1 --2- 172.21.15.17:0/2924816125 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f434c0cbd10 0x7f434c14c7d0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f433c009850 tx=0x7f433c0a8ce0 comp rx=0 tx=0).stop 2024-09-10T21:33:52.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.749+0000 7f434a800640 1 -- 172.21.15.17:0/2924816125 reap_dead start 2024-09-10T21:33:52.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.749+0000 7f434a800640 1 -- 172.21.15.17:0/2924816125 reap_dead start 2024-09-10T21:33:52.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.749+0000 7f434b200640 1 -- 172.21.15.17:0/2924816125 shutdown_connections 2024-09-10T21:33:52.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.749+0000 7f434b200640 1 -- 172.21.15.17:0/2924816125 >> 172.21.15.17:0/2924816125 conn(0x7f434c06cc20 msgr2=0x7f434c155ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:52.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.749+0000 7f434b200640 1 -- 172.21.15.17:0/2924816125 shutdown_connections 2024-09-10T21:33:52.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.749+0000 7f434b200640 1 -- 172.21.15.17:0/2924816125 wait complete. 2024-09-10T21:33:52.775 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-10T21:33:52.775 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-09-10T21:33:52.775 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-10T21:33:52.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.929+0000 7f3c46c00640 1 -- 172.21.15.17:0/3351656076 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3c400c99d0 msgr2=0x7f3c400cb8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:52.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.929+0000 7f3c46c00640 1 --2- 172.21.15.17:0/3351656076 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3c400c99d0 0x7f3c400cb8d0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f3c38009bc0 tx=0x7f3c3801cba0 comp rx=0 tx=0).stop 2024-09-10T21:33:52.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.929+0000 7f3c46c00640 1 -- 172.21.15.17:0/3351656076 shutdown_connections 2024-09-10T21:33:52.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.929+0000 7f3c46c00640 1 --2- 172.21.15.17:0/3351656076 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3c400cbf40 0x7f3c400c9db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:52.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.929+0000 7f3c46c00640 1 --2- 172.21.15.17:0/3351656076 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3c400c99d0 0x7f3c400cb8d0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:52.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.929+0000 7f3c46c00640 1 -- 172.21.15.17:0/3351656076 >> 172.21.15.17:0/3351656076 conn(0x7f3c4006e470 msgr2=0x7f3c4006ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:52.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.929+0000 7f3c46c00640 1 -- 172.21.15.17:0/3351656076 shutdown_connections 2024-09-10T21:33:52.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.929+0000 7f3c46c00640 1 -- 172.21.15.17:0/3351656076 wait complete. 2024-09-10T21:33:52.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.933+0000 7f3c46c00640 1 Processor -- start 2024-09-10T21:33:52.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.933+0000 7f3c46c00640 1 -- start start 2024-09-10T21:33:52.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.933+0000 7f3c46c00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3c400c99d0 0x7f3c400c63c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:52.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.933+0000 7f3c46c00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3c400cbf40 0x7f3c400cfb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:52.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.933+0000 7f3c46c00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3c4015add0 0x7f3c40160080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:52.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.933+0000 7f3c46c00640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f3c400d0800 con 0x7f3c400cbf40 2024-09-10T21:33:52.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.933+0000 7f3c46c00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f3c4015b610 con 0x7f3c4015add0 2024-09-10T21:33:52.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.933+0000 7f3c46c00640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f3c400d0980 con 0x7f3c400c99d0 2024-09-10T21:33:52.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.933+0000 7f3c44e00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3c400cbf40 0x7f3c400cfb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:52.939 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.933+0000 7f3c44e00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3c400cbf40 0x7f3c400cfb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:60608/0 (socket says 172.21.15.17:60608) 2024-09-10T21:33:52.939 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.933+0000 7f3c44e00640 1 -- 172.21.15.17:0/1649116630 learned_addr learned my addr 172.21.15.17:0/1649116630 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:33:52.939 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.933+0000 7f3c45800640 1 --2- 172.21.15.17:0/1649116630 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3c400c99d0 0x7f3c400c63c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:52.939 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.933+0000 7f3c46200640 1 --2- 172.21.15.17:0/1649116630 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3c4015add0 0x7f3c40160080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:52.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.933+0000 7f3c44e00640 1 -- 172.21.15.17:0/1649116630 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3c400c99d0 msgr2=0x7f3c400c63c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:52.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.933+0000 7f3c44e00640 1 --2- 172.21.15.17:0/1649116630 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3c400c99d0 0x7f3c400c63c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:52.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.933+0000 7f3c44e00640 1 -- 172.21.15.17:0/1649116630 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3c4015add0 msgr2=0x7f3c40160080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:52.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.933+0000 7f3c44e00640 1 --2- 172.21.15.17:0/1649116630 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3c4015add0 0x7f3c40160080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:52.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.933+0000 7f3c44e00640 1 -- 172.21.15.17:0/1649116630 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c400c6b00 con 0x7f3c400cbf40 2024-09-10T21:33:52.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.933+0000 7f3c44e00640 1 --2- 172.21.15.17:0/1649116630 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3c400cbf40 0x7f3c400cfb10 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f3c2c00e7a0 tx=0x7f3c2c00ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:52.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.933+0000 7f3c3d600640 1 -- 172.21.15.17:0/1649116630 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c2c00ccc0 con 0x7f3c400cbf40 2024-09-10T21:33:52.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.933+0000 7f3c3d600640 1 -- 172.21.15.17:0/1649116630 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3c2c00ce60 con 0x7f3c400cbf40 2024-09-10T21:33:52.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.933+0000 7f3c46c00640 1 -- 172.21.15.17:0/1649116630 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c401ec650 con 0x7f3c400cbf40 2024-09-10T21:33:52.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.933+0000 7f3c3d600640 1 -- 172.21.15.17:0/1649116630 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c2c004260 con 0x7f3c400cbf40 2024-09-10T21:33:52.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.933+0000 7f3c46c00640 1 -- 172.21.15.17:0/1649116630 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c401ecc10 con 0x7f3c400cbf40 2024-09-10T21:33:52.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.937+0000 7f3c46c00640 1 --2- 172.21.15.17:0/1649116630 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3c401ece50 0x7f3c401605c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:52.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.937+0000 7f3c46c00640 1 -- 172.21.15.17:0/1649116630 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f3c40069ef0 con 0x7f3c401ece50 2024-09-10T21:33:52.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.937+0000 7f3c45800640 1 --2- 172.21.15.17:0/1649116630 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3c401ece50 0x7f3c401605c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:52.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.937+0000 7f3c45800640 1 -- 172.21.15.17:0/1649116630 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3c400cb8d0 con 0x7f3c401ece50 2024-09-10T21:33:52.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.937+0000 7f3c45800640 1 --2- 172.21.15.17:0/1649116630 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3c401ece50 0x7f3c401605c0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f3c400c71e0 tx=0x7f3c38004a50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:52.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.937+0000 7f3c3d600640 1 -- 172.21.15.17:0/1649116630 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c380bb030 con 0x7f3c401ece50 2024-09-10T21:33:52.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.937+0000 7f3c3d600640 1 -- 172.21.15.17:0/1649116630 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f3c2c003600 con 0x7f3c400cbf40 2024-09-10T21:33:52.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.937+0000 7f3c3d600640 1 --2- 172.21.15.17:0/1649116630 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f3c20072510 0x7f3c200749d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:52.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.937+0000 7f3c3d600640 1 -- 172.21.15.17:0/1649116630 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f3c2c093ec0 con 0x7f3c400cbf40 2024-09-10T21:33:52.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.937+0000 7f3c3d600640 1 -- 172.21.15.17:0/1649116630 <== mon.2 v2:172.21.15.31:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f3c38003980 con 0x7f3c401ece50 2024-09-10T21:33:52.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.937+0000 7f3c3d600640 1 -- 172.21.15.17:0/1649116630 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3c401ece50 msgr2=0x7f3c401605c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:52.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.937+0000 7f3c3d600640 1 --2- 172.21.15.17:0/1649116630 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3c401ece50 0x7f3c401605c0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f3c400c71e0 tx=0x7f3c38004a50 comp rx=0 tx=0).stop 2024-09-10T21:33:52.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.937+0000 7f3c3d600640 1 -- 172.21.15.17:0/1649116630 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3c401ece50 msgr2=0x7f3c401605c0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:33:52.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.937+0000 7f3c3d600640 1 --2- 172.21.15.17:0/1649116630 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3c401ece50 0x7f3c401605c0 secure :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f3c400c71e0 tx=0x7f3c38004a50 comp rx=0 tx=0).stop 2024-09-10T21:33:52.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.937+0000 7f3c46200640 1 --2- 172.21.15.17:0/1649116630 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f3c20072510 0x7f3c200749d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:52.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.941+0000 7f3c46200640 1 --2- 172.21.15.17:0/1649116630 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f3c20072510 0x7f3c200749d0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f3c30007fd0 tx=0x7f3c30009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:52.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.945+0000 7f3c46c00640 1 --2- 172.21.15.17:0/1649116630 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3c40158ec0 0x7f3c401574e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:52.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.945+0000 7f3c46c00640 1 -- 172.21.15.17:0/1649116630 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f3c40158610 con 0x7f3c40158ec0 2024-09-10T21:33:52.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.945+0000 7f3c45800640 1 --2- 172.21.15.17:0/1649116630 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3c40158ec0 0x7f3c401574e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:52.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.945+0000 7f3c45800640 1 -- 172.21.15.17:0/1649116630 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f3c400008e0 con 0x7f3c40158ec0 2024-09-10T21:33:52.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.945+0000 7f3c45800640 1 --2- 172.21.15.17:0/1649116630 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3c40158ec0 0x7f3c401574e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f3c38000f90 tx=0x7f3c38009500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:52.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.945+0000 7f3c3d600640 1 -- 172.21.15.17:0/1649116630 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c380b2040 con 0x7f3c40158ec0 2024-09-10T21:33:52.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.945+0000 7f3c3d600640 1 -- 172.21.15.17:0/1649116630 <== mon.2 v2:172.21.15.31:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4875 (secure 0 0 0) 0x7f3c38004250 con 0x7f3c40158ec0 2024-09-10T21:33:52.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.945+0000 7f3c3d600640 1 -- 172.21.15.17:0/1649116630 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3c40158ec0 msgr2=0x7f3c401574e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:52.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.945+0000 7f3c3d600640 1 --2- 172.21.15.17:0/1649116630 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3c40158ec0 0x7f3c401574e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f3c38000f90 tx=0x7f3c38009500 comp rx=0 tx=0).stop 2024-09-10T21:33:52.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.945+0000 7f3c3d600640 1 -- 172.21.15.17:0/1649116630 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3c40158ec0 msgr2=0x7f3c401574e0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:33:52.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.945+0000 7f3c3d600640 1 --2- 172.21.15.17:0/1649116630 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3c40158ec0 0x7f3c401574e0 secure :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f3c38000f90 tx=0x7f3c38009500 comp rx=0 tx=0).stop 2024-09-10T21:33:52.952 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-10T21:33:52.952 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:33:52.952 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:33:52.952 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-10T21:33:52.952 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 80, 2024-09-10T21:33:52.953 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 165254, 2024-09-10T21:33:52.953 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-10T21:33:52.953 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-10T21:33:52.953 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-10T21:33:52.953 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-10T21:33:52.953 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:33:52.953 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 0, 2024-09-10T21:33:52.953 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:33:52.953 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-10T21:33:52.953 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-10T21:33:52.953 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:33:52.953 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:33:52.956 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:33:52.956 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:33:52.957 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:33:52.957 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:33:52.957 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:33:52.957 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:33:52.957 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:33:52.957 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:33:52.957 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:33:52.957 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:33:52.957 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-10T21:33:52.957 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-10T21:33:52.957 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:33:52.957 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:33:52.957 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:33:52.957 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:33:52.958 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:33:52.958 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:33:52.958 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:33:52.958 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:33:52.958 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:33:52.958 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:33:52.958 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:33:52.958 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:33:52.958 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:33:52.958 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-10T21:33:52.958 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-10T21:33:52.958 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-10T21:33:52.958 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-10T21:33:52.959 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-10T21:33:52.959 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "5e2f023f-d6a6-49f4-a256-f4807dce3739", 2024-09-10T21:33:52.959 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:33:52.959 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:33:52.959 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-10T21:33:52.959 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-10T21:33:52.959 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-10T21:33:52.959 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-10T21:33:52.959 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-10T21:33:52.959 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-10T21:33:52.959 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-10T21:33:52.960 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:33:52.960 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-10T21:33:52.960 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:33:52.960 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:33:52.960 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:33:52.960 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:33:52.960 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:33:52.960 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:33:52.960 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:33:52.960 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:33:52.960 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:33:52.960 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:33:52.960 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:33:52.961 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:33:52.961 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-10T21:33:52.961 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:33:52.961 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-10T21:33:52.961 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:33:52.961 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:33:52.961 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:33:52.961 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:33:52.961 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:33:52.961 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:33:52.961 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:33:52.961 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-10T21:33:52.961 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:33:52.962 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:33:52.962 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:33:52.962 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:33:52.962 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-10T21:33:52.962 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:33:52.962 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:33:52.962 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:33:52.962 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:33:52.962 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-10T21:33:52.962 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-10T21:33:52.962 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:33:52.962 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:33:52.962 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:33:52.963 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:33:52.963 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:33:52.963 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:33:52.963 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:33:52.963 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:33:52.963 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:33:52.963 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:33:52.963 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:33:52.963 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3300", 2024-09-10T21:33:52.963 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:33:52.963 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:33:52.963 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:33:52.963 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:33:52.963 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789", 2024-09-10T21:33:52.963 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:33:52.964 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:33:52.964 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:33:52.964 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:33:52.964 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789/0", 2024-09-10T21:33:52.964 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6789/0", 2024-09-10T21:33:52.964 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:33:52.964 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:33:52.964 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:33:52.964 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:33:52.964 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:33:52.964 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:33:52.964 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:33:52.964 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:33:52.964 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:33:52.965 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:33:52.965 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:33:52.965 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3301", 2024-09-10T21:33:52.965 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:33:52.965 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:33:52.965 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:33:52.965 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:33:52.965 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790", 2024-09-10T21:33:52.965 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:33:52.965 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:33:52.965 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:33:52.965 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:33:52.965 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790/0", 2024-09-10T21:33:52.966 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6790/0", 2024-09-10T21:33:52.966 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:33:52.966 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:33:52.966 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:33:52.966 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:33:52.966 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:33:52.966 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:33:52.966 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-10T21:33:52.966 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-10T21:33:52.966 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:33:52.966 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:33:52.966 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:33:52.967 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:33:52.967 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:33:52.967 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:33:52.967 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": [ 2024-09-10T21:33:52.967 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:33:52.967 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:33:52.967 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:33:52.967 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:33:52.967 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:33:52.967 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:33:52.967 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-10T21:33:52.967 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:33:52.967 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:33:52.967 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:33:52.968 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:33:52.968 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:33:52.968 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:33:52.968 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-10T21:33:52.968 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:33:52.968 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:33:52.968 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:33:52.968 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:33:52.968 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:33:52.968 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:33:52.968 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:33:52.968 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-10T21:33:52.968 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-10T21:33:52.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.945+0000 7f3c46c00640 1 -- 172.21.15.17:0/1649116630 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f3c20072510 msgr2=0x7f3c200749d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:52.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.945+0000 7f3c46c00640 1 --2- 172.21.15.17:0/1649116630 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f3c20072510 0x7f3c200749d0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f3c30007fd0 tx=0x7f3c30009040 comp rx=0 tx=0).stop 2024-09-10T21:33:52.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.945+0000 7f3c46c00640 1 -- 172.21.15.17:0/1649116630 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3c400cbf40 msgr2=0x7f3c400cfb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:52.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.945+0000 7f3c46c00640 1 --2- 172.21.15.17:0/1649116630 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3c400cbf40 0x7f3c400cfb10 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f3c2c00e7a0 tx=0x7f3c2c00ec60 comp rx=0 tx=0).stop 2024-09-10T21:33:52.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.945+0000 7f3c46200640 1 -- 172.21.15.17:0/1649116630 reap_dead start 2024-09-10T21:33:52.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.945+0000 7f3c46200640 1 -- 172.21.15.17:0/1649116630 reap_dead start 2024-09-10T21:33:52.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.945+0000 7f3c46c00640 1 -- 172.21.15.17:0/1649116630 shutdown_connections 2024-09-10T21:33:52.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.945+0000 7f3c46c00640 1 -- 172.21.15.17:0/1649116630 >> 172.21.15.17:0/1649116630 conn(0x7f3c4006e470 msgr2=0x7f3c40159510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:52.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.945+0000 7f3c46c00640 1 -- 172.21.15.17:0/1649116630 shutdown_connections 2024-09-10T21:33:52.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:52.949+0000 7f3c46c00640 1 -- 172.21.15.17:0/1649116630 wait complete. 2024-09-10T21:33:52.971 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-10T21:33:52.971 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-09-10T21:33:52.972 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-09-10T21:33:53.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:53.137+0000 7f0662400640 1 -- 172.21.15.17:0/2856991028 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f065c0c7160 msgr2=0x7f065c1572a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:53.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:53.137+0000 7f0662400640 1 --2- 172.21.15.17:0/2856991028 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f065c0c7160 0x7f065c1572a0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f0654009c80 tx=0x7f065401f5a0 comp rx=0 tx=0).stop 2024-09-10T21:33:53.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:53.137+0000 7f0662400640 1 -- 172.21.15.17:0/2856991028 shutdown_connections 2024-09-10T21:33:53.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:53.137+0000 7f0662400640 1 --2- 172.21.15.17:0/2856991028 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f065c0c7160 0x7f065c1572a0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:53.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:53.137+0000 7f0662400640 1 -- 172.21.15.17:0/2856991028 >> 172.21.15.17:0/2856991028 conn(0x7f065c06cdf0 msgr2=0x7f065c06d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:53.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:53.137+0000 7f0662400640 1 -- 172.21.15.17:0/2856991028 shutdown_connections 2024-09-10T21:33:53.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:53.137+0000 7f0662400640 1 -- 172.21.15.17:0/2856991028 wait complete. 2024-09-10T21:33:53.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:53.137+0000 7f0662400640 1 Processor -- start 2024-09-10T21:33:53.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:53.137+0000 7f0662400640 1 -- start start 2024-09-10T21:33:53.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:53.141+0000 7f0662400640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f065c0c7160 0x7f065c14d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:53.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:53.141+0000 7f0662400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f065c1508e0 0x7f065c14dc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:53.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:53.141+0000 7f0662400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f065c150cc0 0x7f065c0c2a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:53.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:53.141+0000 7f0662400640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f065c15a690 con 0x7f065c150cc0 2024-09-10T21:33:53.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:53.141+0000 7f0662400640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f065c15a3b0 con 0x7f065c1508e0 2024-09-10T21:33:53.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:53.141+0000 7f0662400640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f065c0c6820 con 0x7f065c0c7160 2024-09-10T21:33:53.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:53.141+0000 7f0661a00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f065c150cc0 0x7f065c0c2a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:53.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:53.141+0000 7f0661a00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f065c150cc0 0x7f065c0c2a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:60622/0 (socket says 172.21.15.17:60622) 2024-09-10T21:33:53.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:53.141+0000 7f0661a00640 1 -- 172.21.15.17:0/328554970 learned_addr learned my addr 172.21.15.17:0/328554970 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:33:53.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:53.141+0000 7f065be00640 1 --2- 172.21.15.17:0/328554970 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f065c1508e0 0x7f065c14dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:53.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:53.141+0000 7f0661000640 1 --2- 172.21.15.17:0/328554970 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f065c0c7160 0x7f065c14d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:53.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:53.141+0000 7f0661a00640 1 -- 172.21.15.17:0/328554970 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f065c0c7160 msgr2=0x7f065c14d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:53.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:53.141+0000 7f0661a00640 1 --2- 172.21.15.17:0/328554970 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f065c0c7160 0x7f065c14d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:53.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:53.141+0000 7f0661a00640 1 -- 172.21.15.17:0/328554970 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f065c1508e0 msgr2=0x7f065c14dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:53.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:53.141+0000 7f0661a00640 1 --2- 172.21.15.17:0/328554970 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f065c1508e0 0x7f065c14dc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:53.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:53.141+0000 7f0661a00640 1 -- 172.21.15.17:0/328554970 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f065c14ee30 con 0x7f065c150cc0 2024-09-10T21:33:53.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:53.141+0000 7f0661000640 1 --2- 172.21.15.17:0/328554970 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f065c0c7160 0x7f065c14d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:33:53.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:53.141+0000 7f0661a00640 1 --2- 172.21.15.17:0/328554970 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f065c150cc0 0x7f065c0c2a20 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f0654009bd0 tx=0x7f06540af570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:53.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:53.141+0000 7f0658c00640 1 -- 172.21.15.17:0/328554970 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0654004e60 con 0x7f065c150cc0 2024-09-10T21:33:53.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:53.141+0000 7f0658c00640 1 -- 172.21.15.17:0/328554970 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f06540be030 con 0x7f065c150cc0 2024-09-10T21:33:53.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:53.141+0000 7f0658c00640 1 -- 172.21.15.17:0/328554970 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06540afc30 con 0x7f065c150cc0 2024-09-10T21:33:53.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:53.141+0000 7f0662400640 1 -- 172.21.15.17:0/328554970 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f065c14f0d0 con 0x7f065c150cc0 2024-09-10T21:33:53.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:53.141+0000 7f0662400640 1 -- 172.21.15.17:0/328554970 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f065c14f670 con 0x7f065c150cc0 2024-09-10T21:33:53.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:53.141+0000 7f0662400640 1 -- 172.21.15.17:0/328554970 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f065c15abb0 con 0x7f065c150cc0 2024-09-10T21:33:53.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:53.145+0000 7f0658c00640 1 -- 172.21.15.17:0/328554970 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f0654004000 con 0x7f065c150cc0 2024-09-10T21:33:53.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:53.145+0000 7f0658c00640 1 --2- 172.21.15.17:0/328554970 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f063c0725e0 0x7f063c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:33:53.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:53.145+0000 7f0658c00640 1 -- 172.21.15.17:0/328554970 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f06541323c0 con 0x7f065c150cc0 2024-09-10T21:33:53.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:53.145+0000 7f0661000640 1 --2- 172.21.15.17:0/328554970 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f063c0725e0 0x7f063c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:33:53.151 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:53.145+0000 7f0661000640 1 --2- 172.21.15.17:0/328554970 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f063c0725e0 0x7f063c074aa0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f0650000f70 tx=0x7f0650009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:33:53.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:53.149+0000 7f0658c00640 1 -- 172.21.15.17:0/328554970 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f06540ac050 con 0x7f065c150cc0 2024-09-10T21:33:53.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:53.497+0000 7f0662400640 1 -- 172.21.15.17:0/328554970 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) -- 0x7f065c069e60 con 0x7f065c150cc0 2024-09-10T21:33:53.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:53.497+0000 7f0658c00640 1 -- 172.21.15.17:0/328554970 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) ==== 57+0+0 (secure 0 0 0) 0x7f0654019100 con 0x7f065c150cc0 2024-09-10T21:33:53.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:53.501+0000 7f0662400640 1 -- 172.21.15.17:0/328554970 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f063c0725e0 msgr2=0x7f063c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:53.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:53.501+0000 7f0662400640 1 --2- 172.21.15.17:0/328554970 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f063c0725e0 0x7f063c074aa0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f0650000f70 tx=0x7f0650009040 comp rx=0 tx=0).stop 2024-09-10T21:33:53.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:53.501+0000 7f0662400640 1 -- 172.21.15.17:0/328554970 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f065c150cc0 msgr2=0x7f065c0c2a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:33:53.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:53.501+0000 7f0662400640 1 --2- 172.21.15.17:0/328554970 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f065c150cc0 0x7f065c0c2a20 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f0654009bd0 tx=0x7f06540af570 comp rx=0 tx=0).stop 2024-09-10T21:33:53.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:53.501+0000 7f0662400640 1 -- 172.21.15.17:0/328554970 shutdown_connections 2024-09-10T21:33:53.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:53.501+0000 7f0662400640 1 --2- 172.21.15.17:0/328554970 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f063c0725e0 0x7f063c074aa0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:53.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:53.501+0000 7f0662400640 1 --2- 172.21.15.17:0/328554970 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f065c150cc0 0x7f065c0c2a20 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:53.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:53.501+0000 7f0662400640 1 --2- 172.21.15.17:0/328554970 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f065c1508e0 0x7f065c14dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:53.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:53.501+0000 7f0662400640 1 --2- 172.21.15.17:0/328554970 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f065c0c7160 0x7f065c14d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:33:53.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:53.501+0000 7f0662400640 1 -- 172.21.15.17:0/328554970 >> 172.21.15.17:0/328554970 conn(0x7f065c06cdf0 msgr2=0x7f065c0c76d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:33:53.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:53.501+0000 7f0662400640 1 -- 172.21.15.17:0/328554970 shutdown_connections 2024-09-10T21:33:53.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:33:53.501+0000 7f0662400640 1 -- 172.21.15.17:0/328554970 wait complete. 2024-09-10T21:33:53.531 INFO:tasks.mon_thrash.mon_thrasher:waiting for 10.0 secs before continuing thrashing 2024-09-10T21:34:03.532 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-09-10T21:34:03.532 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-10T21:34:03.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:03.689+0000 7fa988a00640 1 -- 172.21.15.17:0/3537181469 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa9841525b0 msgr2=0x7fa984163cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:03.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:03.689+0000 7fa988a00640 1 --2- 172.21.15.17:0/3537181469 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa9841525b0 0x7fa984163cd0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa97c009bf0 tx=0x7fa97c01f3d0 comp rx=0 tx=0).stop 2024-09-10T21:34:03.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:03.689+0000 7fa988a00640 1 -- 172.21.15.17:0/3537181469 shutdown_connections 2024-09-10T21:34:03.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:03.689+0000 7fa988a00640 1 --2- 172.21.15.17:0/3537181469 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa9841525b0 0x7fa984163cd0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:03.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:03.689+0000 7fa988a00640 1 -- 172.21.15.17:0/3537181469 >> 172.21.15.17:0/3537181469 conn(0x7fa984068f90 msgr2=0x7fa98406f8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:03.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:03.689+0000 7fa988a00640 1 -- 172.21.15.17:0/3537181469 shutdown_connections 2024-09-10T21:34:03.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:03.689+0000 7fa988a00640 1 -- 172.21.15.17:0/3537181469 wait complete. 2024-09-10T21:34:03.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:03.689+0000 7fa988a00640 1 Processor -- start 2024-09-10T21:34:03.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:03.693+0000 7fa988a00640 1 -- start start 2024-09-10T21:34:03.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:03.693+0000 7fa988a00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa9840cd990 0x7fa98414c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:03.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:03.693+0000 7fa988a00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9841521d0 0x7fa9841e59a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:03.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:03.693+0000 7fa988a00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa9841525b0 0x7fa9840c4f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:03.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:03.693+0000 7fa988a00640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fa9840c73e0 con 0x7fa9841525b0 2024-09-10T21:34:03.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:03.693+0000 7fa988a00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fa98415b930 con 0x7fa9841521d0 2024-09-10T21:34:03.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:03.693+0000 7fa988a00640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7fa984158340 con 0x7fa9840cd990 2024-09-10T21:34:03.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:03.693+0000 7fa982800640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa9841525b0 0x7fa9840c4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:03.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:03.693+0000 7fa982800640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa9841525b0 0x7fa9840c4f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:51280/0 (socket says 172.21.15.17:51280) 2024-09-10T21:34:03.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:03.693+0000 7fa982800640 1 -- 172.21.15.17:0/2174142523 learned_addr learned my addr 172.21.15.17:0/2174142523 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:34:03.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:03.693+0000 7fa981e00640 1 --2- 172.21.15.17:0/2174142523 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa9840cd990 0x7fa98414c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:03.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:03.693+0000 7fa981400640 1 --2- 172.21.15.17:0/2174142523 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9841521d0 0x7fa9841e59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:03.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:03.693+0000 7fa982800640 1 -- 172.21.15.17:0/2174142523 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa9840cd990 msgr2=0x7fa98414c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:03.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:03.693+0000 7fa982800640 1 --2- 172.21.15.17:0/2174142523 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa9840cd990 0x7fa98414c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:03.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:03.693+0000 7fa982800640 1 -- 172.21.15.17:0/2174142523 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9841521d0 msgr2=0x7fa9841e59a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:03.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:03.693+0000 7fa982800640 1 --2- 172.21.15.17:0/2174142523 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9841521d0 0x7fa9841e59a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:03.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:03.693+0000 7fa982800640 1 -- 172.21.15.17:0/2174142523 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9840cb560 con 0x7fa9841525b0 2024-09-10T21:34:03.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:03.693+0000 7fa982800640 1 --2- 172.21.15.17:0/2174142523 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa9841525b0 0x7fa9840c4f10 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa97c01d000 tx=0x7fa97c003f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:03.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:03.693+0000 7fa97a000640 1 -- 172.21.15.17:0/2174142523 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa97c0b4040 con 0x7fa9841525b0 2024-09-10T21:34:03.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:03.693+0000 7fa97a000640 1 -- 172.21.15.17:0/2174142523 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa97c0bf070 con 0x7fa9841525b0 2024-09-10T21:34:03.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:03.693+0000 7fa988a00640 1 -- 172.21.15.17:0/2174142523 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9840c5f40 con 0x7fa9841525b0 2024-09-10T21:34:03.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:03.693+0000 7fa97a000640 1 -- 172.21.15.17:0/2174142523 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa97c0afc10 con 0x7fa9841525b0 2024-09-10T21:34:03.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:03.693+0000 7fa988a00640 1 -- 172.21.15.17:0/2174142523 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9840cb960 con 0x7fa9841525b0 2024-09-10T21:34:03.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:03.693+0000 7fa988a00640 1 -- 172.21.15.17:0/2174142523 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa984074140 con 0x7fa9841525b0 2024-09-10T21:34:03.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:03.697+0000 7fa97a000640 1 -- 172.21.15.17:0/2174142523 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fa97c004040 con 0x7fa9841525b0 2024-09-10T21:34:03.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:03.697+0000 7fa97a000640 1 --2- 172.21.15.17:0/2174142523 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fa9640725e0 0x7fa964074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:03.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:03.697+0000 7fa97a000640 1 -- 172.21.15.17:0/2174142523 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fa97c1314a0 con 0x7fa9841525b0 2024-09-10T21:34:03.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:03.697+0000 7fa981e00640 1 --2- 172.21.15.17:0/2174142523 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fa9640725e0 0x7fa964074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:03.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:03.697+0000 7fa981e00640 1 --2- 172.21.15.17:0/2174142523 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fa9640725e0 0x7fa964074aa0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fa974007fd0 tx=0x7fa974009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:03.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:03.701+0000 7fa97a000640 1 -- 172.21.15.17:0/2174142523 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fa97c018f30 con 0x7fa9841525b0 2024-09-10T21:34:04.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.009+0000 7fa988a00640 1 -- 172.21.15.17:0/2174142523 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fa984158b90 con 0x7fa9841525b0 2024-09-10T21:34:04.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.009+0000 7fa97a000640 1 -- 172.21.15.17:0/2174142523 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1557 (secure 0 0 0) 0x7fa97c0fc120 con 0x7fa9841525b0 2024-09-10T21:34:04.013 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":80,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":11,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5e2f023f-d6a6-49f4-a256-f4807dce3739","modified":"2024-09-10T21:23:46.892130Z","created":"2024-09-10T21:23:46.892130Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3301","nonce":0},{"type":"v1","addr":"172.21.15.31:6790","nonce":0}]},"addr":"172.21.15.31:6790/0","public_addr":"172.21.15.31:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-10T21:34:04.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.009+0000 7fa988a00640 1 -- 172.21.15.17:0/2174142523 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fa9640725e0 msgr2=0x7fa964074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:04.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.009+0000 7fa988a00640 1 --2- 172.21.15.17:0/2174142523 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fa9640725e0 0x7fa964074aa0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fa974007fd0 tx=0x7fa974009040 comp rx=0 tx=0).stop 2024-09-10T21:34:04.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.009+0000 7fa988a00640 1 -- 172.21.15.17:0/2174142523 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa9841525b0 msgr2=0x7fa9840c4f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:04.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.009+0000 7fa988a00640 1 --2- 172.21.15.17:0/2174142523 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa9841525b0 0x7fa9840c4f10 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa97c01d000 tx=0x7fa97c003f60 comp rx=0 tx=0).stop 2024-09-10T21:34:04.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.009+0000 7fa988a00640 1 -- 172.21.15.17:0/2174142523 shutdown_connections 2024-09-10T21:34:04.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.009+0000 7fa988a00640 1 --2- 172.21.15.17:0/2174142523 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fa9640725e0 0x7fa964074aa0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:04.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.009+0000 7fa988a00640 1 --2- 172.21.15.17:0/2174142523 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa9841525b0 0x7fa9840c4f10 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:04.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.009+0000 7fa988a00640 1 --2- 172.21.15.17:0/2174142523 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9841521d0 0x7fa9841e59a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:04.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.009+0000 7fa988a00640 1 --2- 172.21.15.17:0/2174142523 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa9840cd990 0x7fa98414c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:04.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.009+0000 7fa988a00640 1 -- 172.21.15.17:0/2174142523 >> 172.21.15.17:0/2174142523 conn(0x7fa984068f90 msgr2=0x7fa984155f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:04.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.009+0000 7fa988a00640 1 -- 172.21.15.17:0/2174142523 shutdown_connections 2024-09-10T21:34:04.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.009+0000 7fa988a00640 1 -- 172.21.15.17:0/2174142523 wait complete. 2024-09-10T21:34:04.037 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-09-10T21:34:04.037 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-09-10T21:34:04.037 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-09-10T21:34:04.037 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-10T21:34:04.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.189+0000 7f8271800640 1 -- 172.21.15.17:0/3686285832 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8250004d50 msgr2=0x7f82500051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:04.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.189+0000 7f8271800640 1 --2- 172.21.15.17:0/3686285832 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8250004d50 0x7f82500051d0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f8264009f10 tx=0x7f826401d5d0 comp rx=0 tx=0).stop 2024-09-10T21:34:04.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.189+0000 7f8270e00640 1 -- 172.21.15.17:0/3686285832 reap_dead start 2024-09-10T21:34:04.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.189+0000 7f8271800640 1 -- 172.21.15.17:0/3686285832 shutdown_connections 2024-09-10T21:34:04.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.189+0000 7f8271800640 1 -- 172.21.15.17:0/3686285832 >> 172.21.15.17:0/3686285832 conn(0x7f826c06cc20 msgr2=0x7f826c06d030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:04.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.189+0000 7f8271800640 1 -- 172.21.15.17:0/3686285832 shutdown_connections 2024-09-10T21:34:04.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.189+0000 7f8271800640 1 -- 172.21.15.17:0/3686285832 wait complete. 2024-09-10T21:34:04.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.189+0000 7f8271800640 1 Processor -- start 2024-09-10T21:34:04.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.189+0000 7f8271800640 1 -- start start 2024-09-10T21:34:04.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.193+0000 7f8271800640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8250004d50 0x7f826c0c85e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:04.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.193+0000 7f8271800640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f826c1513b0 0x7f826c0ccb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:04.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.193+0000 7f8271800640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f826c0c8b20 0x7f826c0cd0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:04.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.193+0000 7f8271800640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f826c0cecf0 con 0x7f8250004d50 2024-09-10T21:34:04.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.193+0000 7f8271800640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f826c0c5090 con 0x7f826c1513b0 2024-09-10T21:34:04.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.193+0000 7f8271800640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f826c0cee70 con 0x7f826c0c8b20 2024-09-10T21:34:04.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.193+0000 7f826be00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8250004d50 0x7f826c0c85e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:04.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.193+0000 7f826be00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8250004d50 0x7f826c0c85e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:51294/0 (socket says 172.21.15.17:51294) 2024-09-10T21:34:04.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.193+0000 7f826be00640 1 -- 172.21.15.17:0/1922407595 learned_addr learned my addr 172.21.15.17:0/1922407595 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:34:04.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.193+0000 7f8270e00640 1 --2- 172.21.15.17:0/1922407595 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f826c0c8b20 0x7f826c0cd0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:04.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.193+0000 7f826b400640 1 --2- 172.21.15.17:0/1922407595 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f826c1513b0 0x7f826c0ccb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:04.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.193+0000 7f826be00640 1 -- 172.21.15.17:0/1922407595 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f826c0c8b20 msgr2=0x7f826c0cd0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:04.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.193+0000 7f826be00640 1 --2- 172.21.15.17:0/1922407595 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f826c0c8b20 0x7f826c0cd0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:04.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.193+0000 7f826be00640 1 -- 172.21.15.17:0/1922407595 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f826c1513b0 msgr2=0x7f826c0ccb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:04.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.193+0000 7f826be00640 1 --2- 172.21.15.17:0/1922407595 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f826c1513b0 0x7f826c0ccb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:04.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.193+0000 7f826be00640 1 -- 172.21.15.17:0/1922407595 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f826c1f4610 con 0x7f8250004d50 2024-09-10T21:34:04.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.193+0000 7f826be00640 1 --2- 172.21.15.17:0/1922407595 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8250004d50 0x7f826c0c85e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f825c003600 tx=0x7f825c003a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:04.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.193+0000 7f8263e00640 1 -- 172.21.15.17:0/1922407595 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f825c00f780 con 0x7f8250004d50 2024-09-10T21:34:04.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.193+0000 7f8263e00640 1 -- 172.21.15.17:0/1922407595 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f825c016020 con 0x7f8250004d50 2024-09-10T21:34:04.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.193+0000 7f8263e00640 1 -- 172.21.15.17:0/1922407595 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f825c00fba0 con 0x7f8250004d50 2024-09-10T21:34:04.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.193+0000 7f8271800640 1 -- 172.21.15.17:0/1922407595 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f826c151880 con 0x7f8250004d50 2024-09-10T21:34:04.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.193+0000 7f8271800640 1 -- 172.21.15.17:0/1922407595 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f826c1f4330 con 0x7f8250004d50 2024-09-10T21:34:04.200 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.193+0000 7f8271800640 1 --2- 172.21.15.17:0/1922407595 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f826c1e2fa0 0x7f826c1f07c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:04.200 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.193+0000 7f8271800640 1 -- 172.21.15.17:0/1922407595 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f826c1580e0 con 0x7f826c1e2fa0 2024-09-10T21:34:04.201 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.193+0000 7f826b400640 1 --2- 172.21.15.17:0/1922407595 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f826c1e2fa0 0x7f826c1f07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:04.201 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.193+0000 7f826b400640 1 -- 172.21.15.17:0/1922407595 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f826c071e10 con 0x7f826c1e2fa0 2024-09-10T21:34:04.201 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.193+0000 7f826b400640 1 --2- 172.21.15.17:0/1922407595 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f826c1e2fa0 0x7f826c1f07c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f826c0ca890 tx=0x7f8258013e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:04.201 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.193+0000 7f8263e00640 1 -- 172.21.15.17:0/1922407595 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8258011a00 con 0x7f826c1e2fa0 2024-09-10T21:34:04.201 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.197+0000 7f8263e00640 1 -- 172.21.15.17:0/1922407595 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f8258011ba0 con 0x7f826c1e2fa0 2024-09-10T21:34:04.201 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.197+0000 7f8263e00640 1 -- 172.21.15.17:0/1922407595 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f826c1e2fa0 msgr2=0x7f826c1f07c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:04.201 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.197+0000 7f8263e00640 1 --2- 172.21.15.17:0/1922407595 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f826c1e2fa0 0x7f826c1f07c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f826c0ca890 tx=0x7f8258013e90 comp rx=0 tx=0).stop 2024-09-10T21:34:04.201 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.197+0000 7f8263e00640 1 -- 172.21.15.17:0/1922407595 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f826c1e2fa0 msgr2=0x7f826c1f07c0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:34:04.201 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.197+0000 7f8263e00640 1 --2- 172.21.15.17:0/1922407595 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f826c1e2fa0 0x7f826c1f07c0 secure :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f826c0ca890 tx=0x7f8258013e90 comp rx=0 tx=0).stop 2024-09-10T21:34:04.201 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.197+0000 7f8263e00640 1 -- 172.21.15.17:0/1922407595 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f825c011430 con 0x7f8250004d50 2024-09-10T21:34:04.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.197+0000 7f8263e00640 1 --2- 172.21.15.17:0/1922407595 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f824c0720f0 0x7f824c0745b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:04.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.197+0000 7f8263e00640 1 -- 172.21.15.17:0/1922407595 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f825c094200 con 0x7f8250004d50 2024-09-10T21:34:04.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.197+0000 7f826b400640 1 --2- 172.21.15.17:0/1922407595 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f824c0720f0 0x7f824c0745b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:04.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.197+0000 7f826b400640 1 --2- 172.21.15.17:0/1922407595 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f824c0720f0 0x7f824c0745b0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f825801c000 tx=0x7f825800f000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:04.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.201+0000 7f8271800640 1 --2- 172.21.15.17:0/1922407595 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f826c0ca330 0x7f826c1f0d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:04.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.201+0000 7f8271800640 1 -- 172.21.15.17:0/1922407595 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f826c1f4100 con 0x7f826c0ca330 2024-09-10T21:34:04.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.201+0000 7f8270e00640 1 --2- 172.21.15.17:0/1922407595 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f826c0ca330 0x7f826c1f0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:04.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.201+0000 7f8270e00640 1 -- 172.21.15.17:0/1922407595 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f826c0cc710 con 0x7f826c0ca330 2024-09-10T21:34:04.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.201+0000 7f8270e00640 1 --2- 172.21.15.17:0/1922407595 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f826c0ca330 0x7f826c1f0d00 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f8264009ee0 tx=0x7f8264004f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:04.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.205+0000 7f8263e00640 1 -- 172.21.15.17:0/1922407595 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8264009200 con 0x7f826c0ca330 2024-09-10T21:34:04.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.205+0000 7f8263e00640 1 -- 172.21.15.17:0/1922407595 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4543 (secure 0 0 0) 0x7f8264003b80 con 0x7f826c0ca330 2024-09-10T21:34:04.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.205+0000 7f8263e00640 1 -- 172.21.15.17:0/1922407595 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f826c0ca330 msgr2=0x7f826c1f0d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:04.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.205+0000 7f8263e00640 1 --2- 172.21.15.17:0/1922407595 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f826c0ca330 0x7f826c1f0d00 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f8264009ee0 tx=0x7f8264004f90 comp rx=0 tx=0).stop 2024-09-10T21:34:04.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.205+0000 7f8263e00640 1 -- 172.21.15.17:0/1922407595 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f826c0ca330 msgr2=0x7f826c1f0d00 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:34:04.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.205+0000 7f8263e00640 1 --2- 172.21.15.17:0/1922407595 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f826c0ca330 0x7f826c1f0d00 secure :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f8264009ee0 tx=0x7f8264004f90 comp rx=0 tx=0).stop 2024-09-10T21:34:04.212 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-10T21:34:04.212 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:34:04.212 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:34:04.212 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "leader", 2024-09-10T21:34:04.212 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 80, 2024-09-10T21:34:04.212 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 24811, 2024-09-10T21:34:04.212 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-10T21:34:04.213 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-10T21:34:04.213 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-10T21:34:04.213 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-10T21:34:04.213 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:34:04.213 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 12, 2024-09-10T21:34:04.213 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:34:04.213 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-10T21:34:04.213 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-10T21:34:04.213 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:34:04.213 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:34:04.213 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:34:04.213 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:34:04.213 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:34:04.213 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:34:04.214 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:34:04.214 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:34:04.214 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:34:04.214 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:34:04.214 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:34:04.214 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:34:04.214 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-10T21:34:04.214 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-10T21:34:04.214 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:34:04.214 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:34:04.214 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:34:04.214 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:34:04.214 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:34:04.215 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:34:04.215 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:34:04.215 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:34:04.215 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:34:04.215 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:34:04.215 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:34:04.215 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:34:04.215 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:34:04.215 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-10T21:34:04.215 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-10T21:34:04.215 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-10T21:34:04.215 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-10T21:34:04.215 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-10T21:34:04.215 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "5e2f023f-d6a6-49f4-a256-f4807dce3739", 2024-09-10T21:34:04.215 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:34:04.216 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:34:04.216 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-10T21:34:04.216 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-10T21:34:04.216 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-10T21:34:04.216 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-10T21:34:04.216 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-10T21:34:04.216 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-10T21:34:04.216 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-10T21:34:04.216 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:34:04.216 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-10T21:34:04.216 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:34:04.216 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:34:04.217 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:34:04.217 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:34:04.217 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:34:04.217 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:34:04.217 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:34:04.217 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:34:04.217 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:34:04.217 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:34:04.217 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:34:04.217 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:34:04.217 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-10T21:34:04.217 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:34:04.217 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-10T21:34:04.217 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:34:04.218 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:34:04.218 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:34:04.218 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:34:04.218 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:34:04.218 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:34:04.218 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:34:04.218 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-10T21:34:04.218 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:34:04.218 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:34:04.218 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:34:04.218 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:34:04.218 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-10T21:34:04.219 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:34:04.219 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:34:04.219 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:34:04.219 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:34:04.219 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-10T21:34:04.219 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-10T21:34:04.219 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:34:04.219 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:34:04.219 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:34:04.219 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:34:04.219 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:34:04.219 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:34:04.219 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:34:04.219 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:34:04.220 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:34:04.220 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:34:04.220 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:34:04.220 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3300", 2024-09-10T21:34:04.220 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:34:04.220 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:34:04.220 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:34:04.220 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:34:04.220 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789", 2024-09-10T21:34:04.220 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:34:04.220 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:34:04.221 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:34:04.221 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:34:04.221 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789/0", 2024-09-10T21:34:04.221 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6789/0", 2024-09-10T21:34:04.221 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:34:04.221 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:34:04.221 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:34:04.221 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:34:04.221 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:34:04.221 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:34:04.221 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:34:04.221 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:34:04.221 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:34:04.222 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:34:04.222 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:34:04.222 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3301", 2024-09-10T21:34:04.222 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:34:04.222 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:34:04.222 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:34:04.222 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:34:04.222 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790", 2024-09-10T21:34:04.222 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:34:04.222 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:34:04.222 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:34:04.223 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:34:04.223 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790/0", 2024-09-10T21:34:04.223 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6790/0", 2024-09-10T21:34:04.223 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:34:04.223 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:34:04.223 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:34:04.223 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:34:04.223 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:34:04.223 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:34:04.223 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-10T21:34:04.223 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-10T21:34:04.223 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:34:04.223 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:34:04.223 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:34:04.224 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:34:04.224 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:34:04.224 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:34:04.224 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-10T21:34:04.224 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:34:04.224 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:34:04.224 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:34:04.224 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 2 2024-09-10T21:34:04.224 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:34:04.224 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:34:04.224 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:34:04.224 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-10T21:34:04.224 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-10T21:34:04.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.205+0000 7f8271800640 1 -- 172.21.15.17:0/1922407595 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f824c0720f0 msgr2=0x7f824c0745b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:04.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.205+0000 7f8271800640 1 --2- 172.21.15.17:0/1922407595 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f824c0720f0 0x7f824c0745b0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f825801c000 tx=0x7f825800f000 comp rx=0 tx=0).stop 2024-09-10T21:34:04.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.205+0000 7f8271800640 1 -- 172.21.15.17:0/1922407595 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8250004d50 msgr2=0x7f826c0c85e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:04.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.205+0000 7f8271800640 1 --2- 172.21.15.17:0/1922407595 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8250004d50 0x7f826c0c85e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f825c003600 tx=0x7f825c003a00 comp rx=0 tx=0).stop 2024-09-10T21:34:04.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.205+0000 7f8270e00640 1 -- 172.21.15.17:0/1922407595 reap_dead start 2024-09-10T21:34:04.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.205+0000 7f8270e00640 1 -- 172.21.15.17:0/1922407595 reap_dead start 2024-09-10T21:34:04.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.205+0000 7f8271800640 1 -- 172.21.15.17:0/1922407595 shutdown_connections 2024-09-10T21:34:04.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.205+0000 7f8271800640 1 -- 172.21.15.17:0/1922407595 >> 172.21.15.17:0/1922407595 conn(0x7f826c06cc20 msgr2=0x7f826c0b2f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:04.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.205+0000 7f8271800640 1 -- 172.21.15.17:0/1922407595 shutdown_connections 2024-09-10T21:34:04.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.205+0000 7f8271800640 1 -- 172.21.15.17:0/1922407595 wait complete. 2024-09-10T21:34:04.233 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-10T21:34:04.233 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-09-10T21:34:04.233 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-10T21:34:04.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.381+0000 7f9454000640 1 -- 172.21.15.17:0/4286316663 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f944c0b3eb0 msgr2=0x7f944c163cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:04.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.381+0000 7f9454000640 1 --2- 172.21.15.17:0/4286316663 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f944c0b3eb0 0x7f944c163cf0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f944c1594a0 tx=0x7f944401cae0 comp rx=0 tx=0).stop 2024-09-10T21:34:04.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.381+0000 7f9454000640 1 -- 172.21.15.17:0/4286316663 shutdown_connections 2024-09-10T21:34:04.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.381+0000 7f9454000640 1 --2- 172.21.15.17:0/4286316663 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f944c0b3eb0 0x7f944c163cf0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:04.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.381+0000 7f9454000640 1 -- 172.21.15.17:0/4286316663 >> 172.21.15.17:0/4286316663 conn(0x7f944c0702f0 msgr2=0x7f944c070700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:04.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.381+0000 7f9454000640 1 -- 172.21.15.17:0/4286316663 shutdown_connections 2024-09-10T21:34:04.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.381+0000 7f9454000640 1 -- 172.21.15.17:0/4286316663 wait complete. 2024-09-10T21:34:04.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.381+0000 7f9454000640 1 Processor -- start 2024-09-10T21:34:04.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.381+0000 7f9454000640 1 -- start start 2024-09-10T21:34:04.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.381+0000 7f9454000640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f944c0b3eb0 0x7f944c14ff80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:04.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.381+0000 7f9454000640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f944c152440 0x7f944c152e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:04.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.381+0000 7f9454000640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f944c152820 0x7f944c1f6b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:04.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.381+0000 7f9454000640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f944c0ce630 con 0x7f944c152440 2024-09-10T21:34:04.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.381+0000 7f9454000640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f944c0b42e0 con 0x7f944c152820 2024-09-10T21:34:04.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.381+0000 7f9454000640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f944c0ce7b0 con 0x7f944c0b3eb0 2024-09-10T21:34:04.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.385+0000 7f9450c00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f944c152440 0x7f944c152e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:04.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.385+0000 7f9450c00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f944c152440 0x7f944c152e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:51338/0 (socket says 172.21.15.17:51338) 2024-09-10T21:34:04.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.385+0000 7f9450c00640 1 -- 172.21.15.17:0/281775797 learned_addr learned my addr 172.21.15.17:0/281775797 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:34:04.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.385+0000 7f9451600640 1 --2- 172.21.15.17:0/281775797 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f944c0b3eb0 0x7f944c14ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:04.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.385+0000 7f9452000640 1 --2- 172.21.15.17:0/281775797 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f944c152820 0x7f944c1f6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:04.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.385+0000 7f9450c00640 1 -- 172.21.15.17:0/281775797 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f944c0b3eb0 msgr2=0x7f944c14ff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:04.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.385+0000 7f9450c00640 1 --2- 172.21.15.17:0/281775797 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f944c0b3eb0 0x7f944c14ff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:04.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.385+0000 7f9450c00640 1 -- 172.21.15.17:0/281775797 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f944c152820 msgr2=0x7f944c1f6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:04.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.385+0000 7f9450c00640 1 --2- 172.21.15.17:0/281775797 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f944c152820 0x7f944c1f6b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:04.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.385+0000 7f9450c00640 1 -- 172.21.15.17:0/281775797 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f944c0c6160 con 0x7f944c152440 2024-09-10T21:34:04.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.385+0000 7f9451600640 1 --2- 172.21.15.17:0/281775797 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f944c0b3eb0 0x7f944c14ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:34:04.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.385+0000 7f9450c00640 1 --2- 172.21.15.17:0/281775797 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f944c152440 0x7f944c152e70 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f943800c4e0 tx=0x7f943800c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:04.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.385+0000 7f9449600640 1 -- 172.21.15.17:0/281775797 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94380052d0 con 0x7f944c152440 2024-09-10T21:34:04.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.385+0000 7f9449600640 1 -- 172.21.15.17:0/281775797 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9438005470 con 0x7f944c152440 2024-09-10T21:34:04.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.385+0000 7f9449600640 1 -- 172.21.15.17:0/281775797 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9438012890 con 0x7f944c152440 2024-09-10T21:34:04.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.385+0000 7f9454000640 1 -- 172.21.15.17:0/281775797 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f944c1f76c0 con 0x7f944c152440 2024-09-10T21:34:04.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.385+0000 7f9454000640 1 -- 172.21.15.17:0/281775797 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f944c0c5e80 con 0x7f944c152440 2024-09-10T21:34:04.392 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.385+0000 7f9454000640 1 --2- 172.21.15.17:0/281775797 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f944c1533b0 0x7f944c1f7040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:04.392 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.385+0000 7f9454000640 1 -- 172.21.15.17:0/281775797 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f944c1539f0 con 0x7f944c1533b0 2024-09-10T21:34:04.392 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.385+0000 7f9451600640 1 --2- 172.21.15.17:0/281775797 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f944c1533b0 0x7f944c1f7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:04.392 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.385+0000 7f9451600640 1 -- 172.21.15.17:0/281775797 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f944c0c86c0 con 0x7f944c1533b0 2024-09-10T21:34:04.392 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.385+0000 7f9451600640 1 --2- 172.21.15.17:0/281775797 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f944c1533b0 0x7f944c1f7040 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f9440000f70 tx=0x7f944000eb90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:04.392 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.385+0000 7f9449600640 1 -- 172.21.15.17:0/281775797 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f9438003600 con 0x7f944c152440 2024-09-10T21:34:04.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.389+0000 7f9449600640 1 --2- 172.21.15.17:0/281775797 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f942c0725e0 0x7f942c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:04.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.389+0000 7f9449600640 1 -- 172.21.15.17:0/281775797 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f9438093150 con 0x7f944c152440 2024-09-10T21:34:04.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.389+0000 7f9449600640 1 -- 172.21.15.17:0/281775797 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94400050d0 con 0x7f944c1533b0 2024-09-10T21:34:04.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.389+0000 7f9449600640 1 -- 172.21.15.17:0/281775797 <== mon.1 v2:172.21.15.31:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f9440018070 con 0x7f944c1533b0 2024-09-10T21:34:04.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.389+0000 7f9449600640 1 -- 172.21.15.17:0/281775797 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f944c1533b0 msgr2=0x7f944c1f7040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:04.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.389+0000 7f9449600640 1 --2- 172.21.15.17:0/281775797 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f944c1533b0 0x7f944c1f7040 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f9440000f70 tx=0x7f944000eb90 comp rx=0 tx=0).stop 2024-09-10T21:34:04.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.389+0000 7f9449600640 1 -- 172.21.15.17:0/281775797 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f944c1533b0 msgr2=0x7f944c1f7040 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:34:04.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.389+0000 7f9449600640 1 --2- 172.21.15.17:0/281775797 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f944c1533b0 0x7f944c1f7040 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:04.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.389+0000 7f9452000640 1 --2- 172.21.15.17:0/281775797 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f942c0725e0 0x7f942c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:04.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.389+0000 7f9452000640 1 --2- 172.21.15.17:0/281775797 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f942c0725e0 0x7f942c074aa0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f94440af000 tx=0x7f944401b320 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:04.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.393+0000 7f9454000640 1 --2- 172.21.15.17:0/281775797 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f944c151850 0x7f944c151c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:04.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.393+0000 7f9454000640 1 -- 172.21.15.17:0/281775797 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f944c0b38c0 con 0x7f944c151850 2024-09-10T21:34:04.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.393+0000 7f9451600640 1 --2- 172.21.15.17:0/281775797 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f944c151850 0x7f944c151c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:04.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.397+0000 7f9451600640 1 -- 172.21.15.17:0/281775797 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f944c0c63f0 con 0x7f944c151850 2024-09-10T21:34:04.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.397+0000 7f9451600640 1 --2- 172.21.15.17:0/281775797 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f944c151850 0x7f944c151c60 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f9440017000 tx=0x7f9440000ee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:04.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.397+0000 7f9449600640 1 -- 172.21.15.17:0/281775797 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f944000c8e0 con 0x7f944c151850 2024-09-10T21:34:04.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.397+0000 7f9449600640 1 -- 172.21.15.17:0/281775797 <== mon.1 v2:172.21.15.31:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5043 (secure 0 0 0) 0x7f944000c8e0 con 0x7f944c151850 2024-09-10T21:34:04.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.397+0000 7f9449600640 1 -- 172.21.15.17:0/281775797 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f944c151850 msgr2=0x7f944c151c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:04.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.397+0000 7f9449600640 1 --2- 172.21.15.17:0/281775797 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f944c151850 0x7f944c151c60 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f9440017000 tx=0x7f9440000ee0 comp rx=0 tx=0).stop 2024-09-10T21:34:04.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.397+0000 7f9449600640 1 -- 172.21.15.17:0/281775797 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f944c151850 msgr2=0x7f944c151c60 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:34:04.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.397+0000 7f9449600640 1 --2- 172.21.15.17:0/281775797 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f944c151850 0x7f944c151c60 secure :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f9440017000 tx=0x7f9440000ee0 comp rx=0 tx=0).stop 2024-09-10T21:34:04.402 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-10T21:34:04.402 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:34:04.402 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:34:04.402 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-10T21:34:04.402 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 80, 2024-09-10T21:34:04.403 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 582847, 2024-09-10T21:34:04.403 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-10T21:34:04.403 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-10T21:34:04.403 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-10T21:34:04.403 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-10T21:34:04.403 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:34:04.403 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 12, 2024-09-10T21:34:04.403 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:34:04.403 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-10T21:34:04.403 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-10T21:34:04.403 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:34:04.403 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:34:04.403 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:34:04.403 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:34:04.404 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:34:04.404 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:34:04.404 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:34:04.404 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:34:04.404 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:34:04.404 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:34:04.404 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:34:04.404 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:34:04.404 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-10T21:34:04.404 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-10T21:34:04.404 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:34:04.404 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:34:04.404 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:34:04.405 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:34:04.405 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:34:04.405 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:34:04.405 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:34:04.405 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:34:04.405 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:34:04.405 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:34:04.405 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:34:04.405 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:34:04.406 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:34:04.406 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-10T21:34:04.406 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-10T21:34:04.406 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-10T21:34:04.406 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-10T21:34:04.406 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-10T21:34:04.406 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "5e2f023f-d6a6-49f4-a256-f4807dce3739", 2024-09-10T21:34:04.406 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:34:04.406 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:34:04.406 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-10T21:34:04.406 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-10T21:34:04.406 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-10T21:34:04.406 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-10T21:34:04.406 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-10T21:34:04.407 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-10T21:34:04.407 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-10T21:34:04.407 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:34:04.407 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-10T21:34:04.407 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:34:04.407 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:34:04.407 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:34:04.407 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:34:04.407 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:34:04.407 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:34:04.407 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:34:04.407 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:34:04.407 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:34:04.408 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:34:04.408 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:34:04.408 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:34:04.408 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-10T21:34:04.408 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:34:04.408 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-10T21:34:04.408 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:34:04.408 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:34:04.408 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:34:04.408 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:34:04.408 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:34:04.408 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:34:04.408 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:34:04.409 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-10T21:34:04.409 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:34:04.409 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:34:04.409 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:34:04.409 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:34:04.409 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-10T21:34:04.409 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:34:04.409 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:34:04.409 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:34:04.409 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:34:04.409 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-10T21:34:04.409 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-10T21:34:04.409 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:34:04.410 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:34:04.410 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:34:04.410 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:34:04.410 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:34:04.410 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:34:04.410 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:34:04.410 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:34:04.410 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:34:04.410 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:34:04.410 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:34:04.410 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3300", 2024-09-10T21:34:04.410 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:34:04.410 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:34:04.410 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:34:04.411 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:34:04.411 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789", 2024-09-10T21:34:04.411 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:34:04.411 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:34:04.411 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:34:04.411 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:34:04.411 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789/0", 2024-09-10T21:34:04.411 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6789/0", 2024-09-10T21:34:04.411 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:34:04.411 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:34:04.411 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:34:04.411 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:34:04.411 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:34:04.411 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:34:04.412 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:34:04.412 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:34:04.412 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:34:04.412 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:34:04.412 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:34:04.412 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3301", 2024-09-10T21:34:04.412 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:34:04.412 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:34:04.412 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:34:04.412 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:34:04.412 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790", 2024-09-10T21:34:04.412 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:34:04.412 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:34:04.413 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:34:04.413 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:34:04.413 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790/0", 2024-09-10T21:34:04.413 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6790/0", 2024-09-10T21:34:04.413 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:34:04.413 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:34:04.413 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:34:04.413 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:34:04.413 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:34:04.413 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:34:04.413 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-10T21:34:04.413 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-10T21:34:04.413 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:34:04.413 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:34:04.414 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:34:04.414 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:34:04.414 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:34:04.414 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:34:04.414 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": [ 2024-09-10T21:34:04.414 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:34:04.414 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:34:04.414 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:34:04.414 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 5 2024-09-10T21:34:04.414 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:34:04.414 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:34:04.414 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-10T21:34:04.414 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:34:04.416 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:34:04.416 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:34:04.416 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 7 2024-09-10T21:34:04.416 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:34:04.416 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:34:04.416 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-10T21:34:04.416 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:34:04.416 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:34:04.416 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:34:04.416 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 6 2024-09-10T21:34:04.417 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:34:04.417 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:34:04.417 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": [ 2024-09-10T21:34:04.417 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:34:04.417 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:34:04.417 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:34:04.417 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 2 2024-09-10T21:34:04.417 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:34:04.417 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:34:04.417 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:34:04.417 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-10T21:34:04.417 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-10T21:34:04.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.397+0000 7f9454000640 1 -- 172.21.15.17:0/281775797 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f942c0725e0 msgr2=0x7f942c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:04.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.397+0000 7f9454000640 1 --2- 172.21.15.17:0/281775797 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f942c0725e0 0x7f942c074aa0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f94440af000 tx=0x7f944401b320 comp rx=0 tx=0).stop 2024-09-10T21:34:04.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.397+0000 7f9454000640 1 -- 172.21.15.17:0/281775797 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f944c152440 msgr2=0x7f944c152e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:04.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.397+0000 7f9454000640 1 --2- 172.21.15.17:0/281775797 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f944c152440 0x7f944c152e70 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f943800c4e0 tx=0x7f943800c9a0 comp rx=0 tx=0).stop 2024-09-10T21:34:04.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.397+0000 7f9452000640 1 -- 172.21.15.17:0/281775797 reap_dead start 2024-09-10T21:34:04.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.397+0000 7f9454000640 1 -- 172.21.15.17:0/281775797 shutdown_connections 2024-09-10T21:34:04.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.397+0000 7f9454000640 1 --2- 172.21.15.17:0/281775797 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f944c152440 0x7f944c152e70 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:04.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.397+0000 7f9454000640 1 -- 172.21.15.17:0/281775797 >> 172.21.15.17:0/281775797 conn(0x7f944c0702f0 msgr2=0x7f944c0c5770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:04.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.397+0000 7f9454000640 1 -- 172.21.15.17:0/281775797 shutdown_connections 2024-09-10T21:34:04.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.397+0000 7f9454000640 1 -- 172.21.15.17:0/281775797 wait complete. 2024-09-10T21:34:04.422 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-10T21:34:04.422 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-09-10T21:34:04.423 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-10T21:34:04.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.581+0000 7f18c7200640 1 -- 172.21.15.17:0/2257016914 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f18c80c2f80 msgr2=0x7f18c8163ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:04.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.581+0000 7f18c7200640 1 --2- 172.21.15.17:0/2257016914 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f18c80c2f80 0x7f18c8163ab0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f18c0009bc0 tx=0x7f18c001ca80 comp rx=0 tx=0).stop 2024-09-10T21:34:04.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.581+0000 7f18c7200640 1 -- 172.21.15.17:0/2257016914 shutdown_connections 2024-09-10T21:34:04.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.581+0000 7f18c7200640 1 --2- 172.21.15.17:0/2257016914 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f18c80c2f80 0x7f18c8163ab0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:04.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.581+0000 7f18c7200640 1 -- 172.21.15.17:0/2257016914 >> 172.21.15.17:0/2257016914 conn(0x7f18c8068990 msgr2=0x7f18c8068da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:04.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.581+0000 7f18c7200640 1 -- 172.21.15.17:0/2257016914 shutdown_connections 2024-09-10T21:34:04.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.581+0000 7f18c7200640 1 -- 172.21.15.17:0/2257016914 wait complete. 2024-09-10T21:34:04.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.581+0000 7f18c7200640 1 Processor -- start 2024-09-10T21:34:04.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.581+0000 7f18c7200640 1 -- start start 2024-09-10T21:34:04.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.585+0000 7f18c7200640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18c80c2f80 0x7f18c8140c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:04.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.585+0000 7f18c7200640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f18c80c9520 0x7f18c814dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:04.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.585+0000 7f18c7200640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f18c81523d0 0x7f18c814e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:04.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.585+0000 7f18c7200640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f18c8071320 con 0x7f18c80c9520 2024-09-10T21:34:04.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.585+0000 7f18c7200640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f18c80cb3c0 con 0x7f18c80c2f80 2024-09-10T21:34:04.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.585+0000 7f18c7200640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f18c80714a0 con 0x7f18c81523d0 2024-09-10T21:34:04.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.585+0000 7f18c5400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f18c80c9520 0x7f18c814dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:04.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.585+0000 7f18c5e00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18c80c2f80 0x7f18c8140c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:04.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.585+0000 7f18c5400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f18c80c9520 0x7f18c814dbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:51360/0 (socket says 172.21.15.17:51360) 2024-09-10T21:34:04.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.585+0000 7f18c5400640 1 -- 172.21.15.17:0/4228347357 learned_addr learned my addr 172.21.15.17:0/4228347357 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:34:04.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.585+0000 7f18c6800640 1 --2- 172.21.15.17:0/4228347357 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f18c81523d0 0x7f18c814e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:04.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.585+0000 7f18c5400640 1 -- 172.21.15.17:0/4228347357 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f18c81523d0 msgr2=0x7f18c814e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:04.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.585+0000 7f18c5400640 1 --2- 172.21.15.17:0/4228347357 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f18c81523d0 0x7f18c814e130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:04.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.585+0000 7f18c5400640 1 -- 172.21.15.17:0/4228347357 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18c80c2f80 msgr2=0x7f18c8140c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:04.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.585+0000 7f18c5400640 1 --2- 172.21.15.17:0/4228347357 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18c80c2f80 0x7f18c8140c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:04.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.585+0000 7f18c5400640 1 -- 172.21.15.17:0/4228347357 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18c80c3530 con 0x7f18c80c9520 2024-09-10T21:34:04.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.585+0000 7f18c5e00640 1 --2- 172.21.15.17:0/4228347357 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18c80c2f80 0x7f18c8140c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:34:04.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.585+0000 7f18c5400640 1 --2- 172.21.15.17:0/4228347357 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f18c80c9520 0x7f18c814dbf0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f18b400c450 tx=0x7f18b400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:04.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.585+0000 7f18be000640 1 -- 172.21.15.17:0/4228347357 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18b40052d0 con 0x7f18c80c9520 2024-09-10T21:34:04.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.585+0000 7f18be000640 1 -- 172.21.15.17:0/4228347357 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f18b4005470 con 0x7f18c80c9520 2024-09-10T21:34:04.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.585+0000 7f18be000640 1 -- 172.21.15.17:0/4228347357 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18b4012870 con 0x7f18c80c9520 2024-09-10T21:34:04.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.585+0000 7f18c7200640 1 -- 172.21.15.17:0/4228347357 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18c814d8f0 con 0x7f18c80c9520 2024-09-10T21:34:04.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.585+0000 7f18c7200640 1 -- 172.21.15.17:0/4228347357 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18c806e160 con 0x7f18c80c9520 2024-09-10T21:34:04.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.585+0000 7f18c7200640 1 --2- 172.21.15.17:0/4228347357 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f18c80cbd60 0x7f18c806e360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:04.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.585+0000 7f18c7200640 1 -- 172.21.15.17:0/4228347357 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f18c8069ef0 con 0x7f18c80cbd60 2024-09-10T21:34:04.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.585+0000 7f18c5e00640 1 --2- 172.21.15.17:0/4228347357 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f18c80cbd60 0x7f18c806e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:04.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.585+0000 7f18c5e00640 1 -- 172.21.15.17:0/4228347357 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f18c806df10 con 0x7f18c80cbd60 2024-09-10T21:34:04.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.589+0000 7f18c5e00640 1 --2- 172.21.15.17:0/4228347357 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f18c80cbd60 0x7f18c806e360 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f18c8163f10 tx=0x7f18b800ff70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:04.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.589+0000 7f18be000640 1 -- 172.21.15.17:0/4228347357 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f18b4003600 con 0x7f18c80c9520 2024-09-10T21:34:04.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.589+0000 7f18be000640 1 --2- 172.21.15.17:0/4228347357 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f18a80725e0 0x7f18a8074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:04.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.589+0000 7f18be000640 1 -- 172.21.15.17:0/4228347357 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f18b4092fe0 con 0x7f18c80c9520 2024-09-10T21:34:04.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.589+0000 7f18c6800640 1 --2- 172.21.15.17:0/4228347357 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f18a80725e0 0x7f18a8074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:04.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.589+0000 7f18be000640 1 -- 172.21.15.17:0/4228347357 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18b8018020 con 0x7f18c80cbd60 2024-09-10T21:34:04.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.589+0000 7f18be000640 1 -- 172.21.15.17:0/4228347357 <== mon.2 v2:172.21.15.31:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f18b800d9b0 con 0x7f18c80cbd60 2024-09-10T21:34:04.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.589+0000 7f18be000640 1 -- 172.21.15.17:0/4228347357 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f18c80cbd60 msgr2=0x7f18c806e360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:04.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.589+0000 7f18be000640 1 --2- 172.21.15.17:0/4228347357 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f18c80cbd60 0x7f18c806e360 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f18c8163f10 tx=0x7f18b800ff70 comp rx=0 tx=0).stop 2024-09-10T21:34:04.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.589+0000 7f18be000640 1 -- 172.21.15.17:0/4228347357 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f18c80cbd60 msgr2=0x7f18c806e360 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:34:04.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.589+0000 7f18be000640 1 --2- 172.21.15.17:0/4228347357 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f18c80cbd60 0x7f18c806e360 secure :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f18c8163f10 tx=0x7f18b800ff70 comp rx=0 tx=0).stop 2024-09-10T21:34:04.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.589+0000 7f18c6800640 1 --2- 172.21.15.17:0/4228347357 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f18a80725e0 0x7f18a8074aa0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f18c00af000 tx=0x7f18c001b2c0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:04.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.597+0000 7f18c7200640 1 --2- 172.21.15.17:0/4228347357 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f18c814d2f0 0x7f18c806ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:04.602 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.597+0000 7f18c7200640 1 -- 172.21.15.17:0/4228347357 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f18c81446e0 con 0x7f18c814d2f0 2024-09-10T21:34:04.602 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.597+0000 7f18c5e00640 1 --2- 172.21.15.17:0/4228347357 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f18c814d2f0 0x7f18c806ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:04.602 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.597+0000 7f18c5e00640 1 -- 172.21.15.17:0/4228347357 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f18c80b3510 con 0x7f18c814d2f0 2024-09-10T21:34:04.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.597+0000 7f18c5e00640 1 --2- 172.21.15.17:0/4228347357 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f18c814d2f0 0x7f18c806ef50 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f18b8019000 tx=0x7f18b8004630 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:04.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.597+0000 7f18be000640 1 -- 172.21.15.17:0/4228347357 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18b8018030 con 0x7f18c814d2f0 2024-09-10T21:34:04.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.597+0000 7f18be000640 1 -- 172.21.15.17:0/4228347357 <== mon.2 v2:172.21.15.31:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4876 (secure 0 0 0) 0x7f18b801f220 con 0x7f18c814d2f0 2024-09-10T21:34:04.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.597+0000 7f18be000640 1 -- 172.21.15.17:0/4228347357 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f18c814d2f0 msgr2=0x7f18c806ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:04.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.597+0000 7f18be000640 1 --2- 172.21.15.17:0/4228347357 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f18c814d2f0 0x7f18c806ef50 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f18b8019000 tx=0x7f18b8004630 comp rx=0 tx=0).stop 2024-09-10T21:34:04.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.597+0000 7f18be000640 1 -- 172.21.15.17:0/4228347357 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f18c814d2f0 msgr2=0x7f18c806ef50 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:34:04.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.597+0000 7f18be000640 1 --2- 172.21.15.17:0/4228347357 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f18c814d2f0 0x7f18c806ef50 secure :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f18b8019000 tx=0x7f18b8004630 comp rx=0 tx=0).stop 2024-09-10T21:34:04.604 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-10T21:34:04.604 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:34:04.604 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:34:04.604 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-10T21:34:04.604 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 80, 2024-09-10T21:34:04.604 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 176906, 2024-09-10T21:34:04.605 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-10T21:34:04.605 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-10T21:34:04.605 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-10T21:34:04.605 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-10T21:34:04.605 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:34:04.605 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 12, 2024-09-10T21:34:04.605 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:34:04.605 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-10T21:34:04.605 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-10T21:34:04.605 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:34:04.605 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:34:04.606 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:34:04.606 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:34:04.606 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:34:04.606 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:34:04.606 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:34:04.606 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:34:04.606 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:34:04.606 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:34:04.606 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:34:04.606 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:34:04.606 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-10T21:34:04.607 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-10T21:34:04.607 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:34:04.607 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:34:04.607 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:34:04.607 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:34:04.607 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:34:04.607 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:34:04.607 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:34:04.607 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:34:04.607 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:34:04.607 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:34:04.608 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:34:04.608 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:34:04.608 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:34:04.608 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-10T21:34:04.608 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-10T21:34:04.608 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-10T21:34:04.608 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-10T21:34:04.608 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-10T21:34:04.608 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "5e2f023f-d6a6-49f4-a256-f4807dce3739", 2024-09-10T21:34:04.608 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:34:04.608 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:34:04.609 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-10T21:34:04.609 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-10T21:34:04.609 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-10T21:34:04.609 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-10T21:34:04.609 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-10T21:34:04.609 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-10T21:34:04.609 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-10T21:34:04.609 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:34:04.609 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-10T21:34:04.609 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:34:04.609 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:34:04.610 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:34:04.610 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:34:04.610 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:34:04.610 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:34:04.610 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:34:04.610 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:34:04.610 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:34:04.610 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:34:04.610 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:34:04.610 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:34:04.610 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-10T21:34:04.610 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:34:04.610 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-10T21:34:04.611 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:34:04.611 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:34:04.611 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:34:04.611 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:34:04.611 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:34:04.611 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:34:04.611 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:34:04.611 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-10T21:34:04.611 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:34:04.611 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:34:04.611 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:34:04.612 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:34:04.612 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-10T21:34:04.612 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:34:04.612 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:34:04.612 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:34:04.612 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:34:04.612 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-10T21:34:04.612 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-10T21:34:04.612 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:34:04.612 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:34:04.612 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:34:04.612 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:34:04.613 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:34:04.613 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:34:04.613 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:34:04.613 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:34:04.613 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:34:04.613 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:34:04.613 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:34:04.613 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3300", 2024-09-10T21:34:04.613 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:34:04.613 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:34:04.613 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:34:04.614 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:34:04.614 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789", 2024-09-10T21:34:04.614 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:34:04.614 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:34:04.614 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:34:04.614 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:34:04.614 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789/0", 2024-09-10T21:34:04.614 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6789/0", 2024-09-10T21:34:04.614 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:34:04.614 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:34:04.614 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:34:04.614 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:34:04.614 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:34:04.615 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:34:04.615 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:34:04.615 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:34:04.615 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:34:04.615 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:34:04.615 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:34:04.615 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3301", 2024-09-10T21:34:04.615 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:34:04.615 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:34:04.615 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:34:04.615 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:34:04.616 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790", 2024-09-10T21:34:04.616 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:34:04.616 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:34:04.616 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:34:04.616 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:34:04.616 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790/0", 2024-09-10T21:34:04.616 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6790/0", 2024-09-10T21:34:04.616 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:34:04.616 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:34:04.616 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:34:04.616 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:34:04.616 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:34:04.617 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:34:04.617 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-10T21:34:04.617 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-10T21:34:04.617 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:34:04.617 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:34:04.617 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:34:04.617 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:34:04.617 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:34:04.617 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:34:04.617 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": [ 2024-09-10T21:34:04.617 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:34:04.617 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:34:04.618 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:34:04.618 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:34:04.618 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:34:04.618 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:34:04.618 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-10T21:34:04.618 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:34:04.618 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:34:04.618 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:34:04.618 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:34:04.618 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:34:04.618 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:34:04.619 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-10T21:34:04.619 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:34:04.619 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:34:04.619 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:34:04.619 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:34:04.619 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:34:04.619 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:34:04.619 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:34:04.619 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-10T21:34:04.619 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-10T21:34:04.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.601+0000 7f18c7200640 1 -- 172.21.15.17:0/4228347357 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f18a80725e0 msgr2=0x7f18a8074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:04.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.601+0000 7f18c7200640 1 --2- 172.21.15.17:0/4228347357 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f18a80725e0 0x7f18a8074aa0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f18c00af000 tx=0x7f18c001b2c0 comp rx=0 tx=0).stop 2024-09-10T21:34:04.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.601+0000 7f18c7200640 1 -- 172.21.15.17:0/4228347357 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f18c80c9520 msgr2=0x7f18c814dbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:04.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.601+0000 7f18c7200640 1 --2- 172.21.15.17:0/4228347357 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f18c80c9520 0x7f18c814dbf0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f18b400c450 tx=0x7f18b400c910 comp rx=0 tx=0).stop 2024-09-10T21:34:04.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.601+0000 7f18c6800640 1 -- 172.21.15.17:0/4228347357 reap_dead start 2024-09-10T21:34:04.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.601+0000 7f18c7200640 1 -- 172.21.15.17:0/4228347357 shutdown_connections 2024-09-10T21:34:04.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.601+0000 7f18c7200640 1 --2- 172.21.15.17:0/4228347357 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f18c80c9520 0x7f18c814dbf0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:04.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.601+0000 7f18c7200640 1 -- 172.21.15.17:0/4228347357 >> 172.21.15.17:0/4228347357 conn(0x7f18c8068990 msgr2=0x7f18c8071ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:04.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.601+0000 7f18c7200640 1 -- 172.21.15.17:0/4228347357 shutdown_connections 2024-09-10T21:34:04.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.601+0000 7f18c7200640 1 -- 172.21.15.17:0/4228347357 wait complete. 2024-09-10T21:34:04.625 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-10T21:34:04.625 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['b'] 2024-09-10T21:34:04.625 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2024-09-10T21:34:04.625 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.b 2024-09-10T21:34:04.625 INFO:tasks.mon_thrash.mon_thrasher:killing mon.b 2024-09-10T21:34:04.626 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-10T21:34:04.626 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:34:04.653 INFO:tasks.ceph.mon.b:Stopped 2024-09-10T21:34:04.653 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2024-09-10T21:34:04.654 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-10T21:34:04.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.813+0000 7f79f7200640 1 -- 172.21.15.17:0/326695483 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f79f8150580 msgr2=0x7f79f80c1400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:04.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.813+0000 7f79f7200640 1 --2- 172.21.15.17:0/326695483 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f79f8150580 0x7f79f80c1400 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f79e800aa10 tx=0x7f79e801dc30 comp rx=0 tx=0).stop 2024-09-10T21:34:04.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.813+0000 7f79f7200640 1 -- 172.21.15.17:0/326695483 shutdown_connections 2024-09-10T21:34:04.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.813+0000 7f79f7200640 1 --2- 172.21.15.17:0/326695483 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f79f80c85f0 0x7f79f815a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:04.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.813+0000 7f79f7200640 1 --2- 172.21.15.17:0/326695483 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f79f8150580 0x7f79f80c1400 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:04.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.813+0000 7f79f7200640 1 -- 172.21.15.17:0/326695483 >> 172.21.15.17:0/326695483 conn(0x7f79f80c8ab0 msgr2=0x7f79f80c8ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:04.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.813+0000 7f79f7200640 1 -- 172.21.15.17:0/326695483 shutdown_connections 2024-09-10T21:34:04.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.813+0000 7f79f7200640 1 -- 172.21.15.17:0/326695483 wait complete. 2024-09-10T21:34:04.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.813+0000 7f79f7200640 1 Processor -- start 2024-09-10T21:34:04.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.813+0000 7f79f7200640 1 -- start start 2024-09-10T21:34:04.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.813+0000 7f79f7200640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f79f80c7b10 0x7f79f814c320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:04.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.813+0000 7f79f7200640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f79f80c85f0 0x7f79f815fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:04.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.813+0000 7f79f7200640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f79f8150580 0x7f79f8157e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:04.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.813+0000 7f79f7200640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f79f80c76f0 con 0x7f79f80c85f0 2024-09-10T21:34:04.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.813+0000 7f79f7200640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f79f80c7570 con 0x7f79f8150580 2024-09-10T21:34:04.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.813+0000 7f79f7200640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f79f80cc610 con 0x7f79f80c7b10 2024-09-10T21:34:04.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.817+0000 7f79f5400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f79f80c85f0 0x7f79f815fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:04.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.817+0000 7f79f6800640 1 -- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f79f8150580 msgr2=0x7f79f8157e90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:3300/0 2024-09-10T21:34:04.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.817+0000 7f79f6800640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f79f8150580 0x7f79f8157e90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:34:04.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.817+0000 7f79f5400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f79f80c85f0 0x7f79f815fb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55086/0 (socket says 172.21.15.17:55086) 2024-09-10T21:34:04.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.817+0000 7f79f5400640 1 -- 172.21.15.17:0/1061292802 learned_addr learned my addr 172.21.15.17:0/1061292802 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:34:04.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.817+0000 7f79f5e00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f79f80c7b10 0x7f79f814c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:04.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.817+0000 7f79f5400640 1 -- 172.21.15.17:0/1061292802 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f79f80c7b10 msgr2=0x7f79f814c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:04.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.817+0000 7f79f5400640 1 --2- 172.21.15.17:0/1061292802 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f79f80c7b10 0x7f79f814c320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:04.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.817+0000 7f79f5400640 1 -- 172.21.15.17:0/1061292802 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f79f8150580 msgr2=0x7f79f8157e90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:34:04.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.817+0000 7f79f5400640 1 --2- 172.21.15.17:0/1061292802 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f79f8150580 0x7f79f8157e90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:04.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.817+0000 7f79f5400640 1 -- 172.21.15.17:0/1061292802 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79f815f0b0 con 0x7f79f80c85f0 2024-09-10T21:34:04.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.817+0000 7f79f5400640 1 --2- 172.21.15.17:0/1061292802 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f79f80c85f0 0x7f79f815fb50 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f79e400d950 tx=0x7f79e400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:04.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.817+0000 7f79ee000640 1 -- 172.21.15.17:0/1061292802 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79e4017070 con 0x7f79f80c85f0 2024-09-10T21:34:04.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.817+0000 7f79ee000640 1 -- 172.21.15.17:0/1061292802 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f79e4009d20 con 0x7f79f80c85f0 2024-09-10T21:34:04.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.817+0000 7f79f7200640 1 -- 172.21.15.17:0/1061292802 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79f814b8d0 con 0x7f79f80c85f0 2024-09-10T21:34:04.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.817+0000 7f79ee000640 1 -- 172.21.15.17:0/1061292802 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79e40106c0 con 0x7f79f80c85f0 2024-09-10T21:34:04.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.817+0000 7f79f7200640 1 -- 172.21.15.17:0/1061292802 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79f815f350 con 0x7f79f80c85f0 2024-09-10T21:34:04.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.817+0000 7f79f7200640 1 -- 172.21.15.17:0/1061292802 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79f815a490 con 0x7f79f80c85f0 2024-09-10T21:34:04.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.817+0000 7f79ee000640 1 -- 172.21.15.17:0/1061292802 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f79e4010860 con 0x7f79f80c85f0 2024-09-10T21:34:04.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.821+0000 7f79ee000640 1 --2- 172.21.15.17:0/1061292802 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f79d80725c0 0x7f79d8074a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:04.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.821+0000 7f79ee000640 1 -- 172.21.15.17:0/1061292802 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f79e4093d60 con 0x7f79f80c85f0 2024-09-10T21:34:04.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.821+0000 7f79f5e00640 1 --2- 172.21.15.17:0/1061292802 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f79d80725c0 0x7f79d8074a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:04.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.821+0000 7f79f5e00640 1 --2- 172.21.15.17:0/1061292802 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f79d80725c0 0x7f79d8074a80 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f79e80b0000 tx=0x7f79e801c470 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:04.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:04.825+0000 7f79ee000640 1 -- 172.21.15.17:0/1061292802 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f79e405ea60 con 0x7f79f80c85f0 2024-09-10T21:34:05.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:05.140+0000 7f79f7200640 1 -- 172.21.15.17:0/1061292802 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f79f8156630 con 0x7f79f80c85f0 2024-09-10T21:34:05.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:05.140+0000 7f79ee000640 1 -- 172.21.15.17:0/1061292802 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1557 (secure 0 0 0) 0x7f79e4063490 con 0x7f79f80c85f0 2024-09-10T21:34:05.145 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":80,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":13,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5e2f023f-d6a6-49f4-a256-f4807dce3739","modified":"2024-09-10T21:23:46.892130Z","created":"2024-09-10T21:23:46.892130Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3301","nonce":0},{"type":"v1","addr":"172.21.15.31:6790","nonce":0}]},"addr":"172.21.15.31:6790/0","public_addr":"172.21.15.31:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-10T21:34:05.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:05.140+0000 7f79f7200640 1 -- 172.21.15.17:0/1061292802 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f79d80725c0 msgr2=0x7f79d8074a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:05.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:05.140+0000 7f79f7200640 1 --2- 172.21.15.17:0/1061292802 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f79d80725c0 0x7f79d8074a80 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f79e80b0000 tx=0x7f79e801c470 comp rx=0 tx=0).stop 2024-09-10T21:34:05.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:05.140+0000 7f79f7200640 1 -- 172.21.15.17:0/1061292802 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f79f80c85f0 msgr2=0x7f79f815fb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:05.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:05.140+0000 7f79f7200640 1 --2- 172.21.15.17:0/1061292802 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f79f80c85f0 0x7f79f815fb50 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f79e400d950 tx=0x7f79e400de10 comp rx=0 tx=0).stop 2024-09-10T21:34:05.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:05.140+0000 7f79f7200640 1 -- 172.21.15.17:0/1061292802 shutdown_connections 2024-09-10T21:34:05.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:05.140+0000 7f79f7200640 1 --2- 172.21.15.17:0/1061292802 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f79d80725c0 0x7f79d8074a80 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:05.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:05.140+0000 7f79f7200640 1 --2- 172.21.15.17:0/1061292802 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f79f8150580 0x7f79f8157e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:05.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:05.140+0000 7f79f7200640 1 --2- 172.21.15.17:0/1061292802 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f79f80c85f0 0x7f79f815fb50 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:05.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:05.140+0000 7f79f7200640 1 --2- 172.21.15.17:0/1061292802 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f79f80c7b10 0x7f79f814c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:05.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:05.140+0000 7f79f7200640 1 -- 172.21.15.17:0/1061292802 >> 172.21.15.17:0/1061292802 conn(0x7f79f80c8ab0 msgr2=0x7f79f8074470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:05.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:05.140+0000 7f79f7200640 1 -- 172.21.15.17:0/1061292802 shutdown_connections 2024-09-10T21:34:05.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:05.140+0000 7f79f7200640 1 -- 172.21.15.17:0/1061292802 wait complete. 2024-09-10T21:34:08.171 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-10T21:34:08.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:08.340+0000 7ff0cec00640 1 -- 172.21.15.17:0/1463360189 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff0c80b2d80 msgr2=0x7ff0c806fd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:08.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:08.340+0000 7ff0cec00640 1 --2- 172.21.15.17:0/1463360189 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff0c80b2d80 0x7ff0c806fd00 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff0c80c8440 tx=0x7ff0b401f0d0 comp rx=0 tx=0).stop 2024-09-10T21:34:08.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:08.340+0000 7ff0cec00640 1 -- 172.21.15.17:0/1463360189 shutdown_connections 2024-09-10T21:34:08.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:08.340+0000 7ff0cec00640 1 --2- 172.21.15.17:0/1463360189 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff0c80b2d80 0x7ff0c806fd00 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:08.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:08.340+0000 7ff0cec00640 1 --2- 172.21.15.17:0/1463360189 >> v2:172.21.15.31:3300/0 conn(0x7ff0c81527e0 0x7ff0c8158310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:08.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:08.340+0000 7ff0cec00640 1 -- 172.21.15.17:0/1463360189 >> 172.21.15.17:0/1463360189 conn(0x7ff0c8070ca0 msgr2=0x7ff0c80710b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:08.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:08.340+0000 7ff0cec00640 1 -- 172.21.15.17:0/1463360189 shutdown_connections 2024-09-10T21:34:08.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:08.340+0000 7ff0cec00640 1 -- 172.21.15.17:0/1463360189 wait complete. 2024-09-10T21:34:08.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:08.344+0000 7ff0cec00640 1 Processor -- start 2024-09-10T21:34:08.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:08.344+0000 7ff0cec00640 1 -- start start 2024-09-10T21:34:08.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:08.344+0000 7ff0cec00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff0c80717c0 0x7ff0c80c59a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:08.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:08.344+0000 7ff0cec00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff0c80b2d80 0x7ff0c814a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:08.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:08.344+0000 7ff0cec00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff0c81527e0 0x7ff0c814acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:08.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:08.344+0000 7ff0cec00640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7ff0c8071c50 con 0x7ff0c81527e0 2024-09-10T21:34:08.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:08.344+0000 7ff0cec00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7ff0c806f500 con 0x7ff0c80717c0 2024-09-10T21:34:08.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:08.344+0000 7ff0cec00640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7ff0c806ed40 con 0x7ff0c80b2d80 2024-09-10T21:34:08.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:08.344+0000 7ff0cd800640 1 -- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff0c80717c0 msgr2=0x7ff0c80c59a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:3300/0 2024-09-10T21:34:08.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:08.344+0000 7ff0ce200640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff0c81527e0 0x7ff0c814acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:08.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:08.344+0000 7ff0cd800640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff0c80717c0 0x7ff0c80c59a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:34:08.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:08.344+0000 7ff0ce200640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff0c81527e0 0x7ff0c814acd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55106/0 (socket says 172.21.15.17:55106) 2024-09-10T21:34:08.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:08.344+0000 7ff0ce200640 1 -- 172.21.15.17:0/818134692 learned_addr learned my addr 172.21.15.17:0/818134692 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:34:08.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:08.344+0000 7ff0cce00640 1 --2- 172.21.15.17:0/818134692 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff0c80b2d80 0x7ff0c814a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:08.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:08.344+0000 7ff0ce200640 1 -- 172.21.15.17:0/818134692 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff0c80b2d80 msgr2=0x7ff0c814a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:08.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:08.344+0000 7ff0ce200640 1 --2- 172.21.15.17:0/818134692 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff0c80b2d80 0x7ff0c814a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:08.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:08.344+0000 7ff0ce200640 1 -- 172.21.15.17:0/818134692 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff0c80717c0 msgr2=0x7ff0c80c59a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:34:08.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:08.344+0000 7ff0ce200640 1 --2- 172.21.15.17:0/818134692 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff0c80717c0 0x7ff0c80c59a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:08.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:08.344+0000 7ff0ce200640 1 -- 172.21.15.17:0/818134692 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0c80c6530 con 0x7ff0c81527e0 2024-09-10T21:34:08.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:08.344+0000 7ff0ce200640 1 --2- 172.21.15.17:0/818134692 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff0c81527e0 0x7ff0c814acd0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff0c000b9e0 tx=0x7ff0c000bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:08.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:08.344+0000 7ff0c5600640 1 -- 172.21.15.17:0/818134692 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0c0019070 con 0x7ff0c81527e0 2024-09-10T21:34:08.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:08.344+0000 7ff0c5600640 1 -- 172.21.15.17:0/818134692 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff0c000cc10 con 0x7ff0c81527e0 2024-09-10T21:34:08.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:08.344+0000 7ff0c5600640 1 -- 172.21.15.17:0/818134692 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0c0014440 con 0x7ff0c81527e0 2024-09-10T21:34:08.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:08.344+0000 7ff0cec00640 1 -- 172.21.15.17:0/818134692 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0c80c6810 con 0x7ff0c81527e0 2024-09-10T21:34:08.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:08.344+0000 7ff0cec00640 1 -- 172.21.15.17:0/818134692 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0c80c51a0 con 0x7ff0c81527e0 2024-09-10T21:34:08.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:08.348+0000 7ff0cec00640 1 -- 172.21.15.17:0/818134692 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0c8158310 con 0x7ff0c81527e0 2024-09-10T21:34:08.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:08.348+0000 7ff0c5600640 1 -- 172.21.15.17:0/818134692 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7ff0c001d3b0 con 0x7ff0c81527e0 2024-09-10T21:34:08.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:08.348+0000 7ff0c5600640 1 --2- 172.21.15.17:0/818134692 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7ff0a8072590 0x7ff0a8074a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:08.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:08.348+0000 7ff0c5600640 1 -- 172.21.15.17:0/818134692 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7ff0c0093570 con 0x7ff0c81527e0 2024-09-10T21:34:08.355 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:08.348+0000 7ff0cd800640 1 --2- 172.21.15.17:0/818134692 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7ff0a8072590 0x7ff0a8074a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:08.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:08.352+0000 7ff0cd800640 1 --2- 172.21.15.17:0/818134692 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7ff0a8072590 0x7ff0a8074a50 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7ff0bc001060 tx=0x7ff0bc006d70 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:08.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:08.352+0000 7ff0c5600640 1 -- 172.21.15.17:0/818134692 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7ff0c005e1f0 con 0x7ff0c81527e0 2024-09-10T21:34:08.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:08.656+0000 7ff0cec00640 1 -- 172.21.15.17:0/818134692 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7ff0c80cadb0 con 0x7ff0c81527e0 2024-09-10T21:34:08.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:08.656+0000 7ff0c5600640 1 -- 172.21.15.17:0/818134692 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1557 (secure 0 0 0) 0x7ff0c0062c20 con 0x7ff0c81527e0 2024-09-10T21:34:08.661 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":80,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":16,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5e2f023f-d6a6-49f4-a256-f4807dce3739","modified":"2024-09-10T21:23:46.892130Z","created":"2024-09-10T21:23:46.892130Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3301","nonce":0},{"type":"v1","addr":"172.21.15.31:6790","nonce":0}]},"addr":"172.21.15.31:6790/0","public_addr":"172.21.15.31:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-10T21:34:08.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:08.656+0000 7ff0cec00640 1 -- 172.21.15.17:0/818134692 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7ff0a8072590 msgr2=0x7ff0a8074a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:08.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:08.656+0000 7ff0cec00640 1 --2- 172.21.15.17:0/818134692 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7ff0a8072590 0x7ff0a8074a50 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7ff0bc001060 tx=0x7ff0bc006d70 comp rx=0 tx=0).stop 2024-09-10T21:34:08.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:08.656+0000 7ff0cec00640 1 -- 172.21.15.17:0/818134692 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff0c81527e0 msgr2=0x7ff0c814acd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:08.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:08.656+0000 7ff0cec00640 1 --2- 172.21.15.17:0/818134692 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff0c81527e0 0x7ff0c814acd0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff0c000b9e0 tx=0x7ff0c000bea0 comp rx=0 tx=0).stop 2024-09-10T21:34:08.664 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:08.656+0000 7ff0cec00640 1 -- 172.21.15.17:0/818134692 shutdown_connections 2024-09-10T21:34:08.664 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:08.656+0000 7ff0cec00640 1 --2- 172.21.15.17:0/818134692 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7ff0a8072590 0x7ff0a8074a50 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:08.664 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:08.656+0000 7ff0cec00640 1 --2- 172.21.15.17:0/818134692 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff0c81527e0 0x7ff0c814acd0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:08.664 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:08.656+0000 7ff0cec00640 1 --2- 172.21.15.17:0/818134692 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff0c80b2d80 0x7ff0c814a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:08.664 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:08.656+0000 7ff0cec00640 1 --2- 172.21.15.17:0/818134692 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff0c80717c0 0x7ff0c80c59a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:08.664 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:08.656+0000 7ff0cec00640 1 -- 172.21.15.17:0/818134692 >> 172.21.15.17:0/818134692 conn(0x7ff0c8070ca0 msgr2=0x7ff0c80c7560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:08.664 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:08.656+0000 7ff0cec00640 1 -- 172.21.15.17:0/818134692 shutdown_connections 2024-09-10T21:34:08.664 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:08.656+0000 7ff0cec00640 1 -- 172.21.15.17:0/818134692 wait complete. 2024-09-10T21:34:11.689 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-10T21:34:11.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:11.848+0000 7f7bb6e00640 1 -- 172.21.15.17:0/1180504345 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7bb00c8070 msgr2=0x7f7bb00b2c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:11.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:11.848+0000 7f7bb6e00640 1 --2- 172.21.15.17:0/1180504345 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7bb00c8070 0x7f7bb00b2c60 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f7bb00c3430 tx=0x7f7ba801cd80 comp rx=0 tx=0).stop 2024-09-10T21:34:11.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:11.848+0000 7f7bb6e00640 1 -- 172.21.15.17:0/1180504345 shutdown_connections 2024-09-10T21:34:11.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:11.848+0000 7f7bb6e00640 1 --2- 172.21.15.17:0/1180504345 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7bb00c8070 0x7f7bb00b2c60 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:11.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:11.848+0000 7f7bb6e00640 1 --2- 172.21.15.17:0/1180504345 >> v2:172.21.15.31:3300/0 conn(0x7f7bb006f860 0x7f7bb00c7870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:11.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:11.848+0000 7f7bb6e00640 1 -- 172.21.15.17:0/1180504345 >> 172.21.15.17:0/1180504345 conn(0x7f7bb0070ca0 msgr2=0x7f7bb00710b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:11.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:11.848+0000 7f7bb6e00640 1 -- 172.21.15.17:0/1180504345 shutdown_connections 2024-09-10T21:34:11.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:11.848+0000 7f7bb6e00640 1 -- 172.21.15.17:0/1180504345 wait complete. 2024-09-10T21:34:11.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:11.852+0000 7f7bb6e00640 1 Processor -- start 2024-09-10T21:34:11.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:11.852+0000 7f7bb6e00640 1 -- start start 2024-09-10T21:34:11.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:11.852+0000 7f7bb6e00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7bb006f480 0x7f7bb0146b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:11.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:11.852+0000 7f7bb6e00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7bb006f860 0x7f7bb0147060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:11.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:11.852+0000 7f7bb6e00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f7bb00c8070 0x7f7bb013e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:11.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:11.852+0000 7f7bb6e00640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f7bb0163e60 con 0x7f7bb006f860 2024-09-10T21:34:11.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:11.852+0000 7f7bb6e00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f7bb0163ce0 con 0x7f7bb006f480 2024-09-10T21:34:11.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:11.852+0000 7f7bb6e00640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f7bb0163fe0 con 0x7f7bb00c8070 2024-09-10T21:34:11.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:11.852+0000 7f7bb5a00640 1 -- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7bb006f480 msgr2=0x7f7bb0146b20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:3300/0 2024-09-10T21:34:11.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:11.852+0000 7f7bb5000640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7bb006f860 0x7f7bb0147060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:11.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:11.852+0000 7f7bb5a00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7bb006f480 0x7f7bb0146b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:34:11.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:11.852+0000 7f7bb5000640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7bb006f860 0x7f7bb0147060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55116/0 (socket says 172.21.15.17:55116) 2024-09-10T21:34:11.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:11.852+0000 7f7bb5000640 1 -- 172.21.15.17:0/907805195 learned_addr learned my addr 172.21.15.17:0/907805195 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:34:11.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:11.852+0000 7f7bb6400640 1 --2- 172.21.15.17:0/907805195 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f7bb00c8070 0x7f7bb013e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:11.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:11.852+0000 7f7bb5000640 1 -- 172.21.15.17:0/907805195 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f7bb00c8070 msgr2=0x7f7bb013e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:11.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:11.852+0000 7f7bb5000640 1 --2- 172.21.15.17:0/907805195 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f7bb00c8070 0x7f7bb013e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:11.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:11.852+0000 7f7bb5000640 1 -- 172.21.15.17:0/907805195 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7bb006f480 msgr2=0x7f7bb0146b20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:34:11.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:11.852+0000 7f7bb5000640 1 --2- 172.21.15.17:0/907805195 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7bb006f480 0x7f7bb0146b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:11.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:11.852+0000 7f7bb5000640 1 -- 172.21.15.17:0/907805195 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bb0143bf0 con 0x7f7bb006f860 2024-09-10T21:34:11.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:11.852+0000 7f7bb6400640 1 --2- 172.21.15.17:0/907805195 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f7bb00c8070 0x7f7bb013e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:34:11.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:11.852+0000 7f7bb5000640 1 --2- 172.21.15.17:0/907805195 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7bb006f860 0x7f7bb0147060 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f7b9c00b550 tx=0x7f7b9c00ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:11.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:11.852+0000 7f7bad600640 1 -- 172.21.15.17:0/907805195 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b9c00f850 con 0x7f7bb006f860 2024-09-10T21:34:11.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:11.852+0000 7f7bad600640 1 -- 172.21.15.17:0/907805195 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7b9c00fb00 con 0x7f7bb006f860 2024-09-10T21:34:11.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:11.852+0000 7f7bb6e00640 1 -- 172.21.15.17:0/907805195 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7bb0143ed0 con 0x7f7bb006f860 2024-09-10T21:34:11.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:11.852+0000 7f7bad600640 1 -- 172.21.15.17:0/907805195 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b9c017400 con 0x7f7bb006f860 2024-09-10T21:34:11.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:11.852+0000 7f7bb6e00640 1 -- 172.21.15.17:0/907805195 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7bb0144620 con 0x7f7bb006f860 2024-09-10T21:34:11.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:11.856+0000 7f7bb6e00640 1 -- 172.21.15.17:0/907805195 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7bb00c7870 con 0x7f7bb006f860 2024-09-10T21:34:11.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:11.856+0000 7f7bad600640 1 -- 172.21.15.17:0/907805195 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f7b9c0203f0 con 0x7f7bb006f860 2024-09-10T21:34:11.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:11.856+0000 7f7bad600640 1 --2- 172.21.15.17:0/907805195 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f7b90072590 0x7f7b90074a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:11.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:11.856+0000 7f7bad600640 1 -- 172.21.15.17:0/907805195 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f7b9c0936e0 con 0x7f7bb006f860 2024-09-10T21:34:11.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:11.856+0000 7f7bb5a00640 1 --2- 172.21.15.17:0/907805195 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f7b90072590 0x7f7b90074a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:11.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:11.860+0000 7f7bb5a00640 1 --2- 172.21.15.17:0/907805195 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f7b90072590 0x7f7b90074a50 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f7ba4000bd0 tx=0x7f7ba4007a00 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:11.867 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:11.860+0000 7f7bad600640 1 -- 172.21.15.17:0/907805195 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f7b9c013030 con 0x7f7bb006f860 2024-09-10T21:34:12.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:12.156+0000 7f7bb6e00640 1 -- 172.21.15.17:0/907805195 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f7bb0146880 con 0x7f7bb006f860 2024-09-10T21:34:12.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:12.160+0000 7f7bad600640 1 -- 172.21.15.17:0/907805195 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1557 (secure 0 0 0) 0x7f7b9c05e9b0 con 0x7f7bb006f860 2024-09-10T21:34:12.164 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":80,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":20,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5e2f023f-d6a6-49f4-a256-f4807dce3739","modified":"2024-09-10T21:23:46.892130Z","created":"2024-09-10T21:23:46.892130Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3301","nonce":0},{"type":"v1","addr":"172.21.15.31:6790","nonce":0}]},"addr":"172.21.15.31:6790/0","public_addr":"172.21.15.31:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-10T21:34:12.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:12.160+0000 7f7bb6e00640 1 -- 172.21.15.17:0/907805195 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f7b90072590 msgr2=0x7f7b90074a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:12.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:12.160+0000 7f7bb6e00640 1 --2- 172.21.15.17:0/907805195 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f7b90072590 0x7f7b90074a50 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f7ba4000bd0 tx=0x7f7ba4007a00 comp rx=0 tx=0).stop 2024-09-10T21:34:12.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:12.160+0000 7f7bb6e00640 1 -- 172.21.15.17:0/907805195 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7bb006f860 msgr2=0x7f7bb0147060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:12.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:12.160+0000 7f7bb6e00640 1 --2- 172.21.15.17:0/907805195 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7bb006f860 0x7f7bb0147060 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f7b9c00b550 tx=0x7f7b9c00ba10 comp rx=0 tx=0).stop 2024-09-10T21:34:12.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:12.160+0000 7f7bb6e00640 1 -- 172.21.15.17:0/907805195 shutdown_connections 2024-09-10T21:34:12.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:12.160+0000 7f7bb6e00640 1 --2- 172.21.15.17:0/907805195 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f7b90072590 0x7f7b90074a50 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:12.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:12.160+0000 7f7bb6e00640 1 --2- 172.21.15.17:0/907805195 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f7bb00c8070 0x7f7bb013e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:12.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:12.160+0000 7f7bb6e00640 1 --2- 172.21.15.17:0/907805195 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7bb006f860 0x7f7bb0147060 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:12.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:12.160+0000 7f7bb6e00640 1 --2- 172.21.15.17:0/907805195 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7bb006f480 0x7f7bb0146b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:12.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:12.160+0000 7f7bb6e00640 1 -- 172.21.15.17:0/907805195 >> 172.21.15.17:0/907805195 conn(0x7f7bb0070ca0 msgr2=0x7f7bb006d390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:12.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:12.160+0000 7f7bb6e00640 1 -- 172.21.15.17:0/907805195 shutdown_connections 2024-09-10T21:34:12.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:12.160+0000 7f7bb6e00640 1 -- 172.21.15.17:0/907805195 wait complete. 2024-09-10T21:34:15.188 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-10T21:34:19.151 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:34:19.154 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:34:19.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:19.748+0000 7fec53200640 1 -- 172.21.15.17:0/158053052 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fec2c0036a0 msgr2=0x7fec2c005af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:19.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:19.748+0000 7fec53200640 1 --2- 172.21.15.17:0/158053052 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fec2c0036a0 0x7fec2c005af0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fec3c0064d0 tx=0x7fec3c01d430 comp rx=0 tx=0).stop 2024-09-10T21:34:19.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:19.748+0000 7fec51200640 1 -- 172.21.15.17:0/158053052 reap_dead start 2024-09-10T21:34:19.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:19.748+0000 7fec53200640 1 -- 172.21.15.17:0/158053052 shutdown_connections 2024-09-10T21:34:19.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:19.748+0000 7fec53200640 1 -- 172.21.15.17:0/158053052 >> 172.21.15.17:0/158053052 conn(0x7fec4c0ccf50 msgr2=0x7fec4c0cd360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:19.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:19.748+0000 7fec53200640 1 -- 172.21.15.17:0/158053052 shutdown_connections 2024-09-10T21:34:19.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:19.748+0000 7fec53200640 1 -- 172.21.15.17:0/158053052 wait complete. 2024-09-10T21:34:19.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:19.748+0000 7fec53200640 1 Processor -- start 2024-09-10T21:34:19.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:19.748+0000 7fec53200640 1 -- start start 2024-09-10T21:34:19.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:19.752+0000 7fec53200640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fec2c0036a0 0x7fec4c154fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:19.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:19.752+0000 7fec53200640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fec4c0c42e0 0x7fec4c0c9700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:19.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:19.752+0000 7fec53200640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec4c0c68c0 0x7fec4c145810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:19.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:19.752+0000 7fec53200640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fec4c0c1f70 con 0x7fec2c0036a0 2024-09-10T21:34:19.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:19.752+0000 7fec53200640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fec4c15b170 con 0x7fec4c0c68c0 2024-09-10T21:34:19.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:19.752+0000 7fec53200640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7fec4c0c7950 con 0x7fec4c0c42e0 2024-09-10T21:34:19.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:19.752+0000 7fec4be00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fec2c0036a0 0x7fec4c154fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:19.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:19.752+0000 7fec4be00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fec2c0036a0 0x7fec4c154fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45526/0 (socket says 172.21.15.17:45526) 2024-09-10T21:34:19.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:19.752+0000 7fec4be00640 1 -- 172.21.15.17:0/1030621304 learned_addr learned my addr 172.21.15.17:0/1030621304 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:34:19.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:19.752+0000 7fec51200640 1 -- 172.21.15.17:0/1030621304 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec4c0c68c0 msgr2=0x7fec4c145810 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:3300/0 2024-09-10T21:34:19.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:19.752+0000 7fec51200640 1 --2- 172.21.15.17:0/1030621304 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec4c0c68c0 0x7fec4c145810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:34:19.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:19.752+0000 7fec4b400640 1 --2- 172.21.15.17:0/1030621304 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fec4c0c42e0 0x7fec4c0c9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:19.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:19.752+0000 7fec4be00640 1 -- 172.21.15.17:0/1030621304 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fec4c0c42e0 msgr2=0x7fec4c0c9700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:19.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:19.752+0000 7fec4be00640 1 --2- 172.21.15.17:0/1030621304 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fec4c0c42e0 0x7fec4c0c9700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:19.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:19.752+0000 7fec4be00640 1 -- 172.21.15.17:0/1030621304 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec4c0c68c0 msgr2=0x7fec4c145810 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:34:19.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:19.752+0000 7fec4be00640 1 --2- 172.21.15.17:0/1030621304 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec4c0c68c0 0x7fec4c145810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:19.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:19.752+0000 7fec4be00640 1 -- 172.21.15.17:0/1030621304 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec4c1580d0 con 0x7fec2c0036a0 2024-09-10T21:34:19.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:19.752+0000 7fec4be00640 1 --2- 172.21.15.17:0/1030621304 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fec2c0036a0 0x7fec4c154fa0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fec3c01dcd0 tx=0x7fec3c0ad8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:19.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:19.752+0000 7fec43e00640 1 -- 172.21.15.17:0/1030621304 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec3c0c5070 con 0x7fec2c0036a0 2024-09-10T21:34:19.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:19.752+0000 7fec43e00640 1 -- 172.21.15.17:0/1030621304 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fec3c0a7040 con 0x7fec2c0036a0 2024-09-10T21:34:19.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:19.752+0000 7fec53200640 1 -- 172.21.15.17:0/1030621304 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec4c1555e0 con 0x7fec2c0036a0 2024-09-10T21:34:19.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:19.752+0000 7fec43e00640 1 -- 172.21.15.17:0/1030621304 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec3c0c0680 con 0x7fec2c0036a0 2024-09-10T21:34:19.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:19.752+0000 7fec53200640 1 -- 172.21.15.17:0/1030621304 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec4c157270 con 0x7fec2c0036a0 2024-09-10T21:34:19.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:19.752+0000 7fec53200640 1 -- 172.21.15.17:0/1030621304 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec4c0b28c0 con 0x7fec2c0036a0 2024-09-10T21:34:19.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:19.756+0000 7fec43e00640 1 -- 172.21.15.17:0/1030621304 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fec3c0aa070 con 0x7fec2c0036a0 2024-09-10T21:34:19.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:19.756+0000 7fec43e00640 1 --2- 172.21.15.17:0/1030621304 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fec2c075bf0 0x7fec2c0780b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:19.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:19.756+0000 7fec43e00640 1 -- 172.21.15.17:0/1030621304 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fec3c12ecb0 con 0x7fec2c0036a0 2024-09-10T21:34:19.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:19.756+0000 7fec4b400640 1 --2- 172.21.15.17:0/1030621304 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fec2c075bf0 0x7fec2c0780b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:19.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:19.756+0000 7fec4b400640 1 --2- 172.21.15.17:0/1030621304 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fec2c075bf0 0x7fec2c0780b0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fec4c06d480 tx=0x7fec380135b0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:19.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:19.760+0000 7fec43e00640 1 -- 172.21.15.17:0/1030621304 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fec3c0bb070 con 0x7fec2c0036a0 2024-09-10T21:34:20.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:20.028+0000 7fec53200640 1 -- 172.21.15.17:0/1030621304 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fec4c0c7d90 con 0x7fec2c0036a0 2024-09-10T21:34:20.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:20.032+0000 7fec43e00640 1 -- 172.21.15.17:0/1030621304 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1550 (secure 0 0 0) 0x7fec3c0170f0 con 0x7fec2c0036a0 2024-09-10T21:34:20.036 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":84,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5e2f023f-d6a6-49f4-a256-f4807dce3739","modified":"2024-09-10T21:23:46.892130Z","created":"2024-09-10T21:23:46.892130Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3301","nonce":0},{"type":"v1","addr":"172.21.15.31:6790","nonce":0}]},"addr":"172.21.15.31:6790/0","public_addr":"172.21.15.31:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-10T21:34:20.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:20.032+0000 7fec53200640 1 -- 172.21.15.17:0/1030621304 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fec2c075bf0 msgr2=0x7fec2c0780b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:20.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:20.032+0000 7fec53200640 1 --2- 172.21.15.17:0/1030621304 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fec2c075bf0 0x7fec2c0780b0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fec4c06d480 tx=0x7fec380135b0 comp rx=0 tx=0).stop 2024-09-10T21:34:20.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:20.032+0000 7fec53200640 1 -- 172.21.15.17:0/1030621304 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fec2c0036a0 msgr2=0x7fec4c154fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:20.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:20.032+0000 7fec53200640 1 --2- 172.21.15.17:0/1030621304 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fec2c0036a0 0x7fec4c154fa0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fec3c01dcd0 tx=0x7fec3c0ad8f0 comp rx=0 tx=0).stop 2024-09-10T21:34:20.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:20.032+0000 7fec53200640 1 -- 172.21.15.17:0/1030621304 shutdown_connections 2024-09-10T21:34:20.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:20.032+0000 7fec53200640 1 --2- 172.21.15.17:0/1030621304 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fec2c075bf0 0x7fec2c0780b0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:20.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:20.032+0000 7fec53200640 1 --2- 172.21.15.17:0/1030621304 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fec4c0c68c0 0x7fec4c145810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:20.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:20.032+0000 7fec53200640 1 --2- 172.21.15.17:0/1030621304 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fec4c0c42e0 0x7fec4c0c9700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:20.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:20.032+0000 7fec53200640 1 --2- 172.21.15.17:0/1030621304 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fec2c0036a0 0x7fec4c154fa0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:20.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:20.032+0000 7fec53200640 1 -- 172.21.15.17:0/1030621304 >> 172.21.15.17:0/1030621304 conn(0x7fec4c0ccf50 msgr2=0x7fec4c075c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:20.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:20.032+0000 7fec53200640 1 -- 172.21.15.17:0/1030621304 shutdown_connections 2024-09-10T21:34:20.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:20.032+0000 7fec53200640 1 -- 172.21.15.17:0/1030621304 wait complete. 2024-09-10T21:34:20.061 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2024-09-10T21:34:20.061 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon a 2024-09-10T21:34:20.061 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-10T21:34:20.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:20.224+0000 7fd192000640 1 -- 172.21.15.17:0/3229353840 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd18c152450 msgr2=0x7fd18c072960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:20.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:20.224+0000 7fd192000640 1 --2- 172.21.15.17:0/3229353840 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd18c152450 0x7fd18c072960 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd18c0ce7a0 tx=0x7fd17801c8e0 comp rx=0 tx=0).stop 2024-09-10T21:34:20.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:20.228+0000 7fd192000640 1 -- 172.21.15.17:0/3229353840 shutdown_connections 2024-09-10T21:34:20.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:20.228+0000 7fd192000640 1 --2- 172.21.15.17:0/3229353840 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd18c0cbd10 0x7fd18c158850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:20.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:20.228+0000 7fd192000640 1 --2- 172.21.15.17:0/3229353840 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd18c152450 0x7fd18c072960 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:20.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:20.228+0000 7fd192000640 1 --2- 172.21.15.17:0/3229353840 >> v2:172.21.15.31:3300/0 conn(0x7fd18c152070 0x7fd18c072420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:20.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:20.228+0000 7fd192000640 1 -- 172.21.15.17:0/3229353840 >> 172.21.15.17:0/3229353840 conn(0x7fd18c06cc20 msgr2=0x7fd18c06d030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:20.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:20.228+0000 7fd192000640 1 -- 172.21.15.17:0/3229353840 shutdown_connections 2024-09-10T21:34:20.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:20.228+0000 7fd192000640 1 -- 172.21.15.17:0/3229353840 wait complete. 2024-09-10T21:34:20.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:20.228+0000 7fd192000640 1 Processor -- start 2024-09-10T21:34:20.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:20.228+0000 7fd192000640 1 -- start start 2024-09-10T21:34:20.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:20.228+0000 7fd192000640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd18c0cbd10 0x7fd18c1624c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:20.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:20.228+0000 7fd192000640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd18c152070 0x7fd18c146740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:20.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:20.228+0000 7fd192000640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd18c152450 0x7fd18c1f7a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:20.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:20.228+0000 7fd192000640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fd18c15c060 con 0x7fd18c152070 2024-09-10T21:34:20.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:20.228+0000 7fd192000640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fd18c1591a0 con 0x7fd18c152450 2024-09-10T21:34:20.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:20.228+0000 7fd192000640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7fd18c15c1e0 con 0x7fd18c0cbd10 2024-09-10T21:34:20.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:20.228+0000 7fd18be00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd18c152070 0x7fd18c146740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:20.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:20.228+0000 7fd191600640 1 -- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd18c152450 msgr2=0x7fd18c1f7a70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:3300/0 2024-09-10T21:34:20.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:20.228+0000 7fd191600640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd18c152450 0x7fd18c1f7a70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:34:20.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:20.228+0000 7fd18be00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd18c152070 0x7fd18c146740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45554/0 (socket says 172.21.15.17:45554) 2024-09-10T21:34:20.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:20.228+0000 7fd18be00640 1 -- 172.21.15.17:0/427165769 learned_addr learned my addr 172.21.15.17:0/427165769 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:34:20.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:20.228+0000 7fd190c00640 1 --2- 172.21.15.17:0/427165769 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd18c0cbd10 0x7fd18c1624c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:20.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:20.228+0000 7fd18be00640 1 -- 172.21.15.17:0/427165769 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd18c0cbd10 msgr2=0x7fd18c1624c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:20.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:20.228+0000 7fd18be00640 1 --2- 172.21.15.17:0/427165769 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd18c0cbd10 0x7fd18c1624c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:20.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:20.228+0000 7fd18be00640 1 -- 172.21.15.17:0/427165769 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd18c152450 msgr2=0x7fd18c1f7a70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:34:20.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:20.228+0000 7fd18be00640 1 --2- 172.21.15.17:0/427165769 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd18c152450 0x7fd18c1f7a70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:20.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:20.228+0000 7fd18be00640 1 -- 172.21.15.17:0/427165769 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd18c06fad0 con 0x7fd18c152070 2024-09-10T21:34:20.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:20.228+0000 7fd190c00640 1 --2- 172.21.15.17:0/427165769 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd18c0cbd10 0x7fd18c1624c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:34:20.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:20.232+0000 7fd18be00640 1 --2- 172.21.15.17:0/427165769 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd18c152070 0x7fd18c146740 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd1780b0040 tx=0x7fd1780a5560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:20.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:20.232+0000 7fd188c00640 1 -- 172.21.15.17:0/427165769 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1780b2040 con 0x7fd18c152070 2024-09-10T21:34:20.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:20.232+0000 7fd188c00640 1 -- 172.21.15.17:0/427165769 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd1780043b0 con 0x7fd18c152070 2024-09-10T21:34:20.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:20.232+0000 7fd192000640 1 -- 172.21.15.17:0/427165769 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd18c1498e0 con 0x7fd18c152070 2024-09-10T21:34:20.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:20.232+0000 7fd188c00640 1 -- 172.21.15.17:0/427165769 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1780052d0 con 0x7fd18c152070 2024-09-10T21:34:20.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:20.232+0000 7fd192000640 1 -- 172.21.15.17:0/427165769 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd18c160ab0 con 0x7fd18c152070 2024-09-10T21:34:20.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:20.232+0000 7fd192000640 1 --2- 172.21.15.17:0/427165769 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd18c1f7fb0 0x7fd18c06e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:20.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:20.232+0000 7fd192000640 1 -- 172.21.15.17:0/427165769 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fd18c069ef0 con 0x7fd18c1f7fb0 2024-09-10T21:34:20.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:20.232+0000 7fd190c00640 1 --2- 172.21.15.17:0/427165769 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd18c1f7fb0 0x7fd18c06e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:20.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:20.232+0000 7fd190c00640 1 -- 172.21.15.17:0/427165769 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd18c158850 con 0x7fd18c1f7fb0 2024-09-10T21:34:20.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:20.232+0000 7fd190c00640 1 --2- 172.21.15.17:0/427165769 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd18c1f7fb0 0x7fd18c06e390 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd18000c6f0 tx=0x7fd18000cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:20.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:20.232+0000 7fd188c00640 1 -- 172.21.15.17:0/427165769 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd180015070 con 0x7fd18c1f7fb0 2024-09-10T21:34:20.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:20.232+0000 7fd188c00640 1 -- 172.21.15.17:0/427165769 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fd1780046d0 con 0x7fd18c152070 2024-09-10T21:34:20.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:20.232+0000 7fd188c00640 1 --2- 172.21.15.17:0/427165769 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fd16c0724e0 0x7fd16c0749a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:20.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:20.236+0000 7fd188c00640 1 -- 172.21.15.17:0/427165769 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fd17812cd70 con 0x7fd18c152070 2024-09-10T21:34:20.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:20.236+0000 7fd188c00640 1 -- 172.21.15.17:0/427165769 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fd180007890 con 0x7fd18c1f7fb0 2024-09-10T21:34:20.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:20.236+0000 7fd188c00640 1 -- 172.21.15.17:0/427165769 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd18c1f7fb0 msgr2=0x7fd18c06e390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:20.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:20.236+0000 7fd188c00640 1 --2- 172.21.15.17:0/427165769 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd18c1f7fb0 0x7fd18c06e390 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd18000c6f0 tx=0x7fd18000cbb0 comp rx=0 tx=0).stop 2024-09-10T21:34:20.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:20.236+0000 7fd188c00640 1 -- 172.21.15.17:0/427165769 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd18c1f7fb0 msgr2=0x7fd18c06e390 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:34:20.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:20.236+0000 7fd188c00640 1 --2- 172.21.15.17:0/427165769 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd18c1f7fb0 0x7fd18c06e390 secure :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd18000c6f0 tx=0x7fd18000cbb0 comp rx=0 tx=0).stop 2024-09-10T21:34:20.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:20.236+0000 7fd191600640 1 --2- 172.21.15.17:0/427165769 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fd16c0724e0 0x7fd16c0749a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:20.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:20.236+0000 7fd191600640 1 --2- 172.21.15.17:0/427165769 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fd16c0724e0 0x7fd16c0749a0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fd1840093c0 tx=0x7fd1840096f0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:20.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:20.244+0000 7fd192000640 1 --2- 172.21.15.17:0/427165769 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd18c15b260 0x7fd18c1f7360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:20.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:20.244+0000 7fd192000640 1 -- 172.21.15.17:0/427165769 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fd18c06ddd0 con 0x7fd18c15b260 2024-09-10T21:34:20.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:20.244+0000 7fd190c00640 1 --2- 172.21.15.17:0/427165769 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd18c15b260 0x7fd18c1f7360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:20.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:20.244+0000 7fd190c00640 1 -- 172.21.15.17:0/427165769 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fd18c06e0b0 con 0x7fd18c15b260 2024-09-10T21:34:20.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:20.244+0000 7fd190c00640 1 --2- 172.21.15.17:0/427165769 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd18c15b260 0x7fd18c1f7360 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fd180017000 tx=0x7fd180004810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:20.251 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:20.244+0000 7fd188c00640 1 -- 172.21.15.17:0/427165769 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd180024610 con 0x7fd18c15b260 2024-09-10T21:34:20.251 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:20.244+0000 7fd188c00640 1 -- 172.21.15.17:0/427165769 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5032 (secure 0 0 0) 0x7fd1800113f0 con 0x7fd18c15b260 2024-09-10T21:34:20.251 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:20.244+0000 7fd188c00640 1 -- 172.21.15.17:0/427165769 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd18c15b260 msgr2=0x7fd18c1f7360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:20.251 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:20.244+0000 7fd188c00640 1 --2- 172.21.15.17:0/427165769 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd18c15b260 0x7fd18c1f7360 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fd180017000 tx=0x7fd180004810 comp rx=0 tx=0).stop 2024-09-10T21:34:20.251 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:20.244+0000 7fd188c00640 1 -- 172.21.15.17:0/427165769 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd18c15b260 msgr2=0x7fd18c1f7360 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:34:20.251 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:20.244+0000 7fd188c00640 1 --2- 172.21.15.17:0/427165769 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd18c15b260 0x7fd18c1f7360 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:20.252 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-10T21:34:20.252 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:34:20.252 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:34:20.252 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "leader", 2024-09-10T21:34:20.252 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 84, 2024-09-10T21:34:20.252 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 40851, 2024-09-10T21:34:20.252 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-10T21:34:20.252 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-10T21:34:20.252 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-10T21:34:20.252 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:34:20.252 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 0, 2024-09-10T21:34:20.253 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:34:20.253 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-10T21:34:20.253 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-10T21:34:20.253 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:34:20.253 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:34:20.253 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:34:20.253 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:34:20.253 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:34:20.253 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:34:20.253 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:34:20.253 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:34:20.253 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:34:20.253 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:34:20.254 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:34:20.254 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:34:20.254 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-10T21:34:20.254 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-10T21:34:20.254 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:34:20.254 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:34:20.254 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:34:20.254 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:34:20.254 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:34:20.254 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:34:20.255 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:34:20.255 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:34:20.255 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:34:20.255 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:34:20.255 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:34:20.255 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:34:20.255 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:34:20.255 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-10T21:34:20.255 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-10T21:34:20.255 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-10T21:34:20.255 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-10T21:34:20.256 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-10T21:34:20.256 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "5e2f023f-d6a6-49f4-a256-f4807dce3739", 2024-09-10T21:34:20.256 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:34:20.256 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:34:20.256 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-10T21:34:20.256 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-10T21:34:20.256 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-10T21:34:20.256 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-10T21:34:20.256 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-10T21:34:20.256 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-10T21:34:20.256 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-10T21:34:20.256 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:34:20.256 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-10T21:34:20.257 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:34:20.257 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:34:20.257 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:34:20.257 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:34:20.257 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:34:20.257 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:34:20.257 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:34:20.257 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:34:20.257 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:34:20.257 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:34:20.257 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:34:20.258 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:34:20.258 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-10T21:34:20.258 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:34:20.258 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-10T21:34:20.258 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:34:20.258 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:34:20.258 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:34:20.258 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:34:20.258 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:34:20.259 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:34:20.259 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:34:20.259 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-10T21:34:20.259 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:34:20.259 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:34:20.259 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:34:20.259 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:34:20.259 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-10T21:34:20.259 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:34:20.259 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:34:20.259 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:34:20.259 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:34:20.259 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-10T21:34:20.260 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-10T21:34:20.260 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:34:20.260 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:34:20.260 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:34:20.260 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:34:20.260 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:34:20.260 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:34:20.260 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:34:20.260 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:34:20.260 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:34:20.260 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:34:20.260 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:34:20.260 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3300", 2024-09-10T21:34:20.261 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:34:20.261 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:34:20.261 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:34:20.261 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:34:20.261 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789", 2024-09-10T21:34:20.261 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:34:20.261 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:34:20.261 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:34:20.261 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:34:20.261 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789/0", 2024-09-10T21:34:20.261 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6789/0", 2024-09-10T21:34:20.261 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:34:20.261 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:34:20.261 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:34:20.261 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:34:20.262 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:34:20.262 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:34:20.262 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:34:20.262 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:34:20.262 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:34:20.262 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:34:20.262 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:34:20.262 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3301", 2024-09-10T21:34:20.263 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:34:20.263 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:34:20.263 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:34:20.263 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:34:20.263 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790", 2024-09-10T21:34:20.263 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:34:20.263 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:34:20.263 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:34:20.263 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:34:20.263 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790/0", 2024-09-10T21:34:20.263 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6790/0", 2024-09-10T21:34:20.263 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:34:20.263 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:34:20.264 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:34:20.264 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:34:20.264 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:34:20.264 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:34:20.264 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-10T21:34:20.265 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-10T21:34:20.265 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:34:20.265 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:34:20.265 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:34:20.265 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:34:20.265 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:34:20.265 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:34:20.265 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": [ 2024-09-10T21:34:20.265 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:34:20.265 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:34:20.265 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:34:20.265 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 3 2024-09-10T21:34:20.266 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:34:20.266 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:34:20.266 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-10T21:34:20.266 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:34:20.266 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:34:20.266 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:34:20.266 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 6 2024-09-10T21:34:20.266 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:34:20.266 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:34:20.266 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-10T21:34:20.266 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:34:20.266 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:34:20.266 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:34:20.267 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 6 2024-09-10T21:34:20.267 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:34:20.267 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:34:20.267 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": [ 2024-09-10T21:34:20.267 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:34:20.267 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:34:20.267 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:34:20.267 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 2 2024-09-10T21:34:20.267 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:34:20.267 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:34:20.267 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:34:20.267 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-10T21:34:20.267 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-10T21:34:20.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:20.244+0000 7fd192000640 1 -- 172.21.15.17:0/427165769 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fd16c0724e0 msgr2=0x7fd16c0749a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:20.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:20.244+0000 7fd192000640 1 --2- 172.21.15.17:0/427165769 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fd16c0724e0 0x7fd16c0749a0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fd1840093c0 tx=0x7fd1840096f0 comp rx=0 tx=0).stop 2024-09-10T21:34:20.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:20.244+0000 7fd192000640 1 -- 172.21.15.17:0/427165769 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd18c152070 msgr2=0x7fd18c146740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:20.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:20.244+0000 7fd192000640 1 --2- 172.21.15.17:0/427165769 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd18c152070 0x7fd18c146740 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd1780b0040 tx=0x7fd1780a5560 comp rx=0 tx=0).stop 2024-09-10T21:34:20.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:20.244+0000 7fd191600640 1 -- 172.21.15.17:0/427165769 reap_dead start 2024-09-10T21:34:20.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:20.244+0000 7fd191600640 1 -- 172.21.15.17:0/427165769 reap_dead start 2024-09-10T21:34:20.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:20.244+0000 7fd192000640 1 -- 172.21.15.17:0/427165769 shutdown_connections 2024-09-10T21:34:20.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:20.244+0000 7fd192000640 1 -- 172.21.15.17:0/427165769 >> 172.21.15.17:0/427165769 conn(0x7fd18c06cc20 msgr2=0x7fd18c156a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:20.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:20.244+0000 7fd192000640 1 -- 172.21.15.17:0/427165769 shutdown_connections 2024-09-10T21:34:20.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:20.244+0000 7fd192000640 1 -- 172.21.15.17:0/427165769 wait complete. 2024-09-10T21:34:20.271 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-09-10T21:34:20.271 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon c 2024-09-10T21:34:20.271 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-10T21:34:20.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:20.424+0000 7efd9f000640 1 -- 172.21.15.17:0/661433470 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efd7c004810 msgr2=0x7efd7c004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:20.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:20.424+0000 7efd9f000640 1 --2- 172.21.15.17:0/661433470 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efd7c004810 0x7efd7c004c90 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7efd98074620 tx=0x7efd8001c7c0 comp rx=0 tx=0).stop 2024-09-10T21:34:20.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:20.424+0000 7efd9d000640 1 -- 172.21.15.17:0/661433470 reap_dead start 2024-09-10T21:34:20.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:20.424+0000 7efd9f000640 1 -- 172.21.15.17:0/661433470 shutdown_connections 2024-09-10T21:34:20.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:20.424+0000 7efd9f000640 1 -- 172.21.15.17:0/661433470 >> 172.21.15.17:0/661433470 conn(0x7efd980702f0 msgr2=0x7efd98070700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:20.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:20.424+0000 7efd9f000640 1 -- 172.21.15.17:0/661433470 shutdown_connections 2024-09-10T21:34:20.431 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:20.424+0000 7efd9f000640 1 -- 172.21.15.17:0/661433470 wait complete. 2024-09-10T21:34:20.432 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:20.428+0000 7efd9f000640 1 Processor -- start 2024-09-10T21:34:20.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:20.428+0000 7efd9f000640 1 -- start start 2024-09-10T21:34:20.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:20.428+0000 7efd9f000640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efd7c004810 0x7efd981e43b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:20.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:20.428+0000 7efd9f000640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efd981e3600 0x7efd98152f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:20.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:20.428+0000 7efd9f000640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7efd981566a0 0x7efd981f3db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:20.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:20.428+0000 7efd9f000640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7efd9815af50 con 0x7efd981e3600 2024-09-10T21:34:20.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:20.428+0000 7efd9f000640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7efd9815ac70 con 0x7efd7c004810 2024-09-10T21:34:20.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:20.428+0000 7efd9f000640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7efd98072790 con 0x7efd981566a0 2024-09-10T21:34:20.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:20.428+0000 7efd97e00640 1 -- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efd7c004810 msgr2=0x7efd981e43b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:3300/0 2024-09-10T21:34:20.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:20.428+0000 7efd8fe00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efd981e3600 0x7efd98152f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:20.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:20.428+0000 7efd97e00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efd7c004810 0x7efd981e43b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:34:20.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:20.428+0000 7efd8fe00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efd981e3600 0x7efd98152f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45582/0 (socket says 172.21.15.17:45582) 2024-09-10T21:34:20.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:20.428+0000 7efd8fe00640 1 -- 172.21.15.17:0/595109948 learned_addr learned my addr 172.21.15.17:0/595109948 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:34:20.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:20.428+0000 7efd9d000640 1 --2- 172.21.15.17:0/595109948 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7efd981566a0 0x7efd981f3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:20.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:20.428+0000 7efd8fe00640 1 -- 172.21.15.17:0/595109948 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7efd981566a0 msgr2=0x7efd981f3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:20.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:20.428+0000 7efd8fe00640 1 --2- 172.21.15.17:0/595109948 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7efd981566a0 0x7efd981f3db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:20.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:20.428+0000 7efd8fe00640 1 -- 172.21.15.17:0/595109948 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efd7c004810 msgr2=0x7efd981e43b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:34:20.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:20.428+0000 7efd8fe00640 1 --2- 172.21.15.17:0/595109948 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efd7c004810 0x7efd981e43b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:20.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:20.428+0000 7efd8fe00640 1 -- 172.21.15.17:0/595109948 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd98073080 con 0x7efd981e3600 2024-09-10T21:34:20.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:20.428+0000 7efd8fe00640 1 --2- 172.21.15.17:0/595109948 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efd981e3600 0x7efd98152f90 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7efd80009910 tx=0x7efd80009640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:20.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:20.428+0000 7efd94c00640 1 -- 172.21.15.17:0/595109948 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd800bc070 con 0x7efd981e3600 2024-09-10T21:34:20.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:20.428+0000 7efd94c00640 1 -- 172.21.15.17:0/595109948 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efd80004930 con 0x7efd981e3600 2024-09-10T21:34:20.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:20.428+0000 7efd9f000640 1 -- 172.21.15.17:0/595109948 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd981f04f0 con 0x7efd981e3600 2024-09-10T21:34:20.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:20.428+0000 7efd94c00640 1 -- 172.21.15.17:0/595109948 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd800b2040 con 0x7efd981e3600 2024-09-10T21:34:20.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:20.428+0000 7efd9f000640 1 -- 172.21.15.17:0/595109948 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd981f0ab0 con 0x7efd981e3600 2024-09-10T21:34:20.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:20.432+0000 7efd9f000640 1 --2- 172.21.15.17:0/595109948 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7efd981f42f0 0x7efd981f0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:20.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:20.432+0000 7efd9f000640 1 -- 172.21.15.17:0/595109948 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7efd980c9640 con 0x7efd981f42f0 2024-09-10T21:34:20.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:20.432+0000 7efd97e00640 1 --2- 172.21.15.17:0/595109948 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7efd981f42f0 0x7efd981f0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:20.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:20.432+0000 7efd97e00640 1 -- 172.21.15.17:0/595109948 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7efd981f39d0 con 0x7efd981f42f0 2024-09-10T21:34:20.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:20.432+0000 7efd97e00640 1 --2- 172.21.15.17:0/595109948 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7efd981f42f0 0x7efd981f0cf0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7efd88003650 tx=0x7efd88012870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:20.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:20.432+0000 7efd94c00640 1 -- 172.21.15.17:0/595109948 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd88013920 con 0x7efd981f42f0 2024-09-10T21:34:20.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:20.432+0000 7efd94c00640 1 -- 172.21.15.17:0/595109948 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7efd800b5030 con 0x7efd981e3600 2024-09-10T21:34:20.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:20.432+0000 7efd94c00640 1 --2- 172.21.15.17:0/595109948 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7efd740724e0 0x7efd740749a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:20.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:20.432+0000 7efd94c00640 1 -- 172.21.15.17:0/595109948 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7efd8012d710 con 0x7efd981e3600 2024-09-10T21:34:20.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:20.436+0000 7efd94c00640 1 -- 172.21.15.17:0/595109948 <== mon.2 v2:172.21.15.31:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7efd88010480 con 0x7efd981f42f0 2024-09-10T21:34:20.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:20.436+0000 7efd94c00640 1 -- 172.21.15.17:0/595109948 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7efd981f42f0 msgr2=0x7efd981f0cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:20.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:20.436+0000 7efd94c00640 1 --2- 172.21.15.17:0/595109948 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7efd981f42f0 0x7efd981f0cf0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7efd88003650 tx=0x7efd88012870 comp rx=0 tx=0).stop 2024-09-10T21:34:20.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:20.436+0000 7efd94c00640 1 -- 172.21.15.17:0/595109948 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7efd981f42f0 msgr2=0x7efd981f0cf0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:34:20.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:20.436+0000 7efd94c00640 1 --2- 172.21.15.17:0/595109948 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7efd981f42f0 0x7efd981f0cf0 secure :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7efd88003650 tx=0x7efd88012870 comp rx=0 tx=0).stop 2024-09-10T21:34:20.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:20.436+0000 7efd9d000640 1 --2- 172.21.15.17:0/595109948 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7efd740724e0 0x7efd740749a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:20.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:20.436+0000 7efd9d000640 1 --2- 172.21.15.17:0/595109948 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7efd740724e0 0x7efd740749a0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7efd980c8690 tx=0x7efd90009500 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:20.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:20.444+0000 7efd9f000640 1 --2- 172.21.15.17:0/595109948 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7efd9814f960 0x7efd9814fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:20.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:20.444+0000 7efd9f000640 1 -- 172.21.15.17:0/595109948 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7efd98070a80 con 0x7efd9814f960 2024-09-10T21:34:20.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:20.444+0000 7efd97e00640 1 --2- 172.21.15.17:0/595109948 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7efd9814f960 0x7efd9814fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:20.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:20.444+0000 7efd97e00640 1 -- 172.21.15.17:0/595109948 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7efd980707a0 con 0x7efd9814f960 2024-09-10T21:34:20.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:20.444+0000 7efd97e00640 1 --2- 172.21.15.17:0/595109948 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7efd9814f960 0x7efd9814fd70 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7efd88019000 tx=0x7efd980b4180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:20.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:20.444+0000 7efd94c00640 1 -- 172.21.15.17:0/595109948 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd88010db0 con 0x7efd9814f960 2024-09-10T21:34:20.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:20.444+0000 7efd94c00640 1 -- 172.21.15.17:0/595109948 <== mon.2 v2:172.21.15.31:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4864 (secure 0 0 0) 0x7efd88010d90 con 0x7efd9814f960 2024-09-10T21:34:20.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:20.444+0000 7efd94c00640 1 -- 172.21.15.17:0/595109948 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7efd9814f960 msgr2=0x7efd9814fd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:20.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:20.444+0000 7efd94c00640 1 --2- 172.21.15.17:0/595109948 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7efd9814f960 0x7efd9814fd70 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7efd88019000 tx=0x7efd980b4180 comp rx=0 tx=0).stop 2024-09-10T21:34:20.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:20.444+0000 7efd94c00640 1 -- 172.21.15.17:0/595109948 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7efd9814f960 msgr2=0x7efd9814fd70 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:34:20.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:20.444+0000 7efd94c00640 1 --2- 172.21.15.17:0/595109948 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7efd9814f960 0x7efd9814fd70 secure :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7efd88019000 tx=0x7efd980b4180 comp rx=0 tx=0).stop 2024-09-10T21:34:20.451 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-10T21:34:20.451 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:34:20.451 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:34:20.451 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-10T21:34:20.451 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 84, 2024-09-10T21:34:20.451 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 192753, 2024-09-10T21:34:20.451 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-10T21:34:20.451 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-10T21:34:20.451 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-10T21:34:20.451 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:34:20.452 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 0, 2024-09-10T21:34:20.452 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:34:20.452 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-10T21:34:20.452 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-10T21:34:20.452 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:34:20.452 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:34:20.452 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:34:20.452 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:34:20.452 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:34:20.452 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:34:20.452 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:34:20.452 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:34:20.453 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:34:20.453 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:34:20.453 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:34:20.453 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:34:20.453 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-10T21:34:20.453 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-10T21:34:20.453 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:34:20.453 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:34:20.453 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:34:20.453 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:34:20.453 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:34:20.454 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:34:20.454 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:34:20.454 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:34:20.454 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:34:20.454 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:34:20.454 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:34:20.454 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:34:20.454 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:34:20.454 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-10T21:34:20.454 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-10T21:34:20.454 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-10T21:34:20.454 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-10T21:34:20.455 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-10T21:34:20.455 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "5e2f023f-d6a6-49f4-a256-f4807dce3739", 2024-09-10T21:34:20.455 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:34:20.455 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:34:20.455 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-10T21:34:20.455 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-10T21:34:20.455 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-10T21:34:20.455 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-10T21:34:20.455 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-10T21:34:20.455 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-10T21:34:20.455 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-10T21:34:20.455 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:34:20.455 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-10T21:34:20.456 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:34:20.456 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:34:20.456 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:34:20.456 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:34:20.456 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:34:20.456 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:34:20.456 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:34:20.456 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:34:20.456 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:34:20.456 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:34:20.456 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:34:20.456 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:34:20.457 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-10T21:34:20.457 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:34:20.457 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-10T21:34:20.457 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:34:20.457 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:34:20.457 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:34:20.457 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:34:20.457 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:34:20.457 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:34:20.457 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:34:20.457 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-10T21:34:20.457 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:34:20.458 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:34:20.458 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:34:20.458 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:34:20.458 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-10T21:34:20.458 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:34:20.458 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:34:20.458 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:34:20.458 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:34:20.458 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-10T21:34:20.458 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-10T21:34:20.458 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:34:20.458 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:34:20.458 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:34:20.458 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:34:20.459 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:34:20.459 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:34:20.459 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:34:20.459 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:34:20.459 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:34:20.459 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:34:20.459 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:34:20.459 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3300", 2024-09-10T21:34:20.459 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:34:20.459 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:34:20.459 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:34:20.459 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:34:20.459 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789", 2024-09-10T21:34:20.460 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:34:20.460 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:34:20.460 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:34:20.460 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:34:20.460 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789/0", 2024-09-10T21:34:20.460 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6789/0", 2024-09-10T21:34:20.460 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:34:20.460 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:34:20.460 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:34:20.460 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:34:20.460 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:34:20.460 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:34:20.460 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:34:20.461 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:34:20.461 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:34:20.461 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:34:20.461 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:34:20.461 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3301", 2024-09-10T21:34:20.461 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:34:20.461 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:34:20.461 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:34:20.461 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:34:20.461 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790", 2024-09-10T21:34:20.461 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:34:20.461 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:34:20.461 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:34:20.462 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:34:20.462 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790/0", 2024-09-10T21:34:20.462 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6790/0", 2024-09-10T21:34:20.462 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:34:20.462 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:34:20.462 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:34:20.462 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:34:20.462 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:34:20.462 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:34:20.462 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-10T21:34:20.462 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-10T21:34:20.462 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:34:20.462 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:34:20.462 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:34:20.463 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:34:20.463 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:34:20.463 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:34:20.463 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": [ 2024-09-10T21:34:20.463 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:34:20.463 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:34:20.463 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:34:20.463 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 3 2024-09-10T21:34:20.463 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:34:20.463 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:34:20.463 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-10T21:34:20.463 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:34:20.463 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:34:20.464 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:34:20.464 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 2 2024-09-10T21:34:20.464 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:34:20.464 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:34:20.464 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-10T21:34:20.464 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:34:20.464 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:34:20.464 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:34:20.464 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 2 2024-09-10T21:34:20.464 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:34:20.464 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:34:20.464 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:34:20.464 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-10T21:34:20.465 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-10T21:34:20.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:20.444+0000 7efd9f000640 1 -- 172.21.15.17:0/595109948 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7efd740724e0 msgr2=0x7efd740749a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:20.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:20.444+0000 7efd9f000640 1 --2- 172.21.15.17:0/595109948 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7efd740724e0 0x7efd740749a0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7efd980c8690 tx=0x7efd90009500 comp rx=0 tx=0).stop 2024-09-10T21:34:20.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:20.444+0000 7efd9f000640 1 -- 172.21.15.17:0/595109948 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efd981e3600 msgr2=0x7efd98152f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:20.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:20.444+0000 7efd9f000640 1 --2- 172.21.15.17:0/595109948 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efd981e3600 0x7efd98152f90 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7efd80009910 tx=0x7efd80009640 comp rx=0 tx=0).stop 2024-09-10T21:34:20.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:20.444+0000 7efd9d000640 1 -- 172.21.15.17:0/595109948 reap_dead start 2024-09-10T21:34:20.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:20.444+0000 7efd9d000640 1 -- 172.21.15.17:0/595109948 reap_dead start 2024-09-10T21:34:20.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:20.444+0000 7efd9f000640 1 -- 172.21.15.17:0/595109948 shutdown_connections 2024-09-10T21:34:20.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:20.444+0000 7efd9f000640 1 -- 172.21.15.17:0/595109948 >> 172.21.15.17:0/595109948 conn(0x7efd980702f0 msgr2=0x7efd98072970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:20.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:20.444+0000 7efd9f000640 1 -- 172.21.15.17:0/595109948 shutdown_connections 2024-09-10T21:34:20.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:20.444+0000 7efd9f000640 1 -- 172.21.15.17:0/595109948 wait complete. 2024-09-10T21:34:20.471 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-09-10T21:34:20.471 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2024-09-10T21:34:40.472 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.b 2024-09-10T21:34:40.473 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-10T21:34:40.473 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-10T21:34:40.477 INFO:tasks.ceph.mon.b:Started 2024-09-10T21:34:40.477 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-09-10T21:34:40.478 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-10T21:34:40.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:40.644+0000 7f281f000640 1 -- 172.21.15.17:0/3962878203 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f2818158c20 msgr2=0x7f2818070670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:40.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:40.644+0000 7f281f000640 1 --2- 172.21.15.17:0/3962878203 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f2818158c20 0x7f2818070670 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f2804009840 tx=0x7f280401f010 comp rx=0 tx=0).stop 2024-09-10T21:34:40.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:40.644+0000 7f281f000640 1 -- 172.21.15.17:0/3962878203 shutdown_connections 2024-09-10T21:34:40.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:40.644+0000 7f281f000640 1 --2- 172.21.15.17:0/3962878203 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f2818158c20 0x7f2818070670 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:40.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:40.644+0000 7f281f000640 1 --2- 172.21.15.17:0/3962878203 >> v2:172.21.15.31:3300/0 conn(0x7f28181524d0 0x7f281815bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:40.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:40.644+0000 7f281f000640 1 -- 172.21.15.17:0/3962878203 >> 172.21.15.17:0/3962878203 conn(0x7f28180cdef0 msgr2=0x7f28180ce2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:40.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:40.644+0000 7f281f000640 1 -- 172.21.15.17:0/3962878203 shutdown_connections 2024-09-10T21:34:40.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:40.644+0000 7f281f000640 1 -- 172.21.15.17:0/3962878203 wait complete. 2024-09-10T21:34:40.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:40.644+0000 7f281f000640 1 Processor -- start 2024-09-10T21:34:40.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:40.644+0000 7f281f000640 1 -- start start 2024-09-10T21:34:40.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:40.644+0000 7f281f000640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f28181524d0 0x7f281806f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:40.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:40.644+0000 7f281f000640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2818158c20 0x7f28180c3210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:40.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:40.644+0000 7f281f000640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f281815c440 0x7f28181f42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:40.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:40.644+0000 7f281f000640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f281815b9e0 con 0x7f2818158c20 2024-09-10T21:34:40.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:40.644+0000 7f281f000640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f28180c9350 con 0x7f28181524d0 2024-09-10T21:34:40.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:40.644+0000 7f281f000640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f28180cea50 con 0x7f281815c440 2024-09-10T21:34:40.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:40.648+0000 7f281d200640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2818158c20 0x7f28180c3210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:40.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:40.648+0000 7f281d200640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2818158c20 0x7f28180c3210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:51072/0 (socket says 172.21.15.17:51072) 2024-09-10T21:34:40.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:40.648+0000 7f281d200640 1 -- 172.21.15.17:0/3436729865 learned_addr learned my addr 172.21.15.17:0/3436729865 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:34:40.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:40.648+0000 7f281dc00640 1 --2- 172.21.15.17:0/3436729865 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f28181524d0 0x7f281806f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:40.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:40.648+0000 7f281e600640 1 --2- 172.21.15.17:0/3436729865 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f281815c440 0x7f28181f42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:40.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:40.648+0000 7f281dc00640 1 -- 172.21.15.17:0/3436729865 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f28181524d0 msgr2=0x7f281806f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-09-10T21:34:40.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:40.648+0000 7f281dc00640 1 -- 172.21.15.17:0/3436729865 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f28181524d0 msgr2=0x7f281806f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-10T21:34:40.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:40.648+0000 7f281dc00640 1 --2- 172.21.15.17:0/3436729865 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f28181524d0 0x7f281806f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T21:34:40.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:40.648+0000 7f281d200640 1 -- 172.21.15.17:0/3436729865 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f281815c440 msgr2=0x7f28181f42d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:40.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:40.648+0000 7f281dc00640 1 --2- 172.21.15.17:0/3436729865 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f28181524d0 0x7f281806f750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:34:40.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:40.648+0000 7f281d200640 1 --2- 172.21.15.17:0/3436729865 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f281815c440 0x7f28181f42d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:40.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:40.648+0000 7f281d200640 1 -- 172.21.15.17:0/3436729865 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f28181524d0 msgr2=0x7f281806f750 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:34:40.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:40.648+0000 7f281d200640 1 --2- 172.21.15.17:0/3436729865 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f28181524d0 0x7f281806f750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:40.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:40.648+0000 7f281d200640 1 -- 172.21.15.17:0/3436729865 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28181594e0 con 0x7f2818158c20 2024-09-10T21:34:40.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:40.648+0000 7f281d200640 1 --2- 172.21.15.17:0/3436729865 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2818158c20 0x7f28180c3210 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f2804009790 tx=0x7f28040a8ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:40.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:40.648+0000 7f2815600640 1 -- 172.21.15.17:0/3436729865 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28040be020 con 0x7f2818158c20 2024-09-10T21:34:40.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:40.648+0000 7f2815600640 1 -- 172.21.15.17:0/3436729865 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2804005220 con 0x7f2818158c20 2024-09-10T21:34:40.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:40.648+0000 7f2815600640 1 -- 172.21.15.17:0/3436729865 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28040b5040 con 0x7f2818158c20 2024-09-10T21:34:40.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:40.648+0000 7f281f000640 1 -- 172.21.15.17:0/3436729865 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28181597c0 con 0x7f2818158c20 2024-09-10T21:34:40.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:40.648+0000 7f281f000640 1 -- 172.21.15.17:0/3436729865 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28181f3ad0 con 0x7f2818158c20 2024-09-10T21:34:40.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:40.648+0000 7f281f000640 1 -- 172.21.15.17:0/3436729865 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28180b3e00 con 0x7f2818158c20 2024-09-10T21:34:40.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:40.648+0000 7f2815600640 1 -- 172.21.15.17:0/3436729865 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f28040b8050 con 0x7f2818158c20 2024-09-10T21:34:40.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:40.652+0000 7f2815600640 1 --2- 172.21.15.17:0/3436729865 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f27f80725e0 0x7f27f8074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:40.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:40.652+0000 7f2815600640 1 -- 172.21.15.17:0/3436729865 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f2804130510 con 0x7f2818158c20 2024-09-10T21:34:40.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:40.652+0000 7f281dc00640 1 --2- 172.21.15.17:0/3436729865 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f27f80725e0 0x7f27f8074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:40.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:40.652+0000 7f281dc00640 1 --2- 172.21.15.17:0/3436729865 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f27f80725e0 0x7f27f8074aa0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f280c005fd0 tx=0x7f280c005860 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:40.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:40.656+0000 7f2815600640 1 -- 172.21.15.17:0/3436729865 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f2804018b70 con 0x7f2818158c20 2024-09-10T21:34:40.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:40.916+0000 7f281f000640 1 -- 172.21.15.17:0/3436729865 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f28180b39d0 con 0x7f2818158c20 2024-09-10T21:34:40.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:40.916+0000 7f2815600640 1 -- 172.21.15.17:0/3436729865 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1551 (secure 0 0 0) 0x7f28040fb110 con 0x7f2818158c20 2024-09-10T21:34:40.923 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":84,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"a","quorum_age":21,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5e2f023f-d6a6-49f4-a256-f4807dce3739","modified":"2024-09-10T21:23:46.892130Z","created":"2024-09-10T21:23:46.892130Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3301","nonce":0},{"type":"v1","addr":"172.21.15.31:6790","nonce":0}]},"addr":"172.21.15.31:6790/0","public_addr":"172.21.15.31:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-10T21:34:40.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:40.920+0000 7f281f000640 1 -- 172.21.15.17:0/3436729865 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f27f80725e0 msgr2=0x7f27f8074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:40.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:40.920+0000 7f281f000640 1 --2- 172.21.15.17:0/3436729865 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f27f80725e0 0x7f27f8074aa0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f280c005fd0 tx=0x7f280c005860 comp rx=0 tx=0).stop 2024-09-10T21:34:40.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:40.920+0000 7f281f000640 1 -- 172.21.15.17:0/3436729865 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2818158c20 msgr2=0x7f28180c3210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:40.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:40.920+0000 7f281f000640 1 --2- 172.21.15.17:0/3436729865 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2818158c20 0x7f28180c3210 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f2804009790 tx=0x7f28040a8ca0 comp rx=0 tx=0).stop 2024-09-10T21:34:40.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:40.920+0000 7f281f000640 1 -- 172.21.15.17:0/3436729865 shutdown_connections 2024-09-10T21:34:40.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:40.920+0000 7f281f000640 1 --2- 172.21.15.17:0/3436729865 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f27f80725e0 0x7f27f8074aa0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:40.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:40.920+0000 7f281f000640 1 --2- 172.21.15.17:0/3436729865 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f281815c440 0x7f28181f42d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:40.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:40.920+0000 7f281f000640 1 --2- 172.21.15.17:0/3436729865 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2818158c20 0x7f28180c3210 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:40.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:40.920+0000 7f281f000640 1 --2- 172.21.15.17:0/3436729865 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f28181524d0 0x7f281806f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:40.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:40.920+0000 7f281f000640 1 -- 172.21.15.17:0/3436729865 >> 172.21.15.17:0/3436729865 conn(0x7f28180cdef0 msgr2=0x7f2818156ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:40.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:40.920+0000 7f281f000640 1 -- 172.21.15.17:0/3436729865 shutdown_connections 2024-09-10T21:34:40.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:40.920+0000 7f281f000640 1 -- 172.21.15.17:0/3436729865 wait complete. 2024-09-10T21:34:43.947 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-10T21:34:44.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:44.108+0000 7fc90ea00640 1 -- 172.21.15.17:0/1243148554 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc9080bedb0 msgr2=0x7fc9080c8100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:44.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:44.108+0000 7fc90ea00640 1 --2- 172.21.15.17:0/1243148554 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc9080bedb0 0x7fc9080c8100 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc8fc00aa10 tx=0x7fc8fc01dc30 comp rx=0 tx=0).stop 2024-09-10T21:34:44.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:44.108+0000 7fc90ea00640 1 -- 172.21.15.17:0/1243148554 shutdown_connections 2024-09-10T21:34:44.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:44.108+0000 7fc90ea00640 1 --2- 172.21.15.17:0/1243148554 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc908157880 0x7fc9080c6490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:44.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:44.108+0000 7fc90ea00640 1 --2- 172.21.15.17:0/1243148554 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc9080bedb0 0x7fc9080c8100 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:44.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:44.108+0000 7fc90ea00640 1 -- 172.21.15.17:0/1243148554 >> 172.21.15.17:0/1243148554 conn(0x7fc9080cbbb0 msgr2=0x7fc9080cbfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:44.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:44.108+0000 7fc90ea00640 1 -- 172.21.15.17:0/1243148554 shutdown_connections 2024-09-10T21:34:44.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:44.108+0000 7fc90ea00640 1 -- 172.21.15.17:0/1243148554 wait complete. 2024-09-10T21:34:44.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:44.112+0000 7fc90ea00640 1 Processor -- start 2024-09-10T21:34:44.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:44.112+0000 7fc90ea00640 1 -- start start 2024-09-10T21:34:44.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:44.112+0000 7fc90ea00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc908072b50 0x7fc9081f7010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:44.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:44.112+0000 7fc90ea00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc9080bedb0 0x7fc9080ca390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:44.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:44.112+0000 7fc90ea00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc908157880 0x7fc9080ca8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:44.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:44.112+0000 7fc90ea00640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fc9080bf190 con 0x7fc9080bedb0 2024-09-10T21:34:44.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:44.112+0000 7fc90ea00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fc9080c69d0 con 0x7fc908157880 2024-09-10T21:34:44.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:44.112+0000 7fc90ea00640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7fc9080bf310 con 0x7fc908072b50 2024-09-10T21:34:44.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:44.112+0000 7fc90cc00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc9080bedb0 0x7fc9080ca390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:44.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:44.112+0000 7fc90cc00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc9080bedb0 0x7fc9080ca390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:51086/0 (socket says 172.21.15.17:51086) 2024-09-10T21:34:44.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:44.112+0000 7fc90cc00640 1 -- 172.21.15.17:0/1940107707 learned_addr learned my addr 172.21.15.17:0/1940107707 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:34:44.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:44.112+0000 7fc90d600640 1 --2- 172.21.15.17:0/1940107707 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc908072b50 0x7fc9081f7010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:44.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:44.112+0000 7fc90e000640 1 --2- 172.21.15.17:0/1940107707 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc908157880 0x7fc9080ca8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:44.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:44.112+0000 7fc90e000640 1 -- 172.21.15.17:0/1940107707 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc908157880 msgr2=0x7fc9080ca8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-09-10T21:34:44.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:44.112+0000 7fc90e000640 1 -- 172.21.15.17:0/1940107707 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc908157880 msgr2=0x7fc9080ca8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-10T21:34:44.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:44.112+0000 7fc90e000640 1 --2- 172.21.15.17:0/1940107707 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc908157880 0x7fc9080ca8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T21:34:44.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:44.112+0000 7fc90e000640 1 --2- 172.21.15.17:0/1940107707 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc908157880 0x7fc9080ca8d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:34:44.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:44.112+0000 7fc90cc00640 1 -- 172.21.15.17:0/1940107707 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc908072b50 msgr2=0x7fc9081f7010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:44.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:44.112+0000 7fc90cc00640 1 --2- 172.21.15.17:0/1940107707 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc908072b50 0x7fc9081f7010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:44.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:44.112+0000 7fc90cc00640 1 -- 172.21.15.17:0/1940107707 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc908157880 msgr2=0x7fc9080ca8d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:34:44.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:44.112+0000 7fc90cc00640 1 --2- 172.21.15.17:0/1940107707 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc908157880 0x7fc9080ca8d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:44.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:44.112+0000 7fc90cc00640 1 -- 172.21.15.17:0/1940107707 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9080cb590 con 0x7fc9080bedb0 2024-09-10T21:34:44.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:44.112+0000 7fc90cc00640 1 --2- 172.21.15.17:0/1940107707 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc9080bedb0 0x7fc9080ca390 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fc8f400d780 tx=0x7fc8f400dc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:44.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:44.112+0000 7fc905600640 1 -- 172.21.15.17:0/1940107707 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8f4017070 con 0x7fc9080bedb0 2024-09-10T21:34:44.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:44.112+0000 7fc905600640 1 -- 172.21.15.17:0/1940107707 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc8f400de90 con 0x7fc9080bedb0 2024-09-10T21:34:44.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:44.112+0000 7fc90ea00640 1 -- 172.21.15.17:0/1940107707 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9081f7a70 con 0x7fc9080bedb0 2024-09-10T21:34:44.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:44.112+0000 7fc905600640 1 -- 172.21.15.17:0/1940107707 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8f40099d0 con 0x7fc9080bedb0 2024-09-10T21:34:44.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:44.112+0000 7fc90ea00640 1 -- 172.21.15.17:0/1940107707 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc90814b290 con 0x7fc9080bedb0 2024-09-10T21:34:44.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:44.116+0000 7fc90ea00640 1 -- 172.21.15.17:0/1940107707 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9081f7cb0 con 0x7fc9080bedb0 2024-09-10T21:34:44.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:44.116+0000 7fc905600640 1 -- 172.21.15.17:0/1940107707 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fc8f4009b70 con 0x7fc9080bedb0 2024-09-10T21:34:44.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:44.116+0000 7fc905600640 1 --2- 172.21.15.17:0/1940107707 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fc8e8072610 0x7fc8e8074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:44.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:44.116+0000 7fc905600640 1 -- 172.21.15.17:0/1940107707 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fc8f4093920 con 0x7fc9080bedb0 2024-09-10T21:34:44.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:44.116+0000 7fc90d600640 1 --2- 172.21.15.17:0/1940107707 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fc8e8072610 0x7fc8e8074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:44.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:44.116+0000 7fc90d600640 1 --2- 172.21.15.17:0/1940107707 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fc8e8072610 0x7fc8e8074ad0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fc8fc0b0000 tx=0x7fc8fc01c470 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:44.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:44.120+0000 7fc905600640 1 -- 172.21.15.17:0/1940107707 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fc8f4020020 con 0x7fc9080bedb0 2024-09-10T21:34:44.396 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:44.392+0000 7fc90ea00640 1 -- 172.21.15.17:0/1940107707 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fc908070cf0 con 0x7fc9080bedb0 2024-09-10T21:34:44.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:44.392+0000 7fc905600640 1 -- 172.21.15.17:0/1940107707 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1551 (secure 0 0 0) 0x7fc8f405ebf0 con 0x7fc9080bedb0 2024-09-10T21:34:44.397 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":84,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"a","quorum_age":24,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5e2f023f-d6a6-49f4-a256-f4807dce3739","modified":"2024-09-10T21:23:46.892130Z","created":"2024-09-10T21:23:46.892130Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3301","nonce":0},{"type":"v1","addr":"172.21.15.31:6790","nonce":0}]},"addr":"172.21.15.31:6790/0","public_addr":"172.21.15.31:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-10T21:34:44.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:44.392+0000 7fc90ea00640 1 -- 172.21.15.17:0/1940107707 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fc8e8072610 msgr2=0x7fc8e8074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:44.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:44.392+0000 7fc90ea00640 1 --2- 172.21.15.17:0/1940107707 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fc8e8072610 0x7fc8e8074ad0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fc8fc0b0000 tx=0x7fc8fc01c470 comp rx=0 tx=0).stop 2024-09-10T21:34:44.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:44.392+0000 7fc90ea00640 1 -- 172.21.15.17:0/1940107707 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc9080bedb0 msgr2=0x7fc9080ca390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:44.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:44.392+0000 7fc90ea00640 1 --2- 172.21.15.17:0/1940107707 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc9080bedb0 0x7fc9080ca390 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fc8f400d780 tx=0x7fc8f400dc40 comp rx=0 tx=0).stop 2024-09-10T21:34:44.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:44.392+0000 7fc90ea00640 1 -- 172.21.15.17:0/1940107707 shutdown_connections 2024-09-10T21:34:44.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:44.392+0000 7fc90ea00640 1 --2- 172.21.15.17:0/1940107707 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fc8e8072610 0x7fc8e8074ad0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:44.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:44.392+0000 7fc90ea00640 1 --2- 172.21.15.17:0/1940107707 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc908157880 0x7fc9080ca8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:44.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:44.392+0000 7fc90ea00640 1 --2- 172.21.15.17:0/1940107707 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc9080bedb0 0x7fc9080ca390 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:44.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:44.392+0000 7fc90ea00640 1 --2- 172.21.15.17:0/1940107707 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc908072b50 0x7fc9081f7010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:44.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:44.392+0000 7fc90ea00640 1 -- 172.21.15.17:0/1940107707 >> 172.21.15.17:0/1940107707 conn(0x7fc9080cbbb0 msgr2=0x7fc9080704b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:44.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:44.392+0000 7fc90ea00640 1 -- 172.21.15.17:0/1940107707 shutdown_connections 2024-09-10T21:34:44.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:44.392+0000 7fc90ea00640 1 -- 172.21.15.17:0/1940107707 wait complete. 2024-09-10T21:34:47.420 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-10T21:34:47.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:47.588+0000 7f870cc00640 1 -- 172.21.15.17:0/2314922603 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f870815c440 msgr2=0x7f8708070d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:47.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:47.588+0000 7f870cc00640 1 --2- 172.21.15.17:0/2314922603 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f870815c440 0x7f8708070d80 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f870806b240 tx=0x7f870001f580 comp rx=0 tx=0).stop 2024-09-10T21:34:47.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:47.588+0000 7f870cc00640 1 -- 172.21.15.17:0/2314922603 shutdown_connections 2024-09-10T21:34:47.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:47.588+0000 7f870cc00640 1 --2- 172.21.15.17:0/2314922603 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f870815c440 0x7f8708070d80 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:47.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:47.588+0000 7f870cc00640 1 --2- 172.21.15.17:0/2314922603 >> v2:172.21.15.31:3300/0 conn(0x7f87081524d0 0x7f870815bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:47.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:47.588+0000 7f870cc00640 1 -- 172.21.15.17:0/2314922603 >> 172.21.15.17:0/2314922603 conn(0x7f87080cdef0 msgr2=0x7f87080ce2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:47.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:47.588+0000 7f870cc00640 1 -- 172.21.15.17:0/2314922603 shutdown_connections 2024-09-10T21:34:47.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:47.588+0000 7f870cc00640 1 -- 172.21.15.17:0/2314922603 wait complete. 2024-09-10T21:34:47.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:47.588+0000 7f870cc00640 1 Processor -- start 2024-09-10T21:34:47.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:47.588+0000 7f870cc00640 1 -- start start 2024-09-10T21:34:47.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:47.592+0000 7f870cc00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f87081524d0 0x7f8708143060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:47.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:47.592+0000 7f870cc00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8708158c20 0x7f8708144e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:47.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:47.592+0000 7f870cc00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f870815c440 0x7f8708151450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:47.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:47.592+0000 7f870cc00640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f87080c9350 con 0x7f8708158c20 2024-09-10T21:34:47.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:47.592+0000 7f870cc00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f87080c9070 con 0x7f870815c440 2024-09-10T21:34:47.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:47.592+0000 7f870cc00640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f870815b9e0 con 0x7f87081524d0 2024-09-10T21:34:47.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:47.592+0000 7f8706a00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8708158c20 0x7f8708144e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:47.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:47.592+0000 7f8706a00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8708158c20 0x7f8708144e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:57150/0 (socket says 172.21.15.17:57150) 2024-09-10T21:34:47.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:47.592+0000 7f8706a00640 1 -- 172.21.15.17:0/4214007505 learned_addr learned my addr 172.21.15.17:0/4214007505 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:34:47.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:47.592+0000 7f8707e00640 1 --2- 172.21.15.17:0/4214007505 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f870815c440 0x7f8708151450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:47.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:47.592+0000 7f8707400640 1 --2- 172.21.15.17:0/4214007505 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f87081524d0 0x7f8708143060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:47.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:47.592+0000 7f8707e00640 1 -- 172.21.15.17:0/4214007505 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f870815c440 msgr2=0x7f8708151450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-09-10T21:34:47.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:47.592+0000 7f8707e00640 1 -- 172.21.15.17:0/4214007505 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f870815c440 msgr2=0x7f8708151450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-10T21:34:47.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:47.592+0000 7f8706a00640 1 -- 172.21.15.17:0/4214007505 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f87081524d0 msgr2=0x7f8708143060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:47.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:47.592+0000 7f8707e00640 1 --2- 172.21.15.17:0/4214007505 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f870815c440 0x7f8708151450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T21:34:47.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:47.592+0000 7f8706a00640 1 --2- 172.21.15.17:0/4214007505 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f87081524d0 0x7f8708143060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:47.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:47.592+0000 7f8706a00640 1 -- 172.21.15.17:0/4214007505 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f870815c440 msgr2=0x7f8708151450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:47.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:47.592+0000 7f8707e00640 1 --2- 172.21.15.17:0/4214007505 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f870815c440 0x7f8708151450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:34:47.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:47.592+0000 7f8706a00640 1 --2- 172.21.15.17:0/4214007505 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f870815c440 0x7f8708151450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:47.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:47.592+0000 7f8706a00640 1 -- 172.21.15.17:0/4214007505 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f870815b410 con 0x7f8708158c20 2024-09-10T21:34:47.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:47.592+0000 7f8707400640 1 --2- 172.21.15.17:0/4214007505 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f87081524d0 0x7f8708143060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:34:47.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:47.592+0000 7f8706a00640 1 --2- 172.21.15.17:0/4214007505 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8708158c20 0x7f8708144e20 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f86f400c560 tx=0x7f86f400ca20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:47.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:47.592+0000 7f86ff400640 1 -- 172.21.15.17:0/4214007505 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f4017070 con 0x7f8708158c20 2024-09-10T21:34:47.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:47.592+0000 7f86ff400640 1 -- 172.21.15.17:0/4214007505 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f86f40053c0 con 0x7f8708158c20 2024-09-10T21:34:47.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:47.592+0000 7f870cc00640 1 -- 172.21.15.17:0/4214007505 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8708159500 con 0x7f8708158c20 2024-09-10T21:34:47.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:47.592+0000 7f86ff400640 1 -- 172.21.15.17:0/4214007505 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f4012890 con 0x7f8708158c20 2024-09-10T21:34:47.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:47.592+0000 7f870cc00640 1 -- 172.21.15.17:0/4214007505 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87081eec30 con 0x7f8708158c20 2024-09-10T21:34:47.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:47.592+0000 7f870cc00640 1 -- 172.21.15.17:0/4214007505 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87081ef6c0 con 0x7f8708158c20 2024-09-10T21:34:47.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:47.596+0000 7f86ff400640 1 -- 172.21.15.17:0/4214007505 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f86f4003600 con 0x7f8708158c20 2024-09-10T21:34:47.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:47.596+0000 7f86ff400640 1 --2- 172.21.15.17:0/4214007505 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f86e80725e0 0x7f86e8074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:47.602 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:47.596+0000 7f86ff400640 1 -- 172.21.15.17:0/4214007505 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f86f4092c80 con 0x7f8708158c20 2024-09-10T21:34:47.602 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:47.596+0000 7f8707400640 1 --2- 172.21.15.17:0/4214007505 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f86e80725e0 0x7f86e8074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:47.602 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:47.596+0000 7f8707400640 1 --2- 172.21.15.17:0/4214007505 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f86e80725e0 0x7f86e8074aa0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f86f80058d0 tx=0x7f86f8005860 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:47.606 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:47.600+0000 7f86ff400640 1 -- 172.21.15.17:0/4214007505 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f87081528b0 con 0x7f8708158c20 2024-09-10T21:34:47.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:47.852+0000 7f870cc00640 1 -- 172.21.15.17:0/4214007505 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f87080b3e00 con 0x7f8708158c20 2024-09-10T21:34:47.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:47.852+0000 7f86ff400640 1 -- 172.21.15.17:0/4214007505 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1556 (secure 0 0 0) 0x7f86f405df50 con 0x7f8708158c20 2024-09-10T21:34:47.858 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":86,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":1,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5e2f023f-d6a6-49f4-a256-f4807dce3739","modified":"2024-09-10T21:23:46.892130Z","created":"2024-09-10T21:23:46.892130Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3301","nonce":0},{"type":"v1","addr":"172.21.15.31:6790","nonce":0}]},"addr":"172.21.15.31:6790/0","public_addr":"172.21.15.31:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-10T21:34:47.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:47.856+0000 7f870cc00640 1 -- 172.21.15.17:0/4214007505 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f86e80725e0 msgr2=0x7f86e8074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:47.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:47.856+0000 7f870cc00640 1 --2- 172.21.15.17:0/4214007505 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f86e80725e0 0x7f86e8074aa0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f86f80058d0 tx=0x7f86f8005860 comp rx=0 tx=0).stop 2024-09-10T21:34:47.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:47.856+0000 7f870cc00640 1 -- 172.21.15.17:0/4214007505 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8708158c20 msgr2=0x7f8708144e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:47.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:47.856+0000 7f870cc00640 1 --2- 172.21.15.17:0/4214007505 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8708158c20 0x7f8708144e20 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f86f400c560 tx=0x7f86f400ca20 comp rx=0 tx=0).stop 2024-09-10T21:34:47.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:47.856+0000 7f870cc00640 1 -- 172.21.15.17:0/4214007505 shutdown_connections 2024-09-10T21:34:47.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:47.856+0000 7f870cc00640 1 --2- 172.21.15.17:0/4214007505 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f86e80725e0 0x7f86e8074aa0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:47.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:47.856+0000 7f870cc00640 1 --2- 172.21.15.17:0/4214007505 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f870815c440 0x7f8708151450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:47.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:47.856+0000 7f870cc00640 1 --2- 172.21.15.17:0/4214007505 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8708158c20 0x7f8708144e20 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:47.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:47.856+0000 7f870cc00640 1 --2- 172.21.15.17:0/4214007505 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f87081524d0 0x7f8708143060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:47.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:47.856+0000 7f870cc00640 1 -- 172.21.15.17:0/4214007505 >> 172.21.15.17:0/4214007505 conn(0x7f87080cdef0 msgr2=0x7f8708156860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:47.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:47.856+0000 7f870cc00640 1 -- 172.21.15.17:0/4214007505 shutdown_connections 2024-09-10T21:34:47.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:47.856+0000 7f870cc00640 1 -- 172.21.15.17:0/4214007505 wait complete. 2024-09-10T21:34:47.883 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-09-10T21:34:47.883 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-09-10T21:34:47.883 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-10T21:34:48.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.044+0000 7f6487200640 1 -- 172.21.15.17:0/902534964 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6488152450 msgr2=0x7f6488158850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:48.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.044+0000 7f6487200640 1 --2- 172.21.15.17:0/902534964 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6488152450 0x7f6488158850 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f6474009870 tx=0x7f647401f050 comp rx=0 tx=0).stop 2024-09-10T21:34:48.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.048+0000 7f6487200640 1 -- 172.21.15.17:0/902534964 shutdown_connections 2024-09-10T21:34:48.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.048+0000 7f6487200640 1 --2- 172.21.15.17:0/902534964 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6488152450 0x7f6488158850 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:48.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.048+0000 7f6487200640 1 -- 172.21.15.17:0/902534964 >> 172.21.15.17:0/902534964 conn(0x7f648806cc20 msgr2=0x7f648806d030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:48.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.048+0000 7f6487200640 1 -- 172.21.15.17:0/902534964 shutdown_connections 2024-09-10T21:34:48.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.048+0000 7f6487200640 1 -- 172.21.15.17:0/902534964 wait complete. 2024-09-10T21:34:48.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.048+0000 7f6487200640 1 Processor -- start 2024-09-10T21:34:48.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.048+0000 7f6487200640 1 -- start start 2024-09-10T21:34:48.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.048+0000 7f6487200640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f64880cbd10 0x7f64881e4b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:48.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.048+0000 7f6487200640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6488152070 0x7f6488149860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:48.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.048+0000 7f6487200640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6488152450 0x7f6488149da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:48.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.048+0000 7f6487200640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f64880cecf0 con 0x7f6488152450 2024-09-10T21:34:48.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.048+0000 7f6487200640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f64880c5090 con 0x7f64880cbd10 2024-09-10T21:34:48.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.048+0000 7f6487200640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f64880cee70 con 0x7f6488152070 2024-09-10T21:34:48.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.048+0000 7f6486800640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6488152450 0x7f6488149da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:48.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.048+0000 7f6485400640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6488152070 0x7f6488149860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:48.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.048+0000 7f6485e00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f64880cbd10 0x7f64881e4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:48.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.048+0000 7f6486800640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6488152450 0x7f6488149da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:57152/0 (socket says 172.21.15.17:57152) 2024-09-10T21:34:48.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.048+0000 7f6486800640 1 -- 172.21.15.17:0/950548261 learned_addr learned my addr 172.21.15.17:0/950548261 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:34:48.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.048+0000 7f6485400640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6488152070 0x7f6488149860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3301/0 says I am v2:172.21.15.17:36260/0 (socket says 172.21.15.17:36260) 2024-09-10T21:34:48.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.052+0000 7f6485400640 1 -- 172.21.15.17:0/950548261 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f64880cbd10 msgr2=0x7f64881e4b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:48.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.052+0000 7f6485400640 1 --2- 172.21.15.17:0/950548261 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f64880cbd10 0x7f64881e4b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:48.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.052+0000 7f6485400640 1 -- 172.21.15.17:0/950548261 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6488152450 msgr2=0x7f6488149da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:48.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.052+0000 7f6485400640 1 --2- 172.21.15.17:0/950548261 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6488152450 0x7f6488149da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:48.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.052+0000 7f6485400640 1 -- 172.21.15.17:0/950548261 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64881f0900 con 0x7f6488152070 2024-09-10T21:34:48.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.052+0000 7f6486800640 1 --2- 172.21.15.17:0/950548261 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6488152450 0x7f6488149da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:34:48.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.052+0000 7f6485400640 1 --2- 172.21.15.17:0/950548261 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6488152070 0x7f6488149860 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f64740af000 tx=0x7f6474005ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:48.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.052+0000 7f647e000640 1 -- 172.21.15.17:0/950548261 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64740be070 con 0x7f6488152070 2024-09-10T21:34:48.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.052+0000 7f647e000640 1 -- 172.21.15.17:0/950548261 <== mon.2 v2:172.21.15.31:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6474005340 con 0x7f6488152070 2024-09-10T21:34:48.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.052+0000 7f647e000640 1 -- 172.21.15.17:0/950548261 <== mon.2 v2:172.21.15.31:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64740b5040 con 0x7f6488152070 2024-09-10T21:34:48.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.052+0000 7f6487200640 1 -- 172.21.15.17:0/950548261 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64881e5840 con 0x7f6488152070 2024-09-10T21:34:48.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.052+0000 7f6487200640 1 -- 172.21.15.17:0/950548261 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f648814b480 con 0x7f6488152070 2024-09-10T21:34:48.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.052+0000 7f6487200640 1 --2- 172.21.15.17:0/950548261 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f64881f0de0 0x7f64881f11c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:48.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.052+0000 7f6487200640 1 -- 172.21.15.17:0/950548261 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f64881e24b0 con 0x7f64881f0de0 2024-09-10T21:34:48.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.052+0000 7f6485e00640 1 --2- 172.21.15.17:0/950548261 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f64881f0de0 0x7f64881f11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:48.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.052+0000 7f6485e00640 1 -- 172.21.15.17:0/950548261 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6488071e10 con 0x7f64881f0de0 2024-09-10T21:34:48.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.052+0000 7f6485e00640 1 --2- 172.21.15.17:0/950548261 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f64881f0de0 0x7f64881f11c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f6478000f70 tx=0x7f647800fc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:48.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.052+0000 7f647e000640 1 -- 172.21.15.17:0/950548261 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64780050d0 con 0x7f64881f0de0 2024-09-10T21:34:48.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.052+0000 7f647e000640 1 -- 172.21.15.17:0/950548261 <== mon.2 v2:172.21.15.31:3301/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f64740b8030 con 0x7f6488152070 2024-09-10T21:34:48.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.056+0000 7f647e000640 1 --2- 172.21.15.17:0/950548261 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f6468072510 0x7f64680749d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:48.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.056+0000 7f647e000640 1 -- 172.21.15.17:0/950548261 <== mon.2 v2:172.21.15.31:3301/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f6474130430 con 0x7f6488152070 2024-09-10T21:34:48.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.056+0000 7f6486800640 1 --2- 172.21.15.17:0/950548261 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f6468072510 0x7f64680749d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:48.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.056+0000 7f647e000640 1 -- 172.21.15.17:0/950548261 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f6478005270 con 0x7f64881f0de0 2024-09-10T21:34:48.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.056+0000 7f647e000640 1 -- 172.21.15.17:0/950548261 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f64881f0de0 msgr2=0x7f64881f11c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:48.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.056+0000 7f647e000640 1 --2- 172.21.15.17:0/950548261 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f64881f0de0 0x7f64881f11c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f6478000f70 tx=0x7f647800fc00 comp rx=0 tx=0).stop 2024-09-10T21:34:48.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.056+0000 7f647e000640 1 -- 172.21.15.17:0/950548261 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f64881f0de0 msgr2=0x7f64881f11c0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:34:48.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.056+0000 7f647e000640 1 --2- 172.21.15.17:0/950548261 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f64881f0de0 0x7f64881f11c0 secure :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f6478000f70 tx=0x7f647800fc00 comp rx=0 tx=0).stop 2024-09-10T21:34:48.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.056+0000 7f6486800640 1 --2- 172.21.15.17:0/950548261 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f6468072510 0x7f64680749d0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f6480001d60 tx=0x7f6480001cf0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:48.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.064+0000 7f6487200640 1 --2- 172.21.15.17:0/950548261 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f64881f3e70 0x7f64881f4250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:48.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.064+0000 7f6487200640 1 -- 172.21.15.17:0/950548261 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f6488069ef0 con 0x7f64881f3e70 2024-09-10T21:34:48.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.064+0000 7f6485e00640 1 --2- 172.21.15.17:0/950548261 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f64881f3e70 0x7f64881f4250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:48.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.064+0000 7f6485e00640 1 -- 172.21.15.17:0/950548261 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f64880cb7e0 con 0x7f64881f3e70 2024-09-10T21:34:48.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.064+0000 7f6485e00640 1 --2- 172.21.15.17:0/950548261 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f64881f3e70 0x7f64881f4250 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f6478000f70 tx=0x7f6478001130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:48.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.064+0000 7f647e000640 1 -- 172.21.15.17:0/950548261 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6478007640 con 0x7f64881f3e70 2024-09-10T21:34:48.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.064+0000 7f647e000640 1 -- 172.21.15.17:0/950548261 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5043 (secure 0 0 0) 0x7f6478007640 con 0x7f64881f3e70 2024-09-10T21:34:48.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.064+0000 7f647e000640 1 -- 172.21.15.17:0/950548261 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f64881f3e70 msgr2=0x7f64881f4250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:48.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.064+0000 7f647e000640 1 --2- 172.21.15.17:0/950548261 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f64881f3e70 0x7f64881f4250 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f6478000f70 tx=0x7f6478001130 comp rx=0 tx=0).stop 2024-09-10T21:34:48.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.064+0000 7f647e000640 1 -- 172.21.15.17:0/950548261 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f64881f3e70 msgr2=0x7f64881f4250 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:34:48.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.064+0000 7f647e000640 1 --2- 172.21.15.17:0/950548261 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f64881f3e70 0x7f64881f4250 secure :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f6478000f70 tx=0x7f6478001130 comp rx=0 tx=0).stop 2024-09-10T21:34:48.071 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-10T21:34:48.071 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:34:48.071 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:34:48.071 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "leader", 2024-09-10T21:34:48.071 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 86, 2024-09-10T21:34:48.071 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 68671, 2024-09-10T21:34:48.071 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-10T21:34:48.071 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-10T21:34:48.071 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-10T21:34:48.071 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-10T21:34:48.072 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:34:48.072 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 2, 2024-09-10T21:34:48.072 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:34:48.072 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-10T21:34:48.072 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-10T21:34:48.072 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:34:48.072 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:34:48.072 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:34:48.072 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:34:48.072 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:34:48.072 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:34:48.072 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:34:48.072 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:34:48.073 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:34:48.073 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:34:48.073 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:34:48.073 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:34:48.073 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-10T21:34:48.073 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-10T21:34:48.073 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:34:48.073 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:34:48.073 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:34:48.073 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:34:48.073 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:34:48.073 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:34:48.073 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:34:48.073 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:34:48.074 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:34:48.074 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:34:48.074 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:34:48.074 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:34:48.074 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:34:48.074 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-10T21:34:48.074 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-10T21:34:48.074 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-10T21:34:48.074 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-10T21:34:48.074 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-10T21:34:48.074 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "5e2f023f-d6a6-49f4-a256-f4807dce3739", 2024-09-10T21:34:48.074 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:34:48.074 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:34:48.075 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-10T21:34:48.075 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-10T21:34:48.075 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-10T21:34:48.075 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-10T21:34:48.075 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-10T21:34:48.075 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-10T21:34:48.075 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-10T21:34:48.075 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:34:48.075 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-10T21:34:48.075 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:34:48.075 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:34:48.075 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:34:48.075 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:34:48.075 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:34:48.076 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:34:48.076 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:34:48.076 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:34:48.076 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:34:48.076 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:34:48.076 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:34:48.076 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:34:48.076 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-10T21:34:48.076 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:34:48.076 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-10T21:34:48.076 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:34:48.076 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:34:48.076 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:34:48.077 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:34:48.077 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:34:48.077 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:34:48.077 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:34:48.077 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-10T21:34:48.077 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:34:48.077 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:34:48.077 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:34:48.077 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:34:48.077 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-10T21:34:48.077 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:34:48.077 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:34:48.077 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:34:48.077 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:34:48.078 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-10T21:34:48.078 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-10T21:34:48.078 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:34:48.078 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:34:48.078 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:34:48.078 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:34:48.078 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:34:48.078 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:34:48.078 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:34:48.078 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:34:48.078 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:34:48.078 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:34:48.078 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:34:48.079 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3300", 2024-09-10T21:34:48.079 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:34:48.079 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:34:48.079 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:34:48.079 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:34:48.079 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789", 2024-09-10T21:34:48.079 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:34:48.079 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:34:48.079 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:34:48.079 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:34:48.079 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789/0", 2024-09-10T21:34:48.079 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6789/0", 2024-09-10T21:34:48.079 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:34:48.079 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:34:48.080 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:34:48.080 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:34:48.080 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:34:48.080 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:34:48.080 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:34:48.080 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:34:48.080 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:34:48.080 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:34:48.080 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:34:48.080 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3301", 2024-09-10T21:34:48.080 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:34:48.080 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:34:48.080 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:34:48.080 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:34:48.081 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790", 2024-09-10T21:34:48.081 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:34:48.081 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:34:48.081 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:34:48.081 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:34:48.081 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790/0", 2024-09-10T21:34:48.081 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6790/0", 2024-09-10T21:34:48.081 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:34:48.081 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:34:48.081 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:34:48.081 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:34:48.081 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:34:48.081 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:34:48.082 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-10T21:34:48.082 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-10T21:34:48.082 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:34:48.082 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:34:48.082 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:34:48.082 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:34:48.082 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:34:48.082 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:34:48.082 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": [ 2024-09-10T21:34:48.082 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:34:48.082 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:34:48.082 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:34:48.082 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 3 2024-09-10T21:34:48.082 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:34:48.083 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:34:48.083 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-10T21:34:48.083 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:34:48.083 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:34:48.083 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:34:48.083 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 6 2024-09-10T21:34:48.083 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:34:48.083 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:34:48.083 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-10T21:34:48.083 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:34:48.083 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:34:48.083 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:34:48.083 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 5 2024-09-10T21:34:48.083 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:34:48.084 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:34:48.084 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": [ 2024-09-10T21:34:48.084 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:34:48.084 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:34:48.084 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:34:48.084 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 2 2024-09-10T21:34:48.084 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:34:48.084 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:34:48.084 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:34:48.084 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-10T21:34:48.084 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-10T21:34:48.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.064+0000 7f6487200640 1 -- 172.21.15.17:0/950548261 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f6468072510 msgr2=0x7f64680749d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:48.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.064+0000 7f6487200640 1 --2- 172.21.15.17:0/950548261 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f6468072510 0x7f64680749d0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f6480001d60 tx=0x7f6480001cf0 comp rx=0 tx=0).stop 2024-09-10T21:34:48.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.064+0000 7f6487200640 1 -- 172.21.15.17:0/950548261 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6488152070 msgr2=0x7f6488149860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:48.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.064+0000 7f6487200640 1 --2- 172.21.15.17:0/950548261 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6488152070 0x7f6488149860 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f64740af000 tx=0x7f6474005ca0 comp rx=0 tx=0).stop 2024-09-10T21:34:48.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.064+0000 7f6486800640 1 -- 172.21.15.17:0/950548261 reap_dead start 2024-09-10T21:34:48.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.064+0000 7f6486800640 1 -- 172.21.15.17:0/950548261 reap_dead start 2024-09-10T21:34:48.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.064+0000 7f6487200640 1 -- 172.21.15.17:0/950548261 shutdown_connections 2024-09-10T21:34:48.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.064+0000 7f6487200640 1 -- 172.21.15.17:0/950548261 >> 172.21.15.17:0/950548261 conn(0x7f648806cc20 msgr2=0x7f64880747e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:48.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.064+0000 7f6487200640 1 -- 172.21.15.17:0/950548261 shutdown_connections 2024-09-10T21:34:48.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.064+0000 7f6487200640 1 -- 172.21.15.17:0/950548261 wait complete. 2024-09-10T21:34:48.091 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-10T21:34:48.092 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-09-10T21:34:48.092 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-10T21:34:48.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.268+0000 7f43a1200640 1 -- 172.21.15.17:0/1364316989 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f439c0c9520 msgr2=0x7f439c070c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:48.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.268+0000 7f43a1200640 1 --2- 172.21.15.17:0/1364316989 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f439c0c9520 0x7f439c070c10 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f4388009840 tx=0x7f438801c700 comp rx=0 tx=0).stop 2024-09-10T21:34:48.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.268+0000 7f43a1200640 1 -- 172.21.15.17:0/1364316989 shutdown_connections 2024-09-10T21:34:48.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.268+0000 7f43a1200640 1 --2- 172.21.15.17:0/1364316989 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f439c0c2f80 0x7f439c071150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:48.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.268+0000 7f43a1200640 1 --2- 172.21.15.17:0/1364316989 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f439c0c9520 0x7f439c070c10 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:48.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.268+0000 7f43a1200640 1 -- 172.21.15.17:0/1364316989 >> 172.21.15.17:0/1364316989 conn(0x7f439c068990 msgr2=0x7f439c068da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:48.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.268+0000 7f43a1200640 1 -- 172.21.15.17:0/1364316989 shutdown_connections 2024-09-10T21:34:48.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.268+0000 7f43a1200640 1 -- 172.21.15.17:0/1364316989 wait complete. 2024-09-10T21:34:48.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.268+0000 7f43a1200640 1 Processor -- start 2024-09-10T21:34:48.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.268+0000 7f43a1200640 1 -- start start 2024-09-10T21:34:48.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.272+0000 7f43a1200640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f439c0c2f80 0x7f439c14c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:48.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.272+0000 7f43a1200640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f439c0c9520 0x7f439c13b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:48.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.272+0000 7f43a1200640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f439c1523d0 0x7f439c13bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:48.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.272+0000 7f43a1200640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f439c0cb3c0 con 0x7f439c0c9520 2024-09-10T21:34:48.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.272+0000 7f43a1200640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f439c0ca670 con 0x7f439c1523d0 2024-09-10T21:34:48.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.272+0000 7f43a1200640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f439c1652c0 con 0x7f439c0c2f80 2024-09-10T21:34:48.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.272+0000 7f439aa00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f439c0c9520 0x7f439c13b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:48.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.272+0000 7f439aa00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f439c0c9520 0x7f439c13b810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:57196/0 (socket says 172.21.15.17:57196) 2024-09-10T21:34:48.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.272+0000 7f439aa00640 1 -- 172.21.15.17:0/3207158138 learned_addr learned my addr 172.21.15.17:0/3207158138 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:34:48.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.272+0000 7f439be00640 1 --2- 172.21.15.17:0/3207158138 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f439c1523d0 0x7f439c13bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:48.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.272+0000 7f439b400640 1 --2- 172.21.15.17:0/3207158138 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f439c0c2f80 0x7f439c14c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:48.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.272+0000 7f439aa00640 1 -- 172.21.15.17:0/3207158138 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f439c0c2f80 msgr2=0x7f439c14c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:48.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.272+0000 7f439aa00640 1 --2- 172.21.15.17:0/3207158138 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f439c0c2f80 0x7f439c14c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:48.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.272+0000 7f439aa00640 1 -- 172.21.15.17:0/3207158138 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f439c1523d0 msgr2=0x7f439c13bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:48.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.272+0000 7f439aa00640 1 --2- 172.21.15.17:0/3207158138 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f439c1523d0 0x7f439c13bd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:48.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.272+0000 7f439aa00640 1 -- 172.21.15.17:0/3207158138 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f439c14bc40 con 0x7f439c0c9520 2024-09-10T21:34:48.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.272+0000 7f439aa00640 1 --2- 172.21.15.17:0/3207158138 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f439c0c9520 0x7f439c13b810 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f438801a330 tx=0x7f4388005db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:48.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.272+0000 7f4393400640 1 -- 172.21.15.17:0/3207158138 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43880be020 con 0x7f439c0c9520 2024-09-10T21:34:48.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.272+0000 7f4393400640 1 -- 172.21.15.17:0/3207158138 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4388004130 con 0x7f439c0c9520 2024-09-10T21:34:48.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.272+0000 7f4393400640 1 -- 172.21.15.17:0/3207158138 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43880044e0 con 0x7f439c0c9520 2024-09-10T21:34:48.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.272+0000 7f43a1200640 1 -- 172.21.15.17:0/3207158138 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f439c14df00 con 0x7f439c0c9520 2024-09-10T21:34:48.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.272+0000 7f43a1200640 1 -- 172.21.15.17:0/3207158138 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f439c14c190 con 0x7f439c0c9520 2024-09-10T21:34:48.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.272+0000 7f43a1200640 1 --2- 172.21.15.17:0/3207158138 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f439c14e3e0 0x7f439c14a140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:48.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.272+0000 7f43a1200640 1 -- 172.21.15.17:0/3207158138 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f439c069ef0 con 0x7f439c14e3e0 2024-09-10T21:34:48.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.272+0000 7f439b400640 1 --2- 172.21.15.17:0/3207158138 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f439c14e3e0 0x7f439c14a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:48.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.276+0000 7f439b400640 1 -- 172.21.15.17:0/3207158138 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f439c0c9f60 con 0x7f439c14e3e0 2024-09-10T21:34:48.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.276+0000 7f439b400640 1 --2- 172.21.15.17:0/3207158138 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f439c14e3e0 0x7f439c14a140 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f438c00e410 tx=0x7f438c00e8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:48.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.276+0000 7f4393400640 1 -- 172.21.15.17:0/3207158138 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f438c005020 con 0x7f439c14e3e0 2024-09-10T21:34:48.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.276+0000 7f4393400640 1 -- 172.21.15.17:0/3207158138 <== mon.1 v2:172.21.15.31:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f438c0051c0 con 0x7f439c14e3e0 2024-09-10T21:34:48.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.276+0000 7f4393400640 1 -- 172.21.15.17:0/3207158138 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f439c14e3e0 msgr2=0x7f439c14a140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:48.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.276+0000 7f4393400640 1 --2- 172.21.15.17:0/3207158138 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f439c14e3e0 0x7f439c14a140 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f438c00e410 tx=0x7f438c00e8d0 comp rx=0 tx=0).stop 2024-09-10T21:34:48.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.276+0000 7f4393400640 1 -- 172.21.15.17:0/3207158138 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f439c14e3e0 msgr2=0x7f439c14a140 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:34:48.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.276+0000 7f4393400640 1 --2- 172.21.15.17:0/3207158138 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f439c14e3e0 0x7f439c14a140 secure :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f438c00e410 tx=0x7f438c00e8d0 comp rx=0 tx=0).stop 2024-09-10T21:34:48.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.276+0000 7f4393400640 1 -- 172.21.15.17:0/3207158138 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f4388003600 con 0x7f439c0c9520 2024-09-10T21:34:48.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.276+0000 7f4393400640 1 --2- 172.21.15.17:0/3207158138 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f437c072290 0x7f437c074750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:48.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.276+0000 7f4393400640 1 -- 172.21.15.17:0/3207158138 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f438812e520 con 0x7f439c0c9520 2024-09-10T21:34:48.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.276+0000 7f439b400640 1 --2- 172.21.15.17:0/3207158138 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f437c072290 0x7f437c074750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:48.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.280+0000 7f439b400640 1 --2- 172.21.15.17:0/3207158138 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f437c072290 0x7f437c074750 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f438c00e410 tx=0x7f438c014000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:48.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.288+0000 7f43a1200640 1 --2- 172.21.15.17:0/3207158138 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f439c1519f0 0x7f439c071eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:48.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.288+0000 7f43a1200640 1 -- 172.21.15.17:0/3207158138 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f439c14e100 con 0x7f439c1519f0 2024-09-10T21:34:48.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.288+0000 7f439be00640 1 --2- 172.21.15.17:0/3207158138 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f439c1519f0 0x7f439c071eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:48.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.288+0000 7f439be00640 1 -- 172.21.15.17:0/3207158138 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f439c155b40 con 0x7f439c1519f0 2024-09-10T21:34:48.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.288+0000 7f439be00640 1 --2- 172.21.15.17:0/3207158138 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f439c1519f0 0x7f439c071eb0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f4394009fd0 tx=0x7f439400e720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:48.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.288+0000 7f4393400640 1 -- 172.21.15.17:0/3207158138 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4394017070 con 0x7f439c1519f0 2024-09-10T21:34:48.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.288+0000 7f4393400640 1 -- 172.21.15.17:0/3207158138 <== mon.1 v2:172.21.15.31:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4539 (secure 0 0 0) 0x7f4394003b80 con 0x7f439c1519f0 2024-09-10T21:34:48.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.288+0000 7f4393400640 1 -- 172.21.15.17:0/3207158138 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f439c1519f0 msgr2=0x7f439c071eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:48.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.288+0000 7f4393400640 1 --2- 172.21.15.17:0/3207158138 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f439c1519f0 0x7f439c071eb0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f4394009fd0 tx=0x7f439400e720 comp rx=0 tx=0).stop 2024-09-10T21:34:48.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.288+0000 7f4393400640 1 -- 172.21.15.17:0/3207158138 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f439c1519f0 msgr2=0x7f439c071eb0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:34:48.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.288+0000 7f4393400640 1 --2- 172.21.15.17:0/3207158138 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f439c1519f0 0x7f439c071eb0 secure :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f4394009fd0 tx=0x7f439400e720 comp rx=0 tx=0).stop 2024-09-10T21:34:48.296 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-10T21:34:48.296 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:34:48.296 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:34:48.296 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-10T21:34:48.296 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 86, 2024-09-10T21:34:48.296 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 7675, 2024-09-10T21:34:48.297 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-10T21:34:48.297 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-10T21:34:48.297 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-10T21:34:48.297 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-10T21:34:48.297 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:34:48.297 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 2, 2024-09-10T21:34:48.297 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:34:48.297 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-10T21:34:48.297 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-10T21:34:48.297 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:34:48.297 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:34:48.297 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:34:48.297 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:34:48.298 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:34:48.298 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:34:48.298 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:34:48.298 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:34:48.298 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:34:48.298 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:34:48.298 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:34:48.298 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:34:48.298 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-10T21:34:48.298 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-10T21:34:48.298 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:34:48.298 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:34:48.298 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:34:48.299 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:34:48.299 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:34:48.299 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:34:48.299 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:34:48.299 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:34:48.299 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:34:48.299 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:34:48.299 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:34:48.299 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:34:48.299 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:34:48.299 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-10T21:34:48.299 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-10T21:34:48.299 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-10T21:34:48.299 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-10T21:34:48.300 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-10T21:34:48.300 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "5e2f023f-d6a6-49f4-a256-f4807dce3739", 2024-09-10T21:34:48.300 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:34:48.300 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:34:48.300 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-10T21:34:48.300 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-10T21:34:48.300 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-10T21:34:48.300 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-10T21:34:48.300 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-10T21:34:48.300 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-10T21:34:48.300 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-10T21:34:48.300 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:34:48.300 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-10T21:34:48.301 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:34:48.301 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:34:48.301 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:34:48.301 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:34:48.301 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:34:48.301 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:34:48.301 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:34:48.301 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:34:48.301 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:34:48.301 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:34:48.301 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:34:48.301 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:34:48.301 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-10T21:34:48.301 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:34:48.302 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-10T21:34:48.302 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:34:48.302 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:34:48.302 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:34:48.302 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:34:48.302 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:34:48.302 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:34:48.302 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:34:48.302 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-10T21:34:48.302 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:34:48.302 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:34:48.302 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:34:48.302 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:34:48.302 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-10T21:34:48.303 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:34:48.303 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:34:48.303 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:34:48.303 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:34:48.303 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-10T21:34:48.303 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-10T21:34:48.303 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:34:48.303 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:34:48.303 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:34:48.303 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:34:48.303 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:34:48.303 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:34:48.303 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:34:48.303 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:34:48.304 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:34:48.304 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:34:48.304 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:34:48.304 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3300", 2024-09-10T21:34:48.304 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:34:48.304 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:34:48.304 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:34:48.304 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:34:48.304 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789", 2024-09-10T21:34:48.304 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:34:48.304 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:34:48.304 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:34:48.304 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:34:48.304 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789/0", 2024-09-10T21:34:48.305 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6789/0", 2024-09-10T21:34:48.305 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:34:48.305 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:34:48.305 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:34:48.305 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:34:48.305 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:34:48.305 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:34:48.305 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:34:48.305 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:34:48.305 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:34:48.305 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:34:48.305 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:34:48.305 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3301", 2024-09-10T21:34:48.305 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:34:48.306 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:34:48.306 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:34:48.306 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:34:48.306 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790", 2024-09-10T21:34:48.306 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:34:48.306 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:34:48.306 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:34:48.306 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:34:48.306 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790/0", 2024-09-10T21:34:48.306 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6790/0", 2024-09-10T21:34:48.306 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:34:48.306 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:34:48.306 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:34:48.307 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:34:48.307 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:34:48.307 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:34:48.307 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-10T21:34:48.307 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-10T21:34:48.307 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:34:48.307 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:34:48.307 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:34:48.307 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:34:48.307 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:34:48.307 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:34:48.307 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-10T21:34:48.307 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:34:48.307 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:34:48.308 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:34:48.308 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:34:48.308 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:34:48.308 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:34:48.308 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:34:48.308 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-10T21:34:48.308 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-10T21:34:48.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.292+0000 7f43a1200640 1 -- 172.21.15.17:0/3207158138 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f437c072290 msgr2=0x7f437c074750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:48.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.292+0000 7f43a1200640 1 --2- 172.21.15.17:0/3207158138 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f437c072290 0x7f437c074750 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f438c00e410 tx=0x7f438c014000 comp rx=0 tx=0).stop 2024-09-10T21:34:48.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.292+0000 7f43a1200640 1 -- 172.21.15.17:0/3207158138 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f439c0c9520 msgr2=0x7f439c13b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:48.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.292+0000 7f43a1200640 1 --2- 172.21.15.17:0/3207158138 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f439c0c9520 0x7f439c13b810 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f438801a330 tx=0x7f4388005db0 comp rx=0 tx=0).stop 2024-09-10T21:34:48.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.292+0000 7f439be00640 1 -- 172.21.15.17:0/3207158138 reap_dead start 2024-09-10T21:34:48.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.292+0000 7f43a1200640 1 -- 172.21.15.17:0/3207158138 shutdown_connections 2024-09-10T21:34:48.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.292+0000 7f43a1200640 1 --2- 172.21.15.17:0/3207158138 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f439c0c9520 0x7f439c13b810 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:48.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.292+0000 7f43a1200640 1 -- 172.21.15.17:0/3207158138 >> 172.21.15.17:0/3207158138 conn(0x7f439c068990 msgr2=0x7f439c1496d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:48.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.292+0000 7f43a1200640 1 -- 172.21.15.17:0/3207158138 shutdown_connections 2024-09-10T21:34:48.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.292+0000 7f43a1200640 1 -- 172.21.15.17:0/3207158138 wait complete. 2024-09-10T21:34:48.315 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-10T21:34:48.315 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-09-10T21:34:48.315 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-10T21:34:48.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.488+0000 7f7731400640 1 -- 172.21.15.17:0/1399763924 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f772c074f70 msgr2=0x7f772c0c7130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:48.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.488+0000 7f7731400640 1 --2- 172.21.15.17:0/1399763924 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f772c074f70 0x7f772c0c7130 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f772c071120 tx=0x7f771c01f0b0 comp rx=0 tx=0).stop 2024-09-10T21:34:48.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.488+0000 7f7731400640 1 -- 172.21.15.17:0/1399763924 shutdown_connections 2024-09-10T21:34:48.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.488+0000 7f7731400640 1 --2- 172.21.15.17:0/1399763924 >> v2:172.21.15.31:3300/0 conn(0x7f772c06ed90 0x7f772c15bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:48.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.488+0000 7f7731400640 1 --2- 172.21.15.17:0/1399763924 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f772c070450 0x7f772c06e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:48.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.488+0000 7f7731400640 1 --2- 172.21.15.17:0/1399763924 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f772c074f70 0x7f772c0c7130 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:48.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.488+0000 7f7731400640 1 -- 172.21.15.17:0/1399763924 >> 172.21.15.17:0/1399763924 conn(0x7f772c06dd30 msgr2=0x7f772c06e140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:48.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.488+0000 7f7731400640 1 -- 172.21.15.17:0/1399763924 shutdown_connections 2024-09-10T21:34:48.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.488+0000 7f7731400640 1 -- 172.21.15.17:0/1399763924 wait complete. 2024-09-10T21:34:48.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.488+0000 7f7731400640 1 Processor -- start 2024-09-10T21:34:48.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.488+0000 7f7731400640 1 -- start start 2024-09-10T21:34:48.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.492+0000 7f7731400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f772c06ed90 0x7f772c1e40f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:48.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.492+0000 7f7731400640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f772c070450 0x7f772c1f40b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:48.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.492+0000 7f7731400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f772c074f70 0x7f772c1f45f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:48.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.492+0000 7f7731400640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f772c1592e0 con 0x7f772c074f70 2024-09-10T21:34:48.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.492+0000 7f7731400640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f772c159000 con 0x7f772c06ed90 2024-09-10T21:34:48.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.492+0000 7f7731400640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f772c069b90 con 0x7f772c070450 2024-09-10T21:34:48.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.492+0000 7f772b200640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f772c074f70 0x7f772c1f45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:48.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.492+0000 7f772b200640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f772c074f70 0x7f772c1f45f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:57208/0 (socket says 172.21.15.17:57208) 2024-09-10T21:34:48.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.492+0000 7f772b200640 1 -- 172.21.15.17:0/51093921 learned_addr learned my addr 172.21.15.17:0/51093921 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:34:48.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.492+0000 7f7729e00640 1 --2- 172.21.15.17:0/51093921 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f772c070450 0x7f772c1f40b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:48.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.492+0000 7f772a800640 1 --2- 172.21.15.17:0/51093921 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f772c06ed90 0x7f772c1e40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:48.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.492+0000 7f772b200640 1 -- 172.21.15.17:0/51093921 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f772c070450 msgr2=0x7f772c1f40b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:48.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.492+0000 7f772b200640 1 --2- 172.21.15.17:0/51093921 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f772c070450 0x7f772c1f40b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:48.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.492+0000 7f772b200640 1 -- 172.21.15.17:0/51093921 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f772c06ed90 msgr2=0x7f772c1e40f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:48.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.492+0000 7f772b200640 1 --2- 172.21.15.17:0/51093921 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f772c06ed90 0x7f772c1e40f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:48.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.492+0000 7f772b200640 1 -- 172.21.15.17:0/51093921 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f772c159820 con 0x7f772c074f70 2024-09-10T21:34:48.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.492+0000 7f772b200640 1 --2- 172.21.15.17:0/51093921 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f772c074f70 0x7f772c1f45f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f772400ea50 tx=0x7f772400ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:48.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.492+0000 7f7722a00640 1 -- 172.21.15.17:0/51093921 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f772400cdc0 con 0x7f772c074f70 2024-09-10T21:34:48.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.492+0000 7f7722a00640 1 -- 172.21.15.17:0/51093921 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7724004c40 con 0x7f772c074f70 2024-09-10T21:34:48.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.492+0000 7f7731400640 1 -- 172.21.15.17:0/51093921 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f772c159b00 con 0x7f772c074f70 2024-09-10T21:34:48.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.492+0000 7f7722a00640 1 -- 172.21.15.17:0/51093921 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7724005280 con 0x7f772c074f70 2024-09-10T21:34:48.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.492+0000 7f7731400640 1 -- 172.21.15.17:0/51093921 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f772c15a0c0 con 0x7f772c074f70 2024-09-10T21:34:48.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.492+0000 7f7731400640 1 --2- 172.21.15.17:0/51093921 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f772c1552b0 0x7f772c155690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:48.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.492+0000 7f7731400640 1 -- 172.21.15.17:0/51093921 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f772c06e850 con 0x7f772c1552b0 2024-09-10T21:34:48.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.492+0000 7f772a800640 1 --2- 172.21.15.17:0/51093921 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f772c1552b0 0x7f772c155690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:48.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.492+0000 7f772a800640 1 -- 172.21.15.17:0/51093921 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f772c069e60 con 0x7f772c1552b0 2024-09-10T21:34:48.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.496+0000 7f772a800640 1 --2- 172.21.15.17:0/51093921 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f772c1552b0 0x7f772c155690 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f772c0763f0 tx=0x7f771c009560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:48.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.496+0000 7f7722a00640 1 -- 172.21.15.17:0/51093921 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f771c0bf070 con 0x7f772c1552b0 2024-09-10T21:34:48.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.496+0000 7f7722a00640 1 -- 172.21.15.17:0/51093921 <== mon.2 v2:172.21.15.31:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f771c003800 con 0x7f772c1552b0 2024-09-10T21:34:48.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.496+0000 7f7722a00640 1 -- 172.21.15.17:0/51093921 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f772c1552b0 msgr2=0x7f772c155690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:48.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.496+0000 7f7722a00640 1 --2- 172.21.15.17:0/51093921 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f772c1552b0 0x7f772c155690 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f772c0763f0 tx=0x7f771c009560 comp rx=0 tx=0).stop 2024-09-10T21:34:48.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.496+0000 7f7722a00640 1 -- 172.21.15.17:0/51093921 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f772c1552b0 msgr2=0x7f772c155690 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:34:48.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.496+0000 7f7722a00640 1 --2- 172.21.15.17:0/51093921 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f772c1552b0 0x7f772c155690 secure :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f772c0763f0 tx=0x7f771c009560 comp rx=0 tx=0).stop 2024-09-10T21:34:48.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.496+0000 7f7722a00640 1 -- 172.21.15.17:0/51093921 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f772400f040 con 0x7f772c074f70 2024-09-10T21:34:48.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.496+0000 7f7722a00640 1 --2- 172.21.15.17:0/51093921 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f770c072090 0x7f770c074550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:48.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.496+0000 7f7722a00640 1 -- 172.21.15.17:0/51093921 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f7724093c80 con 0x7f772c074f70 2024-09-10T21:34:48.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.496+0000 7f772a800640 1 --2- 172.21.15.17:0/51093921 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f770c072090 0x7f770c074550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:48.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.496+0000 7f772a800640 1 --2- 172.21.15.17:0/51093921 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f770c072090 0x7f770c074550 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f772c06e9d0 tx=0x7f771c0ba000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:48.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.500+0000 7f7731400640 1 --2- 172.21.15.17:0/51093921 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f772c0aef90 0x7f772c0af370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:48.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.500+0000 7f7731400640 1 -- 172.21.15.17:0/51093921 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f772c070230 con 0x7f772c0aef90 2024-09-10T21:34:48.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.500+0000 7f7729e00640 1 --2- 172.21.15.17:0/51093921 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f772c0aef90 0x7f772c0af370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:48.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.504+0000 7f7729e00640 1 -- 172.21.15.17:0/51093921 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f772c070960 con 0x7f772c0aef90 2024-09-10T21:34:48.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.504+0000 7f7729e00640 1 --2- 172.21.15.17:0/51093921 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f772c0aef90 0x7f772c0af370 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f772c158380 tx=0x7f7718010e00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:48.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.504+0000 7f7722a00640 1 -- 172.21.15.17:0/51093921 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f771800ed80 con 0x7f772c0aef90 2024-09-10T21:34:48.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.504+0000 7f7722a00640 1 -- 172.21.15.17:0/51093921 <== mon.2 v2:172.21.15.31:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4875 (secure 0 0 0) 0x7f771800ed80 con 0x7f772c0aef90 2024-09-10T21:34:48.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.504+0000 7f7722a00640 1 -- 172.21.15.17:0/51093921 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f772c0aef90 msgr2=0x7f772c0af370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:48.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.504+0000 7f7722a00640 1 --2- 172.21.15.17:0/51093921 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f772c0aef90 0x7f772c0af370 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f772c158380 tx=0x7f7718010e00 comp rx=0 tx=0).stop 2024-09-10T21:34:48.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.504+0000 7f7722a00640 1 -- 172.21.15.17:0/51093921 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f772c0aef90 msgr2=0x7f772c0af370 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:34:48.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.504+0000 7f7722a00640 1 --2- 172.21.15.17:0/51093921 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f772c0aef90 0x7f772c0af370 secure :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f772c158380 tx=0x7f7718010e00 comp rx=0 tx=0).stop 2024-09-10T21:34:48.510 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-10T21:34:48.510 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:34:48.510 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:34:48.510 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-10T21:34:48.510 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 86, 2024-09-10T21:34:48.510 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 220813, 2024-09-10T21:34:48.510 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-10T21:34:48.510 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-10T21:34:48.510 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-10T21:34:48.510 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-10T21:34:48.510 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:34:48.510 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 2, 2024-09-10T21:34:48.511 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:34:48.511 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-10T21:34:48.511 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-10T21:34:48.511 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:34:48.511 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:34:48.511 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:34:48.511 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:34:48.511 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:34:48.511 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:34:48.511 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:34:48.511 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:34:48.511 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:34:48.511 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:34:48.511 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:34:48.512 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:34:48.512 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-10T21:34:48.512 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-10T21:34:48.512 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:34:48.512 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:34:48.512 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:34:48.512 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:34:48.512 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:34:48.512 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:34:48.512 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:34:48.512 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:34:48.512 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:34:48.512 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:34:48.513 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:34:48.513 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:34:48.513 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:34:48.513 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-10T21:34:48.513 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-10T21:34:48.513 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-10T21:34:48.513 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-10T21:34:48.513 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-10T21:34:48.513 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "5e2f023f-d6a6-49f4-a256-f4807dce3739", 2024-09-10T21:34:48.513 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:34:48.513 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:34:48.513 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-10T21:34:48.513 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-10T21:34:48.513 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-10T21:34:48.514 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-10T21:34:48.514 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-10T21:34:48.514 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-10T21:34:48.514 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-10T21:34:48.514 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:34:48.514 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-10T21:34:48.514 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:34:48.514 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:34:48.514 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:34:48.514 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:34:48.514 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:34:48.515 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:34:48.515 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:34:48.515 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:34:48.515 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:34:48.515 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:34:48.515 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:34:48.515 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:34:48.515 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-10T21:34:48.515 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:34:48.515 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-10T21:34:48.515 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:34:48.516 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:34:48.516 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:34:48.516 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:34:48.516 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:34:48.516 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:34:48.516 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:34:48.516 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-10T21:34:48.516 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:34:48.516 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:34:48.516 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:34:48.516 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:34:48.516 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-10T21:34:48.516 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:34:48.517 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:34:48.517 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:34:48.517 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:34:48.517 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-10T21:34:48.517 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-10T21:34:48.517 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:34:48.517 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:34:48.517 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:34:48.517 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:34:48.517 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:34:48.517 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:34:48.517 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:34:48.517 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:34:48.517 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:34:48.518 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:34:48.518 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:34:48.518 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3300", 2024-09-10T21:34:48.518 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:34:48.518 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:34:48.518 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:34:48.518 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:34:48.518 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789", 2024-09-10T21:34:48.518 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:34:48.518 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:34:48.518 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:34:48.518 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:34:48.518 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789/0", 2024-09-10T21:34:48.519 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6789/0", 2024-09-10T21:34:48.519 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:34:48.519 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:34:48.519 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:34:48.519 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:34:48.519 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:34:48.519 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:34:48.519 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:34:48.519 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:34:48.519 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:34:48.519 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:34:48.519 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:34:48.519 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3301", 2024-09-10T21:34:48.519 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:34:48.520 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:34:48.520 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:34:48.520 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:34:48.520 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790", 2024-09-10T21:34:48.520 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:34:48.520 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:34:48.520 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:34:48.520 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:34:48.520 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790/0", 2024-09-10T21:34:48.520 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6790/0", 2024-09-10T21:34:48.521 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:34:48.521 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:34:48.521 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:34:48.521 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:34:48.521 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:34:48.521 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:34:48.521 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-10T21:34:48.521 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-10T21:34:48.521 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:34:48.521 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:34:48.522 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:34:48.522 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:34:48.522 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:34:48.522 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:34:48.522 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": [ 2024-09-10T21:34:48.522 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:34:48.522 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:34:48.522 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:34:48.522 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 3 2024-09-10T21:34:48.522 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:34:48.522 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:34:48.522 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-10T21:34:48.522 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:34:48.523 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:34:48.523 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:34:48.523 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 2 2024-09-10T21:34:48.523 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:34:48.523 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:34:48.523 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-10T21:34:48.523 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:34:48.523 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:34:48.523 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:34:48.523 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 2 2024-09-10T21:34:48.523 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:34:48.523 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:34:48.523 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:34:48.523 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-10T21:34:48.524 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-10T21:34:48.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.504+0000 7f7731400640 1 -- 172.21.15.17:0/51093921 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f770c072090 msgr2=0x7f770c074550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:48.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.504+0000 7f7731400640 1 --2- 172.21.15.17:0/51093921 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f770c072090 0x7f770c074550 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f772c06e9d0 tx=0x7f771c0ba000 comp rx=0 tx=0).stop 2024-09-10T21:34:48.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.504+0000 7f7731400640 1 -- 172.21.15.17:0/51093921 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f772c074f70 msgr2=0x7f772c1f45f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:48.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.504+0000 7f7731400640 1 --2- 172.21.15.17:0/51093921 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f772c074f70 0x7f772c1f45f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f772400ea50 tx=0x7f772400ef10 comp rx=0 tx=0).stop 2024-09-10T21:34:48.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.504+0000 7f772b200640 1 -- 172.21.15.17:0/51093921 reap_dead start 2024-09-10T21:34:48.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.504+0000 7f772b200640 1 -- 172.21.15.17:0/51093921 reap_dead start 2024-09-10T21:34:48.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.504+0000 7f7731400640 1 -- 172.21.15.17:0/51093921 shutdown_connections 2024-09-10T21:34:48.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.504+0000 7f7731400640 1 -- 172.21.15.17:0/51093921 >> 172.21.15.17:0/51093921 conn(0x7f772c06dd30 msgr2=0x7f772c0c8ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:48.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.504+0000 7f7731400640 1 -- 172.21.15.17:0/51093921 shutdown_connections 2024-09-10T21:34:48.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.504+0000 7f7731400640 1 -- 172.21.15.17:0/51093921 wait complete. 2024-09-10T21:34:48.529 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-10T21:34:48.529 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-09-10T21:34:48.529 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-09-10T21:34:48.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.688+0000 7f99efe00640 1 -- 172.21.15.17:0/216464369 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f99f0158c20 msgr2=0x7f99f0070670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:48.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.688+0000 7f99efe00640 1 --2- 172.21.15.17:0/216464369 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f99f0158c20 0x7f99f0070670 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f99d8009900 tx=0x7f99d801f0d0 comp rx=0 tx=0).stop 2024-09-10T21:34:48.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.688+0000 7f99efe00640 1 -- 172.21.15.17:0/216464369 shutdown_connections 2024-09-10T21:34:48.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.688+0000 7f99efe00640 1 --2- 172.21.15.17:0/216464369 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f99f0158c20 0x7f99f0070670 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:48.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.688+0000 7f99efe00640 1 -- 172.21.15.17:0/216464369 >> 172.21.15.17:0/216464369 conn(0x7f99f00cdef0 msgr2=0x7f99f00ce2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:48.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.688+0000 7f99efe00640 1 -- 172.21.15.17:0/216464369 shutdown_connections 2024-09-10T21:34:48.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.688+0000 7f99efe00640 1 -- 172.21.15.17:0/216464369 wait complete. 2024-09-10T21:34:48.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.688+0000 7f99efe00640 1 Processor -- start 2024-09-10T21:34:48.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.688+0000 7f99efe00640 1 -- start start 2024-09-10T21:34:48.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.688+0000 7f99efe00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f99f00b3e00 0x7f99f00ccfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:48.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.688+0000 7f99efe00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f99f01524d0 0x7f99f01f2480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:48.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.688+0000 7f99efe00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f99f0158c20 0x7f99f01f29c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:48.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.688+0000 7f99efe00640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f99f00c9350 con 0x7f99f01524d0 2024-09-10T21:34:48.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.688+0000 7f99efe00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f99f00c9070 con 0x7f99f0158c20 2024-09-10T21:34:48.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.688+0000 7f99efe00640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f99f015b9e0 con 0x7f99f00b3e00 2024-09-10T21:34:48.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.692+0000 7f99ee000640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f99f01524d0 0x7f99f01f2480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:48.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.692+0000 7f99ee000640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f99f01524d0 0x7f99f01f2480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:57216/0 (socket says 172.21.15.17:57216) 2024-09-10T21:34:48.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.692+0000 7f99ef400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f99f0158c20 0x7f99f01f29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:48.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.692+0000 7f99ee000640 1 -- 172.21.15.17:0/3229735850 learned_addr learned my addr 172.21.15.17:0/3229735850 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:34:48.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.692+0000 7f99eea00640 1 --2- 172.21.15.17:0/3229735850 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f99f00b3e00 0x7f99f00ccfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:48.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.692+0000 7f99ee000640 1 -- 172.21.15.17:0/3229735850 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f99f00b3e00 msgr2=0x7f99f00ccfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:48.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.692+0000 7f99ee000640 1 --2- 172.21.15.17:0/3229735850 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f99f00b3e00 0x7f99f00ccfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:48.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.692+0000 7f99ee000640 1 -- 172.21.15.17:0/3229735850 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f99f0158c20 msgr2=0x7f99f01f29c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:48.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.692+0000 7f99ee000640 1 --2- 172.21.15.17:0/3229735850 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f99f0158c20 0x7f99f01f29c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:48.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.692+0000 7f99ee000640 1 -- 172.21.15.17:0/3229735850 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99f00715d0 con 0x7f99f01524d0 2024-09-10T21:34:48.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.692+0000 7f99ee000640 1 --2- 172.21.15.17:0/3229735850 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f99f01524d0 0x7f99f01f2480 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f99d80b3040 tx=0x7f99d80a8d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:48.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.692+0000 7f99e6a00640 1 -- 172.21.15.17:0/3229735850 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99d80b5040 con 0x7f99f01524d0 2024-09-10T21:34:48.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.692+0000 7f99e6a00640 1 -- 172.21.15.17:0/3229735850 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f99d8003970 con 0x7f99f01524d0 2024-09-10T21:34:48.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.692+0000 7f99efe00640 1 -- 172.21.15.17:0/3229735850 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99f0150ed0 con 0x7f99f01524d0 2024-09-10T21:34:48.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.692+0000 7f99e6a00640 1 -- 172.21.15.17:0/3229735850 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99d8003d80 con 0x7f99f01524d0 2024-09-10T21:34:48.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.692+0000 7f99efe00640 1 -- 172.21.15.17:0/3229735850 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99f0143570 con 0x7f99f01524d0 2024-09-10T21:34:48.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.692+0000 7f99efe00640 1 -- 172.21.15.17:0/3229735850 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99f015bf00 con 0x7f99f01524d0 2024-09-10T21:34:48.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.692+0000 7f99e6a00640 1 -- 172.21.15.17:0/3229735850 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f99d80b8070 con 0x7f99f01524d0 2024-09-10T21:34:48.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.696+0000 7f99e6a00640 1 --2- 172.21.15.17:0/3229735850 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f99cc0725e0 0x7f99cc074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:48.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.696+0000 7f99e6a00640 1 -- 172.21.15.17:0/3229735850 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f99d81304e0 con 0x7f99f01524d0 2024-09-10T21:34:48.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.696+0000 7f99eea00640 1 --2- 172.21.15.17:0/3229735850 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f99cc0725e0 0x7f99cc074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:48.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.696+0000 7f99eea00640 1 --2- 172.21.15.17:0/3229735850 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f99cc0725e0 0x7f99cc074aa0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f99e0007fd0 tx=0x7f99e0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:48.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.700+0000 7f99e6a00640 1 -- 172.21.15.17:0/3229735850 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f99d8018c30 con 0x7f99f01524d0 2024-09-10T21:34:48.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.960+0000 7f99efe00640 1 -- 172.21.15.17:0/3229735850 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) -- 0x7f99f00c9f60 con 0x7f99f01524d0 2024-09-10T21:34:48.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.964+0000 7f99e6a00640 1 -- 172.21.15.17:0/3229735850 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) ==== 57+0+0 (secure 0 0 0) 0x7f99d80fb240 con 0x7f99f01524d0 2024-09-10T21:34:48.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.964+0000 7f99efe00640 1 -- 172.21.15.17:0/3229735850 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f99cc0725e0 msgr2=0x7f99cc074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:48.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.964+0000 7f99efe00640 1 --2- 172.21.15.17:0/3229735850 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f99cc0725e0 0x7f99cc074aa0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f99e0007fd0 tx=0x7f99e0009040 comp rx=0 tx=0).stop 2024-09-10T21:34:48.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.964+0000 7f99efe00640 1 -- 172.21.15.17:0/3229735850 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f99f01524d0 msgr2=0x7f99f01f2480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:48.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.964+0000 7f99efe00640 1 --2- 172.21.15.17:0/3229735850 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f99f01524d0 0x7f99f01f2480 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f99d80b3040 tx=0x7f99d80a8d60 comp rx=0 tx=0).stop 2024-09-10T21:34:48.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.964+0000 7f99efe00640 1 -- 172.21.15.17:0/3229735850 shutdown_connections 2024-09-10T21:34:48.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.964+0000 7f99efe00640 1 --2- 172.21.15.17:0/3229735850 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f99cc0725e0 0x7f99cc074aa0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:48.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.964+0000 7f99efe00640 1 --2- 172.21.15.17:0/3229735850 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f99f0158c20 0x7f99f01f29c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:48.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.964+0000 7f99efe00640 1 --2- 172.21.15.17:0/3229735850 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f99f01524d0 0x7f99f01f2480 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:48.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.964+0000 7f99efe00640 1 --2- 172.21.15.17:0/3229735850 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f99f00b3e00 0x7f99f00ccfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:48.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.964+0000 7f99efe00640 1 -- 172.21.15.17:0/3229735850 >> 172.21.15.17:0/3229735850 conn(0x7f99f00cdef0 msgr2=0x7f99f0070ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:48.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.964+0000 7f99efe00640 1 -- 172.21.15.17:0/3229735850 shutdown_connections 2024-09-10T21:34:48.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:48.964+0000 7f99efe00640 1 -- 172.21.15.17:0/3229735850 wait complete. 2024-09-10T21:34:48.994 INFO:tasks.mon_thrash.mon_thrasher:waiting for 10.0 secs before continuing thrashing 2024-09-10T21:34:49.207 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:34:49.211 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:34:58.995 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-09-10T21:34:58.995 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-10T21:34:59.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.148+0000 7f84ff400640 1 -- 172.21.15.17:0/925768273 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f84f8152440 msgr2=0x7f84f80b3eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:59.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.148+0000 7f84ff400640 1 --2- 172.21.15.17:0/925768273 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f84f8152440 0x7f84f80b3eb0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f84f81594a0 tx=0x7f84ec01c850 comp rx=0 tx=0).stop 2024-09-10T21:34:59.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.148+0000 7f84ff400640 1 -- 172.21.15.17:0/925768273 shutdown_connections 2024-09-10T21:34:59.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.148+0000 7f84ff400640 1 --2- 172.21.15.17:0/925768273 >> v2:172.21.15.31:3300/0 conn(0x7f84f80cdf20 0x7f84f80ce350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:59.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.148+0000 7f84ff400640 1 --2- 172.21.15.17:0/925768273 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f84f8152440 0x7f84f80b3eb0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:59.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.148+0000 7f84ff400640 1 -- 172.21.15.17:0/925768273 >> 172.21.15.17:0/925768273 conn(0x7f84f80702f0 msgr2=0x7f84f8070700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:59.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.148+0000 7f84ff400640 1 -- 172.21.15.17:0/925768273 shutdown_connections 2024-09-10T21:34:59.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.148+0000 7f84ff400640 1 -- 172.21.15.17:0/925768273 wait complete. 2024-09-10T21:34:59.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.148+0000 7f84ff400640 1 Processor -- start 2024-09-10T21:34:59.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.148+0000 7f84ff400640 1 -- start start 2024-09-10T21:34:59.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.152+0000 7f84ff400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f84f80cdf20 0x7f84f80cd2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:59.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.152+0000 7f84ff400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f84f8152440 0x7f84f806dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:59.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.152+0000 7f84ff400640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f84f8152820 0x7f84f8144950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:59.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.152+0000 7f84ff400640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f84f80cb5d0 con 0x7f84f80cdf20 2024-09-10T21:34:59.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.152+0000 7f84ff400640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f84f80cb450 con 0x7f84f8152440 2024-09-10T21:34:59.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.152+0000 7f84ff400640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f84f80c2ff0 con 0x7f84f8152820 2024-09-10T21:34:59.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.152+0000 7f84fca00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f84f80cdf20 0x7f84f80cd2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:59.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.152+0000 7f84f7e00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f84f8152440 0x7f84f806dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:59.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.152+0000 7f84fd400640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f84f8152820 0x7f84f8144950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:59.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.152+0000 7f84fca00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f84f80cdf20 0x7f84f80cd2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43084/0 (socket says 172.21.15.17:43084) 2024-09-10T21:34:59.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.152+0000 7f84fca00640 1 -- 172.21.15.17:0/4002962450 learned_addr learned my addr 172.21.15.17:0/4002962450 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:34:59.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.152+0000 7f84fd400640 1 -- 172.21.15.17:0/4002962450 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f84f8152440 msgr2=0x7f84f806dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:59.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.152+0000 7f84fd400640 1 --2- 172.21.15.17:0/4002962450 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f84f8152440 0x7f84f806dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:59.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.152+0000 7f84fd400640 1 -- 172.21.15.17:0/4002962450 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f84f80cdf20 msgr2=0x7f84f80cd2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:59.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.152+0000 7f84fd400640 1 --2- 172.21.15.17:0/4002962450 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f84f80cdf20 0x7f84f80cd2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:59.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.152+0000 7f84fd400640 1 -- 172.21.15.17:0/4002962450 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84f8149f60 con 0x7f84f8152820 2024-09-10T21:34:59.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.152+0000 7f84fca00640 1 --2- 172.21.15.17:0/4002962450 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f84f80cdf20 0x7f84f80cd2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:34:59.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.152+0000 7f84fd400640 1 --2- 172.21.15.17:0/4002962450 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f84f8152820 0x7f84f8144950 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f84f000ec20 tx=0x7f84f000c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:59.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.152+0000 7f84f4c00640 1 -- 172.21.15.17:0/4002962450 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84f0003b80 con 0x7f84f8152820 2024-09-10T21:34:59.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.152+0000 7f84f4c00640 1 -- 172.21.15.17:0/4002962450 <== mon.2 v2:172.21.15.31:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f84f0003d20 con 0x7f84f8152820 2024-09-10T21:34:59.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.152+0000 7f84f4c00640 1 -- 172.21.15.17:0/4002962450 <== mon.2 v2:172.21.15.31:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84f0004280 con 0x7f84f8152820 2024-09-10T21:34:59.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.152+0000 7f84ff400640 1 -- 172.21.15.17:0/4002962450 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84f814a8d0 con 0x7f84f8152820 2024-09-10T21:34:59.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.152+0000 7f84ff400640 1 -- 172.21.15.17:0/4002962450 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f84f8147130 con 0x7f84f8152820 2024-09-10T21:34:59.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.152+0000 7f84ff400640 1 -- 172.21.15.17:0/4002962450 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84f814a6e0 con 0x7f84f8152820 2024-09-10T21:34:59.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.156+0000 7f84f4c00640 1 -- 172.21.15.17:0/4002962450 <== mon.2 v2:172.21.15.31:3301/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f84f0004b50 con 0x7f84f8152820 2024-09-10T21:34:59.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.156+0000 7f84f4c00640 1 --2- 172.21.15.17:0/4002962450 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f84d80725e0 0x7f84d8074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:59.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.156+0000 7f84f4c00640 1 -- 172.21.15.17:0/4002962450 <== mon.2 v2:172.21.15.31:3301/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f84f00940b0 con 0x7f84f8152820 2024-09-10T21:34:59.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.156+0000 7f84fca00640 1 --2- 172.21.15.17:0/4002962450 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f84d80725e0 0x7f84d8074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:59.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.156+0000 7f84fca00640 1 --2- 172.21.15.17:0/4002962450 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f84d80725e0 0x7f84d8074aa0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f84ec009230 tx=0x7f84ec01b090 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:59.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.160+0000 7f84f4c00640 1 -- 172.21.15.17:0/4002962450 <== mon.2 v2:172.21.15.31:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f84f005ee20 con 0x7f84f8152820 2024-09-10T21:34:59.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.432+0000 7f84ff400640 1 -- 172.21.15.17:0/4002962450 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f84f8158360 con 0x7f84f8152820 2024-09-10T21:34:59.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.432+0000 7f84f4c00640 1 -- 172.21.15.17:0/4002962450 <== mon.2 v2:172.21.15.31:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1557 (secure 0 0 0) 0x7f84f0063850 con 0x7f84f8152820 2024-09-10T21:34:59.439 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":86,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":13,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5e2f023f-d6a6-49f4-a256-f4807dce3739","modified":"2024-09-10T21:23:46.892130Z","created":"2024-09-10T21:23:46.892130Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3301","nonce":0},{"type":"v1","addr":"172.21.15.31:6790","nonce":0}]},"addr":"172.21.15.31:6790/0","public_addr":"172.21.15.31:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-10T21:34:59.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.436+0000 7f84ff400640 1 -- 172.21.15.17:0/4002962450 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f84d80725e0 msgr2=0x7f84d8074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:59.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.436+0000 7f84ff400640 1 --2- 172.21.15.17:0/4002962450 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f84d80725e0 0x7f84d8074aa0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f84ec009230 tx=0x7f84ec01b090 comp rx=0 tx=0).stop 2024-09-10T21:34:59.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.436+0000 7f84ff400640 1 -- 172.21.15.17:0/4002962450 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f84f8152820 msgr2=0x7f84f8144950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:59.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.436+0000 7f84ff400640 1 --2- 172.21.15.17:0/4002962450 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f84f8152820 0x7f84f8144950 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f84f000ec20 tx=0x7f84f000c6a0 comp rx=0 tx=0).stop 2024-09-10T21:34:59.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.436+0000 7f84ff400640 1 -- 172.21.15.17:0/4002962450 shutdown_connections 2024-09-10T21:34:59.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.436+0000 7f84ff400640 1 --2- 172.21.15.17:0/4002962450 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f84d80725e0 0x7f84d8074aa0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:59.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.436+0000 7f84ff400640 1 --2- 172.21.15.17:0/4002962450 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f84f8152820 0x7f84f8144950 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:59.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.436+0000 7f84ff400640 1 --2- 172.21.15.17:0/4002962450 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f84f8152440 0x7f84f806dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:59.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.436+0000 7f84ff400640 1 --2- 172.21.15.17:0/4002962450 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f84f80cdf20 0x7f84f80cd2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:59.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.436+0000 7f84ff400640 1 -- 172.21.15.17:0/4002962450 >> 172.21.15.17:0/4002962450 conn(0x7f84f80702f0 msgr2=0x7f84f80c3390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:59.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.436+0000 7f84ff400640 1 -- 172.21.15.17:0/4002962450 shutdown_connections 2024-09-10T21:34:59.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.436+0000 7f84ff400640 1 -- 172.21.15.17:0/4002962450 wait complete. 2024-09-10T21:34:59.462 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-09-10T21:34:59.463 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-09-10T21:34:59.463 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-09-10T21:34:59.463 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-10T21:34:59.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.632+0000 7f3792600640 1 -- 172.21.15.17:0/2697842926 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f378c151150 msgr2=0x7f378c157020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:59.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.632+0000 7f3792600640 1 --2- 172.21.15.17:0/2697842926 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f378c151150 0x7f378c157020 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f378c154bf0 tx=0x7f378001f0b0 comp rx=0 tx=0).stop 2024-09-10T21:34:59.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.632+0000 7f3792600640 1 -- 172.21.15.17:0/2697842926 shutdown_connections 2024-09-10T21:34:59.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.632+0000 7f3792600640 1 --2- 172.21.15.17:0/2697842926 >> v2:172.21.15.31:3300/0 conn(0x7f378c0c66b0 0x7f378c070b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:59.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.632+0000 7f3792600640 1 --2- 172.21.15.17:0/2697842926 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f378c0c6290 0x7f378c070610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:59.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.632+0000 7f3792600640 1 --2- 172.21.15.17:0/2697842926 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f378c151150 0x7f378c157020 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:59.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.632+0000 7f3792600640 1 -- 172.21.15.17:0/2697842926 >> 172.21.15.17:0/2697842926 conn(0x7f378c069880 msgr2=0x7f378c06cc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:59.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.632+0000 7f3792600640 1 -- 172.21.15.17:0/2697842926 shutdown_connections 2024-09-10T21:34:59.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.632+0000 7f3792600640 1 -- 172.21.15.17:0/2697842926 wait complete. 2024-09-10T21:34:59.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.636+0000 7f3792600640 1 Processor -- start 2024-09-10T21:34:59.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.636+0000 7f3792600640 1 -- start start 2024-09-10T21:34:59.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.636+0000 7f3792600640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f378c0c6290 0x7f378c1f2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:59.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.636+0000 7f3792600640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f378c0c66b0 0x7f378c146110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:59.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.636+0000 7f3792600640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f378c151150 0x7f378c146650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:59.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.636+0000 7f3792600640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f378c0c5d70 con 0x7f378c0c6290 2024-09-10T21:34:59.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.636+0000 7f3792600640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f378c0c4a00 con 0x7f378c0c66b0 2024-09-10T21:34:59.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.636+0000 7f3792600640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f378c072c20 con 0x7f378c151150 2024-09-10T21:34:59.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.636+0000 7f3791200640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f378c0c6290 0x7f378c1f2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:59.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.636+0000 7f3791200640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f378c0c6290 0x7f378c1f2080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43088/0 (socket says 172.21.15.17:43088) 2024-09-10T21:34:59.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.636+0000 7f3791200640 1 -- 172.21.15.17:0/3576579722 learned_addr learned my addr 172.21.15.17:0/3576579722 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:34:59.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.636+0000 7f378be00640 1 --2- 172.21.15.17:0/3576579722 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f378c0c66b0 0x7f378c146110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:59.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.636+0000 7f3791c00640 1 --2- 172.21.15.17:0/3576579722 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f378c151150 0x7f378c146650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:59.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.636+0000 7f3791200640 1 -- 172.21.15.17:0/3576579722 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f378c151150 msgr2=0x7f378c146650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:59.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.636+0000 7f3791200640 1 --2- 172.21.15.17:0/3576579722 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f378c151150 0x7f378c146650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:59.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.636+0000 7f3791200640 1 -- 172.21.15.17:0/3576579722 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f378c0c66b0 msgr2=0x7f378c146110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:59.644 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.636+0000 7f3791200640 1 --2- 172.21.15.17:0/3576579722 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f378c0c66b0 0x7f378c146110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:59.644 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.636+0000 7f3791200640 1 -- 172.21.15.17:0/3576579722 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f378c1eedd0 con 0x7f378c0c6290 2024-09-10T21:34:59.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.636+0000 7f3791200640 1 --2- 172.21.15.17:0/3576579722 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f378c0c6290 0x7f378c1f2080 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f37800b3040 tx=0x7f3780005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:59.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.636+0000 7f3788c00640 1 -- 172.21.15.17:0/3576579722 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37800b5040 con 0x7f378c0c6290 2024-09-10T21:34:59.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.636+0000 7f3788c00640 1 -- 172.21.15.17:0/3576579722 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3780005320 con 0x7f378c0c6290 2024-09-10T21:34:59.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.636+0000 7f3788c00640 1 -- 172.21.15.17:0/3576579722 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37800046d0 con 0x7f378c0c6290 2024-09-10T21:34:59.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.636+0000 7f3792600640 1 -- 172.21.15.17:0/3576579722 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f378c154570 con 0x7f378c0c6290 2024-09-10T21:34:59.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.636+0000 7f3792600640 1 -- 172.21.15.17:0/3576579722 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f378c1f1880 con 0x7f378c0c6290 2024-09-10T21:34:59.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.640+0000 7f3792600640 1 --2- 172.21.15.17:0/3576579722 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f378c0c76b0 0x7f378c14a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:59.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.640+0000 7f3792600640 1 -- 172.21.15.17:0/3576579722 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f378c0c5860 con 0x7f378c0c76b0 2024-09-10T21:34:59.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.640+0000 7f378be00640 1 --2- 172.21.15.17:0/3576579722 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f378c0c76b0 0x7f378c14a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:59.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.640+0000 7f378be00640 1 -- 172.21.15.17:0/3576579722 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f378c1f25c0 con 0x7f378c0c76b0 2024-09-10T21:34:59.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.640+0000 7f378be00640 1 --2- 172.21.15.17:0/3576579722 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f378c0c76b0 0x7f378c14a500 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f378c153c10 tx=0x7f377800fd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:59.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.640+0000 7f3788c00640 1 -- 172.21.15.17:0/3576579722 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f377800dd40 con 0x7f378c0c76b0 2024-09-10T21:34:59.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.640+0000 7f3788c00640 1 -- 172.21.15.17:0/3576579722 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f3778011440 con 0x7f378c0c76b0 2024-09-10T21:34:59.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.640+0000 7f3788c00640 1 -- 172.21.15.17:0/3576579722 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f378c0c76b0 msgr2=0x7f378c14a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:59.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.640+0000 7f3788c00640 1 --2- 172.21.15.17:0/3576579722 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f378c0c76b0 0x7f378c14a500 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f378c153c10 tx=0x7f377800fd90 comp rx=0 tx=0).stop 2024-09-10T21:34:59.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.640+0000 7f3788c00640 1 -- 172.21.15.17:0/3576579722 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f378c0c76b0 msgr2=0x7f378c14a500 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:34:59.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.640+0000 7f3788c00640 1 --2- 172.21.15.17:0/3576579722 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f378c0c76b0 0x7f378c14a500 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:59.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.640+0000 7f3788c00640 1 -- 172.21.15.17:0/3576579722 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f3780004890 con 0x7f378c0c6290 2024-09-10T21:34:59.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.640+0000 7f3788c00640 1 --2- 172.21.15.17:0/3576579722 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f376c072050 0x7f376c074510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:59.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.640+0000 7f3788c00640 1 -- 172.21.15.17:0/3576579722 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f37801303e0 con 0x7f378c0c6290 2024-09-10T21:34:59.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.640+0000 7f378be00640 1 --2- 172.21.15.17:0/3576579722 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f376c072050 0x7f376c074510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:59.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.644+0000 7f378be00640 1 --2- 172.21.15.17:0/3576579722 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f376c072050 0x7f376c074510 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f377800d750 tx=0x7f3778014000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:59.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.648+0000 7f3792600640 1 --2- 172.21.15.17:0/3576579722 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f378c1f1ac0 0x7f378c14aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:59.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.648+0000 7f3792600640 1 -- 172.21.15.17:0/3576579722 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f378c069e60 con 0x7f378c1f1ac0 2024-09-10T21:34:59.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.648+0000 7f3791c00640 1 --2- 172.21.15.17:0/3576579722 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f378c1f1ac0 0x7f378c14aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:59.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.648+0000 7f3791c00640 1 -- 172.21.15.17:0/3576579722 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f378c072ec0 con 0x7f378c1f1ac0 2024-09-10T21:34:59.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.652+0000 7f3791c00640 1 --2- 172.21.15.17:0/3576579722 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f378c1f1ac0 0x7f378c14aa40 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f378c14ca60 tx=0x7f378400fed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:59.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.652+0000 7f3788c00640 1 -- 172.21.15.17:0/3576579722 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f378400de60 con 0x7f378c1f1ac0 2024-09-10T21:34:59.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.652+0000 7f3788c00640 1 -- 172.21.15.17:0/3576579722 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5044 (secure 0 0 0) 0x7f378400de60 con 0x7f378c1f1ac0 2024-09-10T21:34:59.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.652+0000 7f3788c00640 1 -- 172.21.15.17:0/3576579722 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f378c1f1ac0 msgr2=0x7f378c14aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:59.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.652+0000 7f3788c00640 1 --2- 172.21.15.17:0/3576579722 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f378c1f1ac0 0x7f378c14aa40 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f378c14ca60 tx=0x7f378400fed0 comp rx=0 tx=0).stop 2024-09-10T21:34:59.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.652+0000 7f3788c00640 1 -- 172.21.15.17:0/3576579722 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f378c1f1ac0 msgr2=0x7f378c14aa40 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:34:59.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.652+0000 7f3788c00640 1 --2- 172.21.15.17:0/3576579722 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f378c1f1ac0 0x7f378c14aa40 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:59.658 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-10T21:34:59.658 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:34:59.658 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:34:59.658 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "leader", 2024-09-10T21:34:59.658 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 86, 2024-09-10T21:34:59.658 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 80259, 2024-09-10T21:34:59.658 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-10T21:34:59.658 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-10T21:34:59.658 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-10T21:34:59.658 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-10T21:34:59.658 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:34:59.659 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 13, 2024-09-10T21:34:59.659 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:34:59.659 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-10T21:34:59.659 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-10T21:34:59.659 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:34:59.659 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:34:59.659 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:34:59.659 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:34:59.659 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:34:59.659 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:34:59.659 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:34:59.660 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:34:59.660 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:34:59.660 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:34:59.660 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:34:59.660 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:34:59.660 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-10T21:34:59.660 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-10T21:34:59.660 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:34:59.660 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:34:59.661 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:34:59.661 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:34:59.661 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:34:59.661 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:34:59.661 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:34:59.661 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:34:59.661 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:34:59.661 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:34:59.661 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:34:59.661 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:34:59.661 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:34:59.661 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-10T21:34:59.661 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-10T21:34:59.662 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-10T21:34:59.662 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-10T21:34:59.662 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-10T21:34:59.662 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "5e2f023f-d6a6-49f4-a256-f4807dce3739", 2024-09-10T21:34:59.662 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:34:59.662 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:34:59.662 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-10T21:34:59.662 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-10T21:34:59.662 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-10T21:34:59.662 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-10T21:34:59.662 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-10T21:34:59.662 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-10T21:34:59.663 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-10T21:34:59.663 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:34:59.663 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-10T21:34:59.663 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:34:59.663 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:34:59.663 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:34:59.663 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:34:59.663 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:34:59.663 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:34:59.663 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:34:59.663 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:34:59.663 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:34:59.664 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:34:59.664 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:34:59.664 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:34:59.664 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-10T21:34:59.664 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:34:59.664 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-10T21:34:59.664 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:34:59.664 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:34:59.664 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:34:59.664 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:34:59.664 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:34:59.664 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:34:59.665 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:34:59.665 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-10T21:34:59.665 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:34:59.665 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:34:59.665 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:34:59.665 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:34:59.665 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-10T21:34:59.665 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:34:59.665 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:34:59.665 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:34:59.665 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:34:59.665 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-10T21:34:59.666 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-10T21:34:59.666 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:34:59.666 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:34:59.666 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:34:59.666 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:34:59.666 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:34:59.666 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:34:59.666 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:34:59.666 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:34:59.666 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:34:59.666 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:34:59.666 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:34:59.667 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3300", 2024-09-10T21:34:59.667 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:34:59.667 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:34:59.667 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:34:59.667 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:34:59.667 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789", 2024-09-10T21:34:59.667 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:34:59.667 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:34:59.667 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:34:59.667 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:34:59.667 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789/0", 2024-09-10T21:34:59.667 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6789/0", 2024-09-10T21:34:59.667 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:34:59.668 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:34:59.668 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:34:59.668 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:34:59.668 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:34:59.668 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:34:59.668 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:34:59.668 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:34:59.668 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:34:59.668 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:34:59.668 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:34:59.668 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3301", 2024-09-10T21:34:59.668 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:34:59.668 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:34:59.668 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:34:59.669 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:34:59.669 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790", 2024-09-10T21:34:59.669 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:34:59.669 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:34:59.669 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:34:59.669 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:34:59.669 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790/0", 2024-09-10T21:34:59.669 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6790/0", 2024-09-10T21:34:59.669 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:34:59.669 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:34:59.669 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:34:59.669 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:34:59.670 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:34:59.670 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:34:59.670 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-10T21:34:59.670 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-10T21:34:59.670 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:34:59.670 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:34:59.670 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:34:59.670 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:34:59.670 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:34:59.670 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:34:59.670 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": [ 2024-09-10T21:34:59.670 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:34:59.670 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:34:59.670 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:34:59.671 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 3 2024-09-10T21:34:59.671 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:34:59.671 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:34:59.671 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-10T21:34:59.671 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:34:59.671 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:34:59.671 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:34:59.671 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 6 2024-09-10T21:34:59.671 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:34:59.671 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:34:59.671 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-10T21:34:59.671 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:34:59.672 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:34:59.672 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:34:59.672 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 6 2024-09-10T21:34:59.672 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:34:59.672 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:34:59.672 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": [ 2024-09-10T21:34:59.672 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:34:59.672 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:34:59.672 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:34:59.672 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 2 2024-09-10T21:34:59.672 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:34:59.672 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:34:59.672 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:34:59.672 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-10T21:34:59.673 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-10T21:34:59.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.652+0000 7f3792600640 1 -- 172.21.15.17:0/3576579722 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f376c072050 msgr2=0x7f376c074510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:59.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.652+0000 7f3792600640 1 --2- 172.21.15.17:0/3576579722 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f376c072050 0x7f376c074510 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f377800d750 tx=0x7f3778014000 comp rx=0 tx=0).stop 2024-09-10T21:34:59.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.652+0000 7f3792600640 1 -- 172.21.15.17:0/3576579722 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f378c0c6290 msgr2=0x7f378c1f2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:59.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.652+0000 7f3792600640 1 --2- 172.21.15.17:0/3576579722 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f378c0c6290 0x7f378c1f2080 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f37800b3040 tx=0x7f3780005780 comp rx=0 tx=0).stop 2024-09-10T21:34:59.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.652+0000 7f3791c00640 1 -- 172.21.15.17:0/3576579722 reap_dead start 2024-09-10T21:34:59.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.652+0000 7f3792600640 1 -- 172.21.15.17:0/3576579722 shutdown_connections 2024-09-10T21:34:59.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.652+0000 7f3792600640 1 --2- 172.21.15.17:0/3576579722 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f378c0c6290 0x7f378c1f2080 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:59.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.652+0000 7f3792600640 1 -- 172.21.15.17:0/3576579722 >> 172.21.15.17:0/3576579722 conn(0x7f378c069880 msgr2=0x7f378c147520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:59.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.652+0000 7f3792600640 1 -- 172.21.15.17:0/3576579722 shutdown_connections 2024-09-10T21:34:59.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.652+0000 7f3792600640 1 -- 172.21.15.17:0/3576579722 wait complete. 2024-09-10T21:34:59.678 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-10T21:34:59.678 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-09-10T21:34:59.678 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-10T21:34:59.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.832+0000 7fc1bfe00640 1 -- 172.21.15.17:0/680686228 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc1c0151250 msgr2=0x7fc1c0157100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:59.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.832+0000 7fc1bfe00640 1 --2- 172.21.15.17:0/680686228 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc1c0151250 0x7fc1c0157100 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fc1c006dc40 tx=0x7fc1b001f0b0 comp rx=0 tx=0).stop 2024-09-10T21:34:59.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.832+0000 7fc1bfe00640 1 -- 172.21.15.17:0/680686228 shutdown_connections 2024-09-10T21:34:59.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.832+0000 7fc1bfe00640 1 --2- 172.21.15.17:0/680686228 >> v2:172.21.15.31:3300/0 conn(0x7fc1c0072bb0 0x7fc1c00c6300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:59.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.832+0000 7fc1bfe00640 1 --2- 172.21.15.17:0/680686228 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc1c0151250 0x7fc1c0157100 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:59.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.832+0000 7fc1bfe00640 1 -- 172.21.15.17:0/680686228 >> 172.21.15.17:0/680686228 conn(0x7fc1c006cc20 msgr2=0x7fc1c006d030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:59.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.832+0000 7fc1bfe00640 1 -- 172.21.15.17:0/680686228 shutdown_connections 2024-09-10T21:34:59.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.832+0000 7fc1bfe00640 1 -- 172.21.15.17:0/680686228 wait complete. 2024-09-10T21:34:59.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.836+0000 7fc1bfe00640 1 Processor -- start 2024-09-10T21:34:59.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.836+0000 7fc1bfe00640 1 -- start start 2024-09-10T21:34:59.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.836+0000 7fc1bfe00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc1c0068d20 0x7fc1c01e24a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:59.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.836+0000 7fc1bfe00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc1c0151250 0x7fc1c01474d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:59.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.836+0000 7fc1bfe00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc1c00c9260 0x7fc1c00c9710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:59.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.836+0000 7fc1bfe00640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fc1c00c5b00 con 0x7fc1c00c9260 2024-09-10T21:34:59.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.836+0000 7fc1bfe00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fc1c00c5820 con 0x7fc1c0068d20 2024-09-10T21:34:59.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.836+0000 7fc1bfe00640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7fc1c00c5de0 con 0x7fc1c0151250 2024-09-10T21:34:59.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.836+0000 7fc1bf400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc1c00c9260 0x7fc1c00c9710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:59.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.836+0000 7fc1bea00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc1c0068d20 0x7fc1c01e24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:59.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.836+0000 7fc1bea00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc1c0068d20 0x7fc1c01e24a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.17:56054/0 (socket says 172.21.15.17:56054) 2024-09-10T21:34:59.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.836+0000 7fc1bea00640 1 -- 172.21.15.17:0/2288377101 learned_addr learned my addr 172.21.15.17:0/2288377101 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:34:59.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.836+0000 7fc1be000640 1 --2- 172.21.15.17:0/2288377101 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc1c0151250 0x7fc1c01474d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:59.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.836+0000 7fc1bea00640 1 -- 172.21.15.17:0/2288377101 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc1c0151250 msgr2=0x7fc1c01474d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:59.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.836+0000 7fc1bea00640 1 --2- 172.21.15.17:0/2288377101 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc1c0151250 0x7fc1c01474d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:59.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.836+0000 7fc1bea00640 1 -- 172.21.15.17:0/2288377101 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc1c00c9260 msgr2=0x7fc1c00c9710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:59.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.836+0000 7fc1bea00640 1 --2- 172.21.15.17:0/2288377101 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc1c00c9260 0x7fc1c00c9710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:59.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.836+0000 7fc1bea00640 1 -- 172.21.15.17:0/2288377101 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1c0147d10 con 0x7fc1c0068d20 2024-09-10T21:34:59.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.836+0000 7fc1bea00640 1 --2- 172.21.15.17:0/2288377101 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc1c0068d20 0x7fc1c01e24a0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fc1b00b3040 tx=0x7fc1b0004600 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:59.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.836+0000 7fc1b6a00640 1 -- 172.21.15.17:0/2288377101 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1b00be070 con 0x7fc1c0068d20 2024-09-10T21:34:59.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.836+0000 7fc1b6a00640 1 -- 172.21.15.17:0/2288377101 <== mon.1 v2:172.21.15.31:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc1b00053e0 con 0x7fc1c0068d20 2024-09-10T21:34:59.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.836+0000 7fc1b6a00640 1 -- 172.21.15.17:0/2288377101 <== mon.1 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1b00b5040 con 0x7fc1c0068d20 2024-09-10T21:34:59.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.836+0000 7fc1bfe00640 1 -- 172.21.15.17:0/2288377101 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1c0147fb0 con 0x7fc1c0068d20 2024-09-10T21:34:59.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.840+0000 7fc1bfe00640 1 -- 172.21.15.17:0/2288377101 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1c014bbf0 con 0x7fc1c0068d20 2024-09-10T21:34:59.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.840+0000 7fc1bfe00640 1 --2- 172.21.15.17:0/2288377101 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc1c0072bb0 0x7fc1c01efcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:59.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.840+0000 7fc1bfe00640 1 -- 172.21.15.17:0/2288377101 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fc1c014ba50 con 0x7fc1c0072bb0 2024-09-10T21:34:59.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.840+0000 7fc1be000640 1 --2- 172.21.15.17:0/2288377101 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc1c0072bb0 0x7fc1c01efcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:59.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.840+0000 7fc1be000640 1 -- 172.21.15.17:0/2288377101 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc1c01e4620 con 0x7fc1c0072bb0 2024-09-10T21:34:59.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.844+0000 7fc1be000640 1 --2- 172.21.15.17:0/2288377101 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc1c0072bb0 0x7fc1c01efcd0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fc1ac010970 tx=0x7fc1ac010e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:59.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.844+0000 7fc1b6a00640 1 -- 172.21.15.17:0/2288377101 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1ac00ee70 con 0x7fc1c0072bb0 2024-09-10T21:34:59.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.844+0000 7fc1b6a00640 1 -- 172.21.15.17:0/2288377101 <== mon.1 v2:172.21.15.31:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fc1ac01c480 con 0x7fc1c0072bb0 2024-09-10T21:34:59.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.844+0000 7fc1b6a00640 1 -- 172.21.15.17:0/2288377101 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc1c0072bb0 msgr2=0x7fc1c01efcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:59.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.844+0000 7fc1b6a00640 1 --2- 172.21.15.17:0/2288377101 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc1c0072bb0 0x7fc1c01efcd0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fc1ac010970 tx=0x7fc1ac010e30 comp rx=0 tx=0).stop 2024-09-10T21:34:59.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.844+0000 7fc1b6a00640 1 -- 172.21.15.17:0/2288377101 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc1c0072bb0 msgr2=0x7fc1c01efcd0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:34:59.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.844+0000 7fc1b6a00640 1 --2- 172.21.15.17:0/2288377101 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc1c0072bb0 0x7fc1c01efcd0 secure :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fc1ac010970 tx=0x7fc1ac010e30 comp rx=0 tx=0).stop 2024-09-10T21:34:59.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.844+0000 7fc1b6a00640 1 -- 172.21.15.17:0/2288377101 <== mon.1 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fc1b00b8030 con 0x7fc1c0068d20 2024-09-10T21:34:59.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.844+0000 7fc1b6a00640 1 --2- 172.21.15.17:0/2288377101 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fc1a00720c0 0x7fc1a0074580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:59.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.844+0000 7fc1b6a00640 1 -- 172.21.15.17:0/2288377101 <== mon.1 v2:172.21.15.31:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fc1b0130200 con 0x7fc1c0068d20 2024-09-10T21:34:59.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.844+0000 7fc1be000640 1 --2- 172.21.15.17:0/2288377101 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fc1a00720c0 0x7fc1a0074580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:59.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.848+0000 7fc1be000640 1 --2- 172.21.15.17:0/2288377101 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fc1a00720c0 0x7fc1a0074580 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fc1ac000c10 tx=0x7fc1ac016000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:59.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.852+0000 7fc1bfe00640 1 --2- 172.21.15.17:0/2288377101 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc1c014b470 0x7fc1c01f0210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:34:59.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.852+0000 7fc1bfe00640 1 -- 172.21.15.17:0/2288377101 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fc1c01f0750 con 0x7fc1c014b470 2024-09-10T21:34:59.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.852+0000 7fc1bf400640 1 --2- 172.21.15.17:0/2288377101 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc1c014b470 0x7fc1c01f0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:34:59.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.852+0000 7fc1bf400640 1 -- 172.21.15.17:0/2288377101 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fc1c0070db0 con 0x7fc1c014b470 2024-09-10T21:34:59.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.852+0000 7fc1bf400640 1 --2- 172.21.15.17:0/2288377101 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc1c014b470 0x7fc1c01f0210 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fc1b8007fd0 tx=0x7fc1b800ff70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:34:59.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.852+0000 7fc1b6a00640 1 -- 172.21.15.17:0/2288377101 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1b8003b90 con 0x7fc1c014b470 2024-09-10T21:34:59.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.852+0000 7fc1b6a00640 1 -- 172.21.15.17:0/2288377101 <== mon.1 v2:172.21.15.31:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4541 (secure 0 0 0) 0x7fc1b8003b90 con 0x7fc1c014b470 2024-09-10T21:34:59.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.852+0000 7fc1b6a00640 1 -- 172.21.15.17:0/2288377101 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc1c014b470 msgr2=0x7fc1c01f0210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:59.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.852+0000 7fc1b6a00640 1 --2- 172.21.15.17:0/2288377101 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc1c014b470 0x7fc1c01f0210 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fc1b8007fd0 tx=0x7fc1b800ff70 comp rx=0 tx=0).stop 2024-09-10T21:34:59.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.852+0000 7fc1b6a00640 1 -- 172.21.15.17:0/2288377101 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc1c014b470 msgr2=0x7fc1c01f0210 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:34:59.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.852+0000 7fc1b6a00640 1 --2- 172.21.15.17:0/2288377101 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc1c014b470 0x7fc1c01f0210 secure :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fc1b8007fd0 tx=0x7fc1b800ff70 comp rx=0 tx=0).stop 2024-09-10T21:34:59.859 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-10T21:34:59.859 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:34:59.859 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:34:59.860 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-10T21:34:59.860 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 86, 2024-09-10T21:34:59.860 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 19239, 2024-09-10T21:34:59.860 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-10T21:34:59.860 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-10T21:34:59.860 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-10T21:34:59.860 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-10T21:34:59.860 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:34:59.860 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 13, 2024-09-10T21:34:59.860 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:34:59.860 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-10T21:34:59.860 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-10T21:34:59.861 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:34:59.861 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:34:59.861 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:34:59.861 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:34:59.861 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:34:59.861 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:34:59.861 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:34:59.861 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:34:59.861 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:34:59.862 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:34:59.862 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:34:59.862 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:34:59.862 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-10T21:34:59.862 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-10T21:34:59.862 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:34:59.862 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:34:59.862 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:34:59.862 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:34:59.862 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:34:59.862 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:34:59.863 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:34:59.863 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:34:59.863 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:34:59.863 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:34:59.863 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:34:59.863 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:34:59.863 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:34:59.863 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-10T21:34:59.863 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-10T21:34:59.863 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-10T21:34:59.863 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-10T21:34:59.863 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-10T21:34:59.863 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "5e2f023f-d6a6-49f4-a256-f4807dce3739", 2024-09-10T21:34:59.863 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:34:59.864 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:34:59.864 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-10T21:34:59.864 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-10T21:34:59.864 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-10T21:34:59.864 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-10T21:34:59.864 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-10T21:34:59.864 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-10T21:34:59.864 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-10T21:34:59.864 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:34:59.864 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-10T21:34:59.864 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:34:59.864 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:34:59.864 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:34:59.864 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:34:59.865 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:34:59.865 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:34:59.865 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:34:59.865 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:34:59.865 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:34:59.865 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:34:59.865 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:34:59.865 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:34:59.865 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-10T21:34:59.865 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:34:59.866 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-10T21:34:59.866 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:34:59.866 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:34:59.866 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:34:59.866 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:34:59.866 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:34:59.866 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:34:59.866 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:34:59.866 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-10T21:34:59.866 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:34:59.866 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:34:59.867 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:34:59.867 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:34:59.867 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-10T21:34:59.867 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:34:59.867 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:34:59.867 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:34:59.867 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:34:59.867 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-10T21:34:59.867 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-10T21:34:59.867 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:34:59.867 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:34:59.868 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:34:59.868 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:34:59.868 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:34:59.868 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:34:59.868 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:34:59.868 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:34:59.868 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:34:59.868 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:34:59.868 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:34:59.868 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3300", 2024-09-10T21:34:59.868 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:34:59.868 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:34:59.868 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:34:59.869 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:34:59.869 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789", 2024-09-10T21:34:59.869 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:34:59.869 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:34:59.869 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:34:59.869 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:34:59.869 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789/0", 2024-09-10T21:34:59.869 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6789/0", 2024-09-10T21:34:59.869 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:34:59.869 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:34:59.869 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:34:59.869 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:34:59.869 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:34:59.869 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:34:59.870 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:34:59.870 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:34:59.870 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:34:59.870 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:34:59.870 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:34:59.870 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3301", 2024-09-10T21:34:59.870 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:34:59.870 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:34:59.870 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:34:59.870 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:34:59.870 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790", 2024-09-10T21:34:59.870 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:34:59.870 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:34:59.870 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:34:59.871 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:34:59.871 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790/0", 2024-09-10T21:34:59.871 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6790/0", 2024-09-10T21:34:59.871 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:34:59.871 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:34:59.871 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:34:59.871 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:34:59.871 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:34:59.871 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:34:59.871 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-10T21:34:59.871 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-10T21:34:59.871 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:34:59.871 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:34:59.872 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:34:59.872 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:34:59.872 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:34:59.872 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:34:59.872 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-10T21:34:59.872 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:34:59.872 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:34:59.872 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:34:59.872 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 2 2024-09-10T21:34:59.872 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:34:59.872 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:34:59.872 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:34:59.872 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-10T21:34:59.872 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-10T21:34:59.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.852+0000 7fc1bfe00640 1 -- 172.21.15.17:0/2288377101 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fc1a00720c0 msgr2=0x7fc1a0074580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:59.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.852+0000 7fc1bfe00640 1 --2- 172.21.15.17:0/2288377101 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fc1a00720c0 0x7fc1a0074580 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fc1ac000c10 tx=0x7fc1ac016000 comp rx=0 tx=0).stop 2024-09-10T21:34:59.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.852+0000 7fc1bfe00640 1 -- 172.21.15.17:0/2288377101 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc1c0068d20 msgr2=0x7fc1c01e24a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:34:59.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.852+0000 7fc1bfe00640 1 --2- 172.21.15.17:0/2288377101 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc1c0068d20 0x7fc1c01e24a0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fc1b00b3040 tx=0x7fc1b0004600 comp rx=0 tx=0).stop 2024-09-10T21:34:59.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.852+0000 7fc1bf400640 1 -- 172.21.15.17:0/2288377101 reap_dead start 2024-09-10T21:34:59.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.852+0000 7fc1bfe00640 1 -- 172.21.15.17:0/2288377101 shutdown_connections 2024-09-10T21:34:59.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.852+0000 7fc1bfe00640 1 --2- 172.21.15.17:0/2288377101 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc1c0068d20 0x7fc1c01e24a0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:34:59.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.852+0000 7fc1bfe00640 1 -- 172.21.15.17:0/2288377101 >> 172.21.15.17:0/2288377101 conn(0x7fc1c006cc20 msgr2=0x7fc1c0073b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:34:59.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.852+0000 7fc1bfe00640 1 -- 172.21.15.17:0/2288377101 shutdown_connections 2024-09-10T21:34:59.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:34:59.852+0000 7fc1bfe00640 1 -- 172.21.15.17:0/2288377101 wait complete. 2024-09-10T21:34:59.878 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-10T21:34:59.878 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-09-10T21:34:59.879 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-10T21:35:00.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:00.044+0000 7f1982c00640 1 -- 172.21.15.17:0/292884311 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f197c152460 msgr2=0x7f197c0c8770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:00.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:00.044+0000 7f1982c00640 1 --2- 172.21.15.17:0/292884311 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f197c152460 0x7f197c0c8770 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f1970009840 tx=0x7f197001c700 comp rx=0 tx=0).stop 2024-09-10T21:35:00.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:00.044+0000 7f1982c00640 1 -- 172.21.15.17:0/292884311 shutdown_connections 2024-09-10T21:35:00.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:00.044+0000 7f1982c00640 1 --2- 172.21.15.17:0/292884311 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f197c0c77e0 0x7f197c162bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:00.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:00.044+0000 7f1982c00640 1 --2- 172.21.15.17:0/292884311 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f197c152460 0x7f197c0c8770 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:00.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:00.044+0000 7f1982c00640 1 -- 172.21.15.17:0/292884311 >> 172.21.15.17:0/292884311 conn(0x7f197c0cd690 msgr2=0x7f197c0c98e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:00.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:00.044+0000 7f1982c00640 1 -- 172.21.15.17:0/292884311 shutdown_connections 2024-09-10T21:35:00.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:00.044+0000 7f1982c00640 1 -- 172.21.15.17:0/292884311 wait complete. 2024-09-10T21:35:00.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:00.044+0000 7f1982c00640 1 Processor -- start 2024-09-10T21:35:00.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:00.044+0000 7f1982c00640 1 -- start start 2024-09-10T21:35:00.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:00.048+0000 7f1982c00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f197c06ec30 0x7f197c0c5d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:00.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:00.048+0000 7f1982c00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f197c0c77e0 0x7f197c145680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:00.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:00.048+0000 7f1982c00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f197c152460 0x7f197c145bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:00.055 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:00.048+0000 7f1982c00640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f197c06fbe0 con 0x7f197c152460 2024-09-10T21:35:00.055 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:00.048+0000 7f1982c00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f197c06fa60 con 0x7f197c06ec30 2024-09-10T21:35:00.055 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:00.048+0000 7f1982c00640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f197c0ca5f0 con 0x7f197c0c77e0 2024-09-10T21:35:00.055 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:00.048+0000 7f1982200640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f197c152460 0x7f197c145bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:00.055 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:00.048+0000 7f1982200640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f197c152460 0x7f197c145bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43130/0 (socket says 172.21.15.17:43130) 2024-09-10T21:35:00.055 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:00.048+0000 7f1982200640 1 -- 172.21.15.17:0/1385971733 learned_addr learned my addr 172.21.15.17:0/1385971733 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:35:00.055 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:00.048+0000 7f1981800640 1 --2- 172.21.15.17:0/1385971733 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f197c06ec30 0x7f197c0c5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:00.055 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:00.048+0000 7f1980e00640 1 --2- 172.21.15.17:0/1385971733 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f197c0c77e0 0x7f197c145680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:00.055 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:00.048+0000 7f1982200640 1 -- 172.21.15.17:0/1385971733 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f197c0c77e0 msgr2=0x7f197c145680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:00.055 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:00.048+0000 7f1982200640 1 --2- 172.21.15.17:0/1385971733 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f197c0c77e0 0x7f197c145680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:00.055 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:00.048+0000 7f1982200640 1 -- 172.21.15.17:0/1385971733 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f197c06ec30 msgr2=0x7f197c0c5d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:00.055 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:00.048+0000 7f1982200640 1 --2- 172.21.15.17:0/1385971733 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f197c06ec30 0x7f197c0c5d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:00.055 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:00.048+0000 7f1982200640 1 -- 172.21.15.17:0/1385971733 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f197c0c6240 con 0x7f197c152460 2024-09-10T21:35:00.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:00.048+0000 7f1982200640 1 --2- 172.21.15.17:0/1385971733 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f197c152460 0x7f197c145bc0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f197400eaf0 tx=0x7f197400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:00.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:00.048+0000 7f1979600640 1 -- 172.21.15.17:0/1385971733 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1974004b30 con 0x7f197c152460 2024-09-10T21:35:00.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:00.048+0000 7f1979600640 1 -- 172.21.15.17:0/1385971733 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1974004cf0 con 0x7f197c152460 2024-09-10T21:35:00.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:00.048+0000 7f1982c00640 1 -- 172.21.15.17:0/1385971733 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f197c0c46d0 con 0x7f197c152460 2024-09-10T21:35:00.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:00.048+0000 7f1979600640 1 -- 172.21.15.17:0/1385971733 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1974005280 con 0x7f197c152460 2024-09-10T21:35:00.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:00.048+0000 7f1982c00640 1 -- 172.21.15.17:0/1385971733 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f197c155810 con 0x7f197c152460 2024-09-10T21:35:00.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:00.048+0000 7f1982c00640 1 --2- 172.21.15.17:0/1385971733 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f197c0cc510 0x7f197c1f4620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:00.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:00.048+0000 7f1982c00640 1 -- 172.21.15.17:0/1385971733 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f197c069ef0 con 0x7f197c0cc510 2024-09-10T21:35:00.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:00.052+0000 7f1981800640 1 --2- 172.21.15.17:0/1385971733 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f197c0cc510 0x7f197c1f4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:00.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:00.052+0000 7f1981800640 1 -- 172.21.15.17:0/1385971733 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f197c155c10 con 0x7f197c0cc510 2024-09-10T21:35:00.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:00.052+0000 7f1981800640 1 --2- 172.21.15.17:0/1385971733 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f197c0cc510 0x7f197c1f4620 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f197c0c74a0 tx=0x7f1970005ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:00.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:00.052+0000 7f1979600640 1 -- 172.21.15.17:0/1385971733 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f1974005420 con 0x7f197c152460 2024-09-10T21:35:00.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:00.052+0000 7f1979600640 1 --2- 172.21.15.17:0/1385971733 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f195c072640 0x7f195c074b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:00.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:00.052+0000 7f1979600640 1 -- 172.21.15.17:0/1385971733 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f1974094250 con 0x7f197c152460 2024-09-10T21:35:00.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:00.052+0000 7f1979600640 1 -- 172.21.15.17:0/1385971733 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19700be020 con 0x7f197c0cc510 2024-09-10T21:35:00.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:00.052+0000 7f1979600640 1 -- 172.21.15.17:0/1385971733 <== mon.2 v2:172.21.15.31:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f1970003800 con 0x7f197c0cc510 2024-09-10T21:35:00.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:00.052+0000 7f1979600640 1 -- 172.21.15.17:0/1385971733 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f197c0cc510 msgr2=0x7f197c1f4620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:00.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:00.052+0000 7f1979600640 1 --2- 172.21.15.17:0/1385971733 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f197c0cc510 0x7f197c1f4620 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f197c0c74a0 tx=0x7f1970005ca0 comp rx=0 tx=0).stop 2024-09-10T21:35:00.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:00.052+0000 7f1979600640 1 -- 172.21.15.17:0/1385971733 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f197c0cc510 msgr2=0x7f197c1f4620 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:35:00.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:00.052+0000 7f1979600640 1 --2- 172.21.15.17:0/1385971733 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f197c0cc510 0x7f197c1f4620 secure :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f197c0c74a0 tx=0x7f1970005ca0 comp rx=0 tx=0).stop 2024-09-10T21:35:00.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:00.052+0000 7f1980e00640 1 --2- 172.21.15.17:0/1385971733 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f195c072640 0x7f195c074b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:00.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:00.056+0000 7f1980e00640 1 --2- 172.21.15.17:0/1385971733 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f195c072640 0x7f195c074b00 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f197c0c53c0 tx=0x7f1968001420 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:00.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:00.060+0000 7f1982c00640 1 --2- 172.21.15.17:0/1385971733 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f197c0c4a00 0x7f197c1f4b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:00.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:00.060+0000 7f1982c00640 1 -- 172.21.15.17:0/1385971733 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f197c1541c0 con 0x7f197c0c4a00 2024-09-10T21:35:00.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:00.064+0000 7f1981800640 1 --2- 172.21.15.17:0/1385971733 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f197c0c4a00 0x7f197c1f4b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:00.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:00.064+0000 7f1981800640 1 -- 172.21.15.17:0/1385971733 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f197c1627c0 con 0x7f197c0c4a00 2024-09-10T21:35:00.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:00.064+0000 7f1981800640 1 --2- 172.21.15.17:0/1385971733 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f197c0c4a00 0x7f197c1f4b60 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f197001a330 tx=0x7f19700091e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:00.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:00.064+0000 7f1979600640 1 -- 172.21.15.17:0/1385971733 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19700be030 con 0x7f197c0c4a00 2024-09-10T21:35:00.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:00.064+0000 7f1979600640 1 -- 172.21.15.17:0/1385971733 <== mon.2 v2:172.21.15.31:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4876 (secure 0 0 0) 0x7f19700c6070 con 0x7f197c0c4a00 2024-09-10T21:35:00.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:00.064+0000 7f1979600640 1 -- 172.21.15.17:0/1385971733 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f197c0c4a00 msgr2=0x7f197c1f4b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:00.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:00.064+0000 7f1979600640 1 --2- 172.21.15.17:0/1385971733 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f197c0c4a00 0x7f197c1f4b60 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f197001a330 tx=0x7f19700091e0 comp rx=0 tx=0).stop 2024-09-10T21:35:00.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:00.064+0000 7f1979600640 1 -- 172.21.15.17:0/1385971733 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f197c0c4a00 msgr2=0x7f197c1f4b60 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:35:00.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:00.064+0000 7f1979600640 1 --2- 172.21.15.17:0/1385971733 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f197c0c4a00 0x7f197c1f4b60 secure :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f197001a330 tx=0x7f19700091e0 comp rx=0 tx=0).stop 2024-09-10T21:35:00.070 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-10T21:35:00.070 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:35:00.070 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:35:00.070 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-10T21:35:00.070 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 86, 2024-09-10T21:35:00.070 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 232373, 2024-09-10T21:35:00.071 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-10T21:35:00.071 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-10T21:35:00.071 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-10T21:35:00.071 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-10T21:35:00.071 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:35:00.071 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 14, 2024-09-10T21:35:00.071 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:35:00.071 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-10T21:35:00.071 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-10T21:35:00.071 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:35:00.071 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:35:00.071 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:35:00.071 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:35:00.071 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:35:00.072 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:35:00.072 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:35:00.072 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:35:00.072 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:35:00.072 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:35:00.072 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:35:00.072 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:35:00.072 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-10T21:35:00.072 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-10T21:35:00.072 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:35:00.072 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:35:00.072 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:35:00.072 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:35:00.073 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:35:00.073 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:35:00.073 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:35:00.073 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:35:00.073 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:35:00.073 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:35:00.073 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:35:00.073 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:35:00.073 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:35:00.073 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-10T21:35:00.073 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-10T21:35:00.073 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-10T21:35:00.073 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-10T21:35:00.074 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-10T21:35:00.074 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "5e2f023f-d6a6-49f4-a256-f4807dce3739", 2024-09-10T21:35:00.074 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:35:00.074 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:35:00.074 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-10T21:35:00.074 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-10T21:35:00.074 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-10T21:35:00.074 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-10T21:35:00.074 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-10T21:35:00.074 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-10T21:35:00.074 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-10T21:35:00.074 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:35:00.074 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-10T21:35:00.075 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:35:00.075 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:35:00.075 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:35:00.075 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:35:00.075 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:35:00.075 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:35:00.075 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:35:00.075 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:35:00.075 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:35:00.075 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:35:00.075 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:35:00.075 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:35:00.075 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-10T21:35:00.075 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:35:00.076 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-10T21:35:00.076 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:35:00.076 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:35:00.076 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:35:00.076 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:35:00.076 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:35:00.076 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:35:00.076 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:35:00.076 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-10T21:35:00.076 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:35:00.076 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:35:00.076 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:35:00.076 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:35:00.077 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-10T21:35:00.077 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:35:00.077 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:35:00.077 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:35:00.077 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:35:00.077 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-10T21:35:00.077 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-10T21:35:00.077 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:35:00.077 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:35:00.077 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:35:00.077 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:35:00.077 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:35:00.078 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:35:00.078 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:35:00.078 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:35:00.078 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:35:00.078 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:35:00.078 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:35:00.078 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3300", 2024-09-10T21:35:00.078 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:35:00.078 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:35:00.078 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:35:00.078 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:35:00.079 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789", 2024-09-10T21:35:00.079 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:35:00.079 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:35:00.079 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:35:00.079 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:35:00.079 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789/0", 2024-09-10T21:35:00.079 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6789/0", 2024-09-10T21:35:00.079 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:35:00.079 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:35:00.079 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:35:00.079 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:35:00.079 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:35:00.079 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:35:00.080 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:35:00.080 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:35:00.080 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:35:00.080 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:35:00.080 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:35:00.080 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3301", 2024-09-10T21:35:00.080 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:35:00.080 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:35:00.080 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:35:00.080 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:35:00.081 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790", 2024-09-10T21:35:00.081 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:35:00.081 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:35:00.081 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:35:00.081 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:35:00.081 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790/0", 2024-09-10T21:35:00.081 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6790/0", 2024-09-10T21:35:00.081 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:35:00.081 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:35:00.081 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:35:00.081 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:35:00.081 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:35:00.082 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:35:00.082 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-10T21:35:00.082 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-10T21:35:00.082 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:35:00.082 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:35:00.082 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:35:00.082 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:35:00.082 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:35:00.082 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:35:00.082 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": [ 2024-09-10T21:35:00.082 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:35:00.082 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:35:00.083 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:35:00.083 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 3 2024-09-10T21:35:00.083 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:35:00.083 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:35:00.083 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-10T21:35:00.083 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:35:00.083 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:35:00.083 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:35:00.083 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 2 2024-09-10T21:35:00.083 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:35:00.083 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:35:00.083 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-10T21:35:00.083 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:35:00.083 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:35:00.084 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:35:00.084 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 2 2024-09-10T21:35:00.084 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:35:00.084 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:35:00.084 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:35:00.084 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-10T21:35:00.084 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-10T21:35:00.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:00.064+0000 7f1982c00640 1 -- 172.21.15.17:0/1385971733 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f195c072640 msgr2=0x7f195c074b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:00.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:00.064+0000 7f1982c00640 1 --2- 172.21.15.17:0/1385971733 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f195c072640 0x7f195c074b00 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f197c0c53c0 tx=0x7f1968001420 comp rx=0 tx=0).stop 2024-09-10T21:35:00.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:00.064+0000 7f1982c00640 1 -- 172.21.15.17:0/1385971733 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f197c152460 msgr2=0x7f197c145bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:00.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:00.064+0000 7f1982c00640 1 --2- 172.21.15.17:0/1385971733 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f197c152460 0x7f197c145bc0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f197400eaf0 tx=0x7f197400c2d0 comp rx=0 tx=0).stop 2024-09-10T21:35:00.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:00.064+0000 7f1982200640 1 -- 172.21.15.17:0/1385971733 reap_dead start 2024-09-10T21:35:00.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:00.064+0000 7f1982200640 1 -- 172.21.15.17:0/1385971733 reap_dead start 2024-09-10T21:35:00.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:00.064+0000 7f1982c00640 1 -- 172.21.15.17:0/1385971733 shutdown_connections 2024-09-10T21:35:00.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:00.064+0000 7f1982c00640 1 -- 172.21.15.17:0/1385971733 >> 172.21.15.17:0/1385971733 conn(0x7f197c0cd690 msgr2=0x7f197c0c8080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:00.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:00.064+0000 7f1982c00640 1 -- 172.21.15.17:0/1385971733 shutdown_connections 2024-09-10T21:35:00.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:00.064+0000 7f1982c00640 1 -- 172.21.15.17:0/1385971733 wait complete. 2024-09-10T21:35:00.093 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-10T21:35:00.093 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['c'] 2024-09-10T21:35:00.093 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2024-09-10T21:35:00.093 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.c 2024-09-10T21:35:00.093 INFO:tasks.mon_thrash.mon_thrasher:killing mon.c 2024-09-10T21:35:00.093 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-10T21:35:00.093 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:35:00.115 INFO:tasks.ceph.mon.c:Stopped 2024-09-10T21:35:00.115 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2024-09-10T21:35:00.115 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-10T21:35:00.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:00.276+0000 7fe56b200640 1 -- 172.21.15.17:0/2826109788 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe550004d50 msgr2=0x7fe5500051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:00.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:00.276+0000 7fe56b200640 1 --2- 172.21.15.17:0/2826109788 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe550004d50 0x7fe5500051d0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fe5640013e0 tx=0x7fe56401c950 comp rx=0 tx=0).stop 2024-09-10T21:35:00.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:00.276+0000 7fe56a800640 1 -- 172.21.15.17:0/2826109788 reap_dead start 2024-09-10T21:35:00.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:00.276+0000 7fe56b200640 1 -- 172.21.15.17:0/2826109788 shutdown_connections 2024-09-10T21:35:00.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:00.276+0000 7fe56b200640 1 -- 172.21.15.17:0/2826109788 >> 172.21.15.17:0/2826109788 conn(0x7fe56c070ca0 msgr2=0x7fe56c0710b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:00.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:00.276+0000 7fe56b200640 1 -- 172.21.15.17:0/2826109788 shutdown_connections 2024-09-10T21:35:00.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:00.276+0000 7fe56b200640 1 -- 172.21.15.17:0/2826109788 wait complete. 2024-09-10T21:35:00.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:00.276+0000 7fe56b200640 1 Processor -- start 2024-09-10T21:35:00.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:00.276+0000 7fe56b200640 1 -- start start 2024-09-10T21:35:00.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:00.280+0000 7fe56b200640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe550004d50 0x7fe56c1557c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:00.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:00.280+0000 7fe56b200640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe56c1f0870 0x7fe56c14fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:00.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:00.280+0000 7fe56b200640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe56c1f4550 0x7fe56c0c56a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:00.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:00.280+0000 7fe56b200640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fe56c06f500 con 0x7fe56c1f4550 2024-09-10T21:35:00.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:00.280+0000 7fe56b200640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fe56c06f220 con 0x7fe550004d50 2024-09-10T21:35:00.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:00.280+0000 7fe56b200640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7fe56c06ed40 con 0x7fe56c1f0870 2024-09-10T21:35:00.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:00.280+0000 7fe569400640 1 -- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe56c1f0870 msgr2=0x7fe56c14fd90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:3301/0 2024-09-10T21:35:00.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:00.280+0000 7fe569400640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe56c1f0870 0x7fe56c14fd90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:35:00.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:00.280+0000 7fe56a800640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe56c1f4550 0x7fe56c0c56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:00.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:00.280+0000 7fe56a800640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe56c1f4550 0x7fe56c0c56a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43164/0 (socket says 172.21.15.17:43164) 2024-09-10T21:35:00.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:00.280+0000 7fe56a800640 1 -- 172.21.15.17:0/2929648364 learned_addr learned my addr 172.21.15.17:0/2929648364 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:35:00.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:00.280+0000 7fe569e00640 1 --2- 172.21.15.17:0/2929648364 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe550004d50 0x7fe56c1557c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:00.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:00.280+0000 7fe56a800640 1 -- 172.21.15.17:0/2929648364 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe56c1f0870 msgr2=0x7fe56c14fd90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:35:00.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:00.280+0000 7fe56a800640 1 --2- 172.21.15.17:0/2929648364 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe56c1f0870 0x7fe56c14fd90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:00.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:00.280+0000 7fe56a800640 1 -- 172.21.15.17:0/2929648364 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe550004d50 msgr2=0x7fe56c1557c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:00.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:00.280+0000 7fe56a800640 1 --2- 172.21.15.17:0/2929648364 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe550004d50 0x7fe56c1557c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:00.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:00.280+0000 7fe56a800640 1 -- 172.21.15.17:0/2929648364 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe56c157350 con 0x7fe56c1f4550 2024-09-10T21:35:00.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:00.280+0000 7fe56a800640 1 --2- 172.21.15.17:0/2929648364 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe56c1f4550 0x7fe56c0c56a0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fe56401d1f0 tx=0x7fe5640ad670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:00.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:00.280+0000 7fe562000640 1 -- 172.21.15.17:0/2929648364 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5640b2040 con 0x7fe56c1f4550 2024-09-10T21:35:00.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:00.280+0000 7fe562000640 1 -- 172.21.15.17:0/2929648364 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe564004bd0 con 0x7fe56c1f4550 2024-09-10T21:35:00.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:00.280+0000 7fe562000640 1 -- 172.21.15.17:0/2929648364 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5640094a0 con 0x7fe56c1f4550 2024-09-10T21:35:00.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:00.280+0000 7fe56b200640 1 -- 172.21.15.17:0/2929648364 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe56c153630 con 0x7fe56c1f4550 2024-09-10T21:35:00.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:00.280+0000 7fe56b200640 1 -- 172.21.15.17:0/2929648364 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe56c1f1070 con 0x7fe56c1f4550 2024-09-10T21:35:00.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:00.280+0000 7fe56b200640 1 -- 172.21.15.17:0/2929648364 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe56c158310 con 0x7fe56c1f4550 2024-09-10T21:35:00.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:00.284+0000 7fe562000640 1 -- 172.21.15.17:0/2929648364 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fe5640b5070 con 0x7fe56c1f4550 2024-09-10T21:35:00.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:00.284+0000 7fe562000640 1 --2- 172.21.15.17:0/2929648364 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fe54c0725c0 0x7fe54c074a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:00.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:00.284+0000 7fe562000640 1 -- 172.21.15.17:0/2929648364 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fe5640bd070 con 0x7fe56c1f4550 2024-09-10T21:35:00.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:00.284+0000 7fe569e00640 1 --2- 172.21.15.17:0/2929648364 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fe54c0725c0 0x7fe54c074a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:00.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:00.284+0000 7fe569e00640 1 --2- 172.21.15.17:0/2929648364 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fe54c0725c0 0x7fe54c074a80 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fe55c001250 tx=0x7fe55c0067b0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:00.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:00.288+0000 7fe562000640 1 -- 172.21.15.17:0/2929648364 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fe56c157350 con 0x7fe56c1f4550 2024-09-10T21:35:00.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:00.560+0000 7fe56b200640 1 -- 172.21.15.17:0/2929648364 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fe56c0c7ae0 con 0x7fe56c1f4550 2024-09-10T21:35:00.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:00.560+0000 7fe562000640 1 -- 172.21.15.17:0/2929648364 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1557 (secure 0 0 0) 0x7fe564016660 con 0x7fe56c1f4550 2024-09-10T21:35:00.565 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":86,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":14,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5e2f023f-d6a6-49f4-a256-f4807dce3739","modified":"2024-09-10T21:23:46.892130Z","created":"2024-09-10T21:23:46.892130Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3301","nonce":0},{"type":"v1","addr":"172.21.15.31:6790","nonce":0}]},"addr":"172.21.15.31:6790/0","public_addr":"172.21.15.31:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-10T21:35:00.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:00.560+0000 7fe56b200640 1 -- 172.21.15.17:0/2929648364 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fe54c0725c0 msgr2=0x7fe54c074a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:00.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:00.560+0000 7fe56b200640 1 --2- 172.21.15.17:0/2929648364 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fe54c0725c0 0x7fe54c074a80 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fe55c001250 tx=0x7fe55c0067b0 comp rx=0 tx=0).stop 2024-09-10T21:35:00.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:00.560+0000 7fe56b200640 1 -- 172.21.15.17:0/2929648364 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe56c1f4550 msgr2=0x7fe56c0c56a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:00.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:00.560+0000 7fe56b200640 1 --2- 172.21.15.17:0/2929648364 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe56c1f4550 0x7fe56c0c56a0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fe56401d1f0 tx=0x7fe5640ad670 comp rx=0 tx=0).stop 2024-09-10T21:35:00.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:00.560+0000 7fe56b200640 1 -- 172.21.15.17:0/2929648364 shutdown_connections 2024-09-10T21:35:00.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:00.560+0000 7fe56b200640 1 --2- 172.21.15.17:0/2929648364 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fe54c0725c0 0x7fe54c074a80 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:00.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:00.560+0000 7fe56b200640 1 --2- 172.21.15.17:0/2929648364 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe56c1f4550 0x7fe56c0c56a0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:00.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:00.560+0000 7fe56b200640 1 --2- 172.21.15.17:0/2929648364 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe56c1f0870 0x7fe56c14fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:00.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:00.560+0000 7fe56b200640 1 --2- 172.21.15.17:0/2929648364 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe550004d50 0x7fe56c1557c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:00.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:00.560+0000 7fe56b200640 1 -- 172.21.15.17:0/2929648364 >> 172.21.15.17:0/2929648364 conn(0x7fe56c070ca0 msgr2=0x7fe56c06cc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:00.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:00.560+0000 7fe56b200640 1 -- 172.21.15.17:0/2929648364 shutdown_connections 2024-09-10T21:35:00.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:00.560+0000 7fe56b200640 1 -- 172.21.15.17:0/2929648364 wait complete. 2024-09-10T21:35:03.588 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-10T21:35:03.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:03.744+0000 7f4f75400640 1 -- 172.21.15.17:0/173921964 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4f700717c0 msgr2=0x7f4f7006cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:03.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:03.744+0000 7f4f75400640 1 --2- 172.21.15.17:0/173921964 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4f700717c0 0x7f4f7006cc20 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f4f5c009900 tx=0x7f4f5c01c900 comp rx=0 tx=0).stop 2024-09-10T21:35:03.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:03.744+0000 7f4f75400640 1 -- 172.21.15.17:0/173921964 shutdown_connections 2024-09-10T21:35:03.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:03.744+0000 7f4f75400640 1 --2- 172.21.15.17:0/173921964 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4f700717c0 0x7f4f7006cc20 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:03.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:03.744+0000 7f4f75400640 1 --2- 172.21.15.17:0/173921964 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f4f70154920 0x7f4f7015b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:03.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:03.744+0000 7f4f75400640 1 -- 172.21.15.17:0/173921964 >> 172.21.15.17:0/173921964 conn(0x7f4f70070ca0 msgr2=0x7f4f700710b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:03.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:03.744+0000 7f4f75400640 1 -- 172.21.15.17:0/173921964 shutdown_connections 2024-09-10T21:35:03.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:03.744+0000 7f4f75400640 1 -- 172.21.15.17:0/173921964 wait complete. 2024-09-10T21:35:03.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:03.748+0000 7f4f75400640 1 Processor -- start 2024-09-10T21:35:03.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:03.748+0000 7f4f75400640 1 -- start start 2024-09-10T21:35:03.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:03.748+0000 7f4f75400640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f4f7006d160 0x7f4f701f4220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:03.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:03.748+0000 7f4f75400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4f700717c0 0x7f4f70146290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:03.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:03.748+0000 7f4f75400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4f70154920 0x7f4f700cbde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:03.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:03.748+0000 7f4f75400640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f4f7015dbd0 con 0x7f4f70154920 2024-09-10T21:35:03.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:03.748+0000 7f4f75400640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f4f70154d00 con 0x7f4f700717c0 2024-09-10T21:35:03.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:03.748+0000 7f4f75400640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f4f7015dd50 con 0x7f4f7006d160 2024-09-10T21:35:03.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:03.748+0000 7f4f6fe00640 1 -- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f4f7006d160 msgr2=0x7f4f701f4220 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:3301/0 2024-09-10T21:35:03.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:03.748+0000 7f4f74a00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4f70154920 0x7f4f700cbde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:03.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:03.748+0000 7f4f6fe00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f4f7006d160 0x7f4f701f4220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:35:03.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:03.748+0000 7f4f74a00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4f70154920 0x7f4f700cbde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43176/0 (socket says 172.21.15.17:43176) 2024-09-10T21:35:03.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:03.748+0000 7f4f74a00640 1 -- 172.21.15.17:0/1813689692 learned_addr learned my addr 172.21.15.17:0/1813689692 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:35:03.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:03.748+0000 7f4f6f400640 1 --2- 172.21.15.17:0/1813689692 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4f700717c0 0x7f4f70146290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:03.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:03.748+0000 7f4f74a00640 1 -- 172.21.15.17:0/1813689692 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f4f7006d160 msgr2=0x7f4f701f4220 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:35:03.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:03.748+0000 7f4f74a00640 1 --2- 172.21.15.17:0/1813689692 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f4f7006d160 0x7f4f701f4220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:03.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:03.748+0000 7f4f74a00640 1 -- 172.21.15.17:0/1813689692 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4f700717c0 msgr2=0x7f4f70146290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:03.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:03.748+0000 7f4f74a00640 1 --2- 172.21.15.17:0/1813689692 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4f700717c0 0x7f4f70146290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:03.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:03.748+0000 7f4f74a00640 1 -- 172.21.15.17:0/1813689692 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f70075150 con 0x7f4f70154920 2024-09-10T21:35:03.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:03.748+0000 7f4f74a00640 1 --2- 172.21.15.17:0/1813689692 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4f70154920 0x7f4f700cbde0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f4f6800c7a0 tx=0x7f4f6800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:03.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:03.748+0000 7f4f67e00640 1 -- 172.21.15.17:0/1813689692 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f68003bd0 con 0x7f4f70154920 2024-09-10T21:35:03.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:03.748+0000 7f4f67e00640 1 -- 172.21.15.17:0/1813689692 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4f68003d90 con 0x7f4f70154920 2024-09-10T21:35:03.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:03.748+0000 7f4f67e00640 1 -- 172.21.15.17:0/1813689692 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f6800aab0 con 0x7f4f70154920 2024-09-10T21:35:03.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:03.752+0000 7f4f75400640 1 -- 172.21.15.17:0/1813689692 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f700748b0 con 0x7f4f70154920 2024-09-10T21:35:03.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:03.752+0000 7f4f75400640 1 -- 172.21.15.17:0/1813689692 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f70074e70 con 0x7f4f70154920 2024-09-10T21:35:03.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:03.752+0000 7f4f75400640 1 -- 172.21.15.17:0/1813689692 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f7006cc20 con 0x7f4f70154920 2024-09-10T21:35:03.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:03.752+0000 7f4f67e00640 1 -- 172.21.15.17:0/1813689692 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f4f6800ac50 con 0x7f4f70154920 2024-09-10T21:35:03.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:03.752+0000 7f4f67e00640 1 --2- 172.21.15.17:0/1813689692 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f4f50072590 0x7f4f50074a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:03.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:03.752+0000 7f4f67e00640 1 -- 172.21.15.17:0/1813689692 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f4f680934e0 con 0x7f4f70154920 2024-09-10T21:35:03.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:03.752+0000 7f4f6fe00640 1 --2- 172.21.15.17:0/1813689692 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f4f50072590 0x7f4f50074a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:03.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:03.756+0000 7f4f6fe00640 1 --2- 172.21.15.17:0/1813689692 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f4f50072590 0x7f4f50074a50 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f4f60001060 tx=0x7f4f60006d70 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:03.764 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:03.760+0000 7f4f67e00640 1 -- 172.21.15.17:0/1813689692 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f4f6805e730 con 0x7f4f70154920 2024-09-10T21:35:04.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:04.068+0000 7f4f75400640 1 -- 172.21.15.17:0/1813689692 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f4f70071ce0 con 0x7f4f70154920 2024-09-10T21:35:04.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:04.068+0000 7f4f67e00640 1 -- 172.21.15.17:0/1813689692 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1557 (secure 0 0 0) 0x7f4f6805e030 con 0x7f4f70154920 2024-09-10T21:35:04.074 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":86,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":18,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5e2f023f-d6a6-49f4-a256-f4807dce3739","modified":"2024-09-10T21:23:46.892130Z","created":"2024-09-10T21:23:46.892130Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3301","nonce":0},{"type":"v1","addr":"172.21.15.31:6790","nonce":0}]},"addr":"172.21.15.31:6790/0","public_addr":"172.21.15.31:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-10T21:35:04.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:04.072+0000 7f4f75400640 1 -- 172.21.15.17:0/1813689692 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f4f50072590 msgr2=0x7f4f50074a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:04.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:04.072+0000 7f4f75400640 1 --2- 172.21.15.17:0/1813689692 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f4f50072590 0x7f4f50074a50 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f4f60001060 tx=0x7f4f60006d70 comp rx=0 tx=0).stop 2024-09-10T21:35:04.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:04.072+0000 7f4f75400640 1 -- 172.21.15.17:0/1813689692 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4f70154920 msgr2=0x7f4f700cbde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:04.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:04.072+0000 7f4f75400640 1 --2- 172.21.15.17:0/1813689692 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4f70154920 0x7f4f700cbde0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f4f6800c7a0 tx=0x7f4f6800cc60 comp rx=0 tx=0).stop 2024-09-10T21:35:04.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:04.072+0000 7f4f75400640 1 -- 172.21.15.17:0/1813689692 shutdown_connections 2024-09-10T21:35:04.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:04.072+0000 7f4f75400640 1 --2- 172.21.15.17:0/1813689692 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f4f50072590 0x7f4f50074a50 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:04.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:04.072+0000 7f4f75400640 1 --2- 172.21.15.17:0/1813689692 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4f70154920 0x7f4f700cbde0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:04.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:04.072+0000 7f4f75400640 1 --2- 172.21.15.17:0/1813689692 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4f700717c0 0x7f4f70146290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:04.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:04.072+0000 7f4f75400640 1 --2- 172.21.15.17:0/1813689692 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f4f7006d160 0x7f4f701f4220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:04.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:04.072+0000 7f4f75400640 1 -- 172.21.15.17:0/1813689692 >> 172.21.15.17:0/1813689692 conn(0x7f4f70070ca0 msgr2=0x7f4f70071080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:04.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:04.072+0000 7f4f75400640 1 -- 172.21.15.17:0/1813689692 shutdown_connections 2024-09-10T21:35:04.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:04.072+0000 7f4f75400640 1 -- 172.21.15.17:0/1813689692 wait complete. 2024-09-10T21:35:07.100 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-10T21:35:07.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:07.268+0000 7fe3e9200640 1 -- 172.21.15.17:0/3144903640 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe3e40cb870 msgr2=0x7fe3e40d0000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:07.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:07.268+0000 7fe3e9200640 1 --2- 172.21.15.17:0/3144903640 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe3e40cb870 0x7fe3e40d0000 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fe3e4069550 tx=0x7fe3dc01f5a0 comp rx=0 tx=0).stop 2024-09-10T21:35:07.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:07.268+0000 7fe3e9200640 1 -- 172.21.15.17:0/3144903640 shutdown_connections 2024-09-10T21:35:07.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:07.268+0000 7fe3e9200640 1 --2- 172.21.15.17:0/3144903640 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe3e40cb870 0x7fe3e40d0000 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:07.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:07.268+0000 7fe3e9200640 1 --2- 172.21.15.17:0/3144903640 >> v2:172.21.15.31:3300/0 conn(0x7fe3e40cc5d0 0x7fe3e415a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:07.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:07.268+0000 7fe3e9200640 1 --2- 172.21.15.17:0/3144903640 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe3e40b4580 0x7fe3e40b4960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:07.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:07.268+0000 7fe3e9200640 1 -- 172.21.15.17:0/3144903640 >> 172.21.15.17:0/3144903640 conn(0x7fe3e406d700 msgr2=0x7fe3e4068990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:07.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:07.268+0000 7fe3e9200640 1 -- 172.21.15.17:0/3144903640 shutdown_connections 2024-09-10T21:35:07.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:07.268+0000 7fe3e9200640 1 -- 172.21.15.17:0/3144903640 wait complete. 2024-09-10T21:35:07.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:07.268+0000 7fe3e9200640 1 Processor -- start 2024-09-10T21:35:07.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:07.268+0000 7fe3e9200640 1 -- start start 2024-09-10T21:35:07.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:07.272+0000 7fe3e9200640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe3e40b4580 0x7fe3e41e39e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:07.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:07.272+0000 7fe3e9200640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe3e40cb870 0x7fe3e41e3f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:07.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:07.272+0000 7fe3e9200640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe3e40cc5d0 0x7fe3e40c5e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:07.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:07.272+0000 7fe3e9200640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fe3e415e3c0 con 0x7fe3e40b4580 2024-09-10T21:35:07.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:07.272+0000 7fe3e9200640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fe3e415e0e0 con 0x7fe3e40cc5d0 2024-09-10T21:35:07.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:07.272+0000 7fe3e9200640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7fe3e415e6a0 con 0x7fe3e40cb870 2024-09-10T21:35:07.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:07.272+0000 7fe3e2a00640 1 -- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe3e40cb870 msgr2=0x7fe3e41e3f20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:3301/0 2024-09-10T21:35:07.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:07.272+0000 7fe3e2a00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe3e40cb870 0x7fe3e41e3f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:35:07.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:07.272+0000 7fe3e3400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe3e40b4580 0x7fe3e41e39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:07.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:07.272+0000 7fe3e3400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe3e40b4580 0x7fe3e41e39e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:60446/0 (socket says 172.21.15.17:60446) 2024-09-10T21:35:07.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:07.272+0000 7fe3e3400640 1 -- 172.21.15.17:0/4000491982 learned_addr learned my addr 172.21.15.17:0/4000491982 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:35:07.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:07.272+0000 7fe3e3e00640 1 --2- 172.21.15.17:0/4000491982 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe3e40cc5d0 0x7fe3e40c5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:07.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:07.272+0000 7fe3e3400640 1 -- 172.21.15.17:0/4000491982 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe3e40cb870 msgr2=0x7fe3e41e3f20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:35:07.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:07.272+0000 7fe3e3400640 1 --2- 172.21.15.17:0/4000491982 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe3e40cb870 0x7fe3e41e3f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:07.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:07.272+0000 7fe3e3400640 1 -- 172.21.15.17:0/4000491982 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe3e40cc5d0 msgr2=0x7fe3e40c5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:07.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:07.272+0000 7fe3e3400640 1 --2- 172.21.15.17:0/4000491982 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe3e40cc5d0 0x7fe3e40c5e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:07.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:07.272+0000 7fe3e3400640 1 -- 172.21.15.17:0/4000491982 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3e40c39a0 con 0x7fe3e40b4580 2024-09-10T21:35:07.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:07.272+0000 7fe3e3400640 1 --2- 172.21.15.17:0/4000491982 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe3e40b4580 0x7fe3e41e39e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fe3d400b580 tx=0x7fe3d400ba40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:07.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:07.272+0000 7fe3db400640 1 -- 172.21.15.17:0/4000491982 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3d400f850 con 0x7fe3e40b4580 2024-09-10T21:35:07.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:07.272+0000 7fe3db400640 1 -- 172.21.15.17:0/4000491982 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe3d400fb00 con 0x7fe3e40b4580 2024-09-10T21:35:07.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:07.272+0000 7fe3db400640 1 -- 172.21.15.17:0/4000491982 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3d4017400 con 0x7fe3e40b4580 2024-09-10T21:35:07.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:07.272+0000 7fe3e9200640 1 -- 172.21.15.17:0/4000491982 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3e40cfbb0 con 0x7fe3e40b4580 2024-09-10T21:35:07.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:07.272+0000 7fe3e9200640 1 -- 172.21.15.17:0/4000491982 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3e40c3c40 con 0x7fe3e40b4580 2024-09-10T21:35:07.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:07.272+0000 7fe3e9200640 1 -- 172.21.15.17:0/4000491982 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3e40cb350 con 0x7fe3e40b4580 2024-09-10T21:35:07.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:07.276+0000 7fe3db400640 1 -- 172.21.15.17:0/4000491982 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fe3d40203f0 con 0x7fe3e40b4580 2024-09-10T21:35:07.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:07.276+0000 7fe3db400640 1 --2- 172.21.15.17:0/4000491982 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fe3c40725c0 0x7fe3c4074a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:07.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:07.276+0000 7fe3db400640 1 -- 172.21.15.17:0/4000491982 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fe3d40935e0 con 0x7fe3e40b4580 2024-09-10T21:35:07.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:07.276+0000 7fe3e2a00640 1 --2- 172.21.15.17:0/4000491982 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fe3c40725c0 0x7fe3c4074a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:07.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:07.276+0000 7fe3e2a00640 1 --2- 172.21.15.17:0/4000491982 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fe3c40725c0 0x7fe3c4074a80 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fe3e4157c10 tx=0x7fe3d0005f30 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:07.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:07.280+0000 7fe3db400640 1 -- 172.21.15.17:0/4000491982 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fe3d4013030 con 0x7fe3e40b4580 2024-09-10T21:35:07.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:07.588+0000 7fe3e9200640 1 -- 172.21.15.17:0/4000491982 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fe3e4000910 con 0x7fe3e40b4580 2024-09-10T21:35:07.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:07.592+0000 7fe3db400640 1 -- 172.21.15.17:0/4000491982 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1557 (secure 0 0 0) 0x7fe3d405e930 con 0x7fe3e40b4580 2024-09-10T21:35:07.596 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":86,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":21,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5e2f023f-d6a6-49f4-a256-f4807dce3739","modified":"2024-09-10T21:23:46.892130Z","created":"2024-09-10T21:23:46.892130Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3301","nonce":0},{"type":"v1","addr":"172.21.15.31:6790","nonce":0}]},"addr":"172.21.15.31:6790/0","public_addr":"172.21.15.31:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-10T21:35:07.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:07.592+0000 7fe3e9200640 1 -- 172.21.15.17:0/4000491982 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fe3c40725c0 msgr2=0x7fe3c4074a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:07.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:07.592+0000 7fe3e9200640 1 --2- 172.21.15.17:0/4000491982 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fe3c40725c0 0x7fe3c4074a80 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fe3e4157c10 tx=0x7fe3d0005f30 comp rx=0 tx=0).stop 2024-09-10T21:35:07.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:07.592+0000 7fe3e9200640 1 -- 172.21.15.17:0/4000491982 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe3e40b4580 msgr2=0x7fe3e41e39e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:07.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:07.592+0000 7fe3e9200640 1 --2- 172.21.15.17:0/4000491982 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe3e40b4580 0x7fe3e41e39e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fe3d400b580 tx=0x7fe3d400ba40 comp rx=0 tx=0).stop 2024-09-10T21:35:07.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:07.592+0000 7fe3e9200640 1 -- 172.21.15.17:0/4000491982 shutdown_connections 2024-09-10T21:35:07.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:07.592+0000 7fe3e9200640 1 --2- 172.21.15.17:0/4000491982 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fe3c40725c0 0x7fe3c4074a80 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:07.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:07.592+0000 7fe3e9200640 1 --2- 172.21.15.17:0/4000491982 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe3e40cc5d0 0x7fe3e40c5e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:07.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:07.592+0000 7fe3e9200640 1 --2- 172.21.15.17:0/4000491982 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe3e40cb870 0x7fe3e41e3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:07.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:07.592+0000 7fe3e9200640 1 --2- 172.21.15.17:0/4000491982 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe3e40b4580 0x7fe3e41e39e0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:07.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:07.592+0000 7fe3e9200640 1 -- 172.21.15.17:0/4000491982 >> 172.21.15.17:0/4000491982 conn(0x7fe3e406d700 msgr2=0x7fe3e415d380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:07.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:07.592+0000 7fe3e9200640 1 -- 172.21.15.17:0/4000491982 shutdown_connections 2024-09-10T21:35:07.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:07.592+0000 7fe3e9200640 1 -- 172.21.15.17:0/4000491982 wait complete. 2024-09-10T21:35:10.622 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-10T21:35:14.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:14.672+0000 7f48c5000640 1 -- 172.21.15.17:0/28765384 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f48a40039c0 msgr2=0x7f48a4003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:14.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:14.672+0000 7f48c5000640 1 --2- 172.21.15.17:0/28765384 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f48a40039c0 0x7f48a4003dc0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f48b0001060 tx=0x7f48b001d0c0 comp rx=0 tx=0).stop 2024-09-10T21:35:14.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:14.672+0000 7f48c5000640 1 -- 172.21.15.17:0/28765384 shutdown_connections 2024-09-10T21:35:14.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:14.672+0000 7f48c5000640 1 --2- 172.21.15.17:0/28765384 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f48a40052a0 0x7f48a400b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:14.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:14.672+0000 7f48c5000640 1 --2- 172.21.15.17:0/28765384 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48a4004810 0x7f48a4004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:14.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:14.672+0000 7f48c5000640 1 --2- 172.21.15.17:0/28765384 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f48a40039c0 0x7f48a4003dc0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:14.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:14.672+0000 7f48c5000640 1 -- 172.21.15.17:0/28765384 >> 172.21.15.17:0/28765384 conn(0x7f48c006d700 msgr2=0x7f48c0068990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:14.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:14.672+0000 7f48c5000640 1 -- 172.21.15.17:0/28765384 shutdown_connections 2024-09-10T21:35:14.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:14.672+0000 7f48c5000640 1 -- 172.21.15.17:0/28765384 wait complete. 2024-09-10T21:35:14.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:14.676+0000 7f48c5000640 1 Processor -- start 2024-09-10T21:35:14.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:14.676+0000 7f48c5000640 1 -- start start 2024-09-10T21:35:14.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:14.676+0000 7f48c5000640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48a40039c0 0x7f48c01e72f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:14.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:14.676+0000 7f48c5000640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f48a4004810 0x7f48c00c88a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:14.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:14.676+0000 7f48c5000640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f48a40052a0 0x7f48c01f32d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:14.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:14.676+0000 7f48c5000640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f48c015e3c0 con 0x7f48a4004810 2024-09-10T21:35:14.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:14.676+0000 7f48c5000640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f48c015e0e0 con 0x7f48a40039c0 2024-09-10T21:35:14.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:14.676+0000 7f48c5000640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f48c015e6a0 con 0x7f48a40052a0 2024-09-10T21:35:14.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:14.676+0000 7f48bea00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f48a4004810 0x7f48c00c88a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:14.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:14.676+0000 7f48bfe00640 1 -- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f48a40052a0 msgr2=0x7f48c01f32d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:3301/0 2024-09-10T21:35:14.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:14.676+0000 7f48bfe00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f48a40052a0 0x7f48c01f32d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:35:14.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:14.676+0000 7f48bea00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f48a4004810 0x7f48c00c88a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49178/0 (socket says 172.21.15.17:49178) 2024-09-10T21:35:14.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:14.676+0000 7f48bea00640 1 -- 172.21.15.17:0/3579660558 learned_addr learned my addr 172.21.15.17:0/3579660558 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:35:14.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:14.676+0000 7f48bf400640 1 --2- 172.21.15.17:0/3579660558 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48a40039c0 0x7f48c01e72f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:14.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:14.676+0000 7f48bea00640 1 -- 172.21.15.17:0/3579660558 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f48a40052a0 msgr2=0x7f48c01f32d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:35:14.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:14.676+0000 7f48bea00640 1 --2- 172.21.15.17:0/3579660558 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f48a40052a0 0x7f48c01f32d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:14.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:14.676+0000 7f48bea00640 1 -- 172.21.15.17:0/3579660558 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48a40039c0 msgr2=0x7f48c01e72f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:14.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:14.676+0000 7f48bea00640 1 --2- 172.21.15.17:0/3579660558 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48a40039c0 0x7f48c01e72f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:14.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:14.676+0000 7f48bea00640 1 -- 172.21.15.17:0/3579660558 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48c00c9b60 con 0x7f48a4004810 2024-09-10T21:35:14.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:14.676+0000 7f48bea00640 1 --2- 172.21.15.17:0/3579660558 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f48a4004810 0x7f48c00c88a0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f48ac011c50 tx=0x7f48ac0107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:14.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:14.676+0000 7f48b7400640 1 -- 172.21.15.17:0/3579660558 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48ac004030 con 0x7f48a4004810 2024-09-10T21:35:14.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:14.676+0000 7f48b7400640 1 -- 172.21.15.17:0/3579660558 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f48ac0041d0 con 0x7f48a4004810 2024-09-10T21:35:14.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:14.676+0000 7f48c5000640 1 -- 172.21.15.17:0/3579660558 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48c00c98c0 con 0x7f48a4004810 2024-09-10T21:35:14.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:14.676+0000 7f48b7400640 1 -- 172.21.15.17:0/3579660558 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48ac003600 con 0x7f48a4004810 2024-09-10T21:35:14.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:14.676+0000 7f48c5000640 1 -- 172.21.15.17:0/3579660558 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48c00cf720 con 0x7f48a4004810 2024-09-10T21:35:14.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:14.680+0000 7f48c5000640 1 -- 172.21.15.17:0/3579660558 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48c00caec0 con 0x7f48a4004810 2024-09-10T21:35:14.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:14.680+0000 7f48b7400640 1 -- 172.21.15.17:0/3579660558 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f48ac02f060 con 0x7f48a4004810 2024-09-10T21:35:14.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:14.680+0000 7f48b7400640 1 --2- 172.21.15.17:0/3579660558 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f48a0075dd0 0x7f48a0078290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:14.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:14.680+0000 7f48b7400640 1 -- 172.21.15.17:0/3579660558 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f48ac095b20 con 0x7f48a4004810 2024-09-10T21:35:14.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:14.680+0000 7f48bf400640 1 --2- 172.21.15.17:0/3579660558 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f48a0075dd0 0x7f48a0078290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:14.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:14.680+0000 7f48bf400640 1 --2- 172.21.15.17:0/3579660558 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f48a0075dd0 0x7f48a0078290 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f48b00ac000 tx=0x7f48b0019f00 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:14.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:14.684+0000 7f48b7400640 1 -- 172.21.15.17:0/3579660558 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f48ac060d60 con 0x7f48a4004810 2024-09-10T21:35:14.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:14.980+0000 7f48c5000640 1 -- 172.21.15.17:0/3579660558 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f48c0068da0 con 0x7f48a4004810 2024-09-10T21:35:14.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:14.980+0000 7f48b7400640 1 -- 172.21.15.17:0/3579660558 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1550 (secure 0 0 0) 0x7f48ac060660 con 0x7f48a4004810 2024-09-10T21:35:14.986 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":90,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5e2f023f-d6a6-49f4-a256-f4807dce3739","modified":"2024-09-10T21:23:46.892130Z","created":"2024-09-10T21:23:46.892130Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3301","nonce":0},{"type":"v1","addr":"172.21.15.31:6790","nonce":0}]},"addr":"172.21.15.31:6790/0","public_addr":"172.21.15.31:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-10T21:35:14.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:14.980+0000 7f48c5000640 1 -- 172.21.15.17:0/3579660558 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f48a0075dd0 msgr2=0x7f48a0078290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:14.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:14.980+0000 7f48c5000640 1 --2- 172.21.15.17:0/3579660558 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f48a0075dd0 0x7f48a0078290 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f48b00ac000 tx=0x7f48b0019f00 comp rx=0 tx=0).stop 2024-09-10T21:35:14.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:14.980+0000 7f48c5000640 1 -- 172.21.15.17:0/3579660558 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f48a4004810 msgr2=0x7f48c00c88a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:14.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:14.980+0000 7f48c5000640 1 --2- 172.21.15.17:0/3579660558 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f48a4004810 0x7f48c00c88a0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f48ac011c50 tx=0x7f48ac0107a0 comp rx=0 tx=0).stop 2024-09-10T21:35:14.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:14.980+0000 7f48c5000640 1 -- 172.21.15.17:0/3579660558 shutdown_connections 2024-09-10T21:35:14.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:14.984+0000 7f48c5000640 1 --2- 172.21.15.17:0/3579660558 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f48a0075dd0 0x7f48a0078290 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:14.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:14.984+0000 7f48c5000640 1 --2- 172.21.15.17:0/3579660558 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f48a40052a0 0x7f48c01f32d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:14.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:14.984+0000 7f48c5000640 1 --2- 172.21.15.17:0/3579660558 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f48a4004810 0x7f48c00c88a0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:14.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:14.984+0000 7f48c5000640 1 --2- 172.21.15.17:0/3579660558 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48a40039c0 0x7f48c01e72f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:14.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:14.984+0000 7f48c5000640 1 -- 172.21.15.17:0/3579660558 >> 172.21.15.17:0/3579660558 conn(0x7f48c006d700 msgr2=0x7f48c015d380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:14.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:14.984+0000 7f48c5000640 1 -- 172.21.15.17:0/3579660558 shutdown_connections 2024-09-10T21:35:14.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:14.984+0000 7f48c5000640 1 -- 172.21.15.17:0/3579660558 wait complete. 2024-09-10T21:35:15.011 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2024-09-10T21:35:15.011 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon a 2024-09-10T21:35:15.011 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-10T21:35:15.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:15.172+0000 7f955ec00640 1 -- 172.21.15.17:0/3243239201 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f95580b1b50 msgr2=0x7f9558074d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:15.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:15.172+0000 7f955ec00640 1 --2- 172.21.15.17:0/3243239201 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f95580b1b50 0x7f9558074d40 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f9548009840 tx=0x7f954801c840 comp rx=0 tx=0).stop 2024-09-10T21:35:15.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:15.172+0000 7f955ec00640 1 -- 172.21.15.17:0/3243239201 shutdown_connections 2024-09-10T21:35:15.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:15.172+0000 7f955ec00640 1 --2- 172.21.15.17:0/3243239201 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f95580c9080 0x7f9558157260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:15.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:15.172+0000 7f955ec00640 1 --2- 172.21.15.17:0/3243239201 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f95580b1b50 0x7f9558074d40 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:15.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:15.172+0000 7f955ec00640 1 -- 172.21.15.17:0/3243239201 >> 172.21.15.17:0/3243239201 conn(0x7f95580700d0 msgr2=0x7f95580704e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:15.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:15.172+0000 7f955ec00640 1 -- 172.21.15.17:0/3243239201 shutdown_connections 2024-09-10T21:35:15.180 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:15.172+0000 7f955ec00640 1 -- 172.21.15.17:0/3243239201 wait complete. 2024-09-10T21:35:15.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:15.176+0000 7f955ec00640 1 Processor -- start 2024-09-10T21:35:15.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:15.176+0000 7f955ec00640 1 -- start start 2024-09-10T21:35:15.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:15.176+0000 7f955ec00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f95580b1b50 0x7f9558159300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:15.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:15.176+0000 7f955ec00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f95580c8ca0 0x7f955814b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:15.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:15.176+0000 7f955ec00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f95580c9080 0x7f9558160500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:15.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:15.176+0000 7f955ec00640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f9558163e00 con 0x7f95580c9080 2024-09-10T21:35:15.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:15.176+0000 7f955ec00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f95580c87c0 con 0x7f95580c8ca0 2024-09-10T21:35:15.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:15.176+0000 7f955ec00640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f9558163f80 con 0x7f95580b1b50 2024-09-10T21:35:15.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:15.176+0000 7f9557e00640 1 -- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f95580b1b50 msgr2=0x7f9558159300 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:3301/0 2024-09-10T21:35:15.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:15.176+0000 7f9557e00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f95580b1b50 0x7f9558159300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:35:15.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:15.176+0000 7f955cc00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f95580c9080 0x7f9558160500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:15.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:15.176+0000 7f955cc00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f95580c9080 0x7f9558160500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49194/0 (socket says 172.21.15.17:49194) 2024-09-10T21:35:15.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:15.176+0000 7f955cc00640 1 -- 172.21.15.17:0/4114485233 learned_addr learned my addr 172.21.15.17:0/4114485233 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:35:15.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:15.176+0000 7f9557400640 1 --2- 172.21.15.17:0/4114485233 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f95580c8ca0 0x7f955814b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:15.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:15.176+0000 7f955cc00640 1 -- 172.21.15.17:0/4114485233 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f95580b1b50 msgr2=0x7f9558159300 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:35:15.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:15.176+0000 7f955cc00640 1 --2- 172.21.15.17:0/4114485233 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f95580b1b50 0x7f9558159300 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:15.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:15.176+0000 7f955cc00640 1 -- 172.21.15.17:0/4114485233 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f95580c8ca0 msgr2=0x7f955814b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:15.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:15.176+0000 7f955cc00640 1 --2- 172.21.15.17:0/4114485233 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f95580c8ca0 0x7f955814b390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:15.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:15.176+0000 7f955cc00640 1 -- 172.21.15.17:0/4114485233 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95581505f0 con 0x7f95580c9080 2024-09-10T21:35:15.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:15.176+0000 7f955cc00640 1 --2- 172.21.15.17:0/4114485233 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f95580c9080 0x7f9558160500 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f95480b0040 tx=0x7f95480a6590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:15.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:15.180+0000 7f954fe00640 1 -- 172.21.15.17:0/4114485233 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95480be020 con 0x7f95580c9080 2024-09-10T21:35:15.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:15.180+0000 7f954fe00640 1 -- 172.21.15.17:0/4114485233 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9548004330 con 0x7f95580c9080 2024-09-10T21:35:15.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:15.180+0000 7f955ec00640 1 -- 172.21.15.17:0/4114485233 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95581f6e50 con 0x7f95580c9080 2024-09-10T21:35:15.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:15.180+0000 7f954fe00640 1 -- 172.21.15.17:0/4114485233 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9548003840 con 0x7f95580c9080 2024-09-10T21:35:15.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:15.180+0000 7f955ec00640 1 -- 172.21.15.17:0/4114485233 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f955815ffe0 con 0x7f95580c9080 2024-09-10T21:35:15.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:15.180+0000 7f955ec00640 1 --2- 172.21.15.17:0/4114485233 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9558158390 0x7f9558160a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:15.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:15.180+0000 7f955ec00640 1 -- 172.21.15.17:0/4114485233 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f95581508b0 con 0x7f9558158390 2024-09-10T21:35:15.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:15.180+0000 7f9557e00640 1 --2- 172.21.15.17:0/4114485233 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9558158390 0x7f9558160a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:15.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:15.180+0000 7f9557e00640 1 -- 172.21.15.17:0/4114485233 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9558074d40 con 0x7f9558158390 2024-09-10T21:35:15.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:15.180+0000 7f9557e00640 1 --2- 172.21.15.17:0/4114485233 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9558158390 0x7f9558160a40 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f95500039a0 tx=0x7f955000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:15.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:15.180+0000 7f954fe00640 1 -- 172.21.15.17:0/4114485233 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9550017070 con 0x7f9558158390 2024-09-10T21:35:15.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:15.180+0000 7f954fe00640 1 -- 172.21.15.17:0/4114485233 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f955000a7f0 con 0x7f9558158390 2024-09-10T21:35:15.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:15.180+0000 7f954fe00640 1 -- 172.21.15.17:0/4114485233 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9558158390 msgr2=0x7f9558160a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:15.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:15.180+0000 7f954fe00640 1 --2- 172.21.15.17:0/4114485233 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9558158390 0x7f9558160a40 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f95500039a0 tx=0x7f955000cbf0 comp rx=0 tx=0).stop 2024-09-10T21:35:15.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:15.180+0000 7f954fe00640 1 -- 172.21.15.17:0/4114485233 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9558158390 msgr2=0x7f9558160a40 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:35:15.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:15.180+0000 7f954fe00640 1 --2- 172.21.15.17:0/4114485233 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9558158390 0x7f9558160a40 secure :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f95500039a0 tx=0x7f955000cbf0 comp rx=0 tx=0).stop 2024-09-10T21:35:15.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:15.180+0000 7f954fe00640 1 -- 172.21.15.17:0/4114485233 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f9558151600 con 0x7f95580c9080 2024-09-10T21:35:15.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:15.180+0000 7f954fe00640 1 --2- 172.21.15.17:0/4114485233 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f95380720c0 0x7f9538074580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:15.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:15.184+0000 7f954fe00640 1 -- 172.21.15.17:0/4114485233 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f954812e720 con 0x7f95580c9080 2024-09-10T21:35:15.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:15.184+0000 7f9557e00640 1 --2- 172.21.15.17:0/4114485233 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f95380720c0 0x7f9538074580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:15.189 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:15.184+0000 7f9557e00640 1 --2- 172.21.15.17:0/4114485233 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f95380720c0 0x7f9538074580 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f955000cc30 tx=0x7f9550009030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:15.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:15.192+0000 7f955ec00640 1 --2- 172.21.15.17:0/4114485233 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f95581542c0 0x7f95581546d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:15.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:15.192+0000 7f955ec00640 1 -- 172.21.15.17:0/4114485233 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f9558154c10 con 0x7f95581542c0 2024-09-10T21:35:15.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:15.192+0000 7f9557400640 1 --2- 172.21.15.17:0/4114485233 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f95581542c0 0x7f95581546d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:15.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:15.192+0000 7f9557400640 1 -- 172.21.15.17:0/4114485233 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f95581601b0 con 0x7f95581542c0 2024-09-10T21:35:15.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:15.192+0000 7f9557400640 1 --2- 172.21.15.17:0/4114485233 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f95581542c0 0x7f95581546d0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f955806ee10 tx=0x7f954400eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:15.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:15.192+0000 7f954fe00640 1 -- 172.21.15.17:0/4114485233 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9544017070 con 0x7f95581542c0 2024-09-10T21:35:15.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:15.192+0000 7f954fe00640 1 -- 172.21.15.17:0/4114485233 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5032 (secure 0 0 0) 0x7f954400c6f0 con 0x7f95581542c0 2024-09-10T21:35:15.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:15.192+0000 7f954fe00640 1 -- 172.21.15.17:0/4114485233 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f95581542c0 msgr2=0x7f95581546d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:15.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:15.192+0000 7f954fe00640 1 --2- 172.21.15.17:0/4114485233 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f95581542c0 0x7f95581546d0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f955806ee10 tx=0x7f954400eb90 comp rx=0 tx=0).stop 2024-09-10T21:35:15.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:15.192+0000 7f954fe00640 1 -- 172.21.15.17:0/4114485233 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f95581542c0 msgr2=0x7f95581546d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:35:15.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:15.192+0000 7f954fe00640 1 --2- 172.21.15.17:0/4114485233 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f95581542c0 0x7f95581546d0 secure :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f955806ee10 tx=0x7f954400eb90 comp rx=0 tx=0).stop 2024-09-10T21:35:15.198 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-10T21:35:15.198 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:35:15.198 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:35:15.198 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "leader", 2024-09-10T21:35:15.199 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 90, 2024-09-10T21:35:15.199 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 95799, 2024-09-10T21:35:15.199 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-10T21:35:15.199 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-10T21:35:15.199 INFO:teuthology.orchestra.run.smithi017.stdout: 1 2024-09-10T21:35:15.199 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:35:15.199 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 0, 2024-09-10T21:35:15.199 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:35:15.199 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-10T21:35:15.199 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-10T21:35:15.199 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:35:15.199 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:35:15.199 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:35:15.200 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:35:15.200 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:35:15.200 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:35:15.200 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:35:15.200 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:35:15.200 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:35:15.200 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:35:15.200 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:35:15.200 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:35:15.200 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-10T21:35:15.200 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-10T21:35:15.200 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:35:15.200 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:35:15.200 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:35:15.201 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:35:15.201 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:35:15.201 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:35:15.201 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:35:15.201 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:35:15.201 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:35:15.201 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:35:15.201 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:35:15.201 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:35:15.201 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:35:15.201 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-10T21:35:15.201 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-10T21:35:15.201 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-10T21:35:15.201 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-10T21:35:15.201 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-10T21:35:15.202 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "5e2f023f-d6a6-49f4-a256-f4807dce3739", 2024-09-10T21:35:15.202 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:35:15.202 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:35:15.202 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-10T21:35:15.202 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-10T21:35:15.202 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-10T21:35:15.202 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-10T21:35:15.202 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-10T21:35:15.202 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-10T21:35:15.202 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-10T21:35:15.202 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:35:15.202 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-10T21:35:15.202 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:35:15.202 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:35:15.203 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:35:15.203 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:35:15.203 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:35:15.203 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:35:15.203 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:35:15.203 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:35:15.203 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:35:15.203 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:35:15.203 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:35:15.203 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:35:15.203 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-10T21:35:15.203 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:35:15.204 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-10T21:35:15.204 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:35:15.204 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:35:15.204 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:35:15.204 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:35:15.204 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:35:15.204 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:35:15.204 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:35:15.204 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-10T21:35:15.204 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:35:15.204 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:35:15.204 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:35:15.204 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:35:15.205 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-10T21:35:15.205 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:35:15.205 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:35:15.205 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:35:15.205 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:35:15.205 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-10T21:35:15.205 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-10T21:35:15.205 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:35:15.205 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:35:15.205 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:35:15.205 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:35:15.205 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:35:15.206 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:35:15.206 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:35:15.206 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:35:15.206 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:35:15.206 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:35:15.206 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:35:15.206 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3300", 2024-09-10T21:35:15.206 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:35:15.206 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:35:15.206 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:35:15.206 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:35:15.206 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789", 2024-09-10T21:35:15.206 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:35:15.206 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:35:15.207 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:35:15.207 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:35:15.207 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789/0", 2024-09-10T21:35:15.207 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6789/0", 2024-09-10T21:35:15.207 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:35:15.207 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:35:15.207 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:35:15.207 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:35:15.207 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:35:15.207 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:35:15.207 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:35:15.207 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:35:15.207 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:35:15.208 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:35:15.208 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:35:15.208 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3301", 2024-09-10T21:35:15.208 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:35:15.208 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:35:15.208 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:35:15.208 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:35:15.208 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790", 2024-09-10T21:35:15.208 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:35:15.208 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:35:15.208 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:35:15.208 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:35:15.208 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790/0", 2024-09-10T21:35:15.209 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6790/0", 2024-09-10T21:35:15.209 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:35:15.209 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:35:15.209 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:35:15.209 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:35:15.209 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:35:15.209 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:35:15.209 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-10T21:35:15.209 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-10T21:35:15.209 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:35:15.209 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:35:15.209 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:35:15.209 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:35:15.210 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:35:15.210 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:35:15.210 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": [ 2024-09-10T21:35:15.210 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:35:15.210 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:35:15.210 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:35:15.210 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 6 2024-09-10T21:35:15.210 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:35:15.210 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:35:15.210 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-10T21:35:15.210 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:35:15.210 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:35:15.210 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:35:15.210 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 8 2024-09-10T21:35:15.211 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:35:15.211 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:35:15.211 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-10T21:35:15.211 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:35:15.211 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:35:15.211 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:35:15.211 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 6 2024-09-10T21:35:15.211 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:35:15.211 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:35:15.211 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": [ 2024-09-10T21:35:15.211 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:35:15.211 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:35:15.211 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:35:15.212 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 2 2024-09-10T21:35:15.212 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:35:15.212 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:35:15.212 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:35:15.212 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-10T21:35:15.212 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-10T21:35:15.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:15.192+0000 7f955ec00640 1 -- 172.21.15.17:0/4114485233 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f95380720c0 msgr2=0x7f9538074580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:15.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:15.192+0000 7f955ec00640 1 --2- 172.21.15.17:0/4114485233 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f95380720c0 0x7f9538074580 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f955000cc30 tx=0x7f9550009030 comp rx=0 tx=0).stop 2024-09-10T21:35:15.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:15.192+0000 7f955ec00640 1 -- 172.21.15.17:0/4114485233 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f95580c9080 msgr2=0x7f9558160500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:15.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:15.192+0000 7f955ec00640 1 --2- 172.21.15.17:0/4114485233 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f95580c9080 0x7f9558160500 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f95480b0040 tx=0x7f95480a6590 comp rx=0 tx=0).stop 2024-09-10T21:35:15.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:15.192+0000 7f955cc00640 1 -- 172.21.15.17:0/4114485233 reap_dead start 2024-09-10T21:35:15.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:15.192+0000 7f955cc00640 1 -- 172.21.15.17:0/4114485233 reap_dead start 2024-09-10T21:35:15.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:15.192+0000 7f955ec00640 1 -- 172.21.15.17:0/4114485233 shutdown_connections 2024-09-10T21:35:15.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:15.192+0000 7f955ec00640 1 -- 172.21.15.17:0/4114485233 >> 172.21.15.17:0/4114485233 conn(0x7f95580700d0 msgr2=0x7f95580cbdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:15.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:15.192+0000 7f955ec00640 1 -- 172.21.15.17:0/4114485233 shutdown_connections 2024-09-10T21:35:15.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:15.192+0000 7f955ec00640 1 -- 172.21.15.17:0/4114485233 wait complete. 2024-09-10T21:35:15.220 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-09-10T21:35:15.220 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon b 2024-09-10T21:35:15.220 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-10T21:35:15.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:15.400+0000 7ff54fe00640 1 -- 172.21.15.17:0/1588286085 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff550070820 msgr2=0x7ff550157cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:15.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:15.400+0000 7ff54fe00640 1 --2- 172.21.15.17:0/1588286085 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff550070820 0x7ff550157cc0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7ff5500c4d00 tx=0x7ff54001c8c0 comp rx=0 tx=0).stop 2024-09-10T21:35:15.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:15.400+0000 7ff54fe00640 1 -- 172.21.15.17:0/1588286085 shutdown_connections 2024-09-10T21:35:15.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:15.400+0000 7ff54fe00640 1 --2- 172.21.15.17:0/1588286085 >> v2:172.21.15.31:3300/0 conn(0x7ff550072bb0 0x7ff5500c77b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:15.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:15.400+0000 7ff54fe00640 1 --2- 172.21.15.17:0/1588286085 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff550070820 0x7ff550157cc0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:15.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:15.400+0000 7ff54fe00640 1 -- 172.21.15.17:0/1588286085 >> 172.21.15.17:0/1588286085 conn(0x7ff55006cc20 msgr2=0x7ff55006d030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:15.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:15.400+0000 7ff54fe00640 1 -- 172.21.15.17:0/1588286085 shutdown_connections 2024-09-10T21:35:15.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:15.400+0000 7ff54fe00640 1 -- 172.21.15.17:0/1588286085 wait complete. 2024-09-10T21:35:15.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:15.400+0000 7ff54fe00640 1 Processor -- start 2024-09-10T21:35:15.407 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:15.400+0000 7ff54fe00640 1 -- start start 2024-09-10T21:35:15.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:15.404+0000 7ff54fe00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff550070820 0x7ff550146150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:15.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:15.404+0000 7ff54fe00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff550070c00 0x7ff5500c2910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:15.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:15.404+0000 7ff54fe00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff550072bb0 0x7ff55014c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:15.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:15.404+0000 7ff54fe00640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7ff5500c8120 con 0x7ff550070820 2024-09-10T21:35:15.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:15.404+0000 7ff54fe00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7ff5500c4de0 con 0x7ff550072bb0 2024-09-10T21:35:15.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:15.404+0000 7ff54fe00640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7ff5500c7cf0 con 0x7ff550070c00 2024-09-10T21:35:15.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:15.404+0000 7ff54e000640 1 -- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff550070c00 msgr2=0x7ff5500c2910 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:3301/0 2024-09-10T21:35:15.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:15.404+0000 7ff54ea00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff550070820 0x7ff550146150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:15.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:15.404+0000 7ff54e000640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff550070c00 0x7ff5500c2910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:35:15.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:15.404+0000 7ff54ea00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff550070820 0x7ff550146150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49228/0 (socket says 172.21.15.17:49228) 2024-09-10T21:35:15.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:15.404+0000 7ff54ea00640 1 -- 172.21.15.17:0/3709771489 learned_addr learned my addr 172.21.15.17:0/3709771489 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:35:15.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:15.404+0000 7ff54f400640 1 --2- 172.21.15.17:0/3709771489 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff550072bb0 0x7ff55014c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:15.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:15.404+0000 7ff54ea00640 1 -- 172.21.15.17:0/3709771489 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff550070c00 msgr2=0x7ff5500c2910 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:35:15.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:15.404+0000 7ff54ea00640 1 --2- 172.21.15.17:0/3709771489 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff550070c00 0x7ff5500c2910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:15.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:15.404+0000 7ff54ea00640 1 -- 172.21.15.17:0/3709771489 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff550072bb0 msgr2=0x7ff55014c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:15.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:15.404+0000 7ff54ea00640 1 --2- 172.21.15.17:0/3709771489 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff550072bb0 0x7ff55014c640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:15.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:15.404+0000 7ff54ea00640 1 -- 172.21.15.17:0/3709771489 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5500c2430 con 0x7ff550070820 2024-09-10T21:35:15.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:15.404+0000 7ff54f400640 1 --2- 172.21.15.17:0/3709771489 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff550072bb0 0x7ff55014c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:35:15.411 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:15.404+0000 7ff54ea00640 1 --2- 172.21.15.17:0/3709771489 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff550070820 0x7ff550146150 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7ff5400194d0 tx=0x7ff5400a57b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:15.411 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:15.404+0000 7ff547400640 1 -- 172.21.15.17:0/3709771489 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5400bd070 con 0x7ff550070820 2024-09-10T21:35:15.411 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:15.404+0000 7ff547400640 1 -- 172.21.15.17:0/3709771489 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff5400044b0 con 0x7ff550070820 2024-09-10T21:35:15.411 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:15.404+0000 7ff54fe00640 1 -- 172.21.15.17:0/3709771489 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff55014d1e0 con 0x7ff550070820 2024-09-10T21:35:15.411 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:15.404+0000 7ff547400640 1 -- 172.21.15.17:0/3709771489 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5400b2040 con 0x7ff550070820 2024-09-10T21:35:15.411 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:15.404+0000 7ff54fe00640 1 -- 172.21.15.17:0/3709771489 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff55014da60 con 0x7ff550070820 2024-09-10T21:35:15.411 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:15.404+0000 7ff54fe00640 1 --2- 172.21.15.17:0/3709771489 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff5500c1b70 0x7ff550140ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:15.411 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:15.404+0000 7ff54fe00640 1 -- 172.21.15.17:0/3709771489 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7ff55014d500 con 0x7ff5500c1b70 2024-09-10T21:35:15.411 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:15.404+0000 7ff54e000640 1 --2- 172.21.15.17:0/3709771489 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff5500c1b70 0x7ff550140ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:15.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:15.404+0000 7ff54e000640 1 -- 172.21.15.17:0/3709771489 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff550157cc0 con 0x7ff5500c1b70 2024-09-10T21:35:15.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:15.408+0000 7ff54e000640 1 --2- 172.21.15.17:0/3709771489 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff5500c1b70 0x7ff550140ac0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7ff55014f960 tx=0x7ff53c00da30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:15.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:15.408+0000 7ff547400640 1 -- 172.21.15.17:0/3709771489 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7ff540003840 con 0x7ff550070820 2024-09-10T21:35:15.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:15.408+0000 7ff547400640 1 --2- 172.21.15.17:0/3709771489 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7ff530072590 0x7ff530074a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:15.414 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:15.408+0000 7ff547400640 1 -- 172.21.15.17:0/3709771489 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7ff54012d490 con 0x7ff550070820 2024-09-10T21:35:15.414 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:15.408+0000 7ff547400640 1 -- 172.21.15.17:0/3709771489 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff53c017070 con 0x7ff5500c1b70 2024-09-10T21:35:15.414 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:15.408+0000 7ff547400640 1 -- 172.21.15.17:0/3709771489 <== mon.1 v2:172.21.15.31:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7ff53c005220 con 0x7ff5500c1b70 2024-09-10T21:35:15.414 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:15.408+0000 7ff547400640 1 -- 172.21.15.17:0/3709771489 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff5500c1b70 msgr2=0x7ff550140ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:15.414 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:15.408+0000 7ff547400640 1 --2- 172.21.15.17:0/3709771489 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff5500c1b70 0x7ff550140ac0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7ff55014f960 tx=0x7ff53c00da30 comp rx=0 tx=0).stop 2024-09-10T21:35:15.415 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:15.408+0000 7ff547400640 1 -- 172.21.15.17:0/3709771489 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff5500c1b70 msgr2=0x7ff550140ac0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:35:15.415 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:15.408+0000 7ff547400640 1 --2- 172.21.15.17:0/3709771489 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff5500c1b70 0x7ff550140ac0 secure :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7ff55014f960 tx=0x7ff53c00da30 comp rx=0 tx=0).stop 2024-09-10T21:35:15.415 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:15.408+0000 7ff54f400640 1 --2- 172.21.15.17:0/3709771489 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7ff530072590 0x7ff530074a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:15.415 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:15.408+0000 7ff54f400640 1 --2- 172.21.15.17:0/3709771489 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7ff530072590 0x7ff530074a50 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7ff548007fd0 tx=0x7ff548009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:15.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:15.416+0000 7ff54fe00640 1 --2- 172.21.15.17:0/3709771489 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff55014cb80 0x7ff550141000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:15.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:15.416+0000 7ff54fe00640 1 -- 172.21.15.17:0/3709771489 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7ff55006d000 con 0x7ff55014cb80 2024-09-10T21:35:15.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:15.416+0000 7ff54e000640 1 --2- 172.21.15.17:0/3709771489 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff55014cb80 0x7ff550141000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:15.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:15.416+0000 7ff54e000640 1 -- 172.21.15.17:0/3709771489 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7ff5500712d0 con 0x7ff55014cb80 2024-09-10T21:35:15.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:15.416+0000 7ff54e000640 1 --2- 172.21.15.17:0/3709771489 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff55014cb80 0x7ff550141000 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7ff53c016000 tx=0x7ff53c0046d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:15.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:15.420+0000 7ff547400640 1 -- 172.21.15.17:0/3709771489 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff53c009c40 con 0x7ff55014cb80 2024-09-10T21:35:15.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:15.420+0000 7ff547400640 1 -- 172.21.15.17:0/3709771489 <== mon.1 v2:172.21.15.31:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4529 (secure 0 0 0) 0x7ff53c009c40 con 0x7ff55014cb80 2024-09-10T21:35:15.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:15.420+0000 7ff547400640 1 -- 172.21.15.17:0/3709771489 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff55014cb80 msgr2=0x7ff550141000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:15.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:15.420+0000 7ff547400640 1 --2- 172.21.15.17:0/3709771489 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff55014cb80 0x7ff550141000 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7ff53c016000 tx=0x7ff53c0046d0 comp rx=0 tx=0).stop 2024-09-10T21:35:15.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:15.420+0000 7ff547400640 1 -- 172.21.15.17:0/3709771489 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff55014cb80 msgr2=0x7ff550141000 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:35:15.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:15.420+0000 7ff547400640 1 --2- 172.21.15.17:0/3709771489 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff55014cb80 0x7ff550141000 secure :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7ff53c016000 tx=0x7ff53c0046d0 comp rx=0 tx=0).stop 2024-09-10T21:35:15.425 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-10T21:35:15.425 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:35:15.425 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:35:15.425 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-10T21:35:15.425 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 90, 2024-09-10T21:35:15.425 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 34807, 2024-09-10T21:35:15.426 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-10T21:35:15.426 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-10T21:35:15.426 INFO:teuthology.orchestra.run.smithi017.stdout: 1 2024-09-10T21:35:15.426 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:35:15.426 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 0, 2024-09-10T21:35:15.426 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:35:15.426 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-10T21:35:15.426 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-10T21:35:15.426 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:35:15.426 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:35:15.426 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:35:15.426 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:35:15.426 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:35:15.426 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:35:15.427 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:35:15.427 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:35:15.427 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:35:15.427 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:35:15.427 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:35:15.427 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:35:15.427 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-10T21:35:15.427 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-10T21:35:15.427 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:35:15.427 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:35:15.427 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:35:15.427 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:35:15.427 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:35:15.428 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:35:15.428 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:35:15.428 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:35:15.428 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:35:15.428 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:35:15.428 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:35:15.428 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:35:15.428 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:35:15.428 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-10T21:35:15.428 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-10T21:35:15.428 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-10T21:35:15.428 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-10T21:35:15.428 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-10T21:35:15.429 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "5e2f023f-d6a6-49f4-a256-f4807dce3739", 2024-09-10T21:35:15.429 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:35:15.429 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:35:15.429 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-10T21:35:15.429 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-10T21:35:15.429 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-10T21:35:15.429 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-10T21:35:15.429 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-10T21:35:15.429 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-10T21:35:15.429 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-10T21:35:15.429 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:35:15.429 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-10T21:35:15.429 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:35:15.430 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:35:15.430 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:35:15.430 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:35:15.430 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:35:15.430 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:35:15.430 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:35:15.430 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:35:15.430 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:35:15.430 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:35:15.430 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:35:15.430 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:35:15.430 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-10T21:35:15.430 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:35:15.431 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-10T21:35:15.431 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:35:15.431 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:35:15.431 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:35:15.431 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:35:15.431 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:35:15.431 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:35:15.431 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:35:15.431 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-10T21:35:15.431 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:35:15.431 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:35:15.431 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:35:15.432 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:35:15.432 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-10T21:35:15.432 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:35:15.432 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:35:15.432 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:35:15.432 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:35:15.432 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-10T21:35:15.432 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-10T21:35:15.432 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:35:15.432 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:35:15.432 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:35:15.432 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:35:15.432 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:35:15.433 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:35:15.433 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:35:15.433 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:35:15.433 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:35:15.433 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:35:15.433 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:35:15.433 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3300", 2024-09-10T21:35:15.433 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:35:15.433 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:35:15.433 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:35:15.433 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:35:15.433 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789", 2024-09-10T21:35:15.433 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:35:15.433 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:35:15.434 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:35:15.434 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:35:15.434 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789/0", 2024-09-10T21:35:15.434 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6789/0", 2024-09-10T21:35:15.434 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:35:15.434 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:35:15.434 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:35:15.434 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:35:15.434 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:35:15.434 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:35:15.434 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:35:15.434 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:35:15.434 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:35:15.434 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:35:15.435 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:35:15.435 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3301", 2024-09-10T21:35:15.435 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:35:15.435 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:35:15.435 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:35:15.435 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:35:15.435 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790", 2024-09-10T21:35:15.435 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:35:15.435 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:35:15.435 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:35:15.435 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:35:15.435 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790/0", 2024-09-10T21:35:15.435 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6790/0", 2024-09-10T21:35:15.436 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:35:15.436 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:35:15.436 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:35:15.436 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:35:15.436 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:35:15.436 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:35:15.436 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-10T21:35:15.436 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-10T21:35:15.436 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:35:15.436 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:35:15.436 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:35:15.436 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:35:15.436 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:35:15.437 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:35:15.437 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-10T21:35:15.437 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:35:15.437 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:35:15.437 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:35:15.437 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 2 2024-09-10T21:35:15.437 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:35:15.437 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:35:15.437 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:35:15.437 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-10T21:35:15.437 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-10T21:35:15.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:15.420+0000 7ff54fe00640 1 -- 172.21.15.17:0/3709771489 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7ff530072590 msgr2=0x7ff530074a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:15.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:15.420+0000 7ff54fe00640 1 --2- 172.21.15.17:0/3709771489 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7ff530072590 0x7ff530074a50 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7ff548007fd0 tx=0x7ff548009040 comp rx=0 tx=0).stop 2024-09-10T21:35:15.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:15.420+0000 7ff54fe00640 1 -- 172.21.15.17:0/3709771489 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff550070820 msgr2=0x7ff550146150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:15.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:15.420+0000 7ff54fe00640 1 --2- 172.21.15.17:0/3709771489 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff550070820 0x7ff550146150 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7ff5400194d0 tx=0x7ff5400a57b0 comp rx=0 tx=0).stop 2024-09-10T21:35:15.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:15.420+0000 7ff54f400640 1 -- 172.21.15.17:0/3709771489 reap_dead start 2024-09-10T21:35:15.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:15.420+0000 7ff54f400640 1 -- 172.21.15.17:0/3709771489 reap_dead start 2024-09-10T21:35:15.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:15.420+0000 7ff54fe00640 1 -- 172.21.15.17:0/3709771489 shutdown_connections 2024-09-10T21:35:15.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:15.420+0000 7ff54fe00640 1 -- 172.21.15.17:0/3709771489 >> 172.21.15.17:0/3709771489 conn(0x7ff55006cc20 msgr2=0x7ff5500c4ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:15.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:15.420+0000 7ff54fe00640 1 -- 172.21.15.17:0/3709771489 shutdown_connections 2024-09-10T21:35:15.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:15.420+0000 7ff54fe00640 1 -- 172.21.15.17:0/3709771489 wait complete. 2024-09-10T21:35:15.445 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-09-10T21:35:15.445 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2024-09-10T21:35:19.307 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:35:19.311 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:35:35.446 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.c 2024-09-10T21:35:35.446 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-10T21:35:35.446 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-10T21:35:35.450 INFO:tasks.ceph.mon.c:Started 2024-09-10T21:35:35.451 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-09-10T21:35:35.451 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-10T21:35:35.617 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:35.612+0000 7ff7b6800640 1 -- 172.21.15.17:0/840794320 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff7b00717c0 msgr2=0x7ff7b006cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:35.617 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:35.612+0000 7ff7b6800640 1 --2- 172.21.15.17:0/840794320 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff7b00717c0 0x7ff7b006cc20 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7ff7b006dab0 tx=0x7ff79c01c8c0 comp rx=0 tx=0).stop 2024-09-10T21:35:35.617 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:35.612+0000 7ff7b6800640 1 -- 172.21.15.17:0/840794320 shutdown_connections 2024-09-10T21:35:35.617 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:35.612+0000 7ff7b6800640 1 --2- 172.21.15.17:0/840794320 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff7b00717c0 0x7ff7b006cc20 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:35.617 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:35.612+0000 7ff7b6800640 1 --2- 172.21.15.17:0/840794320 >> v2:172.21.15.31:3300/0 conn(0x7ff7b0154920 0x7ff7b015b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:35.617 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:35.612+0000 7ff7b6800640 1 -- 172.21.15.17:0/840794320 >> 172.21.15.17:0/840794320 conn(0x7ff7b0070ca0 msgr2=0x7ff7b00710b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:35.617 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:35.612+0000 7ff7b6800640 1 -- 172.21.15.17:0/840794320 shutdown_connections 2024-09-10T21:35:35.617 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:35.612+0000 7ff7b6800640 1 -- 172.21.15.17:0/840794320 wait complete. 2024-09-10T21:35:35.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:35.612+0000 7ff7b6800640 1 Processor -- start 2024-09-10T21:35:35.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:35.612+0000 7ff7b6800640 1 -- start start 2024-09-10T21:35:35.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:35.612+0000 7ff7b6800640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff7b006d160 0x7ff7b0153c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:35.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:35.612+0000 7ff7b6800640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff7b00717c0 0x7ff7b00c26b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:35.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:35.612+0000 7ff7b6800640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff7b0154920 0x7ff7b0149810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:35.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:35.612+0000 7ff7b6800640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7ff7b015dbd0 con 0x7ff7b0154920 2024-09-10T21:35:35.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:35.612+0000 7ff7b6800640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7ff7b0154d00 con 0x7ff7b00717c0 2024-09-10T21:35:35.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:35.612+0000 7ff7b6800640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7ff7b015dd50 con 0x7ff7b006d160 2024-09-10T21:35:35.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:35.612+0000 7ff7b5e00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff7b0154920 0x7ff7b0149810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:35.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:35.612+0000 7ff7b4a00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff7b00717c0 0x7ff7b00c26b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:35.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:35.612+0000 7ff7b5e00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff7b0154920 0x7ff7b0149810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:54516/0 (socket says 172.21.15.17:54516) 2024-09-10T21:35:35.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:35.612+0000 7ff7b5e00640 1 -- 172.21.15.17:0/1089419855 learned_addr learned my addr 172.21.15.17:0/1089419855 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:35:35.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:35.612+0000 7ff7b5400640 1 --2- 172.21.15.17:0/1089419855 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff7b006d160 0x7ff7b0153c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:35.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:35.616+0000 7ff7b5400640 1 -- 172.21.15.17:0/1089419855 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff7b006d160 msgr2=0x7ff7b0153c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-09-10T21:35:35.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:35.616+0000 7ff7b5400640 1 -- 172.21.15.17:0/1089419855 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff7b006d160 msgr2=0x7ff7b0153c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-10T21:35:35.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:35.616+0000 7ff7b5400640 1 --2- 172.21.15.17:0/1089419855 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff7b006d160 0x7ff7b0153c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T21:35:35.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:35.616+0000 7ff7b5400640 1 --2- 172.21.15.17:0/1089419855 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff7b006d160 0x7ff7b0153c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:35:35.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:35.616+0000 7ff7b4a00640 1 -- 172.21.15.17:0/1089419855 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff7b006d160 msgr2=0x7ff7b0153c90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:35:35.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:35.616+0000 7ff7b4a00640 1 --2- 172.21.15.17:0/1089419855 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff7b006d160 0x7ff7b0153c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:35.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:35.616+0000 7ff7b4a00640 1 -- 172.21.15.17:0/1089419855 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff7b0154920 msgr2=0x7ff7b0149810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:35.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:35.616+0000 7ff7b4a00640 1 --2- 172.21.15.17:0/1089419855 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff7b0154920 0x7ff7b0149810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:35.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:35.616+0000 7ff7b4a00640 1 -- 172.21.15.17:0/1089419855 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7b014b510 con 0x7ff7b00717c0 2024-09-10T21:35:35.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:35.616+0000 7ff7b5e00640 1 --2- 172.21.15.17:0/1089419855 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff7b0154920 0x7ff7b0149810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:35:35.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:35.616+0000 7ff7b4a00640 1 --2- 172.21.15.17:0/1089419855 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff7b00717c0 0x7ff7b00c26b0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ff79c0ac000 tx=0x7ff79c0a5600 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:35.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:35.616+0000 7ff7ad600640 1 -- 172.21.15.17:0/1089419855 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff79c0b2040 con 0x7ff7b00717c0 2024-09-10T21:35:35.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:35.616+0000 7ff7b6800640 1 -- 172.21.15.17:0/1089419855 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7b014b7b0 con 0x7ff7b00717c0 2024-09-10T21:35:35.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:35.616+0000 7ff7ad600640 1 -- 172.21.15.17:0/1089419855 <== mon.1 v2:172.21.15.31:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff79c0043d0 con 0x7ff7b00717c0 2024-09-10T21:35:35.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:35.616+0000 7ff7ad600640 1 -- 172.21.15.17:0/1089419855 <== mon.1 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff79c005290 con 0x7ff7b00717c0 2024-09-10T21:35:35.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:35.616+0000 7ff7b6800640 1 -- 172.21.15.17:0/1089419855 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7b015d840 con 0x7ff7b00717c0 2024-09-10T21:35:35.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:35.616+0000 7ff7b6800640 1 -- 172.21.15.17:0/1089419855 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7b006cc20 con 0x7ff7b00717c0 2024-09-10T21:35:35.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:35.620+0000 7ff7ad600640 1 -- 172.21.15.17:0/1089419855 <== mon.1 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7ff79c005430 con 0x7ff7b00717c0 2024-09-10T21:35:35.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:35.620+0000 7ff7ad600640 1 --2- 172.21.15.17:0/1089419855 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7ff790072610 0x7ff790074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:35.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:35.620+0000 7ff7ad600640 1 -- 172.21.15.17:0/1089419855 <== mon.1 v2:172.21.15.31:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7ff79c12d790 con 0x7ff7b00717c0 2024-09-10T21:35:35.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:35.620+0000 7ff7b5400640 1 --2- 172.21.15.17:0/1089419855 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7ff790072610 0x7ff790074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:35.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:35.620+0000 7ff7b5400640 1 --2- 172.21.15.17:0/1089419855 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7ff790072610 0x7ff790074ad0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7ff7a4005e80 tx=0x7ff7a4005eb0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:35.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:35.624+0000 7ff7ad600640 1 -- 172.21.15.17:0/1089419855 <== mon.1 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7ff79c0164b0 con 0x7ff7b00717c0 2024-09-10T21:35:35.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:35.944+0000 7ff7b6800640 1 -- 172.21.15.17:0/1089419855 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7ff7b0071ce0 con 0x7ff7b00717c0 2024-09-10T21:35:35.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:35.944+0000 7ff7ad600640 1 -- 172.21.15.17:0/1089419855 <== mon.1 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1551 (secure 0 0 0) 0x7ff79c0f89e0 con 0x7ff7b00717c0 2024-09-10T21:35:35.949 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":90,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":21,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5e2f023f-d6a6-49f4-a256-f4807dce3739","modified":"2024-09-10T21:23:46.892130Z","created":"2024-09-10T21:23:46.892130Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3301","nonce":0},{"type":"v1","addr":"172.21.15.31:6790","nonce":0}]},"addr":"172.21.15.31:6790/0","public_addr":"172.21.15.31:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-10T21:35:35.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:35.944+0000 7ff7b6800640 1 -- 172.21.15.17:0/1089419855 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7ff790072610 msgr2=0x7ff790074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:35.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:35.944+0000 7ff7b6800640 1 --2- 172.21.15.17:0/1089419855 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7ff790072610 0x7ff790074ad0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7ff7a4005e80 tx=0x7ff7a4005eb0 comp rx=0 tx=0).stop 2024-09-10T21:35:35.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:35.944+0000 7ff7b6800640 1 -- 172.21.15.17:0/1089419855 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff7b00717c0 msgr2=0x7ff7b00c26b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:35.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:35.944+0000 7ff7b6800640 1 --2- 172.21.15.17:0/1089419855 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff7b00717c0 0x7ff7b00c26b0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ff79c0ac000 tx=0x7ff79c0a5600 comp rx=0 tx=0).stop 2024-09-10T21:35:35.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:35.944+0000 7ff7b6800640 1 -- 172.21.15.17:0/1089419855 shutdown_connections 2024-09-10T21:35:35.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:35.944+0000 7ff7b6800640 1 --2- 172.21.15.17:0/1089419855 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7ff790072610 0x7ff790074ad0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:35.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:35.944+0000 7ff7b6800640 1 --2- 172.21.15.17:0/1089419855 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff7b0154920 0x7ff7b0149810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:35.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:35.944+0000 7ff7b6800640 1 --2- 172.21.15.17:0/1089419855 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff7b00717c0 0x7ff7b00c26b0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:35.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:35.944+0000 7ff7b6800640 1 --2- 172.21.15.17:0/1089419855 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff7b006d160 0x7ff7b0153c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:35.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:35.944+0000 7ff7b6800640 1 -- 172.21.15.17:0/1089419855 >> 172.21.15.17:0/1089419855 conn(0x7ff7b0070ca0 msgr2=0x7ff7b0151580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:35.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:35.944+0000 7ff7b6800640 1 -- 172.21.15.17:0/1089419855 shutdown_connections 2024-09-10T21:35:35.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:35.944+0000 7ff7b6800640 1 -- 172.21.15.17:0/1089419855 wait complete. 2024-09-10T21:35:38.976 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-10T21:35:39.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:39.132+0000 7f5158600640 1 -- 172.21.15.17:0/1044645216 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5150074f70 msgr2=0x7f51500c7130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:39.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:39.132+0000 7f5158600640 1 --2- 172.21.15.17:0/1044645216 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5150074f70 0x7f51500c7130 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f5150071fd0 tx=0x7f514401c8c0 comp rx=0 tx=0).stop 2024-09-10T21:35:39.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:39.132+0000 7f5158600640 1 -- 172.21.15.17:0/1044645216 shutdown_connections 2024-09-10T21:35:39.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:39.132+0000 7f5158600640 1 --2- 172.21.15.17:0/1044645216 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f515006ed90 0x7f515015bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:39.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:39.132+0000 7f5158600640 1 --2- 172.21.15.17:0/1044645216 >> v2:172.21.15.31:3300/0 conn(0x7f5150070450 0x7f515006e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:39.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:39.132+0000 7f5158600640 1 --2- 172.21.15.17:0/1044645216 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5150074f70 0x7f51500c7130 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:39.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:39.132+0000 7f5158600640 1 -- 172.21.15.17:0/1044645216 >> 172.21.15.17:0/1044645216 conn(0x7f515006dd30 msgr2=0x7f515006e140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:39.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:39.132+0000 7f5158600640 1 -- 172.21.15.17:0/1044645216 shutdown_connections 2024-09-10T21:35:39.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:39.132+0000 7f5158600640 1 -- 172.21.15.17:0/1044645216 wait complete. 2024-09-10T21:35:39.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:39.132+0000 7f5158600640 1 Processor -- start 2024-09-10T21:35:39.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:39.132+0000 7f5158600640 1 -- start start 2024-09-10T21:35:39.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:39.132+0000 7f5158600640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f515006ed90 0x7f51500c5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:39.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:39.132+0000 7f5158600640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5150070450 0x7f51500b21b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:39.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:39.132+0000 7f5158600640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f5150074f70 0x7f5150154250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:39.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:39.132+0000 7f5158600640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f5150069b90 con 0x7f5150070450 2024-09-10T21:35:39.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:39.132+0000 7f5158600640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f515015c890 con 0x7f515006ed90 2024-09-10T21:35:39.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:39.132+0000 7f5158600640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f515006d430 con 0x7f5150074f70 2024-09-10T21:35:39.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:39.132+0000 7f5155200640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5150070450 0x7f51500b21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:39.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:39.132+0000 7f5155200640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5150070450 0x7f51500b21b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:54532/0 (socket says 172.21.15.17:54532) 2024-09-10T21:35:39.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:39.132+0000 7f5155200640 1 -- 172.21.15.17:0/1082266122 learned_addr learned my addr 172.21.15.17:0/1082266122 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:35:39.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:39.132+0000 7f5155c00640 1 --2- 172.21.15.17:0/1082266122 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f515006ed90 0x7f51500c5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:39.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:39.132+0000 7f5156600640 1 --2- 172.21.15.17:0/1082266122 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f5150074f70 0x7f5150154250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:39.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:39.132+0000 7f5155200640 1 -- 172.21.15.17:0/1082266122 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f5150074f70 msgr2=0x7f5150154250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:39.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:39.132+0000 7f5155200640 1 --2- 172.21.15.17:0/1082266122 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f5150074f70 0x7f5150154250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:39.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:39.132+0000 7f5155200640 1 -- 172.21.15.17:0/1082266122 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f515006ed90 msgr2=0x7f51500c5150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:39.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:39.132+0000 7f5155200640 1 --2- 172.21.15.17:0/1082266122 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f515006ed90 0x7f51500c5150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:39.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:39.132+0000 7f5155200640 1 -- 172.21.15.17:0/1082266122 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51500aa490 con 0x7f5150070450 2024-09-10T21:35:39.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:39.132+0000 7f5155c00640 1 --2- 172.21.15.17:0/1082266122 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f515006ed90 0x7f51500c5150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:35:39.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:39.136+0000 7f5155200640 1 --2- 172.21.15.17:0/1082266122 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5150070450 0x7f51500b21b0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f513c00c970 tx=0x7f513c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:39.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:39.136+0000 7f514d600640 1 -- 172.21.15.17:0/1082266122 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f513c007c10 con 0x7f5150070450 2024-09-10T21:35:39.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:39.136+0000 7f514d600640 1 -- 172.21.15.17:0/1082266122 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f513c00f450 con 0x7f5150070450 2024-09-10T21:35:39.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:39.136+0000 7f514d600640 1 -- 172.21.15.17:0/1082266122 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f513c00f8a0 con 0x7f5150070450 2024-09-10T21:35:39.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:39.136+0000 7f5158600640 1 -- 172.21.15.17:0/1082266122 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51500aa730 con 0x7f5150070450 2024-09-10T21:35:39.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:39.136+0000 7f5158600640 1 -- 172.21.15.17:0/1082266122 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51500c60e0 con 0x7f5150070450 2024-09-10T21:35:39.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:39.136+0000 7f5158600640 1 -- 172.21.15.17:0/1082266122 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f515006d830 con 0x7f5150070450 2024-09-10T21:35:39.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:39.136+0000 7f514d600640 1 -- 172.21.15.17:0/1082266122 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f513c00fa40 con 0x7f5150070450 2024-09-10T21:35:39.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:39.140+0000 7f514d600640 1 --2- 172.21.15.17:0/1082266122 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f51300725e0 0x7f5130074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:39.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:39.140+0000 7f514d600640 1 -- 172.21.15.17:0/1082266122 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f513c093da0 con 0x7f5150070450 2024-09-10T21:35:39.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:39.140+0000 7f5155c00640 1 --2- 172.21.15.17:0/1082266122 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f51300725e0 0x7f5130074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:39.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:39.140+0000 7f5155c00640 1 --2- 172.21.15.17:0/1082266122 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f51300725e0 0x7f5130074aa0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f51440af000 tx=0x7f514401b100 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:39.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:39.144+0000 7f514d600640 1 -- 172.21.15.17:0/1082266122 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f513c014030 con 0x7f5150070450 2024-09-10T21:35:39.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:39.388+0000 7f5158600640 1 -- 172.21.15.17:0/1082266122 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f5150069e60 con 0x7f5150070450 2024-09-10T21:35:39.394 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":90,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":24,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5e2f023f-d6a6-49f4-a256-f4807dce3739","modified":"2024-09-10T21:23:46.892130Z","created":"2024-09-10T21:23:46.892130Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3301","nonce":0},{"type":"v1","addr":"172.21.15.31:6790","nonce":0}]},"addr":"172.21.15.31:6790/0","public_addr":"172.21.15.31:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-10T21:35:39.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:39.388+0000 7f514d600640 1 -- 172.21.15.17:0/1082266122 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1551 (secure 0 0 0) 0x7f513c05f070 con 0x7f5150070450 2024-09-10T21:35:39.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:39.392+0000 7f5158600640 1 -- 172.21.15.17:0/1082266122 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f51300725e0 msgr2=0x7f5130074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:39.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:39.392+0000 7f5158600640 1 --2- 172.21.15.17:0/1082266122 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f51300725e0 0x7f5130074aa0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f51440af000 tx=0x7f514401b100 comp rx=0 tx=0).stop 2024-09-10T21:35:39.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:39.392+0000 7f5158600640 1 -- 172.21.15.17:0/1082266122 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5150070450 msgr2=0x7f51500b21b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:39.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:39.392+0000 7f5158600640 1 --2- 172.21.15.17:0/1082266122 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5150070450 0x7f51500b21b0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f513c00c970 tx=0x7f513c00ce30 comp rx=0 tx=0).stop 2024-09-10T21:35:39.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:39.392+0000 7f5158600640 1 -- 172.21.15.17:0/1082266122 shutdown_connections 2024-09-10T21:35:39.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:39.392+0000 7f5158600640 1 --2- 172.21.15.17:0/1082266122 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f51300725e0 0x7f5130074aa0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:39.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:39.392+0000 7f5158600640 1 --2- 172.21.15.17:0/1082266122 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f5150074f70 0x7f5150154250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:39.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:39.392+0000 7f5158600640 1 --2- 172.21.15.17:0/1082266122 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5150070450 0x7f51500b21b0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:39.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:39.392+0000 7f5158600640 1 --2- 172.21.15.17:0/1082266122 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f515006ed90 0x7f51500c5150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:39.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:39.392+0000 7f5158600640 1 -- 172.21.15.17:0/1082266122 >> 172.21.15.17:0/1082266122 conn(0x7f515006dd30 msgr2=0x7f51500cbae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:39.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:39.392+0000 7f5158600640 1 -- 172.21.15.17:0/1082266122 shutdown_connections 2024-09-10T21:35:39.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:39.392+0000 7f5158600640 1 -- 172.21.15.17:0/1082266122 wait complete. 2024-09-10T21:35:42.737 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-10T21:35:42.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:42.900+0000 7f664d200640 1 -- 172.21.15.17:0/1440924561 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f66480c7160 msgr2=0x7f66480c9230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:42.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:42.900+0000 7f664d200640 1 --2- 172.21.15.17:0/1440924561 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f66480c7160 0x7f66480c9230 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f664000acd0 tx=0x7f66400206d0 comp rx=0 tx=0).stop 2024-09-10T21:35:42.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:42.900+0000 7f664d200640 1 -- 172.21.15.17:0/1440924561 shutdown_connections 2024-09-10T21:35:42.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:42.900+0000 7f664d200640 1 --2- 172.21.15.17:0/1440924561 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f66480c7160 0x7f66480c9230 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:42.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:42.900+0000 7f664d200640 1 --2- 172.21.15.17:0/1440924561 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f66481508e0 0x7f66481572a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:42.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:42.900+0000 7f664d200640 1 -- 172.21.15.17:0/1440924561 >> 172.21.15.17:0/1440924561 conn(0x7f664806cdf0 msgr2=0x7f664806d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:42.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:42.900+0000 7f664d200640 1 -- 172.21.15.17:0/1440924561 shutdown_connections 2024-09-10T21:35:42.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:42.900+0000 7f664d200640 1 -- 172.21.15.17:0/1440924561 wait complete. 2024-09-10T21:35:42.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:42.904+0000 7f664d200640 1 Processor -- start 2024-09-10T21:35:42.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:42.904+0000 7f664d200640 1 -- start start 2024-09-10T21:35:42.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:42.904+0000 7f664d200640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f66480c7160 0x7f66480c93e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:42.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:42.904+0000 7f664d200640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f66481508e0 0x7f664814bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:42.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:42.904+0000 7f664d200640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6648150cc0 0x7f664814c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:42.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:42.904+0000 7f664d200640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f664815a690 con 0x7f66480c7160 2024-09-10T21:35:42.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:42.904+0000 7f664d200640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f664815a3b0 con 0x7f6648150cc0 2024-09-10T21:35:42.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:42.904+0000 7f664d200640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f66480c6820 con 0x7f66481508e0 2024-09-10T21:35:42.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:42.904+0000 7f6647400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f66480c7160 0x7f66480c93e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:42.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:42.904+0000 7f6646a00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f66481508e0 0x7f664814bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:42.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:42.904+0000 7f6647400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f66480c7160 0x7f66480c93e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:54540/0 (socket says 172.21.15.17:54540) 2024-09-10T21:35:42.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:42.904+0000 7f6647400640 1 -- 172.21.15.17:0/3192683886 learned_addr learned my addr 172.21.15.17:0/3192683886 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:35:42.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:42.904+0000 7f6647e00640 1 --2- 172.21.15.17:0/3192683886 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6648150cc0 0x7f664814c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:42.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:42.904+0000 7f6646a00640 1 -- 172.21.15.17:0/3192683886 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f66481508e0 msgr2=0x7f664814bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-09-10T21:35:42.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:42.904+0000 7f6646a00640 1 -- 172.21.15.17:0/3192683886 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f66481508e0 msgr2=0x7f664814bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-10T21:35:42.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:42.904+0000 7f6646a00640 1 --2- 172.21.15.17:0/3192683886 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f66481508e0 0x7f664814bdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T21:35:42.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:42.904+0000 7f6646a00640 1 --2- 172.21.15.17:0/3192683886 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f66481508e0 0x7f664814bdc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:35:42.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:42.904+0000 7f6647400640 1 -- 172.21.15.17:0/3192683886 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f66481508e0 msgr2=0x7f664814bdc0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:35:42.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:42.904+0000 7f6647400640 1 --2- 172.21.15.17:0/3192683886 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f66481508e0 0x7f664814bdc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:42.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:42.904+0000 7f6647400640 1 -- 172.21.15.17:0/3192683886 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6648150cc0 msgr2=0x7f664814c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:42.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:42.904+0000 7f6647400640 1 --2- 172.21.15.17:0/3192683886 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6648150cc0 0x7f664814c300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:42.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:42.904+0000 7f6647400640 1 -- 172.21.15.17:0/3192683886 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66481531e0 con 0x7f66480c7160 2024-09-10T21:35:42.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:42.904+0000 7f6647400640 1 --2- 172.21.15.17:0/3192683886 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f66480c7160 0x7f66480c93e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f663800d9b0 tx=0x7f663800de70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:42.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:42.904+0000 7f663f400640 1 -- 172.21.15.17:0/3192683886 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6638017070 con 0x7f66480c7160 2024-09-10T21:35:42.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:42.904+0000 7f663f400640 1 -- 172.21.15.17:0/3192683886 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6638004310 con 0x7f66480c7160 2024-09-10T21:35:42.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:42.904+0000 7f663f400640 1 -- 172.21.15.17:0/3192683886 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6638009c30 con 0x7f66480c7160 2024-09-10T21:35:42.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:42.904+0000 7f664d200640 1 -- 172.21.15.17:0/3192683886 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66481534c0 con 0x7f66480c7160 2024-09-10T21:35:42.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:42.904+0000 7f664d200640 1 -- 172.21.15.17:0/3192683886 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6648153a80 con 0x7f66480c7160 2024-09-10T21:35:42.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:42.908+0000 7f664d200640 1 -- 172.21.15.17:0/3192683886 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f664815abb0 con 0x7f66480c7160 2024-09-10T21:35:42.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:42.908+0000 7f663f400640 1 -- 172.21.15.17:0/3192683886 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f6638003600 con 0x7f66480c7160 2024-09-10T21:35:42.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:42.908+0000 7f663f400640 1 --2- 172.21.15.17:0/3192683886 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f66280725e0 0x7f6628074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:42.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:42.908+0000 7f663f400640 1 -- 172.21.15.17:0/3192683886 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f66380937e0 con 0x7f66480c7160 2024-09-10T21:35:42.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:42.908+0000 7f6646a00640 1 --2- 172.21.15.17:0/3192683886 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f66280725e0 0x7f6628074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:42.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:42.912+0000 7f6646a00640 1 --2- 172.21.15.17:0/3192683886 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f66280725e0 0x7f6628074aa0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f66481ee1c0 tx=0x7f6634009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:42.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:42.912+0000 7f663f400640 1 -- 172.21.15.17:0/3192683886 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f663805e3d0 con 0x7f66480c7160 2024-09-10T21:35:43.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.192+0000 7f664d200640 1 -- 172.21.15.17:0/3192683886 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f6648069e60 con 0x7f66480c7160 2024-09-10T21:35:43.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.192+0000 7f663f400640 1 -- 172.21.15.17:0/3192683886 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1556 (secure 0 0 0) 0x7f6638062e00 con 0x7f66480c7160 2024-09-10T21:35:43.198 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":92,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":2,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5e2f023f-d6a6-49f4-a256-f4807dce3739","modified":"2024-09-10T21:23:46.892130Z","created":"2024-09-10T21:23:46.892130Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3301","nonce":0},{"type":"v1","addr":"172.21.15.31:6790","nonce":0}]},"addr":"172.21.15.31:6790/0","public_addr":"172.21.15.31:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-10T21:35:43.200 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.196+0000 7f664d200640 1 -- 172.21.15.17:0/3192683886 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f66280725e0 msgr2=0x7f6628074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:43.201 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.196+0000 7f664d200640 1 --2- 172.21.15.17:0/3192683886 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f66280725e0 0x7f6628074aa0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f66481ee1c0 tx=0x7f6634009040 comp rx=0 tx=0).stop 2024-09-10T21:35:43.201 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.196+0000 7f664d200640 1 -- 172.21.15.17:0/3192683886 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f66480c7160 msgr2=0x7f66480c93e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:43.201 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.196+0000 7f664d200640 1 --2- 172.21.15.17:0/3192683886 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f66480c7160 0x7f66480c93e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f663800d9b0 tx=0x7f663800de70 comp rx=0 tx=0).stop 2024-09-10T21:35:43.201 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.196+0000 7f664d200640 1 -- 172.21.15.17:0/3192683886 shutdown_connections 2024-09-10T21:35:43.201 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.196+0000 7f664d200640 1 --2- 172.21.15.17:0/3192683886 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f66280725e0 0x7f6628074aa0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:43.201 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.196+0000 7f664d200640 1 --2- 172.21.15.17:0/3192683886 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6648150cc0 0x7f664814c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:43.201 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.196+0000 7f664d200640 1 --2- 172.21.15.17:0/3192683886 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f66481508e0 0x7f664814bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:43.201 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.196+0000 7f664d200640 1 --2- 172.21.15.17:0/3192683886 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f66480c7160 0x7f66480c93e0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:43.201 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.196+0000 7f664d200640 1 -- 172.21.15.17:0/3192683886 >> 172.21.15.17:0/3192683886 conn(0x7f664806cdf0 msgr2=0x7f66480c4a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:43.201 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.196+0000 7f664d200640 1 -- 172.21.15.17:0/3192683886 shutdown_connections 2024-09-10T21:35:43.201 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.196+0000 7f664d200640 1 -- 172.21.15.17:0/3192683886 wait complete. 2024-09-10T21:35:43.224 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-09-10T21:35:43.225 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-09-10T21:35:43.225 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-10T21:35:43.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.388+0000 7f2ef1a00640 1 -- 172.21.15.17:0/692250122 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2eec0cdf20 msgr2=0x7f2eec0ce350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:43.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.388+0000 7f2ef1a00640 1 --2- 172.21.15.17:0/692250122 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2eec0cdf20 0x7f2eec0ce350 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f2eec1594a0 tx=0x7f2ee401cb10 comp rx=0 tx=0).stop 2024-09-10T21:35:43.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.388+0000 7f2ef1a00640 1 -- 172.21.15.17:0/692250122 shutdown_connections 2024-09-10T21:35:43.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.388+0000 7f2ef1a00640 1 --2- 172.21.15.17:0/692250122 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2eec0cdf20 0x7f2eec0ce350 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:43.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.388+0000 7f2ef1a00640 1 --2- 172.21.15.17:0/692250122 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f2eec152440 0x7f2eec0b3eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:43.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.388+0000 7f2ef1a00640 1 -- 172.21.15.17:0/692250122 >> 172.21.15.17:0/692250122 conn(0x7f2eec0702f0 msgr2=0x7f2eec070700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:43.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.388+0000 7f2ef1a00640 1 -- 172.21.15.17:0/692250122 shutdown_connections 2024-09-10T21:35:43.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.388+0000 7f2ef1a00640 1 -- 172.21.15.17:0/692250122 wait complete. 2024-09-10T21:35:43.396 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.388+0000 7f2ef1a00640 1 Processor -- start 2024-09-10T21:35:43.396 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.392+0000 7f2ef1a00640 1 -- start start 2024-09-10T21:35:43.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.392+0000 7f2ef1a00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2eec0cdf20 0x7f2eec162200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:43.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.392+0000 7f2ef1a00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2eec152440 0x7f2eec14c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:43.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.392+0000 7f2ef1a00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f2eec152820 0x7f2eec14af20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:43.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.392+0000 7f2ef1a00640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f2eec0cb5d0 con 0x7f2eec152440 2024-09-10T21:35:43.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.392+0000 7f2ef1a00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f2eec0cb450 con 0x7f2eec0cdf20 2024-09-10T21:35:43.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.392+0000 7f2ef1a00640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f2eec0c2ff0 con 0x7f2eec152820 2024-09-10T21:35:43.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.392+0000 7f2eea000640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2eec152440 0x7f2eec14c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:43.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.392+0000 7f2eea000640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2eec152440 0x7f2eec14c990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:54548/0 (socket says 172.21.15.17:54548) 2024-09-10T21:35:43.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.392+0000 7f2eea000640 1 -- 172.21.15.17:0/1396920669 learned_addr learned my addr 172.21.15.17:0/1396920669 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:35:43.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.392+0000 7f2eeb400640 1 --2- 172.21.15.17:0/1396920669 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f2eec152820 0x7f2eec14af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:43.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.392+0000 7f2eeaa00640 1 --2- 172.21.15.17:0/1396920669 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2eec0cdf20 0x7f2eec162200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:43.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.392+0000 7f2eeb400640 1 -- 172.21.15.17:0/1396920669 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f2eec152820 msgr2=0x7f2eec14af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-09-10T21:35:43.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.392+0000 7f2eeb400640 1 -- 172.21.15.17:0/1396920669 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f2eec152820 msgr2=0x7f2eec14af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-10T21:35:43.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.392+0000 7f2eeb400640 1 --2- 172.21.15.17:0/1396920669 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f2eec152820 0x7f2eec14af20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T21:35:43.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.392+0000 7f2eeb400640 1 --2- 172.21.15.17:0/1396920669 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f2eec152820 0x7f2eec14af20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:35:43.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.392+0000 7f2eea000640 1 -- 172.21.15.17:0/1396920669 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f2eec152820 msgr2=0x7f2eec14af20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:35:43.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.392+0000 7f2eea000640 1 --2- 172.21.15.17:0/1396920669 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f2eec152820 0x7f2eec14af20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:43.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.392+0000 7f2eea000640 1 -- 172.21.15.17:0/1396920669 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2eec0cdf20 msgr2=0x7f2eec162200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:43.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.392+0000 7f2eea000640 1 --2- 172.21.15.17:0/1396920669 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2eec0cdf20 0x7f2eec162200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:43.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.392+0000 7f2eea000640 1 -- 172.21.15.17:0/1396920669 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2eec0ce420 con 0x7f2eec152440 2024-09-10T21:35:43.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.392+0000 7f2eeaa00640 1 --2- 172.21.15.17:0/1396920669 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2eec0cdf20 0x7f2eec162200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:35:43.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.392+0000 7f2eea000640 1 --2- 172.21.15.17:0/1396920669 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2eec152440 0x7f2eec14c990 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f2ed800c500 tx=0x7f2ed800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:43.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.392+0000 7f2ee2a00640 1 -- 172.21.15.17:0/1396920669 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ed8017070 con 0x7f2eec152440 2024-09-10T21:35:43.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.392+0000 7f2ee2a00640 1 -- 172.21.15.17:0/1396920669 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2ed80053c0 con 0x7f2eec152440 2024-09-10T21:35:43.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.392+0000 7f2ef1a00640 1 -- 172.21.15.17:0/1396920669 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2eec0ccc10 con 0x7f2eec152440 2024-09-10T21:35:43.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.392+0000 7f2ee2a00640 1 -- 172.21.15.17:0/1396920669 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ed8012890 con 0x7f2eec152440 2024-09-10T21:35:43.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.392+0000 7f2ef1a00640 1 -- 172.21.15.17:0/1396920669 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2eec14e740 con 0x7f2eec152440 2024-09-10T21:35:43.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.396+0000 7f2ef1a00640 1 --2- 172.21.15.17:0/1396920669 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2eec06e2f0 0x7f2eec14b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:43.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.396+0000 7f2ef1a00640 1 -- 172.21.15.17:0/1396920669 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f2eec162740 con 0x7f2eec06e2f0 2024-09-10T21:35:43.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.396+0000 7f2eeaa00640 1 --2- 172.21.15.17:0/1396920669 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2eec06e2f0 0x7f2eec14b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:43.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.396+0000 7f2eeaa00640 1 -- 172.21.15.17:0/1396920669 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2eec0b3f00 con 0x7f2eec06e2f0 2024-09-10T21:35:43.401 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.396+0000 7f2eeaa00640 1 --2- 172.21.15.17:0/1396920669 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2eec06e2f0 0x7f2eec14b460 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f2edc00bec0 tx=0x7f2edc00e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:43.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.396+0000 7f2ee2a00640 1 -- 172.21.15.17:0/1396920669 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2edc01c680 con 0x7f2eec06e2f0 2024-09-10T21:35:43.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.396+0000 7f2ee2a00640 1 -- 172.21.15.17:0/1396920669 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f2ed8003600 con 0x7f2eec152440 2024-09-10T21:35:43.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.396+0000 7f2ee2a00640 1 --2- 172.21.15.17:0/1396920669 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f2ecc0724e0 0x7f2ecc0749a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:43.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.396+0000 7f2ee2a00640 1 -- 172.21.15.17:0/1396920669 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f2ed8092c90 con 0x7f2eec152440 2024-09-10T21:35:43.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.396+0000 7f2eeb400640 1 --2- 172.21.15.17:0/1396920669 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f2ecc0724e0 0x7f2ecc0749a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:43.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.396+0000 7f2ee2a00640 1 -- 172.21.15.17:0/1396920669 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f2edc01c820 con 0x7f2eec06e2f0 2024-09-10T21:35:43.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.396+0000 7f2ee2a00640 1 -- 172.21.15.17:0/1396920669 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2eec06e2f0 msgr2=0x7f2eec14b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:43.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.396+0000 7f2ee2a00640 1 --2- 172.21.15.17:0/1396920669 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2eec06e2f0 0x7f2eec14b460 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f2edc00bec0 tx=0x7f2edc00e720 comp rx=0 tx=0).stop 2024-09-10T21:35:43.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.396+0000 7f2ee2a00640 1 -- 172.21.15.17:0/1396920669 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2eec06e2f0 msgr2=0x7f2eec14b460 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:35:43.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.396+0000 7f2ee2a00640 1 --2- 172.21.15.17:0/1396920669 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2eec06e2f0 0x7f2eec14b460 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:43.404 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.396+0000 7f2eeb400640 1 --2- 172.21.15.17:0/1396920669 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f2ecc0724e0 0x7f2ecc0749a0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f2ee40af000 tx=0x7f2ee401b350 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:43.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.408+0000 7f2ef1a00640 1 --2- 172.21.15.17:0/1396920669 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2eec14d230 0x7f2eec14d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:43.414 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.408+0000 7f2ef1a00640 1 -- 172.21.15.17:0/1396920669 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f2eec14b9a0 con 0x7f2eec14d230 2024-09-10T21:35:43.414 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.408+0000 7f2eeaa00640 1 --2- 172.21.15.17:0/1396920669 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2eec14d230 0x7f2eec14d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:43.414 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.408+0000 7f2eeaa00640 1 -- 172.21.15.17:0/1396920669 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f2eec158360 con 0x7f2eec14d230 2024-09-10T21:35:43.415 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.408+0000 7f2eeaa00640 1 --2- 172.21.15.17:0/1396920669 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2eec14d230 0x7f2eec14d610 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f2edc011010 tx=0x7f2edc0046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:43.415 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.408+0000 7f2ee2a00640 1 -- 172.21.15.17:0/1396920669 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2edc027070 con 0x7f2eec14d230 2024-09-10T21:35:43.415 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.408+0000 7f2ee2a00640 1 -- 172.21.15.17:0/1396920669 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5044 (secure 0 0 0) 0x7f2edc011cd0 con 0x7f2eec14d230 2024-09-10T21:35:43.415 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.408+0000 7f2ee2a00640 1 -- 172.21.15.17:0/1396920669 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2eec14d230 msgr2=0x7f2eec14d610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:43.415 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.408+0000 7f2ee2a00640 1 --2- 172.21.15.17:0/1396920669 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2eec14d230 0x7f2eec14d610 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f2edc011010 tx=0x7f2edc0046d0 comp rx=0 tx=0).stop 2024-09-10T21:35:43.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.408+0000 7f2ee2a00640 1 -- 172.21.15.17:0/1396920669 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2eec14d230 msgr2=0x7f2eec14d610 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:35:43.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.408+0000 7f2ee2a00640 1 --2- 172.21.15.17:0/1396920669 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2eec14d230 0x7f2eec14d610 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:43.416 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-10T21:35:43.416 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:35:43.416 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:35:43.416 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "leader", 2024-09-10T21:35:43.416 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 92, 2024-09-10T21:35:43.416 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 124014, 2024-09-10T21:35:43.416 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-10T21:35:43.417 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-10T21:35:43.417 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-10T21:35:43.417 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-10T21:35:43.417 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:35:43.417 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 2, 2024-09-10T21:35:43.417 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:35:43.417 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-10T21:35:43.417 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-10T21:35:43.417 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:35:43.417 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:35:43.417 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:35:43.417 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:35:43.417 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:35:43.417 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:35:43.418 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:35:43.418 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:35:43.418 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:35:43.418 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:35:43.418 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:35:43.418 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:35:43.418 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-10T21:35:43.418 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-10T21:35:43.418 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:35:43.418 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:35:43.418 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:35:43.418 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:35:43.418 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:35:43.418 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:35:43.418 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:35:43.419 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:35:43.419 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:35:43.419 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:35:43.419 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:35:43.419 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:35:43.419 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:35:43.419 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-10T21:35:43.419 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-10T21:35:43.419 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-10T21:35:43.419 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-10T21:35:43.419 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-10T21:35:43.419 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "5e2f023f-d6a6-49f4-a256-f4807dce3739", 2024-09-10T21:35:43.419 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:35:43.419 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:35:43.419 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-10T21:35:43.420 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-10T21:35:43.420 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-10T21:35:43.420 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-10T21:35:43.420 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-10T21:35:43.420 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-10T21:35:43.420 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-10T21:35:43.420 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:35:43.420 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-10T21:35:43.420 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:35:43.420 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:35:43.420 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:35:43.420 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:35:43.420 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:35:43.420 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:35:43.421 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:35:43.421 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:35:43.421 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:35:43.421 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:35:43.421 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:35:43.421 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:35:43.421 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-10T21:35:43.421 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:35:43.421 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-10T21:35:43.421 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:35:43.421 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:35:43.421 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:35:43.421 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:35:43.421 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:35:43.422 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:35:43.422 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:35:43.422 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-10T21:35:43.422 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:35:43.422 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:35:43.422 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:35:43.422 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:35:43.422 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-10T21:35:43.422 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:35:43.422 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:35:43.422 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:35:43.422 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:35:43.422 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-10T21:35:43.422 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-10T21:35:43.422 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:35:43.423 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:35:43.423 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:35:43.423 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:35:43.423 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:35:43.423 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:35:43.423 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:35:43.423 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:35:43.423 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:35:43.423 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:35:43.423 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:35:43.423 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3300", 2024-09-10T21:35:43.423 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:35:43.423 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:35:43.423 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:35:43.423 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:35:43.423 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789", 2024-09-10T21:35:43.424 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:35:43.424 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:35:43.424 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:35:43.424 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:35:43.424 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789/0", 2024-09-10T21:35:43.424 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6789/0", 2024-09-10T21:35:43.424 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:35:43.424 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:35:43.424 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:35:43.424 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:35:43.425 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:35:43.425 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:35:43.425 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:35:43.425 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:35:43.425 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:35:43.425 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:35:43.425 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:35:43.425 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3301", 2024-09-10T21:35:43.425 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:35:43.425 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:35:43.425 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:35:43.425 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:35:43.425 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790", 2024-09-10T21:35:43.425 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:35:43.426 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:35:43.426 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:35:43.426 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:35:43.426 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790/0", 2024-09-10T21:35:43.426 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6790/0", 2024-09-10T21:35:43.426 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:35:43.426 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:35:43.426 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:35:43.426 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:35:43.426 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:35:43.426 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:35:43.426 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-10T21:35:43.426 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-10T21:35:43.426 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:35:43.426 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:35:43.427 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:35:43.427 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:35:43.427 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:35:43.427 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:35:43.427 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": [ 2024-09-10T21:35:43.427 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:35:43.427 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:35:43.427 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:35:43.427 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 6 2024-09-10T21:35:43.427 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:35:43.427 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:35:43.427 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-10T21:35:43.427 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:35:43.427 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:35:43.427 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:35:43.427 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 8 2024-09-10T21:35:43.428 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:35:43.428 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:35:43.428 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-10T21:35:43.428 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:35:43.428 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:35:43.428 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:35:43.428 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 6 2024-09-10T21:35:43.428 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:35:43.428 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:35:43.428 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": [ 2024-09-10T21:35:43.428 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:35:43.428 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:35:43.428 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:35:43.428 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 2 2024-09-10T21:35:43.429 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:35:43.429 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:35:43.429 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:35:43.429 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-10T21:35:43.429 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-10T21:35:43.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.412+0000 7f2ef1a00640 1 -- 172.21.15.17:0/1396920669 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f2ecc0724e0 msgr2=0x7f2ecc0749a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:43.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.412+0000 7f2ef1a00640 1 --2- 172.21.15.17:0/1396920669 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f2ecc0724e0 0x7f2ecc0749a0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f2ee40af000 tx=0x7f2ee401b350 comp rx=0 tx=0).stop 2024-09-10T21:35:43.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.412+0000 7f2ef1a00640 1 -- 172.21.15.17:0/1396920669 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2eec152440 msgr2=0x7f2eec14c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:43.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.412+0000 7f2ef1a00640 1 --2- 172.21.15.17:0/1396920669 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2eec152440 0x7f2eec14c990 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f2ed800c500 tx=0x7f2ed800c9c0 comp rx=0 tx=0).stop 2024-09-10T21:35:43.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.412+0000 7f2eeb400640 1 -- 172.21.15.17:0/1396920669 reap_dead start 2024-09-10T21:35:43.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.412+0000 7f2eeb400640 1 -- 172.21.15.17:0/1396920669 reap_dead start 2024-09-10T21:35:43.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.412+0000 7f2ef1a00640 1 -- 172.21.15.17:0/1396920669 shutdown_connections 2024-09-10T21:35:43.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.412+0000 7f2ef1a00640 1 -- 172.21.15.17:0/1396920669 >> 172.21.15.17:0/1396920669 conn(0x7f2eec0702f0 msgr2=0x7f2eec0706d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:43.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.412+0000 7f2ef1a00640 1 -- 172.21.15.17:0/1396920669 shutdown_connections 2024-09-10T21:35:43.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.412+0000 7f2ef1a00640 1 -- 172.21.15.17:0/1396920669 wait complete. 2024-09-10T21:35:43.441 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-10T21:35:43.441 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-09-10T21:35:43.441 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-10T21:35:43.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.620+0000 7f3046e00640 1 -- 172.21.15.17:0/3950422848 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3040152820 msgr2=0x7f30400b3eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:43.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.620+0000 7f3046e00640 1 --2- 172.21.15.17:0/3950422848 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3040152820 0x7f30400b3eb0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f30400c9850 tx=0x7f302c01c830 comp rx=0 tx=0).stop 2024-09-10T21:35:43.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.620+0000 7f3046e00640 1 -- 172.21.15.17:0/3950422848 shutdown_connections 2024-09-10T21:35:43.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.620+0000 7f3046e00640 1 --2- 172.21.15.17:0/3950422848 >> v2:172.21.15.31:3300/0 conn(0x7f30400cdf20 0x7f30400ce350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:43.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.620+0000 7f3046e00640 1 --2- 172.21.15.17:0/3950422848 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3040152820 0x7f30400b3eb0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:43.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.620+0000 7f3046e00640 1 -- 172.21.15.17:0/3950422848 >> 172.21.15.17:0/3950422848 conn(0x7f30400702f0 msgr2=0x7f3040070700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:43.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.620+0000 7f3046e00640 1 -- 172.21.15.17:0/3950422848 shutdown_connections 2024-09-10T21:35:43.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.620+0000 7f3046e00640 1 -- 172.21.15.17:0/3950422848 wait complete. 2024-09-10T21:35:43.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.620+0000 7f3046e00640 1 Processor -- start 2024-09-10T21:35:43.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.620+0000 7f3046e00640 1 -- start start 2024-09-10T21:35:43.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.620+0000 7f3046e00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f30400cdf20 0x7f304006e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:43.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.620+0000 7f3046e00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3040152440 0x7f3040144a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:43.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.620+0000 7f3046e00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3040152820 0x7f30401f35f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:43.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.620+0000 7f3046e00640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f30400cb5d0 con 0x7f3040152440 2024-09-10T21:35:43.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.620+0000 7f3046e00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f30400cb450 con 0x7f3040152820 2024-09-10T21:35:43.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.620+0000 7f3046e00640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f30400c2ff0 con 0x7f30400cdf20 2024-09-10T21:35:43.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.620+0000 7f303f400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3040152440 0x7f3040144a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:43.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.620+0000 7f303f400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3040152440 0x7f3040144a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:54578/0 (socket says 172.21.15.17:54578) 2024-09-10T21:35:43.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.620+0000 7f303f400640 1 -- 172.21.15.17:0/1674984894 learned_addr learned my addr 172.21.15.17:0/1674984894 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:35:43.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.620+0000 7f3044e00640 1 --2- 172.21.15.17:0/1674984894 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3040152820 0x7f30401f35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:43.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.620+0000 7f303fe00640 1 --2- 172.21.15.17:0/1674984894 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f30400cdf20 0x7f304006e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:43.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.620+0000 7f303f400640 1 -- 172.21.15.17:0/1674984894 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f30400cdf20 msgr2=0x7f304006e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:43.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.620+0000 7f303fe00640 1 -- 172.21.15.17:0/1674984894 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f30400cdf20 msgr2=0x7f304006e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-09-10T21:35:43.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.620+0000 7f303fe00640 1 -- 172.21.15.17:0/1674984894 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f30400cdf20 msgr2=0x7f304006e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-10T21:35:43.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.620+0000 7f303fe00640 1 --2- 172.21.15.17:0/1674984894 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f30400cdf20 0x7f304006e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T21:35:43.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.620+0000 7f303fe00640 1 --2- 172.21.15.17:0/1674984894 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f30400cdf20 0x7f304006e640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:35:43.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.620+0000 7f303f400640 1 --2- 172.21.15.17:0/1674984894 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f30400cdf20 0x7f304006e640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:43.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.620+0000 7f303f400640 1 -- 172.21.15.17:0/1674984894 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3040152820 msgr2=0x7f30401f35f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:43.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.620+0000 7f303f400640 1 --2- 172.21.15.17:0/1674984894 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3040152820 0x7f30401f35f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:43.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.620+0000 7f303f400640 1 -- 172.21.15.17:0/1674984894 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3040149bb0 con 0x7f3040152440 2024-09-10T21:35:43.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.624+0000 7f303f400640 1 --2- 172.21.15.17:0/1674984894 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3040152440 0x7f3040144a80 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f302c0b0040 tx=0x7f302c0a56b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:43.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.624+0000 7f3037e00640 1 -- 172.21.15.17:0/1674984894 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f302c0b2040 con 0x7f3040152440 2024-09-10T21:35:43.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.624+0000 7f3037e00640 1 -- 172.21.15.17:0/1674984894 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f302c005070 con 0x7f3040152440 2024-09-10T21:35:43.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.624+0000 7f3046e00640 1 -- 172.21.15.17:0/1674984894 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3040149190 con 0x7f3040152440 2024-09-10T21:35:43.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.624+0000 7f3037e00640 1 -- 172.21.15.17:0/1674984894 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f302c0044f0 con 0x7f3040152440 2024-09-10T21:35:43.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.624+0000 7f3046e00640 1 -- 172.21.15.17:0/1674984894 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3040149750 con 0x7f3040152440 2024-09-10T21:35:43.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.624+0000 7f3046e00640 1 --2- 172.21.15.17:0/1674984894 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3040148860 0x7f30401f3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:43.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.624+0000 7f3046e00640 1 -- 172.21.15.17:0/1674984894 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f30401f32f0 con 0x7f3040148860 2024-09-10T21:35:43.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.624+0000 7f303fe00640 1 --2- 172.21.15.17:0/1674984894 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3040148860 0x7f30401f3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:43.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.624+0000 7f303fe00640 1 -- 172.21.15.17:0/1674984894 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3040158360 con 0x7f3040148860 2024-09-10T21:35:43.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.624+0000 7f303fe00640 1 --2- 172.21.15.17:0/1674984894 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3040148860 0x7f30401f3b30 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f303000e4d0 tx=0x7f303000e990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:43.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.624+0000 7f3037e00640 1 -- 172.21.15.17:0/1674984894 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f303001a070 con 0x7f3040148860 2024-09-10T21:35:43.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.624+0000 7f3037e00640 1 -- 172.21.15.17:0/1674984894 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f302c0054c0 con 0x7f3040152440 2024-09-10T21:35:43.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.628+0000 7f3037e00640 1 --2- 172.21.15.17:0/1674984894 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f30200724e0 0x7f30200749a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:43.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.628+0000 7f3037e00640 1 -- 172.21.15.17:0/1674984894 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f302c12cc30 con 0x7f3040152440 2024-09-10T21:35:43.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.628+0000 7f3037e00640 1 -- 172.21.15.17:0/1674984894 <== mon.1 v2:172.21.15.31:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f3030007240 con 0x7f3040148860 2024-09-10T21:35:43.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.628+0000 7f3037e00640 1 -- 172.21.15.17:0/1674984894 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3040148860 msgr2=0x7f30401f3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:43.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.628+0000 7f3037e00640 1 --2- 172.21.15.17:0/1674984894 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3040148860 0x7f30401f3b30 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f303000e4d0 tx=0x7f303000e990 comp rx=0 tx=0).stop 2024-09-10T21:35:43.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.628+0000 7f3037e00640 1 -- 172.21.15.17:0/1674984894 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3040148860 msgr2=0x7f30401f3b30 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:35:43.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.628+0000 7f3037e00640 1 --2- 172.21.15.17:0/1674984894 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3040148860 0x7f30401f3b30 secure :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f303000e4d0 tx=0x7f303000e990 comp rx=0 tx=0).stop 2024-09-10T21:35:43.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.628+0000 7f3044e00640 1 --2- 172.21.15.17:0/1674984894 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f30200724e0 0x7f30200749a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:43.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.628+0000 7f3044e00640 1 --2- 172.21.15.17:0/1674984894 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f30200724e0 0x7f30200749a0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f304006f290 tx=0x7f3038009530 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:43.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.632+0000 7f3046e00640 1 --2- 172.21.15.17:0/1674984894 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f30401f6810 0x7f30401f6c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:43.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.632+0000 7f3046e00640 1 -- 172.21.15.17:0/1674984894 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f30400ca0e0 con 0x7f30401f6810 2024-09-10T21:35:43.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.632+0000 7f303fe00640 1 --2- 172.21.15.17:0/1674984894 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f30401f6810 0x7f30401f6c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:43.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.636+0000 7f303fe00640 1 -- 172.21.15.17:0/1674984894 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f30400c9e00 con 0x7f30401f6810 2024-09-10T21:35:43.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.636+0000 7f303fe00640 1 --2- 172.21.15.17:0/1674984894 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f30401f6810 0x7f30401f6c20 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f303000e4d0 tx=0x7f3030003770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:43.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.636+0000 7f3037e00640 1 -- 172.21.15.17:0/1674984894 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30300067e0 con 0x7f30401f6810 2024-09-10T21:35:43.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.636+0000 7f3037e00640 1 -- 172.21.15.17:0/1674984894 <== mon.1 v2:172.21.15.31:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4540 (secure 0 0 0) 0x7f30300067e0 con 0x7f30401f6810 2024-09-10T21:35:43.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.636+0000 7f3037e00640 1 -- 172.21.15.17:0/1674984894 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f30401f6810 msgr2=0x7f30401f6c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:43.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.636+0000 7f3037e00640 1 --2- 172.21.15.17:0/1674984894 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f30401f6810 0x7f30401f6c20 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f303000e4d0 tx=0x7f3030003770 comp rx=0 tx=0).stop 2024-09-10T21:35:43.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.636+0000 7f3037e00640 1 -- 172.21.15.17:0/1674984894 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f30401f6810 msgr2=0x7f30401f6c20 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:35:43.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.636+0000 7f3037e00640 1 --2- 172.21.15.17:0/1674984894 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f30401f6810 0x7f30401f6c20 secure :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f303000e4d0 tx=0x7f3030003770 comp rx=0 tx=0).stop 2024-09-10T21:35:43.642 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-10T21:35:43.642 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:35:43.642 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:35:43.642 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-10T21:35:43.642 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 92, 2024-09-10T21:35:43.642 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 63023, 2024-09-10T21:35:43.642 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-10T21:35:43.642 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-10T21:35:43.642 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-10T21:35:43.642 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-10T21:35:43.642 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:35:43.642 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 2, 2024-09-10T21:35:43.643 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:35:43.643 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-10T21:35:43.643 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-10T21:35:43.643 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:35:43.643 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:35:43.643 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:35:43.643 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:35:43.643 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:35:43.643 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:35:43.643 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:35:43.643 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:35:43.643 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:35:43.643 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:35:43.643 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:35:43.644 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:35:43.644 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-10T21:35:43.644 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-10T21:35:43.644 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:35:43.644 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:35:43.644 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:35:43.644 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:35:43.644 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:35:43.644 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:35:43.644 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:35:43.644 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:35:43.644 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:35:43.644 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:35:43.644 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:35:43.644 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:35:43.645 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:35:43.645 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-10T21:35:43.645 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-10T21:35:43.645 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-10T21:35:43.645 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-10T21:35:43.645 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-10T21:35:43.645 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "5e2f023f-d6a6-49f4-a256-f4807dce3739", 2024-09-10T21:35:43.645 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:35:43.645 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:35:43.645 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-10T21:35:43.645 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-10T21:35:43.645 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-10T21:35:43.645 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-10T21:35:43.645 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-10T21:35:43.645 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-10T21:35:43.646 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-10T21:35:43.646 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:35:43.646 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-10T21:35:43.646 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:35:43.646 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:35:43.646 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:35:43.646 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:35:43.646 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:35:43.646 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:35:43.646 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:35:43.646 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:35:43.646 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:35:43.646 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:35:43.647 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:35:43.647 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:35:43.647 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-10T21:35:43.647 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:35:43.647 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-10T21:35:43.647 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:35:43.647 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:35:43.647 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:35:43.647 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:35:43.647 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:35:43.647 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:35:43.647 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:35:43.647 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-10T21:35:43.648 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:35:43.648 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:35:43.648 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:35:43.648 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:35:43.648 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-10T21:35:43.648 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:35:43.648 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:35:43.648 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:35:43.648 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:35:43.648 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-10T21:35:43.648 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-10T21:35:43.648 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:35:43.648 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:35:43.649 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:35:43.649 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:35:43.649 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:35:43.649 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:35:43.649 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:35:43.649 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:35:43.649 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:35:43.649 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:35:43.649 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:35:43.649 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3300", 2024-09-10T21:35:43.649 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:35:43.649 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:35:43.649 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:35:43.650 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:35:43.650 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789", 2024-09-10T21:35:43.650 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:35:43.650 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:35:43.650 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:35:43.650 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:35:43.650 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789/0", 2024-09-10T21:35:43.650 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6789/0", 2024-09-10T21:35:43.650 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:35:43.650 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:35:43.650 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:35:43.650 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:35:43.650 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:35:43.651 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:35:43.651 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:35:43.651 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:35:43.651 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:35:43.651 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:35:43.651 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:35:43.651 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3301", 2024-09-10T21:35:43.651 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:35:43.651 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:35:43.651 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:35:43.651 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:35:43.651 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790", 2024-09-10T21:35:43.651 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:35:43.652 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:35:43.652 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:35:43.652 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:35:43.652 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790/0", 2024-09-10T21:35:43.652 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6790/0", 2024-09-10T21:35:43.652 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:35:43.652 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:35:43.652 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:35:43.652 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:35:43.652 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:35:43.652 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:35:43.652 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-10T21:35:43.652 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-10T21:35:43.652 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:35:43.653 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:35:43.653 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:35:43.653 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:35:43.653 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:35:43.653 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:35:43.653 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-10T21:35:43.653 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:35:43.653 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:35:43.653 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:35:43.653 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 2 2024-09-10T21:35:43.653 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:35:43.653 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:35:43.653 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:35:43.654 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-10T21:35:43.654 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-10T21:35:43.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.636+0000 7f3046e00640 1 -- 172.21.15.17:0/1674984894 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f30200724e0 msgr2=0x7f30200749a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:43.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.636+0000 7f3046e00640 1 --2- 172.21.15.17:0/1674984894 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f30200724e0 0x7f30200749a0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f304006f290 tx=0x7f3038009530 comp rx=0 tx=0).stop 2024-09-10T21:35:43.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.636+0000 7f3046e00640 1 -- 172.21.15.17:0/1674984894 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3040152440 msgr2=0x7f3040144a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:43.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.636+0000 7f3046e00640 1 --2- 172.21.15.17:0/1674984894 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3040152440 0x7f3040144a80 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f302c0b0040 tx=0x7f302c0a56b0 comp rx=0 tx=0).stop 2024-09-10T21:35:43.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.636+0000 7f3044e00640 1 -- 172.21.15.17:0/1674984894 reap_dead start 2024-09-10T21:35:43.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.636+0000 7f3046e00640 1 -- 172.21.15.17:0/1674984894 shutdown_connections 2024-09-10T21:35:43.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.636+0000 7f3046e00640 1 --2- 172.21.15.17:0/1674984894 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3040152440 0x7f3040144a80 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:43.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.636+0000 7f3046e00640 1 -- 172.21.15.17:0/1674984894 >> 172.21.15.17:0/1674984894 conn(0x7f30400702f0 msgr2=0x7f30400c3370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:43.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.636+0000 7f3046e00640 1 -- 172.21.15.17:0/1674984894 shutdown_connections 2024-09-10T21:35:43.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.636+0000 7f3046e00640 1 -- 172.21.15.17:0/1674984894 wait complete. 2024-09-10T21:35:43.669 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-10T21:35:43.669 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-09-10T21:35:43.669 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-10T21:35:43.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.848+0000 7f90e5400640 1 -- 172.21.15.17:0/3994410936 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f90c4004d50 msgr2=0x7f90c40051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:43.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.848+0000 7f90e5400640 1 --2- 172.21.15.17:0/3994410936 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f90c4004d50 0x7f90c40051d0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f90d8009df0 tx=0x7f90d801d150 comp rx=0 tx=0).stop 2024-09-10T21:35:43.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.848+0000 7f90e4a00640 1 -- 172.21.15.17:0/3994410936 reap_dead start 2024-09-10T21:35:43.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.848+0000 7f90e5400640 1 -- 172.21.15.17:0/3994410936 shutdown_connections 2024-09-10T21:35:43.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.848+0000 7f90e5400640 1 -- 172.21.15.17:0/3994410936 >> 172.21.15.17:0/3994410936 conn(0x7f90e0068990 msgr2=0x7f90e0068da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:43.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.848+0000 7f90e5400640 1 -- 172.21.15.17:0/3994410936 shutdown_connections 2024-09-10T21:35:43.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.848+0000 7f90e5400640 1 -- 172.21.15.17:0/3994410936 wait complete. 2024-09-10T21:35:43.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.852+0000 7f90e5400640 1 Processor -- start 2024-09-10T21:35:43.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.852+0000 7f90e5400640 1 -- start start 2024-09-10T21:35:43.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.852+0000 7f90e5400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f90c4004d50 0x7f90e013fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:43.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.852+0000 7f90e5400640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f90e014acc0 0x7f90e00cbe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:43.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.852+0000 7f90e5400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f90e014cc20 0x7f90e014d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:43.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.852+0000 7f90e5400640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f90e0071320 con 0x7f90c4004d50 2024-09-10T21:35:43.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.852+0000 7f90e5400640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f90e00cb3c0 con 0x7f90e014cc20 2024-09-10T21:35:43.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.852+0000 7f90e5400640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f90e00714a0 con 0x7f90e014acc0 2024-09-10T21:35:43.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.852+0000 7f90dfe00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f90c4004d50 0x7f90e013fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:43.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.852+0000 7f90dfe00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f90c4004d50 0x7f90e013fdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:54594/0 (socket says 172.21.15.17:54594) 2024-09-10T21:35:43.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.852+0000 7f90dfe00640 1 -- 172.21.15.17:0/447493474 learned_addr learned my addr 172.21.15.17:0/447493474 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:35:43.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.852+0000 7f90df400640 1 --2- 172.21.15.17:0/447493474 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f90e014acc0 0x7f90e00cbe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:43.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.852+0000 7f90e4a00640 1 --2- 172.21.15.17:0/447493474 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f90e014cc20 0x7f90e014d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:43.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.852+0000 7f90dfe00640 1 -- 172.21.15.17:0/447493474 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f90e014acc0 msgr2=0x7f90e00cbe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:43.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.852+0000 7f90dfe00640 1 --2- 172.21.15.17:0/447493474 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f90e014acc0 0x7f90e00cbe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:43.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.852+0000 7f90dfe00640 1 -- 172.21.15.17:0/447493474 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f90e014cc20 msgr2=0x7f90e014d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:43.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.852+0000 7f90dfe00640 1 --2- 172.21.15.17:0/447493474 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f90e014cc20 0x7f90e014d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:43.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.852+0000 7f90dfe00640 1 -- 172.21.15.17:0/447493474 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90e0143e30 con 0x7f90c4004d50 2024-09-10T21:35:43.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.852+0000 7f90e4a00640 1 --2- 172.21.15.17:0/447493474 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f90e014cc20 0x7f90e014d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:35:43.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.852+0000 7f90dfe00640 1 --2- 172.21.15.17:0/447493474 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f90c4004d50 0x7f90e013fdf0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f90d0011a80 tx=0x7f90d0011f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:43.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.852+0000 7f90d7e00640 1 -- 172.21.15.17:0/447493474 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90d000fe10 con 0x7f90c4004d50 2024-09-10T21:35:43.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.852+0000 7f90d7e00640 1 -- 172.21.15.17:0/447493474 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f90d0016070 con 0x7f90c4004d50 2024-09-10T21:35:43.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.852+0000 7f90d7e00640 1 -- 172.21.15.17:0/447493474 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90d0012780 con 0x7f90c4004d50 2024-09-10T21:35:43.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.852+0000 7f90e5400640 1 -- 172.21.15.17:0/447493474 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90e01f4cf0 con 0x7f90c4004d50 2024-09-10T21:35:43.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.852+0000 7f90e5400640 1 -- 172.21.15.17:0/447493474 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90e01f52b0 con 0x7f90c4004d50 2024-09-10T21:35:43.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.856+0000 7f90e5400640 1 --2- 172.21.15.17:0/447493474 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f90e014b0a0 0x7f90e01f1570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:43.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.856+0000 7f90e5400640 1 -- 172.21.15.17:0/447493474 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f90e0069ef0 con 0x7f90e014b0a0 2024-09-10T21:35:43.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.856+0000 7f90df400640 1 --2- 172.21.15.17:0/447493474 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f90e014b0a0 0x7f90e01f1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:43.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.856+0000 7f90df400640 1 -- 172.21.15.17:0/447493474 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f90e014b0a0 msgr2=0x7f90e01f1570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 20 2024-09-10T21:35:43.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.856+0000 7f90df400640 1 -- 172.21.15.17:0/447493474 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f90e014b0a0 msgr2=0x7f90e01f1570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-10T21:35:43.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.856+0000 7f90df400640 1 --2- 172.21.15.17:0/447493474 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f90e014b0a0 0x7f90e01f1570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T21:35:43.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.856+0000 7f90df400640 1 --2- 172.21.15.17:0/447493474 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f90e014b0a0 0x7f90e01f1570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:35:43.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.856+0000 7f90d7e00640 1 -- 172.21.15.17:0/447493474 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f90d0012920 con 0x7f90c4004d50 2024-09-10T21:35:43.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.856+0000 7f90d7e00640 1 --2- 172.21.15.17:0/447493474 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f90c00725e0 0x7f90c0074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:43.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.860+0000 7f90d7e00640 1 -- 172.21.15.17:0/447493474 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f90d0094160 con 0x7f90c4004d50 2024-09-10T21:35:43.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.860+0000 7f90e4a00640 1 --2- 172.21.15.17:0/447493474 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f90c00725e0 0x7f90c0074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:43.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:43.860+0000 7f90e4a00640 1 --2- 172.21.15.17:0/447493474 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f90c00725e0 0x7f90c0074aa0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f90d80ad000 tx=0x7f90d8019ef0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:44.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:44.060+0000 7f90df400640 1 --2- 172.21.15.17:0/447493474 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f90e014b0a0 0x7f90e01f1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:44.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:44.060+0000 7f90df400640 1 -- 172.21.15.17:0/447493474 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f90e014b0a0 msgr2=0x7f90e01f1570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 21 2024-09-10T21:35:44.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:44.060+0000 7f90df400640 1 -- 172.21.15.17:0/447493474 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f90e014b0a0 msgr2=0x7f90e01f1570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-10T21:35:44.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:44.060+0000 7f90df400640 1 --2- 172.21.15.17:0/447493474 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f90e014b0a0 0x7f90e01f1570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T21:35:44.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:44.060+0000 7f90df400640 1 --2- 172.21.15.17:0/447493474 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f90e014b0a0 0x7f90e01f1570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-10T21:35:44.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:44.464+0000 7f90df400640 1 --2- 172.21.15.17:0/447493474 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f90e014b0a0 0x7f90e01f1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:44.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:44.464+0000 7f90df400640 1 -- 172.21.15.17:0/447493474 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f90e014b0a0 msgr2=0x7f90e01f1570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 21 2024-09-10T21:35:44.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:44.464+0000 7f90df400640 1 -- 172.21.15.17:0/447493474 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f90e014b0a0 msgr2=0x7f90e01f1570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-10T21:35:44.469 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:44.464+0000 7f90df400640 1 --2- 172.21.15.17:0/447493474 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f90e014b0a0 0x7f90e01f1570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T21:35:44.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:44.464+0000 7f90df400640 1 --2- 172.21.15.17:0/447493474 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f90e014b0a0 0x7f90e01f1570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-10T21:35:45.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:45.268+0000 7f90df400640 1 --2- 172.21.15.17:0/447493474 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f90e014b0a0 0x7f90e01f1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:45.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:45.268+0000 7f90df400640 1 -- 172.21.15.17:0/447493474 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f90e014b0a0 msgr2=0x7f90e01f1570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 21 2024-09-10T21:35:45.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:45.268+0000 7f90df400640 1 -- 172.21.15.17:0/447493474 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f90e014b0a0 msgr2=0x7f90e01f1570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-10T21:35:45.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:45.268+0000 7f90df400640 1 --2- 172.21.15.17:0/447493474 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f90e014b0a0 0x7f90e01f1570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T21:35:45.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:45.268+0000 7f90df400640 1 --2- 172.21.15.17:0/447493474 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f90e014b0a0 0x7f90e01f1570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-10T21:35:46.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:46.871+0000 7f90df400640 1 --2- 172.21.15.17:0/447493474 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f90e014b0a0 0x7f90e01f1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:46.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:46.871+0000 7f90df400640 1 -- 172.21.15.17:0/447493474 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f90e00c9f40 con 0x7f90e014b0a0 2024-09-10T21:35:46.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:46.871+0000 7f90df400640 1 --2- 172.21.15.17:0/447493474 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f90e014b0a0 0x7f90e01f1570 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f90e00cef40 tx=0x7f90cc0039d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:46.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:46.871+0000 7f90d7e00640 1 -- 172.21.15.17:0/447493474 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90cc012850 con 0x7f90e014b0a0 2024-09-10T21:35:46.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:46.871+0000 7f90d7e00640 1 -- 172.21.15.17:0/447493474 <== mon.2 v2:172.21.15.31:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f90cc012850 con 0x7f90e014b0a0 2024-09-10T21:35:46.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:46.871+0000 7f90d7e00640 1 -- 172.21.15.17:0/447493474 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f90e014b0a0 msgr2=0x7f90e01f1570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:46.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:46.871+0000 7f90d7e00640 1 --2- 172.21.15.17:0/447493474 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f90e014b0a0 0x7f90e01f1570 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f90e00cef40 tx=0x7f90cc0039d0 comp rx=0 tx=0).stop 2024-09-10T21:35:46.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:46.871+0000 7f90d7e00640 1 -- 172.21.15.17:0/447493474 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f90e014b0a0 msgr2=0x7f90e01f1570 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:35:46.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:46.871+0000 7f90d7e00640 1 --2- 172.21.15.17:0/447493474 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f90e014b0a0 0x7f90e01f1570 secure :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f90e00cef40 tx=0x7f90cc0039d0 comp rx=0 tx=0).stop 2024-09-10T21:35:46.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:46.879+0000 7f90e5400640 1 --2- 172.21.15.17:0/447493474 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f90e01f5480 0x7f90e01f1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:46.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:46.879+0000 7f90e5400640 1 -- 172.21.15.17:0/447493474 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f90e01435d0 con 0x7f90e01f5480 2024-09-10T21:35:46.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:46.879+0000 7f90df400640 1 --2- 172.21.15.17:0/447493474 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f90e01f5480 0x7f90e01f1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:46.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:46.883+0000 7f90df400640 1 -- 172.21.15.17:0/447493474 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f90e0155b40 con 0x7f90e01f5480 2024-09-10T21:35:46.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:46.883+0000 7f90df400640 1 --2- 172.21.15.17:0/447493474 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f90e01f5480 0x7f90e01f1ab0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f90e00cef40 tx=0x7f90e0158120 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:46.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:46.883+0000 7f90d7e00640 1 -- 172.21.15.17:0/447493474 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90cc023c80 con 0x7f90e01f5480 2024-09-10T21:35:46.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:46.883+0000 7f90d7e00640 1 -- 172.21.15.17:0/447493474 <== mon.2 v2:172.21.15.31:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4540 (secure 0 0 0) 0x7f90cc02c400 con 0x7f90e01f5480 2024-09-10T21:35:46.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:46.883+0000 7f90d7e00640 1 -- 172.21.15.17:0/447493474 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f90e01f5480 msgr2=0x7f90e01f1ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:46.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:46.883+0000 7f90d7e00640 1 --2- 172.21.15.17:0/447493474 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f90e01f5480 0x7f90e01f1ab0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f90e00cef40 tx=0x7f90e0158120 comp rx=0 tx=0).stop 2024-09-10T21:35:46.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:46.883+0000 7f90d7e00640 1 -- 172.21.15.17:0/447493474 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f90e01f5480 msgr2=0x7f90e01f1ab0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:35:46.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:46.883+0000 7f90d7e00640 1 --2- 172.21.15.17:0/447493474 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f90e01f5480 0x7f90e01f1ab0 secure :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f90e00cef40 tx=0x7f90e0158120 comp rx=0 tx=0).stop 2024-09-10T21:35:46.890 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-10T21:35:46.890 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:35:46.890 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:35:46.890 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-10T21:35:46.890 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 92, 2024-09-10T21:35:46.890 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 11319, 2024-09-10T21:35:46.890 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-10T21:35:46.890 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-10T21:35:46.890 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-10T21:35:46.890 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-10T21:35:46.890 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:35:46.891 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 5, 2024-09-10T21:35:46.891 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:35:46.891 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-10T21:35:46.891 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-10T21:35:46.891 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:35:46.891 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:35:46.891 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:35:46.891 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:35:46.891 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:35:46.891 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:35:46.891 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:35:46.891 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:35:46.891 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:35:46.891 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:35:46.892 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:35:46.892 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:35:46.892 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-10T21:35:46.892 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-10T21:35:46.892 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:35:46.892 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:35:46.892 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:35:46.892 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:35:46.892 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:35:46.892 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:35:46.892 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:35:46.892 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:35:46.892 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:35:46.892 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:35:46.892 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:35:46.893 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:35:46.893 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:35:46.893 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-10T21:35:46.893 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-10T21:35:46.893 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-10T21:35:46.893 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-10T21:35:46.893 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-10T21:35:46.893 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "5e2f023f-d6a6-49f4-a256-f4807dce3739", 2024-09-10T21:35:46.893 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:35:46.893 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:35:46.893 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-10T21:35:46.893 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-10T21:35:46.893 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-10T21:35:46.893 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-10T21:35:46.893 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-10T21:35:46.894 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-10T21:35:46.894 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-10T21:35:46.894 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:35:46.894 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-10T21:35:46.894 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:35:46.894 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:35:46.894 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:35:46.894 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:35:46.894 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:35:46.894 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:35:46.894 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:35:46.894 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:35:46.894 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:35:46.894 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:35:46.894 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:35:46.895 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:35:46.895 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-10T21:35:46.895 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:35:46.895 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-10T21:35:46.895 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:35:46.895 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:35:46.895 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:35:46.895 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:35:46.895 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:35:46.895 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:35:46.895 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:35:46.895 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-10T21:35:46.895 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:35:46.895 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:35:46.895 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:35:46.896 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:35:46.896 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-10T21:35:46.896 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:35:46.896 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:35:46.896 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:35:46.896 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:35:46.896 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-10T21:35:46.896 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-10T21:35:46.896 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:35:46.896 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:35:46.896 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:35:46.896 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:35:46.896 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:35:46.896 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:35:46.896 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:35:46.897 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:35:46.897 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:35:46.897 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:35:46.897 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:35:46.897 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3300", 2024-09-10T21:35:46.897 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:35:46.897 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:35:46.897 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:35:46.897 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:35:46.897 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789", 2024-09-10T21:35:46.897 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:35:46.897 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:35:46.897 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:35:46.897 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:35:46.897 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789/0", 2024-09-10T21:35:46.898 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6789/0", 2024-09-10T21:35:46.898 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:35:46.898 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:35:46.898 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:35:46.898 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:35:46.898 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:35:46.898 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:35:46.898 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:35:46.898 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:35:46.898 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:35:46.898 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:35:46.898 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:35:46.898 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3301", 2024-09-10T21:35:46.898 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:35:46.899 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:35:46.899 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:35:46.899 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:35:46.899 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790", 2024-09-10T21:35:46.899 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:35:46.899 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:35:46.899 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:35:46.899 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:35:46.899 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790/0", 2024-09-10T21:35:46.899 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6790/0", 2024-09-10T21:35:46.899 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:35:46.899 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:35:46.899 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:35:46.900 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:35:46.900 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:35:46.900 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:35:46.900 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-10T21:35:46.900 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-10T21:35:46.900 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:35:46.900 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:35:46.900 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:35:46.900 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:35:46.900 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:35:46.900 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:35:46.900 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-10T21:35:46.900 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:35:46.900 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:35:46.901 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:35:46.901 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:35:46.901 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:35:46.901 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:35:46.901 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:35:46.901 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-10T21:35:46.901 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-10T21:35:46.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:46.883+0000 7f90e5400640 1 -- 172.21.15.17:0/447493474 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f90c00725e0 msgr2=0x7f90c0074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:46.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:46.883+0000 7f90e5400640 1 --2- 172.21.15.17:0/447493474 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f90c00725e0 0x7f90c0074aa0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f90d80ad000 tx=0x7f90d8019ef0 comp rx=0 tx=0).stop 2024-09-10T21:35:46.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:46.883+0000 7f90e5400640 1 -- 172.21.15.17:0/447493474 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f90c4004d50 msgr2=0x7f90e013fdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:46.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:46.883+0000 7f90e5400640 1 --2- 172.21.15.17:0/447493474 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f90c4004d50 0x7f90e013fdf0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f90d0011a80 tx=0x7f90d0011f40 comp rx=0 tx=0).stop 2024-09-10T21:35:46.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:46.883+0000 7f90e4a00640 1 -- 172.21.15.17:0/447493474 reap_dead start 2024-09-10T21:35:46.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:46.883+0000 7f90e5400640 1 -- 172.21.15.17:0/447493474 shutdown_connections 2024-09-10T21:35:46.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:46.883+0000 7f90e5400640 1 --2- 172.21.15.17:0/447493474 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f90c4004d50 0x7f90e013fdf0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:46.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:46.883+0000 7f90e5400640 1 -- 172.21.15.17:0/447493474 >> 172.21.15.17:0/447493474 conn(0x7f90e0068990 msgr2=0x7f90e014bcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:46.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:46.883+0000 7f90e5400640 1 -- 172.21.15.17:0/447493474 shutdown_connections 2024-09-10T21:35:46.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:46.883+0000 7f90e5400640 1 -- 172.21.15.17:0/447493474 wait complete. 2024-09-10T21:35:46.911 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-10T21:35:46.911 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-09-10T21:35:46.911 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-09-10T21:35:47.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:47.063+0000 7f2fe5000640 1 -- 172.21.15.17:0/1080601392 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2fc4003ab0 msgr2=0x7f2fc4003ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:47.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:47.063+0000 7f2fe5000640 1 --2- 172.21.15.17:0/1080601392 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2fc4003ab0 0x7f2fc4003ed0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f2fd0009b30 tx=0x7f2fd001d210 comp rx=0 tx=0).stop 2024-09-10T21:35:47.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:47.063+0000 7f2fe5000640 1 -- 172.21.15.17:0/1080601392 shutdown_connections 2024-09-10T21:35:47.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:47.063+0000 7f2fe5000640 1 --2- 172.21.15.17:0/1080601392 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2fc4003ab0 0x7f2fc4003ed0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:47.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:47.063+0000 7f2fe5000640 1 -- 172.21.15.17:0/1080601392 >> 172.21.15.17:0/1080601392 conn(0x7f2fe0070ca0 msgr2=0x7f2fe00710b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:47.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:47.063+0000 7f2fe5000640 1 -- 172.21.15.17:0/1080601392 shutdown_connections 2024-09-10T21:35:47.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:47.063+0000 7f2fe5000640 1 -- 172.21.15.17:0/1080601392 wait complete. 2024-09-10T21:35:47.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:47.067+0000 7f2fe5000640 1 Processor -- start 2024-09-10T21:35:47.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:47.067+0000 7f2fe5000640 1 -- start start 2024-09-10T21:35:47.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:47.067+0000 7f2fe5000640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f2fc4003ab0 0x7f2fe0148d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:47.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:47.067+0000 7f2fe5000640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2fe014b0a0 0x7f2fe0152c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:47.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:47.067+0000 7f2fe5000640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2fe00c5570 0x7f2fe00c6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:47.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:47.067+0000 7f2fe5000640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f2fe006f500 con 0x7f2fe014b0a0 2024-09-10T21:35:47.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:47.067+0000 7f2fe5000640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f2fe006f220 con 0x7f2fe00c5570 2024-09-10T21:35:47.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:47.067+0000 7f2fe5000640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f2fe006ed40 con 0x7f2fc4003ab0 2024-09-10T21:35:47.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:47.067+0000 7f2fdea00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2fe014b0a0 0x7f2fe0152c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:47.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:47.067+0000 7f2fdea00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2fe014b0a0 0x7f2fe0152c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:53424/0 (socket says 172.21.15.17:53424) 2024-09-10T21:35:47.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:47.067+0000 7f2fdea00640 1 -- 172.21.15.17:0/3077192614 learned_addr learned my addr 172.21.15.17:0/3077192614 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:35:47.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:47.067+0000 7f2fdfe00640 1 --2- 172.21.15.17:0/3077192614 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2fe00c5570 0x7f2fe00c6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:47.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:47.067+0000 7f2fdf400640 1 --2- 172.21.15.17:0/3077192614 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f2fc4003ab0 0x7f2fe0148d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:47.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:47.067+0000 7f2fdea00640 1 -- 172.21.15.17:0/3077192614 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f2fc4003ab0 msgr2=0x7f2fe0148d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:47.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:47.067+0000 7f2fdea00640 1 --2- 172.21.15.17:0/3077192614 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f2fc4003ab0 0x7f2fe0148d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:47.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:47.067+0000 7f2fdea00640 1 -- 172.21.15.17:0/3077192614 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2fe00c5570 msgr2=0x7f2fe00c6190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:47.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:47.067+0000 7f2fdea00640 1 --2- 172.21.15.17:0/3077192614 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2fe00c5570 0x7f2fe00c6190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:47.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:47.067+0000 7f2fdea00640 1 -- 172.21.15.17:0/3077192614 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fe00c5ab0 con 0x7f2fe014b0a0 2024-09-10T21:35:47.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:47.067+0000 7f2fdea00640 1 --2- 172.21.15.17:0/3077192614 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2fe014b0a0 0x7f2fe0152c00 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f2fcc001b80 tx=0x7f2fcc001830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:47.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:47.067+0000 7f2fd7400640 1 -- 172.21.15.17:0/3077192614 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fcc012850 con 0x7f2fe014b0a0 2024-09-10T21:35:47.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:47.067+0000 7f2fd7400640 1 -- 172.21.15.17:0/3077192614 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2fcc012ae0 con 0x7f2fe014b0a0 2024-09-10T21:35:47.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:47.067+0000 7f2fd7400640 1 -- 172.21.15.17:0/3077192614 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fcc01a400 con 0x7f2fe014b0a0 2024-09-10T21:35:47.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:47.067+0000 7f2fe5000640 1 -- 172.21.15.17:0/3077192614 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fe00c5d70 con 0x7f2fe014b0a0 2024-09-10T21:35:47.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:47.067+0000 7f2fe5000640 1 -- 172.21.15.17:0/3077192614 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fe014ccb0 con 0x7f2fe014b0a0 2024-09-10T21:35:47.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:47.071+0000 7f2fe5000640 1 -- 172.21.15.17:0/3077192614 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2fe0158310 con 0x7f2fe014b0a0 2024-09-10T21:35:47.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:47.071+0000 7f2fd7400640 1 -- 172.21.15.17:0/3077192614 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f2fcc01a5a0 con 0x7f2fe014b0a0 2024-09-10T21:35:47.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:47.071+0000 7f2fd7400640 1 --2- 172.21.15.17:0/3077192614 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f2fc0072610 0x7f2fc0074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:47.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:47.071+0000 7f2fd7400640 1 -- 172.21.15.17:0/3077192614 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f2fcc096a30 con 0x7f2fe014b0a0 2024-09-10T21:35:47.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:47.071+0000 7f2fdf400640 1 --2- 172.21.15.17:0/3077192614 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f2fc0072610 0x7f2fc0074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:47.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:47.075+0000 7f2fdf400640 1 --2- 172.21.15.17:0/3077192614 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f2fc0072610 0x7f2fc0074ad0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f2fd00ae000 tx=0x7f2fd0019fb0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:47.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:47.079+0000 7f2fd7400640 1 -- 172.21.15.17:0/3077192614 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f2fcc016030 con 0x7f2fe014b0a0 2024-09-10T21:35:47.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:47.375+0000 7f2fe5000640 1 -- 172.21.15.17:0/3077192614 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) -- 0x7f2fe0155d30 con 0x7f2fe014b0a0 2024-09-10T21:35:47.381 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:47.375+0000 7f2fd7400640 1 -- 172.21.15.17:0/3077192614 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) ==== 57+0+0 (secure 0 0 0) 0x7f2fcc061d80 con 0x7f2fe014b0a0 2024-09-10T21:35:47.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:47.379+0000 7f2fe5000640 1 -- 172.21.15.17:0/3077192614 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f2fc0072610 msgr2=0x7f2fc0074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:47.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:47.379+0000 7f2fe5000640 1 --2- 172.21.15.17:0/3077192614 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f2fc0072610 0x7f2fc0074ad0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f2fd00ae000 tx=0x7f2fd0019fb0 comp rx=0 tx=0).stop 2024-09-10T21:35:47.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:47.379+0000 7f2fe5000640 1 -- 172.21.15.17:0/3077192614 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2fe014b0a0 msgr2=0x7f2fe0152c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:47.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:47.379+0000 7f2fe5000640 1 --2- 172.21.15.17:0/3077192614 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2fe014b0a0 0x7f2fe0152c00 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f2fcc001b80 tx=0x7f2fcc001830 comp rx=0 tx=0).stop 2024-09-10T21:35:47.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:47.379+0000 7f2fe5000640 1 -- 172.21.15.17:0/3077192614 shutdown_connections 2024-09-10T21:35:47.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:47.379+0000 7f2fe5000640 1 --2- 172.21.15.17:0/3077192614 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f2fc0072610 0x7f2fc0074ad0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:47.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:47.379+0000 7f2fe5000640 1 --2- 172.21.15.17:0/3077192614 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2fe00c5570 0x7f2fe00c6190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:47.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:47.379+0000 7f2fe5000640 1 --2- 172.21.15.17:0/3077192614 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2fe014b0a0 0x7f2fe0152c00 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:47.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:47.379+0000 7f2fe5000640 1 --2- 172.21.15.17:0/3077192614 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f2fc4003ab0 0x7f2fe0148d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:47.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:47.379+0000 7f2fe5000640 1 -- 172.21.15.17:0/3077192614 >> 172.21.15.17:0/3077192614 conn(0x7f2fe0070ca0 msgr2=0x7f2fe0151e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:47.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:47.379+0000 7f2fe5000640 1 -- 172.21.15.17:0/3077192614 shutdown_connections 2024-09-10T21:35:47.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:47.379+0000 7f2fe5000640 1 -- 172.21.15.17:0/3077192614 wait complete. 2024-09-10T21:35:47.408 INFO:tasks.mon_thrash.mon_thrasher:waiting for 10.0 secs before continuing thrashing 2024-09-10T21:35:49.364 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:35:49.368 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:35:57.409 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-09-10T21:35:57.409 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-10T21:35:57.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:57.575+0000 7fcd42200640 1 -- 172.21.15.17:0/1310812109 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fcd3c158c20 msgr2=0x7fcd3c070670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:57.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:57.575+0000 7fcd42200640 1 --2- 172.21.15.17:0/1310812109 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fcd3c158c20 0x7fcd3c070670 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fcd28009900 tx=0x7fcd2801f0d0 comp rx=0 tx=0).stop 2024-09-10T21:35:57.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:57.575+0000 7fcd42200640 1 -- 172.21.15.17:0/1310812109 shutdown_connections 2024-09-10T21:35:57.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:57.575+0000 7fcd42200640 1 --2- 172.21.15.17:0/1310812109 >> v2:172.21.15.31:3300/0 conn(0x7fcd3c0b3e00 0x7fcd3c070bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:57.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:57.575+0000 7fcd42200640 1 --2- 172.21.15.17:0/1310812109 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fcd3c158c20 0x7fcd3c070670 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:57.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:57.575+0000 7fcd42200640 1 -- 172.21.15.17:0/1310812109 >> 172.21.15.17:0/1310812109 conn(0x7fcd3c0cdef0 msgr2=0x7fcd3c0ce2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:57.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:57.575+0000 7fcd42200640 1 -- 172.21.15.17:0/1310812109 shutdown_connections 2024-09-10T21:35:57.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:57.575+0000 7fcd42200640 1 -- 172.21.15.17:0/1310812109 wait complete. 2024-09-10T21:35:57.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:57.579+0000 7fcd42200640 1 Processor -- start 2024-09-10T21:35:57.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:57.579+0000 7fcd42200640 1 -- start start 2024-09-10T21:35:57.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:57.579+0000 7fcd42200640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcd3c0b3e00 0x7fcd3c14ff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:57.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:57.579+0000 7fcd42200640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcd3c1524d0 0x7fcd3c142810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:57.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:57.579+0000 7fcd42200640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fcd3c158c20 0x7fcd3c142d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:57.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:57.579+0000 7fcd42200640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fcd3c0c9350 con 0x7fcd3c0b3e00 2024-09-10T21:35:57.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:57.579+0000 7fcd42200640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fcd3c0c8e50 con 0x7fcd3c1524d0 2024-09-10T21:35:57.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:57.579+0000 7fcd42200640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7fcd3c15b9e0 con 0x7fcd3c158c20 2024-09-10T21:35:57.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:57.579+0000 7fcd40e00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcd3c0b3e00 0x7fcd3c14ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:57.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:57.579+0000 7fcd40e00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcd3c0b3e00 0x7fcd3c14ff60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40310/0 (socket says 172.21.15.17:40310) 2024-09-10T21:35:57.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:57.579+0000 7fcd40e00640 1 -- 172.21.15.17:0/2962714621 learned_addr learned my addr 172.21.15.17:0/2962714621 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:35:57.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:57.579+0000 7fcd3be00640 1 --2- 172.21.15.17:0/2962714621 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcd3c1524d0 0x7fcd3c142810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:57.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:57.579+0000 7fcd41800640 1 --2- 172.21.15.17:0/2962714621 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fcd3c158c20 0x7fcd3c142d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:57.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:57.579+0000 7fcd40e00640 1 -- 172.21.15.17:0/2962714621 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fcd3c158c20 msgr2=0x7fcd3c142d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:57.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:57.579+0000 7fcd40e00640 1 --2- 172.21.15.17:0/2962714621 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fcd3c158c20 0x7fcd3c142d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:57.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:57.579+0000 7fcd40e00640 1 -- 172.21.15.17:0/2962714621 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcd3c1524d0 msgr2=0x7fcd3c142810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:57.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:57.579+0000 7fcd40e00640 1 --2- 172.21.15.17:0/2962714621 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcd3c1524d0 0x7fcd3c142810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:57.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:57.579+0000 7fcd40e00640 1 -- 172.21.15.17:0/2962714621 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd3c151630 con 0x7fcd3c0b3e00 2024-09-10T21:35:57.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:57.583+0000 7fcd40e00640 1 --2- 172.21.15.17:0/2962714621 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcd3c0b3e00 0x7fcd3c14ff60 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fcd3000c500 tx=0x7fcd3000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:57.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:57.583+0000 7fcd38c00640 1 -- 172.21.15.17:0/2962714621 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd30017070 con 0x7fcd3c0b3e00 2024-09-10T21:35:57.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:57.583+0000 7fcd38c00640 1 -- 172.21.15.17:0/2962714621 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcd300053c0 con 0x7fcd3c0b3e00 2024-09-10T21:35:57.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:57.583+0000 7fcd42200640 1 -- 172.21.15.17:0/2962714621 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd3c1518a0 con 0x7fcd3c0b3e00 2024-09-10T21:35:57.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:57.583+0000 7fcd38c00640 1 -- 172.21.15.17:0/2962714621 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd30012890 con 0x7fcd3c0b3e00 2024-09-10T21:35:57.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:57.583+0000 7fcd42200640 1 -- 172.21.15.17:0/2962714621 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd3c150f70 con 0x7fcd3c0b3e00 2024-09-10T21:35:57.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:57.583+0000 7fcd42200640 1 -- 172.21.15.17:0/2962714621 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd3c15bf00 con 0x7fcd3c0b3e00 2024-09-10T21:35:57.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:57.583+0000 7fcd38c00640 1 -- 172.21.15.17:0/2962714621 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fcd30003600 con 0x7fcd3c0b3e00 2024-09-10T21:35:57.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:57.587+0000 7fcd38c00640 1 --2- 172.21.15.17:0/2962714621 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fcd1c0725e0 0x7fcd1c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:57.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:57.587+0000 7fcd38c00640 1 -- 172.21.15.17:0/2962714621 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fcd30092ca0 con 0x7fcd3c0b3e00 2024-09-10T21:35:57.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:57.587+0000 7fcd3be00640 1 --2- 172.21.15.17:0/2962714621 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fcd1c0725e0 0x7fcd1c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:57.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:57.587+0000 7fcd3be00640 1 --2- 172.21.15.17:0/2962714621 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fcd1c0725e0 0x7fcd1c074aa0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fcd280b2000 tx=0x7fcd2801d910 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:57.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:57.591+0000 7fcd38c00640 1 -- 172.21.15.17:0/2962714621 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fcd3c1528b0 con 0x7fcd3c0b3e00 2024-09-10T21:35:57.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:57.855+0000 7fcd42200640 1 -- 172.21.15.17:0/2962714621 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fcd3c146590 con 0x7fcd3c0b3e00 2024-09-10T21:35:57.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:57.855+0000 7fcd38c00640 1 -- 172.21.15.17:0/2962714621 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1557 (secure 0 0 0) 0x7fcd3005d8a0 con 0x7fcd3c0b3e00 2024-09-10T21:35:57.863 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":92,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":16,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5e2f023f-d6a6-49f4-a256-f4807dce3739","modified":"2024-09-10T21:23:46.892130Z","created":"2024-09-10T21:23:46.892130Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3301","nonce":0},{"type":"v1","addr":"172.21.15.31:6790","nonce":0}]},"addr":"172.21.15.31:6790/0","public_addr":"172.21.15.31:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-10T21:35:57.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:57.859+0000 7fcd42200640 1 -- 172.21.15.17:0/2962714621 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fcd1c0725e0 msgr2=0x7fcd1c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:57.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:57.859+0000 7fcd42200640 1 --2- 172.21.15.17:0/2962714621 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fcd1c0725e0 0x7fcd1c074aa0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fcd280b2000 tx=0x7fcd2801d910 comp rx=0 tx=0).stop 2024-09-10T21:35:57.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:57.859+0000 7fcd42200640 1 -- 172.21.15.17:0/2962714621 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcd3c0b3e00 msgr2=0x7fcd3c14ff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:57.865 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:57.859+0000 7fcd42200640 1 --2- 172.21.15.17:0/2962714621 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcd3c0b3e00 0x7fcd3c14ff60 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fcd3000c500 tx=0x7fcd3000c9c0 comp rx=0 tx=0).stop 2024-09-10T21:35:57.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:57.859+0000 7fcd42200640 1 -- 172.21.15.17:0/2962714621 shutdown_connections 2024-09-10T21:35:57.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:57.859+0000 7fcd42200640 1 --2- 172.21.15.17:0/2962714621 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fcd1c0725e0 0x7fcd1c074aa0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:57.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:57.859+0000 7fcd42200640 1 --2- 172.21.15.17:0/2962714621 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fcd3c158c20 0x7fcd3c142d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:57.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:57.859+0000 7fcd42200640 1 --2- 172.21.15.17:0/2962714621 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcd3c1524d0 0x7fcd3c142810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:57.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:57.859+0000 7fcd42200640 1 --2- 172.21.15.17:0/2962714621 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcd3c0b3e00 0x7fcd3c14ff60 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:57.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:57.859+0000 7fcd42200640 1 -- 172.21.15.17:0/2962714621 >> 172.21.15.17:0/2962714621 conn(0x7fcd3c0cdef0 msgr2=0x7fcd3c14deb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:57.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:57.859+0000 7fcd42200640 1 -- 172.21.15.17:0/2962714621 shutdown_connections 2024-09-10T21:35:57.866 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:57.859+0000 7fcd42200640 1 -- 172.21.15.17:0/2962714621 wait complete. 2024-09-10T21:35:57.884 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-09-10T21:35:57.884 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-09-10T21:35:57.884 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-09-10T21:35:57.884 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-10T21:35:58.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.051+0000 7fd475200640 1 -- 172.21.15.17:0/2011489985 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd4700cdf20 msgr2=0x7fd4700ce350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:58.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.051+0000 7fd475200640 1 --2- 172.21.15.17:0/2011489985 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd4700cdf20 0x7fd4700ce350 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fd4700cbaa0 tx=0x7fd46801ccf0 comp rx=0 tx=0).stop 2024-09-10T21:35:58.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.051+0000 7fd475200640 1 -- 172.21.15.17:0/2011489985 shutdown_connections 2024-09-10T21:35:58.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.051+0000 7fd475200640 1 --2- 172.21.15.17:0/2011489985 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd4700cdf20 0x7fd4700ce350 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:58.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.051+0000 7fd475200640 1 --2- 172.21.15.17:0/2011489985 >> v2:172.21.15.31:3300/0 conn(0x7fd470152820 0x7fd4700b3eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:58.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.051+0000 7fd475200640 1 -- 172.21.15.17:0/2011489985 >> 172.21.15.17:0/2011489985 conn(0x7fd4700702f0 msgr2=0x7fd470070700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:58.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.051+0000 7fd475200640 1 -- 172.21.15.17:0/2011489985 shutdown_connections 2024-09-10T21:35:58.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.051+0000 7fd475200640 1 -- 172.21.15.17:0/2011489985 wait complete. 2024-09-10T21:35:58.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.051+0000 7fd475200640 1 Processor -- start 2024-09-10T21:35:58.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.051+0000 7fd475200640 1 -- start start 2024-09-10T21:35:58.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.051+0000 7fd475200640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd4700cdf20 0x7fd470141da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:58.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.055+0000 7fd475200640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd470152440 0x7fd4700c5d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:58.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.055+0000 7fd475200640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd470152820 0x7fd47014a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:58.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.055+0000 7fd475200640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fd4700cb5d0 con 0x7fd470152820 2024-09-10T21:35:58.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.055+0000 7fd475200640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fd4700cb450 con 0x7fd470152440 2024-09-10T21:35:58.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.055+0000 7fd475200640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7fd4700c2ff0 con 0x7fd4700cdf20 2024-09-10T21:35:58.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.055+0000 7fd46e800640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd470152820 0x7fd47014a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:58.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.055+0000 7fd46e800640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd470152820 0x7fd47014a0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40324/0 (socket says 172.21.15.17:40324) 2024-09-10T21:35:58.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.055+0000 7fd46e800640 1 -- 172.21.15.17:0/4168677191 learned_addr learned my addr 172.21.15.17:0/4168677191 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:35:58.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.055+0000 7fd46d400640 1 --2- 172.21.15.17:0/4168677191 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd470152440 0x7fd4700c5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:58.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.055+0000 7fd46de00640 1 --2- 172.21.15.17:0/4168677191 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd4700cdf20 0x7fd470141da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:58.061 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.055+0000 7fd46e800640 1 -- 172.21.15.17:0/4168677191 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd4700cdf20 msgr2=0x7fd470141da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:58.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.055+0000 7fd46e800640 1 --2- 172.21.15.17:0/4168677191 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd4700cdf20 0x7fd470141da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:58.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.055+0000 7fd46e800640 1 -- 172.21.15.17:0/4168677191 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd470152440 msgr2=0x7fd4700c5d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:58.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.055+0000 7fd46e800640 1 --2- 172.21.15.17:0/4168677191 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd470152440 0x7fd4700c5d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:58.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.055+0000 7fd46e800640 1 -- 172.21.15.17:0/4168677191 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd470146640 con 0x7fd470152820 2024-09-10T21:35:58.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.055+0000 7fd46e800640 1 --2- 172.21.15.17:0/4168677191 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd470152820 0x7fd47014a0f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fd46801b3a0 tx=0x7fd468005b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:58.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.055+0000 7fd466000640 1 -- 172.21.15.17:0/4168677191 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4680bc070 con 0x7fd470152820 2024-09-10T21:35:58.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.055+0000 7fd466000640 1 -- 172.21.15.17:0/4168677191 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd468004330 con 0x7fd470152820 2024-09-10T21:35:58.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.055+0000 7fd475200640 1 -- 172.21.15.17:0/4168677191 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4701468c0 con 0x7fd470152820 2024-09-10T21:35:58.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.055+0000 7fd466000640 1 -- 172.21.15.17:0/4168677191 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4680b2040 con 0x7fd470152820 2024-09-10T21:35:58.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.055+0000 7fd475200640 1 -- 172.21.15.17:0/4168677191 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd470146e80 con 0x7fd470152820 2024-09-10T21:35:58.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.055+0000 7fd475200640 1 --2- 172.21.15.17:0/4168677191 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd470145a60 0x7fd47014a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:58.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.055+0000 7fd475200640 1 -- 172.21.15.17:0/4168677191 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fd470158360 con 0x7fd470145a60 2024-09-10T21:35:58.063 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.055+0000 7fd46de00640 1 --2- 172.21.15.17:0/4168677191 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd470145a60 0x7fd47014a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:58.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.055+0000 7fd46de00640 1 -- 172.21.15.17:0/4168677191 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd470155d30 con 0x7fd470145a60 2024-09-10T21:35:58.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.055+0000 7fd46de00640 1 --2- 172.21.15.17:0/4168677191 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd470145a60 0x7fd47014a630 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fd46000f770 tx=0x7fd46000fc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:58.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.055+0000 7fd466000640 1 -- 172.21.15.17:0/4168677191 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd460003970 con 0x7fd470145a60 2024-09-10T21:35:58.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.059+0000 7fd466000640 1 -- 172.21.15.17:0/4168677191 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fd460019070 con 0x7fd470145a60 2024-09-10T21:35:58.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.059+0000 7fd466000640 1 -- 172.21.15.17:0/4168677191 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd470145a60 msgr2=0x7fd47014a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:58.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.059+0000 7fd466000640 1 --2- 172.21.15.17:0/4168677191 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd470145a60 0x7fd47014a630 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fd46000f770 tx=0x7fd46000fc30 comp rx=0 tx=0).stop 2024-09-10T21:35:58.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.059+0000 7fd466000640 1 -- 172.21.15.17:0/4168677191 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd470145a60 msgr2=0x7fd47014a630 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:35:58.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.059+0000 7fd466000640 1 --2- 172.21.15.17:0/4168677191 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd470145a60 0x7fd47014a630 secure :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fd46000f770 tx=0x7fd46000fc30 comp rx=0 tx=0).stop 2024-09-10T21:35:58.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.059+0000 7fd466000640 1 -- 172.21.15.17:0/4168677191 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fd4680b5070 con 0x7fd470152820 2024-09-10T21:35:58.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.059+0000 7fd466000640 1 --2- 172.21.15.17:0/4168677191 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fd4500720f0 0x7fd4500745b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:58.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.059+0000 7fd466000640 1 -- 172.21.15.17:0/4168677191 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fd46812d190 con 0x7fd470152820 2024-09-10T21:35:58.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.059+0000 7fd46de00640 1 --2- 172.21.15.17:0/4168677191 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fd4500720f0 0x7fd4500745b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:58.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.059+0000 7fd46de00640 1 --2- 172.21.15.17:0/4168677191 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fd4500720f0 0x7fd4500745b0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fd460016040 tx=0x7fd460009070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:58.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.063+0000 7fd475200640 1 --2- 172.21.15.17:0/4168677191 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd47014b9f0 0x7fd4701470c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:58.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.063+0000 7fd475200640 1 -- 172.21.15.17:0/4168677191 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fd47014b1b0 con 0x7fd47014b9f0 2024-09-10T21:35:58.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.063+0000 7fd46d400640 1 --2- 172.21.15.17:0/4168677191 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd47014b9f0 0x7fd4701470c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:58.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.067+0000 7fd46d400640 1 -- 172.21.15.17:0/4168677191 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fd470147600 con 0x7fd47014b9f0 2024-09-10T21:35:58.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.067+0000 7fd46d400640 1 --2- 172.21.15.17:0/4168677191 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd47014b9f0 0x7fd4701470c0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fd470161e10 tx=0x7fd45c00bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:58.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.067+0000 7fd466000640 1 -- 172.21.15.17:0/4168677191 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd45c005220 con 0x7fd47014b9f0 2024-09-10T21:35:58.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.067+0000 7fd466000640 1 -- 172.21.15.17:0/4168677191 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5045 (secure 0 0 0) 0x7fd45c005220 con 0x7fd47014b9f0 2024-09-10T21:35:58.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.067+0000 7fd466000640 1 -- 172.21.15.17:0/4168677191 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd47014b9f0 msgr2=0x7fd4701470c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:58.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.067+0000 7fd466000640 1 --2- 172.21.15.17:0/4168677191 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd47014b9f0 0x7fd4701470c0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fd470161e10 tx=0x7fd45c00bf90 comp rx=0 tx=0).stop 2024-09-10T21:35:58.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.067+0000 7fd466000640 1 -- 172.21.15.17:0/4168677191 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd47014b9f0 msgr2=0x7fd4701470c0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:35:58.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.067+0000 7fd466000640 1 --2- 172.21.15.17:0/4168677191 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd47014b9f0 0x7fd4701470c0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:58.074 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-10T21:35:58.074 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:35:58.074 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:35:58.074 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "leader", 2024-09-10T21:35:58.074 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 92, 2024-09-10T21:35:58.074 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 138674, 2024-09-10T21:35:58.074 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-10T21:35:58.074 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-10T21:35:58.074 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-10T21:35:58.074 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-10T21:35:58.074 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:35:58.075 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 16, 2024-09-10T21:35:58.075 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:35:58.075 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-10T21:35:58.075 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-10T21:35:58.075 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:35:58.075 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:35:58.075 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:35:58.075 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:35:58.075 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:35:58.075 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:35:58.075 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:35:58.075 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:35:58.075 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:35:58.075 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:35:58.076 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:35:58.076 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:35:58.076 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-10T21:35:58.076 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-10T21:35:58.076 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:35:58.076 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:35:58.076 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:35:58.076 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:35:58.076 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:35:58.076 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:35:58.076 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:35:58.076 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:35:58.076 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:35:58.076 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:35:58.077 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:35:58.077 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:35:58.077 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:35:58.077 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-10T21:35:58.077 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-10T21:35:58.077 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-10T21:35:58.077 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-10T21:35:58.077 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-10T21:35:58.077 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "5e2f023f-d6a6-49f4-a256-f4807dce3739", 2024-09-10T21:35:58.077 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:35:58.077 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:35:58.077 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-10T21:35:58.077 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-10T21:35:58.078 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-10T21:35:58.078 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-10T21:35:58.078 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-10T21:35:58.078 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-10T21:35:58.078 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-10T21:35:58.078 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:35:58.078 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-10T21:35:58.078 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:35:58.078 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:35:58.078 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:35:58.078 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:35:58.078 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:35:58.078 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:35:58.078 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:35:58.078 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:35:58.079 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:35:58.079 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:35:58.079 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:35:58.079 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:35:58.079 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-10T21:35:58.079 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:35:58.079 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-10T21:35:58.079 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:35:58.079 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:35:58.079 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:35:58.079 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:35:58.079 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:35:58.079 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:35:58.079 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:35:58.080 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-10T21:35:58.080 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:35:58.080 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:35:58.080 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:35:58.080 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:35:58.080 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-10T21:35:58.080 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:35:58.080 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:35:58.080 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:35:58.080 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:35:58.080 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-10T21:35:58.080 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-10T21:35:58.080 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:35:58.081 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:35:58.081 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:35:58.081 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:35:58.081 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:35:58.081 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:35:58.081 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:35:58.081 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:35:58.081 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:35:58.081 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:35:58.081 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:35:58.081 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3300", 2024-09-10T21:35:58.081 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:35:58.081 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:35:58.082 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:35:58.082 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:35:58.082 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789", 2024-09-10T21:35:58.082 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:35:58.082 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:35:58.082 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:35:58.082 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:35:58.082 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789/0", 2024-09-10T21:35:58.082 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6789/0", 2024-09-10T21:35:58.082 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:35:58.082 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:35:58.082 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:35:58.082 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:35:58.082 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:35:58.083 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:35:58.083 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:35:58.083 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:35:58.083 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:35:58.083 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:35:58.083 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:35:58.083 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3301", 2024-09-10T21:35:58.083 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:35:58.083 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:35:58.083 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:35:58.083 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:35:58.083 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790", 2024-09-10T21:35:58.083 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:35:58.083 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:35:58.084 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:35:58.084 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:35:58.084 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790/0", 2024-09-10T21:35:58.084 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6790/0", 2024-09-10T21:35:58.084 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:35:58.084 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:35:58.084 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:35:58.084 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:35:58.084 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:35:58.084 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:35:58.084 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-10T21:35:58.084 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-10T21:35:58.084 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:35:58.085 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:35:58.085 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:35:58.085 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:35:58.085 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:35:58.085 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:35:58.085 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": [ 2024-09-10T21:35:58.085 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:35:58.085 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:35:58.085 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:35:58.085 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 6 2024-09-10T21:35:58.085 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:35:58.085 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:35:58.085 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-10T21:35:58.086 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:35:58.086 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:35:58.086 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:35:58.086 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 8 2024-09-10T21:35:58.086 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:35:58.086 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:35:58.086 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-10T21:35:58.086 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:35:58.086 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:35:58.086 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:35:58.086 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 6 2024-09-10T21:35:58.086 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:35:58.086 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:35:58.086 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": [ 2024-09-10T21:35:58.087 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:35:58.087 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:35:58.087 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:35:58.087 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 2 2024-09-10T21:35:58.087 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:35:58.087 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:35:58.087 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:35:58.087 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-10T21:35:58.087 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-10T21:35:58.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.067+0000 7fd475200640 1 -- 172.21.15.17:0/4168677191 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fd4500720f0 msgr2=0x7fd4500745b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:58.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.067+0000 7fd475200640 1 --2- 172.21.15.17:0/4168677191 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fd4500720f0 0x7fd4500745b0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fd460016040 tx=0x7fd460009070 comp rx=0 tx=0).stop 2024-09-10T21:35:58.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.067+0000 7fd475200640 1 -- 172.21.15.17:0/4168677191 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd470152820 msgr2=0x7fd47014a0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:58.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.067+0000 7fd475200640 1 --2- 172.21.15.17:0/4168677191 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd470152820 0x7fd47014a0f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fd46801b3a0 tx=0x7fd468005b10 comp rx=0 tx=0).stop 2024-09-10T21:35:58.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.067+0000 7fd46e800640 1 -- 172.21.15.17:0/4168677191 reap_dead start 2024-09-10T21:35:58.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.067+0000 7fd46e800640 1 -- 172.21.15.17:0/4168677191 reap_dead start 2024-09-10T21:35:58.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.067+0000 7fd475200640 1 -- 172.21.15.17:0/4168677191 shutdown_connections 2024-09-10T21:35:58.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.067+0000 7fd475200640 1 -- 172.21.15.17:0/4168677191 >> 172.21.15.17:0/4168677191 conn(0x7fd4700702f0 msgr2=0x7fd4700c9a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:58.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.067+0000 7fd475200640 1 -- 172.21.15.17:0/4168677191 shutdown_connections 2024-09-10T21:35:58.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.067+0000 7fd475200640 1 -- 172.21.15.17:0/4168677191 wait complete. 2024-09-10T21:35:58.092 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-10T21:35:58.092 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-09-10T21:35:58.092 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-10T21:35:58.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.271+0000 7f91c4e00640 1 -- 172.21.15.17:0/2232361703 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f91a4004810 msgr2=0x7f91a4004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:58.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.271+0000 7f91c4e00640 1 --2- 172.21.15.17:0/2232361703 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f91a4004810 0x7f91a4004c90 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f91c006f520 tx=0x7f91ac01c7e0 comp rx=0 tx=0).stop 2024-09-10T21:35:58.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.271+0000 7f91bfe00640 1 -- 172.21.15.17:0/2232361703 reap_dead start 2024-09-10T21:35:58.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.271+0000 7f91c4e00640 1 -- 172.21.15.17:0/2232361703 shutdown_connections 2024-09-10T21:35:58.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.271+0000 7f91c4e00640 1 -- 172.21.15.17:0/2232361703 >> 172.21.15.17:0/2232361703 conn(0x7f91c00cd690 msgr2=0x7f91c00c98e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:58.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.271+0000 7f91c4e00640 1 -- 172.21.15.17:0/2232361703 shutdown_connections 2024-09-10T21:35:58.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.271+0000 7f91c4e00640 1 -- 172.21.15.17:0/2232361703 wait complete. 2024-09-10T21:35:58.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.271+0000 7f91c4e00640 1 Processor -- start 2024-09-10T21:35:58.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.271+0000 7f91c4e00640 1 -- start start 2024-09-10T21:35:58.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.271+0000 7f91c4e00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f91a4004810 0x7f91c00c4b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:58.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.271+0000 7f91c4e00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f91c014ca80 0x7f91c00c6420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:58.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.271+0000 7f91c4e00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f91c00c6960 0x7f91c0155340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:58.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.271+0000 7f91c4e00640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f91c00c2710 con 0x7f91c014ca80 2024-09-10T21:35:58.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.271+0000 7f91c4e00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f91c00c2a10 con 0x7f91c00c6960 2024-09-10T21:35:58.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.271+0000 7f91c4e00640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f91c0151830 con 0x7f91a4004810 2024-09-10T21:35:58.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.271+0000 7f91bea00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f91c014ca80 0x7f91c00c6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:58.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.271+0000 7f91bea00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f91c014ca80 0x7f91c00c6420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40356/0 (socket says 172.21.15.17:40356) 2024-09-10T21:35:58.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.271+0000 7f91bea00640 1 -- 172.21.15.17:0/681903682 learned_addr learned my addr 172.21.15.17:0/681903682 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:35:58.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.271+0000 7f91bfe00640 1 --2- 172.21.15.17:0/681903682 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f91c00c6960 0x7f91c0155340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:58.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.271+0000 7f91bf400640 1 --2- 172.21.15.17:0/681903682 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f91a4004810 0x7f91c00c4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:58.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.271+0000 7f91bea00640 1 -- 172.21.15.17:0/681903682 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f91a4004810 msgr2=0x7f91c00c4b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:58.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.271+0000 7f91bea00640 1 --2- 172.21.15.17:0/681903682 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f91a4004810 0x7f91c00c4b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:58.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.271+0000 7f91bea00640 1 -- 172.21.15.17:0/681903682 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f91c00c6960 msgr2=0x7f91c0155340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:58.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.271+0000 7f91bea00640 1 --2- 172.21.15.17:0/681903682 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f91c00c6960 0x7f91c0155340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:58.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.271+0000 7f91bea00640 1 -- 172.21.15.17:0/681903682 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91c00c50b0 con 0x7f91c014ca80 2024-09-10T21:35:58.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.275+0000 7f91bea00640 1 --2- 172.21.15.17:0/681903682 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f91c014ca80 0x7f91c00c6420 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f91ac01c610 tx=0x7f91ac0014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:58.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.275+0000 7f91b7400640 1 -- 172.21.15.17:0/681903682 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91ac0036c0 con 0x7f91c014ca80 2024-09-10T21:35:58.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.275+0000 7f91b7400640 1 -- 172.21.15.17:0/681903682 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f91ac003950 con 0x7f91c014ca80 2024-09-10T21:35:58.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.275+0000 7f91b7400640 1 -- 172.21.15.17:0/681903682 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91ac003d80 con 0x7f91c014ca80 2024-09-10T21:35:58.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.275+0000 7f91c4e00640 1 -- 172.21.15.17:0/681903682 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91c0156030 con 0x7f91c014ca80 2024-09-10T21:35:58.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.275+0000 7f91c4e00640 1 -- 172.21.15.17:0/681903682 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91c00c5940 con 0x7f91c014ca80 2024-09-10T21:35:58.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.275+0000 7f91c4e00640 1 --2- 172.21.15.17:0/681903682 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f91c0155880 0x7f91c00c5b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:58.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.275+0000 7f91c4e00640 1 -- 172.21.15.17:0/681903682 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f91c0069ef0 con 0x7f91c0155880 2024-09-10T21:35:58.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.275+0000 7f91bf400640 1 --2- 172.21.15.17:0/681903682 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f91c0155880 0x7f91c00c5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:58.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.275+0000 7f91bf400640 1 -- 172.21.15.17:0/681903682 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f91c01562d0 con 0x7f91c0155880 2024-09-10T21:35:58.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.275+0000 7f91bf400640 1 --2- 172.21.15.17:0/681903682 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f91c0155880 0x7f91c00c5b80 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f91c015b780 tx=0x7f91b0014b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:58.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.275+0000 7f91b7400640 1 -- 172.21.15.17:0/681903682 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f91ac0b6020 con 0x7f91c014ca80 2024-09-10T21:35:58.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.279+0000 7f91b7400640 1 --2- 172.21.15.17:0/681903682 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f91a0072610 0x7f91a0074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:58.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.279+0000 7f91b7400640 1 -- 172.21.15.17:0/681903682 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91b0012ca0 con 0x7f91c0155880 2024-09-10T21:35:58.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.279+0000 7f91b7400640 1 -- 172.21.15.17:0/681903682 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f91ac12cd00 con 0x7f91c014ca80 2024-09-10T21:35:58.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.279+0000 7f91b7400640 1 -- 172.21.15.17:0/681903682 <== mon.1 v2:172.21.15.31:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f91b0012e40 con 0x7f91c0155880 2024-09-10T21:35:58.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.279+0000 7f91b7400640 1 -- 172.21.15.17:0/681903682 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f91c0155880 msgr2=0x7f91c00c5b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:58.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.279+0000 7f91b7400640 1 --2- 172.21.15.17:0/681903682 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f91c0155880 0x7f91c00c5b80 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f91c015b780 tx=0x7f91b0014b20 comp rx=0 tx=0).stop 2024-09-10T21:35:58.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.279+0000 7f91b7400640 1 -- 172.21.15.17:0/681903682 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f91c0155880 msgr2=0x7f91c00c5b80 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:35:58.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.279+0000 7f91b7400640 1 --2- 172.21.15.17:0/681903682 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f91c0155880 0x7f91c00c5b80 secure :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f91c015b780 tx=0x7f91b0014b20 comp rx=0 tx=0).stop 2024-09-10T21:35:58.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.279+0000 7f91bfe00640 1 --2- 172.21.15.17:0/681903682 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f91a0072610 0x7f91a0074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:58.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.279+0000 7f91bfe00640 1 --2- 172.21.15.17:0/681903682 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f91a0072610 0x7f91a0074ad0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f91c015b950 tx=0x7f91b8009500 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:58.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.287+0000 7f91c4e00640 1 --2- 172.21.15.17:0/681903682 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f91c01f3a80 0x7f91c01f3e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:58.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.287+0000 7f91c4e00640 1 -- 172.21.15.17:0/681903682 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f91c00c60c0 con 0x7f91c01f3a80 2024-09-10T21:35:58.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.287+0000 7f91bf400640 1 --2- 172.21.15.17:0/681903682 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f91c01f3a80 0x7f91c01f3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:58.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.287+0000 7f91bf400640 1 -- 172.21.15.17:0/681903682 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f91c00c5780 con 0x7f91c01f3a80 2024-09-10T21:35:58.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.287+0000 7f91bf400640 1 --2- 172.21.15.17:0/681903682 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f91c01f3a80 0x7f91c01f3e60 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f91b001b000 tx=0x7f91c0157240 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:58.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.287+0000 7f91b7400640 1 -- 172.21.15.17:0/681903682 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91b0029400 con 0x7f91c01f3a80 2024-09-10T21:35:58.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.287+0000 7f91b7400640 1 -- 172.21.15.17:0/681903682 <== mon.1 v2:172.21.15.31:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4541 (secure 0 0 0) 0x7f91b001d3f0 con 0x7f91c01f3a80 2024-09-10T21:35:58.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.287+0000 7f91b7400640 1 -- 172.21.15.17:0/681903682 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f91c01f3a80 msgr2=0x7f91c01f3e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:58.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.287+0000 7f91b7400640 1 --2- 172.21.15.17:0/681903682 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f91c01f3a80 0x7f91c01f3e60 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f91b001b000 tx=0x7f91c0157240 comp rx=0 tx=0).stop 2024-09-10T21:35:58.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.287+0000 7f91b7400640 1 -- 172.21.15.17:0/681903682 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f91c01f3a80 msgr2=0x7f91c01f3e60 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:35:58.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.287+0000 7f91b7400640 1 --2- 172.21.15.17:0/681903682 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f91c01f3a80 0x7f91c01f3e60 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:58.295 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-10T21:35:58.295 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:35:58.295 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:35:58.295 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-10T21:35:58.295 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 92, 2024-09-10T21:35:58.296 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 77675, 2024-09-10T21:35:58.296 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-10T21:35:58.296 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-10T21:35:58.296 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-10T21:35:58.296 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-10T21:35:58.296 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:35:58.296 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 17, 2024-09-10T21:35:58.296 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:35:58.296 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-10T21:35:58.296 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-10T21:35:58.296 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:35:58.296 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:35:58.296 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:35:58.296 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:35:58.297 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:35:58.297 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:35:58.297 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:35:58.297 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:35:58.297 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:35:58.297 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:35:58.297 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:35:58.297 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:35:58.297 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-10T21:35:58.297 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-10T21:35:58.297 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:35:58.297 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:35:58.297 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:35:58.298 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:35:58.298 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:35:58.298 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:35:58.298 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:35:58.298 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:35:58.298 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:35:58.298 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:35:58.298 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:35:58.298 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:35:58.298 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:35:58.298 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-10T21:35:58.298 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-10T21:35:58.298 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-10T21:35:58.298 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-10T21:35:58.299 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-10T21:35:58.299 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "5e2f023f-d6a6-49f4-a256-f4807dce3739", 2024-09-10T21:35:58.299 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:35:58.299 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:35:58.299 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-10T21:35:58.299 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-10T21:35:58.299 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-10T21:35:58.299 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-10T21:35:58.299 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-10T21:35:58.299 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-10T21:35:58.299 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-10T21:35:58.299 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:35:58.299 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-10T21:35:58.299 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:35:58.300 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:35:58.300 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:35:58.300 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:35:58.300 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:35:58.300 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:35:58.300 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:35:58.300 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:35:58.300 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:35:58.300 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:35:58.300 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:35:58.300 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:35:58.300 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-10T21:35:58.300 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:35:58.301 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-10T21:35:58.301 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:35:58.301 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:35:58.301 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:35:58.301 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:35:58.301 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:35:58.301 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:35:58.301 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:35:58.301 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-10T21:35:58.301 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:35:58.301 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:35:58.301 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:35:58.301 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:35:58.301 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-10T21:35:58.302 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:35:58.302 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:35:58.302 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:35:58.302 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:35:58.302 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-10T21:35:58.302 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-10T21:35:58.302 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:35:58.302 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:35:58.302 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:35:58.302 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:35:58.302 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:35:58.302 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:35:58.303 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:35:58.303 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:35:58.303 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:35:58.303 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:35:58.303 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:35:58.303 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3300", 2024-09-10T21:35:58.303 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:35:58.303 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:35:58.303 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:35:58.303 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:35:58.303 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789", 2024-09-10T21:35:58.303 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:35:58.303 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:35:58.303 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:35:58.304 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:35:58.304 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789/0", 2024-09-10T21:35:58.304 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6789/0", 2024-09-10T21:35:58.304 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:35:58.304 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:35:58.304 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:35:58.304 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:35:58.304 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:35:58.304 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:35:58.304 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:35:58.304 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:35:58.304 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:35:58.304 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:35:58.305 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:35:58.305 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3301", 2024-09-10T21:35:58.305 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:35:58.305 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:35:58.305 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:35:58.305 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:35:58.305 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790", 2024-09-10T21:35:58.305 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:35:58.305 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:35:58.305 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:35:58.305 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:35:58.305 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790/0", 2024-09-10T21:35:58.305 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6790/0", 2024-09-10T21:35:58.305 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:35:58.306 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:35:58.306 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:35:58.306 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:35:58.306 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:35:58.306 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:35:58.306 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-10T21:35:58.306 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-10T21:35:58.306 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:35:58.306 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:35:58.306 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:35:58.306 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:35:58.306 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:35:58.306 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:35:58.306 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-10T21:35:58.307 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:35:58.307 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:35:58.307 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:35:58.307 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 2 2024-09-10T21:35:58.307 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:35:58.307 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:35:58.307 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:35:58.307 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-10T21:35:58.307 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-10T21:35:58.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.287+0000 7f91c4e00640 1 -- 172.21.15.17:0/681903682 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f91a0072610 msgr2=0x7f91a0074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:58.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.287+0000 7f91c4e00640 1 --2- 172.21.15.17:0/681903682 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f91a0072610 0x7f91a0074ad0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f91c015b950 tx=0x7f91b8009500 comp rx=0 tx=0).stop 2024-09-10T21:35:58.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.287+0000 7f91c4e00640 1 -- 172.21.15.17:0/681903682 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f91c014ca80 msgr2=0x7f91c00c6420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:58.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.287+0000 7f91c4e00640 1 --2- 172.21.15.17:0/681903682 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f91c014ca80 0x7f91c00c6420 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f91ac01c610 tx=0x7f91ac0014a0 comp rx=0 tx=0).stop 2024-09-10T21:35:58.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.287+0000 7f91bfe00640 1 -- 172.21.15.17:0/681903682 reap_dead start 2024-09-10T21:35:58.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.287+0000 7f91bfe00640 1 -- 172.21.15.17:0/681903682 reap_dead start 2024-09-10T21:35:58.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.287+0000 7f91c4e00640 1 -- 172.21.15.17:0/681903682 shutdown_connections 2024-09-10T21:35:58.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.287+0000 7f91c4e00640 1 -- 172.21.15.17:0/681903682 >> 172.21.15.17:0/681903682 conn(0x7f91c00cd690 msgr2=0x7f91c0157790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:58.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.287+0000 7f91c4e00640 1 -- 172.21.15.17:0/681903682 shutdown_connections 2024-09-10T21:35:58.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.287+0000 7f91c4e00640 1 -- 172.21.15.17:0/681903682 wait complete. 2024-09-10T21:35:58.314 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-10T21:35:58.314 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-09-10T21:35:58.314 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-10T21:35:58.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.491+0000 7f084d600640 1 -- 172.21.15.17:0/3101878055 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f08480cbf40 msgr2=0x7f08480c9db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:58.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.491+0000 7f084d600640 1 --2- 172.21.15.17:0/3101878055 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f08480cbf40 0x7f08480c9db0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f0834009900 tx=0x7f083401c8e0 comp rx=0 tx=0).stop 2024-09-10T21:35:58.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.491+0000 7f084d600640 1 -- 172.21.15.17:0/3101878055 shutdown_connections 2024-09-10T21:35:58.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.491+0000 7f084d600640 1 --2- 172.21.15.17:0/3101878055 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f08480cbf40 0x7f08480c9db0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:58.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.491+0000 7f084d600640 1 --2- 172.21.15.17:0/3101878055 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f08480c99d0 0x7f08480cb8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:58.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.491+0000 7f084d600640 1 -- 172.21.15.17:0/3101878055 >> 172.21.15.17:0/3101878055 conn(0x7f084806e470 msgr2=0x7f084806ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:58.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.491+0000 7f084d600640 1 -- 172.21.15.17:0/3101878055 shutdown_connections 2024-09-10T21:35:58.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.491+0000 7f084d600640 1 -- 172.21.15.17:0/3101878055 wait complete. 2024-09-10T21:35:58.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.495+0000 7f084d600640 1 Processor -- start 2024-09-10T21:35:58.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.495+0000 7f084d600640 1 -- start start 2024-09-10T21:35:58.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.495+0000 7f084d600640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08480c99d0 0x7f0848154b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:58.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.495+0000 7f084d600640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f08480cbf40 0x7f08480c40c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:58.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.495+0000 7f084d600640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f084815add0 0x7f084815cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:58.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.495+0000 7f084d600640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f08480d0800 con 0x7f08480cbf40 2024-09-10T21:35:58.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.495+0000 7f084d600640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f084815b610 con 0x7f08480c99d0 2024-09-10T21:35:58.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.495+0000 7f084d600640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f08480d0980 con 0x7f084815add0 2024-09-10T21:35:58.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.495+0000 7f0847400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f08480cbf40 0x7f08480c40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:58.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.495+0000 7f0847400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f08480cbf40 0x7f08480c40c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40378/0 (socket says 172.21.15.17:40378) 2024-09-10T21:35:58.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.495+0000 7f0847400640 1 -- 172.21.15.17:0/3488256614 learned_addr learned my addr 172.21.15.17:0/3488256614 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:35:58.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.495+0000 7f0847e00640 1 --2- 172.21.15.17:0/3488256614 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08480c99d0 0x7f0848154b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:58.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.495+0000 7f084cc00640 1 --2- 172.21.15.17:0/3488256614 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f084815add0 0x7f084815cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:58.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.495+0000 7f0847400640 1 -- 172.21.15.17:0/3488256614 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f084815add0 msgr2=0x7f084815cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:58.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.495+0000 7f0847400640 1 --2- 172.21.15.17:0/3488256614 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f084815add0 0x7f084815cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:58.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.495+0000 7f0847400640 1 -- 172.21.15.17:0/3488256614 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08480c99d0 msgr2=0x7f0848154b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:58.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.495+0000 7f0847400640 1 --2- 172.21.15.17:0/3488256614 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f08480c99d0 0x7f0848154b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:58.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.495+0000 7f0847400640 1 -- 172.21.15.17:0/3488256614 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f084815ec60 con 0x7f08480cbf40 2024-09-10T21:35:58.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.495+0000 7f084cc00640 1 --2- 172.21.15.17:0/3488256614 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f084815add0 0x7f084815cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:35:58.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.495+0000 7f0847400640 1 --2- 172.21.15.17:0/3488256614 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f08480cbf40 0x7f08480c40c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f0834009850 tx=0x7f0834005760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:58.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.495+0000 7f083fe00640 1 -- 172.21.15.17:0/3488256614 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08340b2040 con 0x7f08480cbf40 2024-09-10T21:35:58.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.495+0000 7f083fe00640 1 -- 172.21.15.17:0/3488256614 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0834004330 con 0x7f08480cbf40 2024-09-10T21:35:58.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.495+0000 7f083fe00640 1 -- 172.21.15.17:0/3488256614 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0834005290 con 0x7f08480cbf40 2024-09-10T21:35:58.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.495+0000 7f084d600640 1 -- 172.21.15.17:0/3488256614 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f084815ef00 con 0x7f08480cbf40 2024-09-10T21:35:58.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.495+0000 7f084d600640 1 -- 172.21.15.17:0/3488256614 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08481576a0 con 0x7f08480cbf40 2024-09-10T21:35:58.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.495+0000 7f084d600640 1 --2- 172.21.15.17:0/3488256614 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f0848164b10 0x7f084815d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:58.504 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.495+0000 7f084d600640 1 -- 172.21.15.17:0/3488256614 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f0848069ef0 con 0x7f0848164b10 2024-09-10T21:35:58.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.499+0000 7f0847e00640 1 --2- 172.21.15.17:0/3488256614 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f0848164b10 0x7f084815d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:58.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.499+0000 7f0847e00640 1 -- 172.21.15.17:0/3488256614 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f08480cb8d0 con 0x7f0848164b10 2024-09-10T21:35:58.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.499+0000 7f0847e00640 1 --2- 172.21.15.17:0/3488256614 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f0848164b10 0x7f084815d490 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f0838011a10 tx=0x7f0838011ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:58.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.499+0000 7f083fe00640 1 -- 172.21.15.17:0/3488256614 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f0834005430 con 0x7f08480cbf40 2024-09-10T21:35:58.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.499+0000 7f083fe00640 1 --2- 172.21.15.17:0/3488256614 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f08280725e0 0x7f0828074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:58.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.499+0000 7f083fe00640 1 -- 172.21.15.17:0/3488256614 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f083412ca80 con 0x7f08480cbf40 2024-09-10T21:35:58.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.499+0000 7f083fe00640 1 -- 172.21.15.17:0/3488256614 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f083800fc20 con 0x7f0848164b10 2024-09-10T21:35:58.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.499+0000 7f083fe00640 1 -- 172.21.15.17:0/3488256614 <== mon.2 v2:172.21.15.31:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f083800fdc0 con 0x7f0848164b10 2024-09-10T21:35:58.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.499+0000 7f083fe00640 1 -- 172.21.15.17:0/3488256614 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f0848164b10 msgr2=0x7f084815d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:58.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.499+0000 7f083fe00640 1 --2- 172.21.15.17:0/3488256614 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f0848164b10 0x7f084815d490 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f0838011a10 tx=0x7f0838011ed0 comp rx=0 tx=0).stop 2024-09-10T21:35:58.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.499+0000 7f083fe00640 1 -- 172.21.15.17:0/3488256614 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f0848164b10 msgr2=0x7f084815d490 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:35:58.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.499+0000 7f083fe00640 1 --2- 172.21.15.17:0/3488256614 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f0848164b10 0x7f084815d490 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:58.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.499+0000 7f084cc00640 1 --2- 172.21.15.17:0/3488256614 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f08280725e0 0x7f0828074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:58.507 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.499+0000 7f084cc00640 1 --2- 172.21.15.17:0/3488256614 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f08280725e0 0x7f0828074aa0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f08400012f0 tx=0x7f0840009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:58.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.507+0000 7f084d600640 1 --2- 172.21.15.17:0/3488256614 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f084815db30 0x7f08481f66f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:58.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.507+0000 7f084d600640 1 -- 172.21.15.17:0/3488256614 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f0848155df0 con 0x7f084815db30 2024-09-10T21:35:58.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.507+0000 7f0847e00640 1 --2- 172.21.15.17:0/3488256614 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f084815db30 0x7f08481f66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:58.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.507+0000 7f0847e00640 1 -- 172.21.15.17:0/3488256614 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f084815a060 con 0x7f084815db30 2024-09-10T21:35:58.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.507+0000 7f0847e00640 1 --2- 172.21.15.17:0/3488256614 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f084815db30 0x7f08481f66f0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f0838011a10 tx=0x7f08380046d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:58.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.507+0000 7f083fe00640 1 -- 172.21.15.17:0/3488256614 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08380047b0 con 0x7f084815db30 2024-09-10T21:35:58.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.507+0000 7f083fe00640 1 -- 172.21.15.17:0/3488256614 <== mon.2 v2:172.21.15.31:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4541 (secure 0 0 0) 0x7f08380047b0 con 0x7f084815db30 2024-09-10T21:35:58.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.507+0000 7f083fe00640 1 -- 172.21.15.17:0/3488256614 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f084815db30 msgr2=0x7f08481f66f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:58.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.507+0000 7f083fe00640 1 --2- 172.21.15.17:0/3488256614 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f084815db30 0x7f08481f66f0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f0838011a10 tx=0x7f08380046d0 comp rx=0 tx=0).stop 2024-09-10T21:35:58.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.507+0000 7f083fe00640 1 -- 172.21.15.17:0/3488256614 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f084815db30 msgr2=0x7f08481f66f0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:35:58.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.507+0000 7f083fe00640 1 --2- 172.21.15.17:0/3488256614 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f084815db30 0x7f08481f66f0 secure :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f0838011a10 tx=0x7f08380046d0 comp rx=0 tx=0).stop 2024-09-10T21:35:58.516 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-10T21:35:58.516 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:35:58.516 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:35:58.516 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-10T21:35:58.516 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 92, 2024-09-10T21:35:58.516 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 22943, 2024-09-10T21:35:58.516 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-10T21:35:58.516 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-10T21:35:58.516 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-10T21:35:58.516 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-10T21:35:58.516 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:35:58.517 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 17, 2024-09-10T21:35:58.517 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:35:58.517 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-10T21:35:58.517 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-10T21:35:58.517 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:35:58.517 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:35:58.517 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:35:58.517 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:35:58.517 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:35:58.517 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:35:58.517 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:35:58.517 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:35:58.517 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:35:58.517 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:35:58.518 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:35:58.518 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:35:58.518 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-10T21:35:58.518 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-10T21:35:58.518 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:35:58.518 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:35:58.518 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:35:58.518 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:35:58.518 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:35:58.518 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:35:58.518 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:35:58.518 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:35:58.518 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:35:58.518 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:35:58.519 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:35:58.519 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:35:58.519 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:35:58.519 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-10T21:35:58.519 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-10T21:35:58.519 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-10T21:35:58.519 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-10T21:35:58.519 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-10T21:35:58.519 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "5e2f023f-d6a6-49f4-a256-f4807dce3739", 2024-09-10T21:35:58.519 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:35:58.519 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:35:58.519 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-10T21:35:58.519 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-10T21:35:58.520 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-10T21:35:58.520 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-10T21:35:58.520 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-10T21:35:58.520 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-10T21:35:58.520 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-10T21:35:58.520 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:35:58.520 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-10T21:35:58.520 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:35:58.520 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:35:58.520 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:35:58.520 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:35:58.520 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:35:58.520 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:35:58.520 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:35:58.521 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:35:58.521 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:35:58.521 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:35:58.521 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:35:58.521 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:35:58.521 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-10T21:35:58.521 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:35:58.521 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-10T21:35:58.521 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:35:58.521 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:35:58.521 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:35:58.521 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:35:58.521 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:35:58.522 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:35:58.522 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:35:58.522 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-10T21:35:58.522 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:35:58.522 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:35:58.522 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:35:58.522 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:35:58.522 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-10T21:35:58.522 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:35:58.522 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:35:58.522 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:35:58.522 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:35:58.522 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-10T21:35:58.523 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-10T21:35:58.523 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:35:58.523 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:35:58.523 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:35:58.523 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:35:58.523 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:35:58.523 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:35:58.523 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:35:58.523 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:35:58.523 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:35:58.523 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:35:58.523 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:35:58.523 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3300", 2024-09-10T21:35:58.523 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:35:58.524 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:35:58.524 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:35:58.524 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:35:58.524 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789", 2024-09-10T21:35:58.524 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:35:58.524 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:35:58.524 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:35:58.524 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:35:58.524 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789/0", 2024-09-10T21:35:58.524 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6789/0", 2024-09-10T21:35:58.524 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:35:58.524 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:35:58.524 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:35:58.524 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:35:58.525 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:35:58.525 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:35:58.525 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:35:58.525 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:35:58.525 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:35:58.525 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:35:58.525 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:35:58.525 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3301", 2024-09-10T21:35:58.525 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:35:58.525 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:35:58.525 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:35:58.525 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:35:58.525 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790", 2024-09-10T21:35:58.526 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:35:58.526 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:35:58.526 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:35:58.526 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:35:58.526 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790/0", 2024-09-10T21:35:58.526 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6790/0", 2024-09-10T21:35:58.526 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:35:58.526 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:35:58.526 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:35:58.526 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:35:58.526 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:35:58.526 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:35:58.526 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-10T21:35:58.526 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-10T21:35:58.526 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:35:58.527 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:35:58.527 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:35:58.527 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:35:58.527 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:35:58.527 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:35:58.527 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-10T21:35:58.527 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:35:58.527 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:35:58.527 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:35:58.527 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:35:58.527 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:35:58.527 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:35:58.527 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:35:58.527 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-10T21:35:58.528 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-10T21:35:58.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.507+0000 7f084d600640 1 -- 172.21.15.17:0/3488256614 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f08280725e0 msgr2=0x7f0828074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:58.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.507+0000 7f084d600640 1 --2- 172.21.15.17:0/3488256614 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f08280725e0 0x7f0828074aa0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f08400012f0 tx=0x7f0840009040 comp rx=0 tx=0).stop 2024-09-10T21:35:58.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.507+0000 7f084d600640 1 -- 172.21.15.17:0/3488256614 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f08480cbf40 msgr2=0x7f08480c40c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:58.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.507+0000 7f084d600640 1 --2- 172.21.15.17:0/3488256614 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f08480cbf40 0x7f08480c40c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f0834009850 tx=0x7f0834005760 comp rx=0 tx=0).stop 2024-09-10T21:35:58.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.507+0000 7f084cc00640 1 -- 172.21.15.17:0/3488256614 reap_dead start 2024-09-10T21:35:58.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.507+0000 7f084d600640 1 -- 172.21.15.17:0/3488256614 shutdown_connections 2024-09-10T21:35:58.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.507+0000 7f084d600640 1 --2- 172.21.15.17:0/3488256614 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f08480cbf40 0x7f08480c40c0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:58.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.507+0000 7f084d600640 1 -- 172.21.15.17:0/3488256614 >> 172.21.15.17:0/3488256614 conn(0x7f084806e470 msgr2=0x7f08480cff50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:58.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.507+0000 7f084d600640 1 -- 172.21.15.17:0/3488256614 shutdown_connections 2024-09-10T21:35:58.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.511+0000 7f084d600640 1 -- 172.21.15.17:0/3488256614 wait complete. 2024-09-10T21:35:58.535 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-10T21:35:58.535 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['c'] 2024-09-10T21:35:58.535 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2024-09-10T21:35:58.535 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.c 2024-09-10T21:35:58.536 INFO:tasks.mon_thrash.mon_thrasher:killing mon.c 2024-09-10T21:35:58.536 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-10T21:35:58.536 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:35:58.552 INFO:tasks.ceph.mon.c:Stopped 2024-09-10T21:35:58.552 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2024-09-10T21:35:58.552 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-10T21:35:58.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.699+0000 7fb149800640 1 -- 172.21.15.17:0/1605213628 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1440c7080 msgr2=0x7fb14406e850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:58.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.699+0000 7fb149800640 1 --2- 172.21.15.17:0/1605213628 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1440c7080 0x7fb14406e850 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb134009900 tx=0x7fb13401c830 comp rx=0 tx=0).stop 2024-09-10T21:35:58.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.699+0000 7fb149800640 1 -- 172.21.15.17:0/1605213628 shutdown_connections 2024-09-10T21:35:58.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.699+0000 7fb149800640 1 --2- 172.21.15.17:0/1605213628 >> v2:172.21.15.17:3300/0 conn(0x7fb14415e7e0 0x7fb1440c79c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:58.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.699+0000 7fb149800640 1 --2- 172.21.15.17:0/1605213628 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fb144070670 0x7fb14406ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:58.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.699+0000 7fb149800640 1 --2- 172.21.15.17:0/1605213628 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1440c7080 0x7fb14406e850 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:58.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.699+0000 7fb149800640 1 -- 172.21.15.17:0/1605213628 >> 172.21.15.17:0/1605213628 conn(0x7fb14406dd30 msgr2=0x7fb14406e140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:58.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.699+0000 7fb149800640 1 -- 172.21.15.17:0/1605213628 shutdown_connections 2024-09-10T21:35:58.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.699+0000 7fb149800640 1 -- 172.21.15.17:0/1605213628 wait complete. 2024-09-10T21:35:58.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.699+0000 7fb149800640 1 Processor -- start 2024-09-10T21:35:58.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.699+0000 7fb149800640 1 -- start start 2024-09-10T21:35:58.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.699+0000 7fb149800640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb144070670 0x7fb1440af4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:58.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.699+0000 7fb149800640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fb1440c7080 0x7fb1441528e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:58.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.699+0000 7fb149800640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb14415e7e0 0x7fb1441f4000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:58.707 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.699+0000 7fb149800640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fb14406d430 con 0x7fb144070670 2024-09-10T21:35:58.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.699+0000 7fb149800640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fb1440c7460 con 0x7fb14415e7e0 2024-09-10T21:35:58.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.699+0000 7fb149800640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7fb14406d5b0 con 0x7fb1440c7080 2024-09-10T21:35:58.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.699+0000 7fb142000640 1 -- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fb1440c7080 msgr2=0x7fb1441528e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:3301/0 2024-09-10T21:35:58.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.699+0000 7fb142a00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb144070670 0x7fb1440af4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:58.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.699+0000 7fb142000640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fb1440c7080 0x7fb1441528e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:35:58.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.699+0000 7fb142a00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb144070670 0x7fb1440af4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40400/0 (socket says 172.21.15.17:40400) 2024-09-10T21:35:58.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.699+0000 7fb142a00640 1 -- 172.21.15.17:0/1823905241 learned_addr learned my addr 172.21.15.17:0/1823905241 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:35:58.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.699+0000 7fb143400640 1 --2- 172.21.15.17:0/1823905241 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb14415e7e0 0x7fb1441f4000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:58.709 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.703+0000 7fb143400640 1 -- 172.21.15.17:0/1823905241 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fb1440c7080 msgr2=0x7fb1441528e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:35:58.709 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.703+0000 7fb143400640 1 --2- 172.21.15.17:0/1823905241 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fb1440c7080 0x7fb1441528e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:58.709 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.703+0000 7fb143400640 1 -- 172.21.15.17:0/1823905241 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb144070670 msgr2=0x7fb1440af4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:58.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.703+0000 7fb143400640 1 --2- 172.21.15.17:0/1823905241 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb144070670 0x7fb1440af4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:58.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.703+0000 7fb143400640 1 -- 172.21.15.17:0/1823905241 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1441f8100 con 0x7fb14415e7e0 2024-09-10T21:35:58.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.703+0000 7fb142a00640 1 --2- 172.21.15.17:0/1823905241 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb144070670 0x7fb1440af4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:35:58.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.703+0000 7fb143400640 1 --2- 172.21.15.17:0/1823905241 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb14415e7e0 0x7fb1441f4000 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb13c00e9c0 tx=0x7fb13c00ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:58.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.703+0000 7fb13aa00640 1 -- 172.21.15.17:0/1823905241 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb13c00cde0 con 0x7fb14415e7e0 2024-09-10T21:35:58.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.703+0000 7fb13aa00640 1 -- 172.21.15.17:0/1823905241 <== mon.1 v2:172.21.15.31:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb13c004b30 con 0x7fb14415e7e0 2024-09-10T21:35:58.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.703+0000 7fb149800640 1 -- 172.21.15.17:0/1823905241 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb144153240 con 0x7fb14415e7e0 2024-09-10T21:35:58.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.703+0000 7fb13aa00640 1 -- 172.21.15.17:0/1823905241 <== mon.1 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb13c005020 con 0x7fb14415e7e0 2024-09-10T21:35:58.710 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.703+0000 7fb149800640 1 -- 172.21.15.17:0/1823905241 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1440aec00 con 0x7fb14415e7e0 2024-09-10T21:35:58.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.703+0000 7fb149800640 1 -- 172.21.15.17:0/1823905241 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1441f7f30 con 0x7fb14415e7e0 2024-09-10T21:35:58.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.703+0000 7fb13aa00640 1 -- 172.21.15.17:0/1823905241 <== mon.1 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fb13c0051c0 con 0x7fb14415e7e0 2024-09-10T21:35:58.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.707+0000 7fb13aa00640 1 --2- 172.21.15.17:0/1823905241 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fb1240725c0 0x7fb124074a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:35:58.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.707+0000 7fb13aa00640 1 -- 172.21.15.17:0/1823905241 <== mon.1 v2:172.21.15.31:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fb13c0940a0 con 0x7fb14415e7e0 2024-09-10T21:35:58.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.707+0000 7fb142a00640 1 --2- 172.21.15.17:0/1823905241 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fb1240725c0 0x7fb124074a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:35:58.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.707+0000 7fb142a00640 1 --2- 172.21.15.17:0/1823905241 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fb1240725c0 0x7fb124074a80 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fb13401d0d0 tx=0x7fb13401b070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:35:58.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.711+0000 7fb13aa00640 1 -- 172.21.15.17:0/1823905241 <== mon.1 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fb13c05f410 con 0x7fb14415e7e0 2024-09-10T21:35:59.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.995+0000 7fb149800640 1 -- 172.21.15.17:0/1823905241 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fb144069e60 con 0x7fb14415e7e0 2024-09-10T21:35:59.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.999+0000 7fb13aa00640 1 -- 172.21.15.17:0/1823905241 <== mon.1 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1557 (secure 0 0 0) 0x7fb13c05ed10 con 0x7fb14415e7e0 2024-09-10T21:35:59.005 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":92,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":17,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5e2f023f-d6a6-49f4-a256-f4807dce3739","modified":"2024-09-10T21:23:46.892130Z","created":"2024-09-10T21:23:46.892130Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3301","nonce":0},{"type":"v1","addr":"172.21.15.31:6790","nonce":0}]},"addr":"172.21.15.31:6790/0","public_addr":"172.21.15.31:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-10T21:35:59.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.999+0000 7fb149800640 1 -- 172.21.15.17:0/1823905241 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fb1240725c0 msgr2=0x7fb124074a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:59.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.999+0000 7fb149800640 1 --2- 172.21.15.17:0/1823905241 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fb1240725c0 0x7fb124074a80 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fb13401d0d0 tx=0x7fb13401b070 comp rx=0 tx=0).stop 2024-09-10T21:35:59.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.999+0000 7fb149800640 1 -- 172.21.15.17:0/1823905241 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb14415e7e0 msgr2=0x7fb1441f4000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:35:59.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.999+0000 7fb149800640 1 --2- 172.21.15.17:0/1823905241 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb14415e7e0 0x7fb1441f4000 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb13c00e9c0 tx=0x7fb13c00ee80 comp rx=0 tx=0).stop 2024-09-10T21:35:59.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.999+0000 7fb149800640 1 -- 172.21.15.17:0/1823905241 shutdown_connections 2024-09-10T21:35:59.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.999+0000 7fb149800640 1 --2- 172.21.15.17:0/1823905241 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fb1240725c0 0x7fb124074a80 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:59.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.999+0000 7fb149800640 1 --2- 172.21.15.17:0/1823905241 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb14415e7e0 0x7fb1441f4000 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:59.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.999+0000 7fb149800640 1 --2- 172.21.15.17:0/1823905241 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fb1440c7080 0x7fb1441528e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:59.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.999+0000 7fb149800640 1 --2- 172.21.15.17:0/1823905241 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb144070670 0x7fb1440af4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:35:59.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.999+0000 7fb149800640 1 -- 172.21.15.17:0/1823905241 >> 172.21.15.17:0/1823905241 conn(0x7fb14406dd30 msgr2=0x7fb1440cb650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:35:59.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.999+0000 7fb149800640 1 -- 172.21.15.17:0/1823905241 shutdown_connections 2024-09-10T21:35:59.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:35:58.999+0000 7fb149800640 1 -- 172.21.15.17:0/1823905241 wait complete. 2024-09-10T21:36:02.030 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-10T21:36:02.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:02.191+0000 7f10a9000640 1 -- 172.21.15.17:0/1910009847 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1088004810 msgr2=0x7f1088004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:02.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:02.191+0000 7f10a9000640 1 --2- 172.21.15.17:0/1910009847 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1088004810 0x7f1088004c90 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f10a4159020 tx=0x7f109001c7a0 comp rx=0 tx=0).stop 2024-09-10T21:36:02.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:02.191+0000 7f10a3e00640 1 -- 172.21.15.17:0/1910009847 reap_dead start 2024-09-10T21:36:02.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:02.191+0000 7f10a9000640 1 -- 172.21.15.17:0/1910009847 shutdown_connections 2024-09-10T21:36:02.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:02.191+0000 7f10a9000640 1 -- 172.21.15.17:0/1910009847 >> 172.21.15.17:0/1910009847 conn(0x7f10a40cdef0 msgr2=0x7f10a40ce2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:36:02.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:02.191+0000 7f10a9000640 1 -- 172.21.15.17:0/1910009847 shutdown_connections 2024-09-10T21:36:02.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:02.191+0000 7f10a9000640 1 -- 172.21.15.17:0/1910009847 wait complete. 2024-09-10T21:36:02.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:02.191+0000 7f10a9000640 1 Processor -- start 2024-09-10T21:36:02.200 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:02.195+0000 7f10a9000640 1 -- start start 2024-09-10T21:36:02.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:02.195+0000 7f10a9000640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1088004810 0x7f10a4147f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:02.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:02.195+0000 7f10a9000640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f10a41f4f60 0x7f10a413e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:02.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:02.195+0000 7f10a9000640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f10a41f53a0 0x7f10a413eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:02.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:02.195+0000 7f10a9000640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f10a40c9350 con 0x7f1088004810 2024-09-10T21:36:02.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:02.195+0000 7f10a9000640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f10a40c8e50 con 0x7f10a41f53a0 2024-09-10T21:36:02.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:02.195+0000 7f10a9000640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f10a415b9e0 con 0x7f10a41f4f60 2024-09-10T21:36:02.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:02.195+0000 7f10a3400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1088004810 0x7f10a4147f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:02.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:02.195+0000 7f10a3400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1088004810 0x7f10a4147f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:40416/0 (socket says 172.21.15.17:40416) 2024-09-10T21:36:02.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:02.195+0000 7f10a3400640 1 -- 172.21.15.17:0/901141242 learned_addr learned my addr 172.21.15.17:0/901141242 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:36:02.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:02.195+0000 7f10a2a00640 1 -- 172.21.15.17:0/901141242 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f10a41f4f60 msgr2=0x7f10a413e630 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:3301/0 2024-09-10T21:36:02.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:02.195+0000 7f10a2a00640 1 --2- 172.21.15.17:0/901141242 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f10a41f4f60 0x7f10a413e630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:36:02.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:02.195+0000 7f10a3e00640 1 --2- 172.21.15.17:0/901141242 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f10a41f53a0 0x7f10a413eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:02.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:02.195+0000 7f10a3400640 1 -- 172.21.15.17:0/901141242 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f10a41f4f60 msgr2=0x7f10a413e630 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:36:02.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:02.195+0000 7f10a3400640 1 --2- 172.21.15.17:0/901141242 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f10a41f4f60 0x7f10a413e630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:02.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:02.195+0000 7f10a3400640 1 -- 172.21.15.17:0/901141242 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f10a41f53a0 msgr2=0x7f10a413eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:02.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:02.195+0000 7f10a3400640 1 --2- 172.21.15.17:0/901141242 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f10a41f53a0 0x7f10a413eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:02.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:02.195+0000 7f10a3400640 1 -- 172.21.15.17:0/901141242 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10a40c5170 con 0x7f1088004810 2024-09-10T21:36:02.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:02.195+0000 7f10a3400640 1 --2- 172.21.15.17:0/901141242 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1088004810 0x7f10a4147f50 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f1094003680 tx=0x7f1094003b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:02.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:02.195+0000 7f109b400640 1 -- 172.21.15.17:0/901141242 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1094015070 con 0x7f1088004810 2024-09-10T21:36:02.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:02.195+0000 7f109b400640 1 -- 172.21.15.17:0/901141242 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f109400fad0 con 0x7f1088004810 2024-09-10T21:36:02.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:02.195+0000 7f109b400640 1 -- 172.21.15.17:0/901141242 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1094011430 con 0x7f1088004810 2024-09-10T21:36:02.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:02.195+0000 7f10a9000640 1 -- 172.21.15.17:0/901141242 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10a4142ba0 con 0x7f1088004810 2024-09-10T21:36:02.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:02.195+0000 7f10a9000640 1 -- 172.21.15.17:0/901141242 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10a415a840 con 0x7f1088004810 2024-09-10T21:36:02.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:02.195+0000 7f10a9000640 1 -- 172.21.15.17:0/901141242 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10a415bf00 con 0x7f1088004810 2024-09-10T21:36:02.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:02.199+0000 7f109b400640 1 -- 172.21.15.17:0/901141242 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f10940115d0 con 0x7f1088004810 2024-09-10T21:36:02.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:02.199+0000 7f109b400640 1 --2- 172.21.15.17:0/901141242 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f1084072590 0x7f1084074a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:02.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:02.199+0000 7f109b400640 1 -- 172.21.15.17:0/901141242 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f10940946d0 con 0x7f1088004810 2024-09-10T21:36:02.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:02.199+0000 7f10a2a00640 1 --2- 172.21.15.17:0/901141242 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f1084072590 0x7f1084074a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:02.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:02.199+0000 7f10a2a00640 1 --2- 172.21.15.17:0/901141242 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f1084072590 0x7f1084074a50 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f10900af000 tx=0x7f10900a7040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:02.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:02.203+0000 7f109b400640 1 -- 172.21.15.17:0/901141242 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f10a41528b0 con 0x7f1088004810 2024-09-10T21:36:02.488 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:02.483+0000 7f10a9000640 1 -- 172.21.15.17:0/901141242 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f10a40c8ad0 con 0x7f1088004810 2024-09-10T21:36:02.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:02.483+0000 7f109b400640 1 -- 172.21.15.17:0/901141242 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1557 (secure 0 0 0) 0x7f109405f880 con 0x7f1088004810 2024-09-10T21:36:02.489 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":92,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":21,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5e2f023f-d6a6-49f4-a256-f4807dce3739","modified":"2024-09-10T21:23:46.892130Z","created":"2024-09-10T21:23:46.892130Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3301","nonce":0},{"type":"v1","addr":"172.21.15.31:6790","nonce":0}]},"addr":"172.21.15.31:6790/0","public_addr":"172.21.15.31:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-10T21:36:02.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:02.483+0000 7f10a9000640 1 -- 172.21.15.17:0/901141242 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f1084072590 msgr2=0x7f1084074a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:02.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:02.483+0000 7f10a9000640 1 --2- 172.21.15.17:0/901141242 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f1084072590 0x7f1084074a50 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f10900af000 tx=0x7f10900a7040 comp rx=0 tx=0).stop 2024-09-10T21:36:02.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:02.483+0000 7f10a9000640 1 -- 172.21.15.17:0/901141242 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1088004810 msgr2=0x7f10a4147f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:02.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:02.483+0000 7f10a9000640 1 --2- 172.21.15.17:0/901141242 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1088004810 0x7f10a4147f50 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f1094003680 tx=0x7f1094003b40 comp rx=0 tx=0).stop 2024-09-10T21:36:02.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:02.483+0000 7f10a9000640 1 -- 172.21.15.17:0/901141242 shutdown_connections 2024-09-10T21:36:02.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:02.483+0000 7f10a9000640 1 --2- 172.21.15.17:0/901141242 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f1084072590 0x7f1084074a50 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:02.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:02.483+0000 7f10a9000640 1 --2- 172.21.15.17:0/901141242 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f10a41f53a0 0x7f10a413eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:02.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:02.483+0000 7f10a9000640 1 --2- 172.21.15.17:0/901141242 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f10a41f4f60 0x7f10a413e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:02.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:02.483+0000 7f10a9000640 1 --2- 172.21.15.17:0/901141242 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1088004810 0x7f10a4147f50 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:02.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:02.483+0000 7f10a9000640 1 -- 172.21.15.17:0/901141242 >> 172.21.15.17:0/901141242 conn(0x7f10a40cdef0 msgr2=0x7f10a4155de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:36:02.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:02.483+0000 7f10a9000640 1 -- 172.21.15.17:0/901141242 shutdown_connections 2024-09-10T21:36:02.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:02.483+0000 7f10a9000640 1 -- 172.21.15.17:0/901141242 wait complete. 2024-09-10T21:36:05.512 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-10T21:36:05.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:05.663+0000 7fd993200640 1 -- 172.21.15.17:0/4279193018 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd98c070450 msgr2=0x7fd98c157470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:05.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:05.663+0000 7fd993200640 1 --2- 172.21.15.17:0/4279193018 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd98c070450 0x7fd98c157470 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fd978009900 tx=0x7fd97801f0f0 comp rx=0 tx=0).stop 2024-09-10T21:36:05.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:05.663+0000 7fd993200640 1 -- 172.21.15.17:0/4279193018 shutdown_connections 2024-09-10T21:36:05.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:05.663+0000 7fd993200640 1 --2- 172.21.15.17:0/4279193018 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd98c0c5ec0 0x7fd98c0c6330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:05.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:05.663+0000 7fd993200640 1 --2- 172.21.15.17:0/4279193018 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd98c070450 0x7fd98c157470 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:05.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:05.663+0000 7fd993200640 1 -- 172.21.15.17:0/4279193018 >> 172.21.15.17:0/4279193018 conn(0x7fd98c06dd30 msgr2=0x7fd98c06e140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:36:05.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:05.663+0000 7fd993200640 1 -- 172.21.15.17:0/4279193018 shutdown_connections 2024-09-10T21:36:05.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:05.663+0000 7fd993200640 1 -- 172.21.15.17:0/4279193018 wait complete. 2024-09-10T21:36:05.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:05.667+0000 7fd993200640 1 Processor -- start 2024-09-10T21:36:05.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:05.667+0000 7fd993200640 1 -- start start 2024-09-10T21:36:05.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:05.667+0000 7fd993200640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd98c070450 0x7fd98c1550d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:05.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:05.667+0000 7fd993200640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd98c0c5ec0 0x7fd98c14e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:05.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:05.667+0000 7fd993200640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd98c0c7210 0x7fd98c1f4f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:05.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:05.667+0000 7fd993200640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fd98c15b3b0 con 0x7fd98c0c5ec0 2024-09-10T21:36:05.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:05.667+0000 7fd993200640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fd98c15b0d0 con 0x7fd98c0c7210 2024-09-10T21:36:05.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:05.667+0000 7fd993200640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7fd98c15b690 con 0x7fd98c070450 2024-09-10T21:36:05.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:05.667+0000 7fd98be00640 1 -- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd98c070450 msgr2=0x7fd98c1550d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:3301/0 2024-09-10T21:36:05.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:05.667+0000 7fd98b400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd98c0c5ec0 0x7fd98c14e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:05.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:05.667+0000 7fd98be00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd98c070450 0x7fd98c1550d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:36:05.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:05.667+0000 7fd98b400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd98c0c5ec0 0x7fd98c14e780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:54096/0 (socket says 172.21.15.17:54096) 2024-09-10T21:36:05.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:05.667+0000 7fd98b400640 1 -- 172.21.15.17:0/3889929043 learned_addr learned my addr 172.21.15.17:0/3889929043 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:36:05.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:05.667+0000 7fd98b400640 1 -- 172.21.15.17:0/3889929043 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd98c070450 msgr2=0x7fd98c1550d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:36:05.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:05.667+0000 7fd98b400640 1 --2- 172.21.15.17:0/3889929043 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd98c070450 0x7fd98c1550d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:05.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:05.667+0000 7fd98b400640 1 -- 172.21.15.17:0/3889929043 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd98c0c7210 msgr2=0x7fd98c1f4f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:05.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:05.667+0000 7fd98b400640 1 --2- 172.21.15.17:0/3889929043 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd98c0c7210 0x7fd98c1f4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:05.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:05.667+0000 7fd98b400640 1 -- 172.21.15.17:0/3889929043 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd98c14ecc0 con 0x7fd98c0c5ec0 2024-09-10T21:36:05.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:05.667+0000 7fd98b400640 1 --2- 172.21.15.17:0/3889929043 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd98c0c5ec0 0x7fd98c14e780 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fd97801d7a0 tx=0x7fd9780036a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:05.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:05.667+0000 7fd983e00640 1 -- 172.21.15.17:0/3889929043 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9780b5040 con 0x7fd98c0c5ec0 2024-09-10T21:36:05.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:05.667+0000 7fd983e00640 1 -- 172.21.15.17:0/3889929043 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd978005310 con 0x7fd98c0c5ec0 2024-09-10T21:36:05.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:05.667+0000 7fd983e00640 1 -- 172.21.15.17:0/3889929043 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd978003890 con 0x7fd98c0c5ec0 2024-09-10T21:36:05.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:05.667+0000 7fd993200640 1 -- 172.21.15.17:0/3889929043 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd98c1f1d90 con 0x7fd98c0c5ec0 2024-09-10T21:36:05.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:05.667+0000 7fd993200640 1 -- 172.21.15.17:0/3889929043 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd98c150770 con 0x7fd98c0c5ec0 2024-09-10T21:36:05.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:05.671+0000 7fd993200640 1 -- 172.21.15.17:0/3889929043 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd98c0c56a0 con 0x7fd98c0c5ec0 2024-09-10T21:36:05.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:05.671+0000 7fd983e00640 1 -- 172.21.15.17:0/3889929043 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fd978003a30 con 0x7fd98c0c5ec0 2024-09-10T21:36:05.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:05.671+0000 7fd983e00640 1 --2- 172.21.15.17:0/3889929043 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fd96c0725c0 0x7fd96c074a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:05.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:05.671+0000 7fd983e00640 1 -- 172.21.15.17:0/3889929043 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fd9780c0070 con 0x7fd98c0c5ec0 2024-09-10T21:36:05.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:05.671+0000 7fd98be00640 1 --2- 172.21.15.17:0/3889929043 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fd96c0725c0 0x7fd96c074a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:05.680 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:05.671+0000 7fd98be00640 1 --2- 172.21.15.17:0/3889929043 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fd96c0725c0 0x7fd96c074a80 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fd97c000bd0 tx=0x7fd97c007a00 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:05.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:05.675+0000 7fd983e00640 1 -- 172.21.15.17:0/3889929043 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fd978018c50 con 0x7fd98c0c5ec0 2024-09-10T21:36:05.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:05.943+0000 7fd993200640 1 -- 172.21.15.17:0/3889929043 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fd98c154e60 con 0x7fd98c0c5ec0 2024-09-10T21:36:05.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:05.943+0000 7fd983e00640 1 -- 172.21.15.17:0/3889929043 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1557 (secure 0 0 0) 0x7fd9780fc0b0 con 0x7fd98c0c5ec0 2024-09-10T21:36:05.950 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":92,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":24,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5e2f023f-d6a6-49f4-a256-f4807dce3739","modified":"2024-09-10T21:23:46.892130Z","created":"2024-09-10T21:23:46.892130Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3301","nonce":0},{"type":"v1","addr":"172.21.15.31:6790","nonce":0}]},"addr":"172.21.15.31:6790/0","public_addr":"172.21.15.31:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-10T21:36:05.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:05.947+0000 7fd993200640 1 -- 172.21.15.17:0/3889929043 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fd96c0725c0 msgr2=0x7fd96c074a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:05.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:05.947+0000 7fd993200640 1 --2- 172.21.15.17:0/3889929043 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fd96c0725c0 0x7fd96c074a80 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fd97c000bd0 tx=0x7fd97c007a00 comp rx=0 tx=0).stop 2024-09-10T21:36:05.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:05.947+0000 7fd993200640 1 -- 172.21.15.17:0/3889929043 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd98c0c5ec0 msgr2=0x7fd98c14e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:05.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:05.947+0000 7fd993200640 1 --2- 172.21.15.17:0/3889929043 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd98c0c5ec0 0x7fd98c14e780 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fd97801d7a0 tx=0x7fd9780036a0 comp rx=0 tx=0).stop 2024-09-10T21:36:05.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:05.947+0000 7fd993200640 1 -- 172.21.15.17:0/3889929043 shutdown_connections 2024-09-10T21:36:05.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:05.947+0000 7fd993200640 1 --2- 172.21.15.17:0/3889929043 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fd96c0725c0 0x7fd96c074a80 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:05.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:05.947+0000 7fd993200640 1 --2- 172.21.15.17:0/3889929043 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd98c0c7210 0x7fd98c1f4f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:05.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:05.947+0000 7fd993200640 1 --2- 172.21.15.17:0/3889929043 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd98c0c5ec0 0x7fd98c14e780 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:05.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:05.947+0000 7fd993200640 1 --2- 172.21.15.17:0/3889929043 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd98c070450 0x7fd98c1550d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:05.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:05.947+0000 7fd993200640 1 -- 172.21.15.17:0/3889929043 >> 172.21.15.17:0/3889929043 conn(0x7fd98c06dd30 msgr2=0x7fd98c0cc2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:36:05.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:05.947+0000 7fd993200640 1 -- 172.21.15.17:0/3889929043 shutdown_connections 2024-09-10T21:36:05.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:05.947+0000 7fd993200640 1 -- 172.21.15.17:0/3889929043 wait complete. 2024-09-10T21:36:08.978 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-10T21:36:12.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:12.767+0000 7efffb200640 1 -- 172.21.15.17:0/781772026 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7effe0004810 msgr2=0x7effe0004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:12.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:12.767+0000 7efffb200640 1 --2- 172.21.15.17:0/781772026 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7effe0004810 0x7effe0004c90 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7efffc0c6350 tx=0x7effe801d3d0 comp rx=0 tx=0).stop 2024-09-10T21:36:12.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:12.767+0000 7efffb200640 1 -- 172.21.15.17:0/781772026 shutdown_connections 2024-09-10T21:36:12.776 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:12.767+0000 7efffb200640 1 --2- 172.21.15.17:0/781772026 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7effe00052f0 0x7effe000b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:12.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:12.767+0000 7efffb200640 1 --2- 172.21.15.17:0/781772026 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7effe0004810 0x7effe0004c90 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:12.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:12.767+0000 7efffb200640 1 --2- 172.21.15.17:0/781772026 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7effe00039c0 0x7effe0003dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:12.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:12.767+0000 7efffb200640 1 -- 172.21.15.17:0/781772026 >> 172.21.15.17:0/781772026 conn(0x7efffc06cdf0 msgr2=0x7efffc06d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:36:12.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:12.771+0000 7efffb200640 1 -- 172.21.15.17:0/781772026 shutdown_connections 2024-09-10T21:36:12.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:12.771+0000 7efffb200640 1 -- 172.21.15.17:0/781772026 wait complete. 2024-09-10T21:36:12.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:12.771+0000 7efffb200640 1 Processor -- start 2024-09-10T21:36:12.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:12.771+0000 7efffb200640 1 -- start start 2024-09-10T21:36:12.780 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:12.771+0000 7efffb200640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7effe00039c0 0x7efffc1ef1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:12.780 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:12.771+0000 7efffb200640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7effe0004810 0x7efffc155a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:12.780 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:12.771+0000 7efffb200640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7effe00052f0 0x7efffc155f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:12.780 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:12.771+0000 7efffb200640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7efffc15a690 con 0x7effe00039c0 2024-09-10T21:36:12.780 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:12.771+0000 7efffb200640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7efffc15a3b0 con 0x7effe00052f0 2024-09-10T21:36:12.780 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:12.771+0000 7efffb200640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7efffc0c6820 con 0x7effe0004810 2024-09-10T21:36:12.780 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:12.771+0000 7efff9e00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7effe00039c0 0x7efffc1ef1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:12.780 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:12.775+0000 7efff9400640 1 -- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7effe0004810 msgr2=0x7efffc155a10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:3301/0 2024-09-10T21:36:12.780 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:12.775+0000 7efff9e00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7effe00039c0 0x7efffc1ef1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:54146/0 (socket says 172.21.15.17:54146) 2024-09-10T21:36:12.781 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:12.775+0000 7efff9e00640 1 -- 172.21.15.17:0/166608759 learned_addr learned my addr 172.21.15.17:0/166608759 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:36:12.781 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:12.775+0000 7efff9400640 1 --2- 172.21.15.17:0/166608759 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7effe0004810 0x7efffc155a10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:36:12.781 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:12.775+0000 7efffa800640 1 --2- 172.21.15.17:0/166608759 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7effe00052f0 0x7efffc155f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:12.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:12.775+0000 7efff9e00640 1 -- 172.21.15.17:0/166608759 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7effe0004810 msgr2=0x7efffc155a10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:36:12.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:12.775+0000 7efff9e00640 1 --2- 172.21.15.17:0/166608759 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7effe0004810 0x7efffc155a10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:12.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:12.775+0000 7efff9e00640 1 -- 172.21.15.17:0/166608759 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7effe00052f0 msgr2=0x7efffc155f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:12.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:12.775+0000 7efff9e00640 1 --2- 172.21.15.17:0/166608759 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7effe00052f0 0x7efffc155f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:12.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:12.775+0000 7efff9e00640 1 -- 172.21.15.17:0/166608759 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efffc1548c0 con 0x7effe00039c0 2024-09-10T21:36:12.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:12.775+0000 7efff9e00640 1 --2- 172.21.15.17:0/166608759 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7effe00039c0 0x7efffc1ef1d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7effec0022c0 tx=0x7effec0023b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:12.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:12.775+0000 7efff2000640 1 -- 172.21.15.17:0/166608759 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effec015070 con 0x7effe00039c0 2024-09-10T21:36:12.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:12.775+0000 7efff2000640 1 -- 172.21.15.17:0/166608759 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7effec00a700 con 0x7effe00039c0 2024-09-10T21:36:12.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:12.775+0000 7efff2000640 1 -- 172.21.15.17:0/166608759 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effec00ab70 con 0x7effe00039c0 2024-09-10T21:36:12.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:12.775+0000 7efffb200640 1 -- 172.21.15.17:0/166608759 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efffc1efc50 con 0x7effe00039c0 2024-09-10T21:36:12.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:12.775+0000 7efffb200640 1 -- 172.21.15.17:0/166608759 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efffc157600 con 0x7effe00039c0 2024-09-10T21:36:12.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:12.775+0000 7efffb200640 1 -- 172.21.15.17:0/166608759 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efffc069e60 con 0x7effe00039c0 2024-09-10T21:36:12.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:12.775+0000 7efff2000640 1 -- 172.21.15.17:0/166608759 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7effec007ce0 con 0x7effe00039c0 2024-09-10T21:36:12.785 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:12.779+0000 7efff2000640 1 --2- 172.21.15.17:0/166608759 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7effdc075cb0 0x7effdc078170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:12.785 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:12.779+0000 7efff2000640 1 -- 172.21.15.17:0/166608759 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7effec0955d0 con 0x7effe00039c0 2024-09-10T21:36:12.785 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:12.779+0000 7efff9400640 1 --2- 172.21.15.17:0/166608759 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7effdc075cb0 0x7effdc078170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:12.786 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:12.779+0000 7efff9400640 1 --2- 172.21.15.17:0/166608759 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7effdc075cb0 0x7effdc078170 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7effe80b0000 tx=0x7effe8006040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:12.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:12.783+0000 7efff2000640 1 -- 172.21.15.17:0/166608759 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7effec0608a0 con 0x7effe00039c0 2024-09-10T21:36:13.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:13.103+0000 7efffb200640 1 -- 172.21.15.17:0/166608759 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7efffc0ca890 con 0x7effe00039c0 2024-09-10T21:36:13.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:13.103+0000 7efff2000640 1 -- 172.21.15.17:0/166608759 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1550 (secure 0 0 0) 0x7effec0601a0 con 0x7effe00039c0 2024-09-10T21:36:13.109 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":96,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5e2f023f-d6a6-49f4-a256-f4807dce3739","modified":"2024-09-10T21:23:46.892130Z","created":"2024-09-10T21:23:46.892130Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3301","nonce":0},{"type":"v1","addr":"172.21.15.31:6790","nonce":0}]},"addr":"172.21.15.31:6790/0","public_addr":"172.21.15.31:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-10T21:36:13.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:13.103+0000 7efffb200640 1 -- 172.21.15.17:0/166608759 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7effdc075cb0 msgr2=0x7effdc078170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:13.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:13.103+0000 7efffb200640 1 --2- 172.21.15.17:0/166608759 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7effdc075cb0 0x7effdc078170 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7effe80b0000 tx=0x7effe8006040 comp rx=0 tx=0).stop 2024-09-10T21:36:13.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:13.103+0000 7efffb200640 1 -- 172.21.15.17:0/166608759 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7effe00039c0 msgr2=0x7efffc1ef1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:13.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:13.103+0000 7efffb200640 1 --2- 172.21.15.17:0/166608759 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7effe00039c0 0x7efffc1ef1d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7effec0022c0 tx=0x7effec0023b0 comp rx=0 tx=0).stop 2024-09-10T21:36:13.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:13.103+0000 7efffb200640 1 -- 172.21.15.17:0/166608759 shutdown_connections 2024-09-10T21:36:13.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:13.103+0000 7efffb200640 1 --2- 172.21.15.17:0/166608759 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7effdc075cb0 0x7effdc078170 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:13.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:13.103+0000 7efffb200640 1 --2- 172.21.15.17:0/166608759 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7effe00052f0 0x7efffc155f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:13.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:13.103+0000 7efffb200640 1 --2- 172.21.15.17:0/166608759 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7effe0004810 0x7efffc155a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:13.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:13.103+0000 7efffb200640 1 --2- 172.21.15.17:0/166608759 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7effe00039c0 0x7efffc1ef1d0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:13.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:13.103+0000 7efffb200640 1 -- 172.21.15.17:0/166608759 >> 172.21.15.17:0/166608759 conn(0x7efffc06cdf0 msgr2=0x7efffc0cc3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:36:13.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:13.103+0000 7efffb200640 1 -- 172.21.15.17:0/166608759 shutdown_connections 2024-09-10T21:36:13.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:13.103+0000 7efffb200640 1 -- 172.21.15.17:0/166608759 wait complete. 2024-09-10T21:36:13.133 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2024-09-10T21:36:13.134 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon a 2024-09-10T21:36:13.134 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-10T21:36:13.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:13.303+0000 7f0597600640 1 -- 172.21.15.17:0/989506894 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0590152820 msgr2=0x7f059015c280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:13.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:13.303+0000 7f0597600640 1 --2- 172.21.15.17:0/989506894 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0590152820 0x7f059015c280 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f05901594a0 tx=0x7f057c01c870 comp rx=0 tx=0).stop 2024-09-10T21:36:13.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:13.303+0000 7f0597600640 1 -- 172.21.15.17:0/989506894 shutdown_connections 2024-09-10T21:36:13.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:13.303+0000 7f0597600640 1 --2- 172.21.15.17:0/989506894 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0590152820 0x7f059015c280 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:13.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:13.303+0000 7f0597600640 1 --2- 172.21.15.17:0/989506894 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f0590152440 0x7f05900b3eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:13.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:13.303+0000 7f0597600640 1 -- 172.21.15.17:0/989506894 >> 172.21.15.17:0/989506894 conn(0x7f05900702f0 msgr2=0x7f0590070700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:36:13.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:13.303+0000 7f0597600640 1 -- 172.21.15.17:0/989506894 shutdown_connections 2024-09-10T21:36:13.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:13.303+0000 7f0597600640 1 -- 172.21.15.17:0/989506894 wait complete. 2024-09-10T21:36:13.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:13.307+0000 7f0597600640 1 Processor -- start 2024-09-10T21:36:13.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:13.307+0000 7f0597600640 1 -- start start 2024-09-10T21:36:13.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:13.307+0000 7f0597600640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f05900cdf20 0x7f0590157100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:13.313 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:13.307+0000 7f0597600640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0590152440 0x7f059014fa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:13.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:13.307+0000 7f0597600640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f0590152820 0x7f059014ff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:13.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:13.307+0000 7f0597600640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f059015c7c0 con 0x7f05900cdf20 2024-09-10T21:36:13.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:13.307+0000 7f0597600640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f05900ce740 con 0x7f0590152440 2024-09-10T21:36:13.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:13.307+0000 7f0597600640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f05900cb450 con 0x7f0590152820 2024-09-10T21:36:13.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:13.307+0000 7f0595600640 1 -- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f0590152820 msgr2=0x7f059014ff90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:3301/0 2024-09-10T21:36:13.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:13.307+0000 7f0595600640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f0590152820 0x7f059014ff90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:36:13.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:13.307+0000 7f0594c00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f05900cdf20 0x7f0590157100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:13.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:13.307+0000 7f0594c00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f05900cdf20 0x7f0590157100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:54156/0 (socket says 172.21.15.17:54156) 2024-09-10T21:36:13.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:13.307+0000 7f0594c00640 1 -- 172.21.15.17:0/4078121633 learned_addr learned my addr 172.21.15.17:0/4078121633 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:36:13.314 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:13.307+0000 7f058fe00640 1 --2- 172.21.15.17:0/4078121633 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0590152440 0x7f059014fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:13.316 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:13.307+0000 7f0594c00640 1 -- 172.21.15.17:0/4078121633 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f0590152820 msgr2=0x7f059014ff90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:36:13.316 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:13.307+0000 7f0594c00640 1 --2- 172.21.15.17:0/4078121633 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f0590152820 0x7f059014ff90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:13.316 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:13.307+0000 7f0594c00640 1 -- 172.21.15.17:0/4078121633 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0590152440 msgr2=0x7f059014fa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:13.316 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:13.307+0000 7f0594c00640 1 --2- 172.21.15.17:0/4078121633 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0590152440 0x7f059014fa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:13.316 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:13.307+0000 7f0594c00640 1 -- 172.21.15.17:0/4078121633 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0590157640 con 0x7f05900cdf20 2024-09-10T21:36:13.316 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:13.307+0000 7f058fe00640 1 --2- 172.21.15.17:0/4078121633 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0590152440 0x7f059014fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:36:13.316 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:13.307+0000 7f0594c00640 1 --2- 172.21.15.17:0/4078121633 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f05900cdf20 0x7f0590157100 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f058400b580 tx=0x7f058400ba40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:13.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:13.307+0000 7f058cc00640 1 -- 172.21.15.17:0/4078121633 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f058400f850 con 0x7f05900cdf20 2024-09-10T21:36:13.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:13.307+0000 7f058cc00640 1 -- 172.21.15.17:0/4078121633 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f058400fb00 con 0x7f05900cdf20 2024-09-10T21:36:13.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:13.307+0000 7f058cc00640 1 -- 172.21.15.17:0/4078121633 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0584017400 con 0x7f05900cdf20 2024-09-10T21:36:13.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:13.307+0000 7f0597600640 1 -- 172.21.15.17:0/4078121633 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05901f72c0 con 0x7f05900cdf20 2024-09-10T21:36:13.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:13.307+0000 7f0597600640 1 -- 172.21.15.17:0/4078121633 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0590151f20 con 0x7f05900cdf20 2024-09-10T21:36:13.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:13.311+0000 7f0597600640 1 --2- 172.21.15.17:0/4078121633 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0590156440 0x7f0590162d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:13.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:13.311+0000 7f0597600640 1 -- 172.21.15.17:0/4078121633 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f0590156820 con 0x7f0590156440 2024-09-10T21:36:13.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:13.311+0000 7f058fe00640 1 --2- 172.21.15.17:0/4078121633 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0590156440 0x7f0590162d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:13.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:13.311+0000 7f058fe00640 1 -- 172.21.15.17:0/4078121633 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f05900c2ff0 con 0x7f0590156440 2024-09-10T21:36:13.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:13.311+0000 7f058fe00640 1 --2- 172.21.15.17:0/4078121633 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0590156440 0x7f0590162d70 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f05900cbaa0 tx=0x7f057c0a61a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:13.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:13.311+0000 7f058cc00640 1 -- 172.21.15.17:0/4078121633 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f057c0bc070 con 0x7f0590156440 2024-09-10T21:36:13.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:13.311+0000 7f058cc00640 1 -- 172.21.15.17:0/4078121633 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f057c0a6510 con 0x7f0590156440 2024-09-10T21:36:13.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:13.311+0000 7f058cc00640 1 -- 172.21.15.17:0/4078121633 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0590156440 msgr2=0x7f0590162d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:13.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:13.311+0000 7f058cc00640 1 --2- 172.21.15.17:0/4078121633 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0590156440 0x7f0590162d70 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f05900cbaa0 tx=0x7f057c0a61a0 comp rx=0 tx=0).stop 2024-09-10T21:36:13.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:13.311+0000 7f058cc00640 1 -- 172.21.15.17:0/4078121633 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0590156440 msgr2=0x7f0590162d70 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:36:13.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:13.311+0000 7f058cc00640 1 --2- 172.21.15.17:0/4078121633 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0590156440 0x7f0590162d70 secure :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f05900cbaa0 tx=0x7f057c0a61a0 comp rx=0 tx=0).stop 2024-09-10T21:36:13.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:13.311+0000 7f058cc00640 1 -- 172.21.15.17:0/4078121633 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f05840203f0 con 0x7f05900cdf20 2024-09-10T21:36:13.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:13.311+0000 7f058cc00640 1 --2- 172.21.15.17:0/4078121633 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f05700720c0 0x7f0570074580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:13.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:13.311+0000 7f058cc00640 1 -- 172.21.15.17:0/4078121633 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f05840936f0 con 0x7f05900cdf20 2024-09-10T21:36:13.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:13.311+0000 7f058fe00640 1 --2- 172.21.15.17:0/4078121633 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f05700720c0 0x7f0570074580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:13.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:13.315+0000 7f058fe00640 1 --2- 172.21.15.17:0/4078121633 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f05700720c0 0x7f0570074580 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f05900cadb0 tx=0x7f057c0b6000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:13.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:13.319+0000 7f0597600640 1 --2- 172.21.15.17:0/4078121633 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f05901632b0 0x7f05901f3060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:13.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:13.319+0000 7f0597600640 1 -- 172.21.15.17:0/4078121633 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f05901f6fd0 con 0x7f05901632b0 2024-09-10T21:36:13.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:13.319+0000 7f0595600640 1 --2- 172.21.15.17:0/4078121633 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f05901632b0 0x7f05901f3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:13.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:13.319+0000 7f0595600640 1 -- 172.21.15.17:0/4078121633 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f05901504d0 con 0x7f05901632b0 2024-09-10T21:36:13.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:13.319+0000 7f0595600640 1 --2- 172.21.15.17:0/4078121633 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f05901632b0 0x7f05901f3060 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f0588001260 tx=0x7f058800cb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:13.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:13.319+0000 7f058cc00640 1 -- 172.21.15.17:0/4078121633 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0588005d50 con 0x7f05901632b0 2024-09-10T21:36:13.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:13.319+0000 7f058cc00640 1 -- 172.21.15.17:0/4078121633 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5033 (secure 0 0 0) 0x7f0588005d50 con 0x7f05901632b0 2024-09-10T21:36:13.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:13.319+0000 7f058cc00640 1 -- 172.21.15.17:0/4078121633 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f05901632b0 msgr2=0x7f05901f3060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:13.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:13.319+0000 7f058cc00640 1 --2- 172.21.15.17:0/4078121633 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f05901632b0 0x7f05901f3060 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f0588001260 tx=0x7f058800cb10 comp rx=0 tx=0).stop 2024-09-10T21:36:13.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:13.319+0000 7f058cc00640 1 -- 172.21.15.17:0/4078121633 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f05901632b0 msgr2=0x7f05901f3060 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:36:13.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:13.319+0000 7f058cc00640 1 --2- 172.21.15.17:0/4078121633 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f05901632b0 0x7f05901f3060 secure :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f0588001260 tx=0x7f058800cb10 comp rx=0 tx=0).stop 2024-09-10T21:36:13.326 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-10T21:36:13.326 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:36:13.327 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:36:13.327 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "leader", 2024-09-10T21:36:13.327 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 96, 2024-09-10T21:36:13.327 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 153926, 2024-09-10T21:36:13.327 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-10T21:36:13.327 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-10T21:36:13.327 INFO:teuthology.orchestra.run.smithi017.stdout: 1 2024-09-10T21:36:13.327 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:36:13.327 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 0, 2024-09-10T21:36:13.327 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:36:13.327 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-10T21:36:13.327 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-10T21:36:13.327 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:36:13.328 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:36:13.328 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:36:13.328 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:36:13.328 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:36:13.328 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:36:13.328 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:36:13.328 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:36:13.328 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:36:13.328 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:36:13.328 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:36:13.328 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:36:13.328 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-10T21:36:13.328 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-10T21:36:13.329 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:36:13.329 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:36:13.329 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:36:13.329 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:36:13.329 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:36:13.329 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:36:13.329 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:36:13.329 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:36:13.329 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:36:13.329 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:36:13.329 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:36:13.329 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:36:13.329 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:36:13.329 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-10T21:36:13.330 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-10T21:36:13.330 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-10T21:36:13.330 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-10T21:36:13.330 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-10T21:36:13.330 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "5e2f023f-d6a6-49f4-a256-f4807dce3739", 2024-09-10T21:36:13.330 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:36:13.330 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:36:13.330 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-10T21:36:13.330 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-10T21:36:13.330 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-10T21:36:13.330 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-10T21:36:13.330 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-10T21:36:13.331 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-10T21:36:13.331 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-10T21:36:13.331 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:36:13.331 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-10T21:36:13.331 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:36:13.331 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:36:13.331 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:36:13.331 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:36:13.331 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:36:13.331 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:36:13.331 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:36:13.331 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:36:13.331 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:36:13.331 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:36:13.331 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:36:13.332 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:36:13.332 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-10T21:36:13.332 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:36:13.332 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-10T21:36:13.332 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:36:13.332 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:36:13.332 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:36:13.332 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:36:13.332 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:36:13.332 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:36:13.332 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:36:13.332 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-10T21:36:13.332 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:36:13.333 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:36:13.333 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:36:13.333 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:36:13.333 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-10T21:36:13.333 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:36:13.333 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:36:13.333 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:36:13.333 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:36:13.333 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-10T21:36:13.333 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-10T21:36:13.333 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:36:13.333 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:36:13.334 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:36:13.334 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:36:13.334 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:36:13.334 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:36:13.334 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:36:13.334 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:36:13.334 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:36:13.334 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:36:13.334 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:36:13.334 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3300", 2024-09-10T21:36:13.334 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:36:13.334 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:36:13.334 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:36:13.334 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:36:13.335 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789", 2024-09-10T21:36:13.335 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:36:13.335 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:36:13.335 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:36:13.335 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:36:13.335 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789/0", 2024-09-10T21:36:13.335 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6789/0", 2024-09-10T21:36:13.335 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:36:13.335 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:36:13.335 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:36:13.335 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:36:13.335 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:36:13.335 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:36:13.335 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:36:13.336 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:36:13.336 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:36:13.336 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:36:13.336 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:36:13.336 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3301", 2024-09-10T21:36:13.336 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:36:13.336 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:36:13.336 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:36:13.336 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:36:13.336 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790", 2024-09-10T21:36:13.336 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:36:13.336 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:36:13.336 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:36:13.337 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:36:13.337 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790/0", 2024-09-10T21:36:13.337 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6790/0", 2024-09-10T21:36:13.337 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:36:13.337 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:36:13.337 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:36:13.337 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:36:13.337 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:36:13.337 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:36:13.337 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-10T21:36:13.337 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-10T21:36:13.338 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:36:13.338 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:36:13.338 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:36:13.338 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:36:13.338 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:36:13.338 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:36:13.338 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": [ 2024-09-10T21:36:13.338 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:36:13.338 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:36:13.338 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:36:13.338 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 6 2024-09-10T21:36:13.338 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:36:13.338 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:36:13.338 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-10T21:36:13.339 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:36:13.339 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:36:13.339 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:36:13.339 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 8 2024-09-10T21:36:13.339 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:36:13.339 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:36:13.339 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-10T21:36:13.339 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:36:13.339 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:36:13.339 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:36:13.339 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 6 2024-09-10T21:36:13.339 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:36:13.339 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:36:13.339 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": [ 2024-09-10T21:36:13.339 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:36:13.340 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:36:13.340 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:36:13.340 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 2 2024-09-10T21:36:13.340 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:36:13.340 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:36:13.340 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:36:13.340 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-10T21:36:13.340 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-10T21:36:13.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:13.319+0000 7f0597600640 1 -- 172.21.15.17:0/4078121633 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f05700720c0 msgr2=0x7f0570074580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:13.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:13.319+0000 7f0597600640 1 --2- 172.21.15.17:0/4078121633 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f05700720c0 0x7f0570074580 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f05900cadb0 tx=0x7f057c0b6000 comp rx=0 tx=0).stop 2024-09-10T21:36:13.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:13.319+0000 7f0597600640 1 -- 172.21.15.17:0/4078121633 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f05900cdf20 msgr2=0x7f0590157100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:13.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:13.319+0000 7f0597600640 1 --2- 172.21.15.17:0/4078121633 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f05900cdf20 0x7f0590157100 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f058400b580 tx=0x7f058400ba40 comp rx=0 tx=0).stop 2024-09-10T21:36:13.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:13.319+0000 7f0595600640 1 -- 172.21.15.17:0/4078121633 reap_dead start 2024-09-10T21:36:13.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:13.319+0000 7f0595600640 1 -- 172.21.15.17:0/4078121633 reap_dead start 2024-09-10T21:36:13.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:13.319+0000 7f0597600640 1 -- 172.21.15.17:0/4078121633 shutdown_connections 2024-09-10T21:36:13.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:13.319+0000 7f0597600640 1 -- 172.21.15.17:0/4078121633 >> 172.21.15.17:0/4078121633 conn(0x7f05900702f0 msgr2=0x7f0590159520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:36:13.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:13.319+0000 7f0597600640 1 -- 172.21.15.17:0/4078121633 shutdown_connections 2024-09-10T21:36:13.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:13.319+0000 7f0597600640 1 -- 172.21.15.17:0/4078121633 wait complete. 2024-09-10T21:36:13.347 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-09-10T21:36:13.347 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon b 2024-09-10T21:36:13.347 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-10T21:36:13.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:13.511+0000 7f6f95400640 1 -- 172.21.15.17:0/1366216124 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6f901546c0 msgr2=0x7f6f900c9c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:13.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:13.511+0000 7f6f95400640 1 --2- 172.21.15.17:0/1366216124 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6f901546c0 0x7f6f900c9c00 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f6f80009900 tx=0x7f6f8001f0f0 comp rx=0 tx=0).stop 2024-09-10T21:36:13.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:13.511+0000 7f6f95400640 1 -- 172.21.15.17:0/1366216124 shutdown_connections 2024-09-10T21:36:13.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:13.511+0000 7f6f95400640 1 --2- 172.21.15.17:0/1366216124 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6f900b3230 0x7f6f900cad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:13.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:13.511+0000 7f6f95400640 1 --2- 172.21.15.17:0/1366216124 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6f901546c0 0x7f6f900c9c00 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:13.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:13.511+0000 7f6f95400640 1 -- 172.21.15.17:0/1366216124 >> 172.21.15.17:0/1366216124 conn(0x7f6f90068b60 msgr2=0x7f6f90068f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:36:13.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:13.511+0000 7f6f95400640 1 -- 172.21.15.17:0/1366216124 shutdown_connections 2024-09-10T21:36:13.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:13.511+0000 7f6f95400640 1 -- 172.21.15.17:0/1366216124 wait complete. 2024-09-10T21:36:13.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:13.511+0000 7f6f95400640 1 Processor -- start 2024-09-10T21:36:13.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:13.511+0000 7f6f95400640 1 -- start start 2024-09-10T21:36:13.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:13.511+0000 7f6f95400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6f900b3230 0x7f6f901f11c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:13.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:13.511+0000 7f6f95400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f900c93d0 0x7f6f900cd790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:13.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:13.511+0000 7f6f95400640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6f901546c0 0x7f6f900ce310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:13.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:13.511+0000 7f6f95400640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f6f900ca760 con 0x7f6f900b3230 2024-09-10T21:36:13.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:13.511+0000 7f6f95400640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f6f9015df50 con 0x7f6f900c93d0 2024-09-10T21:36:13.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:13.511+0000 7f6f95400640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f6f9015dbe0 con 0x7f6f901546c0 2024-09-10T21:36:13.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:13.511+0000 7f6f8e800640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6f900b3230 0x7f6f901f11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:13.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:13.511+0000 7f6f8e800640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6f900b3230 0x7f6f901f11c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:54176/0 (socket says 172.21.15.17:54176) 2024-09-10T21:36:13.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:13.511+0000 7f6f8e800640 1 -- 172.21.15.17:0/2639235158 learned_addr learned my addr 172.21.15.17:0/2639235158 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:36:13.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:13.511+0000 7f6f8f200640 1 -- 172.21.15.17:0/2639235158 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6f901546c0 msgr2=0x7f6f900ce310 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:3301/0 2024-09-10T21:36:13.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:13.511+0000 7f6f8f200640 1 --2- 172.21.15.17:0/2639235158 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6f901546c0 0x7f6f900ce310 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:36:13.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:13.511+0000 7f6f87e00640 1 --2- 172.21.15.17:0/2639235158 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f900c93d0 0x7f6f900cd790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:13.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:13.511+0000 7f6f8e800640 1 -- 172.21.15.17:0/2639235158 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6f901546c0 msgr2=0x7f6f900ce310 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:36:13.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:13.511+0000 7f6f8e800640 1 --2- 172.21.15.17:0/2639235158 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6f901546c0 0x7f6f900ce310 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:13.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:13.511+0000 7f6f8e800640 1 -- 172.21.15.17:0/2639235158 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f900c93d0 msgr2=0x7f6f900cd790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:13.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:13.511+0000 7f6f8e800640 1 --2- 172.21.15.17:0/2639235158 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f900c93d0 0x7f6f900cd790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:13.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:13.515+0000 7f6f8e800640 1 -- 172.21.15.17:0/2639235158 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f900cc060 con 0x7f6f900b3230 2024-09-10T21:36:13.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:13.515+0000 7f6f8e800640 1 --2- 172.21.15.17:0/2639235158 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6f900b3230 0x7f6f901f11c0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f6f800b3040 tx=0x7f6f80003680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:13.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:13.515+0000 7f6f86a00640 1 -- 172.21.15.17:0/2639235158 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f800b5040 con 0x7f6f900b3230 2024-09-10T21:36:13.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:13.515+0000 7f6f86a00640 1 -- 172.21.15.17:0/2639235158 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6f80003b60 con 0x7f6f900b3230 2024-09-10T21:36:13.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:13.515+0000 7f6f86a00640 1 -- 172.21.15.17:0/2639235158 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f800b05f0 con 0x7f6f900b3230 2024-09-10T21:36:13.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:13.515+0000 7f6f95400640 1 -- 172.21.15.17:0/2639235158 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f900cc340 con 0x7f6f900b3230 2024-09-10T21:36:13.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:13.515+0000 7f6f95400640 1 -- 172.21.15.17:0/2639235158 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f900cc900 con 0x7f6f900b3230 2024-09-10T21:36:13.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:13.515+0000 7f6f95400640 1 --2- 172.21.15.17:0/2639235158 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f900ce850 0x7f6f9015cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:13.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:13.515+0000 7f6f95400640 1 -- 172.21.15.17:0/2639235158 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f6f900cdcd0 con 0x7f6f900ce850 2024-09-10T21:36:13.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:13.515+0000 7f6f87e00640 1 --2- 172.21.15.17:0/2639235158 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f900ce850 0x7f6f9015cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:13.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:13.515+0000 7f6f87e00640 1 -- 172.21.15.17:0/2639235158 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6f90154aa0 con 0x7f6f900ce850 2024-09-10T21:36:13.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:13.515+0000 7f6f86a00640 1 -- 172.21.15.17:0/2639235158 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f6f800c3060 con 0x7f6f900b3230 2024-09-10T21:36:13.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:13.515+0000 7f6f87e00640 1 --2- 172.21.15.17:0/2639235158 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f900ce850 0x7f6f9015cab0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f6f90157f40 tx=0x7f6f7800e8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:13.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:13.519+0000 7f6f86a00640 1 --2- 172.21.15.17:0/2639235158 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f6f6c0725c0 0x7f6f6c074a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:13.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:13.519+0000 7f6f86a00640 1 -- 172.21.15.17:0/2639235158 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f6f80130720 con 0x7f6f900b3230 2024-09-10T21:36:13.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:13.519+0000 7f6f86a00640 1 -- 172.21.15.17:0/2639235158 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f78019070 con 0x7f6f900ce850 2024-09-10T21:36:13.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:13.519+0000 7f6f86a00640 1 -- 172.21.15.17:0/2639235158 <== mon.1 v2:172.21.15.31:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f6f780052e0 con 0x7f6f900ce850 2024-09-10T21:36:13.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:13.519+0000 7f6f86a00640 1 -- 172.21.15.17:0/2639235158 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f900ce850 msgr2=0x7f6f9015cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:13.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:13.519+0000 7f6f86a00640 1 --2- 172.21.15.17:0/2639235158 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f900ce850 0x7f6f9015cab0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f6f90157f40 tx=0x7f6f7800e8b0 comp rx=0 tx=0).stop 2024-09-10T21:36:13.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:13.519+0000 7f6f86a00640 1 -- 172.21.15.17:0/2639235158 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f900ce850 msgr2=0x7f6f9015cab0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:36:13.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:13.519+0000 7f6f86a00640 1 --2- 172.21.15.17:0/2639235158 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f900ce850 0x7f6f9015cab0 secure :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f6f90157f40 tx=0x7f6f7800e8b0 comp rx=0 tx=0).stop 2024-09-10T21:36:13.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:13.519+0000 7f6f8f200640 1 --2- 172.21.15.17:0/2639235158 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f6f6c0725c0 0x7f6f6c074a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:13.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:13.519+0000 7f6f8f200640 1 --2- 172.21.15.17:0/2639235158 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f6f6c0725c0 0x7f6f6c074a80 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f6f880013e0 tx=0x7f6f880091f0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:13.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:13.523+0000 7f6f95400640 1 --2- 172.21.15.17:0/2639235158 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f9015cff0 0x7f6f901476e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:13.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:13.523+0000 7f6f95400640 1 -- 172.21.15.17:0/2639235158 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f6f90157f70 con 0x7f6f9015cff0 2024-09-10T21:36:13.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:13.527+0000 7f6f87e00640 1 --2- 172.21.15.17:0/2639235158 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f9015cff0 0x7f6f901476e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:13.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:13.527+0000 7f6f87e00640 1 -- 172.21.15.17:0/2639235158 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f6f90154030 con 0x7f6f9015cff0 2024-09-10T21:36:13.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:13.527+0000 7f6f87e00640 1 --2- 172.21.15.17:0/2639235158 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f9015cff0 0x7f6f901476e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f6f90157f40 tx=0x7f6f78000ee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:13.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:13.527+0000 7f6f86a00640 1 -- 172.21.15.17:0/2639235158 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f78019030 con 0x7f6f9015cff0 2024-09-10T21:36:13.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:13.527+0000 7f6f86a00640 1 -- 172.21.15.17:0/2639235158 <== mon.1 v2:172.21.15.31:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4529 (secure 0 0 0) 0x7f6f78003600 con 0x7f6f9015cff0 2024-09-10T21:36:13.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:13.527+0000 7f6f86a00640 1 -- 172.21.15.17:0/2639235158 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f9015cff0 msgr2=0x7f6f901476e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:13.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:13.527+0000 7f6f86a00640 1 --2- 172.21.15.17:0/2639235158 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f9015cff0 0x7f6f901476e0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f6f90157f40 tx=0x7f6f78000ee0 comp rx=0 tx=0).stop 2024-09-10T21:36:13.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:13.527+0000 7f6f86a00640 1 -- 172.21.15.17:0/2639235158 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f9015cff0 msgr2=0x7f6f901476e0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:36:13.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:13.527+0000 7f6f86a00640 1 --2- 172.21.15.17:0/2639235158 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f9015cff0 0x7f6f901476e0 secure :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f6f90157f40 tx=0x7f6f78000ee0 comp rx=0 tx=0).stop 2024-09-10T21:36:13.534 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-10T21:36:13.534 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:36:13.534 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:36:13.534 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-10T21:36:13.534 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 96, 2024-09-10T21:36:13.534 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 92918, 2024-09-10T21:36:13.534 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-10T21:36:13.535 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-10T21:36:13.535 INFO:teuthology.orchestra.run.smithi017.stdout: 1 2024-09-10T21:36:13.535 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:36:13.535 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 0, 2024-09-10T21:36:13.535 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:36:13.535 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-10T21:36:13.535 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-10T21:36:13.535 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:36:13.535 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:36:13.535 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:36:13.535 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:36:13.535 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:36:13.535 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:36:13.535 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:36:13.536 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:36:13.536 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:36:13.536 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:36:13.536 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:36:13.536 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:36:13.536 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-10T21:36:13.536 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-10T21:36:13.536 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:36:13.536 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:36:13.536 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:36:13.536 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:36:13.536 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:36:13.536 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:36:13.537 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:36:13.537 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:36:13.537 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:36:13.537 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:36:13.537 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:36:13.537 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:36:13.537 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:36:13.537 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-10T21:36:13.537 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-10T21:36:13.537 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-10T21:36:13.537 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-10T21:36:13.537 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-10T21:36:13.537 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "5e2f023f-d6a6-49f4-a256-f4807dce3739", 2024-09-10T21:36:13.538 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:36:13.538 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:36:13.538 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-10T21:36:13.538 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-10T21:36:13.538 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-10T21:36:13.538 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-10T21:36:13.538 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-10T21:36:13.538 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-10T21:36:13.538 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-10T21:36:13.538 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:36:13.538 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-10T21:36:13.538 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:36:13.538 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:36:13.538 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:36:13.539 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:36:13.539 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:36:13.539 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:36:13.539 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:36:13.539 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:36:13.539 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:36:13.539 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:36:13.539 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:36:13.539 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:36:13.539 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-10T21:36:13.539 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:36:13.539 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-10T21:36:13.539 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:36:13.540 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:36:13.540 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:36:13.540 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:36:13.540 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:36:13.540 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:36:13.540 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:36:13.540 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-10T21:36:13.540 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:36:13.540 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:36:13.540 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:36:13.540 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:36:13.540 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-10T21:36:13.541 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:36:13.541 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:36:13.541 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:36:13.541 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:36:13.541 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-10T21:36:13.541 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-10T21:36:13.541 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:36:13.541 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:36:13.541 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:36:13.541 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:36:13.541 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:36:13.541 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:36:13.541 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:36:13.542 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:36:13.542 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:36:13.542 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:36:13.542 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:36:13.542 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3300", 2024-09-10T21:36:13.542 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:36:13.542 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:36:13.542 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:36:13.542 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:36:13.542 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789", 2024-09-10T21:36:13.542 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:36:13.542 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:36:13.542 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:36:13.542 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:36:13.543 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789/0", 2024-09-10T21:36:13.543 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6789/0", 2024-09-10T21:36:13.543 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:36:13.543 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:36:13.543 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:36:13.543 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:36:13.543 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:36:13.543 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:36:13.543 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:36:13.543 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:36:13.543 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:36:13.543 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:36:13.543 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:36:13.543 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3301", 2024-09-10T21:36:13.544 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:36:13.544 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:36:13.544 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:36:13.544 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:36:13.544 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790", 2024-09-10T21:36:13.544 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:36:13.544 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:36:13.544 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:36:13.544 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:36:13.544 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790/0", 2024-09-10T21:36:13.544 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6790/0", 2024-09-10T21:36:13.544 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:36:13.544 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:36:13.544 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:36:13.545 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:36:13.545 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:36:13.545 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:36:13.545 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-10T21:36:13.545 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-10T21:36:13.545 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:36:13.545 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:36:13.545 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:36:13.545 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:36:13.545 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:36:13.545 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:36:13.545 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-10T21:36:13.545 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:36:13.545 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:36:13.546 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:36:13.546 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 2 2024-09-10T21:36:13.546 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:36:13.546 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:36:13.546 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:36:13.546 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-10T21:36:13.546 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-10T21:36:13.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:13.527+0000 7f6f95400640 1 -- 172.21.15.17:0/2639235158 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f6f6c0725c0 msgr2=0x7f6f6c074a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:13.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:13.527+0000 7f6f95400640 1 --2- 172.21.15.17:0/2639235158 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f6f6c0725c0 0x7f6f6c074a80 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f6f880013e0 tx=0x7f6f880091f0 comp rx=0 tx=0).stop 2024-09-10T21:36:13.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:13.527+0000 7f6f95400640 1 -- 172.21.15.17:0/2639235158 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6f900b3230 msgr2=0x7f6f901f11c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:13.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:13.527+0000 7f6f95400640 1 --2- 172.21.15.17:0/2639235158 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6f900b3230 0x7f6f901f11c0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f6f800b3040 tx=0x7f6f80003680 comp rx=0 tx=0).stop 2024-09-10T21:36:13.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:13.527+0000 7f6f8f200640 1 -- 172.21.15.17:0/2639235158 reap_dead start 2024-09-10T21:36:13.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:13.527+0000 7f6f8f200640 1 -- 172.21.15.17:0/2639235158 reap_dead start 2024-09-10T21:36:13.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:13.527+0000 7f6f95400640 1 -- 172.21.15.17:0/2639235158 shutdown_connections 2024-09-10T21:36:13.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:13.527+0000 7f6f95400640 1 -- 172.21.15.17:0/2639235158 >> 172.21.15.17:0/2639235158 conn(0x7f6f90068b60 msgr2=0x7f6f900c69d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:36:13.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:13.527+0000 7f6f95400640 1 -- 172.21.15.17:0/2639235158 shutdown_connections 2024-09-10T21:36:13.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:13.527+0000 7f6f95400640 1 -- 172.21.15.17:0/2639235158 wait complete. 2024-09-10T21:36:13.553 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-09-10T21:36:13.553 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2024-09-10T21:36:19.419 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:36:19.422 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:36:33.554 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.c 2024-09-10T21:36:33.554 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-10T21:36:33.555 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-10T21:36:33.559 INFO:tasks.ceph.mon.c:Started 2024-09-10T21:36:33.559 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-09-10T21:36:33.559 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-10T21:36:33.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:33.723+0000 7fa2dfe00640 1 -- 172.21.15.17:0/3104378440 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa2e015a5a0 msgr2=0x7fa2e00d0000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:33.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:33.723+0000 7fa2dfe00640 1 --2- 172.21.15.17:0/3104378440 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa2e015a5a0 0x7fa2e00d0000 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fa2e0069840 tx=0x7fa2d801f580 comp rx=0 tx=0).stop 2024-09-10T21:36:33.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:33.723+0000 7fa2dfe00640 1 -- 172.21.15.17:0/3104378440 shutdown_connections 2024-09-10T21:36:33.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:33.723+0000 7fa2dfe00640 1 --2- 172.21.15.17:0/3104378440 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa2e015a5a0 0x7fa2e00d0000 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:33.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:33.723+0000 7fa2dfe00640 1 --2- 172.21.15.17:0/3104378440 >> v2:172.21.15.31:3300/0 conn(0x7fa2e00b4580 0x7fa2e00b4960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:33.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:33.723+0000 7fa2dfe00640 1 -- 172.21.15.17:0/3104378440 >> 172.21.15.17:0/3104378440 conn(0x7fa2e006d700 msgr2=0x7fa2e0068990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:36:33.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:33.723+0000 7fa2dfe00640 1 -- 172.21.15.17:0/3104378440 shutdown_connections 2024-09-10T21:36:33.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:33.723+0000 7fa2dfe00640 1 -- 172.21.15.17:0/3104378440 wait complete. 2024-09-10T21:36:33.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:33.723+0000 7fa2dfe00640 1 Processor -- start 2024-09-10T21:36:33.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:33.723+0000 7fa2dfe00640 1 -- start start 2024-09-10T21:36:33.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:33.727+0000 7fa2dfe00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa2e00b4580 0x7fa2e00cf8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:33.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:33.727+0000 7fa2dfe00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa2e00cc5d0 0x7fa2e01e6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:33.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:33.727+0000 7fa2dfe00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa2e015a5a0 0x7fa2e01e67c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:33.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:33.727+0000 7fa2dfe00640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fa2e015e0e0 con 0x7fa2e015a5a0 2024-09-10T21:36:33.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:33.727+0000 7fa2dfe00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fa2e015de20 con 0x7fa2e00cc5d0 2024-09-10T21:36:33.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:33.727+0000 7fa2dfe00640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7fa2e015e3c0 con 0x7fa2e00b4580 2024-09-10T21:36:33.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:33.727+0000 7fa2df400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa2e015a5a0 0x7fa2e01e67c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:33.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:33.727+0000 7fa2de000640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa2e00cc5d0 0x7fa2e01e6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:33.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:33.727+0000 7fa2df400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa2e015a5a0 0x7fa2e01e67c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43114/0 (socket says 172.21.15.17:43114) 2024-09-10T21:36:33.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:33.727+0000 7fa2df400640 1 -- 172.21.15.17:0/1799339391 learned_addr learned my addr 172.21.15.17:0/1799339391 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:36:33.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:33.727+0000 7fa2dea00640 1 --2- 172.21.15.17:0/1799339391 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa2e00b4580 0x7fa2e00cf8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:33.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:33.727+0000 7fa2dea00640 1 -- 172.21.15.17:0/1799339391 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa2e00b4580 msgr2=0x7fa2e00cf8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-09-10T21:36:33.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:33.727+0000 7fa2dea00640 1 -- 172.21.15.17:0/1799339391 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa2e00b4580 msgr2=0x7fa2e00cf8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-10T21:36:33.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:33.727+0000 7fa2dea00640 1 --2- 172.21.15.17:0/1799339391 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa2e00b4580 0x7fa2e00cf8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T21:36:33.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:33.727+0000 7fa2dea00640 1 --2- 172.21.15.17:0/1799339391 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa2e00b4580 0x7fa2e00cf8a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:36:33.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:33.727+0000 7fa2de000640 1 -- 172.21.15.17:0/1799339391 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa2e00b4580 msgr2=0x7fa2e00cf8a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:36:33.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:33.727+0000 7fa2de000640 1 --2- 172.21.15.17:0/1799339391 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa2e00b4580 0x7fa2e00cf8a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:33.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:33.727+0000 7fa2de000640 1 -- 172.21.15.17:0/1799339391 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa2e015a5a0 msgr2=0x7fa2e01e67c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:33.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:33.727+0000 7fa2de000640 1 --2- 172.21.15.17:0/1799339391 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa2e015a5a0 0x7fa2e01e67c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:33.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:33.727+0000 7fa2de000640 1 -- 172.21.15.17:0/1799339391 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2e01e6f20 con 0x7fa2e00cc5d0 2024-09-10T21:36:33.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:33.727+0000 7fa2df400640 1 --2- 172.21.15.17:0/1799339391 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa2e015a5a0 0x7fa2e01e67c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:36:33.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:33.727+0000 7fa2de000640 1 --2- 172.21.15.17:0/1799339391 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa2e00cc5d0 0x7fa2e01e6280 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc00c3c0 tx=0x7fa2cc00c880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:33.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:33.727+0000 7fa2d6a00640 1 -- 172.21.15.17:0/1799339391 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2cc017070 con 0x7fa2e00cc5d0 2024-09-10T21:36:33.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:33.727+0000 7fa2d6a00640 1 -- 172.21.15.17:0/1799339391 <== mon.1 v2:172.21.15.31:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa2cc0053c0 con 0x7fa2e00cc5d0 2024-09-10T21:36:33.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:33.727+0000 7fa2d6a00640 1 -- 172.21.15.17:0/1799339391 <== mon.1 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2cc003840 con 0x7fa2e00cc5d0 2024-09-10T21:36:33.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:33.727+0000 7fa2dfe00640 1 -- 172.21.15.17:0/1799339391 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2e01f0fd0 con 0x7fa2e00cc5d0 2024-09-10T21:36:33.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:33.727+0000 7fa2dfe00640 1 -- 172.21.15.17:0/1799339391 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2e01e7470 con 0x7fa2e00cc5d0 2024-09-10T21:36:33.732 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:33.727+0000 7fa2dfe00640 1 -- 172.21.15.17:0/1799339391 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2e015e980 con 0x7fa2e00cc5d0 2024-09-10T21:36:33.732 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:33.727+0000 7fa2d6a00640 1 -- 172.21.15.17:0/1799339391 <== mon.1 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fa2cc0039e0 con 0x7fa2e00cc5d0 2024-09-10T21:36:33.732 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:33.731+0000 7fa2d6a00640 1 --2- 172.21.15.17:0/1799339391 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fa2c0072610 0x7fa2c0074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:33.732 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:33.731+0000 7fa2d6a00640 1 -- 172.21.15.17:0/1799339391 <== mon.1 v2:172.21.15.31:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fa2cc092bf0 con 0x7fa2e00cc5d0 2024-09-10T21:36:33.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:33.731+0000 7fa2dea00640 1 --2- 172.21.15.17:0/1799339391 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fa2c0072610 0x7fa2c0074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:33.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:33.731+0000 7fa2dea00640 1 --2- 172.21.15.17:0/1799339391 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fa2c0072610 0x7fa2c0074ad0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fa2d00058d0 tx=0x7fa2d0005860 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:33.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:33.735+0000 7fa2d6a00640 1 -- 172.21.15.17:0/1799339391 <== mon.1 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fa2cc05d7e0 con 0x7fa2e00cc5d0 2024-09-10T21:36:33.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:33.995+0000 7fa2dfe00640 1 -- 172.21.15.17:0/1799339391 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fa2e006fa00 con 0x7fa2e00cc5d0 2024-09-10T21:36:33.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:33.995+0000 7fa2d6a00640 1 -- 172.21.15.17:0/1799339391 <== mon.1 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1551 (secure 0 0 0) 0x7fa2cc062210 con 0x7fa2e00cc5d0 2024-09-10T21:36:33.999 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":96,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":21,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5e2f023f-d6a6-49f4-a256-f4807dce3739","modified":"2024-09-10T21:23:46.892130Z","created":"2024-09-10T21:23:46.892130Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3301","nonce":0},{"type":"v1","addr":"172.21.15.31:6790","nonce":0}]},"addr":"172.21.15.31:6790/0","public_addr":"172.21.15.31:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-10T21:36:34.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:33.999+0000 7fa2dfe00640 1 -- 172.21.15.17:0/1799339391 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fa2c0072610 msgr2=0x7fa2c0074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:34.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:33.999+0000 7fa2dfe00640 1 --2- 172.21.15.17:0/1799339391 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fa2c0072610 0x7fa2c0074ad0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fa2d00058d0 tx=0x7fa2d0005860 comp rx=0 tx=0).stop 2024-09-10T21:36:34.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:33.999+0000 7fa2dfe00640 1 -- 172.21.15.17:0/1799339391 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa2e00cc5d0 msgr2=0x7fa2e01e6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:34.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:33.999+0000 7fa2dfe00640 1 --2- 172.21.15.17:0/1799339391 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa2e00cc5d0 0x7fa2e01e6280 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc00c3c0 tx=0x7fa2cc00c880 comp rx=0 tx=0).stop 2024-09-10T21:36:34.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:33.999+0000 7fa2dfe00640 1 -- 172.21.15.17:0/1799339391 shutdown_connections 2024-09-10T21:36:34.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:33.999+0000 7fa2dfe00640 1 --2- 172.21.15.17:0/1799339391 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fa2c0072610 0x7fa2c0074ad0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:34.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:33.999+0000 7fa2dfe00640 1 --2- 172.21.15.17:0/1799339391 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa2e015a5a0 0x7fa2e01e67c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:34.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:33.999+0000 7fa2dfe00640 1 --2- 172.21.15.17:0/1799339391 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa2e00cc5d0 0x7fa2e01e6280 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:34.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:33.999+0000 7fa2dfe00640 1 --2- 172.21.15.17:0/1799339391 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa2e00b4580 0x7fa2e00cf8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:34.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:33.999+0000 7fa2dfe00640 1 -- 172.21.15.17:0/1799339391 >> 172.21.15.17:0/1799339391 conn(0x7fa2e006d700 msgr2=0x7fa2e006ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:36:34.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:33.999+0000 7fa2dfe00640 1 -- 172.21.15.17:0/1799339391 shutdown_connections 2024-09-10T21:36:34.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:33.999+0000 7fa2dfe00640 1 -- 172.21.15.17:0/1799339391 wait complete. 2024-09-10T21:36:37.025 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-10T21:36:37.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:37.183+0000 7fdc86200640 1 -- 172.21.15.17:0/974369997 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdc8006f480 msgr2=0x7fdc800c7870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:37.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:37.183+0000 7fdc86200640 1 --2- 172.21.15.17:0/974369997 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdc8006f480 0x7fdc800c7870 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fdc800ccc00 tx=0x7fdc7401c8c0 comp rx=0 tx=0).stop 2024-09-10T21:36:37.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:37.183+0000 7fdc86200640 1 -- 172.21.15.17:0/974369997 shutdown_connections 2024-09-10T21:36:37.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:37.183+0000 7fdc86200640 1 --2- 172.21.15.17:0/974369997 >> v2:172.21.15.31:3300/0 conn(0x7fdc800c8070 0x7fdc800b2c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:37.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:37.183+0000 7fdc86200640 1 --2- 172.21.15.17:0/974369997 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdc8006f480 0x7fdc800c7870 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:37.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:37.183+0000 7fdc86200640 1 -- 172.21.15.17:0/974369997 >> 172.21.15.17:0/974369997 conn(0x7fdc80070ca0 msgr2=0x7fdc800710b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:36:37.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:37.183+0000 7fdc86200640 1 -- 172.21.15.17:0/974369997 shutdown_connections 2024-09-10T21:36:37.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:37.183+0000 7fdc86200640 1 -- 172.21.15.17:0/974369997 wait complete. 2024-09-10T21:36:37.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:37.187+0000 7fdc86200640 1 Processor -- start 2024-09-10T21:36:37.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:37.187+0000 7fdc86200640 1 -- start start 2024-09-10T21:36:37.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:37.187+0000 7fdc86200640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdc8006f480 0x7fdc800c5080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:37.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:37.187+0000 7fdc86200640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdc8006f860 0x7fdc801f4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:37.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:37.187+0000 7fdc86200640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fdc800c8070 0x7fdc801499d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:37.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:37.187+0000 7fdc86200640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fdc80163e60 con 0x7fdc8006f860 2024-09-10T21:36:37.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:37.187+0000 7fdc86200640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fdc80163ce0 con 0x7fdc8006f480 2024-09-10T21:36:37.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:37.187+0000 7fdc86200640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7fdc80163fe0 con 0x7fdc800c8070 2024-09-10T21:36:37.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:37.187+0000 7fdc7fe00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdc8006f860 0x7fdc801f4ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:37.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:37.187+0000 7fdc7fe00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdc8006f860 0x7fdc801f4ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:60474/0 (socket says 172.21.15.17:60474) 2024-09-10T21:36:37.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:37.187+0000 7fdc7fe00640 1 -- 172.21.15.17:0/2118979304 learned_addr learned my addr 172.21.15.17:0/2118979304 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:36:37.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:37.187+0000 7fdc84e00640 1 --2- 172.21.15.17:0/2118979304 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdc8006f480 0x7fdc800c5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:37.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:37.187+0000 7fdc85800640 1 --2- 172.21.15.17:0/2118979304 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fdc800c8070 0x7fdc801499d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:37.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:37.187+0000 7fdc7fe00640 1 -- 172.21.15.17:0/2118979304 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fdc800c8070 msgr2=0x7fdc801499d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:37.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:37.187+0000 7fdc7fe00640 1 --2- 172.21.15.17:0/2118979304 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fdc800c8070 0x7fdc801499d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:37.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:37.187+0000 7fdc7fe00640 1 -- 172.21.15.17:0/2118979304 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdc8006f480 msgr2=0x7fdc800c5080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:37.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:37.187+0000 7fdc7fe00640 1 --2- 172.21.15.17:0/2118979304 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdc8006f480 0x7fdc800c5080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:37.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:37.187+0000 7fdc7fe00640 1 -- 172.21.15.17:0/2118979304 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc80154ae0 con 0x7fdc8006f860 2024-09-10T21:36:37.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:37.187+0000 7fdc84e00640 1 --2- 172.21.15.17:0/2118979304 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdc8006f480 0x7fdc800c5080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:36:37.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:37.187+0000 7fdc7fe00640 1 --2- 172.21.15.17:0/2118979304 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdc8006f860 0x7fdc801f4ae0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fdc6c00c530 tx=0x7fdc6c00c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:37.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:37.187+0000 7fdc7cc00640 1 -- 172.21.15.17:0/2118979304 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc6c017070 con 0x7fdc8006f860 2024-09-10T21:36:37.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:37.187+0000 7fdc7cc00640 1 -- 172.21.15.17:0/2118979304 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdc6c0053c0 con 0x7fdc8006f860 2024-09-10T21:36:37.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:37.187+0000 7fdc7cc00640 1 -- 172.21.15.17:0/2118979304 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc6c012890 con 0x7fdc8006f860 2024-09-10T21:36:37.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:37.187+0000 7fdc86200640 1 -- 172.21.15.17:0/2118979304 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc80155600 con 0x7fdc8006f860 2024-09-10T21:36:37.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:37.187+0000 7fdc86200640 1 -- 172.21.15.17:0/2118979304 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc8014aa70 con 0x7fdc8006f860 2024-09-10T21:36:37.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:37.191+0000 7fdc86200640 1 -- 172.21.15.17:0/2118979304 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc800c7870 con 0x7fdc8006f860 2024-09-10T21:36:37.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:37.191+0000 7fdc7cc00640 1 -- 172.21.15.17:0/2118979304 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fdc6c003600 con 0x7fdc8006f860 2024-09-10T21:36:37.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:37.191+0000 7fdc7cc00640 1 --2- 172.21.15.17:0/2118979304 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fdc600725e0 0x7fdc60074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:37.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:37.191+0000 7fdc7cc00640 1 -- 172.21.15.17:0/2118979304 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fdc6c092bf0 con 0x7fdc8006f860 2024-09-10T21:36:37.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:37.191+0000 7fdc84e00640 1 --2- 172.21.15.17:0/2118979304 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fdc600725e0 0x7fdc60074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:37.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:37.191+0000 7fdc84e00640 1 --2- 172.21.15.17:0/2118979304 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fdc600725e0 0x7fdc60074aa0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fdc740af000 tx=0x7fdc7401b100 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:37.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:37.199+0000 7fdc7cc00640 1 -- 172.21.15.17:0/2118979304 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fdc6c05dec0 con 0x7fdc8006f860 2024-09-10T21:36:37.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:37.459+0000 7fdc86200640 1 -- 172.21.15.17:0/2118979304 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fdc80071ce0 con 0x7fdc8006f860 2024-09-10T21:36:37.463 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:37.459+0000 7fdc7cc00640 1 -- 172.21.15.17:0/2118979304 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1551 (secure 0 0 0) 0x7fdc6c05d7c0 con 0x7fdc8006f860 2024-09-10T21:36:37.463 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":96,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":24,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5e2f023f-d6a6-49f4-a256-f4807dce3739","modified":"2024-09-10T21:23:46.892130Z","created":"2024-09-10T21:23:46.892130Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3301","nonce":0},{"type":"v1","addr":"172.21.15.31:6790","nonce":0}]},"addr":"172.21.15.31:6790/0","public_addr":"172.21.15.31:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-10T21:36:37.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:37.463+0000 7fdc86200640 1 -- 172.21.15.17:0/2118979304 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fdc600725e0 msgr2=0x7fdc60074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:37.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:37.463+0000 7fdc86200640 1 --2- 172.21.15.17:0/2118979304 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fdc600725e0 0x7fdc60074aa0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fdc740af000 tx=0x7fdc7401b100 comp rx=0 tx=0).stop 2024-09-10T21:36:37.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:37.463+0000 7fdc86200640 1 -- 172.21.15.17:0/2118979304 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdc8006f860 msgr2=0x7fdc801f4ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:37.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:37.463+0000 7fdc86200640 1 --2- 172.21.15.17:0/2118979304 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdc8006f860 0x7fdc801f4ae0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fdc6c00c530 tx=0x7fdc6c00c9f0 comp rx=0 tx=0).stop 2024-09-10T21:36:37.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:37.463+0000 7fdc86200640 1 -- 172.21.15.17:0/2118979304 shutdown_connections 2024-09-10T21:36:37.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:37.463+0000 7fdc86200640 1 --2- 172.21.15.17:0/2118979304 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fdc600725e0 0x7fdc60074aa0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:37.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:37.463+0000 7fdc86200640 1 --2- 172.21.15.17:0/2118979304 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fdc800c8070 0x7fdc801499d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:37.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:37.463+0000 7fdc86200640 1 --2- 172.21.15.17:0/2118979304 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdc8006f860 0x7fdc801f4ae0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:37.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:37.463+0000 7fdc86200640 1 --2- 172.21.15.17:0/2118979304 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdc8006f480 0x7fdc800c5080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:37.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:37.463+0000 7fdc86200640 1 -- 172.21.15.17:0/2118979304 >> 172.21.15.17:0/2118979304 conn(0x7fdc80070ca0 msgr2=0x7fdc800c8d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:36:37.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:37.463+0000 7fdc86200640 1 -- 172.21.15.17:0/2118979304 shutdown_connections 2024-09-10T21:36:37.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:37.463+0000 7fdc86200640 1 -- 172.21.15.17:0/2118979304 wait complete. 2024-09-10T21:36:40.489 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-10T21:36:40.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:40.651+0000 7fe36be00640 1 -- 172.21.15.17:0/1848556074 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe350003c20 msgr2=0x7fe350004020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:40.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:40.651+0000 7fe36be00640 1 --2- 172.21.15.17:0/1848556074 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe350003c20 0x7fe350004020 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fe35c005e60 tx=0x7fe35c01cd40 comp rx=0 tx=0).stop 2024-09-10T21:36:40.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:40.651+0000 7fe36b400640 1 -- 172.21.15.17:0/1848556074 reap_dead start 2024-09-10T21:36:40.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:40.651+0000 7fe36be00640 1 -- 172.21.15.17:0/1848556074 shutdown_connections 2024-09-10T21:36:40.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:40.651+0000 7fe36be00640 1 -- 172.21.15.17:0/1848556074 >> 172.21.15.17:0/1848556074 conn(0x7fe36c0cbbb0 msgr2=0x7fe36c0cbfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:36:40.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:40.651+0000 7fe36be00640 1 -- 172.21.15.17:0/1848556074 shutdown_connections 2024-09-10T21:36:40.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:40.651+0000 7fe36be00640 1 -- 172.21.15.17:0/1848556074 wait complete. 2024-09-10T21:36:40.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:40.651+0000 7fe36be00640 1 Processor -- start 2024-09-10T21:36:40.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:40.651+0000 7fe36be00640 1 -- start start 2024-09-10T21:36:40.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:40.655+0000 7fe36be00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe350003c20 0x7fe36c072dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:40.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:40.655+0000 7fe36be00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe36c0ae8a0 0x7fe36c073720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:40.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:40.655+0000 7fe36be00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe36c073c60 0x7fe36c14c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:40.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:40.655+0000 7fe36be00640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fe36c0c5570 con 0x7fe350003c20 2024-09-10T21:36:40.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:40.655+0000 7fe36be00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fe36c152d00 con 0x7fe36c073c60 2024-09-10T21:36:40.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:40.655+0000 7fe36be00640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7fe36c0c6ba0 con 0x7fe36c0ae8a0 2024-09-10T21:36:40.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:40.655+0000 7fe36aa00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe350003c20 0x7fe36c072dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:40.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:40.655+0000 7fe36aa00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe350003c20 0x7fe36c072dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:60502/0 (socket says 172.21.15.17:60502) 2024-09-10T21:36:40.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:40.655+0000 7fe36aa00640 1 -- 172.21.15.17:0/3644377845 learned_addr learned my addr 172.21.15.17:0/3644377845 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:36:40.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:40.655+0000 7fe363e00640 1 --2- 172.21.15.17:0/3644377845 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe36c0ae8a0 0x7fe36c073720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:40.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:40.655+0000 7fe36aa00640 1 -- 172.21.15.17:0/3644377845 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe36c0ae8a0 msgr2=0x7fe36c073720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:40.658 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:40.655+0000 7fe36aa00640 1 --2- 172.21.15.17:0/3644377845 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe36c0ae8a0 0x7fe36c073720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:40.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:40.655+0000 7fe36aa00640 1 -- 172.21.15.17:0/3644377845 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe36c073c60 msgr2=0x7fe36c14c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:40.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:40.655+0000 7fe36aa00640 1 --2- 172.21.15.17:0/3644377845 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe36c073c60 0x7fe36c14c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:40.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:40.655+0000 7fe36aa00640 1 -- 172.21.15.17:0/3644377845 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe36c0727d0 con 0x7fe350003c20 2024-09-10T21:36:40.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:40.655+0000 7fe36aa00640 1 --2- 172.21.15.17:0/3644377845 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe350003c20 0x7fe36c072dd0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fe35c098650 tx=0x7fe35c0a6f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:40.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:40.655+0000 7fe362a00640 1 -- 172.21.15.17:0/3644377845 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe35c0ab040 con 0x7fe350003c20 2024-09-10T21:36:40.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:40.655+0000 7fe362a00640 1 -- 172.21.15.17:0/3644377845 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe35c0bc070 con 0x7fe350003c20 2024-09-10T21:36:40.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:40.655+0000 7fe362a00640 1 -- 172.21.15.17:0/3644377845 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe35c005050 con 0x7fe350003c20 2024-09-10T21:36:40.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:40.655+0000 7fe36be00640 1 -- 172.21.15.17:0/3644377845 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe36c0afd90 con 0x7fe350003c20 2024-09-10T21:36:40.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:40.655+0000 7fe36be00640 1 -- 172.21.15.17:0/3644377845 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe36c14ccd0 con 0x7fe350003c20 2024-09-10T21:36:40.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:40.659+0000 7fe36be00640 1 -- 172.21.15.17:0/3644377845 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe36c0c74d0 con 0x7fe350003c20 2024-09-10T21:36:40.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:40.663+0000 7fe362a00640 1 -- 172.21.15.17:0/3644377845 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fe35c0051f0 con 0x7fe350003c20 2024-09-10T21:36:40.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:40.663+0000 7fe362a00640 1 --2- 172.21.15.17:0/3644377845 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fe348072610 0x7fe348074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:40.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:40.663+0000 7fe362a00640 1 -- 172.21.15.17:0/3644377845 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fe35c12e4e0 con 0x7fe350003c20 2024-09-10T21:36:40.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:40.663+0000 7fe363e00640 1 --2- 172.21.15.17:0/3644377845 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fe348072610 0x7fe348074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:40.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:40.663+0000 7fe363e00640 1 --2- 172.21.15.17:0/3644377845 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fe348072610 0x7fe348074ad0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fe36c1f1550 tx=0x7fe354006ad0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:40.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:40.667+0000 7fe362a00640 1 -- 172.21.15.17:0/3644377845 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fe35c016a50 con 0x7fe350003c20 2024-09-10T21:36:40.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:40.927+0000 7fe36be00640 1 -- 172.21.15.17:0/3644377845 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fe36c069e60 con 0x7fe350003c20 2024-09-10T21:36:40.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:40.927+0000 7fe362a00640 1 -- 172.21.15.17:0/3644377845 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1556 (secure 0 0 0) 0x7fe35c0f97a0 con 0x7fe350003c20 2024-09-10T21:36:40.929 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":98,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":1,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5e2f023f-d6a6-49f4-a256-f4807dce3739","modified":"2024-09-10T21:23:46.892130Z","created":"2024-09-10T21:23:46.892130Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3301","nonce":0},{"type":"v1","addr":"172.21.15.31:6790","nonce":0}]},"addr":"172.21.15.31:6790/0","public_addr":"172.21.15.31:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-10T21:36:40.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:40.927+0000 7fe36be00640 1 -- 172.21.15.17:0/3644377845 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fe348072610 msgr2=0x7fe348074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:40.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:40.927+0000 7fe36be00640 1 --2- 172.21.15.17:0/3644377845 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fe348072610 0x7fe348074ad0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fe36c1f1550 tx=0x7fe354006ad0 comp rx=0 tx=0).stop 2024-09-10T21:36:40.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:40.927+0000 7fe36be00640 1 -- 172.21.15.17:0/3644377845 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe350003c20 msgr2=0x7fe36c072dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:40.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:40.927+0000 7fe36be00640 1 --2- 172.21.15.17:0/3644377845 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe350003c20 0x7fe36c072dd0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fe35c098650 tx=0x7fe35c0a6f80 comp rx=0 tx=0).stop 2024-09-10T21:36:40.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:40.927+0000 7fe36be00640 1 -- 172.21.15.17:0/3644377845 shutdown_connections 2024-09-10T21:36:40.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:40.927+0000 7fe36be00640 1 --2- 172.21.15.17:0/3644377845 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fe348072610 0x7fe348074ad0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:40.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:40.927+0000 7fe36be00640 1 --2- 172.21.15.17:0/3644377845 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe36c073c60 0x7fe36c14c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:40.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:40.927+0000 7fe36be00640 1 --2- 172.21.15.17:0/3644377845 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe36c0ae8a0 0x7fe36c073720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:40.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:40.927+0000 7fe36be00640 1 --2- 172.21.15.17:0/3644377845 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe350003c20 0x7fe36c072dd0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:40.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:40.927+0000 7fe36be00640 1 -- 172.21.15.17:0/3644377845 >> 172.21.15.17:0/3644377845 conn(0x7fe36c0cbbb0 msgr2=0x7fe36c0745c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:36:40.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:40.931+0000 7fe36be00640 1 -- 172.21.15.17:0/3644377845 shutdown_connections 2024-09-10T21:36:40.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:40.931+0000 7fe36be00640 1 -- 172.21.15.17:0/3644377845 wait complete. 2024-09-10T21:36:40.953 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-09-10T21:36:40.953 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-09-10T21:36:40.953 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-10T21:36:41.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.111+0000 7ff32f600640 1 -- 172.21.15.17:0/696953988 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff32815bf40 msgr2=0x7ff32815c3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:41.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.111+0000 7ff32f600640 1 --2- 172.21.15.17:0/696953988 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff32815bf40 0x7ff32815c3f0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ff320009c80 tx=0x7ff32001cda0 comp rx=0 tx=0).stop 2024-09-10T21:36:41.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.111+0000 7ff32f600640 1 -- 172.21.15.17:0/696953988 shutdown_connections 2024-09-10T21:36:41.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.111+0000 7ff32f600640 1 --2- 172.21.15.17:0/696953988 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff32815bf40 0x7ff32815c3f0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:41.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.111+0000 7ff32f600640 1 --2- 172.21.15.17:0/696953988 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff328074f70 0x7ff3280c7130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:41.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.111+0000 7ff32f600640 1 -- 172.21.15.17:0/696953988 >> 172.21.15.17:0/696953988 conn(0x7ff32806dd30 msgr2=0x7ff32806e140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:36:41.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.111+0000 7ff32f600640 1 -- 172.21.15.17:0/696953988 shutdown_connections 2024-09-10T21:36:41.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.111+0000 7ff32f600640 1 -- 172.21.15.17:0/696953988 wait complete. 2024-09-10T21:36:41.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.111+0000 7ff32f600640 1 Processor -- start 2024-09-10T21:36:41.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.111+0000 7ff32f600640 1 -- start start 2024-09-10T21:36:41.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.115+0000 7ff32f600640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff328070450 0x7ff3281f78b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:41.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.115+0000 7ff32f600640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff328074f70 0x7ff3280aad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:41.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.115+0000 7ff32f600640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff32815bf40 0x7ff32806f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:41.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.115+0000 7ff32f600640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7ff328069b90 con 0x7ff328070450 2024-09-10T21:36:41.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.115+0000 7ff32f600640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7ff3281592e0 con 0x7ff32815bf40 2024-09-10T21:36:41.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.115+0000 7ff32f600640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7ff32806ef60 con 0x7ff328074f70 2024-09-10T21:36:41.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.115+0000 7ff32cc00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff328070450 0x7ff3281f78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:41.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.115+0000 7ff32cc00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff328070450 0x7ff3281f78b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:60516/0 (socket says 172.21.15.17:60516) 2024-09-10T21:36:41.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.115+0000 7ff32cc00640 1 -- 172.21.15.17:0/1996546748 learned_addr learned my addr 172.21.15.17:0/1996546748 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:36:41.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.115+0000 7ff327e00640 1 --2- 172.21.15.17:0/1996546748 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff328074f70 0x7ff3280aad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:41.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.115+0000 7ff32d600640 1 --2- 172.21.15.17:0/1996546748 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff32815bf40 0x7ff32806f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:41.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.115+0000 7ff327e00640 1 -- 172.21.15.17:0/1996546748 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff328074f70 msgr2=0x7ff3280aad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-09-10T21:36:41.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.115+0000 7ff327e00640 1 -- 172.21.15.17:0/1996546748 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff328074f70 msgr2=0x7ff3280aad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-10T21:36:41.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.115+0000 7ff327e00640 1 --2- 172.21.15.17:0/1996546748 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff328074f70 0x7ff3280aad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T21:36:41.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.115+0000 7ff327e00640 1 --2- 172.21.15.17:0/1996546748 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff328074f70 0x7ff3280aad30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:36:41.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.115+0000 7ff32cc00640 1 -- 172.21.15.17:0/1996546748 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff328074f70 msgr2=0x7ff3280aad30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:36:41.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.115+0000 7ff32cc00640 1 --2- 172.21.15.17:0/1996546748 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff328074f70 0x7ff3280aad30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:41.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.115+0000 7ff32cc00640 1 -- 172.21.15.17:0/1996546748 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff32815bf40 msgr2=0x7ff32806f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:41.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.115+0000 7ff32cc00640 1 --2- 172.21.15.17:0/1996546748 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff32815bf40 0x7ff32806f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:41.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.115+0000 7ff32cc00640 1 -- 172.21.15.17:0/1996546748 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff32815adf0 con 0x7ff328070450 2024-09-10T21:36:41.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.115+0000 7ff32cc00640 1 --2- 172.21.15.17:0/1996546748 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff328070450 0x7ff3281f78b0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ff31c005d10 tx=0x7ff31c00dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:41.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.115+0000 7ff324c00640 1 -- 172.21.15.17:0/1996546748 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff31c017070 con 0x7ff328070450 2024-09-10T21:36:41.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.115+0000 7ff324c00640 1 -- 172.21.15.17:0/1996546748 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff31c009d20 con 0x7ff328070450 2024-09-10T21:36:41.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.115+0000 7ff324c00640 1 -- 172.21.15.17:0/1996546748 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff31c005280 con 0x7ff328070450 2024-09-10T21:36:41.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.115+0000 7ff32f600640 1 -- 172.21.15.17:0/1996546748 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff32815ab10 con 0x7ff328070450 2024-09-10T21:36:41.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.115+0000 7ff32f600640 1 -- 172.21.15.17:0/1996546748 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff328161270 con 0x7ff328070450 2024-09-10T21:36:41.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.115+0000 7ff32f600640 1 --2- 172.21.15.17:0/1996546748 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff328161d50 0x7ff3281f3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:41.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.115+0000 7ff32f600640 1 -- 172.21.15.17:0/1996546748 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7ff3280c7130 con 0x7ff328161d50 2024-09-10T21:36:41.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.115+0000 7ff327e00640 1 --2- 172.21.15.17:0/1996546748 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff328161d50 0x7ff3281f3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:41.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.119+0000 7ff327e00640 1 -- 172.21.15.17:0/1996546748 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff328069e60 con 0x7ff328161d50 2024-09-10T21:36:41.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.119+0000 7ff327e00640 1 --2- 172.21.15.17:0/1996546748 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff328161d50 0x7ff3281f3940 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7ff328050fa0 tx=0x7ff31400e890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:41.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.119+0000 7ff324c00640 1 -- 172.21.15.17:0/1996546748 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff314019070 con 0x7ff328161d50 2024-09-10T21:36:41.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.119+0000 7ff324c00640 1 -- 172.21.15.17:0/1996546748 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7ff3140052e0 con 0x7ff328161d50 2024-09-10T21:36:41.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.119+0000 7ff324c00640 1 -- 172.21.15.17:0/1996546748 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff328161d50 msgr2=0x7ff3281f3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:41.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.119+0000 7ff324c00640 1 --2- 172.21.15.17:0/1996546748 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff328161d50 0x7ff3281f3940 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7ff328050fa0 tx=0x7ff31400e890 comp rx=0 tx=0).stop 2024-09-10T21:36:41.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.119+0000 7ff324c00640 1 -- 172.21.15.17:0/1996546748 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff328161d50 msgr2=0x7ff3281f3940 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:36:41.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.119+0000 7ff324c00640 1 --2- 172.21.15.17:0/1996546748 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff328161d50 0x7ff3281f3940 secure :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7ff328050fa0 tx=0x7ff31400e890 comp rx=0 tx=0).stop 2024-09-10T21:36:41.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.119+0000 7ff324c00640 1 -- 172.21.15.17:0/1996546748 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7ff31c005420 con 0x7ff328070450 2024-09-10T21:36:41.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.119+0000 7ff324c00640 1 --2- 172.21.15.17:0/1996546748 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7ff3080720c0 0x7ff308074580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:41.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.119+0000 7ff324c00640 1 -- 172.21.15.17:0/1996546748 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7ff31c093640 con 0x7ff328070450 2024-09-10T21:36:41.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.119+0000 7ff327e00640 1 --2- 172.21.15.17:0/1996546748 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7ff3080720c0 0x7ff308074580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:41.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.119+0000 7ff327e00640 1 --2- 172.21.15.17:0/1996546748 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7ff3080720c0 0x7ff308074580 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7ff314016000 tx=0x7ff314005000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:41.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.127+0000 7ff32f600640 1 --2- 172.21.15.17:0/1996546748 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff328162130 0x7ff3281f3e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:41.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.127+0000 7ff32f600640 1 -- 172.21.15.17:0/1996546748 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7ff328070960 con 0x7ff328162130 2024-09-10T21:36:41.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.127+0000 7ff32d600640 1 --2- 172.21.15.17:0/1996546748 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff328162130 0x7ff3281f3e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:41.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.127+0000 7ff32d600640 1 -- 172.21.15.17:0/1996546748 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7ff3280cc460 con 0x7ff328162130 2024-09-10T21:36:41.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.127+0000 7ff32d600640 1 --2- 172.21.15.17:0/1996546748 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff328162130 0x7ff3281f3e80 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7ff32806f3f0 tx=0x7ff3200099d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:41.131 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.127+0000 7ff324c00640 1 -- 172.21.15.17:0/1996546748 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3200b2040 con 0x7ff328162130 2024-09-10T21:36:41.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.127+0000 7ff324c00640 1 -- 172.21.15.17:0/1996546748 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5044 (secure 0 0 0) 0x7ff320004040 con 0x7ff328162130 2024-09-10T21:36:41.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.127+0000 7ff324c00640 1 -- 172.21.15.17:0/1996546748 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff328162130 msgr2=0x7ff3281f3e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:41.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.127+0000 7ff324c00640 1 --2- 172.21.15.17:0/1996546748 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff328162130 0x7ff3281f3e80 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7ff32806f3f0 tx=0x7ff3200099d0 comp rx=0 tx=0).stop 2024-09-10T21:36:41.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.127+0000 7ff324c00640 1 -- 172.21.15.17:0/1996546748 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff328162130 msgr2=0x7ff3281f3e80 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:36:41.132 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.127+0000 7ff324c00640 1 --2- 172.21.15.17:0/1996546748 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff328162130 0x7ff3281f3e80 secure :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7ff32806f3f0 tx=0x7ff3200099d0 comp rx=0 tx=0).stop 2024-09-10T21:36:41.132 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-10T21:36:41.132 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:36:41.132 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:36:41.133 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "leader", 2024-09-10T21:36:41.133 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 98, 2024-09-10T21:36:41.133 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 181734, 2024-09-10T21:36:41.133 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-10T21:36:41.133 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-10T21:36:41.133 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-10T21:36:41.133 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-10T21:36:41.133 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:36:41.133 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 1, 2024-09-10T21:36:41.133 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:36:41.133 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-10T21:36:41.133 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-10T21:36:41.133 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:36:41.134 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:36:41.134 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:36:41.134 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:36:41.134 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:36:41.134 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:36:41.134 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:36:41.134 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:36:41.134 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:36:41.134 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:36:41.134 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:36:41.134 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:36:41.134 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-10T21:36:41.134 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-10T21:36:41.135 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:36:41.135 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:36:41.135 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:36:41.135 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:36:41.135 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:36:41.135 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:36:41.135 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:36:41.136 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:36:41.136 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:36:41.136 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:36:41.136 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:36:41.136 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:36:41.136 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:36:41.136 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-10T21:36:41.136 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-10T21:36:41.136 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-10T21:36:41.136 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-10T21:36:41.136 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-10T21:36:41.136 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "5e2f023f-d6a6-49f4-a256-f4807dce3739", 2024-09-10T21:36:41.136 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:36:41.137 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:36:41.137 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-10T21:36:41.137 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-10T21:36:41.137 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-10T21:36:41.137 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-10T21:36:41.137 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-10T21:36:41.137 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-10T21:36:41.137 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-10T21:36:41.137 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:36:41.137 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-10T21:36:41.137 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:36:41.137 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:36:41.137 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:36:41.137 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:36:41.138 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:36:41.138 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:36:41.138 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:36:41.138 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:36:41.138 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:36:41.138 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:36:41.138 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:36:41.138 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:36:41.138 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-10T21:36:41.138 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:36:41.138 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-10T21:36:41.138 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:36:41.138 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:36:41.139 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:36:41.139 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:36:41.139 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:36:41.139 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:36:41.139 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:36:41.139 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-10T21:36:41.139 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:36:41.139 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:36:41.139 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:36:41.139 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:36:41.139 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-10T21:36:41.139 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:36:41.139 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:36:41.139 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:36:41.140 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:36:41.140 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-10T21:36:41.140 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-10T21:36:41.140 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:36:41.140 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:36:41.140 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:36:41.140 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:36:41.140 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:36:41.140 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:36:41.140 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:36:41.140 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:36:41.140 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:36:41.140 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:36:41.140 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:36:41.141 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3300", 2024-09-10T21:36:41.141 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:36:41.141 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:36:41.141 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:36:41.141 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:36:41.141 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789", 2024-09-10T21:36:41.141 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:36:41.141 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:36:41.141 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:36:41.141 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:36:41.141 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789/0", 2024-09-10T21:36:41.141 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6789/0", 2024-09-10T21:36:41.141 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:36:41.141 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:36:41.142 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:36:41.142 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:36:41.142 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:36:41.142 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:36:41.142 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:36:41.142 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:36:41.142 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:36:41.142 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:36:41.142 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:36:41.142 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3301", 2024-09-10T21:36:41.142 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:36:41.142 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:36:41.142 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:36:41.142 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:36:41.143 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790", 2024-09-10T21:36:41.143 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:36:41.143 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:36:41.143 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:36:41.143 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:36:41.143 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790/0", 2024-09-10T21:36:41.143 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6790/0", 2024-09-10T21:36:41.143 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:36:41.143 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:36:41.143 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:36:41.143 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:36:41.143 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:36:41.143 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:36:41.143 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-10T21:36:41.144 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-10T21:36:41.144 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:36:41.144 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:36:41.144 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:36:41.144 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:36:41.144 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:36:41.144 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:36:41.144 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": [ 2024-09-10T21:36:41.144 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:36:41.144 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:36:41.144 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:36:41.144 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 6 2024-09-10T21:36:41.144 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:36:41.145 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:36:41.145 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-10T21:36:41.145 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:36:41.145 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:36:41.145 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:36:41.145 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 8 2024-09-10T21:36:41.145 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:36:41.145 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:36:41.145 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-10T21:36:41.145 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:36:41.145 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:36:41.145 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:36:41.145 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 6 2024-09-10T21:36:41.145 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:36:41.146 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:36:41.146 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": [ 2024-09-10T21:36:41.146 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:36:41.146 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:36:41.146 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:36:41.146 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 2 2024-09-10T21:36:41.146 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:36:41.146 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:36:41.146 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:36:41.146 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-10T21:36:41.146 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-10T21:36:41.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.131+0000 7ff32f600640 1 -- 172.21.15.17:0/1996546748 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7ff3080720c0 msgr2=0x7ff308074580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:41.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.131+0000 7ff32f600640 1 --2- 172.21.15.17:0/1996546748 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7ff3080720c0 0x7ff308074580 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7ff314016000 tx=0x7ff314005000 comp rx=0 tx=0).stop 2024-09-10T21:36:41.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.131+0000 7ff32f600640 1 -- 172.21.15.17:0/1996546748 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff328070450 msgr2=0x7ff3281f78b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:41.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.131+0000 7ff32f600640 1 --2- 172.21.15.17:0/1996546748 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff328070450 0x7ff3281f78b0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ff31c005d10 tx=0x7ff31c00dde0 comp rx=0 tx=0).stop 2024-09-10T21:36:41.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.131+0000 7ff32d600640 1 -- 172.21.15.17:0/1996546748 reap_dead start 2024-09-10T21:36:41.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.131+0000 7ff32f600640 1 -- 172.21.15.17:0/1996546748 shutdown_connections 2024-09-10T21:36:41.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.131+0000 7ff32f600640 1 --2- 172.21.15.17:0/1996546748 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff328070450 0x7ff3281f78b0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:41.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.131+0000 7ff32f600640 1 -- 172.21.15.17:0/1996546748 >> 172.21.15.17:0/1996546748 conn(0x7ff32806dd30 msgr2=0x7ff32806e110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:36:41.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.131+0000 7ff32f600640 1 -- 172.21.15.17:0/1996546748 shutdown_connections 2024-09-10T21:36:41.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.131+0000 7ff32f600640 1 -- 172.21.15.17:0/1996546748 wait complete. 2024-09-10T21:36:41.156 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-10T21:36:41.156 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-09-10T21:36:41.156 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-10T21:36:41.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.331+0000 7f5aa3200640 1 -- 172.21.15.17:0/1381104355 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a880051d0 msgr2=0x7f5a8800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:41.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.331+0000 7f5aa3200640 1 --2- 172.21.15.17:0/1381104355 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a880051d0 0x7f5a8800b9d0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f5a9c00c630 tx=0x7f5a9c01d2a0 comp rx=0 tx=0).stop 2024-09-10T21:36:41.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.331+0000 7f5aa3200640 1 -- 172.21.15.17:0/1381104355 shutdown_connections 2024-09-10T21:36:41.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.331+0000 7f5aa3200640 1 --2- 172.21.15.17:0/1381104355 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a880051d0 0x7f5a8800b9d0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:41.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.331+0000 7f5aa3200640 1 -- 172.21.15.17:0/1381104355 >> 172.21.15.17:0/1381104355 conn(0x7f5aa406cc20 msgr2=0x7f5aa406d030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:36:41.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.331+0000 7f5aa3200640 1 -- 172.21.15.17:0/1381104355 shutdown_connections 2024-09-10T21:36:41.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.331+0000 7f5aa3200640 1 -- 172.21.15.17:0/1381104355 wait complete. 2024-09-10T21:36:41.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.331+0000 7f5aa3200640 1 Processor -- start 2024-09-10T21:36:41.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.331+0000 7f5aa3200640 1 -- start start 2024-09-10T21:36:41.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.335+0000 7f5aa3200640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a880051d0 0x7f5aa41e0690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:41.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.335+0000 7f5aa3200640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5aa41e0f70 0x7f5aa40c8a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:41.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.335+0000 7f5aa3200640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f5aa40c8f50 0x7f5aa41de990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:41.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.335+0000 7f5aa3200640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f5aa40c5090 con 0x7f5a880051d0 2024-09-10T21:36:41.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.335+0000 7f5aa3200640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f5aa40c2f10 con 0x7f5aa41e0f70 2024-09-10T21:36:41.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.335+0000 7f5aa3200640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f5aa40c3230 con 0x7f5aa40c8f50 2024-09-10T21:36:41.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.335+0000 7f5aa1e00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a880051d0 0x7f5aa41e0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:41.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.335+0000 7f5aa1e00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a880051d0 0x7f5aa41e0690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:60528/0 (socket says 172.21.15.17:60528) 2024-09-10T21:36:41.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.335+0000 7f5aa1e00640 1 -- 172.21.15.17:0/3705979903 learned_addr learned my addr 172.21.15.17:0/3705979903 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:36:41.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.335+0000 7f5aa1400640 1 --2- 172.21.15.17:0/3705979903 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5aa41e0f70 0x7f5aa40c8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:41.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.335+0000 7f5aa2800640 1 --2- 172.21.15.17:0/3705979903 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f5aa40c8f50 0x7f5aa41de990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:41.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.335+0000 7f5aa1e00640 1 -- 172.21.15.17:0/3705979903 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f5aa40c8f50 msgr2=0x7f5aa41de990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:41.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.335+0000 7f5aa1e00640 1 --2- 172.21.15.17:0/3705979903 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f5aa40c8f50 0x7f5aa41de990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:41.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.335+0000 7f5aa1e00640 1 -- 172.21.15.17:0/3705979903 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5aa41e0f70 msgr2=0x7f5aa40c8a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:41.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.335+0000 7f5aa1e00640 1 --2- 172.21.15.17:0/3705979903 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5aa41e0f70 0x7f5aa40c8a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:41.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.335+0000 7f5aa1e00640 1 -- 172.21.15.17:0/3705979903 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5aa40c94e0 con 0x7f5a880051d0 2024-09-10T21:36:41.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.335+0000 7f5aa2800640 1 --2- 172.21.15.17:0/3705979903 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f5aa40c8f50 0x7f5aa41de990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:36:41.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.335+0000 7f5aa1400640 1 --2- 172.21.15.17:0/3705979903 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5aa41e0f70 0x7f5aa40c8a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:36:41.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.335+0000 7f5aa1e00640 1 --2- 172.21.15.17:0/3705979903 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a880051d0 0x7f5aa41e0690 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f5a94011ab0 tx=0x7f5a94011f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:41.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.335+0000 7f5a9a000640 1 -- 172.21.15.17:0/3705979903 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a9401e070 con 0x7f5a880051d0 2024-09-10T21:36:41.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.335+0000 7f5a9a000640 1 -- 172.21.15.17:0/3705979903 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5a9400fdf0 con 0x7f5a880051d0 2024-09-10T21:36:41.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.335+0000 7f5a9a000640 1 -- 172.21.15.17:0/3705979903 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a94019670 con 0x7f5a880051d0 2024-09-10T21:36:41.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.335+0000 7f5aa3200640 1 -- 172.21.15.17:0/3705979903 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5aa41517e0 con 0x7f5a880051d0 2024-09-10T21:36:41.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.335+0000 7f5aa3200640 1 -- 172.21.15.17:0/3705979903 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5aa40c97d0 con 0x7f5a880051d0 2024-09-10T21:36:41.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.335+0000 7f5aa3200640 1 --2- 172.21.15.17:0/3705979903 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5aa41de3d0 0x7f5aa41deed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:41.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.335+0000 7f5aa3200640 1 -- 172.21.15.17:0/3705979903 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f5aa41e0bd0 con 0x7f5aa41de3d0 2024-09-10T21:36:41.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.335+0000 7f5aa1400640 1 --2- 172.21.15.17:0/3705979903 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5aa41de3d0 0x7f5aa41deed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:41.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.339+0000 7f5aa1400640 1 -- 172.21.15.17:0/3705979903 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5aa40cbd10 con 0x7f5aa41de3d0 2024-09-10T21:36:41.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.339+0000 7f5aa1400640 1 --2- 172.21.15.17:0/3705979903 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5aa41de3d0 0x7f5aa41deed0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f5aa4158050 tx=0x7f5a90001640 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:41.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.339+0000 7f5a9a000640 1 -- 172.21.15.17:0/3705979903 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a90016070 con 0x7f5aa41de3d0 2024-09-10T21:36:41.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.339+0000 7f5a9a000640 1 -- 172.21.15.17:0/3705979903 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f5a94019810 con 0x7f5a880051d0 2024-09-10T21:36:41.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.339+0000 7f5a9a000640 1 --2- 172.21.15.17:0/3705979903 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f5a84072510 0x7f5a840749d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:41.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.339+0000 7f5a9a000640 1 -- 172.21.15.17:0/3705979903 <== mon.1 v2:172.21.15.31:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f5a90012850 con 0x7f5aa41de3d0 2024-09-10T21:36:41.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.339+0000 7f5a9a000640 1 -- 172.21.15.17:0/3705979903 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5aa41de3d0 msgr2=0x7f5aa41deed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:41.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.339+0000 7f5a9a000640 1 --2- 172.21.15.17:0/3705979903 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5aa41de3d0 0x7f5aa41deed0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f5aa4158050 tx=0x7f5a90001640 comp rx=0 tx=0).stop 2024-09-10T21:36:41.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.339+0000 7f5a9a000640 1 -- 172.21.15.17:0/3705979903 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5aa41de3d0 msgr2=0x7f5aa41deed0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:36:41.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.339+0000 7f5a9a000640 1 --2- 172.21.15.17:0/3705979903 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5aa41de3d0 0x7f5aa41deed0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:41.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.339+0000 7f5a9a000640 1 -- 172.21.15.17:0/3705979903 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f5a94094310 con 0x7f5a880051d0 2024-09-10T21:36:41.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.339+0000 7f5aa2800640 1 --2- 172.21.15.17:0/3705979903 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f5a84072510 0x7f5a840749d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:41.343 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.339+0000 7f5aa2800640 1 --2- 172.21.15.17:0/3705979903 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f5a84072510 0x7f5a840749d0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f5a8800c120 tx=0x7f5a9c01a0e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:41.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.347+0000 7f5aa3200640 1 --2- 172.21.15.17:0/3705979903 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5aa41eee30 0x7f5aa41456b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:41.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.347+0000 7f5aa3200640 1 -- 172.21.15.17:0/3705979903 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f5aa4069ef0 con 0x7f5aa41eee30 2024-09-10T21:36:41.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.347+0000 7f5aa1400640 1 --2- 172.21.15.17:0/3705979903 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5aa41eee30 0x7f5aa41456b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:41.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.347+0000 7f5aa1400640 1 -- 172.21.15.17:0/3705979903 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f5aa40cc710 con 0x7f5aa41eee30 2024-09-10T21:36:41.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.347+0000 7f5aa1400640 1 --2- 172.21.15.17:0/3705979903 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5aa41eee30 0x7f5aa41456b0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f5aa40c8740 tx=0x7f5a90001130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:41.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.351+0000 7f5a9a000640 1 -- 172.21.15.17:0/3705979903 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a9001d2d0 con 0x7f5aa41eee30 2024-09-10T21:36:41.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.351+0000 7f5a9a000640 1 -- 172.21.15.17:0/3705979903 <== mon.1 v2:172.21.15.31:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4541 (secure 0 0 0) 0x7f5a9001de40 con 0x7f5aa41eee30 2024-09-10T21:36:41.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.351+0000 7f5a9a000640 1 -- 172.21.15.17:0/3705979903 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5aa41eee30 msgr2=0x7f5aa41456b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:41.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.351+0000 7f5a9a000640 1 --2- 172.21.15.17:0/3705979903 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5aa41eee30 0x7f5aa41456b0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f5aa40c8740 tx=0x7f5a90001130 comp rx=0 tx=0).stop 2024-09-10T21:36:41.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.351+0000 7f5a9a000640 1 -- 172.21.15.17:0/3705979903 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5aa41eee30 msgr2=0x7f5aa41456b0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:36:41.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.351+0000 7f5a9a000640 1 --2- 172.21.15.17:0/3705979903 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5aa41eee30 0x7f5aa41456b0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:41.353 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-10T21:36:41.353 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:36:41.353 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:36:41.353 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-10T21:36:41.353 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 98, 2024-09-10T21:36:41.353 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 120738, 2024-09-10T21:36:41.353 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-10T21:36:41.353 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-10T21:36:41.353 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-10T21:36:41.354 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-10T21:36:41.354 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:36:41.354 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 2, 2024-09-10T21:36:41.354 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:36:41.354 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-10T21:36:41.354 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-10T21:36:41.354 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:36:41.354 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:36:41.354 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:36:41.354 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:36:41.354 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:36:41.354 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:36:41.354 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:36:41.355 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:36:41.355 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:36:41.355 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:36:41.355 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:36:41.355 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:36:41.355 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-10T21:36:41.355 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-10T21:36:41.355 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:36:41.355 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:36:41.355 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:36:41.355 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:36:41.355 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:36:41.355 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:36:41.356 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:36:41.356 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:36:41.356 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:36:41.356 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:36:41.356 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:36:41.356 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:36:41.356 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:36:41.356 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-10T21:36:41.356 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-10T21:36:41.356 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-10T21:36:41.356 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-10T21:36:41.356 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-10T21:36:41.356 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "5e2f023f-d6a6-49f4-a256-f4807dce3739", 2024-09-10T21:36:41.357 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:36:41.357 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:36:41.357 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-10T21:36:41.357 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-10T21:36:41.357 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-10T21:36:41.357 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-10T21:36:41.357 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-10T21:36:41.357 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-10T21:36:41.357 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-10T21:36:41.357 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:36:41.357 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-10T21:36:41.357 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:36:41.357 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:36:41.357 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:36:41.358 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:36:41.358 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:36:41.358 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:36:41.358 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:36:41.358 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:36:41.358 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:36:41.358 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:36:41.358 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:36:41.358 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:36:41.358 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-10T21:36:41.358 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:36:41.358 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-10T21:36:41.358 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:36:41.359 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:36:41.359 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:36:41.359 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:36:41.359 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:36:41.359 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:36:41.359 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:36:41.359 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-10T21:36:41.359 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:36:41.359 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:36:41.359 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:36:41.359 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:36:41.359 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-10T21:36:41.359 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:36:41.360 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:36:41.360 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:36:41.360 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:36:41.360 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-10T21:36:41.360 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-10T21:36:41.360 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:36:41.360 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:36:41.360 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:36:41.360 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:36:41.360 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:36:41.360 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:36:41.360 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:36:41.360 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:36:41.361 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:36:41.361 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:36:41.361 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:36:41.361 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3300", 2024-09-10T21:36:41.361 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:36:41.361 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:36:41.361 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:36:41.361 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:36:41.361 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789", 2024-09-10T21:36:41.361 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:36:41.361 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:36:41.361 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:36:41.361 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:36:41.361 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789/0", 2024-09-10T21:36:41.362 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6789/0", 2024-09-10T21:36:41.362 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:36:41.362 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:36:41.362 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:36:41.362 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:36:41.362 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:36:41.362 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:36:41.362 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:36:41.362 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:36:41.362 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:36:41.362 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:36:41.362 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:36:41.362 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3301", 2024-09-10T21:36:41.362 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:36:41.363 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:36:41.363 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:36:41.363 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:36:41.363 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790", 2024-09-10T21:36:41.363 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:36:41.363 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:36:41.363 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:36:41.363 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:36:41.363 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790/0", 2024-09-10T21:36:41.363 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6790/0", 2024-09-10T21:36:41.363 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:36:41.363 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:36:41.363 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:36:41.363 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:36:41.364 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:36:41.364 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:36:41.364 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-10T21:36:41.364 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-10T21:36:41.364 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:36:41.364 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:36:41.364 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:36:41.364 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:36:41.364 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:36:41.364 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:36:41.364 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-10T21:36:41.364 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:36:41.364 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:36:41.365 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:36:41.365 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 2 2024-09-10T21:36:41.365 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:36:41.365 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:36:41.365 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:36:41.365 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-10T21:36:41.365 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-10T21:36:41.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.351+0000 7f5aa3200640 1 -- 172.21.15.17:0/3705979903 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f5a84072510 msgr2=0x7f5a840749d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:41.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.351+0000 7f5aa3200640 1 --2- 172.21.15.17:0/3705979903 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f5a84072510 0x7f5a840749d0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f5a8800c120 tx=0x7f5a9c01a0e0 comp rx=0 tx=0).stop 2024-09-10T21:36:41.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.351+0000 7f5aa3200640 1 -- 172.21.15.17:0/3705979903 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a880051d0 msgr2=0x7f5aa41e0690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:41.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.351+0000 7f5aa3200640 1 --2- 172.21.15.17:0/3705979903 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a880051d0 0x7f5aa41e0690 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f5a94011ab0 tx=0x7f5a94011f70 comp rx=0 tx=0).stop 2024-09-10T21:36:41.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.351+0000 7f5aa2800640 1 -- 172.21.15.17:0/3705979903 reap_dead start 2024-09-10T21:36:41.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.351+0000 7f5aa2800640 1 -- 172.21.15.17:0/3705979903 reap_dead start 2024-09-10T21:36:41.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.351+0000 7f5aa3200640 1 -- 172.21.15.17:0/3705979903 shutdown_connections 2024-09-10T21:36:41.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.351+0000 7f5aa3200640 1 -- 172.21.15.17:0/3705979903 >> 172.21.15.17:0/3705979903 conn(0x7f5aa406cc20 msgr2=0x7f5aa4075a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:36:41.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.351+0000 7f5aa3200640 1 -- 172.21.15.17:0/3705979903 shutdown_connections 2024-09-10T21:36:41.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.351+0000 7f5aa3200640 1 -- 172.21.15.17:0/3705979903 wait complete. 2024-09-10T21:36:41.371 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-10T21:36:41.372 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-09-10T21:36:41.372 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-10T21:36:41.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.547+0000 7f44c3e00640 1 -- 172.21.15.17:0/2035463154 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f44a80039f0 msgr2=0x7f44a8003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:41.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.547+0000 7f44c3e00640 1 --2- 172.21.15.17:0/2035463154 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f44a80039f0 0x7f44a8003dd0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f44b400baf0 tx=0x7f44b401cd80 comp rx=0 tx=0).stop 2024-09-10T21:36:41.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.547+0000 7f44c3400640 1 -- 172.21.15.17:0/2035463154 reap_dead start 2024-09-10T21:36:41.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.547+0000 7f44c3e00640 1 -- 172.21.15.17:0/2035463154 shutdown_connections 2024-09-10T21:36:41.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.547+0000 7f44c3e00640 1 -- 172.21.15.17:0/2035463154 >> 172.21.15.17:0/2035463154 conn(0x7f44c40cd690 msgr2=0x7f44c40c98e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:36:41.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.547+0000 7f44c3e00640 1 -- 172.21.15.17:0/2035463154 shutdown_connections 2024-09-10T21:36:41.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.547+0000 7f44c3e00640 1 -- 172.21.15.17:0/2035463154 wait complete. 2024-09-10T21:36:41.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.547+0000 7f44c3e00640 1 Processor -- start 2024-09-10T21:36:41.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.547+0000 7f44c3e00640 1 -- start start 2024-09-10T21:36:41.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.551+0000 7f44c3e00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f44a80039f0 0x7f44c41e51a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:41.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.551+0000 7f44c3e00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f44c41e56e0 0x7f44c414fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:41.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.551+0000 7f44c3e00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f44c4150340 0x7f44c4155c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:41.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.551+0000 7f44c3e00640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f44c4151830 con 0x7f44c4150340 2024-09-10T21:36:41.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.551+0000 7f44c3e00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f44c40c2710 con 0x7f44a80039f0 2024-09-10T21:36:41.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.551+0000 7f44c3e00640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f44c41519b0 con 0x7f44c41e56e0 2024-09-10T21:36:41.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.551+0000 7f44c3400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f44c4150340 0x7f44c4155c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:41.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.551+0000 7f44c3400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f44c4150340 0x7f44c4155c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:60534/0 (socket says 172.21.15.17:60534) 2024-09-10T21:36:41.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.551+0000 7f44c3400640 1 -- 172.21.15.17:0/2044617972 learned_addr learned my addr 172.21.15.17:0/2044617972 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:36:41.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.551+0000 7f44c2a00640 1 --2- 172.21.15.17:0/2044617972 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f44a80039f0 0x7f44c41e51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:41.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.551+0000 7f44c2000640 1 --2- 172.21.15.17:0/2044617972 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f44c41e56e0 0x7f44c414fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:41.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.551+0000 7f44c3400640 1 -- 172.21.15.17:0/2044617972 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f44c41e56e0 msgr2=0x7f44c414fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:41.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.551+0000 7f44c2000640 1 -- 172.21.15.17:0/2044617972 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f44c41e56e0 msgr2=0x7f44c414fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-09-10T21:36:41.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.551+0000 7f44c2000640 1 -- 172.21.15.17:0/2044617972 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f44c41e56e0 msgr2=0x7f44c414fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-10T21:36:41.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.551+0000 7f44c2000640 1 --2- 172.21.15.17:0/2044617972 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f44c41e56e0 0x7f44c414fe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T21:36:41.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.551+0000 7f44c2000640 1 --2- 172.21.15.17:0/2044617972 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f44c41e56e0 0x7f44c414fe00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:36:41.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.551+0000 7f44c3400640 1 --2- 172.21.15.17:0/2044617972 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f44c41e56e0 0x7f44c414fe00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:41.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.551+0000 7f44c3400640 1 -- 172.21.15.17:0/2044617972 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f44a80039f0 msgr2=0x7f44c41e51a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:41.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.551+0000 7f44c3400640 1 --2- 172.21.15.17:0/2044617972 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f44a80039f0 0x7f44c41e51a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:41.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.551+0000 7f44c3400640 1 -- 172.21.15.17:0/2044617972 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44c41e3eb0 con 0x7f44c4150340 2024-09-10T21:36:41.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.551+0000 7f44c3400640 1 --2- 172.21.15.17:0/2044617972 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f44c4150340 0x7f44c4155c70 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f44bc002240 tx=0x7f44bc00ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:41.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.551+0000 7f44baa00640 1 -- 172.21.15.17:0/2044617972 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44bc00c9c0 con 0x7f44c4150340 2024-09-10T21:36:41.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.551+0000 7f44baa00640 1 -- 172.21.15.17:0/2044617972 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f44bc017070 con 0x7f44c4150340 2024-09-10T21:36:41.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.551+0000 7f44baa00640 1 -- 172.21.15.17:0/2044617972 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44bc011460 con 0x7f44c4150340 2024-09-10T21:36:41.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.551+0000 7f44c3e00640 1 -- 172.21.15.17:0/2044617972 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44c41569a0 con 0x7f44c4150340 2024-09-10T21:36:41.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.551+0000 7f44c3e00640 1 -- 172.21.15.17:0/2044617972 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44c41551e0 con 0x7f44c4150340 2024-09-10T21:36:41.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.551+0000 7f44c3e00640 1 --2- 172.21.15.17:0/2044617972 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f44c4156b70 0x7f44c41561b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:41.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.551+0000 7f44c3e00640 1 -- 172.21.15.17:0/2044617972 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f44c41e41c0 con 0x7f44c4156b70 2024-09-10T21:36:41.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.551+0000 7f44c2a00640 1 --2- 172.21.15.17:0/2044617972 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f44c4156b70 0x7f44c41561b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:41.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.551+0000 7f44c2a00640 1 -- 172.21.15.17:0/2044617972 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f44c4156b70 msgr2=0x7f44c41561b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 20 2024-09-10T21:36:41.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.551+0000 7f44c2a00640 1 -- 172.21.15.17:0/2044617972 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f44c4156b70 msgr2=0x7f44c41561b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-10T21:36:41.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.551+0000 7f44c2a00640 1 --2- 172.21.15.17:0/2044617972 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f44c4156b70 0x7f44c41561b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T21:36:41.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.551+0000 7f44c2a00640 1 --2- 172.21.15.17:0/2044617972 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f44c4156b70 0x7f44c41561b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:36:41.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.555+0000 7f44baa00640 1 -- 172.21.15.17:0/2044617972 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f44bc011600 con 0x7f44c4150340 2024-09-10T21:36:41.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.555+0000 7f44baa00640 1 --2- 172.21.15.17:0/2044617972 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f44a4072610 0x7f44a4074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:41.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.555+0000 7f44baa00640 1 -- 172.21.15.17:0/2044617972 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f44bc093a60 con 0x7f44c4150340 2024-09-10T21:36:41.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.555+0000 7f44c2000640 1 --2- 172.21.15.17:0/2044617972 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f44a4072610 0x7f44a4074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:41.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.555+0000 7f44c2000640 1 --2- 172.21.15.17:0/2044617972 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f44a4072610 0x7f44a4074ad0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f44b0000c10 tx=0x7f44b0006b20 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:41.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.755+0000 7f44c2a00640 1 --2- 172.21.15.17:0/2044617972 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f44c4156b70 0x7f44c41561b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:41.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.755+0000 7f44c2a00640 1 -- 172.21.15.17:0/2044617972 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f44c4156b70 msgr2=0x7f44c41561b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 21 2024-09-10T21:36:41.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.755+0000 7f44c2a00640 1 -- 172.21.15.17:0/2044617972 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f44c4156b70 msgr2=0x7f44c41561b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-10T21:36:41.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.755+0000 7f44c2a00640 1 --2- 172.21.15.17:0/2044617972 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f44c4156b70 0x7f44c41561b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T21:36:41.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:41.755+0000 7f44c2a00640 1 --2- 172.21.15.17:0/2044617972 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f44c4156b70 0x7f44c41561b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-10T21:36:42.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:42.159+0000 7f44c2a00640 1 --2- 172.21.15.17:0/2044617972 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f44c4156b70 0x7f44c41561b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:42.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:42.159+0000 7f44c2a00640 1 -- 172.21.15.17:0/2044617972 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f44c4156b70 msgr2=0x7f44c41561b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 21 2024-09-10T21:36:42.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:42.159+0000 7f44c2a00640 1 -- 172.21.15.17:0/2044617972 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f44c4156b70 msgr2=0x7f44c41561b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-10T21:36:42.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:42.159+0000 7f44c2a00640 1 --2- 172.21.15.17:0/2044617972 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f44c4156b70 0x7f44c41561b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T21:36:42.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:42.159+0000 7f44c2a00640 1 --2- 172.21.15.17:0/2044617972 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f44c4156b70 0x7f44c41561b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-10T21:36:42.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:42.963+0000 7f44c2a00640 1 --2- 172.21.15.17:0/2044617972 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f44c4156b70 0x7f44c41561b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:42.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:42.963+0000 7f44c2a00640 1 -- 172.21.15.17:0/2044617972 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f44c4156b70 msgr2=0x7f44c41561b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 21 2024-09-10T21:36:42.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:42.963+0000 7f44c2a00640 1 -- 172.21.15.17:0/2044617972 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f44c4156b70 msgr2=0x7f44c41561b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-10T21:36:42.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:42.963+0000 7f44c2a00640 1 --2- 172.21.15.17:0/2044617972 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f44c4156b70 0x7f44c41561b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T21:36:42.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:42.963+0000 7f44c2a00640 1 --2- 172.21.15.17:0/2044617972 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f44c4156b70 0x7f44c41561b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-10T21:36:44.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:44.567+0000 7f44c2a00640 1 --2- 172.21.15.17:0/2044617972 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f44c4156b70 0x7f44c41561b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:44.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:44.567+0000 7f44c2a00640 1 -- 172.21.15.17:0/2044617972 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f44c41e3c10 con 0x7f44c4156b70 2024-09-10T21:36:44.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:44.567+0000 7f44c2a00640 1 --2- 172.21.15.17:0/2044617972 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f44c4156b70 0x7f44c41561b0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f44b4005e60 tx=0x7f44b4005040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:44.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:44.567+0000 7f44baa00640 1 -- 172.21.15.17:0/2044617972 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44b40ab040 con 0x7f44c4156b70 2024-09-10T21:36:44.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:44.567+0000 7f44baa00640 1 -- 172.21.15.17:0/2044617972 <== mon.2 v2:172.21.15.31:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f44b4005580 con 0x7f44c4156b70 2024-09-10T21:36:44.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:44.567+0000 7f44baa00640 1 -- 172.21.15.17:0/2044617972 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f44c4156b70 msgr2=0x7f44c41561b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:44.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:44.567+0000 7f44baa00640 1 --2- 172.21.15.17:0/2044617972 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f44c4156b70 0x7f44c41561b0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f44b4005e60 tx=0x7f44b4005040 comp rx=0 tx=0).stop 2024-09-10T21:36:44.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:44.567+0000 7f44baa00640 1 -- 172.21.15.17:0/2044617972 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f44c4156b70 msgr2=0x7f44c41561b0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:36:44.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:44.567+0000 7f44baa00640 1 --2- 172.21.15.17:0/2044617972 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f44c4156b70 0x7f44c41561b0 secure :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f44b4005e60 tx=0x7f44b4005040 comp rx=0 tx=0).stop 2024-09-10T21:36:44.581 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:44.579+0000 7f44c3e00640 1 --2- 172.21.15.17:0/2044617972 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f44c4146370 0x7f44c4146750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:44.581 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:44.579+0000 7f44c3e00640 1 -- 172.21.15.17:0/2044617972 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f44c4069ef0 con 0x7f44c4146370 2024-09-10T21:36:44.581 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:44.579+0000 7f44c2a00640 1 --2- 172.21.15.17:0/2044617972 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f44c4146370 0x7f44c4146750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:44.581 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:44.579+0000 7f44c2a00640 1 -- 172.21.15.17:0/2044617972 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f44c40cb670 con 0x7f44c4146370 2024-09-10T21:36:44.582 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:44.579+0000 7f44c2a00640 1 --2- 172.21.15.17:0/2044617972 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f44c4146370 0x7f44c4146750 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f44b400ba40 tx=0x7f44b400b260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:44.582 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:44.579+0000 7f44baa00640 1 -- 172.21.15.17:0/2044617972 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44b40d0020 con 0x7f44c4146370 2024-09-10T21:36:44.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:44.579+0000 7f44baa00640 1 -- 172.21.15.17:0/2044617972 <== mon.2 v2:172.21.15.31:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4540 (secure 0 0 0) 0x7f44b40ca770 con 0x7f44c4146370 2024-09-10T21:36:44.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:44.579+0000 7f44baa00640 1 -- 172.21.15.17:0/2044617972 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f44c4146370 msgr2=0x7f44c4146750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:44.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:44.579+0000 7f44baa00640 1 --2- 172.21.15.17:0/2044617972 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f44c4146370 0x7f44c4146750 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f44b400ba40 tx=0x7f44b400b260 comp rx=0 tx=0).stop 2024-09-10T21:36:44.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:44.579+0000 7f44baa00640 1 -- 172.21.15.17:0/2044617972 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f44c4146370 msgr2=0x7f44c4146750 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:36:44.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:44.579+0000 7f44baa00640 1 --2- 172.21.15.17:0/2044617972 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f44c4146370 0x7f44c4146750 secure :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f44b400ba40 tx=0x7f44b400b260 comp rx=0 tx=0).stop 2024-09-10T21:36:44.583 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-10T21:36:44.583 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:36:44.583 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:36:44.583 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-10T21:36:44.584 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 98, 2024-09-10T21:36:44.584 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 10927, 2024-09-10T21:36:44.584 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-10T21:36:44.584 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-10T21:36:44.584 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-10T21:36:44.584 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-10T21:36:44.584 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:36:44.584 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 5, 2024-09-10T21:36:44.584 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:36:44.584 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-10T21:36:44.584 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-10T21:36:44.584 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:36:44.584 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:36:44.584 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:36:44.585 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:36:44.585 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:36:44.585 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:36:44.585 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:36:44.585 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:36:44.585 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:36:44.585 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:36:44.585 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:36:44.585 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:36:44.585 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-10T21:36:44.585 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-10T21:36:44.585 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:36:44.585 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:36:44.585 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:36:44.585 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:36:44.586 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:36:44.586 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:36:44.586 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:36:44.586 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:36:44.586 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:36:44.586 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:36:44.586 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:36:44.586 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:36:44.586 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:36:44.586 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-10T21:36:44.586 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-10T21:36:44.586 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-10T21:36:44.586 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-10T21:36:44.586 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-10T21:36:44.586 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "5e2f023f-d6a6-49f4-a256-f4807dce3739", 2024-09-10T21:36:44.586 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:36:44.587 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:36:44.587 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-10T21:36:44.587 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-10T21:36:44.587 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-10T21:36:44.587 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-10T21:36:44.587 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-10T21:36:44.587 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-10T21:36:44.587 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-10T21:36:44.587 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:36:44.587 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-10T21:36:44.587 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:36:44.587 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:36:44.587 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:36:44.587 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:36:44.587 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:36:44.588 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:36:44.588 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:36:44.588 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:36:44.588 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:36:44.588 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:36:44.588 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:36:44.588 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:36:44.588 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-10T21:36:44.588 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:36:44.588 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-10T21:36:44.588 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:36:44.588 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:36:44.589 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:36:44.589 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:36:44.589 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:36:44.589 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:36:44.589 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:36:44.589 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-10T21:36:44.589 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:36:44.589 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:36:44.589 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:36:44.589 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:36:44.589 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-10T21:36:44.589 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:36:44.589 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:36:44.589 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:36:44.589 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:36:44.590 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-10T21:36:44.590 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-10T21:36:44.590 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:36:44.590 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:36:44.590 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:36:44.590 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:36:44.590 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:36:44.590 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:36:44.590 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:36:44.590 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:36:44.590 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:36:44.590 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:36:44.590 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:36:44.590 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3300", 2024-09-10T21:36:44.590 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:36:44.591 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:36:44.591 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:36:44.591 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:36:44.591 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789", 2024-09-10T21:36:44.591 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:36:44.591 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:36:44.591 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:36:44.591 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:36:44.591 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789/0", 2024-09-10T21:36:44.591 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6789/0", 2024-09-10T21:36:44.591 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:36:44.591 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:36:44.591 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:36:44.591 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:36:44.592 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:36:44.592 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:36:44.592 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:36:44.592 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:36:44.592 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:36:44.592 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:36:44.592 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:36:44.592 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3301", 2024-09-10T21:36:44.592 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:36:44.592 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:36:44.592 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:36:44.592 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:36:44.592 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790", 2024-09-10T21:36:44.592 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:36:44.593 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:36:44.593 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:36:44.593 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:36:44.593 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790/0", 2024-09-10T21:36:44.593 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6790/0", 2024-09-10T21:36:44.593 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:36:44.593 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:36:44.593 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:36:44.593 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:36:44.593 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:36:44.593 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:36:44.593 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-10T21:36:44.593 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-10T21:36:44.593 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:36:44.593 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:36:44.594 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:36:44.594 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:36:44.594 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:36:44.594 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:36:44.594 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-10T21:36:44.594 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:36:44.594 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:36:44.594 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:36:44.594 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:36:44.594 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:36:44.594 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:36:44.594 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:36:44.594 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-10T21:36:44.594 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-10T21:36:44.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:44.579+0000 7f44c3e00640 1 -- 172.21.15.17:0/2044617972 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f44a4072610 msgr2=0x7f44a4074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:44.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:44.579+0000 7f44c3e00640 1 --2- 172.21.15.17:0/2044617972 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f44a4072610 0x7f44a4074ad0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f44b0000c10 tx=0x7f44b0006b20 comp rx=0 tx=0).stop 2024-09-10T21:36:44.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:44.579+0000 7f44c3e00640 1 -- 172.21.15.17:0/2044617972 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f44c4150340 msgr2=0x7f44c4155c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:44.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:44.579+0000 7f44c3e00640 1 --2- 172.21.15.17:0/2044617972 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f44c4150340 0x7f44c4155c70 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f44bc002240 tx=0x7f44bc00ed10 comp rx=0 tx=0).stop 2024-09-10T21:36:44.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:44.579+0000 7f44c3400640 1 -- 172.21.15.17:0/2044617972 reap_dead start 2024-09-10T21:36:44.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:44.579+0000 7f44c3400640 1 -- 172.21.15.17:0/2044617972 reap_dead start 2024-09-10T21:36:44.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:44.579+0000 7f44c3e00640 1 -- 172.21.15.17:0/2044617972 shutdown_connections 2024-09-10T21:36:44.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:44.579+0000 7f44c3e00640 1 -- 172.21.15.17:0/2044617972 >> 172.21.15.17:0/2044617972 conn(0x7f44c40cd690 msgr2=0x7f44c40c9870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:36:44.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:44.579+0000 7f44c3e00640 1 -- 172.21.15.17:0/2044617972 shutdown_connections 2024-09-10T21:36:44.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:44.583+0000 7f44c3e00640 1 -- 172.21.15.17:0/2044617972 wait complete. 2024-09-10T21:36:44.604 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-10T21:36:44.604 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-09-10T21:36:44.604 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-09-10T21:36:44.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:44.755+0000 7fd2d5400640 1 -- 172.21.15.17:0/1202807647 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd2d015bf40 msgr2=0x7fd2d015c3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:44.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:44.755+0000 7fd2d5400640 1 --2- 172.21.15.17:0/1202807647 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd2d015bf40 0x7fd2d015c3f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fd2c8009bc0 tx=0x7fd2c801cba0 comp rx=0 tx=0).stop 2024-09-10T21:36:44.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:44.755+0000 7fd2d5400640 1 -- 172.21.15.17:0/1202807647 shutdown_connections 2024-09-10T21:36:44.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:44.755+0000 7fd2d5400640 1 --2- 172.21.15.17:0/1202807647 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd2d015bf40 0x7fd2d015c3f0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:44.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:44.755+0000 7fd2d5400640 1 --2- 172.21.15.17:0/1202807647 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd2d0074f70 0x7fd2d00c7130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:44.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:44.755+0000 7fd2d5400640 1 -- 172.21.15.17:0/1202807647 >> 172.21.15.17:0/1202807647 conn(0x7fd2d006dd30 msgr2=0x7fd2d006e140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:36:44.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:44.755+0000 7fd2d5400640 1 -- 172.21.15.17:0/1202807647 shutdown_connections 2024-09-10T21:36:44.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:44.755+0000 7fd2d5400640 1 -- 172.21.15.17:0/1202807647 wait complete. 2024-09-10T21:36:44.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:44.755+0000 7fd2d5400640 1 Processor -- start 2024-09-10T21:36:44.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:44.755+0000 7fd2d5400640 1 -- start start 2024-09-10T21:36:44.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:44.755+0000 7fd2d5400640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd2d0070450 0x7fd2d01f5080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:44.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:44.755+0000 7fd2d5400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd2d0074f70 0x7fd2d015f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:44.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:44.755+0000 7fd2d5400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd2d015bf40 0x7fd2d015f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:44.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:44.755+0000 7fd2d5400640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fd2d0069b90 con 0x7fd2d015bf40 2024-09-10T21:36:44.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:44.755+0000 7fd2d5400640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fd2d01592e0 con 0x7fd2d0074f70 2024-09-10T21:36:44.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:44.755+0000 7fd2d5400640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7fd2d006ef60 con 0x7fd2d0070450 2024-09-10T21:36:44.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:44.755+0000 7fd2cf200640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd2d015bf40 0x7fd2d015f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:44.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:44.755+0000 7fd2cf200640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd2d015bf40 0x7fd2d015f610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43048/0 (socket says 172.21.15.17:43048) 2024-09-10T21:36:44.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:44.755+0000 7fd2cf200640 1 -- 172.21.15.17:0/2445713608 learned_addr learned my addr 172.21.15.17:0/2445713608 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:36:44.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:44.755+0000 7fd2ce800640 1 --2- 172.21.15.17:0/2445713608 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd2d0070450 0x7fd2d01f5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:44.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:44.755+0000 7fd2cf200640 1 -- 172.21.15.17:0/2445713608 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd2d0070450 msgr2=0x7fd2d01f5080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:44.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:44.755+0000 7fd2cf200640 1 --2- 172.21.15.17:0/2445713608 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd2d0070450 0x7fd2d01f5080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:44.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:44.759+0000 7fd2cf200640 1 -- 172.21.15.17:0/2445713608 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd2d0074f70 msgr2=0x7fd2d015f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:44.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:44.755+0000 7fd2cde00640 1 --2- 172.21.15.17:0/2445713608 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd2d0074f70 0x7fd2d015f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:44.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:44.759+0000 7fd2cf200640 1 --2- 172.21.15.17:0/2445713608 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd2d0074f70 0x7fd2d015f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:44.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:44.759+0000 7fd2cf200640 1 -- 172.21.15.17:0/2445713608 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2d01f5a30 con 0x7fd2d015bf40 2024-09-10T21:36:44.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:44.759+0000 7fd2cde00640 1 --2- 172.21.15.17:0/2445713608 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd2d0074f70 0x7fd2d015f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:36:44.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:44.759+0000 7fd2cf200640 1 --2- 172.21.15.17:0/2445713608 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd2d015bf40 0x7fd2d015f610 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fd2c801a7d0 tx=0x7fd2c8009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:44.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:44.759+0000 7fd2c6a00640 1 -- 172.21.15.17:0/2445713608 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2c80bb070 con 0x7fd2d015bf40 2024-09-10T21:36:44.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:44.759+0000 7fd2c6a00640 1 -- 172.21.15.17:0/2445713608 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd2c8004d40 con 0x7fd2d015bf40 2024-09-10T21:36:44.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:44.759+0000 7fd2c6a00640 1 -- 172.21.15.17:0/2445713608 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2c80b2040 con 0x7fd2d015bf40 2024-09-10T21:36:44.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:44.759+0000 7fd2d5400640 1 -- 172.21.15.17:0/2445713608 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2d01f5cf0 con 0x7fd2d015bf40 2024-09-10T21:36:44.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:44.759+0000 7fd2d5400640 1 -- 172.21.15.17:0/2445713608 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2d015aca0 con 0x7fd2d015bf40 2024-09-10T21:36:44.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:44.759+0000 7fd2d5400640 1 -- 172.21.15.17:0/2445713608 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2d00c7130 con 0x7fd2d015bf40 2024-09-10T21:36:44.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:44.759+0000 7fd2c6a00640 1 -- 172.21.15.17:0/2445713608 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fd2c80b5030 con 0x7fd2d015bf40 2024-09-10T21:36:44.764 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:44.763+0000 7fd2c6a00640 1 --2- 172.21.15.17:0/2445713608 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fd2b0072610 0x7fd2b0074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:44.764 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:44.763+0000 7fd2c6a00640 1 -- 172.21.15.17:0/2445713608 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fd2c812d090 con 0x7fd2d015bf40 2024-09-10T21:36:44.764 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:44.763+0000 7fd2ce800640 1 --2- 172.21.15.17:0/2445713608 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fd2b0072610 0x7fd2b0074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:44.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:44.763+0000 7fd2ce800640 1 --2- 172.21.15.17:0/2445713608 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fd2b0072610 0x7fd2b0074ad0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fd2c0009b70 tx=0x7fd2c00091e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:44.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:44.767+0000 7fd2c6a00640 1 -- 172.21.15.17:0/2445713608 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fd2c80f1de0 con 0x7fd2d015bf40 2024-09-10T21:36:45.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:45.019+0000 7fd2d5400640 1 -- 172.21.15.17:0/2445713608 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) -- 0x7fd2d0069e60 con 0x7fd2d015bf40 2024-09-10T21:36:45.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:45.023+0000 7fd2c6a00640 1 -- 172.21.15.17:0/2445713608 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) ==== 57+0+0 (secure 0 0 0) 0x7fd2c80a5e20 con 0x7fd2d015bf40 2024-09-10T21:36:45.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:45.023+0000 7fd2d5400640 1 -- 172.21.15.17:0/2445713608 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fd2b0072610 msgr2=0x7fd2b0074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:45.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:45.023+0000 7fd2d5400640 1 --2- 172.21.15.17:0/2445713608 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fd2b0072610 0x7fd2b0074ad0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fd2c0009b70 tx=0x7fd2c00091e0 comp rx=0 tx=0).stop 2024-09-10T21:36:45.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:45.023+0000 7fd2d5400640 1 -- 172.21.15.17:0/2445713608 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd2d015bf40 msgr2=0x7fd2d015f610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:45.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:45.023+0000 7fd2d5400640 1 --2- 172.21.15.17:0/2445713608 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd2d015bf40 0x7fd2d015f610 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fd2c801a7d0 tx=0x7fd2c8009260 comp rx=0 tx=0).stop 2024-09-10T21:36:45.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:45.023+0000 7fd2d5400640 1 -- 172.21.15.17:0/2445713608 shutdown_connections 2024-09-10T21:36:45.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:45.023+0000 7fd2d5400640 1 --2- 172.21.15.17:0/2445713608 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fd2b0072610 0x7fd2b0074ad0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:45.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:45.023+0000 7fd2d5400640 1 --2- 172.21.15.17:0/2445713608 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd2d015bf40 0x7fd2d015f610 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:45.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:45.023+0000 7fd2d5400640 1 --2- 172.21.15.17:0/2445713608 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd2d0074f70 0x7fd2d015f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:45.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:45.023+0000 7fd2d5400640 1 --2- 172.21.15.17:0/2445713608 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd2d0070450 0x7fd2d01f5080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:45.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:45.023+0000 7fd2d5400640 1 -- 172.21.15.17:0/2445713608 >> 172.21.15.17:0/2445713608 conn(0x7fd2d006dd30 msgr2=0x7fd2d006e110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:36:45.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:45.023+0000 7fd2d5400640 1 -- 172.21.15.17:0/2445713608 shutdown_connections 2024-09-10T21:36:45.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:45.023+0000 7fd2d5400640 1 -- 172.21.15.17:0/2445713608 wait complete. 2024-09-10T21:36:45.049 INFO:tasks.mon_thrash.mon_thrasher:waiting for 10.0 secs before continuing thrashing 2024-09-10T21:36:49.474 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:36:49.477 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:36:55.050 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-09-10T21:36:55.050 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-10T21:36:55.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.219+0000 7f15ae400640 1 -- 172.21.15.17:0/354081992 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f15a81524d0 msgr2=0x7f15a815bf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:55.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.219+0000 7f15ae400640 1 --2- 172.21.15.17:0/354081992 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f15a81524d0 0x7f15a815bf00 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f15a815b910 tx=0x7f15a001f5a0 comp rx=0 tx=0).stop 2024-09-10T21:36:55.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.219+0000 7f15ae400640 1 -- 172.21.15.17:0/354081992 shutdown_connections 2024-09-10T21:36:55.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.219+0000 7f15ae400640 1 --2- 172.21.15.17:0/354081992 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f15a81524d0 0x7f15a815bf00 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:55.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.219+0000 7f15ae400640 1 -- 172.21.15.17:0/354081992 >> 172.21.15.17:0/354081992 conn(0x7f15a80cdef0 msgr2=0x7f15a80ce2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:36:55.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.219+0000 7f15ae400640 1 -- 172.21.15.17:0/354081992 shutdown_connections 2024-09-10T21:36:55.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.219+0000 7f15ae400640 1 -- 172.21.15.17:0/354081992 wait complete. 2024-09-10T21:36:55.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.219+0000 7f15ae400640 1 Processor -- start 2024-09-10T21:36:55.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.219+0000 7f15ae400640 1 -- start start 2024-09-10T21:36:55.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.219+0000 7f15ae400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f15a81524d0 0x7f15a81e5c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:55.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.219+0000 7f15ae400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f15a8158c20 0x7f15a806dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:55.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.219+0000 7f15ae400640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f15a815c440 0x7f15a806e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:55.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.219+0000 7f15ae400640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f15a80c9350 con 0x7f15a8158c20 2024-09-10T21:36:55.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.219+0000 7f15ae400640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f15a80c9070 con 0x7f15a81524d0 2024-09-10T21:36:55.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.219+0000 7f15ae400640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f15a815b9e0 con 0x7f15a815c440 2024-09-10T21:36:55.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.219+0000 7f15a7e00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f15a8158c20 0x7f15a806dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:55.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.219+0000 7f15a7e00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f15a8158c20 0x7f15a806dee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:48632/0 (socket says 172.21.15.17:48632) 2024-09-10T21:36:55.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.219+0000 7f15a7e00640 1 -- 172.21.15.17:0/198783891 learned_addr learned my addr 172.21.15.17:0/198783891 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:36:55.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.219+0000 7f15ada00640 1 --2- 172.21.15.17:0/198783891 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f15a815c440 0x7f15a806e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:55.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.219+0000 7f15ad000640 1 --2- 172.21.15.17:0/198783891 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f15a81524d0 0x7f15a81e5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:55.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.223+0000 7f15a7e00640 1 -- 172.21.15.17:0/198783891 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f15a815c440 msgr2=0x7f15a806e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:55.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.223+0000 7f15a7e00640 1 --2- 172.21.15.17:0/198783891 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f15a815c440 0x7f15a806e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:55.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.223+0000 7f15a7e00640 1 -- 172.21.15.17:0/198783891 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f15a81524d0 msgr2=0x7f15a81e5c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:55.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.223+0000 7f15a7e00640 1 --2- 172.21.15.17:0/198783891 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f15a81524d0 0x7f15a81e5c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:55.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.223+0000 7f15a7e00640 1 -- 172.21.15.17:0/198783891 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15a806edd0 con 0x7f15a8158c20 2024-09-10T21:36:55.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.223+0000 7f15a7e00640 1 --2- 172.21.15.17:0/198783891 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f15a8158c20 0x7f15a806dee0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f159400c560 tx=0x7f159400ca20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:55.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.223+0000 7f15a4c00640 1 -- 172.21.15.17:0/198783891 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1594017070 con 0x7f15a8158c20 2024-09-10T21:36:55.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.223+0000 7f15a4c00640 1 -- 172.21.15.17:0/198783891 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f15940053c0 con 0x7f15a8158c20 2024-09-10T21:36:55.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.223+0000 7f15ae400640 1 -- 172.21.15.17:0/198783891 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15a806f940 con 0x7f15a8158c20 2024-09-10T21:36:55.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.223+0000 7f15a4c00640 1 -- 172.21.15.17:0/198783891 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1594012890 con 0x7f15a8158c20 2024-09-10T21:36:55.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.223+0000 7f15ae400640 1 -- 172.21.15.17:0/198783891 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15a814fd00 con 0x7f15a8158c20 2024-09-10T21:36:55.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.223+0000 7f15ae400640 1 -- 172.21.15.17:0/198783891 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15a80c85e0 con 0x7f15a8158c20 2024-09-10T21:36:55.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.223+0000 7f15a4c00640 1 -- 172.21.15.17:0/198783891 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f1594003600 con 0x7f15a8158c20 2024-09-10T21:36:55.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.227+0000 7f15a4c00640 1 --2- 172.21.15.17:0/198783891 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f15880725e0 0x7f1588074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:55.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.227+0000 7f15a4c00640 1 -- 172.21.15.17:0/198783891 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f1594092b20 con 0x7f15a8158c20 2024-09-10T21:36:55.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.227+0000 7f15ad000640 1 --2- 172.21.15.17:0/198783891 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f15880725e0 0x7f1588074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:55.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.227+0000 7f15ad000640 1 --2- 172.21.15.17:0/198783891 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f15880725e0 0x7f1588074aa0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f15a00b3000 tx=0x7f15a001dde0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:55.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.231+0000 7f15a4c00640 1 -- 172.21.15.17:0/198783891 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f159405d840 con 0x7f15a8158c20 2024-09-10T21:36:55.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.543+0000 7f15ae400640 1 -- 172.21.15.17:0/198783891 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f15a80ceaa0 con 0x7f15a8158c20 2024-09-10T21:36:55.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.543+0000 7f15a4c00640 1 -- 172.21.15.17:0/198783891 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1557 (secure 0 0 0) 0x7f1594062270 con 0x7f15a8158c20 2024-09-10T21:36:55.546 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":98,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":16,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5e2f023f-d6a6-49f4-a256-f4807dce3739","modified":"2024-09-10T21:23:46.892130Z","created":"2024-09-10T21:23:46.892130Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3301","nonce":0},{"type":"v1","addr":"172.21.15.31:6790","nonce":0}]},"addr":"172.21.15.31:6790/0","public_addr":"172.21.15.31:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-10T21:36:55.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.547+0000 7f15ae400640 1 -- 172.21.15.17:0/198783891 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f15880725e0 msgr2=0x7f1588074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:55.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.547+0000 7f15ae400640 1 --2- 172.21.15.17:0/198783891 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f15880725e0 0x7f1588074aa0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f15a00b3000 tx=0x7f15a001dde0 comp rx=0 tx=0).stop 2024-09-10T21:36:55.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.547+0000 7f15ae400640 1 -- 172.21.15.17:0/198783891 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f15a8158c20 msgr2=0x7f15a806dee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:55.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.547+0000 7f15ae400640 1 --2- 172.21.15.17:0/198783891 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f15a8158c20 0x7f15a806dee0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f159400c560 tx=0x7f159400ca20 comp rx=0 tx=0).stop 2024-09-10T21:36:55.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.547+0000 7f15ae400640 1 -- 172.21.15.17:0/198783891 shutdown_connections 2024-09-10T21:36:55.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.547+0000 7f15ae400640 1 --2- 172.21.15.17:0/198783891 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f15880725e0 0x7f1588074aa0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:55.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.547+0000 7f15ae400640 1 --2- 172.21.15.17:0/198783891 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f15a815c440 0x7f15a806e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:55.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.547+0000 7f15ae400640 1 --2- 172.21.15.17:0/198783891 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f15a8158c20 0x7f15a806dee0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:55.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.547+0000 7f15ae400640 1 --2- 172.21.15.17:0/198783891 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f15a81524d0 0x7f15a81e5c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:55.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.547+0000 7f15ae400640 1 -- 172.21.15.17:0/198783891 >> 172.21.15.17:0/198783891 conn(0x7f15a80cdef0 msgr2=0x7f15a80c2f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:36:55.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.547+0000 7f15ae400640 1 -- 172.21.15.17:0/198783891 shutdown_connections 2024-09-10T21:36:55.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.547+0000 7f15ae400640 1 -- 172.21.15.17:0/198783891 wait complete. 2024-09-10T21:36:55.572 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-09-10T21:36:55.573 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-09-10T21:36:55.573 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-09-10T21:36:55.573 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-10T21:36:55.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.735+0000 7fecaae00640 1 -- 172.21.15.17:0/1613604448 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feca4158f60 msgr2=0x7feca4072960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:55.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.735+0000 7fecaae00640 1 --2- 172.21.15.17:0/1613604448 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feca4158f60 0x7feca4072960 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fec9c009cb0 tx=0x7fec9c01f5a0 comp rx=0 tx=0).stop 2024-09-10T21:36:55.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.735+0000 7fecaae00640 1 -- 172.21.15.17:0/1613604448 shutdown_connections 2024-09-10T21:36:55.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.735+0000 7fecaae00640 1 --2- 172.21.15.17:0/1613604448 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feca4158f60 0x7feca4072960 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:55.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.735+0000 7fecaae00640 1 --2- 172.21.15.17:0/1613604448 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7feca4152070 0x7feca4072420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:55.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.735+0000 7fecaae00640 1 -- 172.21.15.17:0/1613604448 >> 172.21.15.17:0/1613604448 conn(0x7feca406cc20 msgr2=0x7feca406d030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:36:55.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.735+0000 7fecaae00640 1 -- 172.21.15.17:0/1613604448 shutdown_connections 2024-09-10T21:36:55.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.735+0000 7fecaae00640 1 -- 172.21.15.17:0/1613604448 wait complete. 2024-09-10T21:36:55.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.735+0000 7fecaae00640 1 Processor -- start 2024-09-10T21:36:55.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.739+0000 7fecaae00640 1 -- start start 2024-09-10T21:36:55.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.739+0000 7fecaae00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feca4152070 0x7feca40ca2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:55.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.739+0000 7fecaae00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7feca4152450 0x7feca406de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:55.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.739+0000 7fecaae00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feca4158f60 0x7feca406e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:55.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.739+0000 7fecaae00640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7feca40cecf0 con 0x7feca4158f60 2024-09-10T21:36:55.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.739+0000 7fecaae00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7feca40c5090 con 0x7feca4152070 2024-09-10T21:36:55.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.739+0000 7fecaae00640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7feca40cee70 con 0x7feca4152450 2024-09-10T21:36:55.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.739+0000 7fecaa400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feca4158f60 0x7feca406e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:55.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.739+0000 7fecaa400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feca4158f60 0x7feca406e370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:48652/0 (socket says 172.21.15.17:48652) 2024-09-10T21:36:55.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.739+0000 7fecaa400640 1 -- 172.21.15.17:0/2824301982 learned_addr learned my addr 172.21.15.17:0/2824301982 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:36:55.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.739+0000 7feca9000640 1 --2- 172.21.15.17:0/2824301982 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7feca4152450 0x7feca406de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:55.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.739+0000 7feca9a00640 1 --2- 172.21.15.17:0/2824301982 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feca4152070 0x7feca40ca2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:55.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.739+0000 7fecaa400640 1 -- 172.21.15.17:0/2824301982 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7feca4152450 msgr2=0x7feca406de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:55.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.739+0000 7fecaa400640 1 --2- 172.21.15.17:0/2824301982 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7feca4152450 0x7feca406de30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:55.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.739+0000 7fecaa400640 1 -- 172.21.15.17:0/2824301982 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feca4152070 msgr2=0x7feca40ca2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:55.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.739+0000 7fecaa400640 1 --2- 172.21.15.17:0/2824301982 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feca4152070 0x7feca40ca2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:55.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.739+0000 7fecaa400640 1 -- 172.21.15.17:0/2824301982 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feca414ff70 con 0x7feca4158f60 2024-09-10T21:36:55.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.739+0000 7fecaa400640 1 --2- 172.21.15.17:0/2824301982 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feca4158f60 0x7feca406e370 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fec9c009bd0 tx=0x7fec9c0092c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:55.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.739+0000 7feca1600640 1 -- 172.21.15.17:0/2824301982 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec9c004c50 con 0x7feca4158f60 2024-09-10T21:36:55.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.739+0000 7feca1600640 1 -- 172.21.15.17:0/2824301982 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fec9c0be030 con 0x7feca4158f60 2024-09-10T21:36:55.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.739+0000 7feca1600640 1 -- 172.21.15.17:0/2824301982 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec9c0afb00 con 0x7feca4158f60 2024-09-10T21:36:55.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.739+0000 7fecaae00640 1 -- 172.21.15.17:0/2824301982 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feca40c9780 con 0x7feca4158f60 2024-09-10T21:36:55.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.739+0000 7fecaae00640 1 -- 172.21.15.17:0/2824301982 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feca40c9d40 con 0x7feca4158f60 2024-09-10T21:36:55.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.739+0000 7fecaae00640 1 --2- 172.21.15.17:0/2824301982 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feca406ee20 0x7feca406f200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:55.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.739+0000 7fecaae00640 1 -- 172.21.15.17:0/2824301982 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7feca414fd40 con 0x7feca406ee20 2024-09-10T21:36:55.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.739+0000 7feca9a00640 1 --2- 172.21.15.17:0/2824301982 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feca406ee20 0x7feca406f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:55.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.743+0000 7feca9a00640 1 -- 172.21.15.17:0/2824301982 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7feca40cb7e0 con 0x7feca406ee20 2024-09-10T21:36:55.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.743+0000 7feca9a00640 1 --2- 172.21.15.17:0/2824301982 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feca406ee20 0x7feca406f200 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fec98011a10 tx=0x7fec98011ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:55.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.743+0000 7feca1600640 1 -- 172.21.15.17:0/2824301982 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec9800fce0 con 0x7feca406ee20 2024-09-10T21:36:55.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.743+0000 7feca1600640 1 -- 172.21.15.17:0/2824301982 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fec9800fe80 con 0x7feca406ee20 2024-09-10T21:36:55.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.743+0000 7feca1600640 1 -- 172.21.15.17:0/2824301982 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feca406ee20 msgr2=0x7feca406f200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:55.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.743+0000 7feca1600640 1 --2- 172.21.15.17:0/2824301982 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feca406ee20 0x7feca406f200 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fec98011a10 tx=0x7fec98011ed0 comp rx=0 tx=0).stop 2024-09-10T21:36:55.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.743+0000 7feca1600640 1 -- 172.21.15.17:0/2824301982 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feca406ee20 msgr2=0x7feca406f200 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:36:55.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.743+0000 7feca1600640 1 --2- 172.21.15.17:0/2824301982 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feca406ee20 0x7feca406f200 secure :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fec98011a10 tx=0x7fec98011ed0 comp rx=0 tx=0).stop 2024-09-10T21:36:55.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.743+0000 7feca1600640 1 -- 172.21.15.17:0/2824301982 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fec9c0afca0 con 0x7feca4158f60 2024-09-10T21:36:55.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.743+0000 7feca1600640 1 --2- 172.21.15.17:0/2824301982 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fec840720c0 0x7fec84074580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:55.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.743+0000 7feca1600640 1 -- 172.21.15.17:0/2824301982 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fec9c1320b0 con 0x7feca4158f60 2024-09-10T21:36:55.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.743+0000 7feca9a00640 1 --2- 172.21.15.17:0/2824301982 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fec840720c0 0x7fec84074580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:55.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.743+0000 7feca9a00640 1 --2- 172.21.15.17:0/2824301982 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fec840720c0 0x7fec84074580 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fec98000c10 tx=0x7fec98016000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:55.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.751+0000 7fecaae00640 1 --2- 172.21.15.17:0/2824301982 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feca41f3e40 0x7feca41f4250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:55.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.751+0000 7fecaae00640 1 -- 172.21.15.17:0/2824301982 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7feca40c9f80 con 0x7feca41f3e40 2024-09-10T21:36:55.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.751+0000 7feca9000640 1 --2- 172.21.15.17:0/2824301982 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feca41f3e40 0x7feca41f4250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:55.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.751+0000 7feca9000640 1 -- 172.21.15.17:0/2824301982 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7feca40cbcf0 con 0x7feca41f3e40 2024-09-10T21:36:55.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.751+0000 7feca9000640 1 --2- 172.21.15.17:0/2824301982 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feca41f3e40 0x7feca41f4250 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7feca406d190 tx=0x7fec9000fc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:55.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.751+0000 7feca1600640 1 -- 172.21.15.17:0/2824301982 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec900050d0 con 0x7feca41f3e40 2024-09-10T21:36:55.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.755+0000 7feca1600640 1 -- 172.21.15.17:0/2824301982 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5045 (secure 0 0 0) 0x7fec900050d0 con 0x7feca41f3e40 2024-09-10T21:36:55.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.755+0000 7feca1600640 1 -- 172.21.15.17:0/2824301982 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feca41f3e40 msgr2=0x7feca41f4250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:55.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.755+0000 7feca1600640 1 --2- 172.21.15.17:0/2824301982 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feca41f3e40 0x7feca41f4250 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7feca406d190 tx=0x7fec9000fc00 comp rx=0 tx=0).stop 2024-09-10T21:36:55.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.755+0000 7feca1600640 1 -- 172.21.15.17:0/2824301982 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feca41f3e40 msgr2=0x7feca41f4250 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:36:55.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.755+0000 7feca1600640 1 --2- 172.21.15.17:0/2824301982 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feca41f3e40 0x7feca41f4250 secure :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7feca406d190 tx=0x7fec9000fc00 comp rx=0 tx=0).stop 2024-09-10T21:36:55.757 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-10T21:36:55.757 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:36:55.757 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:36:55.757 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "leader", 2024-09-10T21:36:55.757 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 98, 2024-09-10T21:36:55.757 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 196362, 2024-09-10T21:36:55.757 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-10T21:36:55.757 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-10T21:36:55.757 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-10T21:36:55.757 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-10T21:36:55.757 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:36:55.757 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 16, 2024-09-10T21:36:55.758 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:36:55.758 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-10T21:36:55.758 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-10T21:36:55.758 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:36:55.758 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:36:55.758 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:36:55.758 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:36:55.758 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:36:55.758 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:36:55.758 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:36:55.758 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:36:55.758 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:36:55.758 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:36:55.758 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:36:55.759 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:36:55.759 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-10T21:36:55.759 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-10T21:36:55.759 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:36:55.759 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:36:55.759 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:36:55.759 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:36:55.759 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:36:55.759 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:36:55.759 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:36:55.759 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:36:55.759 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:36:55.759 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:36:55.759 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:36:55.759 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:36:55.760 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:36:55.760 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-10T21:36:55.760 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-10T21:36:55.760 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-10T21:36:55.760 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-10T21:36:55.760 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-10T21:36:55.760 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "5e2f023f-d6a6-49f4-a256-f4807dce3739", 2024-09-10T21:36:55.760 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:36:55.760 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:36:55.760 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-10T21:36:55.761 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-10T21:36:55.761 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-10T21:36:55.761 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-10T21:36:55.761 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-10T21:36:55.761 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-10T21:36:55.761 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-10T21:36:55.761 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:36:55.761 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-10T21:36:55.761 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:36:55.761 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:36:55.761 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:36:55.761 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:36:55.761 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:36:55.762 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:36:55.762 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:36:55.762 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:36:55.762 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:36:55.762 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:36:55.762 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:36:55.762 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:36:55.762 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-10T21:36:55.762 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:36:55.762 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-10T21:36:55.762 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:36:55.762 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:36:55.762 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:36:55.763 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:36:55.763 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:36:55.763 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:36:55.763 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:36:55.763 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-10T21:36:55.763 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:36:55.763 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:36:55.763 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:36:55.763 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:36:55.763 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-10T21:36:55.763 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:36:55.763 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:36:55.763 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:36:55.764 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:36:55.764 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-10T21:36:55.764 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-10T21:36:55.764 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:36:55.764 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:36:55.764 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:36:55.764 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:36:55.764 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:36:55.764 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:36:55.764 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:36:55.764 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:36:55.764 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:36:55.764 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:36:55.764 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:36:55.765 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3300", 2024-09-10T21:36:55.765 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:36:55.765 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:36:55.765 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:36:55.765 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:36:55.765 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789", 2024-09-10T21:36:55.765 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:36:55.765 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:36:55.765 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:36:55.765 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:36:55.765 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789/0", 2024-09-10T21:36:55.765 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6789/0", 2024-09-10T21:36:55.765 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:36:55.766 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:36:55.766 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:36:55.766 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:36:55.766 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:36:55.766 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:36:55.766 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:36:55.766 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:36:55.766 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:36:55.766 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:36:55.766 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:36:55.766 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3301", 2024-09-10T21:36:55.766 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:36:55.766 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:36:55.766 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:36:55.767 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:36:55.767 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790", 2024-09-10T21:36:55.767 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:36:55.767 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:36:55.767 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:36:55.767 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:36:55.767 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790/0", 2024-09-10T21:36:55.767 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6790/0", 2024-09-10T21:36:55.767 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:36:55.767 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:36:55.767 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:36:55.767 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:36:55.767 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:36:55.767 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:36:55.767 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-10T21:36:55.768 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-10T21:36:55.768 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:36:55.768 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:36:55.768 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:36:55.768 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:36:55.768 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:36:55.768 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:36:55.768 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": [ 2024-09-10T21:36:55.768 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:36:55.768 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:36:55.768 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:36:55.768 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 6 2024-09-10T21:36:55.768 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:36:55.768 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:36:55.769 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-10T21:36:55.769 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:36:55.769 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:36:55.769 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:36:55.769 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 8 2024-09-10T21:36:55.769 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:36:55.769 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:36:55.769 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-10T21:36:55.769 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:36:55.769 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:36:55.769 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:36:55.769 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 6 2024-09-10T21:36:55.769 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:36:55.769 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:36:55.770 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": [ 2024-09-10T21:36:55.770 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:36:55.770 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:36:55.770 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:36:55.770 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 2 2024-09-10T21:36:55.770 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:36:55.770 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:36:55.770 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:36:55.770 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-10T21:36:55.770 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-10T21:36:55.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.755+0000 7fecaae00640 1 -- 172.21.15.17:0/2824301982 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fec840720c0 msgr2=0x7fec84074580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:55.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.755+0000 7fecaae00640 1 --2- 172.21.15.17:0/2824301982 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fec840720c0 0x7fec84074580 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fec98000c10 tx=0x7fec98016000 comp rx=0 tx=0).stop 2024-09-10T21:36:55.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.755+0000 7fecaae00640 1 -- 172.21.15.17:0/2824301982 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feca4158f60 msgr2=0x7feca406e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:55.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.755+0000 7fecaae00640 1 --2- 172.21.15.17:0/2824301982 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feca4158f60 0x7feca406e370 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fec9c009bd0 tx=0x7fec9c0092c0 comp rx=0 tx=0).stop 2024-09-10T21:36:55.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.755+0000 7fecaa400640 1 -- 172.21.15.17:0/2824301982 reap_dead start 2024-09-10T21:36:55.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.755+0000 7fecaa400640 1 -- 172.21.15.17:0/2824301982 reap_dead start 2024-09-10T21:36:55.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.755+0000 7fecaae00640 1 -- 172.21.15.17:0/2824301982 shutdown_connections 2024-09-10T21:36:55.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.755+0000 7fecaae00640 1 -- 172.21.15.17:0/2824301982 >> 172.21.15.17:0/2824301982 conn(0x7feca406cc20 msgr2=0x7feca41e0e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:36:55.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.755+0000 7fecaae00640 1 -- 172.21.15.17:0/2824301982 shutdown_connections 2024-09-10T21:36:55.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.755+0000 7fecaae00640 1 -- 172.21.15.17:0/2824301982 wait complete. 2024-09-10T21:36:55.776 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-10T21:36:55.776 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-09-10T21:36:55.777 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-10T21:36:55.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.939+0000 7faa3d200640 1 -- 172.21.15.17:0/2169970092 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faa380c5ec0 msgr2=0x7faa380c62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:55.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.939+0000 7faa3d200640 1 --2- 172.21.15.17:0/2169970092 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faa380c5ec0 0x7faa380c62f0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7faa3815baf0 tx=0x7faa3001f560 comp rx=0 tx=0).stop 2024-09-10T21:36:55.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.939+0000 7faa3d200640 1 -- 172.21.15.17:0/2169970092 shutdown_connections 2024-09-10T21:36:55.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.939+0000 7faa3d200640 1 --2- 172.21.15.17:0/2169970092 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faa380c5ec0 0x7faa380c62f0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:55.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.939+0000 7faa3d200640 1 --2- 172.21.15.17:0/2169970092 >> v2:172.21.15.31:3300/0 conn(0x7faa38070450 0x7faa38157470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:55.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.939+0000 7faa3d200640 1 -- 172.21.15.17:0/2169970092 >> 172.21.15.17:0/2169970092 conn(0x7faa3806dd30 msgr2=0x7faa3806e140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:36:55.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.939+0000 7faa3d200640 1 -- 172.21.15.17:0/2169970092 shutdown_connections 2024-09-10T21:36:55.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.939+0000 7faa3d200640 1 -- 172.21.15.17:0/2169970092 wait complete. 2024-09-10T21:36:55.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.943+0000 7faa3d200640 1 Processor -- start 2024-09-10T21:36:55.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.943+0000 7faa3d200640 1 -- start start 2024-09-10T21:36:55.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.943+0000 7faa3d200640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa38070450 0x7faa3814f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:55.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.943+0000 7faa3d200640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faa380c5ec0 0x7faa380cc400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:55.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.943+0000 7faa3d200640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7faa380c7210 0x7faa380cc940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:55.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.943+0000 7faa3d200640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7faa3815b3b0 con 0x7faa380c5ec0 2024-09-10T21:36:55.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.943+0000 7faa3d200640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7faa3815b0d0 con 0x7faa38070450 2024-09-10T21:36:55.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.943+0000 7faa3d200640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7faa3815b690 con 0x7faa380c7210 2024-09-10T21:36:55.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.943+0000 7faa35400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faa380c5ec0 0x7faa380cc400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:55.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.943+0000 7faa35400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faa380c5ec0 0x7faa380cc400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:48692/0 (socket says 172.21.15.17:48692) 2024-09-10T21:36:55.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.943+0000 7faa35400640 1 -- 172.21.15.17:0/445790452 learned_addr learned my addr 172.21.15.17:0/445790452 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:36:55.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.943+0000 7faa35e00640 1 --2- 172.21.15.17:0/445790452 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa38070450 0x7faa3814f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:55.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.943+0000 7faa36800640 1 --2- 172.21.15.17:0/445790452 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7faa380c7210 0x7faa380cc940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:55.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.943+0000 7faa35400640 1 -- 172.21.15.17:0/445790452 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7faa380c7210 msgr2=0x7faa380cc940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:55.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.943+0000 7faa35400640 1 --2- 172.21.15.17:0/445790452 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7faa380c7210 0x7faa380cc940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:55.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.943+0000 7faa35400640 1 -- 172.21.15.17:0/445790452 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa38070450 msgr2=0x7faa3814f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:55.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.943+0000 7faa35400640 1 --2- 172.21.15.17:0/445790452 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa38070450 0x7faa3814f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:55.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.943+0000 7faa35400640 1 -- 172.21.15.17:0/445790452 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa381f1340 con 0x7faa380c5ec0 2024-09-10T21:36:55.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.943+0000 7faa35400640 1 --2- 172.21.15.17:0/445790452 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faa380c5ec0 0x7faa380cc400 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7faa2800e9c0 tx=0x7faa2800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:55.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.943+0000 7faa2e000640 1 -- 172.21.15.17:0/445790452 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa2800cd20 con 0x7faa380c5ec0 2024-09-10T21:36:55.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.943+0000 7faa2e000640 1 -- 172.21.15.17:0/445790452 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faa28005020 con 0x7faa380c5ec0 2024-09-10T21:36:55.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.943+0000 7faa2e000640 1 -- 172.21.15.17:0/445790452 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa28005470 con 0x7faa380c5ec0 2024-09-10T21:36:55.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.947+0000 7faa3d200640 1 -- 172.21.15.17:0/445790452 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa381f1600 con 0x7faa380c5ec0 2024-09-10T21:36:55.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.947+0000 7faa3d200640 1 -- 172.21.15.17:0/445790452 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa380aff50 con 0x7faa380c5ec0 2024-09-10T21:36:55.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.947+0000 7faa3d200640 1 --2- 172.21.15.17:0/445790452 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa380b05a0 0x7faa38152a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:55.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.947+0000 7faa3d200640 1 -- 172.21.15.17:0/445790452 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7faa380afa20 con 0x7faa380b05a0 2024-09-10T21:36:55.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.947+0000 7faa35e00640 1 --2- 172.21.15.17:0/445790452 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa380b05a0 0x7faa38152a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:55.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.947+0000 7faa35e00640 1 -- 172.21.15.17:0/445790452 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7faa38069b90 con 0x7faa380b05a0 2024-09-10T21:36:55.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.947+0000 7faa35e00640 1 --2- 172.21.15.17:0/445790452 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa380b05a0 0x7faa38152a00 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7faa2400f540 tx=0x7faa2400fa00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:55.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.947+0000 7faa2e000640 1 -- 172.21.15.17:0/445790452 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7faa280047b0 con 0x7faa380c5ec0 2024-09-10T21:36:55.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.947+0000 7faa2e000640 1 --2- 172.21.15.17:0/445790452 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7faa180725e0 0x7faa18074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:55.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.947+0000 7faa2e000640 1 -- 172.21.15.17:0/445790452 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7faa28093ff0 con 0x7faa380c5ec0 2024-09-10T21:36:55.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.947+0000 7faa36800640 1 --2- 172.21.15.17:0/445790452 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7faa180725e0 0x7faa18074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:55.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.947+0000 7faa2e000640 1 -- 172.21.15.17:0/445790452 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa240052e0 con 0x7faa380b05a0 2024-09-10T21:36:55.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.947+0000 7faa2e000640 1 -- 172.21.15.17:0/445790452 <== mon.1 v2:172.21.15.31:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7faa24005480 con 0x7faa380b05a0 2024-09-10T21:36:55.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.947+0000 7faa2e000640 1 -- 172.21.15.17:0/445790452 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa380b05a0 msgr2=0x7faa38152a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:55.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.947+0000 7faa2e000640 1 --2- 172.21.15.17:0/445790452 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa380b05a0 0x7faa38152a00 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7faa2400f540 tx=0x7faa2400fa00 comp rx=0 tx=0).stop 2024-09-10T21:36:55.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.947+0000 7faa2e000640 1 -- 172.21.15.17:0/445790452 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa380b05a0 msgr2=0x7faa38152a00 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:36:55.952 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.947+0000 7faa2e000640 1 --2- 172.21.15.17:0/445790452 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa380b05a0 0x7faa38152a00 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:55.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.951+0000 7faa36800640 1 --2- 172.21.15.17:0/445790452 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7faa180725e0 0x7faa18074aa0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7faa300b2000 tx=0x7faa3001dda0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:55.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.955+0000 7faa3d200640 1 --2- 172.21.15.17:0/445790452 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa380b0120 0x7faa38152f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:55.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.955+0000 7faa3d200640 1 -- 172.21.15.17:0/445790452 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7faa380761e0 con 0x7faa380b0120 2024-09-10T21:36:55.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.959+0000 7faa35e00640 1 --2- 172.21.15.17:0/445790452 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa380b0120 0x7faa38152f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:55.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.959+0000 7faa35e00640 1 -- 172.21.15.17:0/445790452 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7faa38074ed0 con 0x7faa380b0120 2024-09-10T21:36:55.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.959+0000 7faa35e00640 1 --2- 172.21.15.17:0/445790452 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa380b0120 0x7faa38152f40 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7faa24016000 tx=0x7faa24000ee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:55.960 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.959+0000 7faa2e000640 1 -- 172.21.15.17:0/445790452 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa240052e0 con 0x7faa380b0120 2024-09-10T21:36:55.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.959+0000 7faa2e000640 1 -- 172.21.15.17:0/445790452 <== mon.1 v2:172.21.15.31:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4542 (secure 0 0 0) 0x7faa2401c770 con 0x7faa380b0120 2024-09-10T21:36:55.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.959+0000 7faa2e000640 1 -- 172.21.15.17:0/445790452 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa380b0120 msgr2=0x7faa38152f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:55.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.959+0000 7faa2e000640 1 --2- 172.21.15.17:0/445790452 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa380b0120 0x7faa38152f40 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7faa24016000 tx=0x7faa24000ee0 comp rx=0 tx=0).stop 2024-09-10T21:36:55.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.959+0000 7faa2e000640 1 -- 172.21.15.17:0/445790452 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa380b0120 msgr2=0x7faa38152f40 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:36:55.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.959+0000 7faa2e000640 1 --2- 172.21.15.17:0/445790452 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa380b0120 0x7faa38152f40 secure :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7faa24016000 tx=0x7faa24000ee0 comp rx=0 tx=0).stop 2024-09-10T21:36:55.962 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-10T21:36:55.962 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:36:55.962 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:36:55.962 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-10T21:36:55.962 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 98, 2024-09-10T21:36:55.962 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 135346, 2024-09-10T21:36:55.962 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-10T21:36:55.962 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-10T21:36:55.962 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-10T21:36:55.963 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-10T21:36:55.963 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:36:55.963 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 16, 2024-09-10T21:36:55.963 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:36:55.963 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-10T21:36:55.963 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-10T21:36:55.963 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:36:55.963 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:36:55.963 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:36:55.963 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:36:55.963 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:36:55.963 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:36:55.963 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:36:55.963 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:36:55.964 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:36:55.964 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:36:55.964 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:36:55.964 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:36:55.964 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-10T21:36:55.964 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-10T21:36:55.964 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:36:55.964 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:36:55.964 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:36:55.964 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:36:55.964 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:36:55.964 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:36:55.964 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:36:55.965 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:36:55.965 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:36:55.965 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:36:55.965 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:36:55.965 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:36:55.965 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:36:55.965 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-10T21:36:55.965 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-10T21:36:55.965 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-10T21:36:55.965 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-10T21:36:55.965 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-10T21:36:55.965 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "5e2f023f-d6a6-49f4-a256-f4807dce3739", 2024-09-10T21:36:55.965 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:36:55.965 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:36:55.965 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-10T21:36:55.966 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-10T21:36:55.966 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-10T21:36:55.966 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-10T21:36:55.966 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-10T21:36:55.966 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-10T21:36:55.966 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-10T21:36:55.966 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:36:55.966 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-10T21:36:55.966 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:36:55.966 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:36:55.966 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:36:55.966 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:36:55.966 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:36:55.966 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:36:55.967 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:36:55.967 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:36:55.967 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:36:55.967 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:36:55.967 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:36:55.967 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:36:55.967 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-10T21:36:55.967 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:36:55.967 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-10T21:36:55.967 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:36:55.967 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:36:55.967 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:36:55.967 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:36:55.968 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:36:55.968 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:36:55.968 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:36:55.968 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-10T21:36:55.968 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:36:55.968 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:36:55.968 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:36:55.968 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:36:55.968 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-10T21:36:55.968 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:36:55.968 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:36:55.968 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:36:55.968 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:36:55.968 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-10T21:36:55.969 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-10T21:36:55.969 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:36:55.969 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:36:55.969 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:36:55.969 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:36:55.969 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:36:55.969 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:36:55.969 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:36:55.969 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:36:55.969 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:36:55.969 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:36:55.969 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:36:55.969 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3300", 2024-09-10T21:36:55.970 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:36:55.970 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:36:55.970 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:36:55.970 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:36:55.970 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789", 2024-09-10T21:36:55.970 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:36:55.970 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:36:55.970 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:36:55.970 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:36:55.970 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789/0", 2024-09-10T21:36:55.971 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6789/0", 2024-09-10T21:36:55.971 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:36:55.971 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:36:55.971 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:36:55.971 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:36:55.971 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:36:55.971 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:36:55.971 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:36:55.971 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:36:55.971 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:36:55.971 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:36:55.971 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:36:55.972 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3301", 2024-09-10T21:36:55.972 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:36:55.972 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:36:55.972 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:36:55.972 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:36:55.972 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790", 2024-09-10T21:36:55.972 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:36:55.972 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:36:55.972 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:36:55.972 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:36:55.972 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790/0", 2024-09-10T21:36:55.973 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6790/0", 2024-09-10T21:36:55.973 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:36:55.973 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:36:55.973 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:36:55.973 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:36:55.973 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:36:55.973 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:36:55.973 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-10T21:36:55.973 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-10T21:36:55.973 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:36:55.973 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:36:55.974 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:36:55.974 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:36:55.974 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:36:55.974 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:36:55.974 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-10T21:36:55.974 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:36:55.974 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:36:55.974 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:36:55.974 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 2 2024-09-10T21:36:55.974 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:36:55.975 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:36:55.975 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:36:55.975 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-10T21:36:55.975 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-10T21:36:55.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.959+0000 7faa3d200640 1 -- 172.21.15.17:0/445790452 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7faa180725e0 msgr2=0x7faa18074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:55.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.959+0000 7faa3d200640 1 --2- 172.21.15.17:0/445790452 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7faa180725e0 0x7faa18074aa0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7faa300b2000 tx=0x7faa3001dda0 comp rx=0 tx=0).stop 2024-09-10T21:36:55.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.959+0000 7faa3d200640 1 -- 172.21.15.17:0/445790452 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faa380c5ec0 msgr2=0x7faa380cc400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:55.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.959+0000 7faa3d200640 1 --2- 172.21.15.17:0/445790452 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faa380c5ec0 0x7faa380cc400 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7faa2800e9c0 tx=0x7faa2800ee80 comp rx=0 tx=0).stop 2024-09-10T21:36:55.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.959+0000 7faa36800640 1 -- 172.21.15.17:0/445790452 reap_dead start 2024-09-10T21:36:55.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.959+0000 7faa36800640 1 -- 172.21.15.17:0/445790452 reap_dead start 2024-09-10T21:36:55.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.959+0000 7faa3d200640 1 -- 172.21.15.17:0/445790452 shutdown_connections 2024-09-10T21:36:55.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.959+0000 7faa3d200640 1 -- 172.21.15.17:0/445790452 >> 172.21.15.17:0/445790452 conn(0x7faa3806dd30 msgr2=0x7faa3806d100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:36:55.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.959+0000 7faa3d200640 1 -- 172.21.15.17:0/445790452 shutdown_connections 2024-09-10T21:36:55.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:55.959+0000 7faa3d200640 1 -- 172.21.15.17:0/445790452 wait complete. 2024-09-10T21:36:55.983 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-10T21:36:55.983 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-09-10T21:36:55.983 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-10T21:36:56.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:56.135+0000 7f885f200640 1 -- 172.21.15.17:0/757975611 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f886006ec30 msgr2=0x7f886006f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:56.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:56.135+0000 7f885f200640 1 --2- 172.21.15.17:0/757975611 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f886006ec30 0x7f886006f350 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f8860152b10 tx=0x7f884c01c7a0 comp rx=0 tx=0).stop 2024-09-10T21:36:56.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:56.135+0000 7f885f200640 1 -- 172.21.15.17:0/757975611 shutdown_connections 2024-09-10T21:36:56.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:56.135+0000 7f885f200640 1 --2- 172.21.15.17:0/757975611 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f88600c77e0 0x7f886006f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:56.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:56.135+0000 7f885f200640 1 --2- 172.21.15.17:0/757975611 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f886006ec30 0x7f886006f350 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:56.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:56.135+0000 7f885f200640 1 --2- 172.21.15.17:0/757975611 >> v2:172.21.15.31:3300/0 conn(0x7f8860152460 0x7f88600c8770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:56.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:56.135+0000 7f885f200640 1 -- 172.21.15.17:0/757975611 >> 172.21.15.17:0/757975611 conn(0x7f88600cd690 msgr2=0x7f88600c98e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:36:56.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:56.135+0000 7f885f200640 1 -- 172.21.15.17:0/757975611 shutdown_connections 2024-09-10T21:36:56.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:56.135+0000 7f885f200640 1 -- 172.21.15.17:0/757975611 wait complete. 2024-09-10T21:36:56.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:56.135+0000 7f885f200640 1 Processor -- start 2024-09-10T21:36:56.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:56.139+0000 7f885f200640 1 -- start start 2024-09-10T21:36:56.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:56.139+0000 7f885f200640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f886006ec30 0x7f886014fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:56.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:56.139+0000 7f885f200640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f88600c77e0 0x7f8860142860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:56.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:56.139+0000 7f885f200640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8860152460 0x7f8860142da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:56.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:56.139+0000 7f885f200640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f88600ca770 con 0x7f8860152460 2024-09-10T21:36:56.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:56.139+0000 7f885f200640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f88600ca5f0 con 0x7f88600c77e0 2024-09-10T21:36:56.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:56.139+0000 7f885f200640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f88600cc730 con 0x7f886006ec30 2024-09-10T21:36:56.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:56.139+0000 7f885e800640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8860152460 0x7f8860142da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:56.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:56.139+0000 7f885e800640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8860152460 0x7f8860142da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:48708/0 (socket says 172.21.15.17:48708) 2024-09-10T21:36:56.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:56.139+0000 7f885e800640 1 -- 172.21.15.17:0/2693849728 learned_addr learned my addr 172.21.15.17:0/2693849728 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:36:56.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:56.139+0000 7f885de00640 1 --2- 172.21.15.17:0/2693849728 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f886006ec30 0x7f886014fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:56.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:56.139+0000 7f885d400640 1 --2- 172.21.15.17:0/2693849728 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f88600c77e0 0x7f8860142860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:56.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:56.139+0000 7f885e800640 1 -- 172.21.15.17:0/2693849728 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f886006ec30 msgr2=0x7f886014fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:56.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:56.139+0000 7f885e800640 1 --2- 172.21.15.17:0/2693849728 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f886006ec30 0x7f886014fef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:56.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:56.139+0000 7f885e800640 1 -- 172.21.15.17:0/2693849728 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f88600c77e0 msgr2=0x7f8860142860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:56.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:56.139+0000 7f885e800640 1 --2- 172.21.15.17:0/2693849728 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f88600c77e0 0x7f8860142860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:56.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:56.139+0000 7f885e800640 1 -- 172.21.15.17:0/2693849728 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88601f6810 con 0x7f8860152460 2024-09-10T21:36:56.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:56.139+0000 7f885e800640 1 --2- 172.21.15.17:0/2693849728 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8860152460 0x7f8860142da0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f885800d9f0 tx=0x7f885800deb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:56.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:56.139+0000 7f8856000640 1 -- 172.21.15.17:0/2693849728 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8858017070 con 0x7f8860152460 2024-09-10T21:36:56.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:56.139+0000 7f8856000640 1 -- 172.21.15.17:0/2693849728 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8858003980 con 0x7f8860152460 2024-09-10T21:36:56.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:56.139+0000 7f8856000640 1 -- 172.21.15.17:0/2693849728 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f885800be60 con 0x7f8860152460 2024-09-10T21:36:56.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:56.139+0000 7f885f200640 1 -- 172.21.15.17:0/2693849728 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88601f6530 con 0x7f8860152460 2024-09-10T21:36:56.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:56.139+0000 7f885f200640 1 -- 172.21.15.17:0/2693849728 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8860159790 con 0x7f8860152460 2024-09-10T21:36:56.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:56.139+0000 7f885f200640 1 --2- 172.21.15.17:0/2693849728 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8860151580 0x7f88601599d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:56.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:56.139+0000 7f885f200640 1 -- 172.21.15.17:0/2693849728 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f8860069ef0 con 0x7f8860151580 2024-09-10T21:36:56.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:56.139+0000 7f885de00640 1 --2- 172.21.15.17:0/2693849728 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8860151580 0x7f88601599d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:56.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:56.143+0000 7f885de00640 1 -- 172.21.15.17:0/2693849728 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f88600c8770 con 0x7f8860151580 2024-09-10T21:36:56.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:56.143+0000 7f885de00640 1 --2- 172.21.15.17:0/2693849728 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8860151580 0x7f88601599d0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f8850011a10 tx=0x7f8850011ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:56.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:56.143+0000 7f8856000640 1 -- 172.21.15.17:0/2693849728 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f8858010460 con 0x7f8860152460 2024-09-10T21:36:56.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:56.143+0000 7f8856000640 1 --2- 172.21.15.17:0/2693849728 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f88400725e0 0x7f8840074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:56.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:56.143+0000 7f8856000640 1 -- 172.21.15.17:0/2693849728 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f885000fc20 con 0x7f8860151580 2024-09-10T21:36:56.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:56.143+0000 7f8856000640 1 -- 172.21.15.17:0/2693849728 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f88580939b0 con 0x7f8860152460 2024-09-10T21:36:56.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:56.143+0000 7f885d400640 1 --2- 172.21.15.17:0/2693849728 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f88400725e0 0x7f8840074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:56.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:56.143+0000 7f8856000640 1 -- 172.21.15.17:0/2693849728 <== mon.2 v2:172.21.15.31:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f885000fdc0 con 0x7f8860151580 2024-09-10T21:36:56.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:56.143+0000 7f8856000640 1 -- 172.21.15.17:0/2693849728 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8860151580 msgr2=0x7f88601599d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:56.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:56.143+0000 7f8856000640 1 --2- 172.21.15.17:0/2693849728 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8860151580 0x7f88601599d0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f8850011a10 tx=0x7f8850011ed0 comp rx=0 tx=0).stop 2024-09-10T21:36:56.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:56.143+0000 7f8856000640 1 -- 172.21.15.17:0/2693849728 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8860151580 msgr2=0x7f88601599d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:36:56.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:56.143+0000 7f8856000640 1 --2- 172.21.15.17:0/2693849728 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8860151580 0x7f88601599d0 secure :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f8850011a10 tx=0x7f8850011ed0 comp rx=0 tx=0).stop 2024-09-10T21:36:56.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:56.143+0000 7f885d400640 1 --2- 172.21.15.17:0/2693849728 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f88400725e0 0x7f8840074aa0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f884c0ae000 tx=0x7f884c0b3040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:56.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:56.151+0000 7f885f200640 1 --2- 172.21.15.17:0/2693849728 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f886015a270 0x7f886015a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:56.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:56.151+0000 7f885f200640 1 -- 172.21.15.17:0/2693849728 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f88600cc510 con 0x7f886015a270 2024-09-10T21:36:56.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:56.151+0000 7f885de00640 1 --2- 172.21.15.17:0/2693849728 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f886015a270 0x7f886015a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:56.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:56.151+0000 7f885de00640 1 -- 172.21.15.17:0/2693849728 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f8860159f10 con 0x7f886015a270 2024-09-10T21:36:56.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:56.151+0000 7f885de00640 1 --2- 172.21.15.17:0/2693849728 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f886015a270 0x7f886015a650 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f8850011a10 tx=0x7f88500046d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:56.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:56.151+0000 7f8856000640 1 -- 172.21.15.17:0/2693849728 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88500054b0 con 0x7f886015a270 2024-09-10T21:36:56.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:56.151+0000 7f8856000640 1 -- 172.21.15.17:0/2693849728 <== mon.2 v2:172.21.15.31:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4541 (secure 0 0 0) 0x7f88500054b0 con 0x7f886015a270 2024-09-10T21:36:56.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:56.151+0000 7f8856000640 1 -- 172.21.15.17:0/2693849728 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f886015a270 msgr2=0x7f886015a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:56.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:56.151+0000 7f8856000640 1 --2- 172.21.15.17:0/2693849728 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f886015a270 0x7f886015a650 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f8850011a10 tx=0x7f88500046d0 comp rx=0 tx=0).stop 2024-09-10T21:36:56.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:56.151+0000 7f8856000640 1 -- 172.21.15.17:0/2693849728 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f886015a270 msgr2=0x7f886015a650 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:36:56.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:56.151+0000 7f8856000640 1 --2- 172.21.15.17:0/2693849728 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f886015a270 0x7f886015a650 secure :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f8850011a10 tx=0x7f88500046d0 comp rx=0 tx=0).stop 2024-09-10T21:36:56.156 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-10T21:36:56.156 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:36:56.156 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:36:56.156 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-10T21:36:56.156 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 98, 2024-09-10T21:36:56.156 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 22499, 2024-09-10T21:36:56.156 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-10T21:36:56.156 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-10T21:36:56.156 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-10T21:36:56.156 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-10T21:36:56.156 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:36:56.157 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 16, 2024-09-10T21:36:56.157 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:36:56.157 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-10T21:36:56.157 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-10T21:36:56.157 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:36:56.157 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:36:56.157 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:36:56.157 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:36:56.157 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:36:56.157 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:36:56.157 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:36:56.157 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:36:56.157 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:36:56.158 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:36:56.158 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:36:56.158 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:36:56.158 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-10T21:36:56.158 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-10T21:36:56.158 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:36:56.158 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:36:56.158 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:36:56.158 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:36:56.158 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:36:56.158 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:36:56.158 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:36:56.158 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:36:56.158 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:36:56.159 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:36:56.159 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:36:56.159 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:36:56.159 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:36:56.159 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-10T21:36:56.159 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-10T21:36:56.159 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-10T21:36:56.159 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-10T21:36:56.159 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-10T21:36:56.159 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "5e2f023f-d6a6-49f4-a256-f4807dce3739", 2024-09-10T21:36:56.159 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:36:56.159 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:36:56.160 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-10T21:36:56.160 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-10T21:36:56.160 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-10T21:36:56.160 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-10T21:36:56.160 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-10T21:36:56.160 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-10T21:36:56.160 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-10T21:36:56.160 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:36:56.160 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-10T21:36:56.160 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:36:56.160 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:36:56.160 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:36:56.160 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:36:56.160 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:36:56.161 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:36:56.161 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:36:56.161 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:36:56.161 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:36:56.161 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:36:56.161 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:36:56.161 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:36:56.161 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-10T21:36:56.161 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:36:56.161 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-10T21:36:56.161 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:36:56.161 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:36:56.161 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:36:56.162 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:36:56.162 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:36:56.162 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:36:56.162 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:36:56.162 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-10T21:36:56.162 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:36:56.162 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:36:56.162 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:36:56.162 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:36:56.162 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-10T21:36:56.162 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:36:56.162 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:36:56.162 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:36:56.163 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:36:56.163 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-10T21:36:56.163 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-10T21:36:56.163 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:36:56.163 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:36:56.163 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:36:56.163 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:36:56.163 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:36:56.163 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:36:56.163 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:36:56.163 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:36:56.163 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:36:56.163 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:36:56.163 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:36:56.164 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3300", 2024-09-10T21:36:56.164 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:36:56.164 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:36:56.164 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:36:56.164 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:36:56.164 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789", 2024-09-10T21:36:56.164 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:36:56.164 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:36:56.164 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:36:56.164 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:36:56.164 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789/0", 2024-09-10T21:36:56.164 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6789/0", 2024-09-10T21:36:56.164 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:36:56.165 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:36:56.165 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:36:56.165 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:36:56.165 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:36:56.165 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:36:56.165 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:36:56.165 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:36:56.165 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:36:56.165 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:36:56.165 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:36:56.165 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3301", 2024-09-10T21:36:56.165 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:36:56.165 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:36:56.166 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:36:56.166 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:36:56.166 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790", 2024-09-10T21:36:56.166 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:36:56.166 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:36:56.166 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:36:56.166 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:36:56.166 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790/0", 2024-09-10T21:36:56.166 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6790/0", 2024-09-10T21:36:56.166 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:36:56.166 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:36:56.166 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:36:56.166 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:36:56.166 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:36:56.167 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:36:56.167 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-10T21:36:56.167 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-10T21:36:56.167 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:36:56.167 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:36:56.167 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:36:56.167 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:36:56.167 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:36:56.167 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:36:56.167 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-10T21:36:56.167 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:36:56.167 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:36:56.167 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:36:56.167 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:36:56.168 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:36:56.168 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:36:56.168 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:36:56.168 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-10T21:36:56.168 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-10T21:36:56.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:56.151+0000 7f885f200640 1 -- 172.21.15.17:0/2693849728 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f88400725e0 msgr2=0x7f8840074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:56.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:56.151+0000 7f885f200640 1 --2- 172.21.15.17:0/2693849728 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f88400725e0 0x7f8840074aa0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f884c0ae000 tx=0x7f884c0b3040 comp rx=0 tx=0).stop 2024-09-10T21:36:56.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:56.151+0000 7f885f200640 1 -- 172.21.15.17:0/2693849728 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8860152460 msgr2=0x7f8860142da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:56.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:56.151+0000 7f885f200640 1 --2- 172.21.15.17:0/2693849728 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8860152460 0x7f8860142da0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f885800d9f0 tx=0x7f885800deb0 comp rx=0 tx=0).stop 2024-09-10T21:36:56.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:56.151+0000 7f885e800640 1 -- 172.21.15.17:0/2693849728 reap_dead start 2024-09-10T21:36:56.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:56.151+0000 7f885e800640 1 -- 172.21.15.17:0/2693849728 reap_dead start 2024-09-10T21:36:56.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:56.151+0000 7f885f200640 1 -- 172.21.15.17:0/2693849728 shutdown_connections 2024-09-10T21:36:56.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:56.151+0000 7f885f200640 1 -- 172.21.15.17:0/2693849728 >> 172.21.15.17:0/2693849728 conn(0x7f88600cd690 msgr2=0x7f8860146ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:36:56.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:56.151+0000 7f885f200640 1 -- 172.21.15.17:0/2693849728 shutdown_connections 2024-09-10T21:36:56.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:56.151+0000 7f885f200640 1 -- 172.21.15.17:0/2693849728 wait complete. 2024-09-10T21:36:56.176 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-10T21:36:56.176 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['a'] 2024-09-10T21:36:56.176 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2024-09-10T21:36:56.176 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.a 2024-09-10T21:36:56.176 INFO:tasks.mon_thrash.mon_thrasher:killing mon.a 2024-09-10T21:36:56.177 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-10T21:36:56.177 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:36:56.202 INFO:tasks.ceph.mon.a:Stopped 2024-09-10T21:36:56.202 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2024-09-10T21:36:56.202 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-10T21:36:56.411 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:56.407+0000 7f7542400640 1 -- 172.21.15.17:0/911828455 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f753c15b280 msgr2=0x7f753c0d00b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:56.411 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:56.407+0000 7f7542400640 1 --2- 172.21.15.17:0/911828455 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f753c15b280 0x7f753c0d00b0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f753c0b2980 tx=0x7f753401cba0 comp rx=0 tx=0).stop 2024-09-10T21:36:56.411 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:56.407+0000 7f7542400640 1 -- 172.21.15.17:0/911828455 shutdown_connections 2024-09-10T21:36:56.411 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:56.407+0000 7f7542400640 1 --2- 172.21.15.17:0/911828455 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f753c15b280 0x7f753c0d00b0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:56.411 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:56.407+0000 7f7542400640 1 --2- 172.21.15.17:0/911828455 >> v2:172.21.15.31:3300/0 conn(0x7f753c0b2d00 0x7f753c06cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:56.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:56.407+0000 7f7542400640 1 --2- 172.21.15.17:0/911828455 >> v2:172.21.15.17:3300/0 conn(0x7f753c154a20 0x7f753c0c6580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:56.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:56.407+0000 7f7542400640 1 -- 172.21.15.17:0/911828455 >> 172.21.15.17:0/911828455 conn(0x7f753c068990 msgr2=0x7f753c068da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:36:56.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:56.407+0000 7f7542400640 1 -- 172.21.15.17:0/911828455 shutdown_connections 2024-09-10T21:36:56.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:56.407+0000 7f7542400640 1 -- 172.21.15.17:0/911828455 wait complete. 2024-09-10T21:36:56.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:56.411+0000 7f7542400640 1 Processor -- start 2024-09-10T21:36:56.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:56.411+0000 7f7542400640 1 -- start start 2024-09-10T21:36:56.414 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:56.411+0000 7f7542400640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f753c0b2d00 0x7f753c0ca350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:56.414 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:56.411+0000 7f7542400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f753c154a20 0x7f753c0b4f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:56.414 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:56.411+0000 7f7542400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f753c15b280 0x7f753c1f7850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:56.414 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:56.411+0000 7f7542400640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f753c15e310 con 0x7f753c15b280 2024-09-10T21:36:56.414 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:56.411+0000 7f7542400640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f753c0d0c10 con 0x7f753c154a20 2024-09-10T21:36:56.414 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:56.411+0000 7f7542400640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f753c15e490 con 0x7f753c0b2d00 2024-09-10T21:36:56.414 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:56.411+0000 7f753be00640 1 -- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f753c15b280 msgr2=0x7f753c1f7850 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-10T21:36:56.414 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:56.411+0000 7f753be00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f753c15b280 0x7f753c1f7850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:36:56.415 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:56.411+0000 7f753aa00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f753c154a20 0x7f753c0b4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:56.415 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:56.411+0000 7f753aa00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f753c154a20 0x7f753c0b4f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.17:43074/0 (socket says 172.21.15.17:43074) 2024-09-10T21:36:56.415 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:56.411+0000 7f753aa00640 1 -- 172.21.15.17:0/2036673162 learned_addr learned my addr 172.21.15.17:0/2036673162 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:36:56.415 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:56.411+0000 7f753b400640 1 --2- 172.21.15.17:0/2036673162 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f753c0b2d00 0x7f753c0ca350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:56.415 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:56.411+0000 7f753aa00640 1 -- 172.21.15.17:0/2036673162 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f753c0b2d00 msgr2=0x7f753c0ca350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:56.415 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:56.411+0000 7f753aa00640 1 --2- 172.21.15.17:0/2036673162 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f753c0b2d00 0x7f753c0ca350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:56.415 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:56.411+0000 7f753aa00640 1 -- 172.21.15.17:0/2036673162 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f753c15b280 msgr2=0x7f753c1f7850 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:36:56.415 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:56.411+0000 7f753aa00640 1 --2- 172.21.15.17:0/2036673162 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f753c15b280 0x7f753c1f7850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:56.415 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:56.411+0000 7f753aa00640 1 -- 172.21.15.17:0/2036673162 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f753c0c8940 con 0x7f753c154a20 2024-09-10T21:36:56.415 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:56.411+0000 7f753b400640 1 --2- 172.21.15.17:0/2036673162 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f753c0b2d00 0x7f753c0ca350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:36:56.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:56.411+0000 7f753aa00640 1 --2- 172.21.15.17:0/2036673162 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f753c154a20 0x7f753c0b4f20 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f752800ea00 tx=0x7f752800eec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:56.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:56.411+0000 7f7533400640 1 -- 172.21.15.17:0/2036673162 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f752800cde0 con 0x7f753c154a20 2024-09-10T21:36:56.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:56.411+0000 7f7533400640 1 -- 172.21.15.17:0/2036673162 <== mon.1 v2:172.21.15.31:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7528004020 con 0x7f753c154a20 2024-09-10T21:36:56.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:56.411+0000 7f7533400640 1 -- 172.21.15.17:0/2036673162 <== mon.1 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7528004450 con 0x7f753c154a20 2024-09-10T21:36:56.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:56.411+0000 7f7542400640 1 -- 172.21.15.17:0/2036673162 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f753c0c7760 con 0x7f753c154a20 2024-09-10T21:36:56.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:56.411+0000 7f7542400640 1 -- 172.21.15.17:0/2036673162 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f753c0c7d20 con 0x7f753c154a20 2024-09-10T21:36:56.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:56.415+0000 7f7542400640 1 -- 172.21.15.17:0/2036673162 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f753c0c6580 con 0x7f753c154a20 2024-09-10T21:36:56.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:56.415+0000 7f7533400640 1 -- 172.21.15.17:0/2036673162 <== mon.1 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f7528003600 con 0x7f753c154a20 2024-09-10T21:36:56.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:56.415+0000 7f7533400640 1 --2- 172.21.15.17:0/2036673162 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f751c072590 0x7f751c074a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:56.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:56.415+0000 7f7533400640 1 -- 172.21.15.17:0/2036673162 <== mon.1 v2:172.21.15.31:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f7528093fe0 con 0x7f753c154a20 2024-09-10T21:36:56.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:56.415+0000 7f753b400640 1 --2- 172.21.15.17:0/2036673162 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f751c072590 0x7f751c074a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:56.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:56.415+0000 7f753b400640 1 --2- 172.21.15.17:0/2036673162 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f751c072590 0x7f751c074a50 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f752c006fd0 tx=0x7f752c009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:56.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:56.419+0000 7f7533400640 1 -- 172.21.15.17:0/2036673162 <== mon.1 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f752805f240 con 0x7f753c154a20 2024-09-10T21:36:56.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:56.703+0000 7f7542400640 1 -- 172.21.15.17:0/2036673162 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f753c06dbc0 con 0x7f753c154a20 2024-09-10T21:36:56.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:56.703+0000 7f7533400640 1 -- 172.21.15.17:0/2036673162 <== mon.1 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1557 (secure 0 0 0) 0x7f752805eb40 con 0x7f753c154a20 2024-09-10T21:36:56.706 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":98,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":17,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5e2f023f-d6a6-49f4-a256-f4807dce3739","modified":"2024-09-10T21:23:46.892130Z","created":"2024-09-10T21:23:46.892130Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3301","nonce":0},{"type":"v1","addr":"172.21.15.31:6790","nonce":0}]},"addr":"172.21.15.31:6790/0","public_addr":"172.21.15.31:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-10T21:36:56.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:56.707+0000 7f7542400640 1 -- 172.21.15.17:0/2036673162 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f751c072590 msgr2=0x7f751c074a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:56.709 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:56.707+0000 7f7542400640 1 --2- 172.21.15.17:0/2036673162 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f751c072590 0x7f751c074a50 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f752c006fd0 tx=0x7f752c009190 comp rx=0 tx=0).stop 2024-09-10T21:36:56.709 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:56.707+0000 7f7542400640 1 -- 172.21.15.17:0/2036673162 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f753c154a20 msgr2=0x7f753c0b4f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:56.709 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:56.707+0000 7f7542400640 1 --2- 172.21.15.17:0/2036673162 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f753c154a20 0x7f753c0b4f20 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f752800ea00 tx=0x7f752800eec0 comp rx=0 tx=0).stop 2024-09-10T21:36:56.709 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:56.707+0000 7f7542400640 1 -- 172.21.15.17:0/2036673162 shutdown_connections 2024-09-10T21:36:56.709 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:56.707+0000 7f7542400640 1 --2- 172.21.15.17:0/2036673162 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f751c072590 0x7f751c074a50 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:56.709 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:56.707+0000 7f7542400640 1 --2- 172.21.15.17:0/2036673162 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f753c15b280 0x7f753c1f7850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:56.709 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:56.707+0000 7f7542400640 1 --2- 172.21.15.17:0/2036673162 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f753c154a20 0x7f753c0b4f20 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:56.709 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:56.707+0000 7f7542400640 1 --2- 172.21.15.17:0/2036673162 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f753c0b2d00 0x7f753c0ca350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:56.709 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:56.707+0000 7f7542400640 1 -- 172.21.15.17:0/2036673162 >> 172.21.15.17:0/2036673162 conn(0x7f753c068990 msgr2=0x7f753c14fe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:36:56.709 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:56.707+0000 7f7542400640 1 -- 172.21.15.17:0/2036673162 shutdown_connections 2024-09-10T21:36:56.709 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:56.707+0000 7f7542400640 1 -- 172.21.15.17:0/2036673162 wait complete. 2024-09-10T21:36:59.733 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-10T21:36:59.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:59.891+0000 7f6c1b200640 1 -- 172.21.15.17:0/3849017961 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6c1c0c2ea0 msgr2=0x7f6c1c15a8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:59.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:59.891+0000 7f6c1b200640 1 --2- 172.21.15.17:0/3849017961 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6c1c0c2ea0 0x7f6c1c15a8e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f6c14009c80 tx=0x7f6c1401de40 comp rx=0 tx=0).stop 2024-09-10T21:36:59.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:59.891+0000 7f6c1b200640 1 -- 172.21.15.17:0/3849017961 shutdown_connections 2024-09-10T21:36:59.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:59.891+0000 7f6c1b200640 1 --2- 172.21.15.17:0/3849017961 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6c1c0c2ea0 0x7f6c1c15a8e0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:59.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:59.891+0000 7f6c1b200640 1 --2- 172.21.15.17:0/3849017961 >> v2:172.21.15.17:3300/0 conn(0x7f6c1c074de0 0x7f6c1c0c1dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:59.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:59.891+0000 7f6c1b200640 1 -- 172.21.15.17:0/3849017961 >> 172.21.15.17:0/3849017961 conn(0x7f6c1c0c8ab0 msgr2=0x7f6c1c0c8ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:36:59.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:59.891+0000 7f6c1b200640 1 -- 172.21.15.17:0/3849017961 shutdown_connections 2024-09-10T21:36:59.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:59.891+0000 7f6c1b200640 1 -- 172.21.15.17:0/3849017961 wait complete. 2024-09-10T21:36:59.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:59.895+0000 7f6c1b200640 1 Processor -- start 2024-09-10T21:36:59.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:59.895+0000 7f6c1b200640 1 -- start start 2024-09-10T21:36:59.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:59.895+0000 7f6c1b200640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6c1c06eac0 0x7f6c1c0c4850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:59.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:59.895+0000 7f6c1b200640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6c1c074de0 0x7f6c1c1f16c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:59.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:59.895+0000 7f6c1b200640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c1c0c2ea0 0x7f6c1c151cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:59.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:59.895+0000 7f6c1b200640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f6c1c1631a0 con 0x7f6c1c074de0 2024-09-10T21:36:59.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:59.895+0000 7f6c1b200640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f6c1c0c0fd0 con 0x7f6c1c0c2ea0 2024-09-10T21:36:59.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:59.895+0000 7f6c1b200640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f6c1c163320 con 0x7f6c1c06eac0 2024-09-10T21:36:59.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:59.895+0000 7f6c19400640 1 -- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6c1c074de0 msgr2=0x7f6c1c1f16c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-10T21:36:59.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:59.895+0000 7f6c19400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6c1c074de0 0x7f6c1c1f16c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:36:59.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:59.895+0000 7f6c1a800640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c1c0c2ea0 0x7f6c1c151cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:59.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:59.895+0000 7f6c19e00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6c1c06eac0 0x7f6c1c0c4850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:59.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:59.895+0000 7f6c1a800640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c1c0c2ea0 0x7f6c1c151cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.17:43084/0 (socket says 172.21.15.17:43084) 2024-09-10T21:36:59.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:59.895+0000 7f6c1a800640 1 -- 172.21.15.17:0/620749201 learned_addr learned my addr 172.21.15.17:0/620749201 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:36:59.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:59.895+0000 7f6c1a800640 1 -- 172.21.15.17:0/620749201 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6c1c06eac0 msgr2=0x7f6c1c0c4850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:36:59.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:59.895+0000 7f6c1a800640 1 --2- 172.21.15.17:0/620749201 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6c1c06eac0 0x7f6c1c0c4850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:59.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:59.895+0000 7f6c1a800640 1 -- 172.21.15.17:0/620749201 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6c1c074de0 msgr2=0x7f6c1c1f16c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:36:59.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:59.895+0000 7f6c1a800640 1 --2- 172.21.15.17:0/620749201 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6c1c074de0 0x7f6c1c1f16c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:36:59.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:59.895+0000 7f6c1a800640 1 -- 172.21.15.17:0/620749201 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c1c0c64a0 con 0x7f6c1c0c2ea0 2024-09-10T21:36:59.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:59.895+0000 7f6c19e00640 1 --2- 172.21.15.17:0/620749201 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6c1c06eac0 0x7f6c1c0c4850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:36:59.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:59.895+0000 7f6c1a800640 1 --2- 172.21.15.17:0/620749201 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c1c0c2ea0 0x7f6c1c151cb0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f6c14009bd0 tx=0x7f6c140092e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:59.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:59.895+0000 7f6c12000640 1 -- 172.21.15.17:0/620749201 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c140bb020 con 0x7f6c1c0c2ea0 2024-09-10T21:36:59.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:59.895+0000 7f6c12000640 1 -- 172.21.15.17:0/620749201 <== mon.1 v2:172.21.15.31:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6c14004d40 con 0x7f6c1c0c2ea0 2024-09-10T21:36:59.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:59.895+0000 7f6c1b200640 1 -- 172.21.15.17:0/620749201 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c1c0c53e0 con 0x7f6c1c0c2ea0 2024-09-10T21:36:59.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:59.895+0000 7f6c12000640 1 -- 172.21.15.17:0/620749201 <== mon.1 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c140b3040 con 0x7f6c1c0c2ea0 2024-09-10T21:36:59.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:59.899+0000 7f6c1b200640 1 -- 172.21.15.17:0/620749201 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c1c0c5ca0 con 0x7f6c1c0c2ea0 2024-09-10T21:36:59.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:59.899+0000 7f6c1b200640 1 -- 172.21.15.17:0/620749201 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c1c0c6190 con 0x7f6c1c0c2ea0 2024-09-10T21:36:59.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:59.899+0000 7f6c12000640 1 -- 172.21.15.17:0/620749201 <== mon.1 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f6c140a6b00 con 0x7f6c1c0c2ea0 2024-09-10T21:36:59.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:59.899+0000 7f6c12000640 1 --2- 172.21.15.17:0/620749201 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f6bfc072590 0x7f6bfc074a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:36:59.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:59.899+0000 7f6c12000640 1 -- 172.21.15.17:0/620749201 <== mon.1 v2:172.21.15.31:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f6c1412d9f0 con 0x7f6c1c0c2ea0 2024-09-10T21:36:59.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:59.899+0000 7f6c19e00640 1 --2- 172.21.15.17:0/620749201 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f6bfc072590 0x7f6bfc074a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:36:59.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:59.903+0000 7f6c19e00640 1 --2- 172.21.15.17:0/620749201 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f6bfc072590 0x7f6bfc074a50 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f6c0c006fd0 tx=0x7f6c0c009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:36:59.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:36:59.903+0000 7f6c12000640 1 -- 172.21.15.17:0/620749201 <== mon.1 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f6c140c5060 con 0x7f6c1c0c2ea0 2024-09-10T21:37:00.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:00.203+0000 7f6c1b200640 1 -- 172.21.15.17:0/620749201 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f6c1c0c9210 con 0x7f6c1c0c2ea0 2024-09-10T21:37:00.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:00.203+0000 7f6c12000640 1 -- 172.21.15.17:0/620749201 <== mon.1 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1557 (secure 0 0 0) 0x7f6c140a6e20 con 0x7f6c1c0c2ea0 2024-09-10T21:37:00.207 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":98,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":21,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5e2f023f-d6a6-49f4-a256-f4807dce3739","modified":"2024-09-10T21:23:46.892130Z","created":"2024-09-10T21:23:46.892130Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3301","nonce":0},{"type":"v1","addr":"172.21.15.31:6790","nonce":0}]},"addr":"172.21.15.31:6790/0","public_addr":"172.21.15.31:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-10T21:37:00.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:00.207+0000 7f6c1b200640 1 -- 172.21.15.17:0/620749201 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f6bfc072590 msgr2=0x7f6bfc074a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:37:00.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:00.207+0000 7f6c1b200640 1 --2- 172.21.15.17:0/620749201 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f6bfc072590 0x7f6bfc074a50 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f6c0c006fd0 tx=0x7f6c0c009190 comp rx=0 tx=0).stop 2024-09-10T21:37:00.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:00.207+0000 7f6c1b200640 1 -- 172.21.15.17:0/620749201 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c1c0c2ea0 msgr2=0x7f6c1c151cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:37:00.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:00.207+0000 7f6c1b200640 1 --2- 172.21.15.17:0/620749201 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c1c0c2ea0 0x7f6c1c151cb0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f6c14009bd0 tx=0x7f6c140092e0 comp rx=0 tx=0).stop 2024-09-10T21:37:00.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:00.207+0000 7f6c1b200640 1 -- 172.21.15.17:0/620749201 shutdown_connections 2024-09-10T21:37:00.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:00.207+0000 7f6c1b200640 1 --2- 172.21.15.17:0/620749201 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f6bfc072590 0x7f6bfc074a50 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:37:00.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:00.207+0000 7f6c1b200640 1 --2- 172.21.15.17:0/620749201 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6c1c0c2ea0 0x7f6c1c151cb0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:37:00.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:00.207+0000 7f6c1b200640 1 --2- 172.21.15.17:0/620749201 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6c1c074de0 0x7f6c1c1f16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:37:00.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:00.207+0000 7f6c1b200640 1 --2- 172.21.15.17:0/620749201 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6c1c06eac0 0x7f6c1c0c4850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:37:00.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:00.207+0000 7f6c1b200640 1 -- 172.21.15.17:0/620749201 >> 172.21.15.17:0/620749201 conn(0x7f6c1c0c8ab0 msgr2=0x7f6c1c14fdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:37:00.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:00.207+0000 7f6c1b200640 1 -- 172.21.15.17:0/620749201 shutdown_connections 2024-09-10T21:37:00.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:00.207+0000 7f6c1b200640 1 -- 172.21.15.17:0/620749201 wait complete. 2024-09-10T21:37:03.230 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-10T21:37:03.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:03.415+0000 7fe9eb200640 1 -- 172.21.15.17:0/2698193329 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9ec0cddd0 msgr2=0x7fe9ec15b820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:37:03.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:03.415+0000 7fe9eb200640 1 --2- 172.21.15.17:0/2698193329 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9ec0cddd0 0x7fe9ec15b820 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fe9dc009900 tx=0x7fe9dc01c8b0 comp rx=0 tx=0).stop 2024-09-10T21:37:03.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:03.415+0000 7fe9eb200640 1 -- 172.21.15.17:0/2698193329 shutdown_connections 2024-09-10T21:37:03.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:03.415+0000 7fe9eb200640 1 --2- 172.21.15.17:0/2698193329 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe9ec0ce8c0 0x7fe9ec0c2ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:37:03.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:03.415+0000 7fe9eb200640 1 --2- 172.21.15.17:0/2698193329 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9ec0cddd0 0x7fe9ec15b820 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:37:03.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:03.415+0000 7fe9eb200640 1 -- 172.21.15.17:0/2698193329 >> 172.21.15.17:0/2698193329 conn(0x7fe9ec074ac0 msgr2=0x7fe9ec068990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:37:03.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:03.415+0000 7fe9eb200640 1 -- 172.21.15.17:0/2698193329 shutdown_connections 2024-09-10T21:37:03.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:03.415+0000 7fe9eb200640 1 -- 172.21.15.17:0/2698193329 wait complete. 2024-09-10T21:37:03.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:03.419+0000 7fe9eb200640 1 Processor -- start 2024-09-10T21:37:03.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:03.419+0000 7fe9eb200640 1 -- start start 2024-09-10T21:37:03.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:03.419+0000 7fe9eb200640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe9ec071130 0x7fe9ec0c9430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:37:03.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:03.419+0000 7fe9eb200640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe9ec0cddd0 0x7fe9ec0ca450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:37:03.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:03.419+0000 7fe9eb200640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9ec0ce8c0 0x7fe9ec0ca990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:37:03.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:03.419+0000 7fe9eb200640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fe9ec1526b0 con 0x7fe9ec0cddd0 2024-09-10T21:37:03.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:03.419+0000 7fe9eb200640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fe9ec152530 con 0x7fe9ec0ce8c0 2024-09-10T21:37:03.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:03.419+0000 7fe9eb200640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7fe9ec1653d0 con 0x7fe9ec071130 2024-09-10T21:37:03.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:03.419+0000 7fe9e9400640 1 -- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe9ec0cddd0 msgr2=0x7fe9ec0ca450 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-10T21:37:03.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:03.419+0000 7fe9e9400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe9ec0cddd0 0x7fe9ec0ca450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:37:03.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:03.419+0000 7fe9e9e00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe9ec071130 0x7fe9ec0c9430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:37:03.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:03.419+0000 7fe9e9e00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe9ec071130 0x7fe9ec0c9430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3301/0 says I am v2:172.21.15.17:52188/0 (socket says 172.21.15.17:52188) 2024-09-10T21:37:03.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:03.419+0000 7fe9e9e00640 1 -- 172.21.15.17:0/889696118 learned_addr learned my addr 172.21.15.17:0/889696118 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:37:03.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:03.419+0000 7fe9ea800640 1 --2- 172.21.15.17:0/889696118 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9ec0ce8c0 0x7fe9ec0ca990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:37:03.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:03.419+0000 7fe9e9e00640 1 -- 172.21.15.17:0/889696118 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9ec0ce8c0 msgr2=0x7fe9ec0ca990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:37:03.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:03.419+0000 7fe9e9e00640 1 --2- 172.21.15.17:0/889696118 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9ec0ce8c0 0x7fe9ec0ca990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:37:03.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:03.419+0000 7fe9e9e00640 1 -- 172.21.15.17:0/889696118 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe9ec0cddd0 msgr2=0x7fe9ec0ca450 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:37:03.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:03.419+0000 7fe9e9e00640 1 --2- 172.21.15.17:0/889696118 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe9ec0cddd0 0x7fe9ec0ca450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:37:03.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:03.419+0000 7fe9e9e00640 1 -- 172.21.15.17:0/889696118 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9ec151b80 con 0x7fe9ec071130 2024-09-10T21:37:03.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:03.419+0000 7fe9e9e00640 1 --2- 172.21.15.17:0/889696118 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe9ec071130 0x7fe9ec0c9430 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fe9dc009850 tx=0x7fe9dc005e70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:37:03.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:03.419+0000 7fe9e2000640 1 -- 172.21.15.17:0/889696118 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9dc0ba020 con 0x7fe9ec071130 2024-09-10T21:37:03.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:03.419+0000 7fe9e2000640 1 -- 172.21.15.17:0/889696118 <== mon.2 v2:172.21.15.31:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe9dc0053e0 con 0x7fe9ec071130 2024-09-10T21:37:03.424 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:03.419+0000 7fe9e2000640 1 -- 172.21.15.17:0/889696118 <== mon.2 v2:172.21.15.31:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9dc0b2040 con 0x7fe9ec071130 2024-09-10T21:37:03.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:03.419+0000 7fe9eb200640 1 -- 172.21.15.17:0/889696118 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9ec159950 con 0x7fe9ec071130 2024-09-10T21:37:03.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:03.419+0000 7fe9eb200640 1 -- 172.21.15.17:0/889696118 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9ec14e710 con 0x7fe9ec071130 2024-09-10T21:37:03.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:03.423+0000 7fe9eb200640 1 -- 172.21.15.17:0/889696118 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9ec159b20 con 0x7fe9ec071130 2024-09-10T21:37:03.425 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:03.423+0000 7fe9e2000640 1 -- 172.21.15.17:0/889696118 <== mon.2 v2:172.21.15.31:3301/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fe9dc0dbae0 con 0x7fe9ec071130 2024-09-10T21:37:03.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:03.423+0000 7fe9e2000640 1 --2- 172.21.15.17:0/889696118 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fe9cc072580 0x7fe9cc074a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:37:03.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:03.423+0000 7fe9e2000640 1 -- 172.21.15.17:0/889696118 <== mon.2 v2:172.21.15.31:3301/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fe9dc12da00 con 0x7fe9ec071130 2024-09-10T21:37:03.426 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:03.423+0000 7fe9e9400640 1 --2- 172.21.15.17:0/889696118 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fe9cc072580 0x7fe9cc074a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:37:03.427 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:03.423+0000 7fe9e9400640 1 --2- 172.21.15.17:0/889696118 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fe9cc072580 0x7fe9cc074a40 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fe9ec151a90 tx=0x7fe9d8007a00 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:37:03.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:03.427+0000 7fe9e2000640 1 -- 172.21.15.17:0/889696118 <== mon.2 v2:172.21.15.31:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fe9dc0c4060 con 0x7fe9ec071130 2024-09-10T21:37:03.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:03.683+0000 7fe9eb200640 1 -- 172.21.15.17:0/889696118 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fe9ec0c9210 con 0x7fe9ec071130 2024-09-10T21:37:03.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:03.683+0000 7fe9e2000640 1 -- 172.21.15.17:0/889696118 <== mon.2 v2:172.21.15.31:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1557 (secure 0 0 0) 0x7fe9dc0164d0 con 0x7fe9ec071130 2024-09-10T21:37:03.688 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":98,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":24,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5e2f023f-d6a6-49f4-a256-f4807dce3739","modified":"2024-09-10T21:23:46.892130Z","created":"2024-09-10T21:23:46.892130Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3301","nonce":0},{"type":"v1","addr":"172.21.15.31:6790","nonce":0}]},"addr":"172.21.15.31:6790/0","public_addr":"172.21.15.31:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-10T21:37:03.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:03.687+0000 7fe9eb200640 1 -- 172.21.15.17:0/889696118 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fe9cc072580 msgr2=0x7fe9cc074a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:37:03.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:03.687+0000 7fe9eb200640 1 --2- 172.21.15.17:0/889696118 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fe9cc072580 0x7fe9cc074a40 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fe9ec151a90 tx=0x7fe9d8007a00 comp rx=0 tx=0).stop 2024-09-10T21:37:03.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:03.687+0000 7fe9eb200640 1 -- 172.21.15.17:0/889696118 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe9ec071130 msgr2=0x7fe9ec0c9430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:37:03.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:03.687+0000 7fe9eb200640 1 --2- 172.21.15.17:0/889696118 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe9ec071130 0x7fe9ec0c9430 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fe9dc009850 tx=0x7fe9dc005e70 comp rx=0 tx=0).stop 2024-09-10T21:37:03.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:03.687+0000 7fe9eb200640 1 -- 172.21.15.17:0/889696118 shutdown_connections 2024-09-10T21:37:03.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:03.687+0000 7fe9eb200640 1 --2- 172.21.15.17:0/889696118 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fe9cc072580 0x7fe9cc074a40 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:37:03.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:03.687+0000 7fe9eb200640 1 --2- 172.21.15.17:0/889696118 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe9ec0ce8c0 0x7fe9ec0ca990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:37:03.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:03.687+0000 7fe9eb200640 1 --2- 172.21.15.17:0/889696118 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe9ec0cddd0 0x7fe9ec0ca450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:37:03.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:03.687+0000 7fe9eb200640 1 --2- 172.21.15.17:0/889696118 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe9ec071130 0x7fe9ec0c9430 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:37:03.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:03.687+0000 7fe9eb200640 1 -- 172.21.15.17:0/889696118 >> 172.21.15.17:0/889696118 conn(0x7fe9ec074ac0 msgr2=0x7fe9ec0cbd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:37:03.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:03.687+0000 7fe9eb200640 1 -- 172.21.15.17:0/889696118 shutdown_connections 2024-09-10T21:37:03.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:03.687+0000 7fe9eb200640 1 -- 172.21.15.17:0/889696118 wait complete. 2024-09-10T21:37:06.711 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-10T21:37:10.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:10.871+0000 7f0f8f000640 1 -- 172.21.15.17:0/1952878117 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0f70004810 msgr2=0x7f0f70004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:37:10.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:10.871+0000 7f0f8f000640 1 --2- 172.21.15.17:0/1952878117 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0f70004810 0x7f0f70004c90 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f0f74000c10 tx=0x7f0f74020670 comp rx=0 tx=0).stop 2024-09-10T21:37:10.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:10.871+0000 7f0f8f000640 1 -- 172.21.15.17:0/1952878117 shutdown_connections 2024-09-10T21:37:10.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:10.871+0000 7f0f8f000640 1 --2- 172.21.15.17:0/1952878117 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0f700052f0 0x7f0f7000b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:37:10.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:10.871+0000 7f0f8f000640 1 --2- 172.21.15.17:0/1952878117 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0f70004810 0x7f0f70004c90 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:37:10.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:10.871+0000 7f0f8f000640 1 --2- 172.21.15.17:0/1952878117 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f0f700039c0 0x7f0f70003dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:37:10.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:10.871+0000 7f0f8f000640 1 -- 172.21.15.17:0/1952878117 >> 172.21.15.17:0/1952878117 conn(0x7f0f880cdef0 msgr2=0x7f0f880ce2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:37:10.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:10.871+0000 7f0f8f000640 1 -- 172.21.15.17:0/1952878117 shutdown_connections 2024-09-10T21:37:10.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:10.871+0000 7f0f8f000640 1 -- 172.21.15.17:0/1952878117 wait complete. 2024-09-10T21:37:10.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:10.875+0000 7f0f8f000640 1 Processor -- start 2024-09-10T21:37:10.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:10.875+0000 7f0f8f000640 1 -- start start 2024-09-10T21:37:10.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:10.875+0000 7f0f8f000640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f0f700039c0 0x7f0f88145bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:37:10.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:10.875+0000 7f0f8f000640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0f70004810 0x7f0f88159ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:37:10.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:10.875+0000 7f0f8f000640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0f700052f0 0x7f0f88157130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:37:10.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:10.875+0000 7f0f8f000640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f0f880c9350 con 0x7f0f70004810 2024-09-10T21:37:10.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:10.875+0000 7f0f8f000640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f0f880c9070 con 0x7f0f700052f0 2024-09-10T21:37:10.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:10.875+0000 7f0f8f000640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f0f8815b9e0 con 0x7f0f700039c0 2024-09-10T21:37:10.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:10.875+0000 7f0f8d200640 1 -- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0f70004810 msgr2=0x7f0f88159ef0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-10T21:37:10.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:10.875+0000 7f0f8d200640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0f70004810 0x7f0f88159ef0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:37:10.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:10.875+0000 7f0f8dc00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f0f700039c0 0x7f0f88145bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:37:10.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:10.875+0000 7f0f8dc00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f0f700039c0 0x7f0f88145bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3301/0 says I am v2:172.21.15.17:33752/0 (socket says 172.21.15.17:33752) 2024-09-10T21:37:10.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:10.875+0000 7f0f8e600640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0f700052f0 0x7f0f88157130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:37:10.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:10.875+0000 7f0f8dc00640 1 -- 172.21.15.17:0/1433226976 learned_addr learned my addr 172.21.15.17:0/1433226976 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:37:10.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:10.875+0000 7f0f8dc00640 1 -- 172.21.15.17:0/1433226976 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0f700052f0 msgr2=0x7f0f88157130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:37:10.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:10.875+0000 7f0f8dc00640 1 --2- 172.21.15.17:0/1433226976 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0f700052f0 0x7f0f88157130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:37:10.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:10.875+0000 7f0f8dc00640 1 -- 172.21.15.17:0/1433226976 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0f70004810 msgr2=0x7f0f88159ef0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:37:10.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:10.875+0000 7f0f8dc00640 1 --2- 172.21.15.17:0/1433226976 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0f70004810 0x7f0f88159ef0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:37:10.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:10.875+0000 7f0f8dc00640 1 -- 172.21.15.17:0/1433226976 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f881462a0 con 0x7f0f700039c0 2024-09-10T21:37:10.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:10.875+0000 7f0f8dc00640 1 --2- 172.21.15.17:0/1433226976 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f0f700039c0 0x7f0f88145bf0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f0f7c007b00 tx=0x7f0f7c009e50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:37:10.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:10.875+0000 7f0f85600640 1 -- 172.21.15.17:0/1433226976 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f7c012ad0 con 0x7f0f700039c0 2024-09-10T21:37:10.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:10.875+0000 7f0f85600640 1 -- 172.21.15.17:0/1433226976 <== mon.2 v2:172.21.15.31:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0f7c01b070 con 0x7f0f700039c0 2024-09-10T21:37:10.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:10.875+0000 7f0f85600640 1 -- 172.21.15.17:0/1433226976 <== mon.2 v2:172.21.15.31:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f7c016460 con 0x7f0f700039c0 2024-09-10T21:37:10.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:10.875+0000 7f0f8f000640 1 -- 172.21.15.17:0/1433226976 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f8815aa20 con 0x7f0f700039c0 2024-09-10T21:37:10.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:10.875+0000 7f0f8f000640 1 -- 172.21.15.17:0/1433226976 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f8815afe0 con 0x7f0f700039c0 2024-09-10T21:37:10.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:10.879+0000 7f0f8f000640 1 -- 172.21.15.17:0/1433226976 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f8815bf00 con 0x7f0f700039c0 2024-09-10T21:37:10.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:10.879+0000 7f0f85600640 1 -- 172.21.15.17:0/1433226976 <== mon.2 v2:172.21.15.31:3301/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f0f7c016600 con 0x7f0f700039c0 2024-09-10T21:37:10.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:10.879+0000 7f0f85600640 1 --2- 172.21.15.17:0/1433226976 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f0f68072ee0 0x7f0f680753a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:37:10.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:10.879+0000 7f0f85600640 1 -- 172.21.15.17:0/1433226976 <== mon.2 v2:172.21.15.31:3301/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f0f7c093910 con 0x7f0f700039c0 2024-09-10T21:37:10.883 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:10.879+0000 7f0f8d200640 1 --2- 172.21.15.17:0/1433226976 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f0f68072ee0 0x7f0f680753a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:37:10.884 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:10.883+0000 7f0f8d200640 1 --2- 172.21.15.17:0/1433226976 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f0f68072ee0 0x7f0f680753a0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f0f700057f0 tx=0x7f0f7401d510 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:37:10.887 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:10.883+0000 7f0f85600640 1 -- 172.21.15.17:0/1433226976 <== mon.2 v2:172.21.15.31:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f0f7c024020 con 0x7f0f700039c0 2024-09-10T21:37:11.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:11.131+0000 7f0f8f000640 1 -- 172.21.15.17:0/1433226976 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f0f88070850 con 0x7f0f700039c0 2024-09-10T21:37:11.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:11.131+0000 7f0f85600640 1 -- 172.21.15.17:0/1433226976 <== mon.2 v2:172.21.15.31:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1551 (secure 0 0 0) 0x7f0f7c05eb50 con 0x7f0f700039c0 2024-09-10T21:37:11.133 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":100,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":0,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5e2f023f-d6a6-49f4-a256-f4807dce3739","modified":"2024-09-10T21:23:46.892130Z","created":"2024-09-10T21:23:46.892130Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3301","nonce":0},{"type":"v1","addr":"172.21.15.31:6790","nonce":0}]},"addr":"172.21.15.31:6790/0","public_addr":"172.21.15.31:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-10T21:37:11.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:11.131+0000 7f0f8f000640 1 -- 172.21.15.17:0/1433226976 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f0f68072ee0 msgr2=0x7f0f680753a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:37:11.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:11.131+0000 7f0f8f000640 1 --2- 172.21.15.17:0/1433226976 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f0f68072ee0 0x7f0f680753a0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f0f700057f0 tx=0x7f0f7401d510 comp rx=0 tx=0).stop 2024-09-10T21:37:11.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:11.131+0000 7f0f8f000640 1 -- 172.21.15.17:0/1433226976 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f0f700039c0 msgr2=0x7f0f88145bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:37:11.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:11.131+0000 7f0f8f000640 1 --2- 172.21.15.17:0/1433226976 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f0f700039c0 0x7f0f88145bf0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f0f7c007b00 tx=0x7f0f7c009e50 comp rx=0 tx=0).stop 2024-09-10T21:37:11.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:11.131+0000 7f0f8f000640 1 -- 172.21.15.17:0/1433226976 shutdown_connections 2024-09-10T21:37:11.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:11.131+0000 7f0f8f000640 1 --2- 172.21.15.17:0/1433226976 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f0f68072ee0 0x7f0f680753a0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:37:11.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:11.131+0000 7f0f8f000640 1 --2- 172.21.15.17:0/1433226976 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0f700052f0 0x7f0f88157130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:37:11.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:11.131+0000 7f0f8f000640 1 --2- 172.21.15.17:0/1433226976 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0f70004810 0x7f0f88159ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:37:11.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:11.131+0000 7f0f8f000640 1 --2- 172.21.15.17:0/1433226976 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f0f700039c0 0x7f0f88145bf0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:37:11.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:11.131+0000 7f0f8f000640 1 -- 172.21.15.17:0/1433226976 >> 172.21.15.17:0/1433226976 conn(0x7f0f880cdef0 msgr2=0x7f0f881580a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:37:11.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:11.135+0000 7f0f8f000640 1 -- 172.21.15.17:0/1433226976 shutdown_connections 2024-09-10T21:37:11.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:11.135+0000 7f0f8f000640 1 -- 172.21.15.17:0/1433226976 wait complete. 2024-09-10T21:37:11.155 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2024-09-10T21:37:11.155 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon b 2024-09-10T21:37:11.155 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-10T21:37:11.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:11.299+0000 7f4f59400640 1 -- 172.21.15.17:0/3858238105 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f4f54152440 msgr2=0x7f4f540b3eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:37:11.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:11.299+0000 7f4f59400640 1 --2- 172.21.15.17:0/3858238105 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f4f54152440 0x7f4f540b3eb0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f4f541594a0 tx=0x7f4f4401c7b0 comp rx=0 tx=0).stop 2024-09-10T21:37:11.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:11.299+0000 7f4f59400640 1 -- 172.21.15.17:0/3858238105 shutdown_connections 2024-09-10T21:37:11.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:11.299+0000 7f4f59400640 1 --2- 172.21.15.17:0/3858238105 >> v2:172.21.15.17:3300/0 conn(0x7f4f54152820 0x7f4f5415c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:37:11.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:11.299+0000 7f4f59400640 1 --2- 172.21.15.17:0/3858238105 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f4f54152440 0x7f4f540b3eb0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:37:11.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:11.299+0000 7f4f59400640 1 -- 172.21.15.17:0/3858238105 >> 172.21.15.17:0/3858238105 conn(0x7f4f540702f0 msgr2=0x7f4f54070700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:37:11.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:11.299+0000 7f4f59400640 1 -- 172.21.15.17:0/3858238105 shutdown_connections 2024-09-10T21:37:11.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:11.299+0000 7f4f59400640 1 -- 172.21.15.17:0/3858238105 wait complete. 2024-09-10T21:37:11.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:11.303+0000 7f4f59400640 1 Processor -- start 2024-09-10T21:37:11.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:11.303+0000 7f4f59400640 1 -- start start 2024-09-10T21:37:11.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:11.303+0000 7f4f59400640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f4f540cdf20 0x7f4f54159ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:37:11.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:11.303+0000 7f4f59400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4f54152440 0x7f4f54150d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:37:11.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:11.303+0000 7f4f59400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4f54152820 0x7f4f5414b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:37:11.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:11.303+0000 7f4f59400640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f4f5415c7c0 con 0x7f4f54152820 2024-09-10T21:37:11.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:11.303+0000 7f4f59400640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f4f540ce740 con 0x7f4f54152440 2024-09-10T21:37:11.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:11.303+0000 7f4f59400640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f4f540cb450 con 0x7f4f540cdf20 2024-09-10T21:37:11.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:11.303+0000 7f4f53200640 1 -- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4f54152820 msgr2=0x7f4f5414b890 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-10T21:37:11.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:11.303+0000 7f4f53200640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4f54152820 0x7f4f5414b890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:37:11.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:11.303+0000 7f4f52800640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f4f540cdf20 0x7f4f54159ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:37:11.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:11.303+0000 7f4f52800640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f4f540cdf20 0x7f4f54159ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3301/0 says I am v2:172.21.15.17:33770/0 (socket says 172.21.15.17:33770) 2024-09-10T21:37:11.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:11.303+0000 7f4f52800640 1 -- 172.21.15.17:0/1047337058 learned_addr learned my addr 172.21.15.17:0/1047337058 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:37:11.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:11.303+0000 7f4f4be00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4f54152440 0x7f4f54150d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:37:11.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:11.303+0000 7f4f52800640 1 -- 172.21.15.17:0/1047337058 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4f54152440 msgr2=0x7f4f54150d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:37:11.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:11.303+0000 7f4f52800640 1 --2- 172.21.15.17:0/1047337058 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4f54152440 0x7f4f54150d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:37:11.307 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:11.303+0000 7f4f52800640 1 -- 172.21.15.17:0/1047337058 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4f54152820 msgr2=0x7f4f5414b890 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:37:11.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:11.303+0000 7f4f52800640 1 --2- 172.21.15.17:0/1047337058 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4f54152820 0x7f4f5414b890 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:37:11.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:11.303+0000 7f4f52800640 1 -- 172.21.15.17:0/1047337058 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f5414f100 con 0x7f4f540cdf20 2024-09-10T21:37:11.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:11.303+0000 7f4f52800640 1 --2- 172.21.15.17:0/1047337058 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f4f540cdf20 0x7f4f54159ee0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f4f4401a3e0 tx=0x7f4f440a6f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:37:11.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:11.303+0000 7f4f4aa00640 1 -- 172.21.15.17:0/1047337058 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f440b2040 con 0x7f4f540cdf20 2024-09-10T21:37:11.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:11.303+0000 7f4f4aa00640 1 -- 172.21.15.17:0/1047337058 <== mon.2 v2:172.21.15.31:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4f440051e0 con 0x7f4f540cdf20 2024-09-10T21:37:11.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:11.303+0000 7f4f4aa00640 1 -- 172.21.15.17:0/1047337058 <== mon.2 v2:172.21.15.31:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f44005610 con 0x7f4f540cdf20 2024-09-10T21:37:11.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:11.303+0000 7f4f59400640 1 -- 172.21.15.17:0/1047337058 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f5414ee60 con 0x7f4f540cdf20 2024-09-10T21:37:11.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:11.307+0000 7f4f59400640 1 -- 172.21.15.17:0/1047337058 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f540c5b20 con 0x7f4f540cdf20 2024-09-10T21:37:11.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:11.307+0000 7f4f59400640 1 --2- 172.21.15.17:0/1047337058 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4f5414bdd0 0x7f4f540c5d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:37:11.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:11.307+0000 7f4f59400640 1 -- 172.21.15.17:0/1047337058 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f4f540c2ff0 con 0x7f4f5414bdd0 2024-09-10T21:37:11.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:11.307+0000 7f4f4be00640 1 --2- 172.21.15.17:0/1047337058 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4f5414bdd0 0x7f4f540c5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:37:11.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:11.307+0000 7f4f4be00640 1 -- 172.21.15.17:0/1047337058 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4f5414c1b0 con 0x7f4f5414bdd0 2024-09-10T21:37:11.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:11.307+0000 7f4f4be00640 1 --2- 172.21.15.17:0/1047337058 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4f5414bdd0 0x7f4f540c5d60 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f4f54151b10 tx=0x7f4f3c00ec00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:37:11.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:11.307+0000 7f4f4aa00640 1 -- 172.21.15.17:0/1047337058 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f3c017070 con 0x7f4f5414bdd0 2024-09-10T21:37:11.310 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:11.307+0000 7f4f4aa00640 1 -- 172.21.15.17:0/1047337058 <== mon.2 v2:172.21.15.31:3301/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f4f440b5070 con 0x7f4f540cdf20 2024-09-10T21:37:11.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:11.307+0000 7f4f4aa00640 1 --2- 172.21.15.17:0/1047337058 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f4f300724e0 0x7f4f300749a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:37:11.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:11.307+0000 7f4f4aa00640 1 -- 172.21.15.17:0/1047337058 <== mon.2 v2:172.21.15.31:3301/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f4f4412cfd0 con 0x7f4f540cdf20 2024-09-10T21:37:11.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:11.307+0000 7f4f4aa00640 1 -- 172.21.15.17:0/1047337058 <== mon.1 v2:172.21.15.31:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f4f3c00c770 con 0x7f4f5414bdd0 2024-09-10T21:37:11.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:11.307+0000 7f4f4aa00640 1 -- 172.21.15.17:0/1047337058 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4f5414bdd0 msgr2=0x7f4f540c5d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:37:11.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:11.307+0000 7f4f4aa00640 1 --2- 172.21.15.17:0/1047337058 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4f5414bdd0 0x7f4f540c5d60 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f4f54151b10 tx=0x7f4f3c00ec00 comp rx=0 tx=0).stop 2024-09-10T21:37:11.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:11.307+0000 7f4f4aa00640 1 -- 172.21.15.17:0/1047337058 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4f5414bdd0 msgr2=0x7f4f540c5d60 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:37:11.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:11.307+0000 7f4f4aa00640 1 --2- 172.21.15.17:0/1047337058 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4f5414bdd0 0x7f4f540c5d60 secure :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f4f54151b10 tx=0x7f4f3c00ec00 comp rx=0 tx=0).stop 2024-09-10T21:37:11.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:11.307+0000 7f4f53200640 1 --2- 172.21.15.17:0/1047337058 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f4f300724e0 0x7f4f300749a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:37:11.312 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:11.311+0000 7f4f53200640 1 --2- 172.21.15.17:0/1047337058 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f4f300724e0 0x7f4f300749a0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f4f4c003a30 tx=0x7f4f4c001660 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:37:11.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:11.315+0000 7f4f59400640 1 --2- 172.21.15.17:0/1047337058 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4f541f3610 0x7f4f541f3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:37:11.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:11.315+0000 7f4f59400640 1 -- 172.21.15.17:0/1047337058 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f4f54069060 con 0x7f4f541f3610 2024-09-10T21:37:11.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:11.315+0000 7f4f4be00640 1 --2- 172.21.15.17:0/1047337058 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4f541f3610 0x7f4f541f3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:37:11.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:11.315+0000 7f4f4be00640 1 -- 172.21.15.17:0/1047337058 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f4f54068d80 con 0x7f4f541f3610 2024-09-10T21:37:11.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:11.315+0000 7f4f4be00640 1 --2- 172.21.15.17:0/1047337058 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4f541f3610 0x7f4f541f3a20 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f4f3c016000 tx=0x7f4f3c004810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:37:11.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:11.315+0000 7f4f4aa00640 1 -- 172.21.15.17:0/1047337058 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f3c0062b0 con 0x7f4f541f3610 2024-09-10T21:37:11.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:11.315+0000 7f4f4aa00640 1 -- 172.21.15.17:0/1047337058 <== mon.1 v2:172.21.15.31:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5034 (secure 0 0 0) 0x7f4f3c0062b0 con 0x7f4f541f3610 2024-09-10T21:37:11.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:11.315+0000 7f4f4aa00640 1 -- 172.21.15.17:0/1047337058 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4f541f3610 msgr2=0x7f4f541f3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:37:11.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:11.315+0000 7f4f4aa00640 1 --2- 172.21.15.17:0/1047337058 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4f541f3610 0x7f4f541f3a20 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f4f3c016000 tx=0x7f4f3c004810 comp rx=0 tx=0).stop 2024-09-10T21:37:11.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:11.315+0000 7f4f4aa00640 1 -- 172.21.15.17:0/1047337058 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4f541f3610 msgr2=0x7f4f541f3a20 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:37:11.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:11.315+0000 7f4f4aa00640 1 --2- 172.21.15.17:0/1047337058 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4f541f3610 0x7f4f541f3a20 secure :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f4f3c016000 tx=0x7f4f3c004810 comp rx=0 tx=0).stop 2024-09-10T21:37:11.320 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-10T21:37:11.320 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:37:11.320 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:37:11.320 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "leader", 2024-09-10T21:37:11.320 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 100, 2024-09-10T21:37:11.320 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 150701, 2024-09-10T21:37:11.320 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-10T21:37:11.320 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-10T21:37:11.320 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-10T21:37:11.320 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:37:11.320 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 0, 2024-09-10T21:37:11.320 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:37:11.320 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-10T21:37:11.321 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-10T21:37:11.321 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:37:11.321 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:37:11.321 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:37:11.321 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:37:11.321 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:37:11.321 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:37:11.321 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:37:11.321 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:37:11.321 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:37:11.321 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:37:11.321 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:37:11.321 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:37:11.321 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-10T21:37:11.322 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-10T21:37:11.322 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:37:11.322 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:37:11.322 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:37:11.322 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:37:11.322 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:37:11.322 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:37:11.322 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:37:11.322 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:37:11.322 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:37:11.322 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:37:11.322 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:37:11.322 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:37:11.322 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:37:11.322 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-10T21:37:11.323 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-10T21:37:11.323 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-10T21:37:11.323 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-10T21:37:11.323 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-10T21:37:11.323 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "5e2f023f-d6a6-49f4-a256-f4807dce3739", 2024-09-10T21:37:11.323 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:37:11.323 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:37:11.323 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-10T21:37:11.323 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-10T21:37:11.323 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-10T21:37:11.323 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-10T21:37:11.323 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-10T21:37:11.323 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-10T21:37:11.323 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-10T21:37:11.324 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:37:11.324 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-10T21:37:11.324 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:37:11.324 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:37:11.324 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:37:11.324 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:37:11.324 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:37:11.324 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:37:11.324 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:37:11.324 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:37:11.324 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:37:11.324 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:37:11.324 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:37:11.325 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:37:11.325 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-10T21:37:11.325 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:37:11.325 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-10T21:37:11.325 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:37:11.325 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:37:11.325 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:37:11.325 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:37:11.325 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:37:11.325 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:37:11.325 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:37:11.325 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-10T21:37:11.325 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:37:11.325 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:37:11.326 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:37:11.326 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:37:11.326 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-10T21:37:11.326 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:37:11.326 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:37:11.326 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:37:11.326 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:37:11.326 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-10T21:37:11.326 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-10T21:37:11.326 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:37:11.326 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:37:11.326 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:37:11.326 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:37:11.326 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:37:11.326 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:37:11.327 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:37:11.327 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:37:11.327 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:37:11.327 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:37:11.327 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:37:11.327 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3300", 2024-09-10T21:37:11.327 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:37:11.327 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:37:11.327 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:37:11.327 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:37:11.327 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789", 2024-09-10T21:37:11.327 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:37:11.327 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:37:11.327 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:37:11.327 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:37:11.328 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789/0", 2024-09-10T21:37:11.328 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6789/0", 2024-09-10T21:37:11.328 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:37:11.328 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:37:11.328 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:37:11.328 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:37:11.328 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:37:11.328 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:37:11.328 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:37:11.328 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:37:11.328 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:37:11.328 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:37:11.328 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:37:11.329 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3301", 2024-09-10T21:37:11.329 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:37:11.329 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:37:11.329 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:37:11.329 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:37:11.329 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790", 2024-09-10T21:37:11.329 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:37:11.329 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:37:11.329 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:37:11.329 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:37:11.329 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790/0", 2024-09-10T21:37:11.329 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6790/0", 2024-09-10T21:37:11.329 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:37:11.330 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:37:11.330 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:37:11.330 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:37:11.330 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:37:11.330 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:37:11.330 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-10T21:37:11.330 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-10T21:37:11.330 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:37:11.330 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:37:11.330 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:37:11.330 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:37:11.330 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:37:11.330 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:37:11.330 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": [ 2024-09-10T21:37:11.330 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:37:11.331 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:37:11.331 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:37:11.331 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 2 2024-09-10T21:37:11.331 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:37:11.331 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:37:11.331 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-10T21:37:11.331 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:37:11.331 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:37:11.331 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:37:11.331 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 4 2024-09-10T21:37:11.331 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:37:11.331 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:37:11.331 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-10T21:37:11.331 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:37:11.331 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:37:11.332 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:37:11.332 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 4 2024-09-10T21:37:11.332 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:37:11.332 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:37:11.332 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": [ 2024-09-10T21:37:11.332 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:37:11.332 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:37:11.332 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:37:11.332 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:37:11.332 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:37:11.332 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:37:11.332 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:37:11.332 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-10T21:37:11.332 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-10T21:37:11.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:11.315+0000 7f4f59400640 1 -- 172.21.15.17:0/1047337058 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f4f300724e0 msgr2=0x7f4f300749a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:37:11.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:11.315+0000 7f4f59400640 1 --2- 172.21.15.17:0/1047337058 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f4f300724e0 0x7f4f300749a0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f4f4c003a30 tx=0x7f4f4c001660 comp rx=0 tx=0).stop 2024-09-10T21:37:11.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:11.315+0000 7f4f59400640 1 -- 172.21.15.17:0/1047337058 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f4f540cdf20 msgr2=0x7f4f54159ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:37:11.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:11.315+0000 7f4f59400640 1 --2- 172.21.15.17:0/1047337058 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f4f540cdf20 0x7f4f54159ee0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f4f4401a3e0 tx=0x7f4f440a6f90 comp rx=0 tx=0).stop 2024-09-10T21:37:11.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:11.315+0000 7f4f53200640 1 -- 172.21.15.17:0/1047337058 reap_dead start 2024-09-10T21:37:11.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:11.315+0000 7f4f53200640 1 -- 172.21.15.17:0/1047337058 reap_dead start 2024-09-10T21:37:11.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:11.315+0000 7f4f59400640 1 -- 172.21.15.17:0/1047337058 shutdown_connections 2024-09-10T21:37:11.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:11.315+0000 7f4f59400640 1 -- 172.21.15.17:0/1047337058 >> 172.21.15.17:0/1047337058 conn(0x7f4f540702f0 msgr2=0x7f4f540cbeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:37:11.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:11.315+0000 7f4f59400640 1 -- 172.21.15.17:0/1047337058 shutdown_connections 2024-09-10T21:37:11.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:11.315+0000 7f4f59400640 1 -- 172.21.15.17:0/1047337058 wait complete. 2024-09-10T21:37:11.338 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-09-10T21:37:11.338 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon c 2024-09-10T21:37:11.338 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-10T21:37:11.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:11.483+0000 7f51d5800640 1 -- 172.21.15.17:0/2930003467 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f51d0152580 msgr2=0x7f51d00c2ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:37:11.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:11.483+0000 7f51d5800640 1 --2- 172.21.15.17:0/2930003467 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f51d0152580 0x7f51d00c2ff0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f51c0009930 tx=0x7f51c001f0f0 comp rx=0 tx=0).stop 2024-09-10T21:37:11.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:11.483+0000 7f51d5800640 1 -- 172.21.15.17:0/2930003467 shutdown_connections 2024-09-10T21:37:11.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:11.483+0000 7f51d5800640 1 --2- 172.21.15.17:0/2930003467 >> v2:172.21.15.17:3300/0 conn(0x7f51d00c89d0 0x7f51d00cadd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:37:11.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:11.483+0000 7f51d5800640 1 --2- 172.21.15.17:0/2930003467 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f51d0152580 0x7f51d00c2ff0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:37:11.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:11.483+0000 7f51d5800640 1 -- 172.21.15.17:0/2930003467 >> 172.21.15.17:0/2930003467 conn(0x7f51d0068b60 msgr2=0x7f51d0068f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:37:11.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:11.483+0000 7f51d5800640 1 -- 172.21.15.17:0/2930003467 shutdown_connections 2024-09-10T21:37:11.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:11.483+0000 7f51d5800640 1 -- 172.21.15.17:0/2930003467 wait complete. 2024-09-10T21:37:11.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:11.483+0000 7f51d5800640 1 Processor -- start 2024-09-10T21:37:11.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:11.483+0000 7f51d5800640 1 -- start start 2024-09-10T21:37:11.488 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:11.487+0000 7f51d5800640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f51d00c89d0 0x7f51d00c6f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:37:11.488 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:11.487+0000 7f51d5800640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f51d00cb310 0x7f51d00ccf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:37:11.488 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:11.487+0000 7f51d5800640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f51d0152580 0x7f51d00cd490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:37:11.488 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:11.487+0000 7f51d5800640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f51d0070940 con 0x7f51d00cb310 2024-09-10T21:37:11.488 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:11.487+0000 7f51d5800640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f51d0152960 con 0x7f51d00c89d0 2024-09-10T21:37:11.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:11.487+0000 7f51d5800640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f51d00c83b0 con 0x7f51d0152580 2024-09-10T21:37:11.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:11.487+0000 7f51c6000640 1 -- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f51d00cb310 msgr2=0x7f51d00ccf50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-10T21:37:11.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:11.487+0000 7f51c6000640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f51d00cb310 0x7f51d00ccf50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:37:11.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:11.487+0000 7f51cea00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f51d00c89d0 0x7f51d00c6f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:37:11.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:11.487+0000 7f51cf400640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f51d0152580 0x7f51d00cd490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:37:11.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:11.487+0000 7f51cea00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f51d00c89d0 0x7f51d00c6f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.17:37704/0 (socket says 172.21.15.17:37704) 2024-09-10T21:37:11.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:11.487+0000 7f51cea00640 1 -- 172.21.15.17:0/962681369 learned_addr learned my addr 172.21.15.17:0/962681369 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:37:11.490 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:11.487+0000 7f51cea00640 1 -- 172.21.15.17:0/962681369 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f51d0152580 msgr2=0x7f51d00cd490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:37:11.490 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:11.487+0000 7f51cea00640 1 --2- 172.21.15.17:0/962681369 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f51d0152580 0x7f51d00cd490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:37:11.490 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:11.487+0000 7f51cea00640 1 -- 172.21.15.17:0/962681369 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f51d00cb310 msgr2=0x7f51d00ccf50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:37:11.490 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:11.487+0000 7f51cea00640 1 --2- 172.21.15.17:0/962681369 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f51d00cb310 0x7f51d00ccf50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:37:11.490 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:11.487+0000 7f51cea00640 1 -- 172.21.15.17:0/962681369 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51d01507c0 con 0x7f51d00c89d0 2024-09-10T21:37:11.490 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:11.487+0000 7f51cf400640 1 --2- 172.21.15.17:0/962681369 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f51d0152580 0x7f51d00cd490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:37:11.490 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:11.487+0000 7f51cea00640 1 --2- 172.21.15.17:0/962681369 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f51d00c89d0 0x7f51d00c6f70 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f51c0009850 tx=0x7f51c00a8d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:37:11.490 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:11.487+0000 7f51c7400640 1 -- 172.21.15.17:0/962681369 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51c00b5040 con 0x7f51d00c89d0 2024-09-10T21:37:11.490 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:11.487+0000 7f51c7400640 1 -- 172.21.15.17:0/962681369 <== mon.1 v2:172.21.15.31:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f51c0005320 con 0x7f51d00c89d0 2024-09-10T21:37:11.490 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:11.487+0000 7f51d5800640 1 -- 172.21.15.17:0/962681369 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51d0150a60 con 0x7f51d00c89d0 2024-09-10T21:37:11.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:11.487+0000 7f51c7400640 1 -- 172.21.15.17:0/962681369 <== mon.1 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51c00046d0 con 0x7f51d00c89d0 2024-09-10T21:37:11.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:11.487+0000 7f51d5800640 1 -- 172.21.15.17:0/962681369 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51d006f890 con 0x7f51d00c89d0 2024-09-10T21:37:11.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:11.487+0000 7f51d5800640 1 --2- 172.21.15.17:0/962681369 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f51d014c970 0x7f51d0151460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:37:11.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:11.487+0000 7f51d5800640 1 -- 172.21.15.17:0/962681369 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f51d00cadd0 con 0x7f51d014c970 2024-09-10T21:37:11.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:11.487+0000 7f51c6000640 1 --2- 172.21.15.17:0/962681369 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f51d014c970 0x7f51d0151460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:37:11.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:11.487+0000 7f51c6000640 1 -- 172.21.15.17:0/962681369 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f51d00b3490 con 0x7f51d014c970 2024-09-10T21:37:11.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:11.487+0000 7f51c6000640 1 --2- 172.21.15.17:0/962681369 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f51d014c970 0x7f51d0151460 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f51b8000f30 tx=0x7f51b800b8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:37:11.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:11.491+0000 7f51c7400640 1 -- 172.21.15.17:0/962681369 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51b8003d90 con 0x7f51d014c970 2024-09-10T21:37:11.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:11.491+0000 7f51c7400640 1 -- 172.21.15.17:0/962681369 <== mon.1 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f51c0004890 con 0x7f51d00c89d0 2024-09-10T21:37:11.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:11.491+0000 7f51c7400640 1 --2- 172.21.15.17:0/962681369 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f51ac072510 0x7f51ac0749d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:37:11.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:11.491+0000 7f51c7400640 1 -- 172.21.15.17:0/962681369 <== mon.1 v2:172.21.15.31:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f51c0130640 con 0x7f51d00c89d0 2024-09-10T21:37:11.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:11.491+0000 7f51c7400640 1 -- 172.21.15.17:0/962681369 <== mon.2 v2:172.21.15.31:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f51b8015070 con 0x7f51d014c970 2024-09-10T21:37:11.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:11.491+0000 7f51c7400640 1 -- 172.21.15.17:0/962681369 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f51d014c970 msgr2=0x7f51d0151460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:37:11.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:11.491+0000 7f51c7400640 1 --2- 172.21.15.17:0/962681369 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f51d014c970 0x7f51d0151460 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f51b8000f30 tx=0x7f51b800b8e0 comp rx=0 tx=0).stop 2024-09-10T21:37:11.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:11.491+0000 7f51c7400640 1 -- 172.21.15.17:0/962681369 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f51d014c970 msgr2=0x7f51d0151460 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:37:11.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:11.491+0000 7f51c7400640 1 --2- 172.21.15.17:0/962681369 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f51d014c970 0x7f51d0151460 secure :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f51b8000f30 tx=0x7f51b800b8e0 comp rx=0 tx=0).stop 2024-09-10T21:37:11.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:11.491+0000 7f51cf400640 1 --2- 172.21.15.17:0/962681369 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f51ac072510 0x7f51ac0749d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:37:11.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:11.491+0000 7f51cf400640 1 --2- 172.21.15.17:0/962681369 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f51ac072510 0x7f51ac0749d0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f51d00c5550 tx=0x7f51c800d040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:37:11.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:11.495+0000 7f51d5800640 1 --2- 172.21.15.17:0/962681369 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f51d014b6f0 0x7f51d01519a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:37:11.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:11.495+0000 7f51d5800640 1 -- 172.21.15.17:0/962681369 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f51d00c7ef0 con 0x7f51d014b6f0 2024-09-10T21:37:11.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:11.499+0000 7f51c6000640 1 --2- 172.21.15.17:0/962681369 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f51d014b6f0 0x7f51d01519a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:37:11.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:11.499+0000 7f51c6000640 1 -- 172.21.15.17:0/962681369 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f51d0158550 con 0x7f51d014b6f0 2024-09-10T21:37:11.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:11.499+0000 7f51c6000640 1 --2- 172.21.15.17:0/962681369 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f51d014b6f0 0x7f51d01519a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f51b8006fd0 tx=0x7f51b80059e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:37:11.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:11.499+0000 7f51c7400640 1 -- 172.21.15.17:0/962681369 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51b801ae00 con 0x7f51d014b6f0 2024-09-10T21:37:11.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:11.499+0000 7f51c7400640 1 -- 172.21.15.17:0/962681369 <== mon.2 v2:172.21.15.31:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5031 (secure 0 0 0) 0x7f51b8010400 con 0x7f51d014b6f0 2024-09-10T21:37:11.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:11.499+0000 7f51c7400640 1 -- 172.21.15.17:0/962681369 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f51d014b6f0 msgr2=0x7f51d01519a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:37:11.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:11.499+0000 7f51c7400640 1 --2- 172.21.15.17:0/962681369 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f51d014b6f0 0x7f51d01519a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f51b8006fd0 tx=0x7f51b80059e0 comp rx=0 tx=0).stop 2024-09-10T21:37:11.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:11.499+0000 7f51c7400640 1 -- 172.21.15.17:0/962681369 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f51d014b6f0 msgr2=0x7f51d01519a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:37:11.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:11.499+0000 7f51c7400640 1 --2- 172.21.15.17:0/962681369 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f51d014b6f0 0x7f51d01519a0 secure :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f51b8006fd0 tx=0x7f51b80059e0 comp rx=0 tx=0).stop 2024-09-10T21:37:11.503 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-10T21:37:11.503 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:37:11.503 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:37:11.503 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-10T21:37:11.503 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 100, 2024-09-10T21:37:11.503 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 37847, 2024-09-10T21:37:11.503 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-10T21:37:11.503 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-10T21:37:11.503 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-10T21:37:11.503 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:37:11.503 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 0, 2024-09-10T21:37:11.503 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:37:11.503 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-10T21:37:11.503 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-10T21:37:11.504 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:37:11.504 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:37:11.504 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:37:11.504 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:37:11.504 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:37:11.504 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:37:11.504 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:37:11.504 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:37:11.504 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:37:11.504 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:37:11.504 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:37:11.504 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:37:11.504 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-10T21:37:11.504 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-10T21:37:11.505 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:37:11.505 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:37:11.505 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:37:11.505 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:37:11.505 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:37:11.505 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:37:11.505 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:37:11.505 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:37:11.505 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:37:11.505 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:37:11.505 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:37:11.505 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:37:11.505 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:37:11.505 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-10T21:37:11.506 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-10T21:37:11.506 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-10T21:37:11.506 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-10T21:37:11.506 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-10T21:37:11.506 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "5e2f023f-d6a6-49f4-a256-f4807dce3739", 2024-09-10T21:37:11.506 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:37:11.506 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:37:11.506 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-10T21:37:11.506 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-10T21:37:11.506 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-10T21:37:11.506 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-10T21:37:11.506 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-10T21:37:11.506 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-10T21:37:11.507 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-10T21:37:11.507 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:37:11.507 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-10T21:37:11.507 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:37:11.507 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:37:11.507 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:37:11.507 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:37:11.507 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:37:11.507 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:37:11.507 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:37:11.507 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:37:11.507 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:37:11.507 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:37:11.507 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:37:11.507 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:37:11.508 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-10T21:37:11.508 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:37:11.508 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-10T21:37:11.508 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:37:11.508 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:37:11.508 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:37:11.508 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:37:11.508 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:37:11.508 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:37:11.508 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:37:11.508 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-10T21:37:11.508 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:37:11.508 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:37:11.508 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:37:11.508 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:37:11.509 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-10T21:37:11.509 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:37:11.509 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:37:11.509 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:37:11.509 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:37:11.509 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-10T21:37:11.509 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-10T21:37:11.509 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:37:11.509 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:37:11.509 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:37:11.509 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:37:11.509 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:37:11.509 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:37:11.509 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:37:11.510 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:37:11.510 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:37:11.510 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:37:11.510 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:37:11.510 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3300", 2024-09-10T21:37:11.510 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:37:11.510 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:37:11.510 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:37:11.510 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:37:11.510 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789", 2024-09-10T21:37:11.510 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:37:11.510 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:37:11.510 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:37:11.511 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:37:11.511 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789/0", 2024-09-10T21:37:11.511 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6789/0", 2024-09-10T21:37:11.511 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:37:11.511 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:37:11.511 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:37:11.511 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:37:11.511 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:37:11.511 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:37:11.511 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:37:11.511 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:37:11.511 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:37:11.511 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:37:11.511 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:37:11.512 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3301", 2024-09-10T21:37:11.512 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:37:11.512 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:37:11.512 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:37:11.512 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:37:11.512 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790", 2024-09-10T21:37:11.512 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:37:11.512 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:37:11.512 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:37:11.512 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:37:11.512 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790/0", 2024-09-10T21:37:11.512 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6790/0", 2024-09-10T21:37:11.512 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:37:11.512 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:37:11.513 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:37:11.513 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:37:11.513 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:37:11.513 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:37:11.513 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-10T21:37:11.513 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-10T21:37:11.513 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:37:11.513 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:37:11.513 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:37:11.513 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:37:11.513 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:37:11.513 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:37:11.513 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": [ 2024-09-10T21:37:11.513 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:37:11.513 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:37:11.514 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:37:11.514 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 4 2024-09-10T21:37:11.514 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:37:11.514 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:37:11.514 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-10T21:37:11.514 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:37:11.514 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:37:11.514 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:37:11.514 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 4 2024-09-10T21:37:11.514 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:37:11.514 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:37:11.514 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-10T21:37:11.514 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:37:11.514 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:37:11.514 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:37:11.514 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 3 2024-09-10T21:37:11.515 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:37:11.515 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:37:11.515 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": [ 2024-09-10T21:37:11.515 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:37:11.515 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:37:11.515 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:37:11.515 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:37:11.515 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:37:11.515 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:37:11.515 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:37:11.515 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-10T21:37:11.515 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-10T21:37:11.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:11.499+0000 7f51d5800640 1 -- 172.21.15.17:0/962681369 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f51ac072510 msgr2=0x7f51ac0749d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:37:11.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:11.499+0000 7f51d5800640 1 --2- 172.21.15.17:0/962681369 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f51ac072510 0x7f51ac0749d0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f51d00c5550 tx=0x7f51c800d040 comp rx=0 tx=0).stop 2024-09-10T21:37:11.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:11.499+0000 7f51d5800640 1 -- 172.21.15.17:0/962681369 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f51d00c89d0 msgr2=0x7f51d00c6f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:37:11.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:11.499+0000 7f51d5800640 1 --2- 172.21.15.17:0/962681369 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f51d00c89d0 0x7f51d00c6f70 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f51c0009850 tx=0x7f51c00a8d10 comp rx=0 tx=0).stop 2024-09-10T21:37:11.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:11.499+0000 7f51cf400640 1 -- 172.21.15.17:0/962681369 reap_dead start 2024-09-10T21:37:11.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:11.499+0000 7f51cf400640 1 -- 172.21.15.17:0/962681369 reap_dead start 2024-09-10T21:37:11.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:11.499+0000 7f51d5800640 1 -- 172.21.15.17:0/962681369 shutdown_connections 2024-09-10T21:37:11.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:11.499+0000 7f51d5800640 1 -- 172.21.15.17:0/962681369 >> 172.21.15.17:0/962681369 conn(0x7f51d0068b60 msgr2=0x7f51d0157b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:37:11.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:11.499+0000 7f51d5800640 1 -- 172.21.15.17:0/962681369 shutdown_connections 2024-09-10T21:37:11.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:11.499+0000 7f51d5800640 1 -- 172.21.15.17:0/962681369 wait complete. 2024-09-10T21:37:11.521 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-09-10T21:37:11.521 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2024-09-10T21:37:20.865 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:37:20.868 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:37:31.521 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.a 2024-09-10T21:37:31.522 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-10T21:37:31.522 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-10T21:37:31.525 INFO:tasks.ceph.mon.a:Started 2024-09-10T21:37:31.525 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-09-10T21:37:31.525 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-10T21:37:31.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:31.686+0000 7f584f800640 1 -- 172.21.15.17:0/1322625174 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f5848152820 msgr2=0x7f58480b3eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:37:31.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:31.686+0000 7f584f800640 1 --2- 172.21.15.17:0/1322625174 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f5848152820 0x7f58480b3eb0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f5834009900 tx=0x7f583401c850 comp rx=0 tx=0).stop 2024-09-10T21:37:31.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:31.686+0000 7f584f800640 1 -- 172.21.15.17:0/1322625174 shutdown_connections 2024-09-10T21:37:31.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:31.690+0000 7f584f800640 1 --2- 172.21.15.17:0/1322625174 >> v2:172.21.15.31:3300/0 conn(0x7f58480cdf20 0x7f58480ce3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:37:31.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:31.690+0000 7f584f800640 1 --2- 172.21.15.17:0/1322625174 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f5848152820 0x7f58480b3eb0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:37:31.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:31.690+0000 7f584f800640 1 -- 172.21.15.17:0/1322625174 >> 172.21.15.17:0/1322625174 conn(0x7f58480702f0 msgr2=0x7f5848070700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:37:31.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:31.690+0000 7f584f800640 1 -- 172.21.15.17:0/1322625174 shutdown_connections 2024-09-10T21:37:31.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:31.690+0000 7f584f800640 1 -- 172.21.15.17:0/1322625174 wait complete. 2024-09-10T21:37:31.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:31.690+0000 7f584f800640 1 Processor -- start 2024-09-10T21:37:31.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:31.690+0000 7f584f800640 1 -- start start 2024-09-10T21:37:31.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:31.690+0000 7f584f800640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f58480cdf20 0x7f584806f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:37:31.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:31.690+0000 7f584f800640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5848152440 0x7f58481f6660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:37:31.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:31.690+0000 7f584f800640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5848152820 0x7f58480ca1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:37:31.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:31.690+0000 7f584f800640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f58480cb5d0 con 0x7f5848152820 2024-09-10T21:37:31.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:31.690+0000 7f584f800640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f58480cb450 con 0x7f5848152440 2024-09-10T21:37:31.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:31.690+0000 7f584f800640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f58480c2ff0 con 0x7f58480cdf20 2024-09-10T21:37:31.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:31.690+0000 7f584d800640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5848152820 0x7f58480ca1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:37:31.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:31.690+0000 7f584ce00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f58480cdf20 0x7f584806f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:37:31.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:31.690+0000 7f584d800640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5848152820 0x7f58480ca1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:55188/0 (socket says 172.21.15.17:55188) 2024-09-10T21:37:31.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:31.690+0000 7f584d800640 1 -- 172.21.15.17:0/354458016 learned_addr learned my addr 172.21.15.17:0/354458016 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:37:31.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:31.690+0000 7f5847e00640 1 --2- 172.21.15.17:0/354458016 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5848152440 0x7f58481f6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:37:31.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:31.690+0000 7f584d800640 1 -- 172.21.15.17:0/354458016 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5848152820 msgr2=0x7f58480ca1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-09-10T21:37:31.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:31.690+0000 7f584d800640 1 -- 172.21.15.17:0/354458016 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5848152820 msgr2=0x7f58480ca1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-10T21:37:31.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:31.690+0000 7f584d800640 1 --2- 172.21.15.17:0/354458016 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5848152820 0x7f58480ca1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T21:37:31.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:31.690+0000 7f584d800640 1 --2- 172.21.15.17:0/354458016 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5848152820 0x7f58480ca1d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:37:31.696 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:31.690+0000 7f5847e00640 1 -- 172.21.15.17:0/354458016 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f58480cdf20 msgr2=0x7f584806f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:37:31.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:31.690+0000 7f5847e00640 1 --2- 172.21.15.17:0/354458016 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f58480cdf20 0x7f584806f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:37:31.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:31.690+0000 7f5847e00640 1 -- 172.21.15.17:0/354458016 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5848152820 msgr2=0x7f58480ca1d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:37:31.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:31.690+0000 7f5847e00640 1 --2- 172.21.15.17:0/354458016 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5848152820 0x7f58480ca1d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:37:31.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:31.690+0000 7f5847e00640 1 -- 172.21.15.17:0/354458016 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5848162680 con 0x7f5848152440 2024-09-10T21:37:31.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:31.690+0000 7f584ce00640 1 --2- 172.21.15.17:0/354458016 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f58480cdf20 0x7f584806f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:37:31.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:31.690+0000 7f5847e00640 1 --2- 172.21.15.17:0/354458016 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5848152440 0x7f58481f6660 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f58340ac000 tx=0x7f58340a5680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:37:31.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:31.694+0000 7f5844c00640 1 -- 172.21.15.17:0/354458016 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58340b2040 con 0x7f5848152440 2024-09-10T21:37:31.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:31.694+0000 7f5844c00640 1 -- 172.21.15.17:0/354458016 <== mon.1 v2:172.21.15.31:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f58340042e0 con 0x7f5848152440 2024-09-10T21:37:31.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:31.694+0000 7f584f800640 1 -- 172.21.15.17:0/354458016 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58481f3180 con 0x7f5848152440 2024-09-10T21:37:31.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:31.694+0000 7f5844c00640 1 -- 172.21.15.17:0/354458016 <== mon.1 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58340052d0 con 0x7f5848152440 2024-09-10T21:37:31.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:31.694+0000 7f584f800640 1 -- 172.21.15.17:0/354458016 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58481f3720 con 0x7f5848152440 2024-09-10T21:37:31.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:31.694+0000 7f584f800640 1 -- 172.21.15.17:0/354458016 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5848158360 con 0x7f5848152440 2024-09-10T21:37:31.698 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:31.694+0000 7f5844c00640 1 -- 172.21.15.17:0/354458016 <== mon.1 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f58340046d0 con 0x7f5848152440 2024-09-10T21:37:31.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:31.694+0000 7f5844c00640 1 --2- 172.21.15.17:0/354458016 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f5828072630 0x7f5828074af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:37:31.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:31.694+0000 7f5844c00640 1 -- 172.21.15.17:0/354458016 <== mon.1 v2:172.21.15.31:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f583412cff0 con 0x7f5848152440 2024-09-10T21:37:31.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:31.698+0000 7f584ce00640 1 --2- 172.21.15.17:0/354458016 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f5828072630 0x7f5828074af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:37:31.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:31.698+0000 7f584ce00640 1 --2- 172.21.15.17:0/354458016 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f5828072630 0x7f5828074af0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f583c007fd0 tx=0x7f583c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:37:31.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:31.698+0000 7f5844c00640 1 -- 172.21.15.17:0/354458016 <== mon.1 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f58340164d0 con 0x7f5848152440 2024-09-10T21:37:31.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:31.974+0000 7f584f800640 1 -- 172.21.15.17:0/354458016 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f58480cc050 con 0x7f5848152440 2024-09-10T21:37:31.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:31.974+0000 7f5844c00640 1 -- 172.21.15.17:0/354458016 <== mon.1 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1552 (secure 0 0 0) 0x7f58340f82c0 con 0x7f5848152440 2024-09-10T21:37:31.977 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":100,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":21,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5e2f023f-d6a6-49f4-a256-f4807dce3739","modified":"2024-09-10T21:23:46.892130Z","created":"2024-09-10T21:23:46.892130Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3301","nonce":0},{"type":"v1","addr":"172.21.15.31:6790","nonce":0}]},"addr":"172.21.15.31:6790/0","public_addr":"172.21.15.31:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-10T21:37:31.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:31.974+0000 7f584f800640 1 -- 172.21.15.17:0/354458016 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f5828072630 msgr2=0x7f5828074af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:37:31.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:31.974+0000 7f584f800640 1 --2- 172.21.15.17:0/354458016 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f5828072630 0x7f5828074af0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f583c007fd0 tx=0x7f583c009040 comp rx=0 tx=0).stop 2024-09-10T21:37:31.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:31.974+0000 7f584f800640 1 -- 172.21.15.17:0/354458016 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5848152440 msgr2=0x7f58481f6660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:37:31.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:31.974+0000 7f584f800640 1 --2- 172.21.15.17:0/354458016 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5848152440 0x7f58481f6660 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f58340ac000 tx=0x7f58340a5680 comp rx=0 tx=0).stop 2024-09-10T21:37:31.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:31.974+0000 7f584f800640 1 -- 172.21.15.17:0/354458016 shutdown_connections 2024-09-10T21:37:31.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:31.974+0000 7f584f800640 1 --2- 172.21.15.17:0/354458016 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f5828072630 0x7f5828074af0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:37:31.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:31.974+0000 7f584f800640 1 --2- 172.21.15.17:0/354458016 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5848152820 0x7f58480ca1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:37:31.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:31.974+0000 7f584f800640 1 --2- 172.21.15.17:0/354458016 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5848152440 0x7f58481f6660 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:37:31.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:31.974+0000 7f584f800640 1 --2- 172.21.15.17:0/354458016 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f58480cdf20 0x7f584806f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:37:31.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:31.974+0000 7f584f800640 1 -- 172.21.15.17:0/354458016 >> 172.21.15.17:0/354458016 conn(0x7f58480702f0 msgr2=0x7f5848068990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:37:31.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:31.974+0000 7f584f800640 1 -- 172.21.15.17:0/354458016 shutdown_connections 2024-09-10T21:37:31.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:31.974+0000 7f584f800640 1 -- 172.21.15.17:0/354458016 wait complete. 2024-09-10T21:37:35.000 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-10T21:37:35.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:35.154+0000 7f1fb9000640 1 -- 172.21.15.17:0/1515713328 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1f980052f0 msgr2=0x7f1f9800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:37:35.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:35.154+0000 7f1fb9000640 1 --2- 172.21.15.17:0/1515713328 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1f980052f0 0x7f1f9800b9d0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f1fac009df0 tx=0x7f1fac01d170 comp rx=0 tx=0).stop 2024-09-10T21:37:35.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:35.154+0000 7f1fb9000640 1 -- 172.21.15.17:0/1515713328 shutdown_connections 2024-09-10T21:37:35.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:35.154+0000 7f1fb9000640 1 --2- 172.21.15.17:0/1515713328 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1f980052f0 0x7f1f9800b9d0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:37:35.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:35.154+0000 7f1fb9000640 1 -- 172.21.15.17:0/1515713328 >> 172.21.15.17:0/1515713328 conn(0x7f1fb4070ca0 msgr2=0x7f1fb40710b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:37:35.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:35.154+0000 7f1fb9000640 1 -- 172.21.15.17:0/1515713328 shutdown_connections 2024-09-10T21:37:35.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:35.154+0000 7f1fb9000640 1 -- 172.21.15.17:0/1515713328 wait complete. 2024-09-10T21:37:35.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:35.154+0000 7f1fb9000640 1 Processor -- start 2024-09-10T21:37:35.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:35.154+0000 7f1fb9000640 1 -- start start 2024-09-10T21:37:35.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:35.158+0000 7f1fb9000640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1f980052f0 0x7f1fb40c5f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:37:35.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:35.158+0000 7f1fb9000640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1fb41f05f0 0x7f1fb4151b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:37:35.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:35.158+0000 7f1fb9000640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f1fb414f000 0x7f1fb414f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:37:35.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:35.158+0000 7f1fb9000640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f1fb406ed40 con 0x7f1f980052f0 2024-09-10T21:37:35.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:35.158+0000 7f1fb9000640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f1fb406f500 con 0x7f1fb41f05f0 2024-09-10T21:37:35.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:35.158+0000 7f1fb9000640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f1fb406eec0 con 0x7f1fb414f000 2024-09-10T21:37:35.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:35.158+0000 7f1fb3400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1f980052f0 0x7f1fb40c5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:37:35.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:35.158+0000 7f1fb3400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1f980052f0 0x7f1fb40c5f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:48372/0 (socket says 172.21.15.17:48372) 2024-09-10T21:37:35.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:35.158+0000 7f1fb3400640 1 -- 172.21.15.17:0/2774398794 learned_addr learned my addr 172.21.15.17:0/2774398794 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:37:35.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:35.158+0000 7f1fb3400640 1 -- 172.21.15.17:0/2774398794 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1f980052f0 msgr2=0x7f1fb40c5f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-09-10T21:37:35.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:35.158+0000 7f1fb3400640 1 -- 172.21.15.17:0/2774398794 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1f980052f0 msgr2=0x7f1fb40c5f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-10T21:37:35.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:35.158+0000 7f1fb3400640 1 --2- 172.21.15.17:0/2774398794 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1f980052f0 0x7f1fb40c5f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T21:37:35.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:35.158+0000 7f1fb3400640 1 --2- 172.21.15.17:0/2774398794 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1f980052f0 0x7f1fb40c5f00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:37:35.162 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:35.158+0000 7f1fb2a00640 1 --2- 172.21.15.17:0/2774398794 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1fb41f05f0 0x7f1fb4151b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:37:35.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:35.158+0000 7f1fb3e00640 1 --2- 172.21.15.17:0/2774398794 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f1fb414f000 0x7f1fb414f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:37:35.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:35.158+0000 7f1fb2a00640 1 -- 172.21.15.17:0/2774398794 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f1fb414f000 msgr2=0x7f1fb414f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:37:35.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:35.158+0000 7f1fb2a00640 1 --2- 172.21.15.17:0/2774398794 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f1fb414f000 0x7f1fb414f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:37:35.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:35.158+0000 7f1fb2a00640 1 -- 172.21.15.17:0/2774398794 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1f980052f0 msgr2=0x7f1fb40c5f00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:37:35.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:35.158+0000 7f1fb2a00640 1 --2- 172.21.15.17:0/2774398794 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1f980052f0 0x7f1fb40c5f00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:37:35.163 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:35.158+0000 7f1fb2a00640 1 -- 172.21.15.17:0/2774398794 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fb40c6440 con 0x7f1fb41f05f0 2024-09-10T21:37:35.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:35.158+0000 7f1fb2a00640 1 --2- 172.21.15.17:0/2774398794 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1fb41f05f0 0x7f1fb4151b70 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f1fa0003c90 tx=0x7f1fa0001660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:37:35.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:35.158+0000 7f1fab400640 1 -- 172.21.15.17:0/2774398794 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fa0015070 con 0x7f1fb41f05f0 2024-09-10T21:37:35.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:35.158+0000 7f1fab400640 1 -- 172.21.15.17:0/2774398794 <== mon.1 v2:172.21.15.31:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1fa0011630 con 0x7f1fb41f05f0 2024-09-10T21:37:35.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:35.158+0000 7f1fab400640 1 -- 172.21.15.17:0/2774398794 <== mon.1 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fa0011a60 con 0x7f1fb41f05f0 2024-09-10T21:37:35.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:35.158+0000 7f1fb9000640 1 -- 172.21.15.17:0/2774398794 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fb41e2e10 con 0x7f1fb41f05f0 2024-09-10T21:37:35.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:35.158+0000 7f1fb9000640 1 -- 172.21.15.17:0/2774398794 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fb41e3420 con 0x7f1fb41f05f0 2024-09-10T21:37:35.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:35.158+0000 7f1fb9000640 1 -- 172.21.15.17:0/2774398794 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1fb406fd00 con 0x7f1fb41f05f0 2024-09-10T21:37:35.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:35.162+0000 7f1fab400640 1 -- 172.21.15.17:0/2774398794 <== mon.1 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f1fa0011c00 con 0x7f1fb41f05f0 2024-09-10T21:37:35.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:35.162+0000 7f1fab400640 1 --2- 172.21.15.17:0/2774398794 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f1f94072660 0x7f1f94074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:37:35.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:35.162+0000 7f1fab400640 1 -- 172.21.15.17:0/2774398794 <== mon.1 v2:172.21.15.31:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f1fa0093fc0 con 0x7f1fb41f05f0 2024-09-10T21:37:35.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:35.166+0000 7f1fb3400640 1 --2- 172.21.15.17:0/2774398794 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f1f94072660 0x7f1f94074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:37:35.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:35.166+0000 7f1fb3400640 1 --2- 172.21.15.17:0/2774398794 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f1f94072660 0x7f1f94074b20 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f1fa4000fc0 tx=0x7f1fa4001e60 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:37:35.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:35.166+0000 7f1fab400640 1 -- 172.21.15.17:0/2774398794 <== mon.1 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f1fa005eca0 con 0x7f1fb41f05f0 2024-09-10T21:37:35.483 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:35.478+0000 7f1fb9000640 1 -- 172.21.15.17:0/2774398794 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f1fb4155d30 con 0x7f1fb41f05f0 2024-09-10T21:37:35.484 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:35.482+0000 7f1fab400640 1 -- 172.21.15.17:0/2774398794 <== mon.1 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1552 (secure 0 0 0) 0x7f1fa00636d0 con 0x7f1fb41f05f0 2024-09-10T21:37:35.484 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":100,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":24,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5e2f023f-d6a6-49f4-a256-f4807dce3739","modified":"2024-09-10T21:23:46.892130Z","created":"2024-09-10T21:23:46.892130Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3301","nonce":0},{"type":"v1","addr":"172.21.15.31:6790","nonce":0}]},"addr":"172.21.15.31:6790/0","public_addr":"172.21.15.31:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-10T21:37:35.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:35.482+0000 7f1fb9000640 1 -- 172.21.15.17:0/2774398794 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f1f94072660 msgr2=0x7f1f94074b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:37:35.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:35.482+0000 7f1fb9000640 1 --2- 172.21.15.17:0/2774398794 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f1f94072660 0x7f1f94074b20 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f1fa4000fc0 tx=0x7f1fa4001e60 comp rx=0 tx=0).stop 2024-09-10T21:37:35.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:35.482+0000 7f1fb9000640 1 -- 172.21.15.17:0/2774398794 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1fb41f05f0 msgr2=0x7f1fb4151b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:37:35.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:35.482+0000 7f1fb9000640 1 --2- 172.21.15.17:0/2774398794 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1fb41f05f0 0x7f1fb4151b70 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f1fa0003c90 tx=0x7f1fa0001660 comp rx=0 tx=0).stop 2024-09-10T21:37:35.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:35.482+0000 7f1fb9000640 1 -- 172.21.15.17:0/2774398794 shutdown_connections 2024-09-10T21:37:35.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:35.482+0000 7f1fb9000640 1 --2- 172.21.15.17:0/2774398794 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f1f94072660 0x7f1f94074b20 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:37:35.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:35.482+0000 7f1fb9000640 1 --2- 172.21.15.17:0/2774398794 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f1fb414f000 0x7f1fb414f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:37:35.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:35.482+0000 7f1fb9000640 1 --2- 172.21.15.17:0/2774398794 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1fb41f05f0 0x7f1fb4151b70 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:37:35.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:35.482+0000 7f1fb9000640 1 --2- 172.21.15.17:0/2774398794 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1f980052f0 0x7f1fb40c5f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:37:35.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:35.482+0000 7f1fb9000640 1 -- 172.21.15.17:0/2774398794 >> 172.21.15.17:0/2774398794 conn(0x7f1fb4070ca0 msgr2=0x7f1fb4073b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:37:35.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:35.482+0000 7f1fb9000640 1 -- 172.21.15.17:0/2774398794 shutdown_connections 2024-09-10T21:37:35.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:35.482+0000 7f1fb9000640 1 -- 172.21.15.17:0/2774398794 wait complete. 2024-09-10T21:37:38.506 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-10T21:37:48.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:48.274+0000 7efec1c00640 1 -- 172.21.15.17:0/3582598907 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7efe9c006cd0 msgr2=0x7efe9c0140b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:37:48.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:48.274+0000 7efec1c00640 1 --2- 172.21.15.17:0/3582598907 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7efe9c006cd0 0x7efe9c0140b0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7efebc0b3630 tx=0x7efea8016ed0 comp rx=0 tx=0).stop 2024-09-10T21:37:48.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:48.274+0000 7efec1200640 1 -- 172.21.15.17:0/3582598907 reap_dead start 2024-09-10T21:37:48.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:48.274+0000 7efec1c00640 1 -- 172.21.15.17:0/3582598907 shutdown_connections 2024-09-10T21:37:48.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:48.274+0000 7efec1c00640 1 -- 172.21.15.17:0/3582598907 >> 172.21.15.17:0/3582598907 conn(0x7efebc0cdef0 msgr2=0x7efebc0ce2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:37:48.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:48.274+0000 7efec1c00640 1 -- 172.21.15.17:0/3582598907 shutdown_connections 2024-09-10T21:37:48.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:48.274+0000 7efec1c00640 1 -- 172.21.15.17:0/3582598907 wait complete. 2024-09-10T21:37:48.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:48.274+0000 7efec1c00640 1 Processor -- start 2024-09-10T21:37:48.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:48.274+0000 7efec1c00640 1 -- start start 2024-09-10T21:37:48.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:48.278+0000 7efec1c00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efe9c006cd0 0x7efebc153aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:37:48.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:48.278+0000 7efec1c00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7efebc1f2710 0x7efebc0ccfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:37:48.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:48.278+0000 7efec1c00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efebc1459d0 0x7efebc145e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:37:48.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:48.278+0000 7efec1c00640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7efebc0c9350 con 0x7efe9c006cd0 2024-09-10T21:37:48.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:48.278+0000 7efec1c00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7efebc15b9e0 con 0x7efebc1459d0 2024-09-10T21:37:48.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:48.278+0000 7efec1c00640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7efebc0cea50 con 0x7efebc1f2710 2024-09-10T21:37:48.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:48.278+0000 7efebbe00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efe9c006cd0 0x7efebc153aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:37:48.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:48.278+0000 7efebbe00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efe9c006cd0 0x7efebc153aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:56620/0 (socket says 172.21.15.17:56620) 2024-09-10T21:37:48.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:48.278+0000 7efebbe00640 1 -- 172.21.15.17:0/2474959692 learned_addr learned my addr 172.21.15.17:0/2474959692 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:37:48.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:48.278+0000 7efebbe00640 1 -- 172.21.15.17:0/2474959692 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efe9c006cd0 msgr2=0x7efebc153aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-09-10T21:37:48.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:48.278+0000 7efebbe00640 1 -- 172.21.15.17:0/2474959692 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efe9c006cd0 msgr2=0x7efebc153aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-10T21:37:48.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:48.278+0000 7efebbe00640 1 --2- 172.21.15.17:0/2474959692 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efe9c006cd0 0x7efebc153aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T21:37:48.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:48.278+0000 7efec1200640 1 --2- 172.21.15.17:0/2474959692 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efebc1459d0 0x7efebc145e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:37:48.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:48.278+0000 7efebb400640 1 --2- 172.21.15.17:0/2474959692 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7efebc1f2710 0x7efebc0ccfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:37:48.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:48.278+0000 7efebbe00640 1 --2- 172.21.15.17:0/2474959692 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efe9c006cd0 0x7efebc153aa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:37:48.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:48.278+0000 7efec1200640 1 -- 172.21.15.17:0/2474959692 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7efebc1f2710 msgr2=0x7efebc0ccfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:37:48.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:48.278+0000 7efec1200640 1 --2- 172.21.15.17:0/2474959692 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7efebc1f2710 0x7efebc0ccfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:37:48.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:48.278+0000 7efec1200640 1 -- 172.21.15.17:0/2474959692 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efe9c006cd0 msgr2=0x7efebc153aa0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:37:48.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:48.278+0000 7efec1200640 1 --2- 172.21.15.17:0/2474959692 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efe9c006cd0 0x7efebc153aa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:37:48.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:48.278+0000 7efec1200640 1 -- 172.21.15.17:0/2474959692 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efebc15b3d0 con 0x7efebc1459d0 2024-09-10T21:37:48.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:48.278+0000 7efebb400640 1 --2- 172.21.15.17:0/2474959692 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7efebc1f2710 0x7efebc0ccfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:37:48.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:48.278+0000 7efec1200640 1 --2- 172.21.15.17:0/2474959692 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efebc1459d0 0x7efebc145e50 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7efe9c009e50 tx=0x7efeb400c8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:37:48.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:48.278+0000 7efeb3e00640 1 -- 172.21.15.17:0/2474959692 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeb4003980 con 0x7efebc1459d0 2024-09-10T21:37:48.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:48.278+0000 7efeb3e00640 1 -- 172.21.15.17:0/2474959692 <== mon.1 v2:172.21.15.31:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efeb4003b40 con 0x7efebc1459d0 2024-09-10T21:37:48.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:48.278+0000 7efeb3e00640 1 -- 172.21.15.17:0/2474959692 <== mon.1 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeb4004020 con 0x7efebc1459d0 2024-09-10T21:37:48.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:48.278+0000 7efec1c00640 1 -- 172.21.15.17:0/2474959692 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efebc15a850 con 0x7efebc1459d0 2024-09-10T21:37:48.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:48.278+0000 7efec1c00640 1 -- 172.21.15.17:0/2474959692 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efebc15ae10 con 0x7efebc1459d0 2024-09-10T21:37:48.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:48.278+0000 7efec1c00640 1 -- 172.21.15.17:0/2474959692 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efebc0b3f70 con 0x7efebc1459d0 2024-09-10T21:37:48.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:48.282+0000 7efeb3e00640 1 -- 172.21.15.17:0/2474959692 <== mon.1 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7efeb40041c0 con 0x7efebc1459d0 2024-09-10T21:37:48.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:48.282+0000 7efeb3e00640 1 --2- 172.21.15.17:0/2474959692 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7efe9c0775f0 0x7efe9c079ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:37:48.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:48.282+0000 7efeb3e00640 1 -- 172.21.15.17:0/2474959692 <== mon.1 v2:172.21.15.31:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7efeb4092750 con 0x7efebc1459d0 2024-09-10T21:37:48.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:48.282+0000 7efebbe00640 1 --2- 172.21.15.17:0/2474959692 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7efe9c0775f0 0x7efe9c079ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:37:48.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:48.282+0000 7efebbe00640 1 --2- 172.21.15.17:0/2474959692 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7efe9c0775f0 0x7efe9c079ab0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7efebc0c86d0 tx=0x7efeac014040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:37:48.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:48.286+0000 7efeb3e00640 1 -- 172.21.15.17:0/2474959692 <== mon.1 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7efebc1528b0 con 0x7efebc1459d0 2024-09-10T21:37:48.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:48.550+0000 7efec1c00640 1 -- 172.21.15.17:0/2474959692 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7efebc15a630 con 0x7efebc1459d0 2024-09-10T21:37:48.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:48.550+0000 7efeb3e00640 1 -- 172.21.15.17:0/2474959692 <== mon.1 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1551 (secure 0 0 0) 0x7efeb405daa0 con 0x7efebc1459d0 2024-09-10T21:37:48.554 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":106,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":0,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5e2f023f-d6a6-49f4-a256-f4807dce3739","modified":"2024-09-10T21:23:46.892130Z","created":"2024-09-10T21:23:46.892130Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3301","nonce":0},{"type":"v1","addr":"172.21.15.31:6790","nonce":0}]},"addr":"172.21.15.31:6790/0","public_addr":"172.21.15.31:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-10T21:37:48.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:48.554+0000 7efec1c00640 1 -- 172.21.15.17:0/2474959692 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7efe9c0775f0 msgr2=0x7efe9c079ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:37:48.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:48.554+0000 7efec1c00640 1 --2- 172.21.15.17:0/2474959692 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7efe9c0775f0 0x7efe9c079ab0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7efebc0c86d0 tx=0x7efeac014040 comp rx=0 tx=0).stop 2024-09-10T21:37:48.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:48.554+0000 7efec1c00640 1 -- 172.21.15.17:0/2474959692 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efebc1459d0 msgr2=0x7efebc145e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:37:48.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:48.554+0000 7efec1c00640 1 --2- 172.21.15.17:0/2474959692 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efebc1459d0 0x7efebc145e50 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7efe9c009e50 tx=0x7efeb400c8e0 comp rx=0 tx=0).stop 2024-09-10T21:37:48.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:48.554+0000 7efec1c00640 1 -- 172.21.15.17:0/2474959692 shutdown_connections 2024-09-10T21:37:48.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:48.554+0000 7efec1c00640 1 --2- 172.21.15.17:0/2474959692 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7efe9c0775f0 0x7efe9c079ab0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:37:48.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:48.554+0000 7efec1c00640 1 --2- 172.21.15.17:0/2474959692 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efebc1459d0 0x7efebc145e50 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:37:48.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:48.554+0000 7efec1c00640 1 --2- 172.21.15.17:0/2474959692 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7efebc1f2710 0x7efebc0ccfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:37:48.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:48.554+0000 7efec1c00640 1 --2- 172.21.15.17:0/2474959692 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efe9c006cd0 0x7efebc153aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:37:48.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:48.554+0000 7efec1c00640 1 -- 172.21.15.17:0/2474959692 >> 172.21.15.17:0/2474959692 conn(0x7efebc0cdef0 msgr2=0x7efebc14fd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:37:48.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:48.554+0000 7efec1c00640 1 -- 172.21.15.17:0/2474959692 shutdown_connections 2024-09-10T21:37:48.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:48.554+0000 7efec1c00640 1 -- 172.21.15.17:0/2474959692 wait complete. 2024-09-10T21:37:50.951 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:37:50.955 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:37:51.579 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-10T21:37:51.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:51.754+0000 7faed7000640 1 -- 172.21.15.17:0/3561715037 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7faed0152820 msgr2=0x7faed015c280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:37:51.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:51.754+0000 7faed7000640 1 --2- 172.21.15.17:0/3561715037 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7faed0152820 0x7faed015c280 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7faebc009900 tx=0x7faebc01c850 comp rx=0 tx=0).stop 2024-09-10T21:37:51.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:51.754+0000 7faed7000640 1 -- 172.21.15.17:0/3561715037 shutdown_connections 2024-09-10T21:37:51.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:51.754+0000 7faed7000640 1 --2- 172.21.15.17:0/3561715037 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7faed0152820 0x7faed015c280 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:37:51.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:51.754+0000 7faed7000640 1 --2- 172.21.15.17:0/3561715037 >> v2:172.21.15.31:3300/0 conn(0x7faed0152440 0x7faed00b3eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:37:51.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:51.754+0000 7faed7000640 1 -- 172.21.15.17:0/3561715037 >> 172.21.15.17:0/3561715037 conn(0x7faed00702f0 msgr2=0x7faed0070700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:37:51.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:51.754+0000 7faed7000640 1 -- 172.21.15.17:0/3561715037 shutdown_connections 2024-09-10T21:37:51.758 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:51.754+0000 7faed7000640 1 -- 172.21.15.17:0/3561715037 wait complete. 2024-09-10T21:37:51.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:51.758+0000 7faed7000640 1 Processor -- start 2024-09-10T21:37:51.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:51.758+0000 7faed7000640 1 -- start start 2024-09-10T21:37:51.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:51.758+0000 7faed7000640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faed00cdf20 0x7faed014a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:37:51.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:51.758+0000 7faed7000640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7faed0152440 0x7faed006f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:37:51.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:51.758+0000 7faed7000640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faed0152820 0x7faed006f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:37:51.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:51.758+0000 7faed7000640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7faed015c7c0 con 0x7faed00cdf20 2024-09-10T21:37:51.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:51.758+0000 7faed7000640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7faed00ce740 con 0x7faed0152820 2024-09-10T21:37:51.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:51.758+0000 7faed7000640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7faed00cb450 con 0x7faed0152440 2024-09-10T21:37:51.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:51.758+0000 7faecfe00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faed00cdf20 0x7faed014a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:37:51.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:51.758+0000 7faecfe00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faed00cdf20 0x7faed014a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:56634/0 (socket says 172.21.15.17:56634) 2024-09-10T21:37:51.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:51.758+0000 7faecfe00640 1 -- 172.21.15.17:0/1643254186 learned_addr learned my addr 172.21.15.17:0/1643254186 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:37:51.764 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:51.758+0000 7faecfe00640 1 -- 172.21.15.17:0/1643254186 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faed00cdf20 msgr2=0x7faed014a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-09-10T21:37:51.764 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:51.758+0000 7faecfe00640 1 -- 172.21.15.17:0/1643254186 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faed00cdf20 msgr2=0x7faed014a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-10T21:37:51.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:51.758+0000 7faecfe00640 1 --2- 172.21.15.17:0/1643254186 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faed00cdf20 0x7faed014a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T21:37:51.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:51.758+0000 7faecfe00640 1 --2- 172.21.15.17:0/1643254186 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faed00cdf20 0x7faed014a290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:37:51.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:51.758+0000 7faecf400640 1 --2- 172.21.15.17:0/1643254186 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7faed0152440 0x7faed006f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:37:51.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:51.762+0000 7faed5000640 1 --2- 172.21.15.17:0/1643254186 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faed0152820 0x7faed006f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:37:51.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:51.762+0000 7faecf400640 1 -- 172.21.15.17:0/1643254186 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faed0152820 msgr2=0x7faed006f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:37:51.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:51.762+0000 7faecf400640 1 --2- 172.21.15.17:0/1643254186 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faed0152820 0x7faed006f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:37:51.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:51.762+0000 7faecf400640 1 -- 172.21.15.17:0/1643254186 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faed00cdf20 msgr2=0x7faed014a290 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:37:51.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:51.762+0000 7faecf400640 1 --2- 172.21.15.17:0/1643254186 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faed00cdf20 0x7faed014a290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:37:51.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:51.762+0000 7faecf400640 1 -- 172.21.15.17:0/1643254186 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faed014f5e0 con 0x7faed0152440 2024-09-10T21:37:51.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:51.762+0000 7faecf400640 1 --2- 172.21.15.17:0/1643254186 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7faed0152440 0x7faed006f370 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7faebc0ac000 tx=0x7faebc005e50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:37:51.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:51.762+0000 7faec7e00640 1 -- 172.21.15.17:0/1643254186 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faebc0b2040 con 0x7faed0152440 2024-09-10T21:37:51.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:51.762+0000 7faec7e00640 1 -- 172.21.15.17:0/1643254186 <== mon.2 v2:172.21.15.31:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faebc004320 con 0x7faed0152440 2024-09-10T21:37:51.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:51.762+0000 7faed7000640 1 -- 172.21.15.17:0/1643254186 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faed01f3690 con 0x7faed0152440 2024-09-10T21:37:51.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:51.762+0000 7faec7e00640 1 -- 172.21.15.17:0/1643254186 <== mon.2 v2:172.21.15.31:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faebc0044f0 con 0x7faed0152440 2024-09-10T21:37:51.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:51.762+0000 7faed7000640 1 -- 172.21.15.17:0/1643254186 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7faed01f3c50 con 0x7faed0152440 2024-09-10T21:37:51.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:51.762+0000 7faed7000640 1 -- 172.21.15.17:0/1643254186 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faed00c37b0 con 0x7faed0152440 2024-09-10T21:37:51.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:51.766+0000 7faec7e00640 1 -- 172.21.15.17:0/1643254186 <== mon.2 v2:172.21.15.31:3301/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7faebc0052d0 con 0x7faed0152440 2024-09-10T21:37:51.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:51.766+0000 7faec7e00640 1 --2- 172.21.15.17:0/1643254186 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7faeb0072610 0x7faeb0074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:37:51.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:51.766+0000 7faec7e00640 1 -- 172.21.15.17:0/1643254186 <== mon.2 v2:172.21.15.31:3301/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7faebc12cf20 con 0x7faed0152440 2024-09-10T21:37:51.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:51.766+0000 7faecfe00640 1 --2- 172.21.15.17:0/1643254186 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7faeb0072610 0x7faeb0074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:37:51.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:51.766+0000 7faecfe00640 1 --2- 172.21.15.17:0/1643254186 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7faeb0072610 0x7faeb0074ad0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7faec0009660 tx=0x7faec00091e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:37:51.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:51.770+0000 7faec7e00640 1 -- 172.21.15.17:0/1643254186 <== mon.2 v2:172.21.15.31:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7faebc0164d0 con 0x7faed0152440 2024-09-10T21:37:52.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:52.034+0000 7faed7000640 1 -- 172.21.15.17:0/1643254186 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7faed006c7e0 con 0x7faed0152440 2024-09-10T21:37:52.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:52.034+0000 7faec7e00640 1 -- 172.21.15.17:0/1643254186 <== mon.2 v2:172.21.15.31:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1551 (secure 0 0 0) 0x7faebc0f8160 con 0x7faed0152440 2024-09-10T21:37:52.037 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":106,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":4,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5e2f023f-d6a6-49f4-a256-f4807dce3739","modified":"2024-09-10T21:23:46.892130Z","created":"2024-09-10T21:23:46.892130Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3301","nonce":0},{"type":"v1","addr":"172.21.15.31:6790","nonce":0}]},"addr":"172.21.15.31:6790/0","public_addr":"172.21.15.31:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-10T21:37:52.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:52.034+0000 7faed7000640 1 -- 172.21.15.17:0/1643254186 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7faeb0072610 msgr2=0x7faeb0074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:37:52.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:52.034+0000 7faed7000640 1 --2- 172.21.15.17:0/1643254186 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7faeb0072610 0x7faeb0074ad0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7faec0009660 tx=0x7faec00091e0 comp rx=0 tx=0).stop 2024-09-10T21:37:52.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:52.034+0000 7faed7000640 1 -- 172.21.15.17:0/1643254186 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7faed0152440 msgr2=0x7faed006f370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:37:52.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:52.034+0000 7faed7000640 1 --2- 172.21.15.17:0/1643254186 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7faed0152440 0x7faed006f370 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7faebc0ac000 tx=0x7faebc005e50 comp rx=0 tx=0).stop 2024-09-10T21:37:52.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:52.038+0000 7faed7000640 1 -- 172.21.15.17:0/1643254186 shutdown_connections 2024-09-10T21:37:52.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:52.038+0000 7faed7000640 1 --2- 172.21.15.17:0/1643254186 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7faeb0072610 0x7faeb0074ad0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:37:52.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:52.038+0000 7faed7000640 1 --2- 172.21.15.17:0/1643254186 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faed0152820 0x7faed006f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:37:52.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:52.038+0000 7faed7000640 1 --2- 172.21.15.17:0/1643254186 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7faed0152440 0x7faed006f370 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:37:52.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:52.038+0000 7faed7000640 1 --2- 172.21.15.17:0/1643254186 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7faed00cdf20 0x7faed014a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:37:52.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:52.038+0000 7faed7000640 1 -- 172.21.15.17:0/1643254186 >> 172.21.15.17:0/1643254186 conn(0x7faed00702f0 msgr2=0x7faed014eed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:37:52.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:52.038+0000 7faed7000640 1 -- 172.21.15.17:0/1643254186 shutdown_connections 2024-09-10T21:37:52.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:52.038+0000 7faed7000640 1 -- 172.21.15.17:0/1643254186 wait complete. 2024-09-10T21:37:55.060 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-10T21:37:57.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.102+0000 7f9871400640 1 -- 172.21.15.17:0/489707669 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9850003a60 msgr2=0x7f9850003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:37:57.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.102+0000 7f9871400640 1 --2- 172.21.15.17:0/489707669 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9850003a60 0x7f9850003e80 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f985c005e60 tx=0x7f985c01dad0 comp rx=0 tx=0).stop 2024-09-10T21:37:57.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.102+0000 7f9870a00640 1 -- 172.21.15.17:0/489707669 reap_dead start 2024-09-10T21:37:57.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.102+0000 7f9871400640 1 -- 172.21.15.17:0/489707669 shutdown_connections 2024-09-10T21:37:57.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.102+0000 7f9871400640 1 -- 172.21.15.17:0/489707669 >> 172.21.15.17:0/489707669 conn(0x7f986c070ca0 msgr2=0x7f986c0710b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:37:57.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.102+0000 7f9871400640 1 -- 172.21.15.17:0/489707669 shutdown_connections 2024-09-10T21:37:57.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.102+0000 7f9871400640 1 -- 172.21.15.17:0/489707669 wait complete. 2024-09-10T21:37:57.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.102+0000 7f9871400640 1 Processor -- start 2024-09-10T21:37:57.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.102+0000 7f9871400640 1 -- start start 2024-09-10T21:37:57.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.102+0000 7f9871400640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f9850003a60 0x7f986c073f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:37:57.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.102+0000 7f9871400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f986c15c5d0 0x7f986c154d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:37:57.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.102+0000 7f9871400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f986c155240 0x7f986c14f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:37:57.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.102+0000 7f9871400640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f986c06f500 con 0x7f986c15c5d0 2024-09-10T21:37:57.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.102+0000 7f9871400640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f986c06f220 con 0x7f986c155240 2024-09-10T21:37:57.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.102+0000 7f9871400640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f986c06ed40 con 0x7f9850003a60 2024-09-10T21:37:57.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.102+0000 7f986b400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f986c15c5d0 0x7f986c154d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:37:57.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.102+0000 7f986b400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f986c15c5d0 0x7f986c154d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45656/0 (socket says 172.21.15.17:45656) 2024-09-10T21:37:57.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.102+0000 7f986b400640 1 -- 172.21.15.17:0/2486309002 learned_addr learned my addr 172.21.15.17:0/2486309002 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:37:57.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.102+0000 7f986be00640 1 --2- 172.21.15.17:0/2486309002 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f9850003a60 0x7f986c073f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:37:57.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.106+0000 7f9870a00640 1 --2- 172.21.15.17:0/2486309002 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f986c155240 0x7f986c14f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:37:57.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.106+0000 7f986b400640 1 -- 172.21.15.17:0/2486309002 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f9850003a60 msgr2=0x7f986c073f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:37:57.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.106+0000 7f986b400640 1 --2- 172.21.15.17:0/2486309002 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f9850003a60 0x7f986c073f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:37:57.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.106+0000 7f986b400640 1 -- 172.21.15.17:0/2486309002 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f986c155240 msgr2=0x7f986c14f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:37:57.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.106+0000 7f986b400640 1 --2- 172.21.15.17:0/2486309002 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f986c155240 0x7f986c14f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:37:57.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.106+0000 7f986b400640 1 -- 172.21.15.17:0/2486309002 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f986c159180 con 0x7f986c15c5d0 2024-09-10T21:37:57.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.106+0000 7f986b400640 1 --2- 172.21.15.17:0/2486309002 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f986c15c5d0 0x7f986c154d00 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f9858001820 tx=0x7f985800e770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:37:57.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.106+0000 7f9863e00640 1 -- 172.21.15.17:0/2486309002 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9858012850 con 0x7f986c15c5d0 2024-09-10T21:37:57.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.106+0000 7f9863e00640 1 -- 172.21.15.17:0/2486309002 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9858012ae0 con 0x7f986c15c5d0 2024-09-10T21:37:57.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.106+0000 7f9871400640 1 -- 172.21.15.17:0/2486309002 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f986c152d60 con 0x7f986c15c5d0 2024-09-10T21:37:57.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.106+0000 7f9863e00640 1 -- 172.21.15.17:0/2486309002 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f985801a400 con 0x7f986c15c5d0 2024-09-10T21:37:57.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.106+0000 7f9871400640 1 -- 172.21.15.17:0/2486309002 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f986c154500 con 0x7f986c15c5d0 2024-09-10T21:37:57.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.106+0000 7f9871400640 1 -- 172.21.15.17:0/2486309002 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f986c158310 con 0x7f986c15c5d0 2024-09-10T21:37:57.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.110+0000 7f9863e00640 1 -- 172.21.15.17:0/2486309002 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f98580233f0 con 0x7f986c15c5d0 2024-09-10T21:37:57.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.110+0000 7f9863e00640 1 --2- 172.21.15.17:0/2486309002 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f984c072610 0x7f984c074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:37:57.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.110+0000 7f9863e00640 1 -- 172.21.15.17:0/2486309002 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f9858096990 con 0x7f986c15c5d0 2024-09-10T21:37:57.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.110+0000 7f986be00640 1 --2- 172.21.15.17:0/2486309002 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f984c072610 0x7f984c074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:37:57.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.110+0000 7f986be00640 1 --2- 172.21.15.17:0/2486309002 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f984c072610 0x7f984c074ad0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f985c0b1000 tx=0x7f985c0a8040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:37:57.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.114+0000 7f9863e00640 1 -- 172.21.15.17:0/2486309002 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f9858016030 con 0x7f986c15c5d0 2024-09-10T21:37:57.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.414+0000 7f9871400640 1 -- 172.21.15.17:0/2486309002 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f986c0cadb0 con 0x7f986c15c5d0 2024-09-10T21:37:57.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.414+0000 7f9863e00640 1 -- 172.21.15.17:0/2486309002 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1557 (secure 0 0 0) 0x7f985806a420 con 0x7f986c15c5d0 2024-09-10T21:37:57.419 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":110,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5e2f023f-d6a6-49f4-a256-f4807dce3739","modified":"2024-09-10T21:23:46.892130Z","created":"2024-09-10T21:23:46.892130Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3301","nonce":0},{"type":"v1","addr":"172.21.15.31:6790","nonce":0}]},"addr":"172.21.15.31:6790/0","public_addr":"172.21.15.31:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-10T21:37:57.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.418+0000 7f9871400640 1 -- 172.21.15.17:0/2486309002 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f984c072610 msgr2=0x7f984c074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:37:57.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.418+0000 7f9871400640 1 --2- 172.21.15.17:0/2486309002 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f984c072610 0x7f984c074ad0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f985c0b1000 tx=0x7f985c0a8040 comp rx=0 tx=0).stop 2024-09-10T21:37:57.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.418+0000 7f9871400640 1 -- 172.21.15.17:0/2486309002 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f986c15c5d0 msgr2=0x7f986c154d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:37:57.421 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.418+0000 7f9871400640 1 --2- 172.21.15.17:0/2486309002 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f986c15c5d0 0x7f986c154d00 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f9858001820 tx=0x7f985800e770 comp rx=0 tx=0).stop 2024-09-10T21:37:57.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.418+0000 7f9871400640 1 -- 172.21.15.17:0/2486309002 shutdown_connections 2024-09-10T21:37:57.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.418+0000 7f9871400640 1 --2- 172.21.15.17:0/2486309002 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f984c072610 0x7f984c074ad0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:37:57.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.418+0000 7f9871400640 1 --2- 172.21.15.17:0/2486309002 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f986c155240 0x7f986c14f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:37:57.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.418+0000 7f9871400640 1 --2- 172.21.15.17:0/2486309002 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f986c15c5d0 0x7f986c154d00 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:37:57.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.418+0000 7f9871400640 1 --2- 172.21.15.17:0/2486309002 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f9850003a60 0x7f986c073f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:37:57.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.418+0000 7f9871400640 1 -- 172.21.15.17:0/2486309002 >> 172.21.15.17:0/2486309002 conn(0x7f986c070ca0 msgr2=0x7f986c1571c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:37:57.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.418+0000 7f9871400640 1 -- 172.21.15.17:0/2486309002 shutdown_connections 2024-09-10T21:37:57.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.418+0000 7f9871400640 1 -- 172.21.15.17:0/2486309002 wait complete. 2024-09-10T21:37:57.445 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-09-10T21:37:57.445 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-09-10T21:37:57.445 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-10T21:37:57.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.614+0000 7efe1e200640 1 -- 172.21.15.17:0/3349928258 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efe180c7ea0 msgr2=0x7efe180725c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:37:57.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.614+0000 7efe1e200640 1 --2- 172.21.15.17:0/3349928258 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efe180c7ea0 0x7efe180725c0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7efe180c9860 tx=0x7efe0401f090 comp rx=0 tx=0).stop 2024-09-10T21:37:57.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.614+0000 7efe1e200640 1 -- 172.21.15.17:0/3349928258 shutdown_connections 2024-09-10T21:37:57.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.614+0000 7efe1e200640 1 --2- 172.21.15.17:0/3349928258 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7efe18072b00 0x7efe18074770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:37:57.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.614+0000 7efe1e200640 1 --2- 172.21.15.17:0/3349928258 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efe180c7ea0 0x7efe180725c0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:37:57.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.614+0000 7efe1e200640 1 --2- 172.21.15.17:0/3349928258 >> v2:172.21.15.31:3300/0 conn(0x7efe18151530 0x7efe180c2c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:37:57.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.614+0000 7efe1e200640 1 -- 172.21.15.17:0/3349928258 >> 172.21.15.17:0/3349928258 conn(0x7efe180700d0 msgr2=0x7efe180704e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:37:57.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.614+0000 7efe1e200640 1 -- 172.21.15.17:0/3349928258 shutdown_connections 2024-09-10T21:37:57.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.614+0000 7efe1e200640 1 -- 172.21.15.17:0/3349928258 wait complete. 2024-09-10T21:37:57.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.618+0000 7efe1e200640 1 Processor -- start 2024-09-10T21:37:57.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.618+0000 7efe1e200640 1 -- start start 2024-09-10T21:37:57.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.618+0000 7efe1e200640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efe18072b00 0x7efe181561c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:37:57.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.618+0000 7efe1e200640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7efe180c7ea0 0x7efe181e5850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:37:57.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.618+0000 7efe1e200640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe18151530 0x7efe181e5d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:37:57.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.618+0000 7efe1e200640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7efe180c3140 con 0x7efe18072b00 2024-09-10T21:37:57.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.618+0000 7efe1e200640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7efe180c9310 con 0x7efe18151530 2024-09-10T21:37:57.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.618+0000 7efe1e200640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7efe180c1670 con 0x7efe180c7ea0 2024-09-10T21:37:57.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.618+0000 7efe17400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efe18072b00 0x7efe181561c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:37:57.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.618+0000 7efe17400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efe18072b00 0x7efe181561c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45674/0 (socket says 172.21.15.17:45674) 2024-09-10T21:37:57.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.618+0000 7efe17400640 1 -- 172.21.15.17:0/917188428 learned_addr learned my addr 172.21.15.17:0/917188428 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:37:57.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.618+0000 7efe16a00640 1 --2- 172.21.15.17:0/917188428 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7efe180c7ea0 0x7efe181e5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:37:57.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.618+0000 7efe17e00640 1 --2- 172.21.15.17:0/917188428 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe18151530 0x7efe181e5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:37:57.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.618+0000 7efe17400640 1 -- 172.21.15.17:0/917188428 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7efe180c7ea0 msgr2=0x7efe181e5850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:37:57.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.618+0000 7efe17400640 1 --2- 172.21.15.17:0/917188428 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7efe180c7ea0 0x7efe181e5850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:37:57.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.618+0000 7efe17400640 1 -- 172.21.15.17:0/917188428 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe18151530 msgr2=0x7efe181e5d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:37:57.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.618+0000 7efe17400640 1 --2- 172.21.15.17:0/917188428 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efe18151530 0x7efe181e5d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:37:57.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.618+0000 7efe17400640 1 -- 172.21.15.17:0/917188428 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe180ca330 con 0x7efe18072b00 2024-09-10T21:37:57.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.618+0000 7efe17400640 1 --2- 172.21.15.17:0/917188428 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efe18072b00 0x7efe181561c0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7efe0800d9b0 tx=0x7efe0800de70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:37:57.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.618+0000 7efe0f400640 1 -- 172.21.15.17:0/917188428 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe08017070 con 0x7efe18072b00 2024-09-10T21:37:57.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.618+0000 7efe0f400640 1 -- 172.21.15.17:0/917188428 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efe08004310 con 0x7efe18072b00 2024-09-10T21:37:57.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.618+0000 7efe0f400640 1 -- 172.21.15.17:0/917188428 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe08009c30 con 0x7efe18072b00 2024-09-10T21:37:57.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.618+0000 7efe1e200640 1 -- 172.21.15.17:0/917188428 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe181e6c60 con 0x7efe18072b00 2024-09-10T21:37:57.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.618+0000 7efe1e200640 1 -- 172.21.15.17:0/917188428 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe181e1f00 con 0x7efe18072b00 2024-09-10T21:37:57.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.622+0000 7efe1e200640 1 --2- 172.21.15.17:0/917188428 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efe181e5290 0x7efe18148890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:37:57.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.622+0000 7efe1e200640 1 -- 172.21.15.17:0/917188428 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7efe180c8500 con 0x7efe181e5290 2024-09-10T21:37:57.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.622+0000 7efe16a00640 1 --2- 172.21.15.17:0/917188428 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efe181e5290 0x7efe18148890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:37:57.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.622+0000 7efe16a00640 1 -- 172.21.15.17:0/917188428 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7efe181e19d0 con 0x7efe181e5290 2024-09-10T21:37:57.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.622+0000 7efe16a00640 1 --2- 172.21.15.17:0/917188428 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efe181e5290 0x7efe18148890 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7efe1806fc10 tx=0x7efe04009670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:37:57.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.622+0000 7efe0f400640 1 -- 172.21.15.17:0/917188428 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe040b5050 con 0x7efe181e5290 2024-09-10T21:37:57.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.622+0000 7efe0f400640 1 -- 172.21.15.17:0/917188428 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7efe08023060 con 0x7efe18072b00 2024-09-10T21:37:57.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.622+0000 7efe0f400640 1 --2- 172.21.15.17:0/917188428 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7efdf80724e0 0x7efdf80749a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:37:57.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.622+0000 7efe0f400640 1 -- 172.21.15.17:0/917188428 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7efe080930d0 con 0x7efe18072b00 2024-09-10T21:37:57.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.622+0000 7efe0f400640 1 -- 172.21.15.17:0/917188428 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7efe040b6040 con 0x7efe181e5290 2024-09-10T21:37:57.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.622+0000 7efe0f400640 1 -- 172.21.15.17:0/917188428 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efe181e5290 msgr2=0x7efe18148890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:37:57.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.622+0000 7efe0f400640 1 --2- 172.21.15.17:0/917188428 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efe181e5290 0x7efe18148890 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7efe1806fc10 tx=0x7efe04009670 comp rx=0 tx=0).stop 2024-09-10T21:37:57.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.622+0000 7efe0f400640 1 -- 172.21.15.17:0/917188428 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efe181e5290 msgr2=0x7efe18148890 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:37:57.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.622+0000 7efe0f400640 1 --2- 172.21.15.17:0/917188428 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efe181e5290 0x7efe18148890 secure :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7efe1806fc10 tx=0x7efe04009670 comp rx=0 tx=0).stop 2024-09-10T21:37:57.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.626+0000 7efe17e00640 1 --2- 172.21.15.17:0/917188428 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7efdf80724e0 0x7efdf80749a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:37:57.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.626+0000 7efe17e00640 1 --2- 172.21.15.17:0/917188428 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7efdf80724e0 0x7efdf80749a0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7efe181e43f0 tx=0x7efe100094a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:37:57.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.634+0000 7efe1e200640 1 --2- 172.21.15.17:0/917188428 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efe181e6770 0x7efe18148dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:37:57.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.634+0000 7efe1e200640 1 -- 172.21.15.17:0/917188428 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7efe180c0fd0 con 0x7efe181e6770 2024-09-10T21:37:57.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.634+0000 7efe16a00640 1 --2- 172.21.15.17:0/917188428 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efe181e6770 0x7efe18148dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:37:57.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.634+0000 7efe16a00640 1 -- 172.21.15.17:0/917188428 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7efe18154b00 con 0x7efe181e6770 2024-09-10T21:37:57.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.634+0000 7efe16a00640 1 --2- 172.21.15.17:0/917188428 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efe181e6770 0x7efe18148dd0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7efe04009900 tx=0x7efe04009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:37:57.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.634+0000 7efe0f400640 1 -- 172.21.15.17:0/917188428 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe040c3a80 con 0x7efe181e6770 2024-09-10T21:37:57.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.634+0000 7efe0f400640 1 -- 172.21.15.17:0/917188428 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4543 (secure 0 0 0) 0x7efe040c32b0 con 0x7efe181e6770 2024-09-10T21:37:57.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.634+0000 7efe0f400640 1 -- 172.21.15.17:0/917188428 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efe181e6770 msgr2=0x7efe18148dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:37:57.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.634+0000 7efe0f400640 1 --2- 172.21.15.17:0/917188428 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efe181e6770 0x7efe18148dd0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7efe04009900 tx=0x7efe04009260 comp rx=0 tx=0).stop 2024-09-10T21:37:57.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.634+0000 7efe0f400640 1 -- 172.21.15.17:0/917188428 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efe181e6770 msgr2=0x7efe18148dd0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:37:57.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.634+0000 7efe0f400640 1 --2- 172.21.15.17:0/917188428 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efe181e6770 0x7efe18148dd0 secure :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7efe04009900 tx=0x7efe04009260 comp rx=0 tx=0).stop 2024-09-10T21:37:57.638 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-10T21:37:57.638 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:37:57.638 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:37:57.638 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "leader", 2024-09-10T21:37:57.638 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 110, 2024-09-10T21:37:57.638 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 25999, 2024-09-10T21:37:57.639 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-10T21:37:57.639 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-10T21:37:57.639 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-10T21:37:57.639 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-10T21:37:57.639 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:37:57.639 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 0, 2024-09-10T21:37:57.639 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:37:57.639 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-10T21:37:57.639 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-10T21:37:57.639 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:37:57.639 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:37:57.639 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:37:57.639 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:37:57.639 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:37:57.639 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:37:57.640 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:37:57.640 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:37:57.640 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:37:57.640 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:37:57.640 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:37:57.640 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:37:57.640 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-10T21:37:57.640 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-10T21:37:57.640 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:37:57.640 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:37:57.640 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:37:57.640 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:37:57.640 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:37:57.640 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:37:57.641 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:37:57.641 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:37:57.641 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:37:57.641 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:37:57.641 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:37:57.641 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:37:57.641 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:37:57.641 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-10T21:37:57.641 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-10T21:37:57.641 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-10T21:37:57.641 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-10T21:37:57.641 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-10T21:37:57.641 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "5e2f023f-d6a6-49f4-a256-f4807dce3739", 2024-09-10T21:37:57.641 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:37:57.641 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:37:57.642 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-10T21:37:57.642 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-10T21:37:57.642 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-10T21:37:57.642 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-10T21:37:57.642 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-10T21:37:57.642 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-10T21:37:57.642 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-10T21:37:57.642 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:37:57.642 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-10T21:37:57.642 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:37:57.642 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:37:57.642 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:37:57.642 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:37:57.642 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:37:57.643 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:37:57.643 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:37:57.643 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:37:57.643 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:37:57.643 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:37:57.643 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:37:57.643 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:37:57.643 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-10T21:37:57.643 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:37:57.643 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-10T21:37:57.643 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:37:57.643 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:37:57.643 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:37:57.643 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:37:57.644 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:37:57.644 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:37:57.644 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:37:57.644 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-10T21:37:57.644 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:37:57.644 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:37:57.644 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:37:57.644 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:37:57.644 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-10T21:37:57.644 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:37:57.644 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:37:57.644 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:37:57.644 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:37:57.644 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-10T21:37:57.644 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-10T21:37:57.645 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:37:57.645 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:37:57.645 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:37:57.645 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:37:57.645 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:37:57.645 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:37:57.645 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:37:57.645 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:37:57.645 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:37:57.645 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:37:57.645 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:37:57.645 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3300", 2024-09-10T21:37:57.646 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:37:57.646 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:37:57.646 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:37:57.646 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:37:57.646 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789", 2024-09-10T21:37:57.646 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:37:57.646 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:37:57.646 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:37:57.646 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:37:57.646 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789/0", 2024-09-10T21:37:57.646 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6789/0", 2024-09-10T21:37:57.646 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:37:57.646 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:37:57.646 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:37:57.647 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:37:57.647 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:37:57.647 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:37:57.647 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:37:57.647 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:37:57.647 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:37:57.647 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:37:57.647 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:37:57.647 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3301", 2024-09-10T21:37:57.647 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:37:57.647 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:37:57.647 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:37:57.647 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:37:57.647 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790", 2024-09-10T21:37:57.648 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:37:57.648 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:37:57.648 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:37:57.648 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:37:57.648 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790/0", 2024-09-10T21:37:57.648 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6790/0", 2024-09-10T21:37:57.648 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:37:57.648 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:37:57.648 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:37:57.648 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:37:57.648 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:37:57.648 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:37:57.648 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-10T21:37:57.648 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-10T21:37:57.649 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:37:57.649 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:37:57.649 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:37:57.649 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:37:57.649 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:37:57.649 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:37:57.649 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-10T21:37:57.649 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:37:57.649 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:37:57.649 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:37:57.649 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 2 2024-09-10T21:37:57.649 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:37:57.649 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:37:57.650 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:37:57.650 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-10T21:37:57.650 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-10T21:37:57.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.634+0000 7efe1e200640 1 -- 172.21.15.17:0/917188428 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7efdf80724e0 msgr2=0x7efdf80749a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:37:57.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.634+0000 7efe1e200640 1 --2- 172.21.15.17:0/917188428 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7efdf80724e0 0x7efdf80749a0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7efe181e43f0 tx=0x7efe100094a0 comp rx=0 tx=0).stop 2024-09-10T21:37:57.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.634+0000 7efe1e200640 1 -- 172.21.15.17:0/917188428 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efe18072b00 msgr2=0x7efe181561c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:37:57.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.634+0000 7efe1e200640 1 --2- 172.21.15.17:0/917188428 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efe18072b00 0x7efe181561c0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7efe0800d9b0 tx=0x7efe0800de70 comp rx=0 tx=0).stop 2024-09-10T21:37:57.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.634+0000 7efe17e00640 1 -- 172.21.15.17:0/917188428 reap_dead start 2024-09-10T21:37:57.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.634+0000 7efe17e00640 1 -- 172.21.15.17:0/917188428 reap_dead start 2024-09-10T21:37:57.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.634+0000 7efe1e200640 1 -- 172.21.15.17:0/917188428 shutdown_connections 2024-09-10T21:37:57.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.634+0000 7efe1e200640 1 -- 172.21.15.17:0/917188428 >> 172.21.15.17:0/917188428 conn(0x7efe180700d0 msgr2=0x7efe180b18d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:37:57.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.634+0000 7efe1e200640 1 -- 172.21.15.17:0/917188428 shutdown_connections 2024-09-10T21:37:57.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.634+0000 7efe1e200640 1 -- 172.21.15.17:0/917188428 wait complete. 2024-09-10T21:37:57.659 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-10T21:37:57.659 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-09-10T21:37:57.659 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-10T21:37:57.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.838+0000 7f7bf7e00640 1 -- 172.21.15.17:0/2024650674 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7bf8068b20 msgr2=0x7f7bf815c360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:37:57.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.838+0000 7f7bf7e00640 1 --2- 172.21.15.17:0/2024650674 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7bf8068b20 0x7f7bf815c360 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f7bf8155e30 tx=0x7f7bf001cd50 comp rx=0 tx=0).stop 2024-09-10T21:37:57.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.838+0000 7f7bf7e00640 1 -- 172.21.15.17:0/2024650674 shutdown_connections 2024-09-10T21:37:57.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.838+0000 7f7bf7e00640 1 --2- 172.21.15.17:0/2024650674 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7bf8068b20 0x7f7bf815c360 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:37:57.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.838+0000 7f7bf7e00640 1 --2- 172.21.15.17:0/2024650674 >> v2:172.21.15.31:3300/0 conn(0x7f7bf8152540 0x7f7bf80c37e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:37:57.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.838+0000 7f7bf7e00640 1 -- 172.21.15.17:0/2024650674 >> 172.21.15.17:0/2024650674 conn(0x7f7bf806cc20 msgr2=0x7f7bf806d030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:37:57.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.838+0000 7f7bf7e00640 1 -- 172.21.15.17:0/2024650674 shutdown_connections 2024-09-10T21:37:57.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.838+0000 7f7bf7e00640 1 -- 172.21.15.17:0/2024650674 wait complete. 2024-09-10T21:37:57.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.842+0000 7f7bf7e00640 1 Processor -- start 2024-09-10T21:37:57.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.842+0000 7f7bf7e00640 1 -- start start 2024-09-10T21:37:57.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.842+0000 7f7bf7e00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7bf8068b20 0x7f7bf8157410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:37:57.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.842+0000 7f7bf7e00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f7bf8152160 0x7f7bf8074370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:37:57.847 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.842+0000 7f7bf7e00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7bf8152540 0x7f7bf8146f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:37:57.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.842+0000 7f7bf7e00640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f7bf8164bf0 con 0x7f7bf8068b20 2024-09-10T21:37:57.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.842+0000 7f7bf7e00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f7bf8159150 con 0x7f7bf8152540 2024-09-10T21:37:57.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.842+0000 7f7bf7e00640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f7bf8164d70 con 0x7f7bf8152160 2024-09-10T21:37:57.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.842+0000 7f7bf6a00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7bf8068b20 0x7f7bf8157410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:37:57.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.842+0000 7f7bf6a00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7bf8068b20 0x7f7bf8157410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45708/0 (socket says 172.21.15.17:45708) 2024-09-10T21:37:57.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.842+0000 7f7bf6a00640 1 -- 172.21.15.17:0/2752401175 learned_addr learned my addr 172.21.15.17:0/2752401175 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:37:57.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.842+0000 7f7bf7400640 1 --2- 172.21.15.17:0/2752401175 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7bf8152540 0x7f7bf8146f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:37:57.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.842+0000 7f7bf6000640 1 --2- 172.21.15.17:0/2752401175 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f7bf8152160 0x7f7bf8074370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:37:57.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.842+0000 7f7bf6a00640 1 -- 172.21.15.17:0/2752401175 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f7bf8152160 msgr2=0x7f7bf8074370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:37:57.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.842+0000 7f7bf6a00640 1 --2- 172.21.15.17:0/2752401175 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f7bf8152160 0x7f7bf8074370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:37:57.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.842+0000 7f7bf6a00640 1 -- 172.21.15.17:0/2752401175 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7bf8152540 msgr2=0x7f7bf8146f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:37:57.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.842+0000 7f7bf6a00640 1 --2- 172.21.15.17:0/2752401175 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7bf8152540 0x7f7bf8146f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:37:57.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.842+0000 7f7bf6a00640 1 -- 172.21.15.17:0/2752401175 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bf80cbad0 con 0x7f7bf8068b20 2024-09-10T21:37:57.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.842+0000 7f7bf7400640 1 --2- 172.21.15.17:0/2752401175 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7bf8152540 0x7f7bf8146f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:37:57.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.846+0000 7f7bf6a00640 1 --2- 172.21.15.17:0/2752401175 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7bf8068b20 0x7f7bf8157410 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f7be800c530 tx=0x7f7be800c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:37:57.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.846+0000 7f7bef400640 1 -- 172.21.15.17:0/2752401175 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7be8017070 con 0x7f7bf8068b20 2024-09-10T21:37:57.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.846+0000 7f7bef400640 1 -- 172.21.15.17:0/2752401175 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7be80053c0 con 0x7f7bf8068b20 2024-09-10T21:37:57.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.846+0000 7f7bef400640 1 -- 172.21.15.17:0/2752401175 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7be8012890 con 0x7f7bf8068b20 2024-09-10T21:37:57.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.846+0000 7f7bf7e00640 1 -- 172.21.15.17:0/2752401175 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7bf8156170 con 0x7f7bf8068b20 2024-09-10T21:37:57.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.846+0000 7f7bf7e00640 1 -- 172.21.15.17:0/2752401175 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7bf8157950 con 0x7f7bf8068b20 2024-09-10T21:37:57.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.846+0000 7f7bf7e00640 1 --2- 172.21.15.17:0/2752401175 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7bf8147b00 0x7f7bf8147490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:37:57.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.846+0000 7f7bf7e00640 1 -- 172.21.15.17:0/2752401175 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f7bf81518c0 con 0x7f7bf8147b00 2024-09-10T21:37:57.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.846+0000 7f7bf6000640 1 --2- 172.21.15.17:0/2752401175 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7bf8147b00 0x7f7bf8147490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:37:57.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.846+0000 7f7bf6000640 1 -- 172.21.15.17:0/2752401175 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7bf815c360 con 0x7f7bf8147b00 2024-09-10T21:37:57.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.846+0000 7f7bef400640 1 -- 172.21.15.17:0/2752401175 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f7be8003600 con 0x7f7bf8068b20 2024-09-10T21:37:57.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.846+0000 7f7bf6000640 1 --2- 172.21.15.17:0/2752401175 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7bf8147b00 0x7f7bf8147490 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f7bf8148250 tx=0x7f7be4011be0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:37:57.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.850+0000 7f7bef400640 1 --2- 172.21.15.17:0/2752401175 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f7bd80725e0 0x7f7bd8074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:37:57.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.850+0000 7f7bef400640 1 -- 172.21.15.17:0/2752401175 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f7be8092bb0 con 0x7f7bf8068b20 2024-09-10T21:37:57.853 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.850+0000 7f7bef400640 1 -- 172.21.15.17:0/2752401175 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7be401f070 con 0x7f7bf8147b00 2024-09-10T21:37:57.853 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.850+0000 7f7bef400640 1 -- 172.21.15.17:0/2752401175 <== mon.1 v2:172.21.15.31:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f7be400fb80 con 0x7f7bf8147b00 2024-09-10T21:37:57.853 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.850+0000 7f7bef400640 1 -- 172.21.15.17:0/2752401175 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7bf8147b00 msgr2=0x7f7bf8147490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:37:57.853 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.850+0000 7f7bef400640 1 --2- 172.21.15.17:0/2752401175 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7bf8147b00 0x7f7bf8147490 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f7bf8148250 tx=0x7f7be4011be0 comp rx=0 tx=0).stop 2024-09-10T21:37:57.853 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.850+0000 7f7bef400640 1 -- 172.21.15.17:0/2752401175 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7bf8147b00 msgr2=0x7f7bf8147490 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:37:57.853 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.850+0000 7f7bef400640 1 --2- 172.21.15.17:0/2752401175 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7bf8147b00 0x7f7bf8147490 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:37:57.853 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.850+0000 7f7bf7400640 1 --2- 172.21.15.17:0/2752401175 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f7bd80725e0 0x7f7bd8074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:37:57.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.850+0000 7f7bf7400640 1 --2- 172.21.15.17:0/2752401175 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f7bd80725e0 0x7f7bd8074aa0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f7bf00af000 tx=0x7f7bf001b590 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:37:57.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.858+0000 7f7bf7e00640 1 --2- 172.21.15.17:0/2752401175 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7bf81f3080 0x7f7bf81f34a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:37:57.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.858+0000 7f7bf7e00640 1 -- 172.21.15.17:0/2752401175 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f7bf80cbd70 con 0x7f7bf81f3080 2024-09-10T21:37:57.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.858+0000 7f7bf6000640 1 --2- 172.21.15.17:0/2752401175 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7bf81f3080 0x7f7bf81f34a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:37:57.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.858+0000 7f7bf6000640 1 -- 172.21.15.17:0/2752401175 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f7bf814bb90 con 0x7f7bf81f3080 2024-09-10T21:37:57.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.858+0000 7f7bf6000640 1 --2- 172.21.15.17:0/2752401175 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7bf81f3080 0x7f7bf81f34a0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f7be401c000 tx=0x7f7be40046d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:37:57.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.858+0000 7f7bef400640 1 -- 172.21.15.17:0/2752401175 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7be401f030 con 0x7f7bf81f3080 2024-09-10T21:37:57.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.858+0000 7f7bef400640 1 -- 172.21.15.17:0/2752401175 <== mon.1 v2:172.21.15.31:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5043 (secure 0 0 0) 0x7f7be4003810 con 0x7f7bf81f3080 2024-09-10T21:37:57.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.858+0000 7f7bef400640 1 -- 172.21.15.17:0/2752401175 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7bf81f3080 msgr2=0x7f7bf81f34a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:37:57.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.858+0000 7f7bef400640 1 --2- 172.21.15.17:0/2752401175 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7bf81f3080 0x7f7bf81f34a0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f7be401c000 tx=0x7f7be40046d0 comp rx=0 tx=0).stop 2024-09-10T21:37:57.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.858+0000 7f7bef400640 1 -- 172.21.15.17:0/2752401175 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7bf81f3080 msgr2=0x7f7bf81f34a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:37:57.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.858+0000 7f7bef400640 1 --2- 172.21.15.17:0/2752401175 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7bf81f3080 0x7f7bf81f34a0 secure :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f7be401c000 tx=0x7f7be40046d0 comp rx=0 tx=0).stop 2024-09-10T21:37:57.863 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-10T21:37:57.863 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:37:57.863 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:37:57.864 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-10T21:37:57.864 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 110, 2024-09-10T21:37:57.864 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 197244, 2024-09-10T21:37:57.864 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-10T21:37:57.864 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-10T21:37:57.864 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-10T21:37:57.864 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-10T21:37:57.864 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:37:57.864 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 0, 2024-09-10T21:37:57.864 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:37:57.864 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-10T21:37:57.864 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-10T21:37:57.864 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:37:57.865 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:37:57.865 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:37:57.865 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:37:57.865 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:37:57.865 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:37:57.865 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:37:57.865 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:37:57.865 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:37:57.865 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:37:57.865 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:37:57.865 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:37:57.865 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-10T21:37:57.865 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-10T21:37:57.866 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:37:57.866 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:37:57.866 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:37:57.866 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:37:57.866 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:37:57.866 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:37:57.866 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:37:57.866 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:37:57.866 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:37:57.866 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:37:57.866 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:37:57.866 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:37:57.866 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:37:57.867 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-10T21:37:57.867 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-10T21:37:57.867 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-10T21:37:57.867 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-10T21:37:57.867 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-10T21:37:57.867 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "5e2f023f-d6a6-49f4-a256-f4807dce3739", 2024-09-10T21:37:57.867 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:37:57.867 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:37:57.867 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-10T21:37:57.867 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-10T21:37:57.867 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-10T21:37:57.867 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-10T21:37:57.867 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-10T21:37:57.867 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-10T21:37:57.867 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-10T21:37:57.868 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:37:57.868 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-10T21:37:57.868 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:37:57.868 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:37:57.868 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:37:57.868 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:37:57.868 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:37:57.868 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:37:57.868 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:37:57.868 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:37:57.868 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:37:57.868 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:37:57.868 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:37:57.868 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:37:57.869 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-10T21:37:57.869 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:37:57.869 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-10T21:37:57.869 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:37:57.869 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:37:57.869 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:37:57.869 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:37:57.869 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:37:57.869 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:37:57.869 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:37:57.869 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-10T21:37:57.869 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:37:57.869 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:37:57.869 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:37:57.870 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:37:57.870 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-10T21:37:57.870 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:37:57.870 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:37:57.870 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:37:57.870 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:37:57.870 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-10T21:37:57.870 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-10T21:37:57.870 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:37:57.870 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:37:57.870 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:37:57.870 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:37:57.870 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:37:57.871 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:37:57.871 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:37:57.871 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:37:57.871 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:37:57.871 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:37:57.871 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:37:57.871 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3300", 2024-09-10T21:37:57.871 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:37:57.871 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:37:57.871 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:37:57.871 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:37:57.871 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789", 2024-09-10T21:37:57.871 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:37:57.871 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:37:57.872 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:37:57.872 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:37:57.872 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789/0", 2024-09-10T21:37:57.872 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6789/0", 2024-09-10T21:37:57.872 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:37:57.872 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:37:57.872 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:37:57.872 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:37:57.872 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:37:57.872 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:37:57.872 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:37:57.872 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:37:57.872 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:37:57.873 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:37:57.873 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:37:57.873 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3301", 2024-09-10T21:37:57.873 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:37:57.873 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:37:57.873 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:37:57.873 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:37:57.873 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790", 2024-09-10T21:37:57.873 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:37:57.873 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:37:57.873 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:37:57.873 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:37:57.873 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790/0", 2024-09-10T21:37:57.873 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6790/0", 2024-09-10T21:37:57.874 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:37:57.874 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:37:57.874 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:37:57.874 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:37:57.874 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:37:57.874 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:37:57.874 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-10T21:37:57.874 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-10T21:37:57.874 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:37:57.874 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:37:57.874 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:37:57.874 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:37:57.874 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:37:57.874 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:37:57.874 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": [ 2024-09-10T21:37:57.875 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:37:57.875 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:37:57.875 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:37:57.875 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 2 2024-09-10T21:37:57.875 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:37:57.875 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:37:57.875 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-10T21:37:57.875 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:37:57.875 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:37:57.875 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:37:57.875 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 4 2024-09-10T21:37:57.875 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:37:57.875 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:37:57.875 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-10T21:37:57.875 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:37:57.876 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:37:57.876 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:37:57.876 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 4 2024-09-10T21:37:57.876 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:37:57.876 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:37:57.876 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": [ 2024-09-10T21:37:57.876 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:37:57.876 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:37:57.876 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:37:57.876 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:37:57.876 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:37:57.876 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:37:57.876 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:37:57.877 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-10T21:37:57.877 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-10T21:37:57.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.858+0000 7f7bf7e00640 1 -- 172.21.15.17:0/2752401175 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f7bd80725e0 msgr2=0x7f7bd8074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:37:57.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.858+0000 7f7bf7e00640 1 --2- 172.21.15.17:0/2752401175 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f7bd80725e0 0x7f7bd8074aa0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f7bf00af000 tx=0x7f7bf001b590 comp rx=0 tx=0).stop 2024-09-10T21:37:57.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.858+0000 7f7bf7e00640 1 -- 172.21.15.17:0/2752401175 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7bf8068b20 msgr2=0x7f7bf8157410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:37:57.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.858+0000 7f7bf7e00640 1 --2- 172.21.15.17:0/2752401175 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7bf8068b20 0x7f7bf8157410 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f7be800c530 tx=0x7f7be800c9f0 comp rx=0 tx=0).stop 2024-09-10T21:37:57.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.858+0000 7f7bf7400640 1 -- 172.21.15.17:0/2752401175 reap_dead start 2024-09-10T21:37:57.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.858+0000 7f7bf7400640 1 -- 172.21.15.17:0/2752401175 reap_dead start 2024-09-10T21:37:57.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.858+0000 7f7bf7e00640 1 -- 172.21.15.17:0/2752401175 shutdown_connections 2024-09-10T21:37:57.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.858+0000 7f7bf7e00640 1 -- 172.21.15.17:0/2752401175 >> 172.21.15.17:0/2752401175 conn(0x7f7bf806cc20 msgr2=0x7f7bf8150f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:37:57.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.858+0000 7f7bf7e00640 1 -- 172.21.15.17:0/2752401175 shutdown_connections 2024-09-10T21:37:57.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:57.858+0000 7f7bf7e00640 1 -- 172.21.15.17:0/2752401175 wait complete. 2024-09-10T21:37:57.884 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-10T21:37:57.884 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-09-10T21:37:57.884 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-10T21:37:58.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:58.046+0000 7f4f74c00640 1 -- 172.21.15.17:0/3568464269 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4f70151450 msgr2=0x7f4f700ca3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:37:58.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:58.046+0000 7f4f74c00640 1 --2- 172.21.15.17:0/3568464269 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4f70151450 0x7f4f700ca3f0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f4f60009900 tx=0x7f4f6001f0d0 comp rx=0 tx=0).stop 2024-09-10T21:37:58.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:58.046+0000 7f4f74c00640 1 -- 172.21.15.17:0/3568464269 shutdown_connections 2024-09-10T21:37:58.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:58.046+0000 7f4f74c00640 1 --2- 172.21.15.17:0/3568464269 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f4f700cabe0 0x7f4f7015a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:37:58.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:58.046+0000 7f4f74c00640 1 --2- 172.21.15.17:0/3568464269 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4f70151450 0x7f4f700ca3f0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:37:58.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:58.046+0000 7f4f74c00640 1 -- 172.21.15.17:0/3568464269 >> 172.21.15.17:0/3568464269 conn(0x7f4f700cd690 msgr2=0x7f4f700c98e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:37:58.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:58.046+0000 7f4f74c00640 1 -- 172.21.15.17:0/3568464269 shutdown_connections 2024-09-10T21:37:58.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:58.046+0000 7f4f74c00640 1 -- 172.21.15.17:0/3568464269 wait complete. 2024-09-10T21:37:58.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:58.046+0000 7f4f74c00640 1 Processor -- start 2024-09-10T21:37:58.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:58.046+0000 7f4f74c00640 1 -- start start 2024-09-10T21:37:58.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:58.046+0000 7f4f74c00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4f700c2f30 0x7f4f7014fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:37:58.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:58.046+0000 7f4f74c00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f4f700cabe0 0x7f4f70159410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:37:58.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:58.046+0000 7f4f74c00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4f70151450 0x7f4f7006ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:37:58.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:58.046+0000 7f4f74c00640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f4f700c2a10 con 0x7f4f70151450 2024-09-10T21:37:58.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:58.046+0000 7f4f74c00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f4f7006f990 con 0x7f4f700c2f30 2024-09-10T21:37:58.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:58.046+0000 7f4f74c00640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f4f700c2710 con 0x7f4f700cabe0 2024-09-10T21:37:58.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:58.046+0000 7f4f6fe00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4f70151450 0x7f4f7006ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:37:58.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:58.050+0000 7f4f6ea00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f4f700cabe0 0x7f4f70159410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:37:58.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:58.050+0000 7f4f6f400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4f700c2f30 0x7f4f7014fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:37:58.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:58.050+0000 7f4f6fe00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4f70151450 0x7f4f7006ce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45734/0 (socket says 172.21.15.17:45734) 2024-09-10T21:37:58.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:58.050+0000 7f4f6fe00640 1 -- 172.21.15.17:0/2974541645 learned_addr learned my addr 172.21.15.17:0/2974541645 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:37:58.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:58.050+0000 7f4f6fe00640 1 -- 172.21.15.17:0/2974541645 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f4f700cabe0 msgr2=0x7f4f70159410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:37:58.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:58.050+0000 7f4f6fe00640 1 --2- 172.21.15.17:0/2974541645 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f4f700cabe0 0x7f4f70159410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:37:58.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:58.050+0000 7f4f6fe00640 1 -- 172.21.15.17:0/2974541645 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4f700c2f30 msgr2=0x7f4f7014fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:37:58.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:58.050+0000 7f4f6fe00640 1 --2- 172.21.15.17:0/2974541645 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4f700c2f30 0x7f4f7014fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:37:58.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:58.050+0000 7f4f6fe00640 1 -- 172.21.15.17:0/2974541645 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f70159dd0 con 0x7f4f70151450 2024-09-10T21:37:58.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:58.050+0000 7f4f6fe00640 1 --2- 172.21.15.17:0/2974541645 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4f70151450 0x7f4f7006ce80 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f4f6800c780 tx=0x7f4f6800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:37:58.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:58.050+0000 7f4f67400640 1 -- 172.21.15.17:0/2974541645 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f68016020 con 0x7f4f70151450 2024-09-10T21:37:58.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:58.050+0000 7f4f67400640 1 -- 172.21.15.17:0/2974541645 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4f6800aae0 con 0x7f4f70151450 2024-09-10T21:37:58.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:58.050+0000 7f4f74c00640 1 -- 172.21.15.17:0/2974541645 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f70150740 con 0x7f4f70151450 2024-09-10T21:37:58.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:58.050+0000 7f4f67400640 1 -- 172.21.15.17:0/2974541645 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f68012420 con 0x7f4f70151450 2024-09-10T21:37:58.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:58.050+0000 7f4f74c00640 1 -- 172.21.15.17:0/2974541645 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f7015a040 con 0x7f4f70151450 2024-09-10T21:37:58.055 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:58.050+0000 7f4f74c00640 1 --2- 172.21.15.17:0/2974541645 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f4f701583e0 0x7f4f7014bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:37:58.055 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:58.050+0000 7f4f74c00640 1 -- 172.21.15.17:0/2974541645 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f4f70069ef0 con 0x7f4f701583e0 2024-09-10T21:37:58.055 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:58.050+0000 7f4f6f400640 1 --2- 172.21.15.17:0/2974541645 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f4f701583e0 0x7f4f7014bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:37:58.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:58.054+0000 7f4f6f400640 1 -- 172.21.15.17:0/2974541645 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4f700c79a0 con 0x7f4f701583e0 2024-09-10T21:37:58.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:58.054+0000 7f4f6f400640 1 --2- 172.21.15.17:0/2974541645 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f4f701583e0 0x7f4f7014bcc0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f4f700c3e20 tx=0x7f4f60005a30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:37:58.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:58.054+0000 7f4f67400640 1 -- 172.21.15.17:0/2974541645 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f4f6801b400 con 0x7f4f70151450 2024-09-10T21:37:58.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:58.054+0000 7f4f67400640 1 --2- 172.21.15.17:0/2974541645 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f4f500725e0 0x7f4f50074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:37:58.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:58.054+0000 7f4f67400640 1 -- 172.21.15.17:0/2974541645 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f4f680937f0 con 0x7f4f70151450 2024-09-10T21:37:58.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:58.054+0000 7f4f67400640 1 -- 172.21.15.17:0/2974541645 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f600b5050 con 0x7f4f701583e0 2024-09-10T21:37:58.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:58.054+0000 7f4f67400640 1 -- 172.21.15.17:0/2974541645 <== mon.2 v2:172.21.15.31:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f4f600b6040 con 0x7f4f701583e0 2024-09-10T21:37:58.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:58.054+0000 7f4f67400640 1 -- 172.21.15.17:0/2974541645 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f4f701583e0 msgr2=0x7f4f7014bcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:37:58.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:58.054+0000 7f4f67400640 1 --2- 172.21.15.17:0/2974541645 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f4f701583e0 0x7f4f7014bcc0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f4f700c3e20 tx=0x7f4f60005a30 comp rx=0 tx=0).stop 2024-09-10T21:37:58.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:58.054+0000 7f4f67400640 1 -- 172.21.15.17:0/2974541645 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f4f701583e0 msgr2=0x7f4f7014bcc0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:37:58.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:58.054+0000 7f4f67400640 1 --2- 172.21.15.17:0/2974541645 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f4f701583e0 0x7f4f7014bcc0 secure :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f4f700c3e20 tx=0x7f4f60005a30 comp rx=0 tx=0).stop 2024-09-10T21:37:58.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:58.054+0000 7f4f6ea00640 1 --2- 172.21.15.17:0/2974541645 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f4f500725e0 0x7f4f50074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:37:58.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:58.058+0000 7f4f6ea00640 1 --2- 172.21.15.17:0/2974541645 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f4f500725e0 0x7f4f50074aa0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f4f7014d880 tx=0x7f4f5c00b390 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:37:58.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:58.066+0000 7f4f74c00640 1 --2- 172.21.15.17:0/2974541645 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f4f7014c200 0x7f4f70145630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:37:58.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:58.066+0000 7f4f74c00640 1 -- 172.21.15.17:0/2974541645 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f4f7015a840 con 0x7f4f7014c200 2024-09-10T21:37:58.068 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:58.066+0000 7f4f6f400640 1 --2- 172.21.15.17:0/2974541645 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f4f7014c200 0x7f4f70145630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:37:58.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:58.066+0000 7f4f6f400640 1 -- 172.21.15.17:0/2974541645 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f4f700cb670 con 0x7f4f7014c200 2024-09-10T21:37:58.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:58.066+0000 7f4f6f400640 1 --2- 172.21.15.17:0/2974541645 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f4f7014c200 0x7f4f70145630 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f4f6001ef00 tx=0x7f4f60009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:37:58.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:58.066+0000 7f4f67400640 1 -- 172.21.15.17:0/2974541645 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f600c3a80 con 0x7f4f7014c200 2024-09-10T21:37:58.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:58.066+0000 7f4f67400640 1 -- 172.21.15.17:0/2974541645 <== mon.2 v2:172.21.15.31:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5042 (secure 0 0 0) 0x7f4f600c32b0 con 0x7f4f7014c200 2024-09-10T21:37:58.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:58.066+0000 7f4f67400640 1 -- 172.21.15.17:0/2974541645 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f4f7014c200 msgr2=0x7f4f70145630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:37:58.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:58.066+0000 7f4f67400640 1 --2- 172.21.15.17:0/2974541645 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f4f7014c200 0x7f4f70145630 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f4f6001ef00 tx=0x7f4f60009260 comp rx=0 tx=0).stop 2024-09-10T21:37:58.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:58.066+0000 7f4f67400640 1 -- 172.21.15.17:0/2974541645 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f4f7014c200 msgr2=0x7f4f70145630 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:37:58.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:58.066+0000 7f4f67400640 1 --2- 172.21.15.17:0/2974541645 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f4f7014c200 0x7f4f70145630 secure :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f4f6001ef00 tx=0x7f4f60009260 comp rx=0 tx=0).stop 2024-09-10T21:37:58.071 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-10T21:37:58.071 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:37:58.071 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:37:58.071 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-10T21:37:58.071 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 110, 2024-09-10T21:37:58.071 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 84414, 2024-09-10T21:37:58.071 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-10T21:37:58.071 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-10T21:37:58.072 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-10T21:37:58.072 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-10T21:37:58.072 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:37:58.072 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 0, 2024-09-10T21:37:58.072 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:37:58.072 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-10T21:37:58.072 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-10T21:37:58.072 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:37:58.072 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:37:58.072 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:37:58.072 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:37:58.072 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:37:58.073 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:37:58.073 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:37:58.073 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:37:58.073 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:37:58.073 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:37:58.073 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:37:58.073 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:37:58.073 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-10T21:37:58.073 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-10T21:37:58.073 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:37:58.073 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:37:58.073 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:37:58.073 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:37:58.074 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:37:58.074 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:37:58.074 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:37:58.074 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:37:58.074 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:37:58.074 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:37:58.074 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:37:58.074 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:37:58.074 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:37:58.074 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-10T21:37:58.074 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-10T21:37:58.074 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-10T21:37:58.074 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-10T21:37:58.075 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-10T21:37:58.075 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "5e2f023f-d6a6-49f4-a256-f4807dce3739", 2024-09-10T21:37:58.075 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:37:58.075 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:37:58.075 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-10T21:37:58.075 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-10T21:37:58.075 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-10T21:37:58.075 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-10T21:37:58.075 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-10T21:37:58.075 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-10T21:37:58.075 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-10T21:37:58.075 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:37:58.075 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-10T21:37:58.075 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:37:58.075 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:37:58.076 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:37:58.076 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:37:58.076 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:37:58.076 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:37:58.076 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:37:58.076 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:37:58.076 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:37:58.076 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:37:58.076 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:37:58.076 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:37:58.076 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-10T21:37:58.076 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:37:58.076 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-10T21:37:58.076 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:37:58.077 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:37:58.077 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:37:58.077 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:37:58.077 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:37:58.077 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:37:58.077 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:37:58.077 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-10T21:37:58.077 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:37:58.077 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:37:58.077 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:37:58.077 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:37:58.077 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-10T21:37:58.078 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:37:58.078 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:37:58.078 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:37:58.078 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:37:58.078 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-10T21:37:58.078 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-10T21:37:58.078 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:37:58.078 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:37:58.078 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:37:58.078 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:37:58.078 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:37:58.078 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:37:58.078 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:37:58.079 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:37:58.079 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:37:58.079 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:37:58.079 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:37:58.079 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3300", 2024-09-10T21:37:58.079 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:37:58.079 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:37:58.079 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:37:58.079 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:37:58.079 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789", 2024-09-10T21:37:58.079 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:37:58.079 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:37:58.079 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:37:58.080 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:37:58.080 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789/0", 2024-09-10T21:37:58.080 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6789/0", 2024-09-10T21:37:58.080 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:37:58.080 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:37:58.080 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:37:58.080 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:37:58.080 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:37:58.080 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:37:58.080 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:37:58.080 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:37:58.080 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:37:58.081 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:37:58.081 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:37:58.081 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3301", 2024-09-10T21:37:58.081 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:37:58.081 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:37:58.081 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:37:58.081 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:37:58.081 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790", 2024-09-10T21:37:58.081 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:37:58.081 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:37:58.081 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:37:58.081 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:37:58.082 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790/0", 2024-09-10T21:37:58.082 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6790/0", 2024-09-10T21:37:58.082 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:37:58.082 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:37:58.082 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:37:58.082 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:37:58.082 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:37:58.082 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:37:58.082 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-10T21:37:58.082 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-10T21:37:58.082 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:37:58.082 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:37:58.082 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:37:58.082 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:37:58.083 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:37:58.083 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:37:58.083 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": [ 2024-09-10T21:37:58.083 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:37:58.083 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:37:58.083 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:37:58.083 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 4 2024-09-10T21:37:58.083 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:37:58.083 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:37:58.083 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-10T21:37:58.083 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:37:58.083 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:37:58.083 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:37:58.084 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 4 2024-09-10T21:37:58.084 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:37:58.084 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:37:58.084 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-10T21:37:58.084 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:37:58.084 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:37:58.084 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:37:58.084 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 3 2024-09-10T21:37:58.084 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:37:58.084 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:37:58.084 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": [ 2024-09-10T21:37:58.084 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:37:58.085 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:37:58.085 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:37:58.085 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:37:58.085 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:37:58.085 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:37:58.085 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:37:58.085 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-10T21:37:58.085 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-10T21:37:58.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:58.066+0000 7f4f74c00640 1 -- 172.21.15.17:0/2974541645 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f4f500725e0 msgr2=0x7f4f50074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:37:58.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:58.066+0000 7f4f74c00640 1 --2- 172.21.15.17:0/2974541645 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f4f500725e0 0x7f4f50074aa0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f4f7014d880 tx=0x7f4f5c00b390 comp rx=0 tx=0).stop 2024-09-10T21:37:58.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:58.066+0000 7f4f74c00640 1 -- 172.21.15.17:0/2974541645 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4f70151450 msgr2=0x7f4f7006ce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:37:58.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:58.066+0000 7f4f74c00640 1 --2- 172.21.15.17:0/2974541645 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4f70151450 0x7f4f7006ce80 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f4f6800c780 tx=0x7f4f6800cc40 comp rx=0 tx=0).stop 2024-09-10T21:37:58.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:58.066+0000 7f4f6fe00640 1 -- 172.21.15.17:0/2974541645 reap_dead start 2024-09-10T21:37:58.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:58.066+0000 7f4f6fe00640 1 -- 172.21.15.17:0/2974541645 reap_dead start 2024-09-10T21:37:58.086 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:58.066+0000 7f4f74c00640 1 -- 172.21.15.17:0/2974541645 shutdown_connections 2024-09-10T21:37:58.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:58.066+0000 7f4f74c00640 1 -- 172.21.15.17:0/2974541645 >> 172.21.15.17:0/2974541645 conn(0x7f4f700cd690 msgr2=0x7f4f7015af40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:37:58.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:58.070+0000 7f4f74c00640 1 -- 172.21.15.17:0/2974541645 shutdown_connections 2024-09-10T21:37:58.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:58.070+0000 7f4f74c00640 1 -- 172.21.15.17:0/2974541645 wait complete. 2024-09-10T21:37:58.094 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-10T21:37:58.094 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-09-10T21:37:58.094 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-09-10T21:37:58.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:58.262+0000 7ffa44e00640 1 -- 172.21.15.17:0/3001405486 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffa40151250 msgr2=0x7ffa40157bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:37:58.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:58.262+0000 7ffa44e00640 1 --2- 172.21.15.17:0/3001405486 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffa40151250 0x7ffa40157bc0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7ffa40068d60 tx=0x7ffa3001eff0 comp rx=0 tx=0).stop 2024-09-10T21:37:58.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:58.262+0000 7ffa44e00640 1 -- 172.21.15.17:0/3001405486 shutdown_connections 2024-09-10T21:37:58.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:58.262+0000 7ffa44e00640 1 --2- 172.21.15.17:0/3001405486 >> v2:172.21.15.31:3300/0 conn(0x7ffa40072b60 0x7ffa400c60f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:37:58.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:58.262+0000 7ffa44e00640 1 --2- 172.21.15.17:0/3001405486 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffa40151250 0x7ffa40157bc0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:37:58.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:58.262+0000 7ffa44e00640 1 -- 172.21.15.17:0/3001405486 >> 172.21.15.17:0/3001405486 conn(0x7ffa4006cdf0 msgr2=0x7ffa4006d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:37:58.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:58.262+0000 7ffa44e00640 1 -- 172.21.15.17:0/3001405486 shutdown_connections 2024-09-10T21:37:58.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:58.262+0000 7ffa44e00640 1 -- 172.21.15.17:0/3001405486 wait complete. 2024-09-10T21:37:58.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:58.266+0000 7ffa44e00640 1 Processor -- start 2024-09-10T21:37:58.268 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:58.266+0000 7ffa44e00640 1 -- start start 2024-09-10T21:37:58.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:58.266+0000 7ffa44e00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ffa40072b60 0x7ffa400cae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:37:58.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:58.266+0000 7ffa44e00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa400c9100 0x7ffa400cb3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:37:58.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:58.266+0000 7ffa44e00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffa40151250 0x7ffa401559f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:37:58.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:58.266+0000 7ffa44e00640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7ffa400c5610 con 0x7ffa40151250 2024-09-10T21:37:58.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:58.266+0000 7ffa44e00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7ffa400c5330 con 0x7ffa400c9100 2024-09-10T21:37:58.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:58.266+0000 7ffa44e00640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7ffa400c58f0 con 0x7ffa40072b60 2024-09-10T21:37:58.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:58.266+0000 7ffa3fe00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffa40151250 0x7ffa401559f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:37:58.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:58.266+0000 7ffa3fe00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffa40151250 0x7ffa401559f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45754/0 (socket says 172.21.15.17:45754) 2024-09-10T21:37:58.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:58.266+0000 7ffa3fe00640 1 -- 172.21.15.17:0/4011094110 learned_addr learned my addr 172.21.15.17:0/4011094110 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:37:58.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:58.266+0000 7ffa3f400640 1 --2- 172.21.15.17:0/4011094110 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ffa40072b60 0x7ffa400cae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:37:58.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:58.266+0000 7ffa3ea00640 1 --2- 172.21.15.17:0/4011094110 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa400c9100 0x7ffa400cb3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:37:58.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:58.266+0000 7ffa3fe00640 1 -- 172.21.15.17:0/4011094110 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ffa40072b60 msgr2=0x7ffa400cae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:37:58.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:58.266+0000 7ffa3fe00640 1 --2- 172.21.15.17:0/4011094110 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ffa40072b60 0x7ffa400cae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:37:58.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:58.266+0000 7ffa3fe00640 1 -- 172.21.15.17:0/4011094110 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa400c9100 msgr2=0x7ffa400cb3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:37:58.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:58.266+0000 7ffa3fe00640 1 --2- 172.21.15.17:0/4011094110 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa400c9100 0x7ffa400cb3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:37:58.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:58.266+0000 7ffa3fe00640 1 -- 172.21.15.17:0/4011094110 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa40155220 con 0x7ffa40151250 2024-09-10T21:37:58.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:58.266+0000 7ffa3fe00640 1 --2- 172.21.15.17:0/4011094110 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffa40151250 0x7ffa401559f0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7ffa3800cbc0 tx=0x7ffa3800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:37:58.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:58.266+0000 7ffa37400640 1 -- 172.21.15.17:0/4011094110 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa38016020 con 0x7ffa40151250 2024-09-10T21:37:58.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:58.266+0000 7ffa37400640 1 -- 172.21.15.17:0/4011094110 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ffa3800aae0 con 0x7ffa40151250 2024-09-10T21:37:58.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:58.266+0000 7ffa44e00640 1 -- 172.21.15.17:0/4011094110 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa401554c0 con 0x7ffa40151250 2024-09-10T21:37:58.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:58.266+0000 7ffa37400640 1 -- 172.21.15.17:0/4011094110 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa38012420 con 0x7ffa40151250 2024-09-10T21:37:58.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:58.266+0000 7ffa44e00640 1 -- 172.21.15.17:0/4011094110 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa401f1340 con 0x7ffa40151250 2024-09-10T21:37:58.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:58.270+0000 7ffa44e00640 1 -- 172.21.15.17:0/4011094110 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa400c5bd0 con 0x7ffa40151250 2024-09-10T21:37:58.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:58.270+0000 7ffa37400640 1 -- 172.21.15.17:0/4011094110 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7ffa3801b400 con 0x7ffa40151250 2024-09-10T21:37:58.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:58.270+0000 7ffa37400640 1 --2- 172.21.15.17:0/4011094110 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7ffa200725e0 0x7ffa20074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:37:58.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:58.270+0000 7ffa37400640 1 -- 172.21.15.17:0/4011094110 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7ffa38093b30 con 0x7ffa40151250 2024-09-10T21:37:58.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:58.274+0000 7ffa3f400640 1 --2- 172.21.15.17:0/4011094110 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7ffa200725e0 0x7ffa20074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:37:58.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:58.274+0000 7ffa3f400640 1 --2- 172.21.15.17:0/4011094110 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7ffa200725e0 0x7ffa20074aa0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7ffa300b2000 tx=0x7ffa3001d830 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:37:58.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:58.278+0000 7ffa37400640 1 -- 172.21.15.17:0/4011094110 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7ffa38020020 con 0x7ffa40151250 2024-09-10T21:37:58.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:58.558+0000 7ffa44e00640 1 -- 172.21.15.17:0/4011094110 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) -- 0x7ffa40069e60 con 0x7ffa40151250 2024-09-10T21:37:58.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:58.558+0000 7ffa37400640 1 -- 172.21.15.17:0/4011094110 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) ==== 57+0+0 (secure 0 0 0) 0x7ffa3805e730 con 0x7ffa40151250 2024-09-10T21:37:58.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:58.562+0000 7ffa44e00640 1 -- 172.21.15.17:0/4011094110 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7ffa200725e0 msgr2=0x7ffa20074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:37:58.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:58.562+0000 7ffa44e00640 1 --2- 172.21.15.17:0/4011094110 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7ffa200725e0 0x7ffa20074aa0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7ffa300b2000 tx=0x7ffa3001d830 comp rx=0 tx=0).stop 2024-09-10T21:37:58.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:58.562+0000 7ffa44e00640 1 -- 172.21.15.17:0/4011094110 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffa40151250 msgr2=0x7ffa401559f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:37:58.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:58.562+0000 7ffa44e00640 1 --2- 172.21.15.17:0/4011094110 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffa40151250 0x7ffa401559f0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7ffa3800cbc0 tx=0x7ffa3800ccb0 comp rx=0 tx=0).stop 2024-09-10T21:37:58.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:58.562+0000 7ffa44e00640 1 -- 172.21.15.17:0/4011094110 shutdown_connections 2024-09-10T21:37:58.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:58.562+0000 7ffa44e00640 1 --2- 172.21.15.17:0/4011094110 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7ffa200725e0 0x7ffa20074aa0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:37:58.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:58.562+0000 7ffa44e00640 1 --2- 172.21.15.17:0/4011094110 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffa40151250 0x7ffa401559f0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:37:58.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:58.562+0000 7ffa44e00640 1 --2- 172.21.15.17:0/4011094110 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffa400c9100 0x7ffa400cb3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:37:58.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:58.562+0000 7ffa44e00640 1 --2- 172.21.15.17:0/4011094110 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ffa40072b60 0x7ffa400cae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:37:58.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:58.562+0000 7ffa44e00640 1 -- 172.21.15.17:0/4011094110 >> 172.21.15.17:0/4011094110 conn(0x7ffa4006cdf0 msgr2=0x7ffa400c7600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:37:58.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:58.562+0000 7ffa44e00640 1 -- 172.21.15.17:0/4011094110 shutdown_connections 2024-09-10T21:37:58.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:37:58.562+0000 7ffa44e00640 1 -- 172.21.15.17:0/4011094110 wait complete. 2024-09-10T21:37:58.599 INFO:tasks.mon_thrash.mon_thrasher:waiting for 10.0 secs before continuing thrashing 2024-09-10T21:38:08.600 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-09-10T21:38:08.601 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-10T21:38:08.784 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:08.778+0000 7f684e200640 1 -- 172.21.15.17:0/1761542444 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f684806f480 msgr2=0x7f68480c7870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:38:08.784 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:08.778+0000 7f684e200640 1 --2- 172.21.15.17:0/1761542444 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f684806f480 0x7f68480c7870 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f683c009840 tx=0x7f683c01c820 comp rx=0 tx=0).stop 2024-09-10T21:38:08.784 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:08.782+0000 7f684e200640 1 -- 172.21.15.17:0/1761542444 shutdown_connections 2024-09-10T21:38:08.784 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:08.782+0000 7f684e200640 1 --2- 172.21.15.17:0/1761542444 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f684806f480 0x7f68480c7870 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:38:08.785 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:08.782+0000 7f684e200640 1 -- 172.21.15.17:0/1761542444 >> 172.21.15.17:0/1761542444 conn(0x7f6848070ca0 msgr2=0x7f68480710b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:38:08.785 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:08.782+0000 7f684e200640 1 -- 172.21.15.17:0/1761542444 shutdown_connections 2024-09-10T21:38:08.785 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:08.782+0000 7f684e200640 1 -- 172.21.15.17:0/1761542444 wait complete. 2024-09-10T21:38:08.785 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:08.782+0000 7f684e200640 1 Processor -- start 2024-09-10T21:38:08.785 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:08.782+0000 7f684e200640 1 -- start start 2024-09-10T21:38:08.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:08.782+0000 7f684e200640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f684806f480 0x7f6848146570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:38:08.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:08.782+0000 7f684e200640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f684806f860 0x7f68481f46c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:38:08.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:08.782+0000 7f684e200640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f68480c8070 0x7f68481f70e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:38:08.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:08.782+0000 7f684e200640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f68480702a0 con 0x7f684806f480 2024-09-10T21:38:08.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:08.782+0000 7f684e200640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f6848151b40 con 0x7f68480c8070 2024-09-10T21:38:08.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:08.782+0000 7f684e200640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f6848165500 con 0x7f684806f860 2024-09-10T21:38:08.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:08.782+0000 7f684ce00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f684806f480 0x7f6848146570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:38:08.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:08.782+0000 7f684ce00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f684806f480 0x7f6848146570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:58662/0 (socket says 172.21.15.17:58662) 2024-09-10T21:38:08.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:08.782+0000 7f684ce00640 1 -- 172.21.15.17:0/3108773856 learned_addr learned my addr 172.21.15.17:0/3108773856 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:38:08.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:08.782+0000 7f6847e00640 1 --2- 172.21.15.17:0/3108773856 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f684806f860 0x7f68481f46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:38:08.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:08.782+0000 7f684d800640 1 --2- 172.21.15.17:0/3108773856 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f68480c8070 0x7f68481f70e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:38:08.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:08.782+0000 7f684ce00640 1 -- 172.21.15.17:0/3108773856 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f684806f860 msgr2=0x7f68481f46c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:38:08.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:08.782+0000 7f684ce00640 1 --2- 172.21.15.17:0/3108773856 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f684806f860 0x7f68481f46c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:38:08.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:08.782+0000 7f684ce00640 1 -- 172.21.15.17:0/3108773856 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f68480c8070 msgr2=0x7f68481f70e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:38:08.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:08.782+0000 7f684ce00640 1 --2- 172.21.15.17:0/3108773856 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f68480c8070 0x7f68481f70e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:38:08.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:08.782+0000 7f684ce00640 1 -- 172.21.15.17:0/3108773856 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6848144dd0 con 0x7f684806f480 2024-09-10T21:38:08.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:08.782+0000 7f6847e00640 1 --2- 172.21.15.17:0/3108773856 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f684806f860 0x7f68481f46c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:38:08.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:08.786+0000 7f684ce00640 1 --2- 172.21.15.17:0/3108773856 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f684806f480 0x7f6848146570 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f683c009790 tx=0x7f683c0040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:38:08.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:08.786+0000 7f6844c00640 1 -- 172.21.15.17:0/3108773856 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f683c0bb070 con 0x7f684806f480 2024-09-10T21:38:08.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:08.786+0000 7f6844c00640 1 -- 172.21.15.17:0/3108773856 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f683c005030 con 0x7f684806f480 2024-09-10T21:38:08.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:08.786+0000 7f684e200640 1 -- 172.21.15.17:0/3108773856 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6848145070 con 0x7f684806f480 2024-09-10T21:38:08.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:08.786+0000 7f6844c00640 1 -- 172.21.15.17:0/3108773856 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f683c0b2040 con 0x7f684806f480 2024-09-10T21:38:08.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:08.786+0000 7f684e200640 1 -- 172.21.15.17:0/3108773856 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68481455f0 con 0x7f684806f480 2024-09-10T21:38:08.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:08.786+0000 7f684e200640 1 -- 172.21.15.17:0/3108773856 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68480c7870 con 0x7f684806f480 2024-09-10T21:38:08.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:08.786+0000 7f6844c00640 1 -- 172.21.15.17:0/3108773856 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f683c0b5030 con 0x7f684806f480 2024-09-10T21:38:08.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:08.786+0000 7f6844c00640 1 --2- 172.21.15.17:0/3108773856 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f68280725e0 0x7f6828074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:38:08.791 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:08.786+0000 7f6844c00640 1 -- 172.21.15.17:0/3108773856 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f683c12ced0 con 0x7f684806f480 2024-09-10T21:38:08.792 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:08.790+0000 7f6847e00640 1 --2- 172.21.15.17:0/3108773856 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f68280725e0 0x7f6828074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:38:08.792 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:08.790+0000 7f6847e00640 1 --2- 172.21.15.17:0/3108773856 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f68280725e0 0x7f6828074aa0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f68481f17a0 tx=0x7f6834009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:38:08.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:08.794+0000 7f6844c00640 1 -- 172.21.15.17:0/3108773856 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f683c016410 con 0x7f684806f480 2024-09-10T21:38:09.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.070+0000 7f684e200640 1 -- 172.21.15.17:0/3108773856 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f6848071ce0 con 0x7f684806f480 2024-09-10T21:38:09.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.070+0000 7f6844c00640 1 -- 172.21.15.17:0/3108773856 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1558 (secure 0 0 0) 0x7f683c0f8110 con 0x7f684806f480 2024-09-10T21:38:09.073 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":110,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":11,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5e2f023f-d6a6-49f4-a256-f4807dce3739","modified":"2024-09-10T21:23:46.892130Z","created":"2024-09-10T21:23:46.892130Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3301","nonce":0},{"type":"v1","addr":"172.21.15.31:6790","nonce":0}]},"addr":"172.21.15.31:6790/0","public_addr":"172.21.15.31:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-10T21:38:09.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.074+0000 7f684e200640 1 -- 172.21.15.17:0/3108773856 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f68280725e0 msgr2=0x7f6828074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:38:09.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.074+0000 7f684e200640 1 --2- 172.21.15.17:0/3108773856 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f68280725e0 0x7f6828074aa0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f68481f17a0 tx=0x7f6834009040 comp rx=0 tx=0).stop 2024-09-10T21:38:09.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.074+0000 7f684e200640 1 -- 172.21.15.17:0/3108773856 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f684806f480 msgr2=0x7f6848146570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:38:09.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.074+0000 7f684e200640 1 --2- 172.21.15.17:0/3108773856 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f684806f480 0x7f6848146570 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f683c009790 tx=0x7f683c0040c0 comp rx=0 tx=0).stop 2024-09-10T21:38:09.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.074+0000 7f684e200640 1 -- 172.21.15.17:0/3108773856 shutdown_connections 2024-09-10T21:38:09.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.074+0000 7f684e200640 1 --2- 172.21.15.17:0/3108773856 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f68280725e0 0x7f6828074aa0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:38:09.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.074+0000 7f684e200640 1 --2- 172.21.15.17:0/3108773856 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f68480c8070 0x7f68481f70e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:38:09.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.074+0000 7f684e200640 1 --2- 172.21.15.17:0/3108773856 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f684806f860 0x7f68481f46c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:38:09.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.074+0000 7f684e200640 1 --2- 172.21.15.17:0/3108773856 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f684806f480 0x7f6848146570 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:38:09.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.074+0000 7f684e200640 1 -- 172.21.15.17:0/3108773856 >> 172.21.15.17:0/3108773856 conn(0x7f6848070ca0 msgr2=0x7f684814fda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:38:09.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.074+0000 7f684e200640 1 -- 172.21.15.17:0/3108773856 shutdown_connections 2024-09-10T21:38:09.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.074+0000 7f684e200640 1 -- 172.21.15.17:0/3108773856 wait complete. 2024-09-10T21:38:09.097 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-09-10T21:38:09.097 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-09-10T21:38:09.097 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-09-10T21:38:09.097 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-10T21:38:09.269 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.266+0000 7fbbb2a00640 1 -- 172.21.15.17:0/406098329 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbbac072bb0 msgr2=0x7fbbac0c77b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:38:09.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.266+0000 7fbbb2a00640 1 --2- 172.21.15.17:0/406098329 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbbac072bb0 0x7fbbac0c77b0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fbb98009900 tx=0x7fbb9801c8e0 comp rx=0 tx=0).stop 2024-09-10T21:38:09.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.266+0000 7fbbb2a00640 1 -- 172.21.15.17:0/406098329 shutdown_connections 2024-09-10T21:38:09.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.266+0000 7fbbb2a00640 1 --2- 172.21.15.17:0/406098329 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbbac072bb0 0x7fbbac0c77b0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:38:09.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.266+0000 7fbbb2a00640 1 --2- 172.21.15.17:0/406098329 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fbbac070820 0x7fbbac157cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:38:09.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.266+0000 7fbbb2a00640 1 -- 172.21.15.17:0/406098329 >> 172.21.15.17:0/406098329 conn(0x7fbbac06cc20 msgr2=0x7fbbac06d030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:38:09.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.266+0000 7fbbb2a00640 1 -- 172.21.15.17:0/406098329 shutdown_connections 2024-09-10T21:38:09.270 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.266+0000 7fbbb2a00640 1 -- 172.21.15.17:0/406098329 wait complete. 2024-09-10T21:38:09.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.266+0000 7fbbb2a00640 1 Processor -- start 2024-09-10T21:38:09.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.266+0000 7fbbb2a00640 1 -- start start 2024-09-10T21:38:09.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.266+0000 7fbbb2a00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fbbac070820 0x7fbbac147a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:38:09.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.266+0000 7fbbb2a00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbbac070c00 0x7fbbac1f3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:38:09.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.266+0000 7fbbb2a00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbbac072bb0 0x7fbbac1f43f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:38:09.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.266+0000 7fbbb2a00640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fbbac0c8120 con 0x7fbbac070c00 2024-09-10T21:38:09.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.266+0000 7fbbb2a00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fbbac0c4de0 con 0x7fbbac072bb0 2024-09-10T21:38:09.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.266+0000 7fbbb2a00640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7fbbac0c7cf0 con 0x7fbbac070820 2024-09-10T21:38:09.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.270+0000 7fbbb0c00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbbac070c00 0x7fbbac1f3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:38:09.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.270+0000 7fbbb0c00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbbac070c00 0x7fbbac1f3eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:58672/0 (socket says 172.21.15.17:58672) 2024-09-10T21:38:09.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.270+0000 7fbbb0c00640 1 -- 172.21.15.17:0/318761950 learned_addr learned my addr 172.21.15.17:0/318761950 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:38:09.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.270+0000 7fbbb1600640 1 --2- 172.21.15.17:0/318761950 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fbbac070820 0x7fbbac147a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:38:09.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.270+0000 7fbbb2000640 1 --2- 172.21.15.17:0/318761950 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbbac072bb0 0x7fbbac1f43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:38:09.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.270+0000 7fbbb0c00640 1 -- 172.21.15.17:0/318761950 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fbbac070820 msgr2=0x7fbbac147a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:38:09.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.270+0000 7fbbb0c00640 1 --2- 172.21.15.17:0/318761950 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fbbac070820 0x7fbbac147a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:38:09.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.270+0000 7fbbb0c00640 1 -- 172.21.15.17:0/318761950 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbbac072bb0 msgr2=0x7fbbac1f43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:38:09.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.270+0000 7fbbb0c00640 1 --2- 172.21.15.17:0/318761950 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbbac072bb0 0x7fbbac1f43f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:38:09.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.270+0000 7fbbb0c00640 1 -- 172.21.15.17:0/318761950 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbac1f49e0 con 0x7fbbac070c00 2024-09-10T21:38:09.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.270+0000 7fbbb1600640 1 --2- 172.21.15.17:0/318761950 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fbbac070820 0x7fbbac147a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:38:09.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.270+0000 7fbbb2000640 1 --2- 172.21.15.17:0/318761950 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbbac072bb0 0x7fbbac1f43f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:38:09.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.270+0000 7fbbb0c00640 1 --2- 172.21.15.17:0/318761950 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbbac070c00 0x7fbbac1f3eb0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fbb98009850 tx=0x7fbb980a57f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:38:09.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.270+0000 7fbba9600640 1 -- 172.21.15.17:0/318761950 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb980b2040 con 0x7fbbac070c00 2024-09-10T21:38:09.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.270+0000 7fbba9600640 1 -- 172.21.15.17:0/318761950 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbb980043c0 con 0x7fbbac070c00 2024-09-10T21:38:09.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.270+0000 7fbba9600640 1 -- 172.21.15.17:0/318761950 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb98005270 con 0x7fbbac070c00 2024-09-10T21:38:09.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.270+0000 7fbbb2a00640 1 -- 172.21.15.17:0/318761950 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbac1585a0 con 0x7fbbac070c00 2024-09-10T21:38:09.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.270+0000 7fbbb2a00640 1 -- 172.21.15.17:0/318761950 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbac1547a0 con 0x7fbbac070c00 2024-09-10T21:38:09.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.270+0000 7fbbb2a00640 1 --2- 172.21.15.17:0/318761950 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbbac0cb860 0x7fbbac149b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:38:09.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.270+0000 7fbbb2a00640 1 -- 172.21.15.17:0/318761950 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fbbac161a10 con 0x7fbbac0cb860 2024-09-10T21:38:09.276 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.270+0000 7fbbb1600640 1 --2- 172.21.15.17:0/318761950 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbbac0cb860 0x7fbbac149b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:38:09.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.270+0000 7fbbb1600640 1 -- 172.21.15.17:0/318761950 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fbbac157cc0 con 0x7fbbac0cb860 2024-09-10T21:38:09.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.270+0000 7fbbb1600640 1 --2- 172.21.15.17:0/318761950 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbbac0cb860 0x7fbbac149b30 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fbba0010c90 tx=0x7fbba000e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:38:09.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.270+0000 7fbba9600640 1 -- 172.21.15.17:0/318761950 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbba0005020 con 0x7fbbac0cb860 2024-09-10T21:38:09.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.270+0000 7fbba9600640 1 -- 172.21.15.17:0/318761950 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fbba00051c0 con 0x7fbbac0cb860 2024-09-10T21:38:09.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.270+0000 7fbba9600640 1 -- 172.21.15.17:0/318761950 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbbac0cb860 msgr2=0x7fbbac149b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:38:09.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.270+0000 7fbba9600640 1 --2- 172.21.15.17:0/318761950 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbbac0cb860 0x7fbbac149b30 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fbba0010c90 tx=0x7fbba000e720 comp rx=0 tx=0).stop 2024-09-10T21:38:09.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.270+0000 7fbba9600640 1 -- 172.21.15.17:0/318761950 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbbac0cb860 msgr2=0x7fbbac149b30 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:38:09.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.270+0000 7fbba9600640 1 --2- 172.21.15.17:0/318761950 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbbac0cb860 0x7fbbac149b30 secure :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fbba0010c90 tx=0x7fbba000e720 comp rx=0 tx=0).stop 2024-09-10T21:38:09.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.274+0000 7fbba9600640 1 -- 172.21.15.17:0/318761950 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fbb98005410 con 0x7fbbac070c00 2024-09-10T21:38:09.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.274+0000 7fbba9600640 1 --2- 172.21.15.17:0/318761950 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fbb8c0720c0 0x7fbb8c074580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:38:09.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.274+0000 7fbba9600640 1 -- 172.21.15.17:0/318761950 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fbb9812cff0 con 0x7fbbac070c00 2024-09-10T21:38:09.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.274+0000 7fbbb1600640 1 --2- 172.21.15.17:0/318761950 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fbb8c0720c0 0x7fbb8c074580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:38:09.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.274+0000 7fbbb1600640 1 --2- 172.21.15.17:0/318761950 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fbb8c0720c0 0x7fbb8c074580 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fbba000e760 tx=0x7fbba001f040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:38:09.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.282+0000 7fbbb2a00640 1 --2- 172.21.15.17:0/318761950 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbbac158770 0x7fbbac14a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:38:09.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.282+0000 7fbbb2a00640 1 -- 172.21.15.17:0/318761950 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fbbac073110 con 0x7fbbac158770 2024-09-10T21:38:09.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.282+0000 7fbbb2000640 1 --2- 172.21.15.17:0/318761950 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbbac158770 0x7fbbac14a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:38:09.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.282+0000 7fbbb2000640 1 -- 172.21.15.17:0/318761950 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fbbac076660 con 0x7fbbac158770 2024-09-10T21:38:09.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.282+0000 7fbbb2000640 1 --2- 172.21.15.17:0/318761950 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbbac158770 0x7fbbac14a070 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fbbac159e10 tx=0x7fbba400d520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:38:09.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.282+0000 7fbba9600640 1 -- 172.21.15.17:0/318761950 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbba400dde0 con 0x7fbbac158770 2024-09-10T21:38:09.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.282+0000 7fbba9600640 1 -- 172.21.15.17:0/318761950 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4544 (secure 0 0 0) 0x7fbba400dde0 con 0x7fbbac158770 2024-09-10T21:38:09.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.282+0000 7fbba9600640 1 -- 172.21.15.17:0/318761950 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbbac158770 msgr2=0x7fbbac14a070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:38:09.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.282+0000 7fbba9600640 1 --2- 172.21.15.17:0/318761950 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbbac158770 0x7fbbac14a070 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fbbac159e10 tx=0x7fbba400d520 comp rx=0 tx=0).stop 2024-09-10T21:38:09.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.282+0000 7fbba9600640 1 -- 172.21.15.17:0/318761950 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbbac158770 msgr2=0x7fbbac14a070 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:38:09.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.282+0000 7fbba9600640 1 --2- 172.21.15.17:0/318761950 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbbac158770 0x7fbbac14a070 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:38:09.287 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-10T21:38:09.287 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:38:09.287 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:38:09.287 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "leader", 2024-09-10T21:38:09.287 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 110, 2024-09-10T21:38:09.288 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 37647, 2024-09-10T21:38:09.288 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-10T21:38:09.288 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-10T21:38:09.288 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-10T21:38:09.288 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-10T21:38:09.288 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:38:09.288 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 12, 2024-09-10T21:38:09.288 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:38:09.288 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-10T21:38:09.288 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-10T21:38:09.288 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:38:09.288 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:38:09.288 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:38:09.289 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:38:09.289 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:38:09.289 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:38:09.289 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:38:09.289 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:38:09.289 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:38:09.289 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:38:09.289 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:38:09.289 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:38:09.289 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-10T21:38:09.289 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-10T21:38:09.289 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:38:09.289 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:38:09.289 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:38:09.289 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:38:09.290 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:38:09.290 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:38:09.290 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:38:09.290 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:38:09.290 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:38:09.290 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:38:09.290 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:38:09.290 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:38:09.290 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:38:09.290 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-10T21:38:09.290 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-10T21:38:09.290 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-10T21:38:09.290 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-10T21:38:09.290 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-10T21:38:09.290 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "5e2f023f-d6a6-49f4-a256-f4807dce3739", 2024-09-10T21:38:09.291 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:38:09.291 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:38:09.291 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-10T21:38:09.291 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-10T21:38:09.291 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-10T21:38:09.291 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-10T21:38:09.291 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-10T21:38:09.291 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-10T21:38:09.291 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-10T21:38:09.291 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:38:09.291 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-10T21:38:09.291 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:38:09.291 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:38:09.291 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:38:09.292 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:38:09.292 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:38:09.292 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:38:09.292 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:38:09.292 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:38:09.292 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:38:09.292 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:38:09.292 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:38:09.292 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:38:09.292 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-10T21:38:09.292 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:38:09.292 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-10T21:38:09.292 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:38:09.293 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:38:09.293 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:38:09.293 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:38:09.293 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:38:09.293 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:38:09.293 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:38:09.293 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-10T21:38:09.293 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:38:09.293 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:38:09.293 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:38:09.293 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:38:09.293 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-10T21:38:09.293 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:38:09.294 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:38:09.294 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:38:09.294 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:38:09.294 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-10T21:38:09.294 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-10T21:38:09.294 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:38:09.294 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:38:09.294 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:38:09.294 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:38:09.294 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:38:09.294 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:38:09.294 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:38:09.294 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:38:09.294 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:38:09.295 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:38:09.295 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:38:09.295 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3300", 2024-09-10T21:38:09.295 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:38:09.295 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:38:09.295 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:38:09.295 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:38:09.295 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789", 2024-09-10T21:38:09.295 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:38:09.295 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:38:09.295 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:38:09.296 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:38:09.296 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789/0", 2024-09-10T21:38:09.296 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6789/0", 2024-09-10T21:38:09.296 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:38:09.296 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:38:09.296 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:38:09.296 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:38:09.296 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:38:09.296 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:38:09.296 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:38:09.296 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:38:09.296 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:38:09.296 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:38:09.296 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:38:09.297 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3301", 2024-09-10T21:38:09.297 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:38:09.297 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:38:09.297 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:38:09.297 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:38:09.297 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790", 2024-09-10T21:38:09.297 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:38:09.297 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:38:09.297 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:38:09.297 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:38:09.297 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790/0", 2024-09-10T21:38:09.297 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6790/0", 2024-09-10T21:38:09.298 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:38:09.298 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:38:09.298 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:38:09.298 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:38:09.298 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:38:09.298 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:38:09.298 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-10T21:38:09.298 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-10T21:38:09.298 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:38:09.298 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:38:09.298 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:38:09.299 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:38:09.299 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:38:09.299 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:38:09.299 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-10T21:38:09.299 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:38:09.299 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:38:09.299 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:38:09.299 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 2 2024-09-10T21:38:09.299 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:38:09.299 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:38:09.299 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:38:09.299 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-10T21:38:09.299 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-10T21:38:09.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.282+0000 7fbbb2a00640 1 -- 172.21.15.17:0/318761950 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fbb8c0720c0 msgr2=0x7fbb8c074580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:38:09.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.282+0000 7fbbb2a00640 1 --2- 172.21.15.17:0/318761950 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fbb8c0720c0 0x7fbb8c074580 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fbba000e760 tx=0x7fbba001f040 comp rx=0 tx=0).stop 2024-09-10T21:38:09.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.282+0000 7fbbb2a00640 1 -- 172.21.15.17:0/318761950 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbbac070c00 msgr2=0x7fbbac1f3eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:38:09.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.282+0000 7fbbb2000640 1 -- 172.21.15.17:0/318761950 reap_dead start 2024-09-10T21:38:09.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.282+0000 7fbbb2a00640 1 --2- 172.21.15.17:0/318761950 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbbac070c00 0x7fbbac1f3eb0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fbb98009850 tx=0x7fbb980a57f0 comp rx=0 tx=0).stop 2024-09-10T21:38:09.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.282+0000 7fbbb2a00640 1 -- 172.21.15.17:0/318761950 shutdown_connections 2024-09-10T21:38:09.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.282+0000 7fbbb2a00640 1 --2- 172.21.15.17:0/318761950 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbbac070c00 0x7fbbac1f3eb0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:38:09.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.282+0000 7fbbb2a00640 1 -- 172.21.15.17:0/318761950 >> 172.21.15.17:0/318761950 conn(0x7fbbac06cc20 msgr2=0x7fbbac0c55e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:38:09.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.282+0000 7fbbb2a00640 1 -- 172.21.15.17:0/318761950 shutdown_connections 2024-09-10T21:38:09.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.282+0000 7fbbb2a00640 1 -- 172.21.15.17:0/318761950 wait complete. 2024-09-10T21:38:09.310 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-10T21:38:09.310 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-09-10T21:38:09.310 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-10T21:38:09.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.486+0000 7ff73b200640 1 -- 172.21.15.17:0/1520792307 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff73c0ca4a0 msgr2=0x7ff73c070c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:38:09.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.486+0000 7ff73b200640 1 --2- 172.21.15.17:0/1520792307 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff73c0ca4a0 0x7ff73c070c10 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff73c0cde70 tx=0x7ff72801c7a0 comp rx=0 tx=0).stop 2024-09-10T21:38:09.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.490+0000 7ff73b200640 1 -- 172.21.15.17:0/1520792307 shutdown_connections 2024-09-10T21:38:09.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.490+0000 7ff73b200640 1 --2- 172.21.15.17:0/1520792307 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff73c0c9520 0x7ff73c071150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:38:09.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.490+0000 7ff73b200640 1 --2- 172.21.15.17:0/1520792307 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff73c0ca4a0 0x7ff73c070c10 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:38:09.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.490+0000 7ff73b200640 1 --2- 172.21.15.17:0/1520792307 >> v2:172.21.15.31:3300/0 conn(0x7ff73c1523d0 0x7ff73c0c9f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:38:09.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.490+0000 7ff73b200640 1 -- 172.21.15.17:0/1520792307 >> 172.21.15.17:0/1520792307 conn(0x7ff73c068990 msgr2=0x7ff73c068da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:38:09.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.490+0000 7ff73b200640 1 -- 172.21.15.17:0/1520792307 shutdown_connections 2024-09-10T21:38:09.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.490+0000 7ff73b200640 1 -- 172.21.15.17:0/1520792307 wait complete. 2024-09-10T21:38:09.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.490+0000 7ff73b200640 1 Processor -- start 2024-09-10T21:38:09.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.490+0000 7ff73b200640 1 -- start start 2024-09-10T21:38:09.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.490+0000 7ff73b200640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff73c0c9520 0x7ff73c143a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:38:09.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.490+0000 7ff73b200640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff73c0ca4a0 0x7ff73c143f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:38:09.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.490+0000 7ff73b200640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff73c1523d0 0x7ff73c06e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:38:09.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.490+0000 7ff73b200640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7ff73c0726b0 con 0x7ff73c0ca4a0 2024-09-10T21:38:09.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.490+0000 7ff73b200640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7ff73c0cb3c0 con 0x7ff73c0c9520 2024-09-10T21:38:09.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.490+0000 7ff73b200640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7ff73c072830 con 0x7ff73c1523d0 2024-09-10T21:38:09.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.490+0000 7ff739400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff73c0ca4a0 0x7ff73c143f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:38:09.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.490+0000 7ff739e00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff73c0c9520 0x7ff73c143a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:38:09.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.490+0000 7ff73a800640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff73c1523d0 0x7ff73c06e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:38:09.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.490+0000 7ff739400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff73c0ca4a0 0x7ff73c143f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:58726/0 (socket says 172.21.15.17:58726) 2024-09-10T21:38:09.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.490+0000 7ff739400640 1 -- 172.21.15.17:0/2887154917 learned_addr learned my addr 172.21.15.17:0/2887154917 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:38:09.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.490+0000 7ff73a800640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff73c1523d0 0x7ff73c06e830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3301/0 says I am v2:172.21.15.17:39828/0 (socket says 172.21.15.17:39828) 2024-09-10T21:38:09.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.490+0000 7ff739e00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff73c0c9520 0x7ff73c143a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.17:54486/0 (socket says 172.21.15.17:54486) 2024-09-10T21:38:09.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.490+0000 7ff739400640 1 -- 172.21.15.17:0/2887154917 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff73c1523d0 msgr2=0x7ff73c06e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:38:09.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.490+0000 7ff739400640 1 --2- 172.21.15.17:0/2887154917 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff73c1523d0 0x7ff73c06e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:38:09.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.490+0000 7ff739400640 1 -- 172.21.15.17:0/2887154917 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff73c0c9520 msgr2=0x7ff73c143a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:38:09.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.490+0000 7ff739400640 1 --2- 172.21.15.17:0/2887154917 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff73c0c9520 0x7ff73c143a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:38:09.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.490+0000 7ff739400640 1 -- 172.21.15.17:0/2887154917 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff73c148030 con 0x7ff73c0ca4a0 2024-09-10T21:38:09.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.494+0000 7ff739400640 1 --2- 172.21.15.17:0/2887154917 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff73c0ca4a0 0x7ff73c143f80 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff72801ae50 tx=0x7ff7280a5510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:38:09.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.494+0000 7ff732000640 1 -- 172.21.15.17:0/2887154917 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7280ba020 con 0x7ff73c0ca4a0 2024-09-10T21:38:09.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.494+0000 7ff732000640 1 -- 172.21.15.17:0/2887154917 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff728004470 con 0x7ff73c0ca4a0 2024-09-10T21:38:09.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.494+0000 7ff732000640 1 -- 172.21.15.17:0/2887154917 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7280b2040 con 0x7ff73c0ca4a0 2024-09-10T21:38:09.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.494+0000 7ff73b200640 1 -- 172.21.15.17:0/2887154917 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff73c149570 con 0x7ff73c0ca4a0 2024-09-10T21:38:09.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.494+0000 7ff73b200640 1 -- 172.21.15.17:0/2887154917 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff73c1f48c0 con 0x7ff73c0ca4a0 2024-09-10T21:38:09.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.494+0000 7ff73b200640 1 --2- 172.21.15.17:0/2887154917 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff73c06e3a0 0x7ff73c06ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:38:09.498 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.494+0000 7ff73b200640 1 -- 172.21.15.17:0/2887154917 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7ff73c069ef0 con 0x7ff73c06e3a0 2024-09-10T21:38:09.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.494+0000 7ff739e00640 1 --2- 172.21.15.17:0/2887154917 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff73c06e3a0 0x7ff73c06ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:38:09.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.494+0000 7ff739e00640 1 -- 172.21.15.17:0/2887154917 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff73c0c9f60 con 0x7ff73c06e3a0 2024-09-10T21:38:09.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.494+0000 7ff739e00640 1 --2- 172.21.15.17:0/2887154917 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff73c06e3a0 0x7ff73c06ed70 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7ff72c0109a0 tx=0x7ff72c010e60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:38:09.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.494+0000 7ff732000640 1 -- 172.21.15.17:0/2887154917 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7ff728005290 con 0x7ff73c0ca4a0 2024-09-10T21:38:09.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.498+0000 7ff732000640 1 --2- 172.21.15.17:0/2887154917 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7ff71c0725e0 0x7ff71c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:38:09.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.498+0000 7ff732000640 1 -- 172.21.15.17:0/2887154917 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7ff72812d490 con 0x7ff73c0ca4a0 2024-09-10T21:38:09.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.498+0000 7ff732000640 1 -- 172.21.15.17:0/2887154917 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff72c00edb0 con 0x7ff73c06e3a0 2024-09-10T21:38:09.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.498+0000 7ff732000640 1 -- 172.21.15.17:0/2887154917 <== mon.1 v2:172.21.15.31:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7ff72c005020 con 0x7ff73c06e3a0 2024-09-10T21:38:09.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.498+0000 7ff732000640 1 -- 172.21.15.17:0/2887154917 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff73c06e3a0 msgr2=0x7ff73c06ed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:38:09.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.498+0000 7ff732000640 1 --2- 172.21.15.17:0/2887154917 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff73c06e3a0 0x7ff73c06ed70 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7ff72c0109a0 tx=0x7ff72c010e60 comp rx=0 tx=0).stop 2024-09-10T21:38:09.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.498+0000 7ff732000640 1 -- 172.21.15.17:0/2887154917 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff73c06e3a0 msgr2=0x7ff73c06ed70 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:38:09.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.498+0000 7ff732000640 1 --2- 172.21.15.17:0/2887154917 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff73c06e3a0 0x7ff73c06ed70 secure :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7ff72c0109a0 tx=0x7ff72c010e60 comp rx=0 tx=0).stop 2024-09-10T21:38:09.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.498+0000 7ff73a800640 1 --2- 172.21.15.17:0/2887154917 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7ff71c0725e0 0x7ff71c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:38:09.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.498+0000 7ff73a800640 1 --2- 172.21.15.17:0/2887154917 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7ff71c0725e0 0x7ff71c074aa0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7ff734009df0 tx=0x7ff734009540 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:38:09.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.506+0000 7ff73b200640 1 --2- 172.21.15.17:0/2887154917 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff73c068e00 0x7ff73c148710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:38:09.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.506+0000 7ff73b200640 1 -- 172.21.15.17:0/2887154917 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7ff73c155b40 con 0x7ff73c068e00 2024-09-10T21:38:09.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.506+0000 7ff739e00640 1 --2- 172.21.15.17:0/2887154917 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff73c068e00 0x7ff73c148710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:38:09.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.506+0000 7ff739e00640 1 -- 172.21.15.17:0/2887154917 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7ff73c157350 con 0x7ff73c068e00 2024-09-10T21:38:09.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.506+0000 7ff739e00640 1 --2- 172.21.15.17:0/2887154917 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff73c068e00 0x7ff73c148710 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ff72c00bfd0 tx=0x7ff72c0046d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:38:09.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.506+0000 7ff732000640 1 -- 172.21.15.17:0/2887154917 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff72c004cd0 con 0x7ff73c068e00 2024-09-10T21:38:09.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.506+0000 7ff732000640 1 -- 172.21.15.17:0/2887154917 <== mon.1 v2:172.21.15.31:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5044 (secure 0 0 0) 0x7ff72c004cd0 con 0x7ff73c068e00 2024-09-10T21:38:09.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.506+0000 7ff732000640 1 -- 172.21.15.17:0/2887154917 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff73c068e00 msgr2=0x7ff73c148710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:38:09.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.506+0000 7ff732000640 1 --2- 172.21.15.17:0/2887154917 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff73c068e00 0x7ff73c148710 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ff72c00bfd0 tx=0x7ff72c0046d0 comp rx=0 tx=0).stop 2024-09-10T21:38:09.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.506+0000 7ff732000640 1 -- 172.21.15.17:0/2887154917 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff73c068e00 msgr2=0x7ff73c148710 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:38:09.511 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.506+0000 7ff732000640 1 --2- 172.21.15.17:0/2887154917 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff73c068e00 0x7ff73c148710 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:38:09.511 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-10T21:38:09.511 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:38:09.511 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:38:09.512 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-10T21:38:09.512 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 110, 2024-09-10T21:38:09.512 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 208892, 2024-09-10T21:38:09.512 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-10T21:38:09.512 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-10T21:38:09.512 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-10T21:38:09.512 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-10T21:38:09.513 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:38:09.513 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 12, 2024-09-10T21:38:09.513 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:38:09.513 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-10T21:38:09.513 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-10T21:38:09.513 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:38:09.513 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:38:09.513 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:38:09.513 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:38:09.513 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:38:09.514 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:38:09.514 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:38:09.514 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:38:09.514 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:38:09.514 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:38:09.514 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:38:09.514 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:38:09.514 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-10T21:38:09.514 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-10T21:38:09.515 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:38:09.515 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:38:09.515 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:38:09.515 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:38:09.515 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:38:09.515 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:38:09.515 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:38:09.515 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:38:09.515 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:38:09.515 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:38:09.515 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:38:09.515 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:38:09.515 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:38:09.515 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-10T21:38:09.515 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-10T21:38:09.516 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-10T21:38:09.516 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-10T21:38:09.516 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-10T21:38:09.516 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "5e2f023f-d6a6-49f4-a256-f4807dce3739", 2024-09-10T21:38:09.516 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:38:09.516 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:38:09.516 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-10T21:38:09.516 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-10T21:38:09.516 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-10T21:38:09.516 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-10T21:38:09.516 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-10T21:38:09.516 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-10T21:38:09.516 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-10T21:38:09.516 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:38:09.517 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-10T21:38:09.517 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:38:09.517 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:38:09.517 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:38:09.517 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:38:09.517 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:38:09.517 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:38:09.517 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:38:09.517 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:38:09.517 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:38:09.517 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:38:09.517 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:38:09.517 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:38:09.517 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-10T21:38:09.517 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:38:09.518 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-10T21:38:09.518 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:38:09.518 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:38:09.518 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:38:09.518 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:38:09.518 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:38:09.518 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:38:09.518 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:38:09.518 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-10T21:38:09.518 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:38:09.518 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:38:09.518 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:38:09.518 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:38:09.518 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-10T21:38:09.518 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:38:09.519 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:38:09.519 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:38:09.519 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:38:09.519 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-10T21:38:09.519 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-10T21:38:09.519 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:38:09.519 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:38:09.519 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:38:09.519 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:38:09.519 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:38:09.520 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:38:09.520 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:38:09.520 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:38:09.520 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:38:09.520 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:38:09.520 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:38:09.520 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3300", 2024-09-10T21:38:09.520 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:38:09.520 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:38:09.520 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:38:09.520 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:38:09.520 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789", 2024-09-10T21:38:09.520 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:38:09.521 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:38:09.521 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:38:09.521 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:38:09.521 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789/0", 2024-09-10T21:38:09.521 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6789/0", 2024-09-10T21:38:09.521 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:38:09.521 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:38:09.521 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:38:09.521 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:38:09.521 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:38:09.521 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:38:09.521 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:38:09.521 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:38:09.521 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:38:09.522 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:38:09.522 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:38:09.522 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3301", 2024-09-10T21:38:09.522 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:38:09.522 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:38:09.522 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:38:09.522 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:38:09.522 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790", 2024-09-10T21:38:09.522 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:38:09.522 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:38:09.522 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:38:09.522 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:38:09.522 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790/0", 2024-09-10T21:38:09.522 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6790/0", 2024-09-10T21:38:09.522 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:38:09.523 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:38:09.523 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:38:09.523 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:38:09.523 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:38:09.523 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:38:09.523 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-10T21:38:09.523 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-10T21:38:09.523 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:38:09.523 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:38:09.523 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:38:09.523 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:38:09.523 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:38:09.523 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:38:09.524 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": [ 2024-09-10T21:38:09.524 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:38:09.524 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:38:09.524 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:38:09.524 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 2 2024-09-10T21:38:09.524 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:38:09.524 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:38:09.524 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-10T21:38:09.524 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:38:09.524 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:38:09.524 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:38:09.524 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 4 2024-09-10T21:38:09.524 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:38:09.524 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:38:09.524 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-10T21:38:09.525 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:38:09.525 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:38:09.525 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:38:09.525 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 4 2024-09-10T21:38:09.525 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:38:09.525 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:38:09.525 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": [ 2024-09-10T21:38:09.525 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:38:09.525 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:38:09.525 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:38:09.525 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:38:09.526 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:38:09.526 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:38:09.526 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:38:09.526 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-10T21:38:09.526 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-10T21:38:09.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.506+0000 7ff73b200640 1 -- 172.21.15.17:0/2887154917 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7ff71c0725e0 msgr2=0x7ff71c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:38:09.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.506+0000 7ff73b200640 1 --2- 172.21.15.17:0/2887154917 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7ff71c0725e0 0x7ff71c074aa0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7ff734009df0 tx=0x7ff734009540 comp rx=0 tx=0).stop 2024-09-10T21:38:09.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.506+0000 7ff73b200640 1 -- 172.21.15.17:0/2887154917 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff73c0ca4a0 msgr2=0x7ff73c143f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:38:09.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.506+0000 7ff73b200640 1 --2- 172.21.15.17:0/2887154917 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff73c0ca4a0 0x7ff73c143f80 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff72801ae50 tx=0x7ff7280a5510 comp rx=0 tx=0).stop 2024-09-10T21:38:09.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.506+0000 7ff73a800640 1 -- 172.21.15.17:0/2887154917 reap_dead start 2024-09-10T21:38:09.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.506+0000 7ff73b200640 1 -- 172.21.15.17:0/2887154917 shutdown_connections 2024-09-10T21:38:09.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.506+0000 7ff73b200640 1 --2- 172.21.15.17:0/2887154917 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff73c0ca4a0 0x7ff73c143f80 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:38:09.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.506+0000 7ff73b200640 1 -- 172.21.15.17:0/2887154917 >> 172.21.15.17:0/2887154917 conn(0x7ff73c068990 msgr2=0x7ff73c14a390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:38:09.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.506+0000 7ff73b200640 1 -- 172.21.15.17:0/2887154917 shutdown_connections 2024-09-10T21:38:09.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.506+0000 7ff73b200640 1 -- 172.21.15.17:0/2887154917 wait complete. 2024-09-10T21:38:09.531 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-10T21:38:09.531 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-09-10T21:38:09.531 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-10T21:38:09.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.690+0000 7fe7c7e00640 1 -- 172.21.15.17:0/1383210034 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe7c015abb0 msgr2=0x7fe7c01613a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:38:09.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.690+0000 7fe7c7e00640 1 --2- 172.21.15.17:0/1383210034 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe7c015abb0 0x7fe7c01613a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fe7b8009c80 tx=0x7fe7b801cda0 comp rx=0 tx=0).stop 2024-09-10T21:38:09.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.690+0000 7fe7c7e00640 1 -- 172.21.15.17:0/1383210034 shutdown_connections 2024-09-10T21:38:09.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.690+0000 7fe7c7e00640 1 --2- 172.21.15.17:0/1383210034 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe7c015abb0 0x7fe7c01613a0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:38:09.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.690+0000 7fe7c7e00640 1 --2- 172.21.15.17:0/1383210034 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe7c0073cf0 0x7fe7c00740d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:38:09.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.690+0000 7fe7c7e00640 1 -- 172.21.15.17:0/1383210034 >> 172.21.15.17:0/1383210034 conn(0x7fe7c00700d0 msgr2=0x7fe7c00704e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:38:09.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.690+0000 7fe7c7e00640 1 -- 172.21.15.17:0/1383210034 shutdown_connections 2024-09-10T21:38:09.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.690+0000 7fe7c7e00640 1 -- 172.21.15.17:0/1383210034 wait complete. 2024-09-10T21:38:09.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.694+0000 7fe7c7e00640 1 Processor -- start 2024-09-10T21:38:09.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.694+0000 7fe7c7e00640 1 -- start start 2024-09-10T21:38:09.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.694+0000 7fe7c7e00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe7c0073cf0 0x7fe7c0148e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:38:09.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.694+0000 7fe7c7e00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe7c00c0fd0 0x7fe7c0160900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:38:09.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.694+0000 7fe7c7e00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe7c015abb0 0x7fe7c0145920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:38:09.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.694+0000 7fe7c7e00640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fe7c00c7dd0 con 0x7fe7c00c0fd0 2024-09-10T21:38:09.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.694+0000 7fe7c7e00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fe7c00c8370 con 0x7fe7c015abb0 2024-09-10T21:38:09.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.694+0000 7fe7c7e00640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7fe7c00c7f50 con 0x7fe7c0073cf0 2024-09-10T21:38:09.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.694+0000 7fe7c4a00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe7c00c0fd0 0x7fe7c0160900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:38:09.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.694+0000 7fe7c4a00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe7c00c0fd0 0x7fe7c0160900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:58748/0 (socket says 172.21.15.17:58748) 2024-09-10T21:38:09.700 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.694+0000 7fe7c4a00640 1 -- 172.21.15.17:0/2393876503 learned_addr learned my addr 172.21.15.17:0/2393876503 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:38:09.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.694+0000 7fe7c5400640 1 --2- 172.21.15.17:0/2393876503 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe7c0073cf0 0x7fe7c0148e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:38:09.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.694+0000 7fe7c5e00640 1 --2- 172.21.15.17:0/2393876503 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe7c015abb0 0x7fe7c0145920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:38:09.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.698+0000 7fe7c4a00640 1 -- 172.21.15.17:0/2393876503 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe7c0073cf0 msgr2=0x7fe7c0148e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:38:09.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.698+0000 7fe7c4a00640 1 --2- 172.21.15.17:0/2393876503 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe7c0073cf0 0x7fe7c0148e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:38:09.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.698+0000 7fe7c4a00640 1 -- 172.21.15.17:0/2393876503 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe7c015abb0 msgr2=0x7fe7c0145920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:38:09.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.698+0000 7fe7c4a00640 1 --2- 172.21.15.17:0/2393876503 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe7c015abb0 0x7fe7c0145920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:38:09.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.698+0000 7fe7c4a00640 1 -- 172.21.15.17:0/2393876503 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7c01f3500 con 0x7fe7c00c0fd0 2024-09-10T21:38:09.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.698+0000 7fe7c4a00640 1 --2- 172.21.15.17:0/2393876503 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe7c00c0fd0 0x7fe7c0160900 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fe7ac00c480 tx=0x7fe7ac00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:38:09.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.698+0000 7fe7bd600640 1 -- 172.21.15.17:0/2393876503 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7ac016020 con 0x7fe7c00c0fd0 2024-09-10T21:38:09.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.698+0000 7fe7bd600640 1 -- 172.21.15.17:0/2393876503 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe7ac012940 con 0x7fe7c00c0fd0 2024-09-10T21:38:09.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.698+0000 7fe7bd600640 1 -- 172.21.15.17:0/2393876503 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7ac012d90 con 0x7fe7c00c0fd0 2024-09-10T21:38:09.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.698+0000 7fe7c7e00640 1 -- 172.21.15.17:0/2393876503 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7c0147010 con 0x7fe7c00c0fd0 2024-09-10T21:38:09.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.698+0000 7fe7c7e00640 1 -- 172.21.15.17:0/2393876503 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7c01f37c0 con 0x7fe7c00c0fd0 2024-09-10T21:38:09.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.698+0000 7fe7c7e00640 1 --2- 172.21.15.17:0/2393876503 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe7c0146b40 0x7fe7c00c6620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:38:09.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.698+0000 7fe7c7e00640 1 -- 172.21.15.17:0/2393876503 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7fe7c00752a0 con 0x7fe7c0146b40 2024-09-10T21:38:09.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.698+0000 7fe7c5400640 1 --2- 172.21.15.17:0/2393876503 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe7c0146b40 0x7fe7c00c6620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:38:09.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.698+0000 7fe7c5400640 1 -- 172.21.15.17:0/2393876503 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe7c00725c0 con 0x7fe7c0146b40 2024-09-10T21:38:09.703 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.698+0000 7fe7c5400640 1 --2- 172.21.15.17:0/2393876503 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe7c0146b40 0x7fe7c00c6620 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fe7b400ed90 tx=0x7fe7b400d810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:38:09.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.702+0000 7fe7bd600640 1 -- 172.21.15.17:0/2393876503 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7b4004320 con 0x7fe7c0146b40 2024-09-10T21:38:09.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.702+0000 7fe7bd600640 1 -- 172.21.15.17:0/2393876503 <== mon.2 v2:172.21.15.31:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fe7b40044c0 con 0x7fe7c0146b40 2024-09-10T21:38:09.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.702+0000 7fe7bd600640 1 -- 172.21.15.17:0/2393876503 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe7c0146b40 msgr2=0x7fe7c00c6620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:38:09.704 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.702+0000 7fe7bd600640 1 --2- 172.21.15.17:0/2393876503 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe7c0146b40 0x7fe7c00c6620 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fe7b400ed90 tx=0x7fe7b400d810 comp rx=0 tx=0).stop 2024-09-10T21:38:09.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.702+0000 7fe7bd600640 1 -- 172.21.15.17:0/2393876503 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe7c0146b40 msgr2=0x7fe7c00c6620 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:38:09.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.702+0000 7fe7bd600640 1 --2- 172.21.15.17:0/2393876503 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe7c0146b40 0x7fe7c00c6620 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:38:09.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.702+0000 7fe7bd600640 1 -- 172.21.15.17:0/2393876503 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fe7ac01b400 con 0x7fe7c00c0fd0 2024-09-10T21:38:09.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.702+0000 7fe7bd600640 1 --2- 172.21.15.17:0/2393876503 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fe7a00720d0 0x7fe7a0074590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:38:09.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.702+0000 7fe7bd600640 1 -- 172.21.15.17:0/2393876503 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fe7ac093740 con 0x7fe7c00c0fd0 2024-09-10T21:38:09.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.702+0000 7fe7c5400640 1 --2- 172.21.15.17:0/2393876503 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fe7a00720d0 0x7fe7a0074590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:38:09.706 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.702+0000 7fe7c5400640 1 --2- 172.21.15.17:0/2393876503 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fe7a00720d0 0x7fe7a0074590 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fe7b400ddb0 tx=0x7fe7b400f000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:38:09.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.706+0000 7fe7c7e00640 1 --2- 172.21.15.17:0/2393876503 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe7c01f39c0 0x7fe7c014a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:38:09.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.706+0000 7fe7c7e00640 1 -- 172.21.15.17:0/2393876503 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7fe7c00c8150 con 0x7fe7c01f39c0 2024-09-10T21:38:09.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.710+0000 7fe7c5e00640 1 --2- 172.21.15.17:0/2393876503 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe7c01f39c0 0x7fe7c014a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:38:09.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.710+0000 7fe7c5e00640 1 -- 172.21.15.17:0/2393876503 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fe7c015a640 con 0x7fe7c01f39c0 2024-09-10T21:38:09.712 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.710+0000 7fe7c5e00640 1 --2- 172.21.15.17:0/2393876503 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe7c01f39c0 0x7fe7c014a7e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe7c0072e60 tx=0x7fe7b8009350 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:38:09.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.710+0000 7fe7bd600640 1 -- 172.21.15.17:0/2393876503 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7b80b2040 con 0x7fe7c01f39c0 2024-09-10T21:38:09.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.710+0000 7fe7bd600640 1 -- 172.21.15.17:0/2393876503 <== mon.2 v2:172.21.15.31:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5043 (secure 0 0 0) 0x7fe7b8003c40 con 0x7fe7c01f39c0 2024-09-10T21:38:09.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.710+0000 7fe7bd600640 1 -- 172.21.15.17:0/2393876503 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe7c01f39c0 msgr2=0x7fe7c014a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:38:09.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.710+0000 7fe7bd600640 1 --2- 172.21.15.17:0/2393876503 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe7c01f39c0 0x7fe7c014a7e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe7c0072e60 tx=0x7fe7b8009350 comp rx=0 tx=0).stop 2024-09-10T21:38:09.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.710+0000 7fe7bd600640 1 -- 172.21.15.17:0/2393876503 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe7c01f39c0 msgr2=0x7fe7c014a7e0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:38:09.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.710+0000 7fe7bd600640 1 --2- 172.21.15.17:0/2393876503 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe7c01f39c0 0x7fe7c014a7e0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:38:09.714 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-10T21:38:09.715 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:38:09.715 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:38:09.715 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-10T21:38:09.715 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 110, 2024-09-10T21:38:09.715 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 96058, 2024-09-10T21:38:09.715 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-10T21:38:09.715 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-10T21:38:09.715 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-10T21:38:09.716 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-10T21:38:09.716 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:38:09.716 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 12, 2024-09-10T21:38:09.716 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:38:09.716 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-10T21:38:09.716 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-10T21:38:09.716 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:38:09.716 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:38:09.716 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:38:09.716 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:38:09.716 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:38:09.716 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:38:09.716 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:38:09.716 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:38:09.717 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:38:09.717 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:38:09.717 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:38:09.717 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:38:09.717 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-10T21:38:09.717 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-10T21:38:09.717 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:38:09.717 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:38:09.717 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:38:09.717 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:38:09.717 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:38:09.717 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:38:09.717 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:38:09.717 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:38:09.718 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:38:09.718 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:38:09.718 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:38:09.718 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:38:09.718 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:38:09.718 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-10T21:38:09.718 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-10T21:38:09.718 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-10T21:38:09.718 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-10T21:38:09.718 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-10T21:38:09.718 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "5e2f023f-d6a6-49f4-a256-f4807dce3739", 2024-09-10T21:38:09.718 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:38:09.718 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:38:09.718 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-10T21:38:09.719 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-10T21:38:09.719 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-10T21:38:09.719 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-10T21:38:09.719 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-10T21:38:09.719 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-10T21:38:09.719 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-10T21:38:09.719 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:38:09.719 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-10T21:38:09.719 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:38:09.719 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:38:09.719 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:38:09.719 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:38:09.719 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:38:09.719 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:38:09.720 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:38:09.720 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:38:09.720 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:38:09.720 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:38:09.720 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:38:09.720 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:38:09.720 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-10T21:38:09.720 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:38:09.720 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-10T21:38:09.720 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:38:09.720 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:38:09.721 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:38:09.721 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:38:09.721 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:38:09.721 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:38:09.721 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:38:09.721 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-10T21:38:09.721 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:38:09.721 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:38:09.721 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:38:09.721 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:38:09.721 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-10T21:38:09.721 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:38:09.721 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:38:09.722 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:38:09.722 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:38:09.722 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-10T21:38:09.722 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-10T21:38:09.722 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:38:09.722 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:38:09.722 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:38:09.722 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:38:09.722 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:38:09.722 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:38:09.722 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:38:09.722 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:38:09.722 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:38:09.722 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:38:09.723 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:38:09.723 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3300", 2024-09-10T21:38:09.723 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:38:09.723 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:38:09.723 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:38:09.723 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:38:09.723 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789", 2024-09-10T21:38:09.723 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:38:09.723 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:38:09.723 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:38:09.723 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:38:09.723 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789/0", 2024-09-10T21:38:09.723 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6789/0", 2024-09-10T21:38:09.724 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:38:09.724 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:38:09.724 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:38:09.724 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:38:09.724 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:38:09.724 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:38:09.724 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:38:09.724 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:38:09.724 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:38:09.724 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:38:09.724 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:38:09.725 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3301", 2024-09-10T21:38:09.725 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:38:09.725 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:38:09.725 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:38:09.725 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:38:09.725 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790", 2024-09-10T21:38:09.725 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:38:09.725 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:38:09.725 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:38:09.725 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:38:09.725 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790/0", 2024-09-10T21:38:09.725 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6790/0", 2024-09-10T21:38:09.725 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:38:09.726 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:38:09.726 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:38:09.726 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:38:09.726 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:38:09.726 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:38:09.726 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-10T21:38:09.726 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-10T21:38:09.726 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:38:09.726 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:38:09.726 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:38:09.726 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:38:09.726 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:38:09.726 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:38:09.727 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": [ 2024-09-10T21:38:09.727 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:38:09.727 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:38:09.727 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:38:09.727 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 4 2024-09-10T21:38:09.727 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:38:09.727 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:38:09.727 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-10T21:38:09.727 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:38:09.727 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:38:09.727 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:38:09.727 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 4 2024-09-10T21:38:09.727 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:38:09.728 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:38:09.728 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-10T21:38:09.728 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:38:09.728 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:38:09.728 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:38:09.728 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 3 2024-09-10T21:38:09.728 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:38:09.728 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:38:09.728 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": [ 2024-09-10T21:38:09.728 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:38:09.728 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:38:09.728 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:38:09.728 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:38:09.729 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:38:09.729 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:38:09.729 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:38:09.729 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-10T21:38:09.729 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-10T21:38:09.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.710+0000 7fe7c7e00640 1 -- 172.21.15.17:0/2393876503 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fe7a00720d0 msgr2=0x7fe7a0074590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:38:09.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.710+0000 7fe7c7e00640 1 --2- 172.21.15.17:0/2393876503 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fe7a00720d0 0x7fe7a0074590 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fe7b400ddb0 tx=0x7fe7b400f000 comp rx=0 tx=0).stop 2024-09-10T21:38:09.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.710+0000 7fe7c7e00640 1 -- 172.21.15.17:0/2393876503 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe7c00c0fd0 msgr2=0x7fe7c0160900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:38:09.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.710+0000 7fe7c7e00640 1 --2- 172.21.15.17:0/2393876503 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe7c00c0fd0 0x7fe7c0160900 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fe7ac00c480 tx=0x7fe7ac00c940 comp rx=0 tx=0).stop 2024-09-10T21:38:09.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.710+0000 7fe7c5e00640 1 -- 172.21.15.17:0/2393876503 reap_dead start 2024-09-10T21:38:09.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.710+0000 7fe7c7e00640 1 -- 172.21.15.17:0/2393876503 shutdown_connections 2024-09-10T21:38:09.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.710+0000 7fe7c7e00640 1 --2- 172.21.15.17:0/2393876503 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe7c00c0fd0 0x7fe7c0160900 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:38:09.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.710+0000 7fe7c7e00640 1 -- 172.21.15.17:0/2393876503 >> 172.21.15.17:0/2393876503 conn(0x7fe7c00700d0 msgr2=0x7fe7c014c360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:38:09.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.710+0000 7fe7c7e00640 1 -- 172.21.15.17:0/2393876503 shutdown_connections 2024-09-10T21:38:09.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.710+0000 7fe7c7e00640 1 -- 172.21.15.17:0/2393876503 wait complete. 2024-09-10T21:38:09.733 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-10T21:38:09.733 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['a'] 2024-09-10T21:38:09.733 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2024-09-10T21:38:09.733 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.a 2024-09-10T21:38:09.733 INFO:tasks.mon_thrash.mon_thrasher:killing mon.a 2024-09-10T21:38:09.733 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-10T21:38:09.733 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:38:09.752 INFO:tasks.ceph.mon.a:Stopped 2024-09-10T21:38:09.752 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2024-09-10T21:38:09.752 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-10T21:38:09.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.966+0000 7f899f600640 1 -- 172.21.15.17:0/615933694 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f89981546c0 msgr2=0x7f89980ca430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:38:09.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.966+0000 7f899f600640 1 --2- 172.21.15.17:0/615933694 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f89981546c0 0x7f89980ca430 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f898c009870 tx=0x7f898c01c830 comp rx=0 tx=0).stop 2024-09-10T21:38:09.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.966+0000 7f899f600640 1 -- 172.21.15.17:0/615933694 shutdown_connections 2024-09-10T21:38:09.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.966+0000 7f899f600640 1 --2- 172.21.15.17:0/615933694 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f89981546c0 0x7f89980ca430 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:38:09.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.966+0000 7f899f600640 1 -- 172.21.15.17:0/615933694 >> 172.21.15.17:0/615933694 conn(0x7f89980702f0 msgr2=0x7f8998070700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:38:09.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.966+0000 7f899f600640 1 -- 172.21.15.17:0/615933694 shutdown_connections 2024-09-10T21:38:09.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.966+0000 7f899f600640 1 -- 172.21.15.17:0/615933694 wait complete. 2024-09-10T21:38:09.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.966+0000 7f899f600640 1 Processor -- start 2024-09-10T21:38:09.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.966+0000 7f899f600640 1 -- start start 2024-09-10T21:38:09.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.970+0000 7f899f600640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f89980c89d0 0x7f89980cd960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:38:09.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.970+0000 7f899f600640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f89981546c0 0x7f8998142100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:38:09.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.970+0000 7f899f600640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8998154aa0 0x7f89980cc3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:38:09.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.970+0000 7f899f600640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f89980ce630 con 0x7f89980c89d0 2024-09-10T21:38:09.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.970+0000 7f899f600640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f8998072790 con 0x7f8998154aa0 2024-09-10T21:38:09.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.970+0000 7f899f600640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f89980ce7b0 con 0x7f89981546c0 2024-09-10T21:38:09.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.970+0000 7f899cc00640 1 -- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f89980c89d0 msgr2=0x7f89980cd960 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-10T21:38:09.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.970+0000 7f899cc00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f89980c89d0 0x7f89980cd960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:38:09.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.970+0000 7f899d600640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8998154aa0 0x7f89980cc3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:38:09.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.970+0000 7f8997e00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f89981546c0 0x7f8998142100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:38:09.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.970+0000 7f899d600640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8998154aa0 0x7f89980cc3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.17:54520/0 (socket says 172.21.15.17:54520) 2024-09-10T21:38:09.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.970+0000 7f899d600640 1 -- 172.21.15.17:0/1970565790 learned_addr learned my addr 172.21.15.17:0/1970565790 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:38:09.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.970+0000 7f8997e00640 1 -- 172.21.15.17:0/1970565790 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8998154aa0 msgr2=0x7f89980cc3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:38:09.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.970+0000 7f8997e00640 1 --2- 172.21.15.17:0/1970565790 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8998154aa0 0x7f89980cc3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:38:09.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.970+0000 7f8997e00640 1 -- 172.21.15.17:0/1970565790 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f89980c89d0 msgr2=0x7f89980cd960 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:38:09.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.970+0000 7f8997e00640 1 --2- 172.21.15.17:0/1970565790 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f89980c89d0 0x7f89980cd960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:38:09.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.970+0000 7f8997e00640 1 -- 172.21.15.17:0/1970565790 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8998141640 con 0x7f89981546c0 2024-09-10T21:38:09.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.970+0000 7f8997e00640 1 --2- 172.21.15.17:0/1970565790 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f89981546c0 0x7f8998142100 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f898400b400 tx=0x7f898400b8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:38:09.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.970+0000 7f8994c00640 1 -- 172.21.15.17:0/1970565790 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8984015020 con 0x7f89981546c0 2024-09-10T21:38:09.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.970+0000 7f8994c00640 1 -- 172.21.15.17:0/1970565790 <== mon.2 v2:172.21.15.31:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8984011940 con 0x7f89981546c0 2024-09-10T21:38:09.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.970+0000 7f899f600640 1 -- 172.21.15.17:0/1970565790 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8998146eb0 con 0x7f89981546c0 2024-09-10T21:38:09.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.970+0000 7f8994c00640 1 -- 172.21.15.17:0/1970565790 <== mon.2 v2:172.21.15.31:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8984011d90 con 0x7f89981546c0 2024-09-10T21:38:09.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.970+0000 7f899f600640 1 -- 172.21.15.17:0/1970565790 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f8998147470 con 0x7f89981546c0 2024-09-10T21:38:09.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.974+0000 7f899f600640 1 -- 172.21.15.17:0/1970565790 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89980c9740 con 0x7f89981546c0 2024-09-10T21:38:09.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.974+0000 7f8994c00640 1 -- 172.21.15.17:0/1970565790 <== mon.2 v2:172.21.15.31:3301/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f89840193b0 con 0x7f89981546c0 2024-09-10T21:38:09.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.974+0000 7f8994c00640 1 --2- 172.21.15.17:0/1970565790 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f89780725c0 0x7f8978074a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:38:09.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.974+0000 7f8994c00640 1 -- 172.21.15.17:0/1970565790 <== mon.2 v2:172.21.15.31:3301/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f89840934d0 con 0x7f89981546c0 2024-09-10T21:38:09.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.974+0000 7f899cc00640 1 --2- 172.21.15.17:0/1970565790 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f89780725c0 0x7f8978074a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:38:09.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.978+0000 7f899cc00640 1 --2- 172.21.15.17:0/1970565790 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f89780725c0 0x7f8978074a80 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f898c0af000 tx=0x7f898c0a7040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:38:09.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:09.978+0000 7f8994c00640 1 -- 172.21.15.17:0/1970565790 <== mon.2 v2:172.21.15.31:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f898401f030 con 0x7f89981546c0 2024-09-10T21:38:10.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:10.326+0000 7f899f600640 1 -- 172.21.15.17:0/1970565790 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f8998070870 con 0x7f89981546c0 2024-09-10T21:38:10.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:10.330+0000 7f8994c00640 1 -- 172.21.15.17:0/1970565790 <== mon.2 v2:172.21.15.31:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1558 (secure 0 0 0) 0x7f898405e710 con 0x7f89981546c0 2024-09-10T21:38:10.332 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":110,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":13,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5e2f023f-d6a6-49f4-a256-f4807dce3739","modified":"2024-09-10T21:23:46.892130Z","created":"2024-09-10T21:23:46.892130Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3301","nonce":0},{"type":"v1","addr":"172.21.15.31:6790","nonce":0}]},"addr":"172.21.15.31:6790/0","public_addr":"172.21.15.31:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-10T21:38:10.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:10.330+0000 7f899f600640 1 -- 172.21.15.17:0/1970565790 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f89780725c0 msgr2=0x7f8978074a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:38:10.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:10.330+0000 7f899f600640 1 --2- 172.21.15.17:0/1970565790 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f89780725c0 0x7f8978074a80 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f898c0af000 tx=0x7f898c0a7040 comp rx=0 tx=0).stop 2024-09-10T21:38:10.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:10.330+0000 7f899f600640 1 -- 172.21.15.17:0/1970565790 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f89981546c0 msgr2=0x7f8998142100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:38:10.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:10.330+0000 7f899f600640 1 --2- 172.21.15.17:0/1970565790 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f89981546c0 0x7f8998142100 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f898400b400 tx=0x7f898400b8c0 comp rx=0 tx=0).stop 2024-09-10T21:38:10.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:10.330+0000 7f899f600640 1 -- 172.21.15.17:0/1970565790 shutdown_connections 2024-09-10T21:38:10.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:10.330+0000 7f899f600640 1 --2- 172.21.15.17:0/1970565790 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f89780725c0 0x7f8978074a80 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:38:10.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:10.330+0000 7f899f600640 1 --2- 172.21.15.17:0/1970565790 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8998154aa0 0x7f89980cc3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:38:10.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:10.330+0000 7f899f600640 1 --2- 172.21.15.17:0/1970565790 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f89981546c0 0x7f8998142100 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:38:10.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:10.330+0000 7f899f600640 1 --2- 172.21.15.17:0/1970565790 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f89980c89d0 0x7f89980cd960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:38:10.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:10.330+0000 7f899f600640 1 -- 172.21.15.17:0/1970565790 >> 172.21.15.17:0/1970565790 conn(0x7f89980702f0 msgr2=0x7f8998072080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:38:10.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:10.330+0000 7f899f600640 1 -- 172.21.15.17:0/1970565790 shutdown_connections 2024-09-10T21:38:10.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:10.330+0000 7f899f600640 1 -- 172.21.15.17:0/1970565790 wait complete. 2024-09-10T21:38:13.357 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-10T21:38:13.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:13.518+0000 7f6564a00640 1 -- 172.21.15.17:0/2849713998 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f656015c440 msgr2=0x7f6560070bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:38:13.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:13.518+0000 7f6564a00640 1 --2- 172.21.15.17:0/2849713998 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f656015c440 0x7f6560070bb0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f65600b3630 tx=0x7f655801f5a0 comp rx=0 tx=0).stop 2024-09-10T21:38:13.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:13.518+0000 7f6564a00640 1 -- 172.21.15.17:0/2849713998 shutdown_connections 2024-09-10T21:38:13.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:13.518+0000 7f6564a00640 1 --2- 172.21.15.17:0/2849713998 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f656015c440 0x7f6560070bb0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:38:13.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:13.518+0000 7f6564a00640 1 --2- 172.21.15.17:0/2849713998 >> v2:172.21.15.31:3300/0 conn(0x7f6560158c20 0x7f6560070670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:38:13.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:13.518+0000 7f6564a00640 1 --2- 172.21.15.17:0/2849713998 >> v2:172.21.15.17:3300/0 conn(0x7f65601524d0 0x7f656015bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:38:13.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:13.518+0000 7f6564a00640 1 -- 172.21.15.17:0/2849713998 >> 172.21.15.17:0/2849713998 conn(0x7f65600cdef0 msgr2=0x7f65600ce2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:38:13.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:13.518+0000 7f6564a00640 1 -- 172.21.15.17:0/2849713998 shutdown_connections 2024-09-10T21:38:13.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:13.518+0000 7f6564a00640 1 -- 172.21.15.17:0/2849713998 wait complete. 2024-09-10T21:38:13.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:13.522+0000 7f6564a00640 1 Processor -- start 2024-09-10T21:38:13.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:13.522+0000 7f6564a00640 1 -- start start 2024-09-10T21:38:13.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:13.522+0000 7f6564a00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f65601524d0 0x7f656014d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:38:13.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:13.522+0000 7f6564a00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6560158c20 0x7f65600cd540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:38:13.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:13.522+0000 7f6564a00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f656015c440 0x7f6560143820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:38:13.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:13.522+0000 7f6564a00640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f65600c9350 con 0x7f656015c440 2024-09-10T21:38:13.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:13.522+0000 7f6564a00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f65600c9070 con 0x7f65601524d0 2024-09-10T21:38:13.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:13.522+0000 7f6564a00640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f656015b9e0 con 0x7f6560158c20 2024-09-10T21:38:13.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:13.522+0000 7f655f200640 1 -- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f656015c440 msgr2=0x7f6560143820 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-10T21:38:13.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:13.522+0000 7f655f200640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f656015c440 0x7f6560143820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:38:13.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:13.522+0000 7f655de00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6560158c20 0x7f65600cd540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:38:13.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:13.522+0000 7f655e800640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f65601524d0 0x7f656014d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:38:13.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:13.522+0000 7f655de00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6560158c20 0x7f65600cd540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3301/0 says I am v2:172.21.15.17:39910/0 (socket says 172.21.15.17:39910) 2024-09-10T21:38:13.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:13.522+0000 7f655de00640 1 -- 172.21.15.17:0/142663897 learned_addr learned my addr 172.21.15.17:0/142663897 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:38:13.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:13.522+0000 7f655de00640 1 -- 172.21.15.17:0/142663897 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f65601524d0 msgr2=0x7f656014d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:38:13.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:13.522+0000 7f655de00640 1 --2- 172.21.15.17:0/142663897 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f65601524d0 0x7f656014d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:38:13.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:13.522+0000 7f655de00640 1 -- 172.21.15.17:0/142663897 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f656015c440 msgr2=0x7f6560143820 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:38:13.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:13.522+0000 7f655de00640 1 --2- 172.21.15.17:0/142663897 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f656015c440 0x7f6560143820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:38:13.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:13.522+0000 7f655de00640 1 -- 172.21.15.17:0/142663897 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6560155f00 con 0x7f6560158c20 2024-09-10T21:38:13.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:13.522+0000 7f655de00640 1 --2- 172.21.15.17:0/142663897 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6560158c20 0x7f65600cd540 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f654c00e970 tx=0x7f654c00ee30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:38:13.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:13.526+0000 7f6556a00640 1 -- 172.21.15.17:0/142663897 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f654c00cd80 con 0x7f6560158c20 2024-09-10T21:38:13.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:13.526+0000 7f6556a00640 1 -- 172.21.15.17:0/142663897 <== mon.2 v2:172.21.15.31:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f654c005020 con 0x7f6560158c20 2024-09-10T21:38:13.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:13.526+0000 7f6556a00640 1 -- 172.21.15.17:0/142663897 <== mon.2 v2:172.21.15.31:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f654c005450 con 0x7f6560158c20 2024-09-10T21:38:13.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:13.526+0000 7f6564a00640 1 -- 172.21.15.17:0/142663897 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65601516f0 con 0x7f6560158c20 2024-09-10T21:38:13.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:13.526+0000 7f6564a00640 1 -- 172.21.15.17:0/142663897 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f6560151cb0 con 0x7f6560158c20 2024-09-10T21:38:13.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:13.526+0000 7f6564a00640 1 -- 172.21.15.17:0/142663897 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65600b3e00 con 0x7f6560158c20 2024-09-10T21:38:13.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:13.526+0000 7f6556a00640 1 -- 172.21.15.17:0/142663897 <== mon.2 v2:172.21.15.31:3301/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f654c0047b0 con 0x7f6560158c20 2024-09-10T21:38:13.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:13.526+0000 7f6556a00640 1 --2- 172.21.15.17:0/142663897 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f65400725c0 0x7f6540074a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:38:13.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:13.530+0000 7f6556a00640 1 -- 172.21.15.17:0/142663897 <== mon.2 v2:172.21.15.31:3301/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f654c094160 con 0x7f6560158c20 2024-09-10T21:38:13.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:13.530+0000 7f655e800640 1 --2- 172.21.15.17:0/142663897 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f65400725c0 0x7f6540074a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:38:13.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:13.530+0000 7f655e800640 1 --2- 172.21.15.17:0/142663897 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f65400725c0 0x7f6540074a80 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f6550006fd0 tx=0x7f6550009190 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:38:13.536 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:13.534+0000 7f6556a00640 1 -- 172.21.15.17:0/142663897 <== mon.2 v2:172.21.15.31:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f654c05f310 con 0x7f6560158c20 2024-09-10T21:38:13.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:13.794+0000 7f6564a00640 1 -- 172.21.15.17:0/142663897 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f65600c39a0 con 0x7f6560158c20 2024-09-10T21:38:13.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:13.794+0000 7f6556a00640 1 -- 172.21.15.17:0/142663897 <== mon.2 v2:172.21.15.31:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1558 (secure 0 0 0) 0x7f654c05ec10 con 0x7f6560158c20 2024-09-10T21:38:13.798 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":110,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":16,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5e2f023f-d6a6-49f4-a256-f4807dce3739","modified":"2024-09-10T21:23:46.892130Z","created":"2024-09-10T21:23:46.892130Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3301","nonce":0},{"type":"v1","addr":"172.21.15.31:6790","nonce":0}]},"addr":"172.21.15.31:6790/0","public_addr":"172.21.15.31:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-10T21:38:13.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:13.798+0000 7f6564a00640 1 -- 172.21.15.17:0/142663897 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f65400725c0 msgr2=0x7f6540074a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:38:13.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:13.798+0000 7f6564a00640 1 --2- 172.21.15.17:0/142663897 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f65400725c0 0x7f6540074a80 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f6550006fd0 tx=0x7f6550009190 comp rx=0 tx=0).stop 2024-09-10T21:38:13.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:13.798+0000 7f6564a00640 1 -- 172.21.15.17:0/142663897 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6560158c20 msgr2=0x7f65600cd540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:38:13.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:13.798+0000 7f6564a00640 1 --2- 172.21.15.17:0/142663897 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6560158c20 0x7f65600cd540 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f654c00e970 tx=0x7f654c00ee30 comp rx=0 tx=0).stop 2024-09-10T21:38:13.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:13.798+0000 7f6564a00640 1 -- 172.21.15.17:0/142663897 shutdown_connections 2024-09-10T21:38:13.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:13.798+0000 7f6564a00640 1 --2- 172.21.15.17:0/142663897 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f65400725c0 0x7f6540074a80 secure :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f6550006fd0 tx=0x7f6550009190 comp rx=0 tx=0).stop 2024-09-10T21:38:13.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:13.798+0000 7f6564a00640 1 --2- 172.21.15.17:0/142663897 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f656015c440 0x7f6560143820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:38:13.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:13.798+0000 7f6564a00640 1 --2- 172.21.15.17:0/142663897 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6560158c20 0x7f65600cd540 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:38:13.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:13.798+0000 7f6564a00640 1 --2- 172.21.15.17:0/142663897 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f65601524d0 0x7f656014d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:38:13.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:13.798+0000 7f6564a00640 1 -- 172.21.15.17:0/142663897 >> 172.21.15.17:0/142663897 conn(0x7f65600cdef0 msgr2=0x7f65600c2f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:38:13.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:13.798+0000 7f6564a00640 1 -- 172.21.15.17:0/142663897 shutdown_connections 2024-09-10T21:38:13.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:13.798+0000 7f6564a00640 1 -- 172.21.15.17:0/142663897 wait complete. 2024-09-10T21:38:16.830 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-10T21:38:17.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:16.998+0000 7fd657e00640 1 -- 172.21.15.17:0/641573899 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd6580ce8c0 msgr2=0x7fd65815b820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:38:17.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:16.998+0000 7fd657e00640 1 --2- 172.21.15.17:0/641573899 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd6580ce8c0 0x7fd65815b820 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fd640009930 tx=0x7fd64001c8b0 comp rx=0 tx=0).stop 2024-09-10T21:38:17.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:16.998+0000 7fd657e00640 1 -- 172.21.15.17:0/641573899 shutdown_connections 2024-09-10T21:38:17.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:16.998+0000 7fd657e00640 1 --2- 172.21.15.17:0/641573899 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd658071130 0x7fd6580c2ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:38:17.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:16.998+0000 7fd657e00640 1 --2- 172.21.15.17:0/641573899 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd6580ce8c0 0x7fd65815b820 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:38:17.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:16.998+0000 7fd657e00640 1 -- 172.21.15.17:0/641573899 >> 172.21.15.17:0/641573899 conn(0x7fd658074ac0 msgr2=0x7fd658068990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:38:17.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:16.998+0000 7fd657e00640 1 -- 172.21.15.17:0/641573899 shutdown_connections 2024-09-10T21:38:17.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:16.998+0000 7fd657e00640 1 -- 172.21.15.17:0/641573899 wait complete. 2024-09-10T21:38:17.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:17.002+0000 7fd657e00640 1 Processor -- start 2024-09-10T21:38:17.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:17.002+0000 7fd657e00640 1 -- start start 2024-09-10T21:38:17.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:17.002+0000 7fd657e00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd658071130 0x7fd658159b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:38:17.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:17.002+0000 7fd657e00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd6580cddd0 0x7fd65814bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:38:17.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:17.002+0000 7fd657e00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd6580ce8c0 0x7fd658146890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:38:17.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:17.002+0000 7fd657e00640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fd6581526b0 con 0x7fd6580cddd0 2024-09-10T21:38:17.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:17.002+0000 7fd657e00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fd658152530 con 0x7fd6580ce8c0 2024-09-10T21:38:17.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:17.002+0000 7fd657e00640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7fd6581653d0 con 0x7fd658071130 2024-09-10T21:38:17.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:17.002+0000 7fd656000640 1 -- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd6580cddd0 msgr2=0x7fd65814bb20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-10T21:38:17.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:17.002+0000 7fd656000640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd6580cddd0 0x7fd65814bb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:38:17.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:17.002+0000 7fd656a00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd658071130 0x7fd658159b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:38:17.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:17.002+0000 7fd657400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd6580ce8c0 0x7fd658146890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:38:17.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:17.002+0000 7fd657400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd6580ce8c0 0x7fd658146890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.17:58636/0 (socket says 172.21.15.17:58636) 2024-09-10T21:38:17.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:17.002+0000 7fd657400640 1 -- 172.21.15.17:0/3370219301 learned_addr learned my addr 172.21.15.17:0/3370219301 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:38:17.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:17.002+0000 7fd657400640 1 -- 172.21.15.17:0/3370219301 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd658071130 msgr2=0x7fd658159b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:38:17.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:17.002+0000 7fd657400640 1 --2- 172.21.15.17:0/3370219301 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd658071130 0x7fd658159b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:38:17.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:17.002+0000 7fd657400640 1 -- 172.21.15.17:0/3370219301 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd6580cddd0 msgr2=0x7fd65814bb20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:38:17.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:17.002+0000 7fd657400640 1 --2- 172.21.15.17:0/3370219301 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd6580cddd0 0x7fd65814bb20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:38:17.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:17.002+0000 7fd657400640 1 -- 172.21.15.17:0/3370219301 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6580ccd10 con 0x7fd6580ce8c0 2024-09-10T21:38:17.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:17.002+0000 7fd657400640 1 --2- 172.21.15.17:0/3370219301 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd6580ce8c0 0x7fd658146890 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fd648009d30 tx=0x7fd64800c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:38:17.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:17.002+0000 7fd64f400640 1 -- 172.21.15.17:0/3370219301 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd648003b80 con 0x7fd6580ce8c0 2024-09-10T21:38:17.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:17.002+0000 7fd64f400640 1 -- 172.21.15.17:0/3370219301 <== mon.1 v2:172.21.15.31:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd648003d20 con 0x7fd6580ce8c0 2024-09-10T21:38:17.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:17.002+0000 7fd64f400640 1 -- 172.21.15.17:0/3370219301 <== mon.1 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd648004280 con 0x7fd6580ce8c0 2024-09-10T21:38:17.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:17.002+0000 7fd657e00640 1 -- 172.21.15.17:0/3370219301 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6580cca30 con 0x7fd6580ce8c0 2024-09-10T21:38:17.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:17.002+0000 7fd657e00640 1 -- 172.21.15.17:0/3370219301 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd65814dc20 con 0x7fd6580ce8c0 2024-09-10T21:38:17.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:17.006+0000 7fd657e00640 1 -- 172.21.15.17:0/3370219301 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6580c2ea0 con 0x7fd6580ce8c0 2024-09-10T21:38:17.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:17.006+0000 7fd64f400640 1 -- 172.21.15.17:0/3370219301 <== mon.1 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fd648004420 con 0x7fd6580ce8c0 2024-09-10T21:38:17.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:17.006+0000 7fd64f400640 1 --2- 172.21.15.17:0/3370219301 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fd634072590 0x7fd634074a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:38:17.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:17.006+0000 7fd64f400640 1 -- 172.21.15.17:0/3370219301 <== mon.1 v2:172.21.15.31:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fd648094420 con 0x7fd6580ce8c0 2024-09-10T21:38:17.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:17.006+0000 7fd656a00640 1 --2- 172.21.15.17:0/3370219301 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fd634072590 0x7fd634074a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:38:17.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:17.010+0000 7fd656a00640 1 --2- 172.21.15.17:0/3370219301 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fd634072590 0x7fd634074a50 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fd650007fd0 tx=0x7fd650009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:38:17.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:17.010+0000 7fd64f400640 1 -- 172.21.15.17:0/3370219301 <== mon.1 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fd64805f5d0 con 0x7fd6580ce8c0 2024-09-10T21:38:17.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:17.302+0000 7fd657e00640 1 -- 172.21.15.17:0/3370219301 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fd6580c9210 con 0x7fd6580ce8c0 2024-09-10T21:38:17.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:17.302+0000 7fd64f400640 1 -- 172.21.15.17:0/3370219301 <== mon.1 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1558 (secure 0 0 0) 0x7fd64805eed0 con 0x7fd6580ce8c0 2024-09-10T21:38:17.305 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":110,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":20,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5e2f023f-d6a6-49f4-a256-f4807dce3739","modified":"2024-09-10T21:23:46.892130Z","created":"2024-09-10T21:23:46.892130Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3301","nonce":0},{"type":"v1","addr":"172.21.15.31:6790","nonce":0}]},"addr":"172.21.15.31:6790/0","public_addr":"172.21.15.31:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-10T21:38:17.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:17.302+0000 7fd657e00640 1 -- 172.21.15.17:0/3370219301 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fd634072590 msgr2=0x7fd634074a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:38:17.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:17.302+0000 7fd657e00640 1 --2- 172.21.15.17:0/3370219301 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fd634072590 0x7fd634074a50 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fd650007fd0 tx=0x7fd650009040 comp rx=0 tx=0).stop 2024-09-10T21:38:17.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:17.302+0000 7fd657e00640 1 -- 172.21.15.17:0/3370219301 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd6580ce8c0 msgr2=0x7fd658146890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:38:17.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:17.302+0000 7fd657e00640 1 --2- 172.21.15.17:0/3370219301 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd6580ce8c0 0x7fd658146890 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fd648009d30 tx=0x7fd64800c6a0 comp rx=0 tx=0).stop 2024-09-10T21:38:17.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:17.302+0000 7fd657e00640 1 -- 172.21.15.17:0/3370219301 shutdown_connections 2024-09-10T21:38:17.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:17.302+0000 7fd657e00640 1 --2- 172.21.15.17:0/3370219301 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fd634072590 0x7fd634074a50 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:38:17.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:17.302+0000 7fd657e00640 1 --2- 172.21.15.17:0/3370219301 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd6580ce8c0 0x7fd658146890 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:38:17.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:17.302+0000 7fd657e00640 1 --2- 172.21.15.17:0/3370219301 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd6580cddd0 0x7fd65814bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:38:17.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:17.302+0000 7fd657e00640 1 --2- 172.21.15.17:0/3370219301 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd658071130 0x7fd658159b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:38:17.308 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:17.302+0000 7fd657e00640 1 -- 172.21.15.17:0/3370219301 >> 172.21.15.17:0/3370219301 conn(0x7fd658074ac0 msgr2=0x7fd658157130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:38:17.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:17.302+0000 7fd657e00640 1 -- 172.21.15.17:0/3370219301 shutdown_connections 2024-09-10T21:38:17.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:17.306+0000 7fd657e00640 1 -- 172.21.15.17:0/3370219301 wait complete. 2024-09-10T21:38:20.330 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-10T21:38:20.998 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:38:21.002 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:38:24.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:24.890+0000 7f0d21000640 1 -- 172.21.15.17:0/3932864584 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f0cfc0072f0 msgr2=0x7f0cfc009790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:38:24.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:24.890+0000 7f0d21000640 1 --2- 172.21.15.17:0/3932864584 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f0cfc0072f0 0x7f0cfc009790 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f0d080064d0 tx=0x7f0d0801d430 comp rx=0 tx=0).stop 2024-09-10T21:38:24.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:24.890+0000 7f0d1a800640 1 -- 172.21.15.17:0/3932864584 reap_dead start 2024-09-10T21:38:24.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:24.890+0000 7f0d21000640 1 -- 172.21.15.17:0/3932864584 shutdown_connections 2024-09-10T21:38:24.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:24.890+0000 7f0d21000640 1 -- 172.21.15.17:0/3932864584 >> 172.21.15.17:0/3932864584 conn(0x7f0d1c0702f0 msgr2=0x7f0d1c070700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:38:24.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:24.890+0000 7f0d21000640 1 -- 172.21.15.17:0/3932864584 shutdown_connections 2024-09-10T21:38:24.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:24.890+0000 7f0d21000640 1 -- 172.21.15.17:0/3932864584 wait complete. 2024-09-10T21:38:24.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:24.890+0000 7f0d21000640 1 Processor -- start 2024-09-10T21:38:24.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:24.890+0000 7f0d21000640 1 -- start start 2024-09-10T21:38:24.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:24.894+0000 7f0d21000640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f0cfc0072f0 0x7f0d1c162230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:38:24.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:24.894+0000 7f0d21000640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0d1c14b7b0 0x7f0d1c15af70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:38:24.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:24.894+0000 7f0d21000640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0d1c15b4b0 0x7f0d1c159d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:38:24.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:24.894+0000 7f0d21000640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f0d1c0cb5d0 con 0x7f0d1c15b4b0 2024-09-10T21:38:24.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:24.894+0000 7f0d21000640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f0d1c0cb450 con 0x7f0d1c14b7b0 2024-09-10T21:38:24.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:24.894+0000 7f0d21000640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f0d1c0c2ff0 con 0x7f0cfc0072f0 2024-09-10T21:38:24.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:24.894+0000 7f0d1a800640 1 -- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0d1c15b4b0 msgr2=0x7f0d1c159d80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-10T21:38:24.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:24.894+0000 7f0d1a800640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0d1c15b4b0 0x7f0d1c159d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:38:24.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:24.894+0000 7f0d19e00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f0cfc0072f0 0x7f0d1c162230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:38:24.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:24.894+0000 7f0d19400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0d1c14b7b0 0x7f0d1c15af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:38:24.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:24.894+0000 7f0d19e00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f0cfc0072f0 0x7f0d1c162230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3301/0 says I am v2:172.21.15.17:47384/0 (socket says 172.21.15.17:47384) 2024-09-10T21:38:24.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:24.894+0000 7f0d19e00640 1 -- 172.21.15.17:0/113967313 learned_addr learned my addr 172.21.15.17:0/113967313 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:38:24.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:24.894+0000 7f0d19e00640 1 -- 172.21.15.17:0/113967313 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0d1c14b7b0 msgr2=0x7f0d1c15af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:38:24.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:24.894+0000 7f0d19e00640 1 --2- 172.21.15.17:0/113967313 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0d1c14b7b0 0x7f0d1c15af70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:38:24.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:24.894+0000 7f0d19e00640 1 -- 172.21.15.17:0/113967313 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0d1c15b4b0 msgr2=0x7f0d1c159d80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:38:24.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:24.894+0000 7f0d19e00640 1 --2- 172.21.15.17:0/113967313 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0d1c15b4b0 0x7f0d1c159d80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:38:24.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:24.894+0000 7f0d19e00640 1 -- 172.21.15.17:0/113967313 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d1c1f7460 con 0x7f0cfc0072f0 2024-09-10T21:38:24.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:24.894+0000 7f0d19e00640 1 --2- 172.21.15.17:0/113967313 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f0cfc0072f0 0x7f0d1c162230 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c00b9e0 tx=0x7f0d0c00bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:38:24.899 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:24.894+0000 7f0d12000640 1 -- 172.21.15.17:0/113967313 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d0c019070 con 0x7f0cfc0072f0 2024-09-10T21:38:24.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:24.894+0000 7f0d12000640 1 -- 172.21.15.17:0/113967313 <== mon.2 v2:172.21.15.31:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0d0c00ce20 con 0x7f0cfc0072f0 2024-09-10T21:38:24.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:24.894+0000 7f0d12000640 1 -- 172.21.15.17:0/113967313 <== mon.2 v2:172.21.15.31:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d0c014680 con 0x7f0cfc0072f0 2024-09-10T21:38:24.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:24.894+0000 7f0d21000640 1 -- 172.21.15.17:0/113967313 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d1c1514a0 con 0x7f0cfc0072f0 2024-09-10T21:38:24.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:24.894+0000 7f0d21000640 1 -- 172.21.15.17:0/113967313 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d1c151760 con 0x7f0cfc0072f0 2024-09-10T21:38:24.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:24.894+0000 7f0d21000640 1 -- 172.21.15.17:0/113967313 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d1c1583f0 con 0x7f0cfc0072f0 2024-09-10T21:38:24.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:24.898+0000 7f0d12000640 1 -- 172.21.15.17:0/113967313 <== mon.2 v2:172.21.15.31:3301/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f0d0c01d3b0 con 0x7f0cfc0072f0 2024-09-10T21:38:24.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:24.898+0000 7f0d12000640 1 --2- 172.21.15.17:0/113967313 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f0cfc0734d0 0x7f0cfc075990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:38:24.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:24.898+0000 7f0d12000640 1 -- 172.21.15.17:0/113967313 <== mon.2 v2:172.21.15.31:3301/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f0d0c093240 con 0x7f0cfc0072f0 2024-09-10T21:38:24.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:24.898+0000 7f0d19400640 1 --2- 172.21.15.17:0/113967313 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f0cfc0734d0 0x7f0cfc075990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:38:24.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:24.898+0000 7f0d19400640 1 --2- 172.21.15.17:0/113967313 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f0cfc0734d0 0x7f0cfc075990 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f0d1c0c9850 tx=0x7f0d080b1040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:38:24.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:24.902+0000 7f0d12000640 1 -- 172.21.15.17:0/113967313 <== mon.2 v2:172.21.15.31:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f0d0c05dfd0 con 0x7f0cfc0072f0 2024-09-10T21:38:25.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:25.146+0000 7f0d21000640 1 -- 172.21.15.17:0/113967313 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f0d1c069670 con 0x7f0cfc0072f0 2024-09-10T21:38:25.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:25.146+0000 7f0d12000640 1 -- 172.21.15.17:0/113967313 <== mon.2 v2:172.21.15.31:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1551 (secure 0 0 0) 0x7f0d0c062a00 con 0x7f0cfc0072f0 2024-09-10T21:38:25.150 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":112,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":0,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5e2f023f-d6a6-49f4-a256-f4807dce3739","modified":"2024-09-10T21:23:46.892130Z","created":"2024-09-10T21:23:46.892130Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3301","nonce":0},{"type":"v1","addr":"172.21.15.31:6790","nonce":0}]},"addr":"172.21.15.31:6790/0","public_addr":"172.21.15.31:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-10T21:38:25.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:25.150+0000 7f0d21000640 1 -- 172.21.15.17:0/113967313 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f0cfc0734d0 msgr2=0x7f0cfc075990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:38:25.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:25.150+0000 7f0d21000640 1 --2- 172.21.15.17:0/113967313 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f0cfc0734d0 0x7f0cfc075990 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f0d1c0c9850 tx=0x7f0d080b1040 comp rx=0 tx=0).stop 2024-09-10T21:38:25.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:25.150+0000 7f0d21000640 1 -- 172.21.15.17:0/113967313 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f0cfc0072f0 msgr2=0x7f0d1c162230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:38:25.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:25.150+0000 7f0d21000640 1 --2- 172.21.15.17:0/113967313 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f0cfc0072f0 0x7f0d1c162230 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c00b9e0 tx=0x7f0d0c00bea0 comp rx=0 tx=0).stop 2024-09-10T21:38:25.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:25.150+0000 7f0d21000640 1 -- 172.21.15.17:0/113967313 shutdown_connections 2024-09-10T21:38:25.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:25.150+0000 7f0d21000640 1 --2- 172.21.15.17:0/113967313 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f0cfc0734d0 0x7f0cfc075990 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:38:25.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:25.150+0000 7f0d21000640 1 --2- 172.21.15.17:0/113967313 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0d1c15b4b0 0x7f0d1c159d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:38:25.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:25.150+0000 7f0d21000640 1 --2- 172.21.15.17:0/113967313 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0d1c14b7b0 0x7f0d1c15af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:38:25.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:25.150+0000 7f0d21000640 1 --2- 172.21.15.17:0/113967313 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f0cfc0072f0 0x7f0d1c162230 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:38:25.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:25.150+0000 7f0d21000640 1 -- 172.21.15.17:0/113967313 >> 172.21.15.17:0/113967313 conn(0x7f0d1c0702f0 msgr2=0x7f0d1c0c9e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:38:25.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:25.150+0000 7f0d21000640 1 -- 172.21.15.17:0/113967313 shutdown_connections 2024-09-10T21:38:25.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:25.150+0000 7f0d21000640 1 -- 172.21.15.17:0/113967313 wait complete. 2024-09-10T21:38:25.171 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2024-09-10T21:38:25.171 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon b 2024-09-10T21:38:25.172 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-10T21:38:25.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:25.318+0000 7f48fda00640 1 -- 172.21.15.17:0/1313113882 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f48f80c0fd0 msgr2=0x7f48f8150e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:38:25.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:25.318+0000 7f48fda00640 1 --2- 172.21.15.17:0/1313113882 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f48f80c0fd0 0x7f48f8150e30 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f48e4009900 tx=0x7f48e401f0f0 comp rx=0 tx=0).stop 2024-09-10T21:38:25.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:25.318+0000 7f48fda00640 1 -- 172.21.15.17:0/1313113882 shutdown_connections 2024-09-10T21:38:25.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:25.318+0000 7f48fda00640 1 --2- 172.21.15.17:0/1313113882 >> v2:172.21.15.17:3300/0 conn(0x7f48f8073cd0 0x7f48f8074180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:38:25.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:25.318+0000 7f48fda00640 1 --2- 172.21.15.17:0/1313113882 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f48f80c0fd0 0x7f48f8150e30 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:38:25.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:25.318+0000 7f48fda00640 1 -- 172.21.15.17:0/1313113882 >> 172.21.15.17:0/1313113882 conn(0x7f48f80700d0 msgr2=0x7f48f80704e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:38:25.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:25.318+0000 7f48fda00640 1 -- 172.21.15.17:0/1313113882 shutdown_connections 2024-09-10T21:38:25.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:25.318+0000 7f48fda00640 1 -- 172.21.15.17:0/1313113882 wait complete. 2024-09-10T21:38:25.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:25.322+0000 7f48fda00640 1 Processor -- start 2024-09-10T21:38:25.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:25.322+0000 7f48fda00640 1 -- start start 2024-09-10T21:38:25.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:25.322+0000 7f48fda00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f48f8073cd0 0x7f48f81f29d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:38:25.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:25.322+0000 7f48fda00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48f80c0fd0 0x7f48f80c4c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:38:25.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:25.322+0000 7f48fda00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f48f80c7d00 0x7f48f806de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:38:25.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:25.322+0000 7f48fda00640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f48f80c8410 con 0x7f48f80c7d00 2024-09-10T21:38:25.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:25.322+0000 7f48fda00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f48f80c8780 con 0x7f48f80c0fd0 2024-09-10T21:38:25.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:25.322+0000 7f48fda00640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f48f80c1c50 con 0x7f48f8073cd0 2024-09-10T21:38:25.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:25.322+0000 7f48f7400640 1 -- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f48f80c7d00 msgr2=0x7f48f806de90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-10T21:38:25.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:25.322+0000 7f48f7400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f48f80c7d00 0x7f48f806de90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:38:25.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:25.322+0000 7f48f6000640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48f80c0fd0 0x7f48f80c4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:38:25.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:25.322+0000 7f48f6a00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f48f8073cd0 0x7f48f81f29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:38:25.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:25.322+0000 7f48f6000640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48f80c0fd0 0x7f48f80c4c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.17:51712/0 (socket says 172.21.15.17:51712) 2024-09-10T21:38:25.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:25.322+0000 7f48f6000640 1 -- 172.21.15.17:0/1236354221 learned_addr learned my addr 172.21.15.17:0/1236354221 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:38:25.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:25.322+0000 7f48f6a00640 1 -- 172.21.15.17:0/1236354221 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48f80c0fd0 msgr2=0x7f48f80c4c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:38:25.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:25.322+0000 7f48f6a00640 1 --2- 172.21.15.17:0/1236354221 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48f80c0fd0 0x7f48f80c4c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:38:25.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:25.322+0000 7f48f6a00640 1 -- 172.21.15.17:0/1236354221 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f48f80c7d00 msgr2=0x7f48f806de90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:38:25.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:25.322+0000 7f48f6a00640 1 --2- 172.21.15.17:0/1236354221 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f48f80c7d00 0x7f48f806de90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:38:25.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:25.322+0000 7f48f6a00640 1 -- 172.21.15.17:0/1236354221 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48f80c33b0 con 0x7f48f8073cd0 2024-09-10T21:38:25.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:25.322+0000 7f48f6a00640 1 --2- 172.21.15.17:0/1236354221 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f48f8073cd0 0x7f48f81f29d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f48e8006fd0 tx=0x7f48e800b8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:38:25.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:25.322+0000 7f48eea00640 1 -- 172.21.15.17:0/1236354221 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48e8015020 con 0x7f48f8073cd0 2024-09-10T21:38:25.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:25.322+0000 7f48eea00640 1 -- 172.21.15.17:0/1236354221 <== mon.2 v2:172.21.15.31:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f48e8011940 con 0x7f48f8073cd0 2024-09-10T21:38:25.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:25.322+0000 7f48eea00640 1 -- 172.21.15.17:0/1236354221 <== mon.2 v2:172.21.15.31:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48e8011d90 con 0x7f48f8073cd0 2024-09-10T21:38:25.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:25.322+0000 7f48fda00640 1 -- 172.21.15.17:0/1236354221 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48f81f1f30 con 0x7f48f8073cd0 2024-09-10T21:38:25.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:25.322+0000 7f48fda00640 1 -- 172.21.15.17:0/1236354221 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f48f814b700 con 0x7f48f8073cd0 2024-09-10T21:38:25.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:25.326+0000 7f48fda00640 1 --2- 172.21.15.17:0/1236354221 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48f81503e0 0x7f48f8146540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:38:25.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:25.326+0000 7f48fda00640 1 -- 172.21.15.17:0/1236354221 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f48f81f2500 con 0x7f48f81503e0 2024-09-10T21:38:25.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:25.326+0000 7f48f6000640 1 --2- 172.21.15.17:0/1236354221 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48f81503e0 0x7f48f8146540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:38:25.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:25.326+0000 7f48f6000640 1 -- 172.21.15.17:0/1236354221 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f48f8153500 con 0x7f48f81503e0 2024-09-10T21:38:25.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:25.326+0000 7f48f6000640 1 --2- 172.21.15.17:0/1236354221 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48f81503e0 0x7f48f8146540 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f48e4009850 tx=0x7f48e40095f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:38:25.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:25.326+0000 7f48eea00640 1 -- 172.21.15.17:0/1236354221 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48e4003740 con 0x7f48f81503e0 2024-09-10T21:38:25.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:25.326+0000 7f48eea00640 1 -- 172.21.15.17:0/1236354221 <== mon.2 v2:172.21.15.31:3301/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f48e801a400 con 0x7f48f8073cd0 2024-09-10T21:38:25.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:25.326+0000 7f48eea00640 1 --2- 172.21.15.17:0/1236354221 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f48d80724e0 0x7f48d80749a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:38:25.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:25.326+0000 7f48eea00640 1 -- 172.21.15.17:0/1236354221 <== mon.2 v2:172.21.15.31:3301/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f48e8093860 con 0x7f48f8073cd0 2024-09-10T21:38:25.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:25.326+0000 7f48eea00640 1 -- 172.21.15.17:0/1236354221 <== mon.1 v2:172.21.15.31:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f48e40038e0 con 0x7f48f81503e0 2024-09-10T21:38:25.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:25.326+0000 7f48eea00640 1 -- 172.21.15.17:0/1236354221 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48f81503e0 msgr2=0x7f48f8146540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:38:25.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:25.326+0000 7f48eea00640 1 --2- 172.21.15.17:0/1236354221 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48f81503e0 0x7f48f8146540 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f48e4009850 tx=0x7f48e40095f0 comp rx=0 tx=0).stop 2024-09-10T21:38:25.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:25.326+0000 7f48eea00640 1 -- 172.21.15.17:0/1236354221 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48f81503e0 msgr2=0x7f48f8146540 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:38:25.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:25.326+0000 7f48eea00640 1 --2- 172.21.15.17:0/1236354221 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48f81503e0 0x7f48f8146540 secure :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f48e4009850 tx=0x7f48e40095f0 comp rx=0 tx=0).stop 2024-09-10T21:38:25.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:25.326+0000 7f48f7400640 1 --2- 172.21.15.17:0/1236354221 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f48d80724e0 0x7f48d80749a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:38:25.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:25.330+0000 7f48f7400640 1 --2- 172.21.15.17:0/1236354221 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f48d80724e0 0x7f48d80749a0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f48f0001360 tx=0x7f48f0009250 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:38:25.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:25.334+0000 7f48fda00640 1 --2- 172.21.15.17:0/1236354221 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48f814ce80 0x7f48f8146a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:38:25.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:25.334+0000 7f48fda00640 1 -- 172.21.15.17:0/1236354221 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f48f81e50d0 con 0x7f48f814ce80 2024-09-10T21:38:25.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:25.334+0000 7f48f6000640 1 --2- 172.21.15.17:0/1236354221 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48f814ce80 0x7f48f8146a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:38:25.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:25.334+0000 7f48f6000640 1 -- 172.21.15.17:0/1236354221 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f48f814da60 con 0x7f48f814ce80 2024-09-10T21:38:25.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:25.334+0000 7f48f6000640 1 --2- 172.21.15.17:0/1236354221 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48f814ce80 0x7f48f8146a80 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f48e4000c10 tx=0x7f48e4009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:38:25.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:25.334+0000 7f48eea00640 1 -- 172.21.15.17:0/1236354221 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48e40c9670 con 0x7f48f814ce80 2024-09-10T21:38:25.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:25.334+0000 7f48eea00640 1 -- 172.21.15.17:0/1236354221 <== mon.1 v2:172.21.15.31:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5034 (secure 0 0 0) 0x7f48e40c9670 con 0x7f48f814ce80 2024-09-10T21:38:25.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:25.334+0000 7f48eea00640 1 -- 172.21.15.17:0/1236354221 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48f814ce80 msgr2=0x7f48f8146a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:38:25.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:25.334+0000 7f48eea00640 1 --2- 172.21.15.17:0/1236354221 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48f814ce80 0x7f48f8146a80 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f48e4000c10 tx=0x7f48e4009260 comp rx=0 tx=0).stop 2024-09-10T21:38:25.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:25.334+0000 7f48eea00640 1 -- 172.21.15.17:0/1236354221 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48f814ce80 msgr2=0x7f48f8146a80 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:38:25.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:25.334+0000 7f48eea00640 1 --2- 172.21.15.17:0/1236354221 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48f814ce80 0x7f48f8146a80 secure :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f48e4000c10 tx=0x7f48e4009260 comp rx=0 tx=0).stop 2024-09-10T21:38:25.340 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-10T21:38:25.340 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:38:25.340 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:38:25.340 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "leader", 2024-09-10T21:38:25.341 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 112, 2024-09-10T21:38:25.341 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 224720, 2024-09-10T21:38:25.341 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-10T21:38:25.341 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-10T21:38:25.341 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-10T21:38:25.341 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:38:25.341 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 0, 2024-09-10T21:38:25.341 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:38:25.341 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-10T21:38:25.341 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-10T21:38:25.341 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:38:25.342 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:38:25.342 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:38:25.342 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:38:25.342 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:38:25.342 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:38:25.342 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:38:25.342 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:38:25.342 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:38:25.342 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:38:25.342 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:38:25.342 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:38:25.342 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-10T21:38:25.343 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-10T21:38:25.343 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:38:25.343 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:38:25.343 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:38:25.343 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:38:25.343 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:38:25.343 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:38:25.343 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:38:25.343 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:38:25.343 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:38:25.344 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:38:25.344 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:38:25.344 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:38:25.344 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:38:25.344 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-10T21:38:25.344 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-10T21:38:25.344 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-10T21:38:25.344 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-10T21:38:25.344 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-10T21:38:25.344 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "5e2f023f-d6a6-49f4-a256-f4807dce3739", 2024-09-10T21:38:25.344 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:38:25.345 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:38:25.345 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-10T21:38:25.345 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-10T21:38:25.345 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-10T21:38:25.345 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-10T21:38:25.345 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-10T21:38:25.345 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-10T21:38:25.345 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-10T21:38:25.345 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:38:25.345 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-10T21:38:25.346 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:38:25.346 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:38:25.346 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:38:25.346 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:38:25.346 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:38:25.346 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:38:25.346 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:38:25.346 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:38:25.346 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:38:25.346 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:38:25.346 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:38:25.346 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:38:25.347 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-10T21:38:25.347 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:38:25.347 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-10T21:38:25.347 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:38:25.347 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:38:25.347 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:38:25.347 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:38:25.347 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:38:25.347 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:38:25.347 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:38:25.347 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-10T21:38:25.348 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:38:25.348 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:38:25.348 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:38:25.348 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:38:25.348 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-10T21:38:25.348 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:38:25.348 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:38:25.348 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:38:25.348 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:38:25.348 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-10T21:38:25.348 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-10T21:38:25.349 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:38:25.349 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:38:25.349 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:38:25.349 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:38:25.349 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:38:25.349 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:38:25.349 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:38:25.349 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:38:25.349 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:38:25.349 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:38:25.349 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:38:25.350 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3300", 2024-09-10T21:38:25.350 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:38:25.350 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:38:25.350 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:38:25.350 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:38:25.350 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789", 2024-09-10T21:38:25.350 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:38:25.350 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:38:25.350 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:38:25.350 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:38:25.351 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789/0", 2024-09-10T21:38:25.351 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6789/0", 2024-09-10T21:38:25.351 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:38:25.351 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:38:25.351 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:38:25.351 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:38:25.351 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:38:25.351 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:38:25.351 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:38:25.351 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:38:25.351 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:38:25.352 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:38:25.352 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:38:25.352 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3301", 2024-09-10T21:38:25.352 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:38:25.352 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:38:25.352 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:38:25.352 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:38:25.352 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790", 2024-09-10T21:38:25.352 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:38:25.352 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:38:25.352 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:38:25.353 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:38:25.353 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790/0", 2024-09-10T21:38:25.353 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6790/0", 2024-09-10T21:38:25.353 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:38:25.353 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:38:25.353 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:38:25.353 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:38:25.353 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:38:25.353 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:38:25.353 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-10T21:38:25.353 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-10T21:38:25.354 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:38:25.354 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:38:25.354 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:38:25.354 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:38:25.354 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:38:25.354 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:38:25.354 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": [ 2024-09-10T21:38:25.354 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:38:25.354 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:38:25.354 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:38:25.354 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 2 2024-09-10T21:38:25.355 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:38:25.355 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:38:25.355 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-10T21:38:25.355 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:38:25.355 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:38:25.355 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:38:25.355 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 4 2024-09-10T21:38:25.355 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:38:25.355 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:38:25.355 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-10T21:38:25.355 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:38:25.355 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:38:25.356 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:38:25.356 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 4 2024-09-10T21:38:25.356 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:38:25.356 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:38:25.356 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": [ 2024-09-10T21:38:25.356 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:38:25.356 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:38:25.356 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:38:25.356 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:38:25.356 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:38:25.357 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:38:25.357 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:38:25.357 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-10T21:38:25.357 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-10T21:38:25.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:25.334+0000 7f48fda00640 1 -- 172.21.15.17:0/1236354221 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f48d80724e0 msgr2=0x7f48d80749a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:38:25.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:25.334+0000 7f48fda00640 1 --2- 172.21.15.17:0/1236354221 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f48d80724e0 0x7f48d80749a0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f48f0001360 tx=0x7f48f0009250 comp rx=0 tx=0).stop 2024-09-10T21:38:25.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:25.334+0000 7f48fda00640 1 -- 172.21.15.17:0/1236354221 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f48f8073cd0 msgr2=0x7f48f81f29d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:38:25.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:25.334+0000 7f48fda00640 1 --2- 172.21.15.17:0/1236354221 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f48f8073cd0 0x7f48f81f29d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f48e8006fd0 tx=0x7f48e800b8e0 comp rx=0 tx=0).stop 2024-09-10T21:38:25.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:25.338+0000 7f48f7400640 1 -- 172.21.15.17:0/1236354221 reap_dead start 2024-09-10T21:38:25.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:25.338+0000 7f48f7400640 1 -- 172.21.15.17:0/1236354221 reap_dead start 2024-09-10T21:38:25.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:25.338+0000 7f48fda00640 1 -- 172.21.15.17:0/1236354221 shutdown_connections 2024-09-10T21:38:25.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:25.338+0000 7f48fda00640 1 -- 172.21.15.17:0/1236354221 >> 172.21.15.17:0/1236354221 conn(0x7f48f80700d0 msgr2=0x7f48f80c72c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:38:25.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:25.338+0000 7f48fda00640 1 -- 172.21.15.17:0/1236354221 shutdown_connections 2024-09-10T21:38:25.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:25.338+0000 7f48fda00640 1 -- 172.21.15.17:0/1236354221 wait complete. 2024-09-10T21:38:25.359 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-09-10T21:38:25.360 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon c 2024-09-10T21:38:25.360 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-10T21:38:25.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:25.510+0000 7f685dc00640 1 -- 172.21.15.17:0/984502256 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f683c0039c0 msgr2=0x7f683c003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:38:25.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:25.510+0000 7f685dc00640 1 --2- 172.21.15.17:0/984502256 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f683c0039c0 0x7f683c003de0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f68480099a0 tx=0x7f684801c7c0 comp rx=0 tx=0).stop 2024-09-10T21:38:25.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:25.510+0000 7f685dc00640 1 -- 172.21.15.17:0/984502256 shutdown_connections 2024-09-10T21:38:25.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:25.510+0000 7f685dc00640 1 --2- 172.21.15.17:0/984502256 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f683c0039c0 0x7f683c003de0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:38:25.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:25.510+0000 7f685dc00640 1 -- 172.21.15.17:0/984502256 >> 172.21.15.17:0/984502256 conn(0x7f68580702f0 msgr2=0x7f6858070700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:38:25.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:25.510+0000 7f685dc00640 1 -- 172.21.15.17:0/984502256 shutdown_connections 2024-09-10T21:38:25.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:25.510+0000 7f685dc00640 1 -- 172.21.15.17:0/984502256 wait complete. 2024-09-10T21:38:25.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:25.510+0000 7f685dc00640 1 Processor -- start 2024-09-10T21:38:25.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:25.510+0000 7f685dc00640 1 -- start start 2024-09-10T21:38:25.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:25.510+0000 7f685dc00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f683c0039c0 0x7f6858154e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:38:25.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:25.510+0000 7f685dc00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f685815a6a0 0x7f68581f0010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:38:25.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:25.510+0000 7f685dc00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f68581e5570 0x7f68581f3590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:38:25.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:25.510+0000 7f685dc00640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f685815bd60 con 0x7f685815a6a0 2024-09-10T21:38:25.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:25.510+0000 7f685dc00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f685815ba80 con 0x7f68581e5570 2024-09-10T21:38:25.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:25.510+0000 7f685dc00640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f68580b42e0 con 0x7f683c0039c0 2024-09-10T21:38:25.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:25.510+0000 7f684e000640 1 -- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f685815a6a0 msgr2=0x7f68581f0010 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-10T21:38:25.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:25.510+0000 7f684e000640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f685815a6a0 0x7f68581f0010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:38:25.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:25.510+0000 7f6856a00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f683c0039c0 0x7f6858154e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:38:25.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:25.510+0000 7f6856a00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f683c0039c0 0x7f6858154e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3301/0 says I am v2:172.21.15.17:47414/0 (socket says 172.21.15.17:47414) 2024-09-10T21:38:25.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:25.510+0000 7f6856a00640 1 -- 172.21.15.17:0/1697722969 learned_addr learned my addr 172.21.15.17:0/1697722969 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:38:25.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:25.510+0000 7f6857400640 1 --2- 172.21.15.17:0/1697722969 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f68581e5570 0x7f68581f3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:38:25.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:25.514+0000 7f6856a00640 1 -- 172.21.15.17:0/1697722969 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f68581e5570 msgr2=0x7f68581f3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:38:25.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:25.514+0000 7f6856a00640 1 --2- 172.21.15.17:0/1697722969 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f68581e5570 0x7f68581f3590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:38:25.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:25.514+0000 7f6856a00640 1 -- 172.21.15.17:0/1697722969 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f685815a6a0 msgr2=0x7f68581f0010 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:38:25.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:25.514+0000 7f6856a00640 1 --2- 172.21.15.17:0/1697722969 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f685815a6a0 0x7f68581f0010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:38:25.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:25.514+0000 7f6856a00640 1 -- 172.21.15.17:0/1697722969 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6858152ec0 con 0x7f683c0039c0 2024-09-10T21:38:25.517 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:25.514+0000 7f6856a00640 1 --2- 172.21.15.17:0/1697722969 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f683c0039c0 0x7f6858154e50 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f68480981d0 tx=0x7f6848004df0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:38:25.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:25.514+0000 7f684f400640 1 -- 172.21.15.17:0/1697722969 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6848005c60 con 0x7f683c0039c0 2024-09-10T21:38:25.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:25.514+0000 7f684f400640 1 -- 172.21.15.17:0/1697722969 <== mon.2 v2:172.21.15.31:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f68480be070 con 0x7f683c0039c0 2024-09-10T21:38:25.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:25.514+0000 7f684f400640 1 -- 172.21.15.17:0/1697722969 <== mon.2 v2:172.21.15.31:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68480a6bc0 con 0x7f683c0039c0 2024-09-10T21:38:25.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:25.514+0000 7f685dc00640 1 -- 172.21.15.17:0/1697722969 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68581531a0 con 0x7f683c0039c0 2024-09-10T21:38:25.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:25.514+0000 7f685dc00640 1 -- 172.21.15.17:0/1697722969 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f685815b1e0 con 0x7f683c0039c0 2024-09-10T21:38:25.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:25.514+0000 7f685dc00640 1 --2- 172.21.15.17:0/1697722969 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6858154870 0x7f68581f3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:38:25.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:25.514+0000 7f685dc00640 1 -- 172.21.15.17:0/1697722969 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f6858152c30 con 0x7f6858154870 2024-09-10T21:38:25.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:25.514+0000 7f684e000640 1 --2- 172.21.15.17:0/1697722969 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6858154870 0x7f68581f3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:38:25.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:25.514+0000 7f684e000640 1 -- 172.21.15.17:0/1697722969 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f68580c2ff0 con 0x7f6858154870 2024-09-10T21:38:25.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:25.514+0000 7f684e000640 1 --2- 172.21.15.17:0/1697722969 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6858154870 0x7f68581f3ad0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f68580c5ea0 tx=0x7f6840012870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:38:25.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:25.514+0000 7f684f400640 1 -- 172.21.15.17:0/1697722969 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6840013920 con 0x7f6858154870 2024-09-10T21:38:25.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:25.514+0000 7f684f400640 1 -- 172.21.15.17:0/1697722969 <== mon.2 v2:172.21.15.31:3301/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f6848005040 con 0x7f683c0039c0 2024-09-10T21:38:25.520 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:25.518+0000 7f684f400640 1 --2- 172.21.15.17:0/1697722969 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f68340724e0 0x7f68340749a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:38:25.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:25.518+0000 7f684f400640 1 -- 172.21.15.17:0/1697722969 <== mon.2 v2:172.21.15.31:3301/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f684812e2e0 con 0x7f683c0039c0 2024-09-10T21:38:25.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:25.518+0000 7f684f400640 1 -- 172.21.15.17:0/1697722969 <== mon.2 v2:172.21.15.31:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f6840010480 con 0x7f6858154870 2024-09-10T21:38:25.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:25.518+0000 7f684f400640 1 -- 172.21.15.17:0/1697722969 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6858154870 msgr2=0x7f68581f3ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:38:25.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:25.518+0000 7f684f400640 1 --2- 172.21.15.17:0/1697722969 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6858154870 0x7f68581f3ad0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f68580c5ea0 tx=0x7f6840012870 comp rx=0 tx=0).stop 2024-09-10T21:38:25.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:25.518+0000 7f684f400640 1 -- 172.21.15.17:0/1697722969 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6858154870 msgr2=0x7f68581f3ad0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:38:25.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:25.518+0000 7f684f400640 1 --2- 172.21.15.17:0/1697722969 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6858154870 0x7f68581f3ad0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:38:25.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:25.518+0000 7f6857400640 1 --2- 172.21.15.17:0/1697722969 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f68340724e0 0x7f68340749a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:38:25.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:25.518+0000 7f6857400640 1 --2- 172.21.15.17:0/1697722969 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f68340724e0 0x7f68340749a0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f68580712b0 tx=0x7f68500092d0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:38:25.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:25.522+0000 7f685dc00640 1 --2- 172.21.15.17:0/1697722969 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f68581e5980 0x7f68581f4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:38:25.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:25.522+0000 7f685dc00640 1 -- 172.21.15.17:0/1697722969 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f6858155ca0 con 0x7f68581e5980 2024-09-10T21:38:25.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:25.522+0000 7f684e000640 1 --2- 172.21.15.17:0/1697722969 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f68581e5980 0x7f68581f4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:38:25.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:25.526+0000 7f684e000640 1 -- 172.21.15.17:0/1697722969 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f68580c7660 con 0x7f68581e5980 2024-09-10T21:38:25.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:25.526+0000 7f684e000640 1 --2- 172.21.15.17:0/1697722969 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f68581e5980 0x7f68581f4010 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f6840016000 tx=0x7f6858158c50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:38:25.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:25.526+0000 7f684f400640 1 -- 172.21.15.17:0/1697722969 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6840010ab0 con 0x7f68581e5980 2024-09-10T21:38:25.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:25.526+0000 7f684f400640 1 -- 172.21.15.17:0/1697722969 <== mon.2 v2:172.21.15.31:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5032 (secure 0 0 0) 0x7f6840010ab0 con 0x7f68581e5980 2024-09-10T21:38:25.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:25.526+0000 7f684f400640 1 -- 172.21.15.17:0/1697722969 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f68581e5980 msgr2=0x7f68581f4010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:38:25.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:25.526+0000 7f684f400640 1 --2- 172.21.15.17:0/1697722969 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f68581e5980 0x7f68581f4010 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f6840016000 tx=0x7f6858158c50 comp rx=0 tx=0).stop 2024-09-10T21:38:25.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:25.526+0000 7f684f400640 1 -- 172.21.15.17:0/1697722969 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f68581e5980 msgr2=0x7f68581f4010 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:38:25.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:25.526+0000 7f684f400640 1 --2- 172.21.15.17:0/1697722969 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f68581e5980 0x7f68581f4010 secure :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f6840016000 tx=0x7f6858158c50 comp rx=0 tx=0).stop 2024-09-10T21:38:25.530 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-10T21:38:25.530 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:38:25.530 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:38:25.530 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-10T21:38:25.530 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 112, 2024-09-10T21:38:25.530 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 111873, 2024-09-10T21:38:25.530 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-10T21:38:25.530 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-10T21:38:25.530 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-10T21:38:25.530 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:38:25.530 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 0, 2024-09-10T21:38:25.530 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:38:25.530 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-10T21:38:25.531 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-10T21:38:25.531 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:38:25.531 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:38:25.531 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:38:25.531 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:38:25.531 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:38:25.531 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:38:25.531 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:38:25.531 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:38:25.531 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:38:25.531 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:38:25.531 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:38:25.531 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:38:25.531 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-10T21:38:25.532 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-10T21:38:25.532 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:38:25.532 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:38:25.532 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:38:25.532 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:38:25.532 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:38:25.532 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:38:25.532 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:38:25.532 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:38:25.532 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:38:25.532 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:38:25.532 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:38:25.532 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:38:25.532 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:38:25.533 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-10T21:38:25.533 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-10T21:38:25.533 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-10T21:38:25.533 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-10T21:38:25.533 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-10T21:38:25.533 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "5e2f023f-d6a6-49f4-a256-f4807dce3739", 2024-09-10T21:38:25.533 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:38:25.533 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:38:25.533 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-10T21:38:25.533 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-10T21:38:25.533 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-10T21:38:25.533 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-10T21:38:25.533 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-10T21:38:25.534 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-10T21:38:25.534 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-10T21:38:25.534 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:38:25.534 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-10T21:38:25.534 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:38:25.534 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:38:25.534 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:38:25.534 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:38:25.534 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:38:25.534 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:38:25.534 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:38:25.534 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:38:25.534 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:38:25.535 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:38:25.535 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:38:25.535 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:38:25.535 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-10T21:38:25.535 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:38:25.535 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-10T21:38:25.535 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:38:25.535 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:38:25.535 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:38:25.535 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:38:25.535 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:38:25.535 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:38:25.535 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:38:25.535 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-10T21:38:25.536 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:38:25.536 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:38:25.536 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:38:25.536 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:38:25.536 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-10T21:38:25.536 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:38:25.536 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:38:25.536 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:38:25.536 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:38:25.536 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-10T21:38:25.536 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-10T21:38:25.536 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:38:25.536 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:38:25.536 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:38:25.537 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:38:25.537 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:38:25.537 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:38:25.537 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:38:25.537 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:38:25.537 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:38:25.537 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:38:25.537 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:38:25.537 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3300", 2024-09-10T21:38:25.537 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:38:25.537 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:38:25.537 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:38:25.537 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:38:25.537 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789", 2024-09-10T21:38:25.538 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:38:25.538 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:38:25.538 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:38:25.538 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:38:25.538 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789/0", 2024-09-10T21:38:25.538 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6789/0", 2024-09-10T21:38:25.538 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:38:25.538 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:38:25.538 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:38:25.539 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:38:25.539 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:38:25.539 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:38:25.539 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:38:25.539 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:38:25.539 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:38:25.539 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:38:25.539 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:38:25.539 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3301", 2024-09-10T21:38:25.539 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:38:25.539 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:38:25.539 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:38:25.539 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:38:25.539 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790", 2024-09-10T21:38:25.540 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:38:25.540 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:38:25.540 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:38:25.540 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:38:25.540 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790/0", 2024-09-10T21:38:25.540 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6790/0", 2024-09-10T21:38:25.540 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:38:25.540 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:38:25.540 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:38:25.540 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:38:25.540 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:38:25.540 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:38:25.541 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-10T21:38:25.541 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-10T21:38:25.541 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:38:25.541 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:38:25.541 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:38:25.541 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:38:25.541 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:38:25.541 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:38:25.541 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": [ 2024-09-10T21:38:25.541 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:38:25.541 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:38:25.541 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:38:25.541 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 4 2024-09-10T21:38:25.541 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:38:25.542 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:38:25.542 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-10T21:38:25.542 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:38:25.542 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:38:25.542 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:38:25.542 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 4 2024-09-10T21:38:25.542 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:38:25.542 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:38:25.542 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-10T21:38:25.542 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:38:25.542 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:38:25.542 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:38:25.542 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 4 2024-09-10T21:38:25.542 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:38:25.543 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:38:25.543 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": [ 2024-09-10T21:38:25.543 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:38:25.543 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:38:25.543 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:38:25.543 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:38:25.543 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:38:25.543 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:38:25.543 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:38:25.543 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-10T21:38:25.543 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-10T21:38:25.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:25.526+0000 7f685dc00640 1 -- 172.21.15.17:0/1697722969 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f68340724e0 msgr2=0x7f68340749a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:38:25.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:25.526+0000 7f685dc00640 1 --2- 172.21.15.17:0/1697722969 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f68340724e0 0x7f68340749a0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f68580712b0 tx=0x7f68500092d0 comp rx=0 tx=0).stop 2024-09-10T21:38:25.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:25.526+0000 7f685dc00640 1 -- 172.21.15.17:0/1697722969 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f683c0039c0 msgr2=0x7f6858154e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:38:25.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:25.526+0000 7f685dc00640 1 --2- 172.21.15.17:0/1697722969 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f683c0039c0 0x7f6858154e50 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f68480981d0 tx=0x7f6848004df0 comp rx=0 tx=0).stop 2024-09-10T21:38:25.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:25.526+0000 7f6857400640 1 -- 172.21.15.17:0/1697722969 reap_dead start 2024-09-10T21:38:25.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:25.526+0000 7f6857400640 1 -- 172.21.15.17:0/1697722969 reap_dead start 2024-09-10T21:38:25.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:25.526+0000 7f685dc00640 1 -- 172.21.15.17:0/1697722969 shutdown_connections 2024-09-10T21:38:25.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:25.526+0000 7f685dc00640 1 -- 172.21.15.17:0/1697722969 >> 172.21.15.17:0/1697722969 conn(0x7f68580702f0 msgr2=0x7f68580c6110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:38:25.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:25.526+0000 7f685dc00640 1 -- 172.21.15.17:0/1697722969 shutdown_connections 2024-09-10T21:38:25.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:25.526+0000 7f685dc00640 1 -- 172.21.15.17:0/1697722969 wait complete. 2024-09-10T21:38:25.547 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-09-10T21:38:25.547 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2024-09-10T21:38:45.548 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.a 2024-09-10T21:38:45.549 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-10T21:38:45.549 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-10T21:38:45.553 INFO:tasks.ceph.mon.a:Started 2024-09-10T21:38:45.553 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-09-10T21:38:45.553 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-10T21:38:45.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:45.866+0000 7f313be00640 1 -- 172.21.15.17:0/3114677361 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f313c06f480 msgr2=0x7f313c0c7870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:38:45.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:45.866+0000 7f313be00640 1 --2- 172.21.15.17:0/3114677361 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f313c06f480 0x7f313c0c7870 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f313c072990 tx=0x7f312c01c800 comp rx=0 tx=0).stop 2024-09-10T21:38:45.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:45.866+0000 7f313be00640 1 -- 172.21.15.17:0/3114677361 shutdown_connections 2024-09-10T21:38:45.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:45.866+0000 7f313be00640 1 --2- 172.21.15.17:0/3114677361 >> v2:172.21.15.17:3300/0 conn(0x7f313c0c8070 0x7f313c0b2c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:38:45.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:45.866+0000 7f313be00640 1 --2- 172.21.15.17:0/3114677361 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f313c06f480 0x7f313c0c7870 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:38:45.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:45.866+0000 7f313be00640 1 -- 172.21.15.17:0/3114677361 >> 172.21.15.17:0/3114677361 conn(0x7f313c070ca0 msgr2=0x7f313c0710b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:38:45.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:45.866+0000 7f313be00640 1 -- 172.21.15.17:0/3114677361 shutdown_connections 2024-09-10T21:38:45.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:45.866+0000 7f313be00640 1 -- 172.21.15.17:0/3114677361 wait complete. 2024-09-10T21:38:45.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:45.866+0000 7f313be00640 1 Processor -- start 2024-09-10T21:38:45.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:45.866+0000 7f313be00640 1 -- start start 2024-09-10T21:38:45.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:45.870+0000 7f313be00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f313c06f480 0x7f313c14a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:38:45.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:45.870+0000 7f313be00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f313c06f860 0x7f313c074640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:38:45.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:45.870+0000 7f313be00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f313c0c8070 0x7f313c1429b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:38:45.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:45.870+0000 7f313be00640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f313c163e60 con 0x7f313c06f480 2024-09-10T21:38:45.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:45.870+0000 7f313be00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f313c163ce0 con 0x7f313c0c8070 2024-09-10T21:38:45.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:45.870+0000 7f313be00640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f313c163fe0 con 0x7f313c06f860 2024-09-10T21:38:45.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:45.870+0000 7f313aa00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f313c06f480 0x7f313c14a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:38:45.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:45.870+0000 7f313aa00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f313c06f480 0x7f313c14a840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:54134/0 (socket says 172.21.15.17:54134) 2024-09-10T21:38:45.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:45.870+0000 7f313aa00640 1 -- 172.21.15.17:0/3441865606 learned_addr learned my addr 172.21.15.17:0/3441865606 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:38:45.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:45.870+0000 7f313aa00640 1 -- 172.21.15.17:0/3441865606 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f313c06f480 msgr2=0x7f313c14a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-09-10T21:38:45.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:45.870+0000 7f313aa00640 1 -- 172.21.15.17:0/3441865606 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f313c06f480 msgr2=0x7f313c14a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-10T21:38:45.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:45.870+0000 7f313aa00640 1 --2- 172.21.15.17:0/3441865606 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f313c06f480 0x7f313c14a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T21:38:45.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:45.870+0000 7f313aa00640 1 --2- 172.21.15.17:0/3441865606 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f313c06f480 0x7f313c14a840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:38:45.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:45.870+0000 7f313a000640 1 --2- 172.21.15.17:0/3441865606 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f313c06f860 0x7f313c074640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:38:45.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:45.870+0000 7f313b400640 1 --2- 172.21.15.17:0/3441865606 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f313c0c8070 0x7f313c1429b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:38:45.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:45.870+0000 7f313b400640 1 -- 172.21.15.17:0/3441865606 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f313c06f860 msgr2=0x7f313c074640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:38:45.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:45.870+0000 7f313b400640 1 --2- 172.21.15.17:0/3441865606 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f313c06f860 0x7f313c074640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:38:45.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:45.870+0000 7f313b400640 1 -- 172.21.15.17:0/3441865606 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f313c06f480 msgr2=0x7f313c14a840 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:38:45.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:45.870+0000 7f313b400640 1 --2- 172.21.15.17:0/3441865606 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f313c06f480 0x7f313c14a840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:38:45.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:45.870+0000 7f313b400640 1 -- 172.21.15.17:0/3441865606 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f313c150390 con 0x7f313c0c8070 2024-09-10T21:38:45.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:45.870+0000 7f313a000640 1 --2- 172.21.15.17:0/3441865606 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f313c06f860 0x7f313c074640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:38:45.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:45.870+0000 7f313b400640 1 --2- 172.21.15.17:0/3441865606 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f313c0c8070 0x7f313c1429b0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f313400edd0 tx=0x7f313400c6e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:38:45.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:45.870+0000 7f3132a00640 1 -- 172.21.15.17:0/3441865606 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f313401c070 con 0x7f313c0c8070 2024-09-10T21:38:45.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:45.870+0000 7f3132a00640 1 -- 172.21.15.17:0/3441865606 <== mon.1 v2:172.21.15.31:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3134017750 con 0x7f313c0c8070 2024-09-10T21:38:45.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:45.870+0000 7f3132a00640 1 -- 172.21.15.17:0/3441865606 <== mon.1 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3134017ba0 con 0x7f313c0c8070 2024-09-10T21:38:45.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:45.870+0000 7f313be00640 1 -- 172.21.15.17:0/3441865606 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f313c144610 con 0x7f313c0c8070 2024-09-10T21:38:45.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:45.870+0000 7f313be00640 1 -- 172.21.15.17:0/3441865606 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f313c14bd40 con 0x7f313c0c8070 2024-09-10T21:38:45.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:45.870+0000 7f313be00640 1 -- 172.21.15.17:0/3441865606 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f313c0c7870 con 0x7f313c0c8070 2024-09-10T21:38:45.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:45.874+0000 7f3132a00640 1 -- 172.21.15.17:0/3441865606 <== mon.1 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f313400b030 con 0x7f313c0c8070 2024-09-10T21:38:45.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:45.874+0000 7f3132a00640 1 --2- 172.21.15.17:0/3441865606 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f311c0725e0 0x7f311c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:38:45.878 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:45.874+0000 7f3132a00640 1 -- 172.21.15.17:0/3441865606 <== mon.1 v2:172.21.15.31:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f31340941f0 con 0x7f313c0c8070 2024-09-10T21:38:45.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:45.874+0000 7f313aa00640 1 --2- 172.21.15.17:0/3441865606 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f311c0725e0 0x7f311c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:38:45.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:45.874+0000 7f313aa00640 1 --2- 172.21.15.17:0/3441865606 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f311c0725e0 0x7f311c074aa0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f312c01a430 tx=0x7f312c01b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:38:45.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:45.878+0000 7f3132a00640 1 -- 172.21.15.17:0/3441865606 <== mon.1 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f313405eef0 con 0x7f313c0c8070 2024-09-10T21:38:46.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:46.162+0000 7f313be00640 1 -- 172.21.15.17:0/3441865606 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f313c071ce0 con 0x7f313c0c8070 2024-09-10T21:38:46.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:46.162+0000 7f3132a00640 1 -- 172.21.15.17:0/3441865606 <== mon.1 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1552 (secure 0 0 0) 0x7f3134063920 con 0x7f313c0c8070 2024-09-10T21:38:46.167 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":112,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":21,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5e2f023f-d6a6-49f4-a256-f4807dce3739","modified":"2024-09-10T21:23:46.892130Z","created":"2024-09-10T21:23:46.892130Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3301","nonce":0},{"type":"v1","addr":"172.21.15.31:6790","nonce":0}]},"addr":"172.21.15.31:6790/0","public_addr":"172.21.15.31:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-10T21:38:46.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:46.166+0000 7f313be00640 1 -- 172.21.15.17:0/3441865606 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f311c0725e0 msgr2=0x7f311c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:38:46.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:46.166+0000 7f313be00640 1 --2- 172.21.15.17:0/3441865606 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f311c0725e0 0x7f311c074aa0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f312c01a430 tx=0x7f312c01b040 comp rx=0 tx=0).stop 2024-09-10T21:38:46.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:46.166+0000 7f313be00640 1 -- 172.21.15.17:0/3441865606 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f313c0c8070 msgr2=0x7f313c1429b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:38:46.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:46.166+0000 7f313be00640 1 --2- 172.21.15.17:0/3441865606 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f313c0c8070 0x7f313c1429b0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f313400edd0 tx=0x7f313400c6e0 comp rx=0 tx=0).stop 2024-09-10T21:38:46.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:46.166+0000 7f313be00640 1 -- 172.21.15.17:0/3441865606 shutdown_connections 2024-09-10T21:38:46.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:46.166+0000 7f313be00640 1 --2- 172.21.15.17:0/3441865606 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f311c0725e0 0x7f311c074aa0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:38:46.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:46.166+0000 7f313be00640 1 --2- 172.21.15.17:0/3441865606 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f313c0c8070 0x7f313c1429b0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:38:46.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:46.166+0000 7f313be00640 1 --2- 172.21.15.17:0/3441865606 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f313c06f860 0x7f313c074640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:38:46.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:46.166+0000 7f313be00640 1 --2- 172.21.15.17:0/3441865606 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f313c06f480 0x7f313c14a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:38:46.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:46.166+0000 7f313be00640 1 -- 172.21.15.17:0/3441865606 >> 172.21.15.17:0/3441865606 conn(0x7f313c070ca0 msgr2=0x7f313c06d420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:38:46.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:46.166+0000 7f313be00640 1 -- 172.21.15.17:0/3441865606 shutdown_connections 2024-09-10T21:38:46.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:46.166+0000 7f313be00640 1 -- 172.21.15.17:0/3441865606 wait complete. 2024-09-10T21:38:49.212 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-10T21:38:49.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:49.530+0000 7f6a02a00640 1 -- 172.21.15.17:0/1374534186 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f69fc0c99f0 msgr2=0x7f69fc0d0000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:38:49.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:49.530+0000 7f6a02a00640 1 --2- 172.21.15.17:0/1374534186 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f69fc0c99f0 0x7f69fc0d0000 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f69fc0cdb40 tx=0x7f69f001eff0 comp rx=0 tx=0).stop 2024-09-10T21:38:49.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:49.530+0000 7f6a02a00640 1 -- 172.21.15.17:0/1374534186 shutdown_connections 2024-09-10T21:38:49.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:49.530+0000 7f6a02a00640 1 --2- 172.21.15.17:0/1374534186 >> v2:172.21.15.17:3300/0 conn(0x7f69fc0b30a0 0x7f69fc15da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:38:49.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:49.530+0000 7f6a02a00640 1 --2- 172.21.15.17:0/1374534186 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f69fc0c99f0 0x7f69fc0d0000 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:38:49.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:49.530+0000 7f6a02a00640 1 -- 172.21.15.17:0/1374534186 >> 172.21.15.17:0/1374534186 conn(0x7f69fc06d700 msgr2=0x7f69fc068990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:38:49.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:49.530+0000 7f6a02a00640 1 -- 172.21.15.17:0/1374534186 shutdown_connections 2024-09-10T21:38:49.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:49.530+0000 7f6a02a00640 1 -- 172.21.15.17:0/1374534186 wait complete. 2024-09-10T21:38:49.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:49.530+0000 7f6a02a00640 1 Processor -- start 2024-09-10T21:38:49.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:49.530+0000 7f6a02a00640 1 -- start start 2024-09-10T21:38:49.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:49.534+0000 7f6a02a00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f69fc0b30a0 0x7f69fc0cde90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:38:49.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:49.534+0000 7f6a02a00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f69fc0c99f0 0x7f69fc159520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:38:49.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:49.534+0000 7f6a02a00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f69fc0caca0 0x7f69fc1f09f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:38:49.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:49.534+0000 7f6a02a00640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f69fc15abb0 con 0x7f69fc0c99f0 2024-09-10T21:38:49.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:49.534+0000 7f6a02a00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f69fc15a8d0 con 0x7f69fc0caca0 2024-09-10T21:38:49.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:49.534+0000 7f6a02a00640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f69fc0c9dd0 con 0x7f69fc0b30a0 2024-09-10T21:38:49.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:49.534+0000 7f6a00c00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f69fc0c99f0 0x7f69fc159520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:38:49.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:49.534+0000 7f6a00c00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f69fc0c99f0 0x7f69fc159520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:54160/0 (socket says 172.21.15.17:54160) 2024-09-10T21:38:49.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:49.534+0000 7f6a00c00640 1 -- 172.21.15.17:0/4229269519 learned_addr learned my addr 172.21.15.17:0/4229269519 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:38:49.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:49.534+0000 7f6a00c00640 1 -- 172.21.15.17:0/4229269519 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f69fc0c99f0 msgr2=0x7f69fc159520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-09-10T21:38:49.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:49.534+0000 7f6a00c00640 1 -- 172.21.15.17:0/4229269519 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f69fc0c99f0 msgr2=0x7f69fc159520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-10T21:38:49.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:49.534+0000 7f6a00c00640 1 --2- 172.21.15.17:0/4229269519 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f69fc0c99f0 0x7f69fc159520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T21:38:49.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:49.534+0000 7f6a00c00640 1 --2- 172.21.15.17:0/4229269519 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f69fc0c99f0 0x7f69fc159520 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:38:49.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:49.534+0000 7f6a02000640 1 --2- 172.21.15.17:0/4229269519 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f69fc0caca0 0x7f69fc1f09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:38:49.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:49.534+0000 7f6a01600640 1 --2- 172.21.15.17:0/4229269519 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f69fc0b30a0 0x7f69fc0cde90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:38:49.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:49.534+0000 7f6a02000640 1 -- 172.21.15.17:0/4229269519 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f69fc0b30a0 msgr2=0x7f69fc0cde90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:38:49.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:49.534+0000 7f6a02000640 1 --2- 172.21.15.17:0/4229269519 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f69fc0b30a0 0x7f69fc0cde90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:38:49.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:49.534+0000 7f6a02000640 1 -- 172.21.15.17:0/4229269519 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f69fc0c99f0 msgr2=0x7f69fc159520 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:38:49.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:49.534+0000 7f6a02000640 1 --2- 172.21.15.17:0/4229269519 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f69fc0c99f0 0x7f69fc159520 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:38:49.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:49.534+0000 7f6a02000640 1 -- 172.21.15.17:0/4229269519 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69fc0ca990 con 0x7f69fc0caca0 2024-09-10T21:38:49.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:49.534+0000 7f6a02000640 1 --2- 172.21.15.17:0/4229269519 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f69fc0caca0 0x7f69fc1f09f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f69f400ca20 tx=0x7f69f400cb10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:38:49.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:49.534+0000 7f69f9600640 1 -- 172.21.15.17:0/4229269519 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69f4016020 con 0x7f69fc0caca0 2024-09-10T21:38:49.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:49.534+0000 7f69f9600640 1 -- 172.21.15.17:0/4229269519 <== mon.1 v2:172.21.15.31:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f69f400a930 con 0x7f69fc0caca0 2024-09-10T21:38:49.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:49.534+0000 7f69f9600640 1 -- 172.21.15.17:0/4229269519 <== mon.1 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69f400ada0 con 0x7f69fc0caca0 2024-09-10T21:38:49.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:49.534+0000 7f6a02a00640 1 -- 172.21.15.17:0/4229269519 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69fc0cb9b0 con 0x7f69fc0caca0 2024-09-10T21:38:49.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:49.534+0000 7f6a02a00640 1 -- 172.21.15.17:0/4229269519 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69fc1f41f0 con 0x7f69fc0caca0 2024-09-10T21:38:49.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:49.534+0000 7f6a02a00640 1 -- 172.21.15.17:0/4229269519 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69fc15b400 con 0x7f69fc0caca0 2024-09-10T21:38:49.540 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:49.538+0000 7f69f9600640 1 -- 172.21.15.17:0/4229269519 <== mon.1 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f69f401a3b0 con 0x7f69fc0caca0 2024-09-10T21:38:49.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:49.538+0000 7f69f9600640 1 --2- 172.21.15.17:0/4229269519 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f69dc0725e0 0x7f69dc074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:38:49.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:49.538+0000 7f69f9600640 1 -- 172.21.15.17:0/4229269519 <== mon.1 v2:172.21.15.31:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f69f40937d0 con 0x7f69fc0caca0 2024-09-10T21:38:49.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:49.538+0000 7f6a01600640 1 --2- 172.21.15.17:0/4229269519 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f69dc0725e0 0x7f69dc074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:38:49.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:49.538+0000 7f6a01600640 1 --2- 172.21.15.17:0/4229269519 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f69dc0725e0 0x7f69dc074aa0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f69f00b2000 tx=0x7f69f001d830 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:38:49.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:49.542+0000 7f69f9600640 1 -- 172.21.15.17:0/4229269519 <== mon.1 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f69f4020030 con 0x7f69fc0caca0 2024-09-10T21:38:49.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:49.822+0000 7f6a02a00640 1 -- 172.21.15.17:0/4229269519 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f69fc0c3520 con 0x7f69fc0caca0 2024-09-10T21:38:49.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:49.822+0000 7f69f9600640 1 -- 172.21.15.17:0/4229269519 <== mon.1 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1552 (secure 0 0 0) 0x7f69f405ea10 con 0x7f69fc0caca0 2024-09-10T21:38:49.827 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":112,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":25,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5e2f023f-d6a6-49f4-a256-f4807dce3739","modified":"2024-09-10T21:23:46.892130Z","created":"2024-09-10T21:23:46.892130Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3301","nonce":0},{"type":"v1","addr":"172.21.15.31:6790","nonce":0}]},"addr":"172.21.15.31:6790/0","public_addr":"172.21.15.31:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-10T21:38:49.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:49.826+0000 7f6a02a00640 1 -- 172.21.15.17:0/4229269519 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f69dc0725e0 msgr2=0x7f69dc074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:38:49.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:49.826+0000 7f6a02a00640 1 --2- 172.21.15.17:0/4229269519 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f69dc0725e0 0x7f69dc074aa0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f69f00b2000 tx=0x7f69f001d830 comp rx=0 tx=0).stop 2024-09-10T21:38:49.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:49.826+0000 7f6a02a00640 1 -- 172.21.15.17:0/4229269519 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f69fc0caca0 msgr2=0x7f69fc1f09f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:38:49.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:49.826+0000 7f6a02a00640 1 --2- 172.21.15.17:0/4229269519 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f69fc0caca0 0x7f69fc1f09f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f69f400ca20 tx=0x7f69f400cb10 comp rx=0 tx=0).stop 2024-09-10T21:38:49.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:49.826+0000 7f6a02a00640 1 -- 172.21.15.17:0/4229269519 shutdown_connections 2024-09-10T21:38:49.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:49.826+0000 7f6a02a00640 1 --2- 172.21.15.17:0/4229269519 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f69dc0725e0 0x7f69dc074aa0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:38:49.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:49.826+0000 7f6a02a00640 1 --2- 172.21.15.17:0/4229269519 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f69fc0caca0 0x7f69fc1f09f0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:38:49.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:49.826+0000 7f6a02a00640 1 --2- 172.21.15.17:0/4229269519 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f69fc0c99f0 0x7f69fc159520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:38:49.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:49.826+0000 7f6a02a00640 1 --2- 172.21.15.17:0/4229269519 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f69fc0b30a0 0x7f69fc0cde90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:38:49.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:49.826+0000 7f6a02a00640 1 -- 172.21.15.17:0/4229269519 >> 172.21.15.17:0/4229269519 conn(0x7f69fc06d700 msgr2=0x7f69fc0b34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:38:49.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:49.826+0000 7f6a02a00640 1 -- 172.21.15.17:0/4229269519 shutdown_connections 2024-09-10T21:38:49.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:49.826+0000 7f6a02a00640 1 -- 172.21.15.17:0/4229269519 wait complete. 2024-09-10T21:38:51.054 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:38:51.057 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:38:52.851 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-10T21:38:53.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:53.102+0000 7fbcfce00640 1 -- 172.21.15.17:0/247143846 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbcf8152820 msgr2=0x7fbcf815c280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:38:53.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:53.102+0000 7fbcfce00640 1 --2- 172.21.15.17:0/247143846 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbcf8152820 0x7fbcf815c280 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fbcf80cbaa0 tx=0x7fbce401c770 comp rx=0 tx=0).stop 2024-09-10T21:38:53.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:53.102+0000 7fbcfce00640 1 -- 172.21.15.17:0/247143846 shutdown_connections 2024-09-10T21:38:53.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:53.102+0000 7fbcfce00640 1 --2- 172.21.15.17:0/247143846 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbcf8152820 0x7fbcf815c280 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:38:53.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:53.102+0000 7fbcfce00640 1 --2- 172.21.15.17:0/247143846 >> v2:172.21.15.17:3300/0 conn(0x7fbcf8152440 0x7fbcf80b3eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:38:53.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:53.102+0000 7fbcfce00640 1 -- 172.21.15.17:0/247143846 >> 172.21.15.17:0/247143846 conn(0x7fbcf80702f0 msgr2=0x7fbcf8070700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:38:53.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:53.102+0000 7fbcfce00640 1 -- 172.21.15.17:0/247143846 shutdown_connections 2024-09-10T21:38:53.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:53.102+0000 7fbcfce00640 1 -- 172.21.15.17:0/247143846 wait complete. 2024-09-10T21:38:53.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:53.106+0000 7fbcfce00640 1 Processor -- start 2024-09-10T21:38:53.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:53.106+0000 7fbcfce00640 1 -- start start 2024-09-10T21:38:53.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:53.106+0000 7fbcfce00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fbcf80cdf20 0x7fbcf8144570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:38:53.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:53.106+0000 7fbcfce00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbcf8152440 0x7fbcf8143ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:38:53.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:53.106+0000 7fbcfce00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbcf8152820 0x7fbcf81457a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:38:53.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:53.106+0000 7fbcfce00640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fbcf815c7c0 con 0x7fbcf8152440 2024-09-10T21:38:53.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:53.106+0000 7fbcfce00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fbcf80ce740 con 0x7fbcf8152820 2024-09-10T21:38:53.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:53.106+0000 7fbcfce00640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7fbcf80cb450 con 0x7fbcf80cdf20 2024-09-10T21:38:53.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:53.106+0000 7fbcf5400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbcf8152440 0x7fbcf8143ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:38:53.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:53.106+0000 7fbcf5400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbcf8152440 0x7fbcf8143ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:54192/0 (socket says 172.21.15.17:54192) 2024-09-10T21:38:53.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:53.106+0000 7fbcf5400640 1 -- 172.21.15.17:0/158250501 learned_addr learned my addr 172.21.15.17:0/158250501 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:38:53.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:53.106+0000 7fbcf5400640 1 -- 172.21.15.17:0/158250501 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbcf8152440 msgr2=0x7fbcf8143ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-09-10T21:38:53.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:53.106+0000 7fbcf5400640 1 -- 172.21.15.17:0/158250501 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbcf8152440 msgr2=0x7fbcf8143ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-10T21:38:53.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:53.106+0000 7fbcf5400640 1 --2- 172.21.15.17:0/158250501 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbcf8152440 0x7fbcf8143ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T21:38:53.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:53.106+0000 7fbcf5e00640 1 --2- 172.21.15.17:0/158250501 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fbcf80cdf20 0x7fbcf8144570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:38:53.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:53.106+0000 7fbcf5400640 1 --2- 172.21.15.17:0/158250501 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbcf8152440 0x7fbcf8143ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:38:53.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:53.106+0000 7fbcf6800640 1 --2- 172.21.15.17:0/158250501 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbcf8152820 0x7fbcf81457a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:38:53.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:53.106+0000 7fbcf5e00640 1 -- 172.21.15.17:0/158250501 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbcf8152820 msgr2=0x7fbcf81457a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:38:53.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:53.106+0000 7fbcf5e00640 1 --2- 172.21.15.17:0/158250501 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbcf8152820 0x7fbcf81457a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:38:53.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:53.106+0000 7fbcf5e00640 1 -- 172.21.15.17:0/158250501 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbcf8152440 msgr2=0x7fbcf8143ba0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:38:53.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:53.106+0000 7fbcf5e00640 1 --2- 172.21.15.17:0/158250501 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbcf8152440 0x7fbcf8143ba0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:38:53.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:53.106+0000 7fbcf5e00640 1 -- 172.21.15.17:0/158250501 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcf806f4b0 con 0x7fbcf80cdf20 2024-09-10T21:38:53.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:53.106+0000 7fbcf6800640 1 --2- 172.21.15.17:0/158250501 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbcf8152820 0x7fbcf81457a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:38:53.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:53.106+0000 7fbcf5e00640 1 --2- 172.21.15.17:0/158250501 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fbcf80cdf20 0x7fbcf8144570 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fbce800d750 tx=0x7fbce800dc10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:38:53.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:53.106+0000 7fbcee000640 1 -- 172.21.15.17:0/158250501 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbce8017070 con 0x7fbcf80cdf20 2024-09-10T21:38:53.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:53.106+0000 7fbcee000640 1 -- 172.21.15.17:0/158250501 <== mon.2 v2:172.21.15.31:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbce8004330 con 0x7fbcf80cdf20 2024-09-10T21:38:53.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:53.106+0000 7fbcfce00640 1 -- 172.21.15.17:0/158250501 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbcf81f7970 con 0x7fbcf80cdf20 2024-09-10T21:38:53.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:53.106+0000 7fbcee000640 1 -- 172.21.15.17:0/158250501 <== mon.2 v2:172.21.15.31:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbce8009c30 con 0x7fbcf80cdf20 2024-09-10T21:38:53.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:53.106+0000 7fbcfce00640 1 -- 172.21.15.17:0/158250501 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fbcf806f720 con 0x7fbcf80cdf20 2024-09-10T21:38:53.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:53.110+0000 7fbcfce00640 1 -- 172.21.15.17:0/158250501 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbcf80c3040 con 0x7fbcf80cdf20 2024-09-10T21:38:53.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:53.110+0000 7fbcee000640 1 -- 172.21.15.17:0/158250501 <== mon.2 v2:172.21.15.31:3301/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fbce8003600 con 0x7fbcf80cdf20 2024-09-10T21:38:53.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:53.110+0000 7fbcee000640 1 --2- 172.21.15.17:0/158250501 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fbcd80725e0 0x7fbcd8074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:38:53.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:53.110+0000 7fbcee000640 1 -- 172.21.15.17:0/158250501 <== mon.2 v2:172.21.15.31:3301/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fbce8093bc0 con 0x7fbcf80cdf20 2024-09-10T21:38:53.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:53.110+0000 7fbcf5400640 1 --2- 172.21.15.17:0/158250501 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fbcd80725e0 0x7fbcd8074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:38:53.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:53.114+0000 7fbcf5400640 1 --2- 172.21.15.17:0/158250501 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fbcd80725e0 0x7fbcd8074aa0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fbce401a3a0 tx=0x7fbce401afb0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:38:53.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:53.114+0000 7fbcee000640 1 -- 172.21.15.17:0/158250501 <== mon.2 v2:172.21.15.31:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fbce800de60 con 0x7fbcf80cdf20 2024-09-10T21:38:53.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:53.370+0000 7fbcfce00640 1 -- 172.21.15.17:0/158250501 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fbcf80709b0 con 0x7fbcf80cdf20 2024-09-10T21:38:53.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:53.370+0000 7fbcee000640 1 -- 172.21.15.17:0/158250501 <== mon.2 v2:172.21.15.31:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1552 (secure 0 0 0) 0x7fbce805e880 con 0x7fbcf80cdf20 2024-09-10T21:38:53.374 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":112,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":28,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5e2f023f-d6a6-49f4-a256-f4807dce3739","modified":"2024-09-10T21:23:46.892130Z","created":"2024-09-10T21:23:46.892130Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3301","nonce":0},{"type":"v1","addr":"172.21.15.31:6790","nonce":0}]},"addr":"172.21.15.31:6790/0","public_addr":"172.21.15.31:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-10T21:38:53.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:53.374+0000 7fbcfce00640 1 -- 172.21.15.17:0/158250501 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fbcd80725e0 msgr2=0x7fbcd8074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:38:53.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:53.374+0000 7fbcfce00640 1 --2- 172.21.15.17:0/158250501 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fbcd80725e0 0x7fbcd8074aa0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fbce401a3a0 tx=0x7fbce401afb0 comp rx=0 tx=0).stop 2024-09-10T21:38:53.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:53.374+0000 7fbcfce00640 1 -- 172.21.15.17:0/158250501 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fbcf80cdf20 msgr2=0x7fbcf8144570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:38:53.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:53.374+0000 7fbcfce00640 1 --2- 172.21.15.17:0/158250501 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fbcf80cdf20 0x7fbcf8144570 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fbce800d750 tx=0x7fbce800dc10 comp rx=0 tx=0).stop 2024-09-10T21:38:53.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:53.374+0000 7fbcfce00640 1 -- 172.21.15.17:0/158250501 shutdown_connections 2024-09-10T21:38:53.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:53.374+0000 7fbcfce00640 1 --2- 172.21.15.17:0/158250501 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fbcd80725e0 0x7fbcd8074aa0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:38:53.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:53.374+0000 7fbcfce00640 1 --2- 172.21.15.17:0/158250501 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbcf8152820 0x7fbcf81457a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:38:53.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:53.374+0000 7fbcfce00640 1 --2- 172.21.15.17:0/158250501 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbcf8152440 0x7fbcf8143ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:38:53.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:53.374+0000 7fbcfce00640 1 --2- 172.21.15.17:0/158250501 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fbcf80cdf20 0x7fbcf8144570 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:38:53.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:53.374+0000 7fbcfce00640 1 -- 172.21.15.17:0/158250501 >> 172.21.15.17:0/158250501 conn(0x7fbcf80702f0 msgr2=0x7fbcf80cbe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:38:53.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:53.374+0000 7fbcfce00640 1 -- 172.21.15.17:0/158250501 shutdown_connections 2024-09-10T21:38:53.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:53.374+0000 7fbcfce00640 1 -- 172.21.15.17:0/158250501 wait complete. 2024-09-10T21:38:56.415 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-10T21:38:56.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:56.618+0000 7f0d57000640 1 -- 172.21.15.17:0/832179226 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0d380052f0 msgr2=0x7f0d3800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:38:56.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:56.618+0000 7f0d57000640 1 --2- 172.21.15.17:0/832179226 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0d380052f0 0x7f0d3800b9d0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f0d48001df0 tx=0x7f0d4801caf0 comp rx=0 tx=0).stop 2024-09-10T21:38:56.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:56.618+0000 7f0d57000640 1 -- 172.21.15.17:0/832179226 shutdown_connections 2024-09-10T21:38:56.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:56.618+0000 7f0d57000640 1 --2- 172.21.15.17:0/832179226 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0d380052f0 0x7f0d3800b9d0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:38:56.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:56.618+0000 7f0d57000640 1 -- 172.21.15.17:0/832179226 >> 172.21.15.17:0/832179226 conn(0x7f0d50070ca0 msgr2=0x7f0d500710b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:38:56.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:56.618+0000 7f0d57000640 1 -- 172.21.15.17:0/832179226 shutdown_connections 2024-09-10T21:38:56.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:56.618+0000 7f0d57000640 1 -- 172.21.15.17:0/832179226 wait complete. 2024-09-10T21:38:56.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:56.622+0000 7f0d57000640 1 Processor -- start 2024-09-10T21:38:56.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:56.622+0000 7f0d57000640 1 -- start start 2024-09-10T21:38:56.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:56.622+0000 7f0d57000640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0d380052f0 0x7f0d50148130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:38:56.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:56.622+0000 7f0d57000640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0d501e02e0 0x7f0d501e0e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:38:56.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:56.622+0000 7f0d57000640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f0d501e13a0 0x7f0d50141930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:38:56.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:56.622+0000 7f0d57000640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f0d500c7870 con 0x7f0d380052f0 2024-09-10T21:38:56.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:56.622+0000 7f0d57000640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f0d500702a0 con 0x7f0d501e02e0 2024-09-10T21:38:56.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:56.622+0000 7f0d57000640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f0d500c79f0 con 0x7f0d501e13a0 2024-09-10T21:38:56.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:56.622+0000 7f0d55c00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0d380052f0 0x7f0d50148130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:38:56.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:56.622+0000 7f0d55c00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0d380052f0 0x7f0d50148130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49638/0 (socket says 172.21.15.17:49638) 2024-09-10T21:38:56.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:56.622+0000 7f0d55c00640 1 -- 172.21.15.17:0/734102553 learned_addr learned my addr 172.21.15.17:0/734102553 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:38:56.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:56.622+0000 7f0d55c00640 1 -- 172.21.15.17:0/734102553 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0d380052f0 msgr2=0x7f0d50148130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-09-10T21:38:56.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:56.622+0000 7f0d55c00640 1 -- 172.21.15.17:0/734102553 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0d380052f0 msgr2=0x7f0d50148130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-10T21:38:56.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:56.622+0000 7f0d55c00640 1 --2- 172.21.15.17:0/734102553 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0d380052f0 0x7f0d50148130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T21:38:56.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:56.622+0000 7f0d56600640 1 --2- 172.21.15.17:0/734102553 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f0d501e13a0 0x7f0d50141930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:38:56.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:56.622+0000 7f0d55c00640 1 --2- 172.21.15.17:0/734102553 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0d380052f0 0x7f0d50148130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:38:56.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:56.622+0000 7f0d55200640 1 --2- 172.21.15.17:0/734102553 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0d501e02e0 0x7f0d501e0e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:38:56.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:56.622+0000 7f0d56600640 1 -- 172.21.15.17:0/734102553 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0d501e02e0 msgr2=0x7f0d501e0e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:38:56.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:56.622+0000 7f0d56600640 1 --2- 172.21.15.17:0/734102553 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0d501e02e0 0x7f0d501e0e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:38:56.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:56.622+0000 7f0d56600640 1 -- 172.21.15.17:0/734102553 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0d380052f0 msgr2=0x7f0d50148130 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:38:56.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:56.622+0000 7f0d56600640 1 --2- 172.21.15.17:0/734102553 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0d380052f0 0x7f0d50148130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:38:56.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:56.622+0000 7f0d56600640 1 -- 172.21.15.17:0/734102553 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d501e1820 con 0x7f0d501e13a0 2024-09-10T21:38:56.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:56.622+0000 7f0d56600640 1 --2- 172.21.15.17:0/734102553 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f0d501e13a0 0x7f0d50141930 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f0d500cc0a0 tx=0x7f0d480012b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:38:56.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:56.626+0000 7f0d4d600640 1 -- 172.21.15.17:0/734102553 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d480062e0 con 0x7f0d501e13a0 2024-09-10T21:38:56.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:56.626+0000 7f0d57000640 1 -- 172.21.15.17:0/734102553 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d50073930 con 0x7f0d501e13a0 2024-09-10T21:38:56.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:56.626+0000 7f0d57000640 1 -- 172.21.15.17:0/734102553 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d501f0e30 con 0x7f0d501e13a0 2024-09-10T21:38:56.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:56.626+0000 7f0d4d600640 1 -- 172.21.15.17:0/734102553 <== mon.2 v2:172.21.15.31:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0d480b5070 con 0x7f0d501e13a0 2024-09-10T21:38:56.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:56.626+0000 7f0d4d600640 1 -- 172.21.15.17:0/734102553 <== mon.2 v2:172.21.15.31:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d480066e0 con 0x7f0d501e13a0 2024-09-10T21:38:56.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:56.626+0000 7f0d57000640 1 -- 172.21.15.17:0/734102553 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d500b2c60 con 0x7f0d501e13a0 2024-09-10T21:38:56.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:56.626+0000 7f0d4d600640 1 -- 172.21.15.17:0/734102553 <== mon.2 v2:172.21.15.31:3301/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f0d480ba470 con 0x7f0d501e13a0 2024-09-10T21:38:56.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:56.630+0000 7f0d4d600640 1 --2- 172.21.15.17:0/734102553 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f0d300725e0 0x7f0d30074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:38:56.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:56.630+0000 7f0d4d600640 1 -- 172.21.15.17:0/734102553 <== mon.2 v2:172.21.15.31:3301/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f0d4812e5e0 con 0x7f0d501e13a0 2024-09-10T21:38:56.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:56.630+0000 7f0d55c00640 1 --2- 172.21.15.17:0/734102553 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f0d300725e0 0x7f0d30074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:38:56.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:56.630+0000 7f0d55c00640 1 --2- 172.21.15.17:0/734102553 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f0d300725e0 0x7f0d30074aa0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f0d44005ef0 tx=0x7f0d4400b2e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:38:56.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:56.630+0000 7f0d4d600640 1 -- 172.21.15.17:0/734102553 <== mon.2 v2:172.21.15.31:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f0d501e1820 con 0x7f0d501e13a0 2024-09-10T21:38:56.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:56.910+0000 7f0d57000640 1 -- 172.21.15.17:0/734102553 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f0d50071ce0 con 0x7f0d501e13a0 2024-09-10T21:38:56.923 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":112,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":32,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5e2f023f-d6a6-49f4-a256-f4807dce3739","modified":"2024-09-10T21:23:46.892130Z","created":"2024-09-10T21:23:46.892130Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3301","nonce":0},{"type":"v1","addr":"172.21.15.31:6790","nonce":0}]},"addr":"172.21.15.31:6790/0","public_addr":"172.21.15.31:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-10T21:38:56.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:56.910+0000 7f0d4d600640 1 -- 172.21.15.17:0/734102553 <== mon.2 v2:172.21.15.31:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1552 (secure 0 0 0) 0x7f0d48016850 con 0x7f0d501e13a0 2024-09-10T21:38:56.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:56.914+0000 7f0d57000640 1 -- 172.21.15.17:0/734102553 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f0d300725e0 msgr2=0x7f0d30074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:38:56.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:56.914+0000 7f0d57000640 1 --2- 172.21.15.17:0/734102553 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f0d300725e0 0x7f0d30074aa0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f0d44005ef0 tx=0x7f0d4400b2e0 comp rx=0 tx=0).stop 2024-09-10T21:38:56.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:56.914+0000 7f0d57000640 1 -- 172.21.15.17:0/734102553 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f0d501e13a0 msgr2=0x7f0d50141930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:38:56.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:56.914+0000 7f0d57000640 1 --2- 172.21.15.17:0/734102553 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f0d501e13a0 0x7f0d50141930 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f0d500cc0a0 tx=0x7f0d480012b0 comp rx=0 tx=0).stop 2024-09-10T21:38:56.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:56.914+0000 7f0d57000640 1 -- 172.21.15.17:0/734102553 shutdown_connections 2024-09-10T21:38:56.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:56.914+0000 7f0d57000640 1 --2- 172.21.15.17:0/734102553 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f0d300725e0 0x7f0d30074aa0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:38:56.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:56.914+0000 7f0d57000640 1 --2- 172.21.15.17:0/734102553 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f0d501e13a0 0x7f0d50141930 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:38:56.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:56.914+0000 7f0d57000640 1 --2- 172.21.15.17:0/734102553 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0d501e02e0 0x7f0d501e0e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:38:56.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:56.914+0000 7f0d57000640 1 --2- 172.21.15.17:0/734102553 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0d380052f0 0x7f0d50148130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:38:56.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:56.914+0000 7f0d57000640 1 -- 172.21.15.17:0/734102553 >> 172.21.15.17:0/734102553 conn(0x7f0d50070ca0 msgr2=0x7f0d500c8f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:38:56.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:56.914+0000 7f0d57000640 1 -- 172.21.15.17:0/734102553 shutdown_connections 2024-09-10T21:38:56.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:38:56.914+0000 7f0d57000640 1 -- 172.21.15.17:0/734102553 wait complete. 2024-09-10T21:38:59.938 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-10T21:39:00.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.138+0000 7f4a53200640 1 -- 172.21.15.17:0/455789706 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4a54151130 msgr2=0x7f4a540c8790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:39:00.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.138+0000 7f4a53200640 1 --2- 172.21.15.17:0/455789706 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4a54151130 0x7f4a540c8790 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f4a54157bb0 tx=0x7f4a4401c8c0 comp rx=0 tx=0).stop 2024-09-10T21:39:00.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.138+0000 7f4a53200640 1 -- 172.21.15.17:0/455789706 shutdown_connections 2024-09-10T21:39:00.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.138+0000 7f4a53200640 1 --2- 172.21.15.17:0/455789706 >> v2:172.21.15.31:3300/0 conn(0x7f4a54072b60 0x7f4a540c7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:39:00.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.138+0000 7f4a53200640 1 --2- 172.21.15.17:0/455789706 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4a54151130 0x7f4a540c8790 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:39:00.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.138+0000 7f4a53200640 1 -- 172.21.15.17:0/455789706 >> 172.21.15.17:0/455789706 conn(0x7f4a5406cdf0 msgr2=0x7f4a5406d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:39:00.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.138+0000 7f4a53200640 1 -- 172.21.15.17:0/455789706 shutdown_connections 2024-09-10T21:39:00.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.138+0000 7f4a53200640 1 -- 172.21.15.17:0/455789706 wait complete. 2024-09-10T21:39:00.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.142+0000 7f4a53200640 1 Processor -- start 2024-09-10T21:39:00.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.142+0000 7f4a53200640 1 -- start start 2024-09-10T21:39:00.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.142+0000 7f4a53200640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4a54068e00 0x7f4a54155020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:39:00.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.142+0000 7f4a53200640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4a54072b60 0x7f4a5414bb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:39:00.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.142+0000 7f4a53200640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f4a54151130 0x7f4a541f4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:39:00.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.142+0000 7f4a53200640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f4a54151690 con 0x7f4a54068e00 2024-09-10T21:39:00.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.142+0000 7f4a53200640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f4a54151510 con 0x7f4a54072b60 2024-09-10T21:39:00.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.142+0000 7f4a53200640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f4a540c98f0 con 0x7f4a54151130 2024-09-10T21:39:00.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.142+0000 7f4a51e00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4a54068e00 0x7f4a54155020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:39:00.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.142+0000 7f4a51e00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4a54068e00 0x7f4a54155020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49660/0 (socket says 172.21.15.17:49660) 2024-09-10T21:39:00.147 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.142+0000 7f4a51e00640 1 -- 172.21.15.17:0/1883950607 learned_addr learned my addr 172.21.15.17:0/1883950607 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:39:00.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.142+0000 7f4a51400640 1 --2- 172.21.15.17:0/1883950607 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4a54072b60 0x7f4a5414bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:39:00.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.142+0000 7f4a52800640 1 --2- 172.21.15.17:0/1883950607 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f4a54151130 0x7f4a541f4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:39:00.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.142+0000 7f4a51e00640 1 -- 172.21.15.17:0/1883950607 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f4a54151130 msgr2=0x7f4a541f4d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:39:00.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.142+0000 7f4a51e00640 1 --2- 172.21.15.17:0/1883950607 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f4a54151130 0x7f4a541f4d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:39:00.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.142+0000 7f4a51e00640 1 -- 172.21.15.17:0/1883950607 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4a54072b60 msgr2=0x7f4a5414bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:39:00.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.142+0000 7f4a51e00640 1 --2- 172.21.15.17:0/1883950607 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4a54072b60 0x7f4a5414bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:39:00.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.142+0000 7f4a51e00640 1 -- 172.21.15.17:0/1883950607 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a541568d0 con 0x7f4a54068e00 2024-09-10T21:39:00.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.142+0000 7f4a51e00640 1 --2- 172.21.15.17:0/1883950607 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4a54068e00 0x7f4a54155020 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f4a440b0040 tx=0x7f4a440a5690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:39:00.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.142+0000 7f4a4a000640 1 -- 172.21.15.17:0/1883950607 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a440b2040 con 0x7f4a54068e00 2024-09-10T21:39:00.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.142+0000 7f4a4a000640 1 -- 172.21.15.17:0/1883950607 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4a440042c0 con 0x7f4a54068e00 2024-09-10T21:39:00.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.142+0000 7f4a4a000640 1 -- 172.21.15.17:0/1883950607 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a44005290 con 0x7f4a54068e00 2024-09-10T21:39:00.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.142+0000 7f4a53200640 1 -- 172.21.15.17:0/1883950607 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a540713c0 con 0x7f4a54068e00 2024-09-10T21:39:00.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.142+0000 7f4a53200640 1 -- 172.21.15.17:0/1883950607 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a54156b70 con 0x7f4a54068e00 2024-09-10T21:39:00.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.146+0000 7f4a53200640 1 -- 172.21.15.17:0/1883950607 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a54069e60 con 0x7f4a54068e00 2024-09-10T21:39:00.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.146+0000 7f4a4a000640 1 -- 172.21.15.17:0/1883950607 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f4a44005430 con 0x7f4a54068e00 2024-09-10T21:39:00.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.150+0000 7f4a4a000640 1 --2- 172.21.15.17:0/1883950607 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f4a340725e0 0x7f4a34074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:39:00.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.150+0000 7f4a4a000640 1 -- 172.21.15.17:0/1883950607 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f4a4412d1e0 con 0x7f4a54068e00 2024-09-10T21:39:00.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.150+0000 7f4a51400640 1 --2- 172.21.15.17:0/1883950607 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f4a340725e0 0x7f4a34074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:39:00.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.150+0000 7f4a51400640 1 --2- 172.21.15.17:0/1883950607 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f4a340725e0 0x7f4a34074aa0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f4a541488e0 tx=0x7f4a40009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:39:00.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.150+0000 7f4a4a000640 1 -- 172.21.15.17:0/1883950607 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f4a440164b0 con 0x7f4a54068e00 2024-09-10T21:39:00.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.470+0000 7f4a53200640 1 -- 172.21.15.17:0/1883950607 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f4a540c3530 con 0x7f4a54068e00 2024-09-10T21:39:00.472 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.470+0000 7f4a4a000640 1 -- 172.21.15.17:0/1883950607 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1557 (secure 0 0 0) 0x7f4a440044b0 con 0x7f4a54068e00 2024-09-10T21:39:00.472 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":116,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":2,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5e2f023f-d6a6-49f4-a256-f4807dce3739","modified":"2024-09-10T21:23:46.892130Z","created":"2024-09-10T21:23:46.892130Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3301","nonce":0},{"type":"v1","addr":"172.21.15.31:6790","nonce":0}]},"addr":"172.21.15.31:6790/0","public_addr":"172.21.15.31:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-10T21:39:00.475 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.470+0000 7f4a53200640 1 -- 172.21.15.17:0/1883950607 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f4a340725e0 msgr2=0x7f4a34074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:39:00.475 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.470+0000 7f4a53200640 1 --2- 172.21.15.17:0/1883950607 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f4a340725e0 0x7f4a34074aa0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f4a541488e0 tx=0x7f4a40009040 comp rx=0 tx=0).stop 2024-09-10T21:39:00.475 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.470+0000 7f4a53200640 1 -- 172.21.15.17:0/1883950607 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4a54068e00 msgr2=0x7f4a54155020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:39:00.475 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.470+0000 7f4a53200640 1 --2- 172.21.15.17:0/1883950607 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4a54068e00 0x7f4a54155020 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f4a440b0040 tx=0x7f4a440a5690 comp rx=0 tx=0).stop 2024-09-10T21:39:00.475 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.470+0000 7f4a53200640 1 -- 172.21.15.17:0/1883950607 shutdown_connections 2024-09-10T21:39:00.475 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.470+0000 7f4a53200640 1 --2- 172.21.15.17:0/1883950607 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f4a340725e0 0x7f4a34074aa0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:39:00.475 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.470+0000 7f4a53200640 1 --2- 172.21.15.17:0/1883950607 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f4a54151130 0x7f4a541f4d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:39:00.475 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.470+0000 7f4a53200640 1 --2- 172.21.15.17:0/1883950607 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4a54072b60 0x7f4a5414bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:39:00.475 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.470+0000 7f4a53200640 1 --2- 172.21.15.17:0/1883950607 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4a54068e00 0x7f4a54155020 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:39:00.475 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.470+0000 7f4a53200640 1 -- 172.21.15.17:0/1883950607 >> 172.21.15.17:0/1883950607 conn(0x7f4a5406cdf0 msgr2=0x7f4a54072f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:39:00.475 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.470+0000 7f4a53200640 1 -- 172.21.15.17:0/1883950607 shutdown_connections 2024-09-10T21:39:00.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.470+0000 7f4a53200640 1 -- 172.21.15.17:0/1883950607 wait complete. 2024-09-10T21:39:00.499 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-09-10T21:39:00.499 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-09-10T21:39:00.500 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-10T21:39:00.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.662+0000 7f9d39200640 1 -- 172.21.15.17:0/767631693 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9d34072b00 msgr2=0x7f9d34074770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:39:00.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.662+0000 7f9d39200640 1 --2- 172.21.15.17:0/767631693 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9d34072b00 0x7f9d34074770 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f9d3406fc10 tx=0x7f9d2c01f580 comp rx=0 tx=0).stop 2024-09-10T21:39:00.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.666+0000 7f9d39200640 1 -- 172.21.15.17:0/767631693 shutdown_connections 2024-09-10T21:39:00.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.666+0000 7f9d39200640 1 --2- 172.21.15.17:0/767631693 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9d34072b00 0x7f9d34074770 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:39:00.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.666+0000 7f9d39200640 1 --2- 172.21.15.17:0/767631693 >> v2:172.21.15.31:3300/0 conn(0x7f9d340c7ea0 0x7f9d340725c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:39:00.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.666+0000 7f9d39200640 1 -- 172.21.15.17:0/767631693 >> 172.21.15.17:0/767631693 conn(0x7f9d340700d0 msgr2=0x7f9d340704e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:39:00.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.666+0000 7f9d39200640 1 -- 172.21.15.17:0/767631693 shutdown_connections 2024-09-10T21:39:00.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.666+0000 7f9d39200640 1 -- 172.21.15.17:0/767631693 wait complete. 2024-09-10T21:39:00.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.666+0000 7f9d39200640 1 Processor -- start 2024-09-10T21:39:00.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.666+0000 7f9d39200640 1 -- start start 2024-09-10T21:39:00.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.666+0000 7f9d39200640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9d34072b00 0x7f9d34157e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:39:00.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.666+0000 7f9d39200640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f9d340c7ea0 0x7f9d340c7120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:39:00.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.666+0000 7f9d39200640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9d34151530 0x7f9d3414a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:39:00.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.666+0000 7f9d39200640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f9d340c1670 con 0x7f9d34151530 2024-09-10T21:39:00.671 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.666+0000 7f9d39200640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f9d340c9310 con 0x7f9d34072b00 2024-09-10T21:39:00.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.666+0000 7f9d39200640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f9d340c0fd0 con 0x7f9d340c7ea0 2024-09-10T21:39:00.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.666+0000 7f9d32800640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9d34151530 0x7f9d3414a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:39:00.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.666+0000 7f9d32800640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9d34151530 0x7f9d3414a970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49666/0 (socket says 172.21.15.17:49666) 2024-09-10T21:39:00.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.666+0000 7f9d32800640 1 -- 172.21.15.17:0/4114441099 learned_addr learned my addr 172.21.15.17:0/4114441099 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:39:00.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.666+0000 7f9d31e00640 1 --2- 172.21.15.17:0/4114441099 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9d34072b00 0x7f9d34157e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:39:00.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.666+0000 7f9d31400640 1 --2- 172.21.15.17:0/4114441099 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f9d340c7ea0 0x7f9d340c7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:39:00.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.666+0000 7f9d32800640 1 -- 172.21.15.17:0/4114441099 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f9d340c7ea0 msgr2=0x7f9d340c7120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:39:00.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.666+0000 7f9d32800640 1 --2- 172.21.15.17:0/4114441099 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f9d340c7ea0 0x7f9d340c7120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:39:00.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.666+0000 7f9d32800640 1 -- 172.21.15.17:0/4114441099 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9d34072b00 msgr2=0x7f9d34157e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:39:00.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.666+0000 7f9d32800640 1 --2- 172.21.15.17:0/4114441099 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9d34072b00 0x7f9d34157e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:39:00.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.666+0000 7f9d32800640 1 -- 172.21.15.17:0/4114441099 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d3414cbd0 con 0x7f9d34151530 2024-09-10T21:39:00.672 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.666+0000 7f9d31400640 1 --2- 172.21.15.17:0/4114441099 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f9d340c7ea0 0x7f9d340c7120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:39:00.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.666+0000 7f9d32800640 1 --2- 172.21.15.17:0/4114441099 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9d34151530 0x7f9d3414a970 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f9d2c01c190 tx=0x7f9d2c009300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:39:00.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.670+0000 7f9d2a000640 1 -- 172.21.15.17:0/4114441099 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d2c004e60 con 0x7f9d34151530 2024-09-10T21:39:00.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.670+0000 7f9d2a000640 1 -- 172.21.15.17:0/4114441099 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9d2c0be030 con 0x7f9d34151530 2024-09-10T21:39:00.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.670+0000 7f9d2a000640 1 -- 172.21.15.17:0/4114441099 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d2c0afac0 con 0x7f9d34151530 2024-09-10T21:39:00.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.670+0000 7f9d39200640 1 -- 172.21.15.17:0/4114441099 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d3414ceb0 con 0x7f9d34151530 2024-09-10T21:39:00.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.670+0000 7f9d39200640 1 -- 172.21.15.17:0/4114441099 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d3414d470 con 0x7f9d34151530 2024-09-10T21:39:00.674 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.670+0000 7f9d39200640 1 --2- 172.21.15.17:0/4114441099 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9d340c4400 0x7f9d3414aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:39:00.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.670+0000 7f9d39200640 1 -- 172.21.15.17:0/4114441099 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f9d340cbdc0 con 0x7f9d340c4400 2024-09-10T21:39:00.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.670+0000 7f9d31e00640 1 --2- 172.21.15.17:0/4114441099 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9d340c4400 0x7f9d3414aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:39:00.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.670+0000 7f9d31e00640 1 -- 172.21.15.17:0/4114441099 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9d340725c0 con 0x7f9d340c4400 2024-09-10T21:39:00.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.670+0000 7f9d31e00640 1 --2- 172.21.15.17:0/4114441099 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9d340c4400 0x7f9d3414aeb0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f9d2400e3c0 tx=0x7f9d2400e880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:39:00.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.670+0000 7f9d2a000640 1 -- 172.21.15.17:0/4114441099 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d24019070 con 0x7f9d340c4400 2024-09-10T21:39:00.675 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.670+0000 7f9d2a000640 1 -- 172.21.15.17:0/4114441099 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f9d2c0afc60 con 0x7f9d34151530 2024-09-10T21:39:00.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.670+0000 7f9d2a000640 1 --2- 172.21.15.17:0/4114441099 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f9d140724e0 0x7f9d140749a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:39:00.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.670+0000 7f9d2a000640 1 -- 172.21.15.17:0/4114441099 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f9d2c131f60 con 0x7f9d34151530 2024-09-10T21:39:00.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.670+0000 7f9d2a000640 1 -- 172.21.15.17:0/4114441099 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f9d240052e0 con 0x7f9d340c4400 2024-09-10T21:39:00.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.674+0000 7f9d2a000640 1 -- 172.21.15.17:0/4114441099 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9d340c4400 msgr2=0x7f9d3414aeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:39:00.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.674+0000 7f9d2a000640 1 --2- 172.21.15.17:0/4114441099 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9d340c4400 0x7f9d3414aeb0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f9d2400e3c0 tx=0x7f9d2400e880 comp rx=0 tx=0).stop 2024-09-10T21:39:00.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.674+0000 7f9d2a000640 1 -- 172.21.15.17:0/4114441099 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9d340c4400 msgr2=0x7f9d3414aeb0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:39:00.676 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.674+0000 7f9d2a000640 1 --2- 172.21.15.17:0/4114441099 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9d340c4400 0x7f9d3414aeb0 secure :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f9d2400e3c0 tx=0x7f9d2400e880 comp rx=0 tx=0).stop 2024-09-10T21:39:00.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.674+0000 7f9d31400640 1 --2- 172.21.15.17:0/4114441099 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f9d140724e0 0x7f9d140749a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:39:00.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.674+0000 7f9d31400640 1 --2- 172.21.15.17:0/4114441099 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f9d140724e0 0x7f9d140749a0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f9d341e2a20 tx=0x7f9d200091e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:39:00.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.678+0000 7f9d39200640 1 --2- 172.21.15.17:0/4114441099 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9d340c3e30 0x7f9d341ee180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:39:00.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.678+0000 7f9d39200640 1 -- 172.21.15.17:0/4114441099 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f9d340b1580 con 0x7f9d340c3e30 2024-09-10T21:39:00.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.678+0000 7f9d31e00640 1 --2- 172.21.15.17:0/4114441099 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9d340c3e30 0x7f9d341ee180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:39:00.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.678+0000 7f9d31e00640 1 -- 172.21.15.17:0/4114441099 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f9d340c2950 con 0x7f9d340c3e30 2024-09-10T21:39:00.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.678+0000 7f9d31e00640 1 --2- 172.21.15.17:0/4114441099 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9d340c3e30 0x7f9d341ee180 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f9d24016000 tx=0x7f9d24000ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:39:00.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.678+0000 7f9d2a000640 1 -- 172.21.15.17:0/4114441099 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d2401ce00 con 0x7f9d340c3e30 2024-09-10T21:39:00.683 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.678+0000 7f9d2a000640 1 -- 172.21.15.17:0/4114441099 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4543 (secure 0 0 0) 0x7f9d24003600 con 0x7f9d340c3e30 2024-09-10T21:39:00.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.678+0000 7f9d2a000640 1 -- 172.21.15.17:0/4114441099 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9d340c3e30 msgr2=0x7f9d341ee180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:39:00.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.678+0000 7f9d2a000640 1 --2- 172.21.15.17:0/4114441099 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9d340c3e30 0x7f9d341ee180 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f9d24016000 tx=0x7f9d24000ee0 comp rx=0 tx=0).stop 2024-09-10T21:39:00.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.678+0000 7f9d2a000640 1 -- 172.21.15.17:0/4114441099 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9d340c3e30 msgr2=0x7f9d341ee180 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:39:00.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.678+0000 7f9d2a000640 1 --2- 172.21.15.17:0/4114441099 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9d340c3e30 0x7f9d341ee180 secure :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f9d24016000 tx=0x7f9d24000ee0 comp rx=0 tx=0).stop 2024-09-10T21:39:00.684 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-10T21:39:00.684 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:39:00.684 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:39:00.684 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "leader", 2024-09-10T21:39:00.684 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 116, 2024-09-10T21:39:00.684 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 14879, 2024-09-10T21:39:00.685 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-10T21:39:00.685 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-10T21:39:00.685 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-10T21:39:00.685 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-10T21:39:00.685 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:39:00.685 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 2, 2024-09-10T21:39:00.685 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:39:00.685 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-10T21:39:00.685 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-10T21:39:00.685 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:39:00.685 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:39:00.686 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:39:00.686 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:39:00.686 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:39:00.686 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:39:00.686 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:39:00.686 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:39:00.686 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:39:00.686 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:39:00.686 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:39:00.686 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:39:00.686 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-10T21:39:00.686 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-10T21:39:00.687 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:39:00.687 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:39:00.687 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:39:00.687 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:39:00.687 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:39:00.687 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:39:00.687 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:39:00.687 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:39:00.687 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:39:00.687 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:39:00.687 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:39:00.687 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:39:00.688 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:39:00.688 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-10T21:39:00.688 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-10T21:39:00.688 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-10T21:39:00.688 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-10T21:39:00.688 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-10T21:39:00.688 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "5e2f023f-d6a6-49f4-a256-f4807dce3739", 2024-09-10T21:39:00.688 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:39:00.688 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:39:00.689 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-10T21:39:00.689 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-10T21:39:00.689 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-10T21:39:00.689 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-10T21:39:00.689 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-10T21:39:00.689 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-10T21:39:00.689 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-10T21:39:00.689 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:39:00.689 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-10T21:39:00.689 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:39:00.689 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:39:00.689 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:39:00.689 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:39:00.690 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:39:00.690 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:39:00.690 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:39:00.690 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:39:00.690 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:39:00.690 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:39:00.690 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:39:00.690 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:39:00.690 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-10T21:39:00.691 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:39:00.691 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-10T21:39:00.691 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:39:00.691 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:39:00.691 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:39:00.691 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:39:00.691 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:39:00.691 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:39:00.691 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:39:00.691 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-10T21:39:00.691 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:39:00.691 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:39:00.691 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:39:00.692 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:39:00.692 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-10T21:39:00.692 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:39:00.692 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:39:00.692 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:39:00.692 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:39:00.692 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-10T21:39:00.692 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-10T21:39:00.692 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:39:00.692 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:39:00.692 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:39:00.693 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:39:00.693 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:39:00.693 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:39:00.693 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:39:00.693 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:39:00.693 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:39:00.693 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:39:00.693 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:39:00.693 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3300", 2024-09-10T21:39:00.693 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:39:00.693 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:39:00.693 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:39:00.693 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:39:00.694 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789", 2024-09-10T21:39:00.694 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:39:00.694 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:39:00.694 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:39:00.694 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:39:00.694 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789/0", 2024-09-10T21:39:00.694 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6789/0", 2024-09-10T21:39:00.694 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:39:00.694 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:39:00.694 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:39:00.694 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:39:00.694 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:39:00.695 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:39:00.695 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:39:00.695 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:39:00.695 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:39:00.695 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:39:00.695 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:39:00.695 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3301", 2024-09-10T21:39:00.695 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:39:00.695 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:39:00.695 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:39:00.695 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:39:00.695 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790", 2024-09-10T21:39:00.695 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:39:00.696 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:39:00.696 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:39:00.696 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:39:00.696 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790/0", 2024-09-10T21:39:00.696 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6790/0", 2024-09-10T21:39:00.696 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:39:00.696 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:39:00.696 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:39:00.696 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:39:00.696 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:39:00.696 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:39:00.696 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-10T21:39:00.696 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-10T21:39:00.697 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:39:00.697 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:39:00.697 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:39:00.697 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:39:00.697 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:39:00.697 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:39:00.697 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-10T21:39:00.697 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:39:00.697 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:39:00.697 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:39:00.697 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 2 2024-09-10T21:39:00.698 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:39:00.698 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:39:00.698 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:39:00.698 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-10T21:39:00.698 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-10T21:39:00.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.678+0000 7f9d39200640 1 -- 172.21.15.17:0/4114441099 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f9d140724e0 msgr2=0x7f9d140749a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:39:00.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.678+0000 7f9d39200640 1 --2- 172.21.15.17:0/4114441099 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f9d140724e0 0x7f9d140749a0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f9d341e2a20 tx=0x7f9d200091e0 comp rx=0 tx=0).stop 2024-09-10T21:39:00.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.678+0000 7f9d39200640 1 -- 172.21.15.17:0/4114441099 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9d34151530 msgr2=0x7f9d3414a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:39:00.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.678+0000 7f9d39200640 1 --2- 172.21.15.17:0/4114441099 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9d34151530 0x7f9d3414a970 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f9d2c01c190 tx=0x7f9d2c009300 comp rx=0 tx=0).stop 2024-09-10T21:39:00.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.682+0000 7f9d32800640 1 -- 172.21.15.17:0/4114441099 reap_dead start 2024-09-10T21:39:00.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.682+0000 7f9d32800640 1 -- 172.21.15.17:0/4114441099 reap_dead start 2024-09-10T21:39:00.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.682+0000 7f9d39200640 1 -- 172.21.15.17:0/4114441099 shutdown_connections 2024-09-10T21:39:00.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.682+0000 7f9d39200640 1 -- 172.21.15.17:0/4114441099 >> 172.21.15.17:0/4114441099 conn(0x7f9d340700d0 msgr2=0x7f9d3406dd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:39:00.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.682+0000 7f9d39200640 1 -- 172.21.15.17:0/4114441099 shutdown_connections 2024-09-10T21:39:00.699 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.682+0000 7f9d39200640 1 -- 172.21.15.17:0/4114441099 wait complete. 2024-09-10T21:39:00.706 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-10T21:39:00.706 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-09-10T21:39:00.706 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-10T21:39:00.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.882+0000 7f8469800640 1 -- 172.21.15.17:0/385395701 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f84640c9710 msgr2=0x7f8464150db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:39:00.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.882+0000 7f8469800640 1 --2- 172.21.15.17:0/385395701 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f84640c9710 0x7f8464150db0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f84640716f0 tx=0x7f845c01cd50 comp rx=0 tx=0).stop 2024-09-10T21:39:00.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.882+0000 7f8469800640 1 -- 172.21.15.17:0/385395701 shutdown_connections 2024-09-10T21:39:00.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.882+0000 7f8469800640 1 --2- 172.21.15.17:0/385395701 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f84640c9710 0x7f8464150db0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:39:00.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.882+0000 7f8469800640 1 --2- 172.21.15.17:0/385395701 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f84640c7cc0 0x7f8464150870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:39:00.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.882+0000 7f8469800640 1 --2- 172.21.15.17:0/385395701 >> v2:172.21.15.31:3300/0 conn(0x7f84640c9330 0x7f84640c7780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:39:00.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.882+0000 7f8469800640 1 -- 172.21.15.17:0/385395701 >> 172.21.15.17:0/385395701 conn(0x7f846406cc20 msgr2=0x7f846406d030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:39:00.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.882+0000 7f8469800640 1 -- 172.21.15.17:0/385395701 shutdown_connections 2024-09-10T21:39:00.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.886+0000 7f8469800640 1 -- 172.21.15.17:0/385395701 wait complete. 2024-09-10T21:39:00.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.886+0000 7f8469800640 1 Processor -- start 2024-09-10T21:39:00.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.886+0000 7f8469800640 1 -- start start 2024-09-10T21:39:00.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.886+0000 7f8469800640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f84640c7cc0 0x7f84641508a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:39:00.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.886+0000 7f8469800640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f84640c9330 0x7f84641f7020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:39:00.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.886+0000 7f8469800640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f84640c9710 0x7f84641f7560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:39:00.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.886+0000 7f8469800640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f8464070a80 con 0x7f84640c7cc0 2024-09-10T21:39:00.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.886+0000 7f8469800640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f8464070900 con 0x7f84640c9710 2024-09-10T21:39:00.891 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.886+0000 7f8469800640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f8464070c00 con 0x7f84640c9330 2024-09-10T21:39:00.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.886+0000 7f8462a00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f84640c7cc0 0x7f84641508a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:39:00.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.886+0000 7f8462a00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f84640c7cc0 0x7f84641508a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49696/0 (socket says 172.21.15.17:49696) 2024-09-10T21:39:00.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.886+0000 7f8462a00640 1 -- 172.21.15.17:0/2824961154 learned_addr learned my addr 172.21.15.17:0/2824961154 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:39:00.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.886+0000 7f8462000640 1 --2- 172.21.15.17:0/2824961154 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f84640c9330 0x7f84641f7020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:39:00.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.886+0000 7f8463400640 1 --2- 172.21.15.17:0/2824961154 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f84640c9710 0x7f84641f7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:39:00.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.886+0000 7f8462a00640 1 -- 172.21.15.17:0/2824961154 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f84640c9330 msgr2=0x7f84641f7020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:39:00.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.886+0000 7f8462a00640 1 --2- 172.21.15.17:0/2824961154 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f84640c9330 0x7f84641f7020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:39:00.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.886+0000 7f8462a00640 1 -- 172.21.15.17:0/2824961154 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f84640c9710 msgr2=0x7f84641f7560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:39:00.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.886+0000 7f8462a00640 1 --2- 172.21.15.17:0/2824961154 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f84640c9710 0x7f84641f7560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:39:00.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.886+0000 7f8462a00640 1 -- 172.21.15.17:0/2824961154 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8464073f90 con 0x7f84640c7cc0 2024-09-10T21:39:00.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.890+0000 7f8462a00640 1 --2- 172.21.15.17:0/2824961154 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f84640c7cc0 0x7f84641508a0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f845400d950 tx=0x7f845400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:39:00.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.890+0000 7f845aa00640 1 -- 172.21.15.17:0/2824961154 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8454017070 con 0x7f84640c7cc0 2024-09-10T21:39:00.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.890+0000 7f845aa00640 1 -- 172.21.15.17:0/2824961154 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8454009d20 con 0x7f84640c7cc0 2024-09-10T21:39:00.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.890+0000 7f845aa00640 1 -- 172.21.15.17:0/2824961154 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84540106c0 con 0x7f84640c7cc0 2024-09-10T21:39:00.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.890+0000 7f8469800640 1 -- 172.21.15.17:0/2824961154 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8464073cd0 con 0x7f84640c7cc0 2024-09-10T21:39:00.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.890+0000 7f8469800640 1 -- 172.21.15.17:0/2824961154 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84640739f0 con 0x7f84640c7cc0 2024-09-10T21:39:00.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.890+0000 7f8469800640 1 --2- 172.21.15.17:0/2824961154 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8464074b50 0x7f8464140e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:39:00.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.890+0000 7f8469800640 1 -- 172.21.15.17:0/2824961154 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f8464074230 con 0x7f8464074b50 2024-09-10T21:39:00.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.890+0000 7f8462000640 1 --2- 172.21.15.17:0/2824961154 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8464074b50 0x7f8464140e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:39:00.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.890+0000 7f8462000640 1 -- 172.21.15.17:0/2824961154 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8464071160 con 0x7f8464074b50 2024-09-10T21:39:00.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.890+0000 7f845aa00640 1 -- 172.21.15.17:0/2824961154 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f8454010860 con 0x7f84640c7cc0 2024-09-10T21:39:00.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.890+0000 7f8462000640 1 --2- 172.21.15.17:0/2824961154 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8464074b50 0x7f8464140e70 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f845000ed60 tx=0x7f845000d810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:39:00.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.890+0000 7f845aa00640 1 --2- 172.21.15.17:0/2824961154 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f84440725e0 0x7f8444074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:39:00.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.890+0000 7f845aa00640 1 -- 172.21.15.17:0/2824961154 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f8454093b20 con 0x7f84640c7cc0 2024-09-10T21:39:00.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.890+0000 7f845aa00640 1 -- 172.21.15.17:0/2824961154 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8450004030 con 0x7f8464074b50 2024-09-10T21:39:00.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.890+0000 7f845aa00640 1 -- 172.21.15.17:0/2824961154 <== mon.1 v2:172.21.15.31:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f84500041d0 con 0x7f8464074b50 2024-09-10T21:39:00.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.890+0000 7f845aa00640 1 -- 172.21.15.17:0/2824961154 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8464074b50 msgr2=0x7f8464140e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:39:00.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.890+0000 7f845aa00640 1 --2- 172.21.15.17:0/2824961154 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8464074b50 0x7f8464140e70 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f845000ed60 tx=0x7f845000d810 comp rx=0 tx=0).stop 2024-09-10T21:39:00.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.890+0000 7f845aa00640 1 -- 172.21.15.17:0/2824961154 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8464074b50 msgr2=0x7f8464140e70 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:39:00.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.890+0000 7f845aa00640 1 --2- 172.21.15.17:0/2824961154 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8464074b50 0x7f8464140e70 secure :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f845000ed60 tx=0x7f845000d810 comp rx=0 tx=0).stop 2024-09-10T21:39:00.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.894+0000 7f8463400640 1 --2- 172.21.15.17:0/2824961154 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f84440725e0 0x7f8444074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:39:00.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.894+0000 7f8463400640 1 --2- 172.21.15.17:0/2824961154 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f84440725e0 0x7f8444074aa0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f845c0af000 tx=0x7f845c01b590 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:39:00.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.902+0000 7f8469800640 1 --2- 172.21.15.17:0/2824961154 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f84641e8ae0 0x7f84641413b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:39:00.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.902+0000 7f8469800640 1 -- 172.21.15.17:0/2824961154 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f8464145350 con 0x7f84641e8ae0 2024-09-10T21:39:00.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.902+0000 7f8462000640 1 --2- 172.21.15.17:0/2824961154 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f84641e8ae0 0x7f84641413b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:39:00.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.902+0000 7f8462000640 1 -- 172.21.15.17:0/2824961154 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f8464145070 con 0x7f84641e8ae0 2024-09-10T21:39:00.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.902+0000 7f8462000640 1 --2- 172.21.15.17:0/2824961154 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f84641e8ae0 0x7f84641413b0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f845000ed60 tx=0x7f84500046d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:39:00.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.902+0000 7f845aa00640 1 -- 172.21.15.17:0/2824961154 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8450004030 con 0x7f84641e8ae0 2024-09-10T21:39:00.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.902+0000 7f845aa00640 1 -- 172.21.15.17:0/2824961154 <== mon.1 v2:172.21.15.31:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5043 (secure 0 0 0) 0x7f845000c5e0 con 0x7f84641e8ae0 2024-09-10T21:39:00.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.902+0000 7f845aa00640 1 -- 172.21.15.17:0/2824961154 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f84641e8ae0 msgr2=0x7f84641413b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:39:00.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.902+0000 7f845aa00640 1 --2- 172.21.15.17:0/2824961154 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f84641e8ae0 0x7f84641413b0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f845000ed60 tx=0x7f84500046d0 comp rx=0 tx=0).stop 2024-09-10T21:39:00.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.902+0000 7f845aa00640 1 -- 172.21.15.17:0/2824961154 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f84641e8ae0 msgr2=0x7f84641413b0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:39:00.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.902+0000 7f845aa00640 1 --2- 172.21.15.17:0/2824961154 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f84641e8ae0 0x7f84641413b0 secure :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f845000ed60 tx=0x7f84500046d0 comp rx=0 tx=0).stop 2024-09-10T21:39:00.906 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-10T21:39:00.906 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:39:00.907 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:39:00.907 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-10T21:39:00.907 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 116, 2024-09-10T21:39:00.907 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 260287, 2024-09-10T21:39:00.907 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-10T21:39:00.907 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-10T21:39:00.907 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-10T21:39:00.907 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-10T21:39:00.907 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:39:00.907 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 2, 2024-09-10T21:39:00.907 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:39:00.907 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-10T21:39:00.908 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-10T21:39:00.908 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:39:00.908 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:39:00.908 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:39:00.908 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:39:00.908 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:39:00.908 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:39:00.908 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:39:00.908 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:39:00.908 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:39:00.908 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:39:00.908 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:39:00.909 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:39:00.909 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-10T21:39:00.909 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-10T21:39:00.909 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:39:00.909 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:39:00.909 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:39:00.909 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:39:00.909 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:39:00.909 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:39:00.909 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:39:00.909 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:39:00.909 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:39:00.909 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:39:00.910 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:39:00.910 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:39:00.910 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:39:00.910 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-10T21:39:00.910 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-10T21:39:00.910 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-10T21:39:00.910 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-10T21:39:00.910 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-10T21:39:00.910 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "5e2f023f-d6a6-49f4-a256-f4807dce3739", 2024-09-10T21:39:00.910 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:39:00.910 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:39:00.910 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-10T21:39:00.910 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-10T21:39:00.910 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-10T21:39:00.911 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-10T21:39:00.911 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-10T21:39:00.911 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-10T21:39:00.911 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-10T21:39:00.911 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:39:00.911 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-10T21:39:00.911 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:39:00.911 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:39:00.911 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:39:00.911 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:39:00.911 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:39:00.912 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:39:00.912 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:39:00.912 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:39:00.912 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:39:00.912 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:39:00.912 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:39:00.912 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:39:00.912 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-10T21:39:00.912 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:39:00.912 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-10T21:39:00.912 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:39:00.912 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:39:00.912 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:39:00.913 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:39:00.913 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:39:00.913 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:39:00.913 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:39:00.913 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-10T21:39:00.913 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:39:00.913 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:39:00.913 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:39:00.913 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:39:00.913 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-10T21:39:00.913 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:39:00.913 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:39:00.913 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:39:00.914 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:39:00.914 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-10T21:39:00.914 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-10T21:39:00.914 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:39:00.914 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:39:00.914 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:39:00.914 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:39:00.914 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:39:00.914 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:39:00.914 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:39:00.914 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:39:00.914 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:39:00.914 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:39:00.915 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:39:00.915 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3300", 2024-09-10T21:39:00.915 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:39:00.915 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:39:00.915 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:39:00.915 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:39:00.915 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789", 2024-09-10T21:39:00.915 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:39:00.915 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:39:00.915 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:39:00.915 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:39:00.915 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789/0", 2024-09-10T21:39:00.915 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6789/0", 2024-09-10T21:39:00.916 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:39:00.916 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:39:00.916 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:39:00.916 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:39:00.916 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:39:00.916 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:39:00.916 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:39:00.916 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:39:00.916 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:39:00.916 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:39:00.917 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:39:00.917 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3301", 2024-09-10T21:39:00.917 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:39:00.917 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:39:00.917 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:39:00.917 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:39:00.917 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790", 2024-09-10T21:39:00.917 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:39:00.917 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:39:00.917 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:39:00.917 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:39:00.917 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790/0", 2024-09-10T21:39:00.917 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6790/0", 2024-09-10T21:39:00.918 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:39:00.918 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:39:00.918 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:39:00.918 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:39:00.918 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:39:00.918 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:39:00.918 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-10T21:39:00.918 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-10T21:39:00.918 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:39:00.918 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:39:00.918 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:39:00.918 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:39:00.918 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:39:00.918 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:39:00.919 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": [ 2024-09-10T21:39:00.919 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:39:00.919 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:39:00.919 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:39:00.919 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 2 2024-09-10T21:39:00.919 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:39:00.919 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:39:00.919 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-10T21:39:00.919 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:39:00.919 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:39:00.919 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:39:00.919 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 4 2024-09-10T21:39:00.919 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:39:00.919 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:39:00.920 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-10T21:39:00.920 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:39:00.920 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:39:00.920 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:39:00.920 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 4 2024-09-10T21:39:00.921 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:39:00.921 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:39:00.921 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": [ 2024-09-10T21:39:00.921 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:39:00.921 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:39:00.921 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:39:00.922 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:39:00.922 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:39:00.922 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:39:00.922 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:39:00.922 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-10T21:39:00.922 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-10T21:39:00.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.902+0000 7f8469800640 1 -- 172.21.15.17:0/2824961154 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f84440725e0 msgr2=0x7f8444074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:39:00.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.902+0000 7f8469800640 1 --2- 172.21.15.17:0/2824961154 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f84440725e0 0x7f8444074aa0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f845c0af000 tx=0x7f845c01b590 comp rx=0 tx=0).stop 2024-09-10T21:39:00.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.902+0000 7f8469800640 1 -- 172.21.15.17:0/2824961154 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f84640c7cc0 msgr2=0x7f84641508a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:39:00.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.902+0000 7f8469800640 1 --2- 172.21.15.17:0/2824961154 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f84640c7cc0 0x7f84641508a0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f845400d950 tx=0x7f845400de10 comp rx=0 tx=0).stop 2024-09-10T21:39:00.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.902+0000 7f8463400640 1 -- 172.21.15.17:0/2824961154 reap_dead start 2024-09-10T21:39:00.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.902+0000 7f8463400640 1 -- 172.21.15.17:0/2824961154 reap_dead start 2024-09-10T21:39:00.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.902+0000 7f8469800640 1 -- 172.21.15.17:0/2824961154 shutdown_connections 2024-09-10T21:39:00.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.902+0000 7f8469800640 1 -- 172.21.15.17:0/2824961154 >> 172.21.15.17:0/2824961154 conn(0x7f846406cc20 msgr2=0x7f8464155320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:39:00.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.902+0000 7f8469800640 1 -- 172.21.15.17:0/2824961154 shutdown_connections 2024-09-10T21:39:00.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:00.902+0000 7f8469800640 1 -- 172.21.15.17:0/2824961154 wait complete. 2024-09-10T21:39:00.928 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-10T21:39:00.928 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-09-10T21:39:00.928 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-10T21:39:01.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:01.082+0000 7fa9c9400640 1 -- 172.21.15.17:0/2753878824 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa9c40c5ec0 msgr2=0x7fa9c40c62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:39:01.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:01.082+0000 7fa9c9400640 1 --2- 172.21.15.17:0/2753878824 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa9c40c5ec0 0x7fa9c40c62f0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fa9c415baf0 tx=0x7fa9bc01f560 comp rx=0 tx=0).stop 2024-09-10T21:39:01.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:01.082+0000 7fa9c9400640 1 -- 172.21.15.17:0/2753878824 shutdown_connections 2024-09-10T21:39:01.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:01.082+0000 7fa9c9400640 1 --2- 172.21.15.17:0/2753878824 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa9c40c5ec0 0x7fa9c40c62f0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:39:01.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:01.082+0000 7fa9c9400640 1 --2- 172.21.15.17:0/2753878824 >> v2:172.21.15.31:3300/0 conn(0x7fa9c4070450 0x7fa9c4157470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:39:01.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:01.082+0000 7fa9c9400640 1 -- 172.21.15.17:0/2753878824 >> 172.21.15.17:0/2753878824 conn(0x7fa9c406dd30 msgr2=0x7fa9c406e140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:39:01.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:01.082+0000 7fa9c9400640 1 -- 172.21.15.17:0/2753878824 shutdown_connections 2024-09-10T21:39:01.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:01.082+0000 7fa9c9400640 1 -- 172.21.15.17:0/2753878824 wait complete. 2024-09-10T21:39:01.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:01.086+0000 7fa9c9400640 1 Processor -- start 2024-09-10T21:39:01.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:01.086+0000 7fa9c9400640 1 -- start start 2024-09-10T21:39:01.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:01.086+0000 7fa9c9400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9c4070450 0x7fa9c41eff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:39:01.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:01.086+0000 7fa9c9400640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa9c40c5ec0 0x7fa9c41578a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:39:01.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:01.086+0000 7fa9c9400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa9c40c7210 0x7fa9c40b1e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:39:01.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:01.086+0000 7fa9c9400640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fa9c415b3b0 con 0x7fa9c40c7210 2024-09-10T21:39:01.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:01.086+0000 7fa9c9400640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fa9c415b0d0 con 0x7fa9c4070450 2024-09-10T21:39:01.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:01.086+0000 7fa9c9400640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7fa9c415b690 con 0x7fa9c40c5ec0 2024-09-10T21:39:01.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:01.086+0000 7fa9c3200640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa9c40c7210 0x7fa9c40b1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:39:01.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:01.086+0000 7fa9c3200640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa9c40c7210 0x7fa9c40b1e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49714/0 (socket says 172.21.15.17:49714) 2024-09-10T21:39:01.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:01.086+0000 7fa9c3200640 1 -- 172.21.15.17:0/1252817395 learned_addr learned my addr 172.21.15.17:0/1252817395 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:39:01.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:01.086+0000 7fa9c2800640 1 --2- 172.21.15.17:0/1252817395 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9c4070450 0x7fa9c41eff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:39:01.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:01.086+0000 7fa9c1e00640 1 --2- 172.21.15.17:0/1252817395 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa9c40c5ec0 0x7fa9c41578a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:39:01.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:01.086+0000 7fa9c3200640 1 -- 172.21.15.17:0/1252817395 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa9c40c5ec0 msgr2=0x7fa9c41578a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:39:01.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:01.086+0000 7fa9c3200640 1 --2- 172.21.15.17:0/1252817395 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa9c40c5ec0 0x7fa9c41578a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:39:01.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:01.086+0000 7fa9c3200640 1 -- 172.21.15.17:0/1252817395 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9c4070450 msgr2=0x7fa9c41eff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:39:01.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:01.086+0000 7fa9c3200640 1 --2- 172.21.15.17:0/1252817395 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa9c4070450 0x7fa9c41eff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:39:01.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:01.086+0000 7fa9c3200640 1 -- 172.21.15.17:0/1252817395 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9c4156540 con 0x7fa9c40c7210 2024-09-10T21:39:01.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:01.086+0000 7fa9c1e00640 1 --2- 172.21.15.17:0/1252817395 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa9c40c5ec0 0x7fa9c41578a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:39:01.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:01.086+0000 7fa9c3200640 1 --2- 172.21.15.17:0/1252817395 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa9c40c7210 0x7fa9c40b1e00 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fa9bc009bd0 tx=0x7fa9bc009320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:39:01.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:01.086+0000 7fa9baa00640 1 -- 172.21.15.17:0/1252817395 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9bc0b5070 con 0x7fa9c40c7210 2024-09-10T21:39:01.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:01.086+0000 7fa9baa00640 1 -- 172.21.15.17:0/1252817395 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa9bc0b6040 con 0x7fa9c40c7210 2024-09-10T21:39:01.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:01.086+0000 7fa9baa00640 1 -- 172.21.15.17:0/1252817395 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9bc0b0b30 con 0x7fa9c40c7210 2024-09-10T21:39:01.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:01.086+0000 7fa9c9400640 1 -- 172.21.15.17:0/1252817395 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9c4152080 con 0x7fa9c40c7210 2024-09-10T21:39:01.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:01.086+0000 7fa9c9400640 1 -- 172.21.15.17:0/1252817395 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9c4156ae0 con 0x7fa9c40c7210 2024-09-10T21:39:01.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:01.090+0000 7fa9c9400640 1 --2- 172.21.15.17:0/1252817395 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa9c4156d20 0x7fa9c41f2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:39:01.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:01.090+0000 7fa9c9400640 1 -- 172.21.15.17:0/1252817395 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7fa9c4069b90 con 0x7fa9c4156d20 2024-09-10T21:39:01.093 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:01.090+0000 7fa9c2800640 1 --2- 172.21.15.17:0/1252817395 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa9c4156d20 0x7fa9c41f2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:39:01.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:01.090+0000 7fa9c2800640 1 -- 172.21.15.17:0/1252817395 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa9c4074ed0 con 0x7fa9c4156d20 2024-09-10T21:39:01.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:01.090+0000 7fa9baa00640 1 -- 172.21.15.17:0/1252817395 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fa9bc0b0cd0 con 0x7fa9c40c7210 2024-09-10T21:39:01.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:01.090+0000 7fa9c2800640 1 --2- 172.21.15.17:0/1252817395 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa9c4156d20 0x7fa9c41f2000 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fa9b400f540 tx=0x7fa9b400fa00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:39:01.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:01.090+0000 7fa9baa00640 1 --2- 172.21.15.17:0/1252817395 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fa9a40725e0 0x7fa9a4074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:39:01.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:01.090+0000 7fa9baa00640 1 -- 172.21.15.17:0/1252817395 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fa9bc1313a0 con 0x7fa9c40c7210 2024-09-10T21:39:01.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:01.090+0000 7fa9baa00640 1 -- 172.21.15.17:0/1252817395 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9b40052e0 con 0x7fa9c4156d20 2024-09-10T21:39:01.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:01.090+0000 7fa9baa00640 1 -- 172.21.15.17:0/1252817395 <== mon.2 v2:172.21.15.31:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fa9b4005480 con 0x7fa9c4156d20 2024-09-10T21:39:01.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:01.090+0000 7fa9baa00640 1 -- 172.21.15.17:0/1252817395 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa9c4156d20 msgr2=0x7fa9c41f2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:39:01.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:01.090+0000 7fa9baa00640 1 --2- 172.21.15.17:0/1252817395 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa9c4156d20 0x7fa9c41f2000 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fa9b400f540 tx=0x7fa9b400fa00 comp rx=0 tx=0).stop 2024-09-10T21:39:01.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:01.090+0000 7fa9baa00640 1 -- 172.21.15.17:0/1252817395 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa9c4156d20 msgr2=0x7fa9c41f2000 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:39:01.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:01.090+0000 7fa9baa00640 1 --2- 172.21.15.17:0/1252817395 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa9c4156d20 0x7fa9c41f2000 secure :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fa9b400f540 tx=0x7fa9b400fa00 comp rx=0 tx=0).stop 2024-09-10T21:39:01.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:01.090+0000 7fa9c1e00640 1 --2- 172.21.15.17:0/1252817395 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fa9a40725e0 0x7fa9a4074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:39:01.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:01.090+0000 7fa9c1e00640 1 --2- 172.21.15.17:0/1252817395 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fa9a40725e0 0x7fa9a4074aa0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fa9c40b2860 tx=0x7fa9b000b2c0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:39:01.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:01.098+0000 7fa9c9400640 1 --2- 172.21.15.17:0/1252817395 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa9c41f2540 0x7fa9c41f2950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:39:01.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:01.098+0000 7fa9c9400640 1 -- 172.21.15.17:0/1252817395 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7fa9c40746d0 con 0x7fa9c41f2540 2024-09-10T21:39:01.101 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:01.098+0000 7fa9c2800640 1 --2- 172.21.15.17:0/1252817395 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa9c41f2540 0x7fa9c41f2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:39:01.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:01.098+0000 7fa9c2800640 1 -- 172.21.15.17:0/1252817395 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fa9c4150320 con 0x7fa9c41f2540 2024-09-10T21:39:01.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:01.098+0000 7fa9c2800640 1 --2- 172.21.15.17:0/1252817395 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa9c41f2540 0x7fa9c41f2950 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa9b400f540 tx=0x7fa9b4000ee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:39:01.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:01.098+0000 7fa9baa00640 1 -- 172.21.15.17:0/1252817395 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9b40074b0 con 0x7fa9c41f2540 2024-09-10T21:39:01.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:01.098+0000 7fa9baa00640 1 -- 172.21.15.17:0/1252817395 <== mon.2 v2:172.21.15.31:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5043 (secure 0 0 0) 0x7fa9b40074b0 con 0x7fa9c41f2540 2024-09-10T21:39:01.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:01.098+0000 7fa9baa00640 1 -- 172.21.15.17:0/1252817395 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa9c41f2540 msgr2=0x7fa9c41f2950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:39:01.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:01.098+0000 7fa9baa00640 1 --2- 172.21.15.17:0/1252817395 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa9c41f2540 0x7fa9c41f2950 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa9b400f540 tx=0x7fa9b4000ee0 comp rx=0 tx=0).stop 2024-09-10T21:39:01.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:01.098+0000 7fa9baa00640 1 -- 172.21.15.17:0/1252817395 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa9c41f2540 msgr2=0x7fa9c41f2950 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:39:01.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:01.098+0000 7fa9baa00640 1 --2- 172.21.15.17:0/1252817395 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa9c41f2540 0x7fa9c41f2950 secure :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa9b400f540 tx=0x7fa9b4000ee0 comp rx=0 tx=0).stop 2024-09-10T21:39:01.104 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-10T21:39:01.104 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:39:01.104 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:39:01.104 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-10T21:39:01.104 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 116, 2024-09-10T21:39:01.104 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 147445, 2024-09-10T21:39:01.104 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-10T21:39:01.104 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-10T21:39:01.104 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-10T21:39:01.105 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-10T21:39:01.105 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:39:01.105 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 2, 2024-09-10T21:39:01.105 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:39:01.105 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-10T21:39:01.105 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-10T21:39:01.105 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:39:01.105 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:39:01.105 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:39:01.105 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:39:01.105 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:39:01.106 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:39:01.106 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:39:01.106 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:39:01.106 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:39:01.106 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:39:01.106 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:39:01.106 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:39:01.106 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-10T21:39:01.106 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-10T21:39:01.106 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:39:01.106 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:39:01.106 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:39:01.107 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:39:01.107 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:39:01.107 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:39:01.107 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:39:01.107 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:39:01.107 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:39:01.107 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:39:01.107 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:39:01.107 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:39:01.107 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:39:01.107 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-10T21:39:01.108 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-10T21:39:01.108 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-10T21:39:01.108 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-10T21:39:01.108 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-10T21:39:01.108 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "5e2f023f-d6a6-49f4-a256-f4807dce3739", 2024-09-10T21:39:01.108 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:39:01.108 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:39:01.108 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-10T21:39:01.108 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-10T21:39:01.108 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-10T21:39:01.108 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-10T21:39:01.108 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-10T21:39:01.109 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-10T21:39:01.109 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-10T21:39:01.109 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:39:01.109 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-10T21:39:01.109 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:39:01.109 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:39:01.109 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:39:01.109 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:39:01.109 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:39:01.109 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:39:01.110 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:39:01.110 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:39:01.110 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:39:01.110 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:39:01.110 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:39:01.110 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:39:01.110 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-10T21:39:01.110 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:39:01.110 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-10T21:39:01.110 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:39:01.110 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:39:01.110 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:39:01.111 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:39:01.111 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:39:01.111 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:39:01.111 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:39:01.111 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-10T21:39:01.111 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:39:01.111 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:39:01.111 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:39:01.111 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:39:01.111 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-10T21:39:01.111 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:39:01.111 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:39:01.112 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:39:01.112 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:39:01.112 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-10T21:39:01.112 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-10T21:39:01.112 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:39:01.112 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:39:01.112 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:39:01.112 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:39:01.112 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:39:01.112 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:39:01.112 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:39:01.113 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:39:01.113 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:39:01.113 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:39:01.113 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:39:01.113 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3300", 2024-09-10T21:39:01.113 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:39:01.113 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:39:01.113 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:39:01.113 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:39:01.113 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789", 2024-09-10T21:39:01.113 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:39:01.114 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:39:01.114 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:39:01.114 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:39:01.114 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789/0", 2024-09-10T21:39:01.114 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6789/0", 2024-09-10T21:39:01.114 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:39:01.114 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:39:01.114 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:39:01.114 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:39:01.114 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:39:01.115 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:39:01.115 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:39:01.115 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:39:01.115 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:39:01.115 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:39:01.115 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:39:01.115 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3301", 2024-09-10T21:39:01.115 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:39:01.115 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:39:01.115 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:39:01.115 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:39:01.116 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790", 2024-09-10T21:39:01.116 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:39:01.116 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:39:01.116 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:39:01.116 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:39:01.116 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790/0", 2024-09-10T21:39:01.116 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6790/0", 2024-09-10T21:39:01.116 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:39:01.116 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:39:01.116 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:39:01.116 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:39:01.117 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:39:01.117 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:39:01.117 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-10T21:39:01.117 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-10T21:39:01.117 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:39:01.117 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:39:01.117 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:39:01.117 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:39:01.117 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:39:01.117 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:39:01.117 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": [ 2024-09-10T21:39:01.117 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:39:01.118 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:39:01.118 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:39:01.118 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 4 2024-09-10T21:39:01.118 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:39:01.118 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:39:01.118 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-10T21:39:01.118 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:39:01.118 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:39:01.118 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:39:01.118 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 4 2024-09-10T21:39:01.118 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:39:01.118 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:39:01.119 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-10T21:39:01.119 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:39:01.119 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:39:01.119 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:39:01.119 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 3 2024-09-10T21:39:01.119 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:39:01.119 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:39:01.119 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": [ 2024-09-10T21:39:01.119 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:39:01.119 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:39:01.119 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:39:01.119 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:39:01.120 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:39:01.120 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:39:01.120 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:39:01.120 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-10T21:39:01.120 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-10T21:39:01.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:01.098+0000 7fa9c9400640 1 -- 172.21.15.17:0/1252817395 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fa9a40725e0 msgr2=0x7fa9a4074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:39:01.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:01.098+0000 7fa9c9400640 1 --2- 172.21.15.17:0/1252817395 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fa9a40725e0 0x7fa9a4074aa0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fa9c40b2860 tx=0x7fa9b000b2c0 comp rx=0 tx=0).stop 2024-09-10T21:39:01.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:01.098+0000 7fa9c9400640 1 -- 172.21.15.17:0/1252817395 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa9c40c7210 msgr2=0x7fa9c40b1e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:39:01.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:01.098+0000 7fa9c9400640 1 --2- 172.21.15.17:0/1252817395 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa9c40c7210 0x7fa9c40b1e00 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fa9bc009bd0 tx=0x7fa9bc009320 comp rx=0 tx=0).stop 2024-09-10T21:39:01.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:01.098+0000 7fa9c3200640 1 -- 172.21.15.17:0/1252817395 reap_dead start 2024-09-10T21:39:01.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:01.098+0000 7fa9c3200640 1 -- 172.21.15.17:0/1252817395 reap_dead start 2024-09-10T21:39:01.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:01.098+0000 7fa9c9400640 1 -- 172.21.15.17:0/1252817395 shutdown_connections 2024-09-10T21:39:01.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:01.098+0000 7fa9c9400640 1 -- 172.21.15.17:0/1252817395 >> 172.21.15.17:0/1252817395 conn(0x7fa9c406dd30 msgr2=0x7fa9c4150840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:39:01.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:01.098+0000 7fa9c9400640 1 -- 172.21.15.17:0/1252817395 shutdown_connections 2024-09-10T21:39:01.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:01.098+0000 7fa9c9400640 1 -- 172.21.15.17:0/1252817395 wait complete. 2024-09-10T21:39:01.123 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-10T21:39:01.123 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-09-10T21:39:01.123 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-09-10T21:39:01.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:01.286+0000 7f4da7e00640 1 -- 172.21.15.17:0/3508950972 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4d8c004810 msgr2=0x7f4d8c004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:39:01.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:01.286+0000 7f4da7e00640 1 --2- 172.21.15.17:0/3508950972 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4d8c004810 0x7f4d8c004c90 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f4d94001860 tx=0x7f4d9401f050 comp rx=0 tx=0).stop 2024-09-10T21:39:01.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:01.286+0000 7f4da7400640 1 -- 172.21.15.17:0/3508950972 reap_dead start 2024-09-10T21:39:01.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:01.286+0000 7f4da7e00640 1 -- 172.21.15.17:0/3508950972 shutdown_connections 2024-09-10T21:39:01.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:01.286+0000 7f4da7e00640 1 -- 172.21.15.17:0/3508950972 >> 172.21.15.17:0/3508950972 conn(0x7f4da8070ca0 msgr2=0x7f4da80710b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:39:01.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:01.286+0000 7f4da7e00640 1 -- 172.21.15.17:0/3508950972 shutdown_connections 2024-09-10T21:39:01.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:01.286+0000 7f4da7e00640 1 -- 172.21.15.17:0/3508950972 wait complete. 2024-09-10T21:39:01.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:01.286+0000 7f4da7e00640 1 Processor -- start 2024-09-10T21:39:01.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:01.286+0000 7f4da7e00640 1 -- start start 2024-09-10T21:39:01.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:01.286+0000 7f4da7e00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d8c004810 0x7f4da814cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:39:01.292 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:01.286+0000 7f4da7e00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4da814efd0 0x7f4da81f4140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:39:01.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:01.286+0000 7f4da7e00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f4da8153910 0x7f4da81f4680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:39:01.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:01.286+0000 7f4da7e00640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f4da815b050 con 0x7f4da814efd0 2024-09-10T21:39:01.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:01.286+0000 7f4da7e00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f4da80702a0 con 0x7f4d8c004810 2024-09-10T21:39:01.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:01.286+0000 7f4da7e00640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f4da815b1d0 con 0x7f4da8153910 2024-09-10T21:39:01.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:01.290+0000 7f4da6000640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4da814efd0 0x7f4da81f4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:39:01.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:01.290+0000 7f4da6a00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d8c004810 0x7f4da814cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:39:01.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:01.290+0000 7f4da6000640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4da814efd0 0x7f4da81f4140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:49738/0 (socket says 172.21.15.17:49738) 2024-09-10T21:39:01.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:01.290+0000 7f4da6000640 1 -- 172.21.15.17:0/3216541927 learned_addr learned my addr 172.21.15.17:0/3216541927 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:39:01.293 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:01.290+0000 7f4da7400640 1 --2- 172.21.15.17:0/3216541927 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f4da8153910 0x7f4da81f4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:39:01.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:01.290+0000 7f4da6a00640 1 -- 172.21.15.17:0/3216541927 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f4da8153910 msgr2=0x7f4da81f4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:39:01.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:01.290+0000 7f4da6a00640 1 --2- 172.21.15.17:0/3216541927 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f4da8153910 0x7f4da81f4680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:39:01.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:01.290+0000 7f4da6a00640 1 -- 172.21.15.17:0/3216541927 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4da814efd0 msgr2=0x7f4da81f4140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:39:01.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:01.290+0000 7f4da6a00640 1 --2- 172.21.15.17:0/3216541927 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4da814efd0 0x7f4da81f4140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:39:01.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:01.290+0000 7f4da6a00640 1 -- 172.21.15.17:0/3216541927 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4da8150a80 con 0x7f4d8c004810 2024-09-10T21:39:01.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:01.290+0000 7f4da6000640 1 --2- 172.21.15.17:0/3216541927 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4da814efd0 0x7f4da81f4140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:39:01.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:01.290+0000 7f4da6a00640 1 --2- 172.21.15.17:0/3216541927 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d8c004810 0x7f4da814cb10 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f4d980119e0 tx=0x7f4d98011de0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:39:01.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:01.290+0000 7f4d9ea00640 1 -- 172.21.15.17:0/3216541927 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d9800fe10 con 0x7f4d8c004810 2024-09-10T21:39:01.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:01.290+0000 7f4da7e00640 1 -- 172.21.15.17:0/3216541927 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4da814f4c0 con 0x7f4d8c004810 2024-09-10T21:39:01.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:01.290+0000 7f4d9ea00640 1 -- 172.21.15.17:0/3216541927 <== mon.1 v2:172.21.15.31:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4d98004030 con 0x7f4d8c004810 2024-09-10T21:39:01.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:01.290+0000 7f4d9ea00640 1 -- 172.21.15.17:0/3216541927 <== mon.1 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d98004420 con 0x7f4d8c004810 2024-09-10T21:39:01.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:01.290+0000 7f4da7e00640 1 -- 172.21.15.17:0/3216541927 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4da81501e0 con 0x7f4d8c004810 2024-09-10T21:39:01.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:01.290+0000 7f4da7e00640 1 -- 172.21.15.17:0/3216541927 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4da8151980 con 0x7f4d8c004810 2024-09-10T21:39:01.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:01.294+0000 7f4d9ea00640 1 -- 172.21.15.17:0/3216541927 <== mon.1 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f4d98018070 con 0x7f4d8c004810 2024-09-10T21:39:01.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:01.294+0000 7f4d9ea00640 1 --2- 172.21.15.17:0/3216541927 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f4d88072610 0x7f4d88074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:39:01.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:01.294+0000 7f4d9ea00640 1 -- 172.21.15.17:0/3216541927 <== mon.1 v2:172.21.15.31:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f4d98093f50 con 0x7f4d8c004810 2024-09-10T21:39:01.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:01.294+0000 7f4da6000640 1 --2- 172.21.15.17:0/3216541927 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f4d88072610 0x7f4d88074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:39:01.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:01.294+0000 7f4da6000640 1 --2- 172.21.15.17:0/3216541927 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f4d88072610 0x7f4d88074ad0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f4d9401d920 tx=0x7f4d9401be90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:39:01.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:01.298+0000 7f4d9ea00640 1 -- 172.21.15.17:0/3216541927 <== mon.1 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f4d9805f1a0 con 0x7f4d8c004810 2024-09-10T21:39:01.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:01.622+0000 7f4da7e00640 1 -- 172.21.15.17:0/3216541927 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) -- 0x7f4da81f3ea0 con 0x7f4d8c004810 2024-09-10T21:39:01.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:01.622+0000 7f4d9ea00640 1 -- 172.21.15.17:0/3216541927 <== mon.1 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) ==== 57+0+0 (secure 0 0 0) 0x7f4d9805eaa0 con 0x7f4d8c004810 2024-09-10T21:39:01.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:01.626+0000 7f4da7e00640 1 -- 172.21.15.17:0/3216541927 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f4d88072610 msgr2=0x7f4d88074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:39:01.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:01.626+0000 7f4da7e00640 1 --2- 172.21.15.17:0/3216541927 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f4d88072610 0x7f4d88074ad0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f4d9401d920 tx=0x7f4d9401be90 comp rx=0 tx=0).stop 2024-09-10T21:39:01.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:01.626+0000 7f4da7e00640 1 -- 172.21.15.17:0/3216541927 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d8c004810 msgr2=0x7f4da814cb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:39:01.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:01.626+0000 7f4da7e00640 1 --2- 172.21.15.17:0/3216541927 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d8c004810 0x7f4da814cb10 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f4d980119e0 tx=0x7f4d98011de0 comp rx=0 tx=0).stop 2024-09-10T21:39:01.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:01.626+0000 7f4da7e00640 1 -- 172.21.15.17:0/3216541927 shutdown_connections 2024-09-10T21:39:01.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:01.626+0000 7f4da7e00640 1 --2- 172.21.15.17:0/3216541927 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f4d88072610 0x7f4d88074ad0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:39:01.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:01.626+0000 7f4da7e00640 1 --2- 172.21.15.17:0/3216541927 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f4da8153910 0x7f4da81f4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:39:01.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:01.626+0000 7f4da7e00640 1 --2- 172.21.15.17:0/3216541927 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4da814efd0 0x7f4da81f4140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:39:01.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:01.626+0000 7f4da7e00640 1 --2- 172.21.15.17:0/3216541927 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d8c004810 0x7f4da814cb10 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:39:01.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:01.626+0000 7f4da7e00640 1 -- 172.21.15.17:0/3216541927 >> 172.21.15.17:0/3216541927 conn(0x7f4da8070ca0 msgr2=0x7f4da814af30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:39:01.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:01.626+0000 7f4da7e00640 1 -- 172.21.15.17:0/3216541927 shutdown_connections 2024-09-10T21:39:01.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:01.626+0000 7f4da7e00640 1 -- 172.21.15.17:0/3216541927 wait complete. 2024-09-10T21:39:01.655 INFO:tasks.mon_thrash.mon_thrasher:waiting for 10.0 secs before continuing thrashing 2024-09-10T21:39:11.656 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-09-10T21:39:11.657 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-10T21:39:11.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:11.894+0000 7f5a4fe00640 1 -- 172.21.15.17:0/4019445238 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a30004810 msgr2=0x7f5a30004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:39:11.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:11.894+0000 7f5a4fe00640 1 --2- 172.21.15.17:0/4019445238 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a30004810 0x7f5a30004c90 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f5a34009ba0 tx=0x7f5a3401d280 comp rx=0 tx=0).stop 2024-09-10T21:39:11.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:11.894+0000 7f5a4de00640 1 -- 172.21.15.17:0/4019445238 reap_dead start 2024-09-10T21:39:11.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:11.894+0000 7f5a4fe00640 1 -- 172.21.15.17:0/4019445238 shutdown_connections 2024-09-10T21:39:11.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:11.894+0000 7f5a4fe00640 1 -- 172.21.15.17:0/4019445238 >> 172.21.15.17:0/4019445238 conn(0x7f5a480702f0 msgr2=0x7f5a48070700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:39:11.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:11.894+0000 7f5a4fe00640 1 -- 172.21.15.17:0/4019445238 shutdown_connections 2024-09-10T21:39:11.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:11.894+0000 7f5a4fe00640 1 -- 172.21.15.17:0/4019445238 wait complete. 2024-09-10T21:39:11.904 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:11.898+0000 7f5a4fe00640 1 Processor -- start 2024-09-10T21:39:11.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:11.898+0000 7f5a4fe00640 1 -- start start 2024-09-10T21:39:11.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:11.898+0000 7f5a4fe00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a480c7220 0x7f5a4806e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:39:11.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:11.898+0000 7f5a4fe00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f5a4815ca40 0x7f5a4806e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:39:11.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:11.898+0000 7f5a4fe00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5a480c7600 0x7f5a4815cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:39:11.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:11.898+0000 7f5a4fe00640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f5a480cb450 con 0x7f5a480c7220 2024-09-10T21:39:11.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:11.898+0000 7f5a4fe00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f5a480b42c0 con 0x7f5a480c7600 2024-09-10T21:39:11.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:11.898+0000 7f5a4fe00640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f5a480cb5d0 con 0x7f5a4815ca40 2024-09-10T21:39:11.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:11.898+0000 7f5a4d400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a480c7220 0x7f5a4806e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:39:11.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:11.898+0000 7f5a4d400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a480c7220 0x7f5a4806e110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:54404/0 (socket says 172.21.15.17:54404) 2024-09-10T21:39:11.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:11.898+0000 7f5a4d400640 1 -- 172.21.15.17:0/31346518 learned_addr learned my addr 172.21.15.17:0/31346518 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:39:11.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:11.898+0000 7f5a4ca00640 1 --2- 172.21.15.17:0/31346518 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f5a4815ca40 0x7f5a4806e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:39:11.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:11.898+0000 7f5a4de00640 1 --2- 172.21.15.17:0/31346518 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5a480c7600 0x7f5a4815cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:39:11.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:11.898+0000 7f5a4d400640 1 -- 172.21.15.17:0/31346518 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f5a4815ca40 msgr2=0x7f5a4806e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:39:11.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:11.898+0000 7f5a4d400640 1 --2- 172.21.15.17:0/31346518 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f5a4815ca40 0x7f5a4806e650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:39:11.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:11.898+0000 7f5a4d400640 1 -- 172.21.15.17:0/31346518 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5a480c7600 msgr2=0x7f5a4815cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:39:11.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:11.898+0000 7f5a4d400640 1 --2- 172.21.15.17:0/31346518 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5a480c7600 0x7f5a4815cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:39:11.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:11.898+0000 7f5a4d400640 1 -- 172.21.15.17:0/31346518 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a48156b30 con 0x7f5a480c7220 2024-09-10T21:39:11.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:11.898+0000 7f5a4d400640 1 --2- 172.21.15.17:0/31346518 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a480c7220 0x7f5a4806e110 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f5a3c0066b0 tx=0x7f5a3c003ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:39:11.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:11.898+0000 7f5a45600640 1 -- 172.21.15.17:0/31346518 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a3c015020 con 0x7f5a480c7220 2024-09-10T21:39:11.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:11.898+0000 7f5a45600640 1 -- 172.21.15.17:0/31346518 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5a3c011900 con 0x7f5a480c7220 2024-09-10T21:39:11.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:11.898+0000 7f5a4fe00640 1 -- 172.21.15.17:0/31346518 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a4815e6c0 con 0x7f5a480c7220 2024-09-10T21:39:11.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:11.898+0000 7f5a45600640 1 -- 172.21.15.17:0/31346518 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a3c011d50 con 0x7f5a480c7220 2024-09-10T21:39:11.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:11.902+0000 7f5a4fe00640 1 -- 172.21.15.17:0/31346518 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a481570d0 con 0x7f5a480c7220 2024-09-10T21:39:11.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:11.902+0000 7f5a4fe00640 1 -- 172.21.15.17:0/31346518 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a480c2ff0 con 0x7f5a480c7220 2024-09-10T21:39:11.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:11.902+0000 7f5a45600640 1 -- 172.21.15.17:0/31346518 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f5a3c010420 con 0x7f5a480c7220 2024-09-10T21:39:11.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:11.902+0000 7f5a45600640 1 --2- 172.21.15.17:0/31346518 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f5a280725e0 0x7f5a28074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:39:11.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:11.902+0000 7f5a45600640 1 -- 172.21.15.17:0/31346518 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f5a3c094520 con 0x7f5a480c7220 2024-09-10T21:39:11.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:11.902+0000 7f5a4ca00640 1 --2- 172.21.15.17:0/31346518 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f5a280725e0 0x7f5a28074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:39:11.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:11.906+0000 7f5a4ca00640 1 --2- 172.21.15.17:0/31346518 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f5a280725e0 0x7f5a28074aa0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f5a340ae000 tx=0x7f5a3401a020 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:39:11.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:11.910+0000 7f5a45600640 1 -- 172.21.15.17:0/31346518 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f5a3c01f020 con 0x7f5a480c7220 2024-09-10T21:39:12.201 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.198+0000 7f5a4fe00640 1 -- 172.21.15.17:0/31346518 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f5a4806fd00 con 0x7f5a480c7220 2024-09-10T21:39:12.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.198+0000 7f5a45600640 1 -- 172.21.15.17:0/31346518 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1558 (secure 0 0 0) 0x7f5a3c05f090 con 0x7f5a480c7220 2024-09-10T21:39:12.202 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":116,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":13,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5e2f023f-d6a6-49f4-a256-f4807dce3739","modified":"2024-09-10T21:23:46.892130Z","created":"2024-09-10T21:23:46.892130Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3301","nonce":0},{"type":"v1","addr":"172.21.15.31:6790","nonce":0}]},"addr":"172.21.15.31:6790/0","public_addr":"172.21.15.31:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-10T21:39:12.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.202+0000 7f5a4fe00640 1 -- 172.21.15.17:0/31346518 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f5a280725e0 msgr2=0x7f5a28074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:39:12.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.202+0000 7f5a4fe00640 1 --2- 172.21.15.17:0/31346518 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f5a280725e0 0x7f5a28074aa0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f5a340ae000 tx=0x7f5a3401a020 comp rx=0 tx=0).stop 2024-09-10T21:39:12.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.202+0000 7f5a4fe00640 1 -- 172.21.15.17:0/31346518 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a480c7220 msgr2=0x7f5a4806e110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:39:12.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.202+0000 7f5a4fe00640 1 --2- 172.21.15.17:0/31346518 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a480c7220 0x7f5a4806e110 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f5a3c0066b0 tx=0x7f5a3c003ab0 comp rx=0 tx=0).stop 2024-09-10T21:39:12.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.202+0000 7f5a4fe00640 1 -- 172.21.15.17:0/31346518 shutdown_connections 2024-09-10T21:39:12.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.202+0000 7f5a4fe00640 1 --2- 172.21.15.17:0/31346518 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f5a280725e0 0x7f5a28074aa0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:39:12.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.202+0000 7f5a4fe00640 1 --2- 172.21.15.17:0/31346518 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5a480c7600 0x7f5a4815cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:39:12.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.202+0000 7f5a4fe00640 1 --2- 172.21.15.17:0/31346518 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f5a4815ca40 0x7f5a4806e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:39:12.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.202+0000 7f5a4fe00640 1 --2- 172.21.15.17:0/31346518 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a480c7220 0x7f5a4806e110 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:39:12.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.202+0000 7f5a4fe00640 1 -- 172.21.15.17:0/31346518 >> 172.21.15.17:0/31346518 conn(0x7f5a480702f0 msgr2=0x7f5a480c9060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:39:12.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.202+0000 7f5a4fe00640 1 -- 172.21.15.17:0/31346518 shutdown_connections 2024-09-10T21:39:12.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.202+0000 7f5a4fe00640 1 -- 172.21.15.17:0/31346518 wait complete. 2024-09-10T21:39:12.225 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-09-10T21:39:12.225 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-09-10T21:39:12.225 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-09-10T21:39:12.226 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-10T21:39:12.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.402+0000 7f848bc00640 1 -- 172.21.15.17:0/3995875975 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f848406eb90 msgr2=0x7f8484161c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:39:12.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.402+0000 7f848bc00640 1 --2- 172.21.15.17:0/3995875975 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f848406eb90 0x7f8484161c70 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f847c009c80 tx=0x7f847c01cda0 comp rx=0 tx=0).stop 2024-09-10T21:39:12.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.402+0000 7f848bc00640 1 -- 172.21.15.17:0/3995875975 shutdown_connections 2024-09-10T21:39:12.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.402+0000 7f848bc00640 1 --2- 172.21.15.17:0/3995875975 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f848406eb90 0x7f8484161c70 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:39:12.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.402+0000 7f848bc00640 1 --2- 172.21.15.17:0/3995875975 >> v2:172.21.15.31:3300/0 conn(0x7f84841517a0 0x7f848415b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:39:12.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.402+0000 7f848bc00640 1 -- 172.21.15.17:0/3995875975 >> 172.21.15.17:0/3995875975 conn(0x7f84840714a0 msgr2=0x7f84840718b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:39:12.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.402+0000 7f848bc00640 1 -- 172.21.15.17:0/3995875975 shutdown_connections 2024-09-10T21:39:12.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.402+0000 7f848bc00640 1 -- 172.21.15.17:0/3995875975 wait complete. 2024-09-10T21:39:12.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.402+0000 7f848bc00640 1 Processor -- start 2024-09-10T21:39:12.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.402+0000 7f848bc00640 1 -- start start 2024-09-10T21:39:12.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.402+0000 7f848bc00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f848406eb90 0x7f84840ccf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:39:12.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.402+0000 7f848bc00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f84840c45e0 0x7f8484153bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:39:12.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.402+0000 7f848bc00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f84841517a0 0x7f8484159dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:39:12.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.402+0000 7f848bc00640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f848406e770 con 0x7f848406eb90 2024-09-10T21:39:12.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.402+0000 7f848bc00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f848406e5f0 con 0x7f84840c45e0 2024-09-10T21:39:12.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.402+0000 7f848bc00640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f84840c6520 con 0x7f84841517a0 2024-09-10T21:39:12.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.402+0000 7f8489200640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f848406eb90 0x7f84840ccf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:39:12.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.402+0000 7f8489200640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f848406eb90 0x7f84840ccf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:54420/0 (socket says 172.21.15.17:54420) 2024-09-10T21:39:12.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.402+0000 7f8489200640 1 -- 172.21.15.17:0/1017915145 learned_addr learned my addr 172.21.15.17:0/1017915145 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:39:12.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.406+0000 7f8483e00640 1 --2- 172.21.15.17:0/1017915145 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f84840c45e0 0x7f8484153bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:39:12.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.406+0000 7f8489c00640 1 --2- 172.21.15.17:0/1017915145 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f84841517a0 0x7f8484159dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:39:12.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.406+0000 7f8489200640 1 -- 172.21.15.17:0/1017915145 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f84841517a0 msgr2=0x7f8484159dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:39:12.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.406+0000 7f8489200640 1 --2- 172.21.15.17:0/1017915145 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f84841517a0 0x7f8484159dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:39:12.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.406+0000 7f8489200640 1 -- 172.21.15.17:0/1017915145 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f84840c45e0 msgr2=0x7f8484153bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:39:12.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.406+0000 7f8489200640 1 --2- 172.21.15.17:0/1017915145 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f84840c45e0 0x7f8484153bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:39:12.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.406+0000 7f8489200640 1 -- 172.21.15.17:0/1017915145 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84841553c0 con 0x7f848406eb90 2024-09-10T21:39:12.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.406+0000 7f8489200640 1 --2- 172.21.15.17:0/1017915145 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f848406eb90 0x7f84840ccf70 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f847800e7d0 tx=0x7f847800ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:39:12.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.406+0000 7f8480c00640 1 -- 172.21.15.17:0/1017915145 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f847800ccc0 con 0x7f848406eb90 2024-09-10T21:39:12.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.406+0000 7f8480c00640 1 -- 172.21.15.17:0/1017915145 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8478010430 con 0x7f848406eb90 2024-09-10T21:39:12.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.406+0000 7f848bc00640 1 -- 172.21.15.17:0/1017915145 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84840cc9f0 con 0x7f848406eb90 2024-09-10T21:39:12.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.406+0000 7f8480c00640 1 -- 172.21.15.17:0/1017915145 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8478010860 con 0x7f848406eb90 2024-09-10T21:39:12.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.406+0000 7f848bc00640 1 -- 172.21.15.17:0/1017915145 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8484154790 con 0x7f848406eb90 2024-09-10T21:39:12.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.406+0000 7f848bc00640 1 --2- 172.21.15.17:0/1017915145 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8484152180 0x7f84841f6820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:39:12.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.406+0000 7f848bc00640 1 -- 172.21.15.17:0/1017915145 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f8484154b00 con 0x7f8484152180 2024-09-10T21:39:12.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.406+0000 7f8483e00640 1 --2- 172.21.15.17:0/1017915145 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8484152180 0x7f84841f6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:39:12.411 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.406+0000 7f8483e00640 1 -- 172.21.15.17:0/1017915145 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f84840cd4b0 con 0x7f8484152180 2024-09-10T21:39:12.411 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.406+0000 7f8483e00640 1 --2- 172.21.15.17:0/1017915145 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8484152180 0x7f84841f6820 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f84841f43e0 tx=0x7f847000fc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:39:12.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.406+0000 7f8480c00640 1 -- 172.21.15.17:0/1017915145 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84700050d0 con 0x7f8484152180 2024-09-10T21:39:12.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.406+0000 7f8480c00640 1 -- 172.21.15.17:0/1017915145 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f8470005270 con 0x7f8484152180 2024-09-10T21:39:12.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.406+0000 7f8480c00640 1 -- 172.21.15.17:0/1017915145 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8484152180 msgr2=0x7f84841f6820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:39:12.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.406+0000 7f8480c00640 1 --2- 172.21.15.17:0/1017915145 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8484152180 0x7f84841f6820 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f84841f43e0 tx=0x7f847000fc00 comp rx=0 tx=0).stop 2024-09-10T21:39:12.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.406+0000 7f8480c00640 1 -- 172.21.15.17:0/1017915145 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8484152180 msgr2=0x7f84841f6820 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:39:12.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.406+0000 7f8480c00640 1 --2- 172.21.15.17:0/1017915145 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8484152180 0x7f84841f6820 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:39:12.412 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.406+0000 7f8480c00640 1 -- 172.21.15.17:0/1017915145 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f8478010a00 con 0x7f848406eb90 2024-09-10T21:39:12.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.410+0000 7f8480c00640 1 --2- 172.21.15.17:0/1017915145 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f8464072050 0x7f8464074510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:39:12.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.410+0000 7f8480c00640 1 -- 172.21.15.17:0/1017915145 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f8478093fe0 con 0x7f848406eb90 2024-09-10T21:39:12.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.410+0000 7f8483e00640 1 --2- 172.21.15.17:0/1017915145 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f8464072050 0x7f8464074510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:39:12.413 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.410+0000 7f8483e00640 1 --2- 172.21.15.17:0/1017915145 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f8464072050 0x7f8464074510 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f84841f43e0 tx=0x7f8470005000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:39:12.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.414+0000 7f848bc00640 1 --2- 172.21.15.17:0/1017915145 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f84841f6d60 0x7f848414afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:39:12.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.414+0000 7f848bc00640 1 -- 172.21.15.17:0/1017915145 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f84840c5e10 con 0x7f84841f6d60 2024-09-10T21:39:12.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.414+0000 7f8489c00640 1 --2- 172.21.15.17:0/1017915145 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f84841f6d60 0x7f848414afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:39:12.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.414+0000 7f8489c00640 1 -- 172.21.15.17:0/1017915145 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f848406e970 con 0x7f84841f6d60 2024-09-10T21:39:12.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.414+0000 7f8489c00640 1 --2- 172.21.15.17:0/1017915145 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f84841f6d60 0x7f848414afc0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f8484154fa0 tx=0x7f847c004a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:39:12.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.414+0000 7f8480c00640 1 -- 172.21.15.17:0/1017915145 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f847c0b2040 con 0x7f84841f6d60 2024-09-10T21:39:12.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.418+0000 7f8480c00640 1 -- 172.21.15.17:0/1017915145 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4544 (secure 0 0 0) 0x7f847c0b8480 con 0x7f84841f6d60 2024-09-10T21:39:12.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.418+0000 7f8480c00640 1 -- 172.21.15.17:0/1017915145 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f84841f6d60 msgr2=0x7f848414afc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:39:12.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.418+0000 7f8480c00640 1 --2- 172.21.15.17:0/1017915145 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f84841f6d60 0x7f848414afc0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f8484154fa0 tx=0x7f847c004a50 comp rx=0 tx=0).stop 2024-09-10T21:39:12.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.418+0000 7f8480c00640 1 -- 172.21.15.17:0/1017915145 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f84841f6d60 msgr2=0x7f848414afc0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:39:12.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.418+0000 7f8480c00640 1 --2- 172.21.15.17:0/1017915145 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f84841f6d60 0x7f848414afc0 secure :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f8484154fa0 tx=0x7f847c004a50 comp rx=0 tx=0).stop 2024-09-10T21:39:12.420 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-10T21:39:12.421 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:39:12.421 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:39:12.421 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "leader", 2024-09-10T21:39:12.421 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 116, 2024-09-10T21:39:12.421 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 26615, 2024-09-10T21:39:12.421 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-10T21:39:12.421 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-10T21:39:12.421 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-10T21:39:12.421 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-10T21:39:12.421 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:39:12.421 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 13, 2024-09-10T21:39:12.421 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:39:12.422 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-10T21:39:12.422 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-10T21:39:12.422 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:39:12.422 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:39:12.422 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:39:12.422 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:39:12.422 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:39:12.422 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:39:12.422 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:39:12.422 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:39:12.422 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:39:12.422 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:39:12.422 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:39:12.423 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:39:12.423 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-10T21:39:12.423 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-10T21:39:12.423 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:39:12.423 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:39:12.423 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:39:12.423 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:39:12.423 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:39:12.423 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:39:12.423 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:39:12.423 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:39:12.423 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:39:12.423 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:39:12.423 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:39:12.424 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:39:12.424 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:39:12.424 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-10T21:39:12.424 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-10T21:39:12.424 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-10T21:39:12.424 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-10T21:39:12.424 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-10T21:39:12.424 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "5e2f023f-d6a6-49f4-a256-f4807dce3739", 2024-09-10T21:39:12.424 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:39:12.424 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:39:12.424 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-10T21:39:12.424 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-10T21:39:12.424 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-10T21:39:12.425 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-10T21:39:12.425 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-10T21:39:12.425 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-10T21:39:12.425 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-10T21:39:12.425 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:39:12.425 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-10T21:39:12.425 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:39:12.425 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:39:12.425 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:39:12.425 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:39:12.425 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:39:12.425 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:39:12.425 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:39:12.426 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:39:12.426 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:39:12.426 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:39:12.426 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:39:12.426 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:39:12.426 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-10T21:39:12.426 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:39:12.426 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-10T21:39:12.426 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:39:12.426 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:39:12.426 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:39:12.426 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:39:12.426 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:39:12.426 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:39:12.426 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:39:12.427 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-10T21:39:12.427 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:39:12.427 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:39:12.427 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:39:12.427 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:39:12.427 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-10T21:39:12.427 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:39:12.427 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:39:12.427 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:39:12.427 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:39:12.427 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-10T21:39:12.427 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-10T21:39:12.427 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:39:12.427 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:39:12.427 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:39:12.428 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:39:12.428 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:39:12.428 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:39:12.428 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:39:12.428 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:39:12.428 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:39:12.428 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:39:12.428 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:39:12.428 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3300", 2024-09-10T21:39:12.428 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:39:12.428 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:39:12.428 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:39:12.428 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:39:12.428 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789", 2024-09-10T21:39:12.429 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:39:12.429 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:39:12.429 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:39:12.429 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:39:12.429 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789/0", 2024-09-10T21:39:12.429 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6789/0", 2024-09-10T21:39:12.429 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:39:12.429 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:39:12.429 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:39:12.429 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:39:12.429 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:39:12.429 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:39:12.429 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:39:12.429 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:39:12.429 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:39:12.430 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:39:12.430 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:39:12.430 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3301", 2024-09-10T21:39:12.430 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:39:12.430 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:39:12.430 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:39:12.430 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:39:12.430 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790", 2024-09-10T21:39:12.430 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:39:12.430 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:39:12.430 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:39:12.430 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:39:12.430 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790/0", 2024-09-10T21:39:12.430 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6790/0", 2024-09-10T21:39:12.430 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:39:12.431 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:39:12.431 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:39:12.431 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:39:12.431 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:39:12.431 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:39:12.431 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-10T21:39:12.431 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-10T21:39:12.431 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:39:12.431 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:39:12.431 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:39:12.431 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:39:12.431 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:39:12.431 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:39:12.431 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-10T21:39:12.431 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:39:12.432 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:39:12.432 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:39:12.432 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 2 2024-09-10T21:39:12.432 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:39:12.432 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:39:12.432 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:39:12.432 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-10T21:39:12.432 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-10T21:39:12.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.418+0000 7f848bc00640 1 -- 172.21.15.17:0/1017915145 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f8464072050 msgr2=0x7f8464074510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:39:12.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.418+0000 7f848bc00640 1 --2- 172.21.15.17:0/1017915145 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f8464072050 0x7f8464074510 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f84841f43e0 tx=0x7f8470005000 comp rx=0 tx=0).stop 2024-09-10T21:39:12.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.418+0000 7f848bc00640 1 -- 172.21.15.17:0/1017915145 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f848406eb90 msgr2=0x7f84840ccf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:39:12.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.418+0000 7f848bc00640 1 --2- 172.21.15.17:0/1017915145 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f848406eb90 0x7f84840ccf70 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f847800e7d0 tx=0x7f847800ec90 comp rx=0 tx=0).stop 2024-09-10T21:39:12.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.418+0000 7f8489c00640 1 -- 172.21.15.17:0/1017915145 reap_dead start 2024-09-10T21:39:12.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.418+0000 7f8489c00640 1 -- 172.21.15.17:0/1017915145 reap_dead start 2024-09-10T21:39:12.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.418+0000 7f848bc00640 1 -- 172.21.15.17:0/1017915145 shutdown_connections 2024-09-10T21:39:12.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.418+0000 7f848bc00640 1 -- 172.21.15.17:0/1017915145 >> 172.21.15.17:0/1017915145 conn(0x7f84840714a0 msgr2=0x7f8484156bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:39:12.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.418+0000 7f848bc00640 1 -- 172.21.15.17:0/1017915145 shutdown_connections 2024-09-10T21:39:12.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.418+0000 7f848bc00640 1 -- 172.21.15.17:0/1017915145 wait complete. 2024-09-10T21:39:12.440 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-10T21:39:12.440 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-09-10T21:39:12.440 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-10T21:39:12.614 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.610+0000 7fe069200640 1 -- 172.21.15.17:0/1912865479 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe064071190 msgr2=0x7fe0640c8e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:39:12.614 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.610+0000 7fe069200640 1 --2- 172.21.15.17:0/1912865479 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe064071190 0x7fe0640c8e80 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fe0640b31b0 tx=0x7fe05001c8c0 comp rx=0 tx=0).stop 2024-09-10T21:39:12.614 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.610+0000 7fe069200640 1 -- 172.21.15.17:0/1912865479 shutdown_connections 2024-09-10T21:39:12.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.610+0000 7fe069200640 1 --2- 172.21.15.17:0/1912865479 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe064071190 0x7fe0640c8e80 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:39:12.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.610+0000 7fe069200640 1 --2- 172.21.15.17:0/1912865479 >> v2:172.21.15.31:3300/0 conn(0x7fe064157cc0 0x7fe06415afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:39:12.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.610+0000 7fe069200640 1 -- 172.21.15.17:0/1912865479 >> 172.21.15.17:0/1912865479 conn(0x7fe064068990 msgr2=0x7fe064068da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:39:12.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.610+0000 7fe069200640 1 -- 172.21.15.17:0/1912865479 shutdown_connections 2024-09-10T21:39:12.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.610+0000 7fe069200640 1 -- 172.21.15.17:0/1912865479 wait complete. 2024-09-10T21:39:12.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.610+0000 7fe069200640 1 Processor -- start 2024-09-10T21:39:12.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.610+0000 7fe069200640 1 -- start start 2024-09-10T21:39:12.617 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.614+0000 7fe069200640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe0640b2a60 0x7fe064156630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:39:12.617 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.614+0000 7fe069200640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe064157cc0 0x7fe064152040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:39:12.617 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.614+0000 7fe069200640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe06414af70 0x7fe064144910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:39:12.617 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.614+0000 7fe069200640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fe0641580a0 con 0x7fe0640b2a60 2024-09-10T21:39:12.617 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.614+0000 7fe069200640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fe0640727e0 con 0x7fe06414af70 2024-09-10T21:39:12.617 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.614+0000 7fe069200640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7fe064158220 con 0x7fe064157cc0 2024-09-10T21:39:12.617 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.614+0000 7fe063400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe0640b2a60 0x7fe064156630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:39:12.617 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.614+0000 7fe063400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe0640b2a60 0x7fe064156630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:54442/0 (socket says 172.21.15.17:54442) 2024-09-10T21:39:12.617 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.614+0000 7fe063400640 1 -- 172.21.15.17:0/2666040246 learned_addr learned my addr 172.21.15.17:0/2666040246 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:39:12.617 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.614+0000 7fe062a00640 1 --2- 172.21.15.17:0/2666040246 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe064157cc0 0x7fe064152040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:39:12.617 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.614+0000 7fe063e00640 1 --2- 172.21.15.17:0/2666040246 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe06414af70 0x7fe064144910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:39:12.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.614+0000 7fe063400640 1 -- 172.21.15.17:0/2666040246 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe064157cc0 msgr2=0x7fe064152040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:39:12.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.614+0000 7fe063400640 1 --2- 172.21.15.17:0/2666040246 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe064157cc0 0x7fe064152040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:39:12.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.614+0000 7fe063400640 1 -- 172.21.15.17:0/2666040246 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe06414af70 msgr2=0x7fe064144910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:39:12.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.614+0000 7fe063400640 1 --2- 172.21.15.17:0/2666040246 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe06414af70 0x7fe064144910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:39:12.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.614+0000 7fe063400640 1 -- 172.21.15.17:0/2666040246 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe06414a100 con 0x7fe0640b2a60 2024-09-10T21:39:12.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.614+0000 7fe063400640 1 --2- 172.21.15.17:0/2666040246 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe0640b2a60 0x7fe064156630 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fe05400e7d0 tx=0x7fe05400ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:39:12.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.614+0000 7fe05b400640 1 -- 172.21.15.17:0/2666040246 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe05400ccc0 con 0x7fe0640b2a60 2024-09-10T21:39:12.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.614+0000 7fe05b400640 1 -- 172.21.15.17:0/2666040246 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe054010430 con 0x7fe0640b2a60 2024-09-10T21:39:12.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.614+0000 7fe069200640 1 -- 172.21.15.17:0/2666040246 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe06414a3c0 con 0x7fe0640b2a60 2024-09-10T21:39:12.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.614+0000 7fe05b400640 1 -- 172.21.15.17:0/2666040246 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe054010860 con 0x7fe0640b2a60 2024-09-10T21:39:12.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.614+0000 7fe069200640 1 -- 172.21.15.17:0/2666040246 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe06414a980 con 0x7fe0640b2a60 2024-09-10T21:39:12.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.614+0000 7fe069200640 1 --2- 172.21.15.17:0/2666040246 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe064071190 0x7fe064151840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:39:12.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.614+0000 7fe069200640 1 -- 172.21.15.17:0/2666040246 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fe064069ef0 con 0x7fe064071190 2024-09-10T21:39:12.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.614+0000 7fe062a00640 1 --2- 172.21.15.17:0/2666040246 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe064071190 0x7fe064151840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:39:12.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.614+0000 7fe062a00640 1 -- 172.21.15.17:0/2666040246 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe06415afa0 con 0x7fe064071190 2024-09-10T21:39:12.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.618+0000 7fe062a00640 1 --2- 172.21.15.17:0/2666040246 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe064071190 0x7fe064151840 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fe06406dcb0 tx=0x7fe050005d60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:39:12.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.618+0000 7fe05b400640 1 -- 172.21.15.17:0/2666040246 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0500b2050 con 0x7fe064071190 2024-09-10T21:39:12.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.618+0000 7fe05b400640 1 -- 172.21.15.17:0/2666040246 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fe054010a00 con 0x7fe0640b2a60 2024-09-10T21:39:12.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.618+0000 7fe05b400640 1 --2- 172.21.15.17:0/2666040246 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fe0440724e0 0x7fe0440749a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:39:12.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.618+0000 7fe05b400640 1 -- 172.21.15.17:0/2666040246 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fe054093fb0 con 0x7fe0640b2a60 2024-09-10T21:39:12.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.618+0000 7fe05b400640 1 -- 172.21.15.17:0/2666040246 <== mon.1 v2:172.21.15.31:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fe0500b3040 con 0x7fe064071190 2024-09-10T21:39:12.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.618+0000 7fe05b400640 1 -- 172.21.15.17:0/2666040246 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe064071190 msgr2=0x7fe064151840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:39:12.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.618+0000 7fe05b400640 1 --2- 172.21.15.17:0/2666040246 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe064071190 0x7fe064151840 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fe06406dcb0 tx=0x7fe050005d60 comp rx=0 tx=0).stop 2024-09-10T21:39:12.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.618+0000 7fe05b400640 1 -- 172.21.15.17:0/2666040246 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe064071190 msgr2=0x7fe064151840 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:39:12.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.618+0000 7fe05b400640 1 --2- 172.21.15.17:0/2666040246 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe064071190 0x7fe064151840 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:39:12.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.618+0000 7fe063e00640 1 --2- 172.21.15.17:0/2666040246 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fe0440724e0 0x7fe0440749a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:39:12.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.618+0000 7fe063e00640 1 --2- 172.21.15.17:0/2666040246 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fe0440724e0 0x7fe0440749a0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fe05c0012f0 tx=0x7fe05c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:39:12.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.626+0000 7fe069200640 1 --2- 172.21.15.17:0/2666040246 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe06414ab50 0x7fe064144e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:39:12.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.626+0000 7fe069200640 1 -- 172.21.15.17:0/2666040246 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fe06414bc30 con 0x7fe06414ab50 2024-09-10T21:39:12.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.626+0000 7fe062a00640 1 --2- 172.21.15.17:0/2666040246 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe06414ab50 0x7fe064144e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:39:12.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.626+0000 7fe062a00640 1 -- 172.21.15.17:0/2666040246 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fe0640c3ff0 con 0x7fe06414ab50 2024-09-10T21:39:12.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.626+0000 7fe062a00640 1 --2- 172.21.15.17:0/2666040246 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe06414ab50 0x7fe064144e50 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fe05001a4f0 tx=0x7fe050009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:39:12.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.630+0000 7fe05b400640 1 -- 172.21.15.17:0/2666040246 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0500048e0 con 0x7fe06414ab50 2024-09-10T21:39:12.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.630+0000 7fe05b400640 1 -- 172.21.15.17:0/2666040246 <== mon.1 v2:172.21.15.31:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5044 (secure 0 0 0) 0x7fe0500048e0 con 0x7fe06414ab50 2024-09-10T21:39:12.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.630+0000 7fe05b400640 1 -- 172.21.15.17:0/2666040246 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe06414ab50 msgr2=0x7fe064144e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:39:12.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.630+0000 7fe05b400640 1 --2- 172.21.15.17:0/2666040246 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe06414ab50 0x7fe064144e50 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fe05001a4f0 tx=0x7fe050009260 comp rx=0 tx=0).stop 2024-09-10T21:39:12.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.630+0000 7fe05b400640 1 -- 172.21.15.17:0/2666040246 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe06414ab50 msgr2=0x7fe064144e50 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:39:12.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.630+0000 7fe05b400640 1 --2- 172.21.15.17:0/2666040246 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe06414ab50 0x7fe064144e50 secure :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fe05001a4f0 tx=0x7fe050009260 comp rx=0 tx=0).stop 2024-09-10T21:39:12.633 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-10T21:39:12.633 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:39:12.633 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:39:12.633 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-10T21:39:12.633 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 116, 2024-09-10T21:39:12.633 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 272015, 2024-09-10T21:39:12.633 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-10T21:39:12.633 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-10T21:39:12.633 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-10T21:39:12.633 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-10T21:39:12.633 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:39:12.634 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 14, 2024-09-10T21:39:12.634 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:39:12.634 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-10T21:39:12.634 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-10T21:39:12.634 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:39:12.634 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:39:12.634 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:39:12.634 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:39:12.634 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:39:12.634 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:39:12.634 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:39:12.634 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:39:12.634 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:39:12.634 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:39:12.635 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:39:12.635 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:39:12.635 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-10T21:39:12.635 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-10T21:39:12.635 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:39:12.635 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:39:12.635 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:39:12.635 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:39:12.635 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:39:12.635 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:39:12.635 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:39:12.635 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:39:12.635 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:39:12.636 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:39:12.636 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:39:12.636 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:39:12.636 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:39:12.636 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-10T21:39:12.636 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-10T21:39:12.636 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-10T21:39:12.636 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-10T21:39:12.636 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-10T21:39:12.636 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "5e2f023f-d6a6-49f4-a256-f4807dce3739", 2024-09-10T21:39:12.636 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:39:12.636 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:39:12.637 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-10T21:39:12.637 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-10T21:39:12.637 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-10T21:39:12.637 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-10T21:39:12.637 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-10T21:39:12.637 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-10T21:39:12.637 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-10T21:39:12.637 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:39:12.637 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-10T21:39:12.637 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:39:12.637 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:39:12.637 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:39:12.638 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:39:12.638 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:39:12.638 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:39:12.638 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:39:12.638 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:39:12.638 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:39:12.638 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:39:12.638 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:39:12.638 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:39:12.638 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-10T21:39:12.638 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:39:12.638 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-10T21:39:12.638 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:39:12.639 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:39:12.639 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:39:12.639 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:39:12.639 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:39:12.639 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:39:12.639 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:39:12.639 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-10T21:39:12.639 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:39:12.639 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:39:12.639 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:39:12.639 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:39:12.639 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-10T21:39:12.639 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:39:12.640 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:39:12.640 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:39:12.640 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:39:12.640 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-10T21:39:12.640 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-10T21:39:12.640 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:39:12.640 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:39:12.640 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:39:12.640 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:39:12.640 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:39:12.640 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:39:12.640 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:39:12.640 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:39:12.640 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:39:12.640 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:39:12.641 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:39:12.641 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3300", 2024-09-10T21:39:12.641 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:39:12.641 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:39:12.641 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:39:12.641 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:39:12.641 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789", 2024-09-10T21:39:12.641 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:39:12.641 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:39:12.641 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:39:12.641 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:39:12.641 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789/0", 2024-09-10T21:39:12.641 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6789/0", 2024-09-10T21:39:12.641 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:39:12.641 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:39:12.642 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:39:12.642 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:39:12.642 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:39:12.642 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:39:12.642 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:39:12.642 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:39:12.642 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:39:12.642 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:39:12.642 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:39:12.642 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3301", 2024-09-10T21:39:12.642 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:39:12.642 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:39:12.642 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:39:12.642 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:39:12.643 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790", 2024-09-10T21:39:12.643 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:39:12.643 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:39:12.643 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:39:12.643 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:39:12.643 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790/0", 2024-09-10T21:39:12.643 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6790/0", 2024-09-10T21:39:12.643 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:39:12.643 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:39:12.643 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:39:12.643 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:39:12.643 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:39:12.643 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:39:12.644 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-10T21:39:12.644 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-10T21:39:12.644 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:39:12.644 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:39:12.644 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:39:12.644 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:39:12.644 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:39:12.644 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:39:12.644 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": [ 2024-09-10T21:39:12.644 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:39:12.644 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:39:12.644 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:39:12.644 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 2 2024-09-10T21:39:12.645 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:39:12.645 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:39:12.645 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-10T21:39:12.645 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:39:12.645 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:39:12.645 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:39:12.645 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 4 2024-09-10T21:39:12.645 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:39:12.645 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:39:12.645 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-10T21:39:12.645 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:39:12.645 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:39:12.645 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:39:12.645 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 4 2024-09-10T21:39:12.646 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:39:12.646 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:39:12.646 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": [ 2024-09-10T21:39:12.646 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:39:12.646 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:39:12.646 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:39:12.646 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:39:12.646 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:39:12.646 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:39:12.646 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:39:12.646 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-10T21:39:12.646 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-10T21:39:12.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.630+0000 7fe069200640 1 -- 172.21.15.17:0/2666040246 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fe0440724e0 msgr2=0x7fe0440749a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:39:12.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.630+0000 7fe069200640 1 --2- 172.21.15.17:0/2666040246 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fe0440724e0 0x7fe0440749a0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fe05c0012f0 tx=0x7fe05c009040 comp rx=0 tx=0).stop 2024-09-10T21:39:12.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.630+0000 7fe069200640 1 -- 172.21.15.17:0/2666040246 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe0640b2a60 msgr2=0x7fe064156630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:39:12.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.630+0000 7fe069200640 1 --2- 172.21.15.17:0/2666040246 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe0640b2a60 0x7fe064156630 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fe05400e7d0 tx=0x7fe05400ec90 comp rx=0 tx=0).stop 2024-09-10T21:39:12.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.630+0000 7fe063e00640 1 -- 172.21.15.17:0/2666040246 reap_dead start 2024-09-10T21:39:12.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.630+0000 7fe063e00640 1 -- 172.21.15.17:0/2666040246 reap_dead start 2024-09-10T21:39:12.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.630+0000 7fe069200640 1 -- 172.21.15.17:0/2666040246 shutdown_connections 2024-09-10T21:39:12.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.630+0000 7fe069200640 1 -- 172.21.15.17:0/2666040246 >> 172.21.15.17:0/2666040246 conn(0x7fe064068990 msgr2=0x7fe06414e2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:39:12.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.630+0000 7fe069200640 1 -- 172.21.15.17:0/2666040246 shutdown_connections 2024-09-10T21:39:12.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.630+0000 7fe069200640 1 -- 172.21.15.17:0/2666040246 wait complete. 2024-09-10T21:39:12.656 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-10T21:39:12.656 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-09-10T21:39:12.657 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-10T21:39:12.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.821+0000 7fdb32400640 1 -- 172.21.15.17:0/746133711 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdb2c151250 msgr2=0x7fdb2c0705c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:39:12.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.821+0000 7fdb32400640 1 --2- 172.21.15.17:0/746133711 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdb2c151250 0x7fdb2c0705c0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fdb20009840 tx=0x7fdb2001f010 comp rx=0 tx=0).stop 2024-09-10T21:39:12.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.841+0000 7fdb32400640 1 -- 172.21.15.17:0/746133711 shutdown_connections 2024-09-10T21:39:12.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.841+0000 7fdb32400640 1 --2- 172.21.15.17:0/746133711 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fdb2c0c7ff0 0x7fdb2c070b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:39:12.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.841+0000 7fdb32400640 1 --2- 172.21.15.17:0/746133711 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdb2c151250 0x7fdb2c0705c0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:39:12.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.841+0000 7fdb32400640 1 -- 172.21.15.17:0/746133711 >> 172.21.15.17:0/746133711 conn(0x7fdb2c06cc20 msgr2=0x7fdb2c06d030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:39:12.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.841+0000 7fdb32400640 1 -- 172.21.15.17:0/746133711 shutdown_connections 2024-09-10T21:39:12.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.841+0000 7fdb32400640 1 -- 172.21.15.17:0/746133711 wait complete. 2024-09-10T21:39:12.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.841+0000 7fdb32400640 1 Processor -- start 2024-09-10T21:39:12.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.841+0000 7fdb32400640 1 -- start start 2024-09-10T21:39:12.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.845+0000 7fdb32400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdb2c068d90 0x7fdb2c150470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:39:12.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.845+0000 7fdb32400640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fdb2c0c7ff0 0x7fdb2c070800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:39:12.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.845+0000 7fdb32400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdb2c151250 0x7fdb2c1df510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:39:12.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.845+0000 7fdb32400640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fdb2c0c8840 con 0x7fdb2c068d90 2024-09-10T21:39:12.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.845+0000 7fdb32400640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fdb2c0c86c0 con 0x7fdb2c151250 2024-09-10T21:39:12.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.845+0000 7fdb32400640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7fdb2c072bb0 con 0x7fdb2c0c7ff0 2024-09-10T21:39:12.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.845+0000 7fdb31000640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdb2c068d90 0x7fdb2c150470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:39:12.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.845+0000 7fdb31000640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdb2c068d90 0x7fdb2c150470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:54452/0 (socket says 172.21.15.17:54452) 2024-09-10T21:39:12.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.845+0000 7fdb31000640 1 -- 172.21.15.17:0/3842003437 learned_addr learned my addr 172.21.15.17:0/3842003437 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:39:12.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.845+0000 7fdb31a00640 1 --2- 172.21.15.17:0/3842003437 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdb2c151250 0x7fdb2c1df510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:39:12.849 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.845+0000 7fdb2be00640 1 --2- 172.21.15.17:0/3842003437 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fdb2c0c7ff0 0x7fdb2c070800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:39:12.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.845+0000 7fdb31000640 1 -- 172.21.15.17:0/3842003437 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fdb2c0c7ff0 msgr2=0x7fdb2c070800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:39:12.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.845+0000 7fdb31000640 1 --2- 172.21.15.17:0/3842003437 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fdb2c0c7ff0 0x7fdb2c070800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:39:12.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.845+0000 7fdb31000640 1 -- 172.21.15.17:0/3842003437 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdb2c151250 msgr2=0x7fdb2c1df510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:39:12.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.845+0000 7fdb31000640 1 --2- 172.21.15.17:0/3842003437 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdb2c151250 0x7fdb2c1df510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:39:12.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.845+0000 7fdb31000640 1 -- 172.21.15.17:0/3842003437 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb2c1f2680 con 0x7fdb2c068d90 2024-09-10T21:39:12.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.845+0000 7fdb31000640 1 --2- 172.21.15.17:0/3842003437 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdb2c068d90 0x7fdb2c150470 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fdb2001d6c0 tx=0x7fdb20005db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:39:12.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.845+0000 7fdb28c00640 1 -- 172.21.15.17:0/3842003437 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb200be070 con 0x7fdb2c068d90 2024-09-10T21:39:12.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.845+0000 7fdb28c00640 1 -- 172.21.15.17:0/3842003437 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdb200052f0 con 0x7fdb2c068d90 2024-09-10T21:39:12.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.845+0000 7fdb32400640 1 -- 172.21.15.17:0/3842003437 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb2c1f2960 con 0x7fdb2c068d90 2024-09-10T21:39:12.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.845+0000 7fdb28c00640 1 -- 172.21.15.17:0/3842003437 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb200b5040 con 0x7fdb2c068d90 2024-09-10T21:39:12.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.845+0000 7fdb32400640 1 -- 172.21.15.17:0/3842003437 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb2c14fc70 con 0x7fdb2c068d90 2024-09-10T21:39:12.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.845+0000 7fdb32400640 1 --2- 172.21.15.17:0/3842003437 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fdb2c1dfa50 0x7fdb2c14f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:39:12.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.845+0000 7fdb32400640 1 -- 172.21.15.17:0/3842003437 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7fdb2c0c4700 con 0x7fdb2c1dfa50 2024-09-10T21:39:12.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.845+0000 7fdb2be00640 1 --2- 172.21.15.17:0/3842003437 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fdb2c1dfa50 0x7fdb2c14f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:39:12.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.845+0000 7fdb2be00640 1 -- 172.21.15.17:0/3842003437 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fdb2c0c37e0 con 0x7fdb2c1dfa50 2024-09-10T21:39:12.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.845+0000 7fdb2be00640 1 --2- 172.21.15.17:0/3842003437 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fdb2c1dfa50 0x7fdb2c14f010 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fdb2c0c8c70 tx=0x7fdb1800fc00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:39:12.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.849+0000 7fdb28c00640 1 -- 172.21.15.17:0/3842003437 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb180050d0 con 0x7fdb2c1dfa50 2024-09-10T21:39:12.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.849+0000 7fdb28c00640 1 -- 172.21.15.17:0/3842003437 <== mon.2 v2:172.21.15.31:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fdb18018070 con 0x7fdb2c1dfa50 2024-09-10T21:39:12.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.849+0000 7fdb28c00640 1 -- 172.21.15.17:0/3842003437 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fdb2c1dfa50 msgr2=0x7fdb2c14f010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:39:12.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.849+0000 7fdb28c00640 1 --2- 172.21.15.17:0/3842003437 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fdb2c1dfa50 0x7fdb2c14f010 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fdb2c0c8c70 tx=0x7fdb1800fc00 comp rx=0 tx=0).stop 2024-09-10T21:39:12.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.849+0000 7fdb28c00640 1 -- 172.21.15.17:0/3842003437 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fdb2c1dfa50 msgr2=0x7fdb2c14f010 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:39:12.853 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.849+0000 7fdb28c00640 1 --2- 172.21.15.17:0/3842003437 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fdb2c1dfa50 0x7fdb2c14f010 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:39:12.853 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.849+0000 7fdb28c00640 1 -- 172.21.15.17:0/3842003437 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fdb200b8030 con 0x7fdb2c068d90 2024-09-10T21:39:12.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.849+0000 7fdb28c00640 1 --2- 172.21.15.17:0/3842003437 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fdb0c072080 0x7fdb0c074540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:39:12.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.849+0000 7fdb28c00640 1 -- 172.21.15.17:0/3842003437 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fdb20130100 con 0x7fdb2c068d90 2024-09-10T21:39:12.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.849+0000 7fdb2be00640 1 --2- 172.21.15.17:0/3842003437 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fdb0c072080 0x7fdb0c074540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:39:12.855 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.849+0000 7fdb2be00640 1 --2- 172.21.15.17:0/3842003437 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fdb0c072080 0x7fdb0c074540 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fdb18014000 tx=0x7fdb18005000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:39:12.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.857+0000 7fdb32400640 1 --2- 172.21.15.17:0/3842003437 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fdb2c1584c0 0x7fdb2c149550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:39:12.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.857+0000 7fdb32400640 1 -- 172.21.15.17:0/3842003437 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7fdb2c0c84a0 con 0x7fdb2c1584c0 2024-09-10T21:39:12.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.857+0000 7fdb31a00640 1 --2- 172.21.15.17:0/3842003437 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fdb2c1584c0 0x7fdb2c149550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:39:12.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.857+0000 7fdb31a00640 1 -- 172.21.15.17:0/3842003437 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fdb2c156e90 con 0x7fdb2c1584c0 2024-09-10T21:39:12.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.857+0000 7fdb31a00640 1 --2- 172.21.15.17:0/3842003437 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fdb2c1584c0 0x7fdb2c149550 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fdb2c0c3140 tx=0x7fdb24009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:39:12.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.857+0000 7fdb28c00640 1 -- 172.21.15.17:0/3842003437 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb24003980 con 0x7fdb2c1584c0 2024-09-10T21:39:12.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.857+0000 7fdb28c00640 1 -- 172.21.15.17:0/3842003437 <== mon.2 v2:172.21.15.31:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5044 (secure 0 0 0) 0x7fdb24003980 con 0x7fdb2c1584c0 2024-09-10T21:39:12.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.857+0000 7fdb28c00640 1 -- 172.21.15.17:0/3842003437 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fdb2c1584c0 msgr2=0x7fdb2c149550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:39:12.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.857+0000 7fdb28c00640 1 --2- 172.21.15.17:0/3842003437 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fdb2c1584c0 0x7fdb2c149550 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fdb2c0c3140 tx=0x7fdb24009f90 comp rx=0 tx=0).stop 2024-09-10T21:39:12.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.857+0000 7fdb28c00640 1 -- 172.21.15.17:0/3842003437 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fdb2c1584c0 msgr2=0x7fdb2c149550 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:39:12.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.857+0000 7fdb28c00640 1 --2- 172.21.15.17:0/3842003437 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fdb2c1584c0 0x7fdb2c149550 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:39:12.864 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-10T21:39:12.865 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:39:12.865 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:39:12.865 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-10T21:39:12.865 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 116, 2024-09-10T21:39:12.865 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 159205, 2024-09-10T21:39:12.865 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-10T21:39:12.865 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-10T21:39:12.865 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-10T21:39:12.865 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-10T21:39:12.865 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:39:12.865 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 14, 2024-09-10T21:39:12.865 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:39:12.865 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-10T21:39:12.866 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-10T21:39:12.866 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:39:12.866 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:39:12.866 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:39:12.866 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:39:12.866 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:39:12.866 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:39:12.866 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:39:12.866 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:39:12.866 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:39:12.866 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:39:12.866 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:39:12.866 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:39:12.867 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-10T21:39:12.867 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-10T21:39:12.867 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:39:12.867 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:39:12.867 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:39:12.867 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:39:12.867 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:39:12.867 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:39:12.867 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:39:12.867 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:39:12.867 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:39:12.867 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:39:12.867 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:39:12.867 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:39:12.868 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:39:12.868 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-10T21:39:12.868 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-10T21:39:12.868 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-10T21:39:12.868 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-10T21:39:12.868 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-10T21:39:12.868 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "5e2f023f-d6a6-49f4-a256-f4807dce3739", 2024-09-10T21:39:12.868 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:39:12.868 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:39:12.868 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-10T21:39:12.868 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-10T21:39:12.868 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-10T21:39:12.868 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-10T21:39:12.869 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-10T21:39:12.869 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-10T21:39:12.869 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-10T21:39:12.869 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:39:12.869 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-10T21:39:12.869 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:39:12.869 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:39:12.869 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:39:12.869 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:39:12.869 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:39:12.869 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:39:12.869 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:39:12.870 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:39:12.870 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:39:12.870 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:39:12.870 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:39:12.870 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:39:12.870 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-10T21:39:12.870 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:39:12.870 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-10T21:39:12.870 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:39:12.870 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:39:12.870 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:39:12.870 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:39:12.870 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:39:12.871 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:39:12.871 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:39:12.871 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-10T21:39:12.871 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:39:12.871 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:39:12.871 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:39:12.871 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:39:12.871 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-10T21:39:12.871 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:39:12.871 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:39:12.871 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:39:12.871 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:39:12.871 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-10T21:39:12.871 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-10T21:39:12.871 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:39:12.872 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:39:12.872 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:39:12.872 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:39:12.872 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:39:12.872 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:39:12.872 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:39:12.872 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:39:12.872 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:39:12.872 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:39:12.872 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:39:12.872 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3300", 2024-09-10T21:39:12.872 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:39:12.872 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:39:12.872 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:39:12.873 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:39:12.873 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789", 2024-09-10T21:39:12.873 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:39:12.873 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:39:12.873 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:39:12.873 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:39:12.873 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789/0", 2024-09-10T21:39:12.873 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6789/0", 2024-09-10T21:39:12.873 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:39:12.874 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:39:12.874 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:39:12.874 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:39:12.874 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:39:12.874 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:39:12.874 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:39:12.874 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:39:12.874 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:39:12.874 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:39:12.874 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:39:12.874 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3301", 2024-09-10T21:39:12.875 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:39:12.875 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:39:12.875 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:39:12.875 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:39:12.875 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790", 2024-09-10T21:39:12.875 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:39:12.875 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:39:12.875 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:39:12.876 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:39:12.876 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790/0", 2024-09-10T21:39:12.876 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6790/0", 2024-09-10T21:39:12.876 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:39:12.876 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:39:12.876 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:39:12.876 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:39:12.876 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:39:12.876 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:39:12.876 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-10T21:39:12.876 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-10T21:39:12.876 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:39:12.876 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:39:12.876 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:39:12.877 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:39:12.877 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:39:12.877 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:39:12.877 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": [ 2024-09-10T21:39:12.877 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:39:12.877 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:39:12.877 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:39:12.877 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 4 2024-09-10T21:39:12.877 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:39:12.877 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:39:12.877 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-10T21:39:12.877 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:39:12.877 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:39:12.877 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:39:12.878 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 4 2024-09-10T21:39:12.878 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:39:12.878 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:39:12.878 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-10T21:39:12.878 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:39:12.878 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:39:12.878 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:39:12.878 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 3 2024-09-10T21:39:12.878 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:39:12.878 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:39:12.878 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": [ 2024-09-10T21:39:12.878 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:39:12.878 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:39:12.878 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:39:12.878 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:39:12.879 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:39:12.879 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:39:12.879 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:39:12.879 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-10T21:39:12.879 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-10T21:39:12.879 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.861+0000 7fdb32400640 1 -- 172.21.15.17:0/3842003437 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fdb0c072080 msgr2=0x7fdb0c074540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:39:12.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.861+0000 7fdb32400640 1 --2- 172.21.15.17:0/3842003437 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fdb0c072080 0x7fdb0c074540 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fdb18014000 tx=0x7fdb18005000 comp rx=0 tx=0).stop 2024-09-10T21:39:12.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.861+0000 7fdb32400640 1 -- 172.21.15.17:0/3842003437 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdb2c068d90 msgr2=0x7fdb2c150470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:39:12.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.861+0000 7fdb32400640 1 --2- 172.21.15.17:0/3842003437 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdb2c068d90 0x7fdb2c150470 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fdb2001d6c0 tx=0x7fdb20005db0 comp rx=0 tx=0).stop 2024-09-10T21:39:12.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.861+0000 7fdb31a00640 1 -- 172.21.15.17:0/3842003437 reap_dead start 2024-09-10T21:39:12.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.861+0000 7fdb32400640 1 -- 172.21.15.17:0/3842003437 shutdown_connections 2024-09-10T21:39:12.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.861+0000 7fdb32400640 1 --2- 172.21.15.17:0/3842003437 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdb2c068d90 0x7fdb2c150470 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:39:12.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.861+0000 7fdb32400640 1 -- 172.21.15.17:0/3842003437 >> 172.21.15.17:0/3842003437 conn(0x7fdb2c06cc20 msgr2=0x7fdb2c14e2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:39:12.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.861+0000 7fdb32400640 1 -- 172.21.15.17:0/3842003437 shutdown_connections 2024-09-10T21:39:12.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:12.861+0000 7fdb32400640 1 -- 172.21.15.17:0/3842003437 wait complete. 2024-09-10T21:39:12.890 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-10T21:39:12.891 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['b'] 2024-09-10T21:39:12.891 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2024-09-10T21:39:12.891 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.b 2024-09-10T21:39:12.891 INFO:tasks.mon_thrash.mon_thrasher:killing mon.b 2024-09-10T21:39:12.891 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-10T21:39:12.892 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:39:12.913 INFO:tasks.ceph.mon.b:Stopped 2024-09-10T21:39:12.913 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2024-09-10T21:39:12.913 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-10T21:39:13.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:13.073+0000 7f7160a00640 1 -- 172.21.15.17:0/3432039767 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f715c152440 msgr2=0x7f715c0b3eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:39:13.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:13.073+0000 7f7160a00640 1 --2- 172.21.15.17:0/3432039767 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f715c152440 0x7f715c0b3eb0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f714c009870 tx=0x7f714c01c7d0 comp rx=0 tx=0).stop 2024-09-10T21:39:13.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:13.073+0000 7f7160a00640 1 -- 172.21.15.17:0/3432039767 shutdown_connections 2024-09-10T21:39:13.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:13.073+0000 7f7160a00640 1 --2- 172.21.15.17:0/3432039767 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f715c152440 0x7f715c0b3eb0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:39:13.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:13.073+0000 7f7160a00640 1 -- 172.21.15.17:0/3432039767 >> 172.21.15.17:0/3432039767 conn(0x7f715c0702f0 msgr2=0x7f715c070700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:39:13.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:13.073+0000 7f7160a00640 1 -- 172.21.15.17:0/3432039767 shutdown_connections 2024-09-10T21:39:13.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:13.073+0000 7f7160a00640 1 -- 172.21.15.17:0/3432039767 wait complete. 2024-09-10T21:39:13.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:13.073+0000 7f7160a00640 1 Processor -- start 2024-09-10T21:39:13.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:13.073+0000 7f7160a00640 1 -- start start 2024-09-10T21:39:13.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:13.073+0000 7f7160a00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f715c0cdf20 0x7f715c14a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:39:13.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:13.073+0000 7f7160a00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f715c152440 0x7f715c0c71c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:39:13.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:13.073+0000 7f7160a00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f715c152820 0x7f715c14c760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:39:13.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:13.073+0000 7f7160a00640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f715c0cb450 con 0x7f715c152820 2024-09-10T21:39:13.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:13.073+0000 7f7160a00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f715c0ce740 con 0x7f715c0cdf20 2024-09-10T21:39:13.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:13.073+0000 7f7160a00640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f715c0cb5d0 con 0x7f715c152440 2024-09-10T21:39:13.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:13.073+0000 7f7159e00640 1 -- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f715c0cdf20 msgr2=0x7f715c14a1e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:3300/0 2024-09-10T21:39:13.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:13.073+0000 7f7159e00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f715c0cdf20 0x7f715c14a1e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:39:13.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:13.073+0000 7f715a800640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f715c152820 0x7f715c14c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:39:13.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:13.073+0000 7f715a800640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f715c152820 0x7f715c14c760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:54494/0 (socket says 172.21.15.17:54494) 2024-09-10T21:39:13.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:13.073+0000 7f715a800640 1 -- 172.21.15.17:0/803477886 learned_addr learned my addr 172.21.15.17:0/803477886 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:39:13.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:13.073+0000 7f7159400640 1 --2- 172.21.15.17:0/803477886 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f715c152440 0x7f715c0c71c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:39:13.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:13.073+0000 7f715a800640 1 -- 172.21.15.17:0/803477886 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f715c152440 msgr2=0x7f715c0c71c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:39:13.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:13.073+0000 7f715a800640 1 --2- 172.21.15.17:0/803477886 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f715c152440 0x7f715c0c71c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:39:13.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:13.073+0000 7f715a800640 1 -- 172.21.15.17:0/803477886 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f715c0cdf20 msgr2=0x7f715c14a1e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:39:13.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:13.073+0000 7f715a800640 1 --2- 172.21.15.17:0/803477886 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f715c0cdf20 0x7f715c14a1e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:39:13.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:13.073+0000 7f715a800640 1 -- 172.21.15.17:0/803477886 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f715c14a720 con 0x7f715c152820 2024-09-10T21:39:13.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:13.077+0000 7f715a800640 1 --2- 172.21.15.17:0/803477886 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f715c152820 0x7f715c14c760 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f715400cb10 tx=0x7f715400cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:39:13.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:13.077+0000 7f7152000640 1 -- 172.21.15.17:0/803477886 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7154016020 con 0x7f715c152820 2024-09-10T21:39:13.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:13.077+0000 7f7152000640 1 -- 172.21.15.17:0/803477886 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f715400ce50 con 0x7f715c152820 2024-09-10T21:39:13.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:13.077+0000 7f7160a00640 1 -- 172.21.15.17:0/803477886 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f715c06e330 con 0x7f715c152820 2024-09-10T21:39:13.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:13.077+0000 7f7152000640 1 -- 172.21.15.17:0/803477886 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f715400ae00 con 0x7f715c152820 2024-09-10T21:39:13.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:13.077+0000 7f7160a00640 1 -- 172.21.15.17:0/803477886 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f715c0c63d0 con 0x7f715c152820 2024-09-10T21:39:13.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:13.077+0000 7f7160a00640 1 -- 172.21.15.17:0/803477886 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f715c158360 con 0x7f715c152820 2024-09-10T21:39:13.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:13.077+0000 7f7152000640 1 -- 172.21.15.17:0/803477886 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f7154012420 con 0x7f715c152820 2024-09-10T21:39:13.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:13.077+0000 7f7152000640 1 --2- 172.21.15.17:0/803477886 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f713c072590 0x7f713c074a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:39:13.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:13.077+0000 7f7152000640 1 -- 172.21.15.17:0/803477886 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f7154093bc0 con 0x7f715c152820 2024-09-10T21:39:13.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:13.081+0000 7f7159e00640 1 --2- 172.21.15.17:0/803477886 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f713c072590 0x7f713c074a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:39:13.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:13.081+0000 7f7159e00640 1 --2- 172.21.15.17:0/803477886 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f713c072590 0x7f713c074a50 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f714c0af000 tx=0x7f714c0a7040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:39:13.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:13.085+0000 7f7152000640 1 -- 172.21.15.17:0/803477886 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f7154020020 con 0x7f715c152820 2024-09-10T21:39:13.396 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:13.393+0000 7f7160a00640 1 -- 172.21.15.17:0/803477886 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f715c155d30 con 0x7f715c152820 2024-09-10T21:39:13.396 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:13.393+0000 7f7152000640 1 -- 172.21.15.17:0/803477886 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1558 (secure 0 0 0) 0x7f715405ed80 con 0x7f715c152820 2024-09-10T21:39:13.397 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":116,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":14,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5e2f023f-d6a6-49f4-a256-f4807dce3739","modified":"2024-09-10T21:23:46.892130Z","created":"2024-09-10T21:23:46.892130Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3301","nonce":0},{"type":"v1","addr":"172.21.15.31:6790","nonce":0}]},"addr":"172.21.15.31:6790/0","public_addr":"172.21.15.31:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-10T21:39:13.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:13.393+0000 7f7160a00640 1 -- 172.21.15.17:0/803477886 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f713c072590 msgr2=0x7f713c074a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:39:13.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:13.393+0000 7f7160a00640 1 --2- 172.21.15.17:0/803477886 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f713c072590 0x7f713c074a50 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f714c0af000 tx=0x7f714c0a7040 comp rx=0 tx=0).stop 2024-09-10T21:39:13.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:13.393+0000 7f7160a00640 1 -- 172.21.15.17:0/803477886 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f715c152820 msgr2=0x7f715c14c760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:39:13.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:13.393+0000 7f7160a00640 1 --2- 172.21.15.17:0/803477886 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f715c152820 0x7f715c14c760 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f715400cb10 tx=0x7f715400cc00 comp rx=0 tx=0).stop 2024-09-10T21:39:13.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:13.393+0000 7f7160a00640 1 -- 172.21.15.17:0/803477886 shutdown_connections 2024-09-10T21:39:13.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:13.393+0000 7f7160a00640 1 --2- 172.21.15.17:0/803477886 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f713c072590 0x7f713c074a50 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:39:13.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:13.393+0000 7f7160a00640 1 --2- 172.21.15.17:0/803477886 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f715c152820 0x7f715c14c760 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:39:13.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:13.393+0000 7f7160a00640 1 --2- 172.21.15.17:0/803477886 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f715c152440 0x7f715c0c71c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:39:13.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:13.393+0000 7f7160a00640 1 --2- 172.21.15.17:0/803477886 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f715c0cdf20 0x7f715c14a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:39:13.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:13.393+0000 7f7160a00640 1 -- 172.21.15.17:0/803477886 >> 172.21.15.17:0/803477886 conn(0x7f715c0702f0 msgr2=0x7f715c0c9a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:39:13.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:13.393+0000 7f7160a00640 1 -- 172.21.15.17:0/803477886 shutdown_connections 2024-09-10T21:39:13.400 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:13.393+0000 7f7160a00640 1 -- 172.21.15.17:0/803477886 wait complete. 2024-09-10T21:39:16.427 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-10T21:39:16.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:16.585+0000 7f527ce00640 1 -- 172.21.15.17:0/427283164 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f52780ce1f0 msgr2=0x7f5278070e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:39:16.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:16.585+0000 7f527ce00640 1 --2- 172.21.15.17:0/427283164 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f52780ce1f0 0x7f5278070e30 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f5264009900 tx=0x7f526401c870 comp rx=0 tx=0).stop 2024-09-10T21:39:16.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:16.585+0000 7f527ce00640 1 -- 172.21.15.17:0/427283164 shutdown_connections 2024-09-10T21:39:16.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:16.585+0000 7f527ce00640 1 --2- 172.21.15.17:0/427283164 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f52780ce1f0 0x7f5278070e30 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:39:16.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:16.585+0000 7f527ce00640 1 --2- 172.21.15.17:0/427283164 >> v2:172.21.15.31:3300/0 conn(0x7f52780cde10 0x7f527815ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:39:16.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:16.585+0000 7f527ce00640 1 -- 172.21.15.17:0/427283164 >> 172.21.15.17:0/427283164 conn(0x7f5278068990 msgr2=0x7f5278068da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:39:16.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:16.585+0000 7f527ce00640 1 -- 172.21.15.17:0/427283164 shutdown_connections 2024-09-10T21:39:16.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:16.585+0000 7f527ce00640 1 -- 172.21.15.17:0/427283164 wait complete. 2024-09-10T21:39:16.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:16.589+0000 7f527ce00640 1 Processor -- start 2024-09-10T21:39:16.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:16.589+0000 7f527ce00640 1 -- start start 2024-09-10T21:39:16.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:16.589+0000 7f527ce00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f52780cde10 0x7f5278147880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:39:16.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:16.589+0000 7f527ce00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52780ce1f0 0x7f5278147dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:39:16.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:16.589+0000 7f527ce00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f527815b9e0 0x7f52781f3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:39:16.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:16.589+0000 7f527ce00640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f52781549b0 con 0x7f52780cde10 2024-09-10T21:39:16.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:16.589+0000 7f527ce00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f52780ce630 con 0x7f52780ce1f0 2024-09-10T21:39:16.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:16.589+0000 7f527ce00640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f52781510c0 con 0x7f527815b9e0 2024-09-10T21:39:16.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:16.589+0000 7f5276a00640 1 -- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52780ce1f0 msgr2=0x7f5278147dc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:3300/0 2024-09-10T21:39:16.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:16.589+0000 7f5276a00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52780ce1f0 0x7f5278147dc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:39:16.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:16.589+0000 7f5277400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f52780cde10 0x7f5278147880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:39:16.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:16.589+0000 7f5277400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f52780cde10 0x7f5278147880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35030/0 (socket says 172.21.15.17:35030) 2024-09-10T21:39:16.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:16.589+0000 7f5277400640 1 -- 172.21.15.17:0/2629075571 learned_addr learned my addr 172.21.15.17:0/2629075571 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:39:16.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:16.589+0000 7f5277e00640 1 --2- 172.21.15.17:0/2629075571 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f527815b9e0 0x7f52781f3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:39:16.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:16.589+0000 7f5277400640 1 -- 172.21.15.17:0/2629075571 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f527815b9e0 msgr2=0x7f52781f3740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:39:16.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:16.589+0000 7f5277400640 1 --2- 172.21.15.17:0/2629075571 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f527815b9e0 0x7f52781f3740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:39:16.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:16.589+0000 7f5277400640 1 -- 172.21.15.17:0/2629075571 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52780ce1f0 msgr2=0x7f5278147dc0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:39:16.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:16.589+0000 7f5277400640 1 --2- 172.21.15.17:0/2629075571 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52780ce1f0 0x7f5278147dc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:39:16.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:16.589+0000 7f5277400640 1 -- 172.21.15.17:0/2629075571 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52781f7240 con 0x7f52780cde10 2024-09-10T21:39:16.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:16.589+0000 7f5277400640 1 --2- 172.21.15.17:0/2629075571 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f52780cde10 0x7f5278147880 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f526800b4d0 tx=0x7f526800b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:39:16.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:16.589+0000 7f526f400640 1 -- 172.21.15.17:0/2629075571 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5268014070 con 0x7f52780cde10 2024-09-10T21:39:16.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:16.589+0000 7f526f400640 1 -- 172.21.15.17:0/2629075571 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5268010980 con 0x7f52780cde10 2024-09-10T21:39:16.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:16.589+0000 7f527ce00640 1 -- 172.21.15.17:0/2629075571 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52781f74e0 con 0x7f52780cde10 2024-09-10T21:39:16.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:16.589+0000 7f526f400640 1 -- 172.21.15.17:0/2629075571 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5268010dd0 con 0x7f52780cde10 2024-09-10T21:39:16.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:16.589+0000 7f527ce00640 1 -- 172.21.15.17:0/2629075571 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f527814ca40 con 0x7f52780cde10 2024-09-10T21:39:16.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:16.593+0000 7f527ce00640 1 -- 172.21.15.17:0/2629075571 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f527815ae70 con 0x7f52780cde10 2024-09-10T21:39:16.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:16.593+0000 7f526f400640 1 -- 172.21.15.17:0/2629075571 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f526800f3c0 con 0x7f52780cde10 2024-09-10T21:39:16.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:16.593+0000 7f526f400640 1 --2- 172.21.15.17:0/2629075571 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f5258072590 0x7f5258074a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:39:16.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:16.593+0000 7f526f400640 1 -- 172.21.15.17:0/2629075571 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f52680937c0 con 0x7f52780cde10 2024-09-10T21:39:16.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:16.593+0000 7f5276a00640 1 --2- 172.21.15.17:0/2629075571 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f5258072590 0x7f5258074a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:39:16.600 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:16.597+0000 7f5276a00640 1 --2- 172.21.15.17:0/2629075571 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f5258072590 0x7f5258074a50 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f52640af000 tx=0x7f526401b0b0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:39:16.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:16.601+0000 7f526f400640 1 -- 172.21.15.17:0/2629075571 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f526805e330 con 0x7f52780cde10 2024-09-10T21:39:16.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:16.873+0000 7f527ce00640 1 -- 172.21.15.17:0/2629075571 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f52780c9bb0 con 0x7f52780cde10 2024-09-10T21:39:16.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:16.873+0000 7f526f400640 1 -- 172.21.15.17:0/2629075571 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1558 (secure 0 0 0) 0x7f5268062d60 con 0x7f52780cde10 2024-09-10T21:39:16.877 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":116,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":18,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5e2f023f-d6a6-49f4-a256-f4807dce3739","modified":"2024-09-10T21:23:46.892130Z","created":"2024-09-10T21:23:46.892130Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3301","nonce":0},{"type":"v1","addr":"172.21.15.31:6790","nonce":0}]},"addr":"172.21.15.31:6790/0","public_addr":"172.21.15.31:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-10T21:39:16.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:16.873+0000 7f527ce00640 1 -- 172.21.15.17:0/2629075571 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f5258072590 msgr2=0x7f5258074a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:39:16.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:16.873+0000 7f527ce00640 1 --2- 172.21.15.17:0/2629075571 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f5258072590 0x7f5258074a50 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f52640af000 tx=0x7f526401b0b0 comp rx=0 tx=0).stop 2024-09-10T21:39:16.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:16.873+0000 7f527ce00640 1 -- 172.21.15.17:0/2629075571 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f52780cde10 msgr2=0x7f5278147880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:39:16.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:16.873+0000 7f527ce00640 1 --2- 172.21.15.17:0/2629075571 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f52780cde10 0x7f5278147880 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f526800b4d0 tx=0x7f526800b990 comp rx=0 tx=0).stop 2024-09-10T21:39:16.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:16.873+0000 7f527ce00640 1 -- 172.21.15.17:0/2629075571 shutdown_connections 2024-09-10T21:39:16.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:16.873+0000 7f527ce00640 1 --2- 172.21.15.17:0/2629075571 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f5258072590 0x7f5258074a50 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:39:16.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:16.873+0000 7f527ce00640 1 --2- 172.21.15.17:0/2629075571 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f527815b9e0 0x7f52781f3740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:39:16.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:16.873+0000 7f527ce00640 1 --2- 172.21.15.17:0/2629075571 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f52780ce1f0 0x7f5278147dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:39:16.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:16.873+0000 7f527ce00640 1 --2- 172.21.15.17:0/2629075571 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f52780cde10 0x7f5278147880 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:39:16.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:16.873+0000 7f527ce00640 1 -- 172.21.15.17:0/2629075571 >> 172.21.15.17:0/2629075571 conn(0x7f5278068990 msgr2=0x7f5278159b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:39:16.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:16.877+0000 7f527ce00640 1 -- 172.21.15.17:0/2629075571 shutdown_connections 2024-09-10T21:39:16.881 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:16.877+0000 7f527ce00640 1 -- 172.21.15.17:0/2629075571 wait complete. 2024-09-10T21:39:19.905 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-10T21:39:20.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:20.069+0000 7fc519000640 1 -- 172.21.15.17:0/4160886903 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc51406fd00 msgr2=0x7fc5140c7870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:39:20.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:20.069+0000 7fc519000640 1 --2- 172.21.15.17:0/4160886903 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc51406fd00 0x7fc5140c7870 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fc50c009bc0 tx=0x7fc50c01f3d0 comp rx=0 tx=0).stop 2024-09-10T21:39:20.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:20.069+0000 7fc519000640 1 -- 172.21.15.17:0/4160886903 shutdown_connections 2024-09-10T21:39:20.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:20.069+0000 7fc519000640 1 --2- 172.21.15.17:0/4160886903 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc51406fd00 0x7fc5140c7870 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:39:20.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:20.069+0000 7fc519000640 1 --2- 172.21.15.17:0/4160886903 >> v2:172.21.15.31:3300/0 conn(0x7fc5141527e0 0x7fc514158310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:39:20.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:20.069+0000 7fc519000640 1 -- 172.21.15.17:0/4160886903 >> 172.21.15.17:0/4160886903 conn(0x7fc514070ca0 msgr2=0x7fc5140710b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:39:20.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:20.069+0000 7fc519000640 1 -- 172.21.15.17:0/4160886903 shutdown_connections 2024-09-10T21:39:20.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:20.069+0000 7fc519000640 1 -- 172.21.15.17:0/4160886903 wait complete. 2024-09-10T21:39:20.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:20.069+0000 7fc519000640 1 Processor -- start 2024-09-10T21:39:20.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:20.069+0000 7fc519000640 1 -- start start 2024-09-10T21:39:20.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:20.073+0000 7fc519000640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc51406fd00 0x7fc5140c20f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:39:20.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:20.073+0000 7fc519000640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc5140b2d80 0x7fc5141f1260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:39:20.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:20.073+0000 7fc519000640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc5141527e0 0x7fc514159a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:39:20.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:20.073+0000 7fc519000640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fc51406f500 con 0x7fc5140b2d80 2024-09-10T21:39:20.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:20.073+0000 7fc519000640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fc51406f220 con 0x7fc5141527e0 2024-09-10T21:39:20.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:20.073+0000 7fc519000640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7fc51406ed40 con 0x7fc51406fd00 2024-09-10T21:39:20.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:20.073+0000 7fc513e00640 1 -- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc5141527e0 msgr2=0x7fc514159a40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:3300/0 2024-09-10T21:39:20.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:20.073+0000 7fc513e00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc5141527e0 0x7fc514159a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:39:20.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:20.073+0000 7fc512a00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc5140b2d80 0x7fc5141f1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:39:20.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:20.073+0000 7fc513400640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc51406fd00 0x7fc5140c20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:39:20.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:20.073+0000 7fc512a00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc5140b2d80 0x7fc5141f1260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35032/0 (socket says 172.21.15.17:35032) 2024-09-10T21:39:20.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:20.073+0000 7fc512a00640 1 -- 172.21.15.17:0/1865570190 learned_addr learned my addr 172.21.15.17:0/1865570190 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:39:20.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:20.073+0000 7fc513400640 1 -- 172.21.15.17:0/1865570190 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc5141527e0 msgr2=0x7fc514159a40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:39:20.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:20.073+0000 7fc513400640 1 --2- 172.21.15.17:0/1865570190 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc5141527e0 0x7fc514159a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:39:20.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:20.073+0000 7fc513400640 1 -- 172.21.15.17:0/1865570190 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc5140b2d80 msgr2=0x7fc5141f1260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:39:20.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:20.073+0000 7fc513400640 1 --2- 172.21.15.17:0/1865570190 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc5140b2d80 0x7fc5141f1260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:39:20.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:20.073+0000 7fc513400640 1 -- 172.21.15.17:0/1865570190 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5141594d0 con 0x7fc51406fd00 2024-09-10T21:39:20.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:20.073+0000 7fc512a00640 1 --2- 172.21.15.17:0/1865570190 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc5140b2d80 0x7fc5141f1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:39:20.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:20.073+0000 7fc513400640 1 --2- 172.21.15.17:0/1865570190 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc51406fd00 0x7fc5140c20f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fc50400b450 tx=0x7fc50400b910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:39:20.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:20.073+0000 7fc50b400640 1 -- 172.21.15.17:0/1865570190 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc504015020 con 0x7fc51406fd00 2024-09-10T21:39:20.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:20.073+0000 7fc50b400640 1 -- 172.21.15.17:0/1865570190 <== mon.2 v2:172.21.15.31:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc504011980 con 0x7fc51406fd00 2024-09-10T21:39:20.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:20.073+0000 7fc50b400640 1 -- 172.21.15.17:0/1865570190 <== mon.2 v2:172.21.15.31:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc504011dd0 con 0x7fc51406fd00 2024-09-10T21:39:20.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:20.073+0000 7fc519000640 1 -- 172.21.15.17:0/1865570190 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc51415bc00 con 0x7fc51406fd00 2024-09-10T21:39:20.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:20.073+0000 7fc519000640 1 -- 172.21.15.17:0/1865570190 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fc514155af0 con 0x7fc51406fd00 2024-09-10T21:39:20.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:20.073+0000 7fc519000640 1 -- 172.21.15.17:0/1865570190 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc514158310 con 0x7fc51406fd00 2024-09-10T21:39:20.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:20.077+0000 7fc50b400640 1 -- 172.21.15.17:0/1865570190 <== mon.2 v2:172.21.15.31:3301/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fc50401a400 con 0x7fc51406fd00 2024-09-10T21:39:20.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:20.077+0000 7fc50b400640 1 --2- 172.21.15.17:0/1865570190 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fc4f4072590 0x7fc4f4074a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:39:20.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:20.077+0000 7fc50b400640 1 -- 172.21.15.17:0/1865570190 <== mon.2 v2:172.21.15.31:3301/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fc504093a60 con 0x7fc51406fd00 2024-09-10T21:39:20.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:20.077+0000 7fc512a00640 1 --2- 172.21.15.17:0/1865570190 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fc4f4072590 0x7fc4f4074a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:39:20.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:20.081+0000 7fc512a00640 1 --2- 172.21.15.17:0/1865570190 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fc4f4072590 0x7fc4f4074a50 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fc514155aa0 tx=0x7fc500007f60 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:39:20.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:20.081+0000 7fc50b400640 1 -- 172.21.15.17:0/1865570190 <== mon.2 v2:172.21.15.31:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fc50401f020 con 0x7fc51406fd00 2024-09-10T21:39:20.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:20.353+0000 7fc519000640 1 -- 172.21.15.17:0/1865570190 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fc5140cadb0 con 0x7fc51406fd00 2024-09-10T21:39:20.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:20.353+0000 7fc50b400640 1 -- 172.21.15.17:0/1865570190 <== mon.2 v2:172.21.15.31:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1558 (secure 0 0 0) 0x7fc50400d040 con 0x7fc51406fd00 2024-09-10T21:39:20.359 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":116,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":21,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5e2f023f-d6a6-49f4-a256-f4807dce3739","modified":"2024-09-10T21:23:46.892130Z","created":"2024-09-10T21:23:46.892130Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3301","nonce":0},{"type":"v1","addr":"172.21.15.31:6790","nonce":0}]},"addr":"172.21.15.31:6790/0","public_addr":"172.21.15.31:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-10T21:39:20.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:20.357+0000 7fc519000640 1 -- 172.21.15.17:0/1865570190 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fc4f4072590 msgr2=0x7fc4f4074a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:39:20.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:20.357+0000 7fc519000640 1 --2- 172.21.15.17:0/1865570190 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fc4f4072590 0x7fc4f4074a50 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fc514155aa0 tx=0x7fc500007f60 comp rx=0 tx=0).stop 2024-09-10T21:39:20.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:20.357+0000 7fc519000640 1 -- 172.21.15.17:0/1865570190 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc51406fd00 msgr2=0x7fc5140c20f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:39:20.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:20.357+0000 7fc519000640 1 --2- 172.21.15.17:0/1865570190 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc51406fd00 0x7fc5140c20f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fc50400b450 tx=0x7fc50400b910 comp rx=0 tx=0).stop 2024-09-10T21:39:20.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:20.357+0000 7fc519000640 1 -- 172.21.15.17:0/1865570190 shutdown_connections 2024-09-10T21:39:20.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:20.357+0000 7fc519000640 1 --2- 172.21.15.17:0/1865570190 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fc4f4072590 0x7fc4f4074a50 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:39:20.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:20.357+0000 7fc519000640 1 --2- 172.21.15.17:0/1865570190 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc5141527e0 0x7fc514159a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:39:20.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:20.357+0000 7fc519000640 1 --2- 172.21.15.17:0/1865570190 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc5140b2d80 0x7fc5141f1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:39:20.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:20.357+0000 7fc519000640 1 --2- 172.21.15.17:0/1865570190 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc51406fd00 0x7fc5140c20f0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:39:20.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:20.357+0000 7fc519000640 1 -- 172.21.15.17:0/1865570190 >> 172.21.15.17:0/1865570190 conn(0x7fc514070ca0 msgr2=0x7fc514071080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:39:20.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:20.357+0000 7fc519000640 1 -- 172.21.15.17:0/1865570190 shutdown_connections 2024-09-10T21:39:20.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:20.357+0000 7fc519000640 1 -- 172.21.15.17:0/1865570190 wait complete. 2024-09-10T21:39:22.629 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:39:22.632 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:39:23.390 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-10T21:39:27.732 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:27.725+0000 7f16b5a00640 1 -- 172.21.15.17:0/1216894834 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1694004810 msgr2=0x7f1694004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:39:27.732 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:27.725+0000 7f16b5a00640 1 --2- 172.21.15.17:0/1216894834 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1694004810 0x7f1694004c90 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f16b00ced10 tx=0x7f16a001ccf0 comp rx=0 tx=0).stop 2024-09-10T21:39:27.732 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:27.725+0000 7f16b5a00640 1 -- 172.21.15.17:0/1216894834 shutdown_connections 2024-09-10T21:39:27.732 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:27.725+0000 7f16b5a00640 1 --2- 172.21.15.17:0/1216894834 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f16940052f0 0x7f169400b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:39:27.732 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:27.725+0000 7f16b5a00640 1 --2- 172.21.15.17:0/1216894834 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1694004810 0x7f1694004c90 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:39:27.732 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:27.725+0000 7f16b5a00640 1 --2- 172.21.15.17:0/1216894834 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16940039c0 0x7f1694003dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:39:27.732 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:27.725+0000 7f16b5a00640 1 -- 172.21.15.17:0/1216894834 >> 172.21.15.17:0/1216894834 conn(0x7f16b0074ac0 msgr2=0x7f16b0068990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:39:27.732 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:27.725+0000 7f16b5a00640 1 -- 172.21.15.17:0/1216894834 shutdown_connections 2024-09-10T21:39:27.732 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:27.729+0000 7f16b5a00640 1 -- 172.21.15.17:0/1216894834 wait complete. 2024-09-10T21:39:27.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:27.729+0000 7f16b5a00640 1 Processor -- start 2024-09-10T21:39:27.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:27.729+0000 7f16b5a00640 1 -- start start 2024-09-10T21:39:27.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:27.729+0000 7f16b5a00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f16940039c0 0x7f16b00c5650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:39:27.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:27.729+0000 7f16b5a00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1694004810 0x7f16b014bc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:39:27.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:27.729+0000 7f16b5a00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f16940052f0 0x7f16b01595c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:39:27.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:27.729+0000 7f16b5a00640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f16b00706a0 con 0x7f16940052f0 2024-09-10T21:39:27.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:27.729+0000 7f16b5a00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f16b0152740 con 0x7f1694004810 2024-09-10T21:39:27.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:27.729+0000 7f16b5a00640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f16b00c32b0 con 0x7f16940039c0 2024-09-10T21:39:27.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:27.729+0000 7f16af400640 1 -- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1694004810 msgr2=0x7f16b014bc80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:3300/0 2024-09-10T21:39:27.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:27.729+0000 7f16af400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1694004810 0x7f16b014bc80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:39:27.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:27.729+0000 7f16afe00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f16940039c0 0x7f16b00c5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:39:27.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:27.729+0000 7f16b5000640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f16940052f0 0x7f16b01595c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:39:27.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:27.729+0000 7f16afe00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f16940039c0 0x7f16b00c5650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3301/0 says I am v2:172.21.15.17:44866/0 (socket says 172.21.15.17:44866) 2024-09-10T21:39:27.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:27.729+0000 7f16afe00640 1 -- 172.21.15.17:0/1167301177 learned_addr learned my addr 172.21.15.17:0/1167301177 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:39:27.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:27.733+0000 7f16b5000640 1 -- 172.21.15.17:0/1167301177 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f16940039c0 msgr2=0x7f16b00c5650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:39:27.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:27.733+0000 7f16b5000640 1 --2- 172.21.15.17:0/1167301177 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f16940039c0 0x7f16b00c5650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:39:27.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:27.733+0000 7f16b5000640 1 -- 172.21.15.17:0/1167301177 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1694004810 msgr2=0x7f16b014bc80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:39:27.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:27.733+0000 7f16b5000640 1 --2- 172.21.15.17:0/1167301177 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1694004810 0x7f16b014bc80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:39:27.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:27.733+0000 7f16b5000640 1 -- 172.21.15.17:0/1167301177 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16b01575c0 con 0x7f16940052f0 2024-09-10T21:39:27.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:27.733+0000 7f16b5000640 1 --2- 172.21.15.17:0/1167301177 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f16940052f0 0x7f16b01595c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f16b00ceca0 tx=0x7f16a800fad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:39:27.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:27.733+0000 7f16a7e00640 1 -- 172.21.15.17:0/1167301177 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16a8004d30 con 0x7f16940052f0 2024-09-10T21:39:27.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:27.733+0000 7f16b5a00640 1 -- 172.21.15.17:0/1167301177 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16b015a470 con 0x7f16940052f0 2024-09-10T21:39:27.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:27.733+0000 7f16b5a00640 1 -- 172.21.15.17:0/1167301177 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16b00c4b10 con 0x7f16940052f0 2024-09-10T21:39:27.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:27.733+0000 7f16a7e00640 1 -- 172.21.15.17:0/1167301177 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f16a800e900 con 0x7f16940052f0 2024-09-10T21:39:27.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:27.733+0000 7f16a7e00640 1 -- 172.21.15.17:0/1167301177 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16a800ed30 con 0x7f16940052f0 2024-09-10T21:39:27.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:27.733+0000 7f16b5a00640 1 -- 172.21.15.17:0/1167301177 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16880052a0 con 0x7f16940052f0 2024-09-10T21:39:27.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:27.737+0000 7f16a7e00640 1 -- 172.21.15.17:0/1167301177 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f16a800b040 con 0x7f16940052f0 2024-09-10T21:39:27.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:27.737+0000 7f16a7e00640 1 --2- 172.21.15.17:0/1167301177 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f1690072ce0 0x7f16900751a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:39:27.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:27.737+0000 7f16a7e00640 1 -- 172.21.15.17:0/1167301177 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f16a8093f80 con 0x7f16940052f0 2024-09-10T21:39:27.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:27.737+0000 7f16afe00640 1 --2- 172.21.15.17:0/1167301177 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f1690072ce0 0x7f16900751a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:39:27.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:27.741+0000 7f16afe00640 1 --2- 172.21.15.17:0/1167301177 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f1690072ce0 0x7f16900751a0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f169c0010e0 tx=0x7f169c00b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:39:27.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:27.741+0000 7f16a7e00640 1 -- 172.21.15.17:0/1167301177 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f16a805f340 con 0x7f16940052f0 2024-09-10T21:39:28.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:28.025+0000 7f16b5a00640 1 -- 172.21.15.17:0/1167301177 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f1688005590 con 0x7f16940052f0 2024-09-10T21:39:28.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:28.025+0000 7f16a7e00640 1 -- 172.21.15.17:0/1167301177 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1551 (secure 0 0 0) 0x7f16a805ec40 con 0x7f16940052f0 2024-09-10T21:39:28.028 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":120,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5e2f023f-d6a6-49f4-a256-f4807dce3739","modified":"2024-09-10T21:23:46.892130Z","created":"2024-09-10T21:23:46.892130Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3301","nonce":0},{"type":"v1","addr":"172.21.15.31:6790","nonce":0}]},"addr":"172.21.15.31:6790/0","public_addr":"172.21.15.31:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-10T21:39:28.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:28.025+0000 7f16b5a00640 1 -- 172.21.15.17:0/1167301177 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f1690072ce0 msgr2=0x7f16900751a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:39:28.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:28.025+0000 7f16b5a00640 1 --2- 172.21.15.17:0/1167301177 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f1690072ce0 0x7f16900751a0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f169c0010e0 tx=0x7f169c00b040 comp rx=0 tx=0).stop 2024-09-10T21:39:28.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:28.025+0000 7f16b5a00640 1 -- 172.21.15.17:0/1167301177 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f16940052f0 msgr2=0x7f16b01595c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:39:28.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:28.025+0000 7f16b5a00640 1 --2- 172.21.15.17:0/1167301177 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f16940052f0 0x7f16b01595c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f16b00ceca0 tx=0x7f16a800fad0 comp rx=0 tx=0).stop 2024-09-10T21:39:28.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:28.025+0000 7f16b5a00640 1 -- 172.21.15.17:0/1167301177 shutdown_connections 2024-09-10T21:39:28.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:28.025+0000 7f16b5a00640 1 --2- 172.21.15.17:0/1167301177 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f1690072ce0 0x7f16900751a0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:39:28.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:28.025+0000 7f16b5a00640 1 --2- 172.21.15.17:0/1167301177 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f16940052f0 0x7f16b01595c0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:39:28.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:28.025+0000 7f16b5a00640 1 --2- 172.21.15.17:0/1167301177 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1694004810 0x7f16b014bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:39:28.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:28.025+0000 7f16b5a00640 1 --2- 172.21.15.17:0/1167301177 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f16940039c0 0x7f16b00c5650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:39:28.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:28.025+0000 7f16b5a00640 1 -- 172.21.15.17:0/1167301177 >> 172.21.15.17:0/1167301177 conn(0x7f16b0074ac0 msgr2=0x7f16b01560a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:39:28.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:28.025+0000 7f16b5a00640 1 -- 172.21.15.17:0/1167301177 shutdown_connections 2024-09-10T21:39:28.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:28.025+0000 7f16b5a00640 1 -- 172.21.15.17:0/1167301177 wait complete. 2024-09-10T21:39:28.052 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2024-09-10T21:39:28.052 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon a 2024-09-10T21:39:28.052 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-10T21:39:28.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:28.213+0000 7f69a9000640 1 -- 172.21.15.17:0/820709819 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f69880052f0 msgr2=0x7f698800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:39:28.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:28.213+0000 7f69a9000640 1 --2- 172.21.15.17:0/820709819 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f69880052f0 0x7f698800b9d0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f699c0013e0 tx=0x7f699c01c950 comp rx=0 tx=0).stop 2024-09-10T21:39:28.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:28.213+0000 7f69a9000640 1 -- 172.21.15.17:0/820709819 shutdown_connections 2024-09-10T21:39:28.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:28.213+0000 7f69a9000640 1 --2- 172.21.15.17:0/820709819 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f69880052f0 0x7f698800b9d0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:39:28.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:28.213+0000 7f69a9000640 1 -- 172.21.15.17:0/820709819 >> 172.21.15.17:0/820709819 conn(0x7f69a406cc20 msgr2=0x7f69a406d030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:39:28.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:28.213+0000 7f69a9000640 1 -- 172.21.15.17:0/820709819 shutdown_connections 2024-09-10T21:39:28.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:28.213+0000 7f69a9000640 1 -- 172.21.15.17:0/820709819 wait complete. 2024-09-10T21:39:28.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:28.213+0000 7f69a9000640 1 Processor -- start 2024-09-10T21:39:28.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:28.213+0000 7f69a9000640 1 -- start start 2024-09-10T21:39:28.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:28.217+0000 7f69a9000640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f69880052f0 0x7f69a414d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:39:28.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:28.217+0000 7f69a9000640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f69a4159ed0 0x7f69a41578e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:39:28.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:28.217+0000 7f69a9000640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f69a4149e10 0x7f69a414a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:39:28.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:28.217+0000 7f69a9000640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f69a41529e0 con 0x7f69880052f0 2024-09-10T21:39:28.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:28.217+0000 7f69a9000640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f69a4152860 con 0x7f69a4149e10 2024-09-10T21:39:28.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:28.217+0000 7f69a9000640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f69a4068a40 con 0x7f69a4159ed0 2024-09-10T21:39:28.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:28.217+0000 7f69a3400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f69880052f0 0x7f69a414d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:39:28.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:28.217+0000 7f69a3e00640 1 -- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f69a4149e10 msgr2=0x7f69a414a290 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:3300/0 2024-09-10T21:39:28.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:28.217+0000 7f69a3e00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f69a4149e10 0x7f69a414a290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:39:28.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:28.217+0000 7f69a3400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f69880052f0 0x7f69a414d8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41176/0 (socket says 172.21.15.17:41176) 2024-09-10T21:39:28.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:28.217+0000 7f69a3400640 1 -- 172.21.15.17:0/3656975156 learned_addr learned my addr 172.21.15.17:0/3656975156 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:39:28.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:28.217+0000 7f69a2a00640 1 --2- 172.21.15.17:0/3656975156 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f69a4159ed0 0x7f69a41578e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:39:28.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:28.217+0000 7f69a3400640 1 -- 172.21.15.17:0/3656975156 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f69a4159ed0 msgr2=0x7f69a41578e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:39:28.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:28.217+0000 7f69a3400640 1 --2- 172.21.15.17:0/3656975156 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f69a4159ed0 0x7f69a41578e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:39:28.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:28.217+0000 7f69a3400640 1 -- 172.21.15.17:0/3656975156 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f69a4149e10 msgr2=0x7f69a414a290 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:39:28.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:28.217+0000 7f69a3400640 1 --2- 172.21.15.17:0/3656975156 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f69a4149e10 0x7f69a414a290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:39:28.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:28.217+0000 7f69a3400640 1 -- 172.21.15.17:0/3656975156 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69a415a780 con 0x7f69880052f0 2024-09-10T21:39:28.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:28.217+0000 7f69a2a00640 1 --2- 172.21.15.17:0/3656975156 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f69a4159ed0 0x7f69a41578e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:39:28.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:28.217+0000 7f69a3400640 1 --2- 172.21.15.17:0/3656975156 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f69880052f0 0x7f69a414d8b0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f6994004fd0 tx=0x7f699400ea20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:39:28.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:28.217+0000 7f699b400640 1 -- 172.21.15.17:0/3656975156 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6994015070 con 0x7f69880052f0 2024-09-10T21:39:28.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:28.217+0000 7f699b400640 1 -- 172.21.15.17:0/3656975156 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f699400c9e0 con 0x7f69880052f0 2024-09-10T21:39:28.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:28.217+0000 7f699b400640 1 -- 172.21.15.17:0/3656975156 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f699400ce30 con 0x7f69880052f0 2024-09-10T21:39:28.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:28.217+0000 7f69a9000640 1 -- 172.21.15.17:0/3656975156 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69a4156170 con 0x7f69880052f0 2024-09-10T21:39:28.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:28.217+0000 7f69a9000640 1 -- 172.21.15.17:0/3656975156 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69a41f3fb0 con 0x7f69880052f0 2024-09-10T21:39:28.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:28.217+0000 7f69a9000640 1 --2- 172.21.15.17:0/3656975156 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f69a41f0430 0x7f69a41f0810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:39:28.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:28.217+0000 7f69a9000640 1 -- 172.21.15.17:0/3656975156 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f69a40c8f30 con 0x7f69a41f0430 2024-09-10T21:39:28.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:28.217+0000 7f69a2a00640 1 --2- 172.21.15.17:0/3656975156 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f69a41f0430 0x7f69a41f0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:39:28.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:28.217+0000 7f69a2a00640 1 -- 172.21.15.17:0/3656975156 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f69a414edc0 con 0x7f69a41f0430 2024-09-10T21:39:28.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:28.221+0000 7f69a2a00640 1 --2- 172.21.15.17:0/3656975156 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f69a41f0430 0x7f69a41f0810 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f69a414e100 tx=0x7f699000db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:39:28.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:28.221+0000 7f699b400640 1 -- 172.21.15.17:0/3656975156 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f699000b700 con 0x7f69a41f0430 2024-09-10T21:39:28.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:28.221+0000 7f699b400640 1 -- 172.21.15.17:0/3656975156 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f6994011460 con 0x7f69880052f0 2024-09-10T21:39:28.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:28.221+0000 7f699b400640 1 --2- 172.21.15.17:0/3656975156 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f6984072510 0x7f69840749d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:39:28.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:28.221+0000 7f699b400640 1 -- 172.21.15.17:0/3656975156 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f6994093d20 con 0x7f69880052f0 2024-09-10T21:39:28.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:28.221+0000 7f699b400640 1 -- 172.21.15.17:0/3656975156 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f699000b8a0 con 0x7f69a41f0430 2024-09-10T21:39:28.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:28.221+0000 7f699b400640 1 -- 172.21.15.17:0/3656975156 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f69a41f0430 msgr2=0x7f69a41f0810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:39:28.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:28.221+0000 7f699b400640 1 --2- 172.21.15.17:0/3656975156 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f69a41f0430 0x7f69a41f0810 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f69a414e100 tx=0x7f699000db90 comp rx=0 tx=0).stop 2024-09-10T21:39:28.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:28.221+0000 7f699b400640 1 -- 172.21.15.17:0/3656975156 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f69a41f0430 msgr2=0x7f69a41f0810 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:39:28.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:28.221+0000 7f699b400640 1 --2- 172.21.15.17:0/3656975156 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f69a41f0430 0x7f69a41f0810 secure :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f69a414e100 tx=0x7f699000db90 comp rx=0 tx=0).stop 2024-09-10T21:39:28.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:28.221+0000 7f69a3e00640 1 --2- 172.21.15.17:0/3656975156 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f6984072510 0x7f69840749d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:39:28.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:28.221+0000 7f69a3e00640 1 --2- 172.21.15.17:0/3656975156 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f6984072510 0x7f69840749d0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f698800c1c0 tx=0x7f699c0196f0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:39:28.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:28.229+0000 7f69a9000640 1 --2- 172.21.15.17:0/3656975156 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f69a4073c00 0x7f69a4073fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:39:28.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:28.229+0000 7f69a9000640 1 -- 172.21.15.17:0/3656975156 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f69a414cbb0 con 0x7f69a4073c00 2024-09-10T21:39:28.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:28.229+0000 7f69a2a00640 1 --2- 172.21.15.17:0/3656975156 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f69a4073c00 0x7f69a4073fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:39:28.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:28.233+0000 7f69a2a00640 1 -- 172.21.15.17:0/3656975156 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f69a414eae0 con 0x7f69a4073c00 2024-09-10T21:39:28.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:28.233+0000 7f69a2a00640 1 --2- 172.21.15.17:0/3656975156 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f69a4073c00 0x7f69a4073fe0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f6990006e80 tx=0x7f6990004810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:39:28.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:28.233+0000 7f699b400640 1 -- 172.21.15.17:0/3656975156 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6990014640 con 0x7f69a4073c00 2024-09-10T21:39:28.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:28.233+0000 7f699b400640 1 -- 172.21.15.17:0/3656975156 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4699 (secure 0 0 0) 0x7f6990027430 con 0x7f69a4073c00 2024-09-10T21:39:28.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:28.233+0000 7f699b400640 1 -- 172.21.15.17:0/3656975156 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f69a4073c00 msgr2=0x7f69a4073fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:39:28.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:28.233+0000 7f699b400640 1 --2- 172.21.15.17:0/3656975156 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f69a4073c00 0x7f69a4073fe0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f6990006e80 tx=0x7f6990004810 comp rx=0 tx=0).stop 2024-09-10T21:39:28.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:28.233+0000 7f699b400640 1 -- 172.21.15.17:0/3656975156 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f69a4073c00 msgr2=0x7f69a4073fe0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:39:28.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:28.233+0000 7f699b400640 1 --2- 172.21.15.17:0/3656975156 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f69a4073c00 0x7f69a4073fe0 secure :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f6990006e80 tx=0x7f6990004810 comp rx=0 tx=0).stop 2024-09-10T21:39:28.237 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-10T21:39:28.238 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:39:28.238 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:39:28.238 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "leader", 2024-09-10T21:39:28.238 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 120, 2024-09-10T21:39:28.238 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 42435, 2024-09-10T21:39:28.238 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-10T21:39:28.238 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-10T21:39:28.238 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-10T21:39:28.238 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:39:28.238 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 0, 2024-09-10T21:39:28.238 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:39:28.238 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-10T21:39:28.239 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-10T21:39:28.239 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:39:28.239 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:39:28.239 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:39:28.239 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:39:28.239 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:39:28.239 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:39:28.239 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:39:28.239 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:39:28.239 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:39:28.239 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:39:28.239 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:39:28.239 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:39:28.239 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-10T21:39:28.239 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-10T21:39:28.240 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:39:28.240 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:39:28.240 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:39:28.240 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:39:28.240 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:39:28.240 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:39:28.240 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:39:28.240 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:39:28.240 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:39:28.240 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:39:28.240 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:39:28.240 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:39:28.240 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:39:28.241 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-10T21:39:28.241 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-10T21:39:28.241 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-10T21:39:28.241 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-10T21:39:28.241 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-10T21:39:28.241 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "5e2f023f-d6a6-49f4-a256-f4807dce3739", 2024-09-10T21:39:28.241 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:39:28.241 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:39:28.241 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-10T21:39:28.241 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-10T21:39:28.241 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-10T21:39:28.241 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-10T21:39:28.241 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-10T21:39:28.241 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-10T21:39:28.242 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-10T21:39:28.242 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:39:28.242 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-10T21:39:28.242 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:39:28.242 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:39:28.242 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:39:28.242 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:39:28.242 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:39:28.242 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:39:28.242 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:39:28.242 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:39:28.242 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:39:28.242 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:39:28.243 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:39:28.243 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:39:28.243 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-10T21:39:28.243 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:39:28.243 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-10T21:39:28.243 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:39:28.243 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:39:28.243 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:39:28.243 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:39:28.243 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:39:28.243 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:39:28.243 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:39:28.243 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-10T21:39:28.244 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:39:28.244 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:39:28.244 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:39:28.244 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:39:28.244 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-10T21:39:28.244 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:39:28.244 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:39:28.244 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:39:28.244 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:39:28.244 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-10T21:39:28.244 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-10T21:39:28.244 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:39:28.244 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:39:28.244 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:39:28.244 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:39:28.245 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:39:28.245 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:39:28.245 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:39:28.245 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:39:28.245 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:39:28.245 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:39:28.245 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:39:28.245 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3300", 2024-09-10T21:39:28.245 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:39:28.245 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:39:28.245 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:39:28.245 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:39:28.245 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789", 2024-09-10T21:39:28.245 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:39:28.246 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:39:28.246 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:39:28.246 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:39:28.246 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789/0", 2024-09-10T21:39:28.246 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6789/0", 2024-09-10T21:39:28.246 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:39:28.246 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:39:28.246 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:39:28.246 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:39:28.246 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:39:28.246 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:39:28.246 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:39:28.246 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:39:28.246 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:39:28.246 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:39:28.247 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:39:28.247 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3301", 2024-09-10T21:39:28.247 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:39:28.247 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:39:28.247 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:39:28.247 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:39:28.247 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790", 2024-09-10T21:39:28.247 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:39:28.247 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:39:28.247 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:39:28.248 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:39:28.248 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790/0", 2024-09-10T21:39:28.248 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6790/0", 2024-09-10T21:39:28.248 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:39:28.248 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:39:28.248 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:39:28.248 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:39:28.248 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:39:28.248 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:39:28.248 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-10T21:39:28.248 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-10T21:39:28.248 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:39:28.249 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:39:28.249 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:39:28.249 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:39:28.249 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:39:28.249 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:39:28.249 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-10T21:39:28.249 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:39:28.249 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:39:28.249 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:39:28.249 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 2 2024-09-10T21:39:28.249 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:39:28.249 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:39:28.249 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-10T21:39:28.249 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:39:28.249 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:39:28.249 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:39:28.250 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 4 2024-09-10T21:39:28.250 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:39:28.250 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:39:28.250 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:39:28.250 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-10T21:39:28.250 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-10T21:39:28.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:28.233+0000 7f69a9000640 1 -- 172.21.15.17:0/3656975156 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f6984072510 msgr2=0x7f69840749d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:39:28.251 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:28.233+0000 7f69a9000640 1 --2- 172.21.15.17:0/3656975156 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f6984072510 0x7f69840749d0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f698800c1c0 tx=0x7f699c0196f0 comp rx=0 tx=0).stop 2024-09-10T21:39:28.251 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:28.233+0000 7f69a9000640 1 -- 172.21.15.17:0/3656975156 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f69880052f0 msgr2=0x7f69a414d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:39:28.251 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:28.233+0000 7f69a9000640 1 --2- 172.21.15.17:0/3656975156 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f69880052f0 0x7f69a414d8b0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f6994004fd0 tx=0x7f699400ea20 comp rx=0 tx=0).stop 2024-09-10T21:39:28.251 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:28.233+0000 7f69a3e00640 1 -- 172.21.15.17:0/3656975156 reap_dead start 2024-09-10T21:39:28.251 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:28.233+0000 7f69a3e00640 1 -- 172.21.15.17:0/3656975156 reap_dead start 2024-09-10T21:39:28.251 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:28.233+0000 7f69a9000640 1 -- 172.21.15.17:0/3656975156 shutdown_connections 2024-09-10T21:39:28.251 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:28.233+0000 7f69a9000640 1 -- 172.21.15.17:0/3656975156 >> 172.21.15.17:0/3656975156 conn(0x7f69a406cc20 msgr2=0x7f69a40cbad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:39:28.251 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:28.233+0000 7f69a9000640 1 -- 172.21.15.17:0/3656975156 shutdown_connections 2024-09-10T21:39:28.251 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:28.233+0000 7f69a9000640 1 -- 172.21.15.17:0/3656975156 wait complete. 2024-09-10T21:39:28.257 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-09-10T21:39:28.257 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon c 2024-09-10T21:39:28.257 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-10T21:39:28.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:28.425+0000 7fd201a00640 1 -- 172.21.15.17:0/3399609607 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd1e00039c0 msgr2=0x7fd1e0003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:39:28.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:28.425+0000 7fd201a00640 1 --2- 172.21.15.17:0/3399609607 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd1e00039c0 0x7fd1e0003dc0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fd1ec001060 tx=0x7fd1ec01c7a0 comp rx=0 tx=0).stop 2024-09-10T21:39:28.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:28.425+0000 7fd201a00640 1 -- 172.21.15.17:0/3399609607 shutdown_connections 2024-09-10T21:39:28.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:28.425+0000 7fd201a00640 1 --2- 172.21.15.17:0/3399609607 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd1e00039c0 0x7fd1e0003dc0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:39:28.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:28.425+0000 7fd201a00640 1 -- 172.21.15.17:0/3399609607 >> 172.21.15.17:0/3399609607 conn(0x7fd1fc06e470 msgr2=0x7fd1fc06ed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:39:28.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:28.425+0000 7fd201a00640 1 -- 172.21.15.17:0/3399609607 shutdown_connections 2024-09-10T21:39:28.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:28.425+0000 7fd201a00640 1 -- 172.21.15.17:0/3399609607 wait complete. 2024-09-10T21:39:28.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:28.425+0000 7fd201a00640 1 Processor -- start 2024-09-10T21:39:28.430 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:28.425+0000 7fd201a00640 1 -- start start 2024-09-10T21:39:28.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:28.425+0000 7fd201a00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd1e00039c0 0x7fd1fc15cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:39:28.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:28.425+0000 7fd201a00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd1fc1e7550 0x7fd1fc15d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:39:28.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:28.425+0000 7fd201a00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd1fc156a20 0x7fd1fc156ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:39:28.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:28.425+0000 7fd201a00640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fd1fc0d0480 con 0x7fd1fc1e7550 2024-09-10T21:39:28.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:28.425+0000 7fd201a00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fd1fc0d0300 con 0x7fd1fc156a20 2024-09-10T21:39:28.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:28.425+0000 7fd201a00640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7fd1fc0d0600 con 0x7fd1e00039c0 2024-09-10T21:39:28.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:28.429+0000 7fd201000640 1 -- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd1fc156a20 msgr2=0x7fd1fc156ed0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:3300/0 2024-09-10T21:39:28.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:28.429+0000 7fd1fb400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd1fc1e7550 0x7fd1fc15d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:39:28.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:28.429+0000 7fd1fb400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd1fc1e7550 0x7fd1fc15d2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:41216/0 (socket says 172.21.15.17:41216) 2024-09-10T21:39:28.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:28.429+0000 7fd1fb400640 1 -- 172.21.15.17:0/4082223302 learned_addr learned my addr 172.21.15.17:0/4082223302 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:39:28.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:28.429+0000 7fd201000640 1 --2- 172.21.15.17:0/4082223302 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd1fc156a20 0x7fd1fc156ed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:39:28.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:28.429+0000 7fd1fbe00640 1 --2- 172.21.15.17:0/4082223302 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd1e00039c0 0x7fd1fc15cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:39:28.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:28.429+0000 7fd1fb400640 1 -- 172.21.15.17:0/4082223302 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd1e00039c0 msgr2=0x7fd1fc15cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:39:28.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:28.429+0000 7fd1fb400640 1 --2- 172.21.15.17:0/4082223302 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd1e00039c0 0x7fd1fc15cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:39:28.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:28.429+0000 7fd1fb400640 1 -- 172.21.15.17:0/4082223302 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd1fc156a20 msgr2=0x7fd1fc156ed0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:39:28.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:28.429+0000 7fd1fb400640 1 --2- 172.21.15.17:0/4082223302 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd1fc156a20 0x7fd1fc156ed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:39:28.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:28.429+0000 7fd1fb400640 1 -- 172.21.15.17:0/4082223302 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1fc157410 con 0x7fd1fc1e7550 2024-09-10T21:39:28.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:28.429+0000 7fd1fb400640 1 --2- 172.21.15.17:0/4082223302 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd1fc1e7550 0x7fd1fc15d2b0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd1e800d5c0 tx=0x7fd1e800da80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:39:28.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:28.429+0000 7fd1f3e00640 1 -- 172.21.15.17:0/4082223302 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1e8017070 con 0x7fd1fc1e7550 2024-09-10T21:39:28.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:28.429+0000 7fd1f3e00640 1 -- 172.21.15.17:0/4082223302 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd1e800f980 con 0x7fd1fc1e7550 2024-09-10T21:39:28.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:28.429+0000 7fd201a00640 1 -- 172.21.15.17:0/4082223302 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1fc1e7a40 con 0x7fd1fc1e7550 2024-09-10T21:39:28.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:28.429+0000 7fd1f3e00640 1 -- 172.21.15.17:0/4082223302 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1e800fdd0 con 0x7fd1fc1e7550 2024-09-10T21:39:28.435 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:28.429+0000 7fd201a00640 1 -- 172.21.15.17:0/4082223302 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1fc0c4110 con 0x7fd1fc1e7550 2024-09-10T21:39:28.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:28.429+0000 7fd201a00640 1 --2- 172.21.15.17:0/4082223302 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd1fc0c4310 0x7fd1fc0c5d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:39:28.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:28.429+0000 7fd201a00640 1 -- 172.21.15.17:0/4082223302 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7fd1fc069ef0 con 0x7fd1fc0c4310 2024-09-10T21:39:28.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:28.429+0000 7fd1fbe00640 1 --2- 172.21.15.17:0/4082223302 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd1fc0c4310 0x7fd1fc0c5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:39:28.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:28.429+0000 7fd1fbe00640 1 -- 172.21.15.17:0/4082223302 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd1fc0ccb70 con 0x7fd1fc0c4310 2024-09-10T21:39:28.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:28.429+0000 7fd1fbe00640 1 --2- 172.21.15.17:0/4082223302 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd1fc0c4310 0x7fd1fc0c5d80 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fd1e00057f0 tx=0x7fd1ec004f20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:39:28.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:28.429+0000 7fd1f3e00640 1 -- 172.21.15.17:0/4082223302 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1ec004f90 con 0x7fd1fc0c4310 2024-09-10T21:39:28.436 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:28.429+0000 7fd1f3e00640 1 -- 172.21.15.17:0/4082223302 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fd1e8010460 con 0x7fd1fc1e7550 2024-09-10T21:39:28.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:28.433+0000 7fd1f3e00640 1 --2- 172.21.15.17:0/4082223302 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fd1dc072510 0x7fd1dc0749d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:39:28.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:28.433+0000 7fd1f3e00640 1 -- 172.21.15.17:0/4082223302 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fd1e8093770 con 0x7fd1fc1e7550 2024-09-10T21:39:28.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:28.433+0000 7fd1f3e00640 1 -- 172.21.15.17:0/4082223302 <== mon.2 v2:172.21.15.31:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fd1ec005130 con 0x7fd1fc0c4310 2024-09-10T21:39:28.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:28.433+0000 7fd1f3e00640 1 -- 172.21.15.17:0/4082223302 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd1fc0c4310 msgr2=0x7fd1fc0c5d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:39:28.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:28.433+0000 7fd1f3e00640 1 --2- 172.21.15.17:0/4082223302 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd1fc0c4310 0x7fd1fc0c5d80 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fd1e00057f0 tx=0x7fd1ec004f20 comp rx=0 tx=0).stop 2024-09-10T21:39:28.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:28.433+0000 7fd201000640 1 --2- 172.21.15.17:0/4082223302 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fd1dc072510 0x7fd1dc0749d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:39:28.437 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:28.433+0000 7fd1f3e00640 1 -- 172.21.15.17:0/4082223302 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd1fc0c4310 msgr2=0x7fd1fc0c5d80 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:39:28.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:28.433+0000 7fd1f3e00640 1 --2- 172.21.15.17:0/4082223302 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd1fc0c4310 0x7fd1fc0c5d80 secure :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fd1e00057f0 tx=0x7fd1ec004f20 comp rx=0 tx=0).stop 2024-09-10T21:39:28.438 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:28.433+0000 7fd201000640 1 --2- 172.21.15.17:0/4082223302 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fd1dc072510 0x7fd1dc0749d0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fd1fc0b58d0 tx=0x7fd1f4001280 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:39:28.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:28.441+0000 7fd201a00640 1 --2- 172.21.15.17:0/4082223302 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd1fc0c62c0 0x7fd1fc1f6870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:39:28.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:28.441+0000 7fd201a00640 1 -- 172.21.15.17:0/4082223302 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7fd1fc069b30 con 0x7fd1fc0c62c0 2024-09-10T21:39:28.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:28.441+0000 7fd1fbe00640 1 --2- 172.21.15.17:0/4082223302 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd1fc0c62c0 0x7fd1fc1f6870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:39:28.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:28.441+0000 7fd1fbe00640 1 -- 172.21.15.17:0/4082223302 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fd1fc0c37c0 con 0x7fd1fc0c62c0 2024-09-10T21:39:28.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:28.441+0000 7fd1fbe00640 1 --2- 172.21.15.17:0/4082223302 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd1fc0c62c0 0x7fd1fc1f6870 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd1ec001060 tx=0x7fd1ec005eb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:39:28.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:28.441+0000 7fd1f3e00640 1 -- 172.21.15.17:0/4082223302 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1ec0bb8b0 con 0x7fd1fc0c62c0 2024-09-10T21:39:28.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:28.441+0000 7fd1f3e00640 1 -- 172.21.15.17:0/4082223302 <== mon.2 v2:172.21.15.31:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5032 (secure 0 0 0) 0x7fd1ec0c8400 con 0x7fd1fc0c62c0 2024-09-10T21:39:28.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:28.441+0000 7fd1f3e00640 1 -- 172.21.15.17:0/4082223302 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd1fc0c62c0 msgr2=0x7fd1fc1f6870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:39:28.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:28.441+0000 7fd1f3e00640 1 --2- 172.21.15.17:0/4082223302 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd1fc0c62c0 0x7fd1fc1f6870 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd1ec001060 tx=0x7fd1ec005eb0 comp rx=0 tx=0).stop 2024-09-10T21:39:28.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:28.441+0000 7fd1f3e00640 1 -- 172.21.15.17:0/4082223302 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd1fc0c62c0 msgr2=0x7fd1fc1f6870 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:39:28.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:28.441+0000 7fd1f3e00640 1 --2- 172.21.15.17:0/4082223302 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd1fc0c62c0 0x7fd1fc1f6870 secure :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd1ec001060 tx=0x7fd1ec005eb0 comp rx=0 tx=0).stop 2024-09-10T21:39:28.447 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-10T21:39:28.447 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:39:28.447 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:39:28.447 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-10T21:39:28.447 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 120, 2024-09-10T21:39:28.447 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 174788, 2024-09-10T21:39:28.447 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-10T21:39:28.447 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-10T21:39:28.447 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-10T21:39:28.447 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:39:28.447 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 0, 2024-09-10T21:39:28.447 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:39:28.447 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-10T21:39:28.447 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-10T21:39:28.448 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:39:28.448 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:39:28.448 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:39:28.448 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:39:28.448 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:39:28.448 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:39:28.448 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:39:28.448 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:39:28.448 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:39:28.448 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:39:28.449 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:39:28.449 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:39:28.449 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-10T21:39:28.449 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-10T21:39:28.449 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:39:28.449 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:39:28.449 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:39:28.449 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:39:28.450 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:39:28.450 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:39:28.450 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:39:28.450 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:39:28.450 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:39:28.450 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:39:28.450 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:39:28.450 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:39:28.450 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:39:28.450 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-10T21:39:28.450 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-10T21:39:28.450 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-10T21:39:28.450 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-10T21:39:28.451 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-10T21:39:28.451 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "5e2f023f-d6a6-49f4-a256-f4807dce3739", 2024-09-10T21:39:28.451 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:39:28.451 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:39:28.451 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-10T21:39:28.451 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-10T21:39:28.451 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-10T21:39:28.451 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-10T21:39:28.451 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-10T21:39:28.451 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-10T21:39:28.451 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-10T21:39:28.451 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:39:28.451 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-10T21:39:28.451 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:39:28.451 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:39:28.452 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:39:28.452 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:39:28.452 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:39:28.452 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:39:28.452 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:39:28.452 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:39:28.452 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:39:28.452 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:39:28.452 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:39:28.452 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:39:28.452 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-10T21:39:28.452 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:39:28.452 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-10T21:39:28.453 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:39:28.453 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:39:28.453 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:39:28.453 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:39:28.453 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:39:28.453 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:39:28.453 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:39:28.453 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-10T21:39:28.453 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:39:28.453 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:39:28.453 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:39:28.453 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:39:28.453 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-10T21:39:28.453 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:39:28.454 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:39:28.454 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:39:28.454 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:39:28.454 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-10T21:39:28.454 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-10T21:39:28.454 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:39:28.454 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:39:28.454 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:39:28.454 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:39:28.454 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:39:28.454 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:39:28.454 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:39:28.454 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:39:28.454 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:39:28.454 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:39:28.455 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:39:28.455 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3300", 2024-09-10T21:39:28.455 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:39:28.455 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:39:28.455 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:39:28.455 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:39:28.455 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789", 2024-09-10T21:39:28.455 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:39:28.455 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:39:28.455 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:39:28.455 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:39:28.455 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789/0", 2024-09-10T21:39:28.455 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6789/0", 2024-09-10T21:39:28.455 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:39:28.455 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:39:28.456 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:39:28.456 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:39:28.456 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:39:28.456 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:39:28.456 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:39:28.456 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:39:28.456 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:39:28.456 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:39:28.456 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:39:28.456 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3301", 2024-09-10T21:39:28.456 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:39:28.456 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:39:28.456 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:39:28.456 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:39:28.456 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790", 2024-09-10T21:39:28.457 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:39:28.457 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:39:28.457 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:39:28.457 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:39:28.457 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790/0", 2024-09-10T21:39:28.457 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6790/0", 2024-09-10T21:39:28.457 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:39:28.457 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:39:28.457 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:39:28.457 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:39:28.457 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:39:28.457 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:39:28.457 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-10T21:39:28.458 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-10T21:39:28.458 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:39:28.458 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:39:28.458 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:39:28.458 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:39:28.458 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:39:28.458 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:39:28.458 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": [ 2024-09-10T21:39:28.458 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:39:28.458 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:39:28.458 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:39:28.458 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 6 2024-09-10T21:39:28.458 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:39:28.458 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:39:28.458 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-10T21:39:28.459 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:39:28.459 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:39:28.459 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:39:28.459 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 6 2024-09-10T21:39:28.459 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:39:28.459 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:39:28.459 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-10T21:39:28.459 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:39:28.459 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:39:28.459 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:39:28.459 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 4 2024-09-10T21:39:28.459 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:39:28.459 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:39:28.459 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": [ 2024-09-10T21:39:28.459 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:39:28.459 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:39:28.460 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:39:28.460 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 2 2024-09-10T21:39:28.460 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:39:28.460 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:39:28.460 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:39:28.460 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-10T21:39:28.460 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-10T21:39:28.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:28.441+0000 7fd201a00640 1 -- 172.21.15.17:0/4082223302 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fd1dc072510 msgr2=0x7fd1dc0749d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:39:28.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:28.441+0000 7fd201a00640 1 --2- 172.21.15.17:0/4082223302 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fd1dc072510 0x7fd1dc0749d0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fd1fc0b58d0 tx=0x7fd1f4001280 comp rx=0 tx=0).stop 2024-09-10T21:39:28.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:28.441+0000 7fd201a00640 1 -- 172.21.15.17:0/4082223302 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd1fc1e7550 msgr2=0x7fd1fc15d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:39:28.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:28.441+0000 7fd201a00640 1 --2- 172.21.15.17:0/4082223302 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd1fc1e7550 0x7fd1fc15d2b0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd1e800d5c0 tx=0x7fd1e800da80 comp rx=0 tx=0).stop 2024-09-10T21:39:28.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:28.441+0000 7fd201000640 1 -- 172.21.15.17:0/4082223302 reap_dead start 2024-09-10T21:39:28.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:28.441+0000 7fd201000640 1 -- 172.21.15.17:0/4082223302 reap_dead start 2024-09-10T21:39:28.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:28.441+0000 7fd201a00640 1 -- 172.21.15.17:0/4082223302 shutdown_connections 2024-09-10T21:39:28.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:28.441+0000 7fd201a00640 1 -- 172.21.15.17:0/4082223302 >> 172.21.15.17:0/4082223302 conn(0x7fd1fc06e470 msgr2=0x7fd1fc0d0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:39:28.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:28.441+0000 7fd201a00640 1 -- 172.21.15.17:0/4082223302 shutdown_connections 2024-09-10T21:39:28.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:28.441+0000 7fd201a00640 1 -- 172.21.15.17:0/4082223302 wait complete. 2024-09-10T21:39:28.468 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-09-10T21:39:28.468 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2024-09-10T21:39:48.469 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.b 2024-09-10T21:39:48.469 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-10T21:39:48.469 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-10T21:39:48.473 INFO:tasks.ceph.mon.b:Started 2024-09-10T21:39:48.473 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-09-10T21:39:48.473 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-10T21:39:48.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:48.625+0000 7f2b44c00640 1 -- 172.21.15.17:0/2203245281 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f2b40151350 msgr2=0x7f2b400c9100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:39:48.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:48.625+0000 7f2b44c00640 1 --2- 172.21.15.17:0/2203245281 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f2b40151350 0x7f2b400c9100 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f2b2c009900 tx=0x7f2b2c01c850 comp rx=0 tx=0).stop 2024-09-10T21:39:48.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:48.625+0000 7f2b44c00640 1 -- 172.21.15.17:0/2203245281 shutdown_connections 2024-09-10T21:39:48.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:48.625+0000 7f2b44c00640 1 --2- 172.21.15.17:0/2203245281 >> v2:172.21.15.17:3300/0 conn(0x7f2b400c9640 0x7f2b4015a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:39:48.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:48.625+0000 7f2b44c00640 1 --2- 172.21.15.17:0/2203245281 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f2b40151350 0x7f2b400c9100 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:39:48.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:48.625+0000 7f2b44c00640 1 -- 172.21.15.17:0/2203245281 >> 172.21.15.17:0/2203245281 conn(0x7f2b4006cdf0 msgr2=0x7f2b4006d200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:39:48.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:48.625+0000 7f2b44c00640 1 -- 172.21.15.17:0/2203245281 shutdown_connections 2024-09-10T21:39:48.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:48.625+0000 7f2b44c00640 1 -- 172.21.15.17:0/2203245281 wait complete. 2024-09-10T21:39:48.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:48.625+0000 7f2b44c00640 1 Processor -- start 2024-09-10T21:39:48.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:48.625+0000 7f2b44c00640 1 -- start start 2024-09-10T21:39:48.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:48.625+0000 7f2b44c00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2b400c9640 0x7f2b4014b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:39:48.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:48.629+0000 7f2b44c00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2b40150f70 0x7f2b4014e500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:39:48.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:48.629+0000 7f2b44c00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f2b40151350 0x7f2b40144bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:39:48.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:48.629+0000 7f2b44c00640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f2b400c8470 con 0x7f2b400c9640 2024-09-10T21:39:48.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:48.629+0000 7f2b44c00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f2b400c82f0 con 0x7f2b40150f70 2024-09-10T21:39:48.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:48.629+0000 7f2b44c00640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f2b400c6dd0 con 0x7f2b40151350 2024-09-10T21:39:48.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:48.629+0000 7f2b3f400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2b400c9640 0x7f2b4014b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:39:48.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:48.629+0000 7f2b3f400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2b400c9640 0x7f2b4014b360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:34744/0 (socket says 172.21.15.17:34744) 2024-09-10T21:39:48.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:48.629+0000 7f2b3f400640 1 -- 172.21.15.17:0/4109168903 learned_addr learned my addr 172.21.15.17:0/4109168903 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:39:48.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:48.629+0000 7f2b3fe00640 1 --2- 172.21.15.17:0/4109168903 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f2b40151350 0x7f2b40144bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:39:48.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:48.629+0000 7f2b3ea00640 1 --2- 172.21.15.17:0/4109168903 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2b40150f70 0x7f2b4014e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:39:48.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:48.629+0000 7f2b3ea00640 1 -- 172.21.15.17:0/4109168903 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2b40150f70 msgr2=0x7f2b4014e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-09-10T21:39:48.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:48.629+0000 7f2b3ea00640 1 -- 172.21.15.17:0/4109168903 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2b40150f70 msgr2=0x7f2b4014e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-10T21:39:48.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:48.629+0000 7f2b3ea00640 1 --2- 172.21.15.17:0/4109168903 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2b40150f70 0x7f2b4014e500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T21:39:48.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:48.629+0000 7f2b3ea00640 1 --2- 172.21.15.17:0/4109168903 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2b40150f70 0x7f2b4014e500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:39:48.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:48.629+0000 7f2b3f400640 1 -- 172.21.15.17:0/4109168903 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f2b40151350 msgr2=0x7f2b40144bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:39:48.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:48.629+0000 7f2b3f400640 1 --2- 172.21.15.17:0/4109168903 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f2b40151350 0x7f2b40144bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:39:48.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:48.629+0000 7f2b3f400640 1 -- 172.21.15.17:0/4109168903 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2b40150f70 msgr2=0x7f2b4014e500 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:39:48.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:48.629+0000 7f2b3f400640 1 --2- 172.21.15.17:0/4109168903 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2b40150f70 0x7f2b4014e500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:39:48.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:48.629+0000 7f2b3f400640 1 -- 172.21.15.17:0/4109168903 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b4014dd40 con 0x7f2b400c9640 2024-09-10T21:39:48.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:48.629+0000 7f2b3f400640 1 --2- 172.21.15.17:0/4109168903 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2b400c9640 0x7f2b4014b360 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f2b3000c500 tx=0x7f2b3000c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:39:48.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:48.629+0000 7f2b37400640 1 -- 172.21.15.17:0/4109168903 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b30017070 con 0x7f2b400c9640 2024-09-10T21:39:48.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:48.629+0000 7f2b37400640 1 -- 172.21.15.17:0/4109168903 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2b300053c0 con 0x7f2b400c9640 2024-09-10T21:39:48.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:48.629+0000 7f2b44c00640 1 -- 172.21.15.17:0/4109168903 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b4014eea0 con 0x7f2b400c9640 2024-09-10T21:39:48.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:48.629+0000 7f2b37400640 1 -- 172.21.15.17:0/4109168903 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b30003860 con 0x7f2b400c9640 2024-09-10T21:39:48.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:48.629+0000 7f2b44c00640 1 -- 172.21.15.17:0/4109168903 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b40145690 con 0x7f2b400c9640 2024-09-10T21:39:48.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:48.629+0000 7f2b44c00640 1 -- 172.21.15.17:0/4109168903 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b401f45c0 con 0x7f2b400c9640 2024-09-10T21:39:48.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:48.633+0000 7f2b37400640 1 -- 172.21.15.17:0/4109168903 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f2b30003a00 con 0x7f2b400c9640 2024-09-10T21:39:48.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:48.633+0000 7f2b37400640 1 --2- 172.21.15.17:0/4109168903 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f2b200725e0 0x7f2b20074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:39:48.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:48.633+0000 7f2b37400640 1 -- 172.21.15.17:0/4109168903 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f2b30092c70 con 0x7f2b400c9640 2024-09-10T21:39:48.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:48.633+0000 7f2b3ea00640 1 --2- 172.21.15.17:0/4109168903 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f2b200725e0 0x7f2b20074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:39:48.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:48.633+0000 7f2b3ea00640 1 --2- 172.21.15.17:0/4109168903 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f2b200725e0 0x7f2b20074aa0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f2b2c0af000 tx=0x7f2b2c01b0f0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:39:48.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:48.637+0000 7f2b37400640 1 -- 172.21.15.17:0/4109168903 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f2b4014eea0 con 0x7f2b400c9640 2024-09-10T21:39:48.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:48.909+0000 7f2b44c00640 1 -- 172.21.15.17:0/4109168903 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f2b40069e60 con 0x7f2b400c9640 2024-09-10T21:39:48.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:48.909+0000 7f2b37400640 1 -- 172.21.15.17:0/4109168903 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1552 (secure 0 0 0) 0x7f2b3005dfe0 con 0x7f2b400c9640 2024-09-10T21:39:48.913 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":120,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"a","quorum_age":21,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5e2f023f-d6a6-49f4-a256-f4807dce3739","modified":"2024-09-10T21:23:46.892130Z","created":"2024-09-10T21:23:46.892130Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3301","nonce":0},{"type":"v1","addr":"172.21.15.31:6790","nonce":0}]},"addr":"172.21.15.31:6790/0","public_addr":"172.21.15.31:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-10T21:39:48.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:48.909+0000 7f2b44c00640 1 -- 172.21.15.17:0/4109168903 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f2b200725e0 msgr2=0x7f2b20074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:39:48.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:48.909+0000 7f2b44c00640 1 --2- 172.21.15.17:0/4109168903 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f2b200725e0 0x7f2b20074aa0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f2b2c0af000 tx=0x7f2b2c01b0f0 comp rx=0 tx=0).stop 2024-09-10T21:39:48.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:48.909+0000 7f2b44c00640 1 -- 172.21.15.17:0/4109168903 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2b400c9640 msgr2=0x7f2b4014b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:39:48.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:48.909+0000 7f2b44c00640 1 --2- 172.21.15.17:0/4109168903 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2b400c9640 0x7f2b4014b360 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f2b3000c500 tx=0x7f2b3000c900 comp rx=0 tx=0).stop 2024-09-10T21:39:48.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:48.909+0000 7f2b44c00640 1 -- 172.21.15.17:0/4109168903 shutdown_connections 2024-09-10T21:39:48.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:48.909+0000 7f2b44c00640 1 --2- 172.21.15.17:0/4109168903 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f2b200725e0 0x7f2b20074aa0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:39:48.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:48.909+0000 7f2b44c00640 1 --2- 172.21.15.17:0/4109168903 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f2b40151350 0x7f2b40144bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:39:48.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:48.909+0000 7f2b44c00640 1 --2- 172.21.15.17:0/4109168903 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2b40150f70 0x7f2b4014e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:39:48.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:48.909+0000 7f2b44c00640 1 --2- 172.21.15.17:0/4109168903 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2b400c9640 0x7f2b4014b360 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:39:48.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:48.909+0000 7f2b44c00640 1 -- 172.21.15.17:0/4109168903 >> 172.21.15.17:0/4109168903 conn(0x7f2b4006cdf0 msgr2=0x7f2b400cbb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:39:48.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:48.909+0000 7f2b44c00640 1 -- 172.21.15.17:0/4109168903 shutdown_connections 2024-09-10T21:39:48.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:48.909+0000 7f2b44c00640 1 -- 172.21.15.17:0/4109168903 wait complete. 2024-09-10T21:39:51.936 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-10T21:39:52.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:52.105+0000 7f49a7200640 1 -- 172.21.15.17:0/2898874365 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f49a00c7ad0 msgr2=0x7f49a00c3c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:39:52.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:52.105+0000 7f49a7200640 1 --2- 172.21.15.17:0/2898874365 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f49a00c7ad0 0x7f49a00c3c80 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f498c009900 tx=0x7f498c01c820 comp rx=0 tx=0).stop 2024-09-10T21:39:52.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:52.105+0000 7f49a7200640 1 -- 172.21.15.17:0/2898874365 shutdown_connections 2024-09-10T21:39:52.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:52.105+0000 7f49a7200640 1 --2- 172.21.15.17:0/2898874365 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f49a00c7ad0 0x7f49a00c3c80 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:39:52.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:52.105+0000 7f49a7200640 1 -- 172.21.15.17:0/2898874365 >> 172.21.15.17:0/2898874365 conn(0x7f49a006efa0 msgr2=0x7f49a006cc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:39:52.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:52.105+0000 7f49a7200640 1 -- 172.21.15.17:0/2898874365 shutdown_connections 2024-09-10T21:39:52.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:52.105+0000 7f49a7200640 1 -- 172.21.15.17:0/2898874365 wait complete. 2024-09-10T21:39:52.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:52.109+0000 7f49a7200640 1 Processor -- start 2024-09-10T21:39:52.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:52.109+0000 7f49a7200640 1 -- start start 2024-09-10T21:39:52.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:52.109+0000 7f49a7200640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f49a0074270 0x7f49a00c8640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:39:52.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:52.109+0000 7f49a7200640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f49a00c7ad0 0x7f49a0146460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:39:52.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:52.109+0000 7f49a7200640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f49a0151740 0x7f49a01469a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:39:52.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:52.109+0000 7f49a7200640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f49a0164eb0 con 0x7f49a00c7ad0 2024-09-10T21:39:52.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:52.109+0000 7f49a7200640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f49a0164d30 con 0x7f49a0151740 2024-09-10T21:39:52.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:52.109+0000 7f49a7200640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f49a0165030 con 0x7f49a0074270 2024-09-10T21:39:52.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:52.109+0000 7f499f400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f49a00c7ad0 0x7f49a0146460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:39:52.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:52.109+0000 7f499f400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f49a00c7ad0 0x7f49a0146460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:34760/0 (socket says 172.21.15.17:34760) 2024-09-10T21:39:52.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:52.109+0000 7f499f400640 1 -- 172.21.15.17:0/2651428310 learned_addr learned my addr 172.21.15.17:0/2651428310 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:39:52.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:52.109+0000 7f499fe00640 1 --2- 172.21.15.17:0/2651428310 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f49a0074270 0x7f49a00c8640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:39:52.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:52.109+0000 7f49a5200640 1 --2- 172.21.15.17:0/2651428310 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f49a0151740 0x7f49a01469a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:39:52.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:52.109+0000 7f499f400640 1 -- 172.21.15.17:0/2651428310 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f49a0074270 msgr2=0x7f49a00c8640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:39:52.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:52.109+0000 7f499f400640 1 --2- 172.21.15.17:0/2651428310 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f49a0074270 0x7f49a00c8640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:39:52.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:52.109+0000 7f499f400640 1 -- 172.21.15.17:0/2651428310 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f49a0151740 msgr2=0x7f49a01469a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:39:52.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:52.109+0000 7f499f400640 1 --2- 172.21.15.17:0/2651428310 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f49a0151740 0x7f49a01469a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:39:52.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:52.109+0000 7f499f400640 1 -- 172.21.15.17:0/2651428310 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49a0156610 con 0x7f49a00c7ad0 2024-09-10T21:39:52.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:52.109+0000 7f499fe00640 1 --2- 172.21.15.17:0/2651428310 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f49a0074270 0x7f49a00c8640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:39:52.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:52.109+0000 7f499f400640 1 --2- 172.21.15.17:0/2651428310 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f49a00c7ad0 0x7f49a0146460 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f498c009850 tx=0x7f498c004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:39:52.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:52.109+0000 7f4997e00640 1 -- 172.21.15.17:0/2651428310 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f498c0b2040 con 0x7f49a00c7ad0 2024-09-10T21:39:52.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:52.109+0000 7f4997e00640 1 -- 172.21.15.17:0/2651428310 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f498c005020 con 0x7f49a00c7ad0 2024-09-10T21:39:52.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:52.109+0000 7f4997e00640 1 -- 172.21.15.17:0/2651428310 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f498c005430 con 0x7f49a00c7ad0 2024-09-10T21:39:52.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:52.109+0000 7f49a7200640 1 -- 172.21.15.17:0/2651428310 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49a00c9160 con 0x7f49a00c7ad0 2024-09-10T21:39:52.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:52.109+0000 7f49a7200640 1 -- 172.21.15.17:0/2651428310 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49a0156ee0 con 0x7f49a00c7ad0 2024-09-10T21:39:52.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:52.113+0000 7f49a7200640 1 -- 172.21.15.17:0/2651428310 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49a00c3c80 con 0x7f49a00c7ad0 2024-09-10T21:39:52.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:52.113+0000 7f4997e00640 1 -- 172.21.15.17:0/2651428310 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f498c0046d0 con 0x7f49a00c7ad0 2024-09-10T21:39:52.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:52.113+0000 7f4997e00640 1 --2- 172.21.15.17:0/2651428310 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f4980072610 0x7f4980074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:39:52.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:52.113+0000 7f4997e00640 1 -- 172.21.15.17:0/2651428310 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f498c12d660 con 0x7f49a00c7ad0 2024-09-10T21:39:52.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:52.113+0000 7f499fe00640 1 --2- 172.21.15.17:0/2651428310 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f4980072610 0x7f4980074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:39:52.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:52.117+0000 7f499fe00640 1 --2- 172.21.15.17:0/2651428310 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f4980072610 0x7f4980074ad0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f4990001750 tx=0x7f4990009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:39:52.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:52.117+0000 7f4997e00640 1 -- 172.21.15.17:0/2651428310 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f498c0164d0 con 0x7f49a00c7ad0 2024-09-10T21:39:52.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:52.401+0000 7f49a7200640 1 -- 172.21.15.17:0/2651428310 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f49a006c4c0 con 0x7f49a00c7ad0 2024-09-10T21:39:52.406 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:52.401+0000 7f4997e00640 1 -- 172.21.15.17:0/2651428310 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1552 (secure 0 0 0) 0x7f498c0f89d0 con 0x7f49a00c7ad0 2024-09-10T21:39:52.406 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":120,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"a","quorum_age":24,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5e2f023f-d6a6-49f4-a256-f4807dce3739","modified":"2024-09-10T21:23:46.892130Z","created":"2024-09-10T21:23:46.892130Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3301","nonce":0},{"type":"v1","addr":"172.21.15.31:6790","nonce":0}]},"addr":"172.21.15.31:6790/0","public_addr":"172.21.15.31:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-10T21:39:52.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:52.405+0000 7f49a7200640 1 -- 172.21.15.17:0/2651428310 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f4980072610 msgr2=0x7f4980074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:39:52.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:52.405+0000 7f49a7200640 1 --2- 172.21.15.17:0/2651428310 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f4980072610 0x7f4980074ad0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f4990001750 tx=0x7f4990009040 comp rx=0 tx=0).stop 2024-09-10T21:39:52.408 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:52.405+0000 7f49a7200640 1 -- 172.21.15.17:0/2651428310 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f49a00c7ad0 msgr2=0x7f49a0146460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:39:52.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:52.405+0000 7f49a7200640 1 --2- 172.21.15.17:0/2651428310 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f49a00c7ad0 0x7f49a0146460 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f498c009850 tx=0x7f498c004040 comp rx=0 tx=0).stop 2024-09-10T21:39:52.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:52.405+0000 7f49a7200640 1 -- 172.21.15.17:0/2651428310 shutdown_connections 2024-09-10T21:39:52.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:52.405+0000 7f49a7200640 1 --2- 172.21.15.17:0/2651428310 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f4980072610 0x7f4980074ad0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:39:52.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:52.405+0000 7f49a7200640 1 --2- 172.21.15.17:0/2651428310 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f49a0151740 0x7f49a01469a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:39:52.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:52.405+0000 7f49a7200640 1 --2- 172.21.15.17:0/2651428310 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f49a00c7ad0 0x7f49a0146460 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:39:52.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:52.405+0000 7f49a7200640 1 --2- 172.21.15.17:0/2651428310 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f49a0074270 0x7f49a00c8640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:39:52.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:52.405+0000 7f49a7200640 1 -- 172.21.15.17:0/2651428310 >> 172.21.15.17:0/2651428310 conn(0x7f49a006efa0 msgr2=0x7f49a0140200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:39:52.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:52.405+0000 7f49a7200640 1 -- 172.21.15.17:0/2651428310 shutdown_connections 2024-09-10T21:39:52.409 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:52.405+0000 7f49a7200640 1 -- 172.21.15.17:0/2651428310 wait complete. 2024-09-10T21:39:52.688 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:39:52.691 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:39:55.430 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-10T21:39:55.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:55.597+0000 7f88bf200640 1 -- 172.21.15.17:0/2099199025 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f88c0158d80 msgr2=0x7f88c00ce1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:39:55.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:55.597+0000 7f88bf200640 1 --2- 172.21.15.17:0/2099199025 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f88c0158d80 0x7f88c00ce1b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f88c0072f10 tx=0x7f88b801cd80 comp rx=0 tx=0).stop 2024-09-10T21:39:55.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:55.597+0000 7f88bf200640 1 -- 172.21.15.17:0/2099199025 shutdown_connections 2024-09-10T21:39:55.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:55.597+0000 7f88bf200640 1 --2- 172.21.15.17:0/2099199025 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f88c0158d80 0x7f88c00ce1b0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:39:55.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:55.597+0000 7f88bf200640 1 --2- 172.21.15.17:0/2099199025 >> v2:172.21.15.31:3300/0 conn(0x7f88c0071130 0x7f88c00c2ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:39:55.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:55.597+0000 7f88bf200640 1 --2- 172.21.15.17:0/2099199025 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f88c00cddd0 0x7f88c015b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:39:55.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:55.597+0000 7f88bf200640 1 -- 172.21.15.17:0/2099199025 >> 172.21.15.17:0/2099199025 conn(0x7f88c0074ac0 msgr2=0x7f88c0068990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:39:55.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:55.597+0000 7f88bf200640 1 -- 172.21.15.17:0/2099199025 shutdown_connections 2024-09-10T21:39:55.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:55.597+0000 7f88bf200640 1 -- 172.21.15.17:0/2099199025 wait complete. 2024-09-10T21:39:55.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:55.601+0000 7f88bf200640 1 Processor -- start 2024-09-10T21:39:55.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:55.601+0000 7f88bf200640 1 -- start start 2024-09-10T21:39:55.607 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:55.601+0000 7f88bf200640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f88c0071130 0x7f88c015b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:39:55.607 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:55.601+0000 7f88bf200640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f88c00cddd0 0x7f88c01fae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:39:55.607 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:55.601+0000 7f88bf200640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f88c0158d80 0x7f88c0150ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:39:55.607 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:55.601+0000 7f88bf200640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f88c01526b0 con 0x7f88c00cddd0 2024-09-10T21:39:55.607 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:55.601+0000 7f88bf200640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f88c0152530 con 0x7f88c0071130 2024-09-10T21:39:55.607 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:55.601+0000 7f88bf200640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f88c00725c0 con 0x7f88c0158d80 2024-09-10T21:39:55.607 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:55.601+0000 7f88bd400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f88c00cddd0 0x7f88c01fae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:39:55.608 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:55.601+0000 7f88bd400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f88c00cddd0 0x7f88c01fae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45114/0 (socket says 172.21.15.17:45114) 2024-09-10T21:39:55.608 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:55.601+0000 7f88bd400640 1 -- 172.21.15.17:0/269324442 learned_addr learned my addr 172.21.15.17:0/269324442 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:39:55.608 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:55.601+0000 7f88be800640 1 --2- 172.21.15.17:0/269324442 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f88c0158d80 0x7f88c0150ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:39:55.608 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:55.601+0000 7f88bde00640 1 --2- 172.21.15.17:0/269324442 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f88c0071130 0x7f88c015b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:39:55.608 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:55.601+0000 7f88bde00640 1 -- 172.21.15.17:0/269324442 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f88c0071130 msgr2=0x7f88c015b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-09-10T21:39:55.608 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:55.601+0000 7f88bde00640 1 -- 172.21.15.17:0/269324442 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f88c0071130 msgr2=0x7f88c015b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-10T21:39:55.608 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:55.601+0000 7f88bde00640 1 --2- 172.21.15.17:0/269324442 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f88c0071130 0x7f88c015b3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T21:39:55.608 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:55.601+0000 7f88bde00640 1 --2- 172.21.15.17:0/269324442 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f88c0071130 0x7f88c015b3e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:39:55.608 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:55.601+0000 7f88bd400640 1 -- 172.21.15.17:0/269324442 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f88c0158d80 msgr2=0x7f88c0150ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:39:55.608 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:55.601+0000 7f88bd400640 1 --2- 172.21.15.17:0/269324442 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f88c0158d80 0x7f88c0150ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:39:55.608 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:55.601+0000 7f88bd400640 1 -- 172.21.15.17:0/269324442 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f88c0071130 msgr2=0x7f88c015b3e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:39:55.608 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:55.601+0000 7f88bd400640 1 --2- 172.21.15.17:0/269324442 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f88c0071130 0x7f88c015b3e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:39:55.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:55.601+0000 7f88bd400640 1 -- 172.21.15.17:0/269324442 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88c015b920 con 0x7f88c00cddd0 2024-09-10T21:39:55.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:55.601+0000 7f88bd400640 1 --2- 172.21.15.17:0/269324442 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f88c00cddd0 0x7f88c01fae30 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f88ac00d750 tx=0x7f88ac00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:39:55.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:55.601+0000 7f88b6000640 1 -- 172.21.15.17:0/269324442 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88ac017070 con 0x7f88c00cddd0 2024-09-10T21:39:55.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:55.601+0000 7f88b6000640 1 -- 172.21.15.17:0/269324442 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f88ac009cc0 con 0x7f88c00cddd0 2024-09-10T21:39:55.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:55.601+0000 7f88b6000640 1 -- 172.21.15.17:0/269324442 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88ac005280 con 0x7f88c00cddd0 2024-09-10T21:39:55.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:55.605+0000 7f88bf200640 1 -- 172.21.15.17:0/269324442 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88c0155f10 con 0x7f88c00cddd0 2024-09-10T21:39:55.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:55.605+0000 7f88bf200640 1 -- 172.21.15.17:0/269324442 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88c01f84f0 con 0x7f88c00cddd0 2024-09-10T21:39:55.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:55.605+0000 7f88bf200640 1 -- 172.21.15.17:0/269324442 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88c0072a40 con 0x7f88c00cddd0 2024-09-10T21:39:55.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:55.605+0000 7f88b6000640 1 -- 172.21.15.17:0/269324442 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f88ac005420 con 0x7f88c00cddd0 2024-09-10T21:39:55.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:55.605+0000 7f88b6000640 1 --2- 172.21.15.17:0/269324442 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f88a0072610 0x7f88a0074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:39:55.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:55.605+0000 7f88b6000640 1 -- 172.21.15.17:0/269324442 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f88ac093660 con 0x7f88c00cddd0 2024-09-10T21:39:55.612 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:55.609+0000 7f88bde00640 1 --2- 172.21.15.17:0/269324442 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f88a0072610 0x7f88a0074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:39:55.612 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:55.609+0000 7f88bde00640 1 --2- 172.21.15.17:0/269324442 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f88a0072610 0x7f88a0074ad0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f88b00058d0 tx=0x7f88b0005860 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:39:55.616 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:55.613+0000 7f88b6000640 1 -- 172.21.15.17:0/269324442 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f88ac00de60 con 0x7f88c00cddd0 2024-09-10T21:39:55.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:55.917+0000 7f88bf200640 1 -- 172.21.15.17:0/269324442 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f88c0070f10 con 0x7f88c00cddd0 2024-09-10T21:39:55.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:55.917+0000 7f88b6000640 1 -- 172.21.15.17:0/269324442 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1557 (secure 0 0 0) 0x7f88ac05e2e0 con 0x7f88c00cddd0 2024-09-10T21:39:55.920 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":122,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":2,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5e2f023f-d6a6-49f4-a256-f4807dce3739","modified":"2024-09-10T21:23:46.892130Z","created":"2024-09-10T21:23:46.892130Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3301","nonce":0},{"type":"v1","addr":"172.21.15.31:6790","nonce":0}]},"addr":"172.21.15.31:6790/0","public_addr":"172.21.15.31:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-10T21:39:55.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:55.917+0000 7f88bf200640 1 -- 172.21.15.17:0/269324442 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f88a0072610 msgr2=0x7f88a0074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:39:55.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:55.917+0000 7f88bf200640 1 --2- 172.21.15.17:0/269324442 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f88a0072610 0x7f88a0074ad0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f88b00058d0 tx=0x7f88b0005860 comp rx=0 tx=0).stop 2024-09-10T21:39:55.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:55.917+0000 7f88bf200640 1 -- 172.21.15.17:0/269324442 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f88c00cddd0 msgr2=0x7f88c01fae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:39:55.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:55.917+0000 7f88bf200640 1 --2- 172.21.15.17:0/269324442 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f88c00cddd0 0x7f88c01fae30 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f88ac00d750 tx=0x7f88ac00dc10 comp rx=0 tx=0).stop 2024-09-10T21:39:55.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:55.917+0000 7f88bf200640 1 -- 172.21.15.17:0/269324442 shutdown_connections 2024-09-10T21:39:55.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:55.917+0000 7f88bf200640 1 --2- 172.21.15.17:0/269324442 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f88a0072610 0x7f88a0074ad0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:39:55.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:55.917+0000 7f88bf200640 1 --2- 172.21.15.17:0/269324442 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f88c0158d80 0x7f88c0150ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:39:55.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:55.917+0000 7f88bf200640 1 --2- 172.21.15.17:0/269324442 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f88c00cddd0 0x7f88c01fae30 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:39:55.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:55.917+0000 7f88bf200640 1 --2- 172.21.15.17:0/269324442 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f88c0071130 0x7f88c015b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:39:55.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:55.917+0000 7f88bf200640 1 -- 172.21.15.17:0/269324442 >> 172.21.15.17:0/269324442 conn(0x7f88c0074ac0 msgr2=0x7f88c00cc8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:39:55.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:55.917+0000 7f88bf200640 1 -- 172.21.15.17:0/269324442 shutdown_connections 2024-09-10T21:39:55.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:55.917+0000 7f88bf200640 1 -- 172.21.15.17:0/269324442 wait complete. 2024-09-10T21:39:55.946 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-09-10T21:39:55.946 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-09-10T21:39:55.946 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-10T21:39:56.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:56.113+0000 7f6f1da00640 1 -- 172.21.15.17:0/488483701 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6efc004d50 msgr2=0x7f6efc0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:39:56.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:56.113+0000 7f6f1da00640 1 --2- 172.21.15.17:0/488483701 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6efc004d50 0x7f6efc0051d0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f6f10009df0 tx=0x7f6f1001d170 comp rx=0 tx=0).stop 2024-09-10T21:39:56.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:56.113+0000 7f6f17400640 1 -- 172.21.15.17:0/488483701 reap_dead start 2024-09-10T21:39:56.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:56.113+0000 7f6f1da00640 1 -- 172.21.15.17:0/488483701 shutdown_connections 2024-09-10T21:39:56.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:56.113+0000 7f6f1da00640 1 -- 172.21.15.17:0/488483701 >> 172.21.15.17:0/488483701 conn(0x7f6f180702f0 msgr2=0x7f6f18070700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:39:56.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:56.113+0000 7f6f1da00640 1 -- 172.21.15.17:0/488483701 shutdown_connections 2024-09-10T21:39:56.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:56.113+0000 7f6f1da00640 1 -- 172.21.15.17:0/488483701 wait complete. 2024-09-10T21:39:56.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:56.117+0000 7f6f1da00640 1 Processor -- start 2024-09-10T21:39:56.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:56.117+0000 7f6f1da00640 1 -- start start 2024-09-10T21:39:56.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:56.117+0000 7f6f1da00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6efc004d50 0x7f6f18151950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:39:56.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:56.117+0000 7f6f1da00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6f180c56e0 0x7f6f18155440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:39:56.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:56.117+0000 7f6f1da00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f180c5b40 0x7f6f18155980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:39:56.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:56.117+0000 7f6f1da00640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f6f1815bd60 con 0x7f6efc004d50 2024-09-10T21:39:56.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:56.117+0000 7f6f1da00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f6f1815ba80 con 0x7f6f180c5b40 2024-09-10T21:39:56.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:56.117+0000 7f6f1da00640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f6f180cb450 con 0x7f6f180c56e0 2024-09-10T21:39:56.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:56.117+0000 7f6f16a00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6efc004d50 0x7f6f18151950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:39:56.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:56.117+0000 7f6f16a00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6efc004d50 0x7f6f18151950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45136/0 (socket says 172.21.15.17:45136) 2024-09-10T21:39:56.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:56.117+0000 7f6f16a00640 1 -- 172.21.15.17:0/3704568321 learned_addr learned my addr 172.21.15.17:0/3704568321 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:39:56.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:56.117+0000 7f6f0fe00640 1 --2- 172.21.15.17:0/3704568321 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6f180c56e0 0x7f6f18155440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:39:56.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:56.117+0000 7f6f17400640 1 --2- 172.21.15.17:0/3704568321 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f180c5b40 0x7f6f18155980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:39:56.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:56.117+0000 7f6f16a00640 1 -- 172.21.15.17:0/3704568321 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6f180c56e0 msgr2=0x7f6f18155440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:39:56.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:56.117+0000 7f6f17400640 1 -- 172.21.15.17:0/3704568321 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f180c5b40 msgr2=0x7f6f18155980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-09-10T21:39:56.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:56.117+0000 7f6f17400640 1 -- 172.21.15.17:0/3704568321 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f180c5b40 msgr2=0x7f6f18155980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-10T21:39:56.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:56.117+0000 7f6f17400640 1 --2- 172.21.15.17:0/3704568321 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f180c5b40 0x7f6f18155980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T21:39:56.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:56.117+0000 7f6f16a00640 1 --2- 172.21.15.17:0/3704568321 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6f180c56e0 0x7f6f18155440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:39:56.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:56.117+0000 7f6f16a00640 1 -- 172.21.15.17:0/3704568321 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f180c5b40 msgr2=0x7f6f18155980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:39:56.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:56.117+0000 7f6f17400640 1 --2- 172.21.15.17:0/3704568321 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f180c5b40 0x7f6f18155980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:39:56.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:56.117+0000 7f6f16a00640 1 --2- 172.21.15.17:0/3704568321 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6f180c5b40 0x7f6f18155980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:39:56.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:56.117+0000 7f6f16a00640 1 -- 172.21.15.17:0/3704568321 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f181f0e60 con 0x7f6efc004d50 2024-09-10T21:39:56.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:56.117+0000 7f6f16a00640 1 --2- 172.21.15.17:0/3704568321 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6efc004d50 0x7f6f18151950 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f6f08012690 tx=0x7f6f08012a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:39:56.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:56.117+0000 7f6f0ea00640 1 -- 172.21.15.17:0/3704568321 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f08010930 con 0x7f6efc004d50 2024-09-10T21:39:56.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:56.117+0000 7f6f0ea00640 1 -- 172.21.15.17:0/3704568321 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6f08017030 con 0x7f6efc004d50 2024-09-10T21:39:56.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:56.117+0000 7f6f1da00640 1 -- 172.21.15.17:0/3704568321 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f181f1100 con 0x7f6efc004d50 2024-09-10T21:39:56.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:56.117+0000 7f6f0ea00640 1 -- 172.21.15.17:0/3704568321 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f08010d90 con 0x7f6efc004d50 2024-09-10T21:39:56.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:56.117+0000 7f6f1da00640 1 -- 172.21.15.17:0/3704568321 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f180c76f0 con 0x7f6efc004d50 2024-09-10T21:39:56.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:56.121+0000 7f6f1da00640 1 --2- 172.21.15.17:0/3704568321 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6f18153530 0x7f6f181f3950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:39:56.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:56.121+0000 7f6f1da00640 1 -- 172.21.15.17:0/3704568321 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f6f180c7930 con 0x7f6f18153530 2024-09-10T21:39:56.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:56.121+0000 7f6f0fe00640 1 --2- 172.21.15.17:0/3704568321 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6f18153530 0x7f6f181f3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:39:56.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:56.121+0000 7f6f0fe00640 1 -- 172.21.15.17:0/3704568321 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6f180c37b0 con 0x7f6f18153530 2024-09-10T21:39:56.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:56.121+0000 7f6f0fe00640 1 --2- 172.21.15.17:0/3704568321 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6f18153530 0x7f6f181f3950 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6f000129f0 tx=0x7f6f00012eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:39:56.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:56.121+0000 7f6f0ea00640 1 -- 172.21.15.17:0/3704568321 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f0000fa50 con 0x7f6f18153530 2024-09-10T21:39:56.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:56.121+0000 7f6f0ea00640 1 -- 172.21.15.17:0/3704568321 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f6f0000fbf0 con 0x7f6f18153530 2024-09-10T21:39:56.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:56.121+0000 7f6f0ea00640 1 -- 172.21.15.17:0/3704568321 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6f18153530 msgr2=0x7f6f181f3950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:39:56.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:56.121+0000 7f6f0ea00640 1 --2- 172.21.15.17:0/3704568321 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6f18153530 0x7f6f181f3950 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6f000129f0 tx=0x7f6f00012eb0 comp rx=0 tx=0).stop 2024-09-10T21:39:56.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:56.121+0000 7f6f0ea00640 1 -- 172.21.15.17:0/3704568321 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6f18153530 msgr2=0x7f6f181f3950 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:39:56.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:56.121+0000 7f6f0ea00640 1 --2- 172.21.15.17:0/3704568321 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6f18153530 0x7f6f181f3950 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:39:56.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:56.121+0000 7f6f0ea00640 1 -- 172.21.15.17:0/3704568321 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f6f08013450 con 0x7f6efc004d50 2024-09-10T21:39:56.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:56.121+0000 7f6f0ea00640 1 --2- 172.21.15.17:0/3704568321 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f6ef4072050 0x7f6ef4074510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:39:56.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:56.121+0000 7f6f0ea00640 1 -- 172.21.15.17:0/3704568321 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f6f08097010 con 0x7f6efc004d50 2024-09-10T21:39:56.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:56.121+0000 7f6f0fe00640 1 --2- 172.21.15.17:0/3704568321 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f6ef4072050 0x7f6ef4074510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:39:56.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:56.125+0000 7f6f0fe00640 1 --2- 172.21.15.17:0/3704568321 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f6ef4072050 0x7f6ef4074510 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f6f000129f0 tx=0x7f6f00011000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:39:56.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:56.129+0000 7f6f1da00640 1 --2- 172.21.15.17:0/3704568321 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6f181f3e90 0x7f6f181f42b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:39:56.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:56.129+0000 7f6f1da00640 1 -- 172.21.15.17:0/3704568321 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f6f180cd810 con 0x7f6f181f3e90 2024-09-10T21:39:56.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:56.129+0000 7f6f17400640 1 --2- 172.21.15.17:0/3704568321 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6f181f3e90 0x7f6f181f42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:39:56.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:56.129+0000 7f6f17400640 1 -- 172.21.15.17:0/3704568321 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f6f180cd530 con 0x7f6f181f3e90 2024-09-10T21:39:56.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:56.129+0000 7f6f17400640 1 --2- 172.21.15.17:0/3704568321 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6f181f3e90 0x7f6f181f42b0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f6f100093c0 tx=0x7f6f100093f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:39:56.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:56.129+0000 7f6f0ea00640 1 -- 172.21.15.17:0/3704568321 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f100a5e60 con 0x7f6f181f3e90 2024-09-10T21:39:56.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:56.129+0000 7f6f0ea00640 1 -- 172.21.15.17:0/3704568321 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4710 (secure 0 0 0) 0x7f6f100a5e60 con 0x7f6f181f3e90 2024-09-10T21:39:56.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:56.129+0000 7f6f0ea00640 1 -- 172.21.15.17:0/3704568321 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6f181f3e90 msgr2=0x7f6f181f42b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:39:56.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:56.129+0000 7f6f0ea00640 1 --2- 172.21.15.17:0/3704568321 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6f181f3e90 0x7f6f181f42b0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f6f100093c0 tx=0x7f6f100093f0 comp rx=0 tx=0).stop 2024-09-10T21:39:56.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:56.129+0000 7f6f0ea00640 1 -- 172.21.15.17:0/3704568321 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6f181f3e90 msgr2=0x7f6f181f42b0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:39:56.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:56.129+0000 7f6f0ea00640 1 --2- 172.21.15.17:0/3704568321 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6f181f3e90 0x7f6f181f42b0 secure :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f6f100093c0 tx=0x7f6f100093f0 comp rx=0 tx=0).stop 2024-09-10T21:39:56.135 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-10T21:39:56.135 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:39:56.135 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:39:56.136 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "leader", 2024-09-10T21:39:56.136 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 122, 2024-09-10T21:39:56.136 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 70331, 2024-09-10T21:39:56.136 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-10T21:39:56.136 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-10T21:39:56.136 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-10T21:39:56.136 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-10T21:39:56.136 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:39:56.136 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 2, 2024-09-10T21:39:56.136 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:39:56.136 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-10T21:39:56.136 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-10T21:39:56.136 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:39:56.137 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:39:56.137 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:39:56.137 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:39:56.137 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:39:56.137 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:39:56.137 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:39:56.137 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:39:56.137 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:39:56.137 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:39:56.137 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:39:56.137 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:39:56.137 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-10T21:39:56.137 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-10T21:39:56.137 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:39:56.138 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:39:56.138 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:39:56.138 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:39:56.138 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:39:56.138 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:39:56.138 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:39:56.138 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:39:56.138 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:39:56.138 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:39:56.138 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:39:56.138 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:39:56.138 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:39:56.138 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-10T21:39:56.138 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-10T21:39:56.138 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-10T21:39:56.138 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-10T21:39:56.139 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-10T21:39:56.139 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "5e2f023f-d6a6-49f4-a256-f4807dce3739", 2024-09-10T21:39:56.139 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:39:56.139 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:39:56.139 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-10T21:39:56.139 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-10T21:39:56.139 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-10T21:39:56.139 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-10T21:39:56.139 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-10T21:39:56.139 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-10T21:39:56.139 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-10T21:39:56.139 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:39:56.139 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-10T21:39:56.139 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:39:56.140 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:39:56.140 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:39:56.140 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:39:56.140 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:39:56.140 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:39:56.140 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:39:56.140 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:39:56.140 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:39:56.140 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:39:56.140 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:39:56.140 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:39:56.140 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-10T21:39:56.140 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:39:56.140 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-10T21:39:56.141 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:39:56.141 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:39:56.141 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:39:56.141 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:39:56.141 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:39:56.141 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:39:56.141 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:39:56.141 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-10T21:39:56.141 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:39:56.141 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:39:56.141 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:39:56.141 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:39:56.141 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-10T21:39:56.141 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:39:56.142 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:39:56.142 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:39:56.142 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:39:56.142 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-10T21:39:56.142 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-10T21:39:56.142 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:39:56.142 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:39:56.142 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:39:56.142 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:39:56.142 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:39:56.142 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:39:56.142 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:39:56.142 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:39:56.142 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:39:56.142 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:39:56.143 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:39:56.143 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3300", 2024-09-10T21:39:56.143 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:39:56.143 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:39:56.143 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:39:56.143 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:39:56.143 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789", 2024-09-10T21:39:56.143 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:39:56.143 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:39:56.143 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:39:56.143 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:39:56.143 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789/0", 2024-09-10T21:39:56.143 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6789/0", 2024-09-10T21:39:56.143 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:39:56.144 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:39:56.144 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:39:56.144 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:39:56.144 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:39:56.144 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:39:56.144 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:39:56.144 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:39:56.144 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:39:56.144 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:39:56.144 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:39:56.144 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3301", 2024-09-10T21:39:56.144 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:39:56.144 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:39:56.144 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:39:56.145 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:39:56.145 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790", 2024-09-10T21:39:56.145 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:39:56.145 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:39:56.145 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:39:56.145 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:39:56.145 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790/0", 2024-09-10T21:39:56.145 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6790/0", 2024-09-10T21:39:56.145 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:39:56.145 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:39:56.145 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:39:56.145 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:39:56.145 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:39:56.145 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:39:56.146 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-10T21:39:56.146 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-10T21:39:56.146 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:39:56.146 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:39:56.146 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:39:56.146 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:39:56.146 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:39:56.146 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:39:56.146 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-10T21:39:56.146 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:39:56.146 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:39:56.146 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:39:56.146 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 2 2024-09-10T21:39:56.146 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:39:56.147 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:39:56.147 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-10T21:39:56.147 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:39:56.147 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:39:56.147 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:39:56.147 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 4 2024-09-10T21:39:56.147 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:39:56.147 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:39:56.147 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:39:56.147 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-10T21:39:56.147 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-10T21:39:56.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:56.129+0000 7f6f1da00640 1 -- 172.21.15.17:0/3704568321 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f6ef4072050 msgr2=0x7f6ef4074510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:39:56.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:56.129+0000 7f6f1da00640 1 --2- 172.21.15.17:0/3704568321 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f6ef4072050 0x7f6ef4074510 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f6f000129f0 tx=0x7f6f00011000 comp rx=0 tx=0).stop 2024-09-10T21:39:56.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:56.129+0000 7f6f1da00640 1 -- 172.21.15.17:0/3704568321 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6efc004d50 msgr2=0x7f6f18151950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:39:56.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:56.129+0000 7f6f1da00640 1 --2- 172.21.15.17:0/3704568321 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6efc004d50 0x7f6f18151950 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f6f08012690 tx=0x7f6f08012a90 comp rx=0 tx=0).stop 2024-09-10T21:39:56.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:56.129+0000 7f6f17400640 1 -- 172.21.15.17:0/3704568321 reap_dead start 2024-09-10T21:39:56.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:56.129+0000 7f6f1da00640 1 -- 172.21.15.17:0/3704568321 shutdown_connections 2024-09-10T21:39:56.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:56.129+0000 7f6f1da00640 1 --2- 172.21.15.17:0/3704568321 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6efc004d50 0x7f6f18151950 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:39:56.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:56.129+0000 7f6f1da00640 1 -- 172.21.15.17:0/3704568321 >> 172.21.15.17:0/3704568321 conn(0x7f6f180702f0 msgr2=0x7f6f180ca4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:39:56.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:56.129+0000 7f6f1da00640 1 -- 172.21.15.17:0/3704568321 shutdown_connections 2024-09-10T21:39:56.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:56.129+0000 7f6f1da00640 1 -- 172.21.15.17:0/3704568321 wait complete. 2024-09-10T21:39:56.157 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-10T21:39:56.157 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-09-10T21:39:56.157 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-10T21:39:56.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:56.309+0000 7f49d8e00640 1 -- 172.21.15.17:0/2975364460 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f49d4152070 msgr2=0x7f49d4072420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:39:56.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:56.309+0000 7f49d8e00640 1 --2- 172.21.15.17:0/2975364460 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f49d4152070 0x7f49d4072420 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f49c4009870 tx=0x7f49c401f010 comp rx=0 tx=0).stop 2024-09-10T21:39:56.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:56.309+0000 7f49d8e00640 1 -- 172.21.15.17:0/2975364460 shutdown_connections 2024-09-10T21:39:56.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:56.309+0000 7f49d8e00640 1 --2- 172.21.15.17:0/2975364460 >> v2:172.21.15.31:3300/0 conn(0x7f49d4158f60 0x7f49d4072960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:39:56.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:56.309+0000 7f49d8e00640 1 --2- 172.21.15.17:0/2975364460 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f49d4152070 0x7f49d4072420 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:39:56.316 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:56.309+0000 7f49d8e00640 1 -- 172.21.15.17:0/2975364460 >> 172.21.15.17:0/2975364460 conn(0x7f49d406cc20 msgr2=0x7f49d406d030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:39:56.316 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:56.309+0000 7f49d8e00640 1 -- 172.21.15.17:0/2975364460 shutdown_connections 2024-09-10T21:39:56.316 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:56.309+0000 7f49d8e00640 1 -- 172.21.15.17:0/2975364460 wait complete. 2024-09-10T21:39:56.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:56.313+0000 7f49d8e00640 1 Processor -- start 2024-09-10T21:39:56.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:56.313+0000 7f49d8e00640 1 -- start start 2024-09-10T21:39:56.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:56.313+0000 7f49d8e00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f49d4152070 0x7f49d414fb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:39:56.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:56.313+0000 7f49d8e00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f49d4152450 0x7f49d41f04e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:39:56.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:56.313+0000 7f49d8e00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f49d4158f60 0x7f49d41f0a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:39:56.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:56.313+0000 7f49d8e00640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f49d40cecf0 con 0x7f49d4158f60 2024-09-10T21:39:56.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:56.313+0000 7f49d8e00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f49d40c5090 con 0x7f49d4152450 2024-09-10T21:39:56.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:56.313+0000 7f49d8e00640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f49d40cee70 con 0x7f49d4152070 2024-09-10T21:39:56.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:56.317+0000 7f49d3e00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f49d4158f60 0x7f49d41f0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:39:56.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:56.317+0000 7f49d3400640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f49d4152070 0x7f49d414fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:39:56.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:56.317+0000 7f49d3e00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f49d4158f60 0x7f49d41f0a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45162/0 (socket says 172.21.15.17:45162) 2024-09-10T21:39:56.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:56.317+0000 7f49d3e00640 1 -- 172.21.15.17:0/1959190397 learned_addr learned my addr 172.21.15.17:0/1959190397 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:39:56.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:56.317+0000 7f49d3400640 1 -- 172.21.15.17:0/1959190397 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f49d4152450 msgr2=0x7f49d41f04e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:39:56.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:56.317+0000 7f49d3400640 1 --2- 172.21.15.17:0/1959190397 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f49d4152450 0x7f49d41f04e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:39:56.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:56.317+0000 7f49d3400640 1 -- 172.21.15.17:0/1959190397 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f49d4158f60 msgr2=0x7f49d41f0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:39:56.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:56.317+0000 7f49d3400640 1 --2- 172.21.15.17:0/1959190397 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f49d4158f60 0x7f49d41f0a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:39:56.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:56.317+0000 7f49d3400640 1 -- 172.21.15.17:0/1959190397 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49d41517f0 con 0x7f49d4152070 2024-09-10T21:39:56.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:56.317+0000 7f49d3400640 1 --2- 172.21.15.17:0/1959190397 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f49d4152070 0x7f49d414fb00 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f49c40af000 tx=0x7f49c40a8ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:39:56.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:56.317+0000 7f49cb400640 1 -- 172.21.15.17:0/1959190397 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49c40bf070 con 0x7f49d4152070 2024-09-10T21:39:56.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:56.317+0000 7f49cb400640 1 -- 172.21.15.17:0/1959190397 <== mon.2 v2:172.21.15.31:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f49c4005310 con 0x7f49d4152070 2024-09-10T21:39:56.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:56.317+0000 7f49cb400640 1 -- 172.21.15.17:0/1959190397 <== mon.2 v2:172.21.15.31:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49c40b5040 con 0x7f49d4152070 2024-09-10T21:39:56.322 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:56.317+0000 7f49d8e00640 1 -- 172.21.15.17:0/1959190397 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49d41e63e0 con 0x7f49d4152070 2024-09-10T21:39:56.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:56.317+0000 7f49d8e00640 1 -- 172.21.15.17:0/1959190397 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f49d406f840 con 0x7f49d4152070 2024-09-10T21:39:56.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:56.321+0000 7f49d8e00640 1 --2- 172.21.15.17:0/1959190397 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f49d4151110 0x7f49d414ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:39:56.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:56.321+0000 7f49d8e00640 1 -- 172.21.15.17:0/1959190397 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f49d406f6c0 con 0x7f49d4151110 2024-09-10T21:39:56.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:56.321+0000 7f49d2a00640 1 --2- 172.21.15.17:0/1959190397 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f49d4151110 0x7f49d414ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:39:56.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:56.321+0000 7f49d2a00640 1 -- 172.21.15.17:0/1959190397 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f49d4151110 msgr2=0x7f49d414ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 20 2024-09-10T21:39:56.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:56.321+0000 7f49d2a00640 1 -- 172.21.15.17:0/1959190397 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f49d4151110 msgr2=0x7f49d414ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-10T21:39:56.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:56.321+0000 7f49d2a00640 1 --2- 172.21.15.17:0/1959190397 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f49d4151110 0x7f49d414ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T21:39:56.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:56.321+0000 7f49d2a00640 1 --2- 172.21.15.17:0/1959190397 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f49d4151110 0x7f49d414ca30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:39:56.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:56.321+0000 7f49cb400640 1 -- 172.21.15.17:0/1959190397 <== mon.2 v2:172.21.15.31:3301/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f49c40b8070 con 0x7f49d4152070 2024-09-10T21:39:56.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:56.321+0000 7f49cb400640 1 --2- 172.21.15.17:0/1959190397 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f49b40725e0 0x7f49b4074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:39:56.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:56.321+0000 7f49cb400640 1 -- 172.21.15.17:0/1959190397 <== mon.2 v2:172.21.15.31:3301/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f49c4130280 con 0x7f49d4152070 2024-09-10T21:39:56.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:56.321+0000 7f49d3e00640 1 --2- 172.21.15.17:0/1959190397 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f49b40725e0 0x7f49b4074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:39:56.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:56.325+0000 7f49d3e00640 1 --2- 172.21.15.17:0/1959190397 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f49b40725e0 0x7f49b4074aa0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f49cc009e70 tx=0x7f49cc009580 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:39:56.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:56.525+0000 7f49d2a00640 1 --2- 172.21.15.17:0/1959190397 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f49d4151110 0x7f49d414ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:39:56.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:56.525+0000 7f49d2a00640 1 -- 172.21.15.17:0/1959190397 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f49d4151110 msgr2=0x7f49d414ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 21 2024-09-10T21:39:56.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:56.525+0000 7f49d2a00640 1 -- 172.21.15.17:0/1959190397 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f49d4151110 msgr2=0x7f49d414ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-10T21:39:56.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:56.525+0000 7f49d2a00640 1 --2- 172.21.15.17:0/1959190397 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f49d4151110 0x7f49d414ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T21:39:56.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:56.525+0000 7f49d2a00640 1 --2- 172.21.15.17:0/1959190397 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f49d4151110 0x7f49d414ca30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-10T21:39:56.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:56.929+0000 7f49d2a00640 1 --2- 172.21.15.17:0/1959190397 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f49d4151110 0x7f49d414ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:39:56.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:56.929+0000 7f49d2a00640 1 -- 172.21.15.17:0/1959190397 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f49d4151110 msgr2=0x7f49d414ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 21 2024-09-10T21:39:56.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:56.929+0000 7f49d2a00640 1 -- 172.21.15.17:0/1959190397 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f49d4151110 msgr2=0x7f49d414ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-10T21:39:56.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:56.929+0000 7f49d2a00640 1 --2- 172.21.15.17:0/1959190397 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f49d4151110 0x7f49d414ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T21:39:56.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:56.929+0000 7f49d2a00640 1 --2- 172.21.15.17:0/1959190397 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f49d4151110 0x7f49d414ca30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-10T21:39:57.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:57.733+0000 7f49d2a00640 1 --2- 172.21.15.17:0/1959190397 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f49d4151110 0x7f49d414ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:39:57.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:57.733+0000 7f49d2a00640 1 -- 172.21.15.17:0/1959190397 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f49d4151110 msgr2=0x7f49d414ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 21 2024-09-10T21:39:57.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:57.733+0000 7f49d2a00640 1 -- 172.21.15.17:0/1959190397 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f49d4151110 msgr2=0x7f49d414ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-10T21:39:57.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:57.733+0000 7f49d2a00640 1 --2- 172.21.15.17:0/1959190397 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f49d4151110 0x7f49d414ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T21:39:57.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:57.733+0000 7f49d2a00640 1 --2- 172.21.15.17:0/1959190397 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f49d4151110 0x7f49d414ca30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-10T21:39:59.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:59.337+0000 7f49d2a00640 1 --2- 172.21.15.17:0/1959190397 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f49d4151110 0x7f49d414ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:39:59.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:59.337+0000 7f49d2a00640 1 -- 172.21.15.17:0/1959190397 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f49d40cbd80 con 0x7f49d4151110 2024-09-10T21:39:59.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:59.337+0000 7f49d2a00640 1 --2- 172.21.15.17:0/1959190397 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f49d4151110 0x7f49d414ca30 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f49d4151a40 tx=0x7f49c00016d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:39:59.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:59.337+0000 7f49cb400640 1 -- 172.21.15.17:0/1959190397 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49c001a030 con 0x7f49d4151110 2024-09-10T21:39:59.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:59.337+0000 7f49cb400640 1 -- 172.21.15.17:0/1959190397 <== mon.1 v2:172.21.15.31:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f49c0016660 con 0x7f49d4151110 2024-09-10T21:39:59.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:59.337+0000 7f49cb400640 1 -- 172.21.15.17:0/1959190397 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f49d4151110 msgr2=0x7f49d414ca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:39:59.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:59.337+0000 7f49cb400640 1 --2- 172.21.15.17:0/1959190397 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f49d4151110 0x7f49d414ca30 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f49d4151a40 tx=0x7f49c00016d0 comp rx=0 tx=0).stop 2024-09-10T21:39:59.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:59.337+0000 7f49cb400640 1 -- 172.21.15.17:0/1959190397 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f49d4151110 msgr2=0x7f49d414ca30 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:39:59.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:59.337+0000 7f49cb400640 1 --2- 172.21.15.17:0/1959190397 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f49d4151110 0x7f49d414ca30 secure :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f49d4151a40 tx=0x7f49c00016d0 comp rx=0 tx=0).stop 2024-09-10T21:39:59.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:59.345+0000 7f49d8e00640 1 --2- 172.21.15.17:0/1959190397 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f49d41e6960 0x7f49d414cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:39:59.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:59.345+0000 7f49d8e00640 1 -- 172.21.15.17:0/1959190397 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f49d414d4b0 con 0x7f49d41e6960 2024-09-10T21:39:59.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:59.345+0000 7f49d2a00640 1 --2- 172.21.15.17:0/1959190397 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f49d41e6960 0x7f49d414cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:39:59.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:59.345+0000 7f49d2a00640 1 -- 172.21.15.17:0/1959190397 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f49d40cc710 con 0x7f49d41e6960 2024-09-10T21:39:59.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:59.345+0000 7f49d2a00640 1 --2- 172.21.15.17:0/1959190397 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f49d41e6960 0x7f49d414cf70 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f49c001a000 tx=0x7f49c0001090 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:39:59.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:59.345+0000 7f49cb400640 1 -- 172.21.15.17:0/1959190397 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49c0029410 con 0x7f49d41e6960 2024-09-10T21:39:59.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:59.349+0000 7f49cb400640 1 -- 172.21.15.17:0/1959190397 <== mon.1 v2:172.21.15.31:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4541 (secure 0 0 0) 0x7f49c001e3d0 con 0x7f49d41e6960 2024-09-10T21:39:59.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:59.349+0000 7f49cb400640 1 -- 172.21.15.17:0/1959190397 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f49d41e6960 msgr2=0x7f49d414cf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:39:59.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:59.349+0000 7f49cb400640 1 --2- 172.21.15.17:0/1959190397 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f49d41e6960 0x7f49d414cf70 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f49c001a000 tx=0x7f49c0001090 comp rx=0 tx=0).stop 2024-09-10T21:39:59.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:59.349+0000 7f49cb400640 1 -- 172.21.15.17:0/1959190397 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f49d41e6960 msgr2=0x7f49d414cf70 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:39:59.352 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:59.349+0000 7f49cb400640 1 --2- 172.21.15.17:0/1959190397 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f49d41e6960 0x7f49d414cf70 secure :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f49c001a000 tx=0x7f49c0001090 comp rx=0 tx=0).stop 2024-09-10T21:39:59.352 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-10T21:39:59.353 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:39:59.353 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:39:59.353 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-10T21:39:59.353 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 122, 2024-09-10T21:39:59.353 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 10779, 2024-09-10T21:39:59.353 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-10T21:39:59.353 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-10T21:39:59.353 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-10T21:39:59.353 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-10T21:39:59.353 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:39:59.353 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 5, 2024-09-10T21:39:59.354 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:39:59.354 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-10T21:39:59.354 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-10T21:39:59.354 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:39:59.354 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:39:59.354 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:39:59.354 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:39:59.354 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:39:59.354 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:39:59.354 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:39:59.354 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:39:59.354 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:39:59.354 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:39:59.355 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:39:59.355 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:39:59.355 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-10T21:39:59.355 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-10T21:39:59.355 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:39:59.355 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:39:59.355 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:39:59.355 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:39:59.355 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:39:59.355 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:39:59.355 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:39:59.355 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:39:59.355 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:39:59.355 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:39:59.356 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:39:59.356 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:39:59.356 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:39:59.356 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-10T21:39:59.356 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-10T21:39:59.356 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-10T21:39:59.356 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-10T21:39:59.356 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-10T21:39:59.356 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "5e2f023f-d6a6-49f4-a256-f4807dce3739", 2024-09-10T21:39:59.356 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:39:59.356 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:39:59.356 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-10T21:39:59.356 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-10T21:39:59.357 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-10T21:39:59.357 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-10T21:39:59.357 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-10T21:39:59.357 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-10T21:39:59.357 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-10T21:39:59.357 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:39:59.357 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-10T21:39:59.357 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:39:59.357 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:39:59.357 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:39:59.357 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:39:59.357 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:39:59.357 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:39:59.358 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:39:59.358 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:39:59.358 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:39:59.358 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:39:59.358 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:39:59.358 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:39:59.358 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-10T21:39:59.358 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:39:59.358 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-10T21:39:59.358 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:39:59.358 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:39:59.358 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:39:59.358 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:39:59.358 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:39:59.358 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:39:59.359 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:39:59.359 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-10T21:39:59.359 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:39:59.359 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:39:59.359 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:39:59.359 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:39:59.359 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-10T21:39:59.359 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:39:59.359 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:39:59.359 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:39:59.359 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:39:59.359 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-10T21:39:59.359 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-10T21:39:59.360 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:39:59.360 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:39:59.360 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:39:59.360 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:39:59.360 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:39:59.360 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:39:59.360 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:39:59.360 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:39:59.360 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:39:59.360 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:39:59.360 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:39:59.360 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3300", 2024-09-10T21:39:59.360 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:39:59.361 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:39:59.361 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:39:59.361 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:39:59.361 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789", 2024-09-10T21:39:59.361 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:39:59.361 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:39:59.361 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:39:59.361 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:39:59.361 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789/0", 2024-09-10T21:39:59.361 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6789/0", 2024-09-10T21:39:59.361 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:39:59.361 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:39:59.361 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:39:59.362 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:39:59.362 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:39:59.362 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:39:59.362 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:39:59.362 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:39:59.362 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:39:59.362 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:39:59.362 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:39:59.362 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3301", 2024-09-10T21:39:59.362 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:39:59.362 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:39:59.362 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:39:59.362 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:39:59.362 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790", 2024-09-10T21:39:59.363 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:39:59.363 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:39:59.363 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:39:59.363 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:39:59.363 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790/0", 2024-09-10T21:39:59.363 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6790/0", 2024-09-10T21:39:59.363 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:39:59.363 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:39:59.363 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:39:59.363 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:39:59.363 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:39:59.363 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:39:59.363 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-10T21:39:59.364 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-10T21:39:59.364 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:39:59.364 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:39:59.364 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:39:59.364 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:39:59.364 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:39:59.364 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:39:59.364 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-10T21:39:59.364 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:39:59.364 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:39:59.364 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:39:59.364 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:39:59.364 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:39:59.365 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:39:59.365 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:39:59.365 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-10T21:39:59.365 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-10T21:39:59.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:59.349+0000 7f49d8e00640 1 -- 172.21.15.17:0/1959190397 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f49b40725e0 msgr2=0x7f49b4074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:39:59.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:59.349+0000 7f49d8e00640 1 --2- 172.21.15.17:0/1959190397 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f49b40725e0 0x7f49b4074aa0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f49cc009e70 tx=0x7f49cc009580 comp rx=0 tx=0).stop 2024-09-10T21:39:59.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:59.349+0000 7f49d8e00640 1 -- 172.21.15.17:0/1959190397 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f49d4152070 msgr2=0x7f49d414fb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:39:59.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:59.349+0000 7f49d8e00640 1 --2- 172.21.15.17:0/1959190397 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f49d4152070 0x7f49d414fb00 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f49c40af000 tx=0x7f49c40a8ca0 comp rx=0 tx=0).stop 2024-09-10T21:39:59.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:59.349+0000 7f49d3e00640 1 -- 172.21.15.17:0/1959190397 reap_dead start 2024-09-10T21:39:59.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:59.349+0000 7f49d3e00640 1 -- 172.21.15.17:0/1959190397 reap_dead start 2024-09-10T21:39:59.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:59.349+0000 7f49d8e00640 1 -- 172.21.15.17:0/1959190397 shutdown_connections 2024-09-10T21:39:59.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:59.349+0000 7f49d8e00640 1 -- 172.21.15.17:0/1959190397 >> 172.21.15.17:0/1959190397 conn(0x7f49d406cc20 msgr2=0x7f49d4157810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:39:59.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:59.349+0000 7f49d8e00640 1 -- 172.21.15.17:0/1959190397 shutdown_connections 2024-09-10T21:39:59.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:59.349+0000 7f49d8e00640 1 -- 172.21.15.17:0/1959190397 wait complete. 2024-09-10T21:39:59.374 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-10T21:39:59.375 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-09-10T21:39:59.375 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-10T21:39:59.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:59.537+0000 7fd214200640 1 -- 172.21.15.17:0/2282461702 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd20c06fa10 msgr2=0x7fd20c06fec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:39:59.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:59.537+0000 7fd214200640 1 --2- 172.21.15.17:0/2282461702 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd20c06fa10 0x7fd20c06fec0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fd204009c80 tx=0x7fd20401cda0 comp rx=0 tx=0).stop 2024-09-10T21:39:59.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:59.537+0000 7fd214200640 1 -- 172.21.15.17:0/2282461702 shutdown_connections 2024-09-10T21:39:59.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:59.537+0000 7fd214200640 1 --2- 172.21.15.17:0/2282461702 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd20c06fa10 0x7fd20c06fec0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:39:59.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:59.537+0000 7fd214200640 1 --2- 172.21.15.17:0/2282461702 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd20c154840 0x7fd20c15b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:39:59.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:59.537+0000 7fd214200640 1 -- 172.21.15.17:0/2282461702 >> 172.21.15.17:0/2282461702 conn(0x7fd20c0ccf50 msgr2=0x7fd20c0cd360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:39:59.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:59.537+0000 7fd214200640 1 -- 172.21.15.17:0/2282461702 shutdown_connections 2024-09-10T21:39:59.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:59.537+0000 7fd214200640 1 -- 172.21.15.17:0/2282461702 wait complete. 2024-09-10T21:39:59.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:59.537+0000 7fd214200640 1 Processor -- start 2024-09-10T21:39:59.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:59.537+0000 7fd214200640 1 -- start start 2024-09-10T21:39:59.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:59.537+0000 7fd214200640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd20c06fa10 0x7fd20c150ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:39:59.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:59.537+0000 7fd214200640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd20c0cda70 0x7fd20c164240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:39:59.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:59.537+0000 7fd214200640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd20c154840 0x7fd20c1fb210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:39:59.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:59.537+0000 7fd214200640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fd20c154da0 con 0x7fd20c06fa10 2024-09-10T21:39:59.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:59.537+0000 7fd214200640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fd20c154c20 con 0x7fd20c154840 2024-09-10T21:39:59.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:59.537+0000 7fd214200640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7fd20c06d990 con 0x7fd20c0cda70 2024-09-10T21:39:59.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:59.537+0000 7fd211800640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd20c06fa10 0x7fd20c150ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:39:59.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:59.537+0000 7fd211800640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd20c06fa10 0x7fd20c150ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45184/0 (socket says 172.21.15.17:45184) 2024-09-10T21:39:59.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:59.537+0000 7fd211800640 1 -- 172.21.15.17:0/1010859113 learned_addr learned my addr 172.21.15.17:0/1010859113 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:39:59.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:59.537+0000 7fd212200640 1 --2- 172.21.15.17:0/1010859113 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd20c154840 0x7fd20c1fb210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:39:59.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:59.541+0000 7fd210e00640 1 --2- 172.21.15.17:0/1010859113 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd20c0cda70 0x7fd20c164240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:39:59.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:59.541+0000 7fd211800640 1 -- 172.21.15.17:0/1010859113 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd20c0cda70 msgr2=0x7fd20c164240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:39:59.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:59.541+0000 7fd211800640 1 --2- 172.21.15.17:0/1010859113 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd20c0cda70 0x7fd20c164240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:39:59.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:59.541+0000 7fd211800640 1 -- 172.21.15.17:0/1010859113 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd20c154840 msgr2=0x7fd20c1fb210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:39:59.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:59.541+0000 7fd211800640 1 --2- 172.21.15.17:0/1010859113 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd20c154840 0x7fd20c1fb210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:39:59.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:59.541+0000 7fd211800640 1 -- 172.21.15.17:0/1010859113 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd20c153850 con 0x7fd20c06fa10 2024-09-10T21:39:59.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:59.541+0000 7fd212200640 1 --2- 172.21.15.17:0/1010859113 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd20c154840 0x7fd20c1fb210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:39:59.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:59.541+0000 7fd211800640 1 --2- 172.21.15.17:0/1010859113 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd20c06fa10 0x7fd20c150ec0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fd20000e7a0 tx=0x7fd20000ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:39:59.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:59.541+0000 7fd209600640 1 -- 172.21.15.17:0/1010859113 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd20000ccc0 con 0x7fd20c06fa10 2024-09-10T21:39:59.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:59.541+0000 7fd209600640 1 -- 172.21.15.17:0/1010859113 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd200010430 con 0x7fd20c06fa10 2024-09-10T21:39:59.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:59.541+0000 7fd209600640 1 -- 172.21.15.17:0/1010859113 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd200010860 con 0x7fd20c06fa10 2024-09-10T21:39:59.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:59.541+0000 7fd214200640 1 -- 172.21.15.17:0/1010859113 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd20c154390 con 0x7fd20c06fa10 2024-09-10T21:39:59.546 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:59.541+0000 7fd214200640 1 -- 172.21.15.17:0/1010859113 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd20c153dd0 con 0x7fd20c06fa10 2024-09-10T21:39:59.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:59.541+0000 7fd214200640 1 --2- 172.21.15.17:0/1010859113 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd20c152fe0 0x7fd20c1fb750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:39:59.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:59.541+0000 7fd214200640 1 -- 172.21.15.17:0/1010859113 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7fd20c076d80 con 0x7fd20c152fe0 2024-09-10T21:39:59.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:59.541+0000 7fd210e00640 1 --2- 172.21.15.17:0/1010859113 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd20c152fe0 0x7fd20c1fb750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:39:59.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:59.541+0000 7fd210e00640 1 -- 172.21.15.17:0/1010859113 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd20c06fec0 con 0x7fd20c152fe0 2024-09-10T21:39:59.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:59.541+0000 7fd210e00640 1 --2- 172.21.15.17:0/1010859113 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd20c152fe0 0x7fd20c1fb750 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fd20c0cd880 tx=0x7fd1f800e960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:39:59.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:59.541+0000 7fd209600640 1 -- 172.21.15.17:0/1010859113 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1f80052e0 con 0x7fd20c152fe0 2024-09-10T21:39:59.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:59.541+0000 7fd209600640 1 -- 172.21.15.17:0/1010859113 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fd200010a00 con 0x7fd20c06fa10 2024-09-10T21:39:59.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:59.545+0000 7fd209600640 1 --2- 172.21.15.17:0/1010859113 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fd1ec0724e0 0x7fd1ec0749a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:39:59.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:59.545+0000 7fd209600640 1 -- 172.21.15.17:0/1010859113 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fd200094190 con 0x7fd20c06fa10 2024-09-10T21:39:59.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:59.545+0000 7fd209600640 1 -- 172.21.15.17:0/1010859113 <== mon.2 v2:172.21.15.31:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fd1f8005480 con 0x7fd20c152fe0 2024-09-10T21:39:59.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:59.545+0000 7fd209600640 1 -- 172.21.15.17:0/1010859113 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd20c152fe0 msgr2=0x7fd20c1fb750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:39:59.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:59.545+0000 7fd209600640 1 --2- 172.21.15.17:0/1010859113 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd20c152fe0 0x7fd20c1fb750 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fd20c0cd880 tx=0x7fd1f800e960 comp rx=0 tx=0).stop 2024-09-10T21:39:59.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:59.545+0000 7fd209600640 1 -- 172.21.15.17:0/1010859113 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd20c152fe0 msgr2=0x7fd20c1fb750 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:39:59.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:59.545+0000 7fd209600640 1 --2- 172.21.15.17:0/1010859113 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd20c152fe0 0x7fd20c1fb750 secure :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fd20c0cd880 tx=0x7fd1f800e960 comp rx=0 tx=0).stop 2024-09-10T21:39:59.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:59.545+0000 7fd212200640 1 --2- 172.21.15.17:0/1010859113 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fd1ec0724e0 0x7fd1ec0749a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:39:59.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:59.545+0000 7fd212200640 1 --2- 172.21.15.17:0/1010859113 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fd1ec0724e0 0x7fd1ec0749a0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fd2040af000 tx=0x7fd20401b5e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:39:59.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:59.553+0000 7fd214200640 1 --2- 172.21.15.17:0/1010859113 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd20c143660 0x7fd20c143a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:39:59.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:59.553+0000 7fd214200640 1 -- 172.21.15.17:0/1010859113 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7fd20c074de0 con 0x7fd20c143660 2024-09-10T21:39:59.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:59.553+0000 7fd210e00640 1 --2- 172.21.15.17:0/1010859113 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd20c143660 0x7fd20c143a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:39:59.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:59.553+0000 7fd210e00640 1 -- 172.21.15.17:0/1010859113 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fd20c0705b0 con 0x7fd20c143660 2024-09-10T21:39:59.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:59.553+0000 7fd210e00640 1 --2- 172.21.15.17:0/1010859113 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd20c143660 0x7fd20c143a40 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fd1f8007fd0 tx=0x7fd1f8001090 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:39:59.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:59.553+0000 7fd209600640 1 -- 172.21.15.17:0/1010859113 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1f8003810 con 0x7fd20c143660 2024-09-10T21:39:59.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:59.553+0000 7fd209600640 1 -- 172.21.15.17:0/1010859113 <== mon.2 v2:172.21.15.31:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5043 (secure 0 0 0) 0x7fd1f8003810 con 0x7fd20c143660 2024-09-10T21:39:59.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:59.553+0000 7fd209600640 1 -- 172.21.15.17:0/1010859113 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd20c143660 msgr2=0x7fd20c143a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:39:59.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:59.553+0000 7fd209600640 1 --2- 172.21.15.17:0/1010859113 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd20c143660 0x7fd20c143a40 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fd1f8007fd0 tx=0x7fd1f8001090 comp rx=0 tx=0).stop 2024-09-10T21:39:59.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:59.553+0000 7fd209600640 1 -- 172.21.15.17:0/1010859113 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd20c143660 msgr2=0x7fd20c143a40 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:39:59.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:59.553+0000 7fd209600640 1 --2- 172.21.15.17:0/1010859113 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd20c143660 0x7fd20c143a40 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:39:59.559 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-10T21:39:59.560 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:39:59.560 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:39:59.560 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-10T21:39:59.560 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 122, 2024-09-10T21:39:59.560 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 205904, 2024-09-10T21:39:59.560 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-10T21:39:59.560 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-10T21:39:59.560 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-10T21:39:59.560 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-10T21:39:59.560 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:39:59.560 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 5, 2024-09-10T21:39:59.560 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:39:59.561 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-10T21:39:59.561 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-10T21:39:59.561 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:39:59.561 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:39:59.561 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:39:59.561 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:39:59.561 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:39:59.561 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:39:59.561 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:39:59.561 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:39:59.561 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:39:59.561 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:39:59.561 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:39:59.562 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:39:59.562 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-10T21:39:59.562 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-10T21:39:59.562 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:39:59.562 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:39:59.562 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:39:59.562 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:39:59.562 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:39:59.562 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:39:59.562 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:39:59.562 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:39:59.562 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:39:59.562 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:39:59.563 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:39:59.563 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:39:59.563 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:39:59.563 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-10T21:39:59.563 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-10T21:39:59.563 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-10T21:39:59.563 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-10T21:39:59.563 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-10T21:39:59.563 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "5e2f023f-d6a6-49f4-a256-f4807dce3739", 2024-09-10T21:39:59.563 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:39:59.563 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:39:59.563 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-10T21:39:59.563 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-10T21:39:59.564 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-10T21:39:59.564 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-10T21:39:59.564 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-10T21:39:59.564 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-10T21:39:59.564 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-10T21:39:59.564 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:39:59.564 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-10T21:39:59.564 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:39:59.564 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:39:59.564 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:39:59.564 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:39:59.564 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:39:59.564 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:39:59.565 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:39:59.565 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:39:59.565 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:39:59.565 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:39:59.565 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:39:59.565 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:39:59.565 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-10T21:39:59.565 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:39:59.565 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-10T21:39:59.565 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:39:59.565 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:39:59.565 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:39:59.565 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:39:59.566 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:39:59.566 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:39:59.566 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:39:59.566 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-10T21:39:59.566 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:39:59.566 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:39:59.566 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:39:59.566 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:39:59.566 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-10T21:39:59.566 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:39:59.566 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:39:59.566 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:39:59.567 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:39:59.567 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-10T21:39:59.567 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-10T21:39:59.567 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:39:59.567 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:39:59.567 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:39:59.567 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:39:59.567 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:39:59.567 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:39:59.567 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:39:59.567 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:39:59.567 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:39:59.567 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:39:59.568 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:39:59.568 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3300", 2024-09-10T21:39:59.568 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:39:59.568 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:39:59.568 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:39:59.568 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:39:59.568 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789", 2024-09-10T21:39:59.568 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:39:59.568 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:39:59.568 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:39:59.568 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:39:59.568 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789/0", 2024-09-10T21:39:59.568 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6789/0", 2024-09-10T21:39:59.569 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:39:59.569 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:39:59.569 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:39:59.569 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:39:59.569 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:39:59.569 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:39:59.569 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:39:59.569 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:39:59.569 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:39:59.569 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:39:59.569 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:39:59.569 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3301", 2024-09-10T21:39:59.570 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:39:59.570 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:39:59.570 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:39:59.570 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:39:59.570 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790", 2024-09-10T21:39:59.570 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:39:59.570 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:39:59.570 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:39:59.570 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:39:59.570 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790/0", 2024-09-10T21:39:59.570 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6790/0", 2024-09-10T21:39:59.570 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:39:59.570 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:39:59.570 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:39:59.571 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:39:59.571 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:39:59.571 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:39:59.571 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-10T21:39:59.571 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-10T21:39:59.571 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:39:59.571 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:39:59.571 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:39:59.571 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:39:59.571 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:39:59.571 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:39:59.571 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": [ 2024-09-10T21:39:59.571 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:39:59.572 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:39:59.572 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:39:59.572 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 6 2024-09-10T21:39:59.572 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:39:59.572 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:39:59.572 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-10T21:39:59.572 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:39:59.572 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:39:59.572 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:39:59.572 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 6 2024-09-10T21:39:59.572 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:39:59.572 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:39:59.573 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-10T21:39:59.573 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:39:59.573 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:39:59.573 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:39:59.573 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 4 2024-09-10T21:39:59.573 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:39:59.573 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:39:59.573 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": [ 2024-09-10T21:39:59.573 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:39:59.573 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:39:59.573 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:39:59.573 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 2 2024-09-10T21:39:59.573 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:39:59.573 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:39:59.574 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:39:59.574 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-10T21:39:59.574 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-10T21:39:59.574 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:59.553+0000 7fd214200640 1 -- 172.21.15.17:0/1010859113 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fd1ec0724e0 msgr2=0x7fd1ec0749a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:39:59.575 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:59.553+0000 7fd214200640 1 --2- 172.21.15.17:0/1010859113 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fd1ec0724e0 0x7fd1ec0749a0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fd2040af000 tx=0x7fd20401b5e0 comp rx=0 tx=0).stop 2024-09-10T21:39:59.575 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:59.553+0000 7fd214200640 1 -- 172.21.15.17:0/1010859113 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd20c06fa10 msgr2=0x7fd20c150ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:39:59.575 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:59.553+0000 7fd214200640 1 --2- 172.21.15.17:0/1010859113 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd20c06fa10 0x7fd20c150ec0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fd20000e7a0 tx=0x7fd20000ec60 comp rx=0 tx=0).stop 2024-09-10T21:39:59.575 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:59.553+0000 7fd212200640 1 -- 172.21.15.17:0/1010859113 reap_dead start 2024-09-10T21:39:59.575 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:59.553+0000 7fd212200640 1 -- 172.21.15.17:0/1010859113 reap_dead start 2024-09-10T21:39:59.575 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:59.553+0000 7fd214200640 1 -- 172.21.15.17:0/1010859113 shutdown_connections 2024-09-10T21:39:59.575 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:59.553+0000 7fd214200640 1 -- 172.21.15.17:0/1010859113 >> 172.21.15.17:0/1010859113 conn(0x7fd20c0ccf50 msgr2=0x7fd20c14d520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:39:59.575 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:59.553+0000 7fd214200640 1 -- 172.21.15.17:0/1010859113 shutdown_connections 2024-09-10T21:39:59.575 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:59.557+0000 7fd214200640 1 -- 172.21.15.17:0/1010859113 wait complete. 2024-09-10T21:39:59.579 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-10T21:39:59.580 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-09-10T21:39:59.580 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-09-10T21:39:59.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:59.741+0000 7f3a54800640 1 -- 172.21.15.17:0/1150564541 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3a4c152820 msgr2=0x7f3a4c0b3eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:39:59.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:59.741+0000 7f3a54800640 1 --2- 172.21.15.17:0/1150564541 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3a4c152820 0x7f3a4c0b3eb0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f3a4c0c9850 tx=0x7f3a3c01c830 comp rx=0 tx=0).stop 2024-09-10T21:39:59.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:59.741+0000 7f3a54800640 1 -- 172.21.15.17:0/1150564541 shutdown_connections 2024-09-10T21:39:59.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:59.741+0000 7f3a54800640 1 --2- 172.21.15.17:0/1150564541 >> v2:172.21.15.31:3300/0 conn(0x7f3a4c0cdf20 0x7f3a4c0ce350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:39:59.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:59.741+0000 7f3a54800640 1 --2- 172.21.15.17:0/1150564541 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3a4c152820 0x7f3a4c0b3eb0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:39:59.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:59.741+0000 7f3a54800640 1 -- 172.21.15.17:0/1150564541 >> 172.21.15.17:0/1150564541 conn(0x7f3a4c0702f0 msgr2=0x7f3a4c070700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:39:59.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:59.741+0000 7f3a54800640 1 -- 172.21.15.17:0/1150564541 shutdown_connections 2024-09-10T21:39:59.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:59.741+0000 7f3a54800640 1 -- 172.21.15.17:0/1150564541 wait complete. 2024-09-10T21:39:59.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:59.741+0000 7f3a54800640 1 Processor -- start 2024-09-10T21:39:59.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:59.741+0000 7f3a54800640 1 -- start start 2024-09-10T21:39:59.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:59.745+0000 7f3a54800640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3a4c0cdf20 0x7f3a4c14a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:39:59.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:59.745+0000 7f3a54800640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3a4c152440 0x7f3a4c0c60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:39:59.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:59.745+0000 7f3a54800640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3a4c152820 0x7f3a4c1614b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:39:59.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:59.745+0000 7f3a54800640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f3a4c0cb5d0 con 0x7f3a4c152820 2024-09-10T21:39:59.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:59.745+0000 7f3a54800640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f3a4c0cb450 con 0x7f3a4c152440 2024-09-10T21:39:59.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:59.745+0000 7f3a54800640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f3a4c0c2ff0 con 0x7f3a4c0cdf20 2024-09-10T21:39:59.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:59.745+0000 7f3a52800640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3a4c152820 0x7f3a4c1614b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:39:59.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:59.745+0000 7f3a52800640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3a4c152820 0x7f3a4c1614b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45202/0 (socket says 172.21.15.17:45202) 2024-09-10T21:39:59.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:59.745+0000 7f3a52800640 1 -- 172.21.15.17:0/900328793 learned_addr learned my addr 172.21.15.17:0/900328793 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:39:59.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:59.745+0000 7f3a51400640 1 --2- 172.21.15.17:0/900328793 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3a4c152440 0x7f3a4c0c60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:39:59.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:59.745+0000 7f3a51e00640 1 --2- 172.21.15.17:0/900328793 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3a4c0cdf20 0x7f3a4c14a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:39:59.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:59.745+0000 7f3a52800640 1 -- 172.21.15.17:0/900328793 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3a4c0cdf20 msgr2=0x7f3a4c14a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:39:59.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:59.745+0000 7f3a52800640 1 --2- 172.21.15.17:0/900328793 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3a4c0cdf20 0x7f3a4c14a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:39:59.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:59.745+0000 7f3a52800640 1 -- 172.21.15.17:0/900328793 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3a4c152440 msgr2=0x7f3a4c0c60f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:39:59.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:59.745+0000 7f3a52800640 1 --2- 172.21.15.17:0/900328793 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3a4c152440 0x7f3a4c0c60f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:39:59.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:59.745+0000 7f3a52800640 1 -- 172.21.15.17:0/900328793 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a4c0cab90 con 0x7f3a4c152820 2024-09-10T21:39:59.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:59.745+0000 7f3a51e00640 1 --2- 172.21.15.17:0/900328793 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3a4c0cdf20 0x7f3a4c14a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:39:59.750 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:59.745+0000 7f3a52800640 1 --2- 172.21.15.17:0/900328793 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3a4c152820 0x7f3a4c1614b0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f3a4800e970 tx=0x7f3a4800ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:39:59.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:59.745+0000 7f3a46000640 1 -- 172.21.15.17:0/900328793 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a48004d30 con 0x7f3a4c152820 2024-09-10T21:39:59.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:59.745+0000 7f3a46000640 1 -- 172.21.15.17:0/900328793 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3a4800cb20 con 0x7f3a4c152820 2024-09-10T21:39:59.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:59.745+0000 7f3a46000640 1 -- 172.21.15.17:0/900328793 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a48005020 con 0x7f3a4c152820 2024-09-10T21:39:59.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:59.745+0000 7f3a54800640 1 -- 172.21.15.17:0/900328793 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a4c1f73b0 con 0x7f3a4c152820 2024-09-10T21:39:59.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:59.745+0000 7f3a54800640 1 -- 172.21.15.17:0/900328793 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a4c162780 con 0x7f3a4c152820 2024-09-10T21:39:59.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:59.745+0000 7f3a54800640 1 -- 172.21.15.17:0/900328793 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a4c158360 con 0x7f3a4c152820 2024-09-10T21:39:59.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:59.749+0000 7f3a46000640 1 -- 172.21.15.17:0/900328793 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f3a4800f040 con 0x7f3a4c152820 2024-09-10T21:39:59.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:59.749+0000 7f3a46000640 1 --2- 172.21.15.17:0/900328793 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f3a30072610 0x7f3a30074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:39:59.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:59.749+0000 7f3a46000640 1 -- 172.21.15.17:0/900328793 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f3a48093c10 con 0x7f3a4c152820 2024-09-10T21:39:59.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:59.749+0000 7f3a51e00640 1 --2- 172.21.15.17:0/900328793 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f3a30072610 0x7f3a30074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:39:59.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:59.749+0000 7f3a51e00640 1 --2- 172.21.15.17:0/900328793 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f3a30072610 0x7f3a30074ad0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f3a40003ec0 tx=0x7f3a40009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:39:59.757 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:39:59.753+0000 7f3a46000640 1 -- 172.21.15.17:0/900328793 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f3a48016030 con 0x7f3a4c152820 2024-09-10T21:40:00.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:00.029+0000 7f3a54800640 1 -- 172.21.15.17:0/900328793 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) -- 0x7f3a4c1433e0 con 0x7f3a4c152820 2024-09-10T21:40:00.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:00.033+0000 7f3a46000640 1 -- 172.21.15.17:0/900328793 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) ==== 57+0+0 (secure 0 0 0) 0x7f3a4805e800 con 0x7f3a4c152820 2024-09-10T21:40:00.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:00.033+0000 7f3a54800640 1 -- 172.21.15.17:0/900328793 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f3a30072610 msgr2=0x7f3a30074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:40:00.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:00.033+0000 7f3a54800640 1 --2- 172.21.15.17:0/900328793 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f3a30072610 0x7f3a30074ad0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f3a40003ec0 tx=0x7f3a40009040 comp rx=0 tx=0).stop 2024-09-10T21:40:00.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:00.033+0000 7f3a54800640 1 -- 172.21.15.17:0/900328793 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3a4c152820 msgr2=0x7f3a4c1614b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:40:00.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:00.033+0000 7f3a54800640 1 --2- 172.21.15.17:0/900328793 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3a4c152820 0x7f3a4c1614b0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f3a4800e970 tx=0x7f3a4800ee30 comp rx=0 tx=0).stop 2024-09-10T21:40:00.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:00.033+0000 7f3a54800640 1 -- 172.21.15.17:0/900328793 shutdown_connections 2024-09-10T21:40:00.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:00.033+0000 7f3a54800640 1 --2- 172.21.15.17:0/900328793 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f3a30072610 0x7f3a30074ad0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:40:00.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:00.033+0000 7f3a54800640 1 --2- 172.21.15.17:0/900328793 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3a4c152820 0x7f3a4c1614b0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:40:00.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:00.033+0000 7f3a54800640 1 --2- 172.21.15.17:0/900328793 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3a4c152440 0x7f3a4c0c60f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:40:00.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:00.033+0000 7f3a54800640 1 --2- 172.21.15.17:0/900328793 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3a4c0cdf20 0x7f3a4c14a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:40:00.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:00.033+0000 7f3a54800640 1 -- 172.21.15.17:0/900328793 >> 172.21.15.17:0/900328793 conn(0x7f3a4c0702f0 msgr2=0x7f3a4c0c8f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:40:00.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:00.033+0000 7f3a54800640 1 -- 172.21.15.17:0/900328793 shutdown_connections 2024-09-10T21:40:00.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:00.033+0000 7f3a54800640 1 -- 172.21.15.17:0/900328793 wait complete. 2024-09-10T21:40:00.064 INFO:tasks.mon_thrash.mon_thrasher:waiting for 10.0 secs before continuing thrashing 2024-09-10T21:40:10.065 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-09-10T21:40:10.065 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-10T21:40:10.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.225+0000 7f1e57c00640 1 -- 172.21.15.17:0/2257804547 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1e500cdf20 msgr2=0x7f1e500ce3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:40:10.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.225+0000 7f1e57c00640 1 --2- 172.21.15.17:0/2257804547 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1e500cdf20 0x7f1e500ce3b0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f1e50073220 tx=0x7f1e4801ccf0 comp rx=0 tx=0).stop 2024-09-10T21:40:10.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.225+0000 7f1e57c00640 1 -- 172.21.15.17:0/2257804547 shutdown_connections 2024-09-10T21:40:10.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.225+0000 7f1e57c00640 1 --2- 172.21.15.17:0/2257804547 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1e500cdf20 0x7f1e500ce3b0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:40:10.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.225+0000 7f1e57c00640 1 --2- 172.21.15.17:0/2257804547 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f1e50152820 0x7f1e5015c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:40:10.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.225+0000 7f1e57c00640 1 --2- 172.21.15.17:0/2257804547 >> v2:172.21.15.31:3300/0 conn(0x7f1e50152440 0x7f1e500b3eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:40:10.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.225+0000 7f1e57c00640 1 -- 172.21.15.17:0/2257804547 >> 172.21.15.17:0/2257804547 conn(0x7f1e500702f0 msgr2=0x7f1e50070700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:40:10.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.225+0000 7f1e57c00640 1 -- 172.21.15.17:0/2257804547 shutdown_connections 2024-09-10T21:40:10.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.225+0000 7f1e57c00640 1 -- 172.21.15.17:0/2257804547 wait complete. 2024-09-10T21:40:10.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.225+0000 7f1e57c00640 1 Processor -- start 2024-09-10T21:40:10.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.225+0000 7f1e57c00640 1 -- start start 2024-09-10T21:40:10.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.229+0000 7f1e57c00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1e500cdf20 0x7f1e500b2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:40:10.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.229+0000 7f1e57c00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e50152440 0x7f1e501fa9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:40:10.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.229+0000 7f1e57c00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f1e50152820 0x7f1e501faf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:40:10.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.229+0000 7f1e57c00640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f1e500cb450 con 0x7f1e500cdf20 2024-09-10T21:40:10.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.229+0000 7f1e57c00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f1e5015c7c0 con 0x7f1e50152440 2024-09-10T21:40:10.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.229+0000 7f1e57c00640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f1e500cb5d0 con 0x7f1e50152820 2024-09-10T21:40:10.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.229+0000 7f1e55200640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1e500cdf20 0x7f1e500b2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:40:10.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.229+0000 7f1e55200640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1e500cdf20 0x7f1e500b2ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33714/0 (socket says 172.21.15.17:33714) 2024-09-10T21:40:10.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.229+0000 7f1e55200640 1 -- 172.21.15.17:0/4128601911 learned_addr learned my addr 172.21.15.17:0/4128601911 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:40:10.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.229+0000 7f1e4fe00640 1 --2- 172.21.15.17:0/4128601911 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e50152440 0x7f1e501fa9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:40:10.233 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.229+0000 7f1e55c00640 1 --2- 172.21.15.17:0/4128601911 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f1e50152820 0x7f1e501faf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:40:10.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.229+0000 7f1e55200640 1 -- 172.21.15.17:0/4128601911 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f1e50152820 msgr2=0x7f1e501faf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:40:10.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.229+0000 7f1e55200640 1 --2- 172.21.15.17:0/4128601911 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f1e50152820 0x7f1e501faf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:40:10.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.229+0000 7f1e55200640 1 -- 172.21.15.17:0/4128601911 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e50152440 msgr2=0x7f1e501fa9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:40:10.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.229+0000 7f1e55200640 1 --2- 172.21.15.17:0/4128601911 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e50152440 0x7f1e501fa9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:40:10.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.229+0000 7f1e55200640 1 -- 172.21.15.17:0/4128601911 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e501f8310 con 0x7f1e500cdf20 2024-09-10T21:40:10.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.229+0000 7f1e4fe00640 1 --2- 172.21.15.17:0/4128601911 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e50152440 0x7f1e501fa9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:40:10.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.229+0000 7f1e55200640 1 --2- 172.21.15.17:0/4128601911 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1e500cdf20 0x7f1e500b2ef0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f1e4400d770 tx=0x7f1e4400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:40:10.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.229+0000 7f1e4cc00640 1 -- 172.21.15.17:0/4128601911 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e44017070 con 0x7f1e500cdf20 2024-09-10T21:40:10.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.229+0000 7f1e4cc00640 1 -- 172.21.15.17:0/4128601911 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1e44009d20 con 0x7f1e500cdf20 2024-09-10T21:40:10.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.229+0000 7f1e4cc00640 1 -- 172.21.15.17:0/4128601911 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e440106c0 con 0x7f1e500cdf20 2024-09-10T21:40:10.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.229+0000 7f1e57c00640 1 -- 172.21.15.17:0/4128601911 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e5015a5a0 con 0x7f1e500cdf20 2024-09-10T21:40:10.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.229+0000 7f1e57c00640 1 -- 172.21.15.17:0/4128601911 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e5015a860 con 0x7f1e500cdf20 2024-09-10T21:40:10.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.229+0000 7f1e57c00640 1 -- 172.21.15.17:0/4128601911 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e500c30d0 con 0x7f1e500cdf20 2024-09-10T21:40:10.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.233+0000 7f1e4cc00640 1 -- 172.21.15.17:0/4128601911 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f1e44010860 con 0x7f1e500cdf20 2024-09-10T21:40:10.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.233+0000 7f1e4cc00640 1 --2- 172.21.15.17:0/4128601911 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f1e300725e0 0x7f1e30074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:40:10.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.233+0000 7f1e4cc00640 1 -- 172.21.15.17:0/4128601911 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f1e44093a50 con 0x7f1e500cdf20 2024-09-10T21:40:10.238 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.233+0000 7f1e4fe00640 1 --2- 172.21.15.17:0/4128601911 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f1e300725e0 0x7f1e30074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:40:10.239 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.233+0000 7f1e4fe00640 1 --2- 172.21.15.17:0/4128601911 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f1e300725e0 0x7f1e30074aa0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f1e501522f0 tx=0x7f1e3c00b390 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:40:10.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.237+0000 7f1e4cc00640 1 -- 172.21.15.17:0/4128601911 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f1e4405e6d0 con 0x7f1e500cdf20 2024-09-10T21:40:10.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.529+0000 7f1e57c00640 1 -- 172.21.15.17:0/4128601911 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f1e500c8ed0 con 0x7f1e500cdf20 2024-09-10T21:40:10.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.529+0000 7f1e4cc00640 1 -- 172.21.15.17:0/4128601911 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1558 (secure 0 0 0) 0x7f1e44063100 con 0x7f1e500cdf20 2024-09-10T21:40:10.535 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":122,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":16,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5e2f023f-d6a6-49f4-a256-f4807dce3739","modified":"2024-09-10T21:23:46.892130Z","created":"2024-09-10T21:23:46.892130Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3301","nonce":0},{"type":"v1","addr":"172.21.15.31:6790","nonce":0}]},"addr":"172.21.15.31:6790/0","public_addr":"172.21.15.31:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-10T21:40:10.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.533+0000 7f1e57c00640 1 -- 172.21.15.17:0/4128601911 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f1e300725e0 msgr2=0x7f1e30074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:40:10.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.533+0000 7f1e57c00640 1 --2- 172.21.15.17:0/4128601911 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f1e300725e0 0x7f1e30074aa0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f1e501522f0 tx=0x7f1e3c00b390 comp rx=0 tx=0).stop 2024-09-10T21:40:10.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.533+0000 7f1e57c00640 1 -- 172.21.15.17:0/4128601911 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1e500cdf20 msgr2=0x7f1e500b2ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:40:10.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.533+0000 7f1e57c00640 1 --2- 172.21.15.17:0/4128601911 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1e500cdf20 0x7f1e500b2ef0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f1e4400d770 tx=0x7f1e4400dc30 comp rx=0 tx=0).stop 2024-09-10T21:40:10.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.533+0000 7f1e57c00640 1 -- 172.21.15.17:0/4128601911 shutdown_connections 2024-09-10T21:40:10.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.533+0000 7f1e57c00640 1 --2- 172.21.15.17:0/4128601911 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f1e300725e0 0x7f1e30074aa0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:40:10.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.533+0000 7f1e57c00640 1 --2- 172.21.15.17:0/4128601911 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f1e50152820 0x7f1e501faf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:40:10.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.533+0000 7f1e57c00640 1 --2- 172.21.15.17:0/4128601911 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e50152440 0x7f1e501fa9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:40:10.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.533+0000 7f1e57c00640 1 --2- 172.21.15.17:0/4128601911 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1e500cdf20 0x7f1e500b2ef0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:40:10.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.533+0000 7f1e57c00640 1 -- 172.21.15.17:0/4128601911 >> 172.21.15.17:0/4128601911 conn(0x7f1e500702f0 msgr2=0x7f1e50151960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:40:10.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.533+0000 7f1e57c00640 1 -- 172.21.15.17:0/4128601911 shutdown_connections 2024-09-10T21:40:10.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.533+0000 7f1e57c00640 1 -- 172.21.15.17:0/4128601911 wait complete. 2024-09-10T21:40:10.560 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-09-10T21:40:10.560 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-09-10T21:40:10.560 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-09-10T21:40:10.560 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-10T21:40:10.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.721+0000 7f809ca00640 1 -- 172.21.15.17:0/2100808053 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f80981523d0 msgr2=0x7f80980c9f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:40:10.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.721+0000 7f809ca00640 1 --2- 172.21.15.17:0/2100808053 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f80981523d0 0x7f80980c9f60 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f8088009870 tx=0x7f808801c740 comp rx=0 tx=0).stop 2024-09-10T21:40:10.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.721+0000 7f809ca00640 1 -- 172.21.15.17:0/2100808053 shutdown_connections 2024-09-10T21:40:10.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.721+0000 7f809ca00640 1 --2- 172.21.15.17:0/2100808053 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f80981523d0 0x7f80980c9f60 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:40:10.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.721+0000 7f809ca00640 1 -- 172.21.15.17:0/2100808053 >> 172.21.15.17:0/2100808053 conn(0x7f8098068990 msgr2=0x7f8098068da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:40:10.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.721+0000 7f809ca00640 1 -- 172.21.15.17:0/2100808053 shutdown_connections 2024-09-10T21:40:10.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.721+0000 7f809ca00640 1 -- 172.21.15.17:0/2100808053 wait complete. 2024-09-10T21:40:10.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.721+0000 7f809ca00640 1 Processor -- start 2024-09-10T21:40:10.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.721+0000 7f809ca00640 1 -- start start 2024-09-10T21:40:10.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.721+0000 7f809ca00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f80980c9520 0x7f8098156530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:40:10.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.721+0000 7f809ca00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f80980ca4a0 0x7f80981571e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:40:10.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.721+0000 7f809ca00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f80981523d0 0x7f809806ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:40:10.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.721+0000 7f809ca00640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f80980714a0 con 0x7f80980ca4a0 2024-09-10T21:40:10.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.721+0000 7f809ca00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f8098071320 con 0x7f80980c9520 2024-09-10T21:40:10.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.721+0000 7f809ca00640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f80981669e0 con 0x7f80981523d0 2024-09-10T21:40:10.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.725+0000 7f8095e00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f80980ca4a0 0x7f80981571e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:40:10.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.725+0000 7f8095e00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f80980ca4a0 0x7f80981571e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33722/0 (socket says 172.21.15.17:33722) 2024-09-10T21:40:10.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.725+0000 7f8095e00640 1 -- 172.21.15.17:0/4155514444 learned_addr learned my addr 172.21.15.17:0/4155514444 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:40:10.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.725+0000 7f8096800640 1 --2- 172.21.15.17:0/4155514444 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f80980c9520 0x7f8098156530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:40:10.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.725+0000 7f8097200640 1 --2- 172.21.15.17:0/4155514444 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f80981523d0 0x7f809806ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:40:10.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.725+0000 7f8095e00640 1 -- 172.21.15.17:0/4155514444 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f80981523d0 msgr2=0x7f809806ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:40:10.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.725+0000 7f8095e00640 1 --2- 172.21.15.17:0/4155514444 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f80981523d0 0x7f809806ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:40:10.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.725+0000 7f8095e00640 1 -- 172.21.15.17:0/4155514444 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f80980c9520 msgr2=0x7f8098156530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:40:10.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.725+0000 7f8095e00640 1 --2- 172.21.15.17:0/4155514444 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f80980c9520 0x7f8098156530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:40:10.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.725+0000 7f8095e00640 1 -- 172.21.15.17:0/4155514444 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80980cdca0 con 0x7f80980ca4a0 2024-09-10T21:40:10.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.725+0000 7f8096800640 1 --2- 172.21.15.17:0/4155514444 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f80980c9520 0x7f8098156530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:40:10.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.725+0000 7f8095e00640 1 --2- 172.21.15.17:0/4155514444 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f80980ca4a0 0x7f80981571e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f808400c460 tx=0x7f808400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:40:10.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.725+0000 7f808ea00640 1 -- 172.21.15.17:0/4155514444 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8084016020 con 0x7f80980ca4a0 2024-09-10T21:40:10.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.725+0000 7f808ea00640 1 -- 172.21.15.17:0/4155514444 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8084012a00 con 0x7f80980ca4a0 2024-09-10T21:40:10.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.725+0000 7f809ca00640 1 -- 172.21.15.17:0/4155514444 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f809806e6d0 con 0x7f80980ca4a0 2024-09-10T21:40:10.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.725+0000 7f808ea00640 1 -- 172.21.15.17:0/4155514444 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8084011380 con 0x7f80980ca4a0 2024-09-10T21:40:10.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.725+0000 7f809ca00640 1 -- 172.21.15.17:0/4155514444 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8098148070 con 0x7f80980ca4a0 2024-09-10T21:40:10.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.725+0000 7f809ca00640 1 --2- 172.21.15.17:0/4155514444 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f80981f7ad0 0x7f809806f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:40:10.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.725+0000 7f809ca00640 1 -- 172.21.15.17:0/4155514444 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f8098069ef0 con 0x7f80981f7ad0 2024-09-10T21:40:10.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.725+0000 7f8096800640 1 --2- 172.21.15.17:0/4155514444 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f80981f7ad0 0x7f809806f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:40:10.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.725+0000 7f8096800640 1 -- 172.21.15.17:0/4155514444 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f80981f7fc0 con 0x7f80981f7ad0 2024-09-10T21:40:10.732 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.725+0000 7f8096800640 1 --2- 172.21.15.17:0/4155514444 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f80981f7ad0 0x7f809806f3b0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f8098073050 tx=0x7f8088005710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:40:10.732 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.725+0000 7f808ea00640 1 -- 172.21.15.17:0/4155514444 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80880bf070 con 0x7f80981f7ad0 2024-09-10T21:40:10.732 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.725+0000 7f808ea00640 1 -- 172.21.15.17:0/4155514444 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f8084011520 con 0x7f80980ca4a0 2024-09-10T21:40:10.732 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.729+0000 7f808ea00640 1 --2- 172.21.15.17:0/4155514444 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f80780724e0 0x7f80780749a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:40:10.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.729+0000 7f808ea00640 1 -- 172.21.15.17:0/4155514444 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f8084093370 con 0x7f80980ca4a0 2024-09-10T21:40:10.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.729+0000 7f808ea00640 1 -- 172.21.15.17:0/4155514444 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f8088004250 con 0x7f80981f7ad0 2024-09-10T21:40:10.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.729+0000 7f808ea00640 1 -- 172.21.15.17:0/4155514444 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f80981f7ad0 msgr2=0x7f809806f3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:40:10.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.729+0000 7f808ea00640 1 --2- 172.21.15.17:0/4155514444 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f80981f7ad0 0x7f809806f3b0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f8098073050 tx=0x7f8088005710 comp rx=0 tx=0).stop 2024-09-10T21:40:10.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.729+0000 7f808ea00640 1 -- 172.21.15.17:0/4155514444 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f80981f7ad0 msgr2=0x7f809806f3b0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:40:10.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.729+0000 7f808ea00640 1 --2- 172.21.15.17:0/4155514444 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f80981f7ad0 0x7f809806f3b0 secure :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f8098073050 tx=0x7f8088005710 comp rx=0 tx=0).stop 2024-09-10T21:40:10.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.729+0000 7f8097200640 1 --2- 172.21.15.17:0/4155514444 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f80780724e0 0x7f80780749a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:40:10.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.729+0000 7f8097200640 1 --2- 172.21.15.17:0/4155514444 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f80780724e0 0x7f80780749a0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f809806dcb0 tx=0x7f8090009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:40:10.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.737+0000 7f809ca00640 1 --2- 172.21.15.17:0/4155514444 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8098148240 0x7f8098142770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:40:10.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.737+0000 7f809ca00640 1 -- 172.21.15.17:0/4155514444 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f809806ebf0 con 0x7f8098148240 2024-09-10T21:40:10.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.737+0000 7f8096800640 1 --2- 172.21.15.17:0/4155514444 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8098148240 0x7f8098142770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:40:10.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.737+0000 7f8096800640 1 -- 172.21.15.17:0/4155514444 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f8098155b40 con 0x7f8098148240 2024-09-10T21:40:10.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.737+0000 7f8096800640 1 --2- 172.21.15.17:0/4155514444 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8098148240 0x7f8098142770 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f8088000c10 tx=0x7f80880091e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:40:10.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.737+0000 7f808ea00640 1 -- 172.21.15.17:0/4155514444 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8088004ae0 con 0x7f8098148240 2024-09-10T21:40:10.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.737+0000 7f808ea00640 1 -- 172.21.15.17:0/4155514444 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4711 (secure 0 0 0) 0x7f8088009460 con 0x7f8098148240 2024-09-10T21:40:10.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.737+0000 7f808ea00640 1 -- 172.21.15.17:0/4155514444 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8098148240 msgr2=0x7f8098142770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:40:10.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.737+0000 7f808ea00640 1 --2- 172.21.15.17:0/4155514444 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8098148240 0x7f8098142770 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f8088000c10 tx=0x7f80880091e0 comp rx=0 tx=0).stop 2024-09-10T21:40:10.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.737+0000 7f808ea00640 1 -- 172.21.15.17:0/4155514444 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8098148240 msgr2=0x7f8098142770 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:40:10.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.737+0000 7f808ea00640 1 --2- 172.21.15.17:0/4155514444 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8098148240 0x7f8098142770 secure :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f8088000c10 tx=0x7f80880091e0 comp rx=0 tx=0).stop 2024-09-10T21:40:10.742 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-10T21:40:10.742 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:40:10.742 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:40:10.742 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "leader", 2024-09-10T21:40:10.742 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 122, 2024-09-10T21:40:10.742 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 84939, 2024-09-10T21:40:10.742 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-10T21:40:10.742 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-10T21:40:10.743 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-10T21:40:10.743 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-10T21:40:10.743 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:40:10.743 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 16, 2024-09-10T21:40:10.743 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:40:10.743 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-10T21:40:10.743 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-10T21:40:10.743 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:40:10.743 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:40:10.743 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:40:10.743 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:40:10.743 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:40:10.743 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:40:10.744 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:40:10.744 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:40:10.744 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:40:10.744 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:40:10.744 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:40:10.744 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:40:10.744 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-10T21:40:10.744 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-10T21:40:10.744 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:40:10.744 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:40:10.744 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:40:10.744 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:40:10.745 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:40:10.745 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:40:10.745 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:40:10.745 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:40:10.745 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:40:10.745 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:40:10.745 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:40:10.745 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:40:10.745 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:40:10.745 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-10T21:40:10.745 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-10T21:40:10.745 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-10T21:40:10.745 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-10T21:40:10.745 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-10T21:40:10.746 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "5e2f023f-d6a6-49f4-a256-f4807dce3739", 2024-09-10T21:40:10.746 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:40:10.746 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:40:10.746 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-10T21:40:10.746 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-10T21:40:10.746 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-10T21:40:10.746 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-10T21:40:10.746 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-10T21:40:10.746 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-10T21:40:10.746 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-10T21:40:10.746 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:40:10.746 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-10T21:40:10.746 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:40:10.746 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:40:10.747 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:40:10.747 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:40:10.747 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:40:10.747 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:40:10.747 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:40:10.747 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:40:10.747 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:40:10.747 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:40:10.747 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:40:10.747 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:40:10.747 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-10T21:40:10.747 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:40:10.747 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-10T21:40:10.748 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:40:10.748 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:40:10.748 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:40:10.748 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:40:10.748 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:40:10.748 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:40:10.748 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:40:10.748 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-10T21:40:10.748 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:40:10.748 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:40:10.748 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:40:10.748 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:40:10.749 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-10T21:40:10.749 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:40:10.749 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:40:10.749 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:40:10.749 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:40:10.749 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-10T21:40:10.749 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-10T21:40:10.749 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:40:10.749 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:40:10.749 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:40:10.749 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:40:10.749 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:40:10.750 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:40:10.750 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:40:10.750 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:40:10.750 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:40:10.750 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:40:10.750 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:40:10.750 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3300", 2024-09-10T21:40:10.750 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:40:10.750 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:40:10.750 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:40:10.750 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:40:10.750 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789", 2024-09-10T21:40:10.750 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:40:10.750 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:40:10.751 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:40:10.751 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:40:10.751 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789/0", 2024-09-10T21:40:10.751 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6789/0", 2024-09-10T21:40:10.751 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:40:10.751 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:40:10.751 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:40:10.751 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:40:10.751 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:40:10.751 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:40:10.751 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:40:10.751 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:40:10.751 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:40:10.751 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:40:10.752 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:40:10.752 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3301", 2024-09-10T21:40:10.752 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:40:10.752 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:40:10.752 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:40:10.752 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:40:10.752 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790", 2024-09-10T21:40:10.752 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:40:10.752 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:40:10.752 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:40:10.752 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:40:10.752 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790/0", 2024-09-10T21:40:10.752 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6790/0", 2024-09-10T21:40:10.753 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:40:10.753 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:40:10.753 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:40:10.753 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:40:10.753 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:40:10.753 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:40:10.753 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-10T21:40:10.753 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-10T21:40:10.753 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:40:10.753 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:40:10.753 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:40:10.753 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:40:10.753 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:40:10.754 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:40:10.754 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-10T21:40:10.754 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:40:10.754 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:40:10.754 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:40:10.754 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 2 2024-09-10T21:40:10.754 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:40:10.754 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:40:10.754 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-10T21:40:10.754 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:40:10.754 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:40:10.754 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:40:10.754 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 4 2024-09-10T21:40:10.755 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:40:10.755 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:40:10.755 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:40:10.755 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-10T21:40:10.755 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-10T21:40:10.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.737+0000 7f809ca00640 1 -- 172.21.15.17:0/4155514444 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f80780724e0 msgr2=0x7f80780749a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:40:10.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.737+0000 7f809ca00640 1 --2- 172.21.15.17:0/4155514444 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f80780724e0 0x7f80780749a0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f809806dcb0 tx=0x7f8090009040 comp rx=0 tx=0).stop 2024-09-10T21:40:10.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.737+0000 7f809ca00640 1 -- 172.21.15.17:0/4155514444 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f80980ca4a0 msgr2=0x7f80981571e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:40:10.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.737+0000 7f809ca00640 1 --2- 172.21.15.17:0/4155514444 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f80980ca4a0 0x7f80981571e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f808400c460 tx=0x7f808400c920 comp rx=0 tx=0).stop 2024-09-10T21:40:10.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.737+0000 7f8097200640 1 -- 172.21.15.17:0/4155514444 reap_dead start 2024-09-10T21:40:10.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.737+0000 7f809ca00640 1 -- 172.21.15.17:0/4155514444 shutdown_connections 2024-09-10T21:40:10.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.737+0000 7f809ca00640 1 --2- 172.21.15.17:0/4155514444 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f80980ca4a0 0x7f80981571e0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:40:10.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.737+0000 7f809ca00640 1 -- 172.21.15.17:0/4155514444 >> 172.21.15.17:0/4155514444 conn(0x7f8098068990 msgr2=0x7f809814ea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:40:10.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.737+0000 7f809ca00640 1 -- 172.21.15.17:0/4155514444 shutdown_connections 2024-09-10T21:40:10.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.737+0000 7f809ca00640 1 -- 172.21.15.17:0/4155514444 wait complete. 2024-09-10T21:40:10.764 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-10T21:40:10.764 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-09-10T21:40:10.765 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-10T21:40:10.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.945+0000 7ff10d000640 1 -- 172.21.15.17:0/3264096183 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff1080c2f30 msgr2=0x7ff1080c81f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:40:10.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.945+0000 7ff10d000640 1 --2- 172.21.15.17:0/3264096183 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff1080c2f30 0x7ff1080c81f0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7ff100009c80 tx=0x7ff10001f5a0 comp rx=0 tx=0).stop 2024-09-10T21:40:10.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.945+0000 7ff10d000640 1 -- 172.21.15.17:0/3264096183 shutdown_connections 2024-09-10T21:40:10.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.945+0000 7ff10d000640 1 --2- 172.21.15.17:0/3264096183 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff1080c2f30 0x7ff1080c81f0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:40:10.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.945+0000 7ff10d000640 1 --2- 172.21.15.17:0/3264096183 >> v2:172.21.15.31:3300/0 conn(0x7ff108151450 0x7ff1080ca3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:40:10.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.945+0000 7ff10d000640 1 -- 172.21.15.17:0/3264096183 >> 172.21.15.17:0/3264096183 conn(0x7ff1080cd690 msgr2=0x7ff1080c98e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:40:10.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.945+0000 7ff10d000640 1 -- 172.21.15.17:0/3264096183 shutdown_connections 2024-09-10T21:40:10.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.945+0000 7ff10d000640 1 -- 172.21.15.17:0/3264096183 wait complete. 2024-09-10T21:40:10.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.945+0000 7ff10d000640 1 Processor -- start 2024-09-10T21:40:10.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.945+0000 7ff10d000640 1 -- start start 2024-09-10T21:40:10.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.949+0000 7ff10d000640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff1080c2f30 0x7ff1081f12e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:40:10.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.949+0000 7ff10d000640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff1080cabe0 0x7ff1081e2f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:40:10.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.949+0000 7ff10d000640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff108151450 0x7ff10814a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:40:10.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.949+0000 7ff10d000640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7ff1080c2710 con 0x7ff1080cabe0 2024-09-10T21:40:10.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.949+0000 7ff10d000640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7ff1080c2a10 con 0x7ff108151450 2024-09-10T21:40:10.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.949+0000 7ff10d000640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7ff108151830 con 0x7ff1080c2f30 2024-09-10T21:40:10.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.949+0000 7ff106a00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff1080cabe0 0x7ff1081e2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:40:10.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.949+0000 7ff106a00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff1080cabe0 0x7ff1081e2f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33738/0 (socket says 172.21.15.17:33738) 2024-09-10T21:40:10.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.949+0000 7ff106a00640 1 -- 172.21.15.17:0/3789185884 learned_addr learned my addr 172.21.15.17:0/3789185884 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:40:10.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.949+0000 7ff107400640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff1080c2f30 0x7ff1081f12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:40:10.954 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.949+0000 7ff107e00640 1 --2- 172.21.15.17:0/3789185884 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff108151450 0x7ff10814a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:40:10.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.949+0000 7ff106a00640 1 -- 172.21.15.17:0/3789185884 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff1080c2f30 msgr2=0x7ff1081f12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:40:10.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.949+0000 7ff106a00640 1 --2- 172.21.15.17:0/3789185884 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff1080c2f30 0x7ff1081f12e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:40:10.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.949+0000 7ff106a00640 1 -- 172.21.15.17:0/3789185884 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff108151450 msgr2=0x7ff10814a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:40:10.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.949+0000 7ff106a00640 1 --2- 172.21.15.17:0/3789185884 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff108151450 0x7ff10814a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:40:10.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.949+0000 7ff106a00640 1 -- 172.21.15.17:0/3789185884 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1081e3830 con 0x7ff1080cabe0 2024-09-10T21:40:10.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.949+0000 7ff107400640 1 --2- 172.21.15.17:0/3789185884 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff1080c2f30 0x7ff1081f12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:40:10.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.949+0000 7ff107e00640 1 --2- 172.21.15.17:0/3789185884 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff108151450 0x7ff10814a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:40:10.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.949+0000 7ff106a00640 1 --2- 172.21.15.17:0/3789185884 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff1080cabe0 0x7ff1081e2f20 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ff0f400c4b0 tx=0x7ff0f400c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:40:10.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.949+0000 7ff0ff400640 1 -- 172.21.15.17:0/3789185884 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0f40052d0 con 0x7ff1080cabe0 2024-09-10T21:40:10.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.949+0000 7ff0ff400640 1 -- 172.21.15.17:0/3789185884 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff0f4005470 con 0x7ff1080cabe0 2024-09-10T21:40:10.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.949+0000 7ff0ff400640 1 -- 172.21.15.17:0/3789185884 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0f4012870 con 0x7ff1080cabe0 2024-09-10T21:40:10.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.949+0000 7ff10d000640 1 -- 172.21.15.17:0/3789185884 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff10814ed20 con 0x7ff1080cabe0 2024-09-10T21:40:10.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.949+0000 7ff10d000640 1 -- 172.21.15.17:0/3789185884 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1081f0ae0 con 0x7ff1080cabe0 2024-09-10T21:40:10.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.949+0000 7ff10d000640 1 --2- 172.21.15.17:0/3789185884 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff1081e1830 0x7ff10814aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:40:10.956 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.949+0000 7ff10d000640 1 -- 172.21.15.17:0/3789185884 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7ff108069ef0 con 0x7ff1081e1830 2024-09-10T21:40:10.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.949+0000 7ff107400640 1 --2- 172.21.15.17:0/3789185884 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff1081e1830 0x7ff10814aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:40:10.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.953+0000 7ff107400640 1 -- 172.21.15.17:0/3789185884 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff1080c79a0 con 0x7ff1081e1830 2024-09-10T21:40:10.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.953+0000 7ff107400640 1 --2- 172.21.15.17:0/3789185884 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff1081e1830 0x7ff10814aef0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7ff0f8011c90 tx=0x7ff0f800f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:40:10.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.953+0000 7ff0ff400640 1 -- 172.21.15.17:0/3789185884 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7ff0f4003600 con 0x7ff1080cabe0 2024-09-10T21:40:10.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.953+0000 7ff0ff400640 1 --2- 172.21.15.17:0/3789185884 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7ff0e8072610 0x7ff0e8074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:40:10.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.953+0000 7ff0ff400640 1 -- 172.21.15.17:0/3789185884 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7ff0f4093050 con 0x7ff1080cabe0 2024-09-10T21:40:10.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.953+0000 7ff0ff400640 1 -- 172.21.15.17:0/3789185884 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0f801e070 con 0x7ff1081e1830 2024-09-10T21:40:10.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.953+0000 7ff0ff400640 1 -- 172.21.15.17:0/3789185884 <== mon.1 v2:172.21.15.31:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7ff0f800e880 con 0x7ff1081e1830 2024-09-10T21:40:10.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.953+0000 7ff0ff400640 1 -- 172.21.15.17:0/3789185884 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff1081e1830 msgr2=0x7ff10814aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:40:10.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.953+0000 7ff0ff400640 1 --2- 172.21.15.17:0/3789185884 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff1081e1830 0x7ff10814aef0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7ff0f8011c90 tx=0x7ff0f800f790 comp rx=0 tx=0).stop 2024-09-10T21:40:10.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.953+0000 7ff0ff400640 1 -- 172.21.15.17:0/3789185884 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff1081e1830 msgr2=0x7ff10814aef0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:40:10.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.953+0000 7ff0ff400640 1 --2- 172.21.15.17:0/3789185884 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff1081e1830 0x7ff10814aef0 secure :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7ff0f8011c90 tx=0x7ff0f800f790 comp rx=0 tx=0).stop 2024-09-10T21:40:10.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.953+0000 7ff107e00640 1 --2- 172.21.15.17:0/3789185884 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7ff0e8072610 0x7ff0e8074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:40:10.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.953+0000 7ff107e00640 1 --2- 172.21.15.17:0/3789185884 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7ff0e8072610 0x7ff0e8074ad0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7ff1000b3000 tx=0x7ff10001dde0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:40:10.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.961+0000 7ff10d000640 1 --2- 172.21.15.17:0/3789185884 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff10814f400 0x7ff1081f0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:40:10.964 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.961+0000 7ff10d000640 1 -- 172.21.15.17:0/3789185884 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7ff1081f0940 con 0x7ff10814f400 2024-09-10T21:40:10.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.961+0000 7ff107400640 1 --2- 172.21.15.17:0/3789185884 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff10814f400 0x7ff1081f0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:40:10.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.961+0000 7ff107400640 1 -- 172.21.15.17:0/3789185884 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7ff1080cbbd0 con 0x7ff10814f400 2024-09-10T21:40:10.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.961+0000 7ff107400640 1 --2- 172.21.15.17:0/3789185884 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff10814f400 0x7ff1081f0ce0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7ff0f8011c90 tx=0x7ff0f80046d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:40:10.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.961+0000 7ff0ff400640 1 -- 172.21.15.17:0/3789185884 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0f801e040 con 0x7ff10814f400 2024-09-10T21:40:10.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.961+0000 7ff0ff400640 1 -- 172.21.15.17:0/3789185884 <== mon.1 v2:172.21.15.31:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4542 (secure 0 0 0) 0x7ff0f802b420 con 0x7ff10814f400 2024-09-10T21:40:10.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.961+0000 7ff0ff400640 1 -- 172.21.15.17:0/3789185884 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff10814f400 msgr2=0x7ff1081f0ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:40:10.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.961+0000 7ff0ff400640 1 --2- 172.21.15.17:0/3789185884 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff10814f400 0x7ff1081f0ce0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7ff0f8011c90 tx=0x7ff0f80046d0 comp rx=0 tx=0).stop 2024-09-10T21:40:10.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.961+0000 7ff0ff400640 1 -- 172.21.15.17:0/3789185884 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff10814f400 msgr2=0x7ff1081f0ce0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:40:10.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.961+0000 7ff0ff400640 1 --2- 172.21.15.17:0/3789185884 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff10814f400 0x7ff1081f0ce0 secure :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7ff0f8011c90 tx=0x7ff0f80046d0 comp rx=0 tx=0).stop 2024-09-10T21:40:10.967 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-10T21:40:10.967 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:40:10.967 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:40:10.967 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-10T21:40:10.967 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 122, 2024-09-10T21:40:10.967 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 22391, 2024-09-10T21:40:10.968 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-10T21:40:10.968 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-10T21:40:10.968 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-10T21:40:10.968 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-10T21:40:10.968 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:40:10.968 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 17, 2024-09-10T21:40:10.968 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:40:10.968 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-10T21:40:10.968 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-10T21:40:10.968 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:40:10.968 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:40:10.968 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:40:10.968 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:40:10.969 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:40:10.969 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:40:10.969 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:40:10.969 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:40:10.969 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:40:10.969 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:40:10.969 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:40:10.969 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:40:10.969 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-10T21:40:10.969 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-10T21:40:10.970 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:40:10.970 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:40:10.970 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:40:10.970 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:40:10.970 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:40:10.970 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:40:10.970 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:40:10.970 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:40:10.970 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:40:10.970 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:40:10.970 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:40:10.970 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:40:10.970 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:40:10.971 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-10T21:40:10.971 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-10T21:40:10.971 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-10T21:40:10.971 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-10T21:40:10.971 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-10T21:40:10.971 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "5e2f023f-d6a6-49f4-a256-f4807dce3739", 2024-09-10T21:40:10.971 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:40:10.971 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:40:10.971 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-10T21:40:10.971 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-10T21:40:10.971 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-10T21:40:10.971 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-10T21:40:10.971 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-10T21:40:10.971 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-10T21:40:10.972 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-10T21:40:10.972 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:40:10.972 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-10T21:40:10.972 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:40:10.972 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:40:10.972 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:40:10.972 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:40:10.972 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:40:10.972 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:40:10.972 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:40:10.972 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:40:10.972 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:40:10.972 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:40:10.973 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:40:10.973 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:40:10.973 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-10T21:40:10.973 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:40:10.973 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-10T21:40:10.973 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:40:10.973 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:40:10.973 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:40:10.973 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:40:10.973 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:40:10.973 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:40:10.973 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:40:10.973 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-10T21:40:10.973 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:40:10.974 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:40:10.974 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:40:10.974 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:40:10.974 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-10T21:40:10.974 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:40:10.974 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:40:10.974 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:40:10.974 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:40:10.974 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-10T21:40:10.974 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-10T21:40:10.974 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:40:10.974 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:40:10.974 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:40:10.975 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:40:10.975 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:40:10.975 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:40:10.975 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:40:10.975 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:40:10.975 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:40:10.975 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:40:10.975 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:40:10.975 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3300", 2024-09-10T21:40:10.975 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:40:10.975 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:40:10.975 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:40:10.975 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:40:10.975 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789", 2024-09-10T21:40:10.976 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:40:10.976 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:40:10.976 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:40:10.976 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:40:10.976 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789/0", 2024-09-10T21:40:10.976 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6789/0", 2024-09-10T21:40:10.976 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:40:10.976 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:40:10.976 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:40:10.976 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:40:10.976 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:40:10.976 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:40:10.976 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:40:10.977 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:40:10.977 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:40:10.977 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:40:10.977 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:40:10.977 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3301", 2024-09-10T21:40:10.977 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:40:10.977 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:40:10.977 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:40:10.977 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:40:10.977 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790", 2024-09-10T21:40:10.977 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:40:10.977 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:40:10.977 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:40:10.978 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:40:10.978 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790/0", 2024-09-10T21:40:10.978 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6790/0", 2024-09-10T21:40:10.978 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:40:10.978 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:40:10.978 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:40:10.978 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:40:10.978 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:40:10.978 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:40:10.978 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-10T21:40:10.978 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-10T21:40:10.978 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:40:10.978 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:40:10.979 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:40:10.979 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:40:10.979 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:40:10.979 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:40:10.979 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-10T21:40:10.979 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:40:10.979 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:40:10.979 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:40:10.979 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:40:10.979 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:40:10.979 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:40:10.979 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:40:10.979 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-10T21:40:10.979 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-10T21:40:10.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.961+0000 7ff10d000640 1 -- 172.21.15.17:0/3789185884 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7ff0e8072610 msgr2=0x7ff0e8074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:40:10.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.961+0000 7ff10d000640 1 --2- 172.21.15.17:0/3789185884 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7ff0e8072610 0x7ff0e8074ad0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7ff1000b3000 tx=0x7ff10001dde0 comp rx=0 tx=0).stop 2024-09-10T21:40:10.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.961+0000 7ff10d000640 1 -- 172.21.15.17:0/3789185884 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff1080cabe0 msgr2=0x7ff1081e2f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:40:10.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.961+0000 7ff10d000640 1 --2- 172.21.15.17:0/3789185884 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff1080cabe0 0x7ff1081e2f20 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ff0f400c4b0 tx=0x7ff0f400c970 comp rx=0 tx=0).stop 2024-09-10T21:40:10.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.961+0000 7ff107e00640 1 -- 172.21.15.17:0/3789185884 reap_dead start 2024-09-10T21:40:10.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.961+0000 7ff10d000640 1 -- 172.21.15.17:0/3789185884 shutdown_connections 2024-09-10T21:40:10.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.961+0000 7ff10d000640 1 --2- 172.21.15.17:0/3789185884 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff1080cabe0 0x7ff1081e2f20 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:40:10.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.961+0000 7ff10d000640 1 -- 172.21.15.17:0/3789185884 >> 172.21.15.17:0/3789185884 conn(0x7ff1080cd690 msgr2=0x7ff10806d1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:40:10.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.961+0000 7ff10d000640 1 -- 172.21.15.17:0/3789185884 shutdown_connections 2024-09-10T21:40:10.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:10.961+0000 7ff10d000640 1 -- 172.21.15.17:0/3789185884 wait complete. 2024-09-10T21:40:10.989 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-10T21:40:10.989 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-09-10T21:40:10.989 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-10T21:40:11.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:11.145+0000 7f1cc3c00640 1 -- 172.21.15.17:0/4285436160 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1ca4004b70 msgr2=0x7f1ca4004f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:40:11.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:11.145+0000 7f1cc3c00640 1 --2- 172.21.15.17:0/4285436160 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1ca4004b70 0x7f1ca4004f90 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f1cb0009b60 tx=0x7f1cb001d270 comp rx=0 tx=0).stop 2024-09-10T21:40:11.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:11.145+0000 7f1cc3c00640 1 -- 172.21.15.17:0/4285436160 shutdown_connections 2024-09-10T21:40:11.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:11.145+0000 7f1cc3c00640 1 --2- 172.21.15.17:0/4285436160 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1ca4004b70 0x7f1ca4004f90 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:40:11.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:11.145+0000 7f1cc3c00640 1 -- 172.21.15.17:0/4285436160 >> 172.21.15.17:0/4285436160 conn(0x7f1cbc06f0d0 msgr2=0x7f1cbc0cc2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:40:11.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:11.145+0000 7f1cc3c00640 1 -- 172.21.15.17:0/4285436160 shutdown_connections 2024-09-10T21:40:11.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:11.145+0000 7f1cc3c00640 1 -- 172.21.15.17:0/4285436160 wait complete. 2024-09-10T21:40:11.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:11.145+0000 7f1cc3c00640 1 Processor -- start 2024-09-10T21:40:11.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:11.145+0000 7f1cc3c00640 1 -- start start 2024-09-10T21:40:11.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:11.145+0000 7f1cc3c00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1ca4004b70 0x7f1cbc0cc2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:40:11.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:11.145+0000 7f1cc3c00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1cbc0cb390 0x7f1cbc1f3540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:40:11.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:11.145+0000 7f1cc3c00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f1cbc14bd50 0x7f1cbc149bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:40:11.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:11.145+0000 7f1cc3c00640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f1cbc0c70e0 con 0x7f1cbc0cb390 2024-09-10T21:40:11.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:11.145+0000 7f1cc3c00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f1cbc15a8d0 con 0x7f1ca4004b70 2024-09-10T21:40:11.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:11.145+0000 7f1cc3c00640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f1cbc0c7260 con 0x7f1cbc14bd50 2024-09-10T21:40:11.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:11.145+0000 7f1cbbe00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1cbc0cb390 0x7f1cbc1f3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:40:11.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:11.149+0000 7f1cbbe00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1cbc0cb390 0x7f1cbc1f3540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:33750/0 (socket says 172.21.15.17:33750) 2024-09-10T21:40:11.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:11.149+0000 7f1cbbe00640 1 -- 172.21.15.17:0/335501374 learned_addr learned my addr 172.21.15.17:0/335501374 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:40:11.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:11.149+0000 7f1cc1200640 1 --2- 172.21.15.17:0/335501374 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1ca4004b70 0x7f1cbc0cc2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:40:11.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:11.149+0000 7f1cc1c00640 1 --2- 172.21.15.17:0/335501374 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f1cbc14bd50 0x7f1cbc149bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:40:11.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:11.149+0000 7f1cbbe00640 1 -- 172.21.15.17:0/335501374 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f1cbc14bd50 msgr2=0x7f1cbc149bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:40:11.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:11.149+0000 7f1cbbe00640 1 --2- 172.21.15.17:0/335501374 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f1cbc14bd50 0x7f1cbc149bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:40:11.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:11.149+0000 7f1cbbe00640 1 -- 172.21.15.17:0/335501374 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1ca4004b70 msgr2=0x7f1cbc0cc2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:40:11.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:11.149+0000 7f1cbbe00640 1 --2- 172.21.15.17:0/335501374 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1ca4004b70 0x7f1cbc0cc2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:40:11.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:11.149+0000 7f1cbbe00640 1 -- 172.21.15.17:0/335501374 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cbc156b00 con 0x7f1cbc0cb390 2024-09-10T21:40:11.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:11.149+0000 7f1cbbe00640 1 --2- 172.21.15.17:0/335501374 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1cbc0cb390 0x7f1cbc1f3540 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f1ca8006660 tx=0x7f1ca8003a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:40:11.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:11.149+0000 7f1cb8c00640 1 -- 172.21.15.17:0/335501374 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ca8015020 con 0x7f1cbc0cb390 2024-09-10T21:40:11.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:11.149+0000 7f1cb8c00640 1 -- 172.21.15.17:0/335501374 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1ca80119c0 con 0x7f1cbc0cb390 2024-09-10T21:40:11.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:11.149+0000 7f1cb8c00640 1 -- 172.21.15.17:0/335501374 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ca8011df0 con 0x7f1cbc0cb390 2024-09-10T21:40:11.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:11.149+0000 7f1cc3c00640 1 -- 172.21.15.17:0/335501374 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cbc156dc0 con 0x7f1cbc0cb390 2024-09-10T21:40:11.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:11.149+0000 7f1cc3c00640 1 -- 172.21.15.17:0/335501374 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cbc14fb30 con 0x7f1cbc0cb390 2024-09-10T21:40:11.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:11.149+0000 7f1cc3c00640 1 --2- 172.21.15.17:0/335501374 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f1cbc14c130 0x7f1cbc14a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:40:11.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:11.149+0000 7f1cc3c00640 1 -- 172.21.15.17:0/335501374 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f1cbc1506c0 con 0x7f1cbc14c130 2024-09-10T21:40:11.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:11.149+0000 7f1cc1200640 1 --2- 172.21.15.17:0/335501374 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f1cbc14c130 0x7f1cbc14a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:40:11.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:11.149+0000 7f1cc1200640 1 -- 172.21.15.17:0/335501374 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1cbc0ca0e0 con 0x7f1cbc14c130 2024-09-10T21:40:11.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:11.149+0000 7f1cc1200640 1 --2- 172.21.15.17:0/335501374 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f1cbc14c130 0x7f1cbc14a130 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f1cb0009b30 tx=0x7f1cb0005680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:40:11.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:11.149+0000 7f1cb8c00640 1 -- 172.21.15.17:0/335501374 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f1ca8010420 con 0x7f1cbc0cb390 2024-09-10T21:40:11.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:11.153+0000 7f1cb8c00640 1 --2- 172.21.15.17:0/335501374 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f1c9c0725e0 0x7f1c9c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:40:11.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:11.153+0000 7f1cb8c00640 1 -- 172.21.15.17:0/335501374 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f1ca8094070 con 0x7f1cbc0cb390 2024-09-10T21:40:11.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:11.153+0000 7f1cb8c00640 1 -- 172.21.15.17:0/335501374 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb00bb070 con 0x7f1cbc14c130 2024-09-10T21:40:11.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:11.153+0000 7f1cb8c00640 1 -- 172.21.15.17:0/335501374 <== mon.2 v2:172.21.15.31:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f1cb00046d0 con 0x7f1cbc14c130 2024-09-10T21:40:11.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:11.153+0000 7f1cb8c00640 1 -- 172.21.15.17:0/335501374 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f1cbc14c130 msgr2=0x7f1cbc14a130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:40:11.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:11.153+0000 7f1cb8c00640 1 --2- 172.21.15.17:0/335501374 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f1cbc14c130 0x7f1cbc14a130 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f1cb0009b30 tx=0x7f1cb0005680 comp rx=0 tx=0).stop 2024-09-10T21:40:11.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:11.153+0000 7f1cc1c00640 1 --2- 172.21.15.17:0/335501374 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f1c9c0725e0 0x7f1c9c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:40:11.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:11.153+0000 7f1cb8c00640 1 -- 172.21.15.17:0/335501374 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f1cbc14c130 msgr2=0x7f1cbc14a130 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:40:11.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:11.153+0000 7f1cb8c00640 1 --2- 172.21.15.17:0/335501374 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f1cbc14c130 0x7f1cbc14a130 secure :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f1cb0009b30 tx=0x7f1cb0005680 comp rx=0 tx=0).stop 2024-09-10T21:40:11.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:11.153+0000 7f1cc1c00640 1 --2- 172.21.15.17:0/335501374 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f1c9c0725e0 0x7f1c9c074aa0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc0c97d0 tx=0x7f1cb4006f00 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:40:11.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:11.161+0000 7f1cc3c00640 1 --2- 172.21.15.17:0/335501374 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f1cbc0cbd30 0x7f1cbc1effd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:40:11.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:11.161+0000 7f1cc3c00640 1 -- 172.21.15.17:0/335501374 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f1cbc1e6800 con 0x7f1cbc0cbd30 2024-09-10T21:40:11.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:11.161+0000 7f1cc1200640 1 --2- 172.21.15.17:0/335501374 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f1cbc0cbd30 0x7f1cbc1effd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:40:11.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:11.161+0000 7f1cc1200640 1 -- 172.21.15.17:0/335501374 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f1cbc0c6700 con 0x7f1cbc0cbd30 2024-09-10T21:40:11.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:11.165+0000 7f1cc1200640 1 --2- 172.21.15.17:0/335501374 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f1cbc0cbd30 0x7f1cbc1effd0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f1cb001baf0 tx=0x7f1cb00091e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:40:11.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:11.165+0000 7f1cb8c00640 1 -- 172.21.15.17:0/335501374 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb00b1040 con 0x7f1cbc0cbd30 2024-09-10T21:40:11.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:11.165+0000 7f1cb8c00640 1 -- 172.21.15.17:0/335501374 <== mon.2 v2:172.21.15.31:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5044 (secure 0 0 0) 0x7f1cb00ca3f0 con 0x7f1cbc0cbd30 2024-09-10T21:40:11.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:11.165+0000 7f1cb8c00640 1 -- 172.21.15.17:0/335501374 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f1cbc0cbd30 msgr2=0x7f1cbc1effd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:40:11.168 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:11.165+0000 7f1cb8c00640 1 --2- 172.21.15.17:0/335501374 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f1cbc0cbd30 0x7f1cbc1effd0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f1cb001baf0 tx=0x7f1cb00091e0 comp rx=0 tx=0).stop 2024-09-10T21:40:11.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:11.165+0000 7f1cb8c00640 1 -- 172.21.15.17:0/335501374 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f1cbc0cbd30 msgr2=0x7f1cbc1effd0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:40:11.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:11.165+0000 7f1cb8c00640 1 --2- 172.21.15.17:0/335501374 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f1cbc0cbd30 0x7f1cbc1effd0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:40:11.169 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-10T21:40:11.169 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:40:11.169 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:40:11.169 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-10T21:40:11.170 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 122, 2024-09-10T21:40:11.170 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 217516, 2024-09-10T21:40:11.170 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-10T21:40:11.170 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-10T21:40:11.170 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-10T21:40:11.170 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-10T21:40:11.170 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:40:11.170 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 17, 2024-09-10T21:40:11.170 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:40:11.170 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-10T21:40:11.170 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-10T21:40:11.170 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:40:11.170 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:40:11.171 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:40:11.171 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:40:11.171 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:40:11.171 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:40:11.171 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:40:11.171 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:40:11.171 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:40:11.171 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:40:11.171 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:40:11.171 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:40:11.171 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-10T21:40:11.171 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-10T21:40:11.171 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:40:11.171 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:40:11.172 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:40:11.172 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:40:11.172 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:40:11.172 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:40:11.172 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:40:11.172 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:40:11.172 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:40:11.172 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:40:11.172 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:40:11.172 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:40:11.172 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:40:11.172 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-10T21:40:11.172 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-10T21:40:11.173 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-10T21:40:11.173 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-10T21:40:11.173 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-10T21:40:11.173 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "5e2f023f-d6a6-49f4-a256-f4807dce3739", 2024-09-10T21:40:11.173 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:40:11.173 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:40:11.173 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-10T21:40:11.173 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-10T21:40:11.173 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-10T21:40:11.173 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-10T21:40:11.173 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-10T21:40:11.173 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-10T21:40:11.173 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-10T21:40:11.174 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:40:11.174 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-10T21:40:11.174 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:40:11.174 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:40:11.174 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:40:11.174 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:40:11.174 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:40:11.174 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:40:11.174 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:40:11.174 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:40:11.174 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:40:11.174 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:40:11.174 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:40:11.175 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:40:11.175 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-10T21:40:11.175 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:40:11.175 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-10T21:40:11.175 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:40:11.175 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:40:11.175 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:40:11.175 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:40:11.175 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:40:11.175 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:40:11.175 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:40:11.175 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-10T21:40:11.175 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:40:11.175 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:40:11.176 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:40:11.176 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:40:11.176 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-10T21:40:11.176 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:40:11.176 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:40:11.176 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:40:11.176 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:40:11.176 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-10T21:40:11.176 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-10T21:40:11.176 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:40:11.176 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:40:11.176 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:40:11.176 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:40:11.177 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:40:11.177 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:40:11.177 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:40:11.177 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:40:11.177 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:40:11.177 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:40:11.177 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:40:11.177 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3300", 2024-09-10T21:40:11.177 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:40:11.177 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:40:11.177 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:40:11.177 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:40:11.177 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789", 2024-09-10T21:40:11.178 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:40:11.178 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:40:11.178 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:40:11.178 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:40:11.178 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789/0", 2024-09-10T21:40:11.178 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6789/0", 2024-09-10T21:40:11.178 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:40:11.178 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:40:11.178 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:40:11.178 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:40:11.178 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:40:11.178 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:40:11.178 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:40:11.178 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:40:11.179 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:40:11.179 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:40:11.179 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:40:11.179 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3301", 2024-09-10T21:40:11.179 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:40:11.179 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:40:11.179 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:40:11.179 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:40:11.179 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790", 2024-09-10T21:40:11.179 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:40:11.179 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:40:11.179 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:40:11.179 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:40:11.179 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790/0", 2024-09-10T21:40:11.180 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6790/0", 2024-09-10T21:40:11.180 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:40:11.180 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:40:11.180 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:40:11.180 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:40:11.180 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:40:11.180 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:40:11.180 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-10T21:40:11.180 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-10T21:40:11.180 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:40:11.180 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:40:11.180 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:40:11.180 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:40:11.181 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:40:11.181 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:40:11.181 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": [ 2024-09-10T21:40:11.181 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:40:11.181 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:40:11.181 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:40:11.181 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 6 2024-09-10T21:40:11.181 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:40:11.181 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:40:11.181 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-10T21:40:11.181 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:40:11.181 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:40:11.181 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:40:11.182 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 6 2024-09-10T21:40:11.182 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:40:11.182 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:40:11.182 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-10T21:40:11.182 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:40:11.182 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:40:11.182 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:40:11.182 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 4 2024-09-10T21:40:11.182 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:40:11.182 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:40:11.182 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": [ 2024-09-10T21:40:11.182 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:40:11.182 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:40:11.182 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:40:11.182 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 2 2024-09-10T21:40:11.183 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:40:11.183 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:40:11.183 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:40:11.183 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-10T21:40:11.183 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-10T21:40:11.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:11.165+0000 7f1cc3c00640 1 -- 172.21.15.17:0/335501374 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f1c9c0725e0 msgr2=0x7f1c9c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:40:11.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:11.165+0000 7f1cc3c00640 1 --2- 172.21.15.17:0/335501374 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f1c9c0725e0 0x7f1c9c074aa0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc0c97d0 tx=0x7f1cb4006f00 comp rx=0 tx=0).stop 2024-09-10T21:40:11.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:11.165+0000 7f1cc3c00640 1 -- 172.21.15.17:0/335501374 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1cbc0cb390 msgr2=0x7f1cbc1f3540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:40:11.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:11.165+0000 7f1cc3c00640 1 --2- 172.21.15.17:0/335501374 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1cbc0cb390 0x7f1cbc1f3540 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f1ca8006660 tx=0x7f1ca8003a90 comp rx=0 tx=0).stop 2024-09-10T21:40:11.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:11.165+0000 7f1cc1c00640 1 -- 172.21.15.17:0/335501374 reap_dead start 2024-09-10T21:40:11.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:11.165+0000 7f1cc1c00640 1 -- 172.21.15.17:0/335501374 reap_dead start 2024-09-10T21:40:11.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:11.165+0000 7f1cc3c00640 1 -- 172.21.15.17:0/335501374 shutdown_connections 2024-09-10T21:40:11.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:11.165+0000 7f1cc3c00640 1 -- 172.21.15.17:0/335501374 >> 172.21.15.17:0/335501374 conn(0x7f1cbc06f0d0 msgr2=0x7f1cbc1e40d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:40:11.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:11.165+0000 7f1cc3c00640 1 -- 172.21.15.17:0/335501374 shutdown_connections 2024-09-10T21:40:11.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:11.165+0000 7f1cc3c00640 1 -- 172.21.15.17:0/335501374 wait complete. 2024-09-10T21:40:11.193 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-10T21:40:11.193 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['a'] 2024-09-10T21:40:11.193 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: ['c'] 2024-09-10T21:40:11.193 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.a 2024-09-10T21:40:11.193 INFO:tasks.mon_thrash.mon_thrasher:killing mon.a 2024-09-10T21:40:11.193 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-10T21:40:11.194 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:40:11.212 INFO:tasks.ceph.mon.a:Stopped 2024-09-10T21:40:11.212 INFO:tasks.mon_thrash:Sending STOP to mon c 2024-09-10T21:40:11.213 INFO:tasks.ceph.mon.c:Sent signal 19 2024-09-10T21:40:11.213 INFO:tasks.mon_thrash.mon_thrasher:waiting for 15.0 secs to unfreeze mons 2024-09-10T21:40:22.744 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:40:22.747 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:40:26.214 INFO:tasks.mon_thrash:Sending CONT to mon c 2024-09-10T21:40:26.214 INFO:tasks.ceph.mon.c:Sent signal 18 2024-09-10T21:40:26.214 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2024-09-10T21:40:26.215 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-10T21:40:44.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.373+0000 7f9142600640 1 -- 172.21.15.17:0/1973996456 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f911c007990 msgr2=0x7f911c006cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:40:44.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.373+0000 7f9142600640 1 --2- 172.21.15.17:0/1973996456 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f911c007990 0x7f911c006cf0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f913c072990 tx=0x7f913401c7b0 comp rx=0 tx=0).stop 2024-09-10T21:40:44.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.373+0000 7f9142600640 1 -- 172.21.15.17:0/1973996456 shutdown_connections 2024-09-10T21:40:44.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.373+0000 7f9142600640 1 --2- 172.21.15.17:0/1973996456 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f911c007990 0x7f911c006cf0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:40:44.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.373+0000 7f9142600640 1 --2- 172.21.15.17:0/1973996456 >> v2:172.21.15.17:3300/0 conn(0x7f911c012b60 0x7f911c012ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:40:44.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.373+0000 7f9142600640 1 -- 172.21.15.17:0/1973996456 >> 172.21.15.17:0/1973996456 conn(0x7f913c070ca0 msgr2=0x7f913c0710b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:40:44.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.373+0000 7f9142600640 1 -- 172.21.15.17:0/1973996456 shutdown_connections 2024-09-10T21:40:44.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.373+0000 7f9142600640 1 -- 172.21.15.17:0/1973996456 wait complete. 2024-09-10T21:40:44.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.373+0000 7f9142600640 1 Processor -- start 2024-09-10T21:40:44.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.373+0000 7f9142600640 1 -- start start 2024-09-10T21:40:44.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.377+0000 7f9142600640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f911c007990 0x7f913c0af5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:40:44.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.377+0000 7f9142600640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f911c012b60 0x7f913c1e7340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:40:44.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.377+0000 7f9142600640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f911c013530 0x7f913c0b0230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:40:44.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.377+0000 7f9142600640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f913c15e850 con 0x7f911c012b60 2024-09-10T21:40:44.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.377+0000 7f9142600640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f913c15e570 con 0x7f911c013530 2024-09-10T21:40:44.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.377+0000 7f9142600640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f913c154d00 con 0x7f911c007990 2024-09-10T21:40:44.380 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.377+0000 7f913be00640 1 -- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f911c012b60 msgr2=0x7f913c1e7340 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-10T21:40:44.381 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.377+0000 7f913be00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f911c012b60 0x7f913c1e7340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:40:44.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.377+0000 7f9141200640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f911c007990 0x7f913c0af5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:40:44.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.377+0000 7f9141c00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f911c013530 0x7f913c0b0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:40:44.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.377+0000 7f9141200640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f911c007990 0x7f913c0af5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3301/0 says I am v2:172.21.15.17:51728/0 (socket says 172.21.15.17:51728) 2024-09-10T21:40:44.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.377+0000 7f9141200640 1 -- 172.21.15.17:0/2989572997 learned_addr learned my addr 172.21.15.17:0/2989572997 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:40:44.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.377+0000 7f9141200640 1 -- 172.21.15.17:0/2989572997 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f911c013530 msgr2=0x7f913c0b0230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:40:44.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.377+0000 7f9141200640 1 --2- 172.21.15.17:0/2989572997 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f911c013530 0x7f913c0b0230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:40:44.383 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.377+0000 7f9141200640 1 -- 172.21.15.17:0/2989572997 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f911c012b60 msgr2=0x7f913c1e7340 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:40:44.383 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.377+0000 7f9141200640 1 --2- 172.21.15.17:0/2989572997 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f911c012b60 0x7f913c1e7340 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:40:44.383 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.377+0000 7f9141200640 1 -- 172.21.15.17:0/2989572997 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f913c1f0a00 con 0x7f911c007990 2024-09-10T21:40:44.383 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.377+0000 7f9141c00640 1 --2- 172.21.15.17:0/2989572997 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f911c013530 0x7f913c0b0230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:40:44.383 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.377+0000 7f9141200640 1 --2- 172.21.15.17:0/2989572997 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f911c007990 0x7f913c0af5c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f9130012ee0 tx=0x7f91300217f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:40:44.383 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.377+0000 7f9138c00640 1 -- 172.21.15.17:0/2989572997 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f913002b680 con 0x7f911c007990 2024-09-10T21:40:44.383 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.377+0000 7f9138c00640 1 -- 172.21.15.17:0/2989572997 <== mon.2 v2:172.21.15.31:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9130030070 con 0x7f911c007990 2024-09-10T21:40:44.383 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.377+0000 7f9142600640 1 -- 172.21.15.17:0/2989572997 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f913c1f0c70 con 0x7f911c007990 2024-09-10T21:40:44.383 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.377+0000 7f9138c00640 1 -- 172.21.15.17:0/2989572997 <== mon.2 v2:172.21.15.31:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f913002bb50 con 0x7f911c007990 2024-09-10T21:40:44.383 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.377+0000 7f9142600640 1 -- 172.21.15.17:0/2989572997 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f913c0aedc0 con 0x7f911c007990 2024-09-10T21:40:44.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.377+0000 7f9142600640 1 -- 172.21.15.17:0/2989572997 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f913c0ae8f0 con 0x7f911c007990 2024-09-10T21:40:44.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.381+0000 7f9138c00640 1 -- 172.21.15.17:0/2989572997 <== mon.2 v2:172.21.15.31:3301/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f9130053b80 con 0x7f911c007990 2024-09-10T21:40:44.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.381+0000 7f9138c00640 1 --2- 172.21.15.17:0/2989572997 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f911c077df0 0x7f911c07a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:40:44.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.381+0000 7f9138c00640 1 -- 172.21.15.17:0/2989572997 <== mon.2 v2:172.21.15.31:3301/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f9130097190 con 0x7f911c007990 2024-09-10T21:40:44.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.381+0000 7f913be00640 1 --2- 172.21.15.17:0/2989572997 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f911c077df0 0x7f911c07a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:40:44.387 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.381+0000 7f913be00640 1 --2- 172.21.15.17:0/2989572997 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f911c077df0 0x7f911c07a2b0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f9128002600 tx=0x7f9128009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:40:44.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.385+0000 7f9138c00640 1 -- 172.21.15.17:0/2989572997 <== mon.2 v2:172.21.15.31:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f9130026050 con 0x7f911c007990 2024-09-10T21:40:44.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.625+0000 7f9142600640 1 -- 172.21.15.17:0/2989572997 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f913c15dbd0 con 0x7f911c007990 2024-09-10T21:40:44.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.625+0000 7f9138c00640 1 -- 172.21.15.17:0/2989572997 <== mon.2 v2:172.21.15.31:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1551 (secure 0 0 0) 0x7f9130019040 con 0x7f911c007990 2024-09-10T21:40:44.631 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":124,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":0,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5e2f023f-d6a6-49f4-a256-f4807dce3739","modified":"2024-09-10T21:23:46.892130Z","created":"2024-09-10T21:23:46.892130Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3301","nonce":0},{"type":"v1","addr":"172.21.15.31:6790","nonce":0}]},"addr":"172.21.15.31:6790/0","public_addr":"172.21.15.31:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-10T21:40:44.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.629+0000 7f9142600640 1 -- 172.21.15.17:0/2989572997 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f911c077df0 msgr2=0x7f911c07a2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:40:44.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.629+0000 7f9142600640 1 --2- 172.21.15.17:0/2989572997 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f911c077df0 0x7f911c07a2b0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f9128002600 tx=0x7f9128009040 comp rx=0 tx=0).stop 2024-09-10T21:40:44.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.629+0000 7f9142600640 1 -- 172.21.15.17:0/2989572997 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f911c007990 msgr2=0x7f913c0af5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:40:44.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.629+0000 7f9142600640 1 --2- 172.21.15.17:0/2989572997 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f911c007990 0x7f913c0af5c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f9130012ee0 tx=0x7f91300217f0 comp rx=0 tx=0).stop 2024-09-10T21:40:44.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.629+0000 7f9142600640 1 -- 172.21.15.17:0/2989572997 shutdown_connections 2024-09-10T21:40:44.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.629+0000 7f9142600640 1 --2- 172.21.15.17:0/2989572997 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f911c077df0 0x7f911c07a2b0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:40:44.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.629+0000 7f9142600640 1 --2- 172.21.15.17:0/2989572997 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f911c013530 0x7f913c0b0230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:40:44.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.629+0000 7f9142600640 1 --2- 172.21.15.17:0/2989572997 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f911c012b60 0x7f913c1e7340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:40:44.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.629+0000 7f9142600640 1 --2- 172.21.15.17:0/2989572997 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f911c007990 0x7f913c0af5c0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:40:44.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.629+0000 7f9142600640 1 -- 172.21.15.17:0/2989572997 >> 172.21.15.17:0/2989572997 conn(0x7f913c070ca0 msgr2=0x7f913c06cc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:40:44.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.629+0000 7f9142600640 1 -- 172.21.15.17:0/2989572997 shutdown_connections 2024-09-10T21:40:44.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.629+0000 7f9142600640 1 -- 172.21.15.17:0/2989572997 wait complete. 2024-09-10T21:40:44.652 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2024-09-10T21:40:44.652 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon b 2024-09-10T21:40:44.653 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-10T21:40:44.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.797+0000 7f2cbf200640 1 -- 172.21.15.17:0/3150426810 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f2cc00cbd10 msgr2=0x7f2cc0158d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:40:44.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.797+0000 7f2cbf200640 1 --2- 172.21.15.17:0/3150426810 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f2cc00cbd10 0x7f2cc0158d90 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f2cb8009bc0 tx=0x7f2cb801cba0 comp rx=0 tx=0).stop 2024-09-10T21:40:44.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.797+0000 7f2cbf200640 1 -- 172.21.15.17:0/3150426810 shutdown_connections 2024-09-10T21:40:44.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.797+0000 7f2cbf200640 1 --2- 172.21.15.17:0/3150426810 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f2cc00cbd10 0x7f2cc0158d90 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:40:44.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.797+0000 7f2cbf200640 1 --2- 172.21.15.17:0/3150426810 >> v2:172.21.15.31:3300/0 conn(0x7f2cc0152450 0x7f2cc0158850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:40:44.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.797+0000 7f2cbf200640 1 -- 172.21.15.17:0/3150426810 >> 172.21.15.17:0/3150426810 conn(0x7f2cc006cc20 msgr2=0x7f2cc006d030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:40:44.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.797+0000 7f2cbf200640 1 -- 172.21.15.17:0/3150426810 shutdown_connections 2024-09-10T21:40:44.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.797+0000 7f2cbf200640 1 -- 172.21.15.17:0/3150426810 wait complete. 2024-09-10T21:40:44.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.797+0000 7f2cbf200640 1 Processor -- start 2024-09-10T21:40:44.802 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.797+0000 7f2cbf200640 1 -- start start 2024-09-10T21:40:44.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.797+0000 7f2cbf200640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2cc00cbd10 0x7f2cc01429c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:40:44.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.797+0000 7f2cbf200640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f2cc0152070 0x7f2cc006dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:40:44.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.797+0000 7f2cbf200640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2cc0152450 0x7f2cc013e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:40:44.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.797+0000 7f2cbf200640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f2cc0072cb0 con 0x7f2cc00cbd10 2024-09-10T21:40:44.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.797+0000 7f2cbf200640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f2cc0072b30 con 0x7f2cc0152450 2024-09-10T21:40:44.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.797+0000 7f2cbf200640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f2cc0165400 con 0x7f2cc0152070 2024-09-10T21:40:44.803 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.797+0000 7f2cbde00640 1 -- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2cc00cbd10 msgr2=0x7f2cc01429c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-10T21:40:44.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.797+0000 7f2cbde00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2cc00cbd10 0x7f2cc01429c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:40:44.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.797+0000 7f2cb7e00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f2cc0152070 0x7f2cc006dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:40:44.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.797+0000 7f2cbe800640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2cc0152450 0x7f2cc013e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:40:44.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.797+0000 7f2cb7e00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f2cc0152070 0x7f2cc006dd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3301/0 says I am v2:172.21.15.17:56066/0 (socket says 172.21.15.17:56066) 2024-09-10T21:40:44.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.797+0000 7f2cb7e00640 1 -- 172.21.15.17:0/1668762630 learned_addr learned my addr 172.21.15.17:0/1668762630 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:40:44.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.801+0000 7f2cbe800640 1 -- 172.21.15.17:0/1668762630 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f2cc0152070 msgr2=0x7f2cc006dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:40:44.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.801+0000 7f2cbe800640 1 --2- 172.21.15.17:0/1668762630 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f2cc0152070 0x7f2cc006dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:40:44.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.801+0000 7f2cbe800640 1 -- 172.21.15.17:0/1668762630 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2cc00cbd10 msgr2=0x7f2cc01429c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:40:44.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.801+0000 7f2cbe800640 1 --2- 172.21.15.17:0/1668762630 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2cc00cbd10 0x7f2cc01429c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:40:44.806 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.801+0000 7f2cbe800640 1 -- 172.21.15.17:0/1668762630 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2cc01f83e0 con 0x7f2cc0152450 2024-09-10T21:40:44.806 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.801+0000 7f2cbe800640 1 --2- 172.21.15.17:0/1668762630 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2cc0152450 0x7f2cc013e870 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f2cb80ac000 tx=0x7f2cb80092f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:40:44.806 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.801+0000 7f2cb6000640 1 -- 172.21.15.17:0/1668762630 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cb80bb070 con 0x7f2cc0152450 2024-09-10T21:40:44.806 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.801+0000 7f2cb6000640 1 -- 172.21.15.17:0/1668762630 <== mon.1 v2:172.21.15.31:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2cb8004d40 con 0x7f2cc0152450 2024-09-10T21:40:44.806 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.801+0000 7f2cb6000640 1 -- 172.21.15.17:0/1668762630 <== mon.1 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cb80b2040 con 0x7f2cc0152450 2024-09-10T21:40:44.806 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.801+0000 7f2cbf200640 1 -- 172.21.15.17:0/1668762630 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2cc01454a0 con 0x7f2cc0152450 2024-09-10T21:40:44.806 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.801+0000 7f2cbf200640 1 -- 172.21.15.17:0/1668762630 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2cc01f7620 con 0x7f2cc0152450 2024-09-10T21:40:44.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.801+0000 7f2cbf200640 1 --2- 172.21.15.17:0/1668762630 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2cc006e270 0x7f2cc013edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:40:44.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.801+0000 7f2cbf200640 1 -- 172.21.15.17:0/1668762630 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f2cc00cb270 con 0x7f2cc006e270 2024-09-10T21:40:44.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.801+0000 7f2cbde00640 1 --2- 172.21.15.17:0/1668762630 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2cc006e270 0x7f2cc013edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:40:44.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.801+0000 7f2cbde00640 1 -- 172.21.15.17:0/1668762630 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2cc0072420 con 0x7f2cc006e270 2024-09-10T21:40:44.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.801+0000 7f2cbde00640 1 --2- 172.21.15.17:0/1668762630 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2cc006e270 0x7f2cc013edb0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f2cb0000bd0 tx=0x7f2cb000c830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:40:44.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.801+0000 7f2cb6000640 1 -- 172.21.15.17:0/1668762630 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cb00074e0 con 0x7f2cc006e270 2024-09-10T21:40:44.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.801+0000 7f2cb6000640 1 -- 172.21.15.17:0/1668762630 <== mon.1 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f2cb80b5030 con 0x7f2cc0152450 2024-09-10T21:40:44.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.805+0000 7f2cb6000640 1 --2- 172.21.15.17:0/1668762630 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f2c9c072510 0x7f2c9c0749d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:40:44.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.805+0000 7f2cb6000640 1 -- 172.21.15.17:0/1668762630 <== mon.1 v2:172.21.15.31:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f2cb812d0d0 con 0x7f2cc0152450 2024-09-10T21:40:44.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.805+0000 7f2cb6000640 1 -- 172.21.15.17:0/1668762630 <== mon.1 v2:172.21.15.31:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f2cb0004020 con 0x7f2cc006e270 2024-09-10T21:40:44.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.805+0000 7f2cb6000640 1 -- 172.21.15.17:0/1668762630 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2cc006e270 msgr2=0x7f2cc013edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:40:44.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.805+0000 7f2cb6000640 1 --2- 172.21.15.17:0/1668762630 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2cc006e270 0x7f2cc013edb0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f2cb0000bd0 tx=0x7f2cb000c830 comp rx=0 tx=0).stop 2024-09-10T21:40:44.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.805+0000 7f2cb6000640 1 -- 172.21.15.17:0/1668762630 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2cc006e270 msgr2=0x7f2cc013edb0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:40:44.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.805+0000 7f2cb6000640 1 --2- 172.21.15.17:0/1668762630 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2cc006e270 0x7f2cc013edb0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:40:44.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.805+0000 7f2cb7e00640 1 --2- 172.21.15.17:0/1668762630 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f2c9c072510 0x7f2c9c0749d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:40:44.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.805+0000 7f2cb7e00640 1 --2- 172.21.15.17:0/1668762630 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f2c9c072510 0x7f2c9c0749d0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f2cc0143360 tx=0x7f2ca8009260 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:40:44.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.809+0000 7f2cbf200640 1 --2- 172.21.15.17:0/1668762630 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2cc0144d20 0x7f2cc01f7860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:40:44.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.809+0000 7f2cbf200640 1 -- 172.21.15.17:0/1668762630 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f2cc0069ef0 con 0x7f2cc0144d20 2024-09-10T21:40:44.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.809+0000 7f2cbde00640 1 --2- 172.21.15.17:0/1668762630 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2cc0144d20 0x7f2cc01f7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:40:44.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.809+0000 7f2cbde00640 1 -- 172.21.15.17:0/1668762630 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f2cc00cc710 con 0x7f2cc0144d20 2024-09-10T21:40:44.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.813+0000 7f2cbde00640 1 --2- 172.21.15.17:0/1668762630 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2cc0144d20 0x7f2cc01f7860 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f2cb0013000 tx=0x7f2cb0000ee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:40:44.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.813+0000 7f2cb6000640 1 -- 172.21.15.17:0/1668762630 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cb0019880 con 0x7f2cc0144d20 2024-09-10T21:40:44.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.813+0000 7f2cb6000640 1 -- 172.21.15.17:0/1668762630 <== mon.1 v2:172.21.15.31:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4866 (secure 0 0 0) 0x7f2cb0019880 con 0x7f2cc0144d20 2024-09-10T21:40:44.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.813+0000 7f2cb6000640 1 -- 172.21.15.17:0/1668762630 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2cc0144d20 msgr2=0x7f2cc01f7860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:40:44.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.813+0000 7f2cb6000640 1 --2- 172.21.15.17:0/1668762630 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2cc0144d20 0x7f2cc01f7860 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f2cb0013000 tx=0x7f2cb0000ee0 comp rx=0 tx=0).stop 2024-09-10T21:40:44.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.813+0000 7f2cb6000640 1 -- 172.21.15.17:0/1668762630 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2cc0144d20 msgr2=0x7f2cc01f7860 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:40:44.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.813+0000 7f2cb6000640 1 --2- 172.21.15.17:0/1668762630 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2cc0144d20 0x7f2cc01f7860 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:40:44.817 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-10T21:40:44.817 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:40:44.817 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:40:44.817 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "leader", 2024-09-10T21:40:44.818 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 124, 2024-09-10T21:40:44.818 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 56243, 2024-09-10T21:40:44.818 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-10T21:40:44.818 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-10T21:40:44.818 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-10T21:40:44.818 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:40:44.818 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 0, 2024-09-10T21:40:44.818 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:40:44.818 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-10T21:40:44.818 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-10T21:40:44.818 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:40:44.818 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:40:44.819 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:40:44.819 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:40:44.819 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:40:44.819 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:40:44.819 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:40:44.819 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:40:44.819 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:40:44.819 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:40:44.819 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:40:44.819 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:40:44.819 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-10T21:40:44.819 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-10T21:40:44.819 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:40:44.819 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:40:44.820 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:40:44.820 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:40:44.820 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:40:44.820 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:40:44.820 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:40:44.820 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:40:44.820 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:40:44.820 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:40:44.820 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:40:44.820 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:40:44.820 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:40:44.820 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-10T21:40:44.820 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-10T21:40:44.821 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-10T21:40:44.821 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-10T21:40:44.821 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-10T21:40:44.821 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "5e2f023f-d6a6-49f4-a256-f4807dce3739", 2024-09-10T21:40:44.821 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:40:44.821 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:40:44.821 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-10T21:40:44.821 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-10T21:40:44.821 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-10T21:40:44.821 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-10T21:40:44.821 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-10T21:40:44.821 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-10T21:40:44.822 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-10T21:40:44.822 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:40:44.822 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-10T21:40:44.822 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:40:44.822 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:40:44.822 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:40:44.822 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:40:44.822 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:40:44.822 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:40:44.822 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:40:44.822 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:40:44.822 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:40:44.822 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:40:44.822 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:40:44.823 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:40:44.823 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-10T21:40:44.823 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:40:44.823 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-10T21:40:44.823 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:40:44.823 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:40:44.823 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:40:44.823 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:40:44.823 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:40:44.823 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:40:44.823 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:40:44.823 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-10T21:40:44.824 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:40:44.824 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:40:44.824 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:40:44.824 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:40:44.824 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-10T21:40:44.824 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:40:44.824 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:40:44.824 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:40:44.824 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:40:44.824 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-10T21:40:44.824 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-10T21:40:44.824 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:40:44.824 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:40:44.825 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:40:44.825 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:40:44.825 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:40:44.825 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:40:44.825 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:40:44.825 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:40:44.825 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:40:44.825 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:40:44.825 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:40:44.825 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3300", 2024-09-10T21:40:44.825 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:40:44.825 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:40:44.825 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:40:44.826 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:40:44.826 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789", 2024-09-10T21:40:44.826 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:40:44.826 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:40:44.826 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:40:44.826 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:40:44.826 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789/0", 2024-09-10T21:40:44.826 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6789/0", 2024-09-10T21:40:44.826 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:40:44.826 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:40:44.826 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:40:44.826 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:40:44.826 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:40:44.826 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:40:44.827 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:40:44.827 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:40:44.827 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:40:44.827 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:40:44.827 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:40:44.827 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3301", 2024-09-10T21:40:44.827 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:40:44.827 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:40:44.827 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:40:44.828 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:40:44.828 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790", 2024-09-10T21:40:44.828 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:40:44.828 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:40:44.828 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:40:44.828 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:40:44.828 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790/0", 2024-09-10T21:40:44.828 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6790/0", 2024-09-10T21:40:44.828 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:40:44.828 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:40:44.828 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:40:44.828 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:40:44.828 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:40:44.829 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:40:44.829 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-10T21:40:44.829 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-10T21:40:44.829 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:40:44.829 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:40:44.829 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:40:44.829 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:40:44.829 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:40:44.829 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:40:44.829 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": [ 2024-09-10T21:40:44.829 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:40:44.829 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:40:44.829 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:40:44.829 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:40:44.830 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:40:44.830 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:40:44.830 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-10T21:40:44.830 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:40:44.830 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:40:44.830 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:40:44.830 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:40:44.830 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:40:44.830 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:40:44.830 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-10T21:40:44.830 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:40:44.830 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:40:44.830 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:40:44.830 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 3 2024-09-10T21:40:44.831 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:40:44.831 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:40:44.831 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:40:44.831 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-10T21:40:44.831 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-10T21:40:44.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.813+0000 7f2cbf200640 1 -- 172.21.15.17:0/1668762630 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f2c9c072510 msgr2=0x7f2c9c0749d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:40:44.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.813+0000 7f2cbf200640 1 --2- 172.21.15.17:0/1668762630 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f2c9c072510 0x7f2c9c0749d0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f2cc0143360 tx=0x7f2ca8009260 comp rx=0 tx=0).stop 2024-09-10T21:40:44.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.813+0000 7f2cbf200640 1 -- 172.21.15.17:0/1668762630 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2cc0152450 msgr2=0x7f2cc013e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:40:44.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.813+0000 7f2cbe800640 1 -- 172.21.15.17:0/1668762630 reap_dead start 2024-09-10T21:40:44.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.813+0000 7f2cbf200640 1 --2- 172.21.15.17:0/1668762630 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2cc0152450 0x7f2cc013e870 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f2cb80ac000 tx=0x7f2cb80092f0 comp rx=0 tx=0).stop 2024-09-10T21:40:44.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.813+0000 7f2cbf200640 1 -- 172.21.15.17:0/1668762630 shutdown_connections 2024-09-10T21:40:44.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.813+0000 7f2cbf200640 1 --2- 172.21.15.17:0/1668762630 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2cc0152450 0x7f2cc013e870 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:40:44.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.813+0000 7f2cbf200640 1 -- 172.21.15.17:0/1668762630 >> 172.21.15.17:0/1668762630 conn(0x7f2cc006cc20 msgr2=0x7f2cc0156b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:40:44.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.813+0000 7f2cbf200640 1 -- 172.21.15.17:0/1668762630 shutdown_connections 2024-09-10T21:40:44.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.813+0000 7f2cbf200640 1 -- 172.21.15.17:0/1668762630 wait complete. 2024-09-10T21:40:44.835 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-09-10T21:40:44.835 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon c 2024-09-10T21:40:44.836 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-10T21:40:44.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.981+0000 7f488fe00640 1 -- 172.21.15.17:0/4108787424 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f48881526b0 msgr2=0x7f4888159850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:40:44.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.981+0000 7f488fe00640 1 --2- 172.21.15.17:0/4108787424 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f48881526b0 0x7f4888159850 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f487c009870 tx=0x7f487c01f050 comp rx=0 tx=0).stop 2024-09-10T21:40:44.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.981+0000 7f488fe00640 1 -- 172.21.15.17:0/4108787424 shutdown_connections 2024-09-10T21:40:44.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.981+0000 7f488fe00640 1 --2- 172.21.15.17:0/4108787424 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f48881526b0 0x7f4888159850 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:40:44.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.981+0000 7f488fe00640 1 -- 172.21.15.17:0/4108787424 >> 172.21.15.17:0/4108787424 conn(0x7f4888068990 msgr2=0x7f4888068da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:40:44.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.981+0000 7f488fe00640 1 -- 172.21.15.17:0/4108787424 shutdown_connections 2024-09-10T21:40:44.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.981+0000 7f488fe00640 1 -- 172.21.15.17:0/4108787424 wait complete. 2024-09-10T21:40:44.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.981+0000 7f488fe00640 1 Processor -- start 2024-09-10T21:40:44.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.981+0000 7f488fe00640 1 -- start start 2024-09-10T21:40:44.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.981+0000 7f488fe00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4888071760 0x7f488814fee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:40:44.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.981+0000 7f488fe00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48880c4dd0 0x7f48880ca770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:40:44.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.981+0000 7f488fe00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f48881526b0 0x7f488806eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:40:44.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.981+0000 7f488fe00640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f48880c8a40 con 0x7f4888071760 2024-09-10T21:40:44.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.981+0000 7f488fe00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f4888152a90 con 0x7f48880c4dd0 2024-09-10T21:40:44.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.981+0000 7f488fe00640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f48881f5510 con 0x7f48881526b0 2024-09-10T21:40:44.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.981+0000 7f488d400640 1 -- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4888071760 msgr2=0x7f488814fee0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-10T21:40:44.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.981+0000 7f488d400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4888071760 0x7f488814fee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:40:44.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.981+0000 7f488ca00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48880c4dd0 0x7f48880ca770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:40:44.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.981+0000 7f488de00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f48881526b0 0x7f488806eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:40:44.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.981+0000 7f488ca00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48880c4dd0 0x7f48880ca770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.17:34648/0 (socket says 172.21.15.17:34648) 2024-09-10T21:40:44.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.981+0000 7f488ca00640 1 -- 172.21.15.17:0/2165395317 learned_addr learned my addr 172.21.15.17:0/2165395317 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:40:44.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.981+0000 7f488ca00640 1 -- 172.21.15.17:0/2165395317 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f48881526b0 msgr2=0x7f488806eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:40:44.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.981+0000 7f488ca00640 1 --2- 172.21.15.17:0/2165395317 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f48881526b0 0x7f488806eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:40:44.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.981+0000 7f488ca00640 1 -- 172.21.15.17:0/2165395317 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4888071760 msgr2=0x7f488814fee0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:40:44.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.981+0000 7f488ca00640 1 --2- 172.21.15.17:0/2165395317 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4888071760 0x7f488814fee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:40:44.988 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.981+0000 7f488ca00640 1 -- 172.21.15.17:0/2165395317 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f488806f760 con 0x7f48880c4dd0 2024-09-10T21:40:44.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.985+0000 7f488ca00640 1 --2- 172.21.15.17:0/2165395317 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48880c4dd0 0x7f48880ca770 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f487400c460 tx=0x7f487400c920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:40:44.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.985+0000 7f4885600640 1 -- 172.21.15.17:0/2165395317 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4874016020 con 0x7f48880c4dd0 2024-09-10T21:40:44.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.985+0000 7f4885600640 1 -- 172.21.15.17:0/2165395317 <== mon.1 v2:172.21.15.31:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4874012940 con 0x7f48880c4dd0 2024-09-10T21:40:44.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.985+0000 7f4885600640 1 -- 172.21.15.17:0/2165395317 <== mon.1 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4874012d90 con 0x7f48880c4dd0 2024-09-10T21:40:44.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.985+0000 7f488fe00640 1 -- 172.21.15.17:0/2165395317 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f488806fa40 con 0x7f48880c4dd0 2024-09-10T21:40:44.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.985+0000 7f488fe00640 1 -- 172.21.15.17:0/2165395317 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f488806e650 con 0x7f48880c4dd0 2024-09-10T21:40:44.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.985+0000 7f488fe00640 1 --2- 172.21.15.17:0/2165395317 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f4888159850 0x7f488806f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:40:44.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.985+0000 7f488fe00640 1 -- 172.21.15.17:0/2165395317 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f48880c5260 con 0x7f4888159850 2024-09-10T21:40:44.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.985+0000 7f488d400640 1 --2- 172.21.15.17:0/2165395317 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f4888159850 0x7f488806f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:40:44.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.985+0000 7f488d400640 1 -- 172.21.15.17:0/2165395317 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f48880cde50 con 0x7f4888159850 2024-09-10T21:40:44.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.985+0000 7f488d400640 1 --2- 172.21.15.17:0/2165395317 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f4888159850 0x7f488806f0b0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f48880b4520 tx=0x7f487c005d30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:40:44.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.985+0000 7f4885600640 1 -- 172.21.15.17:0/2165395317 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f487c0b5050 con 0x7f4888159850 2024-09-10T21:40:44.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.985+0000 7f4885600640 1 -- 172.21.15.17:0/2165395317 <== mon.1 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f487401b400 con 0x7f48880c4dd0 2024-09-10T21:40:44.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.985+0000 7f4885600640 1 --2- 172.21.15.17:0/2165395317 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f48680724e0 0x7f48680749a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:40:44.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.989+0000 7f4885600640 1 -- 172.21.15.17:0/2165395317 <== mon.1 v2:172.21.15.31:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f4874093780 con 0x7f48880c4dd0 2024-09-10T21:40:44.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.989+0000 7f488de00640 1 --2- 172.21.15.17:0/2165395317 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f48680724e0 0x7f48680749a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:40:44.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.989+0000 7f4885600640 1 -- 172.21.15.17:0/2165395317 <== mon.2 v2:172.21.15.31:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f487c0b6040 con 0x7f4888159850 2024-09-10T21:40:44.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.989+0000 7f4885600640 1 -- 172.21.15.17:0/2165395317 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f4888159850 msgr2=0x7f488806f0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:40:44.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.989+0000 7f4885600640 1 --2- 172.21.15.17:0/2165395317 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f4888159850 0x7f488806f0b0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f48880b4520 tx=0x7f487c005d30 comp rx=0 tx=0).stop 2024-09-10T21:40:44.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.989+0000 7f4885600640 1 -- 172.21.15.17:0/2165395317 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f4888159850 msgr2=0x7f488806f0b0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:40:44.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.989+0000 7f4885600640 1 --2- 172.21.15.17:0/2165395317 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f4888159850 0x7f488806f0b0 secure :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f48880b4520 tx=0x7f487c005d30 comp rx=0 tx=0).stop 2024-09-10T21:40:44.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.989+0000 7f488de00640 1 --2- 172.21.15.17:0/2165395317 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f48680724e0 0x7f48680749a0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f4880001430 tx=0x7f488000d040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:40:44.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.993+0000 7f488fe00640 1 --2- 172.21.15.17:0/2165395317 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f48880cb2c0 0x7f488813f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:40:44.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.993+0000 7f488fe00640 1 -- 172.21.15.17:0/2165395317 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f488806e1d0 con 0x7f48880cb2c0 2024-09-10T21:40:44.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.993+0000 7f488d400640 1 --2- 172.21.15.17:0/2165395317 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f48880cb2c0 0x7f488813f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:40:44.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.993+0000 7f488d400640 1 -- 172.21.15.17:0/2165395317 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f4888071490 con 0x7f48880cb2c0 2024-09-10T21:40:44.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.993+0000 7f488d400640 1 --2- 172.21.15.17:0/2165395317 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f48880cb2c0 0x7f488813f820 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f487c009870 tx=0x7f487c01f050 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:40:44.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.997+0000 7f4885600640 1 -- 172.21.15.17:0/2165395317 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f487c0c3ae0 con 0x7f48880cb2c0 2024-09-10T21:40:45.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.997+0000 7f4885600640 1 -- 172.21.15.17:0/2165395317 <== mon.2 v2:172.21.15.31:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5032 (secure 0 0 0) 0x7f487c0c3ac0 con 0x7f48880cb2c0 2024-09-10T21:40:45.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.997+0000 7f4885600640 1 -- 172.21.15.17:0/2165395317 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f48880cb2c0 msgr2=0x7f488813f820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:40:45.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.997+0000 7f4885600640 1 --2- 172.21.15.17:0/2165395317 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f48880cb2c0 0x7f488813f820 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f487c009870 tx=0x7f487c01f050 comp rx=0 tx=0).stop 2024-09-10T21:40:45.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.997+0000 7f4885600640 1 -- 172.21.15.17:0/2165395317 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f48880cb2c0 msgr2=0x7f488813f820 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:40:45.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.997+0000 7f4885600640 1 --2- 172.21.15.17:0/2165395317 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f48880cb2c0 0x7f488813f820 secure :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f487c009870 tx=0x7f487c01f050 comp rx=0 tx=0).stop 2024-09-10T21:40:45.001 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-10T21:40:45.001 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:40:45.001 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:40:45.001 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-10T21:40:45.001 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 124, 2024-09-10T21:40:45.001 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 251347, 2024-09-10T21:40:45.001 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-10T21:40:45.001 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-10T21:40:45.001 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-10T21:40:45.001 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:40:45.001 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 0, 2024-09-10T21:40:45.001 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:40:45.001 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-10T21:40:45.002 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-10T21:40:45.002 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:40:45.002 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:40:45.002 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:40:45.002 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:40:45.002 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:40:45.002 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:40:45.002 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:40:45.002 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:40:45.002 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:40:45.002 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:40:45.003 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:40:45.003 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:40:45.003 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-10T21:40:45.003 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-10T21:40:45.003 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:40:45.003 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:40:45.003 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:40:45.003 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:40:45.003 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:40:45.003 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:40:45.003 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:40:45.003 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:40:45.004 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:40:45.004 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:40:45.004 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:40:45.004 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:40:45.004 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:40:45.004 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-10T21:40:45.004 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-10T21:40:45.004 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-10T21:40:45.004 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-10T21:40:45.004 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-10T21:40:45.004 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "5e2f023f-d6a6-49f4-a256-f4807dce3739", 2024-09-10T21:40:45.005 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:40:45.005 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:40:45.005 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-10T21:40:45.005 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-10T21:40:45.005 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-10T21:40:45.005 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-10T21:40:45.005 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-10T21:40:45.005 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-10T21:40:45.005 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-10T21:40:45.005 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:40:45.005 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-10T21:40:45.005 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:40:45.005 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:40:45.006 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:40:45.006 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:40:45.006 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:40:45.006 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:40:45.006 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:40:45.006 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:40:45.006 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:40:45.006 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:40:45.006 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:40:45.006 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:40:45.006 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-10T21:40:45.006 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:40:45.006 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-10T21:40:45.006 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:40:45.007 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:40:45.007 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:40:45.007 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:40:45.007 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:40:45.007 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:40:45.007 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:40:45.007 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-10T21:40:45.007 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:40:45.007 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:40:45.007 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:40:45.007 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:40:45.007 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-10T21:40:45.007 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:40:45.007 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:40:45.008 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:40:45.008 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:40:45.008 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-10T21:40:45.008 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-10T21:40:45.008 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:40:45.008 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:40:45.008 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:40:45.008 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:40:45.008 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:40:45.008 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:40:45.008 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:40:45.008 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:40:45.008 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:40:45.009 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:40:45.009 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:40:45.009 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3300", 2024-09-10T21:40:45.009 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:40:45.009 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:40:45.009 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:40:45.009 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:40:45.009 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789", 2024-09-10T21:40:45.009 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:40:45.009 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:40:45.009 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:40:45.009 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:40:45.009 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789/0", 2024-09-10T21:40:45.010 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6789/0", 2024-09-10T21:40:45.010 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:40:45.010 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:40:45.010 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:40:45.010 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:40:45.010 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:40:45.010 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:40:45.010 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:40:45.010 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:40:45.010 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:40:45.010 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:40:45.010 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:40:45.011 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3301", 2024-09-10T21:40:45.011 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:40:45.011 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:40:45.011 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:40:45.011 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:40:45.011 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790", 2024-09-10T21:40:45.011 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:40:45.011 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:40:45.011 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:40:45.011 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:40:45.011 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790/0", 2024-09-10T21:40:45.011 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6790/0", 2024-09-10T21:40:45.012 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:40:45.012 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:40:45.012 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:40:45.012 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:40:45.012 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:40:45.012 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:40:45.012 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-10T21:40:45.012 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-10T21:40:45.012 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:40:45.012 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:40:45.012 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:40:45.013 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:40:45.013 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:40:45.013 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:40:45.013 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": [ 2024-09-10T21:40:45.013 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:40:45.013 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:40:45.013 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:40:45.013 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 5 2024-09-10T21:40:45.013 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:40:45.013 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:40:45.013 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-10T21:40:45.013 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:40:45.014 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:40:45.014 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:40:45.014 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 7 2024-09-10T21:40:45.014 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:40:45.014 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:40:45.014 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-10T21:40:45.014 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:40:45.014 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:40:45.014 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:40:45.014 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 5 2024-09-10T21:40:45.014 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:40:45.014 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:40:45.015 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": [ 2024-09-10T21:40:45.015 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:40:45.015 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:40:45.015 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:40:45.015 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 2 2024-09-10T21:40:45.015 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:40:45.015 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:40:45.015 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:40:45.015 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-10T21:40:45.015 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-10T21:40:45.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.997+0000 7f488fe00640 1 -- 172.21.15.17:0/2165395317 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f48680724e0 msgr2=0x7f48680749a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:40:45.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.997+0000 7f488fe00640 1 --2- 172.21.15.17:0/2165395317 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f48680724e0 0x7f48680749a0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f4880001430 tx=0x7f488000d040 comp rx=0 tx=0).stop 2024-09-10T21:40:45.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.997+0000 7f488fe00640 1 -- 172.21.15.17:0/2165395317 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48880c4dd0 msgr2=0x7f48880ca770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:40:45.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.997+0000 7f488fe00640 1 --2- 172.21.15.17:0/2165395317 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48880c4dd0 0x7f48880ca770 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f487400c460 tx=0x7f487400c920 comp rx=0 tx=0).stop 2024-09-10T21:40:45.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.997+0000 7f488de00640 1 -- 172.21.15.17:0/2165395317 reap_dead start 2024-09-10T21:40:45.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.997+0000 7f488de00640 1 -- 172.21.15.17:0/2165395317 reap_dead start 2024-09-10T21:40:45.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.997+0000 7f488fe00640 1 -- 172.21.15.17:0/2165395317 shutdown_connections 2024-09-10T21:40:45.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.997+0000 7f488fe00640 1 -- 172.21.15.17:0/2165395317 >> 172.21.15.17:0/2165395317 conn(0x7f4888068990 msgr2=0x7f4888157010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:40:45.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.997+0000 7f488fe00640 1 -- 172.21.15.17:0/2165395317 shutdown_connections 2024-09-10T21:40:45.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:40:44.997+0000 7f488fe00640 1 -- 172.21.15.17:0/2165395317 wait complete. 2024-09-10T21:40:45.019 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-09-10T21:40:45.019 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2024-09-10T21:40:52.806 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:40:52.810 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:40:57.740 INFO:tasks.workunit.client.0.smithi017.stdout:fs setup took 838 secs 2024-09-10T21:40:59.234 INFO:tasks.workunit.client.0.smithi017.stdout:Syncing()...1 sec 2024-09-10T21:40:59.236 INFO:tasks.workunit.client.0.smithi017.stdout:Starting Actual Benchmark At: Tue Sep 10 21:40:59 2024 2024-09-10T21:40:59.236 INFO:tasks.workunit.client.0.smithi017.stdout: 2024-09-10T21:41:05.019 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.a 2024-09-10T21:41:05.020 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-10T21:41:05.020 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-10T21:41:05.030 INFO:tasks.ceph.mon.a:Started 2024-09-10T21:41:05.030 INFO:tasks.mon_thrash:Sending STOP to mon c 2024-09-10T21:41:05.030 INFO:tasks.ceph.mon.c:Sent signal 19 2024-09-10T21:41:05.030 INFO:tasks.mon_thrash.mon_thrasher:waiting for 15.0 secs to unfreeze mons 2024-09-10T21:41:20.031 INFO:tasks.mon_thrash:Sending CONT to mon c 2024-09-10T21:41:20.032 INFO:tasks.ceph.mon.c:Sent signal 18 2024-09-10T21:41:20.032 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-09-10T21:41:20.033 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-10T21:41:22.870 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:41:22.879 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:41:23.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:23.036+0000 7fc204800640 1 -- 172.21.15.17:0/2611452441 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc1e4004f40 msgr2=0x7fc1e400ffe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:41:23.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:23.036+0000 7fc204800640 1 --2- 172.21.15.17:0/2611452441 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc1e4004f40 0x7fc1e400ffe0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fc1fc15add0 tx=0x7fc1f801cf20 comp rx=0 tx=0).stop 2024-09-10T21:41:23.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:23.036+0000 7fc202800640 1 -- 172.21.15.17:0/2611452441 reap_dead start 2024-09-10T21:41:23.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:23.040+0000 7fc204800640 1 -- 172.21.15.17:0/2611452441 shutdown_connections 2024-09-10T21:41:23.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:23.040+0000 7fc204800640 1 -- 172.21.15.17:0/2611452441 >> 172.21.15.17:0/2611452441 conn(0x7fc1fc070160 msgr2=0x7fc1fc070570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:41:23.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:23.040+0000 7fc204800640 1 -- 172.21.15.17:0/2611452441 shutdown_connections 2024-09-10T21:41:23.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:23.040+0000 7fc204800640 1 -- 172.21.15.17:0/2611452441 wait complete. 2024-09-10T21:41:23.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:23.044+0000 7fc204800640 1 Processor -- start 2024-09-10T21:41:23.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:23.044+0000 7fc204800640 1 -- start start 2024-09-10T21:41:23.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:23.044+0000 7fc204800640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc1e4004f40 0x7fc1fc1ec890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:41:23.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:23.044+0000 7fc204800640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc1fc14b610 0x7fc1fc154e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:41:23.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:23.044+0000 7fc204800640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc1fc14c0b0 0x7fc1fc148190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:41:23.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:23.044+0000 7fc204800640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fc1fc157650 con 0x7fc1fc14b610 2024-09-10T21:41:23.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:23.044+0000 7fc204800640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fc1fc1541d0 con 0x7fc1fc14c0b0 2024-09-10T21:41:23.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:23.044+0000 7fc204800640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7fc1fc0c91c0 con 0x7fc1e4004f40 2024-09-10T21:41:23.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:23.044+0000 7fc201400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc1fc14b610 0x7fc1fc154e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:41:23.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:23.044+0000 7fc201e00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc1e4004f40 0x7fc1fc1ec890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:41:23.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:23.044+0000 7fc201e00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc1e4004f40 0x7fc1fc1ec890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3301/0 says I am v2:172.21.15.17:55794/0 (socket says 172.21.15.17:55794) 2024-09-10T21:41:23.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:23.044+0000 7fc201e00640 1 -- 172.21.15.17:0/3507543625 learned_addr learned my addr 172.21.15.17:0/3507543625 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:41:23.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:23.044+0000 7fc202800640 1 --2- 172.21.15.17:0/3507543625 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc1fc14c0b0 0x7fc1fc148190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:41:23.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:23.048+0000 7fc202800640 1 -- 172.21.15.17:0/3507543625 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc1e4004f40 msgr2=0x7fc1fc1ec890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:41:23.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:23.048+0000 7fc202800640 1 --2- 172.21.15.17:0/3507543625 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc1e4004f40 0x7fc1fc1ec890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:41:23.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:23.048+0000 7fc202800640 1 -- 172.21.15.17:0/3507543625 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc1fc14b610 msgr2=0x7fc1fc154e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:41:23.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:23.048+0000 7fc202800640 1 --2- 172.21.15.17:0/3507543625 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc1fc14b610 0x7fc1fc154e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:41:23.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:23.048+0000 7fc202800640 1 -- 172.21.15.17:0/3507543625 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1fc149ab0 con 0x7fc1fc14c0b0 2024-09-10T21:41:23.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:23.052+0000 7fc202800640 1 --2- 172.21.15.17:0/3507543625 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc1fc14c0b0 0x7fc1fc148190 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc1e4005730 tx=0x7fc1f8005270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:41:23.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:23.052+0000 7fc1f6000640 1 -- 172.21.15.17:0/3507543625 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1f80ab040 con 0x7fc1fc14c0b0 2024-09-10T21:41:23.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:23.052+0000 7fc1f6000640 1 -- 172.21.15.17:0/3507543625 <== mon.1 v2:172.21.15.31:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc1f8005660 con 0x7fc1fc14c0b0 2024-09-10T21:41:23.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:23.052+0000 7fc1f6000640 1 -- 172.21.15.17:0/3507543625 <== mon.1 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1f80ab040 con 0x7fc1fc14c0b0 2024-09-10T21:41:23.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:23.052+0000 7fc204800640 1 -- 172.21.15.17:0/3507543625 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1fc149d90 con 0x7fc1fc14c0b0 2024-09-10T21:41:23.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:23.052+0000 7fc204800640 1 -- 172.21.15.17:0/3507543625 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1fc1ebdb0 con 0x7fc1fc14c0b0 2024-09-10T21:41:23.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:23.056+0000 7fc204800640 1 -- 172.21.15.17:0/3507543625 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1fc0725c0 con 0x7fc1fc14c0b0 2024-09-10T21:41:23.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:23.056+0000 7fc1f6000640 1 -- 172.21.15.17:0/3507543625 <== mon.1 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fc1f80b8070 con 0x7fc1fc14c0b0 2024-09-10T21:41:23.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:23.060+0000 7fc1f6000640 1 --2- 172.21.15.17:0/3507543625 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fc1e0072810 0x7fc1e0074cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:41:23.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:23.060+0000 7fc1f6000640 1 -- 172.21.15.17:0/3507543625 <== mon.1 v2:172.21.15.31:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fc1f800a2b0 con 0x7fc1fc14c0b0 2024-09-10T21:41:23.067 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:23.060+0000 7fc201e00640 1 --2- 172.21.15.17:0/3507543625 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fc1e0072810 0x7fc1e0074cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:41:23.070 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:23.064+0000 7fc201e00640 1 --2- 172.21.15.17:0/3507543625 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fc1e0072810 0x7fc1e0074cd0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fc1f0001230 tx=0x7fc1f0006730 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:41:23.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:23.076+0000 7fc1f6000640 1 -- 172.21.15.17:0/3507543625 <== mon.1 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fc1f8016c70 con 0x7fc1fc14c0b0 2024-09-10T21:41:23.718 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:23.712+0000 7fc204800640 1 -- 172.21.15.17:0/3507543625 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fc1fc0c7a20 con 0x7fc1fc14c0b0 2024-09-10T21:41:23.720 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:23.716+0000 7fc1f6000640 1 -- 172.21.15.17:0/3507543625 <== mon.1 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1557 (secure 0 0 0) 0x7fc1f80f82b0 con 0x7fc1fc14c0b0 2024-09-10T21:41:23.722 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":130,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5e2f023f-d6a6-49f4-a256-f4807dce3739","modified":"2024-09-10T21:23:46.892130Z","created":"2024-09-10T21:23:46.892130Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3301","nonce":0},{"type":"v1","addr":"172.21.15.31:6790","nonce":0}]},"addr":"172.21.15.31:6790/0","public_addr":"172.21.15.31:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-10T21:41:23.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:23.728+0000 7fc204800640 1 -- 172.21.15.17:0/3507543625 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fc1e0072810 msgr2=0x7fc1e0074cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:41:23.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:23.728+0000 7fc204800640 1 --2- 172.21.15.17:0/3507543625 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fc1e0072810 0x7fc1e0074cd0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fc1f0001230 tx=0x7fc1f0006730 comp rx=0 tx=0).stop 2024-09-10T21:41:23.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:23.728+0000 7fc204800640 1 -- 172.21.15.17:0/3507543625 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc1fc14c0b0 msgr2=0x7fc1fc148190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:41:23.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:23.728+0000 7fc204800640 1 --2- 172.21.15.17:0/3507543625 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc1fc14c0b0 0x7fc1fc148190 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc1e4005730 tx=0x7fc1f8005270 comp rx=0 tx=0).stop 2024-09-10T21:41:23.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:23.728+0000 7fc204800640 1 -- 172.21.15.17:0/3507543625 shutdown_connections 2024-09-10T21:41:23.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:23.728+0000 7fc204800640 1 --2- 172.21.15.17:0/3507543625 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fc1e0072810 0x7fc1e0074cd0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:41:23.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:23.728+0000 7fc204800640 1 --2- 172.21.15.17:0/3507543625 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc1fc14c0b0 0x7fc1fc148190 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:41:23.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:23.728+0000 7fc204800640 1 --2- 172.21.15.17:0/3507543625 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc1fc14b610 0x7fc1fc154e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:41:23.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:23.732+0000 7fc204800640 1 --2- 172.21.15.17:0/3507543625 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc1e4004f40 0x7fc1fc1ec890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:41:23.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:23.732+0000 7fc204800640 1 -- 172.21.15.17:0/3507543625 >> 172.21.15.17:0/3507543625 conn(0x7fc1fc070160 msgr2=0x7fc1fc151a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:41:23.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:23.732+0000 7fc204800640 1 -- 172.21.15.17:0/3507543625 shutdown_connections 2024-09-10T21:41:23.736 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:23.732+0000 7fc204800640 1 -- 172.21.15.17:0/3507543625 wait complete. 2024-09-10T21:41:23.790 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-09-10T21:41:23.790 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-09-10T21:41:23.790 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-10T21:41:24.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:24.216+0000 7f228fe00640 1 -- 172.21.15.17:0/678695447 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f228804f6c0 msgr2=0x7f22880df130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:41:24.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:24.216+0000 7f228fe00640 1 --2- 172.21.15.17:0/678695447 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f228804f6c0 0x7f22880df130 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f2280009840 tx=0x7f228001c820 comp rx=0 tx=0).stop 2024-09-10T21:41:24.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:24.220+0000 7f228fe00640 1 -- 172.21.15.17:0/678695447 shutdown_connections 2024-09-10T21:41:24.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:24.220+0000 7f228fe00640 1 --2- 172.21.15.17:0/678695447 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f228804faa0 0x7f228804fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:41:24.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:24.220+0000 7f228fe00640 1 --2- 172.21.15.17:0/678695447 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f228804f6c0 0x7f22880df130 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:41:24.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:24.220+0000 7f228fe00640 1 -- 172.21.15.17:0/678695447 >> 172.21.15.17:0/678695447 conn(0x7f2288050a10 msgr2=0x7f2288050e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:41:24.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:24.220+0000 7f228fe00640 1 -- 172.21.15.17:0/678695447 shutdown_connections 2024-09-10T21:41:24.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:24.220+0000 7f228fe00640 1 -- 172.21.15.17:0/678695447 wait complete. 2024-09-10T21:41:24.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:24.224+0000 7f228fe00640 1 Processor -- start 2024-09-10T21:41:24.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:24.224+0000 7f228fe00640 1 -- start start 2024-09-10T21:41:24.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:24.224+0000 7f228fe00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2288039420 0x7f228817c080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:41:24.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:24.224+0000 7f228fe00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f228804f6c0 0x7f2288047860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:41:24.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:24.224+0000 7f228fe00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f228804faa0 0x7f22880da580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:41:24.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:24.224+0000 7f228fe00640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f228804f090 con 0x7f2288039420 2024-09-10T21:41:24.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:24.224+0000 7f228fe00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f2288039cc0 con 0x7f228804faa0 2024-09-10T21:41:24.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:24.224+0000 7f228fe00640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f228804f210 con 0x7f228804f6c0 2024-09-10T21:41:24.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:24.224+0000 7f228f400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f228804faa0 0x7f22880da580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:41:24.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:24.224+0000 7f228f400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f228804faa0 0x7f22880da580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.17:51702/0 (socket says 172.21.15.17:51702) 2024-09-10T21:41:24.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:24.224+0000 7f228f400640 1 -- 172.21.15.17:0/3464964532 learned_addr learned my addr 172.21.15.17:0/3464964532 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:41:24.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:24.228+0000 7f228e000640 1 --2- 172.21.15.17:0/3464964532 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f228804f6c0 0x7f2288047860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:41:24.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:24.228+0000 7f228f400640 1 -- 172.21.15.17:0/3464964532 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f228804f6c0 msgr2=0x7f2288047860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:41:24.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:24.228+0000 7f228f400640 1 --2- 172.21.15.17:0/3464964532 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f228804f6c0 0x7f2288047860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:41:24.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:24.228+0000 7f228f400640 1 -- 172.21.15.17:0/3464964532 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2288039420 msgr2=0x7f228817c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:41:24.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:24.228+0000 7f228f400640 1 --2- 172.21.15.17:0/3464964532 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2288039420 0x7f228817c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:41:24.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:24.228+0000 7f228f400640 1 -- 172.21.15.17:0/3464964532 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2288047da0 con 0x7f228804faa0 2024-09-10T21:41:24.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:24.228+0000 7f228e000640 1 --2- 172.21.15.17:0/3464964532 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f228804f6c0 0x7f2288047860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:41:24.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:24.228+0000 7f228f400640 1 --2- 172.21.15.17:0/3464964532 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f228804faa0 0x7f22880da580 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2290066720 tx=0x7f229006a0e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:41:24.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:24.228+0000 7f2287400640 1 -- 172.21.15.17:0/3464964532 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2290090070 con 0x7f228804faa0 2024-09-10T21:41:24.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:24.228+0000 7f2287400640 1 -- 172.21.15.17:0/3464964532 <== mon.1 v2:172.21.15.31:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f22900943b0 con 0x7f228804faa0 2024-09-10T21:41:24.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:24.228+0000 7f2287400640 1 -- 172.21.15.17:0/3464964532 <== mon.1 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2290094800 con 0x7f228804faa0 2024-09-10T21:41:24.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:24.232+0000 7f228fe00640 1 -- 172.21.15.17:0/3464964532 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22880d9200 con 0x7f228804faa0 2024-09-10T21:41:24.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:24.236+0000 7f2287400640 1 -- 172.21.15.17:0/3464964532 <== mon.1 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f22900949a0 con 0x7f228804faa0 2024-09-10T21:41:24.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:24.236+0000 7f228fe00640 1 -- 172.21.15.17:0/3464964532 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22880d97c0 con 0x7f228804faa0 2024-09-10T21:41:24.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:24.236+0000 7f2287400640 1 --2- 172.21.15.17:0/3464964532 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f22700725e0 0x7f2270074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:41:24.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:24.236+0000 7f228fe00640 1 --2- 172.21.15.17:0/3464964532 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f22880d9a00 0x7f22880db7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:41:24.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:24.236+0000 7f228fe00640 1 -- 172.21.15.17:0/3464964532 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f22880df130 con 0x7f22880d9a00 2024-09-10T21:41:24.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:24.240+0000 7f228e000640 1 --2- 172.21.15.17:0/3464964532 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f22880d9a00 0x7f22880db7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:41:24.246 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:24.240+0000 7f228ea00640 1 --2- 172.21.15.17:0/3464964532 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f22700725e0 0x7f2270074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:41:24.246 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:24.240+0000 7f2287400640 1 -- 172.21.15.17:0/3464964532 <== mon.1 v2:172.21.15.31:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f229010c800 con 0x7f228804faa0 2024-09-10T21:41:24.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:24.240+0000 7f228e000640 1 -- 172.21.15.17:0/3464964532 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f22880db2f0 con 0x7f22880d9a00 2024-09-10T21:41:24.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:24.244+0000 7f228ea00640 1 --2- 172.21.15.17:0/3464964532 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f22700725e0 0x7f2270074aa0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f22800af000 tx=0x7f22800195c0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:41:24.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:24.244+0000 7f228e000640 1 --2- 172.21.15.17:0/3464964532 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f22880d9a00 0x7f22880db7c0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f228800dc00 tx=0x7f227c00ed20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:41:24.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:24.244+0000 7f2287400640 1 -- 172.21.15.17:0/3464964532 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f227c00cca0 con 0x7f22880d9a00 2024-09-10T21:41:24.251 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:24.248+0000 7f2287400640 1 -- 172.21.15.17:0/3464964532 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f227c009e30 con 0x7f22880d9a00 2024-09-10T21:41:24.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:24.248+0000 7f2287400640 1 -- 172.21.15.17:0/3464964532 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f22880d9a00 msgr2=0x7f22880db7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:41:24.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:24.248+0000 7f2287400640 1 --2- 172.21.15.17:0/3464964532 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f22880d9a00 0x7f22880db7c0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f228800dc00 tx=0x7f227c00ed20 comp rx=0 tx=0).stop 2024-09-10T21:41:24.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:24.248+0000 7f2287400640 1 -- 172.21.15.17:0/3464964532 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f22880d9a00 msgr2=0x7f22880db7c0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:41:24.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:24.248+0000 7f2287400640 1 --2- 172.21.15.17:0/3464964532 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f22880d9a00 0x7f22880db7c0 secure :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f228800dc00 tx=0x7f227c00ed20 comp rx=0 tx=0).stop 2024-09-10T21:41:24.266 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:24.260+0000 7f228fe00640 1 --2- 172.21.15.17:0/3464964532 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f22880d62c0 0x7f22880d66d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:41:24.266 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:24.260+0000 7f228fe00640 1 -- 172.21.15.17:0/3464964532 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f22880d6c10 con 0x7f22880d62c0 2024-09-10T21:41:24.266 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:24.260+0000 7f228e000640 1 --2- 172.21.15.17:0/3464964532 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f22880d62c0 0x7f22880d66d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:41:24.266 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:24.260+0000 7f228e000640 1 -- 172.21.15.17:0/3464964532 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f22880dc5f0 con 0x7f22880d62c0 2024-09-10T21:41:24.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:24.260+0000 7f228e000640 1 --2- 172.21.15.17:0/3464964532 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f22880d62c0 0x7f22880d66d0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f227c018000 tx=0x7f227c0046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:41:24.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:24.260+0000 7f2287400640 1 -- 172.21.15.17:0/3464964532 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f227c011470 con 0x7f22880d62c0 2024-09-10T21:41:24.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:24.260+0000 7f2287400640 1 -- 172.21.15.17:0/3464964532 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4543 (secure 0 0 0) 0x7f227c011610 con 0x7f22880d62c0 2024-09-10T21:41:24.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:24.260+0000 7f2287400640 1 -- 172.21.15.17:0/3464964532 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f22880d62c0 msgr2=0x7f22880d66d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:41:24.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:24.260+0000 7f2287400640 1 --2- 172.21.15.17:0/3464964532 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f22880d62c0 0x7f22880d66d0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f227c018000 tx=0x7f227c0046d0 comp rx=0 tx=0).stop 2024-09-10T21:41:24.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:24.260+0000 7f2287400640 1 -- 172.21.15.17:0/3464964532 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f22880d62c0 msgr2=0x7f22880d66d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:41:24.267 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:24.260+0000 7f2287400640 1 --2- 172.21.15.17:0/3464964532 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f22880d62c0 0x7f22880d66d0 secure :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f227c018000 tx=0x7f227c0046d0 comp rx=0 tx=0).stop 2024-09-10T21:41:24.272 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-10T21:41:24.272 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:41:24.272 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:41:24.272 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "leader", 2024-09-10T21:41:24.272 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 130, 2024-09-10T21:41:24.272 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 18943, 2024-09-10T21:41:24.273 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-10T21:41:24.273 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-10T21:41:24.273 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-10T21:41:24.273 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-10T21:41:24.273 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:41:24.273 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 1, 2024-09-10T21:41:24.273 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:41:24.273 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-10T21:41:24.273 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-10T21:41:24.273 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:41:24.273 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:41:24.273 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:41:24.273 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:41:24.274 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:41:24.274 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:41:24.274 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:41:24.274 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:41:24.274 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:41:24.274 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:41:24.274 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:41:24.274 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:41:24.274 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-10T21:41:24.274 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-10T21:41:24.274 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:41:24.274 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:41:24.274 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:41:24.274 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:41:24.275 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:41:24.275 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:41:24.275 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:41:24.275 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:41:24.275 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:41:24.275 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:41:24.275 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:41:24.275 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:41:24.275 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:41:24.275 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-10T21:41:24.275 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-10T21:41:24.275 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-10T21:41:24.275 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-10T21:41:24.276 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-10T21:41:24.276 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "5e2f023f-d6a6-49f4-a256-f4807dce3739", 2024-09-10T21:41:24.276 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:41:24.276 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:41:24.276 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-10T21:41:24.276 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-10T21:41:24.276 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-10T21:41:24.276 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-10T21:41:24.276 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-10T21:41:24.276 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-10T21:41:24.276 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-10T21:41:24.276 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:41:24.276 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-10T21:41:24.277 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:41:24.277 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:41:24.277 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:41:24.277 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:41:24.278 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:41:24.278 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:41:24.278 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:41:24.278 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:41:24.278 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:41:24.278 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:41:24.278 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:41:24.278 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:41:24.278 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-10T21:41:24.278 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:41:24.278 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-10T21:41:24.278 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:41:24.278 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:41:24.279 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:41:24.279 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:41:24.279 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:41:24.279 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:41:24.279 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:41:24.279 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-10T21:41:24.279 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:41:24.279 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:41:24.279 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:41:24.279 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:41:24.279 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-10T21:41:24.279 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:41:24.279 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:41:24.280 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:41:24.280 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:41:24.280 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-10T21:41:24.280 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-10T21:41:24.280 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:41:24.280 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:41:24.280 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:41:24.280 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:41:24.280 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:41:24.280 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:41:24.280 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:41:24.280 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:41:24.280 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:41:24.280 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:41:24.281 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:41:24.281 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3300", 2024-09-10T21:41:24.281 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:41:24.281 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:41:24.281 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:41:24.281 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:41:24.281 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789", 2024-09-10T21:41:24.281 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:41:24.281 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:41:24.281 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:41:24.281 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:41:24.281 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789/0", 2024-09-10T21:41:24.281 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6789/0", 2024-09-10T21:41:24.282 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:41:24.282 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:41:24.282 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:41:24.282 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:41:24.282 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:41:24.282 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:41:24.282 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:41:24.282 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:41:24.282 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:41:24.282 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:41:24.282 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:41:24.282 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3301", 2024-09-10T21:41:24.282 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:41:24.282 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:41:24.283 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:41:24.283 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:41:24.283 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790", 2024-09-10T21:41:24.283 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:41:24.283 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:41:24.283 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:41:24.283 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:41:24.283 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790/0", 2024-09-10T21:41:24.283 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6790/0", 2024-09-10T21:41:24.283 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:41:24.283 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:41:24.283 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:41:24.283 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:41:24.283 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:41:24.283 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:41:24.284 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-10T21:41:24.284 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-10T21:41:24.284 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:41:24.284 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:41:24.284 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:41:24.284 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:41:24.284 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:41:24.284 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:41:24.284 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-10T21:41:24.284 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:41:24.284 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:41:24.284 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:41:24.284 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:41:24.284 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:41:24.284 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:41:24.285 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:41:24.285 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-10T21:41:24.285 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-10T21:41:24.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:24.280+0000 7f228fe00640 1 -- 172.21.15.17:0/3464964532 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f22700725e0 msgr2=0x7f2270074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:41:24.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:24.280+0000 7f228fe00640 1 --2- 172.21.15.17:0/3464964532 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f22700725e0 0x7f2270074aa0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f22800af000 tx=0x7f22800195c0 comp rx=0 tx=0).stop 2024-09-10T21:41:24.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:24.280+0000 7f228fe00640 1 -- 172.21.15.17:0/3464964532 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f228804faa0 msgr2=0x7f22880da580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:41:24.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:24.280+0000 7f228fe00640 1 --2- 172.21.15.17:0/3464964532 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f228804faa0 0x7f22880da580 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2290066720 tx=0x7f229006a0e0 comp rx=0 tx=0).stop 2024-09-10T21:41:24.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:24.280+0000 7f228f400640 1 -- 172.21.15.17:0/3464964532 reap_dead start 2024-09-10T21:41:24.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:24.280+0000 7f228f400640 1 -- 172.21.15.17:0/3464964532 reap_dead start 2024-09-10T21:41:24.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:24.284+0000 7f228fe00640 1 -- 172.21.15.17:0/3464964532 shutdown_connections 2024-09-10T21:41:24.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:24.284+0000 7f228fe00640 1 -- 172.21.15.17:0/3464964532 >> 172.21.15.17:0/3464964532 conn(0x7f2288050a10 msgr2=0x7f228804c4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:41:24.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:24.292+0000 7f228fe00640 1 -- 172.21.15.17:0/3464964532 shutdown_connections 2024-09-10T21:41:24.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:24.292+0000 7f228fe00640 1 -- 172.21.15.17:0/3464964532 wait complete. 2024-09-10T21:41:24.346 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-10T21:41:24.346 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-09-10T21:41:24.346 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-10T21:41:24.764 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:24.756+0000 7f67dce00640 1 -- 172.21.15.17:0/365380210 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f67bc0052f0 msgr2=0x7f67bc00b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:41:24.764 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:24.756+0000 7f67dce00640 1 --2- 172.21.15.17:0/365380210 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f67bc0052f0 0x7f67bc00b9d0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f67d0009bf0 tx=0x7f67d001ca20 comp rx=0 tx=0).stop 2024-09-10T21:41:24.764 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:24.760+0000 7f67dce00640 1 -- 172.21.15.17:0/365380210 shutdown_connections 2024-09-10T21:41:24.764 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:24.760+0000 7f67dce00640 1 --2- 172.21.15.17:0/365380210 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f67bc0052f0 0x7f67bc00b9d0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:41:24.764 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:24.760+0000 7f67dce00640 1 -- 172.21.15.17:0/365380210 >> 172.21.15.17:0/365380210 conn(0x7f67d806f1d0 msgr2=0x7f67d806cc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:41:24.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:24.764+0000 7f67dce00640 1 -- 172.21.15.17:0/365380210 shutdown_connections 2024-09-10T21:41:24.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:24.764+0000 7f67dce00640 1 -- 172.21.15.17:0/365380210 wait complete. 2024-09-10T21:41:24.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:24.764+0000 7f67dce00640 1 Processor -- start 2024-09-10T21:41:24.773 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:24.768+0000 7f67dce00640 1 -- start start 2024-09-10T21:41:24.773 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:24.768+0000 7f67dce00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f67bc0052f0 0x7f67d8156ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:41:24.773 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:24.768+0000 7f67dce00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f67d815e470 0x7f67d81fcf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:41:24.773 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:24.768+0000 7f67dce00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67d81574a0 0x7f67d81fd440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:41:24.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:24.768+0000 7f67dce00640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f67d80ccca0 con 0x7f67bc0052f0 2024-09-10T21:41:24.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:24.768+0000 7f67dce00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f67d80cd2a0 con 0x7f67d81574a0 2024-09-10T21:41:24.774 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:24.768+0000 7f67dce00640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f67d80cce20 con 0x7f67d815e470 2024-09-10T21:41:24.777 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:24.768+0000 7f67d5400640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f67d815e470 0x7f67d81fcf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:41:24.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:24.768+0000 7f67d6800640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67d81574a0 0x7f67d81fd440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:41:24.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:24.772+0000 7f67d6800640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67d81574a0 0x7f67d81fd440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.17:51050/0 (socket says 172.21.15.17:51050) 2024-09-10T21:41:24.778 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:24.772+0000 7f67d6800640 1 -- 172.21.15.17:0/1170665457 learned_addr learned my addr 172.21.15.17:0/1170665457 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:41:24.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:24.776+0000 7f67d6800640 1 -- 172.21.15.17:0/1170665457 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f67d815e470 msgr2=0x7f67d81fcf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:41:24.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:24.776+0000 7f67d6800640 1 --2- 172.21.15.17:0/1170665457 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f67d815e470 0x7f67d81fcf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:41:24.784 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:24.776+0000 7f67d6800640 1 -- 172.21.15.17:0/1170665457 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f67bc0052f0 msgr2=0x7f67d8156ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:41:24.784 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:24.776+0000 7f67d6800640 1 --2- 172.21.15.17:0/1170665457 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f67bc0052f0 0x7f67d8156ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:41:24.784 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:24.776+0000 7f67d6800640 1 -- 172.21.15.17:0/1170665457 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67d81fca20 con 0x7f67d81574a0 2024-09-10T21:41:24.784 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:24.776+0000 7f67d6800640 1 --2- 172.21.15.17:0/1170665457 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67d81574a0 0x7f67d81fd440 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f67d001c850 tx=0x7f67d00098e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:41:24.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:24.784+0000 7f67ce000640 1 -- 172.21.15.17:0/1170665457 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67d00bd070 con 0x7f67d81574a0 2024-09-10T21:41:24.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:24.784+0000 7f67ce000640 1 -- 172.21.15.17:0/1170665457 <== mon.1 v2:172.21.15.31:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f67d00b67f0 con 0x7f67d81574a0 2024-09-10T21:41:24.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:24.784+0000 7f67ce000640 1 -- 172.21.15.17:0/1170665457 <== mon.1 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67d00b2040 con 0x7f67d81574a0 2024-09-10T21:41:24.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:24.788+0000 7f67dce00640 1 -- 172.21.15.17:0/1170665457 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67d8158ca0 con 0x7f67d81574a0 2024-09-10T21:41:24.795 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:24.788+0000 7f67dce00640 1 -- 172.21.15.17:0/1170665457 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67d8159260 con 0x7f67d81574a0 2024-09-10T21:41:24.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:24.792+0000 7f67dce00640 1 --2- 172.21.15.17:0/1170665457 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67d812d860 0x7f67d81fff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:41:24.798 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:24.792+0000 7f67dce00640 1 -- 172.21.15.17:0/1170665457 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f67d81fcc20 con 0x7f67d812d860 2024-09-10T21:41:24.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:24.792+0000 7f67d5e00640 1 --2- 172.21.15.17:0/1170665457 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67d812d860 0x7f67d81fff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:41:24.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:24.796+0000 7f67d5e00640 1 -- 172.21.15.17:0/1170665457 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f67d815aae0 con 0x7f67d812d860 2024-09-10T21:41:24.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:24.796+0000 7f67d5e00640 1 --2- 172.21.15.17:0/1170665457 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67d812d860 0x7f67d81fff00 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f67c800c6b0 tx=0x7f67c800cb70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:41:24.806 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:24.796+0000 7f67ce000640 1 -- 172.21.15.17:0/1170665457 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67c8017070 con 0x7f67d812d860 2024-09-10T21:41:24.806 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:24.796+0000 7f67ce000640 1 -- 172.21.15.17:0/1170665457 <== mon.1 v2:172.21.15.31:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f67c8005270 con 0x7f67d812d860 2024-09-10T21:41:24.806 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:24.796+0000 7f67ce000640 1 -- 172.21.15.17:0/1170665457 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67d812d860 msgr2=0x7f67d81fff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:41:24.806 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:24.796+0000 7f67ce000640 1 --2- 172.21.15.17:0/1170665457 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67d812d860 0x7f67d81fff00 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f67c800c6b0 tx=0x7f67c800cb70 comp rx=0 tx=0).stop 2024-09-10T21:41:24.806 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:24.796+0000 7f67ce000640 1 -- 172.21.15.17:0/1170665457 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67d812d860 msgr2=0x7f67d81fff00 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:41:24.806 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:24.796+0000 7f67ce000640 1 --2- 172.21.15.17:0/1170665457 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67d812d860 0x7f67d81fff00 secure :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f67c800c6b0 tx=0x7f67c800cb70 comp rx=0 tx=0).stop 2024-09-10T21:41:24.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:24.804+0000 7f67ce000640 1 -- 172.21.15.17:0/1170665457 <== mon.1 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f67d00b6bc0 con 0x7f67d81574a0 2024-09-10T21:41:24.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:24.808+0000 7f67ce000640 1 --2- 172.21.15.17:0/1170665457 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f67b8072140 0x7f67b8074600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:41:24.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:24.808+0000 7f67d5e00640 1 --2- 172.21.15.17:0/1170665457 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f67b8072140 0x7f67b8074600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:41:24.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:24.808+0000 7f67ce000640 1 -- 172.21.15.17:0/1170665457 <== mon.1 v2:172.21.15.31:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f67d00a59a0 con 0x7f67d81574a0 2024-09-10T21:41:24.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:24.808+0000 7f67d5e00640 1 --2- 172.21.15.17:0/1170665457 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f67b8072140 0x7f67b8074600 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f67c800c6b0 tx=0x7f67c8009030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:41:24.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:24.816+0000 7f67dce00640 1 --2- 172.21.15.17:0/1170665457 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67d8200440 0x7f67d8200850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:41:24.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:24.816+0000 7f67dce00640 1 -- 172.21.15.17:0/1170665457 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f67d8200d90 con 0x7f67d8200440 2024-09-10T21:41:24.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:24.820+0000 7f67d5400640 1 --2- 172.21.15.17:0/1170665457 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67d8200440 0x7f67d8200850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:41:24.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:24.820+0000 7f67d5400640 1 -- 172.21.15.17:0/1170665457 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f67d806d0c0 con 0x7f67d8200440 2024-09-10T21:41:24.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:24.828+0000 7f67d5400640 1 --2- 172.21.15.17:0/1170665457 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67d8200440 0x7f67d8200850 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f67c400f960 tx=0x7f67c400fe20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:41:24.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:24.836+0000 7f67ce000640 1 -- 172.21.15.17:0/1170665457 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67c400de60 con 0x7f67d8200440 2024-09-10T21:41:24.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:24.836+0000 7f67ce000640 1 -- 172.21.15.17:0/1170665457 <== mon.1 v2:172.21.15.31:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4875 (secure 0 0 0) 0x7f67c4005020 con 0x7f67d8200440 2024-09-10T21:41:24.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:24.836+0000 7f67ce000640 1 -- 172.21.15.17:0/1170665457 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67d8200440 msgr2=0x7f67d8200850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:41:24.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:24.836+0000 7f67ce000640 1 --2- 172.21.15.17:0/1170665457 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67d8200440 0x7f67d8200850 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f67c400f960 tx=0x7f67c400fe20 comp rx=0 tx=0).stop 2024-09-10T21:41:24.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:24.836+0000 7f67ce000640 1 -- 172.21.15.17:0/1170665457 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67d8200440 msgr2=0x7f67d8200850 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:41:24.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:24.836+0000 7f67ce000640 1 --2- 172.21.15.17:0/1170665457 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67d8200440 0x7f67d8200850 secure :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f67c400f960 tx=0x7f67c400fe20 comp rx=0 tx=0).stop 2024-09-10T21:41:24.847 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-10T21:41:24.847 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:41:24.847 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:41:24.847 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-10T21:41:24.847 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 130, 2024-09-10T21:41:24.847 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 96262, 2024-09-10T21:41:24.847 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-10T21:41:24.847 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-10T21:41:24.847 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-10T21:41:24.847 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-10T21:41:24.848 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:41:24.848 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 1, 2024-09-10T21:41:24.848 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:41:24.848 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-10T21:41:24.848 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-10T21:41:24.848 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:41:24.848 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:41:24.848 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:41:24.848 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:41:24.848 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:41:24.848 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:41:24.848 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:41:24.848 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:41:24.848 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:41:24.849 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:41:24.849 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:41:24.849 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:41:24.849 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-10T21:41:24.849 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-10T21:41:24.849 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:41:24.849 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:41:24.849 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:41:24.849 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:41:24.849 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:41:24.849 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:41:24.849 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:41:24.849 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:41:24.849 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:41:24.850 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:41:24.850 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:41:24.850 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:41:24.850 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:41:24.850 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-10T21:41:24.850 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-10T21:41:24.850 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-10T21:41:24.850 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-10T21:41:24.850 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-10T21:41:24.850 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "5e2f023f-d6a6-49f4-a256-f4807dce3739", 2024-09-10T21:41:24.850 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:41:24.851 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:41:24.851 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-10T21:41:24.851 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-10T21:41:24.851 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-10T21:41:24.851 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-10T21:41:24.851 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-10T21:41:24.851 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-10T21:41:24.851 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-10T21:41:24.851 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:41:24.851 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-10T21:41:24.851 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:41:24.851 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:41:24.852 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:41:24.852 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:41:24.852 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:41:24.852 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:41:24.852 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:41:24.852 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:41:24.852 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:41:24.853 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:41:24.853 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:41:24.853 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:41:24.853 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-10T21:41:24.853 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:41:24.853 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-10T21:41:24.853 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:41:24.853 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:41:24.853 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:41:24.853 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:41:24.853 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:41:24.853 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:41:24.853 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:41:24.854 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-10T21:41:24.854 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:41:24.854 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:41:24.854 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:41:24.854 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:41:24.854 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-10T21:41:24.854 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:41:24.854 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:41:24.854 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:41:24.854 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:41:24.854 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-10T21:41:24.854 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-10T21:41:24.854 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:41:24.854 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:41:24.855 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:41:24.855 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:41:24.855 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:41:24.855 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:41:24.855 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:41:24.855 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:41:24.855 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:41:24.855 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:41:24.855 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:41:24.855 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3300", 2024-09-10T21:41:24.855 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:41:24.855 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:41:24.855 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:41:24.855 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:41:24.856 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789", 2024-09-10T21:41:24.856 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:41:24.856 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:41:24.856 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:41:24.856 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:41:24.856 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789/0", 2024-09-10T21:41:24.856 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6789/0", 2024-09-10T21:41:24.856 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:41:24.856 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:41:24.856 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:41:24.856 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:41:24.856 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:41:24.856 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:41:24.856 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:41:24.857 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:41:24.857 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:41:24.857 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:41:24.857 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:41:24.857 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3301", 2024-09-10T21:41:24.857 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:41:24.857 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:41:24.857 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:41:24.857 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:41:24.857 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790", 2024-09-10T21:41:24.857 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:41:24.857 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:41:24.857 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:41:24.857 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:41:24.857 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790/0", 2024-09-10T21:41:24.858 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6790/0", 2024-09-10T21:41:24.858 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:41:24.858 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:41:24.858 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:41:24.858 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:41:24.858 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:41:24.858 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:41:24.858 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-10T21:41:24.858 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-10T21:41:24.858 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:41:24.858 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:41:24.858 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:41:24.858 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:41:24.859 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:41:24.859 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:41:24.859 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": [ 2024-09-10T21:41:24.859 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:41:24.859 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:41:24.859 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:41:24.859 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:41:24.859 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:41:24.859 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:41:24.859 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-10T21:41:24.859 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:41:24.859 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:41:24.859 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:41:24.860 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:41:24.860 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:41:24.860 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:41:24.860 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-10T21:41:24.860 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:41:24.860 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:41:24.860 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:41:24.860 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 3 2024-09-10T21:41:24.860 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:41:24.860 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:41:24.860 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:41:24.860 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-10T21:41:24.861 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-10T21:41:24.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:24.844+0000 7f67dce00640 1 -- 172.21.15.17:0/1170665457 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f67b8072140 msgr2=0x7f67b8074600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:41:24.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:24.844+0000 7f67dce00640 1 --2- 172.21.15.17:0/1170665457 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f67b8072140 0x7f67b8074600 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f67c800c6b0 tx=0x7f67c8009030 comp rx=0 tx=0).stop 2024-09-10T21:41:24.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:24.844+0000 7f67dce00640 1 -- 172.21.15.17:0/1170665457 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67d81574a0 msgr2=0x7f67d81fd440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:41:24.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:24.844+0000 7f67dce00640 1 --2- 172.21.15.17:0/1170665457 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f67d81574a0 0x7f67d81fd440 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f67d001c850 tx=0x7f67d00098e0 comp rx=0 tx=0).stop 2024-09-10T21:41:24.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:24.844+0000 7f67d6800640 1 -- 172.21.15.17:0/1170665457 reap_dead start 2024-09-10T21:41:24.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:24.844+0000 7f67d6800640 1 -- 172.21.15.17:0/1170665457 reap_dead start 2024-09-10T21:41:24.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:24.848+0000 7f67dce00640 1 -- 172.21.15.17:0/1170665457 shutdown_connections 2024-09-10T21:41:24.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:24.848+0000 7f67dce00640 1 -- 172.21.15.17:0/1170665457 >> 172.21.15.17:0/1170665457 conn(0x7f67d806f1d0 msgr2=0x7f67d8152770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:41:24.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:24.856+0000 7f67dce00640 1 -- 172.21.15.17:0/1170665457 shutdown_connections 2024-09-10T21:41:24.862 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:24.856+0000 7f67dce00640 1 -- 172.21.15.17:0/1170665457 wait complete. 2024-09-10T21:41:24.933 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-10T21:41:24.933 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-09-10T21:41:24.934 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-10T21:41:25.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:25.248+0000 7f289dc00640 1 -- 172.21.15.17:0/3864032552 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2898162270 msgr2=0x7f28981626a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:41:25.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:25.248+0000 7f289dc00640 1 --2- 172.21.15.17:0/3864032552 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2898162270 0x7f28981626a0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f28980cdbb0 tx=0x7f289001f5a0 comp rx=0 tx=0).stop 2024-09-10T21:41:25.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:25.248+0000 7f289dc00640 1 -- 172.21.15.17:0/3864032552 shutdown_connections 2024-09-10T21:41:25.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:25.248+0000 7f289dc00640 1 --2- 172.21.15.17:0/3864032552 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2898162270 0x7f28981626a0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:41:25.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:25.248+0000 7f289dc00640 1 --2- 172.21.15.17:0/3864032552 >> v2:172.21.15.31:3300/0 conn(0x7f2898158ba0 0x7f28980b4160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:41:25.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:25.248+0000 7f289dc00640 1 -- 172.21.15.17:0/3864032552 >> 172.21.15.17:0/3864032552 conn(0x7f289806e5a0 msgr2=0x7f2898068990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:41:25.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:25.256+0000 7f289dc00640 1 -- 172.21.15.17:0/3864032552 shutdown_connections 2024-09-10T21:41:25.266 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:25.256+0000 7f289dc00640 1 -- 172.21.15.17:0/3864032552 wait complete. 2024-09-10T21:41:25.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:25.280+0000 7f289dc00640 1 Processor -- start 2024-09-10T21:41:25.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:25.284+0000 7f289dc00640 1 -- start start 2024-09-10T21:41:25.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:25.284+0000 7f289dc00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f28980cafa0 0x7f2898150560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:41:25.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:25.284+0000 7f289dc00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2898158ba0 0x7f28981563d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:41:25.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:25.284+0000 7f289dc00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2898162270 0x7f289815b140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:41:25.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:25.284+0000 7f289dc00640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f28980c55b0 con 0x7f2898158ba0 2024-09-10T21:41:25.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:25.284+0000 7f289dc00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f28980c52d0 con 0x7f2898162270 2024-09-10T21:41:25.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:25.284+0000 7f289dc00640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f28980c5890 con 0x7f28980cafa0 2024-09-10T21:41:25.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:25.288+0000 7f2897e00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f28980cafa0 0x7f2898150560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:41:25.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:25.288+0000 7f2897e00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f28980cafa0 0x7f2898150560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3301/0 says I am v2:172.21.15.17:53590/0 (socket says 172.21.15.17:53590) 2024-09-10T21:41:25.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:25.288+0000 7f2897e00640 1 -- 172.21.15.17:0/1820168427 learned_addr learned my addr 172.21.15.17:0/1820168427 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:41:25.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:25.288+0000 7f2897400640 1 --2- 172.21.15.17:0/1820168427 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2898158ba0 0x7f28981563d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:41:25.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:25.292+0000 7f2897e00640 1 -- 172.21.15.17:0/1820168427 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2898162270 msgr2=0x7f289815b140 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-10T21:41:25.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:25.292+0000 7f2897e00640 1 --2- 172.21.15.17:0/1820168427 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2898162270 0x7f289815b140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:41:25.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:25.292+0000 7f2897e00640 1 -- 172.21.15.17:0/1820168427 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2898158ba0 msgr2=0x7f28981563d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:41:25.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:25.292+0000 7f2897e00640 1 --2- 172.21.15.17:0/1820168427 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2898158ba0 0x7f28981563d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:41:25.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:25.292+0000 7f2897e00640 1 -- 172.21.15.17:0/1820168427 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2898150aa0 con 0x7f28980cafa0 2024-09-10T21:41:25.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:25.292+0000 7f2897e00640 1 --2- 172.21.15.17:0/1820168427 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f28980cafa0 0x7f2898150560 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f288400c9a0 tx=0x7f288400ce60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:41:25.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:25.292+0000 7f288fe00640 1 -- 172.21.15.17:0/1820168427 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2884007c70 con 0x7f28980cafa0 2024-09-10T21:41:25.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:25.292+0000 7f288fe00640 1 -- 172.21.15.17:0/1820168427 <== mon.2 v2:172.21.15.31:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2884005020 con 0x7f28980cafa0 2024-09-10T21:41:25.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:25.292+0000 7f288fe00640 1 -- 172.21.15.17:0/1820168427 <== mon.2 v2:172.21.15.31:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2884005430 con 0x7f28980cafa0 2024-09-10T21:41:25.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:25.296+0000 7f289dc00640 1 -- 172.21.15.17:0/1820168427 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f289815fce0 con 0x7f28980cafa0 2024-09-10T21:41:25.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:25.296+0000 7f289dc00640 1 -- 172.21.15.17:0/1820168427 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f28981f86f0 con 0x7f28980cafa0 2024-09-10T21:41:25.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:25.300+0000 7f289dc00640 1 --2- 172.21.15.17:0/1820168427 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f2898156910 0x7f289815f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:41:25.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:25.300+0000 7f289dc00640 1 -- 172.21.15.17:0/1820168427 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f28981f91d0 con 0x7f2898156910 2024-09-10T21:41:25.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:25.300+0000 7f288fe00640 1 -- 172.21.15.17:0/1820168427 <== mon.2 v2:172.21.15.31:3301/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f28840047b0 con 0x7f28980cafa0 2024-09-10T21:41:25.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:25.304+0000 7f288fe00640 1 --2- 172.21.15.17:0/1820168427 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f2878072630 0x7f2878074af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:41:25.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:25.304+0000 7f288fe00640 1 -- 172.21.15.17:0/1820168427 <== mon.2 v2:172.21.15.31:3301/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f2884093a90 con 0x7f28980cafa0 2024-09-10T21:41:25.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:25.304+0000 7f2897400640 1 --2- 172.21.15.17:0/1820168427 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f2898156910 0x7f289815f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:41:25.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:25.304+0000 7f289d200640 1 --2- 172.21.15.17:0/1820168427 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f2878072630 0x7f2878074af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:41:25.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:25.304+0000 7f2897400640 1 -- 172.21.15.17:0/1820168427 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f28981590b0 con 0x7f2898156910 2024-09-10T21:41:25.311 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:25.304+0000 7f2897400640 1 --2- 172.21.15.17:0/1820168427 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f2898156910 0x7f289815f5d0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f2898158580 tx=0x7f288800d980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:41:25.316 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:25.312+0000 7f289d200640 1 --2- 172.21.15.17:0/1820168427 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f2878072630 0x7f2878074af0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f28900b2000 tx=0x7f289001dde0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:41:25.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:25.312+0000 7f288fe00640 1 -- 172.21.15.17:0/1820168427 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2888018020 con 0x7f2898156910 2024-09-10T21:41:25.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:25.312+0000 7f288fe00640 1 -- 172.21.15.17:0/1820168427 <== mon.2 v2:172.21.15.31:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f2888014620 con 0x7f2898156910 2024-09-10T21:41:25.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:25.312+0000 7f288fe00640 1 -- 172.21.15.17:0/1820168427 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f2898156910 msgr2=0x7f289815f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:41:25.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:25.312+0000 7f288fe00640 1 --2- 172.21.15.17:0/1820168427 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f2898156910 0x7f289815f5d0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f2898158580 tx=0x7f288800d980 comp rx=0 tx=0).stop 2024-09-10T21:41:25.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:25.312+0000 7f288fe00640 1 -- 172.21.15.17:0/1820168427 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f2898156910 msgr2=0x7f289815f5d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:41:25.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:25.312+0000 7f288fe00640 1 --2- 172.21.15.17:0/1820168427 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f2898156910 0x7f289815f5d0 secure :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f2898158580 tx=0x7f288800d980 comp rx=0 tx=0).stop 2024-09-10T21:41:25.328 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:25.324+0000 7f289dc00640 1 --2- 172.21.15.17:0/1820168427 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f289815efb0 0x7f28981f88f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:41:25.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:25.324+0000 7f289dc00640 1 -- 172.21.15.17:0/1820168427 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f289815a290 con 0x7f289815efb0 2024-09-10T21:41:25.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:25.324+0000 7f2897400640 1 --2- 172.21.15.17:0/1820168427 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f289815efb0 0x7f28981f88f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:41:25.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:25.324+0000 7f2897400640 1 -- 172.21.15.17:0/1820168427 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f28980cbb70 con 0x7f289815efb0 2024-09-10T21:41:25.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:25.324+0000 7f2897400640 1 --2- 172.21.15.17:0/1820168427 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f289815efb0 0x7f28981f88f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f2888016000 tx=0x7f2888004760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:41:25.341 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-10T21:41:25.341 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:41:25.341 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:41:25.341 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-10T21:41:25.341 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 130, 2024-09-10T21:41:25.342 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 291678, 2024-09-10T21:41:25.342 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-10T21:41:25.342 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-10T21:41:25.342 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-10T21:41:25.342 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-10T21:41:25.342 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:41:25.342 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 2, 2024-09-10T21:41:25.342 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:41:25.342 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-10T21:41:25.343 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-10T21:41:25.343 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:41:25.343 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:41:25.343 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:41:25.343 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:41:25.343 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:41:25.343 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:41:25.343 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:41:25.343 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:41:25.344 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:41:25.344 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:41:25.344 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:41:25.344 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:41:25.344 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-10T21:41:25.344 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-10T21:41:25.344 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:41:25.344 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:41:25.344 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:41:25.344 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:41:25.344 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:41:25.344 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:41:25.344 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:41:25.345 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:41:25.345 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:41:25.345 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:41:25.345 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:41:25.345 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:41:25.345 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:41:25.345 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-10T21:41:25.345 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-10T21:41:25.345 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-10T21:41:25.345 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-10T21:41:25.345 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-10T21:41:25.345 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "5e2f023f-d6a6-49f4-a256-f4807dce3739", 2024-09-10T21:41:25.345 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:41:25.346 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:41:25.346 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-10T21:41:25.346 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-10T21:41:25.346 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-10T21:41:25.346 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-10T21:41:25.346 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-10T21:41:25.346 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-10T21:41:25.346 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-10T21:41:25.346 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:41:25.346 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-10T21:41:25.346 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:41:25.346 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:41:25.346 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:41:25.346 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:41:25.347 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:41:25.347 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:41:25.347 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:41:25.347 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:41:25.347 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:41:25.347 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:41:25.347 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:41:25.347 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:41:25.347 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-10T21:41:25.347 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:41:25.347 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-10T21:41:25.347 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:41:25.347 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:41:25.348 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:41:25.348 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:41:25.348 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:41:25.348 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:41:25.348 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:41:25.348 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-10T21:41:25.348 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:41:25.348 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:41:25.348 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:41:25.348 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:41:25.348 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-10T21:41:25.348 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:41:25.348 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:41:25.348 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:41:25.349 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:41:25.349 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-10T21:41:25.349 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-10T21:41:25.349 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:41:25.349 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:41:25.349 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:41:25.349 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:41:25.349 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:41:25.349 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:41:25.349 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:41:25.349 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:41:25.349 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:41:25.349 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:41:25.349 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:41:25.350 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3300", 2024-09-10T21:41:25.350 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:41:25.350 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:41:25.350 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:41:25.350 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:41:25.350 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789", 2024-09-10T21:41:25.350 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:41:25.350 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:41:25.350 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:41:25.350 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:41:25.350 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789/0", 2024-09-10T21:41:25.350 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6789/0", 2024-09-10T21:41:25.350 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:41:25.351 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:41:25.351 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:41:25.351 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:41:25.351 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:41:25.351 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:41:25.351 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:41:25.351 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:41:25.351 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:41:25.351 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:41:25.351 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:41:25.351 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3301", 2024-09-10T21:41:25.351 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:41:25.351 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:41:25.351 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:41:25.352 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:41:25.352 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790", 2024-09-10T21:41:25.352 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:41:25.352 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:41:25.352 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:41:25.352 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:41:25.352 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790/0", 2024-09-10T21:41:25.352 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6790/0", 2024-09-10T21:41:25.352 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:41:25.352 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:41:25.352 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:41:25.352 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:41:25.352 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:41:25.352 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:41:25.353 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-10T21:41:25.353 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-10T21:41:25.353 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:41:25.353 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:41:25.353 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:41:25.353 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:41:25.353 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:41:25.353 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:41:25.353 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": [ 2024-09-10T21:41:25.353 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:41:25.353 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:41:25.353 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:41:25.353 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 5 2024-09-10T21:41:25.353 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:41:25.354 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:41:25.354 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-10T21:41:25.354 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:41:25.354 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:41:25.354 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:41:25.354 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 7 2024-09-10T21:41:25.354 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:41:25.354 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:41:25.354 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-10T21:41:25.354 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:41:25.354 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:41:25.354 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:41:25.354 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 6 2024-09-10T21:41:25.355 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:41:25.355 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:41:25.355 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": [ 2024-09-10T21:41:25.355 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:41:25.355 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:41:25.355 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:41:25.355 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 2 2024-09-10T21:41:25.355 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:41:25.355 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:41:25.355 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:41:25.355 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-10T21:41:25.355 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-10T21:41:25.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:25.332+0000 7f288fe00640 1 -- 172.21.15.17:0/1820168427 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2888026070 con 0x7f289815efb0 2024-09-10T21:41:25.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:25.332+0000 7f288fe00640 1 -- 172.21.15.17:0/1820168427 <== mon.2 v2:172.21.15.31:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5043 (secure 0 0 0) 0x7f2888012c10 con 0x7f289815efb0 2024-09-10T21:41:25.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:25.332+0000 7f288fe00640 1 -- 172.21.15.17:0/1820168427 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f289815efb0 msgr2=0x7f28981f88f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:41:25.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:25.332+0000 7f288fe00640 1 --2- 172.21.15.17:0/1820168427 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f289815efb0 0x7f28981f88f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f2888016000 tx=0x7f2888004760 comp rx=0 tx=0).stop 2024-09-10T21:41:25.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:25.332+0000 7f288fe00640 1 -- 172.21.15.17:0/1820168427 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f289815efb0 msgr2=0x7f28981f88f0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:41:25.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:25.332+0000 7f288fe00640 1 --2- 172.21.15.17:0/1820168427 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f289815efb0 0x7f28981f88f0 secure :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f2888016000 tx=0x7f2888004760 comp rx=0 tx=0).stop 2024-09-10T21:41:25.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:25.336+0000 7f289dc00640 1 -- 172.21.15.17:0/1820168427 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f2878072630 msgr2=0x7f2878074af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:41:25.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:25.336+0000 7f289dc00640 1 --2- 172.21.15.17:0/1820168427 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f2878072630 0x7f2878074af0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f28900b2000 tx=0x7f289001dde0 comp rx=0 tx=0).stop 2024-09-10T21:41:25.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:25.336+0000 7f289dc00640 1 -- 172.21.15.17:0/1820168427 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f28980cafa0 msgr2=0x7f2898150560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:41:25.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:25.336+0000 7f289dc00640 1 --2- 172.21.15.17:0/1820168427 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f28980cafa0 0x7f2898150560 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f288400c9a0 tx=0x7f288400ce60 comp rx=0 tx=0).stop 2024-09-10T21:41:25.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:25.336+0000 7f289d200640 1 -- 172.21.15.17:0/1820168427 reap_dead start 2024-09-10T21:41:25.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:25.336+0000 7f289d200640 1 -- 172.21.15.17:0/1820168427 reap_dead start 2024-09-10T21:41:25.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:25.344+0000 7f289dc00640 1 -- 172.21.15.17:0/1820168427 shutdown_connections 2024-09-10T21:41:25.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:25.344+0000 7f289dc00640 1 -- 172.21.15.17:0/1820168427 >> 172.21.15.17:0/1820168427 conn(0x7f289806e5a0 msgr2=0x7f2898068990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:41:25.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:25.344+0000 7f289dc00640 1 -- 172.21.15.17:0/1820168427 shutdown_connections 2024-09-10T21:41:25.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:25.344+0000 7f289dc00640 1 -- 172.21.15.17:0/1820168427 wait complete. 2024-09-10T21:41:25.439 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-10T21:41:25.439 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-09-10T21:41:25.439 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-09-10T21:41:25.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:25.816+0000 7f3b96a00640 1 -- 172.21.15.17:0/3498129840 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3b88035200 msgr2=0x7f3b880db030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:41:25.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:25.816+0000 7f3b96a00640 1 --2- 172.21.15.17:0/3498129840 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3b88035200 0x7f3b880db030 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f3b84009880 tx=0x7f3b8401ef70 comp rx=0 tx=0).stop 2024-09-10T21:41:25.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:25.820+0000 7f3b96a00640 1 -- 172.21.15.17:0/3498129840 shutdown_connections 2024-09-10T21:41:25.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:25.820+0000 7f3b96a00640 1 --2- 172.21.15.17:0/3498129840 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3b88035200 0x7f3b880db030 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:41:25.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:25.820+0000 7f3b96a00640 1 -- 172.21.15.17:0/3498129840 >> 172.21.15.17:0/3498129840 conn(0x7f3b8804fc10 msgr2=0x7f3b88050020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:41:25.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:25.820+0000 7f3b96a00640 1 -- 172.21.15.17:0/3498129840 shutdown_connections 2024-09-10T21:41:25.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:25.820+0000 7f3b96a00640 1 -- 172.21.15.17:0/3498129840 wait complete. 2024-09-10T21:41:25.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:25.824+0000 7f3b96a00640 1 Processor -- start 2024-09-10T21:41:25.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:25.824+0000 7f3b96a00640 1 -- start start 2024-09-10T21:41:25.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:25.824+0000 7f3b96a00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3b88035200 0x7f3b880c5b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:41:25.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:25.824+0000 7f3b96a00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3b880355e0 0x7f3b880c32c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:41:25.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:25.824+0000 7f3b96a00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3b8804b460 0x7f3b880c3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:41:25.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:25.824+0000 7f3b96a00640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f3b880456c0 con 0x7f3b880355e0 2024-09-10T21:41:25.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:25.824+0000 7f3b96a00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f3b880453e0 con 0x7f3b8804b460 2024-09-10T21:41:25.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:25.824+0000 7f3b96a00640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f3b8804afc0 con 0x7f3b88035200 2024-09-10T21:41:25.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:25.828+0000 7f3b96000640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3b8804b460 0x7f3b880c3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:41:25.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:25.828+0000 7f3b94c00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3b880355e0 0x7f3b880c32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:41:25.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:25.828+0000 7f3b94c00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3b880355e0 0x7f3b880c32c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:38660/0 (socket says 172.21.15.17:38660) 2024-09-10T21:41:25.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:25.828+0000 7f3b94c00640 1 -- 172.21.15.17:0/1693113601 learned_addr learned my addr 172.21.15.17:0/1693113601 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:41:25.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:25.828+0000 7f3b95600640 1 --2- 172.21.15.17:0/1693113601 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3b88035200 0x7f3b880c5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:41:25.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:25.828+0000 7f3b94c00640 1 -- 172.21.15.17:0/1693113601 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3b88035200 msgr2=0x7f3b880c5b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:41:25.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:25.828+0000 7f3b94c00640 1 --2- 172.21.15.17:0/1693113601 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3b88035200 0x7f3b880c5b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:41:25.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:25.828+0000 7f3b94c00640 1 -- 172.21.15.17:0/1693113601 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3b8804b460 msgr2=0x7f3b880c3800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:41:25.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:25.828+0000 7f3b94c00640 1 --2- 172.21.15.17:0/1693113601 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3b8804b460 0x7f3b880c3800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:41:25.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:25.828+0000 7f3b94c00640 1 -- 172.21.15.17:0/1693113601 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b880d03e0 con 0x7f3b880355e0 2024-09-10T21:41:25.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:25.828+0000 7f3b96000640 1 --2- 172.21.15.17:0/1693113601 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3b8804b460 0x7f3b880c3800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:41:25.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:25.828+0000 7f3b94c00640 1 --2- 172.21.15.17:0/1693113601 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3b880355e0 0x7f3b880c32c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f3b7c00c7a0 tx=0x7f3b7c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:41:25.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:25.832+0000 7f3b8d600640 1 -- 172.21.15.17:0/1693113601 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b7c016020 con 0x7f3b880355e0 2024-09-10T21:41:25.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:25.832+0000 7f3b8d600640 1 -- 172.21.15.17:0/1693113601 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3b7c00ab60 con 0x7f3b880355e0 2024-09-10T21:41:25.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:25.832+0000 7f3b8d600640 1 -- 172.21.15.17:0/1693113601 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b7c012420 con 0x7f3b880355e0 2024-09-10T21:41:25.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:25.832+0000 7f3b96a00640 1 -- 172.21.15.17:0/1693113601 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b8804c1e0 con 0x7f3b880355e0 2024-09-10T21:41:25.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:25.832+0000 7f3b96a00640 1 -- 172.21.15.17:0/1693113601 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b8817bea0 con 0x7f3b880355e0 2024-09-10T21:41:25.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:25.836+0000 7f3b96a00640 1 -- 172.21.15.17:0/1693113601 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b88034990 con 0x7f3b880355e0 2024-09-10T21:41:25.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:25.840+0000 7f3b8d600640 1 -- 172.21.15.17:0/1693113601 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f3b7c01b410 con 0x7f3b880355e0 2024-09-10T21:41:25.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:25.840+0000 7f3b8d600640 1 --2- 172.21.15.17:0/1693113601 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f3b70072630 0x7f3b70074af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:41:25.853 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:25.844+0000 7f3b95600640 1 --2- 172.21.15.17:0/1693113601 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f3b70072630 0x7f3b70074af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:41:25.853 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:25.848+0000 7f3b95600640 1 --2- 172.21.15.17:0/1693113601 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f3b70072630 0x7f3b70074af0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f3b840b2000 tx=0x7f3b8401bd10 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:41:25.856 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:25.852+0000 7f3b8d600640 1 -- 172.21.15.17:0/1693113601 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f3b7c091880 con 0x7f3b880355e0 2024-09-10T21:41:25.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:25.864+0000 7f3b8d600640 1 -- 172.21.15.17:0/1693113601 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f3b7c020070 con 0x7f3b880355e0 2024-09-10T21:41:26.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:26.436+0000 7f3b96a00640 1 -- 172.21.15.17:0/1693113601 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) -- 0x7f3b880346b0 con 0x7f3b880355e0 2024-09-10T21:41:26.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:26.452+0000 7f3b8d600640 1 -- 172.21.15.17:0/1693113601 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) ==== 57+0+0 (secure 0 0 0) 0x7f3b7c05e920 con 0x7f3b880355e0 2024-09-10T21:41:26.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:26.472+0000 7f3b96a00640 1 -- 172.21.15.17:0/1693113601 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f3b70072630 msgr2=0x7f3b70074af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:41:26.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:26.472+0000 7f3b96a00640 1 --2- 172.21.15.17:0/1693113601 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f3b70072630 0x7f3b70074af0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f3b840b2000 tx=0x7f3b8401bd10 comp rx=0 tx=0).stop 2024-09-10T21:41:26.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:26.472+0000 7f3b96a00640 1 -- 172.21.15.17:0/1693113601 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3b880355e0 msgr2=0x7f3b880c32c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:41:26.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:26.472+0000 7f3b96a00640 1 --2- 172.21.15.17:0/1693113601 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3b880355e0 0x7f3b880c32c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f3b7c00c7a0 tx=0x7f3b7c00cc60 comp rx=0 tx=0).stop 2024-09-10T21:41:26.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:26.472+0000 7f3b96a00640 1 -- 172.21.15.17:0/1693113601 shutdown_connections 2024-09-10T21:41:26.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:26.472+0000 7f3b96a00640 1 --2- 172.21.15.17:0/1693113601 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f3b70072630 0x7f3b70074af0 secure :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f3b840b2000 tx=0x7f3b8401bd10 comp rx=0 tx=0).stop 2024-09-10T21:41:26.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:26.472+0000 7f3b96a00640 1 --2- 172.21.15.17:0/1693113601 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3b8804b460 0x7f3b880c3800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:41:26.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:26.472+0000 7f3b96a00640 1 --2- 172.21.15.17:0/1693113601 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3b880355e0 0x7f3b880c32c0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:41:26.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:26.472+0000 7f3b96a00640 1 --2- 172.21.15.17:0/1693113601 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3b88035200 0x7f3b880c5b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:41:26.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:26.472+0000 7f3b96a00640 1 -- 172.21.15.17:0/1693113601 >> 172.21.15.17:0/1693113601 conn(0x7f3b8804fc10 msgr2=0x7f3b8804c740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:41:26.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:26.472+0000 7f3b96a00640 1 -- 172.21.15.17:0/1693113601 shutdown_connections 2024-09-10T21:41:26.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:26.472+0000 7f3b96a00640 1 -- 172.21.15.17:0/1693113601 wait complete. 2024-09-10T21:41:26.657 INFO:tasks.mon_thrash.mon_thrasher:waiting for 10.0 secs before continuing thrashing 2024-09-10T21:41:36.658 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-09-10T21:41:36.658 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-10T21:41:37.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:36.992+0000 7fdeaa200640 1 -- 172.21.15.17:0/1351188518 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdea406e650 msgr2=0x7fdea40c81b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:41:37.000 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:36.996+0000 7fdeaa200640 1 --2- 172.21.15.17:0/1351188518 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdea406e650 0x7fdea40c81b0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fdea4071110 tx=0x7fde9c01f390 comp rx=0 tx=0).stop 2024-09-10T21:41:37.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:36.996+0000 7fdeaa200640 1 -- 172.21.15.17:0/1351188518 shutdown_connections 2024-09-10T21:41:37.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:36.996+0000 7fdeaa200640 1 --2- 172.21.15.17:0/1351188518 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdea406e650 0x7fdea40c81b0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:41:37.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:36.996+0000 7fdeaa200640 1 --2- 172.21.15.17:0/1351188518 >> v2:172.21.15.31:3300/0 conn(0x7fdea40c8ab0 0x7fdea40c8e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:41:37.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:36.996+0000 7fdeaa200640 1 -- 172.21.15.17:0/1351188518 >> 172.21.15.17:0/1351188518 conn(0x7fdea4070160 msgr2=0x7fdea4070570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:41:37.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:36.996+0000 7fdeaa200640 1 -- 172.21.15.17:0/1351188518 shutdown_connections 2024-09-10T21:41:37.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:36.996+0000 7fdeaa200640 1 -- 172.21.15.17:0/1351188518 wait complete. 2024-09-10T21:41:37.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:37.000+0000 7fdeaa200640 1 Processor -- start 2024-09-10T21:41:37.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:37.000+0000 7fdeaa200640 1 -- start start 2024-09-10T21:41:37.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:37.004+0000 7fdeaa200640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdea406e650 0x7fdea40c3de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:41:37.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:37.004+0000 7fdeaa200640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdea40c1040 0x7fdea414e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:41:37.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:37.004+0000 7fdeaa200640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fdea40c8ab0 0x7fdea40c4ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:41:37.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:37.004+0000 7fdeaa200640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fdea406eac0 con 0x7fdea40c1040 2024-09-10T21:41:37.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:37.004+0000 7fdeaa200640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fdea406de50 con 0x7fdea406e650 2024-09-10T21:41:37.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:37.004+0000 7fdeaa200640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7fdea40c86f0 con 0x7fdea40c8ab0 2024-09-10T21:41:37.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:37.004+0000 7fdea3400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdea406e650 0x7fdea40c3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:41:37.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:37.004+0000 7fdea3e00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fdea40c8ab0 0x7fdea40c4ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:41:37.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:37.004+0000 7fdea2a00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdea40c1040 0x7fdea414e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:41:37.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:37.004+0000 7fdea2a00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdea40c1040 0x7fdea414e0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:53668/0 (socket says 172.21.15.17:53668) 2024-09-10T21:41:37.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:37.004+0000 7fdea2a00640 1 -- 172.21.15.17:0/4152301874 learned_addr learned my addr 172.21.15.17:0/4152301874 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:41:37.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:37.004+0000 7fdea3e00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fdea40c8ab0 0x7fdea40c4ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3301/0 says I am v2:172.21.15.17:39710/0 (socket says 172.21.15.17:39710) 2024-09-10T21:41:37.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:37.004+0000 7fdea2a00640 1 -- 172.21.15.17:0/4152301874 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fdea40c8ab0 msgr2=0x7fdea40c4ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:41:37.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:37.004+0000 7fdea2a00640 1 --2- 172.21.15.17:0/4152301874 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fdea40c8ab0 0x7fdea40c4ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:41:37.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:37.004+0000 7fdea2a00640 1 -- 172.21.15.17:0/4152301874 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdea406e650 msgr2=0x7fdea40c3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:41:37.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:37.004+0000 7fdea2a00640 1 --2- 172.21.15.17:0/4152301874 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdea406e650 0x7fdea40c3de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:41:37.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:37.004+0000 7fdea2a00640 1 -- 172.21.15.17:0/4152301874 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdea414e990 con 0x7fdea40c1040 2024-09-10T21:41:37.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:37.004+0000 7fdea2a00640 1 --2- 172.21.15.17:0/4152301874 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdea40c1040 0x7fdea414e0b0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fde9400c4b0 tx=0x7fde9400c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:41:37.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:37.008+0000 7fde9b400640 1 -- 172.21.15.17:0/4152301874 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde94016020 con 0x7fdea40c1040 2024-09-10T21:41:37.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:37.008+0000 7fde9b400640 1 -- 172.21.15.17:0/4152301874 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fde94012920 con 0x7fdea40c1040 2024-09-10T21:41:37.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:37.008+0000 7fde9b400640 1 -- 172.21.15.17:0/4152301874 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde94012d70 con 0x7fdea40c1040 2024-09-10T21:41:37.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:37.016+0000 7fdeaa200640 1 -- 172.21.15.17:0/4152301874 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdea414e5f0 con 0x7fdea40c1040 2024-09-10T21:41:37.021 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:37.016+0000 7fdeaa200640 1 -- 172.21.15.17:0/4152301874 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdea40ca7c0 con 0x7fdea40c1040 2024-09-10T21:41:37.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:37.024+0000 7fdeaa200640 1 -- 172.21.15.17:0/4152301874 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdea40c79f0 con 0x7fdea40c1040 2024-09-10T21:41:37.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:37.052+0000 7fde9b400640 1 -- 172.21.15.17:0/4152301874 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fde94023060 con 0x7fdea40c1040 2024-09-10T21:41:37.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:37.052+0000 7fde9b400640 1 --2- 172.21.15.17:0/4152301874 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fde840726b0 0x7fde84074b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:41:37.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:37.052+0000 7fde9b400640 1 -- 172.21.15.17:0/4152301874 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fde94093c10 con 0x7fdea40c1040 2024-09-10T21:41:37.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:37.068+0000 7fdea3400640 1 --2- 172.21.15.17:0/4152301874 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fde840726b0 0x7fde84074b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:41:37.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:37.068+0000 7fdea3400640 1 --2- 172.21.15.17:0/4152301874 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fde840726b0 0x7fde84074b70 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fde900058d0 tx=0x7fde90005860 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:41:37.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:37.136+0000 7fde9b400640 1 -- 172.21.15.17:0/4152301874 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fde9405ef60 con 0x7fdea40c1040 2024-09-10T21:41:37.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:37.820+0000 7fdeaa200640 1 -- 172.21.15.17:0/4152301874 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fdea4154940 con 0x7fdea40c1040 2024-09-10T21:41:37.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:37.824+0000 7fde9b400640 1 -- 172.21.15.17:0/4152301874 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1558 (secure 0 0 0) 0x7fde9405e860 con 0x7fdea40c1040 2024-09-10T21:41:37.829 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":130,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":14,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5e2f023f-d6a6-49f4-a256-f4807dce3739","modified":"2024-09-10T21:23:46.892130Z","created":"2024-09-10T21:23:46.892130Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3301","nonce":0},{"type":"v1","addr":"172.21.15.31:6790","nonce":0}]},"addr":"172.21.15.31:6790/0","public_addr":"172.21.15.31:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-10T21:41:37.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:37.828+0000 7fdeaa200640 1 -- 172.21.15.17:0/4152301874 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fde840726b0 msgr2=0x7fde84074b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:41:37.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:37.828+0000 7fdeaa200640 1 --2- 172.21.15.17:0/4152301874 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fde840726b0 0x7fde84074b70 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fde900058d0 tx=0x7fde90005860 comp rx=0 tx=0).stop 2024-09-10T21:41:37.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:37.828+0000 7fdeaa200640 1 -- 172.21.15.17:0/4152301874 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdea40c1040 msgr2=0x7fdea414e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:41:37.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:37.828+0000 7fdeaa200640 1 --2- 172.21.15.17:0/4152301874 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdea40c1040 0x7fdea414e0b0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fde9400c4b0 tx=0x7fde9400c970 comp rx=0 tx=0).stop 2024-09-10T21:41:37.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:37.832+0000 7fdeaa200640 1 -- 172.21.15.17:0/4152301874 shutdown_connections 2024-09-10T21:41:37.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:37.832+0000 7fdeaa200640 1 --2- 172.21.15.17:0/4152301874 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fde840726b0 0x7fde84074b70 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:41:37.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:37.832+0000 7fdeaa200640 1 --2- 172.21.15.17:0/4152301874 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fdea40c8ab0 0x7fdea40c4ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:41:37.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:37.832+0000 7fdeaa200640 1 --2- 172.21.15.17:0/4152301874 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdea40c1040 0x7fdea414e0b0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:41:37.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:37.832+0000 7fdeaa200640 1 --2- 172.21.15.17:0/4152301874 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdea406e650 0x7fdea40c3de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:41:37.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:37.832+0000 7fdeaa200640 1 -- 172.21.15.17:0/4152301874 >> 172.21.15.17:0/4152301874 conn(0x7fdea4070160 msgr2=0x7fdea415a5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:41:37.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:37.844+0000 7fdeaa200640 1 -- 172.21.15.17:0/4152301874 shutdown_connections 2024-09-10T21:41:37.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:37.844+0000 7fdeaa200640 1 -- 172.21.15.17:0/4152301874 wait complete. 2024-09-10T21:41:37.946 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-09-10T21:41:37.947 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-09-10T21:41:37.947 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-09-10T21:41:37.947 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-10T21:41:38.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:38.344+0000 7ffbcf200640 1 -- 172.21.15.17:0/1984469386 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ffbd006e3a0 msgr2=0x7ffbd006e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:41:38.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:38.344+0000 7ffbcf200640 1 --2- 172.21.15.17:0/1984469386 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ffbd006e3a0 0x7ffbd006e780 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7ffbc00097f0 tx=0x7ffbc001ed10 comp rx=0 tx=0).stop 2024-09-10T21:41:38.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:38.344+0000 7ffbcf200640 1 -- 172.21.15.17:0/1984469386 shutdown_connections 2024-09-10T21:41:38.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:38.344+0000 7ffbcf200640 1 --2- 172.21.15.17:0/1984469386 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ffbd006e3a0 0x7ffbd006e780 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:41:38.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:38.344+0000 7ffbcf200640 1 -- 172.21.15.17:0/1984469386 >> 172.21.15.17:0/1984469386 conn(0x7ffbd006fe60 msgr2=0x7ffbd0070270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:41:38.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:38.344+0000 7ffbcf200640 1 -- 172.21.15.17:0/1984469386 shutdown_connections 2024-09-10T21:41:38.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:38.344+0000 7ffbcf200640 1 -- 172.21.15.17:0/1984469386 wait complete. 2024-09-10T21:41:38.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:38.348+0000 7ffbcf200640 1 Processor -- start 2024-09-10T21:41:38.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:38.348+0000 7ffbcf200640 1 -- start start 2024-09-10T21:41:38.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:38.348+0000 7ffbcf200640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ffbd006e3a0 0x7ffbd00c36b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:41:38.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:38.348+0000 7ffbcf200640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffbd0070980 0x7ffbd00740a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:41:38.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:38.348+0000 7ffbcf200640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffbd00c7810 0x7ffbd0156b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:41:38.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:38.348+0000 7ffbcf200640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7ffbd015afe0 con 0x7ffbd00c7810 2024-09-10T21:41:38.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:38.348+0000 7ffbcf200640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7ffbd00c6900 con 0x7ffbd0070980 2024-09-10T21:41:38.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:38.348+0000 7ffbcf200640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7ffbd00cb890 con 0x7ffbd006e3a0 2024-09-10T21:41:38.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:38.348+0000 7ffbce800640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffbd00c7810 0x7ffbd0156b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:41:38.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:38.348+0000 7ffbce800640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffbd00c7810 0x7ffbd0156b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:53674/0 (socket says 172.21.15.17:53674) 2024-09-10T21:41:38.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:38.348+0000 7ffbce800640 1 -- 172.21.15.17:0/3136678895 learned_addr learned my addr 172.21.15.17:0/3136678895 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:41:38.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:38.352+0000 7ffbcd400640 1 --2- 172.21.15.17:0/3136678895 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffbd0070980 0x7ffbd00740a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:41:38.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:38.352+0000 7ffbcde00640 1 --2- 172.21.15.17:0/3136678895 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ffbd006e3a0 0x7ffbd00c36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:41:38.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:38.352+0000 7ffbcd400640 1 -- 172.21.15.17:0/3136678895 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ffbd006e3a0 msgr2=0x7ffbd00c36b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:41:38.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:38.352+0000 7ffbcd400640 1 --2- 172.21.15.17:0/3136678895 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ffbd006e3a0 0x7ffbd00c36b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:41:38.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:38.352+0000 7ffbcd400640 1 -- 172.21.15.17:0/3136678895 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffbd00c7810 msgr2=0x7ffbd0156b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:41:38.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:38.352+0000 7ffbcd400640 1 --2- 172.21.15.17:0/3136678895 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffbd00c7810 0x7ffbd0156b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:41:38.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:38.352+0000 7ffbcd400640 1 -- 172.21.15.17:0/3136678895 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbd0152110 con 0x7ffbd0070980 2024-09-10T21:41:38.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:38.352+0000 7ffbce800640 1 --2- 172.21.15.17:0/3136678895 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffbd00c7810 0x7ffbd0156b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:41:38.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:38.352+0000 7ffbcd400640 1 --2- 172.21.15.17:0/3136678895 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffbd0070980 0x7ffbd00740a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ffbbc00c410 tx=0x7ffbbc00c8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:41:38.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:38.356+0000 7ffbc6000640 1 -- 172.21.15.17:0/3136678895 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbbc011790 con 0x7ffbd0070980 2024-09-10T21:41:38.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:38.356+0000 7ffbc6000640 1 -- 172.21.15.17:0/3136678895 <== mon.1 v2:172.21.15.31:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ffbbc017070 con 0x7ffbd0070980 2024-09-10T21:41:38.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:38.356+0000 7ffbc6000640 1 -- 172.21.15.17:0/3136678895 <== mon.1 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbbc011cc0 con 0x7ffbd0070980 2024-09-10T21:41:38.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:38.356+0000 7ffbcf200640 1 -- 172.21.15.17:0/3136678895 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffbd00c8520 con 0x7ffbd0070980 2024-09-10T21:41:38.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:38.356+0000 7ffbcf200640 1 -- 172.21.15.17:0/3136678895 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffbd0156330 con 0x7ffbd0070980 2024-09-10T21:41:38.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:38.356+0000 7ffbcf200640 1 --2- 172.21.15.17:0/3136678895 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffbd0156530 0x7ffbd0157070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:41:38.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:38.356+0000 7ffbcf200640 1 -- 172.21.15.17:0/3136678895 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7ffbd015b160 con 0x7ffbd0156530 2024-09-10T21:41:38.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:38.360+0000 7ffbc6000640 1 -- 172.21.15.17:0/3136678895 <== mon.1 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7ffbbc01c460 con 0x7ffbd0070980 2024-09-10T21:41:38.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:38.360+0000 7ffbc6000640 1 --2- 172.21.15.17:0/3136678895 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7ffbb00726b0 0x7ffbb0074b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:41:38.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:38.360+0000 7ffbc6000640 1 -- 172.21.15.17:0/3136678895 <== mon.1 v2:172.21.15.31:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7ffbbc093340 con 0x7ffbd0070980 2024-09-10T21:41:38.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:38.360+0000 7ffbce800640 1 --2- 172.21.15.17:0/3136678895 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7ffbb00726b0 0x7ffbb0074b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:41:38.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:38.364+0000 7ffbce800640 1 --2- 172.21.15.17:0/3136678895 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7ffbb00726b0 0x7ffbb0074b70 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7ffbc8001e10 tx=0x7ffbc8001da0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:41:38.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:38.368+0000 7ffbcde00640 1 --2- 172.21.15.17:0/3136678895 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffbd0156530 0x7ffbd0157070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:41:38.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:38.368+0000 7ffbcde00640 1 -- 172.21.15.17:0/3136678895 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ffbd01547a0 con 0x7ffbd0156530 2024-09-10T21:41:38.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:38.368+0000 7ffbcde00640 1 --2- 172.21.15.17:0/3136678895 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffbd0156530 0x7ffbd0157070 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7ffbd0076320 tx=0x7ffbc00aafb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:41:38.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:38.368+0000 7ffbc6000640 1 -- 172.21.15.17:0/3136678895 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbc00be020 con 0x7ffbd0156530 2024-09-10T21:41:38.376 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:38.372+0000 7ffbc6000640 1 -- 172.21.15.17:0/3136678895 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7ffbc0004250 con 0x7ffbd0156530 2024-09-10T21:41:38.376 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:38.372+0000 7ffbc6000640 1 -- 172.21.15.17:0/3136678895 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffbd0156530 msgr2=0x7ffbd0157070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:41:38.376 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:38.372+0000 7ffbc6000640 1 --2- 172.21.15.17:0/3136678895 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffbd0156530 0x7ffbd0157070 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7ffbd0076320 tx=0x7ffbc00aafb0 comp rx=0 tx=0).stop 2024-09-10T21:41:38.376 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:38.372+0000 7ffbc6000640 1 -- 172.21.15.17:0/3136678895 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffbd0156530 msgr2=0x7ffbd0157070 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:41:38.376 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:38.372+0000 7ffbc6000640 1 --2- 172.21.15.17:0/3136678895 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffbd0156530 0x7ffbd0157070 secure :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7ffbd0076320 tx=0x7ffbc00aafb0 comp rx=0 tx=0).stop 2024-09-10T21:41:38.411 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:38.404+0000 7ffbcf200640 1 --2- 172.21.15.17:0/3136678895 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffbd014bf20 0x7ffbd014c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:41:38.411 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:38.404+0000 7ffbcf200640 1 -- 172.21.15.17:0/3136678895 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7ffbd0156910 con 0x7ffbd014bf20 2024-09-10T21:41:38.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:38.412+0000 7ffbcde00640 1 --2- 172.21.15.17:0/3136678895 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffbd014bf20 0x7ffbd014c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:41:38.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:38.412+0000 7ffbcde00640 1 -- 172.21.15.17:0/3136678895 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7ffbd012d5d0 con 0x7ffbd014bf20 2024-09-10T21:41:38.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:38.416+0000 7ffbcde00640 1 --2- 172.21.15.17:0/3136678895 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffbd014bf20 0x7ffbd014c330 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7ffbc00097f0 tx=0x7ffbc00091e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:41:38.437 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-10T21:41:38.437 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:41:38.437 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:41:38.437 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "leader", 2024-09-10T21:41:38.438 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 130, 2024-09-10T21:41:38.438 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 33103, 2024-09-10T21:41:38.438 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-10T21:41:38.438 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-10T21:41:38.438 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-10T21:41:38.438 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-10T21:41:38.438 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:41:38.438 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 15, 2024-09-10T21:41:38.438 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:41:38.438 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-10T21:41:38.438 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-10T21:41:38.438 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:41:38.438 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:41:38.439 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:41:38.439 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:41:38.439 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:41:38.439 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:41:38.439 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:41:38.439 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:41:38.439 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:41:38.439 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:41:38.439 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:41:38.439 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:41:38.439 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-10T21:41:38.439 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-10T21:41:38.439 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:41:38.439 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:41:38.440 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:41:38.440 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:41:38.440 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:41:38.440 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:41:38.440 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:41:38.440 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:41:38.440 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:41:38.440 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:41:38.440 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:41:38.440 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:41:38.440 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:41:38.440 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-10T21:41:38.440 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-10T21:41:38.440 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-10T21:41:38.441 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-10T21:41:38.441 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-10T21:41:38.441 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "5e2f023f-d6a6-49f4-a256-f4807dce3739", 2024-09-10T21:41:38.441 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:41:38.441 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:41:38.441 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-10T21:41:38.441 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-10T21:41:38.441 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-10T21:41:38.441 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-10T21:41:38.441 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-10T21:41:38.441 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-10T21:41:38.441 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-10T21:41:38.442 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:41:38.442 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-10T21:41:38.442 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:41:38.442 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:41:38.442 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:41:38.442 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:41:38.442 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:41:38.442 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:41:38.442 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:41:38.442 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:41:38.442 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:41:38.442 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:41:38.442 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:41:38.442 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:41:38.443 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-10T21:41:38.443 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:41:38.443 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-10T21:41:38.443 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:41:38.443 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:41:38.443 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:41:38.443 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:41:38.443 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:41:38.443 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:41:38.443 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:41:38.443 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-10T21:41:38.443 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:41:38.443 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:41:38.443 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:41:38.444 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:41:38.444 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-10T21:41:38.444 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:41:38.444 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:41:38.444 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:41:38.444 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:41:38.444 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-10T21:41:38.444 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-10T21:41:38.444 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:41:38.444 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:41:38.444 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:41:38.444 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:41:38.444 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:41:38.445 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:41:38.445 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:41:38.445 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:41:38.445 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:41:38.445 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:41:38.445 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:41:38.445 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3300", 2024-09-10T21:41:38.445 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:41:38.445 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:41:38.445 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:41:38.445 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:41:38.445 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789", 2024-09-10T21:41:38.445 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:41:38.445 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:41:38.446 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:41:38.446 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:41:38.446 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789/0", 2024-09-10T21:41:38.446 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6789/0", 2024-09-10T21:41:38.446 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:41:38.446 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:41:38.446 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:41:38.446 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:41:38.446 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:41:38.446 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:41:38.446 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:41:38.446 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:41:38.446 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:41:38.446 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:41:38.447 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:41:38.447 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3301", 2024-09-10T21:41:38.447 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:41:38.447 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:41:38.447 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:41:38.447 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:41:38.447 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790", 2024-09-10T21:41:38.447 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:41:38.447 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:41:38.447 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:41:38.447 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:41:38.447 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790/0", 2024-09-10T21:41:38.447 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6790/0", 2024-09-10T21:41:38.447 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:41:38.448 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:41:38.448 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:41:38.448 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:41:38.448 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:41:38.448 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:41:38.448 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-10T21:41:38.448 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-10T21:41:38.448 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:41:38.448 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:41:38.448 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:41:38.448 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:41:38.448 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:41:38.448 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:41:38.449 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-10T21:41:38.449 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:41:38.449 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:41:38.449 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:41:38.449 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:41:38.449 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:41:38.449 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:41:38.449 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:41:38.449 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-10T21:41:38.449 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-10T21:41:38.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:38.432+0000 7ffbc6000640 1 -- 172.21.15.17:0/3136678895 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbc00be070 con 0x7ffbd014bf20 2024-09-10T21:41:38.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:38.432+0000 7ffbc6000640 1 -- 172.21.15.17:0/3136678895 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4544 (secure 0 0 0) 0x7ffbc00c6070 con 0x7ffbd014bf20 2024-09-10T21:41:38.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:38.432+0000 7ffbc6000640 1 -- 172.21.15.17:0/3136678895 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffbd014bf20 msgr2=0x7ffbd014c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:41:38.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:38.432+0000 7ffbc6000640 1 --2- 172.21.15.17:0/3136678895 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffbd014bf20 0x7ffbd014c330 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7ffbc00097f0 tx=0x7ffbc00091e0 comp rx=0 tx=0).stop 2024-09-10T21:41:38.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:38.432+0000 7ffbc6000640 1 -- 172.21.15.17:0/3136678895 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffbd014bf20 msgr2=0x7ffbd014c330 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:41:38.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:38.432+0000 7ffbc6000640 1 --2- 172.21.15.17:0/3136678895 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ffbd014bf20 0x7ffbd014c330 secure :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7ffbc00097f0 tx=0x7ffbc00091e0 comp rx=0 tx=0).stop 2024-09-10T21:41:38.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:38.432+0000 7ffbcf200640 1 -- 172.21.15.17:0/3136678895 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7ffbb00726b0 msgr2=0x7ffbb0074b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:41:38.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:38.432+0000 7ffbcf200640 1 --2- 172.21.15.17:0/3136678895 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7ffbb00726b0 0x7ffbb0074b70 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7ffbc8001e10 tx=0x7ffbc8001da0 comp rx=0 tx=0).stop 2024-09-10T21:41:38.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:38.432+0000 7ffbcf200640 1 -- 172.21.15.17:0/3136678895 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffbd0070980 msgr2=0x7ffbd00740a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:41:38.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:38.432+0000 7ffbcf200640 1 --2- 172.21.15.17:0/3136678895 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffbd0070980 0x7ffbd00740a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ffbbc00c410 tx=0x7ffbbc00c8d0 comp rx=0 tx=0).stop 2024-09-10T21:41:38.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:38.432+0000 7ffbce800640 1 -- 172.21.15.17:0/3136678895 reap_dead start 2024-09-10T21:41:38.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:38.432+0000 7ffbcf200640 1 -- 172.21.15.17:0/3136678895 shutdown_connections 2024-09-10T21:41:38.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:38.432+0000 7ffbcf200640 1 --2- 172.21.15.17:0/3136678895 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ffbd0070980 0x7ffbd00740a0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:41:38.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:38.432+0000 7ffbcf200640 1 -- 172.21.15.17:0/3136678895 >> 172.21.15.17:0/3136678895 conn(0x7ffbd006fe60 msgr2=0x7ffbd006e780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:41:38.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:38.432+0000 7ffbcf200640 1 -- 172.21.15.17:0/3136678895 shutdown_connections 2024-09-10T21:41:38.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:38.432+0000 7ffbcf200640 1 -- 172.21.15.17:0/3136678895 wait complete. 2024-09-10T21:41:38.481 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-10T21:41:38.481 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-09-10T21:41:38.481 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-10T21:41:38.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:38.816+0000 7fba65400640 1 -- 172.21.15.17:0/1795059740 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fba6006e730 msgr2=0x7fba601548b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:41:38.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:38.816+0000 7fba65400640 1 --2- 172.21.15.17:0/1795059740 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fba6006e730 0x7fba601548b0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fba50009840 tx=0x7fba5001c820 comp rx=0 tx=0).stop 2024-09-10T21:41:38.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:38.816+0000 7fba65400640 1 -- 172.21.15.17:0/1795059740 shutdown_connections 2024-09-10T21:41:38.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:38.816+0000 7fba65400640 1 --2- 172.21.15.17:0/1795059740 >> v2:172.21.15.31:3300/0 conn(0x7fba600c6f40 0x7fba6015afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:41:38.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:38.816+0000 7fba65400640 1 --2- 172.21.15.17:0/1795059740 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fba6006e730 0x7fba601548b0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:41:38.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:38.816+0000 7fba65400640 1 -- 172.21.15.17:0/1795059740 >> 172.21.15.17:0/1795059740 conn(0x7fba600700d0 msgr2=0x7fba600704e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:41:38.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:38.816+0000 7fba65400640 1 -- 172.21.15.17:0/1795059740 shutdown_connections 2024-09-10T21:41:38.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:38.816+0000 7fba65400640 1 -- 172.21.15.17:0/1795059740 wait complete. 2024-09-10T21:41:38.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:38.820+0000 7fba65400640 1 Processor -- start 2024-09-10T21:41:38.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:38.820+0000 7fba65400640 1 -- start start 2024-09-10T21:41:38.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:38.820+0000 7fba65400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba6006e730 0x7fba600cc770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:41:38.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:38.820+0000 7fba65400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fba600c6f40 0x7fba60153770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:41:38.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:38.820+0000 7fba65400640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fba60149fe0 0x7fba600cb3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:41:38.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:38.820+0000 7fba65400640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fba600c88a0 con 0x7fba600c6f40 2024-09-10T21:41:38.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:38.820+0000 7fba65400640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fba600c83a0 con 0x7fba6006e730 2024-09-10T21:41:38.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:38.820+0000 7fba65400640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7fba600c0fd0 con 0x7fba60149fe0 2024-09-10T21:41:38.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:38.820+0000 7fba5e800640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba6006e730 0x7fba600cc770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:41:38.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:38.820+0000 7fba5e800640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba6006e730 0x7fba600cc770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.17:51184/0 (socket says 172.21.15.17:51184) 2024-09-10T21:41:38.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:38.820+0000 7fba5e800640 1 -- 172.21.15.17:0/1960213022 learned_addr learned my addr 172.21.15.17:0/1960213022 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:41:38.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:38.824+0000 7fba5f200640 1 --2- 172.21.15.17:0/1960213022 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fba60149fe0 0x7fba600cb3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:41:38.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:38.824+0000 7fba5f200640 1 -- 172.21.15.17:0/1960213022 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba6006e730 msgr2=0x7fba600cc770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:41:38.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:38.824+0000 7fba5f200640 1 --2- 172.21.15.17:0/1960213022 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba6006e730 0x7fba600cc770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:41:38.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:38.824+0000 7fba5f200640 1 -- 172.21.15.17:0/1960213022 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fba600c6f40 msgr2=0x7fba60153770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:41:38.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:38.824+0000 7fba5f200640 1 --2- 172.21.15.17:0/1960213022 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fba600c6f40 0x7fba60153770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:41:38.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:38.824+0000 7fba5f200640 1 -- 172.21.15.17:0/1960213022 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba6014b310 con 0x7fba60149fe0 2024-09-10T21:41:38.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:38.832+0000 7fba5f200640 1 --2- 172.21.15.17:0/1960213022 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fba60149fe0 0x7fba600cb3b0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fba600c9b10 tx=0x7fba4c00c800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:41:38.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:38.840+0000 7fba56a00640 1 -- 172.21.15.17:0/1960213022 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba4c017070 con 0x7fba60149fe0 2024-09-10T21:41:38.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:38.840+0000 7fba56a00640 1 -- 172.21.15.17:0/1960213022 <== mon.2 v2:172.21.15.31:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fba4c003600 con 0x7fba60149fe0 2024-09-10T21:41:38.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:38.840+0000 7fba56a00640 1 -- 172.21.15.17:0/1960213022 <== mon.2 v2:172.21.15.31:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba4c003a50 con 0x7fba60149fe0 2024-09-10T21:41:38.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:38.844+0000 7fba65400640 1 -- 172.21.15.17:0/1960213022 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba600c9ee0 con 0x7fba60149fe0 2024-09-10T21:41:38.851 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:38.844+0000 7fba65400640 1 -- 172.21.15.17:0/1960213022 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fba600cccb0 con 0x7fba60149fe0 2024-09-10T21:41:38.853 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:38.848+0000 7fba65400640 1 --2- 172.21.15.17:0/1960213022 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba600ae4a0 0x7fba600cb8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:41:38.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:38.848+0000 7fba65400640 1 -- 172.21.15.17:0/1960213022 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fba60072cd0 con 0x7fba600ae4a0 2024-09-10T21:41:38.860 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:38.852+0000 7fba5e800640 1 --2- 172.21.15.17:0/1960213022 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba600ae4a0 0x7fba600cb8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:41:38.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:38.852+0000 7fba56a00640 1 -- 172.21.15.17:0/1960213022 <== mon.2 v2:172.21.15.31:3301/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fba4c003bf0 con 0x7fba60149fe0 2024-09-10T21:41:38.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:38.852+0000 7fba5e800640 1 -- 172.21.15.17:0/1960213022 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fba600ca2d0 con 0x7fba600ae4a0 2024-09-10T21:41:38.861 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:38.852+0000 7fba5e800640 1 --2- 172.21.15.17:0/1960213022 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba600ae4a0 0x7fba600cb8f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fba50000c10 tx=0x7fba50005780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:41:38.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:38.856+0000 7fba56a00640 1 --2- 172.21.15.17:0/1960213022 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fba400726b0 0x7fba40074b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:41:38.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:38.856+0000 7fba56a00640 1 -- 172.21.15.17:0/1960213022 <== mon.2 v2:172.21.15.31:3301/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fba4c092ba0 con 0x7fba60149fe0 2024-09-10T21:41:38.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:38.856+0000 7fba56a00640 1 -- 172.21.15.17:0/1960213022 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba500bc020 con 0x7fba600ae4a0 2024-09-10T21:41:38.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:38.856+0000 7fba56a00640 1 -- 172.21.15.17:0/1960213022 <== mon.1 v2:172.21.15.31:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fba500a5900 con 0x7fba600ae4a0 2024-09-10T21:41:38.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:38.856+0000 7fba56a00640 1 -- 172.21.15.17:0/1960213022 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba600ae4a0 msgr2=0x7fba600cb8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:41:38.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:38.856+0000 7fba56a00640 1 --2- 172.21.15.17:0/1960213022 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba600ae4a0 0x7fba600cb8f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fba50000c10 tx=0x7fba50005780 comp rx=0 tx=0).stop 2024-09-10T21:41:38.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:38.856+0000 7fba56a00640 1 -- 172.21.15.17:0/1960213022 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba600ae4a0 msgr2=0x7fba600cb8f0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:41:38.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:38.856+0000 7fba56a00640 1 --2- 172.21.15.17:0/1960213022 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba600ae4a0 0x7fba600cb8f0 secure :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fba50000c10 tx=0x7fba50005780 comp rx=0 tx=0).stop 2024-09-10T21:41:38.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:38.856+0000 7fba5de00640 1 --2- 172.21.15.17:0/1960213022 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fba400726b0 0x7fba40074b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:41:38.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:38.860+0000 7fba5de00640 1 --2- 172.21.15.17:0/1960213022 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fba400726b0 0x7fba40074b70 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fba58009270 tx=0x7fba580091e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:41:38.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:38.868+0000 7fba65400640 1 --2- 172.21.15.17:0/1960213022 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba601488b0 0x7fba60148cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:41:38.877 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:38.872+0000 7fba65400640 1 -- 172.21.15.17:0/1960213022 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fba600c3690 con 0x7fba601488b0 2024-09-10T21:41:38.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:38.872+0000 7fba5e800640 1 --2- 172.21.15.17:0/1960213022 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba601488b0 0x7fba60148cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:41:38.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:38.876+0000 7fba5e800640 1 -- 172.21.15.17:0/1960213022 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fba600ca0b0 con 0x7fba601488b0 2024-09-10T21:41:38.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:38.880+0000 7fba5e800640 1 --2- 172.21.15.17:0/1960213022 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba601488b0 0x7fba60148cc0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fba60072660 tx=0x7fba500091e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:41:38.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:38.880+0000 7fba56a00640 1 -- 172.21.15.17:0/1960213022 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba500bc030 con 0x7fba601488b0 2024-09-10T21:41:38.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:38.880+0000 7fba56a00640 1 -- 172.21.15.17:0/1960213022 <== mon.1 v2:172.21.15.31:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4877 (secure 0 0 0) 0x7fba500c3220 con 0x7fba601488b0 2024-09-10T21:41:38.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:38.880+0000 7fba56a00640 1 -- 172.21.15.17:0/1960213022 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba601488b0 msgr2=0x7fba60148cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:41:38.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:38.880+0000 7fba56a00640 1 --2- 172.21.15.17:0/1960213022 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba601488b0 0x7fba60148cc0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fba60072660 tx=0x7fba500091e0 comp rx=0 tx=0).stop 2024-09-10T21:41:38.886 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:38.880+0000 7fba56a00640 1 -- 172.21.15.17:0/1960213022 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba601488b0 msgr2=0x7fba60148cc0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:41:38.886 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-10T21:41:38.887 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:41:38.887 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:41:38.887 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-10T21:41:38.887 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 130, 2024-09-10T21:41:38.887 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 110310, 2024-09-10T21:41:38.887 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-10T21:41:38.887 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-10T21:41:38.887 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-10T21:41:38.887 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-10T21:41:38.887 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:41:38.887 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 15, 2024-09-10T21:41:38.887 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:41:38.887 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-10T21:41:38.888 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-10T21:41:38.888 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:41:38.888 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:41:38.888 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:41:38.888 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:41:38.888 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:41:38.888 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:41:38.888 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:41:38.888 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:41:38.888 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:41:38.888 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:41:38.888 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:41:38.888 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:41:38.889 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-10T21:41:38.889 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-10T21:41:38.889 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:41:38.889 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:41:38.889 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:41:38.889 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:41:38.889 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:41:38.889 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:41:38.889 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:41:38.889 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:41:38.889 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:41:38.889 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:41:38.890 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:41:38.890 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:41:38.890 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:41:38.890 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-10T21:41:38.890 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-10T21:41:38.890 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-10T21:41:38.890 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-10T21:41:38.890 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-10T21:41:38.890 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "5e2f023f-d6a6-49f4-a256-f4807dce3739", 2024-09-10T21:41:38.890 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:41:38.890 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:41:38.890 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-10T21:41:38.890 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-10T21:41:38.891 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-10T21:41:38.891 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-10T21:41:38.891 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-10T21:41:38.891 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-10T21:41:38.891 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-10T21:41:38.891 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:41:38.891 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-10T21:41:38.891 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:41:38.891 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:41:38.891 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:41:38.891 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:41:38.891 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:41:38.891 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:41:38.891 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:41:38.892 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:41:38.892 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:41:38.892 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:41:38.892 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:41:38.892 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:41:38.892 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-10T21:41:38.892 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:41:38.892 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-10T21:41:38.892 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:41:38.892 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:41:38.892 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:41:38.892 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:41:38.893 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:41:38.893 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:41:38.893 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:41:38.893 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-10T21:41:38.893 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:41:38.893 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:41:38.893 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:41:38.893 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:41:38.893 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-10T21:41:38.893 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:41:38.893 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:41:38.894 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:41:38.894 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:41:38.894 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-10T21:41:38.894 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-10T21:41:38.894 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:41:38.894 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:41:38.894 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:41:38.894 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:41:38.894 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:41:38.894 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:41:38.894 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:41:38.894 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:41:38.894 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:41:38.894 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:41:38.895 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:41:38.895 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3300", 2024-09-10T21:41:38.895 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:41:38.895 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:41:38.895 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:41:38.895 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:41:38.895 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789", 2024-09-10T21:41:38.895 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:41:38.895 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:41:38.895 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:41:38.895 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:41:38.895 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789/0", 2024-09-10T21:41:38.896 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6789/0", 2024-09-10T21:41:38.896 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:41:38.896 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:41:38.896 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:41:38.896 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:41:38.896 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:41:38.896 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:41:38.896 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:41:38.896 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:41:38.896 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:41:38.896 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:41:38.896 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:41:38.896 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3301", 2024-09-10T21:41:38.897 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:41:38.897 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:41:38.897 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:41:38.897 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:41:38.897 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790", 2024-09-10T21:41:38.897 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:41:38.897 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:41:38.897 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:41:38.897 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:41:38.897 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790/0", 2024-09-10T21:41:38.897 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6790/0", 2024-09-10T21:41:38.898 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:41:38.898 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:41:38.898 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:41:38.898 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:41:38.898 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:41:38.898 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:41:38.898 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-10T21:41:38.898 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-10T21:41:38.898 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:41:38.898 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:41:38.898 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:41:38.898 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:41:38.898 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:41:38.899 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:41:38.899 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": [ 2024-09-10T21:41:38.899 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:41:38.899 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:41:38.899 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:41:38.899 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:41:38.899 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:41:38.899 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:41:38.899 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-10T21:41:38.899 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:41:38.899 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:41:38.899 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:41:38.899 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:41:38.899 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:41:38.900 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:41:38.900 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-10T21:41:38.900 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:41:38.900 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:41:38.900 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:41:38.900 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 2 2024-09-10T21:41:38.900 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:41:38.900 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:41:38.900 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:41:38.900 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-10T21:41:38.900 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-10T21:41:38.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:38.880+0000 7fba56a00640 1 --2- 172.21.15.17:0/1960213022 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fba601488b0 0x7fba60148cc0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:41:38.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:38.880+0000 7fba4be00640 1 -- 172.21.15.17:0/1960213022 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fba400726b0 msgr2=0x7fba40074b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:41:38.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:38.880+0000 7fba4be00640 1 --2- 172.21.15.17:0/1960213022 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fba400726b0 0x7fba40074b70 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fba58009270 tx=0x7fba580091e0 comp rx=0 tx=0).stop 2024-09-10T21:41:38.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:38.880+0000 7fba4be00640 1 -- 172.21.15.17:0/1960213022 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fba60149fe0 msgr2=0x7fba600cb3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:41:38.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:38.880+0000 7fba4be00640 1 --2- 172.21.15.17:0/1960213022 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fba60149fe0 0x7fba600cb3b0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fba600c9b10 tx=0x7fba4c00c800 comp rx=0 tx=0).stop 2024-09-10T21:41:38.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:38.880+0000 7fba5f200640 1 -- 172.21.15.17:0/1960213022 reap_dead start 2024-09-10T21:41:38.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:38.880+0000 7fba5f200640 1 -- 172.21.15.17:0/1960213022 reap_dead start 2024-09-10T21:41:38.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:38.880+0000 7fba4be00640 1 -- 172.21.15.17:0/1960213022 shutdown_connections 2024-09-10T21:41:38.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:38.880+0000 7fba4be00640 1 -- 172.21.15.17:0/1960213022 >> 172.21.15.17:0/1960213022 conn(0x7fba600700d0 msgr2=0x7fba60073fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:41:38.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:38.880+0000 7fba4be00640 1 -- 172.21.15.17:0/1960213022 shutdown_connections 2024-09-10T21:41:38.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:38.880+0000 7fba4be00640 1 -- 172.21.15.17:0/1960213022 wait complete. 2024-09-10T21:41:38.937 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-10T21:41:38.937 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-09-10T21:41:38.938 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-10T21:41:39.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:39.324+0000 7f3d46e00640 1 -- 172.21.15.17:0/2296016380 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3d38003ae0 msgr2=0x7f3d38003f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:41:39.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:39.324+0000 7f3d46e00640 1 --2- 172.21.15.17:0/2296016380 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3d38003ae0 0x7f3d38003f00 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f3d2c00c2e0 tx=0x7f3d2c01cf50 comp rx=0 tx=0).stop 2024-09-10T21:41:39.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:39.324+0000 7f3d46e00640 1 -- 172.21.15.17:0/2296016380 shutdown_connections 2024-09-10T21:41:39.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:39.324+0000 7f3d46e00640 1 --2- 172.21.15.17:0/2296016380 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3d38003ae0 0x7f3d38003f00 secure :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f3d2c00c2e0 tx=0x7f3d2c01cf50 comp rx=0 tx=0).stop 2024-09-10T21:41:39.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:39.324+0000 7f3d46e00640 1 -- 172.21.15.17:0/2296016380 >> 172.21.15.17:0/2296016380 conn(0x7f3d40070110 msgr2=0x7f3d40070520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:41:39.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:39.332+0000 7f3d46e00640 1 -- 172.21.15.17:0/2296016380 shutdown_connections 2024-09-10T21:41:39.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:39.332+0000 7f3d46e00640 1 -- 172.21.15.17:0/2296016380 wait complete. 2024-09-10T21:41:39.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:39.336+0000 7f3d46e00640 1 Processor -- start 2024-09-10T21:41:39.346 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:39.340+0000 7f3d46e00640 1 -- start start 2024-09-10T21:41:39.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:39.340+0000 7f3d46e00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3d401e6d80 0x7f3d40153bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:41:39.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:39.340+0000 7f3d46e00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3d40154de0 0x7f3d400c7da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:41:39.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:39.340+0000 7f3d46e00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3d401e7160 0x7f3d400c82e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:41:39.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:39.340+0000 7f3d46e00640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f3d4006e4e0 con 0x7f3d40154de0 2024-09-10T21:41:39.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:39.340+0000 7f3d46e00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f3d4006e200 con 0x7f3d401e6d80 2024-09-10T21:41:39.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:39.340+0000 7f3d46e00640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f3d4006e7c0 con 0x7f3d401e7160 2024-09-10T21:41:39.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:39.344+0000 7f3d3f400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3d401e6d80 0x7f3d40153bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:41:39.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:39.344+0000 7f3d3f400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3d401e6d80 0x7f3d40153bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.17:51222/0 (socket says 172.21.15.17:51222) 2024-09-10T21:41:39.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:39.344+0000 7f3d3f400640 1 -- 172.21.15.17:0/2761098307 learned_addr learned my addr 172.21.15.17:0/2761098307 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:41:39.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:39.344+0000 7f3d3fe00640 1 --2- 172.21.15.17:0/2761098307 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3d401e7160 0x7f3d400c82e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:41:39.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:39.344+0000 7f3d3fe00640 1 -- 172.21.15.17:0/2761098307 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3d401e6d80 msgr2=0x7f3d40153bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:41:39.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:39.344+0000 7f3d3fe00640 1 --2- 172.21.15.17:0/2761098307 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3d401e6d80 0x7f3d40153bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:41:39.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:39.344+0000 7f3d3fe00640 1 -- 172.21.15.17:0/2761098307 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3d40154de0 msgr2=0x7f3d400c7da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:41:39.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:39.344+0000 7f3d3fe00640 1 --2- 172.21.15.17:0/2761098307 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3d40154de0 0x7f3d400c7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:41:39.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:39.344+0000 7f3d3fe00640 1 -- 172.21.15.17:0/2761098307 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d40154110 con 0x7f3d401e7160 2024-09-10T21:41:39.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:39.344+0000 7f3d3fe00640 1 --2- 172.21.15.17:0/2761098307 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3d401e7160 0x7f3d400c82e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f3d40071170 tx=0x7f3d34010e90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:41:39.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:39.344+0000 7f3d33e00640 1 -- 172.21.15.17:0/2761098307 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d34003e50 con 0x7f3d401e7160 2024-09-10T21:41:39.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:39.344+0000 7f3d33e00640 1 -- 172.21.15.17:0/2761098307 <== mon.2 v2:172.21.15.31:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3d34015070 con 0x7f3d401e7160 2024-09-10T21:41:39.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:39.344+0000 7f3d33e00640 1 -- 172.21.15.17:0/2761098307 <== mon.2 v2:172.21.15.31:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d3400ed70 con 0x7f3d401e7160 2024-09-10T21:41:39.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:39.344+0000 7f3d46e00640 1 -- 172.21.15.17:0/2761098307 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d401f4800 con 0x7f3d401e7160 2024-09-10T21:41:39.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:39.348+0000 7f3d46e00640 1 -- 172.21.15.17:0/2761098307 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d401f4da0 con 0x7f3d401e7160 2024-09-10T21:41:39.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:39.348+0000 7f3d46e00640 1 --2- 172.21.15.17:0/2761098307 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3d401f4f70 0x7f3d4014f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:41:39.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:39.348+0000 7f3d46e00640 1 -- 172.21.15.17:0/2761098307 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f3d4006eaa0 con 0x7f3d401f4f70 2024-09-10T21:41:39.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:39.352+0000 7f3d33e00640 1 -- 172.21.15.17:0/2761098307 <== mon.2 v2:172.21.15.31:3301/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f3d34004030 con 0x7f3d401e7160 2024-09-10T21:41:39.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:39.356+0000 7f3d33e00640 1 --2- 172.21.15.17:0/2761098307 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f3d20072880 0x7f3d20074d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:41:39.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:39.356+0000 7f3d33e00640 1 -- 172.21.15.17:0/2761098307 <== mon.2 v2:172.21.15.31:3301/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f3d34093c60 con 0x7f3d401e7160 2024-09-10T21:41:39.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:39.356+0000 7f3d3f400640 1 --2- 172.21.15.17:0/2761098307 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3d401f4f70 0x7f3d4014f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:41:39.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:39.356+0000 7f3d3f400640 1 -- 172.21.15.17:0/2761098307 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3d40070930 con 0x7f3d401f4f70 2024-09-10T21:41:39.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:39.356+0000 7f3d3f400640 1 --2- 172.21.15.17:0/2761098307 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3d401f4f70 0x7f3d4014f0a0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f3d2c01b650 tx=0x7f3d2c006cf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:41:39.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:39.360+0000 7f3d33e00640 1 -- 172.21.15.17:0/2761098307 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d2c0060c0 con 0x7f3d401f4f70 2024-09-10T21:41:39.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:39.360+0000 7f3d3ea00640 1 --2- 172.21.15.17:0/2761098307 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f3d20072880 0x7f3d20074d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:41:39.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:39.360+0000 7f3d3ea00640 1 --2- 172.21.15.17:0/2761098307 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f3d20072880 0x7f3d20074d40 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f3d4015d880 tx=0x7f3d280067b0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:41:39.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:39.360+0000 7f3d33e00640 1 -- 172.21.15.17:0/2761098307 <== mon.2 v2:172.21.15.31:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f3d2c0060c0 con 0x7f3d401f4f70 2024-09-10T21:41:39.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:39.360+0000 7f3d33e00640 1 -- 172.21.15.17:0/2761098307 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3d401f4f70 msgr2=0x7f3d4014f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:41:39.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:39.360+0000 7f3d33e00640 1 --2- 172.21.15.17:0/2761098307 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3d401f4f70 0x7f3d4014f0a0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f3d2c01b650 tx=0x7f3d2c006cf0 comp rx=0 tx=0).stop 2024-09-10T21:41:39.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:39.360+0000 7f3d33e00640 1 -- 172.21.15.17:0/2761098307 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3d401f4f70 msgr2=0x7f3d4014f0a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:41:39.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:39.360+0000 7f3d33e00640 1 --2- 172.21.15.17:0/2761098307 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3d401f4f70 0x7f3d4014f0a0 secure :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f3d2c01b650 tx=0x7f3d2c006cf0 comp rx=0 tx=0).stop 2024-09-10T21:41:39.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:39.388+0000 7f3d46e00640 1 --2- 172.21.15.17:0/2761098307 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3d4014f5e0 0x7f3d401f7d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:41:39.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:39.388+0000 7f3d46e00640 1 -- 172.21.15.17:0/2761098307 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f3d40070650 con 0x7f3d4014f5e0 2024-09-10T21:41:39.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:39.388+0000 7f3d3f400640 1 --2- 172.21.15.17:0/2761098307 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3d4014f5e0 0x7f3d401f7d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:41:39.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:39.388+0000 7f3d3f400640 1 -- 172.21.15.17:0/2761098307 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f3d401f56d0 con 0x7f3d4014f5e0 2024-09-10T21:41:39.397 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:39.392+0000 7f3d3f400640 1 --2- 172.21.15.17:0/2761098307 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3d4014f5e0 0x7f3d401f7d70 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f3d2c0b9000 tx=0x7f3d2c0067b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:41:39.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:39.392+0000 7f3d33e00640 1 -- 172.21.15.17:0/2761098307 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d2c003810 con 0x7f3d4014f5e0 2024-09-10T21:41:39.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:39.396+0000 7f3d33e00640 1 -- 172.21.15.17:0/2761098307 <== mon.2 v2:172.21.15.31:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5044 (secure 0 0 0) 0x7f3d2c0039b0 con 0x7f3d4014f5e0 2024-09-10T21:41:39.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:39.396+0000 7f3d33e00640 1 -- 172.21.15.17:0/2761098307 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3d4014f5e0 msgr2=0x7f3d401f7d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:41:39.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:39.396+0000 7f3d33e00640 1 --2- 172.21.15.17:0/2761098307 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3d4014f5e0 0x7f3d401f7d70 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f3d2c0b9000 tx=0x7f3d2c0067b0 comp rx=0 tx=0).stop 2024-09-10T21:41:39.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:39.396+0000 7f3d33e00640 1 -- 172.21.15.17:0/2761098307 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3d4014f5e0 msgr2=0x7f3d401f7d70 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:41:39.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:39.396+0000 7f3d33e00640 1 --2- 172.21.15.17:0/2761098307 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3d4014f5e0 0x7f3d401f7d70 secure :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f3d2c0b9000 tx=0x7f3d2c0067b0 comp rx=0 tx=0).stop 2024-09-10T21:41:39.403 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-10T21:41:39.403 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:41:39.403 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:41:39.403 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-10T21:41:39.403 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 130, 2024-09-10T21:41:39.404 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 305742, 2024-09-10T21:41:39.404 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-10T21:41:39.404 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-10T21:41:39.404 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-10T21:41:39.404 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-10T21:41:39.404 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:41:39.404 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 16, 2024-09-10T21:41:39.404 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:41:39.404 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-10T21:41:39.404 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-10T21:41:39.404 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:41:39.404 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:41:39.405 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:41:39.405 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:41:39.405 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:41:39.405 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:41:39.405 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:41:39.405 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:41:39.405 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:41:39.405 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:41:39.405 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:41:39.405 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:41:39.405 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-10T21:41:39.405 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-10T21:41:39.405 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:41:39.406 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:41:39.406 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:41:39.406 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:41:39.406 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:41:39.406 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:41:39.406 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:41:39.406 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:41:39.406 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:41:39.406 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:41:39.406 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:41:39.406 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:41:39.406 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:41:39.407 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-10T21:41:39.407 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-10T21:41:39.407 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-10T21:41:39.407 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-10T21:41:39.407 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-10T21:41:39.407 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "5e2f023f-d6a6-49f4-a256-f4807dce3739", 2024-09-10T21:41:39.407 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:41:39.407 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:41:39.407 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-10T21:41:39.407 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-10T21:41:39.407 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-10T21:41:39.407 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-10T21:41:39.407 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-10T21:41:39.408 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-10T21:41:39.408 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-10T21:41:39.408 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:41:39.408 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-10T21:41:39.408 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:41:39.408 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:41:39.408 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:41:39.408 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:41:39.408 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:41:39.408 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:41:39.408 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:41:39.408 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:41:39.408 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:41:39.409 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:41:39.409 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:41:39.409 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:41:39.409 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-10T21:41:39.409 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:41:39.409 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-10T21:41:39.409 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:41:39.409 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:41:39.409 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:41:39.409 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:41:39.409 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:41:39.409 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:41:39.410 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:41:39.410 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-10T21:41:39.410 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:41:39.410 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:41:39.410 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:41:39.410 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:41:39.410 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-10T21:41:39.410 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:41:39.410 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:41:39.410 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:41:39.410 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:41:39.410 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-10T21:41:39.410 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-10T21:41:39.410 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:41:39.411 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:41:39.411 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:41:39.411 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:41:39.411 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:41:39.411 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:41:39.411 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:41:39.411 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:41:39.411 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:41:39.411 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:41:39.411 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:41:39.411 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3300", 2024-09-10T21:41:39.411 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:41:39.411 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:41:39.412 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:41:39.412 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:41:39.412 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789", 2024-09-10T21:41:39.412 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:41:39.412 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:41:39.412 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:41:39.412 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:41:39.412 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789/0", 2024-09-10T21:41:39.412 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6789/0", 2024-09-10T21:41:39.412 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:41:39.412 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:41:39.412 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:41:39.412 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:41:39.413 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:41:39.413 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:41:39.413 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:41:39.413 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:41:39.413 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:41:39.413 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:41:39.413 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:41:39.413 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3301", 2024-09-10T21:41:39.413 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:41:39.413 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:41:39.413 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:41:39.413 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:41:39.414 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790", 2024-09-10T21:41:39.414 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:41:39.414 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:41:39.414 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:41:39.414 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:41:39.414 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790/0", 2024-09-10T21:41:39.414 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6790/0", 2024-09-10T21:41:39.414 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:41:39.414 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:41:39.414 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:41:39.414 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:41:39.414 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:41:39.414 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:41:39.415 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-10T21:41:39.415 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-10T21:41:39.415 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:41:39.415 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:41:39.415 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:41:39.415 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:41:39.415 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:41:39.415 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:41:39.415 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": [ 2024-09-10T21:41:39.415 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:41:39.415 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:41:39.415 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:41:39.415 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 5 2024-09-10T21:41:39.415 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:41:39.416 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:41:39.416 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-10T21:41:39.416 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:41:39.416 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:41:39.416 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:41:39.416 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 7 2024-09-10T21:41:39.416 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:41:39.416 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:41:39.416 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-10T21:41:39.416 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:41:39.416 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:41:39.416 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:41:39.416 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 6 2024-09-10T21:41:39.417 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:41:39.417 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:41:39.417 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": [ 2024-09-10T21:41:39.417 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:41:39.417 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:41:39.417 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:41:39.417 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 2 2024-09-10T21:41:39.417 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:41:39.417 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:41:39.417 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:41:39.417 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-10T21:41:39.417 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-10T21:41:39.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:39.396+0000 7f3d46e00640 1 -- 172.21.15.17:0/2761098307 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f3d20072880 msgr2=0x7f3d20074d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:41:39.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:39.396+0000 7f3d46e00640 1 --2- 172.21.15.17:0/2761098307 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f3d20072880 0x7f3d20074d40 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f3d4015d880 tx=0x7f3d280067b0 comp rx=0 tx=0).stop 2024-09-10T21:41:39.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:39.396+0000 7f3d46e00640 1 -- 172.21.15.17:0/2761098307 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3d401e7160 msgr2=0x7f3d400c82e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:41:39.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:39.396+0000 7f3d46e00640 1 --2- 172.21.15.17:0/2761098307 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3d401e7160 0x7f3d400c82e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f3d40071170 tx=0x7f3d34010e90 comp rx=0 tx=0).stop 2024-09-10T21:41:39.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:39.396+0000 7f3d3fe00640 1 -- 172.21.15.17:0/2761098307 reap_dead start 2024-09-10T21:41:39.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:39.396+0000 7f3d3fe00640 1 -- 172.21.15.17:0/2761098307 reap_dead start 2024-09-10T21:41:39.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:39.396+0000 7f3d46e00640 1 -- 172.21.15.17:0/2761098307 shutdown_connections 2024-09-10T21:41:39.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:39.396+0000 7f3d46e00640 1 -- 172.21.15.17:0/2761098307 >> 172.21.15.17:0/2761098307 conn(0x7f3d40070110 msgr2=0x7f3d4015e660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:41:39.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:39.400+0000 7f3d46e00640 1 -- 172.21.15.17:0/2761098307 shutdown_connections 2024-09-10T21:41:39.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:39.400+0000 7f3d46e00640 1 -- 172.21.15.17:0/2761098307 wait complete. 2024-09-10T21:41:39.468 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-10T21:41:39.469 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['a'] 2024-09-10T21:41:39.469 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2024-09-10T21:41:39.469 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.a 2024-09-10T21:41:39.469 INFO:tasks.mon_thrash.mon_thrasher:killing mon.a 2024-09-10T21:41:39.469 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-10T21:41:39.469 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:41:45.572 INFO:tasks.ceph.mon.a:Stopped 2024-09-10T21:41:45.572 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2024-09-10T21:41:45.572 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-10T21:41:45.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:45.924+0000 7f5f66600640 1 -- 172.21.15.17:0/3534101568 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f5f580d8730 msgr2=0x7f5f580d8b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:41:45.933 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:45.924+0000 7f5f66600640 1 --2- 172.21.15.17:0/3534101568 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f5f580d8730 0x7f5f580d8b10 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f5f58011a90 tx=0x7f5f4c01f0f0 comp rx=0 tx=0).stop 2024-09-10T21:41:45.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:45.924+0000 7f5f66600640 1 -- 172.21.15.17:0/3534101568 shutdown_connections 2024-09-10T21:41:45.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:45.924+0000 7f5f66600640 1 --2- 172.21.15.17:0/3534101568 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f5f580d8730 0x7f5f580d8b10 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:41:45.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:45.924+0000 7f5f66600640 1 -- 172.21.15.17:0/3534101568 >> 172.21.15.17:0/3534101568 conn(0x7f5f58050c10 msgr2=0x7f5f58051020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:41:45.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:45.924+0000 7f5f66600640 1 -- 172.21.15.17:0/3534101568 shutdown_connections 2024-09-10T21:41:45.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:45.924+0000 7f5f66600640 1 -- 172.21.15.17:0/3534101568 wait complete. 2024-09-10T21:41:45.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:45.924+0000 7f5f66600640 1 Processor -- start 2024-09-10T21:41:45.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:45.928+0000 7f5f66600640 1 -- start start 2024-09-10T21:41:45.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:45.928+0000 7f5f66600640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5f58006ea0 0x7f5f580533b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:41:45.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:45.928+0000 7f5f66600640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f5804f270 0x7f5f580d2c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:41:45.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:45.928+0000 7f5f66600640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f5f580d8730 0x7f5f580cca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:41:45.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:45.928+0000 7f5f66600640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f5f580066a0 con 0x7f5f58006ea0 2024-09-10T21:41:45.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:45.928+0000 7f5f66600640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f5f580063c0 con 0x7f5f5804f270 2024-09-10T21:41:45.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:45.928+0000 7f5f66600640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f5f5804f6b0 con 0x7f5f580d8730 2024-09-10T21:41:45.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:45.928+0000 7f5f5fe00640 1 -- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5f58006ea0 msgr2=0x7f5f580533b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-10T21:41:45.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:45.928+0000 7f5f5fe00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5f58006ea0 0x7f5f580533b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:41:45.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:45.928+0000 7f5f5f400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f5804f270 0x7f5f580d2c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:41:45.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:45.928+0000 7f5f5f400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f5804f270 0x7f5f580d2c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.17:49884/0 (socket says 172.21.15.17:49884) 2024-09-10T21:41:45.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:45.928+0000 7f5f5f400640 1 -- 172.21.15.17:0/2417780853 learned_addr learned my addr 172.21.15.17:0/2417780853 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:41:45.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:45.928+0000 7f5f65200640 1 --2- 172.21.15.17:0/2417780853 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f5f580d8730 0x7f5f580cca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:41:45.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:45.928+0000 7f5f5f400640 1 -- 172.21.15.17:0/2417780853 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f5f580d8730 msgr2=0x7f5f580cca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:41:45.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:45.928+0000 7f5f5f400640 1 --2- 172.21.15.17:0/2417780853 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f5f580d8730 0x7f5f580cca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:41:45.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:45.928+0000 7f5f5f400640 1 -- 172.21.15.17:0/2417780853 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5f58006ea0 msgr2=0x7f5f580533b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:41:45.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:45.928+0000 7f5f5f400640 1 --2- 172.21.15.17:0/2417780853 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5f58006ea0 0x7f5f580533b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:41:45.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:45.928+0000 7f5f5f400640 1 -- 172.21.15.17:0/2417780853 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f58177e40 con 0x7f5f5804f270 2024-09-10T21:41:45.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:45.932+0000 7f5f5f400640 1 --2- 172.21.15.17:0/2417780853 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f5804f270 0x7f5f580d2c40 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f5f4800c410 tx=0x7f5f4800c8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:41:45.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:45.932+0000 7f5f5cc00640 1 -- 172.21.15.17:0/2417780853 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f4801d3f0 con 0x7f5f5804f270 2024-09-10T21:41:45.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:45.932+0000 7f5f5cc00640 1 -- 172.21.15.17:0/2417780853 <== mon.1 v2:172.21.15.31:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5f4801d680 con 0x7f5f5804f270 2024-09-10T21:41:45.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:45.932+0000 7f5f5cc00640 1 -- 172.21.15.17:0/2417780853 <== mon.1 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f4801dad0 con 0x7f5f5804f270 2024-09-10T21:41:45.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:45.932+0000 7f5f66600640 1 -- 172.21.15.17:0/2417780853 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f581780e0 con 0x7f5f5804f270 2024-09-10T21:41:45.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:45.932+0000 7f5f66600640 1 -- 172.21.15.17:0/2417780853 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f581786a0 con 0x7f5f5804f270 2024-09-10T21:41:45.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:45.936+0000 7f5f66600640 1 -- 172.21.15.17:0/2417780853 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f5804ee30 con 0x7f5f5804f270 2024-09-10T21:41:45.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:45.936+0000 7f5f5cc00640 1 -- 172.21.15.17:0/2417780853 <== mon.1 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f5f48012060 con 0x7f5f5804f270 2024-09-10T21:41:45.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:45.936+0000 7f5f5cc00640 1 --2- 172.21.15.17:0/2417780853 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f5f400725e0 0x7f5f40074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:41:45.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:45.936+0000 7f5f5cc00640 1 -- 172.21.15.17:0/2417780853 <== mon.1 v2:172.21.15.31:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f5f48093220 con 0x7f5f5804f270 2024-09-10T21:41:45.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:45.944+0000 7f5f5cc00640 1 -- 172.21.15.17:0/2417780853 <== mon.1 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f5f48003810 con 0x7f5f5804f270 2024-09-10T21:41:45.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:45.944+0000 7f5f5fe00640 1 --2- 172.21.15.17:0/2417780853 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f5f400725e0 0x7f5f40074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:41:45.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:45.948+0000 7f5f5fe00640 1 --2- 172.21.15.17:0/2417780853 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f5f400725e0 0x7f5f40074aa0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f5f4c0b2000 tx=0x7f5f4c01d930 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:41:46.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:46.656+0000 7f5f66600640 1 -- 172.21.15.17:0/2417780853 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f5f5804f9e0 con 0x7f5f5804f270 2024-09-10T21:41:46.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:46.656+0000 7f5f5cc00640 1 -- 172.21.15.17:0/2417780853 <== mon.1 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1558 (secure 0 0 0) 0x7f5f4805e560 con 0x7f5f5804f270 2024-09-10T21:41:46.663 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":130,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":23,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5e2f023f-d6a6-49f4-a256-f4807dce3739","modified":"2024-09-10T21:23:46.892130Z","created":"2024-09-10T21:23:46.892130Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3301","nonce":0},{"type":"v1","addr":"172.21.15.31:6790","nonce":0}]},"addr":"172.21.15.31:6790/0","public_addr":"172.21.15.31:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-10T21:41:46.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:46.660+0000 7f5f66600640 1 -- 172.21.15.17:0/2417780853 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f5f400725e0 msgr2=0x7f5f40074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:41:46.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:46.660+0000 7f5f66600640 1 --2- 172.21.15.17:0/2417780853 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f5f400725e0 0x7f5f40074aa0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f5f4c0b2000 tx=0x7f5f4c01d930 comp rx=0 tx=0).stop 2024-09-10T21:41:46.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:46.660+0000 7f5f66600640 1 -- 172.21.15.17:0/2417780853 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f5804f270 msgr2=0x7f5f580d2c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:41:46.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:46.660+0000 7f5f66600640 1 --2- 172.21.15.17:0/2417780853 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f5804f270 0x7f5f580d2c40 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f5f4800c410 tx=0x7f5f4800c8d0 comp rx=0 tx=0).stop 2024-09-10T21:41:46.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:46.664+0000 7f5f66600640 1 -- 172.21.15.17:0/2417780853 shutdown_connections 2024-09-10T21:41:46.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:46.664+0000 7f5f66600640 1 --2- 172.21.15.17:0/2417780853 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f5f400725e0 0x7f5f40074aa0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:41:46.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:46.664+0000 7f5f66600640 1 --2- 172.21.15.17:0/2417780853 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f5f580d8730 0x7f5f580cca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:41:46.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:46.664+0000 7f5f66600640 1 --2- 172.21.15.17:0/2417780853 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f5804f270 0x7f5f580d2c40 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:41:46.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:46.664+0000 7f5f66600640 1 --2- 172.21.15.17:0/2417780853 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5f58006ea0 0x7f5f580533b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:41:46.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:46.664+0000 7f5f66600640 1 -- 172.21.15.17:0/2417780853 >> 172.21.15.17:0/2417780853 conn(0x7f5f58050c10 msgr2=0x7f5f580d5790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:41:46.681 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:46.676+0000 7f5f66600640 1 -- 172.21.15.17:0/2417780853 shutdown_connections 2024-09-10T21:41:46.682 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:46.676+0000 7f5f66600640 1 -- 172.21.15.17:0/2417780853 wait complete. 2024-09-10T21:41:49.741 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-10T21:41:52.960 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:41:52.968 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:41:58.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:58.848+0000 7fd217600640 1 -- 172.21.15.17:0/3212947001 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd1f8004d50 msgr2=0x7fd1f80051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:41:58.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:58.848+0000 7fd217600640 1 --2- 172.21.15.17:0/3212947001 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd1f8004d50 0x7fd1f80051d0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd210076910 tx=0x7fd20801df30 comp rx=0 tx=0).stop 2024-09-10T21:41:58.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:58.848+0000 7fd215600640 1 -- 172.21.15.17:0/3212947001 reap_dead start 2024-09-10T21:41:58.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:58.856+0000 7fd217600640 1 -- 172.21.15.17:0/3212947001 shutdown_connections 2024-09-10T21:41:58.863 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:58.856+0000 7fd217600640 1 -- 172.21.15.17:0/3212947001 >> 172.21.15.17:0/3212947001 conn(0x7fd2100702f0 msgr2=0x7fd210070700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:41:58.864 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:58.860+0000 7fd217600640 1 -- 172.21.15.17:0/3212947001 shutdown_connections 2024-09-10T21:41:58.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:58.860+0000 7fd217600640 1 -- 172.21.15.17:0/3212947001 wait complete. 2024-09-10T21:41:58.882 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:58.868+0000 7fd217600640 1 Processor -- start 2024-09-10T21:41:58.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:58.880+0000 7fd217600640 1 -- start start 2024-09-10T21:41:58.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:58.884+0000 7fd217600640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd1f8004d50 0x7fd21015ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:41:58.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:58.884+0000 7fd217600640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd2100b2c90 0x7fd2100b30f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:41:58.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:58.884+0000 7fd217600640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd21014b310 0x7fd21014b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:41:58.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:58.884+0000 7fd217600640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fd2100b42c0 con 0x7fd1f8004d50 2024-09-10T21:41:58.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:58.884+0000 7fd217600640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fd21015bd60 con 0x7fd2100b2c90 2024-09-10T21:41:58.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:58.884+0000 7fd217600640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7fd2100cb450 con 0x7fd21014b310 2024-09-10T21:41:58.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:58.884+0000 7fd214c00640 1 -- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd1f8004d50 msgr2=0x7fd21015ad90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-10T21:41:58.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:58.888+0000 7fd214c00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd1f8004d50 0x7fd21015ad90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:41:58.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:58.888+0000 7fd20fe00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd2100b2c90 0x7fd2100b30f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:41:58.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:58.888+0000 7fd20fe00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd2100b2c90 0x7fd2100b30f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.17:51798/0 (socket says 172.21.15.17:51798) 2024-09-10T21:41:58.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:58.888+0000 7fd20fe00640 1 -- 172.21.15.17:0/3381239217 learned_addr learned my addr 172.21.15.17:0/3381239217 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:41:58.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:58.888+0000 7fd215600640 1 --2- 172.21.15.17:0/3381239217 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd21014b310 0x7fd21014b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:41:58.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:58.888+0000 7fd20fe00640 1 -- 172.21.15.17:0/3381239217 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd21014b310 msgr2=0x7fd21014b790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:41:58.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:58.888+0000 7fd20fe00640 1 --2- 172.21.15.17:0/3381239217 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd21014b310 0x7fd21014b790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:41:58.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:58.888+0000 7fd20fe00640 1 -- 172.21.15.17:0/3381239217 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd1f8004d50 msgr2=0x7fd21015ad90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:41:58.894 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:58.888+0000 7fd20fe00640 1 --2- 172.21.15.17:0/3381239217 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd1f8004d50 0x7fd21015ad90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:41:58.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:58.888+0000 7fd20fe00640 1 -- 172.21.15.17:0/3381239217 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2101f8cf0 con 0x7fd2100b2c90 2024-09-10T21:41:58.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:58.888+0000 7fd20fe00640 1 --2- 172.21.15.17:0/3381239217 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd2100b2c90 0x7fd2100b30f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd1fc007790 tx=0x7fd1fc007c50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:41:58.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:58.888+0000 7fd20cc00640 1 -- 172.21.15.17:0/3381239217 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1fc01a070 con 0x7fd2100b2c90 2024-09-10T21:41:58.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:58.888+0000 7fd20cc00640 1 -- 172.21.15.17:0/3381239217 <== mon.1 v2:172.21.15.31:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd1fc0127c0 con 0x7fd2100b2c90 2024-09-10T21:41:58.897 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:58.892+0000 7fd20cc00640 1 -- 172.21.15.17:0/3381239217 <== mon.1 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1fc012c10 con 0x7fd2100b2c90 2024-09-10T21:41:58.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:58.896+0000 7fd217600640 1 -- 172.21.15.17:0/3381239217 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2100c56e0 con 0x7fd2100b2c90 2024-09-10T21:41:58.905 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:58.896+0000 7fd217600640 1 -- 172.21.15.17:0/3381239217 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2100c5c60 con 0x7fd2100b2c90 2024-09-10T21:41:58.908 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:58.900+0000 7fd20cc00640 1 -- 172.21.15.17:0/3381239217 <== mon.1 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fd1fc009660 con 0x7fd2100b2c90 2024-09-10T21:41:58.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:58.904+0000 7fd217600640 1 -- 172.21.15.17:0/3381239217 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2100c86c0 con 0x7fd2100b2c90 2024-09-10T21:41:58.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:58.904+0000 7fd20cc00640 1 --2- 172.21.15.17:0/3381239217 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fd1f0075a10 0x7fd1f0077ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:41:58.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:58.908+0000 7fd20cc00640 1 -- 172.21.15.17:0/3381239217 <== mon.1 v2:172.21.15.31:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fd1fc094150 con 0x7fd2100b2c90 2024-09-10T21:41:58.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:58.908+0000 7fd214c00640 1 --2- 172.21.15.17:0/3381239217 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fd1f0075a10 0x7fd1f0077ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:41:58.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:58.916+0000 7fd214c00640 1 --2- 172.21.15.17:0/3381239217 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fd1f0075a10 0x7fd1f0077ed0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fd204000f70 tx=0x7fd204000ee0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:41:58.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:58.920+0000 7fd20cc00640 1 -- 172.21.15.17:0/3381239217 <== mon.1 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fd1fc05ec30 con 0x7fd2100b2c90 2024-09-10T21:41:59.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:59.604+0000 7fd217600640 1 -- 172.21.15.17:0/3381239217 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fd2101589e0 con 0x7fd2100b2c90 2024-09-10T21:41:59.614 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:59.608+0000 7fd20cc00640 1 -- 172.21.15.17:0/3381239217 <== mon.1 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1551 (secure 0 0 0) 0x7fd1fc063660 con 0x7fd2100b2c90 2024-09-10T21:41:59.618 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":134,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":0,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5e2f023f-d6a6-49f4-a256-f4807dce3739","modified":"2024-09-10T21:23:46.892130Z","created":"2024-09-10T21:23:46.892130Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3301","nonce":0},{"type":"v1","addr":"172.21.15.31:6790","nonce":0}]},"addr":"172.21.15.31:6790/0","public_addr":"172.21.15.31:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-10T21:41:59.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:59.616+0000 7fd217600640 1 -- 172.21.15.17:0/3381239217 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fd1f0075a10 msgr2=0x7fd1f0077ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:41:59.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:59.616+0000 7fd217600640 1 --2- 172.21.15.17:0/3381239217 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fd1f0075a10 0x7fd1f0077ed0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fd204000f70 tx=0x7fd204000ee0 comp rx=0 tx=0).stop 2024-09-10T21:41:59.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:59.616+0000 7fd217600640 1 -- 172.21.15.17:0/3381239217 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd2100b2c90 msgr2=0x7fd2100b30f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:41:59.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:59.616+0000 7fd217600640 1 --2- 172.21.15.17:0/3381239217 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd2100b2c90 0x7fd2100b30f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd1fc007790 tx=0x7fd1fc007c50 comp rx=0 tx=0).stop 2024-09-10T21:41:59.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:59.620+0000 7fd217600640 1 -- 172.21.15.17:0/3381239217 shutdown_connections 2024-09-10T21:41:59.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:59.620+0000 7fd217600640 1 --2- 172.21.15.17:0/3381239217 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fd1f0075a10 0x7fd1f0077ed0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:41:59.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:59.620+0000 7fd217600640 1 --2- 172.21.15.17:0/3381239217 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd21014b310 0x7fd21014b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:41:59.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:59.620+0000 7fd217600640 1 --2- 172.21.15.17:0/3381239217 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd2100b2c90 0x7fd2100b30f0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:41:59.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:59.620+0000 7fd217600640 1 --2- 172.21.15.17:0/3381239217 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd1f8004d50 0x7fd21015ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:41:59.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:59.620+0000 7fd217600640 1 -- 172.21.15.17:0/3381239217 >> 172.21.15.17:0/3381239217 conn(0x7fd2100702f0 msgr2=0x7fd2100cc6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:41:59.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:59.624+0000 7fd217600640 1 -- 172.21.15.17:0/3381239217 shutdown_connections 2024-09-10T21:41:59.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:59.624+0000 7fd217600640 1 -- 172.21.15.17:0/3381239217 wait complete. 2024-09-10T21:41:59.669 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2024-09-10T21:41:59.670 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon b 2024-09-10T21:41:59.670 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-10T21:41:59.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:59.980+0000 7f75ba600640 1 -- 172.21.15.17:0/651225664 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f75980039c0 msgr2=0x7f7598003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:41:59.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:59.980+0000 7f75ba600640 1 --2- 172.21.15.17:0/651225664 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f75980039c0 0x7f7598003dc0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f75a00010b0 tx=0x7f75a001c780 comp rx=0 tx=0).stop 2024-09-10T21:41:59.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:59.984+0000 7f75ba600640 1 -- 172.21.15.17:0/651225664 shutdown_connections 2024-09-10T21:41:59.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:59.984+0000 7f75ba600640 1 --2- 172.21.15.17:0/651225664 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f75980039c0 0x7f7598003dc0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:41:59.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:59.984+0000 7f75ba600640 1 -- 172.21.15.17:0/651225664 >> 172.21.15.17:0/651225664 conn(0x7f75b40700d0 msgr2=0x7f75b40704e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:41:59.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:59.984+0000 7f75ba600640 1 -- 172.21.15.17:0/651225664 shutdown_connections 2024-09-10T21:41:59.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:59.984+0000 7f75ba600640 1 -- 172.21.15.17:0/651225664 wait complete. 2024-09-10T21:41:59.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:59.988+0000 7f75ba600640 1 Processor -- start 2024-09-10T21:41:59.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:59.988+0000 7f75ba600640 1 -- start start 2024-09-10T21:41:59.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:59.992+0000 7f75ba600640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f75980039c0 0x7f75b4163fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:41:59.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:59.992+0000 7f75ba600640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f75b4163440 0x7f75b4160e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:41:59.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:59.992+0000 7f75ba600640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f75b4154780 0x7f75b41f7be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:41:59.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:59.992+0000 7f75b3e00640 1 -- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f75b4154780 msgr2=0x7f75b41f7be0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-10T21:41:59.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:59.992+0000 7f75b3e00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f75b4154780 0x7f75b41f7be0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:41:59.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:59.992+0000 7f75ba600640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f75b40b1cd0 con 0x7f75b4154780 2024-09-10T21:41:59.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:59.992+0000 7f75b2a00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f75b4163440 0x7f75b4160e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:41:59.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:59.992+0000 7f75b2a00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f75b4163440 0x7f75b4160e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.17:51822/0 (socket says 172.21.15.17:51822) 2024-09-10T21:41:59.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:59.992+0000 7f75b2a00640 1 -- 172.21.15.17:0/1815751685 learned_addr learned my addr 172.21.15.17:0/1815751685 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:41:59.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:59.992+0000 7f75ba600640 1 -- 172.21.15.17:0/1815751685 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f75b40b1b50 con 0x7f75b4163440 2024-09-10T21:41:59.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:41:59.992+0000 7f75ba600640 1 -- 172.21.15.17:0/1815751685 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f75b40b1e50 con 0x7f75980039c0 2024-09-10T21:42:00.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:00.000+0000 7f75b2a00640 1 -- 172.21.15.17:0/1815751685 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f75980039c0 msgr2=0x7f75b4163fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:00.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:00.000+0000 7f75b2a00640 1 --2- 172.21.15.17:0/1815751685 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f75980039c0 0x7f75b4163fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:00.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:00.000+0000 7f75b2a00640 1 -- 172.21.15.17:0/1815751685 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f75b4154780 msgr2=0x7f75b41f7be0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:42:00.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:00.000+0000 7f75b2a00640 1 --2- 172.21.15.17:0/1815751685 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f75b4154780 0x7f75b41f7be0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:00.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:00.000+0000 7f75b2a00640 1 -- 172.21.15.17:0/1815751685 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75b41530f0 con 0x7f75b4163440 2024-09-10T21:42:00.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:00.000+0000 7f75b2a00640 1 --2- 172.21.15.17:0/1815751685 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f75b4163440 0x7f75b4160e20 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f75a4000fc0 tx=0x7f75a400f820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:00.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:00.000+0000 7f75ab400640 1 -- 172.21.15.17:0/1815751685 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75a401a070 con 0x7f75b4163440 2024-09-10T21:42:00.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:00.004+0000 7f75ba600640 1 -- 172.21.15.17:0/1815751685 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75b41533d0 con 0x7f75b4163440 2024-09-10T21:42:00.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:00.004+0000 7f75ba600640 1 -- 172.21.15.17:0/1815751685 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75b4153990 con 0x7f75b4163440 2024-09-10T21:42:00.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:00.004+0000 7f75ab400640 1 -- 172.21.15.17:0/1815751685 <== mon.1 v2:172.21.15.31:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f75a40084e0 con 0x7f75b4163440 2024-09-10T21:42:00.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:00.004+0000 7f75ab400640 1 -- 172.21.15.17:0/1815751685 <== mon.1 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75a401a070 con 0x7f75b4163440 2024-09-10T21:42:00.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:00.004+0000 7f75ba600640 1 --2- 172.21.15.17:0/1815751685 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f758c002a80 0x7f758c004f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:00.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:00.004+0000 7f75ba600640 1 -- 172.21.15.17:0/1815751685 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f758c007ed0 con 0x7f758c002a80 2024-09-10T21:42:00.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:00.008+0000 7f75b3400640 1 --2- 172.21.15.17:0/1815751685 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f758c002a80 0x7f758c004f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:00.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:00.008+0000 7f75b3400640 1 -- 172.21.15.17:0/1815751685 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f758c0080d0 con 0x7f758c002a80 2024-09-10T21:42:00.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:00.008+0000 7f75b3400640 1 --2- 172.21.15.17:0/1815751685 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f758c002a80 0x7f758c004f40 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f75980057f0 tx=0x7f75a0004ec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:00.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:00.008+0000 7f75ab400640 1 -- 172.21.15.17:0/1815751685 <== mon.1 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f75a4006720 con 0x7f75b4163440 2024-09-10T21:42:00.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:00.012+0000 7f75ab400640 1 --2- 172.21.15.17:0/1815751685 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f7594072990 0x7f7594074e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:00.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:00.012+0000 7f75ab400640 1 -- 172.21.15.17:0/1815751685 <== mon.1 v2:172.21.15.31:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f75a40082e0 con 0x7f75b4163440 2024-09-10T21:42:00.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:00.012+0000 7f75b3e00640 1 --2- 172.21.15.17:0/1815751685 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f7594072990 0x7f7594074e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:00.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:00.012+0000 7f75ab400640 1 -- 172.21.15.17:0/1815751685 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75a00ba020 con 0x7f758c002a80 2024-09-10T21:42:00.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:00.012+0000 7f75ab400640 1 -- 172.21.15.17:0/1815751685 <== mon.1 v2:172.21.15.31:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f75a0005040 con 0x7f758c002a80 2024-09-10T21:42:00.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:00.012+0000 7f75ab400640 1 -- 172.21.15.17:0/1815751685 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f758c002a80 msgr2=0x7f758c004f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:00.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:00.012+0000 7f75ab400640 1 --2- 172.21.15.17:0/1815751685 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f758c002a80 0x7f758c004f40 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f75980057f0 tx=0x7f75a0004ec0 comp rx=0 tx=0).stop 2024-09-10T21:42:00.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:00.012+0000 7f75ab400640 1 -- 172.21.15.17:0/1815751685 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f758c002a80 msgr2=0x7f758c004f40 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:42:00.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:00.012+0000 7f75ab400640 1 --2- 172.21.15.17:0/1815751685 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f758c002a80 0x7f758c004f40 secure :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f75980057f0 tx=0x7f75a0004ec0 comp rx=0 tx=0).stop 2024-09-10T21:42:00.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:00.012+0000 7f75b3e00640 1 --2- 172.21.15.17:0/1815751685 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f7594072990 0x7f7594074e50 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f75b40ccff0 tx=0x7f75ac00b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:00.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:00.024+0000 7f75ba600640 1 --2- 172.21.15.17:0/1815751685 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f758c006f10 0x7f758c0073a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:00.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:00.024+0000 7f75ba600640 1 -- 172.21.15.17:0/1815751685 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f758c00a4b0 con 0x7f758c006f10 2024-09-10T21:42:00.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:00.024+0000 7f75b3400640 1 --2- 172.21.15.17:0/1815751685 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f758c006f10 0x7f758c0073a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:00.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:00.028+0000 7f75b3400640 1 -- 172.21.15.17:0/1815751685 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f758c0078e0 con 0x7f758c006f10 2024-09-10T21:42:00.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:00.040+0000 7f75b3400640 1 --2- 172.21.15.17:0/1815751685 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f758c006f10 0x7f758c0073a0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f75a00b0040 tx=0x7f75a0005eb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:00.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:00.040+0000 7f75ab400640 1 -- 172.21.15.17:0/1815751685 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75a00ba030 con 0x7f758c006f10 2024-09-10T21:42:00.052 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:00.044+0000 7f75ab400640 1 -- 172.21.15.17:0/1815751685 <== mon.1 v2:172.21.15.31:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4867 (secure 0 0 0) 0x7f75a00ad440 con 0x7f758c006f10 2024-09-10T21:42:00.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:00.044+0000 7f75ab400640 1 -- 172.21.15.17:0/1815751685 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f758c006f10 msgr2=0x7f758c0073a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:00.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:00.044+0000 7f75ab400640 1 --2- 172.21.15.17:0/1815751685 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f758c006f10 0x7f758c0073a0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f75a00b0040 tx=0x7f75a0005eb0 comp rx=0 tx=0).stop 2024-09-10T21:42:00.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:00.044+0000 7f75ab400640 1 -- 172.21.15.17:0/1815751685 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f758c006f10 msgr2=0x7f758c0073a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:42:00.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:00.044+0000 7f75ab400640 1 --2- 172.21.15.17:0/1815751685 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f758c006f10 0x7f758c0073a0 secure :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f75a00b0040 tx=0x7f75a0005eb0 comp rx=0 tx=0).stop 2024-09-10T21:42:00.058 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-10T21:42:00.058 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:42:00.059 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:42:00.059 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "leader", 2024-09-10T21:42:00.059 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 134, 2024-09-10T21:42:00.059 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 131469, 2024-09-10T21:42:00.059 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-10T21:42:00.059 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-10T21:42:00.059 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-10T21:42:00.059 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:42:00.059 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 1, 2024-09-10T21:42:00.059 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:42:00.059 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-10T21:42:00.059 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-10T21:42:00.059 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:42:00.060 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:42:00.060 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:42:00.060 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:42:00.060 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:42:00.060 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:42:00.060 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:42:00.060 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:42:00.060 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:42:00.060 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:42:00.060 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:42:00.060 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:42:00.060 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-10T21:42:00.060 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-10T21:42:00.061 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:42:00.061 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:42:00.061 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:42:00.061 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:42:00.061 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:42:00.061 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:42:00.061 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:42:00.061 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:42:00.061 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:42:00.061 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:42:00.061 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:42:00.061 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:42:00.061 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:42:00.062 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-10T21:42:00.062 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-10T21:42:00.062 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-10T21:42:00.062 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-10T21:42:00.062 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-10T21:42:00.062 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "5e2f023f-d6a6-49f4-a256-f4807dce3739", 2024-09-10T21:42:00.062 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:42:00.062 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:42:00.062 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-10T21:42:00.062 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-10T21:42:00.062 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-10T21:42:00.062 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-10T21:42:00.062 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-10T21:42:00.062 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-10T21:42:00.063 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-10T21:42:00.063 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:42:00.063 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-10T21:42:00.063 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:42:00.063 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:42:00.063 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:42:00.063 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:42:00.063 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:42:00.063 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:42:00.063 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:42:00.063 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:42:00.063 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:42:00.063 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:42:00.064 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:42:00.064 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:42:00.064 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-10T21:42:00.064 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:42:00.064 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-10T21:42:00.064 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:42:00.064 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:42:00.064 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:42:00.064 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:42:00.064 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:42:00.064 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:42:00.064 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:42:00.064 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-10T21:42:00.064 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:42:00.065 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:42:00.065 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:42:00.065 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:42:00.065 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-10T21:42:00.065 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:42:00.065 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:42:00.065 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:42:00.065 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:42:00.065 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-10T21:42:00.065 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-10T21:42:00.065 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:42:00.065 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:42:00.065 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:42:00.066 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:42:00.066 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:42:00.066 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:42:00.066 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:42:00.066 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:42:00.066 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:42:00.066 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:42:00.066 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:42:00.066 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3300", 2024-09-10T21:42:00.066 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:42:00.066 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:42:00.066 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:42:00.066 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:42:00.066 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789", 2024-09-10T21:42:00.067 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:42:00.067 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:42:00.067 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:42:00.067 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:42:00.067 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789/0", 2024-09-10T21:42:00.067 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6789/0", 2024-09-10T21:42:00.067 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:42:00.067 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:42:00.067 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:42:00.067 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:42:00.067 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:42:00.067 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:42:00.067 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:42:00.067 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:42:00.068 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:42:00.068 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:42:00.068 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:42:00.068 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3301", 2024-09-10T21:42:00.068 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:42:00.068 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:42:00.068 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:42:00.068 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:42:00.068 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790", 2024-09-10T21:42:00.068 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:42:00.068 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:42:00.068 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:42:00.068 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:42:00.069 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790/0", 2024-09-10T21:42:00.069 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6790/0", 2024-09-10T21:42:00.069 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:42:00.069 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:42:00.069 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:42:00.069 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:42:00.069 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:42:00.069 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:42:00.069 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-10T21:42:00.069 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-10T21:42:00.069 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:42:00.069 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:42:00.069 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:42:00.070 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:42:00.070 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:42:00.070 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:42:00.070 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": [ 2024-09-10T21:42:00.070 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:42:00.070 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:42:00.070 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:42:00.070 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:42:00.070 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:42:00.070 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:42:00.070 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-10T21:42:00.070 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:42:00.070 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:42:00.070 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:42:00.071 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:42:00.071 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:42:00.071 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:42:00.071 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-10T21:42:00.071 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:42:00.071 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:42:00.071 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:42:00.071 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 3 2024-09-10T21:42:00.071 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:42:00.071 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:42:00.071 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:42:00.071 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-10T21:42:00.071 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-10T21:42:00.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:00.060+0000 7f75ba600640 1 -- 172.21.15.17:0/1815751685 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f7594072990 msgr2=0x7f7594074e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:00.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:00.060+0000 7f75ba600640 1 --2- 172.21.15.17:0/1815751685 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f7594072990 0x7f7594074e50 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f75b40ccff0 tx=0x7f75ac00b040 comp rx=0 tx=0).stop 2024-09-10T21:42:00.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:00.060+0000 7f75b3e00640 1 -- 172.21.15.17:0/1815751685 reap_dead start 2024-09-10T21:42:00.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:00.060+0000 7f75ba600640 1 -- 172.21.15.17:0/1815751685 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f75b4163440 msgr2=0x7f75b4160e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:00.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:00.060+0000 7f75ba600640 1 --2- 172.21.15.17:0/1815751685 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f75b4163440 0x7f75b4160e20 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f75a4000fc0 tx=0x7f75a400f820 comp rx=0 tx=0).stop 2024-09-10T21:42:00.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:00.060+0000 7f75ba600640 1 -- 172.21.15.17:0/1815751685 shutdown_connections 2024-09-10T21:42:00.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:00.060+0000 7f75ba600640 1 --2- 172.21.15.17:0/1815751685 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f75b4163440 0x7f75b4160e20 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:00.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:00.060+0000 7f75ba600640 1 -- 172.21.15.17:0/1815751685 >> 172.21.15.17:0/1815751685 conn(0x7f75b40700d0 msgr2=0x7f75b4150190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:00.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:00.064+0000 7f75ba600640 1 -- 172.21.15.17:0/1815751685 shutdown_connections 2024-09-10T21:42:00.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:00.064+0000 7f75ba600640 1 -- 172.21.15.17:0/1815751685 wait complete. 2024-09-10T21:42:00.111 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-09-10T21:42:00.112 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon c 2024-09-10T21:42:00.112 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-10T21:42:00.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:00.472+0000 7f75eba00640 1 -- 172.21.15.17:0/3310094369 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f75e40c7c10 msgr2=0x7f75e4150d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:00.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:00.472+0000 7f75eba00640 1 --2- 172.21.15.17:0/3310094369 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f75e40c7c10 0x7f75e4150d30 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f75d000aa10 tx=0x7f75d001da70 comp rx=0 tx=0).stop 2024-09-10T21:42:00.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:00.472+0000 7f75eba00640 1 -- 172.21.15.17:0/3310094369 shutdown_connections 2024-09-10T21:42:00.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:00.472+0000 7f75eba00640 1 --2- 172.21.15.17:0/3310094369 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f75e40c7c10 0x7f75e4150d30 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:00.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:00.472+0000 7f75eba00640 1 --2- 172.21.15.17:0/3310094369 >> v2:172.21.15.17:3300/0 conn(0x7f75e40c7830 0x7f75e41507f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:00.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:00.472+0000 7f75eba00640 1 -- 172.21.15.17:0/3310094369 >> 172.21.15.17:0/3310094369 conn(0x7f75e406f210 msgr2=0x7f75e406cc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:00.481 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:00.476+0000 7f75eba00640 1 -- 172.21.15.17:0/3310094369 shutdown_connections 2024-09-10T21:42:00.481 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:00.476+0000 7f75eba00640 1 -- 172.21.15.17:0/3310094369 wait complete. 2024-09-10T21:42:00.482 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:00.476+0000 7f75eba00640 1 Processor -- start 2024-09-10T21:42:00.483 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:00.476+0000 7f75eba00640 1 -- start start 2024-09-10T21:42:00.484 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:00.480+0000 7f75eba00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f75e40c7830 0x7f75e40c49c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:00.484 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:00.480+0000 7f75eba00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f75e40c7c10 0x7f75e40c5520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:00.484 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:00.480+0000 7f75eba00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f75e4156e00 0x7f75e4143f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:00.484 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:00.480+0000 7f75eba00640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f75e40cc7e0 con 0x7f75e40c7c10 2024-09-10T21:42:00.484 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:00.480+0000 7f75eba00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f75e40cc660 con 0x7f75e4156e00 2024-09-10T21:42:00.484 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:00.480+0000 7f75eba00640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f75e40cc960 con 0x7f75e40c7830 2024-09-10T21:42:00.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:00.480+0000 7f75e9a00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f75e4156e00 0x7f75e4143f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:00.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:00.480+0000 7f75e9a00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f75e4156e00 0x7f75e4143f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.17:51862/0 (socket says 172.21.15.17:51862) 2024-09-10T21:42:00.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:00.480+0000 7f75e9a00640 1 -- 172.21.15.17:0/2474976238 learned_addr learned my addr 172.21.15.17:0/2474976238 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:42:00.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:00.480+0000 7f75e9a00640 1 -- 172.21.15.17:0/2474976238 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f75e40c7830 msgr2=0x7f75e40c49c0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-10T21:42:00.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:00.480+0000 7f75e9a00640 1 --2- 172.21.15.17:0/2474976238 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f75e40c7830 0x7f75e40c49c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:00.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:00.480+0000 7f75e9a00640 1 -- 172.21.15.17:0/2474976238 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f75e40c7c10 msgr2=0x7f75e40c5520 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:42:00.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:00.480+0000 7f75e3e00640 1 -- 172.21.15.17:0/2474976238 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f75e40c7c10 msgr2=0x7f75e40c5520 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-10T21:42:00.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:00.480+0000 7f75e3e00640 1 --2- 172.21.15.17:0/2474976238 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f75e40c7c10 0x7f75e40c5520 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:42:00.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:00.480+0000 7f75e9a00640 1 --2- 172.21.15.17:0/2474976238 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f75e40c7c10 0x7f75e40c5520 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:00.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:00.480+0000 7f75e9a00640 1 -- 172.21.15.17:0/2474976238 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75e40c5a60 con 0x7f75e4156e00 2024-09-10T21:42:00.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:00.480+0000 7f75e9a00640 1 --2- 172.21.15.17:0/2474976238 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f75e4156e00 0x7f75e4143f40 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f75dc00b9e0 tx=0x7f75dc00bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:00.488 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:00.480+0000 7f75e0c00640 1 -- 172.21.15.17:0/2474976238 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75dc019070 con 0x7f75e4156e00 2024-09-10T21:42:00.488 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:00.484+0000 7f75e0c00640 1 -- 172.21.15.17:0/2474976238 <== mon.1 v2:172.21.15.31:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f75dc00cc10 con 0x7f75e4156e00 2024-09-10T21:42:00.488 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:00.484+0000 7f75e0c00640 1 -- 172.21.15.17:0/2474976238 <== mon.1 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75dc014440 con 0x7f75e4156e00 2024-09-10T21:42:00.488 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:00.484+0000 7f75eba00640 1 -- 172.21.15.17:0/2474976238 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75e4146110 con 0x7f75e4156e00 2024-09-10T21:42:00.488 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:00.484+0000 7f75eba00640 1 -- 172.21.15.17:0/2474976238 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75e4146690 con 0x7f75e4156e00 2024-09-10T21:42:00.490 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:00.484+0000 7f75eba00640 1 --2- 172.21.15.17:0/2474976238 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f75e41f7910 0x7f75e4147510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:00.490 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:00.484+0000 7f75eba00640 1 -- 172.21.15.17:0/2474976238 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f75e4146970 con 0x7f75e41f7910 2024-09-10T21:42:00.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:00.488+0000 7f75e9000640 1 --2- 172.21.15.17:0/2474976238 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f75e41f7910 0x7f75e4147510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:00.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:00.488+0000 7f75e0c00640 1 -- 172.21.15.17:0/2474976238 <== mon.1 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f75dc01d3b0 con 0x7f75e4156e00 2024-09-10T21:42:00.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:00.488+0000 7f75e0c00640 1 --2- 172.21.15.17:0/2474976238 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f75c40725e0 0x7f75c4074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:00.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:00.488+0000 7f75e0c00640 1 -- 172.21.15.17:0/2474976238 <== mon.1 v2:172.21.15.31:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f75dc093620 con 0x7f75e4156e00 2024-09-10T21:42:00.495 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:00.488+0000 7f75e9000640 1 -- 172.21.15.17:0/2474976238 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f75e40ccf00 con 0x7f75e41f7910 2024-09-10T21:42:00.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:00.488+0000 7f75e9000640 1 --2- 172.21.15.17:0/2474976238 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f75e41f7910 0x7f75e4147510 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f75d80010b0 tx=0x7f75d800b780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:00.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:00.488+0000 7f75e3e00640 1 --2- 172.21.15.17:0/2474976238 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f75c40725e0 0x7f75c4074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:00.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:00.492+0000 7f75e0c00640 1 -- 172.21.15.17:0/2474976238 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75d8015020 con 0x7f75e41f7910 2024-09-10T21:42:00.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:00.492+0000 7f75e0c00640 1 -- 172.21.15.17:0/2474976238 <== mon.2 v2:172.21.15.31:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f75d8011830 con 0x7f75e41f7910 2024-09-10T21:42:00.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:00.492+0000 7f75e0c00640 1 -- 172.21.15.17:0/2474976238 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f75e41f7910 msgr2=0x7f75e4147510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:00.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:00.492+0000 7f75e0c00640 1 --2- 172.21.15.17:0/2474976238 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f75e41f7910 0x7f75e4147510 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f75d80010b0 tx=0x7f75d800b780 comp rx=0 tx=0).stop 2024-09-10T21:42:00.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:00.492+0000 7f75e0c00640 1 -- 172.21.15.17:0/2474976238 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f75e41f7910 msgr2=0x7f75e4147510 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:42:00.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:00.492+0000 7f75e0c00640 1 --2- 172.21.15.17:0/2474976238 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f75e41f7910 0x7f75e4147510 secure :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f75d80010b0 tx=0x7f75d800b780 comp rx=0 tx=0).stop 2024-09-10T21:42:00.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:00.492+0000 7f75e3e00640 1 --2- 172.21.15.17:0/2474976238 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f75c40725e0 0x7f75c4074aa0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f75d00b0000 tx=0x7f75d00a8040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:00.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:00.512+0000 7f75eba00640 1 --2- 172.21.15.17:0/2474976238 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f75e41f6d90 0x7f75e41f71a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:00.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:00.512+0000 7f75eba00640 1 -- 172.21.15.17:0/2474976238 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f75e4149cc0 con 0x7f75e41f6d90 2024-09-10T21:42:00.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:00.516+0000 7f75e9000640 1 --2- 172.21.15.17:0/2474976238 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f75e41f6d90 0x7f75e41f71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:00.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:00.516+0000 7f75e9000640 1 -- 172.21.15.17:0/2474976238 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f75e41499e0 con 0x7f75e41f6d90 2024-09-10T21:42:00.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:00.520+0000 7f75e9000640 1 --2- 172.21.15.17:0/2474976238 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f75e41f6d90 0x7f75e41f71a0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f75e4076430 tx=0x7f75d8004810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:00.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:00.520+0000 7f75e0c00640 1 -- 172.21.15.17:0/2474976238 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75d8015040 con 0x7f75e41f6d90 2024-09-10T21:42:00.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:00.520+0000 7f75e0c00640 1 -- 172.21.15.17:0/2474976238 <== mon.2 v2:172.21.15.31:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5032 (secure 0 0 0) 0x7f75d801a630 con 0x7f75e41f6d90 2024-09-10T21:42:00.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:00.520+0000 7f75e0c00640 1 -- 172.21.15.17:0/2474976238 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f75e41f6d90 msgr2=0x7f75e41f71a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:00.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:00.520+0000 7f75e0c00640 1 --2- 172.21.15.17:0/2474976238 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f75e41f6d90 0x7f75e41f71a0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f75e4076430 tx=0x7f75d8004810 comp rx=0 tx=0).stop 2024-09-10T21:42:00.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:00.520+0000 7f75e0c00640 1 -- 172.21.15.17:0/2474976238 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f75e41f6d90 msgr2=0x7f75e41f71a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:42:00.526 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:00.520+0000 7f75e0c00640 1 --2- 172.21.15.17:0/2474976238 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f75e41f6d90 0x7f75e41f71a0 secure :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f75e4076430 tx=0x7f75d8004810 comp rx=0 tx=0).stop 2024-09-10T21:42:00.527 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-10T21:42:00.527 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:42:00.527 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:42:00.527 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-10T21:42:00.527 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 134, 2024-09-10T21:42:00.527 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 326870, 2024-09-10T21:42:00.527 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-10T21:42:00.528 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-10T21:42:00.528 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-10T21:42:00.528 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:42:00.528 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 1, 2024-09-10T21:42:00.528 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:42:00.528 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-10T21:42:00.528 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-10T21:42:00.528 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:42:00.528 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:42:00.528 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:42:00.528 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:42:00.528 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:42:00.529 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:42:00.529 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:42:00.529 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:42:00.529 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:42:00.529 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:42:00.529 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:42:00.529 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:42:00.529 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-10T21:42:00.529 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-10T21:42:00.529 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:42:00.529 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:42:00.529 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:42:00.529 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:42:00.530 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:42:00.530 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:42:00.530 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:42:00.530 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:42:00.530 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:42:00.530 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:42:00.530 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:42:00.530 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:42:00.530 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:42:00.530 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-10T21:42:00.530 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-10T21:42:00.530 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-10T21:42:00.530 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-10T21:42:00.530 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-10T21:42:00.531 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "5e2f023f-d6a6-49f4-a256-f4807dce3739", 2024-09-10T21:42:00.531 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:42:00.531 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:42:00.531 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-10T21:42:00.531 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-10T21:42:00.531 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-10T21:42:00.531 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-10T21:42:00.531 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-10T21:42:00.532 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-10T21:42:00.532 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-10T21:42:00.532 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:42:00.532 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-10T21:42:00.532 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:42:00.532 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:42:00.532 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:42:00.532 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:42:00.532 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:42:00.533 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:42:00.533 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:42:00.533 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:42:00.533 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:42:00.533 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:42:00.533 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:42:00.533 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:42:00.533 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-10T21:42:00.533 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:42:00.533 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-10T21:42:00.533 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:42:00.533 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:42:00.533 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:42:00.534 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:42:00.534 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:42:00.534 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:42:00.534 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:42:00.534 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-10T21:42:00.534 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:42:00.534 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:42:00.534 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:42:00.534 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:42:00.534 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-10T21:42:00.534 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:42:00.534 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:42:00.534 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:42:00.535 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:42:00.535 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-10T21:42:00.535 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-10T21:42:00.535 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:42:00.535 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:42:00.535 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:42:00.535 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:42:00.535 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:42:00.535 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:42:00.535 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:42:00.535 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:42:00.535 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:42:00.535 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:42:00.536 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:42:00.536 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3300", 2024-09-10T21:42:00.536 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:42:00.536 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:42:00.536 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:42:00.536 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:42:00.536 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789", 2024-09-10T21:42:00.536 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:42:00.536 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:42:00.536 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:42:00.537 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:42:00.537 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789/0", 2024-09-10T21:42:00.537 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6789/0", 2024-09-10T21:42:00.537 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:42:00.537 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:42:00.537 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:42:00.537 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:42:00.537 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:42:00.537 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:42:00.537 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:42:00.537 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:42:00.537 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:42:00.537 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:42:00.538 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:42:00.538 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3301", 2024-09-10T21:42:00.538 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:42:00.538 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:42:00.538 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:42:00.538 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:42:00.538 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790", 2024-09-10T21:42:00.538 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:42:00.538 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:42:00.538 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:42:00.538 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:42:00.538 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790/0", 2024-09-10T21:42:00.538 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6790/0", 2024-09-10T21:42:00.539 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:42:00.539 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:42:00.539 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:42:00.539 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:42:00.539 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:42:00.539 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:42:00.539 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-10T21:42:00.539 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-10T21:42:00.539 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:42:00.539 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:42:00.539 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:42:00.539 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:42:00.539 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:42:00.540 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:42:00.540 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": [ 2024-09-10T21:42:00.540 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:42:00.540 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:42:00.540 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:42:00.540 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 5 2024-09-10T21:42:00.540 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:42:00.540 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:42:00.540 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-10T21:42:00.540 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:42:00.540 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:42:00.540 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:42:00.540 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 7 2024-09-10T21:42:00.541 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:42:00.541 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:42:00.541 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-10T21:42:00.541 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:42:00.541 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:42:00.541 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:42:00.541 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 5 2024-09-10T21:42:00.541 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:42:00.541 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:42:00.541 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": [ 2024-09-10T21:42:00.541 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:42:00.541 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:42:00.541 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:42:00.541 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 2 2024-09-10T21:42:00.542 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:42:00.542 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:42:00.542 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:42:00.542 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-10T21:42:00.542 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-10T21:42:00.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:00.520+0000 7f75eba00640 1 -- 172.21.15.17:0/2474976238 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f75c40725e0 msgr2=0x7f75c4074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:00.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:00.520+0000 7f75eba00640 1 --2- 172.21.15.17:0/2474976238 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f75c40725e0 0x7f75c4074aa0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f75d00b0000 tx=0x7f75d00a8040 comp rx=0 tx=0).stop 2024-09-10T21:42:00.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:00.520+0000 7f75eba00640 1 -- 172.21.15.17:0/2474976238 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f75e4156e00 msgr2=0x7f75e4143f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:00.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:00.520+0000 7f75eba00640 1 --2- 172.21.15.17:0/2474976238 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f75e4156e00 0x7f75e4143f40 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f75dc00b9e0 tx=0x7f75dc00bea0 comp rx=0 tx=0).stop 2024-09-10T21:42:00.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:00.520+0000 7f75e9a00640 1 -- 172.21.15.17:0/2474976238 reap_dead start 2024-09-10T21:42:00.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:00.520+0000 7f75e9a00640 1 -- 172.21.15.17:0/2474976238 reap_dead start 2024-09-10T21:42:00.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:00.520+0000 7f75eba00640 1 -- 172.21.15.17:0/2474976238 shutdown_connections 2024-09-10T21:42:00.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:00.520+0000 7f75eba00640 1 -- 172.21.15.17:0/2474976238 >> 172.21.15.17:0/2474976238 conn(0x7f75e406f210 msgr2=0x7f75e41505c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:00.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:00.524+0000 7f75eba00640 1 -- 172.21.15.17:0/2474976238 shutdown_connections 2024-09-10T21:42:00.543 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:00.524+0000 7f75eba00640 1 -- 172.21.15.17:0/2474976238 wait complete. 2024-09-10T21:42:00.564 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-09-10T21:42:00.565 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2024-09-10T21:42:20.565 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.a 2024-09-10T21:42:20.566 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-10T21:42:20.566 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-10T21:42:20.570 INFO:tasks.ceph.mon.a:Started 2024-09-10T21:42:20.570 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-09-10T21:42:20.570 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-10T21:42:20.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:20.776+0000 7f960fa00640 1 -- 172.21.15.17:0/3666822045 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f96080c8d10 msgr2=0x7f96080c1ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:20.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:20.776+0000 7f960fa00640 1 --2- 172.21.15.17:0/3666822045 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f96080c8d10 0x7f96080c1ae0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f960806f1e0 tx=0x7f95fc01f010 comp rx=0 tx=0).stop 2024-09-10T21:42:20.784 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:20.780+0000 7f960fa00640 1 -- 172.21.15.17:0/3666822045 shutdown_connections 2024-09-10T21:42:20.785 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:20.780+0000 7f960fa00640 1 --2- 172.21.15.17:0/3666822045 >> v2:172.21.15.17:3300/0 conn(0x7f96080c90f0 0x7f9608074910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:20.785 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:20.780+0000 7f960fa00640 1 --2- 172.21.15.17:0/3666822045 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f96080c8d10 0x7f96080c1ae0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:20.785 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:20.780+0000 7f960fa00640 1 -- 172.21.15.17:0/3666822045 >> 172.21.15.17:0/3666822045 conn(0x7f9608070160 msgr2=0x7f9608070570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:20.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:20.788+0000 7f960fa00640 1 -- 172.21.15.17:0/3666822045 shutdown_connections 2024-09-10T21:42:20.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:20.796+0000 7f960fa00640 1 -- 172.21.15.17:0/3666822045 wait complete. 2024-09-10T21:42:20.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:20.796+0000 7f960fa00640 1 Processor -- start 2024-09-10T21:42:20.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:20.804+0000 7f960fa00640 1 -- start start 2024-09-10T21:42:20.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:20.812+0000 7f960fa00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f96080c7c10 0x7f96080c49b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:20.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:20.812+0000 7f960fa00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f96080c8d10 0x7f9608161cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:20.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:20.812+0000 7f960fa00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f96080c90f0 0x7f960820eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:20.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:20.812+0000 7f960d000640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f96080c7c10 0x7f96080c49b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:20.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:20.812+0000 7f960d000640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f96080c7c10 0x7f96080c49b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3301/0 says I am v2:172.21.15.17:51396/0 (socket says 172.21.15.17:51396) 2024-09-10T21:42:20.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:20.812+0000 7f960da00640 1 -- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f96080c90f0 msgr2=0x7f960820eac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-10T21:42:20.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:20.812+0000 7f960da00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f96080c90f0 0x7f960820eac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:42:20.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:20.812+0000 7f960d000640 1 -- 172.21.15.17:0/976757980 learned_addr learned my addr 172.21.15.17:0/976757980 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:42:20.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:20.812+0000 7f960fa00640 1 -- 172.21.15.17:0/976757980 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f960806e640 con 0x7f96080c90f0 2024-09-10T21:42:20.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:20.812+0000 7f960fa00640 1 -- 172.21.15.17:0/976757980 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f960806e360 con 0x7f96080c8d10 2024-09-10T21:42:20.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:20.812+0000 7f960fa00640 1 -- 172.21.15.17:0/976757980 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f96080c87c0 con 0x7f96080c7c10 2024-09-10T21:42:20.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:20.816+0000 7f960d000640 1 -- 172.21.15.17:0/976757980 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f96080c8d10 msgr2=0x7f9608161cc0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-10T21:42:20.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:20.816+0000 7f960d000640 1 --2- 172.21.15.17:0/976757980 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f96080c8d10 0x7f9608161cc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:20.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:20.816+0000 7f960d000640 1 -- 172.21.15.17:0/976757980 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f96080c90f0 msgr2=0x7f960820eac0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:42:20.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:20.816+0000 7f960d000640 1 --2- 172.21.15.17:0/976757980 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f96080c90f0 0x7f960820eac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:20.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:20.816+0000 7f960d000640 1 -- 172.21.15.17:0/976757980 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96080c3d30 con 0x7f96080c7c10 2024-09-10T21:42:20.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:20.816+0000 7f960d000640 1 --2- 172.21.15.17:0/976757980 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f96080c7c10 0x7f96080c49b0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f95fc000c10 tx=0x7f95fc0a8d90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:20.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:20.816+0000 7f9604c00640 1 -- 172.21.15.17:0/976757980 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95fc0b5040 con 0x7f96080c7c10 2024-09-10T21:42:20.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:20.816+0000 7f9604c00640 1 -- 172.21.15.17:0/976757980 <== mon.2 v2:172.21.15.31:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f95fc0bf070 con 0x7f96080c7c10 2024-09-10T21:42:20.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:20.816+0000 7f9604c00640 1 -- 172.21.15.17:0/976757980 <== mon.2 v2:172.21.15.31:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95fc0b1b80 con 0x7f96080c7c10 2024-09-10T21:42:20.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:20.820+0000 7f960fa00640 1 -- 172.21.15.17:0/976757980 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96080c3490 con 0x7f96080c7c10 2024-09-10T21:42:20.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:20.820+0000 7f960fa00640 1 -- 172.21.15.17:0/976757980 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f96080c3a50 con 0x7f96080c7c10 2024-09-10T21:42:20.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:20.824+0000 7f9604c00640 1 -- 172.21.15.17:0/976757980 <== mon.2 v2:172.21.15.31:3301/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f95fc0b8030 con 0x7f96080c7c10 2024-09-10T21:42:20.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:20.828+0000 7f9604c00640 1 --2- 172.21.15.17:0/976757980 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f95e80726b0 0x7f95e8074b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:20.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:20.828+0000 7f9604c00640 1 -- 172.21.15.17:0/976757980 <== mon.2 v2:172.21.15.31:3301/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f95fc1308e0 con 0x7f96080c7c10 2024-09-10T21:42:20.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:20.828+0000 7f960fa00640 1 -- 172.21.15.17:0/976757980 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9608072cb0 con 0x7f96080c7c10 2024-09-10T21:42:20.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:20.828+0000 7f9607e00640 1 --2- 172.21.15.17:0/976757980 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f95e80726b0 0x7f95e8074b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:20.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:20.836+0000 7f9607e00640 1 --2- 172.21.15.17:0/976757980 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f95e80726b0 0x7f95e8074b70 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f9608162630 tx=0x7f95f4006c90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:20.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:20.844+0000 7f9604c00640 1 -- 172.21.15.17:0/976757980 <== mon.2 v2:172.21.15.31:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f95fc0c8020 con 0x7f96080c7c10 2024-09-10T21:42:21.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:21.272+0000 7f960fa00640 1 -- 172.21.15.17:0/976757980 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f96080c79f0 con 0x7f96080c7c10 2024-09-10T21:42:21.278 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":134,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":22,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5e2f023f-d6a6-49f4-a256-f4807dce3739","modified":"2024-09-10T21:23:46.892130Z","created":"2024-09-10T21:23:46.892130Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3301","nonce":0},{"type":"v1","addr":"172.21.15.31:6790","nonce":0}]},"addr":"172.21.15.31:6790/0","public_addr":"172.21.15.31:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-10T21:42:21.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:21.272+0000 7f9604c00640 1 -- 172.21.15.17:0/976757980 <== mon.2 v2:172.21.15.31:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1552 (secure 0 0 0) 0x7f95fc0c8020 con 0x7f96080c7c10 2024-09-10T21:42:21.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:21.276+0000 7f960fa00640 1 -- 172.21.15.17:0/976757980 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f95e80726b0 msgr2=0x7f95e8074b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:21.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:21.276+0000 7f960fa00640 1 --2- 172.21.15.17:0/976757980 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f95e80726b0 0x7f95e8074b70 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f9608162630 tx=0x7f95f4006c90 comp rx=0 tx=0).stop 2024-09-10T21:42:21.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:21.276+0000 7f960fa00640 1 -- 172.21.15.17:0/976757980 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f96080c7c10 msgr2=0x7f96080c49b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:21.488 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:21.276+0000 7f960fa00640 1 --2- 172.21.15.17:0/976757980 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f96080c7c10 0x7f96080c49b0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f95fc000c10 tx=0x7f95fc0a8d90 comp rx=0 tx=0).stop 2024-09-10T21:42:21.488 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:21.276+0000 7f960fa00640 1 -- 172.21.15.17:0/976757980 shutdown_connections 2024-09-10T21:42:21.488 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:21.276+0000 7f960fa00640 1 --2- 172.21.15.17:0/976757980 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f95e80726b0 0x7f95e8074b70 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:21.488 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:21.276+0000 7f960fa00640 1 --2- 172.21.15.17:0/976757980 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f96080c90f0 0x7f960820eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:21.488 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:21.276+0000 7f960fa00640 1 --2- 172.21.15.17:0/976757980 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f96080c8d10 0x7f9608161cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:21.488 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:21.276+0000 7f960fa00640 1 --2- 172.21.15.17:0/976757980 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f96080c7c10 0x7f96080c49b0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:21.488 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:21.276+0000 7f960fa00640 1 -- 172.21.15.17:0/976757980 >> 172.21.15.17:0/976757980 conn(0x7f9608070160 msgr2=0x7f96080ca6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:21.488 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:21.276+0000 7f960fa00640 1 -- 172.21.15.17:0/976757980 shutdown_connections 2024-09-10T21:42:21.488 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:21.276+0000 7f960fa00640 1 -- 172.21.15.17:0/976757980 wait complete. 2024-09-10T21:42:23.060 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:42:23.065 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:42:24.555 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-10T21:42:24.806 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:24.800+0000 7f5352e00640 1 -- 172.21.15.17:0/3683902821 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f534c1515f0 msgr2=0x7f534c157a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:24.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:24.800+0000 7f5352e00640 1 --2- 172.21.15.17:0/3683902821 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f534c1515f0 0x7f534c157a40 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f5338009840 tx=0x7f533801c860 comp rx=0 tx=0).stop 2024-09-10T21:42:24.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:24.800+0000 7f5352e00640 1 -- 172.21.15.17:0/3683902821 shutdown_connections 2024-09-10T21:42:24.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:24.800+0000 7f5352e00640 1 --2- 172.21.15.17:0/3683902821 >> v2:172.21.15.31:3300/0 conn(0x7f534c0725c0 0x7f534c072a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:24.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:24.800+0000 7f5352e00640 1 --2- 172.21.15.17:0/3683902821 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f534c1515f0 0x7f534c157a40 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:24.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:24.800+0000 7f5352e00640 1 -- 172.21.15.17:0/3683902821 >> 172.21.15.17:0/3683902821 conn(0x7f534c070160 msgr2=0x7f534c070570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:24.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:24.800+0000 7f5352e00640 1 -- 172.21.15.17:0/3683902821 shutdown_connections 2024-09-10T21:42:24.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:24.800+0000 7f5352e00640 1 -- 172.21.15.17:0/3683902821 wait complete. 2024-09-10T21:42:24.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:24.804+0000 7f5352e00640 1 Processor -- start 2024-09-10T21:42:24.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:24.804+0000 7f5352e00640 1 -- start start 2024-09-10T21:42:24.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:24.804+0000 7f5352e00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f534c0725c0 0x7f534c14af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:24.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:24.804+0000 7f5352e00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f534c1515f0 0x7f534c0c3e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:24.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:24.804+0000 7f5352e00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f534c157f80 0x7f534c1f3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:24.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:24.804+0000 7f5352e00640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f534c157520 con 0x7f534c0725c0 2024-09-10T21:42:24.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:24.804+0000 7f5352e00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f534c0c3080 con 0x7f534c157f80 2024-09-10T21:42:24.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:24.808+0000 7f5352e00640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f534c0c1750 con 0x7f534c1515f0 2024-09-10T21:42:24.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:24.808+0000 7f534b400640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f534c1515f0 0x7f534c0c3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:24.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:24.808+0000 7f534be00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f534c0725c0 0x7f534c14af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:24.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:24.808+0000 7f534be00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f534c0725c0 0x7f534c14af40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:60936/0 (socket says 172.21.15.17:60936) 2024-09-10T21:42:24.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:24.808+0000 7f534be00640 1 -- 172.21.15.17:0/1423388637 learned_addr learned my addr 172.21.15.17:0/1423388637 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:42:24.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:24.808+0000 7f534be00640 1 -- 172.21.15.17:0/1423388637 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f534c1515f0 msgr2=0x7f534c0c3e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:24.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:24.808+0000 7f534be00640 1 --2- 172.21.15.17:0/1423388637 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f534c1515f0 0x7f534c0c3e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:24.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:24.808+0000 7f534be00640 1 -- 172.21.15.17:0/1423388637 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f534c157f80 msgr2=0x7f534c1f3690 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-10T21:42:24.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:24.808+0000 7f534be00640 1 --2- 172.21.15.17:0/1423388637 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f534c157f80 0x7f534c1f3690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:24.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:24.808+0000 7f534be00640 1 -- 172.21.15.17:0/1423388637 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f534c14b480 con 0x7f534c0725c0 2024-09-10T21:42:24.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:24.812+0000 7f534be00640 1 --2- 172.21.15.17:0/1423388637 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f534c0725c0 0x7f534c14af40 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f5338019470 tx=0x7f5338005040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:24.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:24.812+0000 7f5343e00640 1 -- 172.21.15.17:0/1423388637 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53380b2040 con 0x7f534c0725c0 2024-09-10T21:42:24.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:24.816+0000 7f5343e00640 1 -- 172.21.15.17:0/1423388637 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f53380055e0 con 0x7f534c0725c0 2024-09-10T21:42:24.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:24.816+0000 7f5343e00640 1 -- 172.21.15.17:0/1423388637 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5338004910 con 0x7f534c0725c0 2024-09-10T21:42:24.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:24.816+0000 7f5352e00640 1 -- 172.21.15.17:0/1423388637 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f534c1438d0 con 0x7f534c0725c0 2024-09-10T21:42:24.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:24.816+0000 7f5352e00640 1 -- 172.21.15.17:0/1423388637 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f534c1f3f60 con 0x7f534c0725c0 2024-09-10T21:42:24.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:24.816+0000 7f5352e00640 1 -- 172.21.15.17:0/1423388637 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f534c0c7fe0 con 0x7f534c0725c0 2024-09-10T21:42:24.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:24.832+0000 7f5343e00640 1 -- 172.21.15.17:0/1423388637 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f53380c0020 con 0x7f534c0725c0 2024-09-10T21:42:24.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:24.832+0000 7f5343e00640 1 --2- 172.21.15.17:0/1423388637 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f532c072630 0x7f532c074af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:24.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:24.840+0000 7f5343e00640 1 -- 172.21.15.17:0/1423388637 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f53380f46e0 con 0x7f534c0725c0 2024-09-10T21:42:24.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:24.840+0000 7f5343e00640 1 -- 172.21.15.17:0/1423388637 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f534c14b480 con 0x7f534c0725c0 2024-09-10T21:42:24.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:24.844+0000 7f534b400640 1 --2- 172.21.15.17:0/1423388637 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f532c072630 0x7f532c074af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:24.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:24.848+0000 7f534b400640 1 --2- 172.21.15.17:0/1423388637 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f532c072630 0x7f532c074af0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f534c1502b0 tx=0x7f533c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:25.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:25.416+0000 7f5352e00640 1 -- 172.21.15.17:0/1423388637 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f534c1451f0 con 0x7f534c0725c0 2024-09-10T21:42:25.423 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:25.416+0000 7f5343e00640 1 -- 172.21.15.17:0/1423388637 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1557 (secure 0 0 0) 0x7f5338016450 con 0x7f534c0725c0 2024-09-10T21:42:25.423 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":138,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5e2f023f-d6a6-49f4-a256-f4807dce3739","modified":"2024-09-10T21:23:46.892130Z","created":"2024-09-10T21:23:46.892130Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3301","nonce":0},{"type":"v1","addr":"172.21.15.31:6790","nonce":0}]},"addr":"172.21.15.31:6790/0","public_addr":"172.21.15.31:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-10T21:42:25.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:25.420+0000 7f5352e00640 1 -- 172.21.15.17:0/1423388637 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f532c072630 msgr2=0x7f532c074af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:25.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:25.420+0000 7f5352e00640 1 --2- 172.21.15.17:0/1423388637 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f532c072630 0x7f532c074af0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f534c1502b0 tx=0x7f533c009040 comp rx=0 tx=0).stop 2024-09-10T21:42:25.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:25.420+0000 7f5352e00640 1 -- 172.21.15.17:0/1423388637 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f534c0725c0 msgr2=0x7f534c14af40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:25.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:25.420+0000 7f5352e00640 1 --2- 172.21.15.17:0/1423388637 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f534c0725c0 0x7f534c14af40 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f5338019470 tx=0x7f5338005040 comp rx=0 tx=0).stop 2024-09-10T21:42:25.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:25.420+0000 7f5352e00640 1 -- 172.21.15.17:0/1423388637 shutdown_connections 2024-09-10T21:42:25.428 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:25.420+0000 7f5352e00640 1 --2- 172.21.15.17:0/1423388637 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f532c072630 0x7f532c074af0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:25.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:25.420+0000 7f5352e00640 1 --2- 172.21.15.17:0/1423388637 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f534c157f80 0x7f534c1f3690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:25.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:25.420+0000 7f5352e00640 1 --2- 172.21.15.17:0/1423388637 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f534c1515f0 0x7f534c0c3e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:25.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:25.420+0000 7f5352e00640 1 --2- 172.21.15.17:0/1423388637 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f534c0725c0 0x7f534c14af40 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:25.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:25.420+0000 7f5352e00640 1 -- 172.21.15.17:0/1423388637 >> 172.21.15.17:0/1423388637 conn(0x7f534c070160 msgr2=0x7f534c14ee20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:25.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:25.424+0000 7f5352e00640 1 -- 172.21.15.17:0/1423388637 shutdown_connections 2024-09-10T21:42:25.429 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:25.424+0000 7f5352e00640 1 -- 172.21.15.17:0/1423388637 wait complete. 2024-09-10T21:42:25.474 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-09-10T21:42:25.474 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-09-10T21:42:25.474 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-10T21:42:25.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:25.888+0000 7fc494600640 1 -- 172.21.15.17:0/1632930829 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc48c0c0fd0 msgr2=0x7fc48c0c6cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:25.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:25.888+0000 7fc494600640 1 --2- 172.21.15.17:0/1632930829 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc48c0c0fd0 0x7fc48c0c6cd0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fc47800afd0 tx=0x7fc47801ea90 comp rx=0 tx=0).stop 2024-09-10T21:42:25.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:25.900+0000 7fc494600640 1 -- 172.21.15.17:0/1632930829 shutdown_connections 2024-09-10T21:42:25.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:25.900+0000 7fc494600640 1 --2- 172.21.15.17:0/1632930829 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc48c0c0fd0 0x7fc48c0c6cd0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:25.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:25.900+0000 7fc494600640 1 --2- 172.21.15.17:0/1632930829 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc48c153d40 0x7fc48c0c8010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:25.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:25.900+0000 7fc494600640 1 -- 172.21.15.17:0/1632930829 >> 172.21.15.17:0/1632930829 conn(0x7fc48c0700d0 msgr2=0x7fc48c0704e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:25.911 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:25.904+0000 7fc494600640 1 -- 172.21.15.17:0/1632930829 shutdown_connections 2024-09-10T21:42:25.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:25.908+0000 7fc494600640 1 -- 172.21.15.17:0/1632930829 wait complete. 2024-09-10T21:42:25.921 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:25.912+0000 7fc494600640 1 Processor -- start 2024-09-10T21:42:25.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:25.932+0000 7fc494600640 1 -- start start 2024-09-10T21:42:25.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:25.932+0000 7fc494600640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc48c0c0fd0 0x7fc48c0cb3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:25.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:25.932+0000 7fc494600640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc48c0c7210 0x7fc48c1dd180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:25.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:25.932+0000 7fc494600640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc48c153d40 0x7fc48c140830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:25.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:25.932+0000 7fc494600640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fc48c0c13b0 con 0x7fc48c0c0fd0 2024-09-10T21:42:25.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:25.932+0000 7fc494600640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fc48c0c95a0 con 0x7fc48c153d40 2024-09-10T21:42:25.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:25.932+0000 7fc494600640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7fc48c154120 con 0x7fc48c0c7210 2024-09-10T21:42:25.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:25.932+0000 7fc492600640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc48c153d40 0x7fc48c140830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:25.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:25.932+0000 7fc491c00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc48c0c0fd0 0x7fc48c0cb3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:25.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:25.932+0000 7fc491c00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc48c0c0fd0 0x7fc48c0cb3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:60952/0 (socket says 172.21.15.17:60952) 2024-09-10T21:42:25.939 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:25.932+0000 7fc491c00640 1 -- 172.21.15.17:0/4036217964 learned_addr learned my addr 172.21.15.17:0/4036217964 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:42:25.939 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:25.932+0000 7fc491200640 1 --2- 172.21.15.17:0/4036217964 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc48c0c7210 0x7fc48c1dd180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:25.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:25.936+0000 7fc491c00640 1 -- 172.21.15.17:0/4036217964 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc48c0c7210 msgr2=0x7fc48c1dd180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:25.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:25.936+0000 7fc491c00640 1 --2- 172.21.15.17:0/4036217964 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc48c0c7210 0x7fc48c1dd180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:25.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:25.936+0000 7fc491c00640 1 -- 172.21.15.17:0/4036217964 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc48c153d40 msgr2=0x7fc48c140830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:25.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:25.936+0000 7fc491c00640 1 --2- 172.21.15.17:0/4036217964 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc48c153d40 0x7fc48c140830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:25.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:25.936+0000 7fc491c00640 1 -- 172.21.15.17:0/4036217964 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc48c1e26e0 con 0x7fc48c0c0fd0 2024-09-10T21:42:25.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:25.936+0000 7fc492600640 1 --2- 172.21.15.17:0/4036217964 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc48c153d40 0x7fc48c140830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:42:25.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:25.936+0000 7fc491c00640 1 --2- 172.21.15.17:0/4036217964 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc48c0c0fd0 0x7fc48c0cb3a0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fc48000d950 tx=0x7fc48000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:25.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:25.936+0000 7fc489600640 1 -- 172.21.15.17:0/4036217964 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc480017070 con 0x7fc48c0c0fd0 2024-09-10T21:42:25.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:25.936+0000 7fc494600640 1 -- 172.21.15.17:0/4036217964 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc48c1de2c0 con 0x7fc48c0c0fd0 2024-09-10T21:42:25.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:25.936+0000 7fc489600640 1 -- 172.21.15.17:0/4036217964 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc480009d20 con 0x7fc48c0c0fd0 2024-09-10T21:42:25.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:25.936+0000 7fc494600640 1 -- 172.21.15.17:0/4036217964 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc48c1ddfe0 con 0x7fc48c0c0fd0 2024-09-10T21:42:25.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:25.936+0000 7fc489600640 1 -- 172.21.15.17:0/4036217964 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4800106c0 con 0x7fc48c0c0fd0 2024-09-10T21:42:25.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:25.936+0000 7fc494600640 1 --2- 172.21.15.17:0/4036217964 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc48c1dd6c0 0x7fc48c1e2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:25.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:25.936+0000 7fc494600640 1 -- 172.21.15.17:0/4036217964 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fc48c1ef890 con 0x7fc48c1dd6c0 2024-09-10T21:42:25.943 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:25.936+0000 7fc491200640 1 --2- 172.21.15.17:0/4036217964 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc48c1dd6c0 0x7fc48c1e2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:25.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:25.940+0000 7fc491200640 1 -- 172.21.15.17:0/4036217964 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc48c0747d0 con 0x7fc48c1dd6c0 2024-09-10T21:42:25.946 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:25.940+0000 7fc491200640 1 --2- 172.21.15.17:0/4036217964 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc48c1dd6c0 0x7fc48c1e2080 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fc48c0b2530 tx=0x7fc4780040d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:25.948 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:25.944+0000 7fc489600640 1 -- 172.21.15.17:0/4036217964 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fc480010860 con 0x7fc48c0c0fd0 2024-09-10T21:42:25.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:25.944+0000 7fc489600640 1 --2- 172.21.15.17:0/4036217964 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fc46c072610 0x7fc46c074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:25.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:25.944+0000 7fc489600640 1 -- 172.21.15.17:0/4036217964 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc478005230 con 0x7fc48c1dd6c0 2024-09-10T21:42:25.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:25.944+0000 7fc489600640 1 -- 172.21.15.17:0/4036217964 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fc480093ae0 con 0x7fc48c0c0fd0 2024-09-10T21:42:25.950 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:25.944+0000 7fc489600640 1 -- 172.21.15.17:0/4036217964 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fc4780053d0 con 0x7fc48c1dd6c0 2024-09-10T21:42:25.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:25.944+0000 7fc489600640 1 -- 172.21.15.17:0/4036217964 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc48c1dd6c0 msgr2=0x7fc48c1e2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:25.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:25.944+0000 7fc489600640 1 --2- 172.21.15.17:0/4036217964 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc48c1dd6c0 0x7fc48c1e2080 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fc48c0b2530 tx=0x7fc4780040d0 comp rx=0 tx=0).stop 2024-09-10T21:42:25.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:25.944+0000 7fc492600640 1 --2- 172.21.15.17:0/4036217964 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fc46c072610 0x7fc46c074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:25.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:25.944+0000 7fc489600640 1 -- 172.21.15.17:0/4036217964 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc48c1dd6c0 msgr2=0x7fc48c1e2080 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:42:25.951 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:25.944+0000 7fc489600640 1 --2- 172.21.15.17:0/4036217964 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc48c1dd6c0 0x7fc48c1e2080 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:25.957 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:25.948+0000 7fc492600640 1 --2- 172.21.15.17:0/4036217964 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fc46c072610 0x7fc46c074ad0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fc484001580 tx=0x7fc484001510 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:25.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:25.964+0000 7fc494600640 1 --2- 172.21.15.17:0/4036217964 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc48c140d70 0x7fc48c14dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:25.969 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:25.964+0000 7fc494600640 1 -- 172.21.15.17:0/4036217964 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fc48c151930 con 0x7fc48c140d70 2024-09-10T21:42:25.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:25.964+0000 7fc491200640 1 --2- 172.21.15.17:0/4036217964 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc48c140d70 0x7fc48c14dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:25.971 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:25.964+0000 7fc491200640 1 -- 172.21.15.17:0/4036217964 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fc48c0c78a0 con 0x7fc48c140d70 2024-09-10T21:42:25.972 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:25.968+0000 7fc491200640 1 --2- 172.21.15.17:0/4036217964 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc48c140d70 0x7fc48c14dff0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fc47801e8c0 tx=0x7fc47800ab80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:25.973 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:25.968+0000 7fc489600640 1 -- 172.21.15.17:0/4036217964 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc478005230 con 0x7fc48c140d70 2024-09-10T21:42:25.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:25.972+0000 7fc489600640 1 -- 172.21.15.17:0/4036217964 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4542 (secure 0 0 0) 0x7fc4780c3290 con 0x7fc48c140d70 2024-09-10T21:42:25.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:25.972+0000 7fc489600640 1 -- 172.21.15.17:0/4036217964 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc48c140d70 msgr2=0x7fc48c14dff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:25.984 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-10T21:42:25.984 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:42:25.984 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:42:25.984 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "leader", 2024-09-10T21:42:25.984 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 138, 2024-09-10T21:42:25.984 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 5103, 2024-09-10T21:42:25.984 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-10T21:42:25.985 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-10T21:42:25.985 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-10T21:42:25.985 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-10T21:42:25.985 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:42:25.985 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 1, 2024-09-10T21:42:25.985 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:42:25.985 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-10T21:42:25.985 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-10T21:42:25.985 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:42:25.985 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:42:25.985 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:42:25.985 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:42:25.985 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:42:25.986 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:42:25.986 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:42:25.986 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:42:25.986 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:42:25.986 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:42:25.986 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:42:25.986 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:42:25.986 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-10T21:42:25.986 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-10T21:42:25.986 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:42:25.986 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:42:25.986 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:42:25.986 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:42:25.987 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:42:25.987 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:42:25.987 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:42:25.987 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:42:25.987 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:42:25.987 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:42:25.987 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:42:25.987 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:42:25.987 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:42:25.987 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-10T21:42:25.987 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-10T21:42:25.987 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-10T21:42:25.988 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-10T21:42:25.988 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-10T21:42:25.988 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "5e2f023f-d6a6-49f4-a256-f4807dce3739", 2024-09-10T21:42:25.988 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:42:25.988 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:42:25.988 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-10T21:42:25.988 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-10T21:42:25.988 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-10T21:42:25.988 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-10T21:42:25.988 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-10T21:42:25.988 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-10T21:42:25.988 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-10T21:42:25.989 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:42:25.989 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-10T21:42:25.989 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:42:25.989 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:42:25.989 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:42:25.989 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:42:25.989 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:42:25.989 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:42:25.989 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:42:25.989 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:42:25.989 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:42:25.989 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:42:25.990 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:42:25.990 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:42:25.990 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-10T21:42:25.990 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:42:25.990 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-10T21:42:25.990 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:42:25.990 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:42:25.990 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:42:25.990 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:42:25.990 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:42:25.990 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:42:25.990 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:42:25.991 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-10T21:42:25.991 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:42:25.991 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:42:25.991 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:42:25.991 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:42:25.991 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-10T21:42:25.991 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:42:25.991 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:42:25.991 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:42:25.991 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:42:25.991 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-10T21:42:25.991 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-10T21:42:25.992 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:42:25.992 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:42:25.992 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:42:25.992 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:42:25.992 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:42:25.992 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:42:25.992 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:42:25.992 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:42:25.992 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:42:25.992 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:42:25.992 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:42:25.992 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3300", 2024-09-10T21:42:25.993 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:42:25.993 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:42:25.993 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:42:25.993 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:42:25.993 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789", 2024-09-10T21:42:25.993 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:42:25.993 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:42:25.993 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:42:25.993 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:42:25.993 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789/0", 2024-09-10T21:42:25.993 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6789/0", 2024-09-10T21:42:25.993 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:42:25.994 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:42:25.994 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:42:25.994 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:42:25.994 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:42:25.994 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:42:25.994 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:42:25.994 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:42:25.994 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:42:25.994 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:42:25.994 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:42:25.994 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3301", 2024-09-10T21:42:25.994 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:42:25.994 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:42:25.995 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:42:25.995 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:42:25.995 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790", 2024-09-10T21:42:25.995 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:42:25.995 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:42:25.995 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:42:25.995 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:42:25.995 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790/0", 2024-09-10T21:42:25.995 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6790/0", 2024-09-10T21:42:25.995 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:42:25.995 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:42:25.995 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:42:25.995 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:42:25.995 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:42:25.996 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:42:25.996 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-10T21:42:25.996 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-10T21:42:25.996 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:42:25.996 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:42:25.996 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:42:25.996 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:42:25.996 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:42:25.996 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:42:25.996 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-10T21:42:25.996 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:42:25.996 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:42:25.996 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:42:25.997 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 2 2024-09-10T21:42:25.997 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:42:25.997 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:42:25.997 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:42:25.997 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-10T21:42:25.997 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-10T21:42:25.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:25.980+0000 7fc489600640 1 --2- 172.21.15.17:0/4036217964 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc48c140d70 0x7fc48c14dff0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fc47801e8c0 tx=0x7fc47800ab80 comp rx=0 tx=0).stop 2024-09-10T21:42:25.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:25.980+0000 7fc489600640 1 -- 172.21.15.17:0/4036217964 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc48c140d70 msgr2=0x7fc48c14dff0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:42:25.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:25.980+0000 7fc489600640 1 --2- 172.21.15.17:0/4036217964 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc48c140d70 0x7fc48c14dff0 secure :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fc47801e8c0 tx=0x7fc47800ab80 comp rx=0 tx=0).stop 2024-09-10T21:42:25.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:25.984+0000 7fc494600640 1 -- 172.21.15.17:0/4036217964 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fc46c072610 msgr2=0x7fc46c074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:25.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:25.984+0000 7fc494600640 1 --2- 172.21.15.17:0/4036217964 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fc46c072610 0x7fc46c074ad0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fc484001580 tx=0x7fc484001510 comp rx=0 tx=0).stop 2024-09-10T21:42:25.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:25.984+0000 7fc494600640 1 -- 172.21.15.17:0/4036217964 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc48c0c0fd0 msgr2=0x7fc48c0cb3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:25.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:25.984+0000 7fc494600640 1 --2- 172.21.15.17:0/4036217964 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc48c0c0fd0 0x7fc48c0cb3a0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fc48000d950 tx=0x7fc48000de10 comp rx=0 tx=0).stop 2024-09-10T21:42:25.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:25.984+0000 7fc492600640 1 -- 172.21.15.17:0/4036217964 reap_dead start 2024-09-10T21:42:25.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:25.984+0000 7fc492600640 1 -- 172.21.15.17:0/4036217964 reap_dead start 2024-09-10T21:42:25.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:25.984+0000 7fc494600640 1 -- 172.21.15.17:0/4036217964 shutdown_connections 2024-09-10T21:42:25.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:25.984+0000 7fc494600640 1 -- 172.21.15.17:0/4036217964 >> 172.21.15.17:0/4036217964 conn(0x7fc48c0700d0 msgr2=0x7fc48c150560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:25.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:25.984+0000 7fc494600640 1 -- 172.21.15.17:0/4036217964 shutdown_connections 2024-09-10T21:42:25.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:25.988+0000 7fc494600640 1 -- 172.21.15.17:0/4036217964 wait complete. 2024-09-10T21:42:26.052 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-10T21:42:26.052 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-09-10T21:42:26.053 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-10T21:42:26.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:26.320+0000 7f956ee00640 1 -- 172.21.15.17:0/4086001287 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f956004c220 msgr2=0x7f95600ddb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:26.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:26.320+0000 7f956ee00640 1 --2- 172.21.15.17:0/4086001287 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f956004c220 0x7f95600ddb70 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f95540098b0 tx=0x7f955401c690 comp rx=0 tx=0).stop 2024-09-10T21:42:26.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:26.324+0000 7f956ee00640 1 -- 172.21.15.17:0/4086001287 shutdown_connections 2024-09-10T21:42:26.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:26.324+0000 7f956ee00640 1 --2- 172.21.15.17:0/4086001287 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f956004c220 0x7f95600ddb70 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:26.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:26.324+0000 7f956ee00640 1 -- 172.21.15.17:0/4086001287 >> 172.21.15.17:0/4086001287 conn(0x7f956004fde0 msgr2=0x7f95600501f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:26.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:26.324+0000 7f956ee00640 1 -- 172.21.15.17:0/4086001287 shutdown_connections 2024-09-10T21:42:26.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:26.324+0000 7f956ee00640 1 -- 172.21.15.17:0/4086001287 wait complete. 2024-09-10T21:42:26.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:26.328+0000 7f956ee00640 1 Processor -- start 2024-09-10T21:42:26.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:26.328+0000 7f956ee00640 1 -- start start 2024-09-10T21:42:26.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:26.332+0000 7f956ee00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f956004c220 0x7f95600eb070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:26.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:26.332+0000 7f956d000640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f956004c220 0x7f95600eb070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:26.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:26.332+0000 7f956d000640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f956004c220 0x7f95600eb070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:60980/0 (socket says 172.21.15.17:60980) 2024-09-10T21:42:26.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:26.332+0000 7f956d000640 1 -- 172.21.15.17:0/1998306869 learned_addr learned my addr 172.21.15.17:0/1998306869 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:42:26.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:26.332+0000 7f956ee00640 1 --2- 172.21.15.17:0/1998306869 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f95600d4570 0x7f95600e8990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:26.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:26.332+0000 7f9567e00640 1 --2- 172.21.15.17:0/1998306869 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f95600d4570 0x7f95600e8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:26.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:26.332+0000 7f956ee00640 1 --2- 172.21.15.17:0/1998306869 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f95600de0b0 0x7f95600e8ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:26.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:26.332+0000 7f956ee00640 1 -- 172.21.15.17:0/1998306869 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f95600d4950 con 0x7f956004c220 2024-09-10T21:42:26.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:26.332+0000 7f956ee00640 1 -- 172.21.15.17:0/1998306869 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f95600dc180 con 0x7f95600d4570 2024-09-10T21:42:26.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:26.332+0000 7f956ee00640 1 -- 172.21.15.17:0/1998306869 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f956004afa0 con 0x7f95600de0b0 2024-09-10T21:42:26.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:26.332+0000 7f956da00640 1 --2- 172.21.15.17:0/1998306869 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f95600de0b0 0x7f95600e8ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:26.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:26.336+0000 7f956da00640 1 -- 172.21.15.17:0/1998306869 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f95600d4570 msgr2=0x7f95600e8990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:26.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:26.336+0000 7f956da00640 1 --2- 172.21.15.17:0/1998306869 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f95600d4570 0x7f95600e8990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:26.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:26.336+0000 7f956da00640 1 -- 172.21.15.17:0/1998306869 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f956004c220 msgr2=0x7f95600eb070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:26.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:26.336+0000 7f956da00640 1 --2- 172.21.15.17:0/1998306869 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f956004c220 0x7f95600eb070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:26.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:26.336+0000 7f956da00640 1 -- 172.21.15.17:0/1998306869 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95600db720 con 0x7f95600de0b0 2024-09-10T21:42:26.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:26.336+0000 7f956da00640 1 --2- 172.21.15.17:0/1998306869 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f95600de0b0 0x7f95600e8ed0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f954c00c700 tx=0x7f954c00cbc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:26.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:26.336+0000 7f9565600640 1 -- 172.21.15.17:0/1998306869 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f954c017070 con 0x7f95600de0b0 2024-09-10T21:42:26.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:26.336+0000 7f9565600640 1 -- 172.21.15.17:0/1998306869 <== mon.2 v2:172.21.15.31:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f954c005360 con 0x7f95600de0b0 2024-09-10T21:42:26.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:26.336+0000 7f956d000640 1 --2- 172.21.15.17:0/1998306869 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f956004c220 0x7f95600eb070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:42:26.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:26.336+0000 7f956ee00640 1 -- 172.21.15.17:0/1998306869 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9560034a70 con 0x7f95600de0b0 2024-09-10T21:42:26.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:26.336+0000 7f956ee00640 1 -- 172.21.15.17:0/1998306869 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f9560035c80 con 0x7f95600de0b0 2024-09-10T21:42:26.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:26.336+0000 7f9565600640 1 -- 172.21.15.17:0/1998306869 <== mon.2 v2:172.21.15.31:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f954c00a8a0 con 0x7f95600de0b0 2024-09-10T21:42:26.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:26.340+0000 7f956ee00640 1 --2- 172.21.15.17:0/1998306869 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f95600ea860 0x7f95600481c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:26.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:26.340+0000 7f956ee00640 1 -- 172.21.15.17:0/1998306869 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f95600de980 con 0x7f95600ea860 2024-09-10T21:42:26.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:26.340+0000 7f9565600640 1 -- 172.21.15.17:0/1998306869 <== mon.2 v2:172.21.15.31:3301/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f954c00aa40 con 0x7f95600de0b0 2024-09-10T21:42:26.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:26.340+0000 7f9565600640 1 --2- 172.21.15.17:0/1998306869 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f9548072610 0x7f9548074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:26.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:26.340+0000 7f9565600640 1 -- 172.21.15.17:0/1998306869 <== mon.2 v2:172.21.15.31:3301/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f954c092db0 con 0x7f95600de0b0 2024-09-10T21:42:26.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:26.344+0000 7f956d000640 1 --2- 172.21.15.17:0/1998306869 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f95600ea860 0x7f95600481c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:26.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:26.344+0000 7f9567e00640 1 --2- 172.21.15.17:0/1998306869 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f9548072610 0x7f9548074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:26.349 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:26.344+0000 7f956d000640 1 -- 172.21.15.17:0/1998306869 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9560033b80 con 0x7f95600ea860 2024-09-10T21:42:26.350 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:26.344+0000 7f956d000640 1 --2- 172.21.15.17:0/1998306869 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f95600ea860 0x7f95600481c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f955c00eaa0 tx=0x7f955c00ef60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:26.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:26.344+0000 7f9565600640 1 -- 172.21.15.17:0/1998306869 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f955c003b90 con 0x7f95600ea860 2024-09-10T21:42:26.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:26.344+0000 7f9567e00640 1 --2- 172.21.15.17:0/1998306869 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f9548072610 0x7f9548074ad0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f95540ae000 tx=0x7f955401aed0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:26.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:26.344+0000 7f9565600640 1 -- 172.21.15.17:0/1998306869 <== mon.1 v2:172.21.15.31:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f955c018070 con 0x7f95600ea860 2024-09-10T21:42:26.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:26.344+0000 7f9565600640 1 -- 172.21.15.17:0/1998306869 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f95600ea860 msgr2=0x7f95600481c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:26.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:26.344+0000 7f9565600640 1 --2- 172.21.15.17:0/1998306869 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f95600ea860 0x7f95600481c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f955c00eaa0 tx=0x7f955c00ef60 comp rx=0 tx=0).stop 2024-09-10T21:42:26.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:26.344+0000 7f9565600640 1 -- 172.21.15.17:0/1998306869 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f95600ea860 msgr2=0x7f95600481c0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:42:26.351 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:26.344+0000 7f9565600640 1 --2- 172.21.15.17:0/1998306869 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f95600ea860 0x7f95600481c0 secure :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f955c00eaa0 tx=0x7f955c00ef60 comp rx=0 tx=0).stop 2024-09-10T21:42:26.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:26.360+0000 7f956ee00640 1 --2- 172.21.15.17:0/1998306869 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9560048700 0x7f9560184940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:26.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:26.360+0000 7f956ee00640 1 -- 172.21.15.17:0/1998306869 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f9560005c20 con 0x7f9560048700 2024-09-10T21:42:26.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:26.360+0000 7f956d000640 1 --2- 172.21.15.17:0/1998306869 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9560048700 0x7f9560184940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:26.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:26.364+0000 7f956d000640 1 -- 172.21.15.17:0/1998306869 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f956004bd80 con 0x7f9560048700 2024-09-10T21:42:26.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:26.368+0000 7f956d000640 1 --2- 172.21.15.17:0/1998306869 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9560048700 0x7f9560184940 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f955c014000 tx=0x7f955c001370 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:26.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:26.368+0000 7f9565600640 1 -- 172.21.15.17:0/1998306869 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f955c0054e0 con 0x7f9560048700 2024-09-10T21:42:26.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:26.368+0000 7f9565600640 1 -- 172.21.15.17:0/1998306869 <== mon.1 v2:172.21.15.31:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4876 (secure 0 0 0) 0x7f955c005680 con 0x7f9560048700 2024-09-10T21:42:26.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:26.368+0000 7f9565600640 1 -- 172.21.15.17:0/1998306869 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9560048700 msgr2=0x7f9560184940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:26.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:26.368+0000 7f9565600640 1 --2- 172.21.15.17:0/1998306869 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9560048700 0x7f9560184940 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f955c014000 tx=0x7f955c001370 comp rx=0 tx=0).stop 2024-09-10T21:42:26.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:26.368+0000 7f9565600640 1 -- 172.21.15.17:0/1998306869 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9560048700 msgr2=0x7f9560184940 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:42:26.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:26.368+0000 7f9565600640 1 --2- 172.21.15.17:0/1998306869 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9560048700 0x7f9560184940 secure :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f955c014000 tx=0x7f955c001370 comp rx=0 tx=0).stop 2024-09-10T21:42:26.374 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-10T21:42:26.375 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:42:26.375 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:42:26.375 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-10T21:42:26.375 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 138, 2024-09-10T21:42:26.375 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 157797, 2024-09-10T21:42:26.375 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-10T21:42:26.375 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-10T21:42:26.375 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-10T21:42:26.375 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-10T21:42:26.375 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:42:26.375 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 1, 2024-09-10T21:42:26.375 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:42:26.376 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-10T21:42:26.376 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-10T21:42:26.376 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:42:26.376 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:42:26.376 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:42:26.376 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:42:26.376 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:42:26.376 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:42:26.376 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:42:26.376 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:42:26.376 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:42:26.376 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:42:26.377 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:42:26.377 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:42:26.377 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-10T21:42:26.377 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-10T21:42:26.377 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:42:26.377 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:42:26.377 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:42:26.377 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:42:26.377 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:42:26.377 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:42:26.377 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:42:26.377 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:42:26.377 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:42:26.377 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:42:26.378 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:42:26.378 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:42:26.378 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:42:26.378 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-10T21:42:26.378 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-10T21:42:26.378 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-10T21:42:26.378 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-10T21:42:26.378 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-10T21:42:26.378 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "5e2f023f-d6a6-49f4-a256-f4807dce3739", 2024-09-10T21:42:26.378 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:42:26.378 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:42:26.378 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-10T21:42:26.378 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-10T21:42:26.378 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-10T21:42:26.379 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-10T21:42:26.379 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-10T21:42:26.379 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-10T21:42:26.379 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-10T21:42:26.379 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:42:26.379 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-10T21:42:26.379 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:42:26.379 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:42:26.379 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:42:26.379 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:42:26.379 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:42:26.379 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:42:26.379 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:42:26.379 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:42:26.379 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:42:26.380 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:42:26.380 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:42:26.380 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:42:26.380 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-10T21:42:26.380 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:42:26.380 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-10T21:42:26.380 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:42:26.380 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:42:26.380 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:42:26.380 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:42:26.380 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:42:26.380 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:42:26.380 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:42:26.380 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-10T21:42:26.381 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:42:26.381 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:42:26.381 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:42:26.381 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:42:26.381 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-10T21:42:26.381 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:42:26.381 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:42:26.381 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:42:26.381 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:42:26.381 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-10T21:42:26.381 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-10T21:42:26.381 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:42:26.381 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:42:26.381 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:42:26.381 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:42:26.382 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:42:26.382 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:42:26.382 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:42:26.382 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:42:26.382 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:42:26.382 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:42:26.382 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:42:26.382 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3300", 2024-09-10T21:42:26.382 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:42:26.382 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:42:26.382 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:42:26.382 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:42:26.382 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789", 2024-09-10T21:42:26.382 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:42:26.383 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:42:26.383 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:42:26.383 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:42:26.383 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789/0", 2024-09-10T21:42:26.383 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6789/0", 2024-09-10T21:42:26.383 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:42:26.383 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:42:26.383 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:42:26.383 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:42:26.383 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:42:26.383 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:42:26.383 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:42:26.383 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:42:26.384 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:42:26.384 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:42:26.384 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:42:26.384 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3301", 2024-09-10T21:42:26.384 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:42:26.384 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:42:26.384 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:42:26.384 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:42:26.384 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790", 2024-09-10T21:42:26.384 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:42:26.384 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:42:26.384 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:42:26.385 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:42:26.385 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790/0", 2024-09-10T21:42:26.385 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6790/0", 2024-09-10T21:42:26.385 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:42:26.385 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:42:26.385 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:42:26.385 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:42:26.385 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:42:26.385 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:42:26.385 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-10T21:42:26.385 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-10T21:42:26.385 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:42:26.385 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:42:26.386 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:42:26.386 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:42:26.386 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:42:26.386 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:42:26.386 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": [ 2024-09-10T21:42:26.386 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:42:26.386 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:42:26.386 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:42:26.386 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:42:26.386 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:42:26.386 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:42:26.386 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-10T21:42:26.386 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:42:26.387 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:42:26.387 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:42:26.387 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:42:26.387 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:42:26.387 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:42:26.387 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-10T21:42:26.387 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:42:26.387 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:42:26.387 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:42:26.387 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 2 2024-09-10T21:42:26.387 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:42:26.388 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:42:26.388 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:42:26.388 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-10T21:42:26.388 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-10T21:42:26.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:26.368+0000 7f956ee00640 1 -- 172.21.15.17:0/1998306869 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f9548072610 msgr2=0x7f9548074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:26.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:26.368+0000 7f956ee00640 1 --2- 172.21.15.17:0/1998306869 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f9548072610 0x7f9548074ad0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f95540ae000 tx=0x7f955401aed0 comp rx=0 tx=0).stop 2024-09-10T21:42:26.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:26.368+0000 7f956ee00640 1 -- 172.21.15.17:0/1998306869 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f95600de0b0 msgr2=0x7f95600e8ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:26.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:26.368+0000 7f956ee00640 1 --2- 172.21.15.17:0/1998306869 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f95600de0b0 0x7f95600e8ed0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f954c00c700 tx=0x7f954c00cbc0 comp rx=0 tx=0).stop 2024-09-10T21:42:26.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:26.372+0000 7f956da00640 1 -- 172.21.15.17:0/1998306869 reap_dead start 2024-09-10T21:42:26.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:26.372+0000 7f956da00640 1 -- 172.21.15.17:0/1998306869 reap_dead start 2024-09-10T21:42:26.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:26.372+0000 7f956ee00640 1 -- 172.21.15.17:0/1998306869 shutdown_connections 2024-09-10T21:42:26.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:26.372+0000 7f956ee00640 1 -- 172.21.15.17:0/1998306869 >> 172.21.15.17:0/1998306869 conn(0x7f956004fde0 msgr2=0x7f95600d22b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:26.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:26.372+0000 7f956ee00640 1 -- 172.21.15.17:0/1998306869 shutdown_connections 2024-09-10T21:42:26.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:26.372+0000 7f956ee00640 1 -- 172.21.15.17:0/1998306869 wait complete. 2024-09-10T21:42:26.440 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-10T21:42:26.440 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-09-10T21:42:26.440 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-10T21:42:26.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:26.800+0000 7f46d5800640 1 -- 172.21.15.17:0/1303243238 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f46d00cc030 msgr2=0x7f46d00b3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:26.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:26.800+0000 7f46d5800640 1 --2- 172.21.15.17:0/1303243238 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f46d00cc030 0x7f46d00b3690 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f46bc009900 tx=0x7f46bc01f090 comp rx=0 tx=0).stop 2024-09-10T21:42:26.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:26.800+0000 7f46d5800640 1 -- 172.21.15.17:0/1303243238 shutdown_connections 2024-09-10T21:42:26.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:26.800+0000 7f46d5800640 1 --2- 172.21.15.17:0/1303243238 >> v2:172.21.15.17:3300/0 conn(0x7f46d00c5c80 0x7f46d00c6800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:26.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:26.800+0000 7f46d5800640 1 --2- 172.21.15.17:0/1303243238 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f46d00cc030 0x7f46d00b3690 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:26.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:26.800+0000 7f46d5800640 1 -- 172.21.15.17:0/1303243238 >> 172.21.15.17:0/1303243238 conn(0x7f46d0068990 msgr2=0x7f46d0068da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:26.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:26.804+0000 7f46d5800640 1 -- 172.21.15.17:0/1303243238 shutdown_connections 2024-09-10T21:42:26.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:26.804+0000 7f46d5800640 1 -- 172.21.15.17:0/1303243238 wait complete. 2024-09-10T21:42:26.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:26.804+0000 7f46d5800640 1 Processor -- start 2024-09-10T21:42:26.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:26.804+0000 7f46d5800640 1 -- start start 2024-09-10T21:42:26.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:26.808+0000 7f46d5800640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f46d00c5c80 0x7f46d0159a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:26.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:26.808+0000 7f46d5800640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f46d00cc030 0x7f46d01535b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:26.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:26.808+0000 7f46d5800640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46d00cc410 0x7f46d0153af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:26.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:26.808+0000 7f46d5800640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f46d0074cc0 con 0x7f46d00cc410 2024-09-10T21:42:26.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:26.808+0000 7f46d5800640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f46d00749e0 con 0x7f46d00cc030 2024-09-10T21:42:26.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:26.808+0000 7f46d5800640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f46d00c5480 con 0x7f46d00c5c80 2024-09-10T21:42:26.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:26.808+0000 7f46ce000640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f46d00cc030 0x7f46d01535b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:26.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:26.808+0000 7f46ce000640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f46d00cc030 0x7f46d01535b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.17:39894/0 (socket says 172.21.15.17:39894) 2024-09-10T21:42:26.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:26.808+0000 7f46ce000640 1 -- 172.21.15.17:0/2033753935 learned_addr learned my addr 172.21.15.17:0/2033753935 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:42:26.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:26.808+0000 7f46cea00640 1 --2- 172.21.15.17:0/2033753935 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f46d00c5c80 0x7f46d0159a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:26.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:26.808+0000 7f46cf400640 1 --2- 172.21.15.17:0/2033753935 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46d00cc410 0x7f46d0153af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:26.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:26.808+0000 7f46cea00640 1 -- 172.21.15.17:0/2033753935 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f46d00cc030 msgr2=0x7f46d01535b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:26.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:26.808+0000 7f46cea00640 1 --2- 172.21.15.17:0/2033753935 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f46d00cc030 0x7f46d01535b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:26.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:26.808+0000 7f46cea00640 1 -- 172.21.15.17:0/2033753935 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46d00cc410 msgr2=0x7f46d0153af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:26.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:26.808+0000 7f46cea00640 1 --2- 172.21.15.17:0/2033753935 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f46d00cc410 0x7f46d0153af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:26.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:26.808+0000 7f46cea00640 1 -- 172.21.15.17:0/2033753935 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46d00c81a0 con 0x7f46d00c5c80 2024-09-10T21:42:26.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:26.808+0000 7f46cea00640 1 --2- 172.21.15.17:0/2033753935 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f46d00c5c80 0x7f46d0159a80 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f46bc0af000 tx=0x7f46bc0a8d60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:26.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:26.808+0000 7f46c6a00640 1 -- 172.21.15.17:0/2033753935 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46bc0b5040 con 0x7f46d00c5c80 2024-09-10T21:42:26.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:26.808+0000 7f46c6a00640 1 -- 172.21.15.17:0/2033753935 <== mon.2 v2:172.21.15.31:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f46bc0039b0 con 0x7f46d00c5c80 2024-09-10T21:42:26.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:26.808+0000 7f46c6a00640 1 -- 172.21.15.17:0/2033753935 <== mon.2 v2:172.21.15.31:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46bc003de0 con 0x7f46d00c5c80 2024-09-10T21:42:26.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:26.808+0000 7f46d5800640 1 -- 172.21.15.17:0/2033753935 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46d00ca330 con 0x7f46d00c5c80 2024-09-10T21:42:26.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:26.808+0000 7f46d5800640 1 -- 172.21.15.17:0/2033753935 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f46d00c8420 con 0x7f46d00c5c80 2024-09-10T21:42:26.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:26.812+0000 7f46d5800640 1 --2- 172.21.15.17:0/2033753935 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f46d0159330 0x7f46d00c8660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:26.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:26.812+0000 7f46d5800640 1 -- 172.21.15.17:0/2033753935 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f46d00ca0e0 con 0x7f46d0159330 2024-09-10T21:42:26.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:26.812+0000 7f46ce000640 1 --2- 172.21.15.17:0/2033753935 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f46d0159330 0x7f46d00c8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:26.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:26.812+0000 7f46ce000640 1 -- 172.21.15.17:0/2033753935 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f46d00c6800 con 0x7f46d0159330 2024-09-10T21:42:26.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:26.812+0000 7f46c6a00640 1 -- 172.21.15.17:0/2033753935 <== mon.2 v2:172.21.15.31:3301/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f46bc0b8070 con 0x7f46d00c5c80 2024-09-10T21:42:26.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:26.812+0000 7f46ce000640 1 --2- 172.21.15.17:0/2033753935 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f46d0159330 0x7f46d00c8660 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f46d015d7c0 tx=0x7f46c000fc00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:26.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:26.816+0000 7f46c6a00640 1 --2- 172.21.15.17:0/2033753935 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f46b00725e0 0x7f46b0074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:26.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:26.816+0000 7f46c6a00640 1 -- 172.21.15.17:0/2033753935 <== mon.2 v2:172.21.15.31:3301/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f46bc130690 con 0x7f46d00c5c80 2024-09-10T21:42:26.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:26.816+0000 7f46c6a00640 1 -- 172.21.15.17:0/2033753935 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46c00050d0 con 0x7f46d0159330 2024-09-10T21:42:26.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:26.816+0000 7f46c6a00640 1 -- 172.21.15.17:0/2033753935 <== mon.2 v2:172.21.15.31:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f46c0018070 con 0x7f46d0159330 2024-09-10T21:42:26.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:26.816+0000 7f46c6a00640 1 -- 172.21.15.17:0/2033753935 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f46d0159330 msgr2=0x7f46d00c8660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:26.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:26.816+0000 7f46c6a00640 1 --2- 172.21.15.17:0/2033753935 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f46d0159330 0x7f46d00c8660 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f46d015d7c0 tx=0x7f46c000fc00 comp rx=0 tx=0).stop 2024-09-10T21:42:26.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:26.816+0000 7f46c6a00640 1 -- 172.21.15.17:0/2033753935 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f46d0159330 msgr2=0x7f46d00c8660 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:42:26.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:26.816+0000 7f46c6a00640 1 --2- 172.21.15.17:0/2033753935 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f46d0159330 0x7f46d00c8660 secure :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f46d015d7c0 tx=0x7f46c000fc00 comp rx=0 tx=0).stop 2024-09-10T21:42:26.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:26.820+0000 7f46cf400640 1 --2- 172.21.15.17:0/2033753935 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f46b00725e0 0x7f46b0074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:26.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:26.824+0000 7f46cf400640 1 --2- 172.21.15.17:0/2033753935 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f46b00725e0 0x7f46b0074aa0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f46d01599e0 tx=0x7f46c8009410 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:26.858 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-10T21:42:26.859 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:42:26.859 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:42:26.859 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-10T21:42:26.859 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 138, 2024-09-10T21:42:26.859 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 353197, 2024-09-10T21:42:26.859 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-10T21:42:26.859 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-10T21:42:26.859 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-10T21:42:26.859 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-10T21:42:26.859 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:42:26.859 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 2, 2024-09-10T21:42:26.859 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:42:26.859 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-10T21:42:26.860 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-10T21:42:26.860 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:42:26.860 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:42:26.860 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:42:26.860 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:42:26.860 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:42:26.860 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:42:26.860 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:42:26.860 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:42:26.860 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:42:26.860 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:42:26.860 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:42:26.860 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:42:26.861 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-10T21:42:26.861 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-10T21:42:26.861 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:42:26.861 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:42:26.861 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:42:26.861 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:42:26.861 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:42:26.861 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:42:26.861 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:42:26.861 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:42:26.861 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:42:26.861 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:42:26.861 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:42:26.862 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:42:26.862 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:42:26.862 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-10T21:42:26.862 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-10T21:42:26.862 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-10T21:42:26.862 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-10T21:42:26.862 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-10T21:42:26.862 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "5e2f023f-d6a6-49f4-a256-f4807dce3739", 2024-09-10T21:42:26.862 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:42:26.862 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:42:26.862 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-10T21:42:26.862 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-10T21:42:26.862 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-10T21:42:26.862 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-10T21:42:26.863 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-10T21:42:26.863 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-10T21:42:26.863 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-10T21:42:26.863 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:42:26.863 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-10T21:42:26.863 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:42:26.863 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:42:26.863 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:42:26.863 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:42:26.863 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:42:26.863 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:42:26.863 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:42:26.863 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:42:26.864 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:42:26.864 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:42:26.864 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:42:26.864 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:42:26.864 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-10T21:42:26.864 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:42:26.864 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-10T21:42:26.864 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:42:26.864 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:42:26.864 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:42:26.864 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:42:26.864 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:42:26.864 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:42:26.865 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:42:26.865 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-10T21:42:26.865 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:42:26.865 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:42:26.865 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:42:26.865 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:42:26.865 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-10T21:42:26.865 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:42:26.865 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:42:26.865 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:42:26.865 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:42:26.865 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-10T21:42:26.865 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-10T21:42:26.866 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:42:26.866 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:42:26.866 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:42:26.866 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:42:26.866 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:42:26.866 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:42:26.866 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:42:26.866 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:42:26.866 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:42:26.866 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:42:26.866 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:42:26.866 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3300", 2024-09-10T21:42:26.866 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:42:26.866 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:42:26.867 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:42:26.867 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:42:26.867 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789", 2024-09-10T21:42:26.867 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:42:26.867 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:42:26.867 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:42:26.867 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:42:26.867 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789/0", 2024-09-10T21:42:26.867 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6789/0", 2024-09-10T21:42:26.867 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:42:26.867 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:42:26.867 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:42:26.867 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:42:26.868 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:42:26.868 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:42:26.868 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:42:26.868 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:42:26.868 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:42:26.868 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:42:26.868 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:42:26.868 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3301", 2024-09-10T21:42:26.868 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:42:26.868 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:42:26.868 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:42:26.869 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:42:26.869 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790", 2024-09-10T21:42:26.869 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:42:26.869 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:42:26.869 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:42:26.869 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:42:26.869 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790/0", 2024-09-10T21:42:26.869 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6790/0", 2024-09-10T21:42:26.869 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:42:26.869 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:42:26.869 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:42:26.869 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:42:26.869 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:42:26.870 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:42:26.870 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-10T21:42:26.870 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-10T21:42:26.870 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:42:26.870 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:42:26.870 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:42:26.870 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:42:26.870 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:42:26.870 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:42:26.870 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": [ 2024-09-10T21:42:26.870 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:42:26.870 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:42:26.870 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:42:26.871 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 5 2024-09-10T21:42:26.871 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:42:26.871 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:42:26.871 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-10T21:42:26.871 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:42:26.871 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:42:26.871 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:42:26.871 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 7 2024-09-10T21:42:26.871 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:42:26.871 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:42:26.871 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-10T21:42:26.871 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:42:26.871 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:42:26.871 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:42:26.871 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 6 2024-09-10T21:42:26.872 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:42:26.872 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:42:26.872 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": [ 2024-09-10T21:42:26.872 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:42:26.872 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:42:26.872 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:42:26.872 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 2 2024-09-10T21:42:26.872 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:42:26.872 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:42:26.872 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:42:26.872 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-10T21:42:26.872 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-10T21:42:26.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:26.844+0000 7f46d5800640 1 --2- 172.21.15.17:0/2033753935 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f46d01f8710 0x7f46d01f8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:26.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:26.844+0000 7f46d5800640 1 -- 172.21.15.17:0/2033753935 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f46d01558d0 con 0x7f46d01f8710 2024-09-10T21:42:26.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:26.844+0000 7f46ce000640 1 --2- 172.21.15.17:0/2033753935 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f46d01f8710 0x7f46d01f8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:26.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:26.848+0000 7f46ce000640 1 -- 172.21.15.17:0/2033753935 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f46d01555f0 con 0x7f46d01f8710 2024-09-10T21:42:26.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:26.848+0000 7f46ce000640 1 --2- 172.21.15.17:0/2033753935 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f46d01f8710 0x7f46d01f8af0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f46d015d7c0 tx=0x7f46c0001030 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:26.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:26.848+0000 7f46c6a00640 1 -- 172.21.15.17:0/2033753935 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46c001ee10 con 0x7f46d01f8710 2024-09-10T21:42:26.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:26.848+0000 7f46c6a00640 1 -- 172.21.15.17:0/2033753935 <== mon.2 v2:172.21.15.31:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5043 (secure 0 0 0) 0x7f46c0007260 con 0x7f46d01f8710 2024-09-10T21:42:26.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:26.848+0000 7f46c6a00640 1 -- 172.21.15.17:0/2033753935 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f46d01f8710 msgr2=0x7f46d01f8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:26.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:26.848+0000 7f46c6a00640 1 --2- 172.21.15.17:0/2033753935 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f46d01f8710 0x7f46d01f8af0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f46d015d7c0 tx=0x7f46c0001030 comp rx=0 tx=0).stop 2024-09-10T21:42:26.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:26.848+0000 7f46c6a00640 1 -- 172.21.15.17:0/2033753935 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f46d01f8710 msgr2=0x7f46d01f8af0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:42:26.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:26.848+0000 7f46c6a00640 1 --2- 172.21.15.17:0/2033753935 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f46d01f8710 0x7f46d01f8af0 secure :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f46d015d7c0 tx=0x7f46c0001030 comp rx=0 tx=0).stop 2024-09-10T21:42:26.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:26.852+0000 7f46d5800640 1 -- 172.21.15.17:0/2033753935 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f46b00725e0 msgr2=0x7f46b0074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:26.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:26.852+0000 7f46d5800640 1 --2- 172.21.15.17:0/2033753935 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f46b00725e0 0x7f46b0074aa0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f46d01599e0 tx=0x7f46c8009410 comp rx=0 tx=0).stop 2024-09-10T21:42:26.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:26.852+0000 7f46cf400640 1 -- 172.21.15.17:0/2033753935 reap_dead start 2024-09-10T21:42:26.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:26.852+0000 7f46d5800640 1 -- 172.21.15.17:0/2033753935 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f46d00c5c80 msgr2=0x7f46d0159a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:26.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:26.852+0000 7f46d5800640 1 --2- 172.21.15.17:0/2033753935 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f46d00c5c80 0x7f46d0159a80 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f46bc0af000 tx=0x7f46bc0a8d60 comp rx=0 tx=0).stop 2024-09-10T21:42:26.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:26.852+0000 7f46d5800640 1 -- 172.21.15.17:0/2033753935 shutdown_connections 2024-09-10T21:42:26.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:26.852+0000 7f46d5800640 1 --2- 172.21.15.17:0/2033753935 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f46d00c5c80 0x7f46d0159a80 secure :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f46bc0af000 tx=0x7f46bc0a8d60 comp rx=0 tx=0).stop 2024-09-10T21:42:26.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:26.852+0000 7f46d5800640 1 -- 172.21.15.17:0/2033753935 >> 172.21.15.17:0/2033753935 conn(0x7f46d0068990 msgr2=0x7f46d0068d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:26.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:26.852+0000 7f46d5800640 1 -- 172.21.15.17:0/2033753935 shutdown_connections 2024-09-10T21:42:26.874 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:26.852+0000 7f46d5800640 1 -- 172.21.15.17:0/2033753935 wait complete. 2024-09-10T21:42:26.890 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-10T21:42:26.890 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-09-10T21:42:26.891 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-09-10T21:42:27.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:27.176+0000 7f05d7200640 1 -- 172.21.15.17:0/3858200238 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f05d8150580 msgr2=0x7f05d80c1400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:27.181 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:27.176+0000 7f05d7200640 1 --2- 172.21.15.17:0/3858200238 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f05d8150580 0x7f05d80c1400 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f05d0009bc0 tx=0x7f05d001cbc0 comp rx=0 tx=0).stop 2024-09-10T21:42:27.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:27.176+0000 7f05d7200640 1 -- 172.21.15.17:0/3858200238 shutdown_connections 2024-09-10T21:42:27.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:27.176+0000 7f05d7200640 1 --2- 172.21.15.17:0/3858200238 >> v2:172.21.15.17:3300/0 conn(0x7f05d80c85f0 0x7f05d815a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:27.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:27.176+0000 7f05d7200640 1 --2- 172.21.15.17:0/3858200238 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f05d8150580 0x7f05d80c1400 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:27.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:27.176+0000 7f05d7200640 1 -- 172.21.15.17:0/3858200238 >> 172.21.15.17:0/3858200238 conn(0x7f05d80c8ab0 msgr2=0x7f05d80c8ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:27.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:27.176+0000 7f05d7200640 1 -- 172.21.15.17:0/3858200238 shutdown_connections 2024-09-10T21:42:27.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:27.176+0000 7f05d7200640 1 -- 172.21.15.17:0/3858200238 wait complete. 2024-09-10T21:42:27.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:27.180+0000 7f05d7200640 1 Processor -- start 2024-09-10T21:42:27.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:27.184+0000 7f05d7200640 1 -- start start 2024-09-10T21:42:27.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:27.184+0000 7f05d7200640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05d80c7b10 0x7f05d814f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:27.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:27.184+0000 7f05d7200640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f05d80c85f0 0x7f05d8158960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:27.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:27.184+0000 7f05d7200640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f05d8150580 0x7f05d8144860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:27.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:27.184+0000 7f05d7200640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f05d80c76f0 con 0x7f05d8150580 2024-09-10T21:42:27.190 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:27.184+0000 7f05d7200640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f05d80c7570 con 0x7f05d80c7b10 2024-09-10T21:42:27.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:27.184+0000 7f05d7200640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f05d80cc610 con 0x7f05d80c85f0 2024-09-10T21:42:27.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:27.184+0000 7f05d5400640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f05d80c85f0 0x7f05d8158960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:27.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:27.184+0000 7f05d5400640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f05d80c85f0 0x7f05d8158960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3301/0 says I am v2:172.21.15.17:43456/0 (socket says 172.21.15.17:43456) 2024-09-10T21:42:27.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:27.184+0000 7f05d5400640 1 -- 172.21.15.17:0/1797999426 learned_addr learned my addr 172.21.15.17:0/1797999426 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:42:27.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:27.184+0000 7f05d5e00640 1 --2- 172.21.15.17:0/1797999426 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05d80c7b10 0x7f05d814f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:27.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:27.188+0000 7f05d5400640 1 -- 172.21.15.17:0/1797999426 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05d80c7b10 msgr2=0x7f05d814f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:27.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:27.188+0000 7f05d5400640 1 --2- 172.21.15.17:0/1797999426 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05d80c7b10 0x7f05d814f060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:27.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:27.188+0000 7f05d5400640 1 -- 172.21.15.17:0/1797999426 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f05d8150580 msgr2=0x7f05d8144860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:27.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:27.188+0000 7f05d5400640 1 --2- 172.21.15.17:0/1797999426 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f05d8150580 0x7f05d8144860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:27.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:27.188+0000 7f05d5400640 1 -- 172.21.15.17:0/1797999426 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05d80cbf10 con 0x7f05d80c85f0 2024-09-10T21:42:27.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:27.192+0000 7f05d5400640 1 --2- 172.21.15.17:0/1797999426 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f05d80c85f0 0x7f05d8158960 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f05c000d980 tx=0x7f05c000de40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:27.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:27.196+0000 7f05ce000640 1 -- 172.21.15.17:0/1797999426 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05c0017070 con 0x7f05d80c85f0 2024-09-10T21:42:27.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:27.196+0000 7f05ce000640 1 -- 172.21.15.17:0/1797999426 <== mon.2 v2:172.21.15.31:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f05c0009d20 con 0x7f05d80c85f0 2024-09-10T21:42:27.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:27.196+0000 7f05ce000640 1 -- 172.21.15.17:0/1797999426 <== mon.2 v2:172.21.15.31:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05c0004280 con 0x7f05d80c85f0 2024-09-10T21:42:27.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:27.196+0000 7f05d7200640 1 -- 172.21.15.17:0/1797999426 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05d8145a30 con 0x7f05d80c85f0 2024-09-10T21:42:27.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:27.196+0000 7f05d7200640 1 -- 172.21.15.17:0/1797999426 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f05d8145e90 con 0x7f05d80c85f0 2024-09-10T21:42:27.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:27.200+0000 7f05c7400640 1 -- 172.21.15.17:0/1797999426 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05a80052a0 con 0x7f05d80c85f0 2024-09-10T21:42:27.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:27.200+0000 7f05ce000640 1 -- 172.21.15.17:0/1797999426 <== mon.2 v2:172.21.15.31:3301/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f05c0003600 con 0x7f05d80c85f0 2024-09-10T21:42:27.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:27.204+0000 7f05ce000640 1 --2- 172.21.15.17:0/1797999426 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f05b4072660 0x7f05b4074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:27.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:27.204+0000 7f05ce000640 1 -- 172.21.15.17:0/1797999426 <== mon.2 v2:172.21.15.31:3301/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f05c00938d0 con 0x7f05d80c85f0 2024-09-10T21:42:27.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:27.204+0000 7f05d5e00640 1 --2- 172.21.15.17:0/1797999426 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f05b4072660 0x7f05b4074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:27.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:27.208+0000 7f05ce000640 1 -- 172.21.15.17:0/1797999426 <== mon.2 v2:172.21.15.31:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f05c005eb00 con 0x7f05d80c85f0 2024-09-10T21:42:27.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:27.212+0000 7f05d5e00640 1 --2- 172.21.15.17:0/1797999426 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f05b4072660 0x7f05b4074b20 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f05d00af000 tx=0x7f05d000b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:27.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:27.732+0000 7f05c7400640 1 -- 172.21.15.17:0/1797999426 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_command({"prefix": "mon scrub"} v 0) -- 0x7f05a8005da0 con 0x7f05d80c85f0 2024-09-10T21:42:27.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:27.744+0000 7f05ce000640 1 -- 172.21.15.17:0/1797999426 <== mon.2 v2:172.21.15.31:3301/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) ==== 57+0+0 (secure 0 0 0) 0x7f05c005e400 con 0x7f05d80c85f0 2024-09-10T21:42:27.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:27.784+0000 7f05c7400640 1 -- 172.21.15.17:0/1797999426 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f05b4072660 msgr2=0x7f05b4074b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:27.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:27.784+0000 7f05c7400640 1 --2- 172.21.15.17:0/1797999426 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f05b4072660 0x7f05b4074b20 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f05d00af000 tx=0x7f05d000b040 comp rx=0 tx=0).stop 2024-09-10T21:42:27.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:27.784+0000 7f05c7400640 1 -- 172.21.15.17:0/1797999426 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f05d80c85f0 msgr2=0x7f05d8158960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:27.790 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:27.784+0000 7f05c7400640 1 --2- 172.21.15.17:0/1797999426 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f05d80c85f0 0x7f05d8158960 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f05c000d980 tx=0x7f05c000de40 comp rx=0 tx=0).stop 2024-09-10T21:42:27.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:27.784+0000 7f05c7400640 1 -- 172.21.15.17:0/1797999426 shutdown_connections 2024-09-10T21:42:27.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:27.792+0000 7f05c7400640 1 --2- 172.21.15.17:0/1797999426 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f05b4072660 0x7f05b4074b20 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:27.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:27.792+0000 7f05c7400640 1 --2- 172.21.15.17:0/1797999426 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f05d8150580 0x7f05d8144860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:27.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:27.792+0000 7f05c7400640 1 --2- 172.21.15.17:0/1797999426 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f05d80c85f0 0x7f05d8158960 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:27.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:27.792+0000 7f05c7400640 1 --2- 172.21.15.17:0/1797999426 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f05d80c7b10 0x7f05d814f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:27.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:27.792+0000 7f05c7400640 1 -- 172.21.15.17:0/1797999426 >> 172.21.15.17:0/1797999426 conn(0x7f05d80c8ab0 msgr2=0x7f05d8073580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:27.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:27.792+0000 7f05c7400640 1 -- 172.21.15.17:0/1797999426 shutdown_connections 2024-09-10T21:42:27.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:27.792+0000 7f05c7400640 1 -- 172.21.15.17:0/1797999426 wait complete. 2024-09-10T21:42:27.881 INFO:tasks.mon_thrash.mon_thrasher:waiting for 10.0 secs before continuing thrashing 2024-09-10T21:42:37.882 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-09-10T21:42:37.883 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-10T21:42:38.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:38.208+0000 7f5f0d400640 1 -- 172.21.15.17:0/2170114714 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5f000d61e0 msgr2=0x7f5f000d6630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:38.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:38.208+0000 7f5f0d400640 1 --2- 172.21.15.17:0/2170114714 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5f000d61e0 0x7f5f000d6630 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f5ef4009900 tx=0x7f5ef401f0d0 comp rx=0 tx=0).stop 2024-09-10T21:42:38.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:38.208+0000 7f5f0d400640 1 -- 172.21.15.17:0/2170114714 shutdown_connections 2024-09-10T21:42:38.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:38.208+0000 7f5f0d400640 1 --2- 172.21.15.17:0/2170114714 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5f000d61e0 0x7f5f000d6630 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:38.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:38.208+0000 7f5f0d400640 1 --2- 172.21.15.17:0/2170114714 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f5f0004c1e0 0x7f5f000dc360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:38.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:38.208+0000 7f5f0d400640 1 -- 172.21.15.17:0/2170114714 >> 172.21.15.17:0/2170114714 conn(0x7f5f0004fde0 msgr2=0x7f5f000501f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:38.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:38.208+0000 7f5f0d400640 1 -- 172.21.15.17:0/2170114714 shutdown_connections 2024-09-10T21:42:38.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:38.208+0000 7f5f0d400640 1 -- 172.21.15.17:0/2170114714 wait complete. 2024-09-10T21:42:38.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:38.216+0000 7f5f0d400640 1 Processor -- start 2024-09-10T21:42:38.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:38.216+0000 7f5f0d400640 1 -- start start 2024-09-10T21:42:38.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:38.220+0000 7f5f0d400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f0004b6b0 0x7f5f00049a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:38.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:38.220+0000 7f5f0d400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5f0004c1e0 0x7f5f000d3f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:38.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:38.220+0000 7f5f05e00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f0004b6b0 0x7f5f00049a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:38.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:38.220+0000 7f5f05e00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f0004b6b0 0x7f5f00049a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.17:38460/0 (socket says 172.21.15.17:38460) 2024-09-10T21:42:38.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:38.220+0000 7f5f0d400640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f5f000d61e0 0x7f5f000d4440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:38.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:38.220+0000 7f5f0d400640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f5f000dc8a0 con 0x7f5f0004c1e0 2024-09-10T21:42:38.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:38.220+0000 7f5f0d400640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f5f000335c0 con 0x7f5f0004b6b0 2024-09-10T21:42:38.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:38.220+0000 7f5f0d400640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f5f00033ae0 con 0x7f5f000d61e0 2024-09-10T21:42:38.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:38.220+0000 7f5f06800640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f5f000d61e0 0x7f5f000d4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:38.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:38.220+0000 7f5f05e00640 1 -- 172.21.15.17:0/2544424746 learned_addr learned my addr 172.21.15.17:0/2544424746 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:42:38.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:38.220+0000 7f5f05400640 1 --2- 172.21.15.17:0/2544424746 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5f0004c1e0 0x7f5f000d3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:38.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:38.224+0000 7f5f05e00640 1 -- 172.21.15.17:0/2544424746 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f5f000d61e0 msgr2=0x7f5f000d4440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:38.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:38.224+0000 7f5f05e00640 1 --2- 172.21.15.17:0/2544424746 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f5f000d61e0 0x7f5f000d4440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:38.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:38.224+0000 7f5f05e00640 1 -- 172.21.15.17:0/2544424746 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5f0004c1e0 msgr2=0x7f5f000d3f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:38.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:38.224+0000 7f5f05e00640 1 --2- 172.21.15.17:0/2544424746 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5f0004c1e0 0x7f5f000d3f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:38.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:38.224+0000 7f5f05e00640 1 -- 172.21.15.17:0/2544424746 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f00163890 con 0x7f5f0004b6b0 2024-09-10T21:42:38.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:38.224+0000 7f5f05400640 1 --2- 172.21.15.17:0/2544424746 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5f0004c1e0 0x7f5f000d3f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:42:38.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:38.224+0000 7f5f05e00640 1 --2- 172.21.15.17:0/2544424746 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f0004b6b0 0x7f5f00049a60 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f5f080524e0 tx=0x7f5f0806a390 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:38.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:38.228+0000 7f5efea00640 1 -- 172.21.15.17:0/2544424746 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f08090070 con 0x7f5f0004b6b0 2024-09-10T21:42:38.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:38.228+0000 7f5efea00640 1 -- 172.21.15.17:0/2544424746 <== mon.1 v2:172.21.15.31:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5f080946c0 con 0x7f5f0004b6b0 2024-09-10T21:42:38.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:38.228+0000 7f5efea00640 1 -- 172.21.15.17:0/2544424746 <== mon.1 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f08094b50 con 0x7f5f0004b6b0 2024-09-10T21:42:38.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:38.228+0000 7f5f0d400640 1 -- 172.21.15.17:0/2544424746 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f000471f0 con 0x7f5f0004b6b0 2024-09-10T21:42:38.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:38.228+0000 7f5f0d400640 1 -- 172.21.15.17:0/2544424746 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f00162cb0 con 0x7f5f0004b6b0 2024-09-10T21:42:38.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:38.232+0000 7f5f0d400640 1 -- 172.21.15.17:0/2544424746 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ee00052a0 con 0x7f5f0004b6b0 2024-09-10T21:42:38.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:38.236+0000 7f5efea00640 1 -- 172.21.15.17:0/2544424746 <== mon.1 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f5f08094cf0 con 0x7f5f0004b6b0 2024-09-10T21:42:38.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:38.236+0000 7f5efea00640 1 --2- 172.21.15.17:0/2544424746 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f5ee8072660 0x7f5ee8074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:38.242 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:38.236+0000 7f5efea00640 1 -- 172.21.15.17:0/2544424746 <== mon.1 v2:172.21.15.31:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f5f0810f460 con 0x7f5f0004b6b0 2024-09-10T21:42:38.243 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:38.236+0000 7f5f05400640 1 --2- 172.21.15.17:0/2544424746 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f5ee8072660 0x7f5ee8074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:38.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:38.240+0000 7f5efea00640 1 -- 172.21.15.17:0/2544424746 <== mon.1 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f5f080da1a0 con 0x7f5f0004b6b0 2024-09-10T21:42:38.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:38.244+0000 7f5f05400640 1 --2- 172.21.15.17:0/2544424746 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f5ee8072660 0x7f5ee8074b20 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f5f001627c0 tx=0x7f5ef8008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:38.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:38.756+0000 7f5f0d400640 1 -- 172.21.15.17:0/2544424746 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f5ee0005590 con 0x7f5f0004b6b0 2024-09-10T21:42:38.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:38.756+0000 7f5efea00640 1 -- 172.21.15.17:0/2544424746 <== mon.1 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1558 (secure 0 0 0) 0x7f5f080debd0 con 0x7f5f0004b6b0 2024-09-10T21:42:38.763 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":138,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":14,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5e2f023f-d6a6-49f4-a256-f4807dce3739","modified":"2024-09-10T21:23:46.892130Z","created":"2024-09-10T21:23:46.892130Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3301","nonce":0},{"type":"v1","addr":"172.21.15.31:6790","nonce":0}]},"addr":"172.21.15.31:6790/0","public_addr":"172.21.15.31:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-10T21:42:38.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:38.760+0000 7f5f0d400640 1 -- 172.21.15.17:0/2544424746 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f5ee8072660 msgr2=0x7f5ee8074b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:38.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:38.760+0000 7f5f0d400640 1 --2- 172.21.15.17:0/2544424746 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f5ee8072660 0x7f5ee8074b20 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f5f001627c0 tx=0x7f5ef8008040 comp rx=0 tx=0).stop 2024-09-10T21:42:38.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:38.760+0000 7f5f0d400640 1 -- 172.21.15.17:0/2544424746 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f0004b6b0 msgr2=0x7f5f00049a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:38.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:38.760+0000 7f5f0d400640 1 --2- 172.21.15.17:0/2544424746 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f0004b6b0 0x7f5f00049a60 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f5f080524e0 tx=0x7f5f0806a390 comp rx=0 tx=0).stop 2024-09-10T21:42:38.769 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:38.760+0000 7f5f0d400640 1 -- 172.21.15.17:0/2544424746 shutdown_connections 2024-09-10T21:42:38.769 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:38.760+0000 7f5f0d400640 1 --2- 172.21.15.17:0/2544424746 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f5ee8072660 0x7f5ee8074b20 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:38.769 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:38.760+0000 7f5f0d400640 1 --2- 172.21.15.17:0/2544424746 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f5f000d61e0 0x7f5f000d4440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:38.769 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:38.760+0000 7f5f0d400640 1 --2- 172.21.15.17:0/2544424746 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5f0004c1e0 0x7f5f000d3f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:38.769 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:38.760+0000 7f5f0d400640 1 --2- 172.21.15.17:0/2544424746 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5f0004b6b0 0x7f5f00049a60 secure :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f5f080524e0 tx=0x7f5f0806a390 comp rx=0 tx=0).stop 2024-09-10T21:42:38.769 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:38.760+0000 7f5f0d400640 1 -- 172.21.15.17:0/2544424746 >> 172.21.15.17:0/2544424746 conn(0x7f5f0004fde0 msgr2=0x7f5f0004e730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:38.769 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:38.764+0000 7f5f0d400640 1 -- 172.21.15.17:0/2544424746 shutdown_connections 2024-09-10T21:42:38.769 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:38.764+0000 7f5f0d400640 1 -- 172.21.15.17:0/2544424746 wait complete. 2024-09-10T21:42:38.812 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-09-10T21:42:38.812 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-09-10T21:42:38.813 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-09-10T21:42:38.813 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-10T21:42:39.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:39.184+0000 7f13f2c00640 1 -- 172.21.15.17:0/3974153868 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f13ec151500 msgr2=0x7f13ec074e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:39.191 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:39.184+0000 7f13f2c00640 1 --2- 172.21.15.17:0/3974153868 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f13ec151500 0x7f13ec074e10 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f13e0009900 tx=0x7f13e001c900 comp rx=0 tx=0).stop 2024-09-10T21:42:39.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:39.192+0000 7f13f2c00640 1 -- 172.21.15.17:0/3974153868 shutdown_connections 2024-09-10T21:42:39.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:39.192+0000 7f13f2c00640 1 --2- 172.21.15.17:0/3974153868 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f13ec0747e0 0x7f13ec06e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:39.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:39.192+0000 7f13f2c00640 1 --2- 172.21.15.17:0/3974153868 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f13ec151500 0x7f13ec074e10 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:39.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:39.192+0000 7f13f2c00640 1 -- 172.21.15.17:0/3974153868 >> 172.21.15.17:0/3974153868 conn(0x7f13ec06fe60 msgr2=0x7f13ec070270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:39.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:39.196+0000 7f13f2c00640 1 -- 172.21.15.17:0/3974153868 shutdown_connections 2024-09-10T21:42:39.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:39.196+0000 7f13f2c00640 1 -- 172.21.15.17:0/3974153868 wait complete. 2024-09-10T21:42:39.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:39.200+0000 7f13f2c00640 1 Processor -- start 2024-09-10T21:42:39.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:39.204+0000 7f13f2c00640 1 -- start start 2024-09-10T21:42:39.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:39.204+0000 7f13f2c00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f13ec06e680 0x7f13ec153c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:39.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:39.204+0000 7f13f2c00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f13ec0747e0 0x7f13ec0c1c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:39.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:39.204+0000 7f13f2c00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f13ec151500 0x7f13ec201b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:39.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:39.204+0000 7f13f2c00640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f13ec0c8080 con 0x7f13ec0747e0 2024-09-10T21:42:39.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:39.204+0000 7f13f2c00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f13ec1518e0 con 0x7f13ec06e680 2024-09-10T21:42:39.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:39.204+0000 7f13f2c00640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f13ec0c8200 con 0x7f13ec151500 2024-09-10T21:42:39.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:39.212+0000 7f13f0e00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f13ec0747e0 0x7f13ec0c1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:39.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:39.212+0000 7f13f0e00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f13ec0747e0 0x7f13ec0c1c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43098/0 (socket says 172.21.15.17:43098) 2024-09-10T21:42:39.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:39.212+0000 7f13f0e00640 1 -- 172.21.15.17:0/3102699431 learned_addr learned my addr 172.21.15.17:0/3102699431 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:42:39.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:39.212+0000 7f13f2200640 1 --2- 172.21.15.17:0/3102699431 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f13ec151500 0x7f13ec201b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:39.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:39.212+0000 7f13f0e00640 1 -- 172.21.15.17:0/3102699431 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f13ec151500 msgr2=0x7f13ec201b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:39.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:39.212+0000 7f13f0e00640 1 --2- 172.21.15.17:0/3102699431 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f13ec151500 0x7f13ec201b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:39.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:39.212+0000 7f13f0e00640 1 -- 172.21.15.17:0/3102699431 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f13ec06e680 msgr2=0x7f13ec153c30 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-10T21:42:39.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:39.212+0000 7f13f0e00640 1 --2- 172.21.15.17:0/3102699431 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f13ec06e680 0x7f13ec153c30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:39.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:39.212+0000 7f13f0e00640 1 -- 172.21.15.17:0/3102699431 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13ec152730 con 0x7f13ec0747e0 2024-09-10T21:42:39.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:39.212+0000 7f13f0e00640 1 --2- 172.21.15.17:0/3102699431 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f13ec0747e0 0x7f13ec0c1c10 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f13d800b4a0 tx=0x7f13d800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:39.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:39.212+0000 7f13e9600640 1 -- 172.21.15.17:0/3102699431 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13d8014070 con 0x7f13ec0747e0 2024-09-10T21:42:39.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:39.212+0000 7f13e9600640 1 -- 172.21.15.17:0/3102699431 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f13d8010700 con 0x7f13ec0747e0 2024-09-10T21:42:39.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:39.216+0000 7f13e9600640 1 -- 172.21.15.17:0/3102699431 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13d8010b90 con 0x7f13ec0747e0 2024-09-10T21:42:39.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:39.216+0000 7f13f2c00640 1 -- 172.21.15.17:0/3102699431 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13ec152a10 con 0x7f13ec0747e0 2024-09-10T21:42:39.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:39.220+0000 7f13f2c00640 1 -- 172.21.15.17:0/3102699431 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13ec152fd0 con 0x7f13ec0747e0 2024-09-10T21:42:39.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:39.224+0000 7f13f2c00640 1 --2- 172.21.15.17:0/3102699431 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f13ec2020b0 0x7f13ec14b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:39.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:39.224+0000 7f13f2c00640 1 -- 172.21.15.17:0/3102699431 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f13ec1544f0 con 0x7f13ec2020b0 2024-09-10T21:42:39.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:39.228+0000 7f13f1800640 1 --2- 172.21.15.17:0/3102699431 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f13ec2020b0 0x7f13ec14b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:39.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:39.232+0000 7f13f1800640 1 -- 172.21.15.17:0/3102699431 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f13ec06e140 con 0x7f13ec2020b0 2024-09-10T21:42:39.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:39.236+0000 7f13f1800640 1 --2- 172.21.15.17:0/3102699431 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f13ec2020b0 0x7f13ec14b690 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f13ec0c31b0 tx=0x7f13e0005df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:39.244 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:39.236+0000 7f13e9600640 1 -- 172.21.15.17:0/3102699431 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f13d800f360 con 0x7f13ec0747e0 2024-09-10T21:42:39.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:39.240+0000 7f13e9600640 1 --2- 172.21.15.17:0/3102699431 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f13cc072630 0x7f13cc074af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:39.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:39.240+0000 7f13e9600640 1 -- 172.21.15.17:0/3102699431 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f13d8093230 con 0x7f13ec0747e0 2024-09-10T21:42:39.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:39.244+0000 7f13f2200640 1 --2- 172.21.15.17:0/3102699431 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f13cc072630 0x7f13cc074af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:39.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:39.248+0000 7f13f2200640 1 --2- 172.21.15.17:0/3102699431 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f13cc072630 0x7f13cc074af0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f13e40012f0 tx=0x7f13e4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:39.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:39.248+0000 7f13e9600640 1 -- 172.21.15.17:0/3102699431 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13e00b2040 con 0x7f13ec2020b0 2024-09-10T21:42:39.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:39.252+0000 7f13e9600640 1 -- 172.21.15.17:0/3102699431 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f13e00a58d0 con 0x7f13ec2020b0 2024-09-10T21:42:39.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:39.252+0000 7f13e9600640 1 -- 172.21.15.17:0/3102699431 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f13ec2020b0 msgr2=0x7f13ec14b690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:39.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:39.252+0000 7f13e9600640 1 --2- 172.21.15.17:0/3102699431 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f13ec2020b0 0x7f13ec14b690 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f13ec0c31b0 tx=0x7f13e0005df0 comp rx=0 tx=0).stop 2024-09-10T21:42:39.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:39.252+0000 7f13e9600640 1 -- 172.21.15.17:0/3102699431 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f13ec2020b0 msgr2=0x7f13ec14b690 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:42:39.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:39.252+0000 7f13e9600640 1 --2- 172.21.15.17:0/3102699431 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f13ec2020b0 0x7f13ec14b690 secure :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f13ec0c31b0 tx=0x7f13e0005df0 comp rx=0 tx=0).stop 2024-09-10T21:42:39.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:39.288+0000 7f13f2c00640 1 --2- 172.21.15.17:0/3102699431 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f13ec14bbd0 0x7f13ec203850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:39.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:39.288+0000 7f13f2c00640 1 -- 172.21.15.17:0/3102699431 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f13ec1edf50 con 0x7f13ec14bbd0 2024-09-10T21:42:39.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:39.288+0000 7f13f1800640 1 --2- 172.21.15.17:0/3102699431 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f13ec14bbd0 0x7f13ec203850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:39.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:39.292+0000 7f13f1800640 1 -- 172.21.15.17:0/3102699431 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f13ec14f7b0 con 0x7f13ec14bbd0 2024-09-10T21:42:39.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:39.292+0000 7f13f1800640 1 --2- 172.21.15.17:0/3102699431 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f13ec14bbd0 0x7f13ec203850 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f13e0009900 tx=0x7f13e001c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:39.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:39.292+0000 7f13e9600640 1 -- 172.21.15.17:0/3102699431 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13e00b2070 con 0x7f13ec14bbd0 2024-09-10T21:42:39.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:39.296+0000 7f13e9600640 1 -- 172.21.15.17:0/3102699431 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4544 (secure 0 0 0) 0x7f13e0003600 con 0x7f13ec14bbd0 2024-09-10T21:42:39.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:39.296+0000 7f13e9600640 1 -- 172.21.15.17:0/3102699431 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f13ec14bbd0 msgr2=0x7f13ec203850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:39.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:39.296+0000 7f13e9600640 1 --2- 172.21.15.17:0/3102699431 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f13ec14bbd0 0x7f13ec203850 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f13e0009900 tx=0x7f13e001c900 comp rx=0 tx=0).stop 2024-09-10T21:42:39.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:39.296+0000 7f13e9600640 1 -- 172.21.15.17:0/3102699431 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f13ec14bbd0 msgr2=0x7f13ec203850 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:42:39.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:39.296+0000 7f13e9600640 1 --2- 172.21.15.17:0/3102699431 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f13ec14bbd0 0x7f13ec203850 secure :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f13e0009900 tx=0x7f13e001c900 comp rx=0 tx=0).stop 2024-09-10T21:42:39.309 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-10T21:42:39.310 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:42:39.310 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:42:39.310 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "leader", 2024-09-10T21:42:39.310 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 138, 2024-09-10T21:42:39.310 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 18423, 2024-09-10T21:42:39.310 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-10T21:42:39.310 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-10T21:42:39.310 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-10T21:42:39.310 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-10T21:42:39.310 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:42:39.310 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 14, 2024-09-10T21:42:39.310 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:42:39.310 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-10T21:42:39.311 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-10T21:42:39.311 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:42:39.311 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:42:39.311 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:42:39.311 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:42:39.311 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:42:39.311 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:42:39.311 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:42:39.311 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:42:39.311 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:42:39.311 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:42:39.311 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:42:39.311 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:42:39.311 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-10T21:42:39.312 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-10T21:42:39.312 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:42:39.312 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:42:39.312 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:42:39.312 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:42:39.312 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:42:39.312 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:42:39.312 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:42:39.312 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:42:39.312 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:42:39.312 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:42:39.313 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:42:39.313 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:42:39.313 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:42:39.313 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-10T21:42:39.313 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-10T21:42:39.313 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-10T21:42:39.313 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-10T21:42:39.313 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-10T21:42:39.313 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "5e2f023f-d6a6-49f4-a256-f4807dce3739", 2024-09-10T21:42:39.313 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:42:39.313 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:42:39.313 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-10T21:42:39.313 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-10T21:42:39.313 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-10T21:42:39.314 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-10T21:42:39.314 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-10T21:42:39.314 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-10T21:42:39.314 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-10T21:42:39.314 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:42:39.314 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-10T21:42:39.314 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:42:39.314 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:42:39.314 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:42:39.314 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:42:39.314 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:42:39.314 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:42:39.315 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:42:39.315 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:42:39.315 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:42:39.315 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:42:39.315 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:42:39.315 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:42:39.315 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-10T21:42:39.315 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:42:39.315 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-10T21:42:39.315 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:42:39.315 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:42:39.315 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:42:39.315 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:42:39.315 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:42:39.316 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:42:39.316 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:42:39.316 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-10T21:42:39.316 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:42:39.316 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:42:39.316 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:42:39.316 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:42:39.316 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-10T21:42:39.316 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:42:39.316 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:42:39.316 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:42:39.316 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:42:39.317 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-10T21:42:39.317 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-10T21:42:39.317 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:42:39.317 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:42:39.317 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:42:39.317 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:42:39.317 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:42:39.317 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:42:39.317 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:42:39.317 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:42:39.317 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:42:39.317 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:42:39.318 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:42:39.318 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3300", 2024-09-10T21:42:39.318 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:42:39.318 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:42:39.318 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:42:39.318 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:42:39.318 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789", 2024-09-10T21:42:39.318 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:42:39.318 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:42:39.318 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:42:39.319 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:42:39.319 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789/0", 2024-09-10T21:42:39.319 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6789/0", 2024-09-10T21:42:39.319 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:42:39.319 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:42:39.319 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:42:39.319 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:42:39.319 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:42:39.319 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:42:39.319 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:42:39.319 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:42:39.319 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:42:39.319 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:42:39.319 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:42:39.320 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3301", 2024-09-10T21:42:39.320 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:42:39.320 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:42:39.320 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:42:39.320 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:42:39.320 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790", 2024-09-10T21:42:39.320 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:42:39.320 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:42:39.320 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:42:39.320 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:42:39.320 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790/0", 2024-09-10T21:42:39.320 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6790/0", 2024-09-10T21:42:39.320 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:42:39.321 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:42:39.321 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:42:39.321 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:42:39.321 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:42:39.321 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:42:39.321 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-10T21:42:39.321 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-10T21:42:39.321 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:42:39.321 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:42:39.321 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:42:39.321 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:42:39.321 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:42:39.321 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:42:39.322 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-10T21:42:39.322 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:42:39.322 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:42:39.322 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:42:39.322 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 2 2024-09-10T21:42:39.322 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:42:39.322 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:42:39.322 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:42:39.322 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-10T21:42:39.322 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-10T21:42:39.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:39.308+0000 7f13f2c00640 1 -- 172.21.15.17:0/3102699431 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f13cc072630 msgr2=0x7f13cc074af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:39.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:39.308+0000 7f13f2c00640 1 --2- 172.21.15.17:0/3102699431 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f13cc072630 0x7f13cc074af0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f13e40012f0 tx=0x7f13e4009040 comp rx=0 tx=0).stop 2024-09-10T21:42:39.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:39.308+0000 7f13f2200640 1 -- 172.21.15.17:0/3102699431 reap_dead start 2024-09-10T21:42:39.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:39.308+0000 7f13f2c00640 1 -- 172.21.15.17:0/3102699431 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f13ec0747e0 msgr2=0x7f13ec0c1c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:39.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:39.308+0000 7f13f2c00640 1 --2- 172.21.15.17:0/3102699431 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f13ec0747e0 0x7f13ec0c1c10 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f13d800b4a0 tx=0x7f13d800b960 comp rx=0 tx=0).stop 2024-09-10T21:42:39.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:39.312+0000 7f13f2c00640 1 -- 172.21.15.17:0/3102699431 shutdown_connections 2024-09-10T21:42:39.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:39.312+0000 7f13f2c00640 1 --2- 172.21.15.17:0/3102699431 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f13ec0747e0 0x7f13ec0c1c10 secure :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f13d800b4a0 tx=0x7f13d800b960 comp rx=0 tx=0).stop 2024-09-10T21:42:39.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:39.312+0000 7f13f2c00640 1 -- 172.21.15.17:0/3102699431 >> 172.21.15.17:0/3102699431 conn(0x7f13ec06fe60 msgr2=0x7f13ec1ed310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:39.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:39.320+0000 7f13f2c00640 1 -- 172.21.15.17:0/3102699431 shutdown_connections 2024-09-10T21:42:39.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:39.320+0000 7f13f2c00640 1 -- 172.21.15.17:0/3102699431 wait complete. 2024-09-10T21:42:39.449 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-10T21:42:39.449 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-09-10T21:42:39.449 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-10T21:42:39.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:39.800+0000 7fb1a6e00640 1 -- 172.21.15.17:0/196963239 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb1a0158ad0 msgr2=0x7fb1a00cbc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:39.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:39.800+0000 7fb1a6e00640 1 --2- 172.21.15.17:0/196963239 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb1a0158ad0 0x7fb1a00cbc60 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fb1a0074f00 tx=0x7fb19001c830 comp rx=0 tx=0).stop 2024-09-10T21:42:39.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:39.804+0000 7fb1a6e00640 1 -- 172.21.15.17:0/196963239 shutdown_connections 2024-09-10T21:42:39.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:39.804+0000 7fb1a6e00640 1 --2- 172.21.15.17:0/196963239 >> v2:172.21.15.31:3300/0 conn(0x7fb1a00b3150 0x7fb1a00c6780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:39.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:39.804+0000 7fb1a6e00640 1 --2- 172.21.15.17:0/196963239 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb1a0158ad0 0x7fb1a00cbc60 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:39.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:39.804+0000 7fb1a6e00640 1 -- 172.21.15.17:0/196963239 >> 172.21.15.17:0/196963239 conn(0x7fb1a006edb0 msgr2=0x7fb1a006f1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:39.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:39.808+0000 7fb1a6e00640 1 -- 172.21.15.17:0/196963239 shutdown_connections 2024-09-10T21:42:39.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:39.808+0000 7fb1a6e00640 1 -- 172.21.15.17:0/196963239 wait complete. 2024-09-10T21:42:39.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:39.816+0000 7fb1a6e00640 1 Processor -- start 2024-09-10T21:42:39.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:39.820+0000 7fb1a6e00640 1 -- start start 2024-09-10T21:42:39.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:39.820+0000 7fb1a6e00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1a00b3150 0x7fb1a0070680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:39.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:39.820+0000 7fb1a6e00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fb1a00c6cc0 0x7fb1a01fb7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:39.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:39.820+0000 7fb1a6e00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb1a0158ad0 0x7fb1a01fbd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:39.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:39.820+0000 7fb1a6e00640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fb1a00c6280 con 0x7fb1a0158ad0 2024-09-10T21:42:39.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:39.820+0000 7fb1a6e00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fb1a00cd490 con 0x7fb1a00b3150 2024-09-10T21:42:39.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:39.820+0000 7fb1a6e00640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7fb1a00c6400 con 0x7fb1a00c6cc0 2024-09-10T21:42:39.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:39.820+0000 7fb19f400640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fb1a00c6cc0 0x7fb1a01fb7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:39.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:39.820+0000 7fb19fe00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1a00b3150 0x7fb1a0070680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:39.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:39.820+0000 7fb19fe00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1a00b3150 0x7fb1a0070680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.17:38474/0 (socket says 172.21.15.17:38474) 2024-09-10T21:42:39.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:39.820+0000 7fb19fe00640 1 -- 172.21.15.17:0/1856981585 learned_addr learned my addr 172.21.15.17:0/1856981585 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:42:39.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:39.824+0000 7fb1a4e00640 1 --2- 172.21.15.17:0/1856981585 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb1a0158ad0 0x7fb1a01fbd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:39.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:39.824+0000 7fb19f400640 1 -- 172.21.15.17:0/1856981585 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1a00b3150 msgr2=0x7fb1a0070680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:39.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:39.824+0000 7fb19f400640 1 --2- 172.21.15.17:0/1856981585 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1a00b3150 0x7fb1a0070680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:39.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:39.824+0000 7fb19f400640 1 -- 172.21.15.17:0/1856981585 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb1a0158ad0 msgr2=0x7fb1a01fbd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:39.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:39.824+0000 7fb19f400640 1 --2- 172.21.15.17:0/1856981585 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb1a0158ad0 0x7fb1a01fbd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:39.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:39.824+0000 7fb19f400640 1 -- 172.21.15.17:0/1856981585 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1a0151b20 con 0x7fb1a00c6cc0 2024-09-10T21:42:39.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:39.824+0000 7fb19f400640 1 --2- 172.21.15.17:0/1856981585 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fb1a00c6cc0 0x7fb1a01fb7c0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fb18c00ea20 tx=0x7fb18c00eee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:39.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:39.828+0000 7fb197e00640 1 -- 172.21.15.17:0/1856981585 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb18c00cd80 con 0x7fb1a00c6cc0 2024-09-10T21:42:39.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:39.828+0000 7fb197e00640 1 -- 172.21.15.17:0/1856981585 <== mon.2 v2:172.21.15.31:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb18c010430 con 0x7fb1a00c6cc0 2024-09-10T21:42:39.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:39.828+0000 7fb197e00640 1 -- 172.21.15.17:0/1856981585 <== mon.2 v2:172.21.15.31:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb18c010860 con 0x7fb1a00c6cc0 2024-09-10T21:42:39.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:39.828+0000 7fb1a6e00640 1 -- 172.21.15.17:0/1856981585 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1a0151e00 con 0x7fb1a00c6cc0 2024-09-10T21:42:39.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:39.828+0000 7fb1a6e00640 1 -- 172.21.15.17:0/1856981585 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1a0070bc0 con 0x7fb1a00c6cc0 2024-09-10T21:42:39.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:39.828+0000 7fb1a6e00640 1 --2- 172.21.15.17:0/1856981585 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1a0161c40 0x7fb1a0162020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:39.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:39.828+0000 7fb1a6e00640 1 -- 172.21.15.17:0/1856981585 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fb1a0151fd0 con 0x7fb1a0161c40 2024-09-10T21:42:39.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:39.832+0000 7fb19fe00640 1 --2- 172.21.15.17:0/1856981585 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1a0161c40 0x7fb1a0162020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:39.838 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:39.832+0000 7fb197e00640 1 -- 172.21.15.17:0/1856981585 <== mon.2 v2:172.21.15.31:3301/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fb18c010a00 con 0x7fb1a00c6cc0 2024-09-10T21:42:39.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:39.836+0000 7fb197e00640 1 --2- 172.21.15.17:0/1856981585 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fb1800725e0 0x7fb180074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:39.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:39.836+0000 7fb19fe00640 1 -- 172.21.15.17:0/1856981585 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb1a00c6780 con 0x7fb1a0161c40 2024-09-10T21:42:39.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:39.836+0000 7fb197e00640 1 -- 172.21.15.17:0/1856981585 <== mon.2 v2:172.21.15.31:3301/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fb18c094210 con 0x7fb1a00c6cc0 2024-09-10T21:42:39.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:39.836+0000 7fb1a4e00640 1 --2- 172.21.15.17:0/1856981585 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fb1800725e0 0x7fb180074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:39.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:39.836+0000 7fb19fe00640 1 --2- 172.21.15.17:0/1856981585 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1a0161c40 0x7fb1a0162020 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fb190009230 tx=0x7fb1900095a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:39.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:39.840+0000 7fb197e00640 1 -- 172.21.15.17:0/1856981585 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1900b2050 con 0x7fb1a0161c40 2024-09-10T21:42:39.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:39.840+0000 7fb197e00640 1 -- 172.21.15.17:0/1856981585 <== mon.1 v2:172.21.15.31:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fb1900b3040 con 0x7fb1a0161c40 2024-09-10T21:42:39.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:39.840+0000 7fb197e00640 1 -- 172.21.15.17:0/1856981585 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1a0161c40 msgr2=0x7fb1a0162020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:39.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:39.840+0000 7fb197e00640 1 --2- 172.21.15.17:0/1856981585 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1a0161c40 0x7fb1a0162020 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fb190009230 tx=0x7fb1900095a0 comp rx=0 tx=0).stop 2024-09-10T21:42:39.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:39.840+0000 7fb197e00640 1 -- 172.21.15.17:0/1856981585 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1a0161c40 msgr2=0x7fb1a0162020 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:42:39.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:39.840+0000 7fb197e00640 1 --2- 172.21.15.17:0/1856981585 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1a0161c40 0x7fb1a0162020 secure :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fb190009230 tx=0x7fb1900095a0 comp rx=0 tx=0).stop 2024-09-10T21:42:39.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:39.840+0000 7fb1a4e00640 1 --2- 172.21.15.17:0/1856981585 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fb1800725e0 0x7fb180074aa0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fb198001d40 tx=0x7fb198009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:39.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:39.864+0000 7fb1a6e00640 1 --2- 172.21.15.17:0/1856981585 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1a00c7ac0 0x7fb1a00c7ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:39.869 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:39.864+0000 7fb1a6e00640 1 -- 172.21.15.17:0/1856981585 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fb1a015f810 con 0x7fb1a00c7ac0 2024-09-10T21:42:39.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:39.864+0000 7fb19fe00640 1 --2- 172.21.15.17:0/1856981585 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1a00c7ac0 0x7fb1a00c7ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:39.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:39.864+0000 7fb19fe00640 1 -- 172.21.15.17:0/1856981585 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fb1a00cc600 con 0x7fb1a00c7ac0 2024-09-10T21:42:39.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:39.868+0000 7fb19fe00640 1 --2- 172.21.15.17:0/1856981585 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1a00c7ac0 0x7fb1a00c7ed0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fb19001aee0 tx=0x7fb190009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:39.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:39.868+0000 7fb197e00640 1 -- 172.21.15.17:0/1856981585 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1900a5c60 con 0x7fb1a00c7ac0 2024-09-10T21:42:39.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:39.868+0000 7fb197e00640 1 -- 172.21.15.17:0/1856981585 <== mon.1 v2:172.21.15.31:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4877 (secure 0 0 0) 0x7fb1900a5c60 con 0x7fb1a00c7ac0 2024-09-10T21:42:39.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:39.868+0000 7fb197e00640 1 -- 172.21.15.17:0/1856981585 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1a00c7ac0 msgr2=0x7fb1a00c7ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:39.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:39.868+0000 7fb197e00640 1 --2- 172.21.15.17:0/1856981585 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1a00c7ac0 0x7fb1a00c7ed0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fb19001aee0 tx=0x7fb190009260 comp rx=0 tx=0).stop 2024-09-10T21:42:39.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:39.868+0000 7fb197e00640 1 -- 172.21.15.17:0/1856981585 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1a00c7ac0 msgr2=0x7fb1a00c7ed0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:42:39.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:39.868+0000 7fb197e00640 1 --2- 172.21.15.17:0/1856981585 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb1a00c7ac0 0x7fb1a00c7ed0 secure :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fb19001aee0 tx=0x7fb190009260 comp rx=0 tx=0).stop 2024-09-10T21:42:39.874 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-10T21:42:39.874 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:42:39.874 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:42:39.874 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-10T21:42:39.874 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 138, 2024-09-10T21:42:39.874 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 171297, 2024-09-10T21:42:39.874 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-10T21:42:39.874 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-10T21:42:39.874 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-10T21:42:39.874 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-10T21:42:39.874 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:42:39.874 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 15, 2024-09-10T21:42:39.874 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:42:39.875 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-10T21:42:39.875 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-10T21:42:39.875 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:42:39.875 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:42:39.875 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:42:39.875 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:42:39.875 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:42:39.875 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:42:39.875 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:42:39.875 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:42:39.875 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:42:39.875 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:42:39.876 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:42:39.876 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:42:39.876 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-10T21:42:39.876 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-10T21:42:39.876 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:42:39.876 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:42:39.876 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:42:39.876 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:42:39.876 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:42:39.876 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:42:39.876 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:42:39.876 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:42:39.876 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:42:39.876 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:42:39.877 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:42:39.877 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:42:39.877 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:42:39.877 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-10T21:42:39.877 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-10T21:42:39.877 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-10T21:42:39.877 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-10T21:42:39.877 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-10T21:42:39.877 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "5e2f023f-d6a6-49f4-a256-f4807dce3739", 2024-09-10T21:42:39.877 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:42:39.877 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:42:39.877 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-10T21:42:39.877 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-10T21:42:39.878 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-10T21:42:39.878 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-10T21:42:39.878 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-10T21:42:39.878 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-10T21:42:39.878 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-10T21:42:39.878 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:42:39.878 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-10T21:42:39.878 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:42:39.878 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:42:39.878 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:42:39.878 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:42:39.878 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:42:39.878 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:42:39.878 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:42:39.878 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:42:39.879 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:42:39.879 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:42:39.879 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:42:39.879 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:42:39.879 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-10T21:42:39.879 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:42:39.879 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-10T21:42:39.879 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:42:39.879 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:42:39.879 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:42:39.879 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:42:39.879 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:42:39.880 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:42:39.880 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:42:39.880 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-10T21:42:39.880 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:42:39.880 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:42:39.880 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:42:39.880 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:42:39.880 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-10T21:42:39.880 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:42:39.880 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:42:39.880 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:42:39.880 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:42:39.880 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-10T21:42:39.880 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-10T21:42:39.881 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:42:39.881 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:42:39.881 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:42:39.881 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:42:39.881 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:42:39.881 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:42:39.881 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:42:39.881 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:42:39.881 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:42:39.881 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:42:39.881 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:42:39.881 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3300", 2024-09-10T21:42:39.881 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:42:39.882 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:42:39.882 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:42:39.882 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:42:39.882 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789", 2024-09-10T21:42:39.882 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:42:39.882 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:42:39.882 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:42:39.882 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:42:39.882 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789/0", 2024-09-10T21:42:39.882 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6789/0", 2024-09-10T21:42:39.882 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:42:39.882 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:42:39.882 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:42:39.882 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:42:39.882 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:42:39.883 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:42:39.883 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:42:39.883 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:42:39.883 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:42:39.883 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:42:39.883 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:42:39.883 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3301", 2024-09-10T21:42:39.883 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:42:39.883 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:42:39.883 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:42:39.883 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:42:39.883 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790", 2024-09-10T21:42:39.883 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:42:39.884 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:42:39.884 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:42:39.884 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:42:39.884 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790/0", 2024-09-10T21:42:39.884 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6790/0", 2024-09-10T21:42:39.884 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:42:39.884 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:42:39.884 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:42:39.884 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:42:39.884 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:42:39.884 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:42:39.884 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-10T21:42:39.885 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-10T21:42:39.885 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:42:39.885 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:42:39.885 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:42:39.885 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:42:39.885 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:42:39.885 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:42:39.885 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": [ 2024-09-10T21:42:39.885 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:42:39.885 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:42:39.885 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:42:39.885 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:42:39.885 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:42:39.885 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:42:39.886 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-10T21:42:39.886 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:42:39.886 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:42:39.886 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:42:39.886 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:42:39.886 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:42:39.886 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:42:39.886 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-10T21:42:39.886 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:42:39.886 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:42:39.886 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:42:39.886 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 2 2024-09-10T21:42:39.886 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:42:39.887 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:42:39.887 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:42:39.887 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-10T21:42:39.887 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-10T21:42:39.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:39.868+0000 7fb1a6e00640 1 -- 172.21.15.17:0/1856981585 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fb1800725e0 msgr2=0x7fb180074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:39.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:39.868+0000 7fb1a6e00640 1 --2- 172.21.15.17:0/1856981585 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fb1800725e0 0x7fb180074aa0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fb198001d40 tx=0x7fb198009040 comp rx=0 tx=0).stop 2024-09-10T21:42:39.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:39.868+0000 7fb1a6e00640 1 -- 172.21.15.17:0/1856981585 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fb1a00c6cc0 msgr2=0x7fb1a01fb7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:39.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:39.868+0000 7fb1a6e00640 1 --2- 172.21.15.17:0/1856981585 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fb1a00c6cc0 0x7fb1a01fb7c0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fb18c00ea20 tx=0x7fb18c00eee0 comp rx=0 tx=0).stop 2024-09-10T21:42:39.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:39.868+0000 7fb1a4e00640 1 -- 172.21.15.17:0/1856981585 reap_dead start 2024-09-10T21:42:39.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:39.868+0000 7fb1a4e00640 1 -- 172.21.15.17:0/1856981585 reap_dead start 2024-09-10T21:42:39.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:39.868+0000 7fb1a6e00640 1 -- 172.21.15.17:0/1856981585 shutdown_connections 2024-09-10T21:42:39.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:39.868+0000 7fb1a6e00640 1 -- 172.21.15.17:0/1856981585 >> 172.21.15.17:0/1856981585 conn(0x7fb1a006edb0 msgr2=0x7fb1a0154050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:39.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:39.872+0000 7fb1a6e00640 1 -- 172.21.15.17:0/1856981585 shutdown_connections 2024-09-10T21:42:39.888 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:39.880+0000 7fb1a6e00640 1 -- 172.21.15.17:0/1856981585 wait complete. 2024-09-10T21:42:39.996 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-10T21:42:39.996 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-09-10T21:42:39.996 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-10T21:42:40.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:40.604+0000 7f3733200640 1 -- 172.21.15.17:0/157795146 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f372c0c7210 msgr2=0x7f372c0c1e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:40.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:40.604+0000 7f3733200640 1 --2- 172.21.15.17:0/157795146 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f372c0c7210 0x7f372c0c1e10 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f3714005fd0 tx=0x7f3714020280 comp rx=0 tx=0).stop 2024-09-10T21:42:40.614 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:40.608+0000 7f3733200640 1 -- 172.21.15.17:0/157795146 shutdown_connections 2024-09-10T21:42:40.614 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:40.608+0000 7f3733200640 1 --2- 172.21.15.17:0/157795146 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f372c0c7210 0x7f372c0c1e10 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:40.614 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:40.608+0000 7f3733200640 1 --2- 172.21.15.17:0/157795146 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f372c167c20 0x7f372c0c6cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:40.614 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:40.608+0000 7f3733200640 1 -- 172.21.15.17:0/157795146 >> 172.21.15.17:0/157795146 conn(0x7f372c0700d0 msgr2=0x7f372c0704e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:40.614 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:40.608+0000 7f3733200640 1 -- 172.21.15.17:0/157795146 shutdown_connections 2024-09-10T21:42:40.614 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:40.608+0000 7f3733200640 1 -- 172.21.15.17:0/157795146 wait complete. 2024-09-10T21:42:40.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:40.612+0000 7f3733200640 1 Processor -- start 2024-09-10T21:42:40.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:40.612+0000 7f3733200640 1 -- start start 2024-09-10T21:42:40.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:40.616+0000 7f3733200640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f372c0c2350 0x7f372c1f10f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:40.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:40.616+0000 7f3733200640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f372c0c7210 0x7f372c206530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:40.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:40.616+0000 7f3733200640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f372c167c20 0x7f372c1f26a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:40.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:40.616+0000 7f3733200640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f372c0c18f0 con 0x7f372c0c7210 2024-09-10T21:42:40.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:40.616+0000 7f3733200640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f372c06e830 con 0x7f372c0c2350 2024-09-10T21:42:40.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:40.616+0000 7f3733200640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f372c06dc30 con 0x7f372c167c20 2024-09-10T21:42:40.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:40.616+0000 7f3723400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f372c0c7210 0x7f372c206530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:40.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:40.616+0000 7f3723400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f372c0c7210 0x7f372c206530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43110/0 (socket says 172.21.15.17:43110) 2024-09-10T21:42:40.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:40.616+0000 7f3723400640 1 -- 172.21.15.17:0/1289586214 learned_addr learned my addr 172.21.15.17:0/1289586214 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:42:40.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:40.620+0000 7f3723400640 1 -- 172.21.15.17:0/1289586214 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f372c167c20 msgr2=0x7f372c1f26a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-10T21:42:40.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:40.620+0000 7f3723400640 1 --2- 172.21.15.17:0/1289586214 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f372c167c20 0x7f372c1f26a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:40.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:40.620+0000 7f3723400640 1 -- 172.21.15.17:0/1289586214 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f372c0c2350 msgr2=0x7f372c1f10f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:42:40.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:40.620+0000 7f3723400640 1 --2- 172.21.15.17:0/1289586214 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f372c0c2350 0x7f372c1f10f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:40.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:40.620+0000 7f3723400640 1 -- 172.21.15.17:0/1289586214 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f372c1f1820 con 0x7f372c0c7210 2024-09-10T21:42:40.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:40.620+0000 7f3723400640 1 --2- 172.21.15.17:0/1289586214 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f372c0c7210 0x7f372c206530 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f3714009660 tx=0x7f37140b0b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:40.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:40.620+0000 7f3728c00640 1 -- 172.21.15.17:0/1289586214 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37140c1070 con 0x7f372c0c7210 2024-09-10T21:42:40.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:40.620+0000 7f3728c00640 1 -- 172.21.15.17:0/1289586214 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f37140b0d50 con 0x7f372c0c7210 2024-09-10T21:42:40.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:40.620+0000 7f3728c00640 1 -- 172.21.15.17:0/1289586214 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37140c1070 con 0x7f372c0c7210 2024-09-10T21:42:40.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:40.620+0000 7f3733200640 1 -- 172.21.15.17:0/1289586214 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f372c16b570 con 0x7f372c0c7210 2024-09-10T21:42:40.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:40.628+0000 7f3733200640 1 -- 172.21.15.17:0/1289586214 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f372c1f1e30 con 0x7f372c0c7210 2024-09-10T21:42:40.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:40.628+0000 7f3728c00640 1 -- 172.21.15.17:0/1289586214 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f3714004030 con 0x7f372c0c7210 2024-09-10T21:42:40.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:40.628+0000 7f3721600640 1 --2- 172.21.15.17:0/1289586214 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f36fc002a80 0x7f36fc004f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:40.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:40.628+0000 7f3721600640 1 -- 172.21.15.17:0/1289586214 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f36fc007ed0 con 0x7f36fc002a80 2024-09-10T21:42:40.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:40.628+0000 7f372be00640 1 --2- 172.21.15.17:0/1289586214 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f36fc002a80 0x7f36fc004f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:40.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:40.628+0000 7f3728c00640 1 --2- 172.21.15.17:0/1289586214 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f37080727e0 0x7f3708074ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:40.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:40.628+0000 7f372be00640 1 -- 172.21.15.17:0/1289586214 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f36fc0080d0 con 0x7f36fc002a80 2024-09-10T21:42:40.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:40.628+0000 7f3728c00640 1 -- 172.21.15.17:0/1289586214 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f371412f6b0 con 0x7f372c0c7210 2024-09-10T21:42:40.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:40.628+0000 7f372be00640 1 --2- 172.21.15.17:0/1289586214 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f36fc002a80 0x7f36fc004f40 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f372c1696b0 tx=0x7f371c00ea80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:40.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:40.628+0000 7f3728c00640 1 -- 172.21.15.17:0/1289586214 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f371c004230 con 0x7f36fc002a80 2024-09-10T21:42:40.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:40.632+0000 7f3731200640 1 --2- 172.21.15.17:0/1289586214 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f37080727e0 0x7f3708074ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:40.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:40.632+0000 7f3728c00640 1 -- 172.21.15.17:0/1289586214 <== mon.2 v2:172.21.15.31:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f371c0043d0 con 0x7f36fc002a80 2024-09-10T21:42:40.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:40.632+0000 7f3728c00640 1 -- 172.21.15.17:0/1289586214 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f36fc002a80 msgr2=0x7f36fc004f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:40.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:40.632+0000 7f3728c00640 1 --2- 172.21.15.17:0/1289586214 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f36fc002a80 0x7f36fc004f40 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f372c1696b0 tx=0x7f371c00ea80 comp rx=0 tx=0).stop 2024-09-10T21:42:40.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:40.632+0000 7f3728c00640 1 -- 172.21.15.17:0/1289586214 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f36fc002a80 msgr2=0x7f36fc004f40 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:42:40.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:40.632+0000 7f3728c00640 1 --2- 172.21.15.17:0/1289586214 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f36fc002a80 0x7f36fc004f40 secure :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f372c1696b0 tx=0x7f371c00ea80 comp rx=0 tx=0).stop 2024-09-10T21:42:40.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:40.632+0000 7f3731200640 1 --2- 172.21.15.17:0/1289586214 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f37080727e0 0x7f3708074ca0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f37240012f0 tx=0x7f3724007b70 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:40.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:40.652+0000 7f3721600640 1 --2- 172.21.15.17:0/1289586214 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f36fc006ec0 0x7f36fc007350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:40.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:40.652+0000 7f3721600640 1 -- 172.21.15.17:0/1289586214 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f36fc00a4b0 con 0x7f36fc006ec0 2024-09-10T21:42:40.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:40.652+0000 7f372be00640 1 --2- 172.21.15.17:0/1289586214 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f36fc006ec0 0x7f36fc007350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:40.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:40.652+0000 7f372be00640 1 -- 172.21.15.17:0/1289586214 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f36fc007890 con 0x7f36fc006ec0 2024-09-10T21:42:40.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:40.652+0000 7f372be00640 1 --2- 172.21.15.17:0/1289586214 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f36fc006ec0 0x7f36fc007350 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f371c018000 tx=0x7f371c004740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:40.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:40.656+0000 7f3728c00640 1 -- 172.21.15.17:0/1289586214 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f371c005320 con 0x7f36fc006ec0 2024-09-10T21:42:40.661 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-10T21:42:40.661 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:42:40.661 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:42:40.661 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-10T21:42:40.661 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 138, 2024-09-10T21:42:40.661 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 367005, 2024-09-10T21:42:40.662 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-10T21:42:40.662 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-10T21:42:40.662 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-10T21:42:40.662 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-10T21:42:40.662 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:42:40.662 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 16, 2024-09-10T21:42:40.662 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:42:40.662 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-10T21:42:40.662 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-10T21:42:40.662 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:42:40.662 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:42:40.664 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:42:40.664 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:42:40.664 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:42:40.664 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:42:40.664 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:42:40.664 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:42:40.664 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:42:40.664 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:42:40.664 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:42:40.665 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:42:40.665 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-10T21:42:40.665 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-10T21:42:40.665 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:42:40.665 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:42:40.665 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:42:40.665 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:42:40.665 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:42:40.665 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:42:40.665 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:42:40.665 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:42:40.665 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:42:40.665 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:42:40.666 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:42:40.666 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:42:40.666 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:42:40.666 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-10T21:42:40.666 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-10T21:42:40.666 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-10T21:42:40.666 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-10T21:42:40.666 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-10T21:42:40.666 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "5e2f023f-d6a6-49f4-a256-f4807dce3739", 2024-09-10T21:42:40.666 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:42:40.666 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:42:40.666 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-10T21:42:40.666 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-10T21:42:40.666 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-10T21:42:40.667 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-10T21:42:40.667 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-10T21:42:40.667 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-10T21:42:40.667 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-10T21:42:40.667 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:42:40.667 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-10T21:42:40.667 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:42:40.667 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:42:40.667 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:42:40.667 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:42:40.667 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:42:40.667 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:42:40.667 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:42:40.667 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:42:40.668 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:42:40.668 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:42:40.668 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:42:40.668 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:42:40.668 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-10T21:42:40.668 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:42:40.668 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-10T21:42:40.668 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:42:40.668 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:42:40.668 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:42:40.668 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:42:40.668 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:42:40.668 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:42:40.669 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:42:40.669 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-10T21:42:40.669 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:42:40.669 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:42:40.669 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:42:40.669 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:42:40.669 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-10T21:42:40.669 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:42:40.669 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:42:40.669 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:42:40.669 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:42:40.669 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-10T21:42:40.669 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-10T21:42:40.670 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:42:40.670 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:42:40.670 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:42:40.670 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:42:40.670 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:42:40.670 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:42:40.670 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:42:40.670 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:42:40.670 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:42:40.670 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:42:40.670 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:42:40.670 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3300", 2024-09-10T21:42:40.670 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:42:40.670 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:42:40.671 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:42:40.671 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:42:40.671 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789", 2024-09-10T21:42:40.671 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:42:40.671 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:42:40.671 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:42:40.671 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:42:40.671 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789/0", 2024-09-10T21:42:40.671 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6789/0", 2024-09-10T21:42:40.671 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:42:40.671 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:42:40.671 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:42:40.671 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:42:40.672 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:42:40.672 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:42:40.672 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:42:40.672 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:42:40.672 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:42:40.672 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:42:40.672 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:42:40.672 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3301", 2024-09-10T21:42:40.672 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:42:40.672 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:42:40.672 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:42:40.672 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:42:40.672 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790", 2024-09-10T21:42:40.673 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:42:40.673 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:42:40.673 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:42:40.673 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:42:40.673 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790/0", 2024-09-10T21:42:40.673 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6790/0", 2024-09-10T21:42:40.673 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:42:40.673 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:42:40.673 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:42:40.673 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:42:40.673 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:42:40.673 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:42:40.673 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-10T21:42:40.673 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-10T21:42:40.674 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:42:40.674 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:42:40.674 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:42:40.674 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:42:40.674 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:42:40.674 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:42:40.674 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": [ 2024-09-10T21:42:40.674 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:42:40.674 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:42:40.674 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:42:40.674 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 5 2024-09-10T21:42:40.674 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:42:40.674 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:42:40.674 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-10T21:42:40.675 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:42:40.675 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:42:40.675 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:42:40.675 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 7 2024-09-10T21:42:40.675 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:42:40.675 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:42:40.675 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-10T21:42:40.675 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:42:40.675 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:42:40.675 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:42:40.675 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 5 2024-09-10T21:42:40.675 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:42:40.675 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:42:40.676 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": [ 2024-09-10T21:42:40.676 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:42:40.676 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:42:40.676 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:42:40.676 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 2 2024-09-10T21:42:40.676 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:42:40.676 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:42:40.676 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:42:40.676 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-10T21:42:40.676 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-10T21:42:40.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:40.656+0000 7f3728c00640 1 -- 172.21.15.17:0/1289586214 <== mon.2 v2:172.21.15.31:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5044 (secure 0 0 0) 0x7f371c005320 con 0x7f36fc006ec0 2024-09-10T21:42:40.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:40.656+0000 7f3728c00640 1 -- 172.21.15.17:0/1289586214 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f36fc006ec0 msgr2=0x7f36fc007350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:40.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:40.656+0000 7f3728c00640 1 --2- 172.21.15.17:0/1289586214 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f36fc006ec0 0x7f36fc007350 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f371c018000 tx=0x7f371c004740 comp rx=0 tx=0).stop 2024-09-10T21:42:40.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:40.656+0000 7f3728c00640 1 -- 172.21.15.17:0/1289586214 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f36fc006ec0 msgr2=0x7f36fc007350 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:42:40.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:40.656+0000 7f3728c00640 1 --2- 172.21.15.17:0/1289586214 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f36fc006ec0 0x7f36fc007350 secure :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f371c018000 tx=0x7f371c004740 comp rx=0 tx=0).stop 2024-09-10T21:42:40.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:40.656+0000 7f3721600640 1 -- 172.21.15.17:0/1289586214 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f37080727e0 msgr2=0x7f3708074ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:40.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:40.656+0000 7f3721600640 1 --2- 172.21.15.17:0/1289586214 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f37080727e0 0x7f3708074ca0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f37240012f0 tx=0x7f3724007b70 comp rx=0 tx=0).stop 2024-09-10T21:42:40.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:40.656+0000 7f3721600640 1 -- 172.21.15.17:0/1289586214 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f372c0c7210 msgr2=0x7f372c206530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:40.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:40.656+0000 7f3721600640 1 --2- 172.21.15.17:0/1289586214 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f372c0c7210 0x7f372c206530 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f3714009660 tx=0x7f37140b0b60 comp rx=0 tx=0).stop 2024-09-10T21:42:40.677 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:40.660+0000 7f3731200640 1 -- 172.21.15.17:0/1289586214 reap_dead start 2024-09-10T21:42:40.678 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:40.660+0000 7f3731200640 1 -- 172.21.15.17:0/1289586214 reap_dead start 2024-09-10T21:42:40.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:40.680+0000 7f3721600640 1 -- 172.21.15.17:0/1289586214 shutdown_connections 2024-09-10T21:42:40.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:40.680+0000 7f3721600640 1 -- 172.21.15.17:0/1289586214 >> 172.21.15.17:0/1289586214 conn(0x7f372c0700d0 msgr2=0x7f372c164060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:40.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:40.680+0000 7f3721600640 1 -- 172.21.15.17:0/1289586214 shutdown_connections 2024-09-10T21:42:40.684 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:40.680+0000 7f3721600640 1 -- 172.21.15.17:0/1289586214 wait complete. 2024-09-10T21:42:40.741 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-10T21:42:40.742 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['c'] 2024-09-10T21:42:40.742 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2024-09-10T21:42:40.742 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.c 2024-09-10T21:42:40.742 INFO:tasks.mon_thrash.mon_thrasher:killing mon.c 2024-09-10T21:42:40.742 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-10T21:42:40.742 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:42:40.784 INFO:tasks.ceph.mon.c:Stopped 2024-09-10T21:42:40.784 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2024-09-10T21:42:40.785 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-10T21:42:41.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:41.088+0000 7f1751200640 1 -- 172.21.15.17:0/2206997116 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f174c0caca0 msgr2=0x7f174c0cb150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:41.092 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:41.088+0000 7f1751200640 1 --2- 172.21.15.17:0/2206997116 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f174c0caca0 0x7f174c0cb150 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f1744009bc0 tx=0x7f174401f3d0 comp rx=0 tx=0).stop 2024-09-10T21:42:41.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:41.088+0000 7f1751200640 1 -- 172.21.15.17:0/2206997116 shutdown_connections 2024-09-10T21:42:41.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:41.088+0000 7f1751200640 1 --2- 172.21.15.17:0/2206997116 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f174c0caca0 0x7f174c0cb150 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:41.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:41.088+0000 7f1751200640 1 --2- 172.21.15.17:0/2206997116 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f174c0c99f0 0x7f174c0d0000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:41.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:41.088+0000 7f1751200640 1 -- 172.21.15.17:0/2206997116 >> 172.21.15.17:0/2206997116 conn(0x7f174c06d700 msgr2=0x7f174c068990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:41.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:41.092+0000 7f1751200640 1 -- 172.21.15.17:0/2206997116 shutdown_connections 2024-09-10T21:42:41.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:41.092+0000 7f1751200640 1 -- 172.21.15.17:0/2206997116 wait complete. 2024-09-10T21:42:41.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:41.096+0000 7f1751200640 1 Processor -- start 2024-09-10T21:42:41.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:41.108+0000 7f1751200640 1 -- start start 2024-09-10T21:42:41.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:41.108+0000 7f1751200640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f174c0b30a0 0x7f174c0c8d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:41.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:41.108+0000 7f1751200640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f174c0c99f0 0x7f174c0cfb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:41.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:41.108+0000 7f1751200640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f174c0caca0 0x7f174c1f03c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:41.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:41.108+0000 7f1751200640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f174c15a8d0 con 0x7f174c0caca0 2024-09-10T21:42:41.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:41.108+0000 7f1751200640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f174c15a5f0 con 0x7f174c0c99f0 2024-09-10T21:42:41.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:41.108+0000 7f1751200640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f174c15abb0 con 0x7f174c0b30a0 2024-09-10T21:42:41.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:41.112+0000 7f174b400640 1 -- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f174c0b30a0 msgr2=0x7f174c0c8d30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:3301/0 2024-09-10T21:42:41.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:41.112+0000 7f174b400640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f174c0b30a0 0x7f174c0c8d30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:42:41.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:41.112+0000 7f174aa00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f174c0c99f0 0x7f174c0cfb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:41.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:41.116+0000 7f174be00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f174c0caca0 0x7f174c1f03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:41.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:41.116+0000 7f174be00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f174c0caca0 0x7f174c1f03c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:43196/0 (socket says 172.21.15.17:43196) 2024-09-10T21:42:41.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:41.116+0000 7f174be00640 1 -- 172.21.15.17:0/1351737597 learned_addr learned my addr 172.21.15.17:0/1351737597 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:42:41.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:41.116+0000 7f174aa00640 1 -- 172.21.15.17:0/1351737597 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f174c0b30a0 msgr2=0x7f174c0c8d30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:42:41.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:41.116+0000 7f174aa00640 1 --2- 172.21.15.17:0/1351737597 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f174c0b30a0 0x7f174c0c8d30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:41.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:41.116+0000 7f174aa00640 1 -- 172.21.15.17:0/1351737597 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f174c0caca0 msgr2=0x7f174c1f03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:41.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:41.116+0000 7f174aa00640 1 --2- 172.21.15.17:0/1351737597 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f174c0caca0 0x7f174c1f03c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:41.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:41.116+0000 7f174aa00640 1 -- 172.21.15.17:0/1351737597 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f174c1e3b00 con 0x7f174c0c99f0 2024-09-10T21:42:41.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:41.116+0000 7f174be00640 1 --2- 172.21.15.17:0/1351737597 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f174c0caca0 0x7f174c1f03c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:42:41.122 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:41.116+0000 7f174aa00640 1 --2- 172.21.15.17:0/1351737597 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f174c0c99f0 0x7f174c0cfb90 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f174c0b5900 tx=0x7f173800c890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:41.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:41.116+0000 7f1743400640 1 -- 172.21.15.17:0/1351737597 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1738011790 con 0x7f174c0c99f0 2024-09-10T21:42:41.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:41.116+0000 7f1743400640 1 -- 172.21.15.17:0/1351737597 <== mon.1 v2:172.21.15.31:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1738017070 con 0x7f174c0c99f0 2024-09-10T21:42:41.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:41.116+0000 7f1743400640 1 -- 172.21.15.17:0/1351737597 <== mon.1 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1738011ce0 con 0x7f174c0c99f0 2024-09-10T21:42:41.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:41.120+0000 7f1751200640 1 -- 172.21.15.17:0/1351737597 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f174c1e3de0 con 0x7f174c0c99f0 2024-09-10T21:42:41.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:41.120+0000 7f1751200640 1 -- 172.21.15.17:0/1351737597 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f174c1e5a80 con 0x7f174c0c99f0 2024-09-10T21:42:41.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:41.124+0000 7f1751200640 1 -- 172.21.15.17:0/1351737597 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f174c1e4300 con 0x7f174c0c99f0 2024-09-10T21:42:41.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:41.132+0000 7f1743400640 1 -- 172.21.15.17:0/1351737597 <== mon.1 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f173801c460 con 0x7f174c0c99f0 2024-09-10T21:42:41.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:41.132+0000 7f1743400640 1 --2- 172.21.15.17:0/1351737597 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f172c0725e0 0x7f172c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:41.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:41.132+0000 7f1743400640 1 -- 172.21.15.17:0/1351737597 <== mon.1 v2:172.21.15.31:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f1738093da0 con 0x7f174c0c99f0 2024-09-10T21:42:41.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:41.136+0000 7f174b400640 1 --2- 172.21.15.17:0/1351737597 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f172c0725e0 0x7f172c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:41.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:41.140+0000 7f1743400640 1 -- 172.21.15.17:0/1351737597 <== mon.1 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f173805f0f0 con 0x7f174c0c99f0 2024-09-10T21:42:41.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:41.148+0000 7f174b400640 1 --2- 172.21.15.17:0/1351737597 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f172c0725e0 0x7f172c074aa0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f173c001060 tx=0x7f173c006d70 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:41.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:41.724+0000 7f1751200640 1 -- 172.21.15.17:0/1351737597 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f174c0c39c0 con 0x7f174c0c99f0 2024-09-10T21:42:41.734 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:41.728+0000 7f1743400640 1 -- 172.21.15.17:0/1351737597 <== mon.1 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1558 (secure 0 0 0) 0x7f173805e9f0 con 0x7f174c0c99f0 2024-09-10T21:42:41.734 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":138,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":17,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5e2f023f-d6a6-49f4-a256-f4807dce3739","modified":"2024-09-10T21:23:46.892130Z","created":"2024-09-10T21:23:46.892130Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3301","nonce":0},{"type":"v1","addr":"172.21.15.31:6790","nonce":0}]},"addr":"172.21.15.31:6790/0","public_addr":"172.21.15.31:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-10T21:42:41.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:41.732+0000 7f1751200640 1 -- 172.21.15.17:0/1351737597 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f172c0725e0 msgr2=0x7f172c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:41.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:41.736+0000 7f1751200640 1 --2- 172.21.15.17:0/1351737597 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f172c0725e0 0x7f172c074aa0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f173c001060 tx=0x7f173c006d70 comp rx=0 tx=0).stop 2024-09-10T21:42:41.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:41.736+0000 7f1751200640 1 -- 172.21.15.17:0/1351737597 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f174c0c99f0 msgr2=0x7f174c0cfb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:41.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:41.736+0000 7f1751200640 1 --2- 172.21.15.17:0/1351737597 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f174c0c99f0 0x7f174c0cfb90 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f174c0b5900 tx=0x7f173800c890 comp rx=0 tx=0).stop 2024-09-10T21:42:41.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:41.736+0000 7f1751200640 1 -- 172.21.15.17:0/1351737597 shutdown_connections 2024-09-10T21:42:41.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:41.736+0000 7f1751200640 1 --2- 172.21.15.17:0/1351737597 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f172c0725e0 0x7f172c074aa0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:41.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:41.736+0000 7f1751200640 1 --2- 172.21.15.17:0/1351737597 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f174c0caca0 0x7f174c1f03c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:41.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:41.736+0000 7f1751200640 1 --2- 172.21.15.17:0/1351737597 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f174c0c99f0 0x7f174c0cfb90 secure :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f174c0b5900 tx=0x7f173800c890 comp rx=0 tx=0).stop 2024-09-10T21:42:41.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:41.736+0000 7f1751200640 1 --2- 172.21.15.17:0/1351737597 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f174c0b30a0 0x7f174c0c8d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:41.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:41.736+0000 7f1751200640 1 -- 172.21.15.17:0/1351737597 >> 172.21.15.17:0/1351737597 conn(0x7f174c06d700 msgr2=0x7f174c0b34a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:41.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:41.744+0000 7f1751200640 1 -- 172.21.15.17:0/1351737597 shutdown_connections 2024-09-10T21:42:41.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:41.744+0000 7f1751200640 1 -- 172.21.15.17:0/1351737597 wait complete. 2024-09-10T21:42:44.810 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-10T21:42:45.488 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:45.475+0000 7fc95ae00640 1 -- 172.21.15.17:0/2546193922 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc9540c8d10 msgr2=0x7fc9540c1ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:45.488 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:45.475+0000 7fc95ae00640 1 --2- 172.21.15.17:0/2546193922 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc9540c8d10 0x7fc9540c1ae0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fc95406f1e0 tx=0x7fc94401f0f0 comp rx=0 tx=0).stop 2024-09-10T21:42:45.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:45.479+0000 7fc95ae00640 1 -- 172.21.15.17:0/2546193922 shutdown_connections 2024-09-10T21:42:45.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:45.479+0000 7fc95ae00640 1 --2- 172.21.15.17:0/2546193922 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc9540c90f0 0x7fc954074910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:45.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:45.479+0000 7fc95ae00640 1 --2- 172.21.15.17:0/2546193922 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc9540c8d10 0x7fc9540c1ae0 secure :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fc95406f1e0 tx=0x7fc94401f0f0 comp rx=0 tx=0).stop 2024-09-10T21:42:45.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:45.479+0000 7fc95ae00640 1 -- 172.21.15.17:0/2546193922 >> 172.21.15.17:0/2546193922 conn(0x7fc954070160 msgr2=0x7fc954070570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:45.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:45.487+0000 7fc95ae00640 1 -- 172.21.15.17:0/2546193922 shutdown_connections 2024-09-10T21:42:45.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:45.487+0000 7fc95ae00640 1 -- 172.21.15.17:0/2546193922 wait complete. 2024-09-10T21:42:45.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:45.491+0000 7fc95ae00640 1 Processor -- start 2024-09-10T21:42:45.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:45.495+0000 7fc95ae00640 1 -- start start 2024-09-10T21:42:45.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:45.499+0000 7fc95ae00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc9540c7c10 0x7fc9540cc3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:45.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:45.503+0000 7fc95ae00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc9540c90f0 0x7fc9540cc910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:45.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:45.503+0000 7fc95ae00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc95415ff10 0x7fc9541658a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:45.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:45.503+0000 7fc95ae00640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fc95406e640 con 0x7fc9540c7c10 2024-09-10T21:42:45.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:45.503+0000 7fc95ae00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fc95406e100 con 0x7fc95415ff10 2024-09-10T21:42:45.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:45.503+0000 7fc95ae00640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7fc9540c87c0 con 0x7fc9540c90f0 2024-09-10T21:42:45.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:45.503+0000 7fc94be00640 1 -- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc9540c90f0 msgr2=0x7fc9540cc910 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:3301/0 2024-09-10T21:42:45.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:45.503+0000 7fc94be00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc9540c90f0 0x7fc9540cc910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:42:45.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:45.503+0000 7fc958e00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc95415ff10 0x7fc9541658a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:45.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:45.503+0000 7fc958e00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc95415ff10 0x7fc9541658a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.17:52630/0 (socket says 172.21.15.17:52630) 2024-09-10T21:42:45.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:45.503+0000 7fc958e00640 1 -- 172.21.15.17:0/1008813665 learned_addr learned my addr 172.21.15.17:0/1008813665 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:42:45.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:45.503+0000 7fc953e00640 1 --2- 172.21.15.17:0/1008813665 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc9540c7c10 0x7fc9540cc3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:45.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:45.503+0000 7fc953e00640 1 -- 172.21.15.17:0/1008813665 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc9540c90f0 msgr2=0x7fc9540cc910 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:42:45.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:45.503+0000 7fc953e00640 1 --2- 172.21.15.17:0/1008813665 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc9540c90f0 0x7fc9540cc910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:45.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:45.503+0000 7fc953e00640 1 -- 172.21.15.17:0/1008813665 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc95415ff10 msgr2=0x7fc9541658a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:45.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:45.503+0000 7fc953e00640 1 --2- 172.21.15.17:0/1008813665 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc95415ff10 0x7fc9541658a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:45.515 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:45.503+0000 7fc953e00640 1 -- 172.21.15.17:0/1008813665 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc95408bd80 con 0x7fc9540c7c10 2024-09-10T21:42:45.516 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:45.503+0000 7fc958e00640 1 --2- 172.21.15.17:0/1008813665 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc95415ff10 0x7fc9541658a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:42:45.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:45.511+0000 7fc953e00640 1 --2- 172.21.15.17:0/1008813665 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc9540c7c10 0x7fc9540cc3d0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fc9440af000 tx=0x7fc9440a8ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:45.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:45.515+0000 7fc950c00640 1 -- 172.21.15.17:0/1008813665 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9440b5040 con 0x7fc9540c7c10 2024-09-10T21:42:45.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:45.515+0000 7fc950c00640 1 -- 172.21.15.17:0/1008813665 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc944003970 con 0x7fc9540c7c10 2024-09-10T21:42:45.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:45.515+0000 7fc950c00640 1 -- 172.21.15.17:0/1008813665 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9440b5040 con 0x7fc9540c7c10 2024-09-10T21:42:45.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:45.515+0000 7fc95ae00640 1 -- 172.21.15.17:0/1008813665 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc95408bae0 con 0x7fc9540c7c10 2024-09-10T21:42:45.525 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:45.519+0000 7fc95ae00640 1 -- 172.21.15.17:0/1008813665 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc954164e80 con 0x7fc9540c7c10 2024-09-10T21:42:45.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:45.519+0000 7fc950c00640 1 -- 172.21.15.17:0/1008813665 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fc944003b60 con 0x7fc9540c7c10 2024-09-10T21:42:45.528 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:45.523+0000 7fc95ae00640 1 -- 172.21.15.17:0/1008813665 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc954072cb0 con 0x7fc9540c7c10 2024-09-10T21:42:45.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:45.523+0000 7fc950c00640 1 --2- 172.21.15.17:0/1008813665 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fc9300727c0 0x7fc930074c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:45.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:45.523+0000 7fc950c00640 1 -- 172.21.15.17:0/1008813665 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fc94412f3d0 con 0x7fc9540c7c10 2024-09-10T21:42:45.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:45.523+0000 7fc94be00640 1 --2- 172.21.15.17:0/1008813665 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fc9300727c0 0x7fc930074c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:45.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:45.527+0000 7fc94be00640 1 --2- 172.21.15.17:0/1008813665 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fc9300727c0 0x7fc930074c80 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fc95408c370 tx=0x7fc93c006d70 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:45.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:45.551+0000 7fc950c00640 1 -- 172.21.15.17:0/1008813665 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fc944018c50 con 0x7fc9540c7c10 2024-09-10T21:42:46.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:46.119+0000 7fc95ae00640 1 -- 172.21.15.17:0/1008813665 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fc9540c79f0 con 0x7fc9540c7c10 2024-09-10T21:42:46.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:46.123+0000 7fc950c00640 1 -- 172.21.15.17:0/1008813665 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1558 (secure 0 0 0) 0x7fc9440fa070 con 0x7fc9540c7c10 2024-09-10T21:42:46.137 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":138,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":21,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5e2f023f-d6a6-49f4-a256-f4807dce3739","modified":"2024-09-10T21:23:46.892130Z","created":"2024-09-10T21:23:46.892130Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3301","nonce":0},{"type":"v1","addr":"172.21.15.31:6790","nonce":0}]},"addr":"172.21.15.31:6790/0","public_addr":"172.21.15.31:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-10T21:42:46.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:46.131+0000 7fc949600640 1 -- 172.21.15.17:0/1008813665 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fc9300727c0 msgr2=0x7fc930074c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:46.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:46.131+0000 7fc949600640 1 --2- 172.21.15.17:0/1008813665 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fc9300727c0 0x7fc930074c80 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fc95408c370 tx=0x7fc93c006d70 comp rx=0 tx=0).stop 2024-09-10T21:42:46.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:46.131+0000 7fc949600640 1 -- 172.21.15.17:0/1008813665 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc9540c7c10 msgr2=0x7fc9540cc3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:46.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:46.131+0000 7fc949600640 1 --2- 172.21.15.17:0/1008813665 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc9540c7c10 0x7fc9540cc3d0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fc9440af000 tx=0x7fc9440a8ce0 comp rx=0 tx=0).stop 2024-09-10T21:42:46.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:46.131+0000 7fc949600640 1 -- 172.21.15.17:0/1008813665 shutdown_connections 2024-09-10T21:42:46.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:46.131+0000 7fc949600640 1 --2- 172.21.15.17:0/1008813665 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fc9300727c0 0x7fc930074c80 secure :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fc95408c370 tx=0x7fc93c006d70 comp rx=0 tx=0).stop 2024-09-10T21:42:46.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:46.131+0000 7fc949600640 1 --2- 172.21.15.17:0/1008813665 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc95415ff10 0x7fc9541658a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:46.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:46.131+0000 7fc949600640 1 --2- 172.21.15.17:0/1008813665 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc9540c90f0 0x7fc9540cc910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:46.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:46.131+0000 7fc949600640 1 --2- 172.21.15.17:0/1008813665 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc9540c7c10 0x7fc9540cc3d0 secure :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fc9440af000 tx=0x7fc9440a8ce0 comp rx=0 tx=0).stop 2024-09-10T21:42:46.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:46.131+0000 7fc949600640 1 -- 172.21.15.17:0/1008813665 >> 172.21.15.17:0/1008813665 conn(0x7fc954070160 msgr2=0x7fc954160de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:46.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:46.135+0000 7fc949600640 1 -- 172.21.15.17:0/1008813665 shutdown_connections 2024-09-10T21:42:46.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:46.135+0000 7fc949600640 1 -- 172.21.15.17:0/1008813665 wait complete. 2024-09-10T21:42:49.217 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-10T21:42:55.518 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:42:55.522 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:42:55.574 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:55.567+0000 7f28f3e00640 1 -- 172.21.15.17:0/3343561229 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f28d0003b30 msgr2=0x7f28d0006620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:55.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:55.567+0000 7f28f3e00640 1 --2- 172.21.15.17:0/3343561229 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f28d0003b30 0x7f28d0006620 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f28d0006060 tx=0x7f28e401d2b0 comp rx=0 tx=0).stop 2024-09-10T21:42:55.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:55.567+0000 7f28f3e00640 1 -- 172.21.15.17:0/3343561229 shutdown_connections 2024-09-10T21:42:55.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:55.567+0000 7f28f3e00640 1 --2- 172.21.15.17:0/3343561229 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f28d0003b30 0x7f28d0006620 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:55.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:55.567+0000 7f28f3e00640 1 --2- 172.21.15.17:0/3343561229 >> v2:172.21.15.31:3300/0 conn(0x7f28d0006b60 0x7f28d0007010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:55.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:55.567+0000 7f28f3e00640 1 -- 172.21.15.17:0/3343561229 >> 172.21.15.17:0/3343561229 conn(0x7f28f406fe60 msgr2=0x7f28f4070270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:55.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:55.567+0000 7f28f3e00640 1 -- 172.21.15.17:0/3343561229 shutdown_connections 2024-09-10T21:42:55.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:55.567+0000 7f28f3e00640 1 -- 172.21.15.17:0/3343561229 wait complete. 2024-09-10T21:42:55.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:55.743+0000 7f28f3e00640 1 Processor -- start 2024-09-10T21:42:55.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:55.743+0000 7f28f3e00640 1 -- start start 2024-09-10T21:42:55.752 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:55.747+0000 7f28f3e00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f28d0003b30 0x7f28f40c5530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:55.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:55.747+0000 7f28f3e00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f28d0006b60 0x7f28f40c43a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:55.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:55.747+0000 7f28f3e00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f28f40c8220 0x7f28f415a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:55.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:55.747+0000 7f28f3e00640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f28f40c17b0 con 0x7f28f40c8220 2024-09-10T21:42:55.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:55.747+0000 7f28f3e00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f28f406f1d0 con 0x7f28d0003b30 2024-09-10T21:42:55.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:55.747+0000 7f28f3e00640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f28f40b1ee0 con 0x7f28d0006b60 2024-09-10T21:42:55.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:55.747+0000 7f28f3400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f28f40c8220 0x7f28f415a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:55.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:55.747+0000 7f28f3400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f28f40c8220 0x7f28f415a120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37020/0 (socket says 172.21.15.17:37020) 2024-09-10T21:42:55.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:55.747+0000 7f28f3400640 1 -- 172.21.15.17:0/967549428 learned_addr learned my addr 172.21.15.17:0/967549428 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:42:55.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:55.747+0000 7f28f2a00640 1 --2- 172.21.15.17:0/967549428 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f28d0003b30 0x7f28f40c5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:55.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:55.747+0000 7f28f3400640 1 -- 172.21.15.17:0/967549428 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f28d0006b60 msgr2=0x7f28f40c43a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:42:55.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:55.751+0000 7f28f3400640 1 --2- 172.21.15.17:0/967549428 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f28d0006b60 0x7f28f40c43a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:55.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:55.751+0000 7f28f3400640 1 -- 172.21.15.17:0/967549428 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f28d0003b30 msgr2=0x7f28f40c5530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:55.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:55.751+0000 7f28f3400640 1 --2- 172.21.15.17:0/967549428 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f28d0003b30 0x7f28f40c5530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:55.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:55.751+0000 7f28f3400640 1 -- 172.21.15.17:0/967549428 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28f41f7c20 con 0x7f28f40c8220 2024-09-10T21:42:55.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:55.751+0000 7f28f3400640 1 --2- 172.21.15.17:0/967549428 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f28f40c8220 0x7f28f415a120 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f28f4154310 tx=0x7f28ec0019c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:55.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:55.759+0000 7f28eaa00640 1 -- 172.21.15.17:0/967549428 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28ec0107d0 con 0x7f28f40c8220 2024-09-10T21:42:55.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:55.759+0000 7f28eaa00640 1 -- 172.21.15.17:0/967549428 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f28ec010970 con 0x7f28f40c8220 2024-09-10T21:42:55.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:55.763+0000 7f28eaa00640 1 -- 172.21.15.17:0/967549428 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28ec010da0 con 0x7f28f40c8220 2024-09-10T21:42:55.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:55.763+0000 7f28f3e00640 1 -- 172.21.15.17:0/967549428 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28f40c62c0 con 0x7f28f40c8220 2024-09-10T21:42:55.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:55.763+0000 7f28f3e00640 1 -- 172.21.15.17:0/967549428 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28f40c35e0 con 0x7f28f40c8220 2024-09-10T21:42:55.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:55.771+0000 7f28eaa00640 1 -- 172.21.15.17:0/967549428 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f28ec011460 con 0x7f28f40c8220 2024-09-10T21:42:55.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:55.771+0000 7f28f3e00640 1 -- 172.21.15.17:0/967549428 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28f4153780 con 0x7f28f40c8220 2024-09-10T21:42:55.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:55.779+0000 7f28eaa00640 1 --2- 172.21.15.17:0/967549428 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f28d0073ff0 0x7f28d00764b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:55.793 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:55.779+0000 7f28eaa00640 1 -- 172.21.15.17:0/967549428 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f28ec093bc0 con 0x7f28f40c8220 2024-09-10T21:42:55.796 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:55.791+0000 7f28f2a00640 1 --2- 172.21.15.17:0/967549428 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f28d0073ff0 0x7f28d00764b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:55.807 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:55.799+0000 7f28f2a00640 1 --2- 172.21.15.17:0/967549428 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f28d0073ff0 0x7f28d00764b0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f28dc006120 tx=0x7f28dc00b040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:55.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:55.811+0000 7f28eaa00640 1 -- 172.21.15.17:0/967549428 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f28ec05ee80 con 0x7f28f40c8220 2024-09-10T21:42:56.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:56.335+0000 7f28f3e00640 1 -- 172.21.15.17:0/967549428 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f28f4150b80 con 0x7f28f40c8220 2024-09-10T21:42:56.342 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":142,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":1,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5e2f023f-d6a6-49f4-a256-f4807dce3739","modified":"2024-09-10T21:23:46.892130Z","created":"2024-09-10T21:23:46.892130Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3301","nonce":0},{"type":"v1","addr":"172.21.15.31:6790","nonce":0}]},"addr":"172.21.15.31:6790/0","public_addr":"172.21.15.31:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-10T21:42:56.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:56.335+0000 7f28eaa00640 1 -- 172.21.15.17:0/967549428 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1551 (secure 0 0 0) 0x7f28ec05e780 con 0x7f28f40c8220 2024-09-10T21:42:56.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:56.339+0000 7f28f3e00640 1 -- 172.21.15.17:0/967549428 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f28d0073ff0 msgr2=0x7f28d00764b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:56.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:56.339+0000 7f28f3e00640 1 --2- 172.21.15.17:0/967549428 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f28d0073ff0 0x7f28d00764b0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f28dc006120 tx=0x7f28dc00b040 comp rx=0 tx=0).stop 2024-09-10T21:42:56.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:56.339+0000 7f28f3e00640 1 -- 172.21.15.17:0/967549428 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f28f40c8220 msgr2=0x7f28f415a120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:56.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:56.339+0000 7f28f3e00640 1 --2- 172.21.15.17:0/967549428 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f28f40c8220 0x7f28f415a120 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f28f4154310 tx=0x7f28ec0019c0 comp rx=0 tx=0).stop 2024-09-10T21:42:56.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:56.339+0000 7f28f3e00640 1 -- 172.21.15.17:0/967549428 shutdown_connections 2024-09-10T21:42:56.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:56.339+0000 7f28f3e00640 1 --2- 172.21.15.17:0/967549428 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f28d0073ff0 0x7f28d00764b0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:56.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:56.339+0000 7f28f3e00640 1 --2- 172.21.15.17:0/967549428 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f28f40c8220 0x7f28f415a120 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:56.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:56.339+0000 7f28f3e00640 1 --2- 172.21.15.17:0/967549428 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f28d0006b60 0x7f28f40c43a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:56.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:56.339+0000 7f28f3e00640 1 --2- 172.21.15.17:0/967549428 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f28d0003b30 0x7f28f40c5530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:56.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:56.339+0000 7f28f3e00640 1 -- 172.21.15.17:0/967549428 >> 172.21.15.17:0/967549428 conn(0x7f28f406fe60 msgr2=0x7f28f4151e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:56.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:56.339+0000 7f28f3e00640 1 -- 172.21.15.17:0/967549428 shutdown_connections 2024-09-10T21:42:56.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:56.339+0000 7f28f3e00640 1 -- 172.21.15.17:0/967549428 wait complete. 2024-09-10T21:42:56.681 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2024-09-10T21:42:56.681 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon a 2024-09-10T21:42:56.681 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-10T21:42:57.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:57.083+0000 7feb35600640 1 -- 172.21.15.17:0/3709663889 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feb300c9540 msgr2=0x7feb30153940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:57.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:57.083+0000 7feb35600640 1 --2- 172.21.15.17:0/3709663889 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feb300c9540 0x7feb30153940 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7feb300737b0 tx=0x7feb2801f370 comp rx=0 tx=0).stop 2024-09-10T21:42:57.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:57.083+0000 7feb35600640 1 -- 172.21.15.17:0/3709663889 shutdown_connections 2024-09-10T21:42:57.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:57.083+0000 7feb35600640 1 --2- 172.21.15.17:0/3709663889 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feb300c9540 0x7feb30153940 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:57.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:57.083+0000 7feb35600640 1 --2- 172.21.15.17:0/3709663889 >> v2:172.21.15.17:3300/0 conn(0x7feb30074980 0x7feb300b1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:57.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:57.083+0000 7feb35600640 1 -- 172.21.15.17:0/3709663889 >> 172.21.15.17:0/3709663889 conn(0x7feb300700d0 msgr2=0x7feb300704e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:57.091 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:57.083+0000 7feb35600640 1 -- 172.21.15.17:0/3709663889 shutdown_connections 2024-09-10T21:42:57.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:57.083+0000 7feb35600640 1 -- 172.21.15.17:0/3709663889 wait complete. 2024-09-10T21:42:57.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:57.087+0000 7feb35600640 1 Processor -- start 2024-09-10T21:42:57.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:57.099+0000 7feb35600640 1 -- start start 2024-09-10T21:42:57.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:57.103+0000 7feb35600640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feb30074980 0x7feb30152b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:57.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:57.111+0000 7feb35600640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feb300c9160 0x7feb3013b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:57.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:57.111+0000 7feb35600640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7feb300c9540 0x7feb3013be30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:57.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:57.111+0000 7feb35600640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7feb30073e70 con 0x7feb30074980 2024-09-10T21:42:57.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:57.111+0000 7feb35600640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7feb30073b90 con 0x7feb300c9160 2024-09-10T21:42:57.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:57.111+0000 7feb35600640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7feb30073890 con 0x7feb300c9540 2024-09-10T21:42:57.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:57.111+0000 7feb2f400640 1 -- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7feb300c9540 msgr2=0x7feb3013be30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:3301/0 2024-09-10T21:42:57.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:57.111+0000 7feb2f400640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7feb300c9540 0x7feb3013be30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:42:57.120 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:57.115+0000 7feb2e000640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feb300c9160 0x7feb3013b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:57.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:57.115+0000 7feb2e000640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feb300c9160 0x7feb3013b8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.17:37728/0 (socket says 172.21.15.17:37728) 2024-09-10T21:42:57.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:57.115+0000 7feb2e000640 1 -- 172.21.15.17:0/34061484 learned_addr learned my addr 172.21.15.17:0/34061484 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:42:57.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:57.115+0000 7feb2ea00640 1 --2- 172.21.15.17:0/34061484 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feb30074980 0x7feb30152b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:57.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:57.115+0000 7feb2e000640 1 -- 172.21.15.17:0/34061484 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7feb300c9540 msgr2=0x7feb3013be30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:42:57.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:57.115+0000 7feb2e000640 1 --2- 172.21.15.17:0/34061484 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7feb300c9540 0x7feb3013be30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:57.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:57.115+0000 7feb2e000640 1 -- 172.21.15.17:0/34061484 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feb30074980 msgr2=0x7feb30152b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:57.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:57.115+0000 7feb2e000640 1 --2- 172.21.15.17:0/34061484 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feb30074980 0x7feb30152b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:57.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:57.115+0000 7feb2e000640 1 -- 172.21.15.17:0/34061484 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb300ca7f0 con 0x7feb300c9160 2024-09-10T21:42:57.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:57.115+0000 7feb2e000640 1 --2- 172.21.15.17:0/34061484 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feb300c9160 0x7feb3013b8f0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7feb1c00c970 tx=0x7feb1c00ce30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:57.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:57.119+0000 7feb26a00640 1 -- 172.21.15.17:0/34061484 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb1c007c70 con 0x7feb300c9160 2024-09-10T21:42:57.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:57.119+0000 7feb26a00640 1 -- 172.21.15.17:0/34061484 <== mon.1 v2:172.21.15.31:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7feb1c00f450 con 0x7feb300c9160 2024-09-10T21:42:57.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:57.119+0000 7feb26a00640 1 -- 172.21.15.17:0/34061484 <== mon.1 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb1c00f8a0 con 0x7feb300c9160 2024-09-10T21:42:57.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:57.119+0000 7feb35600640 1 -- 172.21.15.17:0/34061484 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb301513d0 con 0x7feb300c9160 2024-09-10T21:42:57.126 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:57.119+0000 7feb35600640 1 -- 172.21.15.17:0/34061484 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb30151e10 con 0x7feb300c9160 2024-09-10T21:42:57.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:57.119+0000 7feb26a00640 1 -- 172.21.15.17:0/34061484 <== mon.1 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7feb1c00fa40 con 0x7feb300c9160 2024-09-10T21:42:57.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:57.123+0000 7feb35600640 1 --2- 172.21.15.17:0/34061484 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feb301515d0 0x7feb301eebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:57.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:57.123+0000 7feb35600640 1 -- 172.21.15.17:0/34061484 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7feb30142ae0 con 0x7feb301515d0 2024-09-10T21:42:57.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:57.127+0000 7feb2ea00640 1 --2- 172.21.15.17:0/34061484 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feb301515d0 0x7feb301eebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:57.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:57.127+0000 7feb26a00640 1 --2- 172.21.15.17:0/34061484 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7feb100725c0 0x7feb10074a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:57.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:57.127+0000 7feb26a00640 1 -- 172.21.15.17:0/34061484 <== mon.1 v2:172.21.15.31:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7feb1c0935d0 con 0x7feb300c9160 2024-09-10T21:42:57.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:57.127+0000 7feb2f400640 1 --2- 172.21.15.17:0/34061484 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7feb100725c0 0x7feb10074a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:57.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:57.127+0000 7feb2f400640 1 --2- 172.21.15.17:0/34061484 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7feb100725c0 0x7feb10074a80 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7feb280b1000 tx=0x7feb2801dbb0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:57.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:57.135+0000 7feb2ea00640 1 -- 172.21.15.17:0/34061484 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7feb300c34c0 con 0x7feb301515d0 2024-09-10T21:42:57.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:57.139+0000 7feb2ea00640 1 --2- 172.21.15.17:0/34061484 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feb301515d0 0x7feb301eebd0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7feb20001ec0 tx=0x7feb20011520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:57.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:57.139+0000 7feb26a00640 1 -- 172.21.15.17:0/34061484 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb2001c020 con 0x7feb301515d0 2024-09-10T21:42:57.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:57.139+0000 7feb26a00640 1 -- 172.21.15.17:0/34061484 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7feb20011e00 con 0x7feb301515d0 2024-09-10T21:42:57.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:57.139+0000 7feb26a00640 1 -- 172.21.15.17:0/34061484 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feb301515d0 msgr2=0x7feb301eebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:57.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:57.139+0000 7feb26a00640 1 --2- 172.21.15.17:0/34061484 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feb301515d0 0x7feb301eebd0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7feb20001ec0 tx=0x7feb20011520 comp rx=0 tx=0).stop 2024-09-10T21:42:57.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:57.139+0000 7feb26a00640 1 -- 172.21.15.17:0/34061484 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feb301515d0 msgr2=0x7feb301eebd0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:42:57.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:57.139+0000 7feb26a00640 1 --2- 172.21.15.17:0/34061484 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feb301515d0 0x7feb301eebd0 secure :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7feb20001ec0 tx=0x7feb20011520 comp rx=0 tx=0).stop 2024-09-10T21:42:57.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:57.179+0000 7feb35600640 1 --2- 172.21.15.17:0/34061484 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feb301ef110 0x7feb301f1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:57.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:57.179+0000 7feb35600640 1 -- 172.21.15.17:0/34061484 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7feb301659b0 con 0x7feb301ef110 2024-09-10T21:42:57.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:57.179+0000 7feb2ea00640 1 --2- 172.21.15.17:0/34061484 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feb301ef110 0x7feb301f1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:57.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:57.179+0000 7feb2ea00640 1 -- 172.21.15.17:0/34061484 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7feb300c7140 con 0x7feb301ef110 2024-09-10T21:42:57.186 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:57.179+0000 7feb2ea00640 1 --2- 172.21.15.17:0/34061484 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feb301ef110 0x7feb301f1ed0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7feb20001ec0 tx=0x7feb20001780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:57.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:57.179+0000 7feb26a00640 1 -- 172.21.15.17:0/34061484 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb2001c030 con 0x7feb301ef110 2024-09-10T21:42:57.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:57.179+0000 7feb26a00640 1 -- 172.21.15.17:0/34061484 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5033 (secure 0 0 0) 0x7feb20018d40 con 0x7feb301ef110 2024-09-10T21:42:57.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:57.179+0000 7feb26a00640 1 -- 172.21.15.17:0/34061484 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feb301ef110 msgr2=0x7feb301f1ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:57.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:57.179+0000 7feb26a00640 1 --2- 172.21.15.17:0/34061484 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feb301ef110 0x7feb301f1ed0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7feb20001ec0 tx=0x7feb20001780 comp rx=0 tx=0).stop 2024-09-10T21:42:57.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:57.183+0000 7feb26a00640 1 -- 172.21.15.17:0/34061484 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feb301ef110 msgr2=0x7feb301f1ed0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:42:57.188 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:57.183+0000 7feb26a00640 1 --2- 172.21.15.17:0/34061484 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7feb301ef110 0x7feb301f1ed0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:57.202 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-10T21:42:57.202 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:42:57.202 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:42:57.202 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "leader", 2024-09-10T21:42:57.202 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 142, 2024-09-10T21:42:57.202 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 36311, 2024-09-10T21:42:57.202 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-10T21:42:57.202 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-10T21:42:57.202 INFO:teuthology.orchestra.run.smithi017.stdout: 1 2024-09-10T21:42:57.203 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:42:57.203 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 2, 2024-09-10T21:42:57.203 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:42:57.203 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-10T21:42:57.203 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-10T21:42:57.203 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:42:57.203 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:42:57.203 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:42:57.203 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:42:57.203 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:42:57.203 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:42:57.204 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:42:57.204 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:42:57.204 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:42:57.204 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:42:57.204 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:42:57.204 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:42:57.204 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-10T21:42:57.204 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-10T21:42:57.204 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:42:57.204 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:42:57.204 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:42:57.204 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:42:57.205 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:42:57.205 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:42:57.205 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:42:57.205 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:42:57.205 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:42:57.205 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:42:57.205 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:42:57.205 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:42:57.205 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:42:57.205 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-10T21:42:57.205 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-10T21:42:57.205 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-10T21:42:57.205 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-10T21:42:57.206 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-10T21:42:57.206 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "5e2f023f-d6a6-49f4-a256-f4807dce3739", 2024-09-10T21:42:57.206 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:42:57.206 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:42:57.206 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-10T21:42:57.206 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-10T21:42:57.206 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-10T21:42:57.206 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-10T21:42:57.206 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-10T21:42:57.206 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-10T21:42:57.206 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-10T21:42:57.206 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:42:57.206 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-10T21:42:57.206 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:42:57.207 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:42:57.207 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:42:57.207 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:42:57.207 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:42:57.207 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:42:57.207 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:42:57.207 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:42:57.207 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:42:57.207 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:42:57.207 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:42:57.207 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:42:57.207 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-10T21:42:57.207 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:42:57.207 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-10T21:42:57.208 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:42:57.208 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:42:57.208 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:42:57.208 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:42:57.208 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:42:57.208 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:42:57.208 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:42:57.208 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-10T21:42:57.208 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:42:57.208 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:42:57.208 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:42:57.208 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:42:57.208 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-10T21:42:57.209 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:42:57.209 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:42:57.209 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:42:57.209 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:42:57.209 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-10T21:42:57.209 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-10T21:42:57.209 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:42:57.209 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:42:57.209 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:42:57.209 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:42:57.209 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:42:57.209 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:42:57.210 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:42:57.210 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:42:57.210 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:42:57.210 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:42:57.210 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:42:57.210 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3300", 2024-09-10T21:42:57.210 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:42:57.210 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:42:57.210 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:42:57.210 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:42:57.210 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789", 2024-09-10T21:42:57.210 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:42:57.210 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:42:57.210 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:42:57.211 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:42:57.211 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789/0", 2024-09-10T21:42:57.211 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6789/0", 2024-09-10T21:42:57.211 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:42:57.211 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:42:57.211 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:42:57.211 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:42:57.211 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:42:57.211 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:42:57.211 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:42:57.211 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:42:57.211 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:42:57.211 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:42:57.211 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:42:57.211 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3301", 2024-09-10T21:42:57.212 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:42:57.212 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:42:57.212 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:42:57.212 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:42:57.212 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790", 2024-09-10T21:42:57.212 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:42:57.212 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:42:57.212 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:42:57.212 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:42:57.212 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790/0", 2024-09-10T21:42:57.212 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6790/0", 2024-09-10T21:42:57.212 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:42:57.212 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:42:57.213 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:42:57.213 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:42:57.213 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:42:57.213 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:42:57.213 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-10T21:42:57.213 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-10T21:42:57.213 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:42:57.213 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:42:57.213 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:42:57.213 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:42:57.213 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:42:57.213 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:42:57.214 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": [ 2024-09-10T21:42:57.214 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:42:57.214 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:42:57.214 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:42:57.214 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 3 2024-09-10T21:42:57.214 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:42:57.214 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:42:57.214 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-10T21:42:57.214 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:42:57.215 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:42:57.215 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:42:57.215 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:42:57.215 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:42:57.215 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:42:57.215 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-10T21:42:57.215 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:42:57.215 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:42:57.215 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:42:57.215 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:42:57.215 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:42:57.215 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:42:57.215 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": [ 2024-09-10T21:42:57.216 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:42:57.216 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:42:57.216 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:42:57.216 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:42:57.216 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:42:57.216 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:42:57.216 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:42:57.216 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-10T21:42:57.216 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-10T21:42:57.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:57.199+0000 7feb35600640 1 -- 172.21.15.17:0/34061484 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7feb100725c0 msgr2=0x7feb10074a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:57.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:57.199+0000 7feb35600640 1 --2- 172.21.15.17:0/34061484 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7feb100725c0 0x7feb10074a80 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7feb280b1000 tx=0x7feb2801dbb0 comp rx=0 tx=0).stop 2024-09-10T21:42:57.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:57.199+0000 7feb35600640 1 -- 172.21.15.17:0/34061484 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feb300c9160 msgr2=0x7feb3013b8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:57.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:57.199+0000 7feb35600640 1 --2- 172.21.15.17:0/34061484 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7feb300c9160 0x7feb3013b8f0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7feb1c00c970 tx=0x7feb1c00ce30 comp rx=0 tx=0).stop 2024-09-10T21:42:57.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:57.203+0000 7feb2f400640 1 -- 172.21.15.17:0/34061484 reap_dead start 2024-09-10T21:42:57.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:57.203+0000 7feb2f400640 1 -- 172.21.15.17:0/34061484 reap_dead start 2024-09-10T21:42:57.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:57.203+0000 7feb35600640 1 -- 172.21.15.17:0/34061484 shutdown_connections 2024-09-10T21:42:57.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:57.203+0000 7feb35600640 1 -- 172.21.15.17:0/34061484 >> 172.21.15.17:0/34061484 conn(0x7feb300700d0 msgr2=0x7feb300cbbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:57.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:57.203+0000 7feb35600640 1 -- 172.21.15.17:0/34061484 shutdown_connections 2024-09-10T21:42:57.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:57.203+0000 7feb35600640 1 -- 172.21.15.17:0/34061484 wait complete. 2024-09-10T21:42:57.258 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-09-10T21:42:57.258 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon b 2024-09-10T21:42:57.258 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-10T21:42:57.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:57.567+0000 7f2604600640 1 -- 172.21.15.17:0/3515419463 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f25fc153d40 msgr2=0x7f25fc0c8010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:57.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:57.567+0000 7f2604600640 1 --2- 172.21.15.17:0/3515419463 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f25fc153d40 0x7f25fc0c8010 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f25fc073750 tx=0x7f25f4023390 comp rx=0 tx=0).stop 2024-09-10T21:42:57.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:57.571+0000 7f2604600640 1 -- 172.21.15.17:0/3515419463 shutdown_connections 2024-09-10T21:42:57.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:57.571+0000 7f2604600640 1 --2- 172.21.15.17:0/3515419463 >> v2:172.21.15.31:3300/0 conn(0x7f25fc06dd10 0x7f25fc06e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:57.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:57.571+0000 7f2604600640 1 --2- 172.21.15.17:0/3515419463 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f25fc153d40 0x7f25fc0c8010 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:57.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:57.571+0000 7f2604600640 1 -- 172.21.15.17:0/3515419463 >> 172.21.15.17:0/3515419463 conn(0x7f25fc0700d0 msgr2=0x7f25fc0704e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:57.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:57.575+0000 7f2604600640 1 -- 172.21.15.17:0/3515419463 shutdown_connections 2024-09-10T21:42:57.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:57.579+0000 7f2604600640 1 -- 172.21.15.17:0/3515419463 wait complete. 2024-09-10T21:42:57.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:57.579+0000 7f2604600640 1 Processor -- start 2024-09-10T21:42:57.586 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:57.579+0000 7f2604600640 1 -- start start 2024-09-10T21:42:57.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:57.583+0000 7f2604600640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f25fc06dd10 0x7f25fc150470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:57.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:57.583+0000 7f2604600640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f25fc0c0fd0 0x7f25fc1ebfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:57.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:57.583+0000 7f2601c00640 1 -- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f25fc06dd10 msgr2=0x7f25fc150470 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:3301/0 2024-09-10T21:42:57.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:57.583+0000 7f2601c00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f25fc06dd10 0x7f25fc150470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:42:57.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:57.583+0000 7f2604600640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f25fc153d40 0x7f25fc140960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:57.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:57.583+0000 7f2604600640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f25fc0c92c0 con 0x7f25fc0c0fd0 2024-09-10T21:42:57.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:57.583+0000 7f2604600640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f25fc0c8fe0 con 0x7f25fc153d40 2024-09-10T21:42:57.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:57.583+0000 7f2604600640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f25fc0c95a0 con 0x7f25fc06dd10 2024-09-10T21:42:57.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:57.583+0000 7f2602600640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f25fc153d40 0x7f25fc140960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:57.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:57.583+0000 7f2602600640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f25fc153d40 0x7f25fc140960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.17:37754/0 (socket says 172.21.15.17:37754) 2024-09-10T21:42:57.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:57.583+0000 7f2602600640 1 -- 172.21.15.17:0/1458457394 learned_addr learned my addr 172.21.15.17:0/1458457394 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:42:57.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:57.583+0000 7f2602600640 1 -- 172.21.15.17:0/1458457394 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f25fc06dd10 msgr2=0x7f25fc150470 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:42:57.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:57.583+0000 7f2602600640 1 --2- 172.21.15.17:0/1458457394 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f25fc06dd10 0x7f25fc150470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:57.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:57.583+0000 7f2602600640 1 -- 172.21.15.17:0/1458457394 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f25fc0c0fd0 msgr2=0x7f25fc1ebfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:57.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:57.583+0000 7f2602600640 1 --2- 172.21.15.17:0/1458457394 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f25fc0c0fd0 0x7f25fc1ebfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:57.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:57.583+0000 7f2602600640 1 -- 172.21.15.17:0/1458457394 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25fc0c1f60 con 0x7f25fc153d40 2024-09-10T21:42:57.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:57.587+0000 7f2602600640 1 --2- 172.21.15.17:0/1458457394 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f25fc153d40 0x7f25fc140960 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f25e800d780 tx=0x7f25e800dc40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:57.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:57.587+0000 7f25f9600640 1 -- 172.21.15.17:0/1458457394 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25e8017070 con 0x7f25fc153d40 2024-09-10T21:42:57.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:57.587+0000 7f25f9600640 1 -- 172.21.15.17:0/1458457394 <== mon.1 v2:172.21.15.31:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f25e800de90 con 0x7f25fc153d40 2024-09-10T21:42:57.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:57.587+0000 7f25f9600640 1 -- 172.21.15.17:0/1458457394 <== mon.1 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25e8005020 con 0x7f25fc153d40 2024-09-10T21:42:57.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:57.587+0000 7f2604600640 1 -- 172.21.15.17:0/1458457394 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25fc0c2600 con 0x7f25fc153d40 2024-09-10T21:42:57.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:57.587+0000 7f2604600640 1 -- 172.21.15.17:0/1458457394 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25fc0c28c0 con 0x7f25fc153d40 2024-09-10T21:42:57.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:57.587+0000 7f2604600640 1 --2- 172.21.15.17:0/1458457394 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f25fc1ec520 0x7f25fc140ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:57.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:57.587+0000 7f2604600640 1 -- 172.21.15.17:0/1458457394 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f25fc0747d0 con 0x7f25fc1ec520 2024-09-10T21:42:57.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:57.587+0000 7f2601c00640 1 --2- 172.21.15.17:0/1458457394 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f25fc1ec520 0x7f25fc140ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:57.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:57.587+0000 7f2601c00640 1 -- 172.21.15.17:0/1458457394 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f25fc0c78a0 con 0x7f25fc1ec520 2024-09-10T21:42:57.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:57.591+0000 7f2601c00640 1 --2- 172.21.15.17:0/1458457394 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f25fc1ec520 0x7f25fc140ea0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f25fc075330 tx=0x7f25f400d570 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:57.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:57.591+0000 7f25f9600640 1 -- 172.21.15.17:0/1458457394 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25f400d6d0 con 0x7f25fc1ec520 2024-09-10T21:42:57.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:57.591+0000 7f25f9600640 1 -- 172.21.15.17:0/1458457394 <== mon.1 v2:172.21.15.31:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f25f40c9070 con 0x7f25fc1ec520 2024-09-10T21:42:57.607 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:57.591+0000 7f25f9600640 1 -- 172.21.15.17:0/1458457394 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f25fc1ec520 msgr2=0x7f25fc140ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:57.607 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:57.591+0000 7f25f9600640 1 --2- 172.21.15.17:0/1458457394 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f25fc1ec520 0x7f25fc140ea0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f25fc075330 tx=0x7f25f400d570 comp rx=0 tx=0).stop 2024-09-10T21:42:57.607 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:57.591+0000 7f25f9600640 1 -- 172.21.15.17:0/1458457394 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f25fc1ec520 msgr2=0x7f25fc140ea0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:42:57.607 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:57.591+0000 7f25f9600640 1 --2- 172.21.15.17:0/1458457394 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f25fc1ec520 0x7f25fc140ea0 secure :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f25fc075330 tx=0x7f25f400d570 comp rx=0 tx=0).stop 2024-09-10T21:42:57.607 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:57.591+0000 7f25f9600640 1 -- 172.21.15.17:0/1458457394 <== mon.1 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f25e80051c0 con 0x7f25fc153d40 2024-09-10T21:42:57.607 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:57.595+0000 7f25f9600640 1 --2- 172.21.15.17:0/1458457394 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f25e40721b0 0x7f25e4074670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:57.607 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:57.595+0000 7f25f9600640 1 -- 172.21.15.17:0/1458457394 <== mon.1 v2:172.21.15.31:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f25e8093790 con 0x7f25fc153d40 2024-09-10T21:42:57.607 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:57.599+0000 7f2601c00640 1 --2- 172.21.15.17:0/1458457394 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f25e40721b0 0x7f25e4074670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:57.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:57.603+0000 7f2601c00640 1 --2- 172.21.15.17:0/1458457394 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f25e40721b0 0x7f25e4074670 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f25fc074950 tx=0x7f25f400f030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:57.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:57.631+0000 7f2604600640 1 --2- 172.21.15.17:0/1458457394 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f25fc14fd40 0x7f25fc146f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:42:57.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:57.631+0000 7f2604600640 1 -- 172.21.15.17:0/1458457394 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f25fc0726a0 con 0x7f25fc14fd40 2024-09-10T21:42:57.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:57.631+0000 7f2601200640 1 --2- 172.21.15.17:0/1458457394 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f25fc14fd40 0x7f25fc146f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:42:57.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:57.635+0000 7f2601200640 1 -- 172.21.15.17:0/1458457394 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f25fc0008e0 con 0x7f25fc14fd40 2024-09-10T21:42:57.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:57.635+0000 7f2601200640 1 --2- 172.21.15.17:0/1458457394 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f25fc14fd40 0x7f25fc146f70 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f25fc1ed130 tx=0x7f25f000c7a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:42:57.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:57.635+0000 7f25f9600640 1 -- 172.21.15.17:0/1458457394 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25f0007020 con 0x7f25fc14fd40 2024-09-10T21:42:57.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:57.635+0000 7f25f9600640 1 -- 172.21.15.17:0/1458457394 <== mon.1 v2:172.21.15.31:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5032 (secure 0 0 0) 0x7f25f00071c0 con 0x7f25fc14fd40 2024-09-10T21:42:57.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:57.635+0000 7f25f9600640 1 -- 172.21.15.17:0/1458457394 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f25fc14fd40 msgr2=0x7f25fc146f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:57.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:57.635+0000 7f25f9600640 1 --2- 172.21.15.17:0/1458457394 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f25fc14fd40 0x7f25fc146f70 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f25fc1ed130 tx=0x7f25f000c7a0 comp rx=0 tx=0).stop 2024-09-10T21:42:57.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:57.635+0000 7f25f9600640 1 -- 172.21.15.17:0/1458457394 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f25fc14fd40 msgr2=0x7f25fc146f70 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:42:57.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:57.635+0000 7f25f9600640 1 --2- 172.21.15.17:0/1458457394 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f25fc14fd40 0x7f25fc146f70 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:42:57.644 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-10T21:42:57.644 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:42:57.644 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:42:57.644 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-10T21:42:57.644 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 142, 2024-09-10T21:42:57.644 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 189065, 2024-09-10T21:42:57.644 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-10T21:42:57.644 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-10T21:42:57.645 INFO:teuthology.orchestra.run.smithi017.stdout: 1 2024-09-10T21:42:57.645 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:42:57.645 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 2, 2024-09-10T21:42:57.645 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:42:57.645 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-10T21:42:57.645 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-10T21:42:57.645 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:42:57.645 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:42:57.645 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:42:57.645 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:42:57.648 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:42:57.648 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:42:57.648 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:42:57.648 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:42:57.648 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:42:57.648 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:42:57.648 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:42:57.648 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:42:57.648 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-10T21:42:57.648 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-10T21:42:57.649 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:42:57.649 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:42:57.649 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:42:57.649 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:42:57.649 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:42:57.649 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:42:57.649 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:42:57.649 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:42:57.649 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:42:57.649 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:42:57.649 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:42:57.649 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:42:57.650 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:42:57.650 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-10T21:42:57.650 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-10T21:42:57.650 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-10T21:42:57.650 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-10T21:42:57.650 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-10T21:42:57.650 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "5e2f023f-d6a6-49f4-a256-f4807dce3739", 2024-09-10T21:42:57.650 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:42:57.650 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:42:57.650 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-10T21:42:57.650 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-10T21:42:57.650 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-10T21:42:57.650 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-10T21:42:57.651 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-10T21:42:57.651 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-10T21:42:57.651 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-10T21:42:57.651 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:42:57.651 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-10T21:42:57.651 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:42:57.651 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:42:57.651 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:42:57.651 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:42:57.651 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:42:57.651 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:42:57.651 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:42:57.651 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:42:57.651 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:42:57.652 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:42:57.652 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:42:57.652 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:42:57.652 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-10T21:42:57.652 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:42:57.652 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-10T21:42:57.652 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:42:57.652 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:42:57.652 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:42:57.652 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:42:57.652 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:42:57.652 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:42:57.652 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:42:57.653 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-10T21:42:57.653 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:42:57.653 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:42:57.653 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:42:57.653 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:42:57.653 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-10T21:42:57.653 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:42:57.653 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:42:57.653 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:42:57.653 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:42:57.653 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-10T21:42:57.653 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-10T21:42:57.653 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:42:57.654 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:42:57.654 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:42:57.654 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:42:57.654 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:42:57.654 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:42:57.654 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:42:57.654 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:42:57.654 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:42:57.654 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:42:57.654 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:42:57.654 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3300", 2024-09-10T21:42:57.654 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:42:57.654 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:42:57.654 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:42:57.655 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:42:57.655 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789", 2024-09-10T21:42:57.655 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:42:57.655 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:42:57.655 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:42:57.655 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:42:57.655 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789/0", 2024-09-10T21:42:57.655 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6789/0", 2024-09-10T21:42:57.655 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:42:57.655 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:42:57.655 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:42:57.655 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:42:57.655 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:42:57.655 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:42:57.655 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:42:57.656 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:42:57.656 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:42:57.656 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:42:57.656 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:42:57.656 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3301", 2024-09-10T21:42:57.656 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:42:57.656 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:42:57.656 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:42:57.656 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:42:57.656 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790", 2024-09-10T21:42:57.656 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:42:57.656 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:42:57.656 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:42:57.657 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:42:57.657 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790/0", 2024-09-10T21:42:57.657 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6790/0", 2024-09-10T21:42:57.657 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:42:57.657 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:42:57.657 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:42:57.657 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:42:57.657 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:42:57.657 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:42:57.657 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-10T21:42:57.657 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-10T21:42:57.657 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:42:57.657 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:42:57.657 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:42:57.658 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:42:57.658 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:42:57.658 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:42:57.658 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": [ 2024-09-10T21:42:57.658 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:42:57.658 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:42:57.658 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:42:57.658 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 3 2024-09-10T21:42:57.658 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:42:57.658 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:42:57.658 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-10T21:42:57.658 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:42:57.658 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:42:57.658 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:42:57.658 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 7 2024-09-10T21:42:57.659 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:42:57.659 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:42:57.659 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-10T21:42:57.659 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:42:57.659 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:42:57.659 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:42:57.659 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 7 2024-09-10T21:42:57.659 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:42:57.659 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:42:57.659 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": [ 2024-09-10T21:42:57.659 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:42:57.659 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:42:57.659 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:42:57.659 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:42:57.659 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:42:57.659 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:42:57.660 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:42:57.660 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-10T21:42:57.660 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-10T21:42:57.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:57.639+0000 7f2604600640 1 -- 172.21.15.17:0/1458457394 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f25e40721b0 msgr2=0x7f25e4074670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:57.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:57.639+0000 7f2604600640 1 --2- 172.21.15.17:0/1458457394 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f25e40721b0 0x7f25e4074670 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f25fc074950 tx=0x7f25f400f030 comp rx=0 tx=0).stop 2024-09-10T21:42:57.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:57.639+0000 7f2604600640 1 -- 172.21.15.17:0/1458457394 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f25fc153d40 msgr2=0x7f25fc140960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:42:57.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:57.639+0000 7f2604600640 1 --2- 172.21.15.17:0/1458457394 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f25fc153d40 0x7f25fc140960 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f25e800d780 tx=0x7f25e800dc40 comp rx=0 tx=0).stop 2024-09-10T21:42:57.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:57.643+0000 7f2602600640 1 -- 172.21.15.17:0/1458457394 reap_dead start 2024-09-10T21:42:57.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:57.643+0000 7f2602600640 1 -- 172.21.15.17:0/1458457394 reap_dead start 2024-09-10T21:42:57.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:57.647+0000 7f2604600640 1 -- 172.21.15.17:0/1458457394 shutdown_connections 2024-09-10T21:42:57.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:57.647+0000 7f2604600640 1 -- 172.21.15.17:0/1458457394 >> 172.21.15.17:0/1458457394 conn(0x7f25fc0700d0 msgr2=0x7f25fc1e22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:42:57.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:57.651+0000 7f2604600640 1 -- 172.21.15.17:0/1458457394 shutdown_connections 2024-09-10T21:42:57.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:42:57.655+0000 7f2604600640 1 -- 172.21.15.17:0/1458457394 wait complete. 2024-09-10T21:42:57.715 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-09-10T21:42:57.716 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2024-09-10T21:43:17.716 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.c 2024-09-10T21:43:17.717 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-10T21:43:17.717 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-10T21:43:17.721 INFO:tasks.ceph.mon.c:Started 2024-09-10T21:43:17.722 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-09-10T21:43:17.722 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-10T21:43:18.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:18.067+0000 7fc1c4800640 1 -- 172.21.15.17:0/3781585852 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc1a4005220 msgr2=0x7fc1a400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:18.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:18.067+0000 7fc1c4800640 1 --2- 172.21.15.17:0/3781585852 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc1a4005220 0x7fc1a400b9d0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc1583d0 tx=0x7fc1b801c830 comp rx=0 tx=0).stop 2024-09-10T21:43:18.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:18.067+0000 7fc1c2800640 1 -- 172.21.15.17:0/3781585852 reap_dead start 2024-09-10T21:43:18.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:18.067+0000 7fc1c4800640 1 -- 172.21.15.17:0/3781585852 shutdown_connections 2024-09-10T21:43:18.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:18.067+0000 7fc1c4800640 1 -- 172.21.15.17:0/3781585852 >> 172.21.15.17:0/3781585852 conn(0x7fc1bc070160 msgr2=0x7fc1bc070570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:43:18.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:18.071+0000 7fc1c4800640 1 -- 172.21.15.17:0/3781585852 shutdown_connections 2024-09-10T21:43:18.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:18.071+0000 7fc1c4800640 1 -- 172.21.15.17:0/3781585852 wait complete. 2024-09-10T21:43:18.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:18.083+0000 7fc1c4800640 1 Processor -- start 2024-09-10T21:43:18.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:18.095+0000 7fc1c4800640 1 -- start start 2024-09-10T21:43:18.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:18.095+0000 7fc1c4800640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc1a4005220 0x7fc1bc0ca6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:18.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:18.095+0000 7fc1c4800640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc1bc1549b0 0x7fc1bc13eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:18.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:18.095+0000 7fc1c4800640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc1bc152050 0x7fc1bc13aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:18.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:18.099+0000 7fc1c4800640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fc1bc0c3080 con 0x7fc1bc1549b0 2024-09-10T21:43:18.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:18.099+0000 7fc1c4800640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fc1bc0c2b40 con 0x7fc1bc152050 2024-09-10T21:43:18.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:18.099+0000 7fc1c4800640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7fc1bc0c8030 con 0x7fc1a4005220 2024-09-10T21:43:18.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:18.099+0000 7fc1c2800640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc1bc152050 0x7fc1bc13aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:18.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:18.099+0000 7fc1c2800640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc1bc152050 0x7fc1bc13aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.17:42078/0 (socket says 172.21.15.17:42078) 2024-09-10T21:43:18.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:18.099+0000 7fc1c2800640 1 -- 172.21.15.17:0/1234818794 learned_addr learned my addr 172.21.15.17:0/1234818794 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:43:18.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:18.099+0000 7fc1c1e00640 1 --2- 172.21.15.17:0/1234818794 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc1a4005220 0x7fc1bc0ca6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:18.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:18.103+0000 7fc1c1400640 1 --2- 172.21.15.17:0/1234818794 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc1bc1549b0 0x7fc1bc13eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:18.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:18.103+0000 7fc1c1e00640 1 -- 172.21.15.17:0/1234818794 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc1a4005220 msgr2=0x7fc1bc0ca6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-09-10T21:43:18.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:18.103+0000 7fc1c1e00640 1 -- 172.21.15.17:0/1234818794 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc1a4005220 msgr2=0x7fc1bc0ca6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-10T21:43:18.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:18.103+0000 7fc1c1e00640 1 --2- 172.21.15.17:0/1234818794 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc1a4005220 0x7fc1bc0ca6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T21:43:18.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:18.103+0000 7fc1c1e00640 1 --2- 172.21.15.17:0/1234818794 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc1a4005220 0x7fc1bc0ca6a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:43:18.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:18.103+0000 7fc1c2800640 1 -- 172.21.15.17:0/1234818794 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc1a4005220 msgr2=0x7fc1bc0ca6a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:43:18.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:18.103+0000 7fc1c2800640 1 --2- 172.21.15.17:0/1234818794 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc1a4005220 0x7fc1bc0ca6a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:18.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:18.103+0000 7fc1c2800640 1 -- 172.21.15.17:0/1234818794 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc1bc1549b0 msgr2=0x7fc1bc13eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:18.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:18.103+0000 7fc1c2800640 1 --2- 172.21.15.17:0/1234818794 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc1bc1549b0 0x7fc1bc13eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:18.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:18.103+0000 7fc1c2800640 1 -- 172.21.15.17:0/1234818794 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1bc13c240 con 0x7fc1bc152050 2024-09-10T21:43:18.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:18.107+0000 7fc1c2800640 1 --2- 172.21.15.17:0/1234818794 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc1bc152050 0x7fc1bc13aa60 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fc1b80983b0 tx=0x7fc1b8006940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:18.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:18.115+0000 7fc1b6000640 1 -- 172.21.15.17:0/1234818794 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1b8001940 con 0x7fc1bc152050 2024-09-10T21:43:18.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:18.115+0000 7fc1b6000640 1 -- 172.21.15.17:0/1234818794 <== mon.1 v2:172.21.15.31:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc1b80b4030 con 0x7fc1bc152050 2024-09-10T21:43:18.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:18.115+0000 7fc1b6000640 1 -- 172.21.15.17:0/1234818794 <== mon.1 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1b80a58d0 con 0x7fc1bc152050 2024-09-10T21:43:18.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:18.115+0000 7fc1c4800640 1 -- 172.21.15.17:0/1234818794 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1bc140a30 con 0x7fc1bc152050 2024-09-10T21:43:18.124 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:18.115+0000 7fc1c4800640 1 -- 172.21.15.17:0/1234818794 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1bc140ff0 con 0x7fc1bc152050 2024-09-10T21:43:18.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:18.119+0000 7fc1b6000640 1 -- 172.21.15.17:0/1234818794 <== mon.1 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fc1b80a5a70 con 0x7fc1bc152050 2024-09-10T21:43:18.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:18.123+0000 7fc1b6000640 1 --2- 172.21.15.17:0/1234818794 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fc1a00726b0 0x7fc1a0074b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:18.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:18.123+0000 7fc1b6000640 1 -- 172.21.15.17:0/1234818794 <== mon.1 v2:172.21.15.31:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fc1b812d6c0 con 0x7fc1bc152050 2024-09-10T21:43:18.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:18.127+0000 7fc1c1e00640 1 --2- 172.21.15.17:0/1234818794 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fc1a00726b0 0x7fc1a0074b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:18.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:18.131+0000 7fc1c4800640 1 -- 172.21.15.17:0/1234818794 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1bc0c2210 con 0x7fc1bc152050 2024-09-10T21:43:18.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:18.131+0000 7fc1c1e00640 1 --2- 172.21.15.17:0/1234818794 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fc1a00726b0 0x7fc1a0074b70 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fc1b0005fd0 tx=0x7fc1b0005860 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:18.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:18.171+0000 7fc1b6000640 1 -- 172.21.15.17:0/1234818794 <== mon.1 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fc1b80be030 con 0x7fc1bc152050 2024-09-10T21:43:18.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:18.839+0000 7fc1c4800640 1 -- 172.21.15.17:0/1234818794 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fc1bc06ea00 con 0x7fc1bc152050 2024-09-10T21:43:18.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:18.839+0000 7fc1b6000640 1 -- 172.21.15.17:0/1234818794 <== mon.1 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1552 (secure 0 0 0) 0x7fc1bc0c6a20 con 0x7fc1bc152050 2024-09-10T21:43:18.845 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":142,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":23,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5e2f023f-d6a6-49f4-a256-f4807dce3739","modified":"2024-09-10T21:23:46.892130Z","created":"2024-09-10T21:23:46.892130Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3301","nonce":0},{"type":"v1","addr":"172.21.15.31:6790","nonce":0}]},"addr":"172.21.15.31:6790/0","public_addr":"172.21.15.31:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-10T21:43:18.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:18.843+0000 7fc1c4800640 1 -- 172.21.15.17:0/1234818794 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fc1a00726b0 msgr2=0x7fc1a0074b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:18.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:18.843+0000 7fc1c4800640 1 --2- 172.21.15.17:0/1234818794 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fc1a00726b0 0x7fc1a0074b70 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fc1b0005fd0 tx=0x7fc1b0005860 comp rx=0 tx=0).stop 2024-09-10T21:43:18.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:18.843+0000 7fc1c4800640 1 -- 172.21.15.17:0/1234818794 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc1bc152050 msgr2=0x7fc1bc13aa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:18.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:18.843+0000 7fc1c4800640 1 --2- 172.21.15.17:0/1234818794 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc1bc152050 0x7fc1bc13aa60 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fc1b80983b0 tx=0x7fc1b8006940 comp rx=0 tx=0).stop 2024-09-10T21:43:18.853 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:18.847+0000 7fc1c4800640 1 -- 172.21.15.17:0/1234818794 shutdown_connections 2024-09-10T21:43:18.853 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:18.847+0000 7fc1c4800640 1 --2- 172.21.15.17:0/1234818794 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fc1a00726b0 0x7fc1a0074b70 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:18.853 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:18.847+0000 7fc1c4800640 1 --2- 172.21.15.17:0/1234818794 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc1bc152050 0x7fc1bc13aa60 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:18.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:18.847+0000 7fc1c4800640 1 --2- 172.21.15.17:0/1234818794 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc1bc1549b0 0x7fc1bc13eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:18.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:18.851+0000 7fc1c4800640 1 --2- 172.21.15.17:0/1234818794 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc1a4005220 0x7fc1bc0ca6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:18.859 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:18.851+0000 7fc1c4800640 1 -- 172.21.15.17:0/1234818794 >> 172.21.15.17:0/1234818794 conn(0x7fc1bc070160 msgr2=0x7fc1bc14eeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:43:18.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:18.859+0000 7fc1c4800640 1 -- 172.21.15.17:0/1234818794 shutdown_connections 2024-09-10T21:43:18.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:18.863+0000 7fc1c4800640 1 -- 172.21.15.17:0/1234818794 wait complete. 2024-09-10T21:43:21.953 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-10T21:43:22.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:22.287+0000 7f7193e00640 1 -- 172.21.15.17:0/180060554 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f718c04baf0 msgr2=0x7f718c006ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:22.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:22.287+0000 7f7193e00640 1 --2- 172.21.15.17:0/180060554 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f718c04baf0 0x7f718c006ec0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f718c00da40 tx=0x7f718401fd40 comp rx=0 tx=0).stop 2024-09-10T21:43:22.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:22.295+0000 7f7193e00640 1 -- 172.21.15.17:0/180060554 shutdown_connections 2024-09-10T21:43:22.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:22.295+0000 7f7193e00640 1 --2- 172.21.15.17:0/180060554 >> v2:172.21.15.17:3300/0 conn(0x7f718c04c3b0 0x7f718c007400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:22.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:22.295+0000 7f7193e00640 1 --2- 172.21.15.17:0/180060554 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f718c04baf0 0x7f718c006ec0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:22.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:22.295+0000 7f7193e00640 1 --2- 172.21.15.17:0/180060554 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f718c0d4980 0x7f718c0db040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:22.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:22.295+0000 7f7193e00640 1 -- 172.21.15.17:0/180060554 >> 172.21.15.17:0/180060554 conn(0x7f718c050060 msgr2=0x7f718c050470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:43:22.298 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:22.295+0000 7f7193e00640 1 -- 172.21.15.17:0/180060554 shutdown_connections 2024-09-10T21:43:22.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:22.295+0000 7f7193e00640 1 -- 172.21.15.17:0/180060554 wait complete. 2024-09-10T21:43:22.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:22.299+0000 7f7193e00640 1 Processor -- start 2024-09-10T21:43:22.306 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:22.303+0000 7f7193e00640 1 -- start start 2024-09-10T21:43:22.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:22.311+0000 7f7193e00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f718c04baf0 0x7f718c175500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:22.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:22.311+0000 7f7193e00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f718c04c3b0 0x7f718c162a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:22.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:22.311+0000 7f7193e00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f718c0d4980 0x7f718c163db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:22.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:22.311+0000 7f7193e00640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f718c0066c0 con 0x7f718c0d4980 2024-09-10T21:43:22.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:22.311+0000 7f7193e00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f718c0063e0 con 0x7f718c04c3b0 2024-09-10T21:43:22.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:22.311+0000 7f7193e00640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f718c0069a0 con 0x7f718c04baf0 2024-09-10T21:43:22.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:22.311+0000 7f7192a00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f718c04baf0 0x7f718c175500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:22.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:22.311+0000 7f7192a00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f718c04baf0 0x7f718c175500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3301/0 says I am v2:172.21.15.17:39538/0 (socket says 172.21.15.17:39538) 2024-09-10T21:43:22.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:22.311+0000 7f7192a00640 1 -- 172.21.15.17:0/2880444883 learned_addr learned my addr 172.21.15.17:0/2880444883 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:43:22.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:22.311+0000 7f7192a00640 1 -- 172.21.15.17:0/2880444883 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f718c04baf0 msgr2=0x7f718c175500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-09-10T21:43:22.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:22.311+0000 7f7192a00640 1 -- 172.21.15.17:0/2880444883 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f718c04baf0 msgr2=0x7f718c175500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-10T21:43:22.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:22.311+0000 7f7192a00640 1 --2- 172.21.15.17:0/2880444883 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f718c04baf0 0x7f718c175500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T21:43:22.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:22.311+0000 7f7192a00640 1 --2- 172.21.15.17:0/2880444883 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f718c04baf0 0x7f718c175500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:43:22.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:22.315+0000 7f7193400640 1 --2- 172.21.15.17:0/2880444883 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f718c0d4980 0x7f718c163db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:22.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:22.315+0000 7f7192000640 1 --2- 172.21.15.17:0/2880444883 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f718c04c3b0 0x7f718c162a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:22.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:22.315+0000 7f7192000640 1 -- 172.21.15.17:0/2880444883 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f718c04baf0 msgr2=0x7f718c175500 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:43:22.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:22.315+0000 7f7192000640 1 --2- 172.21.15.17:0/2880444883 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f718c04baf0 0x7f718c175500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:22.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:22.315+0000 7f7192000640 1 -- 172.21.15.17:0/2880444883 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f718c0d4980 msgr2=0x7f718c163db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:22.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:22.315+0000 7f7192000640 1 --2- 172.21.15.17:0/2880444883 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f718c0d4980 0x7f718c163db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:22.319 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:22.315+0000 7f7192000640 1 -- 172.21.15.17:0/2880444883 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f718c175a40 con 0x7f718c04c3b0 2024-09-10T21:43:22.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:22.315+0000 7f7193400640 1 --2- 172.21.15.17:0/2880444883 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f718c0d4980 0x7f718c163db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:43:22.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:22.323+0000 7f7192000640 1 --2- 172.21.15.17:0/2880444883 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f718c04c3b0 0x7f718c162a90 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f71840af000 tx=0x7f7184007ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:22.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:22.331+0000 7f718aa00640 1 -- 172.21.15.17:0/2880444883 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7184004250 con 0x7f718c04c3b0 2024-09-10T21:43:22.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:22.331+0000 7f718aa00640 1 -- 172.21.15.17:0/2880444883 <== mon.1 v2:172.21.15.31:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f71840043f0 con 0x7f718c04c3b0 2024-09-10T21:43:22.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:22.331+0000 7f718aa00640 1 -- 172.21.15.17:0/2880444883 <== mon.1 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7184003860 con 0x7f718c04c3b0 2024-09-10T21:43:22.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:22.331+0000 7f7193e00640 1 -- 172.21.15.17:0/2880444883 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f718c0325a0 con 0x7f718c04c3b0 2024-09-10T21:43:22.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:22.331+0000 7f7193e00640 1 -- 172.21.15.17:0/2880444883 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f718c165c80 con 0x7f718c04c3b0 2024-09-10T21:43:22.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:22.335+0000 7f7193e00640 1 -- 172.21.15.17:0/2880444883 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f718c004a40 con 0x7f718c04c3b0 2024-09-10T21:43:22.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:22.335+0000 7f718aa00640 1 -- 172.21.15.17:0/2880444883 <== mon.1 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f71840ba070 con 0x7f718c04c3b0 2024-09-10T21:43:22.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:22.339+0000 7f718aa00640 1 --2- 172.21.15.17:0/2880444883 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f71740726e0 0x7f7174074ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:22.342 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:22.339+0000 7f718aa00640 1 -- 172.21.15.17:0/2880444883 <== mon.1 v2:172.21.15.31:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f71841305b0 con 0x7f718c04c3b0 2024-09-10T21:43:22.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:22.343+0000 7f7192a00640 1 --2- 172.21.15.17:0/2880444883 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f71740726e0 0x7f7174074ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:22.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:22.343+0000 7f718aa00640 1 -- 172.21.15.17:0/2880444883 <== mon.1 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f71840198a0 con 0x7f718c04c3b0 2024-09-10T21:43:22.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:22.359+0000 7f7192a00640 1 --2- 172.21.15.17:0/2880444883 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f71740726e0 0x7f7174074ba0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f7180005fd0 tx=0x7f7180005f00 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:23.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:23.059+0000 7f7193e00640 1 -- 172.21.15.17:0/2880444883 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f718c0095d0 con 0x7f718c04c3b0 2024-09-10T21:43:23.069 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:23.063+0000 7f718aa00640 1 -- 172.21.15.17:0/2880444883 <== mon.1 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1557 (secure 0 0 0) 0x7f71840fb990 con 0x7f718c04c3b0 2024-09-10T21:43:23.072 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":144,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":1,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5e2f023f-d6a6-49f4-a256-f4807dce3739","modified":"2024-09-10T21:23:46.892130Z","created":"2024-09-10T21:23:46.892130Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3301","nonce":0},{"type":"v1","addr":"172.21.15.31:6790","nonce":0}]},"addr":"172.21.15.31:6790/0","public_addr":"172.21.15.31:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-10T21:43:23.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:23.075+0000 7f7193e00640 1 -- 172.21.15.17:0/2880444883 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f71740726e0 msgr2=0x7f7174074ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:23.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:23.075+0000 7f7193e00640 1 --2- 172.21.15.17:0/2880444883 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f71740726e0 0x7f7174074ba0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f7180005fd0 tx=0x7f7180005f00 comp rx=0 tx=0).stop 2024-09-10T21:43:23.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:23.075+0000 7f7193e00640 1 -- 172.21.15.17:0/2880444883 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f718c04c3b0 msgr2=0x7f718c162a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:23.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:23.075+0000 7f7193e00640 1 --2- 172.21.15.17:0/2880444883 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f718c04c3b0 0x7f718c162a90 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f71840af000 tx=0x7f7184007ca0 comp rx=0 tx=0).stop 2024-09-10T21:43:23.077 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:23.075+0000 7f7193e00640 1 -- 172.21.15.17:0/2880444883 shutdown_connections 2024-09-10T21:43:23.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:23.075+0000 7f7193e00640 1 --2- 172.21.15.17:0/2880444883 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f71740726e0 0x7f7174074ba0 secure :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f7180005fd0 tx=0x7f7180005f00 comp rx=0 tx=0).stop 2024-09-10T21:43:23.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:23.075+0000 7f7193e00640 1 --2- 172.21.15.17:0/2880444883 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f718c0d4980 0x7f718c163db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:23.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:23.075+0000 7f7193e00640 1 --2- 172.21.15.17:0/2880444883 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f718c04c3b0 0x7f718c162a90 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:23.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:23.075+0000 7f7193e00640 1 --2- 172.21.15.17:0/2880444883 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f718c04baf0 0x7f718c175500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:23.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:23.075+0000 7f7193e00640 1 -- 172.21.15.17:0/2880444883 >> 172.21.15.17:0/2880444883 conn(0x7f718c050060 msgr2=0x7f718c0d9210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:43:23.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:23.079+0000 7f7193e00640 1 -- 172.21.15.17:0/2880444883 shutdown_connections 2024-09-10T21:43:23.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:23.079+0000 7f7193e00640 1 -- 172.21.15.17:0/2880444883 wait complete. 2024-09-10T21:43:23.182 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-09-10T21:43:23.182 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-09-10T21:43:23.182 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-10T21:43:23.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:23.515+0000 7efee1e00640 1 -- 172.21.15.17:0/3194506352 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7efec0006400 msgr2=0x7efec000cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:23.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:23.515+0000 7efee1e00640 1 --2- 172.21.15.17:0/3194506352 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7efec0006400 0x7efec000cae0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7efed400c670 tx=0x7efed401fb50 comp rx=0 tx=0).stop 2024-09-10T21:43:23.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:23.515+0000 7efee1e00640 1 -- 172.21.15.17:0/3194506352 shutdown_connections 2024-09-10T21:43:23.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:23.515+0000 7efee1e00640 1 --2- 172.21.15.17:0/3194506352 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7efec0006400 0x7efec000cae0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:23.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:23.515+0000 7efee1e00640 1 -- 172.21.15.17:0/3194506352 >> 172.21.15.17:0/3194506352 conn(0x7efedc06cc20 msgr2=0x7efedc06d030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:43:23.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:23.527+0000 7efee1e00640 1 -- 172.21.15.17:0/3194506352 shutdown_connections 2024-09-10T21:43:23.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:23.527+0000 7efee1e00640 1 -- 172.21.15.17:0/3194506352 wait complete. 2024-09-10T21:43:23.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:23.535+0000 7efee1e00640 1 Processor -- start 2024-09-10T21:43:23.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:23.539+0000 7efee1e00640 1 -- start start 2024-09-10T21:43:23.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:23.539+0000 7efee1e00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efec0006400 0x7efedc14c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:23.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:23.539+0000 7efee1e00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7efedc154170 0x7efedc149a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:23.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:23.539+0000 7efee1e00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efedc158d70 0x7efedc1591f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:23.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:23.539+0000 7efee1e00640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7efedc1579c0 con 0x7efec0006400 2024-09-10T21:43:23.544 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:23.539+0000 7efee1e00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7efedc157fd0 con 0x7efedc158d70 2024-09-10T21:43:23.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:23.539+0000 7efee1e00640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7efedc150e30 con 0x7efedc154170 2024-09-10T21:43:23.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:23.543+0000 7efedb400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efedc158d70 0x7efedc1591f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:23.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:23.543+0000 7efedaa00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efec0006400 0x7efedc14c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:23.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:23.547+0000 7efedb400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efedc158d70 0x7efedc1591f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.17:42110/0 (socket says 172.21.15.17:42110) 2024-09-10T21:43:23.548 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:23.547+0000 7efedb400640 1 -- 172.21.15.17:0/670439348 learned_addr learned my addr 172.21.15.17:0/670439348 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:43:23.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:23.547+0000 7efeda000640 1 --2- 172.21.15.17:0/670439348 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7efedc154170 0x7efedc149a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:23.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:23.547+0000 7efeda000640 1 -- 172.21.15.17:0/670439348 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efedc158d70 msgr2=0x7efedc1591f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:23.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:23.547+0000 7efeda000640 1 --2- 172.21.15.17:0/670439348 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efedc158d70 0x7efedc1591f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:23.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:23.547+0000 7efeda000640 1 -- 172.21.15.17:0/670439348 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efec0006400 msgr2=0x7efedc14c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:23.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:23.547+0000 7efeda000640 1 --2- 172.21.15.17:0/670439348 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efec0006400 0x7efedc14c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:23.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:23.547+0000 7efeda000640 1 -- 172.21.15.17:0/670439348 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efedc159c30 con 0x7efedc154170 2024-09-10T21:43:23.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:23.551+0000 7efeda000640 1 --2- 172.21.15.17:0/670439348 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7efedc154170 0x7efedc149a30 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7efedc0c74c0 tx=0x7efec800b820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:23.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:23.555+0000 7efed3400640 1 -- 172.21.15.17:0/670439348 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efec8015020 con 0x7efedc154170 2024-09-10T21:43:23.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:23.555+0000 7efee1e00640 1 -- 172.21.15.17:0/670439348 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efedc14c250 con 0x7efedc154170 2024-09-10T21:43:23.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:23.555+0000 7efed3400640 1 -- 172.21.15.17:0/670439348 <== mon.2 v2:172.21.15.31:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efec8011920 con 0x7efedc154170 2024-09-10T21:43:23.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:23.555+0000 7efed3400640 1 -- 172.21.15.17:0/670439348 <== mon.2 v2:172.21.15.31:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efec8011d70 con 0x7efedc154170 2024-09-10T21:43:23.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:23.555+0000 7efee1e00640 1 -- 172.21.15.17:0/670439348 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7efedc1f50d0 con 0x7efedc154170 2024-09-10T21:43:23.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:23.559+0000 7efee1e00640 1 --2- 172.21.15.17:0/670439348 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efeb4002a80 0x7efeb4004f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:23.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:23.559+0000 7efee1e00640 1 -- 172.21.15.17:0/670439348 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7efeb4007ed0 con 0x7efeb4002a80 2024-09-10T21:43:23.562 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:23.559+0000 7efed3400640 1 -- 172.21.15.17:0/670439348 <== mon.2 v2:172.21.15.31:3301/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7efec8035a50 con 0x7efedc154170 2024-09-10T21:43:23.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:23.559+0000 7efed3400640 1 --2- 172.21.15.17:0/670439348 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7efebc072610 0x7efebc074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:23.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:23.559+0000 7efed3400640 1 -- 172.21.15.17:0/670439348 <== mon.2 v2:172.21.15.31:3301/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7efec8092970 con 0x7efedc154170 2024-09-10T21:43:23.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:23.563+0000 7efedb400640 1 --2- 172.21.15.17:0/670439348 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7efebc072610 0x7efebc074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:23.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:23.563+0000 7efedaa00640 1 --2- 172.21.15.17:0/670439348 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efeb4002a80 0x7efeb4004f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:23.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:23.563+0000 7efedb400640 1 --2- 172.21.15.17:0/670439348 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7efebc072610 0x7efebc074ad0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7efed4000f90 tx=0x7efed401c940 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:23.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:23.563+0000 7efedaa00640 1 -- 172.21.15.17:0/670439348 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7efeb40080d0 con 0x7efeb4002a80 2024-09-10T21:43:23.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:23.563+0000 7efedaa00640 1 --2- 172.21.15.17:0/670439348 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efeb4002a80 0x7efeb4004f40 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7efedc0cd630 tx=0x7efecc00ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:23.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:23.563+0000 7efed3400640 1 -- 172.21.15.17:0/670439348 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efecc017070 con 0x7efeb4002a80 2024-09-10T21:43:23.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:23.567+0000 7efed3400640 1 -- 172.21.15.17:0/670439348 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7efecc00c720 con 0x7efeb4002a80 2024-09-10T21:43:23.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:23.567+0000 7efed3400640 1 -- 172.21.15.17:0/670439348 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efeb4002a80 msgr2=0x7efeb4004f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:23.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:23.567+0000 7efed3400640 1 --2- 172.21.15.17:0/670439348 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efeb4002a80 0x7efeb4004f40 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7efedc0cd630 tx=0x7efecc00ec00 comp rx=0 tx=0).stop 2024-09-10T21:43:23.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:23.567+0000 7efed3400640 1 -- 172.21.15.17:0/670439348 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efeb4002a80 msgr2=0x7efeb4004f40 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:43:23.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:23.567+0000 7efed3400640 1 --2- 172.21.15.17:0/670439348 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efeb4002a80 0x7efeb4004f40 secure :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7efedc0cd630 tx=0x7efecc00ec00 comp rx=0 tx=0).stop 2024-09-10T21:43:23.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:23.579+0000 7efee1e00640 1 --2- 172.21.15.17:0/670439348 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efeb4006ec0 0x7efeb4007350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:23.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:23.579+0000 7efee1e00640 1 -- 172.21.15.17:0/670439348 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7efeb400a4b0 con 0x7efeb4006ec0 2024-09-10T21:43:23.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:23.579+0000 7efedaa00640 1 --2- 172.21.15.17:0/670439348 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efeb4006ec0 0x7efeb4007350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:23.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:23.599+0000 7efedaa00640 1 -- 172.21.15.17:0/670439348 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7efeb4007890 con 0x7efeb4006ec0 2024-09-10T21:43:23.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:23.615+0000 7efedaa00640 1 --2- 172.21.15.17:0/670439348 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efeb4006ec0 0x7efeb4007350 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7efedc06b6a0 tx=0x7efecc004760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:23.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:23.627+0000 7efed3400640 1 -- 172.21.15.17:0/670439348 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efecc017070 con 0x7efeb4006ec0 2024-09-10T21:43:23.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:23.635+0000 7efed3400640 1 -- 172.21.15.17:0/670439348 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5044 (secure 0 0 0) 0x7efecc006310 con 0x7efeb4006ec0 2024-09-10T21:43:23.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:23.635+0000 7efed3400640 1 -- 172.21.15.17:0/670439348 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efeb4006ec0 msgr2=0x7efeb4007350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:23.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:23.635+0000 7efed3400640 1 --2- 172.21.15.17:0/670439348 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efeb4006ec0 0x7efeb4007350 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7efedc06b6a0 tx=0x7efecc004760 comp rx=0 tx=0).stop 2024-09-10T21:43:23.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:23.635+0000 7efed3400640 1 -- 172.21.15.17:0/670439348 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efeb4006ec0 msgr2=0x7efeb4007350 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:43:23.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:23.635+0000 7efed3400640 1 --2- 172.21.15.17:0/670439348 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efeb4006ec0 0x7efeb4007350 secure :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7efedc06b6a0 tx=0x7efecc004760 comp rx=0 tx=0).stop 2024-09-10T21:43:23.644 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-10T21:43:23.645 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:43:23.645 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:43:23.645 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "leader", 2024-09-10T21:43:23.645 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 144, 2024-09-10T21:43:23.645 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 62759, 2024-09-10T21:43:23.645 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-10T21:43:23.645 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-10T21:43:23.645 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-10T21:43:23.645 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-10T21:43:23.645 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:43:23.645 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 2, 2024-09-10T21:43:23.645 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:43:23.646 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-10T21:43:23.646 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-10T21:43:23.646 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:43:23.646 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:43:23.646 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:43:23.646 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:43:23.646 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:43:23.646 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:43:23.646 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:43:23.646 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:43:23.646 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:43:23.646 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:43:23.646 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:43:23.647 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:43:23.647 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-10T21:43:23.647 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-10T21:43:23.647 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:43:23.647 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:43:23.647 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:43:23.647 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:43:23.647 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:43:23.647 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:43:23.647 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:43:23.647 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:43:23.647 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:43:23.647 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:43:23.647 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:43:23.648 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:43:23.648 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:43:23.648 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-10T21:43:23.648 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-10T21:43:23.648 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-10T21:43:23.648 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-10T21:43:23.648 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-10T21:43:23.648 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "5e2f023f-d6a6-49f4-a256-f4807dce3739", 2024-09-10T21:43:23.648 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:43:23.648 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:43:23.648 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-10T21:43:23.648 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-10T21:43:23.648 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-10T21:43:23.648 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-10T21:43:23.649 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-10T21:43:23.649 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-10T21:43:23.649 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-10T21:43:23.649 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:43:23.649 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-10T21:43:23.649 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:43:23.649 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:43:23.649 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:43:23.649 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:43:23.649 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:43:23.649 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:43:23.649 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:43:23.649 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:43:23.649 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:43:23.650 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:43:23.650 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:43:23.650 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:43:23.650 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-10T21:43:23.650 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:43:23.650 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-10T21:43:23.650 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:43:23.650 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:43:23.650 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:43:23.650 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:43:23.650 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:43:23.650 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:43:23.650 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:43:23.651 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-10T21:43:23.651 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:43:23.651 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:43:23.651 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:43:23.651 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:43:23.651 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-10T21:43:23.651 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:43:23.651 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:43:23.651 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:43:23.651 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:43:23.651 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-10T21:43:23.651 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-10T21:43:23.651 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:43:23.652 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:43:23.652 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:43:23.652 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:43:23.652 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:43:23.652 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:43:23.652 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:43:23.652 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:43:23.652 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:43:23.652 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:43:23.652 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:43:23.652 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3300", 2024-09-10T21:43:23.652 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:43:23.652 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:43:23.652 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:43:23.653 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:43:23.653 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789", 2024-09-10T21:43:23.653 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:43:23.653 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:43:23.653 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:43:23.653 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:43:23.653 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789/0", 2024-09-10T21:43:23.653 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6789/0", 2024-09-10T21:43:23.653 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:43:23.653 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:43:23.653 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:43:23.653 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:43:23.653 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:43:23.654 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:43:23.654 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:43:23.654 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:43:23.654 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:43:23.654 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:43:23.654 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:43:23.654 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3301", 2024-09-10T21:43:23.654 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:43:23.654 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:43:23.654 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:43:23.654 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:43:23.654 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790", 2024-09-10T21:43:23.654 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:43:23.654 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:43:23.655 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:43:23.655 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:43:23.655 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790/0", 2024-09-10T21:43:23.655 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6790/0", 2024-09-10T21:43:23.655 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:43:23.655 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:43:23.655 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:43:23.655 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:43:23.655 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:43:23.655 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:43:23.655 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-10T21:43:23.655 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-10T21:43:23.655 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:43:23.655 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:43:23.656 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:43:23.656 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:43:23.656 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:43:23.656 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:43:23.656 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": [ 2024-09-10T21:43:23.656 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:43:23.656 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:43:23.656 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:43:23.656 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 3 2024-09-10T21:43:23.656 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:43:23.656 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:43:23.656 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-10T21:43:23.656 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:43:23.656 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:43:23.657 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:43:23.657 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:43:23.657 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:43:23.657 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:43:23.657 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-10T21:43:23.657 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:43:23.657 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:43:23.657 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:43:23.657 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:43:23.657 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:43:23.657 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:43:23.657 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": [ 2024-09-10T21:43:23.657 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:43:23.658 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:43:23.658 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:43:23.658 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:43:23.658 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:43:23.658 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:43:23.658 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:43:23.658 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-10T21:43:23.658 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-10T21:43:23.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:23.643+0000 7efee1e00640 1 -- 172.21.15.17:0/670439348 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7efebc072610 msgr2=0x7efebc074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:23.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:23.643+0000 7efee1e00640 1 --2- 172.21.15.17:0/670439348 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7efebc072610 0x7efebc074ad0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7efed4000f90 tx=0x7efed401c940 comp rx=0 tx=0).stop 2024-09-10T21:43:23.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:23.643+0000 7efee1e00640 1 -- 172.21.15.17:0/670439348 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7efedc154170 msgr2=0x7efedc149a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:23.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:23.643+0000 7efee1e00640 1 --2- 172.21.15.17:0/670439348 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7efedc154170 0x7efedc149a30 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7efedc0c74c0 tx=0x7efec800b820 comp rx=0 tx=0).stop 2024-09-10T21:43:23.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:23.643+0000 7efedb400640 1 -- 172.21.15.17:0/670439348 reap_dead start 2024-09-10T21:43:23.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:23.643+0000 7efedb400640 1 -- 172.21.15.17:0/670439348 reap_dead start 2024-09-10T21:43:23.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:23.643+0000 7efee1e00640 1 -- 172.21.15.17:0/670439348 shutdown_connections 2024-09-10T21:43:23.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:23.643+0000 7efee1e00640 1 -- 172.21.15.17:0/670439348 >> 172.21.15.17:0/670439348 conn(0x7efedc06cc20 msgr2=0x7efedc0cbc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:43:23.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:23.647+0000 7efee1e00640 1 -- 172.21.15.17:0/670439348 shutdown_connections 2024-09-10T21:43:23.659 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:23.655+0000 7efee1e00640 1 -- 172.21.15.17:0/670439348 wait complete. 2024-09-10T21:43:23.733 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-10T21:43:23.733 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-09-10T21:43:23.733 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-10T21:43:24.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:24.023+0000 7f4213e00640 1 -- 172.21.15.17:0/1127196532 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f420c04b460 msgr2=0x7f420c04adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:24.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:24.023+0000 7f4213e00640 1 --2- 172.21.15.17:0/1127196532 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f420c04b460 0x7f420c04adf0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f421406bb20 tx=0x7f421409c560 comp rx=0 tx=0).stop 2024-09-10T21:43:24.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:24.027+0000 7f4213e00640 1 -- 172.21.15.17:0/1127196532 shutdown_connections 2024-09-10T21:43:24.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:24.027+0000 7f4213e00640 1 --2- 172.21.15.17:0/1127196532 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f420c04b460 0x7f420c04adf0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:24.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:24.027+0000 7f4213e00640 1 --2- 172.21.15.17:0/1127196532 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f420c0355e0 0x7f420c04a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:24.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:24.027+0000 7f4213e00640 1 -- 172.21.15.17:0/1127196532 >> 172.21.15.17:0/1127196532 conn(0x7f420c04fc10 msgr2=0x7f420c050020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:43:24.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:24.027+0000 7f4213e00640 1 -- 172.21.15.17:0/1127196532 shutdown_connections 2024-09-10T21:43:24.029 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:24.027+0000 7f4213e00640 1 -- 172.21.15.17:0/1127196532 wait complete. 2024-09-10T21:43:24.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:24.031+0000 7f4213e00640 1 Processor -- start 2024-09-10T21:43:24.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:24.031+0000 7f4213e00640 1 -- start start 2024-09-10T21:43:24.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:24.035+0000 7f4213e00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f420c035200 0x7f420c0d5680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:24.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:24.035+0000 7f4213e00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f420c0355e0 0x7f420c0d4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:24.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:24.035+0000 7f4213e00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f420c04b460 0x7f420c0c9500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:24.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:24.039+0000 7f4212a00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f420c035200 0x7f420c0d5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:24.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:24.039+0000 7f4212a00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f420c035200 0x7f420c0d5680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.17:42124/0 (socket says 172.21.15.17:42124) 2024-09-10T21:43:24.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:24.039+0000 7f4212a00640 1 -- 172.21.15.17:0/3065158258 learned_addr learned my addr 172.21.15.17:0/3065158258 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:43:24.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:24.039+0000 7f4213400640 1 --2- 172.21.15.17:0/3065158258 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f420c04b460 0x7f420c0c9500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:24.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:24.039+0000 7f4213e00640 1 -- 172.21.15.17:0/3065158258 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f420c050730 con 0x7f420c0355e0 2024-09-10T21:43:24.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:24.039+0000 7f4213e00640 1 -- 172.21.15.17:0/3065158258 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f420c0456c0 con 0x7f420c035200 2024-09-10T21:43:24.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:24.039+0000 7f4213e00640 1 -- 172.21.15.17:0/3065158258 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f420c0508b0 con 0x7f420c04b460 2024-09-10T21:43:24.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:24.039+0000 7f4212000640 1 --2- 172.21.15.17:0/3065158258 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f420c0355e0 0x7f420c0d4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:24.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:24.043+0000 7f4212000640 1 -- 172.21.15.17:0/3065158258 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f420c04b460 msgr2=0x7f420c0c9500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:24.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:24.043+0000 7f4212000640 1 --2- 172.21.15.17:0/3065158258 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f420c04b460 0x7f420c0c9500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:24.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:24.043+0000 7f4212000640 1 -- 172.21.15.17:0/3065158258 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f420c035200 msgr2=0x7f420c0d5680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:24.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:24.043+0000 7f4212000640 1 --2- 172.21.15.17:0/3065158258 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f420c035200 0x7f420c0d5680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:24.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:24.043+0000 7f4212000640 1 -- 172.21.15.17:0/3065158258 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f420c034840 con 0x7f420c0355e0 2024-09-10T21:43:24.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:24.043+0000 7f4212a00640 1 --2- 172.21.15.17:0/3065158258 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f420c035200 0x7f420c0d5680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:43:24.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:24.043+0000 7f4212000640 1 --2- 172.21.15.17:0/3065158258 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f420c0355e0 0x7f420c0d4ea0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f41fc00e9c0 tx=0x7f41fc00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:24.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:24.059+0000 7f420aa00640 1 -- 172.21.15.17:0/3065158258 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41fc00ccc0 con 0x7f420c0355e0 2024-09-10T21:43:24.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:24.059+0000 7f420aa00640 1 -- 172.21.15.17:0/3065158258 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f41fc010430 con 0x7f420c0355e0 2024-09-10T21:43:24.060 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:24.059+0000 7f420aa00640 1 -- 172.21.15.17:0/3065158258 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41fc010860 con 0x7f420c0355e0 2024-09-10T21:43:24.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:24.059+0000 7f4213e00640 1 -- 172.21.15.17:0/3065158258 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f420c0c9df0 con 0x7f420c0355e0 2024-09-10T21:43:24.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:24.059+0000 7f4213e00640 1 -- 172.21.15.17:0/3065158258 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f420c0c8fe0 con 0x7f420c0355e0 2024-09-10T21:43:24.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:24.063+0000 7f4213e00640 1 --2- 172.21.15.17:0/3065158258 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f420c033de0 0x7f420c184330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:24.065 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:24.063+0000 7f4213e00640 1 -- 172.21.15.17:0/3065158258 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f420c0c8e60 con 0x7f420c033de0 2024-09-10T21:43:24.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:24.071+0000 7f4212a00640 1 --2- 172.21.15.17:0/3065158258 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f420c033de0 0x7f420c184330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:24.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:24.083+0000 7f420aa00640 1 -- 172.21.15.17:0/3065158258 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f41fc010a00 con 0x7f420c0355e0 2024-09-10T21:43:24.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:24.083+0000 7f4212a00640 1 -- 172.21.15.17:0/3065158258 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f420c00dc30 con 0x7f420c033de0 2024-09-10T21:43:24.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:24.083+0000 7f4212a00640 1 --2- 172.21.15.17:0/3065158258 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f420c033de0 0x7f420c184330 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f420400d680 tx=0x7f420400db40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:24.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:24.087+0000 7f420aa00640 1 --2- 172.21.15.17:0/3065158258 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f41f00726e0 0x7f41f0074ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:24.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:24.095+0000 7f420aa00640 1 -- 172.21.15.17:0/3065158258 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f41fc094060 con 0x7f420c0355e0 2024-09-10T21:43:24.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:24.095+0000 7f420aa00640 1 -- 172.21.15.17:0/3065158258 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4204016070 con 0x7f420c033de0 2024-09-10T21:43:24.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:24.095+0000 7f420aa00640 1 -- 172.21.15.17:0/3065158258 <== mon.1 v2:172.21.15.31:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f4204005220 con 0x7f420c033de0 2024-09-10T21:43:24.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:24.095+0000 7f420aa00640 1 -- 172.21.15.17:0/3065158258 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f420c033de0 msgr2=0x7f420c184330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:24.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:24.095+0000 7f420aa00640 1 --2- 172.21.15.17:0/3065158258 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f420c033de0 0x7f420c184330 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f420400d680 tx=0x7f420400db40 comp rx=0 tx=0).stop 2024-09-10T21:43:24.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:24.095+0000 7f420aa00640 1 -- 172.21.15.17:0/3065158258 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f420c033de0 msgr2=0x7f420c184330 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:43:24.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:24.095+0000 7f420aa00640 1 --2- 172.21.15.17:0/3065158258 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f420c033de0 0x7f420c184330 secure :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f420400d680 tx=0x7f420400db40 comp rx=0 tx=0).stop 2024-09-10T21:43:24.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:24.099+0000 7f4213400640 1 --2- 172.21.15.17:0/3065158258 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f41f00726e0 0x7f41f0074ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:24.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:24.107+0000 7f4213400640 1 --2- 172.21.15.17:0/3065158258 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f41f00726e0 0x7f41f0074ba0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f420c050cf0 tx=0x7f421409adf0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:24.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:24.107+0000 7f4213e00640 1 --2- 172.21.15.17:0/3065158258 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f420c0341c0 0x7f420c184870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:24.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:24.107+0000 7f4213e00640 1 -- 172.21.15.17:0/3065158258 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f420c0c9220 con 0x7f420c0341c0 2024-09-10T21:43:24.114 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:24.111+0000 7f4212a00640 1 --2- 172.21.15.17:0/3065158258 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f420c0341c0 0x7f420c184870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:24.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:24.115+0000 7f4212a00640 1 -- 172.21.15.17:0/3065158258 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f420c0cb3b0 con 0x7f420c0341c0 2024-09-10T21:43:24.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:24.119+0000 7f4212a00640 1 --2- 172.21.15.17:0/3065158258 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f420c0341c0 0x7f420c184870 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f420400d680 tx=0x7f4204004630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:24.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:24.119+0000 7f420aa00640 1 -- 172.21.15.17:0/3065158258 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f420400b840 con 0x7f420c0341c0 2024-09-10T21:43:24.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:24.119+0000 7f420aa00640 1 -- 172.21.15.17:0/3065158258 <== mon.1 v2:172.21.15.31:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5043 (secure 0 0 0) 0x7f420400b840 con 0x7f420c0341c0 2024-09-10T21:43:24.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:24.119+0000 7f420aa00640 1 -- 172.21.15.17:0/3065158258 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f420c0341c0 msgr2=0x7f420c184870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:24.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:24.119+0000 7f420aa00640 1 --2- 172.21.15.17:0/3065158258 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f420c0341c0 0x7f420c184870 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f420400d680 tx=0x7f4204004630 comp rx=0 tx=0).stop 2024-09-10T21:43:24.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:24.119+0000 7f420aa00640 1 -- 172.21.15.17:0/3065158258 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f420c0341c0 msgr2=0x7f420c184870 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:43:24.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:24.119+0000 7f420aa00640 1 --2- 172.21.15.17:0/3065158258 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f420c0341c0 0x7f420c184870 secure :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f420400d680 tx=0x7f4204004630 comp rx=0 tx=0).stop 2024-09-10T21:43:24.122 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-10T21:43:24.122 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:43:24.122 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:43:24.122 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-10T21:43:24.122 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 144, 2024-09-10T21:43:24.122 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 215544, 2024-09-10T21:43:24.122 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-10T21:43:24.122 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-10T21:43:24.122 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-10T21:43:24.123 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-10T21:43:24.123 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:43:24.123 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 2, 2024-09-10T21:43:24.123 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:43:24.123 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-10T21:43:24.123 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-10T21:43:24.123 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:43:24.123 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:43:24.123 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:43:24.123 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:43:24.123 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:43:24.123 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:43:24.123 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:43:24.124 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:43:24.124 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:43:24.124 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:43:24.124 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:43:24.124 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:43:24.124 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-10T21:43:24.124 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-10T21:43:24.124 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:43:24.124 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:43:24.124 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:43:24.124 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:43:24.124 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:43:24.124 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:43:24.125 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:43:24.125 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:43:24.125 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:43:24.125 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:43:24.125 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:43:24.125 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:43:24.125 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:43:24.125 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-10T21:43:24.125 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-10T21:43:24.125 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-10T21:43:24.125 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-10T21:43:24.125 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-10T21:43:24.125 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "5e2f023f-d6a6-49f4-a256-f4807dce3739", 2024-09-10T21:43:24.126 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:43:24.126 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:43:24.126 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-10T21:43:24.126 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-10T21:43:24.126 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-10T21:43:24.126 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-10T21:43:24.126 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-10T21:43:24.126 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-10T21:43:24.126 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-10T21:43:24.126 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:43:24.126 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-10T21:43:24.126 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:43:24.126 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:43:24.127 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:43:24.127 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:43:24.127 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:43:24.127 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:43:24.127 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:43:24.127 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:43:24.127 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:43:24.127 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:43:24.127 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:43:24.127 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:43:24.127 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-10T21:43:24.127 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:43:24.127 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-10T21:43:24.127 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:43:24.128 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:43:24.128 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:43:24.128 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:43:24.128 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:43:24.128 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:43:24.128 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:43:24.128 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-10T21:43:24.128 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:43:24.128 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:43:24.128 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:43:24.128 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:43:24.128 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-10T21:43:24.128 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:43:24.129 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:43:24.129 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:43:24.129 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:43:24.129 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-10T21:43:24.129 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-10T21:43:24.129 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:43:24.129 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:43:24.129 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:43:24.129 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:43:24.129 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:43:24.129 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:43:24.129 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:43:24.129 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:43:24.129 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:43:24.130 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:43:24.130 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:43:24.130 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3300", 2024-09-10T21:43:24.130 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:43:24.130 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:43:24.130 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:43:24.130 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:43:24.130 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789", 2024-09-10T21:43:24.130 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:43:24.130 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:43:24.130 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:43:24.130 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:43:24.130 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789/0", 2024-09-10T21:43:24.130 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6789/0", 2024-09-10T21:43:24.131 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:43:24.131 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:43:24.131 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:43:24.131 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:43:24.131 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:43:24.131 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:43:24.131 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:43:24.131 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:43:24.131 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:43:24.131 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:43:24.131 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:43:24.131 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3301", 2024-09-10T21:43:24.131 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:43:24.131 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:43:24.132 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:43:24.132 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:43:24.132 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790", 2024-09-10T21:43:24.132 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:43:24.132 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:43:24.132 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:43:24.132 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:43:24.132 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790/0", 2024-09-10T21:43:24.132 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6790/0", 2024-09-10T21:43:24.132 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:43:24.132 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:43:24.132 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:43:24.132 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:43:24.133 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:43:24.133 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:43:24.133 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-10T21:43:24.133 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-10T21:43:24.133 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:43:24.133 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:43:24.133 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:43:24.133 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:43:24.133 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:43:24.133 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:43:24.133 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": [ 2024-09-10T21:43:24.133 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:43:24.133 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:43:24.134 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:43:24.134 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 3 2024-09-10T21:43:24.134 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:43:24.134 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:43:24.134 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-10T21:43:24.134 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:43:24.134 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:43:24.134 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:43:24.134 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 7 2024-09-10T21:43:24.134 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:43:24.134 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:43:24.134 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-10T21:43:24.134 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:43:24.134 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:43:24.135 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:43:24.135 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 6 2024-09-10T21:43:24.135 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:43:24.135 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:43:24.135 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": [ 2024-09-10T21:43:24.135 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:43:24.135 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:43:24.135 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:43:24.135 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:43:24.135 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:43:24.135 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:43:24.135 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:43:24.135 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-10T21:43:24.135 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-10T21:43:24.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:24.119+0000 7f4213e00640 1 -- 172.21.15.17:0/3065158258 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f41f00726e0 msgr2=0x7f41f0074ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:24.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:24.119+0000 7f4213e00640 1 --2- 172.21.15.17:0/3065158258 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f41f00726e0 0x7f41f0074ba0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f420c050cf0 tx=0x7f421409adf0 comp rx=0 tx=0).stop 2024-09-10T21:43:24.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:24.119+0000 7f4213e00640 1 -- 172.21.15.17:0/3065158258 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f420c0355e0 msgr2=0x7f420c0d4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:24.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:24.119+0000 7f4213e00640 1 --2- 172.21.15.17:0/3065158258 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f420c0355e0 0x7f420c0d4ea0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f41fc00e9c0 tx=0x7f41fc00ee80 comp rx=0 tx=0).stop 2024-09-10T21:43:24.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:24.119+0000 7f4213400640 1 -- 172.21.15.17:0/3065158258 reap_dead start 2024-09-10T21:43:24.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:24.119+0000 7f4213400640 1 -- 172.21.15.17:0/3065158258 reap_dead start 2024-09-10T21:43:24.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:24.123+0000 7f4213e00640 1 -- 172.21.15.17:0/3065158258 shutdown_connections 2024-09-10T21:43:24.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:24.123+0000 7f4213e00640 1 -- 172.21.15.17:0/3065158258 >> 172.21.15.17:0/3065158258 conn(0x7f420c04fc10 msgr2=0x7f420c04c620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:43:24.148 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:24.143+0000 7f4213e00640 1 -- 172.21.15.17:0/3065158258 shutdown_connections 2024-09-10T21:43:24.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:24.143+0000 7f4213e00640 1 -- 172.21.15.17:0/3065158258 wait complete. 2024-09-10T21:43:24.209 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-10T21:43:24.209 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-09-10T21:43:24.209 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-10T21:43:24.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:24.511+0000 7fbc97e00640 1 -- 172.21.15.17:0/2206608524 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fbc78004f50 msgr2=0x7fbc780099c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:24.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:24.511+0000 7fbc97e00640 1 --2- 172.21.15.17:0/2206608524 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fbc78004f50 0x7fbc780099c0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fbc7c00c730 tx=0x7fbc7c01c420 comp rx=0 tx=0).stop 2024-09-10T21:43:24.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:24.511+0000 7fbc97e00640 1 -- 172.21.15.17:0/2206608524 shutdown_connections 2024-09-10T21:43:24.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:24.511+0000 7fbc97e00640 1 --2- 172.21.15.17:0/2206608524 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fbc78004f50 0x7fbc780099c0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:24.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:24.511+0000 7fbc97e00640 1 -- 172.21.15.17:0/2206608524 >> 172.21.15.17:0/2206608524 conn(0x7fbc900700d0 msgr2=0x7fbc900704e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:43:24.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:24.515+0000 7fbc97e00640 1 -- 172.21.15.17:0/2206608524 shutdown_connections 2024-09-10T21:43:24.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:24.519+0000 7fbc97e00640 1 -- 172.21.15.17:0/2206608524 wait complete. 2024-09-10T21:43:24.523 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:24.519+0000 7fbc97e00640 1 Processor -- start 2024-09-10T21:43:24.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:24.527+0000 7fbc97e00640 1 -- start start 2024-09-10T21:43:24.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:24.527+0000 7fbc97e00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fbc78004f50 0x7fbc901ed320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:24.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:24.527+0000 7fbc97e00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbc90207e90 0x7fbc90210170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:24.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:24.527+0000 7fbc97e00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbc902106b0 0x7fbc902111d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:24.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:24.527+0000 7fbc97e00640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fbc9006dc30 con 0x7fbc902106b0 2024-09-10T21:43:24.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:24.527+0000 7fbc97e00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fbc900c18f0 con 0x7fbc90207e90 2024-09-10T21:43:24.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:24.527+0000 7fbc97e00640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7fbc90168000 con 0x7fbc78004f50 2024-09-10T21:43:24.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:24.531+0000 7fbc95400640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fbc78004f50 0x7fbc901ed320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:24.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:24.531+0000 7fbc95400640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fbc78004f50 0x7fbc901ed320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3301/0 says I am v2:172.21.15.17:39594/0 (socket says 172.21.15.17:39594) 2024-09-10T21:43:24.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:24.531+0000 7fbc95400640 1 -- 172.21.15.17:0/1006119173 learned_addr learned my addr 172.21.15.17:0/1006119173 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:43:24.533 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:24.531+0000 7fbc94a00640 1 --2- 172.21.15.17:0/1006119173 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbc90207e90 0x7fbc90210170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:24.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:24.531+0000 7fbc95400640 1 -- 172.21.15.17:0/1006119173 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbc90207e90 msgr2=0x7fbc90210170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:24.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:24.531+0000 7fbc95400640 1 --2- 172.21.15.17:0/1006119173 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbc90207e90 0x7fbc90210170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:24.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:24.531+0000 7fbc95400640 1 -- 172.21.15.17:0/1006119173 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbc902106b0 msgr2=0x7fbc902111d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:24.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:24.531+0000 7fbc95400640 1 --2- 172.21.15.17:0/1006119173 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbc902106b0 0x7fbc902111d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:24.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:24.531+0000 7fbc95400640 1 -- 172.21.15.17:0/1006119173 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc90210b30 con 0x7fbc78004f50 2024-09-10T21:43:24.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:24.531+0000 7fbc95400640 1 --2- 172.21.15.17:0/1006119173 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fbc78004f50 0x7fbc901ed320 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fbc8400d9e0 tx=0x7fbc8400dea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:24.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:24.531+0000 7fbc8d600640 1 -- 172.21.15.17:0/1006119173 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc84008c50 con 0x7fbc78004f50 2024-09-10T21:43:24.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:24.531+0000 7fbc8d600640 1 -- 172.21.15.17:0/1006119173 <== mon.2 v2:172.21.15.31:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbc84008e10 con 0x7fbc78004f50 2024-09-10T21:43:24.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:24.531+0000 7fbc8d600640 1 -- 172.21.15.17:0/1006119173 <== mon.2 v2:172.21.15.31:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc84004290 con 0x7fbc78004f50 2024-09-10T21:43:24.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:24.531+0000 7fbc97e00640 1 -- 172.21.15.17:0/1006119173 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc900ccaf0 con 0x7fbc78004f50 2024-09-10T21:43:24.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:24.531+0000 7fbc97e00640 1 -- 172.21.15.17:0/1006119173 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc901f0ef0 con 0x7fbc78004f50 2024-09-10T21:43:24.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:24.535+0000 7fbc97e00640 1 --2- 172.21.15.17:0/1006119173 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fbc9020fbb0 0x7fbc90211710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:24.545 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:24.535+0000 7fbc97e00640 1 -- 172.21.15.17:0/1006119173 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7fbc900c6190 con 0x7fbc9020fbb0 2024-09-10T21:43:24.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:24.543+0000 7fbc94a00640 1 --2- 172.21.15.17:0/1006119173 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fbc9020fbb0 0x7fbc90211710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:24.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:24.543+0000 7fbc94a00640 1 -- 172.21.15.17:0/1006119173 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fbc901f10f0 con 0x7fbc9020fbb0 2024-09-10T21:43:24.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:24.543+0000 7fbc94a00640 1 --2- 172.21.15.17:0/1006119173 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fbc9020fbb0 0x7fbc90211710 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fbc7c005000 tx=0x7fbc7c00c300 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:24.551 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:24.543+0000 7fbc8d600640 1 -- 172.21.15.17:0/1006119173 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc7c004840 con 0x7fbc9020fbb0 2024-09-10T21:43:24.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:24.547+0000 7fbc8d600640 1 -- 172.21.15.17:0/1006119173 <== mon.2 v2:172.21.15.31:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fbc7c0049e0 con 0x7fbc9020fbb0 2024-09-10T21:43:24.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:24.547+0000 7fbc8d600640 1 -- 172.21.15.17:0/1006119173 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fbc9020fbb0 msgr2=0x7fbc90211710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:24.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:24.547+0000 7fbc8d600640 1 --2- 172.21.15.17:0/1006119173 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fbc9020fbb0 0x7fbc90211710 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fbc7c005000 tx=0x7fbc7c00c300 comp rx=0 tx=0).stop 2024-09-10T21:43:24.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:24.547+0000 7fbc8d600640 1 -- 172.21.15.17:0/1006119173 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fbc9020fbb0 msgr2=0x7fbc90211710 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:43:24.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:24.547+0000 7fbc8d600640 1 --2- 172.21.15.17:0/1006119173 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fbc9020fbb0 0x7fbc90211710 secure :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fbc7c005000 tx=0x7fbc7c00c300 comp rx=0 tx=0).stop 2024-09-10T21:43:24.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:24.551+0000 7fbc8d600640 1 -- 172.21.15.17:0/1006119173 <== mon.2 v2:172.21.15.31:3301/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fbc84003600 con 0x7fbc78004f50 2024-09-10T21:43:24.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:24.559+0000 7fbc8d600640 1 --2- 172.21.15.17:0/1006119173 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fbc70072150 0x7fbc70074610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:24.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:24.559+0000 7fbc8d600640 1 -- 172.21.15.17:0/1006119173 <== mon.2 v2:172.21.15.31:3301/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fbc84093c20 con 0x7fbc78004f50 2024-09-10T21:43:24.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:24.559+0000 7fbc97e00640 1 --2- 172.21.15.17:0/1006119173 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fbc90211c50 0x7fbc9020dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:24.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:24.559+0000 7fbc97e00640 1 -- 172.21.15.17:0/1006119173 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7fbc900c2c40 con 0x7fbc90211c50 2024-09-10T21:43:24.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:24.563+0000 7fbc94a00640 1 --2- 172.21.15.17:0/1006119173 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fbc70072150 0x7fbc70074610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:24.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:24.563+0000 7fbc95e00640 1 --2- 172.21.15.17:0/1006119173 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fbc90211c50 0x7fbc9020dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:24.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:24.563+0000 7fbc95e00640 1 -- 172.21.15.17:0/1006119173 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fbc900c6cd0 con 0x7fbc90211c50 2024-09-10T21:43:24.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:24.563+0000 7fbc94a00640 1 --2- 172.21.15.17:0/1006119173 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fbc70072150 0x7fbc70074610 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fbc900b2450 tx=0x7fbc7c0af000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:24.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:24.563+0000 7fbc95e00640 1 --2- 172.21.15.17:0/1006119173 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fbc90211c50 0x7fbc9020dcc0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fbc900710b0 tx=0x7fbc88004e40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:24.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:24.563+0000 7fbc8d600640 1 -- 172.21.15.17:0/1006119173 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc88018030 con 0x7fbc90211c50 2024-09-10T21:43:24.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:24.563+0000 7fbc8d600640 1 -- 172.21.15.17:0/1006119173 <== mon.2 v2:172.21.15.31:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4540 (secure 0 0 0) 0x7fbc88007390 con 0x7fbc90211c50 2024-09-10T21:43:24.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:24.563+0000 7fbc8d600640 1 -- 172.21.15.17:0/1006119173 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fbc90211c50 msgr2=0x7fbc9020dcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:24.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:24.563+0000 7fbc8d600640 1 --2- 172.21.15.17:0/1006119173 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fbc90211c50 0x7fbc9020dcc0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fbc900710b0 tx=0x7fbc88004e40 comp rx=0 tx=0).stop 2024-09-10T21:43:24.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:24.563+0000 7fbc8d600640 1 -- 172.21.15.17:0/1006119173 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fbc90211c50 msgr2=0x7fbc9020dcc0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:43:24.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:24.563+0000 7fbc8d600640 1 --2- 172.21.15.17:0/1006119173 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fbc90211c50 0x7fbc9020dcc0 secure :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fbc900710b0 tx=0x7fbc88004e40 comp rx=0 tx=0).stop 2024-09-10T21:43:24.576 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-10T21:43:24.576 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:43:24.576 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:43:24.576 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-10T21:43:24.576 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 144, 2024-09-10T21:43:24.576 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 6691, 2024-09-10T21:43:24.576 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-10T21:43:24.576 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-10T21:43:24.576 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-10T21:43:24.577 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-10T21:43:24.577 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:43:24.577 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 3, 2024-09-10T21:43:24.577 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:43:24.577 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-10T21:43:24.577 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-10T21:43:24.577 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:43:24.577 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:43:24.577 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:43:24.577 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:43:24.577 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:43:24.578 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:43:24.578 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:43:24.578 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:43:24.578 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:43:24.578 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:43:24.578 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:43:24.578 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:43:24.578 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-10T21:43:24.578 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-10T21:43:24.579 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:43:24.579 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:43:24.579 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:43:24.579 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:43:24.579 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:43:24.579 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:43:24.579 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:43:24.579 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:43:24.580 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:43:24.580 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:43:24.580 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:43:24.580 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:43:24.580 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:43:24.581 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-10T21:43:24.581 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-10T21:43:24.581 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-10T21:43:24.581 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-10T21:43:24.581 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-10T21:43:24.582 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "5e2f023f-d6a6-49f4-a256-f4807dce3739", 2024-09-10T21:43:24.582 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:43:24.582 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:43:24.582 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-10T21:43:24.582 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-10T21:43:24.583 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-10T21:43:24.583 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-10T21:43:24.583 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-10T21:43:24.583 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-10T21:43:24.583 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-10T21:43:24.583 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:43:24.583 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-10T21:43:24.584 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:43:24.584 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:43:24.584 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:43:24.584 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:43:24.584 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:43:24.584 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:43:24.584 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:43:24.584 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:43:24.585 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:43:24.585 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:43:24.585 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:43:24.585 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:43:24.585 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-10T21:43:24.585 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:43:24.585 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-10T21:43:24.585 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:43:24.585 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:43:24.585 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:43:24.586 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:43:24.586 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:43:24.586 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:43:24.586 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:43:24.586 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-10T21:43:24.586 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:43:24.586 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:43:24.586 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:43:24.586 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:43:24.587 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-10T21:43:24.587 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:43:24.587 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:43:24.587 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:43:24.587 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:43:24.587 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-10T21:43:24.587 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-10T21:43:24.588 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:43:24.588 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:43:24.588 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:43:24.588 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:43:24.588 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:43:24.588 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:43:24.588 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:43:24.588 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:43:24.588 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:43:24.588 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:43:24.588 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:43:24.589 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3300", 2024-09-10T21:43:24.589 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:43:24.589 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:43:24.589 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:43:24.589 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:43:24.590 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789", 2024-09-10T21:43:24.590 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:43:24.590 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:43:24.590 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:43:24.590 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:43:24.591 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789/0", 2024-09-10T21:43:24.591 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6789/0", 2024-09-10T21:43:24.591 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:43:24.591 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:43:24.591 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:43:24.592 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:43:24.592 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:43:24.592 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:43:24.592 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:43:24.593 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:43:24.593 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:43:24.593 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:43:24.593 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:43:24.594 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3301", 2024-09-10T21:43:24.594 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:43:24.594 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:43:24.594 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:43:24.595 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:43:24.595 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790", 2024-09-10T21:43:24.595 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:43:24.595 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:43:24.595 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:43:24.596 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:43:24.596 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790/0", 2024-09-10T21:43:24.596 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6790/0", 2024-09-10T21:43:24.596 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:43:24.596 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:43:24.597 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:43:24.597 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:43:24.597 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:43:24.597 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:43:24.597 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-10T21:43:24.597 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-10T21:43:24.598 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:43:24.598 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:43:24.598 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:43:24.598 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:43:24.598 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:43:24.599 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:43:24.599 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-10T21:43:24.599 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:43:24.599 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:43:24.599 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:43:24.600 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 2 2024-09-10T21:43:24.600 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:43:24.600 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:43:24.600 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:43:24.600 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-10T21:43:24.601 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-10T21:43:24.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:24.567+0000 7fbc97e00640 1 -- 172.21.15.17:0/1006119173 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fbc70072150 msgr2=0x7fbc70074610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:24.601 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:24.567+0000 7fbc97e00640 1 --2- 172.21.15.17:0/1006119173 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fbc70072150 0x7fbc70074610 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fbc900b2450 tx=0x7fbc7c0af000 comp rx=0 tx=0).stop 2024-09-10T21:43:24.602 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:24.567+0000 7fbc97e00640 1 -- 172.21.15.17:0/1006119173 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fbc78004f50 msgr2=0x7fbc901ed320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:24.602 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:24.567+0000 7fbc97e00640 1 --2- 172.21.15.17:0/1006119173 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fbc78004f50 0x7fbc901ed320 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fbc8400d9e0 tx=0x7fbc8400dea0 comp rx=0 tx=0).stop 2024-09-10T21:43:24.602 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:24.567+0000 7fbc95e00640 1 -- 172.21.15.17:0/1006119173 reap_dead start 2024-09-10T21:43:24.602 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:24.567+0000 7fbc97e00640 1 -- 172.21.15.17:0/1006119173 shutdown_connections 2024-09-10T21:43:24.602 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:24.567+0000 7fbc97e00640 1 -- 172.21.15.17:0/1006119173 >> 172.21.15.17:0/1006119173 conn(0x7fbc900700d0 msgr2=0x7fbc901f35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:43:24.602 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:24.571+0000 7fbc97e00640 1 -- 172.21.15.17:0/1006119173 shutdown_connections 2024-09-10T21:43:24.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:24.571+0000 7fbc97e00640 1 -- 172.21.15.17:0/1006119173 wait complete. 2024-09-10T21:43:24.618 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-10T21:43:24.618 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-09-10T21:43:24.619 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-09-10T21:43:24.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:24.987+0000 7f13e2600640 1 -- 172.21.15.17:0/1419247135 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f13dc154890 msgr2=0x7f13dc06e5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:24.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:24.987+0000 7f13e2600640 1 --2- 172.21.15.17:0/1419247135 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f13dc154890 0x7f13dc06e5c0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f13cc009840 tx=0x7f13cc01f010 comp rx=0 tx=0).stop 2024-09-10T21:43:24.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:24.991+0000 7f13e2600640 1 -- 172.21.15.17:0/1419247135 shutdown_connections 2024-09-10T21:43:24.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:24.991+0000 7f13e2600640 1 --2- 172.21.15.17:0/1419247135 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f13dc154890 0x7f13dc06e5c0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:24.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:24.991+0000 7f13e2600640 1 -- 172.21.15.17:0/1419247135 >> 172.21.15.17:0/1419247135 conn(0x7f13dc070160 msgr2=0x7f13dc070570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:43:24.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:24.991+0000 7f13e2600640 1 -- 172.21.15.17:0/1419247135 shutdown_connections 2024-09-10T21:43:24.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:24.991+0000 7f13e2600640 1 -- 172.21.15.17:0/1419247135 wait complete. 2024-09-10T21:43:24.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:24.995+0000 7f13e2600640 1 Processor -- start 2024-09-10T21:43:24.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:24.995+0000 7f13e2600640 1 -- start start 2024-09-10T21:43:25.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:25.003+0000 7f13e2600640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f13dc0c1040 0x7f13dc15c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:25.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:25.007+0000 7f13e2600640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f13dc0c81b0 0x7f13dc1f88d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:25.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:25.007+0000 7f13e2600640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f13dc154890 0x7f13dc152890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:25.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:25.007+0000 7f13e2600640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f13dc06e0a0 con 0x7f13dc0c1040 2024-09-10T21:43:25.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:25.007+0000 7f13e2600640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f13dc06ddc0 con 0x7f13dc0c81b0 2024-09-10T21:43:25.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:25.007+0000 7f13e2600640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f13dc154c70 con 0x7f13dc154890 2024-09-10T21:43:25.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:25.011+0000 7f13daa00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f13dc0c81b0 0x7f13dc1f88d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:25.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:25.011+0000 7f13daa00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f13dc0c81b0 0x7f13dc1f88d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.17:59306/0 (socket says 172.21.15.17:59306) 2024-09-10T21:43:25.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:25.011+0000 7f13daa00640 1 -- 172.21.15.17:0/868430105 learned_addr learned my addr 172.21.15.17:0/868430105 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:43:25.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:25.011+0000 7f13dbe00640 1 --2- 172.21.15.17:0/868430105 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f13dc154890 0x7f13dc152890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:25.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:25.015+0000 7f13daa00640 1 -- 172.21.15.17:0/868430105 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f13dc154890 msgr2=0x7f13dc152890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:25.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:25.015+0000 7f13daa00640 1 --2- 172.21.15.17:0/868430105 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f13dc154890 0x7f13dc152890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:25.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:25.015+0000 7f13daa00640 1 -- 172.21.15.17:0/868430105 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f13dc0c1040 msgr2=0x7f13dc15c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:25.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:25.015+0000 7f13daa00640 1 --2- 172.21.15.17:0/868430105 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f13dc0c1040 0x7f13dc15c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:25.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:25.015+0000 7f13daa00640 1 -- 172.21.15.17:0/868430105 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13dc150f10 con 0x7f13dc0c81b0 2024-09-10T21:43:25.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:25.015+0000 7f13daa00640 1 --2- 172.21.15.17:0/868430105 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f13dc0c81b0 0x7f13dc1f88d0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f13c800c500 tx=0x7f13c800c9c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:25.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:25.023+0000 7f13d3400640 1 -- 172.21.15.17:0/868430105 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13c8017070 con 0x7f13dc0c81b0 2024-09-10T21:43:25.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:25.023+0000 7f13d3400640 1 -- 172.21.15.17:0/868430105 <== mon.1 v2:172.21.15.31:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f13c8003600 con 0x7f13dc0c81b0 2024-09-10T21:43:25.025 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:25.023+0000 7f13d3400640 1 -- 172.21.15.17:0/868430105 <== mon.1 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13c8003a90 con 0x7f13dc0c81b0 2024-09-10T21:43:25.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:25.023+0000 7f13e2600640 1 -- 172.21.15.17:0/868430105 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13dc1f8e10 con 0x7f13dc0c81b0 2024-09-10T21:43:25.026 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:25.023+0000 7f13e2600640 1 -- 172.21.15.17:0/868430105 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13dc14e1b0 con 0x7f13dc0c81b0 2024-09-10T21:43:25.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:25.031+0000 7f13e2600640 1 -- 172.21.15.17:0/868430105 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13dc0b1b80 con 0x7f13dc0c81b0 2024-09-10T21:43:25.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:25.035+0000 7f13d3400640 1 -- 172.21.15.17:0/868430105 <== mon.1 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f13c8003c30 con 0x7f13dc0c81b0 2024-09-10T21:43:25.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:25.035+0000 7f13d3400640 1 --2- 172.21.15.17:0/868430105 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f13bc0726b0 0x7f13bc074b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:25.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:25.039+0000 7f13d3400640 1 -- 172.21.15.17:0/868430105 <== mon.1 v2:172.21.15.31:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f13c8092df0 con 0x7f13dc0c81b0 2024-09-10T21:43:25.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:25.047+0000 7f13db400640 1 --2- 172.21.15.17:0/868430105 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f13bc0726b0 0x7f13bc074b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:25.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:25.047+0000 7f13db400640 1 --2- 172.21.15.17:0/868430105 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f13bc0726b0 0x7f13bc074b70 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f13cc0b2000 tx=0x7f13cc000bd0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:25.062 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:25.055+0000 7f13d3400640 1 -- 172.21.15.17:0/868430105 <== mon.1 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f13c805daf0 con 0x7f13dc0c81b0 2024-09-10T21:43:25.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:25.591+0000 7f13e2600640 1 -- 172.21.15.17:0/868430105 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) -- 0x7f13dc0c79f0 con 0x7f13dc0c81b0 2024-09-10T21:43:25.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:25.603+0000 7f13d3400640 1 -- 172.21.15.17:0/868430105 <== mon.1 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) ==== 57+0+0 (secure 0 0 0) 0x7f13c8062520 con 0x7f13dc0c81b0 2024-09-10T21:43:25.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:25.607+0000 7f13e2600640 1 -- 172.21.15.17:0/868430105 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f13bc0726b0 msgr2=0x7f13bc074b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:25.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:25.607+0000 7f13e2600640 1 --2- 172.21.15.17:0/868430105 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f13bc0726b0 0x7f13bc074b70 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f13cc0b2000 tx=0x7f13cc000bd0 comp rx=0 tx=0).stop 2024-09-10T21:43:25.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:25.607+0000 7f13e2600640 1 -- 172.21.15.17:0/868430105 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f13dc0c81b0 msgr2=0x7f13dc1f88d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:25.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:25.607+0000 7f13e2600640 1 --2- 172.21.15.17:0/868430105 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f13dc0c81b0 0x7f13dc1f88d0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f13c800c500 tx=0x7f13c800c9c0 comp rx=0 tx=0).stop 2024-09-10T21:43:25.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:25.607+0000 7f13e2600640 1 -- 172.21.15.17:0/868430105 shutdown_connections 2024-09-10T21:43:25.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:25.607+0000 7f13e2600640 1 --2- 172.21.15.17:0/868430105 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f13bc0726b0 0x7f13bc074b70 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:25.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:25.607+0000 7f13e2600640 1 --2- 172.21.15.17:0/868430105 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f13dc154890 0x7f13dc152890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:25.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:25.607+0000 7f13e2600640 1 --2- 172.21.15.17:0/868430105 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f13dc0c81b0 0x7f13dc1f88d0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:25.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:25.607+0000 7f13e2600640 1 --2- 172.21.15.17:0/868430105 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f13dc0c1040 0x7f13dc15c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:25.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:25.607+0000 7f13e2600640 1 -- 172.21.15.17:0/868430105 >> 172.21.15.17:0/868430105 conn(0x7f13dc070160 msgr2=0x7f13dc0c9330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:43:25.612 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:25.607+0000 7f13e2600640 1 -- 172.21.15.17:0/868430105 shutdown_connections 2024-09-10T21:43:25.612 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:25.607+0000 7f13e2600640 1 -- 172.21.15.17:0/868430105 wait complete. 2024-09-10T21:43:25.660 INFO:tasks.mon_thrash.mon_thrasher:waiting for 10.0 secs before continuing thrashing 2024-09-10T21:43:26.914 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:43:26.920 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:43:35.661 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-09-10T21:43:35.661 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-10T21:43:35.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:35.979+0000 7f10af200640 1 -- 172.21.15.17:0/2403449738 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f10a80d8780 msgr2=0x7f10a804ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:35.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:35.979+0000 7f10af200640 1 --2- 172.21.15.17:0/2403449738 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f10a80d8780 0x7f10a804ee70 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f10b006b290 tx=0x7f10b0099110 comp rx=0 tx=0).stop 2024-09-10T21:43:35.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:35.983+0000 7f10af200640 1 -- 172.21.15.17:0/2403449738 shutdown_connections 2024-09-10T21:43:35.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:35.983+0000 7f10af200640 1 --2- 172.21.15.17:0/2403449738 >> v2:172.21.15.31:3300/0 conn(0x7f10a8036860 0x7f10a8036ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:35.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:35.983+0000 7f10af200640 1 --2- 172.21.15.17:0/2403449738 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f10a80d8780 0x7f10a804ee70 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:35.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:35.983+0000 7f10af200640 1 -- 172.21.15.17:0/2403449738 >> 172.21.15.17:0/2403449738 conn(0x7f10a8053f10 msgr2=0x7f10a8054320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:43:35.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:35.983+0000 7f10af200640 1 -- 172.21.15.17:0/2403449738 shutdown_connections 2024-09-10T21:43:35.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:35.983+0000 7f10af200640 1 -- 172.21.15.17:0/2403449738 wait complete. 2024-09-10T21:43:35.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:35.987+0000 7f10af200640 1 Processor -- start 2024-09-10T21:43:35.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:35.987+0000 7f10af200640 1 -- start start 2024-09-10T21:43:35.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:35.987+0000 7f10af200640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f10a8036860 0x7f10a80c9570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:35.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:35.987+0000 7f10af200640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f10a804f430 0x7f10a804c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:35.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:35.987+0000 7f10af200640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f10a80d8780 0x7f10a8052970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:35.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:35.987+0000 7f10af200640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f10a80e2540 con 0x7f10a8036860 2024-09-10T21:43:35.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:35.987+0000 7f10af200640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f10a80d8b60 con 0x7f10a80d8780 2024-09-10T21:43:35.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:35.987+0000 7f10af200640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f10a80c9ab0 con 0x7f10a804f430 2024-09-10T21:43:35.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:35.991+0000 7f10ade00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f10a8036860 0x7f10a80c9570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:35.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:35.991+0000 7f10ade00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f10a8036860 0x7f10a80c9570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:46384/0 (socket says 172.21.15.17:46384) 2024-09-10T21:43:35.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:35.991+0000 7f10ade00640 1 -- 172.21.15.17:0/2849664348 learned_addr learned my addr 172.21.15.17:0/2849664348 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:43:35.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:35.991+0000 7f10ae800640 1 --2- 172.21.15.17:0/2849664348 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f10a80d8780 0x7f10a8052970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:35.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:35.991+0000 7f10ad400640 1 --2- 172.21.15.17:0/2849664348 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f10a804f430 0x7f10a804c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:35.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:35.991+0000 7f10ae800640 1 -- 172.21.15.17:0/2849664348 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f10a804f430 msgr2=0x7f10a804c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:35.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:35.991+0000 7f10ae800640 1 --2- 172.21.15.17:0/2849664348 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f10a804f430 0x7f10a804c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:35.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:35.991+0000 7f10ae800640 1 -- 172.21.15.17:0/2849664348 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f10a8036860 msgr2=0x7f10a80c9570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:35.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:35.991+0000 7f10ae800640 1 --2- 172.21.15.17:0/2849664348 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f10a8036860 0x7f10a80c9570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:35.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:35.991+0000 7f10ae800640 1 -- 172.21.15.17:0/2849664348 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10a804c980 con 0x7f10a80d8780 2024-09-10T21:43:35.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:35.991+0000 7f10ae800640 1 --2- 172.21.15.17:0/2849664348 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f10a80d8780 0x7f10a8052970 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f109c00c410 tx=0x7f109c00c8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:36.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:35.999+0000 7f10a6000640 1 -- 172.21.15.17:0/2849664348 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f109c017070 con 0x7f10a80d8780 2024-09-10T21:43:36.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:35.999+0000 7f10a6000640 1 -- 172.21.15.17:0/2849664348 <== mon.1 v2:172.21.15.31:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f109c003600 con 0x7f10a80d8780 2024-09-10T21:43:36.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:35.999+0000 7f10a6000640 1 -- 172.21.15.17:0/2849664348 <== mon.1 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f109c003a90 con 0x7f10a80d8780 2024-09-10T21:43:36.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:35.999+0000 7f10af200640 1 -- 172.21.15.17:0/2849664348 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10a804e2c0 con 0x7f10a80d8780 2024-09-10T21:43:36.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:35.999+0000 7f10af200640 1 -- 172.21.15.17:0/2849664348 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10a80d6e40 con 0x7f10a80d8780 2024-09-10T21:43:36.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:36.003+0000 7f10af200640 1 -- 172.21.15.17:0/2849664348 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10a80d6c50 con 0x7f10a80d8780 2024-09-10T21:43:36.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:36.003+0000 7f10a6000640 1 -- 172.21.15.17:0/2849664348 <== mon.1 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f109c003c30 con 0x7f10a80d8780 2024-09-10T21:43:36.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:36.007+0000 7f10a6000640 1 --2- 172.21.15.17:0/2849664348 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f1090072660 0x7f1090074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:36.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:36.007+0000 7f10a6000640 1 -- 172.21.15.17:0/2849664348 <== mon.1 v2:172.21.15.31:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f109c092ae0 con 0x7f10a80d8780 2024-09-10T21:43:36.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:36.007+0000 7f10ade00640 1 --2- 172.21.15.17:0/2849664348 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f1090072660 0x7f1090074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:36.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:36.011+0000 7f10ade00640 1 --2- 172.21.15.17:0/2849664348 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f1090072660 0x7f1090074b20 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f10b006a080 tx=0x7f10b00979a0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:36.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:36.015+0000 7f10a6000640 1 -- 172.21.15.17:0/2849664348 <== mon.1 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f109c05d8e0 con 0x7f10a80d8780 2024-09-10T21:43:36.595 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":144,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":15,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5e2f023f-d6a6-49f4-a256-f4807dce3739","modified":"2024-09-10T21:23:46.892130Z","created":"2024-09-10T21:23:46.892130Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3301","nonce":0},{"type":"v1","addr":"172.21.15.31:6790","nonce":0}]},"addr":"172.21.15.31:6790/0","public_addr":"172.21.15.31:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-10T21:43:36.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:36.583+0000 7f10af200640 1 -- 172.21.15.17:0/2849664348 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f10a8036ca0 con 0x7f10a80d8780 2024-09-10T21:43:36.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:36.587+0000 7f10a6000640 1 -- 172.21.15.17:0/2849664348 <== mon.1 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1558 (secure 0 0 0) 0x7f109c062310 con 0x7f10a80d8780 2024-09-10T21:43:36.602 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:36.599+0000 7f10af200640 1 -- 172.21.15.17:0/2849664348 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f1090072660 msgr2=0x7f1090074b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:36.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:36.599+0000 7f10af200640 1 --2- 172.21.15.17:0/2849664348 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f1090072660 0x7f1090074b20 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f10b006a080 tx=0x7f10b00979a0 comp rx=0 tx=0).stop 2024-09-10T21:43:36.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:36.599+0000 7f10af200640 1 -- 172.21.15.17:0/2849664348 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f10a80d8780 msgr2=0x7f10a8052970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:36.603 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:36.599+0000 7f10af200640 1 --2- 172.21.15.17:0/2849664348 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f10a80d8780 0x7f10a8052970 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f109c00c410 tx=0x7f109c00c8d0 comp rx=0 tx=0).stop 2024-09-10T21:43:36.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:36.603+0000 7f10af200640 1 -- 172.21.15.17:0/2849664348 shutdown_connections 2024-09-10T21:43:36.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:36.603+0000 7f10af200640 1 --2- 172.21.15.17:0/2849664348 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f1090072660 0x7f1090074b20 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:36.605 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:36.603+0000 7f10af200640 1 --2- 172.21.15.17:0/2849664348 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f10a80d8780 0x7f10a8052970 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:36.606 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:36.603+0000 7f10af200640 1 --2- 172.21.15.17:0/2849664348 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f10a804f430 0x7f10a804c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:36.606 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:36.603+0000 7f10af200640 1 --2- 172.21.15.17:0/2849664348 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f10a8036860 0x7f10a80c9570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:36.606 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:36.603+0000 7f10af200640 1 -- 172.21.15.17:0/2849664348 >> 172.21.15.17:0/2849664348 conn(0x7f10a8053f10 msgr2=0x7f10a8005d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:43:36.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:36.603+0000 7f10af200640 1 -- 172.21.15.17:0/2849664348 shutdown_connections 2024-09-10T21:43:36.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:36.607+0000 7f10af200640 1 -- 172.21.15.17:0/2849664348 wait complete. 2024-09-10T21:43:36.660 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-09-10T21:43:36.660 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-09-10T21:43:36.660 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-09-10T21:43:36.660 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-10T21:43:36.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:36.971+0000 7f1e7ce00640 1 -- 172.21.15.17:0/2611891424 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e78167c20 msgr2=0x7f1e780c6cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:36.974 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:36.971+0000 7f1e7ce00640 1 --2- 172.21.15.17:0/2611891424 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e78167c20 0x7f1e780c6cd0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f1e6800afd0 tx=0x7f1e6801fae0 comp rx=0 tx=0).stop 2024-09-10T21:43:36.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:36.971+0000 7f1e7ce00640 1 -- 172.21.15.17:0/2611891424 shutdown_connections 2024-09-10T21:43:36.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:36.971+0000 7f1e7ce00640 1 --2- 172.21.15.17:0/2611891424 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e78167c20 0x7f1e780c6cd0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:36.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:36.971+0000 7f1e7ce00640 1 -- 172.21.15.17:0/2611891424 >> 172.21.15.17:0/2611891424 conn(0x7f1e780700d0 msgr2=0x7f1e780704e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:43:36.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:36.975+0000 7f1e7ce00640 1 -- 172.21.15.17:0/2611891424 shutdown_connections 2024-09-10T21:43:36.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:36.975+0000 7f1e7ce00640 1 -- 172.21.15.17:0/2611891424 wait complete. 2024-09-10T21:43:36.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:36.979+0000 7f1e7ce00640 1 Processor -- start 2024-09-10T21:43:36.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:36.979+0000 7f1e7ce00640 1 -- start start 2024-09-10T21:43:36.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:36.979+0000 7f1e7ce00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e780c2520 0x7f1e781f57f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:36.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:36.979+0000 7f1e7ce00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1e780c7210 0x7f1e781ef110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:36.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:36.979+0000 7f1e7ce00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f1e78167c20 0x7f1e781ee1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:36.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:36.979+0000 7f1e7ce00640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f1e7806dc30 con 0x7f1e780c7210 2024-09-10T21:43:36.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:36.979+0000 7f1e7ce00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f1e780c18f0 con 0x7f1e780c2520 2024-09-10T21:43:36.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:36.979+0000 7f1e7ce00640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f1e78168000 con 0x7f1e78167c20 2024-09-10T21:43:36.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:36.983+0000 7f1e76800640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f1e78167c20 0x7f1e781ee1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:36.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:36.983+0000 7f1e76800640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f1e78167c20 0x7f1e781ee1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3301/0 says I am v2:172.21.15.17:37212/0 (socket says 172.21.15.17:37212) 2024-09-10T21:43:36.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:36.983+0000 7f1e76800640 1 -- 172.21.15.17:0/1413700507 learned_addr learned my addr 172.21.15.17:0/1413700507 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:43:36.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:36.983+0000 7f1e76800640 1 -- 172.21.15.17:0/1413700507 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e780c2520 msgr2=0x7f1e781f57f0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-10T21:43:36.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:36.983+0000 7f1e76800640 1 --2- 172.21.15.17:0/1413700507 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1e780c2520 0x7f1e781f57f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:36.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:36.983+0000 7f1e76800640 1 -- 172.21.15.17:0/1413700507 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1e780c7210 msgr2=0x7f1e781ef110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:36.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:36.983+0000 7f1e76800640 1 --2- 172.21.15.17:0/1413700507 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1e780c7210 0x7f1e781ef110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:36.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:36.983+0000 7f1e76800640 1 -- 172.21.15.17:0/1413700507 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e781ef650 con 0x7f1e78167c20 2024-09-10T21:43:36.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:36.983+0000 7f1e76800640 1 --2- 172.21.15.17:0/1413700507 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f1e78167c20 0x7f1e781ee1c0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f1e7000cb10 tx=0x7f1e7000a060 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:36.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:36.987+0000 7f1e6e000640 1 -- 172.21.15.17:0/1413700507 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e70003d90 con 0x7f1e78167c20 2024-09-10T21:43:36.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:36.987+0000 7f1e6e000640 1 -- 172.21.15.17:0/1413700507 <== mon.2 v2:172.21.15.31:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1e7000a800 con 0x7f1e78167c20 2024-09-10T21:43:36.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:36.987+0000 7f1e6e000640 1 -- 172.21.15.17:0/1413700507 <== mon.2 v2:172.21.15.31:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e7000ac90 con 0x7f1e78167c20 2024-09-10T21:43:36.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:36.987+0000 7f1e7ce00640 1 -- 172.21.15.17:0/1413700507 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e781ee700 con 0x7f1e78167c20 2024-09-10T21:43:36.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:36.987+0000 7f1e7ce00640 1 -- 172.21.15.17:0/1413700507 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e782090b0 con 0x7f1e78167c20 2024-09-10T21:43:36.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:36.991+0000 7f1e63400640 1 --2- 172.21.15.17:0/1413700507 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1e78209b90 0x7f1e782092b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:36.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:36.991+0000 7f1e63400640 1 -- 172.21.15.17:0/1413700507 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f1e781edc90 con 0x7f1e78209b90 2024-09-10T21:43:36.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:36.995+0000 7f1e75e00640 1 --2- 172.21.15.17:0/1413700507 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1e78209b90 0x7f1e782092b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:36.998 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:36.995+0000 7f1e75e00640 1 -- 172.21.15.17:0/1413700507 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1e780c6cd0 con 0x7f1e78209b90 2024-09-10T21:43:36.999 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:36.995+0000 7f1e75e00640 1 --2- 172.21.15.17:0/1413700507 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1e78209b90 0x7f1e782092b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1e7806f1d0 tx=0x7f1e68005b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:37.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:37.003+0000 7f1e6e000640 1 -- 172.21.15.17:0/1413700507 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e68004300 con 0x7f1e78209b90 2024-09-10T21:43:37.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:37.003+0000 7f1e6e000640 1 -- 172.21.15.17:0/1413700507 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f1e680044a0 con 0x7f1e78209b90 2024-09-10T21:43:37.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:37.003+0000 7f1e6e000640 1 -- 172.21.15.17:0/1413700507 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1e78209b90 msgr2=0x7f1e782092b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:37.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:37.003+0000 7f1e6e000640 1 --2- 172.21.15.17:0/1413700507 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1e78209b90 0x7f1e782092b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1e7806f1d0 tx=0x7f1e68005b10 comp rx=0 tx=0).stop 2024-09-10T21:43:37.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:37.003+0000 7f1e6e000640 1 -- 172.21.15.17:0/1413700507 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1e78209b90 msgr2=0x7f1e782092b0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:43:37.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:37.003+0000 7f1e6e000640 1 --2- 172.21.15.17:0/1413700507 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1e78209b90 0x7f1e782092b0 secure :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1e7806f1d0 tx=0x7f1e68005b10 comp rx=0 tx=0).stop 2024-09-10T21:43:37.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:37.011+0000 7f1e6e000640 1 -- 172.21.15.17:0/1413700507 <== mon.2 v2:172.21.15.31:3301/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f1e70005070 con 0x7f1e78167c20 2024-09-10T21:43:37.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:37.023+0000 7f1e6e000640 1 --2- 172.21.15.17:0/1413700507 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f1e58072130 0x7f1e580745f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:37.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:37.027+0000 7f1e6e000640 1 -- 172.21.15.17:0/1413700507 <== mon.2 v2:172.21.15.31:3301/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f1e70093550 con 0x7f1e78167c20 2024-09-10T21:43:37.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:37.027+0000 7f1e75e00640 1 --2- 172.21.15.17:0/1413700507 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f1e58072130 0x7f1e580745f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:37.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:37.035+0000 7f1e75e00640 1 --2- 172.21.15.17:0/1413700507 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f1e58072130 0x7f1e580745f0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f1e781ede10 tx=0x7f1e680b9000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:37.055 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-10T21:43:37.055 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:43:37.055 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:43:37.055 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "leader", 2024-09-10T21:43:37.055 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 144, 2024-09-10T21:43:37.055 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 76183, 2024-09-10T21:43:37.055 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-10T21:43:37.055 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-10T21:43:37.056 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-10T21:43:37.056 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-10T21:43:37.056 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:43:37.056 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 15, 2024-09-10T21:43:37.056 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:43:37.056 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-10T21:43:37.056 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-10T21:43:37.056 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:43:37.056 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:43:37.056 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:43:37.057 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:43:37.057 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:43:37.057 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:43:37.057 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:43:37.057 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:43:37.057 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:43:37.057 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:43:37.057 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:43:37.057 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:43:37.057 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-10T21:43:37.057 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-10T21:43:37.057 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:43:37.058 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:43:37.058 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:43:37.058 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:43:37.058 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:43:37.058 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:43:37.058 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:43:37.058 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:43:37.058 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:43:37.058 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:43:37.058 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:43:37.058 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:43:37.058 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:43:37.059 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-10T21:43:37.059 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-10T21:43:37.059 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-10T21:43:37.059 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-10T21:43:37.059 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-10T21:43:37.059 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "5e2f023f-d6a6-49f4-a256-f4807dce3739", 2024-09-10T21:43:37.059 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:43:37.059 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:43:37.059 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-10T21:43:37.059 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-10T21:43:37.060 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-10T21:43:37.060 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-10T21:43:37.060 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-10T21:43:37.060 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-10T21:43:37.060 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-10T21:43:37.060 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:43:37.060 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-10T21:43:37.060 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:43:37.060 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:43:37.060 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:43:37.060 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:43:37.060 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:43:37.061 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:43:37.061 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:43:37.061 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:43:37.061 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:43:37.061 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:43:37.061 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:43:37.061 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:43:37.061 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-10T21:43:37.061 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:43:37.061 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-10T21:43:37.061 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:43:37.061 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:43:37.061 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:43:37.062 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:43:37.062 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:43:37.062 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:43:37.062 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:43:37.062 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-10T21:43:37.062 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:43:37.062 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:43:37.062 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:43:37.062 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:43:37.062 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-10T21:43:37.062 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:43:37.062 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:43:37.062 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:43:37.062 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:43:37.063 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-10T21:43:37.063 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-10T21:43:37.063 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:43:37.063 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:43:37.063 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:43:37.063 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:43:37.063 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:43:37.063 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:43:37.063 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:43:37.063 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:43:37.063 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:43:37.063 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:43:37.063 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:43:37.064 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3300", 2024-09-10T21:43:37.064 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:43:37.064 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:43:37.064 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:43:37.064 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:43:37.064 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789", 2024-09-10T21:43:37.064 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:43:37.064 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:43:37.064 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:43:37.064 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:43:37.065 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789/0", 2024-09-10T21:43:37.065 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6789/0", 2024-09-10T21:43:37.065 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:43:37.065 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:43:37.065 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:43:37.065 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:43:37.065 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:43:37.065 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:43:37.065 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:43:37.065 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:43:37.065 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:43:37.065 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:43:37.065 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:43:37.066 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3301", 2024-09-10T21:43:37.066 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:43:37.066 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:43:37.066 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:43:37.066 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:43:37.066 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790", 2024-09-10T21:43:37.066 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:43:37.066 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:43:37.066 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:43:37.066 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:43:37.066 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790/0", 2024-09-10T21:43:37.066 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6790/0", 2024-09-10T21:43:37.067 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:43:37.067 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:43:37.067 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:43:37.067 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:43:37.067 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:43:37.067 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:43:37.067 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-10T21:43:37.067 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-10T21:43:37.067 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:43:37.067 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:43:37.067 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:43:37.067 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:43:37.067 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:43:37.067 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:43:37.068 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": [ 2024-09-10T21:43:37.068 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:43:37.068 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:43:37.068 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:43:37.068 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 3 2024-09-10T21:43:37.068 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:43:37.068 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:43:37.068 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-10T21:43:37.068 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:43:37.069 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:43:37.069 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:43:37.069 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:43:37.069 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:43:37.069 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:43:37.069 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-10T21:43:37.069 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:43:37.069 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:43:37.069 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:43:37.069 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:43:37.069 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:43:37.070 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:43:37.070 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": [ 2024-09-10T21:43:37.070 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:43:37.070 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:43:37.070 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:43:37.070 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:43:37.070 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:43:37.070 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:43:37.070 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:43:37.070 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-10T21:43:37.070 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-10T21:43:37.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:37.043+0000 7f1e63400640 1 --2- 172.21.15.17:0/1413700507 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1e781640f0 0x7f1e781644d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:37.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:37.043+0000 7f1e63400640 1 -- 172.21.15.17:0/1413700507 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f1e78074040 con 0x7f1e781640f0 2024-09-10T21:43:37.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:37.047+0000 7f1e75400640 1 --2- 172.21.15.17:0/1413700507 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1e781640f0 0x7f1e781644d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:37.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:37.047+0000 7f1e75400640 1 -- 172.21.15.17:0/1413700507 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f1e78070700 con 0x7f1e781640f0 2024-09-10T21:43:37.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:37.051+0000 7f1e75400640 1 --2- 172.21.15.17:0/1413700507 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1e781640f0 0x7f1e781644d0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f1e781f3860 tx=0x7f1e6400c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:37.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:37.051+0000 7f1e6e000640 1 -- 172.21.15.17:0/1413700507 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e640052d0 con 0x7f1e781640f0 2024-09-10T21:43:37.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:37.051+0000 7f1e6e000640 1 -- 172.21.15.17:0/1413700507 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5045 (secure 0 0 0) 0x7f1e640052d0 con 0x7f1e781640f0 2024-09-10T21:43:37.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:37.051+0000 7f1e6e000640 1 -- 172.21.15.17:0/1413700507 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1e781640f0 msgr2=0x7f1e781644d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:37.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:37.051+0000 7f1e6e000640 1 --2- 172.21.15.17:0/1413700507 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1e781640f0 0x7f1e781644d0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f1e781f3860 tx=0x7f1e6400c810 comp rx=0 tx=0).stop 2024-09-10T21:43:37.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:37.051+0000 7f1e6e000640 1 -- 172.21.15.17:0/1413700507 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1e781640f0 msgr2=0x7f1e781644d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:43:37.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:37.051+0000 7f1e6e000640 1 --2- 172.21.15.17:0/1413700507 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1e781640f0 0x7f1e781644d0 secure :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f1e781f3860 tx=0x7f1e6400c810 comp rx=0 tx=0).stop 2024-09-10T21:43:37.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:37.055+0000 7f1e63400640 1 -- 172.21.15.17:0/1413700507 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f1e58072130 msgr2=0x7f1e580745f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:37.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:37.055+0000 7f1e63400640 1 --2- 172.21.15.17:0/1413700507 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f1e58072130 0x7f1e580745f0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f1e781ede10 tx=0x7f1e680b9000 comp rx=0 tx=0).stop 2024-09-10T21:43:37.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:37.055+0000 7f1e76800640 1 -- 172.21.15.17:0/1413700507 reap_dead start 2024-09-10T21:43:37.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:37.055+0000 7f1e63400640 1 -- 172.21.15.17:0/1413700507 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f1e78167c20 msgr2=0x7f1e781ee1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:37.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:37.055+0000 7f1e63400640 1 --2- 172.21.15.17:0/1413700507 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f1e78167c20 0x7f1e781ee1c0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f1e7000cb10 tx=0x7f1e7000a060 comp rx=0 tx=0).stop 2024-09-10T21:43:37.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:37.055+0000 7f1e76800640 1 -- 172.21.15.17:0/1413700507 reap_dead start 2024-09-10T21:43:37.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:37.055+0000 7f1e63400640 1 -- 172.21.15.17:0/1413700507 shutdown_connections 2024-09-10T21:43:37.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:37.059+0000 7f1e63400640 1 -- 172.21.15.17:0/1413700507 >> 172.21.15.17:0/1413700507 conn(0x7f1e780700d0 msgr2=0x7f1e78166ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:43:37.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:37.059+0000 7f1e63400640 1 -- 172.21.15.17:0/1413700507 shutdown_connections 2024-09-10T21:43:37.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:37.059+0000 7f1e63400640 1 -- 172.21.15.17:0/1413700507 wait complete. 2024-09-10T21:43:37.119 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-10T21:43:37.119 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-09-10T21:43:37.120 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-10T21:43:37.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:37.467+0000 7f645d200640 1 -- 172.21.15.17:0/170735072 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f645816ab30 msgr2=0x7f64580c6d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:37.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:37.467+0000 7f645d200640 1 --2- 172.21.15.17:0/170735072 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f645816ab30 0x7f64580c6d60 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f6458076320 tx=0x7f644801c800 comp rx=0 tx=0).stop 2024-09-10T21:43:37.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:37.467+0000 7f645d200640 1 -- 172.21.15.17:0/170735072 shutdown_connections 2024-09-10T21:43:37.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:37.467+0000 7f645d200640 1 --2- 172.21.15.17:0/170735072 >> v2:172.21.15.17:3300/0 conn(0x7f645816af10 0x7f6458164610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:37.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:37.467+0000 7f645d200640 1 --2- 172.21.15.17:0/170735072 >> v2:172.21.15.31:3300/0 conn(0x7f645808c2c0 0x7f64580c72a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:37.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:37.467+0000 7f645d200640 1 --2- 172.21.15.17:0/170735072 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f645816ab30 0x7f64580c6d60 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:37.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:37.467+0000 7f645d200640 1 -- 172.21.15.17:0/170735072 >> 172.21.15.17:0/170735072 conn(0x7f645806fe60 msgr2=0x7f6458070270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:43:37.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:37.467+0000 7f645d200640 1 -- 172.21.15.17:0/170735072 shutdown_connections 2024-09-10T21:43:37.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:37.467+0000 7f645d200640 1 -- 172.21.15.17:0/170735072 wait complete. 2024-09-10T21:43:37.473 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:37.471+0000 7f645d200640 1 Processor -- start 2024-09-10T21:43:37.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:37.475+0000 7f645d200640 1 -- start start 2024-09-10T21:43:37.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:37.475+0000 7f645d200640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f645808c2c0 0x7f64580cb8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:37.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:37.475+0000 7f645d200640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f645816ab30 0x7f6458074320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:37.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:37.475+0000 7f645d200640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f645816af10 0x7f6458161770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:37.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:37.475+0000 7f645d200640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f6458070980 con 0x7f645816ab30 2024-09-10T21:43:37.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:37.475+0000 7f645d200640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f6458070d10 con 0x7f645816af10 2024-09-10T21:43:37.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:37.475+0000 7f645d200640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f6458070b00 con 0x7f645808c2c0 2024-09-10T21:43:37.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:37.475+0000 7f6457400640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f645808c2c0 0x7f64580cb8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:37.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:37.475+0000 7f6457400640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f645808c2c0 0x7f64580cb8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3301/0 says I am v2:172.21.15.17:37220/0 (socket says 172.21.15.17:37220) 2024-09-10T21:43:37.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:37.475+0000 7f6457400640 1 -- 172.21.15.17:0/570549915 learned_addr learned my addr 172.21.15.17:0/570549915 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:43:37.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:37.475+0000 7f644ea00640 1 --2- 172.21.15.17:0/570549915 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f645816ab30 0x7f6458074320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:37.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:37.475+0000 7f6457e00640 1 --2- 172.21.15.17:0/570549915 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f645816af10 0x7f6458161770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:37.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:37.475+0000 7f6457400640 1 -- 172.21.15.17:0/570549915 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f645816af10 msgr2=0x7f6458161770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:37.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:37.475+0000 7f6457400640 1 --2- 172.21.15.17:0/570549915 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f645816af10 0x7f6458161770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:37.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:37.479+0000 7f6457400640 1 -- 172.21.15.17:0/570549915 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f645816ab30 msgr2=0x7f6458074320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:37.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:37.479+0000 7f6457400640 1 --2- 172.21.15.17:0/570549915 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f645816ab30 0x7f6458074320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:37.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:37.479+0000 7f6457400640 1 -- 172.21.15.17:0/570549915 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6458074860 con 0x7f645808c2c0 2024-09-10T21:43:37.481 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:37.479+0000 7f644ea00640 1 --2- 172.21.15.17:0/570549915 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f645816ab30 0x7f6458074320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:43:37.481 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:37.479+0000 7f6457400640 1 --2- 172.21.15.17:0/570549915 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f645808c2c0 0x7f64580cb8e0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f6448009790 tx=0x7f6448005040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:37.481 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:37.479+0000 7f644fe00640 1 -- 172.21.15.17:0/570549915 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64480b2040 con 0x7f645808c2c0 2024-09-10T21:43:37.481 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:37.479+0000 7f644fe00640 1 -- 172.21.15.17:0/570549915 <== mon.2 v2:172.21.15.31:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f64480055e0 con 0x7f645808c2c0 2024-09-10T21:43:37.481 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:37.479+0000 7f644fe00640 1 -- 172.21.15.17:0/570549915 <== mon.2 v2:172.21.15.31:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6448004270 con 0x7f645808c2c0 2024-09-10T21:43:37.484 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:37.479+0000 7f645d200640 1 -- 172.21.15.17:0/570549915 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6458074df0 con 0x7f645808c2c0 2024-09-10T21:43:37.484 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:37.479+0000 7f645d200640 1 -- 172.21.15.17:0/570549915 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f6458075090 con 0x7f645808c2c0 2024-09-10T21:43:37.484 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:37.483+0000 7f644cc00640 1 --2- 172.21.15.17:0/570549915 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f64581690e0 0x7f64581694c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:37.484 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:37.483+0000 7f644cc00640 1 -- 172.21.15.17:0/570549915 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f64580c8360 con 0x7f64581690e0 2024-09-10T21:43:37.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:37.483+0000 7f644ea00640 1 --2- 172.21.15.17:0/570549915 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f64581690e0 0x7f64581694c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:37.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:37.483+0000 7f644ea00640 1 -- 172.21.15.17:0/570549915 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6458164050 con 0x7f64581690e0 2024-09-10T21:43:37.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:37.483+0000 7f644ea00640 1 --2- 172.21.15.17:0/570549915 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f64581690e0 0x7f64581694c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f64580c21f0 tx=0x7f6440008cf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:37.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:37.483+0000 7f644fe00640 1 -- 172.21.15.17:0/570549915 <== mon.2 v2:172.21.15.31:3301/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f64480bf060 con 0x7f645808c2c0 2024-09-10T21:43:37.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:37.487+0000 7f644fe00640 1 --2- 172.21.15.17:0/570549915 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f6434072630 0x7f6434074af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:37.490 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:37.487+0000 7f644fe00640 1 -- 172.21.15.17:0/570549915 <== mon.2 v2:172.21.15.31:3301/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f644812ca40 con 0x7f645808c2c0 2024-09-10T21:43:37.490 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:37.487+0000 7f644fe00640 1 -- 172.21.15.17:0/570549915 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6440007c20 con 0x7f64581690e0 2024-09-10T21:43:37.490 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:37.487+0000 7f6457e00640 1 --2- 172.21.15.17:0/570549915 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f6434072630 0x7f6434074af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:37.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:37.487+0000 7f644fe00640 1 -- 172.21.15.17:0/570549915 <== mon.1 v2:172.21.15.31:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f6440019070 con 0x7f64581690e0 2024-09-10T21:43:37.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:37.487+0000 7f644fe00640 1 -- 172.21.15.17:0/570549915 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f64581690e0 msgr2=0x7f64581694c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:37.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:37.487+0000 7f644fe00640 1 --2- 172.21.15.17:0/570549915 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f64581690e0 0x7f64581694c0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f64580c21f0 tx=0x7f6440008cf0 comp rx=0 tx=0).stop 2024-09-10T21:43:37.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:37.487+0000 7f644fe00640 1 -- 172.21.15.17:0/570549915 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f64581690e0 msgr2=0x7f64581694c0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:43:37.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:37.487+0000 7f644fe00640 1 --2- 172.21.15.17:0/570549915 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f64581690e0 0x7f64581694c0 secure :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f64580c21f0 tx=0x7f6440008cf0 comp rx=0 tx=0).stop 2024-09-10T21:43:37.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:37.487+0000 7f6457e00640 1 --2- 172.21.15.17:0/570549915 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f6434072630 0x7f6434074af0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f6450009ec0 tx=0x7f6450010040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:37.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:37.503+0000 7f644cc00640 1 --2- 172.21.15.17:0/570549915 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f64580c9d10 0x7f64580ca0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:37.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:37.503+0000 7f644cc00640 1 -- 172.21.15.17:0/570549915 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f6458163d70 con 0x7f64580c9d10 2024-09-10T21:43:37.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:37.511+0000 7f644ea00640 1 --2- 172.21.15.17:0/570549915 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f64580c9d10 0x7f64580ca0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:37.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:37.511+0000 7f644ea00640 1 -- 172.21.15.17:0/570549915 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f64580c1c40 con 0x7f64580c9d10 2024-09-10T21:43:37.522 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-10T21:43:37.522 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:43:37.523 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:43:37.523 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-10T21:43:37.523 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 144, 2024-09-10T21:43:37.523 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 228944, 2024-09-10T21:43:37.523 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-10T21:43:37.523 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-10T21:43:37.523 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-10T21:43:37.523 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-10T21:43:37.523 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:43:37.523 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 16, 2024-09-10T21:43:37.523 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:43:37.523 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-10T21:43:37.524 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-10T21:43:37.524 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:43:37.524 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:43:37.524 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:43:37.524 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:43:37.524 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:43:37.524 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:43:37.524 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:43:37.524 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:43:37.524 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:43:37.524 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:43:37.524 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:43:37.524 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:43:37.525 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-10T21:43:37.525 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-10T21:43:37.525 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:43:37.525 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:43:37.525 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:43:37.525 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:43:37.525 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:43:37.525 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:43:37.525 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:43:37.525 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:43:37.525 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:43:37.525 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:43:37.525 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:43:37.526 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:43:37.526 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:43:37.526 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-10T21:43:37.526 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-10T21:43:37.526 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-10T21:43:37.526 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-10T21:43:37.526 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-10T21:43:37.526 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "5e2f023f-d6a6-49f4-a256-f4807dce3739", 2024-09-10T21:43:37.526 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:43:37.526 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:43:37.526 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-10T21:43:37.527 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-10T21:43:37.527 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-10T21:43:37.527 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-10T21:43:37.527 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-10T21:43:37.527 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-10T21:43:37.527 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-10T21:43:37.527 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:43:37.527 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-10T21:43:37.527 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:43:37.527 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:43:37.527 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:43:37.527 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:43:37.527 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:43:37.528 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:43:37.528 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:43:37.528 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:43:37.528 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:43:37.528 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:43:37.528 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:43:37.528 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:43:37.528 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-10T21:43:37.528 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:43:37.528 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-10T21:43:37.528 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:43:37.528 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:43:37.528 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:43:37.529 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:43:37.529 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:43:37.529 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:43:37.529 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:43:37.529 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-10T21:43:37.529 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:43:37.529 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:43:37.529 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:43:37.529 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:43:37.529 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-10T21:43:37.529 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:43:37.529 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:43:37.529 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:43:37.530 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:43:37.530 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-10T21:43:37.530 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-10T21:43:37.530 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:43:37.530 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:43:37.530 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:43:37.530 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:43:37.530 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:43:37.530 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:43:37.530 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:43:37.530 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:43:37.530 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:43:37.530 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:43:37.531 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:43:37.531 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3300", 2024-09-10T21:43:37.531 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:43:37.531 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:43:37.531 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:43:37.531 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:43:37.531 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789", 2024-09-10T21:43:37.531 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:43:37.531 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:43:37.531 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:43:37.531 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:43:37.531 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789/0", 2024-09-10T21:43:37.531 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6789/0", 2024-09-10T21:43:37.531 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:43:37.532 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:43:37.532 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:43:37.532 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:43:37.532 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:43:37.532 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:43:37.532 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:43:37.532 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:43:37.532 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:43:37.532 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:43:37.532 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:43:37.532 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3301", 2024-09-10T21:43:37.532 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:43:37.532 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:43:37.533 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:43:37.533 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:43:37.533 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790", 2024-09-10T21:43:37.533 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:43:37.533 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:43:37.533 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:43:37.533 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:43:37.533 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790/0", 2024-09-10T21:43:37.533 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6790/0", 2024-09-10T21:43:37.533 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:43:37.533 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:43:37.533 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:43:37.533 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:43:37.533 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:43:37.534 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:43:37.534 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-10T21:43:37.534 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-10T21:43:37.534 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:43:37.534 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:43:37.534 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:43:37.534 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:43:37.534 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:43:37.534 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:43:37.534 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": [ 2024-09-10T21:43:37.534 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:43:37.534 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:43:37.534 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:43:37.535 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 3 2024-09-10T21:43:37.535 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:43:37.535 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:43:37.535 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-10T21:43:37.535 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:43:37.535 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:43:37.535 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:43:37.535 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 6 2024-09-10T21:43:37.535 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:43:37.535 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:43:37.535 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-10T21:43:37.535 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:43:37.535 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:43:37.535 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:43:37.536 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 6 2024-09-10T21:43:37.536 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:43:37.536 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:43:37.536 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": [ 2024-09-10T21:43:37.536 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:43:37.536 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:43:37.536 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:43:37.536 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:43:37.536 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:43:37.536 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:43:37.536 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:43:37.536 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-10T21:43:37.536 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-10T21:43:37.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:37.515+0000 7f644ea00640 1 --2- 172.21.15.17:0/570549915 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f64580c9d10 0x7f64580ca0f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f6440016040 tx=0x7f64400046d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:37.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:37.519+0000 7f644fe00640 1 -- 172.21.15.17:0/570549915 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6440004220 con 0x7f64580c9d10 2024-09-10T21:43:37.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:37.519+0000 7f644fe00640 1 -- 172.21.15.17:0/570549915 <== mon.1 v2:172.21.15.31:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5044 (secure 0 0 0) 0x7f64400043c0 con 0x7f64580c9d10 2024-09-10T21:43:37.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:37.519+0000 7f644fe00640 1 -- 172.21.15.17:0/570549915 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f64580c9d10 msgr2=0x7f64580ca0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:37.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:37.519+0000 7f644fe00640 1 --2- 172.21.15.17:0/570549915 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f64580c9d10 0x7f64580ca0f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f6440016040 tx=0x7f64400046d0 comp rx=0 tx=0).stop 2024-09-10T21:43:37.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:37.519+0000 7f644fe00640 1 -- 172.21.15.17:0/570549915 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f64580c9d10 msgr2=0x7f64580ca0f0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:43:37.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:37.519+0000 7f644fe00640 1 --2- 172.21.15.17:0/570549915 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f64580c9d10 0x7f64580ca0f0 secure :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f6440016040 tx=0x7f64400046d0 comp rx=0 tx=0).stop 2024-09-10T21:43:37.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:37.519+0000 7f644cc00640 1 -- 172.21.15.17:0/570549915 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f6434072630 msgr2=0x7f6434074af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:37.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:37.519+0000 7f644cc00640 1 --2- 172.21.15.17:0/570549915 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f6434072630 0x7f6434074af0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f6450009ec0 tx=0x7f6450010040 comp rx=0 tx=0).stop 2024-09-10T21:43:37.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:37.519+0000 7f644cc00640 1 -- 172.21.15.17:0/570549915 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f645808c2c0 msgr2=0x7f64580cb8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:37.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:37.519+0000 7f644cc00640 1 --2- 172.21.15.17:0/570549915 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f645808c2c0 0x7f64580cb8e0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f6448009790 tx=0x7f6448005040 comp rx=0 tx=0).stop 2024-09-10T21:43:37.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:37.519+0000 7f6457e00640 1 -- 172.21.15.17:0/570549915 reap_dead start 2024-09-10T21:43:37.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:37.519+0000 7f6457e00640 1 -- 172.21.15.17:0/570549915 reap_dead start 2024-09-10T21:43:37.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:37.523+0000 7f644cc00640 1 -- 172.21.15.17:0/570549915 shutdown_connections 2024-09-10T21:43:37.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:37.523+0000 7f644cc00640 1 -- 172.21.15.17:0/570549915 >> 172.21.15.17:0/570549915 conn(0x7f645806fe60 msgr2=0x7f64581643e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:43:37.538 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:37.523+0000 7f644cc00640 1 -- 172.21.15.17:0/570549915 shutdown_connections 2024-09-10T21:43:37.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:37.523+0000 7f644cc00640 1 -- 172.21.15.17:0/570549915 wait complete. 2024-09-10T21:43:37.573 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-10T21:43:37.573 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-09-10T21:43:37.573 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-10T21:43:37.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:37.895+0000 7f605d000640 1 -- 172.21.15.17:0/2033511915 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f605806dd10 msgr2=0x7f605806e180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:37.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:37.895+0000 7f605d000640 1 --2- 172.21.15.17:0/2033511915 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f605806dd10 0x7f605806e180 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f6058073750 tx=0x7f604401f030 comp rx=0 tx=0).stop 2024-09-10T21:43:37.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:37.899+0000 7f605d000640 1 -- 172.21.15.17:0/2033511915 shutdown_connections 2024-09-10T21:43:37.902 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:37.899+0000 7f605d000640 1 --2- 172.21.15.17:0/2033511915 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f605806dd10 0x7f605806e180 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:37.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:37.899+0000 7f605d000640 1 --2- 172.21.15.17:0/2033511915 >> v2:172.21.15.17:3300/0 conn(0x7f6058153d40 0x7f60580c8010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:37.903 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:37.899+0000 7f605d000640 1 -- 172.21.15.17:0/2033511915 >> 172.21.15.17:0/2033511915 conn(0x7f60580700d0 msgr2=0x7f60580704e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:43:37.909 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:37.907+0000 7f605d000640 1 -- 172.21.15.17:0/2033511915 shutdown_connections 2024-09-10T21:43:37.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:37.907+0000 7f605d000640 1 -- 172.21.15.17:0/2033511915 wait complete. 2024-09-10T21:43:37.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:37.911+0000 7f605d000640 1 Processor -- start 2024-09-10T21:43:37.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:37.911+0000 7f605d000640 1 -- start start 2024-09-10T21:43:37.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:37.915+0000 7f605d000640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f605806dd10 0x7f60581f0050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:37.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:37.915+0000 7f605d000640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f60580c0fd0 0x7f6058144050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:37.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:37.915+0000 7f605d000640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6058153d40 0x7f6058149910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:37.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:37.915+0000 7f605d000640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f60580c92c0 con 0x7f60580c0fd0 2024-09-10T21:43:37.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:37.915+0000 7f605d000640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f60580c8fe0 con 0x7f605806dd10 2024-09-10T21:43:37.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:37.915+0000 7f605d000640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f60580c95a0 con 0x7f6058153d40 2024-09-10T21:43:37.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:37.915+0000 7f6056800640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6058153d40 0x7f6058149910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:37.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:37.915+0000 7f6056800640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6058153d40 0x7f6058149910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3301/0 says I am v2:172.21.15.17:37224/0 (socket says 172.21.15.17:37224) 2024-09-10T21:43:37.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:37.915+0000 7f6056800640 1 -- 172.21.15.17:0/927233546 learned_addr learned my addr 172.21.15.17:0/927233546 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:43:37.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:37.915+0000 7f6055e00640 1 --2- 172.21.15.17:0/927233546 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f605806dd10 0x7f60581f0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:37.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:37.915+0000 7f6056800640 1 -- 172.21.15.17:0/927233546 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f605806dd10 msgr2=0x7f60581f0050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:37.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:37.915+0000 7f6056800640 1 --2- 172.21.15.17:0/927233546 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f605806dd10 0x7f60581f0050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:37.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:37.915+0000 7f6056800640 1 -- 172.21.15.17:0/927233546 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f60580c0fd0 msgr2=0x7f6058144050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:37.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:37.915+0000 7f6056800640 1 --2- 172.21.15.17:0/927233546 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f60580c0fd0 0x7f6058144050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:37.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:37.915+0000 7f6056800640 1 -- 172.21.15.17:0/927233546 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60581531a0 con 0x7f6058153d40 2024-09-10T21:43:37.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:37.919+0000 7f6056800640 1 --2- 172.21.15.17:0/927233546 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6058153d40 0x7f6058149910 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f604401cc60 tx=0x7f6044005dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:37.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:37.919+0000 7f604e000640 1 -- 172.21.15.17:0/927233546 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60440be070 con 0x7f6058153d40 2024-09-10T21:43:37.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:37.919+0000 7f604e000640 1 -- 172.21.15.17:0/927233546 <== mon.2 v2:172.21.15.31:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6044005320 con 0x7f6058153d40 2024-09-10T21:43:37.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:37.919+0000 7f605d000640 1 -- 172.21.15.17:0/927233546 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6058143820 con 0x7f6058153d40 2024-09-10T21:43:37.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:37.919+0000 7f605d000640 1 -- 172.21.15.17:0/927233546 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f6058143540 con 0x7f6058153d40 2024-09-10T21:43:37.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:37.919+0000 7f604e000640 1 -- 172.21.15.17:0/927233546 <== mon.2 v2:172.21.15.31:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60440b5040 con 0x7f6058153d40 2024-09-10T21:43:37.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:37.923+0000 7f605d000640 1 --2- 172.21.15.17:0/927233546 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6030002a80 0x7f6030004f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:37.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:37.923+0000 7f605d000640 1 -- 172.21.15.17:0/927233546 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f6030007ed0 con 0x7f6030002a80 2024-09-10T21:43:37.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:37.927+0000 7f604e000640 1 -- 172.21.15.17:0/927233546 <== mon.2 v2:172.21.15.31:3301/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f60440b8030 con 0x7f6058153d40 2024-09-10T21:43:37.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:37.927+0000 7f6055e00640 1 --2- 172.21.15.17:0/927233546 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6030002a80 0x7f6030004f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:37.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:37.927+0000 7f6055e00640 1 -- 172.21.15.17:0/927233546 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f60300080d0 con 0x7f6030002a80 2024-09-10T21:43:37.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:37.931+0000 7f604e000640 1 --2- 172.21.15.17:0/927233546 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f603c072800 0x7f603c074cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:37.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:37.931+0000 7f604e000640 1 -- 172.21.15.17:0/927233546 <== mon.2 v2:172.21.15.31:3301/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f6044130400 con 0x7f6058153d40 2024-09-10T21:43:37.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:37.931+0000 7f6055400640 1 --2- 172.21.15.17:0/927233546 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f603c072800 0x7f603c074cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:37.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:37.935+0000 7f6055400640 1 --2- 172.21.15.17:0/927233546 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f603c072800 0x7f603c074cc0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f6058142ec0 tx=0x7f6048007a00 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:37.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:37.935+0000 7f6055e00640 1 --2- 172.21.15.17:0/927233546 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6030002a80 0x7f6030004f40 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f6050010b20 tx=0x7f605000e350 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:37.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:37.935+0000 7f604e000640 1 -- 172.21.15.17:0/927233546 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6050003980 con 0x7f6030002a80 2024-09-10T21:43:37.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:37.935+0000 7f604e000640 1 -- 172.21.15.17:0/927233546 <== mon.2 v2:172.21.15.31:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f6050003980 con 0x7f6030002a80 2024-09-10T21:43:37.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:37.939+0000 7f604e000640 1 -- 172.21.15.17:0/927233546 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6030002a80 msgr2=0x7f6030004f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:37.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:37.939+0000 7f604e000640 1 --2- 172.21.15.17:0/927233546 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6030002a80 0x7f6030004f40 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f6050010b20 tx=0x7f605000e350 comp rx=0 tx=0).stop 2024-09-10T21:43:37.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:37.939+0000 7f604e000640 1 -- 172.21.15.17:0/927233546 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6030002a80 msgr2=0x7f6030004f40 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:43:37.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:37.939+0000 7f604e000640 1 --2- 172.21.15.17:0/927233546 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6030002a80 0x7f6030004f40 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:37.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:37.951+0000 7f605d000640 1 --2- 172.21.15.17:0/927233546 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6030006f40 0x7f60300073d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:37.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:37.951+0000 7f605d000640 1 -- 172.21.15.17:0/927233546 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f603000a4b0 con 0x7f6030006f40 2024-09-10T21:43:37.955 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:37.951+0000 7f6055e00640 1 --2- 172.21.15.17:0/927233546 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6030006f40 0x7f60300073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:37.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:37.955+0000 7f6055e00640 1 -- 172.21.15.17:0/927233546 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f6030007910 con 0x7f6030006f40 2024-09-10T21:43:37.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:37.963+0000 7f6055e00640 1 --2- 172.21.15.17:0/927233546 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6030006f40 0x7f60300073d0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f6050010b20 tx=0x7f6050004a50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:37.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:37.963+0000 7f604e000640 1 -- 172.21.15.17:0/927233546 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6050004800 con 0x7f6030006f40 2024-09-10T21:43:37.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:37.963+0000 7f604e000640 1 -- 172.21.15.17:0/927233546 <== mon.2 v2:172.21.15.31:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4709 (secure 0 0 0) 0x7f6050004800 con 0x7f6030006f40 2024-09-10T21:43:37.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:37.963+0000 7f604e000640 1 -- 172.21.15.17:0/927233546 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6030006f40 msgr2=0x7f60300073d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:37.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:37.963+0000 7f604e000640 1 --2- 172.21.15.17:0/927233546 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6030006f40 0x7f60300073d0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f6050010b20 tx=0x7f6050004a50 comp rx=0 tx=0).stop 2024-09-10T21:43:37.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:37.967+0000 7f604e000640 1 -- 172.21.15.17:0/927233546 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6030006f40 msgr2=0x7f60300073d0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:43:37.970 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:37.967+0000 7f604e000640 1 --2- 172.21.15.17:0/927233546 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6030006f40 0x7f60300073d0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:37.970 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-10T21:43:37.970 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:43:37.970 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:43:37.970 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-10T21:43:37.970 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 144, 2024-09-10T21:43:37.971 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 20091, 2024-09-10T21:43:37.971 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-10T21:43:37.971 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-10T21:43:37.971 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-10T21:43:37.971 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-10T21:43:37.971 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:43:37.971 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 16, 2024-09-10T21:43:37.971 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:43:37.971 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-10T21:43:37.971 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-10T21:43:37.971 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:43:37.971 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:43:37.971 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:43:37.971 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:43:37.972 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:43:37.972 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:43:37.972 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:43:37.972 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:43:37.972 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:43:37.972 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:43:37.972 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:43:37.972 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:43:37.972 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-10T21:43:37.972 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-10T21:43:37.972 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:43:37.972 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:43:37.972 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:43:37.973 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:43:37.973 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:43:37.973 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:43:37.973 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:43:37.973 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:43:37.973 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:43:37.973 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:43:37.973 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:43:37.973 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:43:37.973 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:43:37.973 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-10T21:43:37.973 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-10T21:43:37.973 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-10T21:43:37.974 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-10T21:43:37.974 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-10T21:43:37.974 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "5e2f023f-d6a6-49f4-a256-f4807dce3739", 2024-09-10T21:43:37.974 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:43:37.974 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:43:37.974 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-10T21:43:37.974 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-10T21:43:37.974 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-10T21:43:37.974 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-10T21:43:37.974 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-10T21:43:37.975 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-10T21:43:37.975 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-10T21:43:37.975 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:43:37.975 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-10T21:43:37.975 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:43:37.975 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:43:37.975 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:43:37.975 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:43:37.975 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:43:37.975 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:43:37.975 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:43:37.975 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:43:37.975 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:43:37.975 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:43:37.976 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:43:37.976 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:43:37.976 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-10T21:43:37.976 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:43:37.976 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-10T21:43:37.976 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:43:37.976 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:43:37.976 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:43:37.976 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:43:37.976 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:43:37.977 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:43:37.977 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:43:37.977 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-10T21:43:37.977 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:43:37.977 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:43:37.977 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:43:37.977 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:43:37.977 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-10T21:43:37.977 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:43:37.977 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:43:37.977 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:43:37.977 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:43:37.977 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-10T21:43:37.978 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-10T21:43:37.978 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:43:37.978 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:43:37.978 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:43:37.978 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:43:37.978 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:43:37.978 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:43:37.978 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:43:37.978 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:43:37.978 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:43:37.978 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:43:37.978 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:43:37.979 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3300", 2024-09-10T21:43:37.979 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:43:37.979 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:43:37.979 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:43:37.979 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:43:37.979 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789", 2024-09-10T21:43:37.979 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:43:37.979 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:43:37.979 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:43:37.979 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:43:37.979 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789/0", 2024-09-10T21:43:37.979 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6789/0", 2024-09-10T21:43:37.979 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:43:37.979 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:43:37.980 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:43:37.980 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:43:37.980 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:43:37.980 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:43:37.980 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:43:37.980 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:43:37.980 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:43:37.980 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:43:37.980 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:43:37.980 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3301", 2024-09-10T21:43:37.980 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:43:37.980 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:43:37.980 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:43:37.981 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:43:37.981 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790", 2024-09-10T21:43:37.981 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:43:37.981 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:43:37.981 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:43:37.981 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:43:37.981 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790/0", 2024-09-10T21:43:37.981 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6790/0", 2024-09-10T21:43:37.981 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:43:37.981 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:43:37.981 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:43:37.981 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:43:37.981 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:43:37.982 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:43:37.982 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-10T21:43:37.982 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-10T21:43:37.982 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:43:37.982 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:43:37.982 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:43:37.982 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:43:37.982 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:43:37.982 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:43:37.982 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-10T21:43:37.982 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:43:37.982 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:43:37.982 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:43:37.983 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:43:37.983 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:43:37.983 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:43:37.983 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-10T21:43:37.983 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:43:37.983 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:43:37.983 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:43:37.983 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 2 2024-09-10T21:43:37.983 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:43:37.983 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:43:37.983 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:43:37.983 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-10T21:43:37.983 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-10T21:43:37.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:37.983+0000 7f605d000640 1 -- 172.21.15.17:0/927233546 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f603c072800 msgr2=0x7f603c074cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:37.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:37.983+0000 7f605d000640 1 --2- 172.21.15.17:0/927233546 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f603c072800 0x7f603c074cc0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f6058142ec0 tx=0x7f6048007a00 comp rx=0 tx=0).stop 2024-09-10T21:43:37.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:37.983+0000 7f605d000640 1 -- 172.21.15.17:0/927233546 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6058153d40 msgr2=0x7f6058149910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:37.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:37.983+0000 7f605d000640 1 --2- 172.21.15.17:0/927233546 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6058153d40 0x7f6058149910 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f604401cc60 tx=0x7f6044005dc0 comp rx=0 tx=0).stop 2024-09-10T21:43:37.985 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:37.983+0000 7f6056800640 1 -- 172.21.15.17:0/927233546 reap_dead start 2024-09-10T21:43:37.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:37.983+0000 7f6056800640 1 -- 172.21.15.17:0/927233546 reap_dead start 2024-09-10T21:43:37.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:37.983+0000 7f605d000640 1 -- 172.21.15.17:0/927233546 shutdown_connections 2024-09-10T21:43:37.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:37.983+0000 7f605d000640 1 -- 172.21.15.17:0/927233546 >> 172.21.15.17:0/927233546 conn(0x7f60580700d0 msgr2=0x7f60580cb460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:43:37.990 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:37.987+0000 7f605d000640 1 -- 172.21.15.17:0/927233546 shutdown_connections 2024-09-10T21:43:37.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:37.987+0000 7f605d000640 1 -- 172.21.15.17:0/927233546 wait complete. 2024-09-10T21:43:38.055 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-10T21:43:38.056 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['b'] 2024-09-10T21:43:38.056 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2024-09-10T21:43:38.056 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.b 2024-09-10T21:43:38.056 INFO:tasks.mon_thrash.mon_thrasher:killing mon.b 2024-09-10T21:43:38.056 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-10T21:43:38.056 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:43:38.104 INFO:tasks.ceph.mon.b:Stopped 2024-09-10T21:43:38.105 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2024-09-10T21:43:38.105 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-10T21:43:38.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:38.475+0000 7fdf70e00640 1 -- 172.21.15.17:0/3535047018 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fdf6c0c8290 msgr2=0x7fdf6c06fe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:38.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:38.475+0000 7fdf70e00640 1 --2- 172.21.15.17:0/3535047018 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fdf6c0c8290 0x7fdf6c06fe80 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fdf5800aa10 tx=0x7fdf58020440 comp rx=0 tx=0).stop 2024-09-10T21:43:38.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:38.483+0000 7fdf70e00640 1 -- 172.21.15.17:0/3535047018 shutdown_connections 2024-09-10T21:43:38.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:38.483+0000 7fdf70e00640 1 --2- 172.21.15.17:0/3535047018 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fdf6c0c8290 0x7fdf6c06fe80 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:38.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:38.483+0000 7fdf70e00640 1 --2- 172.21.15.17:0/3535047018 >> v2:172.21.15.31:3300/0 conn(0x7fdf6c150b50 0x7fdf6c072b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:38.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:38.483+0000 7fdf70e00640 1 -- 172.21.15.17:0/3535047018 >> 172.21.15.17:0/3535047018 conn(0x7fdf6c06cdf0 msgr2=0x7fdf6c06d1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:43:38.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:38.487+0000 7fdf70e00640 1 -- 172.21.15.17:0/3535047018 shutdown_connections 2024-09-10T21:43:38.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:38.495+0000 7fdf70e00640 1 -- 172.21.15.17:0/3535047018 wait complete. 2024-09-10T21:43:38.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:38.495+0000 7fdf70e00640 1 Processor -- start 2024-09-10T21:43:38.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:38.507+0000 7fdf70e00640 1 -- start start 2024-09-10T21:43:38.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:38.507+0000 7fdf70e00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fdf6c0c7bf0 0x7fdf6c14e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:38.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:38.507+0000 7fdf70e00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdf6c0c8290 0x7fdf6c14e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:38.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:38.507+0000 7fdf70e00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdf6c150b50 0x7fdf6c151800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:38.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:38.507+0000 7fdf70e00640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fdf6c150f30 con 0x7fdf6c0c8290 2024-09-10T21:43:38.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:38.507+0000 7fdf70e00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fdf6c157630 con 0x7fdf6c150b50 2024-09-10T21:43:38.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:38.507+0000 7fdf70e00640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7fdf6c0c98f0 con 0x7fdf6c0c7bf0 2024-09-10T21:43:38.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:38.515+0000 7fdf6b400640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fdf6c0c7bf0 0x7fdf6c14e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:38.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:38.515+0000 7fdf6be00640 1 -- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdf6c150b50 msgr2=0x7fdf6c151800 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:3300/0 2024-09-10T21:43:38.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:38.515+0000 7fdf6be00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdf6c150b50 0x7fdf6c151800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:43:38.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:38.515+0000 7fdf6b400640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fdf6c0c7bf0 0x7fdf6c14e0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3301/0 says I am v2:172.21.15.17:37302/0 (socket says 172.21.15.17:37302) 2024-09-10T21:43:38.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:38.515+0000 7fdf6b400640 1 -- 172.21.15.17:0/2518174069 learned_addr learned my addr 172.21.15.17:0/2518174069 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:43:38.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:38.515+0000 7fdf6b400640 1 -- 172.21.15.17:0/2518174069 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdf6c150b50 msgr2=0x7fdf6c151800 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:43:38.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:38.515+0000 7fdf6b400640 1 --2- 172.21.15.17:0/2518174069 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdf6c150b50 0x7fdf6c151800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:38.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:38.515+0000 7fdf6b400640 1 -- 172.21.15.17:0/2518174069 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdf6c0c8290 msgr2=0x7fdf6c14e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:38.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:38.515+0000 7fdf6b400640 1 --2- 172.21.15.17:0/2518174069 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdf6c0c8290 0x7fdf6c14e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:38.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:38.515+0000 7fdf6b400640 1 -- 172.21.15.17:0/2518174069 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf6c152cd0 con 0x7fdf6c0c7bf0 2024-09-10T21:43:38.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:38.523+0000 7fdf6b400640 1 --2- 172.21.15.17:0/2518174069 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fdf6c0c7bf0 0x7fdf6c14e0f0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fdf6400ba80 tx=0x7fdf6400bf40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:38.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:38.527+0000 7fdf63400640 1 -- 172.21.15.17:0/2518174069 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf64019070 con 0x7fdf6c0c7bf0 2024-09-10T21:43:38.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:38.527+0000 7fdf63400640 1 -- 172.21.15.17:0/2518174069 <== mon.2 v2:172.21.15.31:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdf6400cda0 con 0x7fdf6c0c7bf0 2024-09-10T21:43:38.529 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:38.527+0000 7fdf63400640 1 -- 172.21.15.17:0/2518174069 <== mon.2 v2:172.21.15.31:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf640146a0 con 0x7fdf6c0c7bf0 2024-09-10T21:43:38.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:38.527+0000 7fdf70e00640 1 -- 172.21.15.17:0/2518174069 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf6c0cc210 con 0x7fdf6c0c7bf0 2024-09-10T21:43:38.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:38.527+0000 7fdf70e00640 1 -- 172.21.15.17:0/2518174069 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf6c152710 con 0x7fdf6c0c7bf0 2024-09-10T21:43:38.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:38.531+0000 7fdf63400640 1 -- 172.21.15.17:0/2518174069 <== mon.2 v2:172.21.15.31:3301/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fdf6401d3f0 con 0x7fdf6c0c7bf0 2024-09-10T21:43:38.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:38.531+0000 7fdf70e00640 1 -- 172.21.15.17:0/2518174069 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf6c072b60 con 0x7fdf6c0c7bf0 2024-09-10T21:43:38.534 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:38.531+0000 7fdf63400640 1 --2- 172.21.15.17:0/2518174069 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fdf4c0725e0 0x7fdf4c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:38.535 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:38.531+0000 7fdf63400640 1 -- 172.21.15.17:0/2518174069 <== mon.2 v2:172.21.15.31:3301/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fdf64093400 con 0x7fdf6c0c7bf0 2024-09-10T21:43:38.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:38.539+0000 7fdf6aa00640 1 --2- 172.21.15.17:0/2518174069 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fdf4c0725e0 0x7fdf4c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:38.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:38.539+0000 7fdf63400640 1 -- 172.21.15.17:0/2518174069 <== mon.2 v2:172.21.15.31:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fdf6405e6c0 con 0x7fdf6c0c7bf0 2024-09-10T21:43:38.547 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:38.543+0000 7fdf6aa00640 1 --2- 172.21.15.17:0/2518174069 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fdf4c0725e0 0x7fdf4c074aa0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fdf6c076f10 tx=0x7fdf5c006cd0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:38.989 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:38.987+0000 7fdf70e00640 1 -- 172.21.15.17:0/2518174069 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fdf6c14ec10 con 0x7fdf6c0c7bf0 2024-09-10T21:43:39.001 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:38.999+0000 7fdf63400640 1 -- 172.21.15.17:0/2518174069 <== mon.2 v2:172.21.15.31:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1558 (secure 0 0 0) 0x7fdf6405dfc0 con 0x7fdf6c0c7bf0 2024-09-10T21:43:39.001 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":144,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":17,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5e2f023f-d6a6-49f4-a256-f4807dce3739","modified":"2024-09-10T21:23:46.892130Z","created":"2024-09-10T21:23:46.892130Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3301","nonce":0},{"type":"v1","addr":"172.21.15.31:6790","nonce":0}]},"addr":"172.21.15.31:6790/0","public_addr":"172.21.15.31:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-10T21:43:39.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:39.007+0000 7fdf70e00640 1 -- 172.21.15.17:0/2518174069 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fdf4c0725e0 msgr2=0x7fdf4c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:39.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:39.007+0000 7fdf70e00640 1 --2- 172.21.15.17:0/2518174069 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fdf4c0725e0 0x7fdf4c074aa0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fdf6c076f10 tx=0x7fdf5c006cd0 comp rx=0 tx=0).stop 2024-09-10T21:43:39.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:39.007+0000 7fdf70e00640 1 -- 172.21.15.17:0/2518174069 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fdf6c0c7bf0 msgr2=0x7fdf6c14e0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:39.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:39.007+0000 7fdf70e00640 1 --2- 172.21.15.17:0/2518174069 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fdf6c0c7bf0 0x7fdf6c14e0f0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fdf6400ba80 tx=0x7fdf6400bf40 comp rx=0 tx=0).stop 2024-09-10T21:43:39.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:39.007+0000 7fdf70e00640 1 -- 172.21.15.17:0/2518174069 shutdown_connections 2024-09-10T21:43:39.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:39.007+0000 7fdf70e00640 1 --2- 172.21.15.17:0/2518174069 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fdf4c0725e0 0x7fdf4c074aa0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:39.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:39.007+0000 7fdf70e00640 1 --2- 172.21.15.17:0/2518174069 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdf6c150b50 0x7fdf6c151800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:39.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:39.007+0000 7fdf70e00640 1 --2- 172.21.15.17:0/2518174069 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdf6c0c8290 0x7fdf6c14e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:39.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:39.007+0000 7fdf70e00640 1 --2- 172.21.15.17:0/2518174069 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fdf6c0c7bf0 0x7fdf6c14e0f0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:39.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:39.007+0000 7fdf70e00640 1 -- 172.21.15.17:0/2518174069 >> 172.21.15.17:0/2518174069 conn(0x7fdf6c06cdf0 msgr2=0x7fdf6c06d1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:43:39.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:39.007+0000 7fdf70e00640 1 -- 172.21.15.17:0/2518174069 shutdown_connections 2024-09-10T21:43:39.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:39.007+0000 7fdf70e00640 1 -- 172.21.15.17:0/2518174069 wait complete. 2024-09-10T21:43:42.064 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-10T21:43:42.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:42.355+0000 7f9d22800640 1 -- 172.21.15.17:0/62396728 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9d1c15a380 msgr2=0x7f9d1c161170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:42.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:42.355+0000 7f9d22800640 1 --2- 172.21.15.17:0/62396728 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9d1c15a380 0x7f9d1c161170 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f9d1c0c6e70 tx=0x7f9d1401ddd0 comp rx=0 tx=0).stop 2024-09-10T21:43:42.360 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:42.355+0000 7f9d22800640 1 -- 172.21.15.17:0/62396728 shutdown_connections 2024-09-10T21:43:42.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:42.355+0000 7f9d22800640 1 --2- 172.21.15.17:0/62396728 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9d1c15a380 0x7f9d1c161170 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:42.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:42.355+0000 7f9d22800640 1 --2- 172.21.15.17:0/62396728 >> v2:172.21.15.31:3300/0 conn(0x7f9d1c073ac0 0x7f9d1c159e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:42.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:42.355+0000 7f9d22800640 1 -- 172.21.15.17:0/62396728 >> 172.21.15.17:0/62396728 conn(0x7f9d1c0cbbb0 msgr2=0x7f9d1c0cbfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:43:42.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:42.355+0000 7f9d22800640 1 -- 172.21.15.17:0/62396728 shutdown_connections 2024-09-10T21:43:42.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:42.355+0000 7f9d22800640 1 -- 172.21.15.17:0/62396728 wait complete. 2024-09-10T21:43:42.361 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:42.359+0000 7f9d22800640 1 Processor -- start 2024-09-10T21:43:42.362 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:42.359+0000 7f9d22800640 1 -- start start 2024-09-10T21:43:42.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:42.359+0000 7f9d22800640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9d1c073ac0 0x7f9d1c14ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:42.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:42.359+0000 7f9d22800640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f9d1c157370 0x7f9d1c0beab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:42.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:42.363+0000 7f9d22800640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9d1c15a380 0x7f9d1c0beff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:42.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:42.363+0000 7f9d21400640 1 -- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9d1c073ac0 msgr2=0x7f9d1c14ccc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:3300/0 2024-09-10T21:43:42.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:42.363+0000 7f9d21400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9d1c073ac0 0x7f9d1c14ccc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:43:42.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:42.363+0000 7f9d20a00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f9d1c157370 0x7f9d1c0beab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:42.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:42.363+0000 7f9d20a00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f9d1c157370 0x7f9d1c0beab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3301/0 says I am v2:172.21.15.17:37310/0 (socket says 172.21.15.17:37310) 2024-09-10T21:43:42.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:42.363+0000 7f9d20a00640 1 -- 172.21.15.17:0/4265609569 learned_addr learned my addr 172.21.15.17:0/4265609569 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:43:42.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:42.363+0000 7f9d22800640 1 -- 172.21.15.17:0/4265609569 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f9d1c0c83d0 con 0x7f9d1c15a380 2024-09-10T21:43:42.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:42.363+0000 7f9d22800640 1 -- 172.21.15.17:0/4265609569 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f9d1c06e9d0 con 0x7f9d1c073ac0 2024-09-10T21:43:42.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:42.363+0000 7f9d22800640 1 -- 172.21.15.17:0/4265609569 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f9d1c157750 con 0x7f9d1c157370 2024-09-10T21:43:42.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:42.363+0000 7f9d21e00640 1 --2- 172.21.15.17:0/4265609569 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9d1c15a380 0x7f9d1c0beff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:42.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:42.363+0000 7f9d20a00640 1 -- 172.21.15.17:0/4265609569 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9d1c073ac0 msgr2=0x7f9d1c14ccc0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:43:42.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:42.363+0000 7f9d20a00640 1 --2- 172.21.15.17:0/4265609569 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9d1c073ac0 0x7f9d1c14ccc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:42.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:42.363+0000 7f9d20a00640 1 -- 172.21.15.17:0/4265609569 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9d1c15a380 msgr2=0x7f9d1c0beff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:42.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:42.363+0000 7f9d20a00640 1 --2- 172.21.15.17:0/4265609569 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9d1c15a380 0x7f9d1c0beff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:42.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:42.363+0000 7f9d20a00640 1 -- 172.21.15.17:0/4265609569 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d1c1f0500 con 0x7f9d1c157370 2024-09-10T21:43:42.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:42.363+0000 7f9d20a00640 1 --2- 172.21.15.17:0/4265609569 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f9d1c157370 0x7f9d1c0beab0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f9d0800b580 tx=0x7f9d0800ba40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:42.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:42.367+0000 7f9d19600640 1 -- 172.21.15.17:0/4265609569 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d0800f850 con 0x7f9d1c157370 2024-09-10T21:43:42.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:42.367+0000 7f9d19600640 1 -- 172.21.15.17:0/4265609569 <== mon.2 v2:172.21.15.31:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9d0800fb00 con 0x7f9d1c157370 2024-09-10T21:43:42.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:42.367+0000 7f9d19600640 1 -- 172.21.15.17:0/4265609569 <== mon.2 v2:172.21.15.31:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d08017400 con 0x7f9d1c157370 2024-09-10T21:43:42.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:42.367+0000 7f9d22800640 1 -- 172.21.15.17:0/4265609569 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d1c150f70 con 0x7f9d1c157370 2024-09-10T21:43:42.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:42.367+0000 7f9d22800640 1 -- 172.21.15.17:0/4265609569 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d1c151530 con 0x7f9d1c157370 2024-09-10T21:43:42.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:42.371+0000 7f9d22800640 1 -- 172.21.15.17:0/4265609569 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d1c0c3ab0 con 0x7f9d1c157370 2024-09-10T21:43:42.376 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:42.375+0000 7f9d19600640 1 -- 172.21.15.17:0/4265609569 <== mon.2 v2:172.21.15.31:3301/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f9d0801f030 con 0x7f9d1c157370 2024-09-10T21:43:42.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:42.375+0000 7f9d19600640 1 --2- 172.21.15.17:0/4265609569 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f9cfc0725e0 0x7f9cfc074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:42.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:42.375+0000 7f9d19600640 1 -- 172.21.15.17:0/4265609569 <== mon.2 v2:172.21.15.31:3301/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f9d080937e0 con 0x7f9d1c157370 2024-09-10T21:43:42.381 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:42.379+0000 7f9d21400640 1 --2- 172.21.15.17:0/4265609569 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f9cfc0725e0 0x7f9cfc074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:42.381 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:42.379+0000 7f9d19600640 1 -- 172.21.15.17:0/4265609569 <== mon.2 v2:172.21.15.31:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f9d08026080 con 0x7f9d1c157370 2024-09-10T21:43:42.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:42.387+0000 7f9d21400640 1 --2- 172.21.15.17:0/4265609569 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f9cfc0725e0 0x7f9cfc074aa0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f9d100036b0 tx=0x7f9d10007ae0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:42.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:42.631+0000 7f9d22800640 1 -- 172.21.15.17:0/4265609569 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f9d1c0bf770 con 0x7f9d1c157370 2024-09-10T21:43:42.635 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":144,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":21,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5e2f023f-d6a6-49f4-a256-f4807dce3739","modified":"2024-09-10T21:23:46.892130Z","created":"2024-09-10T21:23:46.892130Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3301","nonce":0},{"type":"v1","addr":"172.21.15.31:6790","nonce":0}]},"addr":"172.21.15.31:6790/0","public_addr":"172.21.15.31:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-10T21:43:42.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:42.635+0000 7f9d19600640 1 -- 172.21.15.17:0/4265609569 <== mon.2 v2:172.21.15.31:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1558 (secure 0 0 0) 0x7f9d08013030 con 0x7f9d1c157370 2024-09-10T21:43:42.737 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:42.635+0000 7f9d22800640 1 -- 172.21.15.17:0/4265609569 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f9cfc0725e0 msgr2=0x7f9cfc074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:42.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:42.635+0000 7f9d22800640 1 --2- 172.21.15.17:0/4265609569 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f9cfc0725e0 0x7f9cfc074aa0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f9d100036b0 tx=0x7f9d10007ae0 comp rx=0 tx=0).stop 2024-09-10T21:43:42.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:42.635+0000 7f9d22800640 1 -- 172.21.15.17:0/4265609569 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f9d1c157370 msgr2=0x7f9d1c0beab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:42.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:42.635+0000 7f9d22800640 1 --2- 172.21.15.17:0/4265609569 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f9d1c157370 0x7f9d1c0beab0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f9d0800b580 tx=0x7f9d0800ba40 comp rx=0 tx=0).stop 2024-09-10T21:43:42.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:42.635+0000 7f9d22800640 1 -- 172.21.15.17:0/4265609569 shutdown_connections 2024-09-10T21:43:42.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:42.635+0000 7f9d22800640 1 --2- 172.21.15.17:0/4265609569 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f9cfc0725e0 0x7f9cfc074aa0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:42.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:42.635+0000 7f9d22800640 1 --2- 172.21.15.17:0/4265609569 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9d1c15a380 0x7f9d1c0beff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:42.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:42.635+0000 7f9d22800640 1 --2- 172.21.15.17:0/4265609569 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f9d1c157370 0x7f9d1c0beab0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:42.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:42.635+0000 7f9d22800640 1 --2- 172.21.15.17:0/4265609569 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9d1c073ac0 0x7f9d1c14ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:42.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:42.635+0000 7f9d22800640 1 -- 172.21.15.17:0/4265609569 >> 172.21.15.17:0/4265609569 conn(0x7f9d1c0cbbb0 msgr2=0x7f9d1c1543a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:43:42.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:42.635+0000 7f9d22800640 1 -- 172.21.15.17:0/4265609569 shutdown_connections 2024-09-10T21:43:42.740 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:42.635+0000 7f9d22800640 1 -- 172.21.15.17:0/4265609569 wait complete. 2024-09-10T21:43:45.803 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-10T21:43:46.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:46.087+0000 7f89eea00640 1 -- 172.21.15.17:0/1196032390 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f89e80c1040 msgr2=0x7f89e806e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:46.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:46.087+0000 7f89eea00640 1 --2- 172.21.15.17:0/1196032390 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f89e80c1040 0x7f89e806e650 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f89d4009840 tx=0x7f89d401f050 comp rx=0 tx=0).stop 2024-09-10T21:43:46.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:46.087+0000 7f89eea00640 1 -- 172.21.15.17:0/1196032390 shutdown_connections 2024-09-10T21:43:46.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:46.087+0000 7f89eea00640 1 --2- 172.21.15.17:0/1196032390 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f89e80c1040 0x7f89e806e650 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:46.089 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:46.087+0000 7f89eea00640 1 --2- 172.21.15.17:0/1196032390 >> v2:172.21.15.31:3300/0 conn(0x7f89e80c8ab0 0x7f89e80c8e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:46.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:46.087+0000 7f89eea00640 1 -- 172.21.15.17:0/1196032390 >> 172.21.15.17:0/1196032390 conn(0x7f89e8070160 msgr2=0x7f89e8070570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:43:46.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:46.087+0000 7f89eea00640 1 -- 172.21.15.17:0/1196032390 shutdown_connections 2024-09-10T21:43:46.094 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:46.087+0000 7f89eea00640 1 -- 172.21.15.17:0/1196032390 wait complete. 2024-09-10T21:43:46.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:46.095+0000 7f89eea00640 1 Processor -- start 2024-09-10T21:43:46.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:46.099+0000 7f89eea00640 1 -- start start 2024-09-10T21:43:46.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:46.099+0000 7f89eea00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f89e80c1040 0x7f89e8152b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:46.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:46.099+0000 7f89eea00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f89e80c8ab0 0x7f89e80ca080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:46.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:46.099+0000 7f89eea00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f89e815a5d0 0x7f89e814cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:46.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:46.099+0000 7f89eea00640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f89e806de50 con 0x7f89e815a5d0 2024-09-10T21:43:46.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:46.099+0000 7f89eea00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f89e806db70 con 0x7f89e80c1040 2024-09-10T21:43:46.103 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:46.099+0000 7f89eea00640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f89e815aa80 con 0x7f89e80c8ab0 2024-09-10T21:43:46.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:46.103+0000 7f89eca00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f89e815a5d0 0x7f89e814cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:46.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:46.103+0000 7f89eca00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f89e815a5d0 0x7f89e814cab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35184/0 (socket says 172.21.15.17:35184) 2024-09-10T21:43:46.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:46.103+0000 7f89eca00640 1 -- 172.21.15.17:0/1815895886 learned_addr learned my addr 172.21.15.17:0/1815895886 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:43:46.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:46.103+0000 7f89e7400640 1 --2- 172.21.15.17:0/1815895886 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f89e80c8ab0 0x7f89e80ca080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:46.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:46.103+0000 7f89e7e00640 1 -- 172.21.15.17:0/1815895886 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f89e80c1040 msgr2=0x7f89e8152b70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:3300/0 2024-09-10T21:43:46.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:46.103+0000 7f89e7e00640 1 --2- 172.21.15.17:0/1815895886 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f89e80c1040 0x7f89e8152b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:43:46.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:46.103+0000 7f89e7400640 1 -- 172.21.15.17:0/1815895886 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f89e80c1040 msgr2=0x7f89e8152b70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:43:46.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:46.103+0000 7f89e7400640 1 --2- 172.21.15.17:0/1815895886 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f89e80c1040 0x7f89e8152b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:46.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:46.103+0000 7f89e7400640 1 -- 172.21.15.17:0/1815895886 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f89e815a5d0 msgr2=0x7f89e814cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:46.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:46.103+0000 7f89e7400640 1 --2- 172.21.15.17:0/1815895886 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f89e815a5d0 0x7f89e814cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:46.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:46.103+0000 7f89e7400640 1 -- 172.21.15.17:0/1815895886 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89e81530b0 con 0x7f89e80c8ab0 2024-09-10T21:43:46.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:46.103+0000 7f89e7400640 1 --2- 172.21.15.17:0/1815895886 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f89e80c8ab0 0x7f89e80ca080 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f89d40af000 tx=0x7f89d4005cc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:46.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:46.103+0000 7f89dfe00640 1 -- 172.21.15.17:0/1815895886 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89d40bd020 con 0x7f89e80c8ab0 2024-09-10T21:43:46.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:46.103+0000 7f89dfe00640 1 -- 172.21.15.17:0/1815895886 <== mon.2 v2:172.21.15.31:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f89d4004320 con 0x7f89e80c8ab0 2024-09-10T21:43:46.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:46.103+0000 7f89dfe00640 1 -- 172.21.15.17:0/1815895886 <== mon.2 v2:172.21.15.31:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89d40b5040 con 0x7f89e80c8ab0 2024-09-10T21:43:46.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:46.103+0000 7f89eea00640 1 -- 172.21.15.17:0/1815895886 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89e80caa50 con 0x7f89e80c8ab0 2024-09-10T21:43:46.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:46.103+0000 7f89eea00640 1 -- 172.21.15.17:0/1815895886 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f89e80cc010 con 0x7f89e80c8ab0 2024-09-10T21:43:46.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:46.107+0000 7f89eea00640 1 -- 172.21.15.17:0/1815895886 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89e80cbe00 con 0x7f89e80c8ab0 2024-09-10T21:43:46.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:46.107+0000 7f89dfe00640 1 -- 172.21.15.17:0/1815895886 <== mon.2 v2:172.21.15.31:3301/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f89d40b8070 con 0x7f89e80c8ab0 2024-09-10T21:43:46.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:46.111+0000 7f89dfe00640 1 --2- 172.21.15.17:0/1815895886 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f89c8072610 0x7f89c8074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:46.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:46.111+0000 7f89dfe00640 1 -- 172.21.15.17:0/1815895886 <== mon.2 v2:172.21.15.31:3301/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f89d4130480 con 0x7f89e80c8ab0 2024-09-10T21:43:46.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:46.111+0000 7f89e7e00640 1 --2- 172.21.15.17:0/1815895886 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f89c8072610 0x7f89c8074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:46.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:46.111+0000 7f89e7e00640 1 --2- 172.21.15.17:0/1815895886 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f89c8072610 0x7f89c8074ad0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f89d8001060 tx=0x7f89d8006d70 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:46.125 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:46.119+0000 7f89dfe00640 1 -- 172.21.15.17:0/1815895886 <== mon.2 v2:172.21.15.31:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f89d4018bb0 con 0x7f89e80c8ab0 2024-09-10T21:43:46.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:46.619+0000 7f89eea00640 1 -- 172.21.15.17:0/1815895886 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f89e80c79f0 con 0x7f89e80c8ab0 2024-09-10T21:43:46.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:46.619+0000 7f89dfe00640 1 -- 172.21.15.17:0/1815895886 <== mon.2 v2:172.21.15.31:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1558 (secure 0 0 0) 0x7f89d40a8e20 con 0x7f89e80c8ab0 2024-09-10T21:43:46.631 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":144,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":25,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5e2f023f-d6a6-49f4-a256-f4807dce3739","modified":"2024-09-10T21:23:46.892130Z","created":"2024-09-10T21:23:46.892130Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3301","nonce":0},{"type":"v1","addr":"172.21.15.31:6790","nonce":0}]},"addr":"172.21.15.31:6790/0","public_addr":"172.21.15.31:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-10T21:43:46.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:46.635+0000 7f89eea00640 1 -- 172.21.15.17:0/1815895886 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f89c8072610 msgr2=0x7f89c8074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:46.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:46.635+0000 7f89eea00640 1 --2- 172.21.15.17:0/1815895886 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f89c8072610 0x7f89c8074ad0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f89d8001060 tx=0x7f89d8006d70 comp rx=0 tx=0).stop 2024-09-10T21:43:46.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:46.635+0000 7f89eea00640 1 -- 172.21.15.17:0/1815895886 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f89e80c8ab0 msgr2=0x7f89e80ca080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:46.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:46.635+0000 7f89eea00640 1 --2- 172.21.15.17:0/1815895886 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f89e80c8ab0 0x7f89e80ca080 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f89d40af000 tx=0x7f89d4005cc0 comp rx=0 tx=0).stop 2024-09-10T21:43:46.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:46.635+0000 7f89eea00640 1 -- 172.21.15.17:0/1815895886 shutdown_connections 2024-09-10T21:43:46.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:46.635+0000 7f89eea00640 1 --2- 172.21.15.17:0/1815895886 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f89c8072610 0x7f89c8074ad0 secure :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f89d8001060 tx=0x7f89d8006d70 comp rx=0 tx=0).stop 2024-09-10T21:43:46.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:46.635+0000 7f89eea00640 1 --2- 172.21.15.17:0/1815895886 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f89e815a5d0 0x7f89e814cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:46.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:46.635+0000 7f89eea00640 1 --2- 172.21.15.17:0/1815895886 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f89e80c8ab0 0x7f89e80ca080 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:46.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:46.635+0000 7f89eea00640 1 --2- 172.21.15.17:0/1815895886 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f89e80c1040 0x7f89e8152b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:46.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:46.635+0000 7f89eea00640 1 -- 172.21.15.17:0/1815895886 >> 172.21.15.17:0/1815895886 conn(0x7f89e8070160 msgr2=0x7f89e80c7e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:43:46.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:46.635+0000 7f89eea00640 1 -- 172.21.15.17:0/1815895886 shutdown_connections 2024-09-10T21:43:46.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:46.635+0000 7f89eea00640 1 -- 172.21.15.17:0/1815895886 wait complete. 2024-09-10T21:43:49.693 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-10T21:43:53.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:53.195+0000 7ff892e00640 1 -- 172.21.15.17:0/39259022 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff86c007360 msgr2=0x7ff86c009820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:53.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:53.195+0000 7ff892e00640 1 --2- 172.21.15.17:0/39259022 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff86c007360 0x7ff86c009820 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff878001060 tx=0x7ff87801ca20 comp rx=0 tx=0).stop 2024-09-10T21:43:53.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:53.203+0000 7ff890e00640 1 -- 172.21.15.17:0/39259022 reap_dead start 2024-09-10T21:43:53.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:53.203+0000 7ff892e00640 1 -- 172.21.15.17:0/39259022 shutdown_connections 2024-09-10T21:43:53.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:53.203+0000 7ff892e00640 1 --2- 172.21.15.17:0/39259022 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff86c007360 0x7ff86c009820 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:53.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:53.203+0000 7ff892e00640 1 -- 172.21.15.17:0/39259022 >> 172.21.15.17:0/39259022 conn(0x7ff88c068990 msgr2=0x7ff88c068da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:43:53.214 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:53.207+0000 7ff892e00640 1 -- 172.21.15.17:0/39259022 shutdown_connections 2024-09-10T21:43:53.214 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:53.207+0000 7ff892e00640 1 -- 172.21.15.17:0/39259022 wait complete. 2024-09-10T21:43:53.214 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:53.211+0000 7ff892e00640 1 Processor -- start 2024-09-10T21:43:53.214 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:53.211+0000 7ff892e00640 1 -- start start 2024-09-10T21:43:53.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:53.211+0000 7ff892e00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff86c007360 0x7ff88c0b5030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:53.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:53.215+0000 7ff892e00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff88c156ee0 0x7ff88c0c8840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:53.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:53.215+0000 7ff88be00640 1 -- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff86c007360 msgr2=0x7ff88c0b5030 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:3300/0 2024-09-10T21:43:53.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:53.215+0000 7ff88be00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff86c007360 0x7ff88c0b5030 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:43:53.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:53.215+0000 7ff892e00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff88c0c8d80 0x7ff88c150090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:53.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:53.215+0000 7ff88b400640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff88c156ee0 0x7ff88c0c8840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:53.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:53.215+0000 7ff88b400640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff88c156ee0 0x7ff88c0c8840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3301/0 says I am v2:172.21.15.17:48836/0 (socket says 172.21.15.17:48836) 2024-09-10T21:43:53.217 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:53.215+0000 7ff88b400640 1 -- 172.21.15.17:0/2494680947 learned_addr learned my addr 172.21.15.17:0/2494680947 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:43:53.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:53.215+0000 7ff892e00640 1 -- 172.21.15.17:0/2494680947 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7ff88c167a20 con 0x7ff88c0c8d80 2024-09-10T21:43:53.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:53.215+0000 7ff890e00640 1 --2- 172.21.15.17:0/2494680947 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff88c0c8d80 0x7ff88c150090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:53.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:53.215+0000 7ff892e00640 1 -- 172.21.15.17:0/2494680947 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7ff88c1678a0 con 0x7ff86c007360 2024-09-10T21:43:53.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:53.215+0000 7ff892e00640 1 -- 172.21.15.17:0/2494680947 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7ff88c167ba0 con 0x7ff88c156ee0 2024-09-10T21:43:53.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:53.215+0000 7ff88b400640 1 -- 172.21.15.17:0/2494680947 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff86c007360 msgr2=0x7ff88c0b5030 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:43:53.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:53.215+0000 7ff88b400640 1 --2- 172.21.15.17:0/2494680947 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff86c007360 0x7ff88c0b5030 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:53.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:53.215+0000 7ff88b400640 1 -- 172.21.15.17:0/2494680947 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff88c0c8d80 msgr2=0x7ff88c150090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:53.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:53.215+0000 7ff88b400640 1 --2- 172.21.15.17:0/2494680947 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff88c0c8d80 0x7ff88c150090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:53.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:53.215+0000 7ff88b400640 1 -- 172.21.15.17:0/2494680947 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff88c157c20 con 0x7ff88c156ee0 2024-09-10T21:43:53.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:53.219+0000 7ff88b400640 1 --2- 172.21.15.17:0/2494680947 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff88c156ee0 0x7ff88c0c8840 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff87801a650 tx=0x7ff878009300 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:53.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:53.219+0000 7ff883e00640 1 -- 172.21.15.17:0/2494680947 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8780a5b10 con 0x7ff88c156ee0 2024-09-10T21:43:53.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:53.219+0000 7ff892e00640 1 -- 172.21.15.17:0/2494680947 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff88c1553c0 con 0x7ff88c156ee0 2024-09-10T21:43:53.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:53.219+0000 7ff892e00640 1 -- 172.21.15.17:0/2494680947 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7ff88c155980 con 0x7ff88c156ee0 2024-09-10T21:43:53.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:53.219+0000 7ff883e00640 1 -- 172.21.15.17:0/2494680947 <== mon.2 v2:172.21.15.31:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff8780bb070 con 0x7ff88c156ee0 2024-09-10T21:43:53.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:53.219+0000 7ff883e00640 1 -- 172.21.15.17:0/2494680947 <== mon.2 v2:172.21.15.31:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8780a5b10 con 0x7ff88c156ee0 2024-09-10T21:43:53.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:53.223+0000 7ff881600640 1 -- 172.21.15.17:0/2494680947 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff88c0c6580 con 0x7ff88c156ee0 2024-09-10T21:43:53.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:53.223+0000 7ff883e00640 1 -- 172.21.15.17:0/2494680947 <== mon.2 v2:172.21.15.31:3301/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7ff8780deb80 con 0x7ff88c156ee0 2024-09-10T21:43:53.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:53.227+0000 7ff883e00640 1 --2- 172.21.15.17:0/2494680947 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7ff86c0732a0 0x7ff86c075760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:53.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:53.227+0000 7ff88be00640 1 --2- 172.21.15.17:0/2494680947 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7ff86c0732a0 0x7ff86c075760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:53.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:53.231+0000 7ff88be00640 1 --2- 172.21.15.17:0/2494680947 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7ff86c0732a0 0x7ff86c075760 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7ff87c005ac0 tx=0x7ff87c005a30 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:53.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:53.231+0000 7ff883e00640 1 -- 172.21.15.17:0/2494680947 <== mon.2 v2:172.21.15.31:3301/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7ff8780a5da0 con 0x7ff88c156ee0 2024-09-10T21:43:53.237 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:53.231+0000 7ff883e00640 1 -- 172.21.15.17:0/2494680947 <== mon.2 v2:172.21.15.31:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7ff8780166a0 con 0x7ff88c156ee0 2024-09-10T21:43:53.785 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:53.783+0000 7ff881600640 1 -- 172.21.15.17:0/2494680947 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7ff88c153c30 con 0x7ff88c156ee0 2024-09-10T21:43:53.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:53.783+0000 7ff883e00640 1 -- 172.21.15.17:0/2494680947 <== mon.2 v2:172.21.15.31:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1551 (secure 0 0 0) 0x7ff8780f8aa0 con 0x7ff88c156ee0 2024-09-10T21:43:53.788 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":148,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5e2f023f-d6a6-49f4-a256-f4807dce3739","modified":"2024-09-10T21:23:46.892130Z","created":"2024-09-10T21:23:46.892130Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3301","nonce":0},{"type":"v1","addr":"172.21.15.31:6790","nonce":0}]},"addr":"172.21.15.31:6790/0","public_addr":"172.21.15.31:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-10T21:43:53.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:53.807+0000 7ff881600640 1 -- 172.21.15.17:0/2494680947 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7ff86c0732a0 msgr2=0x7ff86c075760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:53.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:53.807+0000 7ff881600640 1 --2- 172.21.15.17:0/2494680947 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7ff86c0732a0 0x7ff86c075760 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7ff87c005ac0 tx=0x7ff87c005a30 comp rx=0 tx=0).stop 2024-09-10T21:43:53.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:53.807+0000 7ff881600640 1 -- 172.21.15.17:0/2494680947 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff88c156ee0 msgr2=0x7ff88c0c8840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:53.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:53.807+0000 7ff881600640 1 --2- 172.21.15.17:0/2494680947 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff88c156ee0 0x7ff88c0c8840 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff87801a650 tx=0x7ff878009300 comp rx=0 tx=0).stop 2024-09-10T21:43:53.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:53.819+0000 7ff881600640 1 -- 172.21.15.17:0/2494680947 shutdown_connections 2024-09-10T21:43:53.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:53.819+0000 7ff881600640 1 --2- 172.21.15.17:0/2494680947 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7ff86c0732a0 0x7ff86c075760 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:53.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:53.819+0000 7ff881600640 1 --2- 172.21.15.17:0/2494680947 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff88c0c8d80 0x7ff88c150090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:53.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:53.819+0000 7ff881600640 1 --2- 172.21.15.17:0/2494680947 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff88c156ee0 0x7ff88c0c8840 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:53.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:53.819+0000 7ff881600640 1 --2- 172.21.15.17:0/2494680947 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff86c007360 0x7ff88c0b5030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:53.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:53.819+0000 7ff881600640 1 -- 172.21.15.17:0/2494680947 >> 172.21.15.17:0/2494680947 conn(0x7ff88c068990 msgr2=0x7ff88c151b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:43:53.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:53.819+0000 7ff881600640 1 -- 172.21.15.17:0/2494680947 shutdown_connections 2024-09-10T21:43:53.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:53.819+0000 7ff881600640 1 -- 172.21.15.17:0/2494680947 wait complete. 2024-09-10T21:43:53.856 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2024-09-10T21:43:53.857 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon a 2024-09-10T21:43:53.857 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-10T21:43:54.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:54.131+0000 7fa354a00640 1 -- 172.21.15.17:0/2068642508 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa35006dc30 msgr2=0x7fa35006e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:54.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:54.131+0000 7fa354a00640 1 --2- 172.21.15.17:0/2068642508 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa35006dc30 0x7fa35006e010 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fa3500c6c40 tx=0x7fa34001eef0 comp rx=0 tx=0).stop 2024-09-10T21:43:54.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:54.143+0000 7fa354a00640 1 -- 172.21.15.17:0/2068642508 shutdown_connections 2024-09-10T21:43:54.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:54.143+0000 7fa354a00640 1 --2- 172.21.15.17:0/2068642508 >> v2:172.21.15.31:3300/0 conn(0x7fa350154400 0x7fa350154850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:54.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:54.143+0000 7fa354a00640 1 --2- 172.21.15.17:0/2068642508 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa35006dc30 0x7fa35006e010 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:54.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:54.143+0000 7fa354a00640 1 -- 172.21.15.17:0/2068642508 >> 172.21.15.17:0/2068642508 conn(0x7fa3500700d0 msgr2=0x7fa3500704e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:43:54.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:54.151+0000 7fa354a00640 1 -- 172.21.15.17:0/2068642508 shutdown_connections 2024-09-10T21:43:54.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:54.151+0000 7fa354a00640 1 -- 172.21.15.17:0/2068642508 wait complete. 2024-09-10T21:43:54.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:54.151+0000 7fa354a00640 1 Processor -- start 2024-09-10T21:43:54.160 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:54.159+0000 7fa354a00640 1 -- start start 2024-09-10T21:43:54.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:54.159+0000 7fa354a00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa35006dc30 0x7fa350155810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:54.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:54.159+0000 7fa354a00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa35006e7a0 0x7fa350156b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:54.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:54.159+0000 7fa354a00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa350154400 0x7fa3501570d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:54.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:54.159+0000 7fa354a00640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fa3500c8220 con 0x7fa350154400 2024-09-10T21:43:54.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:54.159+0000 7fa354a00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fa3500c8870 con 0x7fa35006dc30 2024-09-10T21:43:54.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:54.159+0000 7fa354a00640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7fa3500c8500 con 0x7fa35006e7a0 2024-09-10T21:43:54.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:54.163+0000 7fa34e800640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa350154400 0x7fa3501570d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:54.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:54.163+0000 7fa34de00640 1 -- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa35006dc30 msgr2=0x7fa350155810 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:3300/0 2024-09-10T21:43:54.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:54.163+0000 7fa34e800640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa350154400 0x7fa3501570d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35196/0 (socket says 172.21.15.17:35196) 2024-09-10T21:43:54.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:54.163+0000 7fa34e800640 1 -- 172.21.15.17:0/494027899 learned_addr learned my addr 172.21.15.17:0/494027899 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:43:54.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:54.163+0000 7fa34de00640 1 --2- 172.21.15.17:0/494027899 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa35006dc30 0x7fa350155810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:43:54.166 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:54.163+0000 7fa34d400640 1 --2- 172.21.15.17:0/494027899 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa35006e7a0 0x7fa350156b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:54.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:54.163+0000 7fa34e800640 1 -- 172.21.15.17:0/494027899 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa35006e7a0 msgr2=0x7fa350156b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:54.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:54.163+0000 7fa34e800640 1 --2- 172.21.15.17:0/494027899 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa35006e7a0 0x7fa350156b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:54.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:54.163+0000 7fa34e800640 1 -- 172.21.15.17:0/494027899 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa35006dc30 msgr2=0x7fa350155810 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:43:54.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:54.163+0000 7fa34e800640 1 --2- 172.21.15.17:0/494027899 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa35006dc30 0x7fa350155810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:54.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:54.163+0000 7fa34e800640 1 -- 172.21.15.17:0/494027899 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3501563a0 con 0x7fa350154400 2024-09-10T21:43:54.167 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:54.163+0000 7fa34e800640 1 --2- 172.21.15.17:0/494027899 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa350154400 0x7fa3501570d0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fa34800b9c0 tx=0x7fa34800be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:54.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:54.167+0000 7fa346000640 1 -- 172.21.15.17:0/494027899 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa348019070 con 0x7fa350154400 2024-09-10T21:43:54.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:54.167+0000 7fa354a00640 1 -- 172.21.15.17:0/494027899 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3501fdc10 con 0x7fa350154400 2024-09-10T21:43:54.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:54.167+0000 7fa354a00640 1 -- 172.21.15.17:0/494027899 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3501fe1d0 con 0x7fa350154400 2024-09-10T21:43:54.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:54.167+0000 7fa346000640 1 -- 172.21.15.17:0/494027899 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa34800cc30 con 0x7fa350154400 2024-09-10T21:43:54.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:54.167+0000 7fa354a00640 1 --2- 172.21.15.17:0/494027899 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa3501fe410 0x7fa350152730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:54.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:54.167+0000 7fa354a00640 1 -- 172.21.15.17:0/494027899 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fa35006e010 con 0x7fa3501fe410 2024-09-10T21:43:54.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:54.167+0000 7fa346000640 1 -- 172.21.15.17:0/494027899 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa34800ce60 con 0x7fa350154400 2024-09-10T21:43:54.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:54.167+0000 7fa34de00640 1 --2- 172.21.15.17:0/494027899 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa3501fe410 0x7fa350152730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:54.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:54.171+0000 7fa34de00640 1 -- 172.21.15.17:0/494027899 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa3500cafd0 con 0x7fa3501fe410 2024-09-10T21:43:54.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:54.179+0000 7fa346000640 1 -- 172.21.15.17:0/494027899 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fa3480146a0 con 0x7fa350154400 2024-09-10T21:43:54.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:54.179+0000 7fa34de00640 1 --2- 172.21.15.17:0/494027899 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa3501fe410 0x7fa350152730 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fa3500c9b40 tx=0x7fa3400a8ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:54.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:54.179+0000 7fa346000640 1 --2- 172.21.15.17:0/494027899 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fa32c072590 0x7fa32c074a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:54.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:54.179+0000 7fa346000640 1 -- 172.21.15.17:0/494027899 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fa348093570 con 0x7fa350154400 2024-09-10T21:43:54.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:54.179+0000 7fa346000640 1 -- 172.21.15.17:0/494027899 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3400bd030 con 0x7fa3501fe410 2024-09-10T21:43:54.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:54.179+0000 7fa346000640 1 -- 172.21.15.17:0/494027899 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fa3400a8c30 con 0x7fa3501fe410 2024-09-10T21:43:54.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:54.179+0000 7fa346000640 1 -- 172.21.15.17:0/494027899 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa3501fe410 msgr2=0x7fa350152730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:54.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:54.179+0000 7fa346000640 1 --2- 172.21.15.17:0/494027899 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa3501fe410 0x7fa350152730 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fa3500c9b40 tx=0x7fa3400a8ac0 comp rx=0 tx=0).stop 2024-09-10T21:43:54.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:54.179+0000 7fa346000640 1 -- 172.21.15.17:0/494027899 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa3501fe410 msgr2=0x7fa350152730 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:43:54.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:54.179+0000 7fa346000640 1 --2- 172.21.15.17:0/494027899 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa3501fe410 0x7fa350152730 secure :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fa3500c9b40 tx=0x7fa3400a8ac0 comp rx=0 tx=0).stop 2024-09-10T21:43:54.185 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:54.183+0000 7fa34d400640 1 --2- 172.21.15.17:0/494027899 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fa32c072590 0x7fa32c074a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:54.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:54.183+0000 7fa34d400640 1 --2- 172.21.15.17:0/494027899 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fa32c072590 0x7fa32c074a50 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fa350154260 tx=0x7fa3380095e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:54.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:54.207+0000 7fa354a00640 1 --2- 172.21.15.17:0/494027899 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa350152c70 0x7fa3500aa4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:54.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:54.207+0000 7fa354a00640 1 -- 172.21.15.17:0/494027899 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fa35015a2b0 con 0x7fa350152c70 2024-09-10T21:43:54.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:54.223+0000 7fa34de00640 1 --2- 172.21.15.17:0/494027899 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa350152c70 0x7fa3500aa4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:54.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:54.243+0000 7fa34de00640 1 -- 172.21.15.17:0/494027899 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fa3500c3160 con 0x7fa350152c70 2024-09-10T21:43:54.274 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:54.271+0000 7fa34de00640 1 --2- 172.21.15.17:0/494027899 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa350152c70 0x7fa3500aa4f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa340009230 tx=0x7fa34001eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:54.278 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:54.271+0000 7fa346000640 1 -- 172.21.15.17:0/494027899 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3400bd040 con 0x7fa350152c70 2024-09-10T21:43:54.281 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-10T21:43:54.282 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:43:54.282 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:43:54.282 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "leader", 2024-09-10T21:43:54.282 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 148, 2024-09-10T21:43:54.282 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 93403, 2024-09-10T21:43:54.282 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-10T21:43:54.282 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-10T21:43:54.282 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-10T21:43:54.282 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:43:54.282 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 1, 2024-09-10T21:43:54.282 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:43:54.282 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-10T21:43:54.283 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-10T21:43:54.283 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:43:54.283 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:43:54.283 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:43:54.283 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:43:54.283 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:43:54.283 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:43:54.283 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:43:54.283 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:43:54.283 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:43:54.283 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:43:54.283 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:43:54.283 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:43:54.283 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-10T21:43:54.284 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-10T21:43:54.284 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:43:54.284 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:43:54.284 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:43:54.284 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:43:54.284 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:43:54.284 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:43:54.284 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:43:54.284 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:43:54.284 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:43:54.284 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:43:54.284 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:43:54.284 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:43:54.285 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:43:54.285 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-10T21:43:54.285 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-10T21:43:54.285 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-10T21:43:54.285 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-10T21:43:54.285 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-10T21:43:54.285 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "5e2f023f-d6a6-49f4-a256-f4807dce3739", 2024-09-10T21:43:54.285 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:43:54.285 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:43:54.285 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-10T21:43:54.285 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-10T21:43:54.285 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-10T21:43:54.285 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-10T21:43:54.286 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-10T21:43:54.286 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-10T21:43:54.286 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-10T21:43:54.286 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:43:54.286 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-10T21:43:54.286 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:43:54.286 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:43:54.286 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:43:54.286 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:43:54.286 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:43:54.286 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:43:54.286 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:43:54.286 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:43:54.286 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:43:54.286 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:43:54.287 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:43:54.287 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:43:54.287 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-10T21:43:54.287 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:43:54.287 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-10T21:43:54.287 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:43:54.287 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:43:54.287 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:43:54.287 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:43:54.287 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:43:54.287 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:43:54.287 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:43:54.287 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-10T21:43:54.288 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:43:54.288 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:43:54.288 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:43:54.288 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:43:54.288 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-10T21:43:54.288 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:43:54.288 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:43:54.288 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:43:54.288 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:43:54.288 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-10T21:43:54.288 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-10T21:43:54.288 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:43:54.288 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:43:54.288 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:43:54.289 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:43:54.289 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:43:54.289 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:43:54.289 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:43:54.289 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:43:54.289 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:43:54.289 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:43:54.289 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:43:54.289 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3300", 2024-09-10T21:43:54.289 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:43:54.289 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:43:54.289 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:43:54.289 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:43:54.290 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789", 2024-09-10T21:43:54.290 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:43:54.290 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:43:54.290 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:43:54.290 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:43:54.290 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789/0", 2024-09-10T21:43:54.290 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6789/0", 2024-09-10T21:43:54.290 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:43:54.290 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:43:54.290 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:43:54.290 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:43:54.290 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:43:54.290 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:43:54.291 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:43:54.291 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:43:54.291 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:43:54.291 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:43:54.291 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:43:54.291 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3301", 2024-09-10T21:43:54.291 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:43:54.291 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:43:54.291 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:43:54.291 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:43:54.291 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790", 2024-09-10T21:43:54.291 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:43:54.291 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:43:54.291 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:43:54.292 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:43:54.292 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790/0", 2024-09-10T21:43:54.292 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6790/0", 2024-09-10T21:43:54.292 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:43:54.292 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:43:54.292 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:43:54.292 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:43:54.292 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:43:54.292 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:43:54.292 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-10T21:43:54.292 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-10T21:43:54.292 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:43:54.292 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:43:54.292 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:43:54.293 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:43:54.293 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:43:54.293 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:43:54.293 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": [ 2024-09-10T21:43:54.293 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:43:54.293 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:43:54.293 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:43:54.293 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 4 2024-09-10T21:43:54.293 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:43:54.293 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:43:54.293 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-10T21:43:54.293 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:43:54.293 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:43:54.293 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:43:54.294 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 3 2024-09-10T21:43:54.294 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:43:54.294 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:43:54.294 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-10T21:43:54.294 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:43:54.294 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:43:54.294 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:43:54.294 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 3 2024-09-10T21:43:54.294 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:43:54.294 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:43:54.294 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": [ 2024-09-10T21:43:54.294 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:43:54.294 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:43:54.294 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:43:54.294 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:43:54.294 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:43:54.295 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:43:54.295 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:43:54.295 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-10T21:43:54.295 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-10T21:43:54.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:54.275+0000 7fa346000640 1 -- 172.21.15.17:0/494027899 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5033 (secure 0 0 0) 0x7fa3400c3810 con 0x7fa350152c70 2024-09-10T21:43:54.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:54.275+0000 7fa346000640 1 -- 172.21.15.17:0/494027899 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa350152c70 msgr2=0x7fa3500aa4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:54.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:54.275+0000 7fa346000640 1 --2- 172.21.15.17:0/494027899 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa350152c70 0x7fa3500aa4f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa340009230 tx=0x7fa34001eef0 comp rx=0 tx=0).stop 2024-09-10T21:43:54.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:54.275+0000 7fa346000640 1 -- 172.21.15.17:0/494027899 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa350152c70 msgr2=0x7fa3500aa4f0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:43:54.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:54.275+0000 7fa346000640 1 --2- 172.21.15.17:0/494027899 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa350152c70 0x7fa3500aa4f0 secure :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa340009230 tx=0x7fa34001eef0 comp rx=0 tx=0).stop 2024-09-10T21:43:54.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:54.279+0000 7fa354a00640 1 -- 172.21.15.17:0/494027899 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fa32c072590 msgr2=0x7fa32c074a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:54.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:54.279+0000 7fa354a00640 1 --2- 172.21.15.17:0/494027899 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fa32c072590 0x7fa32c074a50 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fa350154260 tx=0x7fa3380095e0 comp rx=0 tx=0).stop 2024-09-10T21:43:54.296 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:54.279+0000 7fa354a00640 1 -- 172.21.15.17:0/494027899 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa350154400 msgr2=0x7fa3501570d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:54.297 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:54.279+0000 7fa354a00640 1 --2- 172.21.15.17:0/494027899 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa350154400 0x7fa3501570d0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fa34800b9c0 tx=0x7fa34800be80 comp rx=0 tx=0).stop 2024-09-10T21:43:54.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:54.283+0000 7fa34e800640 1 -- 172.21.15.17:0/494027899 reap_dead start 2024-09-10T21:43:54.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:54.283+0000 7fa34e800640 1 -- 172.21.15.17:0/494027899 reap_dead start 2024-09-10T21:43:54.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:54.287+0000 7fa354a00640 1 -- 172.21.15.17:0/494027899 shutdown_connections 2024-09-10T21:43:54.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:54.287+0000 7fa354a00640 1 -- 172.21.15.17:0/494027899 >> 172.21.15.17:0/494027899 conn(0x7fa3500700d0 msgr2=0x7fa350073ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:43:54.304 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:54.299+0000 7fa354a00640 1 -- 172.21.15.17:0/494027899 shutdown_connections 2024-09-10T21:43:54.305 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:54.303+0000 7fa354a00640 1 -- 172.21.15.17:0/494027899 wait complete. 2024-09-10T21:43:54.389 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-09-10T21:43:54.390 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon c 2024-09-10T21:43:54.390 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-10T21:43:54.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:54.719+0000 7f3e99400640 1 -- 172.21.15.17:0/1548404657 <== mon.2 v2:172.21.15.31:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e840b2070 con 0x7f3e9c0c9080 2024-09-10T21:43:54.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:54.723+0000 7f3ea1200640 1 -- 172.21.15.17:0/1548404657 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3e9c0c9080 msgr2=0x7f3e9c157260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:54.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:54.723+0000 7f3ea1200640 1 --2- 172.21.15.17:0/1548404657 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3e9c0c9080 0x7f3e9c157260 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f3e84009900 tx=0x7f3e8401c8e0 comp rx=0 tx=0).stop 2024-09-10T21:43:54.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:54.723+0000 7f3ea1200640 1 -- 172.21.15.17:0/1548404657 shutdown_connections 2024-09-10T21:43:54.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:54.723+0000 7f3ea1200640 1 --2- 172.21.15.17:0/1548404657 >> v2:172.21.15.17:3300/0 conn(0x7f3e9c0b1b50 0x7f3e9c074d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:54.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:54.723+0000 7f3ea1200640 1 --2- 172.21.15.17:0/1548404657 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3e9c0c9080 0x7f3e9c157260 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:54.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:54.723+0000 7f3ea1200640 1 -- 172.21.15.17:0/1548404657 >> 172.21.15.17:0/1548404657 conn(0x7f3e9c0700d0 msgr2=0x7f3e9c0704e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:43:54.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:54.723+0000 7f3ea1200640 1 -- 172.21.15.17:0/1548404657 shutdown_connections 2024-09-10T21:43:54.726 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:54.723+0000 7f3ea1200640 1 -- 172.21.15.17:0/1548404657 wait complete. 2024-09-10T21:43:54.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:54.723+0000 7f3ea1200640 1 Processor -- start 2024-09-10T21:43:54.738 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:54.731+0000 7f3ea1200640 1 -- start start 2024-09-10T21:43:54.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:54.735+0000 7f3ea1200640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3e9c0b1b50 0x7f3e9c1566b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:54.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:54.735+0000 7f3ea1200640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3e9c0c8ca0 0x7f3e9c0c1b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:54.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:54.735+0000 7f3ea1200640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e9c0c9080 0x7f3e9c0c31c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:54.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:54.735+0000 7f3e91400640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3e9c0c8ca0 0x7f3e9c0c1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:54.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:54.735+0000 7f3e91400640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3e9c0c8ca0 0x7f3e9c0c1b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3301/0 says I am v2:172.21.15.17:48082/0 (socket says 172.21.15.17:48082) 2024-09-10T21:43:54.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:54.735+0000 7f3e99e00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3e9c0b1b50 0x7f3e9c1566b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:54.739 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:54.735+0000 7f3e99e00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3e9c0b1b50 0x7f3e9c1566b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:35692/0 (socket says 172.21.15.17:35692) 2024-09-10T21:43:54.742 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:54.739+0000 7f3ea1200640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f3e9c163e00 con 0x7f3e9c0b1b50 2024-09-10T21:43:54.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:54.739+0000 7f3ea1200640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f3e9c0c87c0 con 0x7f3e9c0c9080 2024-09-10T21:43:54.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:54.739+0000 7f3ea1200640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f3e9c163f80 con 0x7f3e9c0c8ca0 2024-09-10T21:43:54.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:54.739+0000 7f3e91400640 1 -- 172.21.15.17:0/4166820534 learned_addr learned my addr 172.21.15.17:0/4166820534 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:43:54.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:54.739+0000 7f3e9a800640 1 -- 172.21.15.17:0/4166820534 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e9c0c9080 msgr2=0x7f3e9c0c31c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:3300/0 2024-09-10T21:43:54.743 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:54.739+0000 7f3e9a800640 1 --2- 172.21.15.17:0/4166820534 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e9c0c9080 0x7f3e9c0c31c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:43:54.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:54.739+0000 7f3e91400640 1 -- 172.21.15.17:0/4166820534 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e9c0c9080 msgr2=0x7f3e9c0c31c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:43:54.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:54.739+0000 7f3e91400640 1 --2- 172.21.15.17:0/4166820534 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3e9c0c9080 0x7f3e9c0c31c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:54.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:54.739+0000 7f3e91400640 1 -- 172.21.15.17:0/4166820534 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3e9c0b1b50 msgr2=0x7f3e9c1566b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:54.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:54.739+0000 7f3e91400640 1 --2- 172.21.15.17:0/4166820534 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3e9c0b1b50 0x7f3e9c1566b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:43:54.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:54.739+0000 7f3e91400640 1 -- 172.21.15.17:0/4166820534 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e9c2007c0 con 0x7f3e9c0c8ca0 2024-09-10T21:43:54.744 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:54.743+0000 7f3e99e00640 1 --2- 172.21.15.17:0/4166820534 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3e9c0b1b50 0x7f3e9c1566b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2024-09-10T21:43:54.745 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:54.743+0000 7f3e91400640 1 --2- 172.21.15.17:0/4166820534 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3e9c0c8ca0 0x7f3e9c0c1b20 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f3e840b0040 tx=0x7f3e84005de0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:54.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:54.743+0000 7f3e92a00640 1 -- 172.21.15.17:0/4166820534 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e840b2040 con 0x7f3e9c0c8ca0 2024-09-10T21:43:54.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:54.743+0000 7f3e92a00640 1 -- 172.21.15.17:0/4166820534 <== mon.2 v2:172.21.15.31:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3e84004330 con 0x7f3e9c0c8ca0 2024-09-10T21:43:54.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:54.743+0000 7f3e92a00640 1 -- 172.21.15.17:0/4166820534 <== mon.2 v2:172.21.15.31:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e84005220 con 0x7f3e9c0c8ca0 2024-09-10T21:43:54.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:54.743+0000 7f3ea1200640 1 -- 172.21.15.17:0/4166820534 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e9c200aa0 con 0x7f3e9c0c8ca0 2024-09-10T21:43:54.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:54.743+0000 7f3ea1200640 1 -- 172.21.15.17:0/4166820534 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e9c0c2c50 con 0x7f3e9c0c8ca0 2024-09-10T21:43:54.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:54.743+0000 7f3ea1200640 1 --2- 172.21.15.17:0/4166820534 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3e9c0c3700 0x7f3e9c0c3f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:54.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:54.743+0000 7f3ea1200640 1 -- 172.21.15.17:0/4166820534 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f3e9c0c2fc0 con 0x7f3e9c0c3700 2024-09-10T21:43:54.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:54.747+0000 7f3e99e00640 1 --2- 172.21.15.17:0/4166820534 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3e9c0c3700 0x7f3e9c0c3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:54.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:54.751+0000 7f3e99e00640 1 -- 172.21.15.17:0/4166820534 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3e9c074d40 con 0x7f3e9c0c3700 2024-09-10T21:43:54.755 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:54.751+0000 7f3e99e00640 1 --2- 172.21.15.17:0/4166820534 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3e9c0c3700 0x7f3e9c0c3f30 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f3e8c00d6f0 tx=0x7f3e8c00dbb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:54.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:54.755+0000 7f3e92a00640 1 -- 172.21.15.17:0/4166820534 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e8c005270 con 0x7f3e9c0c3700 2024-09-10T21:43:54.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:54.755+0000 7f3e92a00640 1 -- 172.21.15.17:0/4166820534 <== mon.2 v2:172.21.15.31:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f3e8c017070 con 0x7f3e9c0c3700 2024-09-10T21:43:54.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:54.755+0000 7f3e92a00640 1 -- 172.21.15.17:0/4166820534 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3e9c0c3700 msgr2=0x7f3e9c0c3f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:54.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:54.755+0000 7f3e92a00640 1 --2- 172.21.15.17:0/4166820534 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3e9c0c3700 0x7f3e9c0c3f30 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f3e8c00d6f0 tx=0x7f3e8c00dbb0 comp rx=0 tx=0).stop 2024-09-10T21:43:54.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:54.755+0000 7f3e92a00640 1 -- 172.21.15.17:0/4166820534 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3e9c0c3700 msgr2=0x7f3e9c0c3f30 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:43:54.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:54.755+0000 7f3e92a00640 1 --2- 172.21.15.17:0/4166820534 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3e9c0c3700 0x7f3e9c0c3f30 secure :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f3e8c00d6f0 tx=0x7f3e8c00dbb0 comp rx=0 tx=0).stop 2024-09-10T21:43:54.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:54.759+0000 7f3e92a00640 1 -- 172.21.15.17:0/4166820534 <== mon.2 v2:172.21.15.31:3301/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f3e9c151600 con 0x7f3e9c0c8ca0 2024-09-10T21:43:54.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:54.759+0000 7f3e92a00640 1 --2- 172.21.15.17:0/4166820534 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f3e780720f0 0x7f3e780745b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:54.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:54.763+0000 7f3e92a00640 1 -- 172.21.15.17:0/4166820534 <== mon.2 v2:172.21.15.31:3301/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f3e8412d2d0 con 0x7f3e9c0c8ca0 2024-09-10T21:43:54.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:54.775+0000 7f3e99e00640 1 --2- 172.21.15.17:0/4166820534 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f3e780720f0 0x7f3e780745b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:54.779 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:54.775+0000 7f3e99e00640 1 --2- 172.21.15.17:0/4166820534 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f3e780720f0 0x7f3e780745b0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f3e8c00b4f0 tx=0x7f3e8c005000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:54.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:54.779+0000 7f3ea1200640 1 --2- 172.21.15.17:0/4166820534 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3e9c0c4470 0x7f3e9c203190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:43:54.782 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:54.779+0000 7f3ea1200640 1 -- 172.21.15.17:0/4166820534 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f3e9c0c9670 con 0x7f3e9c0c4470 2024-09-10T21:43:54.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:54.787+0000 7f3e9a800640 1 --2- 172.21.15.17:0/4166820534 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3e9c0c4470 0x7f3e9c203190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:43:54.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:54.787+0000 7f3e9a800640 1 -- 172.21.15.17:0/4166820534 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f3e9c156bf0 con 0x7f3e9c0c4470 2024-09-10T21:43:54.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:54.787+0000 7f3e9a800640 1 --2- 172.21.15.17:0/4166820534 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3e9c0c4470 0x7f3e9c203190 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f3e940098c0 tx=0x7f3e9400ef50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:43:54.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:54.787+0000 7f3e92a00640 1 -- 172.21.15.17:0/4166820534 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e9400ce30 con 0x7f3e9c0c4470 2024-09-10T21:43:54.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:54.791+0000 7f3e92a00640 1 -- 172.21.15.17:0/4166820534 <== mon.2 v2:172.21.15.31:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5031 (secure 0 0 0) 0x7f3e9400ce30 con 0x7f3e9c0c4470 2024-09-10T21:43:54.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:54.791+0000 7f3e92a00640 1 -- 172.21.15.17:0/4166820534 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3e9c0c4470 msgr2=0x7f3e9c203190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:54.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:54.791+0000 7f3e92a00640 1 --2- 172.21.15.17:0/4166820534 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3e9c0c4470 0x7f3e9c203190 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f3e940098c0 tx=0x7f3e9400ef50 comp rx=0 tx=0).stop 2024-09-10T21:43:54.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:54.791+0000 7f3e92a00640 1 -- 172.21.15.17:0/4166820534 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3e9c0c4470 msgr2=0x7f3e9c203190 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:43:54.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:54.791+0000 7f3e92a00640 1 --2- 172.21.15.17:0/4166820534 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3e9c0c4470 0x7f3e9c203190 secure :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f3e940098c0 tx=0x7f3e9400ef50 comp rx=0 tx=0).stop 2024-09-10T21:43:54.794 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-10T21:43:54.795 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:43:54.795 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:43:54.795 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-10T21:43:54.795 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 148, 2024-09-10T21:43:54.795 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 36915, 2024-09-10T21:43:54.795 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-10T21:43:54.795 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-10T21:43:54.795 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-10T21:43:54.795 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:43:54.795 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 1, 2024-09-10T21:43:54.795 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:43:54.795 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-10T21:43:54.796 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-10T21:43:54.796 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:43:54.796 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:43:54.796 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:43:54.796 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:43:54.796 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:43:54.796 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:43:54.796 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:43:54.796 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:43:54.796 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:43:54.796 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:43:54.796 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:43:54.797 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:43:54.797 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-10T21:43:54.797 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-10T21:43:54.797 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:43:54.797 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:43:54.797 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:43:54.797 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:43:54.797 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:43:54.797 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:43:54.797 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:43:54.797 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:43:54.797 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:43:54.797 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:43:54.797 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:43:54.798 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:43:54.798 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:43:54.798 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-10T21:43:54.798 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-10T21:43:54.798 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-10T21:43:54.798 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-10T21:43:54.798 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-10T21:43:54.798 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "5e2f023f-d6a6-49f4-a256-f4807dce3739", 2024-09-10T21:43:54.798 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:43:54.798 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:43:54.798 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-10T21:43:54.798 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-10T21:43:54.799 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-10T21:43:54.799 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-10T21:43:54.799 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-10T21:43:54.799 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-10T21:43:54.799 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-10T21:43:54.799 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:43:54.799 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-10T21:43:54.799 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:43:54.799 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:43:54.799 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:43:54.799 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:43:54.799 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:43:54.799 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:43:54.800 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:43:54.800 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:43:54.800 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:43:54.800 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:43:54.800 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:43:54.800 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:43:54.800 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-10T21:43:54.800 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:43:54.800 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-10T21:43:54.800 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:43:54.801 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:43:54.801 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:43:54.801 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:43:54.801 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:43:54.801 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:43:54.801 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:43:54.801 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-10T21:43:54.801 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:43:54.801 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:43:54.801 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:43:54.801 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:43:54.801 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-10T21:43:54.802 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:43:54.802 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:43:54.802 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:43:54.802 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:43:54.802 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-10T21:43:54.802 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-10T21:43:54.802 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:43:54.802 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:43:54.802 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:43:54.802 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:43:54.802 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:43:54.802 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:43:54.802 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:43:54.803 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:43:54.803 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:43:54.803 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:43:54.803 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:43:54.803 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3300", 2024-09-10T21:43:54.803 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:43:54.803 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:43:54.803 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:43:54.803 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:43:54.803 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789", 2024-09-10T21:43:54.803 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:43:54.803 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:43:54.803 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:43:54.803 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:43:54.804 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789/0", 2024-09-10T21:43:54.804 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6789/0", 2024-09-10T21:43:54.804 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:43:54.804 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:43:54.804 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:43:54.804 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:43:54.804 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:43:54.804 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:43:54.804 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:43:54.805 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:43:54.805 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:43:54.805 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:43:54.805 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:43:54.805 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3301", 2024-09-10T21:43:54.805 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:43:54.805 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:43:54.805 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:43:54.805 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:43:54.805 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790", 2024-09-10T21:43:54.805 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:43:54.805 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:43:54.805 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:43:54.805 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:43:54.806 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790/0", 2024-09-10T21:43:54.806 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6790/0", 2024-09-10T21:43:54.806 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:43:54.806 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:43:54.806 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:43:54.806 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:43:54.806 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:43:54.806 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:43:54.806 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-10T21:43:54.806 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-10T21:43:54.806 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:43:54.806 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:43:54.806 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:43:54.807 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:43:54.807 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:43:54.807 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:43:54.807 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": [ 2024-09-10T21:43:54.807 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:43:54.807 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:43:54.807 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:43:54.807 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 2 2024-09-10T21:43:54.807 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:43:54.807 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:43:54.808 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-10T21:43:54.808 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:43:54.808 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:43:54.808 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:43:54.808 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 5 2024-09-10T21:43:54.808 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:43:54.808 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:43:54.808 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-10T21:43:54.808 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:43:54.808 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:43:54.808 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:43:54.808 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 6 2024-09-10T21:43:54.809 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:43:54.809 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:43:54.809 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": [ 2024-09-10T21:43:54.809 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:43:54.809 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:43:54.809 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:43:54.809 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:43:54.809 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:43:54.809 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:43:54.809 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:43:54.809 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-10T21:43:54.809 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-10T21:43:54.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:54.795+0000 7f3ea1200640 1 -- 172.21.15.17:0/4166820534 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f3e780720f0 msgr2=0x7f3e780745b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:54.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:54.795+0000 7f3ea1200640 1 --2- 172.21.15.17:0/4166820534 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f3e780720f0 0x7f3e780745b0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f3e8c00b4f0 tx=0x7f3e8c005000 comp rx=0 tx=0).stop 2024-09-10T21:43:54.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:54.795+0000 7f3ea1200640 1 -- 172.21.15.17:0/4166820534 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3e9c0c8ca0 msgr2=0x7f3e9c0c1b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:43:54.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:54.795+0000 7f3ea1200640 1 --2- 172.21.15.17:0/4166820534 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3e9c0c8ca0 0x7f3e9c0c1b20 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f3e840b0040 tx=0x7f3e84005de0 comp rx=0 tx=0).stop 2024-09-10T21:43:54.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:54.795+0000 7f3e9a800640 1 -- 172.21.15.17:0/4166820534 reap_dead start 2024-09-10T21:43:54.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:54.799+0000 7f3ea1200640 1 -- 172.21.15.17:0/4166820534 shutdown_connections 2024-09-10T21:43:54.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:54.799+0000 7f3ea1200640 1 -- 172.21.15.17:0/4166820534 >> 172.21.15.17:0/4166820534 conn(0x7f3e9c0700d0 msgr2=0x7f3e9c14b1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:43:54.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:54.799+0000 7f3ea1200640 1 -- 172.21.15.17:0/4166820534 shutdown_connections 2024-09-10T21:43:54.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:43:54.799+0000 7f3ea1200640 1 -- 172.21.15.17:0/4166820534 wait complete. 2024-09-10T21:43:54.871 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-09-10T21:43:54.872 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2024-09-10T21:43:56.994 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:43:56.999 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:44:14.872 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.b 2024-09-10T21:44:14.872 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-10T21:44:14.873 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-10T21:44:14.878 INFO:tasks.ceph.mon.b:Started 2024-09-10T21:44:14.878 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-09-10T21:44:14.878 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-10T21:44:15.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:15.191+0000 7f2c8a000640 1 -- 172.21.15.17:0/2825793724 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f2c68005220 msgr2=0x7f2c6800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:15.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:15.191+0000 7f2c8a000640 1 --2- 172.21.15.17:0/2825793724 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f2c68005220 0x7f2c6800b9d0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f2c7c009df0 tx=0x7f2c7c01d170 comp rx=0 tx=0).stop 2024-09-10T21:44:15.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:15.191+0000 7f2c8a000640 1 -- 172.21.15.17:0/2825793724 shutdown_connections 2024-09-10T21:44:15.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:15.191+0000 7f2c8a000640 1 --2- 172.21.15.17:0/2825793724 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f2c68005220 0x7f2c6800b9d0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:15.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:15.191+0000 7f2c8a000640 1 -- 172.21.15.17:0/2825793724 >> 172.21.15.17:0/2825793724 conn(0x7f2c84068990 msgr2=0x7f2c84068da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:44:15.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:15.191+0000 7f2c8a000640 1 -- 172.21.15.17:0/2825793724 shutdown_connections 2024-09-10T21:44:15.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:15.199+0000 7f2c8a000640 1 -- 172.21.15.17:0/2825793724 wait complete. 2024-09-10T21:44:15.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:15.199+0000 7f2c8a000640 1 Processor -- start 2024-09-10T21:44:15.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:15.207+0000 7f2c8a000640 1 -- start start 2024-09-10T21:44:15.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:15.211+0000 7f2c8a000640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c68005220 0x7f2c84153280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:15.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:15.211+0000 7f2c8a000640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2c840cf1f0 0x7f2c841505c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:15.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:15.211+0000 7f2c8a000640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f2c842010f0 0x7f2c84201570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:15.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:15.211+0000 7f2c8a000640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f2c840cacc0 con 0x7f2c840cf1f0 2024-09-10T21:44:15.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:15.211+0000 7f2c8a000640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f2c840cab40 con 0x7f2c68005220 2024-09-10T21:44:15.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:15.211+0000 7f2c8a000640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f2c840cae40 con 0x7f2c842010f0 2024-09-10T21:44:15.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:15.211+0000 7f2c83400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c68005220 0x7f2c84153280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:15.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:15.211+0000 7f2c83400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c68005220 0x7f2c84153280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.17:38674/0 (socket says 172.21.15.17:38674) 2024-09-10T21:44:15.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:15.211+0000 7f2c83400640 1 -- 172.21.15.17:0/2700034435 learned_addr learned my addr 172.21.15.17:0/2700034435 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:44:15.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:15.211+0000 7f2c83400640 1 -- 172.21.15.17:0/2700034435 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c68005220 msgr2=0x7f2c84153280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-09-10T21:44:15.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:15.211+0000 7f2c83400640 1 -- 172.21.15.17:0/2700034435 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c68005220 msgr2=0x7f2c84153280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-10T21:44:15.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:15.211+0000 7f2c83400640 1 --2- 172.21.15.17:0/2700034435 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c68005220 0x7f2c84153280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T21:44:15.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:15.211+0000 7f2c83400640 1 --2- 172.21.15.17:0/2700034435 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c68005220 0x7f2c84153280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:44:15.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:15.211+0000 7f2c82a00640 1 --2- 172.21.15.17:0/2700034435 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2c840cf1f0 0x7f2c841505c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:15.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:15.215+0000 7f2c82a00640 1 -- 172.21.15.17:0/2700034435 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f2c842010f0 msgr2=0x7f2c84201570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:15.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:15.215+0000 7f2c82a00640 1 --2- 172.21.15.17:0/2700034435 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f2c842010f0 0x7f2c84201570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:15.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:15.215+0000 7f2c82a00640 1 -- 172.21.15.17:0/2700034435 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c68005220 msgr2=0x7f2c84153280 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:44:15.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:15.215+0000 7f2c82a00640 1 --2- 172.21.15.17:0/2700034435 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c68005220 0x7f2c84153280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:15.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:15.215+0000 7f2c82a00640 1 -- 172.21.15.17:0/2700034435 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c84152440 con 0x7f2c840cf1f0 2024-09-10T21:44:15.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:15.215+0000 7f2c82a00640 1 --2- 172.21.15.17:0/2700034435 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2c840cf1f0 0x7f2c841505c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f2c84052940 tx=0x7f2c6c00c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:44:15.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:15.215+0000 7f2c7b400640 1 -- 172.21.15.17:0/2700034435 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c6c011790 con 0x7f2c840cf1f0 2024-09-10T21:44:15.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:15.215+0000 7f2c7b400640 1 -- 172.21.15.17:0/2700034435 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2c6c017070 con 0x7f2c840cf1f0 2024-09-10T21:44:15.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:15.215+0000 7f2c7b400640 1 -- 172.21.15.17:0/2700034435 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c6c011cc0 con 0x7f2c840cf1f0 2024-09-10T21:44:15.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:15.215+0000 7f2c8a000640 1 -- 172.21.15.17:0/2700034435 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c84151fa0 con 0x7f2c840cf1f0 2024-09-10T21:44:15.220 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:15.215+0000 7f2c8a000640 1 -- 172.21.15.17:0/2700034435 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c8414ecd0 con 0x7f2c840cf1f0 2024-09-10T21:44:15.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:15.219+0000 7f2c7b400640 1 -- 172.21.15.17:0/2700034435 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f2c6c01c460 con 0x7f2c840cf1f0 2024-09-10T21:44:15.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:15.219+0000 7f2c7b400640 1 --2- 172.21.15.17:0/2700034435 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f2c60072630 0x7f2c60074af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:15.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:15.219+0000 7f2c7b400640 1 -- 172.21.15.17:0/2700034435 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f2c6c093280 con 0x7f2c840cf1f0 2024-09-10T21:44:15.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:15.223+0000 7f2c83400640 1 --2- 172.21.15.17:0/2700034435 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f2c60072630 0x7f2c60074af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:15.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:15.223+0000 7f2c8a000640 1 -- 172.21.15.17:0/2700034435 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c840c53d0 con 0x7f2c840cf1f0 2024-09-10T21:44:15.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:15.223+0000 7f2c83400640 1 --2- 172.21.15.17:0/2700034435 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f2c60072630 0x7f2c60074af0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f2c74001060 tx=0x7f2c7400c040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:44:15.248 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:15.243+0000 7f2c7b400640 1 -- 172.21.15.17:0/2700034435 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f2c6c05e640 con 0x7f2c840cf1f0 2024-09-10T21:44:15.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:15.831+0000 7f2c8a000640 1 -- 172.21.15.17:0/2700034435 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f2c841e3430 con 0x7f2c840cf1f0 2024-09-10T21:44:15.839 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:15.835+0000 7f2c7b400640 1 -- 172.21.15.17:0/2700034435 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1552 (secure 0 0 0) 0x7f2c6c05df40 con 0x7f2c840cf1f0 2024-09-10T21:44:15.840 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":148,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"a","quorum_age":22,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5e2f023f-d6a6-49f4-a256-f4807dce3739","modified":"2024-09-10T21:23:46.892130Z","created":"2024-09-10T21:23:46.892130Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3301","nonce":0},{"type":"v1","addr":"172.21.15.31:6790","nonce":0}]},"addr":"172.21.15.31:6790/0","public_addr":"172.21.15.31:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-10T21:44:15.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:15.843+0000 7f2c8a000640 1 -- 172.21.15.17:0/2700034435 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f2c60072630 msgr2=0x7f2c60074af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:15.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:15.843+0000 7f2c8a000640 1 --2- 172.21.15.17:0/2700034435 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f2c60072630 0x7f2c60074af0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f2c74001060 tx=0x7f2c7400c040 comp rx=0 tx=0).stop 2024-09-10T21:44:15.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:15.843+0000 7f2c8a000640 1 -- 172.21.15.17:0/2700034435 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2c840cf1f0 msgr2=0x7f2c841505c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:15.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:15.843+0000 7f2c8a000640 1 --2- 172.21.15.17:0/2700034435 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2c840cf1f0 0x7f2c841505c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f2c84052940 tx=0x7f2c6c00c820 comp rx=0 tx=0).stop 2024-09-10T21:44:15.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:15.843+0000 7f2c8a000640 1 -- 172.21.15.17:0/2700034435 shutdown_connections 2024-09-10T21:44:15.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:15.843+0000 7f2c8a000640 1 --2- 172.21.15.17:0/2700034435 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f2c60072630 0x7f2c60074af0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:15.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:15.843+0000 7f2c8a000640 1 --2- 172.21.15.17:0/2700034435 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f2c842010f0 0x7f2c84201570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:15.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:15.843+0000 7f2c8a000640 1 --2- 172.21.15.17:0/2700034435 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2c840cf1f0 0x7f2c841505c0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:15.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:15.843+0000 7f2c8a000640 1 --2- 172.21.15.17:0/2700034435 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2c68005220 0x7f2c84153280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:15.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:15.843+0000 7f2c8a000640 1 -- 172.21.15.17:0/2700034435 >> 172.21.15.17:0/2700034435 conn(0x7f2c84068990 msgr2=0x7f2c840d0080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:44:15.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:15.843+0000 7f2c8a000640 1 -- 172.21.15.17:0/2700034435 shutdown_connections 2024-09-10T21:44:15.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:15.843+0000 7f2c8a000640 1 -- 172.21.15.17:0/2700034435 wait complete. 2024-09-10T21:44:18.967 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-10T21:44:19.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:19.323+0000 7fce1e800640 1 -- 172.21.15.17:0/854547498 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcdfc003a10 msgr2=0x7fcdfc003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:19.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:19.323+0000 7fce1e800640 1 --2- 172.21.15.17:0/854547498 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcdfc003a10 0x7fcdfc003e10 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fce04006fc0 tx=0x7fce0401f6d0 comp rx=0 tx=0).stop 2024-09-10T21:44:19.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:19.323+0000 7fce1e800640 1 -- 172.21.15.17:0/854547498 shutdown_connections 2024-09-10T21:44:19.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:19.323+0000 7fce1e800640 1 --2- 172.21.15.17:0/854547498 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fcdfc003a10 0x7fcdfc003e10 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:19.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:19.323+0000 7fce1e800640 1 -- 172.21.15.17:0/854547498 >> 172.21.15.17:0/854547498 conn(0x7fce1806fe60 msgr2=0x7fce18070270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:44:19.326 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:19.323+0000 7fce1e800640 1 -- 172.21.15.17:0/854547498 shutdown_connections 2024-09-10T21:44:19.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:19.323+0000 7fce1e800640 1 -- 172.21.15.17:0/854547498 wait complete. 2024-09-10T21:44:19.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:19.327+0000 7fce1e800640 1 Processor -- start 2024-09-10T21:44:19.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:19.327+0000 7fce1e800640 1 -- start start 2024-09-10T21:44:19.333 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:19.331+0000 7fce1e800640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcdfc003a10 0x7fce180c4640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:19.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:19.331+0000 7fce1e800640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fce180c28e0 0x7fce180c12f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:19.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:19.331+0000 7fce17400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcdfc003a10 0x7fce180c4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:19.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:19.331+0000 7fce17400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcdfc003a10 0x7fce180c4640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.17:38720/0 (socket says 172.21.15.17:38720) 2024-09-10T21:44:19.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:19.331+0000 7fce16a00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fce180c28e0 0x7fce180c12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:19.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:19.331+0000 7fce16a00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fce180c28e0 0x7fce180c12f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45158/0 (socket says 172.21.15.17:45158) 2024-09-10T21:44:19.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:19.335+0000 7fce17400640 1 -- 172.21.15.17:0/3869001440 learned_addr learned my addr 172.21.15.17:0/3869001440 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:44:19.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:19.335+0000 7fce1e800640 1 --2- 172.21.15.17:0/3869001440 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fce180c2d60 0x7fce180c1830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:19.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:19.335+0000 7fce1e800640 1 -- 172.21.15.17:0/3869001440 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fce1806e330 con 0x7fce180c28e0 2024-09-10T21:44:19.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:19.335+0000 7fce17e00640 1 --2- 172.21.15.17:0/3869001440 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fce180c2d60 0x7fce180c1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:19.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:19.335+0000 7fce1e800640 1 -- 172.21.15.17:0/3869001440 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fce1806e1b0 con 0x7fcdfc003a10 2024-09-10T21:44:19.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:19.335+0000 7fce1e800640 1 -- 172.21.15.17:0/3869001440 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7fce180c6860 con 0x7fce180c2d60 2024-09-10T21:44:19.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:19.335+0000 7fce16a00640 1 -- 172.21.15.17:0/3869001440 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fce180c2d60 msgr2=0x7fce180c1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:19.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:19.335+0000 7fce16a00640 1 --2- 172.21.15.17:0/3869001440 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fce180c2d60 0x7fce180c1830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:19.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:19.335+0000 7fce16a00640 1 -- 172.21.15.17:0/3869001440 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcdfc003a10 msgr2=0x7fce180c4640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:19.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:19.335+0000 7fce16a00640 1 --2- 172.21.15.17:0/3869001440 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcdfc003a10 0x7fce180c4640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:19.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:19.335+0000 7fce16a00640 1 -- 172.21.15.17:0/3869001440 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce180c1d70 con 0x7fce180c28e0 2024-09-10T21:44:19.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:19.335+0000 7fce17400640 1 --2- 172.21.15.17:0/3869001440 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcdfc003a10 0x7fce180c4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:44:19.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:19.335+0000 7fce16a00640 1 --2- 172.21.15.17:0/3869001440 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fce180c28e0 0x7fce180c12f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fce1000cb20 tx=0x7fce1000cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:44:19.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:19.339+0000 7fce0f400640 1 -- 172.21.15.17:0/3869001440 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce10016020 con 0x7fce180c28e0 2024-09-10T21:44:19.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:19.339+0000 7fce0f400640 1 -- 172.21.15.17:0/3869001440 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fce1000a850 con 0x7fce180c28e0 2024-09-10T21:44:19.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:19.339+0000 7fce0f400640 1 -- 172.21.15.17:0/3869001440 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce1000acc0 con 0x7fce180c28e0 2024-09-10T21:44:19.340 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:19.339+0000 7fce1e800640 1 -- 172.21.15.17:0/3869001440 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce1814fa20 con 0x7fce180c28e0 2024-09-10T21:44:19.341 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:19.339+0000 7fce1e800640 1 -- 172.21.15.17:0/3869001440 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce180c3390 con 0x7fce180c28e0 2024-09-10T21:44:19.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:19.343+0000 7fce0f400640 1 -- 172.21.15.17:0/3869001440 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fce1001b400 con 0x7fce180c28e0 2024-09-10T21:44:19.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:19.343+0000 7fce0cc00640 1 -- 172.21.15.17:0/3869001440 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce18156e70 con 0x7fce180c28e0 2024-09-10T21:44:19.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:19.343+0000 7fce0f400640 1 --2- 172.21.15.17:0/3869001440 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fcdf80726e0 0x7fcdf8074ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:19.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:19.347+0000 7fce0f400640 1 -- 172.21.15.17:0/3869001440 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fce10093af0 con 0x7fce180c28e0 2024-09-10T21:44:19.348 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:19.347+0000 7fce17400640 1 --2- 172.21.15.17:0/3869001440 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fcdf80726e0 0x7fcdf8074ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:19.357 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:19.355+0000 7fce17400640 1 --2- 172.21.15.17:0/3869001440 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fcdf80726e0 0x7fcdf8074ba0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fce0401dfa0 tx=0x7fce040b0040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:44:19.359 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:19.355+0000 7fce0f400640 1 -- 172.21.15.17:0/3869001440 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fce10020020 con 0x7fce180c28e0 2024-09-10T21:44:19.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:19.995+0000 7fce0cc00640 1 -- 172.21.15.17:0/3869001440 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fce181530e0 con 0x7fce180c28e0 2024-09-10T21:44:19.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:19.995+0000 7fce0f400640 1 -- 172.21.15.17:0/3869001440 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1557 (secure 0 0 0) 0x7fce10020020 con 0x7fce180c28e0 2024-09-10T21:44:20.005 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":150,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":1,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5e2f023f-d6a6-49f4-a256-f4807dce3739","modified":"2024-09-10T21:23:46.892130Z","created":"2024-09-10T21:23:46.892130Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3301","nonce":0},{"type":"v1","addr":"172.21.15.31:6790","nonce":0}]},"addr":"172.21.15.31:6790/0","public_addr":"172.21.15.31:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-10T21:44:20.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:20.011+0000 7fce0cc00640 1 -- 172.21.15.17:0/3869001440 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fcdf80726e0 msgr2=0x7fcdf8074ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:20.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:20.011+0000 7fce0cc00640 1 --2- 172.21.15.17:0/3869001440 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fcdf80726e0 0x7fcdf8074ba0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fce0401dfa0 tx=0x7fce040b0040 comp rx=0 tx=0).stop 2024-09-10T21:44:20.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:20.015+0000 7fce0cc00640 1 -- 172.21.15.17:0/3869001440 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fce180c28e0 msgr2=0x7fce180c12f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:20.015 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:20.015+0000 7fce0cc00640 1 --2- 172.21.15.17:0/3869001440 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fce180c28e0 0x7fce180c12f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fce1000cb20 tx=0x7fce1000cc10 comp rx=0 tx=0).stop 2024-09-10T21:44:20.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:20.015+0000 7fce0cc00640 1 -- 172.21.15.17:0/3869001440 shutdown_connections 2024-09-10T21:44:20.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:20.015+0000 7fce0cc00640 1 --2- 172.21.15.17:0/3869001440 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fcdf80726e0 0x7fcdf8074ba0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:20.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:20.015+0000 7fce0cc00640 1 --2- 172.21.15.17:0/3869001440 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fce180c2d60 0x7fce180c1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:20.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:20.015+0000 7fce0cc00640 1 --2- 172.21.15.17:0/3869001440 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fce180c28e0 0x7fce180c12f0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:20.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:20.015+0000 7fce0cc00640 1 --2- 172.21.15.17:0/3869001440 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fcdfc003a10 0x7fce180c4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:20.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:20.015+0000 7fce0cc00640 1 -- 172.21.15.17:0/3869001440 >> 172.21.15.17:0/3869001440 conn(0x7fce1806fe60 msgr2=0x7fce18151c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:44:20.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:20.015+0000 7fce0cc00640 1 -- 172.21.15.17:0/3869001440 shutdown_connections 2024-09-10T21:44:20.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:20.015+0000 7fce0cc00640 1 -- 172.21.15.17:0/3869001440 wait complete. 2024-09-10T21:44:20.110 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-09-10T21:44:20.110 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-09-10T21:44:20.110 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-10T21:44:20.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:20.475+0000 7f394fe00640 1 -- 172.21.15.17:0/3983549733 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f39480f6740 msgr2=0x7f394804a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:20.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:20.475+0000 7f3955600640 1 -- 172.21.15.17:0/3983549733 <== mon.2 v2:172.21.15.31:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f393c0b2070 con 0x7f39480f6740 2024-09-10T21:44:20.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:20.475+0000 7f394fe00640 1 --2- 172.21.15.17:0/3983549733 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f39480f6740 0x7f394804a8f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f393c009900 tx=0x7f393c01c8d0 comp rx=0 tx=0).stop 2024-09-10T21:44:20.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:20.475+0000 7f394fe00640 1 -- 172.21.15.17:0/3983549733 shutdown_connections 2024-09-10T21:44:20.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:20.475+0000 7f394fe00640 1 --2- 172.21.15.17:0/3983549733 >> v2:172.21.15.17:3300/0 conn(0x7f3948044e70 0x7f394804ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:20.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:20.475+0000 7f394fe00640 1 --2- 172.21.15.17:0/3983549733 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f39480f6740 0x7f394804a8f0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:20.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:20.475+0000 7f394fe00640 1 -- 172.21.15.17:0/3983549733 >> 172.21.15.17:0/3983549733 conn(0x7f394804fde0 msgr2=0x7f39480501f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:44:20.478 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:20.475+0000 7f394fe00640 1 -- 172.21.15.17:0/3983549733 shutdown_connections 2024-09-10T21:44:20.479 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:20.475+0000 7f394fe00640 1 -- 172.21.15.17:0/3983549733 wait complete. 2024-09-10T21:44:20.481 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:20.479+0000 7f394fe00640 1 Processor -- start 2024-09-10T21:44:20.481 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:20.479+0000 7f394fe00640 1 -- start start 2024-09-10T21:44:20.483 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:20.483+0000 7f394fe00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3948033c30 0x7f39480fcba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:20.484 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:20.483+0000 7f394e000640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3948033c30 0x7f39480fcba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:20.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:20.483+0000 7f394e000640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3948033c30 0x7f39480fcba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:45166/0 (socket says 172.21.15.17:45166) 2024-09-10T21:44:20.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:20.483+0000 7f394fe00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3948044e70 0x7f39480f9470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:20.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:20.483+0000 7f394fe00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39480f6740 0x7f3948199730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:20.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:20.483+0000 7f394fe00640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f3948100ec0 con 0x7f3948033c30 2024-09-10T21:44:20.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:20.483+0000 7f394fe00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f3948100d40 con 0x7f39480f6740 2024-09-10T21:44:20.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:20.483+0000 7f394fe00640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f3948101040 con 0x7f3948044e70 2024-09-10T21:44:20.485 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:20.483+0000 7f394e000640 1 -- 172.21.15.17:0/2468153976 learned_addr learned my addr 172.21.15.17:0/2468153976 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:44:20.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:20.483+0000 7f394ea00640 1 --2- 172.21.15.17:0/2468153976 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39480f6740 0x7f3948199730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:20.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:20.483+0000 7f394d600640 1 --2- 172.21.15.17:0/2468153976 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3948044e70 0x7f39480f9470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:20.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:20.483+0000 7f394d600640 1 -- 172.21.15.17:0/2468153976 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39480f6740 msgr2=0x7f3948199730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:20.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:20.483+0000 7f394d600640 1 --2- 172.21.15.17:0/2468153976 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f39480f6740 0x7f3948199730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:20.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:20.483+0000 7f394d600640 1 -- 172.21.15.17:0/2468153976 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3948033c30 msgr2=0x7f39480fcba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:20.486 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:20.483+0000 7f394d600640 1 --2- 172.21.15.17:0/2468153976 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3948033c30 0x7f39480fcba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:20.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:20.487+0000 7f394d600640 1 -- 172.21.15.17:0/2468153976 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3948109b00 con 0x7f3948044e70 2024-09-10T21:44:20.489 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:20.487+0000 7f394d600640 1 --2- 172.21.15.17:0/2468153976 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3948044e70 0x7f39480f9470 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f393c0ac000 tx=0x7f393c004220 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:44:20.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:20.487+0000 7f3946a00640 1 -- 172.21.15.17:0/2468153976 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f393c0b2040 con 0x7f3948044e70 2024-09-10T21:44:20.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:20.487+0000 7f3946a00640 1 -- 172.21.15.17:0/2468153976 <== mon.2 v2:172.21.15.31:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f393c004410 con 0x7f3948044e70 2024-09-10T21:44:20.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:20.491+0000 7f3946a00640 1 -- 172.21.15.17:0/2468153976 <== mon.2 v2:172.21.15.31:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f393c005280 con 0x7f3948044e70 2024-09-10T21:44:20.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:20.491+0000 7f394fe00640 1 -- 172.21.15.17:0/2468153976 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3948049430 con 0x7f3948044e70 2024-09-10T21:44:20.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:20.491+0000 7f394fe00640 1 -- 172.21.15.17:0/2468153976 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f39480499b0 con 0x7f3948044e70 2024-09-10T21:44:20.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:20.495+0000 7f394fe00640 1 --2- 172.21.15.17:0/2468153976 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f394804afe0 0x7f3948199c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:20.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:20.495+0000 7f394fe00640 1 -- 172.21.15.17:0/2468153976 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f394804b4c0 con 0x7f394804afe0 2024-09-10T21:44:20.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:20.495+0000 7f394e000640 1 --2- 172.21.15.17:0/2468153976 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f394804afe0 0x7f3948199c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:20.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:20.499+0000 7f394e000640 1 -- 172.21.15.17:0/2468153976 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f394804efe0 con 0x7f394804afe0 2024-09-10T21:44:20.503 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:20.499+0000 7f394e000640 1 --2- 172.21.15.17:0/2468153976 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f394804afe0 0x7f3948199c70 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f394000f7a0 tx=0x7f394000fc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:44:20.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:20.503+0000 7f3946a00640 1 -- 172.21.15.17:0/2468153976 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39400050d0 con 0x7f394804afe0 2024-09-10T21:44:20.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:20.507+0000 7f3946a00640 1 -- 172.21.15.17:0/2468153976 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f3940018070 con 0x7f394804afe0 2024-09-10T21:44:20.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:20.507+0000 7f3946a00640 1 -- 172.21.15.17:0/2468153976 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f394804afe0 msgr2=0x7f3948199c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:20.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:20.507+0000 7f3946a00640 1 --2- 172.21.15.17:0/2468153976 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f394804afe0 0x7f3948199c70 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f394000f7a0 tx=0x7f394000fc60 comp rx=0 tx=0).stop 2024-09-10T21:44:20.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:20.507+0000 7f3946a00640 1 -- 172.21.15.17:0/2468153976 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f394804afe0 msgr2=0x7f3948199c70 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:44:20.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:20.507+0000 7f3946a00640 1 --2- 172.21.15.17:0/2468153976 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f394804afe0 0x7f3948199c70 secure :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f394000f7a0 tx=0x7f394000fc60 comp rx=0 tx=0).stop 2024-09-10T21:44:20.512 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:20.507+0000 7f3946a00640 1 -- 172.21.15.17:0/2468153976 <== mon.2 v2:172.21.15.31:3301/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f393c005420 con 0x7f3948044e70 2024-09-10T21:44:20.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:20.515+0000 7f3946a00640 1 --2- 172.21.15.17:0/2468153976 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f39300720c0 0x7f3930074580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:20.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:20.515+0000 7f3946a00640 1 -- 172.21.15.17:0/2468153976 <== mon.2 v2:172.21.15.31:3301/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f393c12d3f0 con 0x7f3948044e70 2024-09-10T21:44:20.524 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:20.523+0000 7f394e000640 1 --2- 172.21.15.17:0/2468153976 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f39300720c0 0x7f3930074580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:20.541 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:20.531+0000 7f394e000640 1 --2- 172.21.15.17:0/2468153976 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f39300720c0 0x7f3930074580 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f394000fca0 tx=0x7f3940005000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:44:20.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:20.575+0000 7f394fe00640 1 --2- 172.21.15.17:0/2468153976 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3948049bf0 0x7f39480ed6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:20.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:20.575+0000 7f394fe00640 1 -- 172.21.15.17:0/2468153976 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f3948008a30 con 0x7f3948049bf0 2024-09-10T21:44:20.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:20.579+0000 7f394ea00640 1 --2- 172.21.15.17:0/2468153976 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3948049bf0 0x7f39480ed6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:20.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:20.583+0000 7f394ea00640 1 -- 172.21.15.17:0/2468153976 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f39481097c0 con 0x7f3948049bf0 2024-09-10T21:44:20.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:20.583+0000 7f394ea00640 1 --2- 172.21.15.17:0/2468153976 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3948049bf0 0x7f39480ed6b0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f3950051d40 tx=0x7f3950064710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:44:20.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:20.591+0000 7f3946a00640 1 -- 172.21.15.17:0/2468153976 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f395006c8d0 con 0x7f3948049bf0 2024-09-10T21:44:20.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:20.591+0000 7f3946a00640 1 -- 172.21.15.17:0/2468153976 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5045 (secure 0 0 0) 0x7f395007f070 con 0x7f3948049bf0 2024-09-10T21:44:20.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:20.591+0000 7f3946a00640 1 -- 172.21.15.17:0/2468153976 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3948049bf0 msgr2=0x7f39480ed6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:20.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:20.591+0000 7f3946a00640 1 --2- 172.21.15.17:0/2468153976 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3948049bf0 0x7f39480ed6b0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f3950051d40 tx=0x7f3950064710 comp rx=0 tx=0).stop 2024-09-10T21:44:20.598 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:20.591+0000 7f3946a00640 1 -- 172.21.15.17:0/2468153976 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3948049bf0 msgr2=0x7f39480ed6b0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:44:20.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:20.591+0000 7f3946a00640 1 --2- 172.21.15.17:0/2468153976 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3948049bf0 0x7f39480ed6b0 secure :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f3950051d40 tx=0x7f3950064710 comp rx=0 tx=0).stop 2024-09-10T21:44:20.599 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-10T21:44:20.599 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:44:20.599 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:44:20.599 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "leader", 2024-09-10T21:44:20.599 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 150, 2024-09-10T21:44:20.599 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 119722, 2024-09-10T21:44:20.599 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-10T21:44:20.599 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-10T21:44:20.600 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-10T21:44:20.600 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-10T21:44:20.600 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:44:20.600 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 1, 2024-09-10T21:44:20.600 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:44:20.600 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-10T21:44:20.600 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-10T21:44:20.600 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:44:20.600 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:44:20.600 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:44:20.600 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:44:20.600 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:44:20.600 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:44:20.601 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:44:20.601 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:44:20.601 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:44:20.601 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:44:20.601 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:44:20.601 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:44:20.601 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-10T21:44:20.601 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-10T21:44:20.601 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:44:20.601 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:44:20.601 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:44:20.601 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:44:20.601 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:44:20.601 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:44:20.602 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:44:20.602 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:44:20.602 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:44:20.602 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:44:20.602 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:44:20.602 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:44:20.602 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:44:20.602 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-10T21:44:20.602 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-10T21:44:20.602 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-10T21:44:20.602 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-10T21:44:20.602 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-10T21:44:20.602 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "5e2f023f-d6a6-49f4-a256-f4807dce3739", 2024-09-10T21:44:20.603 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:44:20.603 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:44:20.603 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-10T21:44:20.603 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-10T21:44:20.603 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-10T21:44:20.603 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-10T21:44:20.604 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-10T21:44:20.604 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-10T21:44:20.604 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-10T21:44:20.604 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:44:20.604 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-10T21:44:20.604 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:44:20.604 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:44:20.604 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:44:20.604 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:44:20.604 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:44:20.604 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:44:20.604 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:44:20.605 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:44:20.605 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:44:20.605 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:44:20.605 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:44:20.605 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:44:20.605 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-10T21:44:20.605 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:44:20.605 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-10T21:44:20.605 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:44:20.605 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:44:20.606 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:44:20.606 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:44:20.606 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:44:20.606 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:44:20.606 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:44:20.606 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-10T21:44:20.606 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:44:20.606 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:44:20.607 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:44:20.607 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:44:20.607 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-10T21:44:20.607 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:44:20.607 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:44:20.607 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:44:20.607 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:44:20.607 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-10T21:44:20.607 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-10T21:44:20.607 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:44:20.607 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:44:20.607 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:44:20.607 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:44:20.608 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:44:20.608 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:44:20.608 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:44:20.608 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:44:20.608 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:44:20.608 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:44:20.608 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:44:20.608 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3300", 2024-09-10T21:44:20.608 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:44:20.608 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:44:20.608 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:44:20.608 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:44:20.608 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789", 2024-09-10T21:44:20.609 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:44:20.609 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:44:20.610 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:44:20.610 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:44:20.610 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789/0", 2024-09-10T21:44:20.610 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6789/0", 2024-09-10T21:44:20.610 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:44:20.610 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:44:20.610 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:44:20.610 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:44:20.610 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:44:20.611 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:44:20.611 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:44:20.611 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:44:20.611 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:44:20.611 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:44:20.611 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:44:20.611 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3301", 2024-09-10T21:44:20.611 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:44:20.611 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:44:20.611 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:44:20.611 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:44:20.612 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790", 2024-09-10T21:44:20.612 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:44:20.612 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:44:20.612 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:44:20.612 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:44:20.612 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790/0", 2024-09-10T21:44:20.612 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6790/0", 2024-09-10T21:44:20.612 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:44:20.612 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:44:20.612 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:44:20.613 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:44:20.613 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:44:20.613 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:44:20.613 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-10T21:44:20.613 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-10T21:44:20.613 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:44:20.613 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:44:20.613 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:44:20.613 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:44:20.613 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:44:20.613 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:44:20.614 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": [ 2024-09-10T21:44:20.614 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:44:20.614 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:44:20.614 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:44:20.614 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 4 2024-09-10T21:44:20.614 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:44:20.614 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:44:20.614 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-10T21:44:20.614 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:44:20.614 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:44:20.614 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:44:20.614 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 3 2024-09-10T21:44:20.614 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:44:20.615 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:44:20.615 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-10T21:44:20.615 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:44:20.615 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:44:20.615 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:44:20.615 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 2 2024-09-10T21:44:20.615 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:44:20.615 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:44:20.615 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": [ 2024-09-10T21:44:20.615 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:44:20.615 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:44:20.615 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:44:20.615 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:44:20.615 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:44:20.615 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:44:20.616 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:44:20.616 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-10T21:44:20.616 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-10T21:44:20.616 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:20.595+0000 7f393be00640 1 -- 172.21.15.17:0/2468153976 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f39300720c0 msgr2=0x7f3930074580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:20.616 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:20.595+0000 7f393be00640 1 --2- 172.21.15.17:0/2468153976 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f39300720c0 0x7f3930074580 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f394000fca0 tx=0x7f3940005000 comp rx=0 tx=0).stop 2024-09-10T21:44:20.617 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:20.595+0000 7f393be00640 1 -- 172.21.15.17:0/2468153976 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3948044e70 msgr2=0x7f39480f9470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:20.617 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:20.595+0000 7f393be00640 1 --2- 172.21.15.17:0/2468153976 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3948044e70 0x7f39480f9470 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f393c0ac000 tx=0x7f393c004220 comp rx=0 tx=0).stop 2024-09-10T21:44:20.617 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:20.595+0000 7f394ea00640 1 -- 172.21.15.17:0/2468153976 reap_dead start 2024-09-10T21:44:20.617 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:20.595+0000 7f393be00640 1 -- 172.21.15.17:0/2468153976 shutdown_connections 2024-09-10T21:44:20.617 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:20.595+0000 7f393be00640 1 -- 172.21.15.17:0/2468153976 >> 172.21.15.17:0/2468153976 conn(0x7f394804fde0 msgr2=0x7f394804c7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:44:20.617 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:20.599+0000 7f393be00640 1 -- 172.21.15.17:0/2468153976 shutdown_connections 2024-09-10T21:44:20.617 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:20.603+0000 7f393be00640 1 -- 172.21.15.17:0/2468153976 wait complete. 2024-09-10T21:44:20.694 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-10T21:44:20.694 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-09-10T21:44:20.694 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-10T21:44:21.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:21.095+0000 7f3492600640 1 -- 172.21.15.17:0/605599716 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f348c0c0fd0 msgr2=0x7f348c0c6cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:21.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:21.095+0000 7f3492600640 1 --2- 172.21.15.17:0/605599716 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f348c0c0fd0 0x7f348c0c6cd0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f348c0c9b70 tx=0x7f3478022f90 comp rx=0 tx=0).stop 2024-09-10T21:44:21.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:21.095+0000 7f3492600640 1 -- 172.21.15.17:0/605599716 shutdown_connections 2024-09-10T21:44:21.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:21.095+0000 7f3492600640 1 --2- 172.21.15.17:0/605599716 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f348c0c0fd0 0x7f348c0c6cd0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:21.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:21.095+0000 7f3492600640 1 --2- 172.21.15.17:0/605599716 >> v2:172.21.15.31:3300/0 conn(0x7f348c153d40 0x7f348c0c8010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:21.097 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:21.095+0000 7f3492600640 1 -- 172.21.15.17:0/605599716 >> 172.21.15.17:0/605599716 conn(0x7f348c0700d0 msgr2=0x7f348c0704e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:44:21.098 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:21.095+0000 7f3492600640 1 -- 172.21.15.17:0/605599716 shutdown_connections 2024-09-10T21:44:21.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:21.095+0000 7f3492600640 1 -- 172.21.15.17:0/605599716 wait complete. 2024-09-10T21:44:21.104 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:21.099+0000 7f3492600640 1 Processor -- start 2024-09-10T21:44:21.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:21.103+0000 7f3492600640 1 -- start start 2024-09-10T21:44:21.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:21.107+0000 7f3492600640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f348c0c0fd0 0x7f348c1dde90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:21.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:21.107+0000 7f3492600640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f348c0c7210 0x7f348c1de3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:21.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:21.107+0000 7f3492600640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f348c153d40 0x7f348c1f3070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:21.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:21.107+0000 7f3492600640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f348c0c95a0 con 0x7f348c153d40 2024-09-10T21:44:21.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:21.107+0000 7f3492600640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f348c0c92c0 con 0x7f348c0c0fd0 2024-09-10T21:44:21.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:21.107+0000 7f3492600640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f348c154120 con 0x7f348c0c7210 2024-09-10T21:44:21.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:21.107+0000 7f348b400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f348c0c0fd0 0x7f348c1dde90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:21.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:21.107+0000 7f348aa00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f348c0c7210 0x7f348c1de3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:21.109 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:21.107+0000 7f348aa00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f348c0c7210 0x7f348c1de3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3301/0 says I am v2:172.21.15.17:50714/0 (socket says 172.21.15.17:50714) 2024-09-10T21:44:21.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:21.107+0000 7f348aa00640 1 -- 172.21.15.17:0/2090247624 learned_addr learned my addr 172.21.15.17:0/2090247624 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:44:21.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:21.107+0000 7f348b400640 1 --2- 172.21.15.17:0/2090247624 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f348c0c0fd0 0x7f348c1dde90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.17:38752/0 (socket says 172.21.15.17:38752) 2024-09-10T21:44:21.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:21.107+0000 7f348be00640 1 --2- 172.21.15.17:0/2090247624 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f348c153d40 0x7f348c1f3070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:21.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:21.107+0000 7f348aa00640 1 -- 172.21.15.17:0/2090247624 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f348c0c0fd0 msgr2=0x7f348c1dde90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:21.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:21.107+0000 7f348aa00640 1 --2- 172.21.15.17:0/2090247624 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f348c0c0fd0 0x7f348c1dde90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:21.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:21.107+0000 7f348aa00640 1 -- 172.21.15.17:0/2090247624 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f348c153d40 msgr2=0x7f348c1f3070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:21.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:21.107+0000 7f348aa00640 1 --2- 172.21.15.17:0/2090247624 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f348c153d40 0x7f348c1f3070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:21.110 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:21.107+0000 7f348aa00640 1 -- 172.21.15.17:0/2090247624 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f348c1fde60 con 0x7f348c0c7210 2024-09-10T21:44:21.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:21.107+0000 7f348aa00640 1 --2- 172.21.15.17:0/2090247624 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f348c0c7210 0x7f348c1de3d0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f3478021640 tx=0x7f3478005040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:44:21.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:21.107+0000 7f348b400640 1 --2- 172.21.15.17:0/2090247624 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f348c0c0fd0 0x7f348c1dde90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:44:21.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:21.107+0000 7f3483400640 1 -- 172.21.15.17:0/2090247624 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34780c1070 con 0x7f348c0c7210 2024-09-10T21:44:21.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:21.107+0000 7f3483400640 1 -- 172.21.15.17:0/2090247624 <== mon.2 v2:172.21.15.31:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f34780055c0 con 0x7f348c0c7210 2024-09-10T21:44:21.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:21.107+0000 7f3483400640 1 -- 172.21.15.17:0/2090247624 <== mon.2 v2:172.21.15.31:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34780048d0 con 0x7f348c0c7210 2024-09-10T21:44:21.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:21.107+0000 7f3492600640 1 -- 172.21.15.17:0/2090247624 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f348c1e1850 con 0x7f348c0c7210 2024-09-10T21:44:21.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:21.107+0000 7f3492600640 1 -- 172.21.15.17:0/2090247624 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f348c1e1d60 con 0x7f348c0c7210 2024-09-10T21:44:21.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:21.111+0000 7f3492600640 1 --2- 172.21.15.17:0/2090247624 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f348c1e1f60 0x7f348c1f35b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:21.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:21.111+0000 7f3492600640 1 -- 172.21.15.17:0/2090247624 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f348c0ca1f0 con 0x7f348c1e1f60 2024-09-10T21:44:21.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:21.111+0000 7f348b400640 1 --2- 172.21.15.17:0/2090247624 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f348c1e1f60 0x7f348c1f35b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:21.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:21.111+0000 7f348b400640 1 -- 172.21.15.17:0/2090247624 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f348c1fe360 con 0x7f348c1e1f60 2024-09-10T21:44:21.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:21.111+0000 7f348b400640 1 --2- 172.21.15.17:0/2090247624 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f348c1e1f60 0x7f348c1f35b0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f347c0087d0 tx=0x7f347c008c90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:44:21.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:21.111+0000 7f3483400640 1 -- 172.21.15.17:0/2090247624 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f347c007bd0 con 0x7f348c1e1f60 2024-09-10T21:44:21.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:21.111+0000 7f3483400640 1 -- 172.21.15.17:0/2090247624 <== mon.1 v2:172.21.15.31:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f347c019070 con 0x7f348c1e1f60 2024-09-10T21:44:21.116 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:21.111+0000 7f3483400640 1 -- 172.21.15.17:0/2090247624 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f348c1e1f60 msgr2=0x7f348c1f35b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:21.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:21.111+0000 7f3483400640 1 --2- 172.21.15.17:0/2090247624 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f348c1e1f60 0x7f348c1f35b0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f347c0087d0 tx=0x7f347c008c90 comp rx=0 tx=0).stop 2024-09-10T21:44:21.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:21.111+0000 7f3483400640 1 -- 172.21.15.17:0/2090247624 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f348c1e1f60 msgr2=0x7f348c1f35b0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:44:21.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:21.111+0000 7f3483400640 1 --2- 172.21.15.17:0/2090247624 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f348c1e1f60 0x7f348c1f35b0 secure :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f347c0087d0 tx=0x7f347c008c90 comp rx=0 tx=0).stop 2024-09-10T21:44:21.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:21.111+0000 7f3483400640 1 -- 172.21.15.17:0/2090247624 <== mon.2 v2:172.21.15.31:3301/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f3478004a70 con 0x7f348c0c7210 2024-09-10T21:44:21.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:21.115+0000 7f3483400640 1 --2- 172.21.15.17:0/2090247624 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f346c0720f0 0x7f346c0745b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:21.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:21.115+0000 7f3483400640 1 -- 172.21.15.17:0/2090247624 <== mon.2 v2:172.21.15.31:3301/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f34781307d0 con 0x7f348c0c7210 2024-09-10T21:44:21.117 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:21.115+0000 7f348b400640 1 --2- 172.21.15.17:0/2090247624 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f346c0720f0 0x7f346c0745b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:21.119 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:21.115+0000 7f348b400640 1 --2- 172.21.15.17:0/2090247624 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f346c0720f0 0x7f346c0745b0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f347c016040 tx=0x7f347c00c000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:44:21.128 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:21.127+0000 7f3492600640 1 --2- 172.21.15.17:0/2090247624 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f348c1dc890 0x7f348c1dcca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:21.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:21.127+0000 7f3492600640 1 -- 172.21.15.17:0/2090247624 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f348c0747d0 con 0x7f348c1dc890 2024-09-10T21:44:21.129 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:21.127+0000 7f348be00640 1 --2- 172.21.15.17:0/2090247624 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f348c1dc890 0x7f348c1dcca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:21.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:21.127+0000 7f348be00640 1 -- 172.21.15.17:0/2090247624 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f348c0c78a0 con 0x7f348c1dc890 2024-09-10T21:44:21.130 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:21.127+0000 7f348be00640 1 --2- 172.21.15.17:0/2090247624 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f348c1dc890 0x7f348c1dcca0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f348400f8e0 tx=0x7f348400fda0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:44:21.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:21.131+0000 7f3483400640 1 -- 172.21.15.17:0/2090247624 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3484018030 con 0x7f348c1dc890 2024-09-10T21:44:21.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:21.131+0000 7f3483400640 1 -- 172.21.15.17:0/2090247624 <== mon.1 v2:172.21.15.31:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4540 (secure 0 0 0) 0x7f3484006b60 con 0x7f348c1dc890 2024-09-10T21:44:21.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:21.131+0000 7f3483400640 1 -- 172.21.15.17:0/2090247624 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f348c1dc890 msgr2=0x7f348c1dcca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:21.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:21.131+0000 7f3483400640 1 --2- 172.21.15.17:0/2090247624 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f348c1dc890 0x7f348c1dcca0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f348400f8e0 tx=0x7f348400fda0 comp rx=0 tx=0).stop 2024-09-10T21:44:21.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:21.131+0000 7f3483400640 1 -- 172.21.15.17:0/2090247624 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f348c1dc890 msgr2=0x7f348c1dcca0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:44:21.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:21.131+0000 7f3483400640 1 --2- 172.21.15.17:0/2090247624 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f348c1dc890 0x7f348c1dcca0 secure :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f348400f8e0 tx=0x7f348400fda0 comp rx=0 tx=0).stop 2024-09-10T21:44:21.134 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-10T21:44:21.134 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:44:21.134 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:44:21.135 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-10T21:44:21.135 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 150, 2024-09-10T21:44:21.135 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 6107, 2024-09-10T21:44:21.135 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-10T21:44:21.135 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-10T21:44:21.135 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-10T21:44:21.135 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-10T21:44:21.135 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:44:21.136 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 2, 2024-09-10T21:44:21.136 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:44:21.136 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-10T21:44:21.136 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-10T21:44:21.136 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:44:21.136 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:44:21.136 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:44:21.136 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:44:21.136 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:44:21.136 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:44:21.136 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:44:21.137 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:44:21.137 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:44:21.137 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:44:21.137 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:44:21.137 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:44:21.137 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-10T21:44:21.137 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-10T21:44:21.137 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:44:21.137 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:44:21.137 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:44:21.138 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:44:21.138 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:44:21.138 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:44:21.138 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:44:21.138 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:44:21.138 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:44:21.138 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:44:21.138 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:44:21.138 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:44:21.138 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:44:21.138 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-10T21:44:21.139 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-10T21:44:21.139 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-10T21:44:21.139 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-10T21:44:21.139 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-10T21:44:21.139 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "5e2f023f-d6a6-49f4-a256-f4807dce3739", 2024-09-10T21:44:21.139 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:44:21.139 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:44:21.139 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-10T21:44:21.139 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-10T21:44:21.139 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-10T21:44:21.139 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-10T21:44:21.139 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-10T21:44:21.140 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-10T21:44:21.140 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-10T21:44:21.140 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:44:21.140 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-10T21:44:21.140 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:44:21.140 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:44:21.140 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:44:21.140 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:44:21.140 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:44:21.141 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:44:21.141 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:44:21.141 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:44:21.141 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:44:21.141 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:44:21.141 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:44:21.141 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:44:21.141 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-10T21:44:21.141 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:44:21.142 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-10T21:44:21.142 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:44:21.142 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:44:21.142 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:44:21.142 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:44:21.142 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:44:21.142 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:44:21.142 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:44:21.142 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-10T21:44:21.142 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:44:21.143 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:44:21.143 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:44:21.143 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:44:21.143 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-10T21:44:21.143 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:44:21.143 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:44:21.143 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:44:21.143 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:44:21.143 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-10T21:44:21.143 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-10T21:44:21.143 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:44:21.143 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:44:21.144 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:44:21.144 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:44:21.144 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:44:21.144 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:44:21.144 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:44:21.144 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:44:21.144 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:44:21.144 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:44:21.144 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:44:21.144 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3300", 2024-09-10T21:44:21.144 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:44:21.144 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:44:21.145 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:44:21.145 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:44:21.145 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789", 2024-09-10T21:44:21.145 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:44:21.145 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:44:21.145 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:44:21.145 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:44:21.145 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789/0", 2024-09-10T21:44:21.145 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6789/0", 2024-09-10T21:44:21.145 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:44:21.145 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:44:21.145 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:44:21.145 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:44:21.145 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:44:21.146 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:44:21.146 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:44:21.146 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:44:21.146 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:44:21.146 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:44:21.146 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:44:21.146 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3301", 2024-09-10T21:44:21.146 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:44:21.146 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:44:21.146 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:44:21.146 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:44:21.146 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790", 2024-09-10T21:44:21.146 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:44:21.146 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:44:21.146 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:44:21.147 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:44:21.147 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790/0", 2024-09-10T21:44:21.147 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6790/0", 2024-09-10T21:44:21.147 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:44:21.147 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:44:21.147 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:44:21.147 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:44:21.147 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:44:21.147 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:44:21.147 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-10T21:44:21.147 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-10T21:44:21.147 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:44:21.147 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:44:21.148 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:44:21.148 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:44:21.148 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:44:21.148 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:44:21.148 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-10T21:44:21.148 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:44:21.148 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:44:21.148 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:44:21.148 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:44:21.148 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:44:21.148 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:44:21.148 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:44:21.148 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-10T21:44:21.148 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-10T21:44:21.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:21.131+0000 7f3492600640 1 -- 172.21.15.17:0/2090247624 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f346c0720f0 msgr2=0x7f346c0745b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:21.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:21.131+0000 7f3492600640 1 --2- 172.21.15.17:0/2090247624 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f346c0720f0 0x7f346c0745b0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f347c016040 tx=0x7f347c00c000 comp rx=0 tx=0).stop 2024-09-10T21:44:21.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:21.131+0000 7f3492600640 1 -- 172.21.15.17:0/2090247624 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f348c0c7210 msgr2=0x7f348c1de3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:21.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:21.131+0000 7f3492600640 1 --2- 172.21.15.17:0/2090247624 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f348c0c7210 0x7f348c1de3d0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f3478021640 tx=0x7f3478005040 comp rx=0 tx=0).stop 2024-09-10T21:44:21.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:21.131+0000 7f348be00640 1 -- 172.21.15.17:0/2090247624 reap_dead start 2024-09-10T21:44:21.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:21.131+0000 7f3492600640 1 -- 172.21.15.17:0/2090247624 shutdown_connections 2024-09-10T21:44:21.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:21.131+0000 7f3492600640 1 -- 172.21.15.17:0/2090247624 >> 172.21.15.17:0/2090247624 conn(0x7f348c0700d0 msgr2=0x7f348c0c1880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:44:21.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:21.135+0000 7f3492600640 1 -- 172.21.15.17:0/2090247624 shutdown_connections 2024-09-10T21:44:21.150 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:21.135+0000 7f3492600640 1 -- 172.21.15.17:0/2090247624 wait complete. 2024-09-10T21:44:21.268 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-10T21:44:21.268 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-09-10T21:44:21.268 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-10T21:44:21.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:21.543+0000 7faa16e00640 1 -- 172.21.15.17:0/3173885996 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa9f80051d0 msgr2=0x7fa9f800b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:21.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:21.543+0000 7faa16e00640 1 --2- 172.21.15.17:0/3173885996 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa9f80051d0 0x7fa9f800b9d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7faa04009ee0 tx=0x7faa0401d650 comp rx=0 tx=0).stop 2024-09-10T21:44:21.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:21.547+0000 7faa16e00640 1 -- 172.21.15.17:0/3173885996 shutdown_connections 2024-09-10T21:44:21.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:21.551+0000 7faa16e00640 1 --2- 172.21.15.17:0/3173885996 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa9f80051d0 0x7fa9f800b9d0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:21.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:21.551+0000 7faa16e00640 1 -- 172.21.15.17:0/3173885996 >> 172.21.15.17:0/3173885996 conn(0x7faa10070110 msgr2=0x7faa10070520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:44:21.555 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:21.551+0000 7faa16e00640 1 -- 172.21.15.17:0/3173885996 shutdown_connections 2024-09-10T21:44:21.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:21.555+0000 7faa16e00640 1 -- 172.21.15.17:0/3173885996 wait complete. 2024-09-10T21:44:21.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:21.555+0000 7faa16e00640 1 Processor -- start 2024-09-10T21:44:21.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:21.555+0000 7faa16e00640 1 -- start start 2024-09-10T21:44:21.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:21.559+0000 7faa16e00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa9f80051d0 0x7faa100af1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:21.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:21.559+0000 7faa16e00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa10200820 0x7faa1015de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:21.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:21.559+0000 7faa16e00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7faa10200c80 0x7faa10152850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:21.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:21.559+0000 7faa16e00640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7faa100c8640 con 0x7fa9f80051d0 2024-09-10T21:44:21.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:21.559+0000 7faa16e00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7faa100c8360 con 0x7faa10200820 2024-09-10T21:44:21.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:21.559+0000 7faa16e00640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7faa100c10d0 con 0x7faa10200c80 2024-09-10T21:44:21.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:21.559+0000 7faa0fe00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7faa10200c80 0x7faa10152850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:21.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:21.559+0000 7faa0fe00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7faa10200c80 0x7faa10152850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3301/0 says I am v2:172.21.15.17:50736/0 (socket says 172.21.15.17:50736) 2024-09-10T21:44:21.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:21.559+0000 7faa0fe00640 1 -- 172.21.15.17:0/1055165763 learned_addr learned my addr 172.21.15.17:0/1055165763 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:44:21.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:21.559+0000 7faa0f400640 1 --2- 172.21.15.17:0/1055165763 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa9f80051d0 0x7faa100af1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:21.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:21.559+0000 7faa0ea00640 1 --2- 172.21.15.17:0/1055165763 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa10200820 0x7faa1015de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:21.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:21.563+0000 7faa0fe00640 1 -- 172.21.15.17:0/1055165763 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa10200820 msgr2=0x7faa1015de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:21.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:21.563+0000 7faa0fe00640 1 --2- 172.21.15.17:0/1055165763 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7faa10200820 0x7faa1015de00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:21.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:21.563+0000 7faa0fe00640 1 -- 172.21.15.17:0/1055165763 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa9f80051d0 msgr2=0x7faa100af1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:21.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:21.563+0000 7faa0fe00640 1 --2- 172.21.15.17:0/1055165763 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa9f80051d0 0x7faa100af1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:21.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:21.563+0000 7faa0fe00640 1 -- 172.21.15.17:0/1055165763 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa10167290 con 0x7faa10200c80 2024-09-10T21:44:21.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:21.563+0000 7faa0f400640 1 --2- 172.21.15.17:0/1055165763 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa9f80051d0 0x7faa100af1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:44:21.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:21.563+0000 7faa0fe00640 1 --2- 172.21.15.17:0/1055165763 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7faa10200c80 0x7faa10152850 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7faa0401def0 tx=0x7faa040a7af0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:44:21.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:21.563+0000 7faa03e00640 1 -- 172.21.15.17:0/1055165763 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa04009ab0 con 0x7faa10200c80 2024-09-10T21:44:21.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:21.563+0000 7faa03e00640 1 -- 172.21.15.17:0/1055165763 <== mon.2 v2:172.21.15.31:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faa04004a50 con 0x7faa10200c80 2024-09-10T21:44:21.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:21.563+0000 7faa03e00640 1 -- 172.21.15.17:0/1055165763 <== mon.2 v2:172.21.15.31:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa04004c40 con 0x7faa10200c80 2024-09-10T21:44:21.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:21.563+0000 7faa16e00640 1 -- 172.21.15.17:0/1055165763 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa100afa20 con 0x7faa10200c80 2024-09-10T21:44:21.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:21.563+0000 7faa16e00640 1 -- 172.21.15.17:0/1055165763 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7faa10201420 con 0x7faa10200c80 2024-09-10T21:44:21.576 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:21.571+0000 7faa03e00640 1 -- 172.21.15.17:0/1055165763 <== mon.2 v2:172.21.15.31:3301/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7faa040bd030 con 0x7faa10200c80 2024-09-10T21:44:21.576 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:21.571+0000 7faa16e00640 1 --2- 172.21.15.17:0/1055165763 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa9ec002a80 0x7fa9ec004f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:21.576 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:21.571+0000 7faa16e00640 1 -- 172.21.15.17:0/1055165763 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7fa9ec007ed0 con 0x7fa9ec002a80 2024-09-10T21:44:21.576 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:21.571+0000 7faa0f400640 1 --2- 172.21.15.17:0/1055165763 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa9ec002a80 0x7fa9ec004f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:21.576 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:21.571+0000 7faa0f400640 1 -- 172.21.15.17:0/1055165763 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa9ec0080d0 con 0x7fa9ec002a80 2024-09-10T21:44:21.576 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:21.575+0000 7faa0f400640 1 --2- 172.21.15.17:0/1055165763 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa9ec002a80 0x7fa9ec004f40 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7faa1006be40 tx=0x7fa9fc00eb90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:44:21.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:21.575+0000 7faa03e00640 1 --2- 172.21.15.17:0/1055165763 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fa9f00726e0 0x7fa9f0074ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:21.577 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:21.575+0000 7faa03e00640 1 -- 172.21.15.17:0/1055165763 <== mon.2 v2:172.21.15.31:3301/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7faa04002020 con 0x7faa10200c80 2024-09-10T21:44:21.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:21.575+0000 7faa03e00640 1 -- 172.21.15.17:0/1055165763 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9fc00c610 con 0x7fa9ec002a80 2024-09-10T21:44:21.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:21.575+0000 7faa03e00640 1 -- 172.21.15.17:0/1055165763 <== mon.2 v2:172.21.15.31:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fa9fc01a070 con 0x7fa9ec002a80 2024-09-10T21:44:21.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:21.575+0000 7faa0ea00640 1 --2- 172.21.15.17:0/1055165763 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fa9f00726e0 0x7fa9f0074ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:21.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:21.575+0000 7faa03e00640 1 -- 172.21.15.17:0/1055165763 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa9ec002a80 msgr2=0x7fa9ec004f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:21.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:21.575+0000 7faa03e00640 1 --2- 172.21.15.17:0/1055165763 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa9ec002a80 0x7fa9ec004f40 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7faa1006be40 tx=0x7fa9fc00eb90 comp rx=0 tx=0).stop 2024-09-10T21:44:21.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:21.575+0000 7faa03e00640 1 -- 172.21.15.17:0/1055165763 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa9ec002a80 msgr2=0x7fa9ec004f40 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:44:21.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:21.575+0000 7faa03e00640 1 --2- 172.21.15.17:0/1055165763 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa9ec002a80 0x7fa9ec004f40 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:21.580 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:21.579+0000 7faa0ea00640 1 --2- 172.21.15.17:0/1055165763 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fa9f00726e0 0x7fa9f0074ba0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7faa08002020 tx=0x7faa08009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:44:21.617 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:21.607+0000 7faa16e00640 1 --2- 172.21.15.17:0/1055165763 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa9ec006f40 0x7fa9ec0073d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:21.617 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:21.607+0000 7faa16e00640 1 -- 172.21.15.17:0/1055165763 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7fa9ec00a4b0 con 0x7fa9ec006f40 2024-09-10T21:44:21.617 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:21.607+0000 7faa0f400640 1 --2- 172.21.15.17:0/1055165763 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa9ec006f40 0x7fa9ec0073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:21.617 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:21.611+0000 7faa0f400640 1 -- 172.21.15.17:0/1055165763 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fa9ec007910 con 0x7fa9ec006f40 2024-09-10T21:44:21.617 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:21.611+0000 7faa0f400640 1 --2- 172.21.15.17:0/1055165763 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa9ec006f40 0x7fa9ec0073d0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7faa1006be40 tx=0x7fa9fc001090 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:44:21.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:21.615+0000 7faa03e00640 1 -- 172.21.15.17:0/1055165763 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9fc015c40 con 0x7fa9ec006f40 2024-09-10T21:44:21.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:21.615+0000 7faa03e00640 1 -- 172.21.15.17:0/1055165763 <== mon.2 v2:172.21.15.31:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5042 (secure 0 0 0) 0x7fa9fc015c40 con 0x7fa9ec006f40 2024-09-10T21:44:21.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:21.615+0000 7faa03e00640 1 -- 172.21.15.17:0/1055165763 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa9ec006f40 msgr2=0x7fa9ec0073d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:21.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:21.615+0000 7faa03e00640 1 --2- 172.21.15.17:0/1055165763 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa9ec006f40 0x7fa9ec0073d0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7faa1006be40 tx=0x7fa9fc001090 comp rx=0 tx=0).stop 2024-09-10T21:44:21.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:21.615+0000 7faa03e00640 1 -- 172.21.15.17:0/1055165763 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa9ec006f40 msgr2=0x7fa9ec0073d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:44:21.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:21.615+0000 7faa03e00640 1 --2- 172.21.15.17:0/1055165763 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa9ec006f40 0x7fa9ec0073d0 secure :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7faa1006be40 tx=0x7fa9fc001090 comp rx=0 tx=0).stop 2024-09-10T21:44:21.623 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-10T21:44:21.623 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:44:21.623 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:44:21.623 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-10T21:44:21.623 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 150, 2024-09-10T21:44:21.623 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 63747, 2024-09-10T21:44:21.623 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-10T21:44:21.624 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-10T21:44:21.624 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-10T21:44:21.624 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-10T21:44:21.624 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:44:21.624 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 2, 2024-09-10T21:44:21.624 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:44:21.624 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-10T21:44:21.624 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-10T21:44:21.624 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:44:21.624 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:44:21.624 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:44:21.624 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:44:21.625 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:44:21.625 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:44:21.625 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:44:21.625 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:44:21.625 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:44:21.625 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:44:21.625 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:44:21.625 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:44:21.625 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-10T21:44:21.625 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-10T21:44:21.625 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:44:21.626 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:44:21.626 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:44:21.626 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:44:21.626 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:44:21.626 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:44:21.626 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:44:21.626 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:44:21.626 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:44:21.626 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:44:21.626 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:44:21.626 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:44:21.626 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:44:21.627 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-10T21:44:21.627 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-10T21:44:21.627 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-10T21:44:21.627 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-10T21:44:21.627 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-10T21:44:21.627 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "5e2f023f-d6a6-49f4-a256-f4807dce3739", 2024-09-10T21:44:21.627 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:44:21.627 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:44:21.627 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-10T21:44:21.627 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-10T21:44:21.627 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-10T21:44:21.628 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-10T21:44:21.628 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-10T21:44:21.628 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-10T21:44:21.628 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-10T21:44:21.628 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:44:21.628 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-10T21:44:21.628 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:44:21.628 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:44:21.628 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:44:21.628 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:44:21.628 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:44:21.628 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:44:21.628 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:44:21.628 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:44:21.629 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:44:21.629 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:44:21.629 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:44:21.629 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:44:21.629 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-10T21:44:21.629 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:44:21.629 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-10T21:44:21.629 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:44:21.629 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:44:21.629 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:44:21.629 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:44:21.629 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:44:21.629 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:44:21.629 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:44:21.630 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-10T21:44:21.630 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:44:21.630 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:44:21.630 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:44:21.630 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:44:21.630 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-10T21:44:21.630 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:44:21.630 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:44:21.630 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:44:21.630 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:44:21.630 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-10T21:44:21.630 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-10T21:44:21.630 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:44:21.630 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:44:21.631 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:44:21.631 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:44:21.631 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:44:21.631 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:44:21.631 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:44:21.631 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:44:21.631 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:44:21.631 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:44:21.631 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:44:21.631 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3300", 2024-09-10T21:44:21.631 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:44:21.631 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:44:21.631 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:44:21.631 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:44:21.632 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789", 2024-09-10T21:44:21.632 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:44:21.632 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:44:21.632 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:44:21.632 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:44:21.632 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789/0", 2024-09-10T21:44:21.632 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6789/0", 2024-09-10T21:44:21.632 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:44:21.632 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:44:21.632 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:44:21.633 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:44:21.633 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:44:21.633 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:44:21.633 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:44:21.633 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:44:21.633 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:44:21.633 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:44:21.633 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:44:21.633 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3301", 2024-09-10T21:44:21.633 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:44:21.633 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:44:21.633 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:44:21.633 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:44:21.634 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790", 2024-09-10T21:44:21.634 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:44:21.634 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:44:21.634 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:44:21.634 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:44:21.634 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790/0", 2024-09-10T21:44:21.634 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6790/0", 2024-09-10T21:44:21.634 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:44:21.634 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:44:21.634 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:44:21.634 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:44:21.634 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:44:21.634 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:44:21.635 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-10T21:44:21.635 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-10T21:44:21.635 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:44:21.635 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:44:21.635 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:44:21.635 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:44:21.635 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:44:21.635 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:44:21.635 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": [ 2024-09-10T21:44:21.635 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:44:21.635 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:44:21.636 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:44:21.636 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 2 2024-09-10T21:44:21.636 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:44:21.636 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:44:21.636 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-10T21:44:21.636 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:44:21.636 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:44:21.636 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:44:21.636 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 5 2024-09-10T21:44:21.636 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:44:21.636 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:44:21.636 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-10T21:44:21.637 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:44:21.637 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:44:21.637 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:44:21.637 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 6 2024-09-10T21:44:21.637 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:44:21.637 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:44:21.637 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": [ 2024-09-10T21:44:21.637 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:44:21.637 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:44:21.637 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:44:21.637 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:44:21.637 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:44:21.637 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:44:21.637 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:44:21.638 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-10T21:44:21.638 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-10T21:44:21.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:21.635+0000 7faa16e00640 1 -- 172.21.15.17:0/1055165763 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fa9f00726e0 msgr2=0x7fa9f0074ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:21.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:21.635+0000 7faa16e00640 1 --2- 172.21.15.17:0/1055165763 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fa9f00726e0 0x7fa9f0074ba0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7faa08002020 tx=0x7faa08009040 comp rx=0 tx=0).stop 2024-09-10T21:44:21.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:21.635+0000 7faa16e00640 1 -- 172.21.15.17:0/1055165763 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7faa10200c80 msgr2=0x7faa10152850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:21.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:21.635+0000 7faa16e00640 1 --2- 172.21.15.17:0/1055165763 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7faa10200c80 0x7faa10152850 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7faa0401def0 tx=0x7faa040a7af0 comp rx=0 tx=0).stop 2024-09-10T21:44:21.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:21.639+0000 7faa0fe00640 1 -- 172.21.15.17:0/1055165763 reap_dead start 2024-09-10T21:44:21.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:21.639+0000 7faa0fe00640 1 -- 172.21.15.17:0/1055165763 reap_dead start 2024-09-10T21:44:21.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:21.639+0000 7faa16e00640 1 -- 172.21.15.17:0/1055165763 shutdown_connections 2024-09-10T21:44:21.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:21.639+0000 7faa16e00640 1 -- 172.21.15.17:0/1055165763 >> 172.21.15.17:0/1055165763 conn(0x7faa10070110 msgr2=0x7faa100c1720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:44:21.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:21.639+0000 7faa16e00640 1 -- 172.21.15.17:0/1055165763 shutdown_connections 2024-09-10T21:44:21.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:21.643+0000 7faa16e00640 1 -- 172.21.15.17:0/1055165763 wait complete. 2024-09-10T21:44:21.715 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-10T21:44:21.715 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-09-10T21:44:21.715 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-09-10T21:44:22.023 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:22.019+0000 7f4227e00640 1 -- 172.21.15.17:0/3578450339 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f420c0051d0 msgr2=0x7f420c0079b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:22.023 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:22.019+0000 7f4227e00640 1 --2- 172.21.15.17:0/3578450339 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f420c0051d0 0x7f420c0079b0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f422806ba20 tx=0x7f4228099830 comp rx=0 tx=0).stop 2024-09-10T21:44:22.023 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:22.019+0000 7f422d600640 1 -- 172.21.15.17:0/3578450339 reap_dead start 2024-09-10T21:44:22.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:22.023+0000 7f4227e00640 1 -- 172.21.15.17:0/3578450339 shutdown_connections 2024-09-10T21:44:22.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:22.023+0000 7f4227e00640 1 -- 172.21.15.17:0/3578450339 >> 172.21.15.17:0/3578450339 conn(0x7f422004fe10 msgr2=0x7f42200501f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:44:22.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:22.027+0000 7f4227e00640 1 -- 172.21.15.17:0/3578450339 shutdown_connections 2024-09-10T21:44:22.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:22.027+0000 7f4227e00640 1 -- 172.21.15.17:0/3578450339 wait complete. 2024-09-10T21:44:22.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:22.027+0000 7f4227e00640 1 Processor -- start 2024-09-10T21:44:22.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:22.039+0000 7f4227e00640 1 -- start start 2024-09-10T21:44:22.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:22.039+0000 7f4227e00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f420c0051d0 0x7f4220048960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:22.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:22.039+0000 7f4227e00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f42200cc440 0x7f42200dcea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:22.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:22.039+0000 7f4227e00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f422004f870 0x7f42200dd3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:22.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:22.039+0000 7f4227e00640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f422004b1a0 con 0x7f420c0051d0 2024-09-10T21:44:22.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:22.039+0000 7f4227e00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f42200d51f0 con 0x7f42200cc440 2024-09-10T21:44:22.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:22.039+0000 7f4227e00640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f42200068c0 con 0x7f422004f870 2024-09-10T21:44:22.043 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:22.039+0000 7f422d600640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f422004f870 0x7f42200dd3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:22.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:22.039+0000 7f422d600640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f422004f870 0x7f42200dd3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3301/0 says I am v2:172.21.15.17:50788/0 (socket says 172.21.15.17:50788) 2024-09-10T21:44:22.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:22.039+0000 7f422d600640 1 -- 172.21.15.17:0/2201044143 learned_addr learned my addr 172.21.15.17:0/2201044143 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:44:22.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:22.039+0000 7f4226000640 1 --2- 172.21.15.17:0/2201044143 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f42200cc440 0x7f42200dcea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:22.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:22.043+0000 7f4226a00640 1 --2- 172.21.15.17:0/2201044143 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f420c0051d0 0x7f4220048960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:22.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:22.043+0000 7f4226000640 1 -- 172.21.15.17:0/2201044143 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f422004f870 msgr2=0x7f42200dd3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:22.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:22.043+0000 7f4226000640 1 --2- 172.21.15.17:0/2201044143 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f422004f870 0x7f42200dd3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:22.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:22.043+0000 7f4226000640 1 -- 172.21.15.17:0/2201044143 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f420c0051d0 msgr2=0x7f4220048960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:22.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:22.043+0000 7f4226000640 1 --2- 172.21.15.17:0/2201044143 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f420c0051d0 0x7f4220048960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:22.045 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:22.043+0000 7f4226000640 1 -- 172.21.15.17:0/2201044143 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f422004d610 con 0x7f42200cc440 2024-09-10T21:44:22.046 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:22.043+0000 7f4226000640 1 --2- 172.21.15.17:0/2201044143 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f42200cc440 0x7f42200dcea0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f4220050a90 tx=0x7f421800b820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:44:22.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:22.043+0000 7f421ea00640 1 -- 172.21.15.17:0/2201044143 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4218015020 con 0x7f42200cc440 2024-09-10T21:44:22.047 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:22.043+0000 7f421ea00640 1 -- 172.21.15.17:0/2201044143 <== mon.1 v2:172.21.15.31:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4218011920 con 0x7f42200cc440 2024-09-10T21:44:22.048 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:22.043+0000 7f421ea00640 1 -- 172.21.15.17:0/2201044143 <== mon.1 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4218011d70 con 0x7f42200cc440 2024-09-10T21:44:22.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:22.047+0000 7f4227e00640 1 -- 172.21.15.17:0/2201044143 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4220184df0 con 0x7f42200cc440 2024-09-10T21:44:22.055 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:22.051+0000 7f4227e00640 1 -- 172.21.15.17:0/2201044143 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4220049e90 con 0x7f42200cc440 2024-09-10T21:44:22.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:22.051+0000 7f421ea00640 1 -- 172.21.15.17:0/2201044143 <== mon.1 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f42180349f0 con 0x7f42200cc440 2024-09-10T21:44:22.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:22.055+0000 7f421ea00640 1 --2- 172.21.15.17:0/2201044143 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f42080726e0 0x7f4208074ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:22.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:22.055+0000 7f421ea00640 1 -- 172.21.15.17:0/2201044143 <== mon.1 v2:172.21.15.31:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f4218092a20 con 0x7f42200cc440 2024-09-10T21:44:22.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:22.055+0000 7f4226a00640 1 --2- 172.21.15.17:0/2201044143 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f42080726e0 0x7f4208074ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:22.059 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:22.055+0000 7f4227e00640 1 -- 172.21.15.17:0/2201044143 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42200499c0 con 0x7f42200cc440 2024-09-10T21:44:22.064 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:22.063+0000 7f4226a00640 1 --2- 172.21.15.17:0/2201044143 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f42080726e0 0x7f4208074ba0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f4214004db0 tx=0x7f4214009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:44:22.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:22.079+0000 7f421ea00640 1 -- 172.21.15.17:0/2201044143 <== mon.1 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f421805d720 con 0x7f42200cc440 2024-09-10T21:44:22.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:22.603+0000 7f4227e00640 1 -- 172.21.15.17:0/2201044143 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) -- 0x7f422004b950 con 0x7f42200cc440 2024-09-10T21:44:22.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:22.615+0000 7f421ea00640 1 -- 172.21.15.17:0/2201044143 <== mon.1 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) ==== 57+0+0 (secure 0 0 0) 0x7f4218062150 con 0x7f42200cc440 2024-09-10T21:44:22.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:22.635+0000 7f4227e00640 1 -- 172.21.15.17:0/2201044143 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f42080726e0 msgr2=0x7f4208074ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:22.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:22.635+0000 7f4227e00640 1 --2- 172.21.15.17:0/2201044143 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f42080726e0 0x7f4208074ba0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f4214004db0 tx=0x7f4214009040 comp rx=0 tx=0).stop 2024-09-10T21:44:22.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:22.635+0000 7f4227e00640 1 -- 172.21.15.17:0/2201044143 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f42200cc440 msgr2=0x7f42200dcea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:22.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:22.635+0000 7f4227e00640 1 --2- 172.21.15.17:0/2201044143 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f42200cc440 0x7f42200dcea0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f4220050a90 tx=0x7f421800b820 comp rx=0 tx=0).stop 2024-09-10T21:44:22.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:22.635+0000 7f4227e00640 1 -- 172.21.15.17:0/2201044143 shutdown_connections 2024-09-10T21:44:22.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:22.635+0000 7f4227e00640 1 --2- 172.21.15.17:0/2201044143 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f42080726e0 0x7f4208074ba0 secure :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f4214004db0 tx=0x7f4214009040 comp rx=0 tx=0).stop 2024-09-10T21:44:22.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:22.635+0000 7f4227e00640 1 --2- 172.21.15.17:0/2201044143 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f422004f870 0x7f42200dd3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:22.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:22.635+0000 7f4227e00640 1 --2- 172.21.15.17:0/2201044143 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f42200cc440 0x7f42200dcea0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:22.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:22.635+0000 7f4227e00640 1 --2- 172.21.15.17:0/2201044143 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f420c0051d0 0x7f4220048960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:22.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:22.635+0000 7f4227e00640 1 -- 172.21.15.17:0/2201044143 >> 172.21.15.17:0/2201044143 conn(0x7f422004fe10 msgr2=0x7f42200d12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:44:22.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:22.639+0000 7f4227e00640 1 -- 172.21.15.17:0/2201044143 shutdown_connections 2024-09-10T21:44:22.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:22.639+0000 7f4227e00640 1 -- 172.21.15.17:0/2201044143 wait complete. 2024-09-10T21:44:22.719 INFO:tasks.mon_thrash.mon_thrasher:waiting for 10.0 secs before continuing thrashing 2024-09-10T21:44:29.518 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:44:29.525 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:44:32.720 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-09-10T21:44:32.721 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-10T21:44:33.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:33.066+0000 7fdb69000640 1 -- 172.21.15.17:0/466461710 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdb6415e840 msgr2=0x7fdb640c1e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:33.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:33.066+0000 7fdb69000640 1 --2- 172.21.15.17:0/466461710 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdb6415e840 0x7fdb640c1e90 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fdb50009900 tx=0x7fdb5001f0f0 comp rx=0 tx=0).stop 2024-09-10T21:44:33.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:33.070+0000 7fdb69000640 1 -- 172.21.15.17:0/466461710 shutdown_connections 2024-09-10T21:44:33.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:33.070+0000 7fdb69000640 1 --2- 172.21.15.17:0/466461710 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fdb6406e460 0x7fdb64164ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:33.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:33.070+0000 7fdb69000640 1 --2- 172.21.15.17:0/466461710 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdb6415e840 0x7fdb640c1e90 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:33.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:33.070+0000 7fdb69000640 1 -- 172.21.15.17:0/466461710 >> 172.21.15.17:0/466461710 conn(0x7fdb640ca390 msgr2=0x7fdb640cce70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:44:33.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:33.070+0000 7fdb69000640 1 -- 172.21.15.17:0/466461710 shutdown_connections 2024-09-10T21:44:33.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:33.070+0000 7fdb69000640 1 -- 172.21.15.17:0/466461710 wait complete. 2024-09-10T21:44:33.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:33.074+0000 7fdb69000640 1 Processor -- start 2024-09-10T21:44:33.078 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:33.074+0000 7fdb69000640 1 -- start start 2024-09-10T21:44:33.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:33.074+0000 7fdb69000640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdb6406e460 0x7fdb640af400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:33.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:33.074+0000 7fdb69000640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fdb64157c90 0x7fdb6415bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:33.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:33.074+0000 7fdb69000640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdb6415e840 0x7fdb640b07a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:33.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:33.074+0000 7fdb63400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdb6406e460 0x7fdb640af400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:33.079 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:33.074+0000 7fdb62a00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fdb64157c90 0x7fdb6415bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:33.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:33.074+0000 7fdb63400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdb6406e460 0x7fdb640af400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.17:46178/0 (socket says 172.21.15.17:46178) 2024-09-10T21:44:33.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:33.074+0000 7fdb62a00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fdb64157c90 0x7fdb6415bc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3301/0 says I am v2:172.21.15.17:44884/0 (socket says 172.21.15.17:44884) 2024-09-10T21:44:33.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:33.074+0000 7fdb62a00640 1 -- 172.21.15.17:0/1880655062 learned_addr learned my addr 172.21.15.17:0/1880655062 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:44:33.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:33.078+0000 7fdb63e00640 1 --2- 172.21.15.17:0/1880655062 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdb6415e840 0x7fdb640b07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:33.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:33.078+0000 7fdb69000640 1 -- 172.21.15.17:0/1880655062 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fdb640c3d10 con 0x7fdb6415e840 2024-09-10T21:44:33.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:33.078+0000 7fdb69000640 1 -- 172.21.15.17:0/1880655062 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fdb640b22b0 con 0x7fdb6406e460 2024-09-10T21:44:33.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:33.078+0000 7fdb69000640 1 -- 172.21.15.17:0/1880655062 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7fdb64158070 con 0x7fdb64157c90 2024-09-10T21:44:33.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:33.078+0000 7fdb63400640 1 -- 172.21.15.17:0/1880655062 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fdb64157c90 msgr2=0x7fdb6415bc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:33.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:33.078+0000 7fdb63400640 1 --2- 172.21.15.17:0/1880655062 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fdb64157c90 0x7fdb6415bc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:33.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:33.078+0000 7fdb63400640 1 -- 172.21.15.17:0/1880655062 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdb6415e840 msgr2=0x7fdb640b07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:33.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:33.078+0000 7fdb63400640 1 --2- 172.21.15.17:0/1880655062 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdb6415e840 0x7fdb640b07a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:33.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:33.078+0000 7fdb63400640 1 -- 172.21.15.17:0/1880655062 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb640b1990 con 0x7fdb6406e460 2024-09-10T21:44:33.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:33.078+0000 7fdb63400640 1 --2- 172.21.15.17:0/1880655062 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdb6406e460 0x7fdb640af400 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fdb5400c440 tx=0x7fdb5400c900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:44:33.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:33.078+0000 7fdb5b400640 1 -- 172.21.15.17:0/1880655062 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb54011790 con 0x7fdb6406e460 2024-09-10T21:44:33.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:33.078+0000 7fdb5b400640 1 -- 172.21.15.17:0/1880655062 <== mon.1 v2:172.21.15.31:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdb54017070 con 0x7fdb6406e460 2024-09-10T21:44:33.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:33.078+0000 7fdb69000640 1 -- 172.21.15.17:0/1880655062 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb640ccbf0 con 0x7fdb6406e460 2024-09-10T21:44:33.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:33.078+0000 7fdb5b400640 1 -- 172.21.15.17:0/1880655062 <== mon.1 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb54011cc0 con 0x7fdb6406e460 2024-09-10T21:44:33.083 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:33.078+0000 7fdb69000640 1 -- 172.21.15.17:0/1880655062 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb64074d10 con 0x7fdb6406e460 2024-09-10T21:44:33.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:33.082+0000 7fdb69000640 1 -- 172.21.15.17:0/1880655062 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb64070570 con 0x7fdb6406e460 2024-09-10T21:44:33.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:33.082+0000 7fdb5b400640 1 -- 172.21.15.17:0/1880655062 <== mon.1 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fdb5401c460 con 0x7fdb6406e460 2024-09-10T21:44:33.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:33.082+0000 7fdb5b400640 1 --2- 172.21.15.17:0/1880655062 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fdb44072610 0x7fdb44074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:33.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:33.086+0000 7fdb5b400640 1 -- 172.21.15.17:0/1880655062 <== mon.1 v2:172.21.15.31:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fdb54093350 con 0x7fdb6406e460 2024-09-10T21:44:33.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:33.086+0000 7fdb62a00640 1 --2- 172.21.15.17:0/1880655062 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fdb44072610 0x7fdb44074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:33.090 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:33.086+0000 7fdb62a00640 1 --2- 172.21.15.17:0/1880655062 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fdb44072610 0x7fdb44074ad0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fdb5001ef20 tx=0x7fdb5001d930 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:44:33.102 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:33.094+0000 7fdb5b400640 1 -- 172.21.15.17:0/1880655062 <== mon.1 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fdb5405e6c0 con 0x7fdb6406e460 2024-09-10T21:44:33.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:33.682+0000 7fdb69000640 1 -- 172.21.15.17:0/1880655062 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fdb640c92c0 con 0x7fdb6406e460 2024-09-10T21:44:33.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:33.682+0000 7fdb5b400640 1 -- 172.21.15.17:0/1880655062 <== mon.1 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1558 (secure 0 0 0) 0x7fdb5405dfc0 con 0x7fdb6406e460 2024-09-10T21:44:33.690 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":150,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":14,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5e2f023f-d6a6-49f4-a256-f4807dce3739","modified":"2024-09-10T21:23:46.892130Z","created":"2024-09-10T21:23:46.892130Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3301","nonce":0},{"type":"v1","addr":"172.21.15.31:6790","nonce":0}]},"addr":"172.21.15.31:6790/0","public_addr":"172.21.15.31:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-10T21:44:33.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:33.690+0000 7fdb69000640 1 -- 172.21.15.17:0/1880655062 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fdb44072610 msgr2=0x7fdb44074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:33.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:33.690+0000 7fdb69000640 1 --2- 172.21.15.17:0/1880655062 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fdb44072610 0x7fdb44074ad0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fdb5001ef20 tx=0x7fdb5001d930 comp rx=0 tx=0).stop 2024-09-10T21:44:33.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:33.690+0000 7fdb69000640 1 -- 172.21.15.17:0/1880655062 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdb6406e460 msgr2=0x7fdb640af400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:33.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:33.690+0000 7fdb69000640 1 --2- 172.21.15.17:0/1880655062 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdb6406e460 0x7fdb640af400 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fdb5400c440 tx=0x7fdb5400c900 comp rx=0 tx=0).stop 2024-09-10T21:44:33.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:33.690+0000 7fdb69000640 1 -- 172.21.15.17:0/1880655062 shutdown_connections 2024-09-10T21:44:33.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:33.690+0000 7fdb69000640 1 --2- 172.21.15.17:0/1880655062 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fdb44072610 0x7fdb44074ad0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:33.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:33.690+0000 7fdb69000640 1 --2- 172.21.15.17:0/1880655062 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdb6415e840 0x7fdb640b07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:33.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:33.690+0000 7fdb69000640 1 --2- 172.21.15.17:0/1880655062 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fdb64157c90 0x7fdb6415bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:33.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:33.690+0000 7fdb69000640 1 --2- 172.21.15.17:0/1880655062 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdb6406e460 0x7fdb640af400 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:33.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:33.690+0000 7fdb69000640 1 -- 172.21.15.17:0/1880655062 >> 172.21.15.17:0/1880655062 conn(0x7fdb640ca390 msgr2=0x7fdb64074470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:44:33.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:33.690+0000 7fdb69000640 1 -- 172.21.15.17:0/1880655062 shutdown_connections 2024-09-10T21:44:33.695 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:33.690+0000 7fdb69000640 1 -- 172.21.15.17:0/1880655062 wait complete. 2024-09-10T21:44:33.781 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-09-10T21:44:33.782 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-09-10T21:44:33.782 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-09-10T21:44:33.782 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-10T21:44:34.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:34.102+0000 7fe324200640 1 -- 172.21.15.17:0/3217970895 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe31c156750 msgr2=0x7fe31c156b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:34.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:34.102+0000 7fe324200640 1 --2- 172.21.15.17:0/3217970895 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe31c156750 0x7fe31c156b80 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fe31c06f1d0 tx=0x7fe31001c8a0 comp rx=0 tx=0).stop 2024-09-10T21:44:34.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:34.106+0000 7fe324200640 1 -- 172.21.15.17:0/3217970895 shutdown_connections 2024-09-10T21:44:34.112 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:34.106+0000 7fe324200640 1 --2- 172.21.15.17:0/3217970895 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe31c156750 0x7fe31c156b80 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:34.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:34.106+0000 7fe324200640 1 --2- 172.21.15.17:0/3217970895 >> v2:172.21.15.17:3300/0 conn(0x7fe31c074980 0x7fe31c0b1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:34.113 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:34.106+0000 7fe324200640 1 -- 172.21.15.17:0/3217970895 >> 172.21.15.17:0/3217970895 conn(0x7fe31c0700d0 msgr2=0x7fe31c0704e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:44:34.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:34.114+0000 7fe324200640 1 -- 172.21.15.17:0/3217970895 shutdown_connections 2024-09-10T21:44:34.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:34.114+0000 7fe324200640 1 -- 172.21.15.17:0/3217970895 wait complete. 2024-09-10T21:44:34.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:34.114+0000 7fe324200640 1 Processor -- start 2024-09-10T21:44:34.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:34.122+0000 7fe324200640 1 -- start start 2024-09-10T21:44:34.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:34.122+0000 7fe324200640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe31c074980 0x7fe31c1403f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:34.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:34.122+0000 7fe324200640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe31c153940 0x7fe31c141380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:34.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:34.122+0000 7fe324200640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe31c156750 0x7fe31c1f8050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:34.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:34.122+0000 7fe324200640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fe31c0c7140 con 0x7fe31c074980 2024-09-10T21:44:34.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:34.122+0000 7fe324200640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fe31c154160 con 0x7fe31c156750 2024-09-10T21:44:34.127 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:34.122+0000 7fe324200640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7fe31c15cfd0 con 0x7fe31c153940 2024-09-10T21:44:34.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:34.126+0000 7fe320e00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe31c153940 0x7fe31c141380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:34.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:34.126+0000 7fe322200640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe31c156750 0x7fe31c1f8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:34.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:34.126+0000 7fe322200640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe31c156750 0x7fe31c1f8050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.17:46196/0 (socket says 172.21.15.17:46196) 2024-09-10T21:44:34.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:34.126+0000 7fe322200640 1 -- 172.21.15.17:0/1425711122 learned_addr learned my addr 172.21.15.17:0/1425711122 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:44:34.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:34.126+0000 7fe321800640 1 --2- 172.21.15.17:0/1425711122 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe31c074980 0x7fe31c1403f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:34.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:34.130+0000 7fe322200640 1 -- 172.21.15.17:0/1425711122 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe31c153940 msgr2=0x7fe31c141380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:34.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:34.130+0000 7fe322200640 1 --2- 172.21.15.17:0/1425711122 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe31c153940 0x7fe31c141380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:34.133 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:34.130+0000 7fe322200640 1 -- 172.21.15.17:0/1425711122 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe31c074980 msgr2=0x7fe31c1403f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:34.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:34.130+0000 7fe322200640 1 --2- 172.21.15.17:0/1425711122 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe31c074980 0x7fe31c1403f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:34.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:34.130+0000 7fe322200640 1 -- 172.21.15.17:0/1425711122 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe31c151940 con 0x7fe31c156750 2024-09-10T21:44:34.134 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:34.130+0000 7fe321800640 1 --2- 172.21.15.17:0/1425711122 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe31c074980 0x7fe31c1403f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:44:34.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:34.130+0000 7fe322200640 1 --2- 172.21.15.17:0/1425711122 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe31c156750 0x7fe31c1f8050 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fe31400c7b0 tx=0x7fe31400cc70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:44:34.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:34.134+0000 7fe319600640 1 -- 172.21.15.17:0/1425711122 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe314017070 con 0x7fe31c156750 2024-09-10T21:44:34.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:34.134+0000 7fe319600640 1 -- 172.21.15.17:0/1425711122 <== mon.1 v2:172.21.15.31:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe314005070 con 0x7fe31c156750 2024-09-10T21:44:34.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:34.134+0000 7fe319600640 1 -- 172.21.15.17:0/1425711122 <== mon.1 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3140054c0 con 0x7fe31c156750 2024-09-10T21:44:34.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:34.134+0000 7fe324200640 1 -- 172.21.15.17:0/1425711122 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe31c13e430 con 0x7fe31c156750 2024-09-10T21:44:34.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:34.134+0000 7fe324200640 1 -- 172.21.15.17:0/1425711122 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe31c0ca8c0 con 0x7fe31c156750 2024-09-10T21:44:34.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:34.134+0000 7fe324200640 1 --2- 172.21.15.17:0/1425711122 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe31c1f8590 0x7fe31c13b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:34.137 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:34.134+0000 7fe324200640 1 -- 172.21.15.17:0/1425711122 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fe31c13e1e0 con 0x7fe31c1f8590 2024-09-10T21:44:34.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:34.134+0000 7fe321800640 1 --2- 172.21.15.17:0/1425711122 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe31c1f8590 0x7fe31c13b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:34.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:34.134+0000 7fe321800640 1 -- 172.21.15.17:0/1425711122 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe31c14dc30 con 0x7fe31c1f8590 2024-09-10T21:44:34.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:34.138+0000 7fe321800640 1 --2- 172.21.15.17:0/1425711122 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe31c1f8590 0x7fe31c13b8a0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fe30800d700 tx=0x7fe30800dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:44:34.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:34.138+0000 7fe319600640 1 -- 172.21.15.17:0/1425711122 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe308018030 con 0x7fe31c1f8590 2024-09-10T21:44:34.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:34.138+0000 7fe319600640 1 -- 172.21.15.17:0/1425711122 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fe308009710 con 0x7fe31c1f8590 2024-09-10T21:44:34.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:34.138+0000 7fe319600640 1 -- 172.21.15.17:0/1425711122 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe31c1f8590 msgr2=0x7fe31c13b8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:34.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:34.138+0000 7fe319600640 1 --2- 172.21.15.17:0/1425711122 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe31c1f8590 0x7fe31c13b8a0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fe30800d700 tx=0x7fe30800dbc0 comp rx=0 tx=0).stop 2024-09-10T21:44:34.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:34.138+0000 7fe319600640 1 -- 172.21.15.17:0/1425711122 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe31c1f8590 msgr2=0x7fe31c13b8a0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:44:34.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:34.138+0000 7fe319600640 1 --2- 172.21.15.17:0/1425711122 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe31c1f8590 0x7fe31c13b8a0 secure :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fe30800d700 tx=0x7fe30800dbc0 comp rx=0 tx=0).stop 2024-09-10T21:44:34.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:34.138+0000 7fe319600640 1 -- 172.21.15.17:0/1425711122 <== mon.1 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fe314005660 con 0x7fe31c156750 2024-09-10T21:44:34.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:34.138+0000 7fe319600640 1 --2- 172.21.15.17:0/1425711122 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fe2fc0722b0 0x7fe2fc074770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:34.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:34.138+0000 7fe319600640 1 -- 172.21.15.17:0/1425711122 <== mon.1 v2:172.21.15.31:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fe314092f90 con 0x7fe31c156750 2024-09-10T21:44:34.164 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:34.158+0000 7fe321800640 1 --2- 172.21.15.17:0/1425711122 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fe2fc0722b0 0x7fe2fc074770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:34.165 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:34.158+0000 7fe321800640 1 --2- 172.21.15.17:0/1425711122 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fe2fc0722b0 0x7fe2fc074770 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fe30800dc00 tx=0x7fe308012000 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:44:34.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:34.210+0000 7fe324200640 1 --2- 172.21.15.17:0/1425711122 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe31c13bde0 0x7fe31c1fb450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:34.216 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:34.210+0000 7fe324200640 1 -- 172.21.15.17:0/1425711122 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fe31c073390 con 0x7fe31c13bde0 2024-09-10T21:44:34.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:34.218+0000 7fe320e00640 1 --2- 172.21.15.17:0/1425711122 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe31c13bde0 0x7fe31c1fb450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:34.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:34.218+0000 7fe320e00640 1 -- 172.21.15.17:0/1425711122 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fe31c075e50 con 0x7fe31c13bde0 2024-09-10T21:44:34.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:34.222+0000 7fe320e00640 1 --2- 172.21.15.17:0/1425711122 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe31c13bde0 0x7fe31c1fb450 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fe31c1570c0 tx=0x7fe310005db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:44:34.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:34.222+0000 7fe319600640 1 -- 172.21.15.17:0/1425711122 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3100b2040 con 0x7fe31c13bde0 2024-09-10T21:44:34.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:34.226+0000 7fe319600640 1 -- 172.21.15.17:0/1425711122 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5046 (secure 0 0 0) 0x7fe310005220 con 0x7fe31c13bde0 2024-09-10T21:44:34.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:34.226+0000 7fe319600640 1 -- 172.21.15.17:0/1425711122 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe31c13bde0 msgr2=0x7fe31c1fb450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:34.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:34.226+0000 7fe319600640 1 --2- 172.21.15.17:0/1425711122 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe31c13bde0 0x7fe31c1fb450 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fe31c1570c0 tx=0x7fe310005db0 comp rx=0 tx=0).stop 2024-09-10T21:44:34.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:34.226+0000 7fe319600640 1 -- 172.21.15.17:0/1425711122 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe31c13bde0 msgr2=0x7fe31c1fb450 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:44:34.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:34.226+0000 7fe319600640 1 --2- 172.21.15.17:0/1425711122 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe31c13bde0 0x7fe31c1fb450 secure :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fe31c1570c0 tx=0x7fe310005db0 comp rx=0 tx=0).stop 2024-09-10T21:44:34.230 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-10T21:44:34.230 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:44:34.230 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:44:34.230 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "leader", 2024-09-10T21:44:34.231 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 150, 2024-09-10T21:44:34.231 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 133354, 2024-09-10T21:44:34.231 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-10T21:44:34.231 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-10T21:44:34.231 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-10T21:44:34.231 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-10T21:44:34.231 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:44:34.231 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 15, 2024-09-10T21:44:34.232 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:44:34.232 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-10T21:44:34.232 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-10T21:44:34.232 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:44:34.232 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:44:34.232 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:44:34.232 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:44:34.233 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:44:34.233 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:44:34.233 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:44:34.233 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:44:34.233 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:44:34.233 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:44:34.234 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:44:34.234 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:44:34.234 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-10T21:44:34.234 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-10T21:44:34.234 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:44:34.234 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:44:34.235 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:44:34.235 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:44:34.235 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:44:34.235 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:44:34.235 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:44:34.235 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:44:34.235 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:44:34.236 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:44:34.236 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:44:34.237 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:44:34.237 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:44:34.237 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-10T21:44:34.237 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-10T21:44:34.237 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-10T21:44:34.237 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-10T21:44:34.238 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-10T21:44:34.238 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "5e2f023f-d6a6-49f4-a256-f4807dce3739", 2024-09-10T21:44:34.238 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:44:34.238 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:44:34.238 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-10T21:44:34.238 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-10T21:44:34.239 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-10T21:44:34.239 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-10T21:44:34.239 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-10T21:44:34.239 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-10T21:44:34.239 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-10T21:44:34.239 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:44:34.239 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-10T21:44:34.240 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:44:34.240 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:44:34.240 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:44:34.240 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:44:34.240 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:44:34.240 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:44:34.240 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:44:34.241 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:44:34.241 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:44:34.241 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:44:34.241 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:44:34.241 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:44:34.241 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-10T21:44:34.242 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:44:34.242 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-10T21:44:34.242 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:44:34.242 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:44:34.242 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:44:34.242 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:44:34.243 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:44:34.243 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:44:34.243 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:44:34.243 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-10T21:44:34.243 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:44:34.243 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:44:34.243 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:44:34.243 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:44:34.244 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-10T21:44:34.244 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:44:34.244 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:44:34.244 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:44:34.244 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:44:34.244 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-10T21:44:34.244 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-10T21:44:34.244 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:44:34.244 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:44:34.244 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:44:34.244 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:44:34.245 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:44:34.245 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:44:34.245 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:44:34.245 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:44:34.245 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:44:34.245 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:44:34.245 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:44:34.245 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3300", 2024-09-10T21:44:34.245 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:44:34.245 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:44:34.245 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:44:34.246 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:44:34.246 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789", 2024-09-10T21:44:34.246 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:44:34.246 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:44:34.246 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:44:34.246 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:44:34.246 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789/0", 2024-09-10T21:44:34.246 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6789/0", 2024-09-10T21:44:34.246 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:44:34.246 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:44:34.246 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:44:34.246 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:44:34.246 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:44:34.246 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:44:34.247 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:44:34.247 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:44:34.247 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:44:34.247 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:44:34.247 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:44:34.247 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3301", 2024-09-10T21:44:34.247 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:44:34.247 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:44:34.247 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:44:34.247 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:44:34.247 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790", 2024-09-10T21:44:34.247 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:44:34.247 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:44:34.248 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:44:34.248 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:44:34.248 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790/0", 2024-09-10T21:44:34.248 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6790/0", 2024-09-10T21:44:34.248 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:44:34.248 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:44:34.248 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:44:34.248 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:44:34.248 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:44:34.248 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:44:34.248 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-10T21:44:34.248 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-10T21:44:34.249 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:44:34.249 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:44:34.249 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:44:34.249 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:44:34.249 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:44:34.249 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:44:34.249 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": [ 2024-09-10T21:44:34.249 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:44:34.249 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:44:34.249 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:44:34.249 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 4 2024-09-10T21:44:34.249 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:44:34.249 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:44:34.250 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-10T21:44:34.250 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:44:34.250 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:44:34.250 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:44:34.250 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 3 2024-09-10T21:44:34.250 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:44:34.250 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:44:34.250 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-10T21:44:34.250 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:44:34.250 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:44:34.250 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:44:34.250 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 2 2024-09-10T21:44:34.250 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:44:34.250 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:44:34.251 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": [ 2024-09-10T21:44:34.251 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:44:34.251 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:44:34.251 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:44:34.251 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:44:34.251 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:44:34.251 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:44:34.251 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:44:34.251 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-10T21:44:34.251 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-10T21:44:34.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:34.226+0000 7fe324200640 1 -- 172.21.15.17:0/1425711122 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fe2fc0722b0 msgr2=0x7fe2fc074770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:34.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:34.226+0000 7fe324200640 1 --2- 172.21.15.17:0/1425711122 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fe2fc0722b0 0x7fe2fc074770 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fe30800dc00 tx=0x7fe308012000 comp rx=0 tx=0).stop 2024-09-10T21:44:34.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:34.226+0000 7fe324200640 1 -- 172.21.15.17:0/1425711122 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe31c156750 msgr2=0x7fe31c1f8050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:34.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:34.226+0000 7fe324200640 1 --2- 172.21.15.17:0/1425711122 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe31c156750 0x7fe31c1f8050 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fe31400c7b0 tx=0x7fe31400cc70 comp rx=0 tx=0).stop 2024-09-10T21:44:34.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:34.226+0000 7fe322200640 1 -- 172.21.15.17:0/1425711122 reap_dead start 2024-09-10T21:44:34.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:34.226+0000 7fe322200640 1 -- 172.21.15.17:0/1425711122 reap_dead start 2024-09-10T21:44:34.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:34.226+0000 7fe324200640 1 -- 172.21.15.17:0/1425711122 shutdown_connections 2024-09-10T21:44:34.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:34.226+0000 7fe324200640 1 -- 172.21.15.17:0/1425711122 >> 172.21.15.17:0/1425711122 conn(0x7fe31c0700d0 msgr2=0x7fe31c0caf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:44:34.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:34.226+0000 7fe324200640 1 -- 172.21.15.17:0/1425711122 shutdown_connections 2024-09-10T21:44:34.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:34.234+0000 7fe324200640 1 -- 172.21.15.17:0/1425711122 wait complete. 2024-09-10T21:44:34.273 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-10T21:44:34.273 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-09-10T21:44:34.273 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-10T21:44:34.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:34.558+0000 7f8f2a400640 1 -- 172.21.15.17:0/2950429084 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8f240c9540 msgr2=0x7f8f24153940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:34.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:34.558+0000 7f8f2a400640 1 --2- 172.21.15.17:0/2950429084 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8f240c9540 0x7f8f24153940 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f8f1c00bbe0 tx=0x7f8f1c023310 comp rx=0 tx=0).stop 2024-09-10T21:44:34.574 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:34.570+0000 7f8f2a400640 1 -- 172.21.15.17:0/2950429084 shutdown_connections 2024-09-10T21:44:34.574 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:34.570+0000 7f8f2a400640 1 --2- 172.21.15.17:0/2950429084 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8f240c9540 0x7f8f24153940 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:34.574 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:34.570+0000 7f8f2a400640 1 --2- 172.21.15.17:0/2950429084 >> v2:172.21.15.17:3300/0 conn(0x7f8f24074980 0x7f8f240b1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:34.574 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:34.570+0000 7f8f2a400640 1 -- 172.21.15.17:0/2950429084 >> 172.21.15.17:0/2950429084 conn(0x7f8f240700d0 msgr2=0x7f8f240704e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:44:34.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:34.578+0000 7f8f2a400640 1 -- 172.21.15.17:0/2950429084 shutdown_connections 2024-09-10T21:44:34.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:34.578+0000 7f8f2a400640 1 -- 172.21.15.17:0/2950429084 wait complete. 2024-09-10T21:44:34.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:34.582+0000 7f8f2a400640 1 Processor -- start 2024-09-10T21:44:34.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:34.586+0000 7f8f2a400640 1 -- start start 2024-09-10T21:44:34.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:34.586+0000 7f8f2a400640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8f24074980 0x7f8f241f80e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:34.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:34.590+0000 7f8f2a400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f240c9160 0x7f8f24154aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:34.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:34.590+0000 7f8f23400640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8f24074980 0x7f8f241f80e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:34.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:34.590+0000 7f8f23400640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8f24074980 0x7f8f241f80e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3301/0 says I am v2:172.21.15.17:44904/0 (socket says 172.21.15.17:44904) 2024-09-10T21:44:34.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:34.590+0000 7f8f22a00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f240c9160 0x7f8f24154aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:34.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:34.590+0000 7f8f23400640 1 -- 172.21.15.17:0/1303512845 learned_addr learned my addr 172.21.15.17:0/1303512845 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:44:34.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:34.590+0000 7f8f2a400640 1 --2- 172.21.15.17:0/1303512845 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8f240c9540 0x7f8f24154fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:34.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:34.590+0000 7f8f2a400640 1 -- 172.21.15.17:0/1303512845 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f8f24073e70 con 0x7f8f240c9540 2024-09-10T21:44:34.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:34.590+0000 7f8f2a400640 1 -- 172.21.15.17:0/1303512845 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f8f24073b90 con 0x7f8f240c9160 2024-09-10T21:44:34.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:34.590+0000 7f8f2a400640 1 -- 172.21.15.17:0/1303512845 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f8f24073890 con 0x7f8f24074980 2024-09-10T21:44:34.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:34.590+0000 7f8f23e00640 1 --2- 172.21.15.17:0/1303512845 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8f240c9540 0x7f8f24154fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:34.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:34.590+0000 7f8f22a00640 1 -- 172.21.15.17:0/1303512845 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8f24074980 msgr2=0x7f8f241f80e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:34.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:34.590+0000 7f8f22a00640 1 --2- 172.21.15.17:0/1303512845 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8f24074980 0x7f8f241f80e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:34.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:34.590+0000 7f8f22a00640 1 -- 172.21.15.17:0/1303512845 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8f240c9540 msgr2=0x7f8f24154fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:34.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:34.590+0000 7f8f22a00640 1 --2- 172.21.15.17:0/1303512845 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8f240c9540 0x7f8f24154fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:34.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:34.590+0000 7f8f22a00640 1 -- 172.21.15.17:0/1303512845 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f24153b70 con 0x7f8f240c9160 2024-09-10T21:44:34.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:34.594+0000 7f8f23400640 1 --2- 172.21.15.17:0/1303512845 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8f24074980 0x7f8f241f80e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:44:34.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:34.594+0000 7f8f22a00640 1 --2- 172.21.15.17:0/1303512845 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f240c9160 0x7f8f24154aa0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f8f10007860 tx=0x7f8f10007d20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:44:34.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:34.594+0000 7f8f1b400640 1 -- 172.21.15.17:0/1303512845 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f10017030 con 0x7f8f240c9160 2024-09-10T21:44:34.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:34.594+0000 7f8f1b400640 1 -- 172.21.15.17:0/1303512845 <== mon.1 v2:172.21.15.31:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8f10007e50 con 0x7f8f240c9160 2024-09-10T21:44:34.599 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:34.594+0000 7f8f1b400640 1 -- 172.21.15.17:0/1303512845 <== mon.1 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f1000db10 con 0x7f8f240c9160 2024-09-10T21:44:34.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:34.598+0000 7f8f2a400640 1 -- 172.21.15.17:0/1303512845 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f241f8a20 con 0x7f8f240c9160 2024-09-10T21:44:34.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:34.598+0000 7f8f2a400640 1 -- 172.21.15.17:0/1303512845 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f2414cb80 con 0x7f8f240c9160 2024-09-10T21:44:34.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:34.602+0000 7f8f18c00640 1 --2- 172.21.15.17:0/1303512845 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8efc002a80 0x7f8efc004f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:34.604 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:34.602+0000 7f8f18c00640 1 -- 172.21.15.17:0/1303512845 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f8efc007ed0 con 0x7f8efc002a80 2024-09-10T21:44:34.606 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:34.602+0000 7f8f23400640 1 --2- 172.21.15.17:0/1303512845 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8efc002a80 0x7f8efc004f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:34.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:34.610+0000 7f8f23400640 1 -- 172.21.15.17:0/1303512845 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8efc0080d0 con 0x7f8efc002a80 2024-09-10T21:44:34.613 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:34.610+0000 7f8f23400640 1 --2- 172.21.15.17:0/1303512845 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8efc002a80 0x7f8efc004f40 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f8f14001250 tx=0x7f8f1400db80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:44:34.613 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:34.610+0000 7f8f1b400640 1 -- 172.21.15.17:0/1303512845 <== mon.1 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f8f1000dcb0 con 0x7f8f240c9160 2024-09-10T21:44:34.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:34.610+0000 7f8f1b400640 1 --2- 172.21.15.17:0/1303512845 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f8f04072660 0x7f8f04074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:34.616 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:34.614+0000 7f8f23e00640 1 --2- 172.21.15.17:0/1303512845 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f8f04072660 0x7f8f04074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:34.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:34.614+0000 7f8f1b400640 1 -- 172.21.15.17:0/1303512845 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f14016070 con 0x7f8efc002a80 2024-09-10T21:44:34.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:34.614+0000 7f8f23e00640 1 --2- 172.21.15.17:0/1303512845 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f8f04072660 0x7f8f04074b20 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f8f1c0b5000 tx=0x7f8f1c021b50 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:44:34.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:34.618+0000 7f8f1b400640 1 -- 172.21.15.17:0/1303512845 <== mon.1 v2:172.21.15.31:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f8f10092ee0 con 0x7f8f240c9160 2024-09-10T21:44:34.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:34.618+0000 7f8f1b400640 1 -- 172.21.15.17:0/1303512845 <== mon.1 v2:172.21.15.31:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f8f1400b830 con 0x7f8efc002a80 2024-09-10T21:44:34.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:34.618+0000 7f8f1b400640 1 -- 172.21.15.17:0/1303512845 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8efc002a80 msgr2=0x7f8efc004f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:34.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:34.618+0000 7f8f1b400640 1 --2- 172.21.15.17:0/1303512845 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8efc002a80 0x7f8efc004f40 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f8f14001250 tx=0x7f8f1400db80 comp rx=0 tx=0).stop 2024-09-10T21:44:34.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:34.618+0000 7f8f1b400640 1 -- 172.21.15.17:0/1303512845 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8efc002a80 msgr2=0x7f8efc004f40 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:44:34.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:34.618+0000 7f8f1b400640 1 --2- 172.21.15.17:0/1303512845 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8efc002a80 0x7f8efc004f40 secure :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f8f14001250 tx=0x7f8f1400db80 comp rx=0 tx=0).stop 2024-09-10T21:44:34.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:34.630+0000 7f8f18c00640 1 --2- 172.21.15.17:0/1303512845 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8efc006f60 0x7f8efc0073f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:34.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:34.630+0000 7f8f18c00640 1 -- 172.21.15.17:0/1303512845 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f8efc00a4b0 con 0x7f8efc006f60 2024-09-10T21:44:34.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:34.630+0000 7f8f23400640 1 --2- 172.21.15.17:0/1303512845 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8efc006f60 0x7f8efc0073f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:34.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:34.634+0000 7f8f23400640 1 -- 172.21.15.17:0/1303512845 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f8efc007930 con 0x7f8efc006f60 2024-09-10T21:44:34.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:34.638+0000 7f8f23400640 1 --2- 172.21.15.17:0/1303512845 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8efc006f60 0x7f8efc0073f0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f8f14001ec0 tx=0x7f8f14001340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:44:34.644 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:34.642+0000 7f8f1b400640 1 -- 172.21.15.17:0/1303512845 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f1401bde0 con 0x7f8efc006f60 2024-09-10T21:44:34.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:34.646+0000 7f8f1b400640 1 -- 172.21.15.17:0/1303512845 <== mon.1 v2:172.21.15.31:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4542 (secure 0 0 0) 0x7f8f140123f0 con 0x7f8efc006f60 2024-09-10T21:44:34.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:34.646+0000 7f8f1b400640 1 -- 172.21.15.17:0/1303512845 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8efc006f60 msgr2=0x7f8efc0073f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:34.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:34.646+0000 7f8f1b400640 1 --2- 172.21.15.17:0/1303512845 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8efc006f60 0x7f8efc0073f0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f8f14001ec0 tx=0x7f8f14001340 comp rx=0 tx=0).stop 2024-09-10T21:44:34.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:34.646+0000 7f8f1b400640 1 -- 172.21.15.17:0/1303512845 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8efc006f60 msgr2=0x7f8efc0073f0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:44:34.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:34.646+0000 7f8f1b400640 1 --2- 172.21.15.17:0/1303512845 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8efc006f60 0x7f8efc0073f0 secure :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f8f14001ec0 tx=0x7f8f14001340 comp rx=0 tx=0).stop 2024-09-10T21:44:34.654 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-10T21:44:34.654 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:44:34.654 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:44:34.654 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-10T21:44:34.654 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 150, 2024-09-10T21:44:34.654 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 19623, 2024-09-10T21:44:34.654 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-10T21:44:34.654 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-10T21:44:34.654 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-10T21:44:34.654 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-10T21:44:34.654 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:44:34.654 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 15, 2024-09-10T21:44:34.655 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:44:34.655 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-10T21:44:34.655 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-10T21:44:34.655 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:44:34.655 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:44:34.655 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:44:34.655 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:44:34.655 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:44:34.655 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:44:34.655 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:44:34.655 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:44:34.655 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:44:34.655 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:44:34.656 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:44:34.656 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:44:34.656 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-10T21:44:34.656 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-10T21:44:34.656 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:44:34.656 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:44:34.656 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:44:34.656 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:44:34.656 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:44:34.656 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:44:34.656 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:44:34.656 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:44:34.656 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:44:34.656 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:44:34.657 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:44:34.657 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:44:34.657 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:44:34.657 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-10T21:44:34.657 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-10T21:44:34.657 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-10T21:44:34.657 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-10T21:44:34.657 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-10T21:44:34.657 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "5e2f023f-d6a6-49f4-a256-f4807dce3739", 2024-09-10T21:44:34.657 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:44:34.657 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:44:34.657 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-10T21:44:34.657 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-10T21:44:34.657 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-10T21:44:34.658 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-10T21:44:34.658 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-10T21:44:34.658 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-10T21:44:34.658 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-10T21:44:34.658 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:44:34.658 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-10T21:44:34.658 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:44:34.658 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:44:34.658 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:44:34.658 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:44:34.658 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:44:34.658 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:44:34.658 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:44:34.659 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:44:34.659 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:44:34.659 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:44:34.659 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:44:34.659 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:44:34.659 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-10T21:44:34.659 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:44:34.659 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-10T21:44:34.659 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:44:34.659 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:44:34.659 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:44:34.659 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:44:34.659 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:44:34.659 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:44:34.660 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:44:34.660 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-10T21:44:34.660 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:44:34.660 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:44:34.660 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:44:34.660 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:44:34.660 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-10T21:44:34.660 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:44:34.660 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:44:34.660 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:44:34.660 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:44:34.660 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-10T21:44:34.660 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-10T21:44:34.660 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:44:34.661 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:44:34.661 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:44:34.661 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:44:34.661 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:44:34.661 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:44:34.661 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:44:34.661 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:44:34.661 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:44:34.661 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:44:34.661 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:44:34.661 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3300", 2024-09-10T21:44:34.661 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:44:34.661 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:44:34.661 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:44:34.662 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:44:34.662 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789", 2024-09-10T21:44:34.662 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:44:34.662 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:44:34.662 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:44:34.662 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:44:34.662 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789/0", 2024-09-10T21:44:34.662 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6789/0", 2024-09-10T21:44:34.662 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:44:34.662 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:44:34.662 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:44:34.662 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:44:34.662 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:44:34.663 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:44:34.663 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:44:34.663 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:44:34.663 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:44:34.663 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:44:34.663 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:44:34.663 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3301", 2024-09-10T21:44:34.663 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:44:34.663 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:44:34.663 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:44:34.663 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:44:34.663 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790", 2024-09-10T21:44:34.663 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:44:34.663 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:44:34.664 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:44:34.664 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:44:34.664 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790/0", 2024-09-10T21:44:34.664 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6790/0", 2024-09-10T21:44:34.664 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:44:34.664 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:44:34.664 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:44:34.664 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:44:34.664 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:44:34.664 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:44:34.664 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-10T21:44:34.664 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-10T21:44:34.664 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:44:34.665 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:44:34.665 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:44:34.665 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:44:34.665 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:44:34.665 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:44:34.665 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-10T21:44:34.665 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:44:34.665 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:44:34.665 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:44:34.665 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 2 2024-09-10T21:44:34.665 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:44:34.665 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:44:34.665 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:44:34.665 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-10T21:44:34.666 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-10T21:44:34.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:34.654+0000 7f8f18c00640 1 -- 172.21.15.17:0/1303512845 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f8f04072660 msgr2=0x7f8f04074b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:34.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:34.654+0000 7f8f18c00640 1 --2- 172.21.15.17:0/1303512845 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f8f04072660 0x7f8f04074b20 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f8f1c0b5000 tx=0x7f8f1c021b50 comp rx=0 tx=0).stop 2024-09-10T21:44:34.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:34.654+0000 7f8f23e00640 1 -- 172.21.15.17:0/1303512845 reap_dead start 2024-09-10T21:44:34.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:34.654+0000 7f8f18c00640 1 -- 172.21.15.17:0/1303512845 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f240c9160 msgr2=0x7f8f24154aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:34.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:34.654+0000 7f8f18c00640 1 --2- 172.21.15.17:0/1303512845 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f240c9160 0x7f8f24154aa0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f8f10007860 tx=0x7f8f10007d20 comp rx=0 tx=0).stop 2024-09-10T21:44:34.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:34.654+0000 7f8f18c00640 1 -- 172.21.15.17:0/1303512845 shutdown_connections 2024-09-10T21:44:34.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:34.654+0000 7f8f18c00640 1 --2- 172.21.15.17:0/1303512845 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8f240c9160 0x7f8f24154aa0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:34.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:34.654+0000 7f8f18c00640 1 -- 172.21.15.17:0/1303512845 >> 172.21.15.17:0/1303512845 conn(0x7f8f240700d0 msgr2=0x7f8f240ca190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:44:34.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:34.654+0000 7f8f18c00640 1 -- 172.21.15.17:0/1303512845 shutdown_connections 2024-09-10T21:44:34.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:34.658+0000 7f8f18c00640 1 -- 172.21.15.17:0/1303512845 wait complete. 2024-09-10T21:44:34.704 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-10T21:44:34.704 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-09-10T21:44:34.704 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-10T21:44:35.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:35.102+0000 7fbed0e00640 1 -- 172.21.15.17:0/3815582918 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fbec4035f40 msgr2=0x7fbec4046ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:35.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:35.102+0000 7fbed0e00640 1 --2- 172.21.15.17:0/3815582918 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fbec4035f40 0x7fbec4046ec0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fbecc06b290 tx=0x7fbecc098ff0 comp rx=0 tx=0).stop 2024-09-10T21:44:35.105 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:35.102+0000 7fbed0e00640 1 -- 172.21.15.17:0/3815582918 shutdown_connections 2024-09-10T21:44:35.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:35.102+0000 7fbed0e00640 1 --2- 172.21.15.17:0/3815582918 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fbec4035f40 0x7fbec4046ec0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:35.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:35.102+0000 7fbed0e00640 1 --2- 172.21.15.17:0/3815582918 >> v2:172.21.15.31:3300/0 conn(0x7fbec4036490 0x7fbec4046980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:35.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:35.102+0000 7fbed0e00640 1 --2- 172.21.15.17:0/3815582918 >> v2:172.21.15.17:3300/0 conn(0x7fbec40d87e0 0x7fbec4045880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:35.106 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:35.102+0000 7fbed0e00640 1 -- 172.21.15.17:0/3815582918 >> 172.21.15.17:0/3815582918 conn(0x7fbec40504a0 msgr2=0x7fbec40508b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:44:35.118 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:35.114+0000 7fbed0e00640 1 -- 172.21.15.17:0/3815582918 shutdown_connections 2024-09-10T21:44:35.121 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:35.118+0000 7fbed0e00640 1 -- 172.21.15.17:0/3815582918 wait complete. 2024-09-10T21:44:35.123 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:35.118+0000 7fbed0e00640 1 Processor -- start 2024-09-10T21:44:35.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:35.134+0000 7fbed0e00640 1 -- start start 2024-09-10T21:44:35.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:35.134+0000 7fbed0e00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbec4035f40 0x7fbec40cb0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:35.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:35.134+0000 7fbed0e00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fbec40d87e0 0x7fbec40d2a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:35.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:35.134+0000 7fbed0e00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbec40d2f40 0x7fbec40e0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:35.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:35.134+0000 7fbed0e00640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fbec40de860 con 0x7fbec4035f40 2024-09-10T21:44:35.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:35.134+0000 7fbed0e00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fbec40de6e0 con 0x7fbec40d2f40 2024-09-10T21:44:35.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:35.134+0000 7fbed0e00640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7fbec40de9e0 con 0x7fbec40d87e0 2024-09-10T21:44:35.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:35.134+0000 7fbecbe00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbec40d2f40 0x7fbec40e0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:35.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:35.138+0000 7fbecbe00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbec40d2f40 0x7fbec40e0ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.17:36436/0 (socket says 172.21.15.17:36436) 2024-09-10T21:44:35.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:35.138+0000 7fbecbe00640 1 -- 172.21.15.17:0/2302988254 learned_addr learned my addr 172.21.15.17:0/2302988254 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:44:35.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:35.138+0000 7fbecbe00640 1 -- 172.21.15.17:0/2302988254 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fbec40d87e0 msgr2=0x7fbec40d2a00 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-10T21:44:35.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:35.138+0000 7fbecbe00640 1 --2- 172.21.15.17:0/2302988254 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fbec40d87e0 0x7fbec40d2a00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:35.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:35.138+0000 7fbecbe00640 1 -- 172.21.15.17:0/2302988254 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbec4035f40 msgr2=0x7fbec40cb0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:35.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:35.138+0000 7fbecbe00640 1 --2- 172.21.15.17:0/2302988254 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbec4035f40 0x7fbec40cb0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:35.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:35.138+0000 7fbecbe00640 1 -- 172.21.15.17:0/2302988254 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbec40303e0 con 0x7fbec40d2f40 2024-09-10T21:44:35.144 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:35.138+0000 7fbecbe00640 1 --2- 172.21.15.17:0/2302988254 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbec40d2f40 0x7fbec40e0ca0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fbecc052200 tx=0x7fbecc06a720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:44:35.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:35.142+0000 7fbebf400640 1 -- 172.21.15.17:0/2302988254 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbecc063a00 con 0x7fbec40d2f40 2024-09-10T21:44:35.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:35.142+0000 7fbebf400640 1 -- 172.21.15.17:0/2302988254 <== mon.1 v2:172.21.15.31:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbecc063c90 con 0x7fbec40d2f40 2024-09-10T21:44:35.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:35.142+0000 7fbed0e00640 1 -- 172.21.15.17:0/2302988254 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbec4049890 con 0x7fbec40d2f40 2024-09-10T21:44:35.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:35.142+0000 7fbebf400640 1 -- 172.21.15.17:0/2302988254 <== mon.1 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbecc1346b0 con 0x7fbec40d2f40 2024-09-10T21:44:35.146 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:35.142+0000 7fbed0e00640 1 -- 172.21.15.17:0/2302988254 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbec404a310 con 0x7fbec40d2f40 2024-09-10T21:44:35.149 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:35.146+0000 7fbed0e00640 1 --2- 172.21.15.17:0/2302988254 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fbea8002a80 0x7fbea8004f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:35.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:35.146+0000 7fbed0e00640 1 -- 172.21.15.17:0/2302988254 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7fbea8007ed0 con 0x7fbea8002a80 2024-09-10T21:44:35.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:35.150+0000 7fbecb400640 1 --2- 172.21.15.17:0/2302988254 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fbea8002a80 0x7fbea8004f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:35.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:35.150+0000 7fbecb400640 1 -- 172.21.15.17:0/2302988254 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fbea80080d0 con 0x7fbea8002a80 2024-09-10T21:44:35.154 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:35.150+0000 7fbecb400640 1 --2- 172.21.15.17:0/2302988254 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fbea8002a80 0x7fbea8004f40 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fbec000b340 tx=0x7fbec000b800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:44:35.155 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:35.154+0000 7fbebf400640 1 -- 172.21.15.17:0/2302988254 <== mon.1 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fbecc151af0 con 0x7fbec40d2f40 2024-09-10T21:44:35.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:35.154+0000 7fbebf400640 1 --2- 172.21.15.17:0/2302988254 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fbeac0725e0 0x7fbeac074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:35.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:35.154+0000 7fbebf400640 1 -- 172.21.15.17:0/2302988254 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbec0015020 con 0x7fbea8002a80 2024-09-10T21:44:35.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:35.154+0000 7fbebf400640 1 -- 172.21.15.17:0/2302988254 <== mon.1 v2:172.21.15.31:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fbecc1a8d10 con 0x7fbec40d2f40 2024-09-10T21:44:35.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:35.154+0000 7fbebf400640 1 -- 172.21.15.17:0/2302988254 <== mon.2 v2:172.21.15.31:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fbec00118f0 con 0x7fbea8002a80 2024-09-10T21:44:35.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:35.154+0000 7fbebf400640 1 -- 172.21.15.17:0/2302988254 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fbea8002a80 msgr2=0x7fbea8004f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:35.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:35.154+0000 7fbebf400640 1 --2- 172.21.15.17:0/2302988254 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fbea8002a80 0x7fbea8004f40 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fbec000b340 tx=0x7fbec000b800 comp rx=0 tx=0).stop 2024-09-10T21:44:35.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:35.154+0000 7fbebf400640 1 -- 172.21.15.17:0/2302988254 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fbea8002a80 msgr2=0x7fbea8004f40 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:44:35.159 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:35.154+0000 7fbebf400640 1 --2- 172.21.15.17:0/2302988254 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fbea8002a80 0x7fbea8004f40 secure :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fbec000b340 tx=0x7fbec000b800 comp rx=0 tx=0).stop 2024-09-10T21:44:35.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:35.158+0000 7fbecaa00640 1 --2- 172.21.15.17:0/2302988254 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fbeac0725e0 0x7fbeac074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:35.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:35.166+0000 7fbecaa00640 1 --2- 172.21.15.17:0/2302988254 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fbeac0725e0 0x7fbeac074aa0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fbec40311c0 tx=0x7fbeb8005b90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:44:35.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:35.170+0000 7fbed0e00640 1 --2- 172.21.15.17:0/2302988254 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fbea8006f60 0x7fbea80073f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:35.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:35.170+0000 7fbed0e00640 1 -- 172.21.15.17:0/2302988254 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7fbea800a4b0 con 0x7fbea8006f60 2024-09-10T21:44:35.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:35.178+0000 7fbecb400640 1 --2- 172.21.15.17:0/2302988254 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fbea8006f60 0x7fbea80073f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:35.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:35.182+0000 7fbecb400640 1 -- 172.21.15.17:0/2302988254 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fbea8007930 con 0x7fbea8006f60 2024-09-10T21:44:35.187 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:35.182+0000 7fbecb400640 1 --2- 172.21.15.17:0/2302988254 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fbea8006f60 0x7fbea80073f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fbec0016000 tx=0x7fbec0004740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:44:35.189 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-10T21:44:35.190 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:44:35.190 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:44:35.190 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-10T21:44:35.190 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 150, 2024-09-10T21:44:35.190 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 77318, 2024-09-10T21:44:35.190 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-10T21:44:35.190 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-10T21:44:35.190 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-10T21:44:35.190 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-10T21:44:35.190 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:44:35.190 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 16, 2024-09-10T21:44:35.190 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:44:35.190 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-10T21:44:35.191 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-10T21:44:35.191 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:44:35.191 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:44:35.191 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:44:35.191 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:44:35.191 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:44:35.191 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:44:35.191 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:44:35.191 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:44:35.192 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:44:35.192 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:44:35.192 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:44:35.192 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:44:35.192 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-10T21:44:35.192 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-10T21:44:35.192 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:44:35.192 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:44:35.192 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:44:35.192 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:44:35.192 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:44:35.192 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:44:35.192 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:44:35.192 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:44:35.193 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:44:35.193 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:44:35.193 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:44:35.193 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:44:35.193 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:44:35.193 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-10T21:44:35.193 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-10T21:44:35.193 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-10T21:44:35.193 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-10T21:44:35.193 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-10T21:44:35.193 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "5e2f023f-d6a6-49f4-a256-f4807dce3739", 2024-09-10T21:44:35.193 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:44:35.193 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:44:35.193 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-10T21:44:35.194 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-10T21:44:35.194 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-10T21:44:35.194 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-10T21:44:35.194 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-10T21:44:35.194 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-10T21:44:35.194 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-10T21:44:35.194 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:44:35.194 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-10T21:44:35.194 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:44:35.194 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:44:35.194 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:44:35.194 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:44:35.194 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:44:35.194 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:44:35.195 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:44:35.195 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:44:35.195 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:44:35.195 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:44:35.195 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:44:35.195 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:44:35.195 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-10T21:44:35.195 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:44:35.195 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-10T21:44:35.195 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:44:35.195 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:44:35.195 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:44:35.195 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:44:35.196 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:44:35.196 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:44:35.196 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:44:35.196 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-10T21:44:35.196 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:44:35.196 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:44:35.196 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:44:35.196 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:44:35.196 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-10T21:44:35.196 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:44:35.196 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:44:35.196 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:44:35.196 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:44:35.197 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-10T21:44:35.197 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-10T21:44:35.197 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:44:35.197 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:44:35.197 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:44:35.197 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:44:35.197 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:44:35.197 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:44:35.197 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:44:35.197 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:44:35.197 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:44:35.197 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:44:35.197 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:44:35.198 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3300", 2024-09-10T21:44:35.198 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:44:35.198 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:44:35.198 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:44:35.198 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:44:35.198 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789", 2024-09-10T21:44:35.198 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:44:35.198 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:44:35.198 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:44:35.198 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:44:35.199 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789/0", 2024-09-10T21:44:35.199 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6789/0", 2024-09-10T21:44:35.199 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:44:35.199 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:44:35.199 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:44:35.199 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:44:35.199 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:44:35.199 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:44:35.199 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:44:35.199 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:44:35.199 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:44:35.199 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:44:35.199 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:44:35.200 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3301", 2024-09-10T21:44:35.200 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:44:35.200 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:44:35.200 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:44:35.200 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:44:35.200 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790", 2024-09-10T21:44:35.200 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:44:35.200 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:44:35.200 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:44:35.200 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:44:35.200 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790/0", 2024-09-10T21:44:35.200 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6790/0", 2024-09-10T21:44:35.200 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:44:35.200 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:44:35.201 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:44:35.201 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:44:35.201 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:44:35.201 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:44:35.201 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-10T21:44:35.201 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-10T21:44:35.201 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:44:35.201 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:44:35.201 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:44:35.202 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:44:35.202 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:44:35.202 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:44:35.202 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": [ 2024-09-10T21:44:35.202 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:44:35.202 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:44:35.202 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:44:35.202 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 2 2024-09-10T21:44:35.202 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:44:35.202 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:44:35.202 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-10T21:44:35.202 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:44:35.202 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:44:35.202 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:44:35.202 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 5 2024-09-10T21:44:35.203 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:44:35.203 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:44:35.203 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-10T21:44:35.203 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:44:35.203 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:44:35.203 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:44:35.203 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 5 2024-09-10T21:44:35.203 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:44:35.203 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:44:35.203 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": [ 2024-09-10T21:44:35.203 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:44:35.203 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:44:35.203 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:44:35.204 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:44:35.204 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:44:35.204 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:44:35.204 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:44:35.204 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-10T21:44:35.204 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-10T21:44:35.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:35.186+0000 7fbebf400640 1 -- 172.21.15.17:0/2302988254 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbec0015040 con 0x7fbea8006f60 2024-09-10T21:44:35.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:35.186+0000 7fbebf400640 1 -- 172.21.15.17:0/2302988254 <== mon.2 v2:172.21.15.31:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5043 (secure 0 0 0) 0x7fbec001c220 con 0x7fbea8006f60 2024-09-10T21:44:35.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:35.186+0000 7fbebf400640 1 -- 172.21.15.17:0/2302988254 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fbea8006f60 msgr2=0x7fbea80073f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:35.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:35.186+0000 7fbebf400640 1 --2- 172.21.15.17:0/2302988254 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fbea8006f60 0x7fbea80073f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fbec0016000 tx=0x7fbec0004740 comp rx=0 tx=0).stop 2024-09-10T21:44:35.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:35.186+0000 7fbebf400640 1 -- 172.21.15.17:0/2302988254 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fbea8006f60 msgr2=0x7fbea80073f0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:44:35.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:35.186+0000 7fbebf400640 1 --2- 172.21.15.17:0/2302988254 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fbea8006f60 0x7fbea80073f0 secure :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fbec0016000 tx=0x7fbec0004740 comp rx=0 tx=0).stop 2024-09-10T21:44:35.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:35.190+0000 7fbed0e00640 1 -- 172.21.15.17:0/2302988254 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fbeac0725e0 msgr2=0x7fbeac074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:35.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:35.190+0000 7fbed0e00640 1 --2- 172.21.15.17:0/2302988254 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fbeac0725e0 0x7fbeac074aa0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fbec40311c0 tx=0x7fbeb8005b90 comp rx=0 tx=0).stop 2024-09-10T21:44:35.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:35.190+0000 7fbed0e00640 1 -- 172.21.15.17:0/2302988254 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbec40d2f40 msgr2=0x7fbec40e0ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:35.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:35.190+0000 7fbed0e00640 1 --2- 172.21.15.17:0/2302988254 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbec40d2f40 0x7fbec40e0ca0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fbecc052200 tx=0x7fbecc06a720 comp rx=0 tx=0).stop 2024-09-10T21:44:35.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:35.194+0000 7fbecbe00640 1 -- 172.21.15.17:0/2302988254 reap_dead start 2024-09-10T21:44:35.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:35.194+0000 7fbecbe00640 1 -- 172.21.15.17:0/2302988254 reap_dead start 2024-09-10T21:44:35.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:35.194+0000 7fbed0e00640 1 -- 172.21.15.17:0/2302988254 shutdown_connections 2024-09-10T21:44:35.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:35.194+0000 7fbed0e00640 1 -- 172.21.15.17:0/2302988254 >> 172.21.15.17:0/2302988254 conn(0x7fbec40504a0 msgr2=0x7fbec404e360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:44:35.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:35.194+0000 7fbed0e00640 1 -- 172.21.15.17:0/2302988254 shutdown_connections 2024-09-10T21:44:35.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:35.194+0000 7fbed0e00640 1 -- 172.21.15.17:0/2302988254 wait complete. 2024-09-10T21:44:35.249 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-10T21:44:35.249 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['c'] 2024-09-10T21:44:35.249 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: [] 2024-09-10T21:44:35.249 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.c 2024-09-10T21:44:35.249 INFO:tasks.mon_thrash.mon_thrasher:killing mon.c 2024-09-10T21:44:35.250 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-10T21:44:35.250 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:44:35.279 INFO:tasks.ceph.mon.c:Stopped 2024-09-10T21:44:35.279 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2024-09-10T21:44:35.279 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-10T21:44:35.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:35.626+0000 7f5654a00640 1 -- 172.21.15.17:0/1138777642 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5658064650 con 0x7f563c003d60 2024-09-10T21:44:35.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:35.630+0000 7f5657200640 1 -- 172.21.15.17:0/1138777642 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f563c003d60 msgr2=0x7f563c0041e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:35.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:35.630+0000 7f5657200640 1 --2- 172.21.15.17:0/1138777642 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f563c003d60 0x7f563c0041e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f565806bf70 tx=0x7f56580990c0 comp rx=0 tx=0).stop 2024-09-10T21:44:35.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:35.630+0000 7f565d200640 1 -- 172.21.15.17:0/1138777642 reap_dead start 2024-09-10T21:44:35.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:35.630+0000 7f5657200640 1 -- 172.21.15.17:0/1138777642 shutdown_connections 2024-09-10T21:44:35.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:35.630+0000 7f5657200640 1 -- 172.21.15.17:0/1138777642 >> 172.21.15.17:0/1138777642 conn(0x7f565004fde0 msgr2=0x7f56500501c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:44:35.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:35.630+0000 7f5657200640 1 -- 172.21.15.17:0/1138777642 shutdown_connections 2024-09-10T21:44:35.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:35.630+0000 7f5657200640 1 -- 172.21.15.17:0/1138777642 wait complete. 2024-09-10T21:44:35.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:35.634+0000 7f5657200640 1 Processor -- start 2024-09-10T21:44:35.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:35.634+0000 7f5657200640 1 -- start start 2024-09-10T21:44:35.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:35.638+0000 7f5657200640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f563c003d60 0x7f56500d5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:35.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:35.638+0000 7f5657200640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5650047590 0x7f56500479b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:35.644 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:35.638+0000 7f5657200640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f56500cd650 0x7f56500cdad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:35.644 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:35.638+0000 7f5657200640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f56500508d0 con 0x7f5650047590 2024-09-10T21:44:35.644 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:35.638+0000 7f5657200640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f56500db050 con 0x7f56500cd650 2024-09-10T21:44:35.644 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:35.638+0000 7f5657200640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f565004be60 con 0x7f563c003d60 2024-09-10T21:44:35.644 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:35.638+0000 7f565d200640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f56500cd650 0x7f56500cdad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:35.644 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:35.638+0000 7f565d200640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f56500cd650 0x7f56500cdad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.17:36476/0 (socket says 172.21.15.17:36476) 2024-09-10T21:44:35.644 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:35.638+0000 7f565d200640 1 -- 172.21.15.17:0/2821543538 learned_addr learned my addr 172.21.15.17:0/2821543538 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:44:35.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:35.638+0000 7f5655400640 1 --2- 172.21.15.17:0/2821543538 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5650047590 0x7f56500479b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:35.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:35.638+0000 7f5655e00640 1 -- 172.21.15.17:0/2821543538 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f563c003d60 msgr2=0x7f56500d5140 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:3301/0 2024-09-10T21:44:35.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:35.642+0000 7f5655e00640 1 --2- 172.21.15.17:0/2821543538 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f563c003d60 0x7f56500d5140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:44:35.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:35.642+0000 7f565d200640 1 -- 172.21.15.17:0/2821543538 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f563c003d60 msgr2=0x7f56500d5140 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:44:35.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:35.642+0000 7f565d200640 1 --2- 172.21.15.17:0/2821543538 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f563c003d60 0x7f56500d5140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:35.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:35.642+0000 7f565d200640 1 -- 172.21.15.17:0/2821543538 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5650047590 msgr2=0x7f56500479b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:35.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:35.642+0000 7f565d200640 1 --2- 172.21.15.17:0/2821543538 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5650047590 0x7f56500479b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:35.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:35.642+0000 7f565d200640 1 -- 172.21.15.17:0/2821543538 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5650181f80 con 0x7f56500cd650 2024-09-10T21:44:35.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:35.642+0000 7f565d200640 1 --2- 172.21.15.17:0/2821543538 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f56500cd650 0x7f56500cdad0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f565806bf70 tx=0x7f56580669e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:44:35.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:35.642+0000 7f564e000640 1 -- 172.21.15.17:0/2821543538 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f565812e040 con 0x7f56500cd650 2024-09-10T21:44:35.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:35.642+0000 7f5657200640 1 -- 172.21.15.17:0/2821543538 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5650048860 con 0x7f56500cd650 2024-09-10T21:44:35.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:35.642+0000 7f5657200640 1 -- 172.21.15.17:0/2821543538 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5650048e00 con 0x7f56500cd650 2024-09-10T21:44:35.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:35.642+0000 7f564e000640 1 -- 172.21.15.17:0/2821543538 <== mon.1 v2:172.21.15.31:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5658063930 con 0x7f56500cd650 2024-09-10T21:44:35.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:35.642+0000 7f564e000640 1 -- 172.21.15.17:0/2821543538 <== mon.1 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f565812e040 con 0x7f56500cd650 2024-09-10T21:44:35.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:35.646+0000 7f5657200640 1 -- 172.21.15.17:0/2821543538 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56300052a0 con 0x7f56500cd650 2024-09-10T21:44:35.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:35.646+0000 7f564e000640 1 -- 172.21.15.17:0/2821543538 <== mon.1 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f5658131070 con 0x7f56500cd650 2024-09-10T21:44:35.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:35.646+0000 7f564e000640 1 --2- 172.21.15.17:0/2821543538 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f563c076ef0 0x7f563c0793b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:35.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:35.646+0000 7f564e000640 1 -- 172.21.15.17:0/2821543538 <== mon.1 v2:172.21.15.31:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f56581a8a40 con 0x7f56500cd650 2024-09-10T21:44:35.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:35.650+0000 7f5655e00640 1 --2- 172.21.15.17:0/2821543538 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f563c076ef0 0x7f563c0793b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:35.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:35.654+0000 7f5655e00640 1 --2- 172.21.15.17:0/2821543538 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f563c076ef0 0x7f563c0793b0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f5648006a60 tx=0x7f56480069f0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:44:35.661 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:35.654+0000 7f564e000640 1 -- 172.21.15.17:0/2821543538 <== mon.1 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f5658092f90 con 0x7f56500cd650 2024-09-10T21:44:36.309 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:36.306+0000 7f5657200640 1 -- 172.21.15.17:0/2821543538 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f5630005590 con 0x7f56500cd650 2024-09-10T21:44:36.320 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:36.314+0000 7f564e000640 1 -- 172.21.15.17:0/2821543538 <== mon.1 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1558 (secure 0 0 0) 0x7f565817c3a0 con 0x7f56500cd650 2024-09-10T21:44:36.321 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":150,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":17,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5e2f023f-d6a6-49f4-a256-f4807dce3739","modified":"2024-09-10T21:23:46.892130Z","created":"2024-09-10T21:23:46.892130Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3301","nonce":0},{"type":"v1","addr":"172.21.15.31:6790","nonce":0}]},"addr":"172.21.15.31:6790/0","public_addr":"172.21.15.31:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-10T21:44:36.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:36.322+0000 7f5657200640 1 -- 172.21.15.17:0/2821543538 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f563c076ef0 msgr2=0x7f563c0793b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:36.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:36.322+0000 7f5657200640 1 --2- 172.21.15.17:0/2821543538 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f563c076ef0 0x7f563c0793b0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f5648006a60 tx=0x7f56480069f0 comp rx=0 tx=0).stop 2024-09-10T21:44:36.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:36.322+0000 7f5657200640 1 -- 172.21.15.17:0/2821543538 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f56500cd650 msgr2=0x7f56500cdad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:36.327 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:36.322+0000 7f5657200640 1 --2- 172.21.15.17:0/2821543538 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f56500cd650 0x7f56500cdad0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f565806bf70 tx=0x7f56580669e0 comp rx=0 tx=0).stop 2024-09-10T21:44:36.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:36.326+0000 7f5657200640 1 -- 172.21.15.17:0/2821543538 shutdown_connections 2024-09-10T21:44:36.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:36.326+0000 7f5657200640 1 --2- 172.21.15.17:0/2821543538 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f563c076ef0 0x7f563c0793b0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:36.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:36.326+0000 7f5657200640 1 --2- 172.21.15.17:0/2821543538 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f56500cd650 0x7f56500cdad0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:36.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:36.326+0000 7f5657200640 1 --2- 172.21.15.17:0/2821543538 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5650047590 0x7f56500479b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:36.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:36.326+0000 7f5657200640 1 --2- 172.21.15.17:0/2821543538 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f563c003d60 0x7f56500d5140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:36.329 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:36.326+0000 7f5657200640 1 -- 172.21.15.17:0/2821543538 >> 172.21.15.17:0/2821543538 conn(0x7f565004fde0 msgr2=0x7f5650034050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:44:36.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:36.326+0000 7f5657200640 1 -- 172.21.15.17:0/2821543538 shutdown_connections 2024-09-10T21:44:36.330 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:36.326+0000 7f5657200640 1 -- 172.21.15.17:0/2821543538 wait complete. 2024-09-10T21:44:39.386 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-10T21:44:39.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:39.702+0000 7f02c0200640 1 -- 172.21.15.17:0/1215096418 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f02b8153ed0 msgr2=0x7f02b8072fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:39.705 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:39.702+0000 7f02c0200640 1 --2- 172.21.15.17:0/1215096418 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f02b8153ed0 0x7f02b8072fe0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f02b000b6c0 tx=0x7f02b001d3d0 comp rx=0 tx=0).stop 2024-09-10T21:44:39.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:39.706+0000 7f02c0200640 1 -- 172.21.15.17:0/1215096418 shutdown_connections 2024-09-10T21:44:39.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:39.706+0000 7f02c0200640 1 --2- 172.21.15.17:0/1215096418 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f02b8074e70 0x7f02b8156d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:39.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:39.706+0000 7f02c0200640 1 --2- 172.21.15.17:0/1215096418 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f02b8153ed0 0x7f02b8072fe0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:39.708 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:39.706+0000 7f02c0200640 1 -- 172.21.15.17:0/1215096418 >> 172.21.15.17:0/1215096418 conn(0x7f02b8070160 msgr2=0x7f02b8070570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:44:39.709 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:39.706+0000 7f02c0200640 1 -- 172.21.15.17:0/1215096418 shutdown_connections 2024-09-10T21:44:39.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:39.706+0000 7f02c0200640 1 -- 172.21.15.17:0/1215096418 wait complete. 2024-09-10T21:44:39.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:39.710+0000 7f02c0200640 1 Processor -- start 2024-09-10T21:44:39.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:39.710+0000 7f02c0200640 1 -- start start 2024-09-10T21:44:39.713 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:39.710+0000 7f02c0200640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f02b8074e70 0x7f02b81638c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:39.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:39.710+0000 7f02c0200640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f02b80c7a80 0x7f02b8150600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:39.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:39.710+0000 7f02c0200640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f02b8153ed0 0x7f02b80c2870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:39.714 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:39.710+0000 7f02bce00640 1 -- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f02b80c7a80 msgr2=0x7f02b8150600 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:3301/0 2024-09-10T21:44:39.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:39.710+0000 7f02bce00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f02b80c7a80 0x7f02b8150600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:44:39.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:39.710+0000 7f02c0200640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f02b80c9680 con 0x7f02b8153ed0 2024-09-10T21:44:39.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:39.710+0000 7f02c0200640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f02b80c86e0 con 0x7f02b8074e70 2024-09-10T21:44:39.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:39.710+0000 7f02c0200640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f02b80c9800 con 0x7f02b80c7a80 2024-09-10T21:44:39.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:39.710+0000 7f02bd800640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f02b8074e70 0x7f02b81638c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:39.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:39.710+0000 7f02bd800640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f02b8074e70 0x7f02b81638c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.17:36502/0 (socket says 172.21.15.17:36502) 2024-09-10T21:44:39.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:39.710+0000 7f02bd800640 1 -- 172.21.15.17:0/3576237697 learned_addr learned my addr 172.21.15.17:0/3576237697 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:44:39.715 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:39.710+0000 7f02be200640 1 --2- 172.21.15.17:0/3576237697 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f02b8153ed0 0x7f02b80c2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:39.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:39.714+0000 7f02bd800640 1 -- 172.21.15.17:0/3576237697 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f02b80c7a80 msgr2=0x7f02b8150600 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:44:39.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:39.714+0000 7f02bd800640 1 --2- 172.21.15.17:0/3576237697 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f02b80c7a80 0x7f02b8150600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:39.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:39.714+0000 7f02bd800640 1 -- 172.21.15.17:0/3576237697 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f02b8153ed0 msgr2=0x7f02b80c2870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:39.716 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:39.714+0000 7f02bd800640 1 --2- 172.21.15.17:0/3576237697 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f02b8153ed0 0x7f02b80c2870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:39.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:39.714+0000 7f02bd800640 1 -- 172.21.15.17:0/3576237697 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02b8165630 con 0x7f02b8074e70 2024-09-10T21:44:39.717 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:39.714+0000 7f02bd800640 1 --2- 172.21.15.17:0/3576237697 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f02b8074e70 0x7f02b81638c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f02b001b000 tx=0x7f02b0004860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:44:39.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:39.714+0000 7f02b5600640 1 -- 172.21.15.17:0/3576237697 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02b0003a40 con 0x7f02b8074e70 2024-09-10T21:44:39.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:39.714+0000 7f02b5600640 1 -- 172.21.15.17:0/3576237697 <== mon.1 v2:172.21.15.31:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f02b0009200 con 0x7f02b8074e70 2024-09-10T21:44:39.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:39.714+0000 7f02b5600640 1 -- 172.21.15.17:0/3576237697 <== mon.1 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02b0009630 con 0x7f02b8074e70 2024-09-10T21:44:39.721 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:39.714+0000 7f02c0200640 1 -- 172.21.15.17:0/3576237697 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02b8165910 con 0x7f02b8074e70 2024-09-10T21:44:39.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:39.714+0000 7f02c0200640 1 -- 172.21.15.17:0/3576237697 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02b8165ed0 con 0x7f02b8074e70 2024-09-10T21:44:39.722 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:39.714+0000 7f02c0200640 1 -- 172.21.15.17:0/3576237697 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02b8156d20 con 0x7f02b8074e70 2024-09-10T21:44:39.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:39.718+0000 7f02b5600640 1 -- 172.21.15.17:0/3576237697 <== mon.1 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f02b00097d0 con 0x7f02b8074e70 2024-09-10T21:44:39.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:39.718+0000 7f02b5600640 1 --2- 172.21.15.17:0/3576237697 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f02980725e0 0x7f0298074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:39.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:39.718+0000 7f02b5600640 1 -- 172.21.15.17:0/3576237697 <== mon.1 v2:172.21.15.31:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f02b012e790 con 0x7f02b8074e70 2024-09-10T21:44:39.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:39.726+0000 7f02bce00640 1 --2- 172.21.15.17:0/3576237697 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f02980725e0 0x7f0298074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:39.741 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:39.738+0000 7f02bce00640 1 --2- 172.21.15.17:0/3576237697 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f02980725e0 0x7f0298074aa0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f02a40036b0 tx=0x7f02a4007ae0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:44:39.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:39.750+0000 7f02b5600640 1 -- 172.21.15.17:0/3576237697 <== mon.1 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f02b0017050 con 0x7f02b8074e70 2024-09-10T21:44:40.483 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:40.478+0000 7f02c0200640 1 -- 172.21.15.17:0/3576237697 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f02b8160df0 con 0x7f02b8074e70 2024-09-10T21:44:40.484 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:40.482+0000 7f02b5600640 1 -- 172.21.15.17:0/3576237697 <== mon.1 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1558 (secure 0 0 0) 0x7f02b00f9ad0 con 0x7f02b8074e70 2024-09-10T21:44:40.496 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":150,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":21,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5e2f023f-d6a6-49f4-a256-f4807dce3739","modified":"2024-09-10T21:23:46.892130Z","created":"2024-09-10T21:23:46.892130Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3301","nonce":0},{"type":"v1","addr":"172.21.15.31:6790","nonce":0}]},"addr":"172.21.15.31:6790/0","public_addr":"172.21.15.31:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-10T21:44:40.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:40.506+0000 7f02c0200640 1 -- 172.21.15.17:0/3576237697 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f02980725e0 msgr2=0x7f0298074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:40.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:40.506+0000 7f02c0200640 1 --2- 172.21.15.17:0/3576237697 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f02980725e0 0x7f0298074aa0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f02a40036b0 tx=0x7f02a4007ae0 comp rx=0 tx=0).stop 2024-09-10T21:44:40.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:40.506+0000 7f02c0200640 1 -- 172.21.15.17:0/3576237697 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f02b8074e70 msgr2=0x7f02b81638c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:40.513 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:40.506+0000 7f02c0200640 1 --2- 172.21.15.17:0/3576237697 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f02b8074e70 0x7f02b81638c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f02b001b000 tx=0x7f02b0004860 comp rx=0 tx=0).stop 2024-09-10T21:44:40.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:40.510+0000 7f02c0200640 1 -- 172.21.15.17:0/3576237697 shutdown_connections 2024-09-10T21:44:40.518 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:40.510+0000 7f02c0200640 1 --2- 172.21.15.17:0/3576237697 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f02980725e0 0x7f0298074aa0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:40.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:40.510+0000 7f02c0200640 1 --2- 172.21.15.17:0/3576237697 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f02b8153ed0 0x7f02b80c2870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:40.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:40.510+0000 7f02c0200640 1 --2- 172.21.15.17:0/3576237697 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f02b80c7a80 0x7f02b8150600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:40.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:40.510+0000 7f02c0200640 1 --2- 172.21.15.17:0/3576237697 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f02b8074e70 0x7f02b81638c0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:40.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:40.510+0000 7f02c0200640 1 -- 172.21.15.17:0/3576237697 >> 172.21.15.17:0/3576237697 conn(0x7f02b8070160 msgr2=0x7f02b8070540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:44:40.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:40.514+0000 7f02c0200640 1 -- 172.21.15.17:0/3576237697 shutdown_connections 2024-09-10T21:44:40.527 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:40.518+0000 7f02c0200640 1 -- 172.21.15.17:0/3576237697 wait complete. 2024-09-10T21:44:43.595 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-10T21:44:43.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:43.894+0000 7f096ce00640 1 -- 172.21.15.17:0/2846052013 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0968153df0 msgr2=0x7f09680c8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:43.898 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:43.894+0000 7f096ce00640 1 --2- 172.21.15.17:0/2846052013 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0968153df0 0x7f09680c8ab0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f09580091e0 tx=0x7f095801d950 comp rx=0 tx=0).stop 2024-09-10T21:44:43.900 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:43.894+0000 7f096ce00640 1 -- 172.21.15.17:0/2846052013 shutdown_connections 2024-09-10T21:44:43.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:43.898+0000 7f096ce00640 1 --2- 172.21.15.17:0/2846052013 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0968153df0 0x7f09680c8ab0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:43.901 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:43.898+0000 7f096ce00640 1 -- 172.21.15.17:0/2846052013 >> 172.21.15.17:0/2846052013 conn(0x7f0968070160 msgr2=0x7f0968070570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:44:43.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:43.902+0000 7f096ce00640 1 -- 172.21.15.17:0/2846052013 shutdown_connections 2024-09-10T21:44:43.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:43.902+0000 7f096ce00640 1 -- 172.21.15.17:0/2846052013 wait complete. 2024-09-10T21:44:43.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:43.902+0000 7f096ce00640 1 Processor -- start 2024-09-10T21:44:43.910 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:43.906+0000 7f096ce00640 1 -- start start 2024-09-10T21:44:43.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:43.910+0000 7f096ce00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f096806e8a0 0x7f09681fb7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:43.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:43.910+0000 7f096ce00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f09680c17a0 0x7f09681f8b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:43.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:43.910+0000 7f096ce00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f0968153df0 0x7f09681fac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:43.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:43.910+0000 7f096ce00640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f09680c7010 con 0x7f096806e8a0 2024-09-10T21:44:43.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:43.910+0000 7f096ce00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f09680c6e90 con 0x7f09680c17a0 2024-09-10T21:44:43.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:43.910+0000 7f096ce00640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f09680c7190 con 0x7f0968153df0 2024-09-10T21:44:43.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:43.910+0000 7f0966800640 1 -- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f0968153df0 msgr2=0x7f09681fac90 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:3301/0 2024-09-10T21:44:43.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:43.910+0000 7f0966800640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f0968153df0 0x7f09681fac90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:44:43.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:43.910+0000 7f0965400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f09680c17a0 0x7f09681f8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:43.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:43.910+0000 7f0965400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f09680c17a0 0x7f09681f8b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.17:36506/0 (socket says 172.21.15.17:36506) 2024-09-10T21:44:43.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:43.910+0000 7f0965400640 1 -- 172.21.15.17:0/4150244216 learned_addr learned my addr 172.21.15.17:0/4150244216 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:44:43.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:43.910+0000 7f0965e00640 1 --2- 172.21.15.17:0/4150244216 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f096806e8a0 0x7f09681fb7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:43.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:43.914+0000 7f0965e00640 1 -- 172.21.15.17:0/4150244216 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f0968153df0 msgr2=0x7f09681fac90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:44:43.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:43.914+0000 7f0965e00640 1 --2- 172.21.15.17:0/4150244216 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f0968153df0 0x7f09681fac90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:43.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:43.914+0000 7f0965e00640 1 -- 172.21.15.17:0/4150244216 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f09680c17a0 msgr2=0x7f09681f8b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:43.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:43.914+0000 7f0965e00640 1 --2- 172.21.15.17:0/4150244216 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f09680c17a0 0x7f09681f8b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:43.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:43.914+0000 7f0965e00640 1 -- 172.21.15.17:0/4150244216 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09681fbe80 con 0x7f096806e8a0 2024-09-10T21:44:43.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:43.914+0000 7f0965400640 1 --2- 172.21.15.17:0/4150244216 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f09680c17a0 0x7f09681f8b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:44:43.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:43.918+0000 7f0965e00640 1 --2- 172.21.15.17:0/4150244216 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f096806e8a0 0x7f09681fb7b0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f096806f1e0 tx=0x7f095801dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:44:43.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:43.918+0000 7f095e000640 1 -- 172.21.15.17:0/4150244216 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09580b3040 con 0x7f096806e8a0 2024-09-10T21:44:43.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:43.922+0000 7f096ce00640 1 -- 172.21.15.17:0/4150244216 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09681fc160 con 0x7f096806e8a0 2024-09-10T21:44:43.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:43.922+0000 7f096ce00640 1 -- 172.21.15.17:0/4150244216 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09681f8360 con 0x7f096806e8a0 2024-09-10T21:44:43.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:43.922+0000 7f095e000640 1 -- 172.21.15.17:0/4150244216 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f09580a6720 con 0x7f096806e8a0 2024-09-10T21:44:43.927 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:43.922+0000 7f095e000640 1 -- 172.21.15.17:0/4150244216 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0958004490 con 0x7f096806e8a0 2024-09-10T21:44:43.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:43.934+0000 7f095e000640 1 -- 172.21.15.17:0/4150244216 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f09580b6070 con 0x7f096806e8a0 2024-09-10T21:44:43.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:43.934+0000 7f095e000640 1 --2- 172.21.15.17:0/4150244216 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f09480725c0 0x7f0948074a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:43.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:43.934+0000 7f095e000640 1 -- 172.21.15.17:0/4150244216 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f095812d6f0 con 0x7f096806e8a0 2024-09-10T21:44:43.941 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:43.934+0000 7f096ce00640 1 -- 172.21.15.17:0/4150244216 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09400052a0 con 0x7f096806e8a0 2024-09-10T21:44:43.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:43.938+0000 7f0965400640 1 --2- 172.21.15.17:0/4150244216 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f09480725c0 0x7f0948074a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:43.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:43.942+0000 7f0965400640 1 --2- 172.21.15.17:0/4150244216 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f09480725c0 0x7f0948074a80 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f0968161260 tx=0x7f0954009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:44:43.958 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:43.954+0000 7f095e000640 1 -- 172.21.15.17:0/4150244216 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f09580be060 con 0x7f096806e8a0 2024-09-10T21:44:44.569 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:44.566+0000 7f096ce00640 1 -- 172.21.15.17:0/4150244216 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f0940005590 con 0x7f096806e8a0 2024-09-10T21:44:44.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:44.586+0000 7f095e000640 1 -- 172.21.15.17:0/4150244216 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1558 (secure 0 0 0) 0x7f0958017540 con 0x7f096806e8a0 2024-09-10T21:44:44.590 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":150,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":25,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5e2f023f-d6a6-49f4-a256-f4807dce3739","modified":"2024-09-10T21:23:46.892130Z","created":"2024-09-10T21:23:46.892130Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3301","nonce":0},{"type":"v1","addr":"172.21.15.31:6790","nonce":0}]},"addr":"172.21.15.31:6790/0","public_addr":"172.21.15.31:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-10T21:44:44.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:44.594+0000 7f096ce00640 1 -- 172.21.15.17:0/4150244216 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f09480725c0 msgr2=0x7f0948074a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:44.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:44.594+0000 7f096ce00640 1 --2- 172.21.15.17:0/4150244216 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f09480725c0 0x7f0948074a80 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f0968161260 tx=0x7f0954009040 comp rx=0 tx=0).stop 2024-09-10T21:44:44.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:44.594+0000 7f096ce00640 1 -- 172.21.15.17:0/4150244216 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f096806e8a0 msgr2=0x7f09681fb7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:44.596 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:44.594+0000 7f096ce00640 1 --2- 172.21.15.17:0/4150244216 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f096806e8a0 0x7f09681fb7b0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f096806f1e0 tx=0x7f095801dbc0 comp rx=0 tx=0).stop 2024-09-10T21:44:44.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:44.594+0000 7f096ce00640 1 -- 172.21.15.17:0/4150244216 shutdown_connections 2024-09-10T21:44:44.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:44.594+0000 7f096ce00640 1 --2- 172.21.15.17:0/4150244216 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f09480725c0 0x7f0948074a80 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:44.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:44.594+0000 7f096ce00640 1 --2- 172.21.15.17:0/4150244216 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f0968153df0 0x7f09681fac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:44.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:44.594+0000 7f096ce00640 1 --2- 172.21.15.17:0/4150244216 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f09680c17a0 0x7f09681f8b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:44.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:44.594+0000 7f096ce00640 1 --2- 172.21.15.17:0/4150244216 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f096806e8a0 0x7f09681fb7b0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:44.597 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:44.594+0000 7f096ce00640 1 -- 172.21.15.17:0/4150244216 >> 172.21.15.17:0/4150244216 conn(0x7f0968070160 msgr2=0x7f096814d1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:44:44.613 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:44.606+0000 7f096ce00640 1 -- 172.21.15.17:0/4150244216 shutdown_connections 2024-09-10T21:44:44.613 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:44.610+0000 7f096ce00640 1 -- 172.21.15.17:0/4150244216 wait complete. 2024-09-10T21:44:47.680 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-10T21:44:50.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:50.990+0000 7f4822c00640 1 -- 172.21.15.17:0/1666146252 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4800004dc0 msgr2=0x7f4800005240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:50.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:50.990+0000 7f4822c00640 1 --2- 172.21.15.17:0/1666146252 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4800004dc0 0x7f4800005240 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f481c076a20 tx=0x7f481401db90 comp rx=0 tx=0).stop 2024-09-10T21:44:50.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:50.990+0000 7f4822c00640 1 -- 172.21.15.17:0/1666146252 shutdown_connections 2024-09-10T21:44:50.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:50.990+0000 7f4822c00640 1 --2- 172.21.15.17:0/1666146252 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4800004dc0 0x7f4800005240 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:50.995 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:50.990+0000 7f4822c00640 1 --2- 172.21.15.17:0/1666146252 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f481c0c17a0 0x7f48000039c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:50.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:50.994+0000 7f4822c00640 1 --2- 172.21.15.17:0/1666146252 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f48000040b0 0x7f4800004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:50.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:50.994+0000 7f4822c00640 1 -- 172.21.15.17:0/1666146252 >> 172.21.15.17:0/1666146252 conn(0x7f481c070160 msgr2=0x7f481c070570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:44:51.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:51.002+0000 7f4822c00640 1 -- 172.21.15.17:0/1666146252 shutdown_connections 2024-09-10T21:44:51.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:51.002+0000 7f4822c00640 1 -- 172.21.15.17:0/1666146252 wait complete. 2024-09-10T21:44:51.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:51.002+0000 7f4822c00640 1 Processor -- start 2024-09-10T21:44:51.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:51.002+0000 7f4822c00640 1 -- start start 2024-09-10T21:44:51.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:51.006+0000 7f4822c00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f48000040b0 0x7f481c0c4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:51.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:51.006+0000 7f4822c00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4800004dc0 0x7f481c0c3d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:51.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:51.006+0000 7f4822c00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f481c0c17a0 0x7f481c0c9f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:51.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:51.006+0000 7f4822c00640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f481c0c6e90 con 0x7f48000040b0 2024-09-10T21:44:51.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:51.006+0000 7f4822c00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f481c157650 con 0x7f4800004dc0 2024-09-10T21:44:51.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:51.006+0000 7f4822c00640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f481c0c7010 con 0x7f481c0c17a0 2024-09-10T21:44:51.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:51.006+0000 7f4820c00640 1 -- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f481c0c17a0 msgr2=0x7f481c0c9f20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:3301/0 2024-09-10T21:44:51.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:51.006+0000 7f4820c00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f481c0c17a0 0x7f481c0c9f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:44:51.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:51.006+0000 7f481b400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4800004dc0 0x7f481c0c3d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:51.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:51.006+0000 7f481b400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4800004dc0 0x7f481c0c3d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.17:51930/0 (socket says 172.21.15.17:51930) 2024-09-10T21:44:51.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:51.006+0000 7f481b400640 1 -- 172.21.15.17:0/2646617246 learned_addr learned my addr 172.21.15.17:0/2646617246 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:44:51.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:51.006+0000 7f481be00640 1 --2- 172.21.15.17:0/2646617246 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f48000040b0 0x7f481c0c4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:51.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:51.010+0000 7f481be00640 1 -- 172.21.15.17:0/2646617246 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f481c0c17a0 msgr2=0x7f481c0c9f20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:44:51.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:51.010+0000 7f481be00640 1 --2- 172.21.15.17:0/2646617246 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f481c0c17a0 0x7f481c0c9f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:51.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:51.010+0000 7f481be00640 1 -- 172.21.15.17:0/2646617246 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4800004dc0 msgr2=0x7f481c0c3d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:51.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:51.010+0000 7f481be00640 1 --2- 172.21.15.17:0/2646617246 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4800004dc0 0x7f481c0c3d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:51.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:51.010+0000 7f481be00640 1 -- 172.21.15.17:0/2646617246 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f481c0cb610 con 0x7f48000040b0 2024-09-10T21:44:51.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:51.010+0000 7f481be00640 1 --2- 172.21.15.17:0/2646617246 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f48000040b0 0x7f481c0c4ee0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f480c005f20 tx=0x7f480c0054e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:44:51.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:51.010+0000 7f4813e00640 1 -- 172.21.15.17:0/2646617246 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f480c003d10 con 0x7f48000040b0 2024-09-10T21:44:51.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:51.014+0000 7f4813e00640 1 -- 172.21.15.17:0/2646617246 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f480c013480 con 0x7f48000040b0 2024-09-10T21:44:51.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:51.014+0000 7f4813e00640 1 -- 172.21.15.17:0/2646617246 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f480c013890 con 0x7f48000040b0 2024-09-10T21:44:51.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:51.014+0000 7f4822c00640 1 -- 172.21.15.17:0/2646617246 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f481c0cb8d0 con 0x7f48000040b0 2024-09-10T21:44:51.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:51.014+0000 7f4822c00640 1 -- 172.21.15.17:0/2646617246 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f481c0c3270 con 0x7f48000040b0 2024-09-10T21:44:51.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:51.014+0000 7f4811600640 1 -- 172.21.15.17:0/2646617246 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47f00052a0 con 0x7f48000040b0 2024-09-10T21:44:51.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:51.022+0000 7f4813e00640 1 -- 172.21.15.17:0/2646617246 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f480c013a30 con 0x7f48000040b0 2024-09-10T21:44:51.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:51.022+0000 7f4813e00640 1 --2- 172.21.15.17:0/2646617246 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f47fc072f00 0x7f47fc0753c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:51.027 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:51.022+0000 7f4813e00640 1 -- 172.21.15.17:0/2646617246 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f480c0978e0 con 0x7f48000040b0 2024-09-10T21:44:51.031 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:51.026+0000 7f481b400640 1 --2- 172.21.15.17:0/2646617246 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f47fc072f00 0x7f47fc0753c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:51.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:51.034+0000 7f481b400640 1 --2- 172.21.15.17:0/2646617246 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f47fc072f00 0x7f47fc0753c0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f481c13e500 tx=0x7f4808009280 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:44:51.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:51.042+0000 7f4813e00640 1 -- 172.21.15.17:0/2646617246 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f480c062c30 con 0x7f48000040b0 2024-09-10T21:44:51.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:51.550+0000 7f4811600640 1 -- 172.21.15.17:0/2646617246 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f47f0005590 con 0x7f48000040b0 2024-09-10T21:44:51.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:51.550+0000 7f4813e00640 1 -- 172.21.15.17:0/2646617246 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1551 (secure 0 0 0) 0x7f480c06b350 con 0x7f48000040b0 2024-09-10T21:44:51.558 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":154,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":1,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5e2f023f-d6a6-49f4-a256-f4807dce3739","modified":"2024-09-10T21:23:46.892130Z","created":"2024-09-10T21:23:46.892130Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3301","nonce":0},{"type":"v1","addr":"172.21.15.31:6790","nonce":0}]},"addr":"172.21.15.31:6790/0","public_addr":"172.21.15.31:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-10T21:44:51.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:51.566+0000 7f4811600640 1 -- 172.21.15.17:0/2646617246 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f47fc072f00 msgr2=0x7f47fc0753c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:51.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:51.566+0000 7f4811600640 1 --2- 172.21.15.17:0/2646617246 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f47fc072f00 0x7f47fc0753c0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f481c13e500 tx=0x7f4808009280 comp rx=0 tx=0).stop 2024-09-10T21:44:51.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:51.566+0000 7f4811600640 1 -- 172.21.15.17:0/2646617246 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f48000040b0 msgr2=0x7f481c0c4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:51.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:51.566+0000 7f4811600640 1 --2- 172.21.15.17:0/2646617246 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f48000040b0 0x7f481c0c4ee0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f480c005f20 tx=0x7f480c0054e0 comp rx=0 tx=0).stop 2024-09-10T21:44:51.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:51.574+0000 7f4811600640 1 -- 172.21.15.17:0/2646617246 shutdown_connections 2024-09-10T21:44:51.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:51.574+0000 7f4811600640 1 --2- 172.21.15.17:0/2646617246 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f47fc072f00 0x7f47fc0753c0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:51.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:51.574+0000 7f4811600640 1 --2- 172.21.15.17:0/2646617246 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f481c0c17a0 0x7f481c0c9f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:51.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:51.574+0000 7f4811600640 1 --2- 172.21.15.17:0/2646617246 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4800004dc0 0x7f481c0c3d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:51.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:51.574+0000 7f4811600640 1 --2- 172.21.15.17:0/2646617246 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f48000040b0 0x7f481c0c4ee0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:51.579 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:51.574+0000 7f4811600640 1 -- 172.21.15.17:0/2646617246 >> 172.21.15.17:0/2646617246 conn(0x7f481c070160 msgr2=0x7f481c074730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:44:51.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:51.578+0000 7f4811600640 1 -- 172.21.15.17:0/2646617246 shutdown_connections 2024-09-10T21:44:51.583 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:51.582+0000 7f4811600640 1 -- 172.21.15.17:0/2646617246 wait complete. 2024-09-10T21:44:51.629 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2024-09-10T21:44:51.630 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon a 2024-09-10T21:44:51.630 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-10T21:44:51.982 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:51.978+0000 7efcdde00640 1 -- 172.21.15.17:0/868875373 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efcd004b430 msgr2=0x7efcd00fcbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:51.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:51.978+0000 7efcdde00640 1 --2- 172.21.15.17:0/868875373 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efcd004b430 0x7efcd00fcbf0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7efcc4009900 tx=0x7efcc401c660 comp rx=0 tx=0).stop 2024-09-10T21:44:51.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:51.978+0000 7efcdde00640 1 -- 172.21.15.17:0/868875373 shutdown_connections 2024-09-10T21:44:51.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:51.978+0000 7efcdde00640 1 --2- 172.21.15.17:0/868875373 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efcd004b430 0x7efcd00fcbf0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:51.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:51.978+0000 7efcdde00640 1 --2- 172.21.15.17:0/868875373 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7efcd0033bf0 0x7efcd00f62e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:51.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:51.982+0000 7efcdde00640 1 -- 172.21.15.17:0/868875373 >> 172.21.15.17:0/868875373 conn(0x7efcd004fc10 msgr2=0x7efcd0050020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:44:51.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:51.982+0000 7efcdde00640 1 -- 172.21.15.17:0/868875373 shutdown_connections 2024-09-10T21:44:51.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:51.982+0000 7efcdde00640 1 -- 172.21.15.17:0/868875373 wait complete. 2024-09-10T21:44:52.002 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:51.994+0000 7efcdde00640 1 Processor -- start 2024-09-10T21:44:52.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:51.998+0000 7efcdde00640 1 -- start start 2024-09-10T21:44:52.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:52.002+0000 7efcdde00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7efcd0033bf0 0x7efcd00f6d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:52.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:52.002+0000 7efcdde00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efcd004a8b0 0x7efcd00fd590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:52.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:52.002+0000 7efcdde00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efcd004b430 0x7efcd00f7a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:52.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:52.002+0000 7efcdde00640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7efcd0050730 con 0x7efcd004a8b0 2024-09-10T21:44:52.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:52.002+0000 7efcdde00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7efcd004b0e0 con 0x7efcd004b430 2024-09-10T21:44:52.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:52.002+0000 7efcdde00640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7efcd00508b0 con 0x7efcd0033bf0 2024-09-10T21:44:52.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:52.002+0000 7efcdca00640 1 -- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7efcd0033bf0 msgr2=0x7efcd00f6d20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:3301/0 2024-09-10T21:44:52.007 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:52.002+0000 7efcdca00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7efcd0033bf0 0x7efcd00f6d20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:44:52.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:52.002+0000 7efcd7e00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efcd004a8b0 0x7efcd00fd590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:52.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:52.002+0000 7efcd7e00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efcd004a8b0 0x7efcd00fd590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:60308/0 (socket says 172.21.15.17:60308) 2024-09-10T21:44:52.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:52.002+0000 7efcd7e00640 1 -- 172.21.15.17:0/3499029848 learned_addr learned my addr 172.21.15.17:0/3499029848 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:44:52.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:52.006+0000 7efcdd400640 1 --2- 172.21.15.17:0/3499029848 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efcd004b430 0x7efcd00f7a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:52.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:52.010+0000 7efcd7e00640 1 -- 172.21.15.17:0/3499029848 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7efcd0033bf0 msgr2=0x7efcd00f6d20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:44:52.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:52.010+0000 7efcd7e00640 1 --2- 172.21.15.17:0/3499029848 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7efcd0033bf0 0x7efcd00f6d20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:52.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:52.010+0000 7efcd7e00640 1 -- 172.21.15.17:0/3499029848 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efcd004b430 msgr2=0x7efcd00f7a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:52.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:52.010+0000 7efcd7e00640 1 --2- 172.21.15.17:0/3499029848 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efcd004b430 0x7efcd00f7a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:52.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:52.010+0000 7efcd7e00640 1 -- 172.21.15.17:0/3499029848 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcd00f7260 con 0x7efcd004a8b0 2024-09-10T21:44:52.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:52.010+0000 7efcdd400640 1 --2- 172.21.15.17:0/3499029848 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efcd004b430 0x7efcd00f7a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:44:52.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:52.010+0000 7efcd7e00640 1 --2- 172.21.15.17:0/3499029848 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efcd004a8b0 0x7efcd00fd590 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7efcc4009850 tx=0x7efcc40a6ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:44:52.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:52.014+0000 7efcd4c00640 1 -- 172.21.15.17:0/3499029848 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcc4004310 con 0x7efcd004a8b0 2024-09-10T21:44:52.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:52.014+0000 7efcd4c00640 1 -- 172.21.15.17:0/3499029848 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efcc40044b0 con 0x7efcd004a8b0 2024-09-10T21:44:52.018 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:52.014+0000 7efcd4c00640 1 -- 172.21.15.17:0/3499029848 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcc4003860 con 0x7efcd004a8b0 2024-09-10T21:44:52.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:52.014+0000 7efcdde00640 1 -- 172.21.15.17:0/3499029848 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efcd00ff2f0 con 0x7efcd004a8b0 2024-09-10T21:44:52.020 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:52.018+0000 7efcdde00640 1 -- 172.21.15.17:0/3499029848 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efcd00ff8b0 con 0x7efcd004a8b0 2024-09-10T21:44:52.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:52.018+0000 7efcdde00640 1 --2- 172.21.15.17:0/3499029848 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efcd00f98a0 0x7efcd00f7f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:52.024 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:52.018+0000 7efcdde00640 1 -- 172.21.15.17:0/3499029848 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7efcd0034710 con 0x7efcd00f98a0 2024-09-10T21:44:52.028 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:52.022+0000 7efcdca00640 1 --2- 172.21.15.17:0/3499029848 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efcd00f98a0 0x7efcd00f7f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:52.032 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:52.026+0000 7efcd4c00640 1 -- 172.21.15.17:0/3499029848 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7efcc4003a00 con 0x7efcd004a8b0 2024-09-10T21:44:52.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:52.030+0000 7efcd4c00640 1 --2- 172.21.15.17:0/3499029848 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7efcb8072590 0x7efcb8074a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:52.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:52.030+0000 7efcdd400640 1 --2- 172.21.15.17:0/3499029848 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7efcb8072590 0x7efcb8074a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:52.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:52.030+0000 7efcd4c00640 1 -- 172.21.15.17:0/3499029848 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7efcc40a6850 con 0x7efcd004a8b0 2024-09-10T21:44:52.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:52.030+0000 7efcdd400640 1 --2- 172.21.15.17:0/3499029848 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7efcb8072590 0x7efcb8074a50 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7efcd8052940 tx=0x7efcd806f040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:44:52.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:52.034+0000 7efcdca00640 1 -- 172.21.15.17:0/3499029848 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7efcd00008e0 con 0x7efcd00f98a0 2024-09-10T21:44:52.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:52.038+0000 7efcdca00640 1 --2- 172.21.15.17:0/3499029848 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efcd00f98a0 0x7efcd00f7f90 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7efccc009fd0 tx=0x7efccc00b7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:44:52.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:52.038+0000 7efcd4c00640 1 -- 172.21.15.17:0/3499029848 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efccc016070 con 0x7efcd00f98a0 2024-09-10T21:44:52.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:52.046+0000 7efcd4c00640 1 -- 172.21.15.17:0/3499029848 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7efccc003c70 con 0x7efcd00f98a0 2024-09-10T21:44:52.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:52.046+0000 7efcd4c00640 1 -- 172.21.15.17:0/3499029848 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efcd00f98a0 msgr2=0x7efcd00f7f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:52.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:52.046+0000 7efcd4c00640 1 --2- 172.21.15.17:0/3499029848 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efcd00f98a0 0x7efcd00f7f90 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7efccc009fd0 tx=0x7efccc00b7a0 comp rx=0 tx=0).stop 2024-09-10T21:44:52.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:52.046+0000 7efcd4c00640 1 -- 172.21.15.17:0/3499029848 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efcd00f98a0 msgr2=0x7efcd00f7f90 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:44:52.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:52.046+0000 7efcd4c00640 1 --2- 172.21.15.17:0/3499029848 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efcd00f98a0 0x7efcd00f7f90 secure :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7efccc009fd0 tx=0x7efccc00b7a0 comp rx=0 tx=0).stop 2024-09-10T21:44:52.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:52.062+0000 7efcdde00640 1 --2- 172.21.15.17:0/3499029848 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efcd00ffab0 0x7efcd01a96d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:52.066 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:52.062+0000 7efcdde00640 1 -- 172.21.15.17:0/3499029848 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7efcd0005130 con 0x7efcd00ffab0 2024-09-10T21:44:52.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:52.070+0000 7efcdca00640 1 --2- 172.21.15.17:0/3499029848 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efcd00ffab0 0x7efcd01a96d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:52.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:52.070+0000 7efcdca00640 1 -- 172.21.15.17:0/3499029848 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7efcd00fdad0 con 0x7efcd00ffab0 2024-09-10T21:44:52.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:52.070+0000 7efcdca00640 1 --2- 172.21.15.17:0/3499029848 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efcd00ffab0 0x7efcd01a96d0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7efccc009fd0 tx=0x7efccc0058c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:44:52.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:52.070+0000 7efcd4c00640 1 -- 172.21.15.17:0/3499029848 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efccc016070 con 0x7efcd00ffab0 2024-09-10T21:44:52.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:52.074+0000 7efcd4c00640 1 -- 172.21.15.17:0/3499029848 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5034 (secure 0 0 0) 0x7efccc011400 con 0x7efcd00ffab0 2024-09-10T21:44:52.075 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:52.074+0000 7efcd4c00640 1 -- 172.21.15.17:0/3499029848 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efcd00ffab0 msgr2=0x7efcd01a96d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:52.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:52.074+0000 7efcd4c00640 1 --2- 172.21.15.17:0/3499029848 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efcd00ffab0 0x7efcd01a96d0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7efccc009fd0 tx=0x7efccc0058c0 comp rx=0 tx=0).stop 2024-09-10T21:44:52.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:52.074+0000 7efcd4c00640 1 -- 172.21.15.17:0/3499029848 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efcd00ffab0 msgr2=0x7efcd01a96d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:44:52.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:52.074+0000 7efcd4c00640 1 --2- 172.21.15.17:0/3499029848 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efcd00ffab0 0x7efcd01a96d0 secure :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7efccc009fd0 tx=0x7efccc0058c0 comp rx=0 tx=0).stop 2024-09-10T21:44:52.077 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-10T21:44:52.077 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:44:52.077 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:44:52.077 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "leader", 2024-09-10T21:44:52.077 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 154, 2024-09-10T21:44:52.077 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 151202, 2024-09-10T21:44:52.077 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-10T21:44:52.078 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-10T21:44:52.078 INFO:teuthology.orchestra.run.smithi017.stdout: 1 2024-09-10T21:44:52.078 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:44:52.078 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 1, 2024-09-10T21:44:52.078 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:44:52.078 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-10T21:44:52.078 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-10T21:44:52.078 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:44:52.078 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:44:52.078 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:44:52.078 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:44:52.078 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:44:52.078 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:44:52.078 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:44:52.079 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:44:52.079 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:44:52.079 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:44:52.079 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:44:52.079 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:44:52.079 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-10T21:44:52.079 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-10T21:44:52.079 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:44:52.079 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:44:52.079 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:44:52.079 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:44:52.079 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:44:52.079 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:44:52.079 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:44:52.080 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:44:52.080 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:44:52.080 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:44:52.080 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:44:52.080 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:44:52.080 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:44:52.080 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-10T21:44:52.080 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-10T21:44:52.080 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-10T21:44:52.080 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-10T21:44:52.080 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-10T21:44:52.080 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "5e2f023f-d6a6-49f4-a256-f4807dce3739", 2024-09-10T21:44:52.080 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:44:52.080 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:44:52.081 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-10T21:44:52.081 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-10T21:44:52.081 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-10T21:44:52.081 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-10T21:44:52.081 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-10T21:44:52.081 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-10T21:44:52.081 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-10T21:44:52.081 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:44:52.081 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-10T21:44:52.081 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:44:52.081 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:44:52.081 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:44:52.081 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:44:52.081 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:44:52.082 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:44:52.082 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:44:52.082 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:44:52.082 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:44:52.082 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:44:52.082 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:44:52.082 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:44:52.082 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-10T21:44:52.082 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:44:52.082 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-10T21:44:52.082 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:44:52.082 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:44:52.082 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:44:52.082 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:44:52.083 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:44:52.083 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:44:52.083 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:44:52.083 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-10T21:44:52.083 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:44:52.083 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:44:52.083 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:44:52.083 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:44:52.083 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-10T21:44:52.083 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:44:52.083 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:44:52.083 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:44:52.083 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:44:52.083 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-10T21:44:52.084 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-10T21:44:52.084 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:44:52.084 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:44:52.084 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:44:52.084 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:44:52.084 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:44:52.084 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:44:52.084 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:44:52.084 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:44:52.084 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:44:52.084 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:44:52.084 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:44:52.084 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3300", 2024-09-10T21:44:52.084 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:44:52.085 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:44:52.085 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:44:52.085 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:44:52.085 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789", 2024-09-10T21:44:52.089 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:44:52.089 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:44:52.089 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:44:52.089 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:44:52.089 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789/0", 2024-09-10T21:44:52.089 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6789/0", 2024-09-10T21:44:52.089 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:44:52.089 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:44:52.089 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:44:52.089 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:44:52.089 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:44:52.090 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:44:52.090 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:44:52.090 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:44:52.090 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:44:52.090 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:44:52.090 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:44:52.090 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3301", 2024-09-10T21:44:52.090 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:44:52.090 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:44:52.090 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:44:52.090 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:44:52.090 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790", 2024-09-10T21:44:52.090 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:44:52.090 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:44:52.090 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:44:52.091 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:44:52.091 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790/0", 2024-09-10T21:44:52.091 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6790/0", 2024-09-10T21:44:52.091 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:44:52.091 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:44:52.091 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:44:52.091 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:44:52.091 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:44:52.091 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:44:52.091 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-10T21:44:52.091 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-10T21:44:52.091 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:44:52.091 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:44:52.091 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:44:52.092 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:44:52.092 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:44:52.092 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:44:52.092 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": [ 2024-09-10T21:44:52.092 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:44:52.092 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:44:52.092 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:44:52.092 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 4 2024-09-10T21:44:52.092 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:44:52.092 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:44:52.092 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-10T21:44:52.092 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:44:52.092 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:44:52.092 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:44:52.093 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 7 2024-09-10T21:44:52.093 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:44:52.093 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:44:52.093 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-10T21:44:52.093 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:44:52.093 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:44:52.093 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:44:52.093 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 5 2024-09-10T21:44:52.093 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:44:52.093 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:44:52.093 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": [ 2024-09-10T21:44:52.093 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:44:52.093 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:44:52.093 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:44:52.094 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:44:52.094 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:44:52.094 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:44:52.094 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:44:52.094 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-10T21:44:52.094 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-10T21:44:52.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:52.090+0000 7efcdde00640 1 -- 172.21.15.17:0/3499029848 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7efcb8072590 msgr2=0x7efcb8074a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:52.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:52.090+0000 7efcdde00640 1 --2- 172.21.15.17:0/3499029848 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7efcb8072590 0x7efcb8074a50 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7efcd8052940 tx=0x7efcd806f040 comp rx=0 tx=0).stop 2024-09-10T21:44:52.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:52.090+0000 7efcdde00640 1 -- 172.21.15.17:0/3499029848 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efcd004a8b0 msgr2=0x7efcd00fd590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:52.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:52.090+0000 7efcdde00640 1 --2- 172.21.15.17:0/3499029848 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efcd004a8b0 0x7efcd00fd590 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7efcc4009850 tx=0x7efcc40a6ba0 comp rx=0 tx=0).stop 2024-09-10T21:44:52.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:52.090+0000 7efcdd400640 1 -- 172.21.15.17:0/3499029848 reap_dead start 2024-09-10T21:44:52.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:52.090+0000 7efcdd400640 1 -- 172.21.15.17:0/3499029848 reap_dead start 2024-09-10T21:44:52.095 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:52.090+0000 7efcdde00640 1 -- 172.21.15.17:0/3499029848 shutdown_connections 2024-09-10T21:44:52.096 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:52.090+0000 7efcdde00640 1 -- 172.21.15.17:0/3499029848 >> 172.21.15.17:0/3499029848 conn(0x7efcd004fc10 msgr2=0x7efcd00f9fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:44:52.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:52.094+0000 7efcdde00640 1 -- 172.21.15.17:0/3499029848 shutdown_connections 2024-09-10T21:44:52.099 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:52.094+0000 7efcdde00640 1 -- 172.21.15.17:0/3499029848 wait complete. 2024-09-10T21:44:52.159 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-09-10T21:44:52.160 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon b 2024-09-10T21:44:52.160 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-10T21:44:52.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:52.454+0000 7f339a400640 1 -- 172.21.15.17:0/1060480843 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3378003a10 msgr2=0x7f3378003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:52.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:52.454+0000 7f339a400640 1 --2- 172.21.15.17:0/1060480843 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3378003a10 0x7f3378003e30 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f33840099a0 tx=0x7f338401c7c0 comp rx=0 tx=0).stop 2024-09-10T21:44:52.461 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:52.458+0000 7f3393e00640 1 -- 172.21.15.17:0/1060480843 reap_dead start 2024-09-10T21:44:52.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:52.462+0000 7f339a400640 1 -- 172.21.15.17:0/1060480843 shutdown_connections 2024-09-10T21:44:52.464 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:52.462+0000 7f339a400640 1 -- 172.21.15.17:0/1060480843 >> 172.21.15.17:0/1060480843 conn(0x7f33940700d0 msgr2=0x7f33940704e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:44:52.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:52.466+0000 7f339a400640 1 -- 172.21.15.17:0/1060480843 shutdown_connections 2024-09-10T21:44:52.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:52.466+0000 7f339a400640 1 -- 172.21.15.17:0/1060480843 wait complete. 2024-09-10T21:44:52.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:52.466+0000 7f339a400640 1 Processor -- start 2024-09-10T21:44:52.488 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:52.478+0000 7f339a400640 1 -- start start 2024-09-10T21:44:52.490 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:52.486+0000 7f339a400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3378003a10 0x7f33940c3610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:52.490 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:52.486+0000 7f339a400640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3394154420 0x7f33941ddb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:52.490 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:52.486+0000 7f339a400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f33941df3c0 0x7f33941de0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:52.490 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:52.486+0000 7f339a400640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f33940c13b0 con 0x7f33941df3c0 2024-09-10T21:44:52.490 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:52.486+0000 7f339a400640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f33940c95a0 con 0x7f3378003a10 2024-09-10T21:44:52.490 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:52.486+0000 7f339a400640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f3394154120 con 0x7f3394154420 2024-09-10T21:44:52.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:52.486+0000 7f338be00640 1 -- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3394154420 msgr2=0x7f33941ddb80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:3301/0 2024-09-10T21:44:52.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:52.486+0000 7f338be00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3394154420 0x7f33941ddb80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:44:52.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:52.486+0000 7f3393400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3378003a10 0x7f33940c3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:52.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:52.486+0000 7f3393400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3378003a10 0x7f33940c3610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.17:51962/0 (socket says 172.21.15.17:51962) 2024-09-10T21:44:52.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:52.486+0000 7f3393400640 1 -- 172.21.15.17:0/3174412354 learned_addr learned my addr 172.21.15.17:0/3174412354 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:44:52.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:52.490+0000 7f3393e00640 1 --2- 172.21.15.17:0/3174412354 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f33941df3c0 0x7f33941de0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:52.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:52.490+0000 7f3393400640 1 -- 172.21.15.17:0/3174412354 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3394154420 msgr2=0x7f33941ddb80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:44:52.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:52.490+0000 7f3393400640 1 --2- 172.21.15.17:0/3174412354 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3394154420 0x7f33941ddb80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:52.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:52.490+0000 7f3393400640 1 -- 172.21.15.17:0/3174412354 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f33941df3c0 msgr2=0x7f33941de0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:52.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:52.490+0000 7f3393400640 1 --2- 172.21.15.17:0/3174412354 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f33941df3c0 0x7f33941de0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:44:52.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:52.490+0000 7f3393400640 1 -- 172.21.15.17:0/3174412354 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33941571a0 con 0x7f3378003a10 2024-09-10T21:44:52.494 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:52.490+0000 7f3393400640 1 --2- 172.21.15.17:0/3174412354 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3378003a10 0x7f33940c3610 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f338401c5f0 tx=0x7f33840a7f70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:44:52.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:52.494+0000 7f338b400640 1 -- 172.21.15.17:0/3174412354 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33840bc070 con 0x7f3378003a10 2024-09-10T21:44:52.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:52.494+0000 7f339a400640 1 -- 172.21.15.17:0/3174412354 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33940c5df0 con 0x7f3378003a10 2024-09-10T21:44:52.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:52.494+0000 7f339a400640 1 -- 172.21.15.17:0/3174412354 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33940c6090 con 0x7f3378003a10 2024-09-10T21:44:52.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:52.494+0000 7f338b400640 1 -- 172.21.15.17:0/3174412354 <== mon.1 v2:172.21.15.31:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f33840a5680 con 0x7f3378003a10 2024-09-10T21:44:52.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:52.494+0000 7f338b400640 1 -- 172.21.15.17:0/3174412354 <== mon.1 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33840bc070 con 0x7f3378003a10 2024-09-10T21:44:52.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:52.498+0000 7f339a400640 1 --2- 172.21.15.17:0/3174412354 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f336c002a80 0x7f336c004f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:52.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:52.498+0000 7f339a400640 1 -- 172.21.15.17:0/3174412354 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f336c007ed0 con 0x7f336c002a80 2024-09-10T21:44:52.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:52.498+0000 7f338b400640 1 -- 172.21.15.17:0/3174412354 <== mon.1 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f3384004850 con 0x7f3378003a10 2024-09-10T21:44:52.508 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:52.502+0000 7f338b400640 1 --2- 172.21.15.17:0/3174412354 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f3370072990 0x7f3370074e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:52.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:52.502+0000 7f338b400640 1 -- 172.21.15.17:0/3174412354 <== mon.1 v2:172.21.15.31:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f33840a5970 con 0x7f3378003a10 2024-09-10T21:44:52.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:52.502+0000 7f338be00640 1 --2- 172.21.15.17:0/3174412354 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f336c002a80 0x7f336c004f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:52.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:52.502+0000 7f338be00640 1 -- 172.21.15.17:0/3174412354 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f336c0080d0 con 0x7f336c002a80 2024-09-10T21:44:52.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:52.506+0000 7f338be00640 1 --2- 172.21.15.17:0/3174412354 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f336c002a80 0x7f336c004f40 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f3394154860 tx=0x7f337c00eb50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:44:52.509 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:52.506+0000 7f338b400640 1 -- 172.21.15.17:0/3174412354 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f337c004230 con 0x7f336c002a80 2024-09-10T21:44:52.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:52.506+0000 7f338b400640 1 -- 172.21.15.17:0/3174412354 <== mon.1 v2:172.21.15.31:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f337c0043d0 con 0x7f336c002a80 2024-09-10T21:44:52.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:52.506+0000 7f338b400640 1 -- 172.21.15.17:0/3174412354 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f336c002a80 msgr2=0x7f336c004f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:52.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:52.506+0000 7f338b400640 1 --2- 172.21.15.17:0/3174412354 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f336c002a80 0x7f336c004f40 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f3394154860 tx=0x7f337c00eb50 comp rx=0 tx=0).stop 2024-09-10T21:44:52.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:52.506+0000 7f338b400640 1 -- 172.21.15.17:0/3174412354 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f336c002a80 msgr2=0x7f336c004f40 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:44:52.510 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:52.506+0000 7f338b400640 1 --2- 172.21.15.17:0/3174412354 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f336c002a80 0x7f336c004f40 secure :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f3394154860 tx=0x7f337c00eb50 comp rx=0 tx=0).stop 2024-09-10T21:44:52.514 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:52.510+0000 7f3393e00640 1 --2- 172.21.15.17:0/3174412354 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f3370072990 0x7f3370074e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:52.522 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:52.518+0000 7f3393e00640 1 --2- 172.21.15.17:0/3174412354 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f3370072990 0x7f3370074e50 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f3394073750 tx=0x7f338c009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:44:52.530 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:52.526+0000 7f339a400640 1 --2- 172.21.15.17:0/3174412354 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f336c006f40 0x7f336c0073d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:44:52.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:52.526+0000 7f339a400640 1 -- 172.21.15.17:0/3174412354 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f336c00a4b0 con 0x7f336c006f40 2024-09-10T21:44:52.531 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:52.530+0000 7f338be00640 1 --2- 172.21.15.17:0/3174412354 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f336c006f40 0x7f336c0073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:44:52.532 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:52.530+0000 7f338be00640 1 -- 172.21.15.17:0/3174412354 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f336c007910 con 0x7f336c006f40 2024-09-10T21:44:52.537 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:52.534+0000 7f338be00640 1 --2- 172.21.15.17:0/3174412354 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f336c006f40 0x7f336c0073d0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f337c00efd0 tx=0x7f337c0046d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:44:52.539 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:52.534+0000 7f338b400640 1 -- 172.21.15.17:0/3174412354 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f337c003800 con 0x7f336c006f40 2024-09-10T21:44:52.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:52.538+0000 7f338b400640 1 -- 172.21.15.17:0/3174412354 <== mon.1 v2:172.21.15.31:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5031 (secure 0 0 0) 0x7f337c003800 con 0x7f336c006f40 2024-09-10T21:44:52.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:52.538+0000 7f338b400640 1 -- 172.21.15.17:0/3174412354 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f336c006f40 msgr2=0x7f336c0073d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:52.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:52.538+0000 7f338b400640 1 --2- 172.21.15.17:0/3174412354 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f336c006f40 0x7f336c0073d0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f337c00efd0 tx=0x7f337c0046d0 comp rx=0 tx=0).stop 2024-09-10T21:44:52.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:52.538+0000 7f338b400640 1 -- 172.21.15.17:0/3174412354 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f336c006f40 msgr2=0x7f336c0073d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:44:52.542 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:52.538+0000 7f338b400640 1 --2- 172.21.15.17:0/3174412354 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f336c006f40 0x7f336c0073d0 secure :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f337c00efd0 tx=0x7f337c0046d0 comp rx=0 tx=0).stop 2024-09-10T21:44:52.542 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-10T21:44:52.543 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:44:52.543 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:44:52.543 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-10T21:44:52.543 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 154, 2024-09-10T21:44:52.543 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 37519, 2024-09-10T21:44:52.543 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-10T21:44:52.543 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-10T21:44:52.543 INFO:teuthology.orchestra.run.smithi017.stdout: 1 2024-09-10T21:44:52.543 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:44:52.543 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 2, 2024-09-10T21:44:52.543 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:44:52.543 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-10T21:44:52.543 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-10T21:44:52.544 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:44:52.544 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:44:52.544 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:44:52.544 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:44:52.544 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:44:52.544 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:44:52.544 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:44:52.544 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:44:52.544 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:44:52.544 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:44:52.544 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:44:52.544 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:44:52.544 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-10T21:44:52.545 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-10T21:44:52.545 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:44:52.545 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:44:52.545 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:44:52.545 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:44:52.545 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:44:52.545 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:44:52.545 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:44:52.545 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:44:52.545 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:44:52.545 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:44:52.545 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:44:52.545 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:44:52.545 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:44:52.546 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-10T21:44:52.546 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-10T21:44:52.546 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-10T21:44:52.546 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-10T21:44:52.546 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-10T21:44:52.546 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "5e2f023f-d6a6-49f4-a256-f4807dce3739", 2024-09-10T21:44:52.546 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:44:52.546 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:44:52.546 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-10T21:44:52.546 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-10T21:44:52.546 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-10T21:44:52.546 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-10T21:44:52.546 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-10T21:44:52.547 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-10T21:44:52.547 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-10T21:44:52.547 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:44:52.547 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-10T21:44:52.547 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:44:52.547 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:44:52.547 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:44:52.547 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:44:52.547 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:44:52.547 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:44:52.547 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:44:52.547 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:44:52.547 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:44:52.548 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:44:52.548 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:44:52.548 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:44:52.548 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-10T21:44:52.548 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:44:52.548 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-10T21:44:52.548 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:44:52.548 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:44:52.548 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:44:52.548 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:44:52.548 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:44:52.548 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:44:52.548 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:44:52.549 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-10T21:44:52.549 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:44:52.549 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:44:52.549 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:44:52.549 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:44:52.549 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-10T21:44:52.549 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:44:52.549 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:44:52.549 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:44:52.549 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:44:52.549 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-10T21:44:52.549 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-10T21:44:52.549 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:44:52.549 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:44:52.550 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:44:52.550 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:44:52.550 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:44:52.550 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:44:52.550 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:44:52.550 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:44:52.550 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:44:52.550 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:44:52.550 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:44:52.550 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3300", 2024-09-10T21:44:52.550 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:44:52.550 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:44:52.550 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:44:52.551 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:44:52.551 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789", 2024-09-10T21:44:52.551 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:44:52.551 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:44:52.551 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:44:52.551 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:44:52.551 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789/0", 2024-09-10T21:44:52.551 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6789/0", 2024-09-10T21:44:52.551 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:44:52.551 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:44:52.551 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:44:52.551 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:44:52.551 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:44:52.552 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:44:52.552 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:44:52.552 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:44:52.552 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:44:52.552 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:44:52.552 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:44:52.552 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3301", 2024-09-10T21:44:52.552 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:44:52.552 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:44:52.552 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:44:52.552 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:44:52.552 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790", 2024-09-10T21:44:52.552 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:44:52.553 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:44:52.553 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:44:52.553 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:44:52.553 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790/0", 2024-09-10T21:44:52.553 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6790/0", 2024-09-10T21:44:52.553 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:44:52.553 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:44:52.553 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:44:52.553 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:44:52.553 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:44:52.553 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:44:52.553 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-10T21:44:52.553 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-10T21:44:52.553 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:44:52.554 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:44:52.554 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:44:52.554 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:44:52.554 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:44:52.554 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:44:52.554 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": [ 2024-09-10T21:44:52.554 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:44:52.554 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:44:52.554 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:44:52.554 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 2 2024-09-10T21:44:52.554 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:44:52.554 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:44:52.554 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-10T21:44:52.554 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:44:52.555 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:44:52.555 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:44:52.555 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:44:52.555 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:44:52.555 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:44:52.555 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-10T21:44:52.555 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:44:52.555 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:44:52.555 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:44:52.555 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 4 2024-09-10T21:44:52.555 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:44:52.555 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:44:52.555 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": [ 2024-09-10T21:44:52.555 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:44:52.556 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:44:52.556 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:44:52.556 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:44:52.556 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:44:52.556 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:44:52.556 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:44:52.556 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-10T21:44:52.556 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-10T21:44:52.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:52.542+0000 7f339a400640 1 -- 172.21.15.17:0/3174412354 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f3370072990 msgr2=0x7f3370074e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:52.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:52.542+0000 7f339a400640 1 --2- 172.21.15.17:0/3174412354 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f3370072990 0x7f3370074e50 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f3394073750 tx=0x7f338c009040 comp rx=0 tx=0).stop 2024-09-10T21:44:52.557 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:52.542+0000 7f339a400640 1 -- 172.21.15.17:0/3174412354 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3378003a10 msgr2=0x7f33940c3610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:44:52.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:52.542+0000 7f339a400640 1 --2- 172.21.15.17:0/3174412354 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3378003a10 0x7f33940c3610 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f338401c5f0 tx=0x7f33840a7f70 comp rx=0 tx=0).stop 2024-09-10T21:44:52.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:52.542+0000 7f3393e00640 1 -- 172.21.15.17:0/3174412354 reap_dead start 2024-09-10T21:44:52.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:52.542+0000 7f3393e00640 1 -- 172.21.15.17:0/3174412354 reap_dead start 2024-09-10T21:44:52.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:52.542+0000 7f339a400640 1 -- 172.21.15.17:0/3174412354 shutdown_connections 2024-09-10T21:44:52.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:52.542+0000 7f339a400640 1 -- 172.21.15.17:0/3174412354 >> 172.21.15.17:0/3174412354 conn(0x7f33940700d0 msgr2=0x7f3394150b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:44:52.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:52.542+0000 7f339a400640 1 -- 172.21.15.17:0/3174412354 shutdown_connections 2024-09-10T21:44:52.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:44:52.542+0000 7f339a400640 1 -- 172.21.15.17:0/3174412354 wait complete. 2024-09-10T21:44:52.615 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-09-10T21:44:52.616 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2024-09-10T21:45:00.876 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:45:00.880 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:45:12.617 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.c 2024-09-10T21:45:12.617 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-10T21:45:12.617 DEBUG:teuthology.orchestra.run.smithi031:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-10T21:45:13.922 INFO:tasks.ceph.mon.c:Started 2024-09-10T21:45:13.922 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-09-10T21:45:13.922 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-10T21:45:14.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:14.334+0000 7f5848800640 1 -- 172.21.15.17:0/365789172 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5840074e70 msgr2=0x7f58400c94b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:14.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:14.334+0000 7f5848800640 1 --2- 172.21.15.17:0/365789172 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5840074e70 0x7f58400c94b0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f58400c9eb0 tx=0x7f583c01c740 comp rx=0 tx=0).stop 2024-09-10T21:45:14.344 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:14.342+0000 7f5848800640 1 -- 172.21.15.17:0/365789172 shutdown_connections 2024-09-10T21:45:14.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:14.342+0000 7f5848800640 1 --2- 172.21.15.17:0/365789172 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5840074e70 0x7f58400c94b0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:14.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:14.342+0000 7f5848800640 1 --2- 172.21.15.17:0/365789172 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f58400c7a80 0x7f58400c8f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:14.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:14.342+0000 7f5848800640 1 --2- 172.21.15.17:0/365789172 >> v2:172.21.15.17:3300/0 conn(0x7f5840153ed0 0x7f5840072fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:14.345 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:14.342+0000 7f5848800640 1 -- 172.21.15.17:0/365789172 >> 172.21.15.17:0/365789172 conn(0x7f5840070160 msgr2=0x7f5840070570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:45:14.353 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:14.350+0000 7f5848800640 1 -- 172.21.15.17:0/365789172 shutdown_connections 2024-09-10T21:45:14.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:14.350+0000 7f5848800640 1 -- 172.21.15.17:0/365789172 wait complete. 2024-09-10T21:45:14.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:14.362+0000 7f5848800640 1 Processor -- start 2024-09-10T21:45:14.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:14.362+0000 7f5848800640 1 -- start start 2024-09-10T21:45:14.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:14.362+0000 7f5848800640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f5840074e70 0x7f58400c9e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:14.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:14.362+0000 7f5848800640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f58400c7a80 0x7f58401f8d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:14.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:14.362+0000 7f5848800640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5840153ed0 0x7f58401407a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:14.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:14.362+0000 7f5848800640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f5840159ca0 con 0x7f5840153ed0 2024-09-10T21:45:14.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:14.362+0000 7f5848800640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f58400c86e0 con 0x7f58400c7a80 2024-09-10T21:45:14.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:14.362+0000 7f5848800640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f5840159e20 con 0x7f5840074e70 2024-09-10T21:45:14.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:14.366+0000 7f5845e00640 1 -- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f5840074e70 msgr2=0x7f58400c9e20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.31:3301/0 2024-09-10T21:45:14.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:14.366+0000 7f5845e00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f5840074e70 0x7f58400c9e20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:45:14.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:14.366+0000 7f5846800640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5840153ed0 0x7f58401407a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:14.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:14.366+0000 7f5846800640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5840153ed0 0x7f58401407a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:56920/0 (socket says 172.21.15.17:56920) 2024-09-10T21:45:14.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:14.366+0000 7f5846800640 1 -- 172.21.15.17:0/3405994790 learned_addr learned my addr 172.21.15.17:0/3405994790 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:45:14.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:14.366+0000 7f5845400640 1 --2- 172.21.15.17:0/3405994790 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f58400c7a80 0x7f58401f8d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:14.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:14.366+0000 7f5845400640 1 -- 172.21.15.17:0/3405994790 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f5840074e70 msgr2=0x7f58400c9e20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:45:14.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:14.366+0000 7f5845400640 1 --2- 172.21.15.17:0/3405994790 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f5840074e70 0x7f58400c9e20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:14.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:14.366+0000 7f5845400640 1 -- 172.21.15.17:0/3405994790 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5840153ed0 msgr2=0x7f58401407a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:14.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:14.366+0000 7f5845400640 1 --2- 172.21.15.17:0/3405994790 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5840153ed0 0x7f58401407a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:14.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:14.370+0000 7f5845400640 1 -- 172.21.15.17:0/3405994790 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58400c3210 con 0x7f58400c7a80 2024-09-10T21:45:14.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:14.370+0000 7f5845400640 1 --2- 172.21.15.17:0/3405994790 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f58400c7a80 0x7f58401f8d00 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f583000b450 tx=0x7f583000b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:45:14.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:14.370+0000 7f583a000640 1 -- 172.21.15.17:0/3405994790 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5830015020 con 0x7f58400c7a80 2024-09-10T21:45:14.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:14.370+0000 7f5848800640 1 -- 172.21.15.17:0/3405994790 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58400cb0a0 con 0x7f58400c7a80 2024-09-10T21:45:14.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:14.370+0000 7f5848800640 1 -- 172.21.15.17:0/3405994790 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58400ca740 con 0x7f58400c7a80 2024-09-10T21:45:14.376 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:14.374+0000 7f5848800640 1 -- 172.21.15.17:0/3405994790 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5840072fe0 con 0x7f58400c7a80 2024-09-10T21:45:14.376 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:14.374+0000 7f583a000640 1 -- 172.21.15.17:0/3405994790 <== mon.1 v2:172.21.15.31:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5830011980 con 0x7f58400c7a80 2024-09-10T21:45:14.376 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:14.374+0000 7f583a000640 1 -- 172.21.15.17:0/3405994790 <== mon.1 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5830015020 con 0x7f58400c7a80 2024-09-10T21:45:14.381 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:14.378+0000 7f583a000640 1 -- 172.21.15.17:0/3405994790 <== mon.1 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f5830011b70 con 0x7f58400c7a80 2024-09-10T21:45:14.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:14.378+0000 7f583a000640 1 --2- 172.21.15.17:0/3405994790 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f5824072790 0x7f5824074c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:14.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:14.378+0000 7f583a000640 1 -- 172.21.15.17:0/3405994790 <== mon.1 v2:172.21.15.31:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f5830093ba0 con 0x7f58400c7a80 2024-09-10T21:45:14.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:14.382+0000 7f5845e00640 1 --2- 172.21.15.17:0/3405994790 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f5824072790 0x7f5824074c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:14.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:14.382+0000 7f5845e00640 1 --2- 172.21.15.17:0/3405994790 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f5824072790 0x7f5824074c50 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f583400d590 tx=0x7f58340091e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:45:14.399 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:14.394+0000 7f583a000640 1 -- 172.21.15.17:0/3405994790 <== mon.1 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f583001f020 con 0x7f58400c7a80 2024-09-10T21:45:15.259 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":154,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":25,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5e2f023f-d6a6-49f4-a256-f4807dce3739","modified":"2024-09-10T21:23:46.892130Z","created":"2024-09-10T21:23:46.892130Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3301","nonce":0},{"type":"v1","addr":"172.21.15.31:6790","nonce":0}]},"addr":"172.21.15.31:6790/0","public_addr":"172.21.15.31:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-10T21:45:15.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:15.238+0000 7f5848800640 1 -- 172.21.15.17:0/3405994790 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f58400c7e60 con 0x7f58400c7a80 2024-09-10T21:45:15.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:15.246+0000 7f583a000640 1 -- 172.21.15.17:0/3405994790 <== mon.1 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1552 (secure 0 0 0) 0x7f583000d040 con 0x7f58400c7a80 2024-09-10T21:45:15.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:15.266+0000 7f5848800640 1 -- 172.21.15.17:0/3405994790 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f5824072790 msgr2=0x7f5824074c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:15.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:15.266+0000 7f5848800640 1 --2- 172.21.15.17:0/3405994790 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f5824072790 0x7f5824074c50 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f583400d590 tx=0x7f58340091e0 comp rx=0 tx=0).stop 2024-09-10T21:45:15.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:15.266+0000 7f5848800640 1 -- 172.21.15.17:0/3405994790 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f58400c7a80 msgr2=0x7f58401f8d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:15.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:15.266+0000 7f5848800640 1 --2- 172.21.15.17:0/3405994790 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f58400c7a80 0x7f58401f8d00 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f583000b450 tx=0x7f583000b910 comp rx=0 tx=0).stop 2024-09-10T21:45:15.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:15.278+0000 7f5848800640 1 -- 172.21.15.17:0/3405994790 shutdown_connections 2024-09-10T21:45:15.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:15.278+0000 7f5848800640 1 --2- 172.21.15.17:0/3405994790 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f5824072790 0x7f5824074c50 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:15.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:15.278+0000 7f5848800640 1 --2- 172.21.15.17:0/3405994790 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5840153ed0 0x7f58401407a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:15.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:15.278+0000 7f5848800640 1 --2- 172.21.15.17:0/3405994790 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f58400c7a80 0x7f58401f8d00 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:15.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:15.278+0000 7f5848800640 1 --2- 172.21.15.17:0/3405994790 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f5840074e70 0x7f58400c9e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:15.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:15.278+0000 7f5848800640 1 -- 172.21.15.17:0/3405994790 >> 172.21.15.17:0/3405994790 conn(0x7f5840070160 msgr2=0x7f5840155c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:45:15.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:15.278+0000 7f5848800640 1 -- 172.21.15.17:0/3405994790 shutdown_connections 2024-09-10T21:45:15.282 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:15.278+0000 7f5848800640 1 -- 172.21.15.17:0/3405994790 wait complete. 2024-09-10T21:45:18.338 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-10T21:45:18.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:18.638+0000 7fe917c00640 1 -- 172.21.15.17:0/2516393851 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe8f4003a60 msgr2=0x7fe8f4003e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:18.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:18.638+0000 7fe917c00640 1 --2- 172.21.15.17:0/2516393851 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe8f4003a60 0x7fe8f4003e80 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fe90800c310 tx=0x7fe90801f720 comp rx=0 tx=0).stop 2024-09-10T21:45:18.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:18.638+0000 7fe917c00640 1 -- 172.21.15.17:0/2516393851 shutdown_connections 2024-09-10T21:45:18.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:18.638+0000 7fe917c00640 1 --2- 172.21.15.17:0/2516393851 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe8f4003a60 0x7fe8f4003e80 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:18.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:18.638+0000 7fe917c00640 1 -- 172.21.15.17:0/2516393851 >> 172.21.15.17:0/2516393851 conn(0x7fe910070160 msgr2=0x7fe910070570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:45:18.652 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:18.646+0000 7fe917c00640 1 -- 172.21.15.17:0/2516393851 shutdown_connections 2024-09-10T21:45:18.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:18.650+0000 7fe917c00640 1 -- 172.21.15.17:0/2516393851 wait complete. 2024-09-10T21:45:18.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:18.666+0000 7fe917c00640 1 Processor -- start 2024-09-10T21:45:18.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:18.670+0000 7fe917c00640 1 -- start start 2024-09-10T21:45:18.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:18.674+0000 7fe917c00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe8f4003a60 0x7fe910154ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:18.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:18.674+0000 7fe917c00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe9100b01a0 0x7fe9100b05c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:18.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:18.674+0000 7fe917c00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe910150bd0 0x7fe910151060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:18.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:18.674+0000 7fe917c00640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fe91015ad00 con 0x7fe8f4003a60 2024-09-10T21:45:18.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:18.674+0000 7fe917c00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fe9100c6e90 con 0x7fe910150bd0 2024-09-10T21:45:18.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:18.674+0000 7fe917c00640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7fe9100c73c0 con 0x7fe9100b01a0 2024-09-10T21:45:18.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:18.674+0000 7fe915200640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe8f4003a60 0x7fe910154ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:18.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:18.674+0000 7fe915200640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe8f4003a60 0x7fe910154ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:56020/0 (socket says 172.21.15.17:56020) 2024-09-10T21:45:18.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:18.674+0000 7fe915200640 1 -- 172.21.15.17:0/2240351131 learned_addr learned my addr 172.21.15.17:0/2240351131 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:45:18.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:18.674+0000 7fe915200640 1 -- 172.21.15.17:0/2240351131 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe9100b01a0 msgr2=0x7fe9100b05c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:45:18.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:18.674+0000 7fe915200640 1 --2- 172.21.15.17:0/2240351131 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe9100b01a0 0x7fe9100b05c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:18.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:18.674+0000 7fe915200640 1 -- 172.21.15.17:0/2240351131 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe910150bd0 msgr2=0x7fe910151060 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:45:18.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:18.674+0000 7fe915200640 1 --2- 172.21.15.17:0/2240351131 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe910150bd0 0x7fe910151060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:18.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:18.674+0000 7fe915200640 1 -- 172.21.15.17:0/2240351131 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9100c1680 con 0x7fe8f4003a60 2024-09-10T21:45:18.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:18.674+0000 7fe915200640 1 --2- 172.21.15.17:0/2240351131 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe8f4003a60 0x7fe910154ed0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fe9080b1040 tx=0x7fe908001df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:45:18.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:18.674+0000 7fe90cc00640 1 -- 172.21.15.17:0/2240351131 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9080b3070 con 0x7fe8f4003a60 2024-09-10T21:45:18.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:18.674+0000 7fe90cc00640 1 -- 172.21.15.17:0/2240351131 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe908003600 con 0x7fe8f4003a60 2024-09-10T21:45:18.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:18.674+0000 7fe90cc00640 1 -- 172.21.15.17:0/2240351131 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe908006c60 con 0x7fe8f4003a60 2024-09-10T21:45:18.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:18.674+0000 7fe917c00640 1 -- 172.21.15.17:0/2240351131 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe910157760 con 0x7fe8f4003a60 2024-09-10T21:45:18.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:18.674+0000 7fe917c00640 1 -- 172.21.15.17:0/2240351131 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9100c1920 con 0x7fe8f4003a60 2024-09-10T21:45:18.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:18.678+0000 7fe917c00640 1 -- 172.21.15.17:0/2240351131 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8e80052a0 con 0x7fe8f4003a60 2024-09-10T21:45:18.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:18.678+0000 7fe90cc00640 1 -- 172.21.15.17:0/2240351131 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fe908003a10 con 0x7fe8f4003a60 2024-09-10T21:45:18.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:18.682+0000 7fe90cc00640 1 --2- 172.21.15.17:0/2240351131 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fe8f00725e0 0x7fe8f0074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:18.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:18.682+0000 7fe90fe00640 1 --2- 172.21.15.17:0/2240351131 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fe8f00725e0 0x7fe8f0074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:18.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:18.682+0000 7fe90cc00640 1 -- 172.21.15.17:0/2240351131 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fe908131200 con 0x7fe8f4003a60 2024-09-10T21:45:18.689 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:18.686+0000 7fe90fe00640 1 --2- 172.21.15.17:0/2240351131 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fe8f00725e0 0x7fe8f0074aa0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fe910072570 tx=0x7fe8fc006b90 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:45:18.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:18.686+0000 7fe90cc00640 1 -- 172.21.15.17:0/2240351131 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fe9080194d0 con 0x7fe8f4003a60 2024-09-10T21:45:19.356 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:19.354+0000 7fe917c00640 1 -- 172.21.15.17:0/2240351131 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fe8e8005860 con 0x7fe8f4003a60 2024-09-10T21:45:19.358 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:19.354+0000 7fe90cc00640 1 -- 172.21.15.17:0/2240351131 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1552 (secure 0 0 0) 0x7fe9080fc570 con 0x7fe8f4003a60 2024-09-10T21:45:19.362 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":154,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":29,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5e2f023f-d6a6-49f4-a256-f4807dce3739","modified":"2024-09-10T21:23:46.892130Z","created":"2024-09-10T21:23:46.892130Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3301","nonce":0},{"type":"v1","addr":"172.21.15.31:6790","nonce":0}]},"addr":"172.21.15.31:6790/0","public_addr":"172.21.15.31:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-10T21:45:19.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:19.358+0000 7fe917c00640 1 -- 172.21.15.17:0/2240351131 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fe8f00725e0 msgr2=0x7fe8f0074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:19.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:19.358+0000 7fe917c00640 1 --2- 172.21.15.17:0/2240351131 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fe8f00725e0 0x7fe8f0074aa0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fe910072570 tx=0x7fe8fc006b90 comp rx=0 tx=0).stop 2024-09-10T21:45:19.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:19.358+0000 7fe917c00640 1 -- 172.21.15.17:0/2240351131 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe8f4003a60 msgr2=0x7fe910154ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:19.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:19.358+0000 7fe917c00640 1 --2- 172.21.15.17:0/2240351131 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe8f4003a60 0x7fe910154ed0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fe9080b1040 tx=0x7fe908001df0 comp rx=0 tx=0).stop 2024-09-10T21:45:19.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:19.366+0000 7fe917c00640 1 -- 172.21.15.17:0/2240351131 shutdown_connections 2024-09-10T21:45:19.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:19.366+0000 7fe917c00640 1 --2- 172.21.15.17:0/2240351131 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fe8f00725e0 0x7fe8f0074aa0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:19.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:19.366+0000 7fe917c00640 1 --2- 172.21.15.17:0/2240351131 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe910150bd0 0x7fe910151060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:19.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:19.366+0000 7fe917c00640 1 --2- 172.21.15.17:0/2240351131 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe9100b01a0 0x7fe9100b05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:19.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:19.366+0000 7fe917c00640 1 --2- 172.21.15.17:0/2240351131 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe8f4003a60 0x7fe910154ed0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:19.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:19.366+0000 7fe917c00640 1 -- 172.21.15.17:0/2240351131 >> 172.21.15.17:0/2240351131 conn(0x7fe910070160 msgr2=0x7fe910073400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:45:19.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:19.370+0000 7fe917c00640 1 -- 172.21.15.17:0/2240351131 shutdown_connections 2024-09-10T21:45:19.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:19.370+0000 7fe917c00640 1 -- 172.21.15.17:0/2240351131 wait complete. 2024-09-10T21:45:22.515 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-10T21:45:22.914 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:22.910+0000 7fe547200640 1 -- 172.21.15.17:0/610642945 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe5400c7b10 msgr2=0x7fe540150320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:22.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:22.910+0000 7fe547200640 1 --2- 172.21.15.17:0/610642945 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe5400c7b10 0x7fe540150320 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe5400c8230 tx=0x7fe53c01f580 comp rx=0 tx=0).stop 2024-09-10T21:45:22.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:22.910+0000 7fe547200640 1 -- 172.21.15.17:0/610642945 shutdown_connections 2024-09-10T21:45:22.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:22.910+0000 7fe547200640 1 --2- 172.21.15.17:0/610642945 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe5400c7b10 0x7fe540150320 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:22.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:22.910+0000 7fe547200640 1 --2- 172.21.15.17:0/610642945 >> v2:172.21.15.17:3300/0 conn(0x7fe5400c85f0 0x7fe54015a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:22.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:22.910+0000 7fe547200640 1 --2- 172.21.15.17:0/610642945 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe54006a0e0 0x7fe5400c1400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:22.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:22.910+0000 7fe547200640 1 -- 172.21.15.17:0/610642945 >> 172.21.15.17:0/610642945 conn(0x7fe5400c8ab0 msgr2=0x7fe5400c8ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:45:22.915 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:22.910+0000 7fe547200640 1 -- 172.21.15.17:0/610642945 shutdown_connections 2024-09-10T21:45:22.916 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:22.910+0000 7fe547200640 1 -- 172.21.15.17:0/610642945 wait complete. 2024-09-10T21:45:22.917 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:22.914+0000 7fe547200640 1 Processor -- start 2024-09-10T21:45:22.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:22.914+0000 7fe547200640 1 -- start start 2024-09-10T21:45:22.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:22.914+0000 7fe547200640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe54006a0e0 0x7fe54014bc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:22.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:22.914+0000 7fe547200640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe5400c7b10 0x7fe540145750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:22.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:22.914+0000 7fe547200640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe5400c85f0 0x7fe540145c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:22.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:22.914+0000 7fe547200640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fe5400c6fb0 con 0x7fe54006a0e0 2024-09-10T21:45:22.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:22.914+0000 7fe547200640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fe540159f70 con 0x7fe5400c85f0 2024-09-10T21:45:22.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:22.914+0000 7fe547200640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7fe5400c7570 con 0x7fe5400c7b10 2024-09-10T21:45:22.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:22.918+0000 7fe545e00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe54006a0e0 0x7fe54014bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:22.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:22.918+0000 7fe545400640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe5400c7b10 0x7fe540145750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:22.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:22.918+0000 7fe545400640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe5400c7b10 0x7fe540145750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3301/0 says I am v2:172.21.15.17:56082/0 (socket says 172.21.15.17:56082) 2024-09-10T21:45:22.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:22.918+0000 7fe545400640 1 -- 172.21.15.17:0/864149694 learned_addr learned my addr 172.21.15.17:0/864149694 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:45:22.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:22.918+0000 7fe545400640 1 -- 172.21.15.17:0/864149694 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe5400c7b10 msgr2=0x7fe540145750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 18 2024-09-10T21:45:22.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:22.918+0000 7fe545400640 1 -- 172.21.15.17:0/864149694 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe5400c7b10 msgr2=0x7fe540145750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-10T21:45:22.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:22.918+0000 7fe545400640 1 --2- 172.21.15.17:0/864149694 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe5400c7b10 0x7fe540145750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T21:45:22.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:22.918+0000 7fe545400640 1 --2- 172.21.15.17:0/864149694 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe5400c7b10 0x7fe540145750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:45:22.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:22.918+0000 7fe545e00640 1 -- 172.21.15.17:0/864149694 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe5400c7b10 msgr2=0x7fe540145750 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:45:22.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:22.918+0000 7fe545e00640 1 --2- 172.21.15.17:0/864149694 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe5400c7b10 0x7fe540145750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:22.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:22.918+0000 7fe545e00640 1 -- 172.21.15.17:0/864149694 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe5400c85f0 msgr2=0x7fe540145c90 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-10T21:45:22.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:22.918+0000 7fe545e00640 1 --2- 172.21.15.17:0/864149694 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe5400c85f0 0x7fe540145c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:22.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:22.918+0000 7fe545e00640 1 -- 172.21.15.17:0/864149694 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5401f0120 con 0x7fe54006a0e0 2024-09-10T21:45:22.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:22.918+0000 7fe545e00640 1 --2- 172.21.15.17:0/864149694 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe54006a0e0 0x7fe54014bc10 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fe53400b3f0 tx=0x7fe53400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:45:22.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:22.926+0000 7fe53a000640 1 -- 172.21.15.17:0/864149694 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe534015020 con 0x7fe54006a0e0 2024-09-10T21:45:22.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:22.926+0000 7fe53a000640 1 -- 172.21.15.17:0/864149694 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe534011920 con 0x7fe54006a0e0 2024-09-10T21:45:22.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:22.926+0000 7fe53a000640 1 -- 172.21.15.17:0/864149694 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe534011d70 con 0x7fe54006a0e0 2024-09-10T21:45:22.928 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:22.926+0000 7fe547200640 1 -- 172.21.15.17:0/864149694 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe54014c790 con 0x7fe54006a0e0 2024-09-10T21:45:22.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:22.926+0000 7fe547200640 1 -- 172.21.15.17:0/864149694 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5401ef360 con 0x7fe54006a0e0 2024-09-10T21:45:22.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:22.934+0000 7fe547200640 1 -- 172.21.15.17:0/864149694 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5400c1400 con 0x7fe54006a0e0 2024-09-10T21:45:22.942 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:22.938+0000 7fe53a000640 1 -- 172.21.15.17:0/864149694 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fe53401a400 con 0x7fe54006a0e0 2024-09-10T21:45:22.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:22.942+0000 7fe53a000640 1 --2- 172.21.15.17:0/864149694 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fe5240725e0 0x7fe524074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:22.944 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:22.942+0000 7fe545400640 1 --2- 172.21.15.17:0/864149694 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fe5240725e0 0x7fe524074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:22.945 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:22.942+0000 7fe53a000640 1 -- 172.21.15.17:0/864149694 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fe534093930 con 0x7fe54006a0e0 2024-09-10T21:45:22.949 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:22.946+0000 7fe545400640 1 --2- 172.21.15.17:0/864149694 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fe5240725e0 0x7fe524074aa0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fe540051080 tx=0x7fe530008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:45:22.959 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:22.954+0000 7fe53a000640 1 -- 172.21.15.17:0/864149694 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fe53401f020 con 0x7fe54006a0e0 2024-09-10T21:45:23.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:23.554+0000 7fe547200640 1 -- 172.21.15.17:0/864149694 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fe5400741e0 con 0x7fe54006a0e0 2024-09-10T21:45:23.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:23.558+0000 7fe53a000640 1 -- 172.21.15.17:0/864149694 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1552 (secure 0 0 0) 0x7fe53405eb70 con 0x7fe54006a0e0 2024-09-10T21:45:23.564 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":154,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":33,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5e2f023f-d6a6-49f4-a256-f4807dce3739","modified":"2024-09-10T21:23:46.892130Z","created":"2024-09-10T21:23:46.892130Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3301","nonce":0},{"type":"v1","addr":"172.21.15.31:6790","nonce":0}]},"addr":"172.21.15.31:6790/0","public_addr":"172.21.15.31:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-10T21:45:23.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:23.566+0000 7fe547200640 1 -- 172.21.15.17:0/864149694 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fe5240725e0 msgr2=0x7fe524074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:23.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:23.566+0000 7fe547200640 1 --2- 172.21.15.17:0/864149694 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fe5240725e0 0x7fe524074aa0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fe540051080 tx=0x7fe530008040 comp rx=0 tx=0).stop 2024-09-10T21:45:23.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:23.566+0000 7fe547200640 1 -- 172.21.15.17:0/864149694 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe54006a0e0 msgr2=0x7fe54014bc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:23.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:23.566+0000 7fe547200640 1 --2- 172.21.15.17:0/864149694 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe54006a0e0 0x7fe54014bc10 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fe53400b3f0 tx=0x7fe53400b8b0 comp rx=0 tx=0).stop 2024-09-10T21:45:23.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:23.578+0000 7fe547200640 1 -- 172.21.15.17:0/864149694 shutdown_connections 2024-09-10T21:45:23.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:23.578+0000 7fe547200640 1 --2- 172.21.15.17:0/864149694 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fe5240725e0 0x7fe524074aa0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:23.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:23.578+0000 7fe547200640 1 --2- 172.21.15.17:0/864149694 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe5400c85f0 0x7fe540145c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:23.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:23.578+0000 7fe547200640 1 --2- 172.21.15.17:0/864149694 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe5400c7b10 0x7fe540145750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:23.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:23.578+0000 7fe547200640 1 --2- 172.21.15.17:0/864149694 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe54006a0e0 0x7fe54014bc10 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:23.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:23.578+0000 7fe547200640 1 -- 172.21.15.17:0/864149694 >> 172.21.15.17:0/864149694 conn(0x7fe5400c8ab0 msgr2=0x7fe540073580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:45:23.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:23.578+0000 7fe547200640 1 -- 172.21.15.17:0/864149694 shutdown_connections 2024-09-10T21:45:23.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:23.578+0000 7fe547200640 1 -- 172.21.15.17:0/864149694 wait complete. 2024-09-10T21:45:26.658 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-10T21:45:30.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:30.550+0000 7fbad2200640 1 -- 172.21.15.17:0/1878133161 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fbaac003a60 msgr2=0x7fbaac003e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:30.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:30.550+0000 7fbad2200640 1 --2- 172.21.15.17:0/1878133161 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fbaac003a60 0x7fbaac003e60 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fbab400c260 tx=0x7fbab401cf20 comp rx=0 tx=0).stop 2024-09-10T21:45:30.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:30.550+0000 7fbad2200640 1 -- 172.21.15.17:0/1878133161 shutdown_connections 2024-09-10T21:45:30.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:30.550+0000 7fbad2200640 1 --2- 172.21.15.17:0/1878133161 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbaac0053d0 0x7fbaac00b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:30.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:30.550+0000 7fbad2200640 1 --2- 172.21.15.17:0/1878133161 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbaac004810 0x7fbaac004c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:30.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:30.550+0000 7fbad2200640 1 --2- 172.21.15.17:0/1878133161 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fbaac003a60 0x7fbaac003e60 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:30.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:30.550+0000 7fbad2200640 1 -- 172.21.15.17:0/1878133161 >> 172.21.15.17:0/1878133161 conn(0x7fbacc06ed90 msgr2=0x7fbacc06f1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:45:30.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:30.550+0000 7fbad2200640 1 -- 172.21.15.17:0/1878133161 shutdown_connections 2024-09-10T21:45:30.554 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:30.550+0000 7fbad2200640 1 -- 172.21.15.17:0/1878133161 wait complete. 2024-09-10T21:45:30.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:30.554+0000 7fbad2200640 1 Processor -- start 2024-09-10T21:45:30.556 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:30.554+0000 7fbad2200640 1 -- start start 2024-09-10T21:45:30.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:30.554+0000 7fbad2200640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbaac003a60 0x7fbacc1f11e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:30.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:30.554+0000 7fbad2200640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbaac004810 0x7fbacc14c260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:30.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:30.554+0000 7fbad2200640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fbaac0053d0 0x7fbacc0d07f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:30.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:30.554+0000 7fbad2200640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fbacc1514c0 con 0x7fbaac004810 2024-09-10T21:45:30.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:30.554+0000 7fbad2200640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fbacc151340 con 0x7fbaac003a60 2024-09-10T21:45:30.558 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:30.554+0000 7fbad2200640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7fbacc15a5b0 con 0x7fbaac0053d0 2024-09-10T21:45:30.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:30.554+0000 7fbad1800640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fbaac0053d0 0x7fbacc0d07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:30.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:30.554+0000 7fbad0e00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbaac003a60 0x7fbacc1f11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:30.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:30.554+0000 7fbad1800640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fbaac0053d0 0x7fbacc0d07f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3301/0 says I am v2:172.21.15.17:46418/0 (socket says 172.21.15.17:46418) 2024-09-10T21:45:30.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:30.554+0000 7fbad1800640 1 -- 172.21.15.17:0/1426158295 learned_addr learned my addr 172.21.15.17:0/1426158295 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:45:30.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:30.558+0000 7fbad1800640 1 -- 172.21.15.17:0/1426158295 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fbaac0053d0 msgr2=0x7fbacc0d07f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 16 2024-09-10T21:45:30.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:30.558+0000 7fbad1800640 1 -- 172.21.15.17:0/1426158295 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fbaac0053d0 msgr2=0x7fbacc0d07f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-10T21:45:30.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:30.558+0000 7fbad1800640 1 --2- 172.21.15.17:0/1426158295 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fbaac0053d0 0x7fbacc0d07f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T21:45:30.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:30.558+0000 7fbad1800640 1 --2- 172.21.15.17:0/1426158295 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fbaac0053d0 0x7fbacc0d07f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:45:30.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:30.558+0000 7fbac3e00640 1 --2- 172.21.15.17:0/1426158295 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbaac004810 0x7fbacc14c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:30.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:30.558+0000 7fbad0e00640 1 -- 172.21.15.17:0/1426158295 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fbaac0053d0 msgr2=0x7fbacc0d07f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:45:30.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:30.558+0000 7fbad0e00640 1 --2- 172.21.15.17:0/1426158295 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fbaac0053d0 0x7fbacc0d07f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:30.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:30.558+0000 7fbad0e00640 1 -- 172.21.15.17:0/1426158295 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbaac004810 msgr2=0x7fbacc14c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:30.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:30.558+0000 7fbad0e00640 1 --2- 172.21.15.17:0/1426158295 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbaac004810 0x7fbacc14c260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:30.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:30.558+0000 7fbad0e00640 1 -- 172.21.15.17:0/1426158295 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbacc155020 con 0x7fbaac003a60 2024-09-10T21:45:30.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:30.558+0000 7fbad0e00640 1 --2- 172.21.15.17:0/1426158295 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbaac003a60 0x7fbacc1f11e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fbab40a9000 tx=0x7fbab4001d00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:45:30.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:30.562+0000 7fbac9600640 1 -- 172.21.15.17:0/1426158295 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbab4004230 con 0x7fbaac003a60 2024-09-10T21:45:30.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:30.562+0000 7fbac9600640 1 -- 172.21.15.17:0/1426158295 <== mon.1 v2:172.21.15.31:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbab4004500 con 0x7fbaac003a60 2024-09-10T21:45:30.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:30.562+0000 7fbac9600640 1 -- 172.21.15.17:0/1426158295 <== mon.1 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbab4004950 con 0x7fbaac003a60 2024-09-10T21:45:30.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:30.562+0000 7fbad2200640 1 -- 172.21.15.17:0/1426158295 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbacc1552e0 con 0x7fbaac003a60 2024-09-10T21:45:30.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:30.562+0000 7fbad2200640 1 -- 172.21.15.17:0/1426158295 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbacc1f0700 con 0x7fbaac003a60 2024-09-10T21:45:30.574 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:30.570+0000 7fbac2000640 1 -- 172.21.15.17:0/1426158295 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbacc0cc390 con 0x7fbaac003a60 2024-09-10T21:45:30.580 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:30.570+0000 7fbac9600640 1 -- 172.21.15.17:0/1426158295 <== mon.1 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fbab40c1030 con 0x7fbaac003a60 2024-09-10T21:45:30.580 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:30.574+0000 7fbac9600640 1 --2- 172.21.15.17:0/1426158295 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fbaa8075670 0x7fbaa8077b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:30.580 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:30.574+0000 7fbac9600640 1 -- 172.21.15.17:0/1426158295 <== mon.1 v2:172.21.15.31:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fbab412d870 con 0x7fbaac003a60 2024-09-10T21:45:30.581 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:30.578+0000 7fbac3e00640 1 --2- 172.21.15.17:0/1426158295 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fbaa8075670 0x7fbaa8077b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:30.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:30.582+0000 7fbac9600640 1 -- 172.21.15.17:0/1426158295 <== mon.1 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fbab4016d10 con 0x7fbaac003a60 2024-09-10T21:45:30.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:30.586+0000 7fbac3e00640 1 --2- 172.21.15.17:0/1426158295 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fbaa8075670 0x7fbaa8077b30 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fbacc0d1050 tx=0x7fbabc012580 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:45:31.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:31.310+0000 7fbac2000640 1 -- 172.21.15.17:0/1426158295 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fbacc0cc390 con 0x7fbaac003a60 2024-09-10T21:45:31.323 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:31.318+0000 7fbac9600640 1 -- 172.21.15.17:0/1426158295 <== mon.1 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1557 (secure 0 0 0) 0x7fbab40f8ff0 con 0x7fbaac003a60 2024-09-10T21:45:31.327 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":160,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5e2f023f-d6a6-49f4-a256-f4807dce3739","modified":"2024-09-10T21:23:46.892130Z","created":"2024-09-10T21:23:46.892130Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3301","nonce":0},{"type":"v1","addr":"172.21.15.31:6790","nonce":0}]},"addr":"172.21.15.31:6790/0","public_addr":"172.21.15.31:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-10T21:45:31.334 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:31.330+0000 7fbac2000640 1 -- 172.21.15.17:0/1426158295 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fbaa8075670 msgr2=0x7fbaa8077b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:31.335 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:31.334+0000 7fbac2000640 1 --2- 172.21.15.17:0/1426158295 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fbaa8075670 0x7fbaa8077b30 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fbacc0d1050 tx=0x7fbabc012580 comp rx=0 tx=0).stop 2024-09-10T21:45:31.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:31.334+0000 7fbac2000640 1 -- 172.21.15.17:0/1426158295 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbaac003a60 msgr2=0x7fbacc1f11e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:31.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:31.334+0000 7fbac2000640 1 --2- 172.21.15.17:0/1426158295 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbaac003a60 0x7fbacc1f11e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fbab40a9000 tx=0x7fbab4001d00 comp rx=0 tx=0).stop 2024-09-10T21:45:31.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:31.334+0000 7fbac2000640 1 -- 172.21.15.17:0/1426158295 shutdown_connections 2024-09-10T21:45:31.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:31.334+0000 7fbac2000640 1 --2- 172.21.15.17:0/1426158295 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fbaa8075670 0x7fbaa8077b30 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:31.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:31.334+0000 7fbac2000640 1 --2- 172.21.15.17:0/1426158295 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fbaac0053d0 0x7fbacc0d07f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:31.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:31.334+0000 7fbac2000640 1 --2- 172.21.15.17:0/1426158295 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbaac004810 0x7fbacc14c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:31.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:31.334+0000 7fbac2000640 1 --2- 172.21.15.17:0/1426158295 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbaac003a60 0x7fbacc1f11e0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:31.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:31.334+0000 7fbac2000640 1 -- 172.21.15.17:0/1426158295 >> 172.21.15.17:0/1426158295 conn(0x7fbacc06ed90 msgr2=0x7fbacc0c66b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:45:31.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:31.334+0000 7fbac2000640 1 -- 172.21.15.17:0/1426158295 shutdown_connections 2024-09-10T21:45:31.338 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:31.334+0000 7fbac2000640 1 -- 172.21.15.17:0/1426158295 wait complete. 2024-09-10T21:45:31.402 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-09-10T21:45:31.402 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-09-10T21:45:31.402 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-10T21:45:31.645 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:45:31.649 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:45:31.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:31.818+0000 7f8cd2c00640 1 -- 172.21.15.17:0/2051976277 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8ccc0c6ac0 msgr2=0x7f8ccc070980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:31.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:31.818+0000 7f8cd2c00640 1 --2- 172.21.15.17:0/2051976277 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8ccc0c6ac0 0x7f8ccc070980 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f8cc0009840 tx=0x7f8cc001f030 comp rx=0 tx=0).stop 2024-09-10T21:45:31.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:31.818+0000 7f8cd2c00640 1 -- 172.21.15.17:0/2051976277 shutdown_connections 2024-09-10T21:45:31.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:31.818+0000 7f8cd2c00640 1 --2- 172.21.15.17:0/2051976277 >> v2:172.21.15.31:3300/0 conn(0x7f8ccc15a240 0x7f8ccc15a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:31.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:31.818+0000 7f8cd2c00640 1 --2- 172.21.15.17:0/2051976277 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8ccc0c6ac0 0x7f8ccc070980 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:31.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:31.818+0000 7f8cd2c00640 1 -- 172.21.15.17:0/2051976277 >> 172.21.15.17:0/2051976277 conn(0x7f8ccc06fe60 msgr2=0x7f8ccc070270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:45:31.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:31.818+0000 7f8cd2c00640 1 -- 172.21.15.17:0/2051976277 shutdown_connections 2024-09-10T21:45:31.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:31.818+0000 7f8cd2c00640 1 -- 172.21.15.17:0/2051976277 wait complete. 2024-09-10T21:45:31.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:31.822+0000 7f8cd2c00640 1 Processor -- start 2024-09-10T21:45:31.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:31.822+0000 7f8cd2c00640 1 -- start start 2024-09-10T21:45:31.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:31.826+0000 7f8cd2c00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8ccc0c6ac0 0x7f8ccc153800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:31.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:31.826+0000 7f8cd2c00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8ccc0c6ea0 0x7f8ccc152a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:31.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:31.826+0000 7f8cd2c00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8ccc15a240 0x7f8ccc1586c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:31.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:31.826+0000 7f8cd2c00640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f8ccc150780 con 0x7f8ccc0c6ac0 2024-09-10T21:45:31.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:31.826+0000 7f8cd2c00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f8ccc0c8480 con 0x7f8ccc15a240 2024-09-10T21:45:31.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:31.826+0000 7f8cd2c00640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f8ccc150900 con 0x7f8ccc0c6ea0 2024-09-10T21:45:31.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:31.826+0000 7f8cd0e00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8ccc0c6ea0 0x7f8ccc152a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:31.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:31.826+0000 7f8cd0e00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8ccc0c6ea0 0x7f8ccc152a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3301/0 says I am v2:172.21.15.17:46438/0 (socket says 172.21.15.17:46438) 2024-09-10T21:45:31.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:31.826+0000 7f8cd0e00640 1 -- 172.21.15.17:0/2927227510 learned_addr learned my addr 172.21.15.17:0/2927227510 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:45:31.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:31.826+0000 7f8cd1800640 1 --2- 172.21.15.17:0/2927227510 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8ccc0c6ac0 0x7f8ccc153800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:31.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:31.826+0000 7f8cd0e00640 1 -- 172.21.15.17:0/2927227510 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8ccc15a240 msgr2=0x7f8ccc1586c0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-10T21:45:31.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:31.826+0000 7f8cd0e00640 1 --2- 172.21.15.17:0/2927227510 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8ccc15a240 0x7f8ccc1586c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:31.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:31.826+0000 7f8cd0e00640 1 -- 172.21.15.17:0/2927227510 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8ccc0c6ac0 msgr2=0x7f8ccc153800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:31.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:31.826+0000 7f8cd0e00640 1 --2- 172.21.15.17:0/2927227510 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8ccc0c6ac0 0x7f8ccc153800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:31.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:31.826+0000 7f8cd0e00640 1 -- 172.21.15.17:0/2927227510 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ccc06d990 con 0x7f8ccc0c6ea0 2024-09-10T21:45:31.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:31.826+0000 7f8cd0e00640 1 --2- 172.21.15.17:0/2927227510 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8ccc0c6ea0 0x7f8ccc152a20 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f8cb800e7a0 tx=0x7f8cb800ec60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:45:31.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:31.830+0000 7f8cc9600640 1 -- 172.21.15.17:0/2927227510 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cb800ccc0 con 0x7f8ccc0c6ea0 2024-09-10T21:45:31.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:31.830+0000 7f8cc9600640 1 -- 172.21.15.17:0/2927227510 <== mon.2 v2:172.21.15.31:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8cb8010430 con 0x7f8ccc0c6ea0 2024-09-10T21:45:31.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:31.830+0000 7f8cc9600640 1 -- 172.21.15.17:0/2927227510 <== mon.2 v2:172.21.15.31:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cb8010880 con 0x7f8ccc0c6ea0 2024-09-10T21:45:31.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:31.830+0000 7f8cd2c00640 1 -- 172.21.15.17:0/2927227510 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ccc1577d0 con 0x7f8ccc0c6ea0 2024-09-10T21:45:31.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:31.830+0000 7f8cd2c00640 1 -- 172.21.15.17:0/2927227510 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ccc151bb0 con 0x7f8ccc0c6ea0 2024-09-10T21:45:31.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:31.834+0000 7f8cbea00640 1 --2- 172.21.15.17:0/2927227510 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8ccc152f60 0x7f8ccc141d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:31.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:31.834+0000 7f8cbea00640 1 -- 172.21.15.17:0/2927227510 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f8ccc157f70 con 0x7f8ccc152f60 2024-09-10T21:45:31.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:31.838+0000 7f8cc9600640 1 -- 172.21.15.17:0/2927227510 <== mon.2 v2:172.21.15.31:3301/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f8cb8010a20 con 0x7f8ccc0c6ea0 2024-09-10T21:45:31.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:31.838+0000 7f8cc9600640 1 --2- 172.21.15.17:0/2927227510 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f8cac072630 0x7f8cac074af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:31.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:31.838+0000 7f8cc9600640 1 -- 172.21.15.17:0/2927227510 <== mon.2 v2:172.21.15.31:3301/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f8cb8094140 con 0x7f8ccc0c6ea0 2024-09-10T21:45:31.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:31.842+0000 7f8cd1800640 1 --2- 172.21.15.17:0/2927227510 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8ccc152f60 0x7f8ccc141d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:31.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:31.850+0000 7f8cd1800640 1 -- 172.21.15.17:0/2927227510 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8ccc072470 con 0x7f8ccc152f60 2024-09-10T21:45:31.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:31.850+0000 7f8cd1800640 1 --2- 172.21.15.17:0/2927227510 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8ccc152f60 0x7f8ccc141d40 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f8ccc0c99e0 tx=0x7f8cc00a9bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:45:31.853 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:31.850+0000 7f8cc9600640 1 -- 172.21.15.17:0/2927227510 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cc00bf070 con 0x7f8ccc152f60 2024-09-10T21:45:31.853 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:31.850+0000 7f8cc9600640 1 -- 172.21.15.17:0/2927227510 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f8cc0005240 con 0x7f8ccc152f60 2024-09-10T21:45:31.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:31.850+0000 7f8cc9600640 1 -- 172.21.15.17:0/2927227510 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8ccc152f60 msgr2=0x7f8ccc141d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:31.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:31.850+0000 7f8cd2200640 1 --2- 172.21.15.17:0/2927227510 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f8cac072630 0x7f8cac074af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:31.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:31.850+0000 7f8cc9600640 1 --2- 172.21.15.17:0/2927227510 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8ccc152f60 0x7f8ccc141d40 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f8ccc0c99e0 tx=0x7f8cc00a9bb0 comp rx=0 tx=0).stop 2024-09-10T21:45:31.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:31.850+0000 7f8cc9600640 1 -- 172.21.15.17:0/2927227510 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8ccc152f60 msgr2=0x7f8ccc141d40 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:45:31.854 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:31.850+0000 7f8cc9600640 1 --2- 172.21.15.17:0/2927227510 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8ccc152f60 0x7f8ccc141d40 secure :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f8ccc0c99e0 tx=0x7f8cc00a9bb0 comp rx=0 tx=0).stop 2024-09-10T21:45:31.870 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:31.866+0000 7f8cd2200640 1 --2- 172.21.15.17:0/2927227510 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f8cac072630 0x7f8cac074af0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f8cc4001ba0 tx=0x7f8cc4007d20 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:45:31.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:31.866+0000 7f8cbea00640 1 --2- 172.21.15.17:0/2927227510 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8ccc158c00 0x7f8ccc1f18e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:31.871 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:31.866+0000 7f8cbea00640 1 -- 172.21.15.17:0/2927227510 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f8ccc14fb10 con 0x7f8ccc158c00 2024-09-10T21:45:31.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:31.874+0000 7f8cd1800640 1 --2- 172.21.15.17:0/2927227510 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8ccc158c00 0x7f8ccc1f18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:31.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:31.890+0000 7f8cd1800640 1 -- 172.21.15.17:0/2927227510 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f8ccc0c8260 con 0x7f8ccc158c00 2024-09-10T21:45:31.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:31.890+0000 7f8cd1800640 1 --2- 172.21.15.17:0/2927227510 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8ccc158c00 0x7f8ccc1f18e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f8cc001f8d0 tx=0x7f8cc00091e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:45:31.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:31.890+0000 7f8cc9600640 1 -- 172.21.15.17:0/2927227510 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cc00bf070 con 0x7f8ccc158c00 2024-09-10T21:45:31.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:31.890+0000 7f8cc9600640 1 -- 172.21.15.17:0/2927227510 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5045 (secure 0 0 0) 0x7f8cc0000e50 con 0x7f8ccc158c00 2024-09-10T21:45:31.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:31.890+0000 7f8cc9600640 1 -- 172.21.15.17:0/2927227510 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8ccc158c00 msgr2=0x7f8ccc1f18e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:31.895 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:31.890+0000 7f8cc9600640 1 --2- 172.21.15.17:0/2927227510 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8ccc158c00 0x7f8ccc1f18e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f8cc001f8d0 tx=0x7f8cc00091e0 comp rx=0 tx=0).stop 2024-09-10T21:45:31.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:31.890+0000 7f8cc9600640 1 -- 172.21.15.17:0/2927227510 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8ccc158c00 msgr2=0x7f8ccc1f18e0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:45:31.896 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:31.890+0000 7f8cc9600640 1 --2- 172.21.15.17:0/2927227510 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8ccc158c00 0x7f8ccc1f18e0 secure :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f8cc001f8d0 tx=0x7f8cc00091e0 comp rx=0 tx=0).stop 2024-09-10T21:45:31.897 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-10T21:45:31.897 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:45:31.897 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:45:31.897 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "leader", 2024-09-10T21:45:31.897 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 160, 2024-09-10T21:45:31.897 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 191022, 2024-09-10T21:45:31.897 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-10T21:45:31.897 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-10T21:45:31.898 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-10T21:45:31.898 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-10T21:45:31.898 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:45:31.898 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 1, 2024-09-10T21:45:31.898 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:45:31.898 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-10T21:45:31.898 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-10T21:45:31.898 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:45:31.898 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:45:31.898 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:45:31.898 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:45:31.898 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:45:31.899 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:45:31.899 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:45:31.899 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:45:31.899 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:45:31.899 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:45:31.899 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:45:31.899 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:45:31.899 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-10T21:45:31.899 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-10T21:45:31.899 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:45:31.899 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:45:31.899 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:45:31.899 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:45:31.900 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:45:31.900 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:45:31.900 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:45:31.900 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:45:31.900 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:45:31.900 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:45:31.900 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:45:31.900 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:45:31.900 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:45:31.900 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-10T21:45:31.900 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-10T21:45:31.900 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-10T21:45:31.901 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-10T21:45:31.901 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-10T21:45:31.901 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "5e2f023f-d6a6-49f4-a256-f4807dce3739", 2024-09-10T21:45:31.901 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:45:31.901 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:45:31.901 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-10T21:45:31.901 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-10T21:45:31.901 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-10T21:45:31.901 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-10T21:45:31.901 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-10T21:45:31.901 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-10T21:45:31.901 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-10T21:45:31.901 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:45:31.902 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-10T21:45:31.902 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:45:31.902 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:45:31.902 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:45:31.902 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:45:31.902 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:45:31.902 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:45:31.902 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:45:31.902 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:45:31.903 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:45:31.903 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:45:31.903 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:45:31.903 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:45:31.903 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-10T21:45:31.903 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:45:31.903 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-10T21:45:31.903 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:45:31.903 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:45:31.903 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:45:31.903 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:45:31.903 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:45:31.903 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:45:31.904 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:45:31.904 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-10T21:45:31.904 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:45:31.904 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:45:31.904 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:45:31.904 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:45:31.904 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-10T21:45:31.904 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:45:31.904 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:45:31.904 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:45:31.904 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:45:31.904 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-10T21:45:31.904 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-10T21:45:31.905 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:45:31.905 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:45:31.905 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:45:31.905 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:45:31.905 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:45:31.905 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:45:31.905 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:45:31.905 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:45:31.905 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:45:31.905 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:45:31.905 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:45:31.905 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3300", 2024-09-10T21:45:31.905 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:45:31.906 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:45:31.906 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:45:31.906 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:45:31.906 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789", 2024-09-10T21:45:31.906 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:45:31.906 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:45:31.906 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:45:31.906 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:45:31.906 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789/0", 2024-09-10T21:45:31.906 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6789/0", 2024-09-10T21:45:31.906 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:45:31.906 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:45:31.906 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:45:31.906 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:45:31.907 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:45:31.907 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:45:31.907 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:45:31.907 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:45:31.907 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:45:31.907 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:45:31.907 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:45:31.907 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3301", 2024-09-10T21:45:31.907 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:45:31.907 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:45:31.907 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:45:31.907 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:45:31.907 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790", 2024-09-10T21:45:31.908 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:45:31.908 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:45:31.908 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:45:31.908 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:45:31.908 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790/0", 2024-09-10T21:45:31.908 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6790/0", 2024-09-10T21:45:31.908 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:45:31.908 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:45:31.908 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:45:31.908 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:45:31.908 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:45:31.908 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:45:31.908 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-10T21:45:31.909 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-10T21:45:31.909 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:45:31.909 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:45:31.909 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:45:31.909 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:45:31.909 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:45:31.909 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:45:31.909 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": [ 2024-09-10T21:45:31.909 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:45:31.909 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:45:31.909 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:45:31.909 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 3 2024-09-10T21:45:31.909 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:45:31.910 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:45:31.910 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-10T21:45:31.910 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:45:31.910 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:45:31.910 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:45:31.910 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 7 2024-09-10T21:45:31.910 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:45:31.910 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:45:31.910 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-10T21:45:31.910 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:45:31.910 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:45:31.910 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:45:31.910 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 4 2024-09-10T21:45:31.911 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:45:31.911 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:45:31.911 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": [ 2024-09-10T21:45:31.911 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:45:31.911 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:45:31.911 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:45:31.911 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:45:31.911 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:45:31.911 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:45:31.911 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:45:31.911 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-10T21:45:31.911 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-10T21:45:31.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:31.894+0000 7f8cbea00640 1 -- 172.21.15.17:0/2927227510 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f8cac072630 msgr2=0x7f8cac074af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:31.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:31.898+0000 7f8cbea00640 1 --2- 172.21.15.17:0/2927227510 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f8cac072630 0x7f8cac074af0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f8cc4001ba0 tx=0x7f8cc4007d20 comp rx=0 tx=0).stop 2024-09-10T21:45:31.912 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:31.898+0000 7f8cbea00640 1 -- 172.21.15.17:0/2927227510 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8ccc0c6ea0 msgr2=0x7f8ccc152a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:31.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:31.898+0000 7f8cbea00640 1 --2- 172.21.15.17:0/2927227510 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8ccc0c6ea0 0x7f8ccc152a20 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f8cb800e7a0 tx=0x7f8cb800ec60 comp rx=0 tx=0).stop 2024-09-10T21:45:31.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:31.898+0000 7f8cd2200640 1 -- 172.21.15.17:0/2927227510 reap_dead start 2024-09-10T21:45:31.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:31.898+0000 7f8cbea00640 1 -- 172.21.15.17:0/2927227510 shutdown_connections 2024-09-10T21:45:31.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:31.898+0000 7f8cbea00640 1 --2- 172.21.15.17:0/2927227510 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8ccc0c6ea0 0x7f8ccc152a20 secure :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f8cb800e7a0 tx=0x7f8cb800ec60 comp rx=0 tx=0).stop 2024-09-10T21:45:31.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:31.898+0000 7f8cbea00640 1 -- 172.21.15.17:0/2927227510 >> 172.21.15.17:0/2927227510 conn(0x7f8ccc06fe60 msgr2=0x7f8ccc154980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:45:31.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:31.906+0000 7f8cbea00640 1 -- 172.21.15.17:0/2927227510 shutdown_connections 2024-09-10T21:45:31.913 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:31.906+0000 7f8cbea00640 1 -- 172.21.15.17:0/2927227510 wait complete. 2024-09-10T21:45:31.963 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-10T21:45:31.963 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-09-10T21:45:31.963 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-10T21:45:32.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:32.266+0000 7f7b15e00640 1 -- 172.21.15.17:0/964895853 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f7b1006e3a0 msgr2=0x7f7b1006e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:32.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:32.266+0000 7f7b15e00640 1 --2- 172.21.15.17:0/964895853 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f7b1006e3a0 0x7f7b1006e780 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f7b00009900 tx=0x7f7b0001c850 comp rx=0 tx=0).stop 2024-09-10T21:45:32.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:32.266+0000 7f7b15e00640 1 -- 172.21.15.17:0/964895853 shutdown_connections 2024-09-10T21:45:32.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:32.266+0000 7f7b15e00640 1 --2- 172.21.15.17:0/964895853 >> v2:172.21.15.17:3300/0 conn(0x7f7b10070980 0x7f7b1006ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:32.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:32.266+0000 7f7b15e00640 1 --2- 172.21.15.17:0/964895853 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f7b1006e3a0 0x7f7b1006e780 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:32.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:32.266+0000 7f7b15e00640 1 -- 172.21.15.17:0/964895853 >> 172.21.15.17:0/964895853 conn(0x7f7b1006fe60 msgr2=0x7f7b10070270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:45:32.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:32.270+0000 7f7b15e00640 1 -- 172.21.15.17:0/964895853 shutdown_connections 2024-09-10T21:45:32.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:32.270+0000 7f7b15e00640 1 -- 172.21.15.17:0/964895853 wait complete. 2024-09-10T21:45:32.275 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:32.270+0000 7f7b15e00640 1 Processor -- start 2024-09-10T21:45:32.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:32.278+0000 7f7b15e00640 1 -- start start 2024-09-10T21:45:32.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:32.278+0000 7f7b15e00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7b1006e3a0 0x7f7b100c3500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:32.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:32.282+0000 7f7b15e00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7b10070980 0x7f7b1015c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:32.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:32.282+0000 7f7b15e00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f7b100c7100 0x7f7b101f8090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:32.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:32.282+0000 7f7b15e00640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f7b101502f0 con 0x7f7b10070980 2024-09-10T21:45:32.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:32.282+0000 7f7b15e00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f7b10150170 con 0x7f7b1006e3a0 2024-09-10T21:45:32.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:32.282+0000 7f7b15e00640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f7b10150470 con 0x7f7b100c7100 2024-09-10T21:45:32.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:32.282+0000 7f7b15400640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f7b100c7100 0x7f7b101f8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:32.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:32.282+0000 7f7b14a00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7b1006e3a0 0x7f7b100c3500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:32.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:32.282+0000 7f7b14a00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7b1006e3a0 0x7f7b100c3500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.17:37420/0 (socket says 172.21.15.17:37420) 2024-09-10T21:45:32.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:32.282+0000 7f7b14a00640 1 -- 172.21.15.17:0/670534404 learned_addr learned my addr 172.21.15.17:0/670534404 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:45:32.289 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:32.286+0000 7f7b0fe00640 1 --2- 172.21.15.17:0/670534404 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7b10070980 0x7f7b1015c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:32.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:32.286+0000 7f7b14a00640 1 -- 172.21.15.17:0/670534404 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f7b100c7100 msgr2=0x7f7b101f8090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:32.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:32.286+0000 7f7b14a00640 1 --2- 172.21.15.17:0/670534404 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f7b100c7100 0x7f7b101f8090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:32.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:32.286+0000 7f7b14a00640 1 -- 172.21.15.17:0/670534404 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7b10070980 msgr2=0x7f7b1015c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:32.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:32.286+0000 7f7b14a00640 1 --2- 172.21.15.17:0/670534404 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7b10070980 0x7f7b1015c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:32.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:32.286+0000 7f7b14a00640 1 -- 172.21.15.17:0/670534404 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b10169a60 con 0x7f7b1006e3a0 2024-09-10T21:45:32.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:32.286+0000 7f7b15400640 1 --2- 172.21.15.17:0/670534404 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f7b100c7100 0x7f7b101f8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:45:32.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:32.286+0000 7f7b14a00640 1 --2- 172.21.15.17:0/670534404 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7b1006e3a0 0x7f7b100c3500 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f7b00000c10 tx=0x7f7b000a6630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:45:32.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:32.286+0000 7f7b0cc00640 1 -- 172.21.15.17:0/670534404 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b000ac070 con 0x7f7b1006e3a0 2024-09-10T21:45:32.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:32.286+0000 7f7b0cc00640 1 -- 172.21.15.17:0/670534404 <== mon.1 v2:172.21.15.31:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7b000054d0 con 0x7f7b1006e3a0 2024-09-10T21:45:32.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:32.286+0000 7f7b15e00640 1 -- 172.21.15.17:0/670534404 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b10169d00 con 0x7f7b1006e3a0 2024-09-10T21:45:32.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:32.290+0000 7f7b15e00640 1 -- 172.21.15.17:0/670534404 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b1016a2a0 con 0x7f7b1006e3a0 2024-09-10T21:45:32.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:32.290+0000 7f7b0cc00640 1 -- 172.21.15.17:0/670534404 <== mon.1 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b00004930 con 0x7f7b1006e3a0 2024-09-10T21:45:32.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:32.290+0000 7f7b15e00640 1 --2- 172.21.15.17:0/670534404 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7b101f77d0 0x7f7b100cb140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:32.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:32.290+0000 7f7b15e00640 1 -- 172.21.15.17:0/670534404 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f7b1015ccb0 con 0x7f7b101f77d0 2024-09-10T21:45:32.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:32.290+0000 7f7b0fe00640 1 --2- 172.21.15.17:0/670534404 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7b101f77d0 0x7f7b100cb140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:32.300 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:32.294+0000 7f7b0fe00640 1 -- 172.21.15.17:0/670534404 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7b100c7570 con 0x7f7b101f77d0 2024-09-10T21:45:32.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:32.294+0000 7f7b0fe00640 1 --2- 172.21.15.17:0/670534404 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7b101f77d0 0x7f7b100cb140 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f7b1015a130 tx=0x7f7afc00db90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:45:32.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:32.294+0000 7f7b0cc00640 1 -- 172.21.15.17:0/670534404 <== mon.1 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f7b00004ad0 con 0x7f7b1006e3a0 2024-09-10T21:45:32.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:32.298+0000 7f7b0cc00640 1 --2- 172.21.15.17:0/670534404 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f7af0072630 0x7f7af0074af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:32.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:32.298+0000 7f7b0cc00640 1 -- 172.21.15.17:0/670534404 <== mon.1 v2:172.21.15.31:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f7b0012dba0 con 0x7f7b1006e3a0 2024-09-10T21:45:32.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:32.298+0000 7f7b0cc00640 1 -- 172.21.15.17:0/670534404 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7afc018030 con 0x7f7b101f77d0 2024-09-10T21:45:32.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:32.298+0000 7f7b0cc00640 1 -- 172.21.15.17:0/670534404 <== mon.1 v2:172.21.15.31:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f7afc0096e0 con 0x7f7b101f77d0 2024-09-10T21:45:32.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:32.298+0000 7f7b0cc00640 1 -- 172.21.15.17:0/670534404 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7b101f77d0 msgr2=0x7f7b100cb140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:32.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:32.298+0000 7f7b0cc00640 1 --2- 172.21.15.17:0/670534404 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7b101f77d0 0x7f7b100cb140 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f7b1015a130 tx=0x7f7afc00db90 comp rx=0 tx=0).stop 2024-09-10T21:45:32.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:32.298+0000 7f7b0cc00640 1 -- 172.21.15.17:0/670534404 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7b101f77d0 msgr2=0x7f7b100cb140 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:45:32.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:32.298+0000 7f7b0cc00640 1 --2- 172.21.15.17:0/670534404 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7b101f77d0 0x7f7b100cb140 secure :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f7b1015a130 tx=0x7f7afc00db90 comp rx=0 tx=0).stop 2024-09-10T21:45:32.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:32.298+0000 7f7b15400640 1 --2- 172.21.15.17:0/670534404 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f7af0072630 0x7f7af0074af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:32.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:32.310+0000 7f7b15e00640 1 --2- 172.21.15.17:0/670534404 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7b101f7bb0 0x7f7b1014b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:32.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:32.310+0000 7f7b15e00640 1 -- 172.21.15.17:0/670534404 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f7b1014e3d0 con 0x7f7b101f7bb0 2024-09-10T21:45:32.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:32.314+0000 7f7b15400640 1 --2- 172.21.15.17:0/670534404 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f7af0072630 0x7f7af0074af0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f7b08009f60 tx=0x7f7b080094f0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:45:32.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:32.314+0000 7f7b0fe00640 1 --2- 172.21.15.17:0/670534404 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7b101f7bb0 0x7f7b1014b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:32.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:32.318+0000 7f7b0fe00640 1 -- 172.21.15.17:0/670534404 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f7b1014e0f0 con 0x7f7b101f7bb0 2024-09-10T21:45:32.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:32.326+0000 7f7b0fe00640 1 --2- 172.21.15.17:0/670534404 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7b101f7bb0 0x7f7b1014b790 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f7afc016000 tx=0x7f7afc0047e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:45:32.364 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-10T21:45:32.364 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:45:32.364 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:45:32.364 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-10T21:45:32.365 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 160, 2024-09-10T21:45:32.365 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 77327, 2024-09-10T21:45:32.365 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-10T21:45:32.365 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-10T21:45:32.365 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-10T21:45:32.365 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-10T21:45:32.365 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:45:32.365 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 1, 2024-09-10T21:45:32.365 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:45:32.365 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-10T21:45:32.365 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-10T21:45:32.365 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:45:32.365 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:45:32.366 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:45:32.366 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:45:32.366 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:45:32.366 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:45:32.366 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:45:32.366 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:45:32.366 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:45:32.366 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:45:32.366 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:45:32.366 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:45:32.366 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-10T21:45:32.366 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-10T21:45:32.366 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:45:32.366 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:45:32.367 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:45:32.367 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:45:32.367 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:45:32.367 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:45:32.367 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:45:32.367 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:45:32.367 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:45:32.367 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:45:32.367 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:45:32.367 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:45:32.367 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:45:32.368 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-10T21:45:32.368 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-10T21:45:32.368 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-10T21:45:32.368 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-10T21:45:32.368 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-10T21:45:32.368 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "5e2f023f-d6a6-49f4-a256-f4807dce3739", 2024-09-10T21:45:32.368 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:45:32.368 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:45:32.368 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-10T21:45:32.368 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-10T21:45:32.369 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-10T21:45:32.369 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-10T21:45:32.369 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-10T21:45:32.369 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-10T21:45:32.369 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-10T21:45:32.369 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:45:32.369 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-10T21:45:32.369 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:45:32.369 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:45:32.369 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:45:32.369 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:45:32.369 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:45:32.369 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:45:32.370 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:45:32.370 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:45:32.370 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:45:32.370 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:45:32.370 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:45:32.370 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:45:32.370 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-10T21:45:32.370 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:45:32.370 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-10T21:45:32.371 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:45:32.371 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:45:32.371 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:45:32.371 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:45:32.371 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:45:32.371 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:45:32.371 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:45:32.371 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-10T21:45:32.371 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:45:32.371 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:45:32.371 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:45:32.371 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:45:32.371 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-10T21:45:32.371 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:45:32.372 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:45:32.372 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:45:32.372 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:45:32.372 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-10T21:45:32.372 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-10T21:45:32.372 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:45:32.372 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:45:32.372 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:45:32.372 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:45:32.372 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:45:32.372 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:45:32.373 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:45:32.373 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:45:32.373 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:45:32.373 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:45:32.373 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:45:32.373 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3300", 2024-09-10T21:45:32.373 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:45:32.373 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:45:32.373 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:45:32.373 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:45:32.373 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789", 2024-09-10T21:45:32.373 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:45:32.374 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:45:32.374 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:45:32.374 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:45:32.374 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789/0", 2024-09-10T21:45:32.374 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6789/0", 2024-09-10T21:45:32.375 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:45:32.375 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:45:32.375 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:45:32.375 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:45:32.375 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:45:32.375 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:45:32.375 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:45:32.376 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:45:32.376 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:45:32.376 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:45:32.376 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:45:32.376 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3301", 2024-09-10T21:45:32.376 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:45:32.376 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:45:32.376 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:45:32.376 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:45:32.377 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790", 2024-09-10T21:45:32.377 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:45:32.377 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:45:32.377 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:45:32.377 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:45:32.377 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790/0", 2024-09-10T21:45:32.377 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6790/0", 2024-09-10T21:45:32.377 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:45:32.377 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:45:32.377 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:45:32.377 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:45:32.378 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:45:32.378 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:45:32.378 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-10T21:45:32.378 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-10T21:45:32.378 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:45:32.378 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:45:32.379 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:45:32.379 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:45:32.379 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:45:32.379 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:45:32.379 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": [ 2024-09-10T21:45:32.379 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:45:32.379 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:45:32.379 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:45:32.379 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 2 2024-09-10T21:45:32.379 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:45:32.379 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:45:32.380 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-10T21:45:32.380 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:45:32.380 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:45:32.380 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:45:32.380 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:45:32.380 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:45:32.380 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:45:32.380 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-10T21:45:32.380 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:45:32.381 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:45:32.381 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:45:32.381 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 4 2024-09-10T21:45:32.381 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:45:32.381 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:45:32.381 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": [ 2024-09-10T21:45:32.381 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:45:32.381 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:45:32.381 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:45:32.381 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:45:32.381 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:45:32.381 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:45:32.381 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:45:32.381 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-10T21:45:32.382 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-10T21:45:32.383 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:32.354+0000 7f7b0cc00640 1 -- 172.21.15.17:0/670534404 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7afc024070 con 0x7f7b101f7bb0 2024-09-10T21:45:32.383 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:32.354+0000 7f7b0cc00640 1 -- 172.21.15.17:0/670534404 <== mon.1 v2:172.21.15.31:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5042 (secure 0 0 0) 0x7f7afc01eae0 con 0x7f7b101f7bb0 2024-09-10T21:45:32.383 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:32.354+0000 7f7b0cc00640 1 -- 172.21.15.17:0/670534404 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7b101f7bb0 msgr2=0x7f7b1014b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:32.383 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:32.354+0000 7f7b0cc00640 1 --2- 172.21.15.17:0/670534404 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7b101f7bb0 0x7f7b1014b790 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f7afc016000 tx=0x7f7afc0047e0 comp rx=0 tx=0).stop 2024-09-10T21:45:32.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:32.354+0000 7f7b0cc00640 1 -- 172.21.15.17:0/670534404 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7b101f7bb0 msgr2=0x7f7b1014b790 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:45:32.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:32.354+0000 7f7b0cc00640 1 --2- 172.21.15.17:0/670534404 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7b101f7bb0 0x7f7b1014b790 secure :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f7afc016000 tx=0x7f7afc0047e0 comp rx=0 tx=0).stop 2024-09-10T21:45:32.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:32.362+0000 7f7b15e00640 1 -- 172.21.15.17:0/670534404 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f7af0072630 msgr2=0x7f7af0074af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:32.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:32.362+0000 7f7b15e00640 1 --2- 172.21.15.17:0/670534404 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f7af0072630 0x7f7af0074af0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f7b08009f60 tx=0x7f7b080094f0 comp rx=0 tx=0).stop 2024-09-10T21:45:32.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:32.362+0000 7f7b15e00640 1 -- 172.21.15.17:0/670534404 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7b1006e3a0 msgr2=0x7f7b100c3500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:32.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:32.362+0000 7f7b15e00640 1 --2- 172.21.15.17:0/670534404 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7b1006e3a0 0x7f7b100c3500 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f7b00000c10 tx=0x7f7b000a6630 comp rx=0 tx=0).stop 2024-09-10T21:45:32.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:32.370+0000 7f7b15400640 1 -- 172.21.15.17:0/670534404 reap_dead start 2024-09-10T21:45:32.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:32.370+0000 7f7b15400640 1 -- 172.21.15.17:0/670534404 reap_dead start 2024-09-10T21:45:32.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:32.378+0000 7f7b15e00640 1 -- 172.21.15.17:0/670534404 shutdown_connections 2024-09-10T21:45:32.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:32.378+0000 7f7b15e00640 1 -- 172.21.15.17:0/670534404 >> 172.21.15.17:0/670534404 conn(0x7f7b1006fe60 msgr2=0x7f7b1016ab40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:45:32.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:32.394+0000 7f7b15e00640 1 -- 172.21.15.17:0/670534404 shutdown_connections 2024-09-10T21:45:32.398 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:32.394+0000 7f7b15e00640 1 -- 172.21.15.17:0/670534404 wait complete. 2024-09-10T21:45:32.477 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-10T21:45:32.478 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-09-10T21:45:32.478 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-10T21:45:32.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:32.818+0000 7f163e800640 1 -- 172.21.15.17:0/128672355 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1638074980 msgr2=0x7f16380b1ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:32.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:32.818+0000 7f163e800640 1 --2- 172.21.15.17:0/128672355 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1638074980 0x7f16380b1ee0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f1638157880 tx=0x7f1624022fd0 comp rx=0 tx=0).stop 2024-09-10T21:45:32.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:32.818+0000 7f163e800640 1 -- 172.21.15.17:0/128672355 shutdown_connections 2024-09-10T21:45:32.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:32.822+0000 7f163e800640 1 --2- 172.21.15.17:0/128672355 >> v2:172.21.15.17:3300/0 conn(0x7f16380c9540 0x7f1638153940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:32.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:32.822+0000 7f163e800640 1 --2- 172.21.15.17:0/128672355 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1638074980 0x7f16380b1ee0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:32.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:32.822+0000 7f163e800640 1 -- 172.21.15.17:0/128672355 >> 172.21.15.17:0/128672355 conn(0x7f16380700d0 msgr2=0x7f16380704e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:45:32.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:32.822+0000 7f163e800640 1 -- 172.21.15.17:0/128672355 shutdown_connections 2024-09-10T21:45:32.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:32.822+0000 7f163e800640 1 -- 172.21.15.17:0/128672355 wait complete. 2024-09-10T21:45:32.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:32.826+0000 7f163e800640 1 Processor -- start 2024-09-10T21:45:32.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:32.830+0000 7f163e800640 1 -- start start 2024-09-10T21:45:32.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:32.838+0000 7f163e800640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1638074980 0x7f163814c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:32.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:32.838+0000 7f163e800640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f16380c9160 0x7f163814eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:32.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:32.838+0000 7f163e800640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16380c9540 0x7f16381477c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:32.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:32.838+0000 7f163e800640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f1638073e70 con 0x7f1638074980 2024-09-10T21:45:32.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:32.838+0000 7f163e800640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f1638073b90 con 0x7f16380c9540 2024-09-10T21:45:32.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:32.838+0000 7f163e800640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f1638073890 con 0x7f16380c9160 2024-09-10T21:45:32.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:32.838+0000 7f1636a00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f16380c9160 0x7f163814eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:32.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:32.838+0000 7f1636a00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f16380c9160 0x7f163814eb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3301/0 says I am v2:172.21.15.17:46454/0 (socket says 172.21.15.17:46454) 2024-09-10T21:45:32.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:32.838+0000 7f1636a00640 1 -- 172.21.15.17:0/3534600463 learned_addr learned my addr 172.21.15.17:0/3534600463 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:45:32.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:32.838+0000 7f1637e00640 1 --2- 172.21.15.17:0/3534600463 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16380c9540 0x7f16381477c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:32.843 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:32.838+0000 7f1637400640 1 --2- 172.21.15.17:0/3534600463 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1638074980 0x7f163814c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:32.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:32.842+0000 7f1636a00640 1 -- 172.21.15.17:0/3534600463 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16380c9540 msgr2=0x7f16381477c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:32.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:32.842+0000 7f1636a00640 1 --2- 172.21.15.17:0/3534600463 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f16380c9540 0x7f16381477c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:32.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:32.842+0000 7f1636a00640 1 -- 172.21.15.17:0/3534600463 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1638074980 msgr2=0x7f163814c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:32.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:32.842+0000 7f1636a00640 1 --2- 172.21.15.17:0/3534600463 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1638074980 0x7f163814c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:32.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:32.842+0000 7f1636a00640 1 -- 172.21.15.17:0/3534600463 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16380c4820 con 0x7f16380c9160 2024-09-10T21:45:32.844 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:32.842+0000 7f1636a00640 1 --2- 172.21.15.17:0/3534600463 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f16380c9160 0x7f163814eb10 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f162400d880 tx=0x7f16240056f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:45:32.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:32.842+0000 7f162f400640 1 -- 172.21.15.17:0/3534600463 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16240b9070 con 0x7f16380c9160 2024-09-10T21:45:32.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:32.842+0000 7f162f400640 1 -- 172.21.15.17:0/3534600463 <== mon.2 v2:172.21.15.31:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1624004ac0 con 0x7f16380c9160 2024-09-10T21:45:32.845 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:32.842+0000 7f162f400640 1 -- 172.21.15.17:0/3534600463 <== mon.2 v2:172.21.15.31:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16240b4440 con 0x7f16380c9160 2024-09-10T21:45:32.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:32.842+0000 7f163e800640 1 -- 172.21.15.17:0/3534600463 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16380c4ac0 con 0x7f16380c9160 2024-09-10T21:45:32.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:32.842+0000 7f163e800640 1 -- 172.21.15.17:0/3534600463 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f16380c5e90 con 0x7f16380c9160 2024-09-10T21:45:32.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:32.842+0000 7f163e800640 1 --2- 172.21.15.17:0/3534600463 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f1638147d00 0x7f16381f7de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:32.846 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:32.842+0000 7f163e800640 1 -- 172.21.15.17:0/3534600463 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f16380c7140 con 0x7f1638147d00 2024-09-10T21:45:32.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:32.842+0000 7f1637400640 1 --2- 172.21.15.17:0/3534600463 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f1638147d00 0x7f16381f7de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:32.850 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:32.846+0000 7f162f400640 1 -- 172.21.15.17:0/3534600463 <== mon.2 v2:172.21.15.31:3301/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f16240acb80 con 0x7f16380c9160 2024-09-10T21:45:32.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:32.846+0000 7f162f400640 1 --2- 172.21.15.17:0/3534600463 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f16180726b0 0x7f1618074b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:32.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:32.846+0000 7f162f400640 1 -- 172.21.15.17:0/3534600463 <== mon.2 v2:172.21.15.31:3301/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f16241302d0 con 0x7f16380c9160 2024-09-10T21:45:32.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:32.850+0000 7f1637400640 1 -- 172.21.15.17:0/3534600463 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1638073390 con 0x7f1638147d00 2024-09-10T21:45:32.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:32.850+0000 7f1637400640 1 --2- 172.21.15.17:0/3534600463 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f1638147d00 0x7f16381f7de0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f1628000fc0 tx=0x7f162800ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:45:32.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:32.850+0000 7f162f400640 1 -- 172.21.15.17:0/3534600463 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16280052e0 con 0x7f1638147d00 2024-09-10T21:45:32.857 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:32.850+0000 7f1637e00640 1 --2- 172.21.15.17:0/3534600463 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f16180726b0 0x7f1618074b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:32.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:32.854+0000 7f162f400640 1 -- 172.21.15.17:0/3534600463 <== mon.2 v2:172.21.15.31:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f1628018070 con 0x7f1638147d00 2024-09-10T21:45:32.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:32.854+0000 7f162f400640 1 -- 172.21.15.17:0/3534600463 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f1638147d00 msgr2=0x7f16381f7de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:32.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:32.854+0000 7f162f400640 1 --2- 172.21.15.17:0/3534600463 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f1638147d00 0x7f16381f7de0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f1628000fc0 tx=0x7f162800ecd0 comp rx=0 tx=0).stop 2024-09-10T21:45:32.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:32.854+0000 7f162f400640 1 -- 172.21.15.17:0/3534600463 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f1638147d00 msgr2=0x7f16381f7de0 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:45:32.858 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:32.854+0000 7f162f400640 1 --2- 172.21.15.17:0/3534600463 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f1638147d00 0x7f16381f7de0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:32.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:32.870+0000 7f1637e00640 1 --2- 172.21.15.17:0/3534600463 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f16180726b0 0x7f1618074b70 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f1630005c50 tx=0x7f1630005be0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:45:32.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:32.878+0000 7f163e800640 1 --2- 172.21.15.17:0/3534600463 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f16381f8320 0x7f16381f8730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:32.880 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:32.878+0000 7f163e800640 1 -- 172.21.15.17:0/3534600463 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f16381f8c70 con 0x7f16381f8320 2024-09-10T21:45:32.885 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:32.882+0000 7f1637400640 1 --2- 172.21.15.17:0/3534600463 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f16381f8320 0x7f16381f8730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:32.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:32.882+0000 7f1637400640 1 -- 172.21.15.17:0/3534600463 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f163814ab00 con 0x7f16381f8320 2024-09-10T21:45:32.889 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:32.882+0000 7f1637400640 1 --2- 172.21.15.17:0/3534600463 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f16381f8320 0x7f16381f8730 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f1628000fc0 tx=0x7f1628000ee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:45:32.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:32.886+0000 7f162f400640 1 -- 172.21.15.17:0/3534600463 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1628003a30 con 0x7f16381f8320 2024-09-10T21:45:32.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:32.886+0000 7f162f400640 1 -- 172.21.15.17:0/3534600463 <== mon.2 v2:172.21.15.31:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4708 (secure 0 0 0) 0x7f1628003bd0 con 0x7f16381f8320 2024-09-10T21:45:32.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:32.886+0000 7f162f400640 1 -- 172.21.15.17:0/3534600463 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f16381f8320 msgr2=0x7f16381f8730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:32.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:32.886+0000 7f162f400640 1 --2- 172.21.15.17:0/3534600463 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f16381f8320 0x7f16381f8730 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f1628000fc0 tx=0x7f1628000ee0 comp rx=0 tx=0).stop 2024-09-10T21:45:32.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:32.886+0000 7f162f400640 1 -- 172.21.15.17:0/3534600463 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f16381f8320 msgr2=0x7f16381f8730 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:45:32.890 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:32.886+0000 7f162f400640 1 --2- 172.21.15.17:0/3534600463 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f16381f8320 0x7f16381f8730 secure :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f1628000fc0 tx=0x7f1628000ee0 comp rx=0 tx=0).stop 2024-09-10T21:45:32.890 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-10T21:45:32.890 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:45:32.891 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:45:32.891 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-10T21:45:32.891 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 160, 2024-09-10T21:45:32.891 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 11303, 2024-09-10T21:45:32.891 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-10T21:45:32.891 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-10T21:45:32.891 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-10T21:45:32.891 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-10T21:45:32.891 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:45:32.892 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 2, 2024-09-10T21:45:32.892 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:45:32.892 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-10T21:45:32.892 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-10T21:45:32.892 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:45:32.892 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:45:32.892 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:45:32.892 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:45:32.892 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:45:32.892 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:45:32.892 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:45:32.892 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:45:32.893 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:45:32.893 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:45:32.893 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:45:32.893 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:45:32.893 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-10T21:45:32.893 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-10T21:45:32.893 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:45:32.894 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:45:32.894 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:45:32.894 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:45:32.894 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:45:32.894 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:45:32.894 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:45:32.894 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:45:32.894 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:45:32.895 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:45:32.895 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:45:32.895 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:45:32.895 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:45:32.895 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-10T21:45:32.895 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-10T21:45:32.895 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-10T21:45:32.895 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-10T21:45:32.895 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-10T21:45:32.895 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "5e2f023f-d6a6-49f4-a256-f4807dce3739", 2024-09-10T21:45:32.896 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:45:32.896 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:45:32.896 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-10T21:45:32.896 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-10T21:45:32.896 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-10T21:45:32.896 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-10T21:45:32.896 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-10T21:45:32.896 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-10T21:45:32.896 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-10T21:45:32.896 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:45:32.896 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-10T21:45:32.896 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:45:32.897 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:45:32.897 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:45:32.897 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:45:32.897 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:45:32.897 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:45:32.897 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:45:32.897 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:45:32.897 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:45:32.897 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:45:32.897 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:45:32.897 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:45:32.897 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-10T21:45:32.897 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:45:32.898 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-10T21:45:32.898 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:45:32.898 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:45:32.898 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:45:32.898 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:45:32.898 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:45:32.898 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:45:32.898 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:45:32.898 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-10T21:45:32.898 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:45:32.898 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:45:32.898 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:45:32.898 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:45:32.898 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-10T21:45:32.899 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:45:32.899 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:45:32.899 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:45:32.899 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:45:32.899 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-10T21:45:32.899 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-10T21:45:32.899 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:45:32.899 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:45:32.899 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:45:32.899 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:45:32.899 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:45:32.899 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:45:32.899 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:45:32.899 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:45:32.900 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:45:32.900 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:45:32.900 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:45:32.900 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3300", 2024-09-10T21:45:32.900 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:45:32.900 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:45:32.900 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:45:32.900 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:45:32.900 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789", 2024-09-10T21:45:32.900 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:45:32.900 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:45:32.900 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:45:32.900 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:45:32.900 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789/0", 2024-09-10T21:45:32.901 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6789/0", 2024-09-10T21:45:32.901 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:45:32.901 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:45:32.901 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:45:32.901 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:45:32.901 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:45:32.901 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:45:32.901 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:45:32.901 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:45:32.901 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:45:32.901 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:45:32.901 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:45:32.901 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3301", 2024-09-10T21:45:32.902 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:45:32.902 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:45:32.902 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:45:32.902 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:45:32.902 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790", 2024-09-10T21:45:32.902 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:45:32.902 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:45:32.902 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:45:32.902 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:45:32.902 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790/0", 2024-09-10T21:45:32.903 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6790/0", 2024-09-10T21:45:32.903 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:45:32.903 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:45:32.903 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:45:32.903 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:45:32.903 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:45:32.903 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:45:32.903 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-10T21:45:32.903 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-10T21:45:32.903 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:45:32.903 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:45:32.904 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:45:32.904 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:45:32.904 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:45:32.904 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:45:32.904 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": [ 2024-09-10T21:45:32.904 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:45:32.904 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:45:32.904 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:45:32.904 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:45:32.904 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:45:32.904 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:45:32.904 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-10T21:45:32.904 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:45:32.905 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:45:32.905 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:45:32.905 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 2 2024-09-10T21:45:32.905 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:45:32.905 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:45:32.905 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:45:32.905 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-10T21:45:32.905 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-10T21:45:32.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:32.886+0000 7f163e800640 1 -- 172.21.15.17:0/3534600463 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f16180726b0 msgr2=0x7f1618074b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:32.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:32.886+0000 7f163e800640 1 --2- 172.21.15.17:0/3534600463 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f16180726b0 0x7f1618074b70 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f1630005c50 tx=0x7f1630005be0 comp rx=0 tx=0).stop 2024-09-10T21:45:32.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:32.886+0000 7f163e800640 1 -- 172.21.15.17:0/3534600463 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f16380c9160 msgr2=0x7f163814eb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:32.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:32.886+0000 7f163e800640 1 --2- 172.21.15.17:0/3534600463 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f16380c9160 0x7f163814eb10 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f162400d880 tx=0x7f16240056f0 comp rx=0 tx=0).stop 2024-09-10T21:45:32.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:32.886+0000 7f1637e00640 1 -- 172.21.15.17:0/3534600463 reap_dead start 2024-09-10T21:45:32.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:32.886+0000 7f1637e00640 1 -- 172.21.15.17:0/3534600463 reap_dead start 2024-09-10T21:45:32.906 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:32.886+0000 7f163e800640 1 -- 172.21.15.17:0/3534600463 shutdown_connections 2024-09-10T21:45:32.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:32.886+0000 7f163e800640 1 -- 172.21.15.17:0/3534600463 >> 172.21.15.17:0/3534600463 conn(0x7f16380700d0 msgr2=0x7f1638150070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:45:32.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:32.890+0000 7f163e800640 1 -- 172.21.15.17:0/3534600463 shutdown_connections 2024-09-10T21:45:32.907 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:32.890+0000 7f163e800640 1 -- 172.21.15.17:0/3534600463 wait complete. 2024-09-10T21:45:33.000 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-10T21:45:33.000 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-09-10T21:45:33.000 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-09-10T21:45:33.299 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:33.294+0000 7fb4c8a00640 1 -- 172.21.15.17:0/2751432747 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4b80a5970 con 0x7fb4b0003ae0 2024-09-10T21:45:33.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:33.294+0000 7fb4d0c00640 1 -- 172.21.15.17:0/2751432747 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb4b0003ae0 msgr2=0x7fb4b0003ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:33.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:33.294+0000 7fb4d0c00640 1 --2- 172.21.15.17:0/2751432747 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb4b0003ae0 0x7fb4b0003ec0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fb4b800a9e0 tx=0x7fb4b801c910 comp rx=0 tx=0).stop 2024-09-10T21:45:33.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:33.298+0000 7fb4ca800640 1 -- 172.21.15.17:0/2751432747 reap_dead start 2024-09-10T21:45:33.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:33.298+0000 7fb4d0c00640 1 -- 172.21.15.17:0/2751432747 shutdown_connections 2024-09-10T21:45:33.301 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:33.298+0000 7fb4d0c00640 1 -- 172.21.15.17:0/2751432747 >> 172.21.15.17:0/2751432747 conn(0x7fb4cc070160 msgr2=0x7fb4cc070570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:45:33.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:33.298+0000 7fb4d0c00640 1 -- 172.21.15.17:0/2751432747 shutdown_connections 2024-09-10T21:45:33.315 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:33.298+0000 7fb4d0c00640 1 -- 172.21.15.17:0/2751432747 wait complete. 2024-09-10T21:45:33.317 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:33.314+0000 7fb4d0c00640 1 Processor -- start 2024-09-10T21:45:33.318 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:33.314+0000 7fb4d0c00640 1 -- start start 2024-09-10T21:45:33.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:33.326+0000 7fb4d0c00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fb4b0003ae0 0x7fb4cc0ccb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:33.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:33.326+0000 7fb4d0c00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4cc1e4510 0x7fb4cc0c1680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:33.331 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:33.326+0000 7fb4d0c00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb4cc152750 0x7fb4cc1f4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:33.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:33.326+0000 7fb4d0c00640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fb4cc06e0a0 con 0x7fb4cc152750 2024-09-10T21:45:33.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:33.326+0000 7fb4d0c00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fb4cc15b4e0 con 0x7fb4cc1e4510 2024-09-10T21:45:33.332 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:33.326+0000 7fb4d0c00640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7fb4cc154c70 con 0x7fb4b0003ae0 2024-09-10T21:45:33.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:33.330+0000 7fb4c9400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4cc1e4510 0x7fb4cc0c1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:33.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:33.330+0000 7fb4c9400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4cc1e4510 0x7fb4cc0c1680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.17:37490/0 (socket says 172.21.15.17:37490) 2024-09-10T21:45:33.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:33.330+0000 7fb4c9400640 1 -- 172.21.15.17:0/4210829990 learned_addr learned my addr 172.21.15.17:0/4210829990 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:45:33.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:33.330+0000 7fb4c9400640 1 -- 172.21.15.17:0/4210829990 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fb4b0003ae0 msgr2=0x7fb4cc0ccb10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:45:33.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:33.330+0000 7fb4c9400640 1 --2- 172.21.15.17:0/4210829990 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fb4b0003ae0 0x7fb4cc0ccb10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:33.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:33.330+0000 7fb4c9400640 1 -- 172.21.15.17:0/4210829990 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb4cc152750 msgr2=0x7fb4cc1f4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:33.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:33.330+0000 7fb4c9400640 1 --2- 172.21.15.17:0/4210829990 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb4cc152750 0x7fb4cc1f4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:33.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:33.330+0000 7fb4c9400640 1 -- 172.21.15.17:0/4210829990 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4cc153030 con 0x7fb4cc1e4510 2024-09-10T21:45:33.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:33.330+0000 7fb4c9400640 1 --2- 172.21.15.17:0/4210829990 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4cc1e4510 0x7fb4cc0c1680 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fb4bc00e8f0 tx=0x7fb4bc00edb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:45:33.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:33.330+0000 7fb4c2000640 1 -- 172.21.15.17:0/4210829990 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4bc00cc40 con 0x7fb4cc1e4510 2024-09-10T21:45:33.336 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:33.330+0000 7fb4c2000640 1 -- 172.21.15.17:0/4210829990 <== mon.1 v2:172.21.15.31:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb4bc00ce00 con 0x7fb4cc1e4510 2024-09-10T21:45:33.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:33.330+0000 7fb4c2000640 1 -- 172.21.15.17:0/4210829990 <== mon.1 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4bc004290 con 0x7fb4cc1e4510 2024-09-10T21:45:33.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:33.330+0000 7fb4d0c00640 1 -- 172.21.15.17:0/4210829990 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4cc0cac00 con 0x7fb4cc1e4510 2024-09-10T21:45:33.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:33.330+0000 7fb4d0c00640 1 -- 172.21.15.17:0/4210829990 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4cc0cb1c0 con 0x7fb4cc1e4510 2024-09-10T21:45:33.337 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:33.334+0000 7fb4d0c00640 1 -- 172.21.15.17:0/4210829990 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4cc0c8780 con 0x7fb4cc1e4510 2024-09-10T21:45:33.339 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:33.334+0000 7fb4c2000640 1 -- 172.21.15.17:0/4210829990 <== mon.1 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fb4bc03a950 con 0x7fb4cc1e4510 2024-09-10T21:45:33.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:33.338+0000 7fb4c2000640 1 --2- 172.21.15.17:0/4210829990 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fb4ac072610 0x7fb4ac074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:33.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:33.338+0000 7fb4c2000640 1 -- 172.21.15.17:0/4210829990 <== mon.1 v2:172.21.15.31:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fb4bc0940d0 con 0x7fb4cc1e4510 2024-09-10T21:45:33.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:33.342+0000 7fb4c2000640 1 -- 172.21.15.17:0/4210829990 <== mon.1 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fb4bc05f390 con 0x7fb4cc1e4510 2024-09-10T21:45:33.347 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:33.342+0000 7fb4c9e00640 1 --2- 172.21.15.17:0/4210829990 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fb4ac072610 0x7fb4ac074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:33.354 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:33.350+0000 7fb4c9e00640 1 --2- 172.21.15.17:0/4210829990 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fb4ac072610 0x7fb4ac074ad0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fb4b800a9b0 tx=0x7fb4b80196b0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:45:33.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:33.990+0000 7fb4d0c00640 1 -- 172.21.15.17:0/4210829990 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) -- 0x7fb4cc0b1b80 con 0x7fb4cc1e4510 2024-09-10T21:45:34.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:34.002+0000 7fb4c2000640 1 -- 172.21.15.17:0/4210829990 <== mon.1 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) ==== 57+0+0 (secure 0 0 0) 0x7fb4bc05ec90 con 0x7fb4cc1e4510 2024-09-10T21:45:34.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:34.006+0000 7fb4d0c00640 1 -- 172.21.15.17:0/4210829990 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fb4ac072610 msgr2=0x7fb4ac074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:34.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:34.006+0000 7fb4d0c00640 1 --2- 172.21.15.17:0/4210829990 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fb4ac072610 0x7fb4ac074ad0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fb4b800a9b0 tx=0x7fb4b80196b0 comp rx=0 tx=0).stop 2024-09-10T21:45:34.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:34.006+0000 7fb4d0c00640 1 -- 172.21.15.17:0/4210829990 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4cc1e4510 msgr2=0x7fb4cc0c1680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:34.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:34.006+0000 7fb4d0c00640 1 --2- 172.21.15.17:0/4210829990 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4cc1e4510 0x7fb4cc0c1680 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fb4bc00e8f0 tx=0x7fb4bc00edb0 comp rx=0 tx=0).stop 2024-09-10T21:45:34.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:34.014+0000 7fb4d0c00640 1 -- 172.21.15.17:0/4210829990 shutdown_connections 2024-09-10T21:45:34.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:34.014+0000 7fb4d0c00640 1 --2- 172.21.15.17:0/4210829990 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fb4ac072610 0x7fb4ac074ad0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:34.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:34.014+0000 7fb4d0c00640 1 --2- 172.21.15.17:0/4210829990 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb4cc152750 0x7fb4cc1f4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:34.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:34.014+0000 7fb4d0c00640 1 --2- 172.21.15.17:0/4210829990 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb4cc1e4510 0x7fb4cc0c1680 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:34.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:34.014+0000 7fb4d0c00640 1 --2- 172.21.15.17:0/4210829990 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fb4b0003ae0 0x7fb4cc0ccb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:34.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:34.014+0000 7fb4d0c00640 1 -- 172.21.15.17:0/4210829990 >> 172.21.15.17:0/4210829990 conn(0x7fb4cc070160 msgr2=0x7fb4cc070570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:45:34.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:34.018+0000 7fb4d0c00640 1 -- 172.21.15.17:0/4210829990 shutdown_connections 2024-09-10T21:45:34.022 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:34.018+0000 7fb4d0c00640 1 -- 172.21.15.17:0/4210829990 wait complete. 2024-09-10T21:45:34.064 INFO:tasks.mon_thrash.mon_thrasher:waiting for 10.0 secs before continuing thrashing 2024-09-10T21:45:44.065 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-09-10T21:45:44.066 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-10T21:45:44.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:44.358+0000 7f5dc9c00640 1 -- 172.21.15.17:0/4074490642 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5da8004810 msgr2=0x7f5da8004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:44.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:44.358+0000 7f5dc9c00640 1 --2- 172.21.15.17:0/4074490642 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5da8004810 0x7f5da8004c90 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f5db400bbc0 tx=0x7f5db401cde0 comp rx=0 tx=0).stop 2024-09-10T21:45:44.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:44.362+0000 7f5dc9c00640 1 -- 172.21.15.17:0/4074490642 shutdown_connections 2024-09-10T21:45:44.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:44.362+0000 7f5dc9c00640 1 --2- 172.21.15.17:0/4074490642 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5da8004810 0x7f5da8004c90 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:44.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:44.362+0000 7f5dc9c00640 1 -- 172.21.15.17:0/4074490642 >> 172.21.15.17:0/4074490642 conn(0x7f5dc40702f0 msgr2=0x7f5dc4070700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:45:44.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:44.362+0000 7f5dc9c00640 1 -- 172.21.15.17:0/4074490642 shutdown_connections 2024-09-10T21:45:44.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:44.362+0000 7f5dc9c00640 1 -- 172.21.15.17:0/4074490642 wait complete. 2024-09-10T21:45:44.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:44.362+0000 7f5dc9c00640 1 Processor -- start 2024-09-10T21:45:44.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:44.362+0000 7f5dc9c00640 1 -- start start 2024-09-10T21:45:44.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:44.366+0000 7f5dc9c00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5da8004810 0x7f5dc41f8850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:44.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:44.366+0000 7f5dc9c00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f5dc41e4cf0 0x7f5dc41e4440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:44.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:44.366+0000 7f5dc9c00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5dc41e51d0 0x7f5dc40c7a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:44.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:44.366+0000 7f5dc9c00640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f5dc415af50 con 0x7f5dc41e51d0 2024-09-10T21:45:44.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:44.366+0000 7f5dc9c00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f5dc415ac70 con 0x7f5da8004810 2024-09-10T21:45:44.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:44.366+0000 7f5dc9c00640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f5dc40c9640 con 0x7f5dc41e4cf0 2024-09-10T21:45:44.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:44.366+0000 7f5dc2a00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5da8004810 0x7f5dc41f8850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:44.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:44.366+0000 7f5dc2a00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5da8004810 0x7f5dc41f8850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.17:41804/0 (socket says 172.21.15.17:41804) 2024-09-10T21:45:44.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:44.366+0000 7f5dc2a00640 1 -- 172.21.15.17:0/2136698883 learned_addr learned my addr 172.21.15.17:0/2136698883 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:45:44.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:44.366+0000 7f5dc2000640 1 --2- 172.21.15.17:0/2136698883 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f5dc41e4cf0 0x7f5dc41e4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:44.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:44.366+0000 7f5dc2a00640 1 -- 172.21.15.17:0/2136698883 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f5dc41e4cf0 msgr2=0x7f5dc41e4440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:44.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:44.366+0000 7f5dc2a00640 1 --2- 172.21.15.17:0/2136698883 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f5dc41e4cf0 0x7f5dc41e4440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:44.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:44.366+0000 7f5dc2a00640 1 -- 172.21.15.17:0/2136698883 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5dc41e51d0 msgr2=0x7f5dc40c7a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:44.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:44.366+0000 7f5dc2a00640 1 --2- 172.21.15.17:0/2136698883 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5dc41e51d0 0x7f5dc40c7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:44.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:44.366+0000 7f5dc2a00640 1 -- 172.21.15.17:0/2136698883 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5dc41e3a50 con 0x7f5da8004810 2024-09-10T21:45:44.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:44.366+0000 7f5dc2a00640 1 --2- 172.21.15.17:0/2136698883 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5da8004810 0x7f5dc41f8850 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f5dac00c730 tx=0x7f5dac00cbf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:45:44.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:44.366+0000 7f5dbaa00640 1 -- 172.21.15.17:0/2136698883 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dac005480 con 0x7f5da8004810 2024-09-10T21:45:44.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:44.366+0000 7f5dbaa00640 1 -- 172.21.15.17:0/2136698883 <== mon.1 v2:172.21.15.31:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5dac00ce40 con 0x7f5da8004810 2024-09-10T21:45:44.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:44.366+0000 7f5dbaa00640 1 -- 172.21.15.17:0/2136698883 <== mon.1 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dac003840 con 0x7f5da8004810 2024-09-10T21:45:44.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:44.370+0000 7f5dc9c00640 1 -- 172.21.15.17:0/2136698883 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5dc406e640 con 0x7f5da8004810 2024-09-10T21:45:44.372 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:44.370+0000 7f5dc9c00640 1 -- 172.21.15.17:0/2136698883 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5dc406ec00 con 0x7f5da8004810 2024-09-10T21:45:44.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:44.370+0000 7f5dc9c00640 1 -- 172.21.15.17:0/2136698883 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5dc406f360 con 0x7f5da8004810 2024-09-10T21:45:44.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:44.374+0000 7f5dbaa00640 1 -- 172.21.15.17:0/2136698883 <== mon.1 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f5dac0039e0 con 0x7f5da8004810 2024-09-10T21:45:44.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:44.374+0000 7f5dbaa00640 1 --2- 172.21.15.17:0/2136698883 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f5da00725e0 0x7f5da0074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:44.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:44.374+0000 7f5dbaa00640 1 -- 172.21.15.17:0/2136698883 <== mon.1 v2:172.21.15.31:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f5dac093220 con 0x7f5da8004810 2024-09-10T21:45:44.381 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:44.378+0000 7f5dc2000640 1 --2- 172.21.15.17:0/2136698883 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f5da00725e0 0x7f5da0074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:44.381 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:44.378+0000 7f5dc2000640 1 --2- 172.21.15.17:0/2136698883 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f5da00725e0 0x7f5da0074aa0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f5db40af000 tx=0x7f5db40b1040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:45:44.388 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:44.386+0000 7f5dbaa00640 1 -- 172.21.15.17:0/2136698883 <== mon.1 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f5dac05e590 con 0x7f5da8004810 2024-09-10T21:45:45.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:45.066+0000 7f5dc9c00640 1 -- 172.21.15.17:0/2136698883 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f5dc4070aa0 con 0x7f5da8004810 2024-09-10T21:45:45.115 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:45.102+0000 7f5dbaa00640 1 -- 172.21.15.17:0/2136698883 <== mon.1 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1558 (secure 0 0 0) 0x7f5dac05de90 con 0x7f5da8004810 2024-09-10T21:45:45.141 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":160,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":14,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5e2f023f-d6a6-49f4-a256-f4807dce3739","modified":"2024-09-10T21:23:46.892130Z","created":"2024-09-10T21:23:46.892130Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3301","nonce":0},{"type":"v1","addr":"172.21.15.31:6790","nonce":0}]},"addr":"172.21.15.31:6790/0","public_addr":"172.21.15.31:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-10T21:45:45.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:45.146+0000 7f5dc9c00640 1 -- 172.21.15.17:0/2136698883 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f5da00725e0 msgr2=0x7f5da0074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:45.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:45.146+0000 7f5dc9c00640 1 --2- 172.21.15.17:0/2136698883 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f5da00725e0 0x7f5da0074aa0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f5db40af000 tx=0x7f5db40b1040 comp rx=0 tx=0).stop 2024-09-10T21:45:45.152 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:45.146+0000 7f5dc9c00640 1 -- 172.21.15.17:0/2136698883 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5da8004810 msgr2=0x7f5dc41f8850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:45.153 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:45.146+0000 7f5dc9c00640 1 --2- 172.21.15.17:0/2136698883 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5da8004810 0x7f5dc41f8850 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f5dac00c730 tx=0x7f5dac00cbf0 comp rx=0 tx=0).stop 2024-09-10T21:45:45.156 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:45.150+0000 7f5dc9c00640 1 -- 172.21.15.17:0/2136698883 shutdown_connections 2024-09-10T21:45:45.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:45.150+0000 7f5dc9c00640 1 --2- 172.21.15.17:0/2136698883 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f5da00725e0 0x7f5da0074aa0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:45.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:45.150+0000 7f5dc9c00640 1 --2- 172.21.15.17:0/2136698883 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5dc41e51d0 0x7f5dc40c7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:45.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:45.150+0000 7f5dc9c00640 1 --2- 172.21.15.17:0/2136698883 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f5dc41e4cf0 0x7f5dc41e4440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:45.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:45.150+0000 7f5dc9c00640 1 --2- 172.21.15.17:0/2136698883 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5da8004810 0x7f5dc41f8850 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:45.157 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:45.150+0000 7f5dc9c00640 1 -- 172.21.15.17:0/2136698883 >> 172.21.15.17:0/2136698883 conn(0x7f5dc40702f0 msgr2=0x7f5dc40cdee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:45:45.158 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:45.154+0000 7f5dc9c00640 1 -- 172.21.15.17:0/2136698883 shutdown_connections 2024-09-10T21:45:45.161 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:45.158+0000 7f5dc9c00640 1 -- 172.21.15.17:0/2136698883 wait complete. 2024-09-10T21:45:45.255 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-09-10T21:45:45.255 INFO:tasks.mon_thrash.mon_thrasher:making sure all monitors are in the quorum 2024-09-10T21:45:45.255 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-09-10T21:45:45.255 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-10T21:45:45.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:45.562+0000 7fe137000640 1 -- 172.21.15.17:0/3305393422 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe1300cf4c0 msgr2=0x7fe1301691c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:45.568 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:45.562+0000 7fe137000640 1 --2- 172.21.15.17:0/3305393422 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe1300cf4c0 0x7fe1301691c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fe128009c80 tx=0x7fe12801cd30 comp rx=0 tx=0).stop 2024-09-10T21:45:45.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:45.566+0000 7fe137000640 1 -- 172.21.15.17:0/3305393422 shutdown_connections 2024-09-10T21:45:45.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:45.566+0000 7fe137000640 1 --2- 172.21.15.17:0/3305393422 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe1300cf4c0 0x7fe1301691c0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:45.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:45.566+0000 7fe137000640 1 --2- 172.21.15.17:0/3305393422 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe1300d0020 0x7fe1300c9bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:45.571 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:45.566+0000 7fe137000640 1 -- 172.21.15.17:0/3305393422 >> 172.21.15.17:0/3305393422 conn(0x7fe1300d4630 msgr2=0x7fe1300d4a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:45:45.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:45.570+0000 7fe137000640 1 -- 172.21.15.17:0/3305393422 shutdown_connections 2024-09-10T21:45:45.575 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:45.570+0000 7fe137000640 1 -- 172.21.15.17:0/3305393422 wait complete. 2024-09-10T21:45:45.575 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:45.570+0000 7fe137000640 1 Processor -- start 2024-09-10T21:45:45.578 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:45.574+0000 7fe137000640 1 -- start start 2024-09-10T21:45:45.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:45.586+0000 7fe137000640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe1300cf4c0 0x7fe1300cd970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:45.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:45.586+0000 7fe137000640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe1300d0020 0x7fe1300cac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:45.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:45.586+0000 7fe135c00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe1300cf4c0 0x7fe1300cd970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:45.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:45.586+0000 7fe135c00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe1300cf4c0 0x7fe1300cd970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3301/0 says I am v2:172.21.15.17:58400/0 (socket says 172.21.15.17:58400) 2024-09-10T21:45:45.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:45.586+0000 7fe135c00640 1 -- 172.21.15.17:0/3449051154 learned_addr learned my addr 172.21.15.17:0/3449051154 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:45:45.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:45.586+0000 7fe135200640 1 --2- 172.21.15.17:0/3449051154 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe1300d0020 0x7fe1300cac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:45.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:45.586+0000 7fe137000640 1 --2- 172.21.15.17:0/3449051154 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe130158d80 0x7fe1300cb7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:45.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:45.586+0000 7fe137000640 1 -- 172.21.15.17:0/3449051154 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fe13006d4b0 con 0x7fe130158d80 2024-09-10T21:45:45.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:45.586+0000 7fe137000640 1 -- 172.21.15.17:0/3449051154 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fe13006d330 con 0x7fe1300d0020 2024-09-10T21:45:45.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:45.586+0000 7fe137000640 1 -- 172.21.15.17:0/3449051154 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7fe1300d0b20 con 0x7fe1300cf4c0 2024-09-10T21:45:45.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:45.586+0000 7fe136600640 1 --2- 172.21.15.17:0/3449051154 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe130158d80 0x7fe1300cb7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:45.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:45.590+0000 7fe135200640 1 -- 172.21.15.17:0/3449051154 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe1300cf4c0 msgr2=0x7fe1300cd970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:45.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:45.590+0000 7fe135200640 1 --2- 172.21.15.17:0/3449051154 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe1300cf4c0 0x7fe1300cd970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:45.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:45.590+0000 7fe135200640 1 -- 172.21.15.17:0/3449051154 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe130158d80 msgr2=0x7fe1300cb7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:45.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:45.590+0000 7fe135200640 1 --2- 172.21.15.17:0/3449051154 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe130158d80 0x7fe1300cb7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:45.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:45.590+0000 7fe135200640 1 -- 172.21.15.17:0/3449051154 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1302012e0 con 0x7fe1300d0020 2024-09-10T21:45:45.595 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:45.590+0000 7fe135200640 1 --2- 172.21.15.17:0/3449051154 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe1300d0020 0x7fe1300cac30 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fe12000b440 tx=0x7fe12000b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:45:45.602 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:45.590+0000 7fe12d600640 1 -- 172.21.15.17:0/3449051154 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe120015020 con 0x7fe1300d0020 2024-09-10T21:45:45.602 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:45.590+0000 7fe12d600640 1 -- 172.21.15.17:0/3449051154 <== mon.1 v2:172.21.15.31:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe120011920 con 0x7fe1300d0020 2024-09-10T21:45:45.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:45.598+0000 7fe12d600640 1 -- 172.21.15.17:0/3449051154 <== mon.1 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe120011d70 con 0x7fe1300d0020 2024-09-10T21:45:45.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:45.602+0000 7fe137000640 1 -- 172.21.15.17:0/3449051154 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1302015a0 con 0x7fe1300d0020 2024-09-10T21:45:45.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:45.606+0000 7fe137000640 1 -- 172.21.15.17:0/3449051154 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe13015c530 con 0x7fe1300d0020 2024-09-10T21:45:45.609 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:45.606+0000 7fe137000640 1 --2- 172.21.15.17:0/3449051154 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe1301532f0 0x7fe1300ca260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:45.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:45.606+0000 7fe137000640 1 -- 172.21.15.17:0/3449051154 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fe1300c9bc0 con 0x7fe1301532f0 2024-09-10T21:45:45.610 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:45.606+0000 7fe135c00640 1 --2- 172.21.15.17:0/3449051154 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe1301532f0 0x7fe1300ca260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:45.644 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:45.634+0000 7fe135c00640 1 -- 172.21.15.17:0/3449051154 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe1300c9790 con 0x7fe1301532f0 2024-09-10T21:45:45.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:45.642+0000 7fe135c00640 1 --2- 172.21.15.17:0/3449051154 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe1301532f0 0x7fe1300ca260 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fe124007fd0 tx=0x7fe12400fc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:45:45.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:45.642+0000 7fe12d600640 1 -- 172.21.15.17:0/3449051154 <== mon.1 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fe12001a400 con 0x7fe1300d0020 2024-09-10T21:45:45.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:45.646+0000 7fe12d600640 1 --2- 172.21.15.17:0/3449051154 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fe114072660 0x7fe114074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:45.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:45.646+0000 7fe12d600640 1 -- 172.21.15.17:0/3449051154 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1240050d0 con 0x7fe1301532f0 2024-09-10T21:45:45.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:45.646+0000 7fe12d600640 1 -- 172.21.15.17:0/3449051154 <== mon.1 v2:172.21.15.31:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fe1200936c0 con 0x7fe1300d0020 2024-09-10T21:45:45.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:45.646+0000 7fe12d600640 1 -- 172.21.15.17:0/3449051154 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fe124018070 con 0x7fe1301532f0 2024-09-10T21:45:45.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:45.646+0000 7fe12d600640 1 -- 172.21.15.17:0/3449051154 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe1301532f0 msgr2=0x7fe1300ca260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:45.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:45.646+0000 7fe12d600640 1 --2- 172.21.15.17:0/3449051154 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe1301532f0 0x7fe1300ca260 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fe124007fd0 tx=0x7fe12400fc00 comp rx=0 tx=0).stop 2024-09-10T21:45:45.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:45.646+0000 7fe12d600640 1 -- 172.21.15.17:0/3449051154 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe1301532f0 msgr2=0x7fe1300ca260 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:45:45.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:45.646+0000 7fe12d600640 1 --2- 172.21.15.17:0/3449051154 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe1301532f0 0x7fe1300ca260 secure :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fe124007fd0 tx=0x7fe12400fc00 comp rx=0 tx=0).stop 2024-09-10T21:45:45.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:45.646+0000 7fe136600640 1 --2- 172.21.15.17:0/3449051154 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fe114072660 0x7fe114074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:45.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:45.650+0000 7fe136600640 1 --2- 172.21.15.17:0/3449051154 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fe114072660 0x7fe114074b20 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fe1280af000 tx=0x7fe12801b570 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:45:45.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:45.674+0000 7fe137000640 1 --2- 172.21.15.17:0/3449051154 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe130095370 0x7fe1300cbd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:45.679 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:45.674+0000 7fe137000640 1 -- 172.21.15.17:0/3449051154 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fe1300cfe00 con 0x7fe130095370 2024-09-10T21:45:45.683 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-10T21:45:45.683 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:45:45.683 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:45:45.683 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "leader", 2024-09-10T21:45:45.683 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 160, 2024-09-10T21:45:45.684 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 204810, 2024-09-10T21:45:45.684 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-10T21:45:45.684 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-10T21:45:45.684 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-10T21:45:45.684 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-10T21:45:45.684 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:45:45.684 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 14, 2024-09-10T21:45:45.684 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:45:45.684 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-10T21:45:45.684 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-10T21:45:45.684 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:45:45.685 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:45:45.685 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:45:45.685 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:45:45.685 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:45:45.685 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:45:45.685 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:45:45.685 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:45:45.685 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:45:45.685 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:45:45.686 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:45:45.686 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:45:45.686 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-10T21:45:45.686 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-10T21:45:45.686 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:45:45.686 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:45:45.686 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:45:45.686 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:45:45.686 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:45:45.687 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:45:45.687 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:45:45.687 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:45:45.687 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:45:45.687 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:45:45.687 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:45:45.687 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:45:45.687 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:45:45.687 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-10T21:45:45.687 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-10T21:45:45.687 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-10T21:45:45.687 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-10T21:45:45.687 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-10T21:45:45.688 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "5e2f023f-d6a6-49f4-a256-f4807dce3739", 2024-09-10T21:45:45.688 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:45:45.688 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:45:45.688 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-10T21:45:45.688 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-10T21:45:45.688 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-10T21:45:45.688 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-10T21:45:45.688 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-10T21:45:45.688 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-10T21:45:45.688 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-10T21:45:45.688 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:45:45.688 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-10T21:45:45.688 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:45:45.689 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:45:45.689 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:45:45.689 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:45:45.689 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:45:45.689 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:45:45.689 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:45:45.689 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:45:45.689 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:45:45.689 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:45:45.689 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:45:45.689 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:45:45.689 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-10T21:45:45.689 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:45:45.690 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-10T21:45:45.690 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:45:45.690 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:45:45.690 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:45:45.690 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:45:45.690 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:45:45.690 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:45:45.690 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:45:45.690 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-10T21:45:45.690 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:45:45.690 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:45:45.690 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:45:45.690 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:45:45.691 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-10T21:45:45.691 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:45:45.691 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:45:45.691 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:45:45.691 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:45:45.691 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-10T21:45:45.691 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-10T21:45:45.691 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:45:45.691 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:45:45.691 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:45:45.691 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:45:45.691 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:45:45.692 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:45:45.692 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:45:45.692 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:45:45.692 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:45:45.692 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:45:45.692 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:45:45.692 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3300", 2024-09-10T21:45:45.692 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:45:45.692 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:45:45.692 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:45:45.692 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:45:45.692 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789", 2024-09-10T21:45:45.692 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:45:45.692 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:45:45.693 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:45:45.693 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:45:45.693 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789/0", 2024-09-10T21:45:45.693 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6789/0", 2024-09-10T21:45:45.693 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:45:45.693 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:45:45.693 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:45:45.693 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:45:45.693 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:45:45.693 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:45:45.693 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:45:45.693 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:45:45.694 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:45:45.694 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:45:45.695 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:45:45.695 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3301", 2024-09-10T21:45:45.695 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:45:45.695 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:45:45.695 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:45:45.695 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:45:45.696 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790", 2024-09-10T21:45:45.696 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:45:45.696 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:45:45.696 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:45:45.696 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:45:45.696 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790/0", 2024-09-10T21:45:45.696 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6790/0", 2024-09-10T21:45:45.696 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:45:45.696 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:45:45.696 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:45:45.696 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:45:45.696 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:45:45.697 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:45:45.697 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-10T21:45:45.697 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-10T21:45:45.697 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:45:45.697 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:45:45.697 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:45:45.697 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:45:45.697 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:45:45.697 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:45:45.697 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": [ 2024-09-10T21:45:45.698 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:45:45.698 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:45:45.698 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:45:45.698 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 3 2024-09-10T21:45:45.698 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:45:45.698 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:45:45.698 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-10T21:45:45.698 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:45:45.698 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:45:45.698 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:45:45.698 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 7 2024-09-10T21:45:45.698 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:45:45.698 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:45:45.699 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-10T21:45:45.699 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:45:45.699 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:45:45.699 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:45:45.699 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 4 2024-09-10T21:45:45.699 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:45:45.699 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:45:45.699 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": [ 2024-09-10T21:45:45.699 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:45:45.699 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:45:45.699 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:45:45.699 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:45:45.699 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:45:45.700 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:45:45.700 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:45:45.700 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-10T21:45:45.700 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-10T21:45:45.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:45.678+0000 7fe135c00640 1 --2- 172.21.15.17:0/3449051154 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe130095370 0x7fe1300cbd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:45.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:45.678+0000 7fe135c00640 1 -- 172.21.15.17:0/3449051154 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fe130050ef0 con 0x7fe130095370 2024-09-10T21:45:45.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:45.678+0000 7fe135c00640 1 --2- 172.21.15.17:0/3449051154 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe130095370 0x7fe1300cbd30 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fe124007fd0 tx=0x7fe124001090 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:45:45.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:45.678+0000 7fe12d600640 1 -- 172.21.15.17:0/3449051154 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe12400d750 con 0x7fe130095370 2024-09-10T21:45:45.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:45.678+0000 7fe12d600640 1 -- 172.21.15.17:0/3449051154 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5046 (secure 0 0 0) 0x7fe12400d750 con 0x7fe130095370 2024-09-10T21:45:45.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:45.678+0000 7fe12d600640 1 -- 172.21.15.17:0/3449051154 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe130095370 msgr2=0x7fe1300cbd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:45.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:45.678+0000 7fe12d600640 1 --2- 172.21.15.17:0/3449051154 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe130095370 0x7fe1300cbd30 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fe124007fd0 tx=0x7fe124001090 comp rx=0 tx=0).stop 2024-09-10T21:45:45.701 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:45.678+0000 7fe12d600640 1 -- 172.21.15.17:0/3449051154 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe130095370 msgr2=0x7fe1300cbd30 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:45:45.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:45.678+0000 7fe12d600640 1 --2- 172.21.15.17:0/3449051154 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe130095370 0x7fe1300cbd30 secure :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fe124007fd0 tx=0x7fe124001090 comp rx=0 tx=0).stop 2024-09-10T21:45:45.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:45.682+0000 7fe137000640 1 -- 172.21.15.17:0/3449051154 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fe114072660 msgr2=0x7fe114074b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:45.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:45.682+0000 7fe137000640 1 --2- 172.21.15.17:0/3449051154 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fe114072660 0x7fe114074b20 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fe1280af000 tx=0x7fe12801b570 comp rx=0 tx=0).stop 2024-09-10T21:45:45.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:45.682+0000 7fe137000640 1 -- 172.21.15.17:0/3449051154 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe1300d0020 msgr2=0x7fe1300cac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:45.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:45.682+0000 7fe137000640 1 --2- 172.21.15.17:0/3449051154 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe1300d0020 0x7fe1300cac30 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fe12000b440 tx=0x7fe12000b840 comp rx=0 tx=0).stop 2024-09-10T21:45:45.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:45.682+0000 7fe136600640 1 -- 172.21.15.17:0/3449051154 reap_dead start 2024-09-10T21:45:45.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:45.682+0000 7fe136600640 1 -- 172.21.15.17:0/3449051154 reap_dead start 2024-09-10T21:45:45.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:45.682+0000 7fe137000640 1 -- 172.21.15.17:0/3449051154 shutdown_connections 2024-09-10T21:45:45.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:45.682+0000 7fe137000640 1 -- 172.21.15.17:0/3449051154 >> 172.21.15.17:0/3449051154 conn(0x7fe1300d4630 msgr2=0x7fe130152be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:45:45.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:45.682+0000 7fe137000640 1 -- 172.21.15.17:0/3449051154 shutdown_connections 2024-09-10T21:45:45.702 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:45.682+0000 7fe137000640 1 -- 172.21.15.17:0/3449051154 wait complete. 2024-09-10T21:45:45.763 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-10T21:45:45.763 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-09-10T21:45:45.764 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-10T21:45:46.182 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:46.178+0000 7f9eef400640 1 -- 172.21.15.17:0/2707704560 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ee806e7a0 msgr2=0x7f9ee806e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:46.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:46.178+0000 7f9eef400640 1 --2- 172.21.15.17:0/2707704560 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ee806e7a0 0x7f9ee806e010 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f9ed40097c0 tx=0x7f9ed401c680 comp rx=0 tx=0).stop 2024-09-10T21:45:46.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:46.178+0000 7f9eef400640 1 -- 172.21.15.17:0/2707704560 shutdown_connections 2024-09-10T21:45:46.183 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:46.178+0000 7f9eef400640 1 --2- 172.21.15.17:0/2707704560 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ee806e7a0 0x7f9ee806e010 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:46.184 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:46.178+0000 7f9eef400640 1 -- 172.21.15.17:0/2707704560 >> 172.21.15.17:0/2707704560 conn(0x7f9ee80700d0 msgr2=0x7f9ee80704e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:45:46.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:46.190+0000 7f9eef400640 1 -- 172.21.15.17:0/2707704560 shutdown_connections 2024-09-10T21:45:46.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:46.190+0000 7f9eef400640 1 -- 172.21.15.17:0/2707704560 wait complete. 2024-09-10T21:45:46.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:46.190+0000 7f9eef400640 1 Processor -- start 2024-09-10T21:45:46.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:46.194+0000 7f9eef400640 1 -- start start 2024-09-10T21:45:46.201 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:46.198+0000 7f9eef400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9ee806dc30 0x7f9ee80c53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:46.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:46.198+0000 7f9eef400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ee806e7a0 0x7f9ee8200bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:46.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:46.198+0000 7f9eeca00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9ee806dc30 0x7f9ee80c53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:46.202 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:46.198+0000 7f9eeca00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9ee806dc30 0x7f9ee80c53d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:53204/0 (socket says 172.21.15.17:53204) 2024-09-10T21:45:46.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:46.198+0000 7f9eef400640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f9ee8154400 0x7f9ee80c3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:46.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:46.198+0000 7f9eef400640 1 -- 172.21.15.17:0/2482710443 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f9ee80c8870 con 0x7f9ee806dc30 2024-09-10T21:45:46.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:46.198+0000 7f9eef400640 1 -- 172.21.15.17:0/2482710443 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f9ee815b4d0 con 0x7f9ee806e7a0 2024-09-10T21:45:46.203 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:46.198+0000 7f9eef400640 1 -- 172.21.15.17:0/2482710443 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f9ee80c8500 con 0x7f9ee8154400 2024-09-10T21:45:46.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:46.198+0000 7f9eeca00640 1 -- 172.21.15.17:0/2482710443 learned_addr learned my addr 172.21.15.17:0/2482710443 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:45:46.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:46.198+0000 7f9eed400640 1 --2- 172.21.15.17:0/2482710443 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f9ee8154400 0x7f9ee80c3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:46.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:46.202+0000 7f9eed400640 1 -- 172.21.15.17:0/2482710443 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ee806e7a0 msgr2=0x7f9ee8200bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:46.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:46.202+0000 7f9eed400640 1 --2- 172.21.15.17:0/2482710443 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ee806e7a0 0x7f9ee8200bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:46.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:46.202+0000 7f9eed400640 1 -- 172.21.15.17:0/2482710443 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9ee806dc30 msgr2=0x7f9ee80c53d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:46.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:46.202+0000 7f9eed400640 1 --2- 172.21.15.17:0/2482710443 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f9ee806dc30 0x7f9ee80c53d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:46.204 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:46.202+0000 7f9eed400640 1 -- 172.21.15.17:0/2482710443 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ee82015a0 con 0x7f9ee8154400 2024-09-10T21:45:46.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:46.202+0000 7f9eed400640 1 --2- 172.21.15.17:0/2482710443 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f9ee8154400 0x7f9ee80c3fc0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f9ed800c730 tx=0x7f9ed800cbf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:45:46.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:46.210+0000 7f9ee4c00640 1 -- 172.21.15.17:0/2482710443 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ed8017070 con 0x7f9ee8154400 2024-09-10T21:45:46.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:46.210+0000 7f9ee4c00640 1 -- 172.21.15.17:0/2482710443 <== mon.2 v2:172.21.15.31:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9ed8005380 con 0x7f9ee8154400 2024-09-10T21:45:46.218 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:46.210+0000 7f9ee4c00640 1 -- 172.21.15.17:0/2482710443 <== mon.2 v2:172.21.15.31:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ed800a8a0 con 0x7f9ee8154400 2024-09-10T21:45:46.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:46.214+0000 7f9eef400640 1 -- 172.21.15.17:0/2482710443 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ee80c34e0 con 0x7f9ee8154400 2024-09-10T21:45:46.219 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:46.214+0000 7f9eef400640 1 -- 172.21.15.17:0/2482710443 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ee806a210 con 0x7f9ee8154400 2024-09-10T21:45:46.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:46.222+0000 7f9ee4c00640 1 -- 172.21.15.17:0/2482710443 <== mon.2 v2:172.21.15.31:3301/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f9ed800aa40 con 0x7f9ee8154400 2024-09-10T21:45:46.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:46.222+0000 7f9ee4c00640 1 --2- 172.21.15.17:0/2482710443 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f9ec80726e0 0x7f9ec8074ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:46.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:46.222+0000 7f9ee4c00640 1 -- 172.21.15.17:0/2482710443 <== mon.2 v2:172.21.15.31:3301/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f9ed8092f40 con 0x7f9ee8154400 2024-09-10T21:45:46.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:46.226+0000 7f9eef400640 1 --2- 172.21.15.17:0/2482710443 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ee80c3b30 0x7f9ee80c4500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:46.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:46.226+0000 7f9eef400640 1 -- 172.21.15.17:0/2482710443 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f9ee80c5910 con 0x7f9ee80c3b30 2024-09-10T21:45:46.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:46.226+0000 7f9eeca00640 1 --2- 172.21.15.17:0/2482710443 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f9ec80726e0 0x7f9ec8074ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:46.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:46.226+0000 7f9ee7e00640 1 --2- 172.21.15.17:0/2482710443 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ee80c3b30 0x7f9ee80c4500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:46.236 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:46.234+0000 7f9ee7e00640 1 -- 172.21.15.17:0/2482710443 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9ee8201880 con 0x7f9ee80c3b30 2024-09-10T21:45:46.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:46.234+0000 7f9ee7e00640 1 --2- 172.21.15.17:0/2482710443 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ee80c3b30 0x7f9ee80c4500 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9ee8071130 tx=0x7f9ed4003830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:45:46.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:46.234+0000 7f9ee4c00640 1 -- 172.21.15.17:0/2482710443 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ed40be020 con 0x7f9ee80c3b30 2024-09-10T21:45:46.240 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:46.234+0000 7f9ee4c00640 1 -- 172.21.15.17:0/2482710443 <== mon.1 v2:172.21.15.31:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f9ed4003cb0 con 0x7f9ee80c3b30 2024-09-10T21:45:46.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:46.234+0000 7f9ee4c00640 1 -- 172.21.15.17:0/2482710443 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ee80c3b30 msgr2=0x7f9ee80c4500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:46.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:46.234+0000 7f9ee4c00640 1 --2- 172.21.15.17:0/2482710443 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ee80c3b30 0x7f9ee80c4500 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9ee8071130 tx=0x7f9ed4003830 comp rx=0 tx=0).stop 2024-09-10T21:45:46.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:46.234+0000 7f9ee4c00640 1 -- 172.21.15.17:0/2482710443 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ee80c3b30 msgr2=0x7f9ee80c4500 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:45:46.241 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:46.234+0000 7f9ee4c00640 1 --2- 172.21.15.17:0/2482710443 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ee80c3b30 0x7f9ee80c4500 secure :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9ee8071130 tx=0x7f9ed4003830 comp rx=0 tx=0).stop 2024-09-10T21:45:46.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:46.246+0000 7f9eeca00640 1 --2- 172.21.15.17:0/2482710443 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f9ec80726e0 0x7f9ec8074ba0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f9ee00012f0 tx=0x7f9ee000c550 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:45:46.251 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:46.246+0000 7f9eef400640 1 --2- 172.21.15.17:0/2482710443 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ee80c5fb0 0x7f9ee8150330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:46.251 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:46.250+0000 7f9eef400640 1 -- 172.21.15.17:0/2482710443 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f9ee806e010 con 0x7f9ee80c5fb0 2024-09-10T21:45:46.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:46.250+0000 7f9ee7e00640 1 --2- 172.21.15.17:0/2482710443 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ee80c5fb0 0x7f9ee8150330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:46.261 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:46.258+0000 7f9ee7e00640 1 -- 172.21.15.17:0/2482710443 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f9ee80c3160 con 0x7f9ee80c5fb0 2024-09-10T21:45:46.271 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:46.266+0000 7f9ee7e00640 1 --2- 172.21.15.17:0/2482710443 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ee80c5fb0 0x7f9ee8150330 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f9ed40b0040 tx=0x7f9ed40091e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:45:46.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:46.266+0000 7f9ee4c00640 1 -- 172.21.15.17:0/2482710443 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ed40be070 con 0x7f9ee80c5fb0 2024-09-10T21:45:46.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:46.266+0000 7f9ee4c00640 1 -- 172.21.15.17:0/2482710443 <== mon.1 v2:172.21.15.31:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5043 (secure 0 0 0) 0x7f9ed40c2910 con 0x7f9ee80c5fb0 2024-09-10T21:45:46.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:46.266+0000 7f9ee4c00640 1 -- 172.21.15.17:0/2482710443 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ee80c5fb0 msgr2=0x7f9ee8150330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:46.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:46.266+0000 7f9ee4c00640 1 --2- 172.21.15.17:0/2482710443 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ee80c5fb0 0x7f9ee8150330 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f9ed40b0040 tx=0x7f9ed40091e0 comp rx=0 tx=0).stop 2024-09-10T21:45:46.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:46.266+0000 7f9ee4c00640 1 -- 172.21.15.17:0/2482710443 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ee80c5fb0 msgr2=0x7f9ee8150330 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:45:46.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:46.266+0000 7f9ee4c00640 1 --2- 172.21.15.17:0/2482710443 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f9ee80c5fb0 0x7f9ee8150330 secure :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f9ed40b0040 tx=0x7f9ed40091e0 comp rx=0 tx=0).stop 2024-09-10T21:45:46.273 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-10T21:45:46.273 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:45:46.273 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:45:46.273 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-10T21:45:46.273 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 160, 2024-09-10T21:45:46.273 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 91250, 2024-09-10T21:45:46.273 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-10T21:45:46.273 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-10T21:45:46.273 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-10T21:45:46.273 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-10T21:45:46.273 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:45:46.274 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 15, 2024-09-10T21:45:46.274 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:45:46.274 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-10T21:45:46.274 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-10T21:45:46.274 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:45:46.274 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:45:46.274 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:45:46.274 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:45:46.274 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:45:46.274 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:45:46.275 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:45:46.275 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:45:46.275 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:45:46.275 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:45:46.275 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:45:46.275 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:45:46.275 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-10T21:45:46.275 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-10T21:45:46.275 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:45:46.275 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:45:46.276 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:45:46.276 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:45:46.276 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:45:46.276 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:45:46.276 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:45:46.276 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:45:46.276 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:45:46.276 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:45:46.276 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:45:46.276 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:45:46.276 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:45:46.277 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-10T21:45:46.277 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-10T21:45:46.277 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-10T21:45:46.277 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-10T21:45:46.277 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-10T21:45:46.277 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "5e2f023f-d6a6-49f4-a256-f4807dce3739", 2024-09-10T21:45:46.277 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:45:46.277 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:45:46.278 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-10T21:45:46.278 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-10T21:45:46.278 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-10T21:45:46.278 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-10T21:45:46.278 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-10T21:45:46.278 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-10T21:45:46.278 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-10T21:45:46.278 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:45:46.278 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-10T21:45:46.278 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:45:46.278 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:45:46.278 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:45:46.279 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:45:46.279 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:45:46.279 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:45:46.279 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:45:46.279 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:45:46.279 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:45:46.279 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:45:46.279 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:45:46.279 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:45:46.279 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-10T21:45:46.279 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:45:46.280 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-10T21:45:46.280 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:45:46.280 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:45:46.280 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:45:46.280 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:45:46.280 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:45:46.280 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:45:46.280 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:45:46.280 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-10T21:45:46.280 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:45:46.280 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:45:46.280 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:45:46.281 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:45:46.281 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-10T21:45:46.281 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:45:46.281 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:45:46.281 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:45:46.281 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:45:46.281 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-10T21:45:46.281 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-10T21:45:46.281 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:45:46.281 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:45:46.281 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:45:46.281 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:45:46.282 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:45:46.282 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:45:46.282 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:45:46.282 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:45:46.282 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:45:46.282 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:45:46.282 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:45:46.282 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3300", 2024-09-10T21:45:46.282 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:45:46.282 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:45:46.282 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:45:46.282 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:45:46.283 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789", 2024-09-10T21:45:46.283 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:45:46.283 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:45:46.283 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:45:46.283 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:45:46.283 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789/0", 2024-09-10T21:45:46.283 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6789/0", 2024-09-10T21:45:46.283 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:45:46.283 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:45:46.283 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:45:46.283 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:45:46.284 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:45:46.284 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:45:46.284 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:45:46.284 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:45:46.284 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:45:46.284 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:45:46.284 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:45:46.284 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3301", 2024-09-10T21:45:46.284 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:45:46.284 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:45:46.284 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:45:46.284 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:45:46.285 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790", 2024-09-10T21:45:46.285 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:45:46.285 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:45:46.285 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:45:46.285 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:45:46.285 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790/0", 2024-09-10T21:45:46.285 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6790/0", 2024-09-10T21:45:46.285 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:45:46.285 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:45:46.285 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:45:46.285 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:45:46.286 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:45:46.286 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:45:46.286 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-10T21:45:46.286 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-10T21:45:46.286 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:45:46.286 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:45:46.286 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:45:46.286 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:45:46.286 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:45:46.286 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:45:46.286 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": [ 2024-09-10T21:45:46.287 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:45:46.287 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:45:46.287 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:45:46.287 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 2 2024-09-10T21:45:46.287 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:45:46.287 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:45:46.287 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-10T21:45:46.287 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:45:46.287 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:45:46.287 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:45:46.287 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:45:46.288 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:45:46.288 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:45:46.288 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-10T21:45:46.288 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:45:46.288 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:45:46.288 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:45:46.288 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 3 2024-09-10T21:45:46.288 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:45:46.288 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:45:46.288 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": [ 2024-09-10T21:45:46.288 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:45:46.288 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:45:46.289 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:45:46.289 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:45:46.289 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:45:46.289 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:45:46.289 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:45:46.289 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-10T21:45:46.289 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-10T21:45:46.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:46.278+0000 7f9eef400640 1 -- 172.21.15.17:0/2482710443 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f9ec80726e0 msgr2=0x7f9ec8074ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:46.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:46.278+0000 7f9eef400640 1 --2- 172.21.15.17:0/2482710443 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f9ec80726e0 0x7f9ec8074ba0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f9ee00012f0 tx=0x7f9ee000c550 comp rx=0 tx=0).stop 2024-09-10T21:45:46.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:46.278+0000 7f9eef400640 1 -- 172.21.15.17:0/2482710443 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f9ee8154400 msgr2=0x7f9ee80c3fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:46.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:46.278+0000 7f9eef400640 1 --2- 172.21.15.17:0/2482710443 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f9ee8154400 0x7f9ee80c3fc0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f9ed800c730 tx=0x7f9ed800cbf0 comp rx=0 tx=0).stop 2024-09-10T21:45:46.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:46.282+0000 7f9eed400640 1 -- 172.21.15.17:0/2482710443 reap_dead start 2024-09-10T21:45:46.290 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:46.282+0000 7f9eed400640 1 -- 172.21.15.17:0/2482710443 reap_dead start 2024-09-10T21:45:46.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:46.286+0000 7f9eef400640 1 -- 172.21.15.17:0/2482710443 shutdown_connections 2024-09-10T21:45:46.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:46.286+0000 7f9eef400640 1 -- 172.21.15.17:0/2482710443 >> 172.21.15.17:0/2482710443 conn(0x7f9ee80700d0 msgr2=0x7f9ee80c8d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:45:46.294 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:46.286+0000 7f9eef400640 1 -- 172.21.15.17:0/2482710443 shutdown_connections 2024-09-10T21:45:46.295 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:46.286+0000 7f9eef400640 1 -- 172.21.15.17:0/2482710443 wait complete. 2024-09-10T21:45:46.336 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-10T21:45:46.336 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-09-10T21:45:46.336 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-10T21:45:46.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:46.682+0000 7f5431c00640 1 -- 172.21.15.17:0/4192508489 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f542404c2d0 msgr2=0x7f5424044c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:46.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:46.682+0000 7f5431c00640 1 --2- 172.21.15.17:0/4192508489 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f542404c2d0 0x7f5424044c60 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f542c07ba90 tx=0x7f542c09d370 comp rx=0 tx=0).stop 2024-09-10T21:45:46.685 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:46.682+0000 7f5431c00640 1 -- 172.21.15.17:0/4192508489 shutdown_connections 2024-09-10T21:45:46.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:46.682+0000 7f5431c00640 1 --2- 172.21.15.17:0/4192508489 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f542404c2d0 0x7f5424044c60 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:46.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:46.682+0000 7f5431c00640 1 --2- 172.21.15.17:0/4192508489 >> v2:172.21.15.17:3300/0 conn(0x7f542404b3a0 0x7f5424005c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:46.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:46.682+0000 7f5431c00640 1 --2- 172.21.15.17:0/4192508489 >> v2:172.21.15.31:3300/0 conn(0x7f542404afc0 0x7f54240da590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:46.686 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:46.682+0000 7f5431c00640 1 -- 172.21.15.17:0/4192508489 >> 172.21.15.17:0/4192508489 conn(0x7f542404fde0 msgr2=0x7f54240501f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:45:46.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:46.686+0000 7f5431c00640 1 -- 172.21.15.17:0/4192508489 shutdown_connections 2024-09-10T21:45:46.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:46.686+0000 7f5431c00640 1 -- 172.21.15.17:0/4192508489 wait complete. 2024-09-10T21:45:46.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:46.686+0000 7f5431c00640 1 Processor -- start 2024-09-10T21:45:46.711 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:46.706+0000 7f5431c00640 1 -- start start 2024-09-10T21:45:46.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:46.718+0000 7f5431c00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f542404afc0 0x7f54240cd190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:46.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:46.718+0000 7f5431c00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f542404b3a0 0x7f54240d97e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:46.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:46.718+0000 7f5431c00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f542404c2d0 0x7f5424181370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:46.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:46.718+0000 7f5431c00640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f54240070d0 con 0x7f542404afc0 2024-09-10T21:45:46.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:46.718+0000 7f5431c00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f5424006df0 con 0x7f542404b3a0 2024-09-10T21:45:46.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:46.718+0000 7f5431c00640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f5424050900 con 0x7f542404c2d0 2024-09-10T21:45:46.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:46.718+0000 7f542be00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f542404c2d0 0x7f5424181370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:46.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:46.718+0000 7f542be00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f542404c2d0 0x7f5424181370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3301/0 says I am v2:172.21.15.17:58442/0 (socket says 172.21.15.17:58442) 2024-09-10T21:45:46.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:46.718+0000 7f542be00640 1 -- 172.21.15.17:0/793368414 learned_addr learned my addr 172.21.15.17:0/793368414 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:45:46.723 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:46.718+0000 7f542aa00640 1 --2- 172.21.15.17:0/793368414 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f542404b3a0 0x7f54240d97e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:46.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:46.722+0000 7f542b400640 1 --2- 172.21.15.17:0/793368414 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f542404afc0 0x7f54240cd190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:46.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:46.722+0000 7f542aa00640 1 -- 172.21.15.17:0/793368414 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f542404c2d0 msgr2=0x7f5424181370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:46.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:46.722+0000 7f542aa00640 1 --2- 172.21.15.17:0/793368414 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f542404c2d0 0x7f5424181370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:46.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:46.722+0000 7f542aa00640 1 -- 172.21.15.17:0/793368414 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f542404afc0 msgr2=0x7f54240cd190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:46.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:46.722+0000 7f542aa00640 1 --2- 172.21.15.17:0/793368414 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f542404afc0 0x7f54240cd190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:45:46.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:46.722+0000 7f542aa00640 1 -- 172.21.15.17:0/793368414 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f542404f460 con 0x7f542404b3a0 2024-09-10T21:45:46.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:46.726+0000 7f542aa00640 1 --2- 172.21.15.17:0/793368414 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f542404b3a0 0x7f54240d97e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f542000bd40 tx=0x7f542000be30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:45:46.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:46.730+0000 7f541fe00640 1 -- 172.21.15.17:0/793368414 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5420019070 con 0x7f542404b3a0 2024-09-10T21:45:46.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:46.730+0000 7f541fe00640 1 -- 172.21.15.17:0/793368414 <== mon.1 v2:172.21.15.31:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5420014440 con 0x7f542404b3a0 2024-09-10T21:45:46.735 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:46.730+0000 7f541fe00640 1 -- 172.21.15.17:0/793368414 <== mon.1 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54200148d0 con 0x7f542404b3a0 2024-09-10T21:45:46.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:46.742+0000 7f5431c00640 1 -- 172.21.15.17:0/793368414 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f542404eea0 con 0x7f542404b3a0 2024-09-10T21:45:46.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:46.742+0000 7f5431c00640 1 -- 172.21.15.17:0/793368414 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f542404dcf0 con 0x7f542404b3a0 2024-09-10T21:45:46.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:46.746+0000 7f5431c00640 1 --2- 172.21.15.17:0/793368414 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f542404e9b0 0x7f54241818b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:46.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:46.746+0000 7f5431c00640 1 -- 172.21.15.17:0/793368414 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f542404e7d0 con 0x7f542404e9b0 2024-09-10T21:45:46.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:46.746+0000 7f541fe00640 1 -- 172.21.15.17:0/793368414 <== mon.1 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f542001d3b0 con 0x7f542404b3a0 2024-09-10T21:45:46.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:46.746+0000 7f541fe00640 1 --2- 172.21.15.17:0/793368414 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f540c072630 0x7f540c074af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:46.751 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:46.746+0000 7f541fe00640 1 -- 172.21.15.17:0/793368414 <== mon.1 v2:172.21.15.31:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f5420093920 con 0x7f542404b3a0 2024-09-10T21:45:46.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:46.750+0000 7f542b400640 1 --2- 172.21.15.17:0/793368414 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f542404e9b0 0x7f54241818b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:46.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:46.754+0000 7f542be00640 1 --2- 172.21.15.17:0/793368414 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f540c072630 0x7f540c074af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:46.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:46.754+0000 7f542b400640 1 -- 172.21.15.17:0/793368414 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5424005c30 con 0x7f542404e9b0 2024-09-10T21:45:46.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:46.758+0000 7f542b400640 1 --2- 172.21.15.17:0/793368414 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f542404e9b0 0x7f54241818b0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f541800f930 tx=0x7f541800fdf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:45:46.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:46.762+0000 7f542be00640 1 --2- 172.21.15.17:0/793368414 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f540c072630 0x7f540c074af0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f542c052200 tx=0x7f542c08e040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:45:46.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:46.762+0000 7f541fe00640 1 -- 172.21.15.17:0/793368414 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f541800dd40 con 0x7f542404e9b0 2024-09-10T21:45:46.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:46.762+0000 7f541fe00640 1 -- 172.21.15.17:0/793368414 <== mon.2 v2:172.21.15.31:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f5418011440 con 0x7f542404e9b0 2024-09-10T21:45:46.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:46.762+0000 7f541fe00640 1 -- 172.21.15.17:0/793368414 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f542404e9b0 msgr2=0x7f54241818b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:46.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:46.762+0000 7f541fe00640 1 --2- 172.21.15.17:0/793368414 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f542404e9b0 0x7f54241818b0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f541800f930 tx=0x7f541800fdf0 comp rx=0 tx=0).stop 2024-09-10T21:45:46.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:46.762+0000 7f541fe00640 1 -- 172.21.15.17:0/793368414 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f542404e9b0 msgr2=0x7f54241818b0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:45:46.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:46.762+0000 7f541fe00640 1 --2- 172.21.15.17:0/793368414 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f542404e9b0 0x7f54241818b0 secure :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f541800f930 tx=0x7f541800fdf0 comp rx=0 tx=0).stop 2024-09-10T21:45:46.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:46.790+0000 7f5431c00640 1 --2- 172.21.15.17:0/793368414 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f542404df30 0x7f542417e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:45:46.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:46.790+0000 7f5431c00640 1 -- 172.21.15.17:0/793368414 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f542404b930 con 0x7f542404df30 2024-09-10T21:45:46.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:46.798+0000 7f542b400640 1 --2- 172.21.15.17:0/793368414 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f542404df30 0x7f542417e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:45:46.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:46.806+0000 7f542b400640 1 -- 172.21.15.17:0/793368414 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f54240d1410 con 0x7f542404df30 2024-09-10T21:45:46.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:46.814+0000 7f542b400640 1 --2- 172.21.15.17:0/793368414 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f542404df30 0x7f542417e6b0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f541800f930 tx=0x7f54180046d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:45:46.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:46.818+0000 7f541fe00640 1 -- 172.21.15.17:0/793368414 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5418026070 con 0x7f542404df30 2024-09-10T21:45:46.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:46.818+0000 7f541fe00640 1 -- 172.21.15.17:0/793368414 <== mon.2 v2:172.21.15.31:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4709 (secure 0 0 0) 0x7f5418024070 con 0x7f542404df30 2024-09-10T21:45:46.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:46.818+0000 7f541fe00640 1 -- 172.21.15.17:0/793368414 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f542404df30 msgr2=0x7f542417e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:46.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:46.818+0000 7f541fe00640 1 --2- 172.21.15.17:0/793368414 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f542404df30 0x7f542417e6b0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f541800f930 tx=0x7f54180046d0 comp rx=0 tx=0).stop 2024-09-10T21:45:46.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:46.818+0000 7f541fe00640 1 -- 172.21.15.17:0/793368414 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f542404df30 msgr2=0x7f542417e6b0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:45:46.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:46.818+0000 7f541fe00640 1 --2- 172.21.15.17:0/793368414 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f542404df30 0x7f542417e6b0 secure :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f541800f930 tx=0x7f54180046d0 comp rx=0 tx=0).stop 2024-09-10T21:45:46.825 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-10T21:45:46.825 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:45:46.825 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:45:46.825 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-10T21:45:46.826 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 160, 2024-09-10T21:45:46.826 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 25235, 2024-09-10T21:45:46.826 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-10T21:45:46.826 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-10T21:45:46.826 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-10T21:45:46.826 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-10T21:45:46.826 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:45:46.826 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 15, 2024-09-10T21:45:46.826 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:45:46.826 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-10T21:45:46.826 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-10T21:45:46.826 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:45:46.826 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:45:46.827 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:45:46.827 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:45:46.827 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:45:46.827 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:45:46.827 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:45:46.827 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:45:46.827 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:45:46.827 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:45:46.827 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:45:46.827 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:45:46.827 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-10T21:45:46.827 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-10T21:45:46.827 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:45:46.827 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:45:46.827 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:45:46.828 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:45:46.828 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:45:46.828 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:45:46.828 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:45:46.828 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:45:46.828 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:45:46.828 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:45:46.828 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:45:46.828 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:45:46.828 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:45:46.828 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-10T21:45:46.828 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-10T21:45:46.828 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-10T21:45:46.829 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-10T21:45:46.829 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-10T21:45:46.829 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "5e2f023f-d6a6-49f4-a256-f4807dce3739", 2024-09-10T21:45:46.829 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:45:46.829 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:45:46.829 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-10T21:45:46.829 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-10T21:45:46.829 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-10T21:45:46.829 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-10T21:45:46.829 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-10T21:45:46.829 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-10T21:45:46.829 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-10T21:45:46.830 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:45:46.830 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-10T21:45:46.830 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:45:46.830 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:45:46.830 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:45:46.830 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:45:46.830 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:45:46.830 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:45:46.830 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:45:46.830 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:45:46.830 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:45:46.830 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:45:46.831 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:45:46.831 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:45:46.831 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-10T21:45:46.831 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:45:46.831 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-10T21:45:46.831 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:45:46.831 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:45:46.831 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:45:46.831 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:45:46.831 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:45:46.831 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:45:46.831 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:45:46.832 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-10T21:45:46.832 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:45:46.832 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:45:46.832 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:45:46.832 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:45:46.832 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-10T21:45:46.832 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:45:46.832 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:45:46.832 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:45:46.832 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:45:46.832 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-10T21:45:46.832 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-10T21:45:46.832 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:45:46.833 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:45:46.833 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:45:46.833 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:45:46.833 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:45:46.833 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:45:46.833 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:45:46.833 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:45:46.833 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:45:46.833 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:45:46.833 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:45:46.833 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3300", 2024-09-10T21:45:46.833 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:45:46.834 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:45:46.834 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:45:46.834 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:45:46.834 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789", 2024-09-10T21:45:46.834 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:45:46.834 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:45:46.834 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:45:46.834 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:45:46.834 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789/0", 2024-09-10T21:45:46.834 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6789/0", 2024-09-10T21:45:46.834 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:45:46.834 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:45:46.834 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:45:46.834 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:45:46.835 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:45:46.835 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:45:46.835 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:45:46.835 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:45:46.835 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:45:46.835 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:45:46.835 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:45:46.835 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3301", 2024-09-10T21:45:46.835 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:45:46.835 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:45:46.835 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:45:46.835 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:45:46.835 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790", 2024-09-10T21:45:46.836 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:45:46.836 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:45:46.836 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:45:46.836 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:45:46.836 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790/0", 2024-09-10T21:45:46.836 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6790/0", 2024-09-10T21:45:46.836 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:45:46.836 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:45:46.836 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:45:46.836 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:45:46.837 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:45:46.837 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:45:46.837 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-10T21:45:46.837 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-10T21:45:46.837 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:45:46.837 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:45:46.837 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:45:46.837 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:45:46.838 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:45:46.838 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:45:46.838 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": [ 2024-09-10T21:45:46.838 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:45:46.838 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:45:46.838 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:45:46.838 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:45:46.838 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:45:46.838 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:45:46.838 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-10T21:45:46.838 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:45:46.838 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:45:46.838 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:45:46.839 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:45:46.839 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:45:46.839 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:45:46.839 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:45:46.839 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-10T21:45:46.839 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-10T21:45:46.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:46.826+0000 7f5431c00640 1 -- 172.21.15.17:0/793368414 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f540c072630 msgr2=0x7f540c074af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:46.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:46.826+0000 7f5431c00640 1 --2- 172.21.15.17:0/793368414 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f540c072630 0x7f540c074af0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f542c052200 tx=0x7f542c08e040 comp rx=0 tx=0).stop 2024-09-10T21:45:46.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:46.826+0000 7f5431c00640 1 -- 172.21.15.17:0/793368414 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f542404b3a0 msgr2=0x7f54240d97e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:45:46.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:46.826+0000 7f5431c00640 1 --2- 172.21.15.17:0/793368414 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f542404b3a0 0x7f54240d97e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f542000bd40 tx=0x7f542000be30 comp rx=0 tx=0).stop 2024-09-10T21:45:46.840 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:46.826+0000 7f542be00640 1 -- 172.21.15.17:0/793368414 reap_dead start 2024-09-10T21:45:46.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:46.826+0000 7f542be00640 1 -- 172.21.15.17:0/793368414 reap_dead start 2024-09-10T21:45:46.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:46.834+0000 7f5431c00640 1 -- 172.21.15.17:0/793368414 shutdown_connections 2024-09-10T21:45:46.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:46.834+0000 7f5431c00640 1 -- 172.21.15.17:0/793368414 >> 172.21.15.17:0/793368414 conn(0x7f542404fde0 msgr2=0x7f54240d1be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:45:46.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:46.834+0000 7f5431c00640 1 -- 172.21.15.17:0/793368414 shutdown_connections 2024-09-10T21:45:46.841 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:45:46.834+0000 7f5431c00640 1 -- 172.21.15.17:0/793368414 wait complete. 2024-09-10T21:45:46.954 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-10T21:45:46.955 INFO:tasks.mon_thrash.mon_thrasher:monitors to thrash: ['a'] 2024-09-10T21:45:46.955 INFO:tasks.mon_thrash.mon_thrasher:monitors to freeze: ['b'] 2024-09-10T21:45:46.955 INFO:tasks.mon_thrash.mon_thrasher:thrashing mon.a 2024-09-10T21:45:46.955 INFO:tasks.mon_thrash.mon_thrasher:killing mon.a 2024-09-10T21:45:46.955 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-10T21:45:46.955 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:45:47.016 INFO:tasks.ceph.mon.a:Stopped 2024-09-10T21:45:47.016 INFO:tasks.mon_thrash:Sending STOP to mon b 2024-09-10T21:45:47.016 INFO:tasks.ceph.mon.b:Sent signal 19 2024-09-10T21:45:47.016 INFO:tasks.mon_thrash.mon_thrasher:waiting for 15.0 secs to unfreeze mons 2024-09-10T21:46:02.018 INFO:tasks.mon_thrash:Sending CONT to mon b 2024-09-10T21:46:02.018 INFO:tasks.ceph.mon.b:Sent signal 18 2024-09-10T21:46:02.018 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 2 2024-09-10T21:46:02.019 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-10T21:46:02.680 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:46:02.687 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:46:08.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:08.386+0000 7f00a4c00640 1 -- 172.21.15.17:0/2580484272 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f0080008e20 msgr2=0x7f00800086f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:08.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:08.386+0000 7f00a4c00640 1 --2- 172.21.15.17:0/2580484272 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f0080008e20 0x7f00800086f0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f00a006ed40 tx=0x7f009801d840 comp rx=0 tx=0).stop 2024-09-10T21:46:08.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:08.386+0000 7f00a4c00640 1 -- 172.21.15.17:0/2580484272 shutdown_connections 2024-09-10T21:46:08.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:08.386+0000 7f00a4c00640 1 --2- 172.21.15.17:0/2580484272 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f0080008e20 0x7f00800086f0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:08.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:08.386+0000 7f00a4c00640 1 --2- 172.21.15.17:0/2580484272 >> v2:172.21.15.17:3300/0 conn(0x7f0080009f80 0x7f0080005b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:08.396 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:08.386+0000 7f00a4c00640 1 -- 172.21.15.17:0/2580484272 >> 172.21.15.17:0/2580484272 conn(0x7f00a0070160 msgr2=0x7f00a0070570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:46:08.396 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:08.390+0000 7f00a4c00640 1 -- 172.21.15.17:0/2580484272 shutdown_connections 2024-09-10T21:46:08.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:08.398+0000 7f00a4c00640 1 -- 172.21.15.17:0/2580484272 wait complete. 2024-09-10T21:46:08.405 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:08.402+0000 7f00a4c00640 1 Processor -- start 2024-09-10T21:46:08.414 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:08.406+0000 7f00a4c00640 1 -- start start 2024-09-10T21:46:08.414 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:08.406+0000 7f00a4c00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0080008e20 0x7f00a01fe370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:08.414 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:08.406+0000 7f00a4c00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0080009f80 0x7f00a0201840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:08.414 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:08.406+0000 7f00a4c00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f008000a390 0x7f00a0152c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:08.414 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:08.406+0000 7f00a4c00640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f00a0154c70 con 0x7f0080008e20 2024-09-10T21:46:08.415 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:08.406+0000 7f00a4c00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f00a006e0a0 con 0x7f0080009f80 2024-09-10T21:46:08.415 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:08.406+0000 7f00a4c00640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f00a00c8600 con 0x7f008000a390 2024-09-10T21:46:08.415 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:08.406+0000 7f009d400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0080009f80 0x7f00a0201840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:08.415 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:08.406+0000 7f009d400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0080009f80 0x7f00a0201840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.17:34132/0 (socket says 172.21.15.17:34132) 2024-09-10T21:46:08.415 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:08.406+0000 7f009d400640 1 -- 172.21.15.17:0/3838592864 learned_addr learned my addr 172.21.15.17:0/3838592864 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:46:08.415 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:08.410+0000 7f009e800640 1 --2- 172.21.15.17:0/3838592864 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f008000a390 0x7f00a0152c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:08.415 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:08.410+0000 7f009de00640 1 -- 172.21.15.17:0/3838592864 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0080008e20 msgr2=0x7f00a01fe370 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-10T21:46:08.415 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:08.410+0000 7f009de00640 1 --2- 172.21.15.17:0/3838592864 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0080008e20 0x7f00a01fe370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:46:08.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:08.410+0000 7f009e800640 1 -- 172.21.15.17:0/3838592864 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0080009f80 msgr2=0x7f00a0201840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:08.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:08.410+0000 7f009e800640 1 --2- 172.21.15.17:0/3838592864 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0080009f80 0x7f00a0201840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:08.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:08.410+0000 7f009e800640 1 -- 172.21.15.17:0/3838592864 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0080008e20 msgr2=0x7f00a01fe370 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:46:08.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:08.410+0000 7f009e800640 1 --2- 172.21.15.17:0/3838592864 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0080008e20 0x7f00a01fe370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:08.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:08.410+0000 7f009e800640 1 -- 172.21.15.17:0/3838592864 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00a00c3490 con 0x7f008000a390 2024-09-10T21:46:08.416 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:08.414+0000 7f009e800640 1 --2- 172.21.15.17:0/3838592864 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f008000a390 0x7f00a0152c40 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f00980ae000 tx=0x7f0098004c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:46:08.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:08.418+0000 7f0096000640 1 -- 172.21.15.17:0/3838592864 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00980bf070 con 0x7f008000a390 2024-09-10T21:46:08.419 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:08.418+0000 7f0096000640 1 -- 172.21.15.17:0/3838592864 <== mon.2 v2:172.21.15.31:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0098005540 con 0x7f008000a390 2024-09-10T21:46:08.420 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:08.418+0000 7f0096000640 1 -- 172.21.15.17:0/3838592864 <== mon.2 v2:172.21.15.31:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00980ac040 con 0x7f008000a390 2024-09-10T21:46:08.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:08.418+0000 7f00a4c00640 1 -- 172.21.15.17:0/3838592864 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00a00c3770 con 0x7f008000a390 2024-09-10T21:46:08.422 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:08.418+0000 7f00a4c00640 1 -- 172.21.15.17:0/3838592864 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f00a0201040 con 0x7f008000a390 2024-09-10T21:46:08.433 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:08.426+0000 7f008b400640 1 -- 172.21.15.17:0/3838592864 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00740052a0 con 0x7f008000a390 2024-09-10T21:46:08.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:08.430+0000 7f0096000640 1 -- 172.21.15.17:0/3838592864 <== mon.2 v2:172.21.15.31:3301/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f00980b9070 con 0x7f008000a390 2024-09-10T21:46:08.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:08.438+0000 7f0096000640 1 --2- 172.21.15.17:0/3838592864 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f0080077320 0x7f00800797e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:08.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:08.438+0000 7f0096000640 1 -- 172.21.15.17:0/3838592864 <== mon.2 v2:172.21.15.31:3301/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f009812ee10 con 0x7f008000a390 2024-09-10T21:46:08.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:08.438+0000 7f0096000640 1 -- 172.21.15.17:0/3838592864 <== mon.2 v2:172.21.15.31:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f0098017620 con 0x7f008000a390 2024-09-10T21:46:08.442 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:08.438+0000 7f009de00640 1 --2- 172.21.15.17:0/3838592864 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f0080077320 0x7f00800797e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:08.480 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:08.478+0000 7f009de00640 1 --2- 172.21.15.17:0/3838592864 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f0080077320 0x7f00800797e0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f0090000f70 tx=0x7f0090000f00 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:46:08.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:08.786+0000 7f008b400640 1 -- 172.21.15.17:0/3838592864 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f0074005590 con 0x7f008000a390 2024-09-10T21:46:08.789 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":162,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":1,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5e2f023f-d6a6-49f4-a256-f4807dce3739","modified":"2024-09-10T21:23:46.892130Z","created":"2024-09-10T21:23:46.892130Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3301","nonce":0},{"type":"v1","addr":"172.21.15.31:6790","nonce":0}]},"addr":"172.21.15.31:6790/0","public_addr":"172.21.15.31:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-10T21:46:08.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:08.786+0000 7f0096000640 1 -- 172.21.15.17:0/3838592864 <== mon.2 v2:172.21.15.31:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1551 (secure 0 0 0) 0x7f00980fa040 con 0x7f008000a390 2024-09-10T21:46:08.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:08.786+0000 7f008b400640 1 -- 172.21.15.17:0/3838592864 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f0080077320 msgr2=0x7f00800797e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:08.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:08.786+0000 7f008b400640 1 --2- 172.21.15.17:0/3838592864 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f0080077320 0x7f00800797e0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f0090000f70 tx=0x7f0090000f00 comp rx=0 tx=0).stop 2024-09-10T21:46:08.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:08.786+0000 7f008b400640 1 -- 172.21.15.17:0/3838592864 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f008000a390 msgr2=0x7f00a0152c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:08.892 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:08.786+0000 7f008b400640 1 --2- 172.21.15.17:0/3838592864 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f008000a390 0x7f00a0152c40 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f00980ae000 tx=0x7f0098004c80 comp rx=0 tx=0).stop 2024-09-10T21:46:08.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:08.786+0000 7f008b400640 1 -- 172.21.15.17:0/3838592864 shutdown_connections 2024-09-10T21:46:08.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:08.786+0000 7f008b400640 1 --2- 172.21.15.17:0/3838592864 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f0080077320 0x7f00800797e0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:08.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:08.786+0000 7f008b400640 1 --2- 172.21.15.17:0/3838592864 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f008000a390 0x7f00a0152c40 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:08.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:08.786+0000 7f008b400640 1 --2- 172.21.15.17:0/3838592864 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f0080009f80 0x7f00a0201840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:08.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:08.786+0000 7f008b400640 1 --2- 172.21.15.17:0/3838592864 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f0080008e20 0x7f00a01fe370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:08.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:08.786+0000 7f008b400640 1 -- 172.21.15.17:0/3838592864 >> 172.21.15.17:0/3838592864 conn(0x7f00a0070160 msgr2=0x7f00a00ca6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:46:08.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:08.790+0000 7f008b400640 1 -- 172.21.15.17:0/3838592864 shutdown_connections 2024-09-10T21:46:08.893 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:08.790+0000 7f008b400640 1 -- 172.21.15.17:0/3838592864 wait complete. 2024-09-10T21:46:08.947 INFO:tasks.mon_thrash.ceph_manager:quorum is size 2 2024-09-10T21:46:08.947 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon b 2024-09-10T21:46:08.947 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-10T21:46:10.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:10.738+0000 7efd35a00640 1 -- 172.21.15.17:0/109495968 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efd30167eb0 msgr2=0x7efd3006e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:10.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:10.738+0000 7efd35a00640 1 --2- 172.21.15.17:0/109495968 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efd30167eb0 0x7efd3006e560 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7efd30071050 tx=0x7efd2001f5a0 comp rx=0 tx=0).stop 2024-09-10T21:46:10.753 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:10.750+0000 7efd35a00640 1 -- 172.21.15.17:0/109495968 shutdown_connections 2024-09-10T21:46:10.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:10.750+0000 7efd35a00640 1 --2- 172.21.15.17:0/109495968 >> v2:172.21.15.17:3300/0 conn(0x7efd300c8b40 0x7efd300c6e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:10.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:10.750+0000 7efd35a00640 1 --2- 172.21.15.17:0/109495968 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efd30167eb0 0x7efd3006e560 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:10.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:10.750+0000 7efd35a00640 1 -- 172.21.15.17:0/109495968 >> 172.21.15.17:0/109495968 conn(0x7efd30070110 msgr2=0x7efd30070520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:46:10.754 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:10.750+0000 7efd35a00640 1 -- 172.21.15.17:0/109495968 shutdown_connections 2024-09-10T21:46:10.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:10.750+0000 7efd35a00640 1 -- 172.21.15.17:0/109495968 wait complete. 2024-09-10T21:46:10.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:10.758+0000 7efd35a00640 1 Processor -- start 2024-09-10T21:46:10.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:10.758+0000 7efd35a00640 1 -- start start 2024-09-10T21:46:10.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:10.762+0000 7efd35a00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efd300c7380 0x7efd301efc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:10.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:10.762+0000 7efd35a00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efd300c8b40 0x7efd301edca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:10.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:10.762+0000 7efd35a00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7efd30167eb0 0x7efd301ee1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:10.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:10.762+0000 7efd35a00640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7efd300c8730 con 0x7efd300c8b40 2024-09-10T21:46:10.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:10.762+0000 7efd35a00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7efd3006eaa0 con 0x7efd300c7380 2024-09-10T21:46:10.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:10.762+0000 7efd35a00640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7efd300c88b0 con 0x7efd30167eb0 2024-09-10T21:46:10.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:10.762+0000 7efd2d600640 1 -- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efd300c8b40 msgr2=0x7efd301edca0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-10T21:46:10.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:10.762+0000 7efd2d600640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efd300c8b40 0x7efd301edca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:46:10.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:10.762+0000 7efd2e000640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efd300c7380 0x7efd301efc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:10.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:10.762+0000 7efd2e000640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efd300c7380 0x7efd301efc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.17:34148/0 (socket says 172.21.15.17:34148) 2024-09-10T21:46:10.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:10.762+0000 7efd2e000640 1 -- 172.21.15.17:0/2961914994 learned_addr learned my addr 172.21.15.17:0/2961914994 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:46:10.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:10.762+0000 7efd2ea00640 1 --2- 172.21.15.17:0/2961914994 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7efd30167eb0 0x7efd301ee1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:10.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:10.766+0000 7efd2e000640 1 -- 172.21.15.17:0/2961914994 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7efd30167eb0 msgr2=0x7efd301ee1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:10.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:10.766+0000 7efd2e000640 1 --2- 172.21.15.17:0/2961914994 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7efd30167eb0 0x7efd301ee1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:10.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:10.766+0000 7efd2e000640 1 -- 172.21.15.17:0/2961914994 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efd300c8b40 msgr2=0x7efd301edca0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:46:10.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:10.766+0000 7efd2e000640 1 --2- 172.21.15.17:0/2961914994 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efd300c8b40 0x7efd301edca0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:10.769 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:10.766+0000 7efd2e000640 1 -- 172.21.15.17:0/2961914994 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd301ed660 con 0x7efd300c7380 2024-09-10T21:46:10.769 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:10.766+0000 7efd2e000640 1 --2- 172.21.15.17:0/2961914994 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efd300c7380 0x7efd301efc90 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7efd20000f90 tx=0x7efd200b0590 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:46:10.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:10.766+0000 7efd26a00640 1 -- 172.21.15.17:0/2961914994 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd200ac070 con 0x7efd300c7380 2024-09-10T21:46:10.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:10.766+0000 7efd26a00640 1 -- 172.21.15.17:0/2961914994 <== mon.1 v2:172.21.15.31:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efd200b1c10 con 0x7efd300c7380 2024-09-10T21:46:10.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:10.766+0000 7efd26a00640 1 -- 172.21.15.17:0/2961914994 <== mon.1 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd200b5040 con 0x7efd300c7380 2024-09-10T21:46:10.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:10.766+0000 7efd35a00640 1 -- 172.21.15.17:0/2961914994 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd301ed940 con 0x7efd300c7380 2024-09-10T21:46:10.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:10.770+0000 7efd35a00640 1 -- 172.21.15.17:0/2961914994 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd300cca10 con 0x7efd300c7380 2024-09-10T21:46:10.787 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:10.774+0000 7efd35a00640 1 --2- 172.21.15.17:0/2961914994 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efd30166670 0x7efd301f38d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:10.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:10.774+0000 7efd35a00640 1 -- 172.21.15.17:0/2961914994 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7efd300c6e40 con 0x7efd30166670 2024-09-10T21:46:10.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:10.778+0000 7efd26a00640 1 -- 172.21.15.17:0/2961914994 <== mon.1 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7efd200bb470 con 0x7efd300c7380 2024-09-10T21:46:10.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:10.778+0000 7efd2d600640 1 --2- 172.21.15.17:0/2961914994 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efd30166670 0x7efd301f38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:10.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:10.782+0000 7efd26a00640 1 --2- 172.21.15.17:0/2961914994 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7efd100725c0 0x7efd10074a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:10.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:10.782+0000 7efd26a00640 1 -- 172.21.15.17:0/2961914994 <== mon.1 v2:172.21.15.31:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7efd20131510 con 0x7efd300c7380 2024-09-10T21:46:10.788 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:10.782+0000 7efd2ea00640 1 --2- 172.21.15.17:0/2961914994 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7efd100725c0 0x7efd10074a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:10.789 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:10.786+0000 7efd2ea00640 1 --2- 172.21.15.17:0/2961914994 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7efd100725c0 0x7efd10074a80 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7efd14006fd0 tx=0x7efd14008040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:46:10.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:10.790+0000 7efd2d600640 1 -- 172.21.15.17:0/2961914994 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7efd300748e0 con 0x7efd30166670 2024-09-10T21:46:10.794 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:10.790+0000 7efd2d600640 1 --2- 172.21.15.17:0/2961914994 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efd30166670 0x7efd301f38d0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7efd301672d0 tx=0x7efd1c00b8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:46:10.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:10.794+0000 7efd26a00640 1 -- 172.21.15.17:0/2961914994 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd1c016070 con 0x7efd30166670 2024-09-10T21:46:10.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:10.798+0000 7efd26a00640 1 -- 172.21.15.17:0/2961914994 <== mon.1 v2:172.21.15.31:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7efd1c003d80 con 0x7efd30166670 2024-09-10T21:46:10.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:10.798+0000 7efd26a00640 1 -- 172.21.15.17:0/2961914994 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efd30166670 msgr2=0x7efd301f38d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:10.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:10.798+0000 7efd26a00640 1 --2- 172.21.15.17:0/2961914994 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efd30166670 0x7efd301f38d0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7efd301672d0 tx=0x7efd1c00b8e0 comp rx=0 tx=0).stop 2024-09-10T21:46:10.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:10.798+0000 7efd26a00640 1 -- 172.21.15.17:0/2961914994 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efd30166670 msgr2=0x7efd301f38d0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:46:10.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:10.798+0000 7efd26a00640 1 --2- 172.21.15.17:0/2961914994 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efd30166670 0x7efd301f38d0 secure :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7efd301672d0 tx=0x7efd1c00b8e0 comp rx=0 tx=0).stop 2024-09-10T21:46:10.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:10.814+0000 7efd35a00640 1 --2- 172.21.15.17:0/2961914994 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efd300c79d0 0x7efd301f3e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:10.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:10.814+0000 7efd35a00640 1 -- 172.21.15.17:0/2961914994 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7efd30165290 con 0x7efd300c79d0 2024-09-10T21:46:10.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:10.818+0000 7efd2d600640 1 --2- 172.21.15.17:0/2961914994 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efd300c79d0 0x7efd301f3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:10.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:10.818+0000 7efd2d600640 1 -- 172.21.15.17:0/2961914994 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7efd300707d0 con 0x7efd300c79d0 2024-09-10T21:46:10.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:10.822+0000 7efd2d600640 1 --2- 172.21.15.17:0/2961914994 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efd300c79d0 0x7efd301f3e10 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7efd30074540 tx=0x7efd1c005990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:46:10.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:10.822+0000 7efd26a00640 1 -- 172.21.15.17:0/2961914994 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd1c016030 con 0x7efd300c79d0 2024-09-10T21:46:10.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:10.822+0000 7efd26a00640 1 -- 172.21.15.17:0/2961914994 <== mon.1 v2:172.21.15.31:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5034 (secure 0 0 0) 0x7efd1c0103a0 con 0x7efd300c79d0 2024-09-10T21:46:10.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:10.822+0000 7efd26a00640 1 -- 172.21.15.17:0/2961914994 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efd300c79d0 msgr2=0x7efd301f3e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:10.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:10.822+0000 7efd26a00640 1 --2- 172.21.15.17:0/2961914994 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efd300c79d0 0x7efd301f3e10 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7efd30074540 tx=0x7efd1c005990 comp rx=0 tx=0).stop 2024-09-10T21:46:10.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:10.822+0000 7efd26a00640 1 -- 172.21.15.17:0/2961914994 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efd300c79d0 msgr2=0x7efd301f3e10 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:46:10.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:10.822+0000 7efd26a00640 1 --2- 172.21.15.17:0/2961914994 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efd300c79d0 0x7efd301f3e10 secure :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7efd30074540 tx=0x7efd1c005990 comp rx=0 tx=0).stop 2024-09-10T21:46:10.837 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-10T21:46:10.837 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:46:10.838 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:46:10.838 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "leader", 2024-09-10T21:46:10.838 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 162, 2024-09-10T21:46:10.838 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 115802, 2024-09-10T21:46:10.838 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-10T21:46:10.838 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-10T21:46:10.838 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-10T21:46:10.838 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:46:10.838 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 3, 2024-09-10T21:46:10.838 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:46:10.838 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-10T21:46:10.838 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-10T21:46:10.838 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:46:10.839 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:46:10.839 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:46:10.839 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:46:10.839 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:46:10.839 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:46:10.839 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:46:10.839 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:46:10.839 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:46:10.839 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:46:10.839 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:46:10.839 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:46:10.839 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-10T21:46:10.839 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-10T21:46:10.839 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:46:10.840 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:46:10.840 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:46:10.840 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:46:10.840 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:46:10.840 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:46:10.840 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:46:10.840 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:46:10.840 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:46:10.840 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:46:10.840 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:46:10.840 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:46:10.840 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:46:10.841 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-10T21:46:10.841 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-10T21:46:10.841 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-10T21:46:10.841 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-10T21:46:10.841 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-10T21:46:10.841 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "5e2f023f-d6a6-49f4-a256-f4807dce3739", 2024-09-10T21:46:10.841 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:46:10.841 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:46:10.841 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-10T21:46:10.841 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-10T21:46:10.841 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-10T21:46:10.841 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-10T21:46:10.841 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-10T21:46:10.841 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-10T21:46:10.842 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-10T21:46:10.842 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:46:10.842 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-10T21:46:10.842 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:46:10.842 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:46:10.842 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:46:10.842 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:46:10.842 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:46:10.842 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:46:10.842 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:46:10.843 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:46:10.843 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:46:10.843 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:46:10.843 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:46:10.843 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:46:10.843 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-10T21:46:10.843 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:46:10.843 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-10T21:46:10.843 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:46:10.843 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:46:10.843 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:46:10.843 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:46:10.843 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:46:10.844 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:46:10.844 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:46:10.844 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-10T21:46:10.844 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:46:10.844 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:46:10.844 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:46:10.844 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:46:10.844 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-10T21:46:10.844 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:46:10.844 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:46:10.844 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:46:10.844 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:46:10.844 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-10T21:46:10.845 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-10T21:46:10.845 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:46:10.845 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:46:10.845 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:46:10.845 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:46:10.845 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:46:10.845 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:46:10.845 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:46:10.845 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:46:10.845 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:46:10.845 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:46:10.845 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:46:10.845 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3300", 2024-09-10T21:46:10.846 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:46:10.846 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:46:10.846 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:46:10.846 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:46:10.846 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789", 2024-09-10T21:46:10.846 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:46:10.846 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:46:10.846 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:46:10.846 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:46:10.846 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789/0", 2024-09-10T21:46:10.846 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6789/0", 2024-09-10T21:46:10.846 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:46:10.846 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:46:10.846 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:46:10.847 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:46:10.847 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:46:10.847 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:46:10.847 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:46:10.847 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:46:10.847 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:46:10.847 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:46:10.847 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:46:10.847 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3301", 2024-09-10T21:46:10.847 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:46:10.847 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:46:10.847 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:46:10.847 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:46:10.847 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790", 2024-09-10T21:46:10.848 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:46:10.848 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:46:10.848 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:46:10.848 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:46:10.848 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790/0", 2024-09-10T21:46:10.848 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6790/0", 2024-09-10T21:46:10.848 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:46:10.848 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:46:10.848 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:46:10.848 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:46:10.848 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:46:10.848 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:46:10.848 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-10T21:46:10.848 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-10T21:46:10.849 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:46:10.849 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:46:10.849 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:46:10.849 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:46:10.849 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:46:10.849 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:46:10.849 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": [ 2024-09-10T21:46:10.849 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:46:10.849 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:46:10.849 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:46:10.849 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 3 2024-09-10T21:46:10.849 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:46:10.849 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:46:10.850 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-10T21:46:10.850 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:46:10.850 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:46:10.850 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:46:10.850 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 8 2024-09-10T21:46:10.850 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:46:10.850 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:46:10.850 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-10T21:46:10.850 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:46:10.850 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:46:10.850 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:46:10.850 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 7 2024-09-10T21:46:10.850 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:46:10.851 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:46:10.851 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": [ 2024-09-10T21:46:10.851 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:46:10.851 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:46:10.851 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:46:10.851 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 2 2024-09-10T21:46:10.851 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:46:10.851 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:46:10.851 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:46:10.851 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-10T21:46:10.851 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-10T21:46:10.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:10.838+0000 7efd35a00640 1 -- 172.21.15.17:0/2961914994 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7efd100725c0 msgr2=0x7efd10074a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:10.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:10.838+0000 7efd35a00640 1 --2- 172.21.15.17:0/2961914994 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7efd100725c0 0x7efd10074a80 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7efd14006fd0 tx=0x7efd14008040 comp rx=0 tx=0).stop 2024-09-10T21:46:10.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:10.838+0000 7efd35a00640 1 -- 172.21.15.17:0/2961914994 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efd300c7380 msgr2=0x7efd301efc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:10.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:10.838+0000 7efd35a00640 1 --2- 172.21.15.17:0/2961914994 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efd300c7380 0x7efd301efc90 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7efd20000f90 tx=0x7efd200b0590 comp rx=0 tx=0).stop 2024-09-10T21:46:10.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:10.838+0000 7efd2ea00640 1 -- 172.21.15.17:0/2961914994 reap_dead start 2024-09-10T21:46:10.852 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:10.838+0000 7efd2ea00640 1 -- 172.21.15.17:0/2961914994 reap_dead start 2024-09-10T21:46:10.853 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:10.838+0000 7efd35a00640 1 -- 172.21.15.17:0/2961914994 shutdown_connections 2024-09-10T21:46:10.853 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:10.838+0000 7efd35a00640 1 -- 172.21.15.17:0/2961914994 >> 172.21.15.17:0/2961914994 conn(0x7efd30070110 msgr2=0x7efd300c5800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:46:10.853 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:10.846+0000 7efd35a00640 1 -- 172.21.15.17:0/2961914994 shutdown_connections 2024-09-10T21:46:10.853 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:10.846+0000 7efd35a00640 1 -- 172.21.15.17:0/2961914994 wait complete. 2024-09-10T21:46:10.896 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-09-10T21:46:10.896 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 2 for mon c 2024-09-10T21:46:10.896 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-10T21:46:11.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:11.222+0000 7ff417c00640 1 -- 172.21.15.17:0/3217393721 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff41006dc30 msgr2=0x7ff41006e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:11.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:11.222+0000 7ff417c00640 1 --2- 172.21.15.17:0/3217393721 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff41006dc30 0x7ff41006e010 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7ff404009900 tx=0x7ff40402ff40 comp rx=0 tx=0).stop 2024-09-10T21:46:11.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:11.226+0000 7ff417c00640 1 -- 172.21.15.17:0/3217393721 shutdown_connections 2024-09-10T21:46:11.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:11.226+0000 7ff417c00640 1 --2- 172.21.15.17:0/3217393721 >> v2:172.21.15.31:3300/0 conn(0x7ff410154400 0x7ff4101548b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:11.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:11.226+0000 7ff417c00640 1 --2- 172.21.15.17:0/3217393721 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff41006dc30 0x7ff41006e010 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:11.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:11.226+0000 7ff417c00640 1 -- 172.21.15.17:0/3217393721 >> 172.21.15.17:0/3217393721 conn(0x7ff4100700d0 msgr2=0x7ff4100704e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:46:11.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:11.226+0000 7ff417c00640 1 -- 172.21.15.17:0/3217393721 shutdown_connections 2024-09-10T21:46:11.230 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:11.226+0000 7ff417c00640 1 -- 172.21.15.17:0/3217393721 wait complete. 2024-09-10T21:46:11.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:11.230+0000 7ff417c00640 1 Processor -- start 2024-09-10T21:46:11.232 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:11.230+0000 7ff417c00640 1 -- start start 2024-09-10T21:46:11.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:11.230+0000 7ff417c00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff41006dc30 0x7ff410157550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:11.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:11.230+0000 7ff417c00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff41006e7a0 0x7ff41015a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:11.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:11.230+0000 7ff417c00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff410154400 0x7ff4100c5620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:11.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:11.230+0000 7ff417c00640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7ff4100c8220 con 0x7ff410154400 2024-09-10T21:46:11.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:11.230+0000 7ff417c00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7ff4100c8870 con 0x7ff41006e7a0 2024-09-10T21:46:11.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:11.230+0000 7ff417c00640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7ff4100c8500 con 0x7ff41006dc30 2024-09-10T21:46:11.246 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:11.230+0000 7ff415c00640 1 -- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff410154400 msgr2=0x7ff4100c5620 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.17:3300/0 2024-09-10T21:46:11.246 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:11.230+0000 7ff415c00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff410154400 0x7ff4100c5620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:46:11.246 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:11.230+0000 7ff40fe00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff41006e7a0 0x7ff41015a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:11.246 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:11.230+0000 7ff40fe00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff41006e7a0 0x7ff41015a100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.17:34186/0 (socket says 172.21.15.17:34186) 2024-09-10T21:46:11.246 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:11.230+0000 7ff40fe00640 1 -- 172.21.15.17:0/299873048 learned_addr learned my addr 172.21.15.17:0/299873048 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:46:11.246 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:11.230+0000 7ff415200640 1 --2- 172.21.15.17:0/299873048 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff41006dc30 0x7ff410157550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:11.246 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:11.234+0000 7ff415200640 1 -- 172.21.15.17:0/299873048 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff41006e7a0 msgr2=0x7ff41015a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:11.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:11.234+0000 7ff415200640 1 --2- 172.21.15.17:0/299873048 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff41006e7a0 0x7ff41015a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:11.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:11.234+0000 7ff415200640 1 -- 172.21.15.17:0/299873048 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff410154400 msgr2=0x7ff4100c5620 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:46:11.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:11.234+0000 7ff415200640 1 --2- 172.21.15.17:0/299873048 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff410154400 0x7ff4100c5620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:11.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:11.234+0000 7ff415200640 1 -- 172.21.15.17:0/299873048 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4101ec3a0 con 0x7ff41006dc30 2024-09-10T21:46:11.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:11.238+0000 7ff415200640 1 --2- 172.21.15.17:0/299873048 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff41006dc30 0x7ff410157550 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ff41015b650 tx=0x7ff404005a50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:46:11.251 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:11.238+0000 7ff40cc00640 1 -- 172.21.15.17:0/299873048 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4040d0070 con 0x7ff41006dc30 2024-09-10T21:46:11.251 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:11.238+0000 7ff40cc00640 1 -- 172.21.15.17:0/299873048 <== mon.2 v2:172.21.15.31:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff4040038f0 con 0x7ff41006dc30 2024-09-10T21:46:11.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:11.238+0000 7ff40cc00640 1 -- 172.21.15.17:0/299873048 <== mon.2 v2:172.21.15.31:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4040c6040 con 0x7ff41006dc30 2024-09-10T21:46:11.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:11.238+0000 7ff417c00640 1 -- 172.21.15.17:0/299873048 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4101ec680 con 0x7ff41006dc30 2024-09-10T21:46:11.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:11.238+0000 7ff417c00640 1 -- 172.21.15.17:0/299873048 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4101ecc40 con 0x7ff41006dc30 2024-09-10T21:46:11.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:11.242+0000 7ff417c00640 1 --2- 172.21.15.17:0/299873048 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff4100b1770 0x7ff4100c5b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:11.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:11.242+0000 7ff417c00640 1 -- 172.21.15.17:0/299873048 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7ff4100c3160 con 0x7ff4100b1770 2024-09-10T21:46:11.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:11.242+0000 7ff40fe00640 1 --2- 172.21.15.17:0/299873048 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff4100b1770 0x7ff4100c5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:11.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:11.242+0000 7ff40fe00640 1 -- 172.21.15.17:0/299873048 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff410154080 con 0x7ff4100b1770 2024-09-10T21:46:11.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:11.242+0000 7ff40fe00640 1 --2- 172.21.15.17:0/299873048 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff4100b1770 0x7ff4100c5b60 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff4100c5050 tx=0x7ff3fc00fc00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:46:11.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:11.242+0000 7ff40cc00640 1 -- 172.21.15.17:0/299873048 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3fc018030 con 0x7ff4100b1770 2024-09-10T21:46:11.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:11.242+0000 7ff40cc00640 1 -- 172.21.15.17:0/299873048 <== mon.2 v2:172.21.15.31:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7ff3fc007020 con 0x7ff4100b1770 2024-09-10T21:46:11.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:11.242+0000 7ff40cc00640 1 -- 172.21.15.17:0/299873048 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff4100b1770 msgr2=0x7ff4100c5b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:11.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:11.242+0000 7ff40cc00640 1 --2- 172.21.15.17:0/299873048 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff4100b1770 0x7ff4100c5b60 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff4100c5050 tx=0x7ff3fc00fc00 comp rx=0 tx=0).stop 2024-09-10T21:46:11.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:11.242+0000 7ff40cc00640 1 -- 172.21.15.17:0/299873048 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff4100b1770 msgr2=0x7ff4100c5b60 unknown :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:46:11.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:11.242+0000 7ff40cc00640 1 --2- 172.21.15.17:0/299873048 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff4100b1770 0x7ff4100c5b60 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:11.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:11.246+0000 7ff40cc00640 1 -- 172.21.15.17:0/299873048 <== mon.2 v2:172.21.15.31:3301/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7ff4040c9070 con 0x7ff41006dc30 2024-09-10T21:46:11.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:11.246+0000 7ff40cc00640 1 --2- 172.21.15.17:0/299873048 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7ff3f0072050 0x7ff3f0074510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:11.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:11.246+0000 7ff40cc00640 1 -- 172.21.15.17:0/299873048 <== mon.2 v2:172.21.15.31:3301/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7ff404140fe0 con 0x7ff41006dc30 2024-09-10T21:46:11.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:11.246+0000 7ff40fe00640 1 --2- 172.21.15.17:0/299873048 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7ff3f0072050 0x7ff3f0074510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:11.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:11.250+0000 7ff40fe00640 1 --2- 172.21.15.17:0/299873048 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7ff3f0072050 0x7ff3f0074510 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7ff4100c5050 tx=0x7ff3fc009030 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:46:11.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:11.274+0000 7ff417c00640 1 --2- 172.21.15.17:0/299873048 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff4100c5100 0x7ff41014b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:11.277 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:11.274+0000 7ff417c00640 1 -- 172.21.15.17:0/299873048 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7ff4100c78a0 con 0x7ff4100c5100 2024-09-10T21:46:11.279 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:11.274+0000 7ff415c00640 1 --2- 172.21.15.17:0/299873048 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff4100c5100 0x7ff41014b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:11.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:11.282+0000 7ff415c00640 1 -- 172.21.15.17:0/299873048 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7ff41006e010 con 0x7ff4100c5100 2024-09-10T21:46:11.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:11.282+0000 7ff415c00640 1 --2- 172.21.15.17:0/299873048 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff4100c5100 0x7ff41014b6e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7ff40800e9c0 tx=0x7ff40800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:46:11.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:11.282+0000 7ff40cc00640 1 -- 172.21.15.17:0/299873048 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff40800cd80 con 0x7ff4100c5100 2024-09-10T21:46:11.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:11.282+0000 7ff40cc00640 1 -- 172.21.15.17:0/299873048 <== mon.2 v2:172.21.15.31:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4697 (secure 0 0 0) 0x7ff40800cd80 con 0x7ff4100c5100 2024-09-10T21:46:11.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:11.282+0000 7ff40cc00640 1 -- 172.21.15.17:0/299873048 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff4100c5100 msgr2=0x7ff41014b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:11.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:11.282+0000 7ff40cc00640 1 --2- 172.21.15.17:0/299873048 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff4100c5100 0x7ff41014b6e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7ff40800e9c0 tx=0x7ff40800ee80 comp rx=0 tx=0).stop 2024-09-10T21:46:11.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:11.282+0000 7ff40cc00640 1 -- 172.21.15.17:0/299873048 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff4100c5100 msgr2=0x7ff41014b6e0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:46:11.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:11.282+0000 7ff40cc00640 1 --2- 172.21.15.17:0/299873048 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff4100c5100 0x7ff41014b6e0 secure :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7ff40800e9c0 tx=0x7ff40800ee80 comp rx=0 tx=0).stop 2024-09-10T21:46:11.288 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-10T21:46:11.288 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:46:11.288 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:46:11.288 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-10T21:46:11.288 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 162, 2024-09-10T21:46:11.288 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 49699, 2024-09-10T21:46:11.288 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-10T21:46:11.288 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-10T21:46:11.288 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-10T21:46:11.288 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:46:11.289 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 4, 2024-09-10T21:46:11.289 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:46:11.289 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-10T21:46:11.289 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-10T21:46:11.289 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:46:11.289 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:46:11.289 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:46:11.289 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:46:11.289 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:46:11.289 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:46:11.289 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:46:11.289 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:46:11.289 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:46:11.289 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:46:11.290 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:46:11.290 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:46:11.290 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-10T21:46:11.290 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-10T21:46:11.290 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:46:11.290 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:46:11.290 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:46:11.290 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:46:11.290 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:46:11.290 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:46:11.290 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:46:11.290 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:46:11.290 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:46:11.290 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:46:11.291 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:46:11.291 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:46:11.291 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:46:11.291 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-10T21:46:11.291 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-10T21:46:11.291 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-10T21:46:11.291 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-10T21:46:11.291 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-10T21:46:11.291 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "5e2f023f-d6a6-49f4-a256-f4807dce3739", 2024-09-10T21:46:11.291 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:46:11.291 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:46:11.291 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-10T21:46:11.291 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-10T21:46:11.291 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-10T21:46:11.292 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-10T21:46:11.292 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-10T21:46:11.292 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-10T21:46:11.292 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-10T21:46:11.292 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:46:11.292 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-10T21:46:11.292 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:46:11.292 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:46:11.292 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:46:11.292 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:46:11.292 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:46:11.292 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:46:11.292 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:46:11.292 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:46:11.293 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:46:11.293 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:46:11.293 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:46:11.293 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:46:11.293 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-10T21:46:11.293 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:46:11.293 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-10T21:46:11.293 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:46:11.293 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:46:11.293 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:46:11.293 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:46:11.294 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:46:11.294 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:46:11.294 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:46:11.294 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-10T21:46:11.294 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:46:11.294 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:46:11.294 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:46:11.294 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:46:11.294 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-10T21:46:11.294 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:46:11.294 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:46:11.294 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:46:11.294 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:46:11.295 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-10T21:46:11.295 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-10T21:46:11.295 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:46:11.295 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:46:11.295 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:46:11.295 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:46:11.295 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:46:11.295 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:46:11.295 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:46:11.295 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:46:11.295 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:46:11.296 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:46:11.296 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:46:11.296 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3300", 2024-09-10T21:46:11.296 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:46:11.296 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:46:11.296 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:46:11.296 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:46:11.296 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789", 2024-09-10T21:46:11.296 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:46:11.296 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:46:11.296 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:46:11.296 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:46:11.296 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789/0", 2024-09-10T21:46:11.297 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6789/0", 2024-09-10T21:46:11.297 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:46:11.297 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:46:11.297 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:46:11.297 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:46:11.298 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:46:11.298 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:46:11.298 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:46:11.298 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:46:11.298 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:46:11.298 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:46:11.298 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:46:11.298 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3301", 2024-09-10T21:46:11.298 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:46:11.298 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:46:11.298 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:46:11.298 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:46:11.298 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790", 2024-09-10T21:46:11.298 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:46:11.299 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:46:11.299 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:46:11.299 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:46:11.299 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790/0", 2024-09-10T21:46:11.299 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6790/0", 2024-09-10T21:46:11.299 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:46:11.299 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:46:11.299 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:46:11.299 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:46:11.299 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:46:11.299 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:46:11.299 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-10T21:46:11.299 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-10T21:46:11.299 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:46:11.300 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:46:11.300 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:46:11.300 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:46:11.300 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:46:11.300 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:46:11.300 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": [ 2024-09-10T21:46:11.300 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:46:11.300 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:46:11.300 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:46:11.300 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 3 2024-09-10T21:46:11.300 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:46:11.300 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:46:11.301 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-10T21:46:11.301 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:46:11.301 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:46:11.301 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:46:11.301 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 2 2024-09-10T21:46:11.301 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:46:11.301 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:46:11.301 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:46:11.301 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-10T21:46:11.301 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-10T21:46:11.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:11.286+0000 7ff417c00640 1 -- 172.21.15.17:0/299873048 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7ff3f0072050 msgr2=0x7ff3f0074510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:11.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:11.286+0000 7ff417c00640 1 --2- 172.21.15.17:0/299873048 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7ff3f0072050 0x7ff3f0074510 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7ff4100c5050 tx=0x7ff3fc009030 comp rx=0 tx=0).stop 2024-09-10T21:46:11.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:11.286+0000 7ff415c00640 1 -- 172.21.15.17:0/299873048 reap_dead start 2024-09-10T21:46:11.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:11.286+0000 7ff417c00640 1 -- 172.21.15.17:0/299873048 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff41006dc30 msgr2=0x7ff410157550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:11.302 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:11.286+0000 7ff417c00640 1 --2- 172.21.15.17:0/299873048 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff41006dc30 0x7ff410157550 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ff41015b650 tx=0x7ff404005a50 comp rx=0 tx=0).stop 2024-09-10T21:46:11.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:11.286+0000 7ff417c00640 1 -- 172.21.15.17:0/299873048 shutdown_connections 2024-09-10T21:46:11.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:11.286+0000 7ff417c00640 1 --2- 172.21.15.17:0/299873048 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff41006dc30 0x7ff410157550 secure :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7ff41015b650 tx=0x7ff404005a50 comp rx=0 tx=0).stop 2024-09-10T21:46:11.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:11.286+0000 7ff417c00640 1 -- 172.21.15.17:0/299873048 >> 172.21.15.17:0/299873048 conn(0x7ff4100700d0 msgr2=0x7ff4100cb600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:46:11.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:11.294+0000 7ff417c00640 1 -- 172.21.15.17:0/299873048 shutdown_connections 2024-09-10T21:46:11.303 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:11.294+0000 7ff417c00640 1 -- 172.21.15.17:0/299873048 wait complete. 2024-09-10T21:46:11.334 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 2 2024-09-10T21:46:11.334 INFO:tasks.mon_thrash.mon_thrasher:waiting for 20.0 secs before reviving monitors 2024-09-10T21:46:27.019 INFO:tasks.workunit.client.0.smithi017.stdout:Syncing()...14 sec 2024-09-10T21:46:27.019 INFO:tasks.workunit.client.0.smithi017.stdout:FFSB benchmark finished at: Tue Sep 10 21:46:27 2024 2024-09-10T21:46:27.019 INFO:tasks.workunit.client.0.smithi017.stdout: 2024-09-10T21:46:27.019 INFO:tasks.workunit.client.0.smithi017.stdout:Results: 2024-09-10T21:46:27.019 INFO:tasks.workunit.client.0.smithi017.stdout:Benchmark took 327.78 sec 2024-09-10T21:46:27.020 INFO:tasks.workunit.client.0.smithi017.stdout: 2024-09-10T21:46:27.020 INFO:tasks.workunit.client.0.smithi017.stdout:Total Results 2024-09-10T21:46:27.020 INFO:tasks.workunit.client.0.smithi017.stdout:=============== 2024-09-10T21:46:27.020 INFO:tasks.workunit.client.0.smithi017.stdout: Op Name Transactions Trans/sec % Trans % Op Weight Throughput 2024-09-10T21:46:27.020 INFO:tasks.workunit.client.0.smithi017.stdout: ======= ============ ========= ======= =========== ========== 2024-09-10T21:46:27.020 INFO:tasks.workunit.client.0.smithi017.stdout: write : 723200 2206.33 100.000% 100.000% 8.62MB/sec 2024-09-10T21:46:27.020 INFO:tasks.workunit.client.0.smithi017.stdout:- 2024-09-10T21:46:27.020 INFO:tasks.workunit.client.0.smithi017.stdout:2206.33 Transactions per Second 2024-09-10T21:46:27.020 INFO:tasks.workunit.client.0.smithi017.stdout: 2024-09-10T21:46:27.020 INFO:tasks.workunit.client.0.smithi017.stdout:Throughput Results 2024-09-10T21:46:27.020 INFO:tasks.workunit.client.0.smithi017.stdout:=================== 2024-09-10T21:46:27.020 INFO:tasks.workunit.client.0.smithi017.stdout:Write Throughput: 8.62MB/sec 2024-09-10T21:46:27.027 INFO:tasks.workunit.client.0.smithi017.stdout: 2024-09-10T21:46:27.027 INFO:tasks.workunit.client.0.smithi017.stdout:System Call Latency statistics in millisecs 2024-09-10T21:46:27.028 INFO:tasks.workunit.client.0.smithi017.stdout:===== 2024-09-10T21:46:27.028 INFO:tasks.workunit.client.0.smithi017.stdout: Min Avg Max Total Calls 2024-09-10T21:46:27.028 INFO:tasks.workunit.client.0.smithi017.stdout: ======== ======== ======== ============ 2024-09-10T21:46:27.028 INFO:tasks.workunit.client.0.smithi017.stdout:[ open] 0.781000 202.148441 6766.895020 565 2024-09-10T21:46:27.028 INFO:tasks.workunit.client.0.smithi017.stdout: msec_range[0] 5000.000000 - 10000.000000 : 6 2024-09-10T21:46:27.028 INFO:tasks.workunit.client.0.smithi017.stdout: msec_range[1] 2000.000000 - 5000.000000 : 13 2024-09-10T21:46:27.028 INFO:tasks.workunit.client.0.smithi017.stdout: msec_range[2] 1000.000000 - 2000.000000 : 3 2024-09-10T21:46:27.028 INFO:tasks.workunit.client.0.smithi017.stdout: msec_range[3] 500.000000 - 1000.000000 : 4 2024-09-10T21:46:27.028 INFO:tasks.workunit.client.0.smithi017.stdout: msec_range[4] 200.000000 - 500.000000 : 1 2024-09-10T21:46:27.028 INFO:tasks.workunit.client.0.smithi017.stdout: msec_range[5] 100.000000 - 200.000000 : 2 2024-09-10T21:46:27.028 INFO:tasks.workunit.client.0.smithi017.stdout: msec_range[6] 50.000000 - 100.000000 : 99 2024-09-10T21:46:27.028 INFO:tasks.workunit.client.0.smithi017.stdout: msec_range[7] 20.000000 - 50.000000 : 235 2024-09-10T21:46:27.028 INFO:tasks.workunit.client.0.smithi017.stdout: msec_range[8] 10.000000 - 20.000000 : 127 2024-09-10T21:46:27.029 INFO:tasks.workunit.client.0.smithi017.stdout: msec_range[9] 5.000000 - 10.000000 : 54 2024-09-10T21:46:27.029 INFO:tasks.workunit.client.0.smithi017.stdout: msec_range[10] 2.000000 - 5.000000 : 13 2024-09-10T21:46:27.029 INFO:tasks.workunit.client.0.smithi017.stdout: msec_range[11] 1.000000 - 2.000000 : 6 2024-09-10T21:46:27.029 INFO:tasks.workunit.client.0.smithi017.stdout: msec_range[12] 0.500000 - 1.000000 : 2 2024-09-10T21:46:27.029 INFO:tasks.workunit.client.0.smithi017.stdout: msec_range[13] 0.200000 - 0.500000 : 0 2024-09-10T21:46:27.029 INFO:tasks.workunit.client.0.smithi017.stdout: msec_range[14] 0.100000 - 0.200000 : 0 2024-09-10T21:46:27.029 INFO:tasks.workunit.client.0.smithi017.stdout: msec_range[15] 0.050000 - 0.100000 : 0 2024-09-10T21:46:27.029 INFO:tasks.workunit.client.0.smithi017.stdout: msec_range[16] 0.020000 - 0.050000 : 0 2024-09-10T21:46:27.029 INFO:tasks.workunit.client.0.smithi017.stdout: msec_range[17] 0.010000 - 0.020000 : 0 2024-09-10T21:46:27.029 INFO:tasks.workunit.client.0.smithi017.stdout: msec_range[18] 0.000000 - 0.010000 : 0 2024-09-10T21:46:27.029 INFO:tasks.workunit.client.0.smithi017.stdout: 2024-09-10T21:46:27.029 INFO:tasks.workunit.client.0.smithi017.stdout:[ write] 0.163000 13.471963 700.559998 723200 2024-09-10T21:46:27.029 INFO:tasks.workunit.client.0.smithi017.stdout: msec_range[0] 5000.000000 - 10000.000000 : 0 2024-09-10T21:46:27.029 INFO:tasks.workunit.client.0.smithi017.stdout: msec_range[1] 2000.000000 - 5000.000000 : 0 2024-09-10T21:46:27.029 INFO:tasks.workunit.client.0.smithi017.stdout: msec_range[2] 1000.000000 - 2000.000000 : 0 2024-09-10T21:46:27.030 INFO:tasks.workunit.client.0.smithi017.stdout: msec_range[3] 500.000000 - 1000.000000 : 80 2024-09-10T21:46:27.030 INFO:tasks.workunit.client.0.smithi017.stdout: msec_range[4] 200.000000 - 500.000000 : 1041 2024-09-10T21:46:27.030 INFO:tasks.workunit.client.0.smithi017.stdout: msec_range[5] 100.000000 - 200.000000 : 4051 2024-09-10T21:46:27.030 INFO:tasks.workunit.client.0.smithi017.stdout: msec_range[6] 50.000000 - 100.000000 : 30924 2024-09-10T21:46:27.030 INFO:tasks.workunit.client.0.smithi017.stdout: msec_range[7] 20.000000 - 50.000000 : 120467 2024-09-10T21:46:27.030 INFO:tasks.workunit.client.0.smithi017.stdout: msec_range[8] 10.000000 - 20.000000 : 121107 2024-09-10T21:46:27.030 INFO:tasks.workunit.client.0.smithi017.stdout: msec_range[9] 5.000000 - 10.000000 : 107565 2024-09-10T21:46:27.030 INFO:tasks.workunit.client.0.smithi017.stdout: msec_range[10] 2.000000 - 5.000000 : 105638 2024-09-10T21:46:27.030 INFO:tasks.workunit.client.0.smithi017.stdout: msec_range[11] 1.000000 - 2.000000 : 70458 2024-09-10T21:46:27.030 INFO:tasks.workunit.client.0.smithi017.stdout: msec_range[12] 0.500000 - 1.000000 : 87181 2024-09-10T21:46:27.030 INFO:tasks.workunit.client.0.smithi017.stdout: msec_range[13] 0.200000 - 0.500000 : 73674 2024-09-10T21:46:27.030 INFO:tasks.workunit.client.0.smithi017.stdout: msec_range[14] 0.100000 - 0.200000 : 1014 2024-09-10T21:46:27.030 INFO:tasks.workunit.client.0.smithi017.stdout: msec_range[15] 0.050000 - 0.100000 : 0 2024-09-10T21:46:27.030 INFO:tasks.workunit.client.0.smithi017.stdout: msec_range[16] 0.020000 - 0.050000 : 0 2024-09-10T21:46:27.030 INFO:tasks.workunit.client.0.smithi017.stdout: msec_range[17] 0.010000 - 0.020000 : 0 2024-09-10T21:46:27.031 INFO:tasks.workunit.client.0.smithi017.stdout: msec_range[18] 0.000000 - 0.010000 : 0 2024-09-10T21:46:27.031 INFO:tasks.workunit.client.0.smithi017.stdout: 2024-09-10T21:46:27.031 INFO:tasks.workunit.client.0.smithi017.stdout:[ lseek] 0.000000 0.002052 11.479000 723200 2024-09-10T21:46:27.031 INFO:tasks.workunit.client.0.smithi017.stdout: msec_range[0] 5000.000000 - 10000.000000 : 0 2024-09-10T21:46:27.031 INFO:tasks.workunit.client.0.smithi017.stdout: msec_range[1] 2000.000000 - 5000.000000 : 0 2024-09-10T21:46:27.031 INFO:tasks.workunit.client.0.smithi017.stdout: msec_range[2] 1000.000000 - 2000.000000 : 0 2024-09-10T21:46:27.031 INFO:tasks.workunit.client.0.smithi017.stdout: msec_range[3] 500.000000 - 1000.000000 : 0 2024-09-10T21:46:27.031 INFO:tasks.workunit.client.0.smithi017.stdout: msec_range[4] 200.000000 - 500.000000 : 0 2024-09-10T21:46:27.031 INFO:tasks.workunit.client.0.smithi017.stdout: msec_range[5] 100.000000 - 200.000000 : 0 2024-09-10T21:46:27.031 INFO:tasks.workunit.client.0.smithi017.stdout: msec_range[6] 50.000000 - 100.000000 : 0 2024-09-10T21:46:27.031 INFO:tasks.workunit.client.0.smithi017.stdout: msec_range[7] 20.000000 - 50.000000 : 0 2024-09-10T21:46:27.031 INFO:tasks.workunit.client.0.smithi017.stdout: msec_range[8] 10.000000 - 20.000000 : 1 2024-09-10T21:46:27.031 INFO:tasks.workunit.client.0.smithi017.stdout: msec_range[9] 5.000000 - 10.000000 : 8 2024-09-10T21:46:27.031 INFO:tasks.workunit.client.0.smithi017.stdout: msec_range[10] 2.000000 - 5.000000 : 13 2024-09-10T21:46:27.031 INFO:tasks.workunit.client.0.smithi017.stdout: msec_range[11] 1.000000 - 2.000000 : 35 2024-09-10T21:46:27.031 INFO:tasks.workunit.client.0.smithi017.stdout: msec_range[12] 0.500000 - 1.000000 : 82 2024-09-10T21:46:27.032 INFO:tasks.workunit.client.0.smithi017.stdout: msec_range[13] 0.200000 - 0.500000 : 127 2024-09-10T21:46:27.032 INFO:tasks.workunit.client.0.smithi017.stdout: msec_range[14] 0.100000 - 0.200000 : 133 2024-09-10T21:46:27.032 INFO:tasks.workunit.client.0.smithi017.stdout: msec_range[15] 0.050000 - 0.100000 : 191 2024-09-10T21:46:27.032 INFO:tasks.workunit.client.0.smithi017.stdout: msec_range[16] 0.020000 - 0.050000 : 770 2024-09-10T21:46:27.032 INFO:tasks.workunit.client.0.smithi017.stdout: msec_range[17] 0.010000 - 0.020000 : 385 2024-09-10T21:46:27.032 INFO:tasks.workunit.client.0.smithi017.stdout: msec_range[18] 0.000000 - 0.010000 : 721455 2024-09-10T21:46:27.032 INFO:tasks.workunit.client.0.smithi017.stdout: 2024-09-10T21:46:27.032 INFO:tasks.workunit.client.0.smithi017.stdout:[ close] 0.036000 4.674860 115.752998 565 2024-09-10T21:46:27.032 INFO:tasks.workunit.client.0.smithi017.stdout: msec_range[0] 5000.000000 - 10000.000000 : 0 2024-09-10T21:46:27.032 INFO:tasks.workunit.client.0.smithi017.stdout: msec_range[1] 2000.000000 - 5000.000000 : 0 2024-09-10T21:46:27.032 INFO:tasks.workunit.client.0.smithi017.stdout: msec_range[2] 1000.000000 - 2000.000000 : 0 2024-09-10T21:46:27.032 INFO:tasks.workunit.client.0.smithi017.stdout: msec_range[3] 500.000000 - 1000.000000 : 0 2024-09-10T21:46:27.032 INFO:tasks.workunit.client.0.smithi017.stdout: msec_range[4] 200.000000 - 500.000000 : 0 2024-09-10T21:46:27.033 INFO:tasks.workunit.client.0.smithi017.stdout: msec_range[5] 100.000000 - 200.000000 : 1 2024-09-10T21:46:27.033 INFO:tasks.workunit.client.0.smithi017.stdout: msec_range[6] 50.000000 - 100.000000 : 9 2024-09-10T21:46:27.033 INFO:tasks.workunit.client.0.smithi017.stdout: msec_range[7] 20.000000 - 50.000000 : 26 2024-09-10T21:46:27.033 INFO:tasks.workunit.client.0.smithi017.stdout: msec_range[8] 10.000000 - 20.000000 : 39 2024-09-10T21:46:27.033 INFO:tasks.workunit.client.0.smithi017.stdout: msec_range[9] 5.000000 - 10.000000 : 41 2024-09-10T21:46:27.033 INFO:tasks.workunit.client.0.smithi017.stdout: msec_range[10] 2.000000 - 5.000000 : 63 2024-09-10T21:46:27.033 INFO:tasks.workunit.client.0.smithi017.stdout: msec_range[11] 1.000000 - 2.000000 : 54 2024-09-10T21:46:27.033 INFO:tasks.workunit.client.0.smithi017.stdout: msec_range[12] 0.500000 - 1.000000 : 66 2024-09-10T21:46:27.033 INFO:tasks.workunit.client.0.smithi017.stdout: msec_range[13] 0.200000 - 0.500000 : 110 2024-09-10T21:46:27.033 INFO:tasks.workunit.client.0.smithi017.stdout: msec_range[14] 0.100000 - 0.200000 : 64 2024-09-10T21:46:27.033 INFO:tasks.workunit.client.0.smithi017.stdout: msec_range[15] 0.050000 - 0.100000 : 88 2024-09-10T21:46:27.033 INFO:tasks.workunit.client.0.smithi017.stdout: msec_range[16] 0.020000 - 0.050000 : 4 2024-09-10T21:46:27.033 INFO:tasks.workunit.client.0.smithi017.stdout: msec_range[17] 0.010000 - 0.020000 : 0 2024-09-10T21:46:27.034 INFO:tasks.workunit.client.0.smithi017.stdout: msec_range[18] 0.000000 - 0.010000 : 0 2024-09-10T21:46:27.034 INFO:tasks.workunit.client.0.smithi017.stdout: 2024-09-10T21:46:27.034 INFO:tasks.workunit.client.0.smithi017.stdout: 2024-09-10T21:46:27.034 INFO:tasks.workunit.client.0.smithi017.stdout: 2024-09-10T21:46:27.034 INFO:tasks.workunit.client.0.smithi017.stdout: 2024-09-10T21:46:27.034 INFO:tasks.workunit.client.0.smithi017.stdout:0.4% User Time 2024-09-10T21:46:27.034 INFO:tasks.workunit.client.0.smithi017.stdout:26.9% System Time 2024-09-10T21:46:27.034 INFO:tasks.workunit.client.0.smithi017.stdout:27.3% CPU Utilization 2024-09-10T21:46:27.034 INFO:tasks.workunit.client.0.smithi017.stderr:+ cd .. 2024-09-10T21:46:27.043 INFO:tasks.workunit.client.0.smithi017.stderr:+ rm -r tmp ffsb 2024-09-10T21:46:29.842 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-09-10T21:46:29.842 DEBUG:teuthology.orchestra.run.smithi017:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-09-10T21:46:29.870 INFO:tasks.workunit:Stopping ['suites/ffsb.sh'] on client.0... 2024-09-10T21:46:29.871 DEBUG:teuthology.orchestra.run.smithi017:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-09-10T21:46:31.140 DEBUG:teuthology.parallel:result is None 2024-09-10T21:46:31.140 DEBUG:teuthology.orchestra.run.smithi017:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-09-10T21:46:31.224 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-09-10T21:46:31.225 DEBUG:teuthology.run_tasks:Unwinding manager mon_thrash 2024-09-10T21:46:31.235 INFO:tasks.mon_thrash:joining mon_thrasher 2024-09-10T21:46:31.335 INFO:tasks.mon_thrash.mon_thrasher:reviving mon.a 2024-09-10T21:46:31.335 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-10T21:46:31.335 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-10T21:46:31.342 INFO:tasks.ceph.mon.a:Started 2024-09-10T21:46:31.342 INFO:tasks.mon_thrash:Sending STOP to mon b 2024-09-10T21:46:31.342 INFO:tasks.ceph.mon.b:Sent signal 19 2024-09-10T21:46:31.342 INFO:tasks.mon_thrash.mon_thrasher:waiting for 15.0 secs to unfreeze mons 2024-09-10T21:46:33.770 DEBUG:teuthology.orchestra.run.smithi017:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:46:33.776 DEBUG:teuthology.orchestra.run.smithi031:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-10T21:46:33.857 INFO:tasks.ceph.osd.1.smithi017.stderr:2024-09-10T21:46:33.853+0000 7f7e5ae00640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 78282) UID: 0 2024-09-10T21:46:33.858 INFO:tasks.ceph.osd.2.smithi017.stderr:2024-09-10T21:46:33.853+0000 7feb7a600640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 78282) UID: 0 2024-09-10T21:46:33.859 INFO:tasks.ceph.osd.3.smithi017.stderr:2024-09-10T21:46:33.853+0000 7f7d76a00640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 78282) UID: 0 2024-09-10T21:46:33.877 INFO:tasks.ceph.osd.0.smithi017.stderr:2024-09-10T21:46:33.853+0000 7f740f600640 -1 received signal: Hangup from killall ceph-osd -1 (PID: 78282) UID: 0 2024-09-10T21:46:46.343 INFO:tasks.mon_thrash:Sending CONT to mon b 2024-09-10T21:46:46.344 INFO:tasks.ceph.mon.b:Sent signal 18 2024-09-10T21:46:46.344 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-09-10T21:46:46.345 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-10T21:46:51.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:51.373+0000 7f4d47000640 1 -- 172.21.15.17:0/1837875524 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4d24004810 msgr2=0x7f4d24004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:51.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:51.373+0000 7f4d47000640 1 --2- 172.21.15.17:0/1837875524 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4d24004810 0x7f4d24004c90 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f4d38006450 tx=0x7f4d3801dc70 comp rx=0 tx=0).stop 2024-09-10T21:46:51.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:51.373+0000 7f4d47000640 1 -- 172.21.15.17:0/1837875524 shutdown_connections 2024-09-10T21:46:51.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:51.373+0000 7f4d47000640 1 --2- 172.21.15.17:0/1837875524 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d240053d0 0x7f4d2400b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:51.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:51.373+0000 7f4d47000640 1 --2- 172.21.15.17:0/1837875524 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4d24004810 0x7f4d24004c90 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:51.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:51.373+0000 7f4d47000640 1 --2- 172.21.15.17:0/1837875524 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f4d24003a10 0x7f4d24003e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:51.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:51.373+0000 7f4d47000640 1 -- 172.21.15.17:0/1837875524 >> 172.21.15.17:0/1837875524 conn(0x7f4d40070160 msgr2=0x7f4d40070570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:46:51.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:51.373+0000 7f4d47000640 1 -- 172.21.15.17:0/1837875524 shutdown_connections 2024-09-10T21:46:51.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:51.373+0000 7f4d47000640 1 -- 172.21.15.17:0/1837875524 wait complete. 2024-09-10T21:46:51.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:51.373+0000 7f4d47000640 1 Processor -- start 2024-09-10T21:46:51.379 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:51.373+0000 7f4d47000640 1 -- start start 2024-09-10T21:46:51.381 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:51.377+0000 7f4d47000640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f4d24003a10 0x7f4d401ee8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:51.381 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:51.377+0000 7f4d47000640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4d24004810 0x7f4d40145c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:51.381 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:51.377+0000 7f4d47000640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d240053d0 0x7f4d40144820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:51.381 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:51.377+0000 7f4d47000640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f4d4015ba00 con 0x7f4d24004810 2024-09-10T21:46:51.381 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:51.377+0000 7f4d47000640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f4d4006dca0 con 0x7f4d240053d0 2024-09-10T21:46:51.381 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:51.377+0000 7f4d47000640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f4d400c6c90 con 0x7f4d24003a10 2024-09-10T21:46:51.381 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:51.377+0000 7f4d3f400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4d24004810 0x7f4d40145c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:51.381 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:51.377+0000 7f4d3f400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4d24004810 0x7f4d40145c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:54338/0 (socket says 172.21.15.17:54338) 2024-09-10T21:46:51.381 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:51.377+0000 7f4d3f400640 1 -- 172.21.15.17:0/65758535 learned_addr learned my addr 172.21.15.17:0/65758535 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:46:51.381 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:51.377+0000 7f4d3fe00640 1 --2- 172.21.15.17:0/65758535 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f4d24003a10 0x7f4d401ee8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:51.381 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:51.377+0000 7f4d45000640 1 --2- 172.21.15.17:0/65758535 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d240053d0 0x7f4d40144820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:51.381 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:51.377+0000 7f4d3f400640 1 -- 172.21.15.17:0/65758535 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4d24004810 msgr2=0x7f4d40145c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 17 2024-09-10T21:46:51.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:51.377+0000 7f4d3f400640 1 -- 172.21.15.17:0/65758535 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4d24004810 msgr2=0x7f4d40145c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-10T21:46:51.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:51.377+0000 7f4d3f400640 1 --2- 172.21.15.17:0/65758535 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4d24004810 0x7f4d40145c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T21:46:51.382 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:51.377+0000 7f4d3f400640 1 --2- 172.21.15.17:0/65758535 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4d24004810 0x7f4d40145c00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T21:46:51.383 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:51.377+0000 7f4d3fe00640 1 -- 172.21.15.17:0/65758535 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d240053d0 msgr2=0x7f4d40144820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:51.383 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:51.377+0000 7f4d3fe00640 1 --2- 172.21.15.17:0/65758535 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d240053d0 0x7f4d40144820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:51.383 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:51.377+0000 7f4d3fe00640 1 -- 172.21.15.17:0/65758535 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4d24004810 msgr2=0x7f4d40145c00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T21:46:51.383 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:51.377+0000 7f4d3fe00640 1 --2- 172.21.15.17:0/65758535 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4d24004810 0x7f4d40145c00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:51.383 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:51.377+0000 7f4d3fe00640 1 -- 172.21.15.17:0/65758535 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d4014fe90 con 0x7f4d24003a10 2024-09-10T21:46:51.383 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:51.377+0000 7f4d45000640 1 --2- 172.21.15.17:0/65758535 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d240053d0 0x7f4d40144820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:46:51.383 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:51.377+0000 7f4d3fe00640 1 --2- 172.21.15.17:0/65758535 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f4d24003a10 0x7f4d401ee8a0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f4d2c005e60 tx=0x7f4d2c012f00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:46:51.383 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:51.377+0000 7f4d37e00640 1 -- 172.21.15.17:0/65758535 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d2c01d440 con 0x7f4d24003a10 2024-09-10T21:46:51.383 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:51.377+0000 7f4d37e00640 1 -- 172.21.15.17:0/65758535 <== mon.2 v2:172.21.15.31:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4d2c01d5e0 con 0x7f4d24003a10 2024-09-10T21:46:51.383 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:51.377+0000 7f4d37e00640 1 -- 172.21.15.17:0/65758535 <== mon.2 v2:172.21.15.31:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d2c01da30 con 0x7f4d24003a10 2024-09-10T21:46:51.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:51.377+0000 7f4d47000640 1 -- 172.21.15.17:0/65758535 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d400c9ec0 con 0x7f4d24003a10 2024-09-10T21:46:51.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:51.377+0000 7f4d47000640 1 -- 172.21.15.17:0/65758535 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d400ca480 con 0x7f4d24003a10 2024-09-10T21:46:51.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:51.377+0000 7f4d47000640 1 -- 172.21.15.17:0/65758535 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d400c7180 con 0x7f4d24003a10 2024-09-10T21:46:51.384 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:51.381+0000 7f4d37e00640 1 -- 172.21.15.17:0/65758535 <== mon.2 v2:172.21.15.31:3301/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f4d2c01dbd0 con 0x7f4d24003a10 2024-09-10T21:46:51.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:51.381+0000 7f4d37e00640 1 --2- 172.21.15.17:0/65758535 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f4d20075bd0 0x7f4d20078090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:51.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:51.381+0000 7f4d37e00640 1 -- 172.21.15.17:0/65758535 <== mon.2 v2:172.21.15.31:3301/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f4d2c0953f0 con 0x7f4d24003a10 2024-09-10T21:46:51.385 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:51.381+0000 7f4d3f400640 1 --2- 172.21.15.17:0/65758535 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f4d20075bd0 0x7f4d20078090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:51.386 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:51.381+0000 7f4d3f400640 1 --2- 172.21.15.17:0/65758535 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f4d20075bd0 0x7f4d20078090 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f4d380063a0 tx=0x7f4d38006040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:46:51.390 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:51.385+0000 7f4d37e00640 1 -- 172.21.15.17:0/65758535 <== mon.2 v2:172.21.15.31:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f4d2c060630 con 0x7f4d24003a10 2024-09-10T21:46:51.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:51.633+0000 7f4d47000640 1 -- 172.21.15.17:0/65758535 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f4d40151c20 con 0x7f4d24003a10 2024-09-10T21:46:51.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:51.633+0000 7f4d37e00640 1 -- 172.21.15.17:0/65758535 <== mon.2 v2:172.21.15.31:3301/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1551 (secure 0 0 0) 0x7f4d2c05ff30 con 0x7f4d24003a10 2024-09-10T21:46:51.637 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":166,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":0,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5e2f023f-d6a6-49f4-a256-f4807dce3739","modified":"2024-09-10T21:23:46.892130Z","created":"2024-09-10T21:23:46.892130Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3301","nonce":0},{"type":"v1","addr":"172.21.15.31:6790","nonce":0}]},"addr":"172.21.15.31:6790/0","public_addr":"172.21.15.31:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-10T21:46:51.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:51.633+0000 7f4d47000640 1 -- 172.21.15.17:0/65758535 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f4d20075bd0 msgr2=0x7f4d20078090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:51.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:51.633+0000 7f4d47000640 1 --2- 172.21.15.17:0/65758535 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f4d20075bd0 0x7f4d20078090 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f4d380063a0 tx=0x7f4d38006040 comp rx=0 tx=0).stop 2024-09-10T21:46:51.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:51.633+0000 7f4d47000640 1 -- 172.21.15.17:0/65758535 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f4d24003a10 msgr2=0x7f4d401ee8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:51.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:51.633+0000 7f4d47000640 1 --2- 172.21.15.17:0/65758535 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f4d24003a10 0x7f4d401ee8a0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f4d2c005e60 tx=0x7f4d2c012f00 comp rx=0 tx=0).stop 2024-09-10T21:46:51.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:51.633+0000 7f4d47000640 1 -- 172.21.15.17:0/65758535 shutdown_connections 2024-09-10T21:46:51.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:51.633+0000 7f4d47000640 1 --2- 172.21.15.17:0/65758535 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f4d20075bd0 0x7f4d20078090 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:51.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:51.633+0000 7f4d47000640 1 --2- 172.21.15.17:0/65758535 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f4d240053d0 0x7f4d40144820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:51.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:51.633+0000 7f4d47000640 1 --2- 172.21.15.17:0/65758535 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f4d24004810 0x7f4d40145c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:51.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:51.633+0000 7f4d47000640 1 --2- 172.21.15.17:0/65758535 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f4d24003a10 0x7f4d401ee8a0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:51.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:51.633+0000 7f4d47000640 1 -- 172.21.15.17:0/65758535 >> 172.21.15.17:0/65758535 conn(0x7f4d40070160 msgr2=0x7f4d40070540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:46:51.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:51.633+0000 7f4d47000640 1 -- 172.21.15.17:0/65758535 shutdown_connections 2024-09-10T21:46:51.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:51.633+0000 7f4d47000640 1 -- 172.21.15.17:0/65758535 wait complete. 2024-09-10T21:46:54.660 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-10T21:46:57.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:57.821+0000 7fbfb2600640 1 -- 172.21.15.17:0/3163211919 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbf8c007290 msgr2=0x7fbf8c009750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:57.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:57.821+0000 7fbfb2600640 1 --2- 172.21.15.17:0/3163211919 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbf8c007290 0x7fbf8c009750 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fbf98001790 tx=0x7fbf9801c9a0 comp rx=0 tx=0).stop 2024-09-10T21:46:57.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:57.821+0000 7fbfb1c00640 1 -- 172.21.15.17:0/3163211919 reap_dead start 2024-09-10T21:46:57.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:57.821+0000 7fbfb2600640 1 -- 172.21.15.17:0/3163211919 shutdown_connections 2024-09-10T21:46:57.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:57.821+0000 7fbfb2600640 1 --2- 172.21.15.17:0/3163211919 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbf8c007290 0x7fbf8c009750 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:57.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:57.821+0000 7fbfb2600640 1 -- 172.21.15.17:0/3163211919 >> 172.21.15.17:0/3163211919 conn(0x7fbfac074ac0 msgr2=0x7fbfac068990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:46:57.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:57.821+0000 7fbfb2600640 1 -- 172.21.15.17:0/3163211919 shutdown_connections 2024-09-10T21:46:57.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:57.821+0000 7fbfb2600640 1 -- 172.21.15.17:0/3163211919 wait complete. 2024-09-10T21:46:57.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:57.821+0000 7fbfb2600640 1 Processor -- start 2024-09-10T21:46:57.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:57.821+0000 7fbfb2600640 1 -- start start 2024-09-10T21:46:57.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:57.825+0000 7fbfb2600640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbf8c007290 0x7fbfac14da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:57.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:57.825+0000 7fbfb2600640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbfac14dfd0 0x7fbfac1551c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:57.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:57.825+0000 7fbfb2600640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fbfac155700 0x7fbfac0c9430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:57.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:57.825+0000 7fbfb2600640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fbfac1526b0 con 0x7fbfac14dfd0 2024-09-10T21:46:57.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:57.825+0000 7fbfb2600640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fbfac152530 con 0x7fbf8c007290 2024-09-10T21:46:57.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:57.825+0000 7fbfb2600640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7fbfac1653d0 con 0x7fbfac155700 2024-09-10T21:46:57.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:57.825+0000 7fbfabe00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbfac14dfd0 0x7fbfac1551c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:57.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:57.825+0000 7fbfabe00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbfac14dfd0 0x7fbfac1551c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37060/0 (socket says 172.21.15.17:37060) 2024-09-10T21:46:57.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:57.825+0000 7fbfabe00640 1 -- 172.21.15.17:0/439682372 learned_addr learned my addr 172.21.15.17:0/439682372 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:46:57.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:57.825+0000 7fbfb1200640 1 --2- 172.21.15.17:0/439682372 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbf8c007290 0x7fbfac14da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:57.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:57.825+0000 7fbfb1c00640 1 --2- 172.21.15.17:0/439682372 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fbfac155700 0x7fbfac0c9430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:57.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:57.825+0000 7fbfabe00640 1 -- 172.21.15.17:0/439682372 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fbfac155700 msgr2=0x7fbfac0c9430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:57.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:57.825+0000 7fbfabe00640 1 --2- 172.21.15.17:0/439682372 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fbfac155700 0x7fbfac0c9430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:57.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:57.825+0000 7fbfabe00640 1 -- 172.21.15.17:0/439682372 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbf8c007290 msgr2=0x7fbfac14da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:57.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:57.825+0000 7fbfabe00640 1 --2- 172.21.15.17:0/439682372 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbf8c007290 0x7fbfac14da90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:57.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:57.825+0000 7fbfabe00640 1 -- 172.21.15.17:0/439682372 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfac152c40 con 0x7fbfac14dfd0 2024-09-10T21:46:57.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:57.825+0000 7fbfb1200640 1 --2- 172.21.15.17:0/439682372 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbf8c007290 0x7fbfac14da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:46:57.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:57.825+0000 7fbfabe00640 1 --2- 172.21.15.17:0/439682372 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbfac14dfd0 0x7fbfac1551c0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fbf98000c10 tx=0x7fbf98006720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:46:57.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:57.825+0000 7fbfa8c00640 1 -- 172.21.15.17:0/439682372 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf98006330 con 0x7fbfac14dfd0 2024-09-10T21:46:57.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:57.825+0000 7fbfa8c00640 1 -- 172.21.15.17:0/439682372 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbf980b4070 con 0x7fbfac14dfd0 2024-09-10T21:46:57.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:57.825+0000 7fbfa8c00640 1 -- 172.21.15.17:0/439682372 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf980a5cc0 con 0x7fbfac14dfd0 2024-09-10T21:46:57.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:57.825+0000 7fbfb2600640 1 -- 172.21.15.17:0/439682372 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfac155b60 con 0x7fbfac14dfd0 2024-09-10T21:46:57.831 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:57.825+0000 7fbfb2600640 1 -- 172.21.15.17:0/439682372 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfac0cb480 con 0x7fbfac14dfd0 2024-09-10T21:46:57.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:57.829+0000 7fbfb2600640 1 -- 172.21.15.17:0/439682372 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbfac0cab70 con 0x7fbfac14dfd0 2024-09-10T21:46:57.832 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:57.829+0000 7fbfa8c00640 1 -- 172.21.15.17:0/439682372 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fbf98003600 con 0x7fbfac14dfd0 2024-09-10T21:46:57.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:57.829+0000 7fbfa8c00640 1 --2- 172.21.15.17:0/439682372 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fbf8c073260 0x7fbf8c075720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:57.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:57.829+0000 7fbfa8c00640 1 -- 172.21.15.17:0/439682372 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fbf9812d110 con 0x7fbfac14dfd0 2024-09-10T21:46:57.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:57.829+0000 7fbfb1200640 1 --2- 172.21.15.17:0/439682372 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fbf8c073260 0x7fbf8c075720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:57.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:57.829+0000 7fbfb1200640 1 --2- 172.21.15.17:0/439682372 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fbf8c073260 0x7fbf8c075720 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fbfa0005d10 tx=0x7fbfa0015040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:46:57.842 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:57.837+0000 7fbfa8c00640 1 -- 172.21.15.17:0/439682372 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fbf98016650 con 0x7fbfac14dfd0 2024-09-10T21:46:58.084 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.081+0000 7fbfb2600640 1 -- 172.21.15.17:0/439682372 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7fbfac15b820 con 0x7fbfac14dfd0 2024-09-10T21:46:58.085 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.081+0000 7fbfa8c00640 1 -- 172.21.15.17:0/439682372 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1557 (secure 0 0 0) 0x7fbf980f7d00 con 0x7fbfac14dfd0 2024-09-10T21:46:58.085 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":172,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5e2f023f-d6a6-49f4-a256-f4807dce3739","modified":"2024-09-10T21:23:46.892130Z","created":"2024-09-10T21:23:46.892130Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3301","nonce":0},{"type":"v1","addr":"172.21.15.31:6790","nonce":0}]},"addr":"172.21.15.31:6790/0","public_addr":"172.21.15.31:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-10T21:46:58.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.081+0000 7fbfb2600640 1 -- 172.21.15.17:0/439682372 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fbf8c073260 msgr2=0x7fbf8c075720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:58.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.081+0000 7fbfb2600640 1 --2- 172.21.15.17:0/439682372 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fbf8c073260 0x7fbf8c075720 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fbfa0005d10 tx=0x7fbfa0015040 comp rx=0 tx=0).stop 2024-09-10T21:46:58.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.081+0000 7fbfb2600640 1 -- 172.21.15.17:0/439682372 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbfac14dfd0 msgr2=0x7fbfac1551c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:58.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.081+0000 7fbfb2600640 1 --2- 172.21.15.17:0/439682372 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbfac14dfd0 0x7fbfac1551c0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fbf98000c10 tx=0x7fbf98006720 comp rx=0 tx=0).stop 2024-09-10T21:46:58.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.081+0000 7fbfb2600640 1 -- 172.21.15.17:0/439682372 shutdown_connections 2024-09-10T21:46:58.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.081+0000 7fbfb2600640 1 --2- 172.21.15.17:0/439682372 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fbf8c073260 0x7fbf8c075720 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:58.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.081+0000 7fbfb2600640 1 --2- 172.21.15.17:0/439682372 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fbfac155700 0x7fbfac0c9430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:58.087 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.081+0000 7fbfb2600640 1 --2- 172.21.15.17:0/439682372 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbfac14dfd0 0x7fbfac1551c0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:58.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.081+0000 7fbfb2600640 1 --2- 172.21.15.17:0/439682372 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbf8c007290 0x7fbfac14da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:58.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.081+0000 7fbfb2600640 1 -- 172.21.15.17:0/439682372 >> 172.21.15.17:0/439682372 conn(0x7fbfac074ac0 msgr2=0x7fbfac071db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:46:58.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.081+0000 7fbfb2600640 1 -- 172.21.15.17:0/439682372 shutdown_connections 2024-09-10T21:46:58.088 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.081+0000 7fbfb2600640 1 -- 172.21.15.17:0/439682372 wait complete. 2024-09-10T21:46:58.107 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-09-10T21:46:58.108 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon a 2024-09-10T21:46:58.108 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-09-10T21:46:58.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.253+0000 7f61dc600640 1 -- 172.21.15.17:0/3281389193 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f61d4152820 msgr2=0x7f61d40b3eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:58.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.253+0000 7f61dc600640 1 --2- 172.21.15.17:0/3281389193 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f61d4152820 0x7f61d40b3eb0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f61d41594a0 tx=0x7f61c001c790 comp rx=0 tx=0).stop 2024-09-10T21:46:58.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.253+0000 7f61dc600640 1 -- 172.21.15.17:0/3281389193 shutdown_connections 2024-09-10T21:46:58.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.253+0000 7f61dc600640 1 --2- 172.21.15.17:0/3281389193 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f61d4152820 0x7f61d40b3eb0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:58.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.253+0000 7f61dc600640 1 -- 172.21.15.17:0/3281389193 >> 172.21.15.17:0/3281389193 conn(0x7f61d40702f0 msgr2=0x7f61d4070700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:46:58.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.253+0000 7f61dc600640 1 -- 172.21.15.17:0/3281389193 shutdown_connections 2024-09-10T21:46:58.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.253+0000 7f61dc600640 1 -- 172.21.15.17:0/3281389193 wait complete. 2024-09-10T21:46:58.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.253+0000 7f61dc600640 1 Processor -- start 2024-09-10T21:46:58.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.253+0000 7f61dc600640 1 -- start start 2024-09-10T21:46:58.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.253+0000 7f61dc600640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f61d40cdf20 0x7f61d415ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:58.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.253+0000 7f61dc600640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f61d4152440 0x7f61d414dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:58.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.257+0000 7f61dc600640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f61d4152820 0x7f61d40c3410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:58.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.257+0000 7f61dc600640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f61d40cb450 con 0x7f61d40cdf20 2024-09-10T21:46:58.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.257+0000 7f61dc600640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f61d40ce740 con 0x7f61d4152820 2024-09-10T21:46:58.260 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.257+0000 7f61dc600640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f61d40cb5d0 con 0x7f61d4152440 2024-09-10T21:46:58.261 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.257+0000 7f61d9c00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f61d40cdf20 0x7f61d415ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:58.261 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.257+0000 7f61d9c00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f61d40cdf20 0x7f61d415ae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37074/0 (socket says 172.21.15.17:37074) 2024-09-10T21:46:58.261 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.257+0000 7f61d9c00640 1 -- 172.21.15.17:0/2485173527 learned_addr learned my addr 172.21.15.17:0/2485173527 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:46:58.261 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.257+0000 7f61d9200640 1 --2- 172.21.15.17:0/2485173527 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f61d4152440 0x7f61d414dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:58.261 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.257+0000 7f61da600640 1 --2- 172.21.15.17:0/2485173527 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f61d4152820 0x7f61d40c3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:58.261 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.257+0000 7f61d9c00640 1 -- 172.21.15.17:0/2485173527 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f61d4152440 msgr2=0x7f61d414dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:58.261 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.257+0000 7f61d9c00640 1 --2- 172.21.15.17:0/2485173527 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f61d4152440 0x7f61d414dc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:58.261 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.257+0000 7f61d9c00640 1 -- 172.21.15.17:0/2485173527 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f61d4152820 msgr2=0x7f61d40c3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:58.261 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.257+0000 7f61d9c00640 1 --2- 172.21.15.17:0/2485173527 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f61d4152820 0x7f61d40c3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:58.261 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.257+0000 7f61d9c00640 1 -- 172.21.15.17:0/2485173527 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61d414f570 con 0x7f61d40cdf20 2024-09-10T21:46:58.261 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.257+0000 7f61d9c00640 1 --2- 172.21.15.17:0/2485173527 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f61d40cdf20 0x7f61d415ae70 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f61c800c480 tx=0x7f61c800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:46:58.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.257+0000 7f61d1600640 1 -- 172.21.15.17:0/2485173527 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61c8012830 con 0x7f61d40cdf20 2024-09-10T21:46:58.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.257+0000 7f61d1600640 1 -- 172.21.15.17:0/2485173527 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f61c8017070 con 0x7f61d40cdf20 2024-09-10T21:46:58.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.257+0000 7f61d1600640 1 -- 172.21.15.17:0/2485173527 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61c8012d80 con 0x7f61d40cdf20 2024-09-10T21:46:58.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.257+0000 7f61dc600640 1 -- 172.21.15.17:0/2485173527 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61d414d4c0 con 0x7f61d40cdf20 2024-09-10T21:46:58.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.257+0000 7f61dc600640 1 -- 172.21.15.17:0/2485173527 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61d414f290 con 0x7f61d40cdf20 2024-09-10T21:46:58.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.257+0000 7f61dc600640 1 --2- 172.21.15.17:0/2485173527 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f61d414d690 0x7f61d40c3950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:58.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.257+0000 7f61dc600640 1 -- 172.21.15.17:0/2485173527 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f61d406e450 con 0x7f61d414d690 2024-09-10T21:46:58.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.257+0000 7f61d9200640 1 --2- 172.21.15.17:0/2485173527 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f61d414d690 0x7f61d40c3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:58.263 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.257+0000 7f61d9200640 1 -- 172.21.15.17:0/2485173527 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f61d4158360 con 0x7f61d414d690 2024-09-10T21:46:58.264 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.257+0000 7f61d9200640 1 --2- 172.21.15.17:0/2485173527 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f61d414d690 0x7f61d40c3950 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f61d40c9850 tx=0x7f61c00050d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:46:58.264 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.257+0000 7f61d1600640 1 -- 172.21.15.17:0/2485173527 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61c00b2040 con 0x7f61d414d690 2024-09-10T21:46:58.264 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.257+0000 7f61d1600640 1 -- 172.21.15.17:0/2485173527 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f61c801c460 con 0x7f61d40cdf20 2024-09-10T21:46:58.265 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.261+0000 7f61d1600640 1 --2- 172.21.15.17:0/2485173527 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f61b40724e0 0x7f61b40749a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:58.265 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.261+0000 7f61d1600640 1 -- 172.21.15.17:0/2485173527 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f61c8093520 con 0x7f61d40cdf20 2024-09-10T21:46:58.265 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.261+0000 7f61d1600640 1 -- 172.21.15.17:0/2485173527 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7f61c0005440 con 0x7f61d414d690 2024-09-10T21:46:58.265 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.261+0000 7f61d1600640 1 -- 172.21.15.17:0/2485173527 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f61d414d690 msgr2=0x7f61d40c3950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:58.265 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.261+0000 7f61d1600640 1 --2- 172.21.15.17:0/2485173527 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f61d414d690 0x7f61d40c3950 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f61d40c9850 tx=0x7f61c00050d0 comp rx=0 tx=0).stop 2024-09-10T21:46:58.265 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.261+0000 7f61d1600640 1 -- 172.21.15.17:0/2485173527 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f61d414d690 msgr2=0x7f61d40c3950 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:46:58.265 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.261+0000 7f61d1600640 1 --2- 172.21.15.17:0/2485173527 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f61d414d690 0x7f61d40c3950 secure :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f61d40c9850 tx=0x7f61c00050d0 comp rx=0 tx=0).stop 2024-09-10T21:46:58.265 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.261+0000 7f61da600640 1 --2- 172.21.15.17:0/2485173527 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f61b40724e0 0x7f61b40749a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:58.265 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.261+0000 7f61da600640 1 --2- 172.21.15.17:0/2485173527 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f61b40724e0 0x7f61b40749a0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f61cc0012f0 tx=0x7f61cc009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:46:58.272 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.265+0000 7f61dc600640 1 --2- 172.21.15.17:0/2485173527 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f61d414e1c0 0x7f61d414bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:58.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.265+0000 7f61dc600640 1 -- 172.21.15.17:0/2485173527 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f61d40cc510 con 0x7f61d414e1c0 2024-09-10T21:46:58.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.265+0000 7f61d9200640 1 --2- 172.21.15.17:0/2485173527 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f61d414e1c0 0x7f61d414bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:58.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.265+0000 7f61d9200640 1 -- 172.21.15.17:0/2485173527 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7f61d40cc230 con 0x7f61d414e1c0 2024-09-10T21:46:58.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.265+0000 7f61d9200640 1 --2- 172.21.15.17:0/2485173527 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f61d414e1c0 0x7f61d414bac0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f61c001a3c0 tx=0x7f61c00091e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:46:58.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.265+0000 7f61d1600640 1 -- 172.21.15.17:0/2485173527 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61c00048e0 con 0x7f61d414e1c0 2024-09-10T21:46:58.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.269+0000 7f61d1600640 1 -- 172.21.15.17:0/2485173527 <== mon.0 v2:172.21.15.17:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4543 (secure 0 0 0) 0x7f61c00048e0 con 0x7f61d414e1c0 2024-09-10T21:46:58.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.269+0000 7f61d1600640 1 -- 172.21.15.17:0/2485173527 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f61d414e1c0 msgr2=0x7f61d414bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:58.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.269+0000 7f61d1600640 1 --2- 172.21.15.17:0/2485173527 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f61d414e1c0 0x7f61d414bac0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f61c001a3c0 tx=0x7f61c00091e0 comp rx=0 tx=0).stop 2024-09-10T21:46:58.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.269+0000 7f61d1600640 1 -- 172.21.15.17:0/2485173527 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f61d414e1c0 msgr2=0x7f61d414bac0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:46:58.273 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.269+0000 7f61d1600640 1 --2- 172.21.15.17:0/2485173527 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f61d414e1c0 0x7f61d414bac0 secure :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f61c001a3c0 tx=0x7f61c00091e0 comp rx=0 tx=0).stop 2024-09-10T21:46:58.274 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-10T21:46:58.274 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:46:58.274 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:46:58.274 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "leader", 2024-09-10T21:46:58.274 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 172, 2024-09-10T21:46:58.274 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 26627, 2024-09-10T21:46:58.274 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-10T21:46:58.274 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-10T21:46:58.274 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-10T21:46:58.274 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-10T21:46:58.274 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:46:58.274 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 1, 2024-09-10T21:46:58.275 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:46:58.275 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-10T21:46:58.275 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-10T21:46:58.275 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:46:58.275 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:46:58.275 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:46:58.275 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:46:58.275 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:46:58.275 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:46:58.275 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:46:58.275 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:46:58.275 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:46:58.276 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:46:58.276 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:46:58.276 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:46:58.276 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-10T21:46:58.276 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-10T21:46:58.276 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:46:58.276 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:46:58.276 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:46:58.276 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:46:58.276 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:46:58.276 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:46:58.276 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:46:58.276 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:46:58.276 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:46:58.277 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:46:58.277 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:46:58.277 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:46:58.277 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:46:58.277 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-10T21:46:58.277 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-10T21:46:58.277 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-10T21:46:58.277 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-10T21:46:58.277 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-10T21:46:58.277 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "5e2f023f-d6a6-49f4-a256-f4807dce3739", 2024-09-10T21:46:58.277 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:46:58.277 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:46:58.278 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-10T21:46:58.278 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-10T21:46:58.278 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-10T21:46:58.278 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-10T21:46:58.278 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-10T21:46:58.278 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-10T21:46:58.278 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-10T21:46:58.278 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:46:58.278 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-10T21:46:58.278 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:46:58.278 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:46:58.278 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:46:58.278 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:46:58.279 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:46:58.279 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:46:58.279 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:46:58.279 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:46:58.279 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:46:58.279 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:46:58.279 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:46:58.279 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:46:58.279 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-10T21:46:58.279 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:46:58.279 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-10T21:46:58.279 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:46:58.279 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:46:58.279 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:46:58.280 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:46:58.280 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:46:58.280 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:46:58.280 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:46:58.280 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-10T21:46:58.280 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:46:58.280 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:46:58.280 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:46:58.280 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:46:58.280 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-10T21:46:58.280 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:46:58.280 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:46:58.280 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:46:58.280 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:46:58.281 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-10T21:46:58.281 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-10T21:46:58.281 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:46:58.281 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:46:58.281 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:46:58.281 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:46:58.281 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:46:58.281 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:46:58.281 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:46:58.281 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:46:58.281 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:46:58.281 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:46:58.281 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:46:58.281 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3300", 2024-09-10T21:46:58.281 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:46:58.282 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:46:58.282 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:46:58.282 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:46:58.282 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789", 2024-09-10T21:46:58.282 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:46:58.282 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:46:58.282 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:46:58.282 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:46:58.282 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789/0", 2024-09-10T21:46:58.282 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6789/0", 2024-09-10T21:46:58.282 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:46:58.282 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:46:58.282 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:46:58.282 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:46:58.283 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:46:58.283 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:46:58.283 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:46:58.283 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:46:58.283 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:46:58.283 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:46:58.283 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:46:58.283 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3301", 2024-09-10T21:46:58.283 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:46:58.283 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:46:58.283 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:46:58.283 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:46:58.283 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790", 2024-09-10T21:46:58.284 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:46:58.284 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:46:58.284 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:46:58.284 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:46:58.284 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790/0", 2024-09-10T21:46:58.284 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6790/0", 2024-09-10T21:46:58.284 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:46:58.284 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:46:58.284 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:46:58.284 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:46:58.284 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:46:58.284 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:46:58.284 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-10T21:46:58.284 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-10T21:46:58.285 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:46:58.285 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:46:58.285 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:46:58.285 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:46:58.285 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:46:58.285 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:46:58.285 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-10T21:46:58.285 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:46:58.285 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:46:58.285 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:46:58.285 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 2 2024-09-10T21:46:58.285 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:46:58.285 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:46:58.285 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:46:58.286 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-10T21:46:58.286 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-10T21:46:58.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.269+0000 7f61dc600640 1 -- 172.21.15.17:0/2485173527 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f61b40724e0 msgr2=0x7f61b40749a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:58.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.269+0000 7f61dc600640 1 --2- 172.21.15.17:0/2485173527 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f61b40724e0 0x7f61b40749a0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f61cc0012f0 tx=0x7f61cc009040 comp rx=0 tx=0).stop 2024-09-10T21:46:58.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.269+0000 7f61dc600640 1 -- 172.21.15.17:0/2485173527 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f61d40cdf20 msgr2=0x7f61d415ae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:58.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.269+0000 7f61dc600640 1 --2- 172.21.15.17:0/2485173527 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f61d40cdf20 0x7f61d415ae70 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f61c800c480 tx=0x7f61c800c940 comp rx=0 tx=0).stop 2024-09-10T21:46:58.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.269+0000 7f61da600640 1 -- 172.21.15.17:0/2485173527 reap_dead start 2024-09-10T21:46:58.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.269+0000 7f61da600640 1 -- 172.21.15.17:0/2485173527 reap_dead start 2024-09-10T21:46:58.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.269+0000 7f61dc600640 1 -- 172.21.15.17:0/2485173527 shutdown_connections 2024-09-10T21:46:58.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.269+0000 7f61dc600640 1 -- 172.21.15.17:0/2485173527 >> 172.21.15.17:0/2485173527 conn(0x7f61d40702f0 msgr2=0x7f61d4157830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:46:58.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.269+0000 7f61dc600640 1 -- 172.21.15.17:0/2485173527 shutdown_connections 2024-09-10T21:46:58.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.269+0000 7f61dc600640 1 -- 172.21.15.17:0/2485173527 wait complete. 2024-09-10T21:46:58.291 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-10T21:46:58.291 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon b 2024-09-10T21:46:58.291 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-09-10T21:46:58.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.437+0000 7fd59e800640 1 -- 172.21.15.17:0/3026102308 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd598073cf0 msgr2=0x7fd5980740d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:58.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.437+0000 7fd59e800640 1 --2- 172.21.15.17:0/3026102308 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd598073cf0 0x7fd5980740d0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fd588009900 tx=0x7fd58801c8e0 comp rx=0 tx=0).stop 2024-09-10T21:46:58.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.437+0000 7fd59e800640 1 -- 172.21.15.17:0/3026102308 shutdown_connections 2024-09-10T21:46:58.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.437+0000 7fd59e800640 1 --2- 172.21.15.17:0/3026102308 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd598073cf0 0x7fd5980740d0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:58.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.437+0000 7fd59e800640 1 -- 172.21.15.17:0/3026102308 >> 172.21.15.17:0/3026102308 conn(0x7fd5980700d0 msgr2=0x7fd5980704e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:46:58.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.437+0000 7fd59e800640 1 -- 172.21.15.17:0/3026102308 shutdown_connections 2024-09-10T21:46:58.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.437+0000 7fd59e800640 1 -- 172.21.15.17:0/3026102308 wait complete. 2024-09-10T21:46:58.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.441+0000 7fd59e800640 1 Processor -- start 2024-09-10T21:46:58.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.441+0000 7fd59e800640 1 -- start start 2024-09-10T21:46:58.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.441+0000 7fd59e800640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd598073cf0 0x7fd598143250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:58.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.441+0000 7fd59e800640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd5980c0fd0 0x7fd5981f3960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:58.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.441+0000 7fd59e800640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd59815abb0 0x7fd5980cb920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:58.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.441+0000 7fd59e800640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fd5980c7dd0 con 0x7fd5980c0fd0 2024-09-10T21:46:58.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.441+0000 7fd59e800640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fd5980c8370 con 0x7fd598073cf0 2024-09-10T21:46:58.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.441+0000 7fd59e800640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7fd5980c7f50 con 0x7fd59815abb0 2024-09-10T21:46:58.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.441+0000 7fd596a00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd5980c0fd0 0x7fd5981f3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:58.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.441+0000 7fd596a00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd5980c0fd0 0x7fd5981f3960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37116/0 (socket says 172.21.15.17:37116) 2024-09-10T21:46:58.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.441+0000 7fd596a00640 1 -- 172.21.15.17:0/257301044 learned_addr learned my addr 172.21.15.17:0/257301044 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:46:58.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.441+0000 7fd597e00640 1 --2- 172.21.15.17:0/257301044 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd59815abb0 0x7fd5980cb920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:58.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.441+0000 7fd597400640 1 --2- 172.21.15.17:0/257301044 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd598073cf0 0x7fd598143250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:58.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.441+0000 7fd596a00640 1 -- 172.21.15.17:0/257301044 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd59815abb0 msgr2=0x7fd5980cb920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:58.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.441+0000 7fd596a00640 1 --2- 172.21.15.17:0/257301044 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd59815abb0 0x7fd5980cb920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:58.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.441+0000 7fd596a00640 1 -- 172.21.15.17:0/257301044 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd598073cf0 msgr2=0x7fd598143250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:58.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.441+0000 7fd596a00640 1 --2- 172.21.15.17:0/257301044 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd598073cf0 0x7fd598143250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:58.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.441+0000 7fd596a00640 1 -- 172.21.15.17:0/257301044 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5980cc6e0 con 0x7fd5980c0fd0 2024-09-10T21:46:58.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.441+0000 7fd596a00640 1 --2- 172.21.15.17:0/257301044 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd5980c0fd0 0x7fd5981f3960 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fd58400c500 tx=0x7fd58400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:46:58.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.441+0000 7fd58f400640 1 -- 172.21.15.17:0/257301044 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd584017070 con 0x7fd5980c0fd0 2024-09-10T21:46:58.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.441+0000 7fd58f400640 1 -- 172.21.15.17:0/257301044 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd5840053c0 con 0x7fd5980c0fd0 2024-09-10T21:46:58.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.441+0000 7fd58f400640 1 -- 172.21.15.17:0/257301044 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd584012890 con 0x7fd5980c0fd0 2024-09-10T21:46:58.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.441+0000 7fd59e800640 1 -- 172.21.15.17:0/257301044 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5981f7120 con 0x7fd5980c0fd0 2024-09-10T21:46:58.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.441+0000 7fd59e800640 1 -- 172.21.15.17:0/257301044 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd598157be0 con 0x7fd5980c0fd0 2024-09-10T21:46:58.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.445+0000 7fd59e800640 1 --2- 172.21.15.17:0/257301044 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd5981f7be0 0x7fd5980cbe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:58.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.445+0000 7fd59e800640 1 -- 172.21.15.17:0/257301044 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fd5980cc3a0 con 0x7fd5981f7be0 2024-09-10T21:46:58.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.445+0000 7fd597400640 1 --2- 172.21.15.17:0/257301044 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd5981f7be0 0x7fd5980cbe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:58.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.445+0000 7fd597400640 1 -- 172.21.15.17:0/257301044 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd5981f7920 con 0x7fd5981f7be0 2024-09-10T21:46:58.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.445+0000 7fd597400640 1 --2- 172.21.15.17:0/257301044 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd5981f7be0 0x7fd5980cbe60 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fd5980cd070 tx=0x7fd588005040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:46:58.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.445+0000 7fd58f400640 1 -- 172.21.15.17:0/257301044 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5880b2040 con 0x7fd5981f7be0 2024-09-10T21:46:58.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.445+0000 7fd58f400640 1 -- 172.21.15.17:0/257301044 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fd584003600 con 0x7fd5980c0fd0 2024-09-10T21:46:58.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.445+0000 7fd58f400640 1 --2- 172.21.15.17:0/257301044 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fd5780724e0 0x7fd5780749a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:58.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.445+0000 7fd58f400640 1 -- 172.21.15.17:0/257301044 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fd584092c30 con 0x7fd5980c0fd0 2024-09-10T21:46:58.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.445+0000 7fd58f400640 1 -- 172.21.15.17:0/257301044 <== mon.1 v2:172.21.15.31:3300/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7fd588005470 con 0x7fd5981f7be0 2024-09-10T21:46:58.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.445+0000 7fd58f400640 1 -- 172.21.15.17:0/257301044 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd5981f7be0 msgr2=0x7fd5980cbe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:58.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.445+0000 7fd58f400640 1 --2- 172.21.15.17:0/257301044 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd5981f7be0 0x7fd5980cbe60 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fd5980cd070 tx=0x7fd588005040 comp rx=0 tx=0).stop 2024-09-10T21:46:58.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.445+0000 7fd58f400640 1 -- 172.21.15.17:0/257301044 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd5981f7be0 msgr2=0x7fd5980cbe60 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:46:58.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.445+0000 7fd58f400640 1 --2- 172.21.15.17:0/257301044 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd5981f7be0 0x7fd5980cbe60 secure :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fd5980cd070 tx=0x7fd588005040 comp rx=0 tx=0).stop 2024-09-10T21:46:58.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.445+0000 7fd597e00640 1 --2- 172.21.15.17:0/257301044 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fd5780724e0 0x7fd5780749a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:58.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.445+0000 7fd597e00640 1 --2- 172.21.15.17:0/257301044 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fd5780724e0 0x7fd5780749a0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fd59815b010 tx=0x7fd590003d50 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:46:58.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.453+0000 7fd59e800640 1 --2- 172.21.15.17:0/257301044 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd598158890 0x7fd598141980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:58.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.453+0000 7fd59e800640 1 -- 172.21.15.17:0/257301044 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fd5981584e0 con 0x7fd598158890 2024-09-10T21:46:58.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.453+0000 7fd597400640 1 --2- 172.21.15.17:0/257301044 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd598158890 0x7fd598141980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:58.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.453+0000 7fd597400640 1 -- 172.21.15.17:0/257301044 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7fd5981511e0 con 0x7fd598158890 2024-09-10T21:46:58.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.453+0000 7fd597400640 1 --2- 172.21.15.17:0/257301044 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd598158890 0x7fd598141980 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fd58801af90 tx=0x7fd588009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:46:58.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.453+0000 7fd58f400640 1 -- 172.21.15.17:0/257301044 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5880a5d70 con 0x7fd598158890 2024-09-10T21:46:58.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.453+0000 7fd58f400640 1 -- 172.21.15.17:0/257301044 <== mon.1 v2:172.21.15.31:3300/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+5043 (secure 0 0 0) 0x7fd5880046d0 con 0x7fd598158890 2024-09-10T21:46:58.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.453+0000 7fd58f400640 1 -- 172.21.15.17:0/257301044 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd598158890 msgr2=0x7fd598141980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:58.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.453+0000 7fd58f400640 1 --2- 172.21.15.17:0/257301044 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd598158890 0x7fd598141980 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fd58801af90 tx=0x7fd588009260 comp rx=0 tx=0).stop 2024-09-10T21:46:58.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.453+0000 7fd58f400640 1 -- 172.21.15.17:0/257301044 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd598158890 msgr2=0x7fd598141980 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:46:58.459 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.453+0000 7fd58f400640 1 --2- 172.21.15.17:0/257301044 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd598158890 0x7fd598141980 secure :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fd58801af90 tx=0x7fd588009260 comp rx=0 tx=0).stop 2024-09-10T21:46:58.459 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-10T21:46:58.460 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:46:58.460 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:46:58.460 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-10T21:46:58.460 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 172, 2024-09-10T21:46:58.460 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 163434, 2024-09-10T21:46:58.460 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-10T21:46:58.460 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-10T21:46:58.460 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-10T21:46:58.460 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-10T21:46:58.460 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:46:58.460 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 1, 2024-09-10T21:46:58.461 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:46:58.461 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-10T21:46:58.461 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-10T21:46:58.461 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:46:58.461 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:46:58.461 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:46:58.461 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:46:58.461 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:46:58.461 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:46:58.461 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:46:58.461 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:46:58.461 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:46:58.461 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:46:58.461 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:46:58.462 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:46:58.462 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-10T21:46:58.462 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-10T21:46:58.462 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:46:58.462 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:46:58.462 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:46:58.462 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:46:58.462 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:46:58.462 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:46:58.462 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:46:58.462 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:46:58.462 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:46:58.462 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:46:58.463 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:46:58.463 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:46:58.463 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:46:58.463 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-10T21:46:58.463 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-10T21:46:58.463 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-10T21:46:58.463 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-10T21:46:58.463 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-10T21:46:58.463 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "5e2f023f-d6a6-49f4-a256-f4807dce3739", 2024-09-10T21:46:58.463 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:46:58.463 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:46:58.463 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-10T21:46:58.463 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-10T21:46:58.464 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-10T21:46:58.464 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-10T21:46:58.464 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-10T21:46:58.464 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-10T21:46:58.464 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-10T21:46:58.464 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:46:58.464 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-10T21:46:58.464 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:46:58.464 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:46:58.464 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:46:58.464 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:46:58.464 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:46:58.464 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:46:58.465 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:46:58.465 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:46:58.465 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:46:58.465 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:46:58.465 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:46:58.465 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:46:58.465 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-10T21:46:58.465 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:46:58.465 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-10T21:46:58.465 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:46:58.465 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:46:58.465 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:46:58.465 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:46:58.466 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:46:58.466 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:46:58.466 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:46:58.466 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-10T21:46:58.466 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:46:58.466 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:46:58.466 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:46:58.466 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:46:58.466 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-10T21:46:58.466 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:46:58.466 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:46:58.466 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:46:58.466 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:46:58.467 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-10T21:46:58.467 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-10T21:46:58.467 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:46:58.467 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:46:58.467 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:46:58.467 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:46:58.467 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:46:58.467 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:46:58.467 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:46:58.467 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:46:58.467 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:46:58.467 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:46:58.467 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:46:58.467 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3300", 2024-09-10T21:46:58.468 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:46:58.468 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:46:58.468 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:46:58.468 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:46:58.468 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789", 2024-09-10T21:46:58.468 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:46:58.468 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:46:58.468 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:46:58.468 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:46:58.468 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789/0", 2024-09-10T21:46:58.468 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6789/0", 2024-09-10T21:46:58.469 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:46:58.469 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:46:58.469 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:46:58.469 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:46:58.469 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:46:58.470 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:46:58.470 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:46:58.470 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:46:58.470 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:46:58.470 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:46:58.470 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:46:58.471 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3301", 2024-09-10T21:46:58.471 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:46:58.471 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:46:58.471 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:46:58.471 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:46:58.471 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790", 2024-09-10T21:46:58.471 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:46:58.472 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:46:58.472 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:46:58.472 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:46:58.472 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790/0", 2024-09-10T21:46:58.472 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6790/0", 2024-09-10T21:46:58.472 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:46:58.472 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:46:58.472 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:46:58.472 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:46:58.472 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:46:58.472 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:46:58.472 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-10T21:46:58.472 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-10T21:46:58.473 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:46:58.473 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:46:58.473 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:46:58.473 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:46:58.473 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:46:58.473 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:46:58.473 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": [ 2024-09-10T21:46:58.473 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:46:58.473 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:46:58.473 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:46:58.473 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 3 2024-09-10T21:46:58.473 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:46:58.474 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:46:58.474 INFO:teuthology.orchestra.run.smithi017.stdout: "osd": [ 2024-09-10T21:46:58.474 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:46:58.474 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:46:58.474 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:46:58.474 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 8 2024-09-10T21:46:58.474 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:46:58.474 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:46:58.474 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-10T21:46:58.474 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:46:58.474 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:46:58.474 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:46:58.474 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 6 2024-09-10T21:46:58.474 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:46:58.474 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:46:58.475 INFO:teuthology.orchestra.run.smithi017.stdout: "mgr": [ 2024-09-10T21:46:58.475 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:46:58.475 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:46:58.475 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:46:58.475 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 2 2024-09-10T21:46:58.475 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:46:58.475 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:46:58.475 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:46:58.475 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-10T21:46:58.475 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-10T21:46:58.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.453+0000 7fd59e800640 1 -- 172.21.15.17:0/257301044 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fd5780724e0 msgr2=0x7fd5780749a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:58.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.453+0000 7fd59e800640 1 --2- 172.21.15.17:0/257301044 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fd5780724e0 0x7fd5780749a0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fd59815b010 tx=0x7fd590003d50 comp rx=0 tx=0).stop 2024-09-10T21:46:58.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.453+0000 7fd59e800640 1 -- 172.21.15.17:0/257301044 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd5980c0fd0 msgr2=0x7fd5981f3960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:58.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.453+0000 7fd59e800640 1 --2- 172.21.15.17:0/257301044 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd5980c0fd0 0x7fd5981f3960 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fd58400c500 tx=0x7fd58400c9c0 comp rx=0 tx=0).stop 2024-09-10T21:46:58.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.453+0000 7fd597e00640 1 -- 172.21.15.17:0/257301044 reap_dead start 2024-09-10T21:46:58.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.453+0000 7fd597e00640 1 -- 172.21.15.17:0/257301044 reap_dead start 2024-09-10T21:46:58.476 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.453+0000 7fd59e800640 1 -- 172.21.15.17:0/257301044 shutdown_connections 2024-09-10T21:46:58.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.453+0000 7fd59e800640 1 -- 172.21.15.17:0/257301044 >> 172.21.15.17:0/257301044 conn(0x7fd5980700d0 msgr2=0x7fd59806dc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:46:58.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.453+0000 7fd59e800640 1 -- 172.21.15.17:0/257301044 shutdown_connections 2024-09-10T21:46:58.477 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.453+0000 7fd59e800640 1 -- 172.21.15.17:0/257301044 wait complete. 2024-09-10T21:46:58.479 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-10T21:46:58.479 INFO:tasks.mon_thrash.mon_thrasher:waiting for quorum size 3 for mon c 2024-09-10T21:46:58.479 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-09-10T21:46:58.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.629+0000 7ff9e3000640 1 -- 172.21.15.17:0/492182561 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff9dc0c6b40 msgr2=0x7ff9dc06dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:58.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.629+0000 7ff9e3000640 1 --2- 172.21.15.17:0/492182561 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff9dc0c6b40 0x7ff9dc06dd30 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff9c4009870 tx=0x7ff9c401c750 comp rx=0 tx=0).stop 2024-09-10T21:46:58.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.629+0000 7ff9e3000640 1 -- 172.21.15.17:0/492182561 shutdown_connections 2024-09-10T21:46:58.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.629+0000 7ff9e3000640 1 --2- 172.21.15.17:0/492182561 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff9dc0c6b40 0x7ff9dc06dd30 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:58.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.629+0000 7ff9e3000640 1 -- 172.21.15.17:0/492182561 >> 172.21.15.17:0/492182561 conn(0x7ff9dc0700d0 msgr2=0x7ff9dc0704e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:46:58.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.629+0000 7ff9e3000640 1 -- 172.21.15.17:0/492182561 shutdown_connections 2024-09-10T21:46:58.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.629+0000 7ff9e3000640 1 -- 172.21.15.17:0/492182561 wait complete. 2024-09-10T21:46:58.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.629+0000 7ff9e3000640 1 Processor -- start 2024-09-10T21:46:58.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.629+0000 7ff9e3000640 1 -- start start 2024-09-10T21:46:58.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.629+0000 7ff9e3000640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff9dc0c6b40 0x7ff9dc0c4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:58.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.629+0000 7ff9e3000640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff9dc0c8490 0x7ff9dc1f4be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:58.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.629+0000 7ff9e3000640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff9dc1522b0 0x7ff9dc14ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:58.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.629+0000 7ff9e3000640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7ff9dc152690 con 0x7ff9dc0c6b40 2024-09-10T21:46:58.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.629+0000 7ff9e3000640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7ff9dc0725c0 con 0x7ff9dc1522b0 2024-09-10T21:46:58.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.629+0000 7ff9e3000640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7ff9dc0afa40 con 0x7ff9dc0c8490 2024-09-10T21:46:58.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.629+0000 7ff9dbe00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff9dc0c6b40 0x7ff9dc0c4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:58.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.629+0000 7ff9dbe00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff9dc0c6b40 0x7ff9dc0c4fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37132/0 (socket says 172.21.15.17:37132) 2024-09-10T21:46:58.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.629+0000 7ff9dbe00640 1 -- 172.21.15.17:0/154556446 learned_addr learned my addr 172.21.15.17:0/154556446 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:46:58.639 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.629+0000 7ff9e1000640 1 --2- 172.21.15.17:0/154556446 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff9dc1522b0 0x7ff9dc14ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:58.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.629+0000 7ff9d3400640 1 --2- 172.21.15.17:0/154556446 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff9dc0c8490 0x7ff9dc1f4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:58.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.629+0000 7ff9dbe00640 1 -- 172.21.15.17:0/154556446 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff9dc0c8490 msgr2=0x7ff9dc1f4be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:58.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.629+0000 7ff9dbe00640 1 --2- 172.21.15.17:0/154556446 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff9dc0c8490 0x7ff9dc1f4be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:58.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.629+0000 7ff9dbe00640 1 -- 172.21.15.17:0/154556446 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff9dc1522b0 msgr2=0x7ff9dc14ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:58.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.629+0000 7ff9dbe00640 1 --2- 172.21.15.17:0/154556446 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff9dc1522b0 0x7ff9dc14ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:58.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.629+0000 7ff9dbe00640 1 -- 172.21.15.17:0/154556446 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9dc0b1950 con 0x7ff9dc0c6b40 2024-09-10T21:46:58.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.633+0000 7ff9dbe00640 1 --2- 172.21.15.17:0/154556446 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff9dc0c6b40 0x7ff9dc0c4fc0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7ff9cc00c780 tx=0x7ff9cc00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:46:58.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.633+0000 7ff9d8c00640 1 -- 172.21.15.17:0/154556446 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9cc003b80 con 0x7ff9dc0c6b40 2024-09-10T21:46:58.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.633+0000 7ff9d8c00640 1 -- 172.21.15.17:0/154556446 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff9cc003d20 con 0x7ff9dc0c6b40 2024-09-10T21:46:58.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.633+0000 7ff9d8c00640 1 -- 172.21.15.17:0/154556446 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9cc00aab0 con 0x7ff9dc0c6b40 2024-09-10T21:46:58.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.633+0000 7ff9e3000640 1 -- 172.21.15.17:0/154556446 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9dc0af480 con 0x7ff9dc0c6b40 2024-09-10T21:46:58.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.633+0000 7ff9e3000640 1 -- 172.21.15.17:0/154556446 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9dc0b1150 con 0x7ff9dc0c6b40 2024-09-10T21:46:58.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.633+0000 7ff9e3000640 1 --2- 172.21.15.17:0/154556446 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff9dc1f5120 0x7ff9dc0b08b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:58.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.633+0000 7ff9e3000640 1 -- 172.21.15.17:0/154556446 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7ff9dc152810 con 0x7ff9dc1f5120 2024-09-10T21:46:58.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.633+0000 7ff9d3400640 1 --2- 172.21.15.17:0/154556446 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff9dc1f5120 0x7ff9dc0b08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:58.642 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.633+0000 7ff9d3400640 1 -- 172.21.15.17:0/154556446 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff9dc0c7d30 con 0x7ff9dc1f5120 2024-09-10T21:46:58.643 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.633+0000 7ff9d3400640 1 --2- 172.21.15.17:0/154556446 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff9dc1f5120 0x7ff9dc0b08b0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff9dc0c6ab0 tx=0x7ff9c4005aa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:46:58.644 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.633+0000 7ff9d8c00640 1 -- 172.21.15.17:0/154556446 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9c40bc070 con 0x7ff9dc1f5120 2024-09-10T21:46:58.644 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.633+0000 7ff9d8c00640 1 -- 172.21.15.17:0/154556446 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7ff9cc00ac50 con 0x7ff9dc0c6b40 2024-09-10T21:46:58.644 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.637+0000 7ff9d8c00640 1 --2- 172.21.15.17:0/154556446 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7ff9b80724e0 0x7ff9b80749a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:58.644 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.637+0000 7ff9d8c00640 1 -- 172.21.15.17:0/154556446 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7ff9cc0932c0 con 0x7ff9dc0c6b40 2024-09-10T21:46:58.644 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.637+0000 7ff9d8c00640 1 -- 172.21.15.17:0/154556446 <== mon.2 v2:172.21.15.31:3301/0 2 ==== command_reply(tid 1: 0 ) ==== 8+0+9324 (secure 0 0 0) 0x7ff9c4003800 con 0x7ff9dc1f5120 2024-09-10T21:46:58.644 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.637+0000 7ff9d8c00640 1 -- 172.21.15.17:0/154556446 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff9dc1f5120 msgr2=0x7ff9dc0b08b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:58.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.637+0000 7ff9d8c00640 1 --2- 172.21.15.17:0/154556446 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff9dc1f5120 0x7ff9dc0b08b0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff9dc0c6ab0 tx=0x7ff9c4005aa0 comp rx=0 tx=0).stop 2024-09-10T21:46:58.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.637+0000 7ff9d8c00640 1 -- 172.21.15.17:0/154556446 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff9dc1f5120 msgr2=0x7ff9dc0b08b0 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:46:58.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.637+0000 7ff9d8c00640 1 --2- 172.21.15.17:0/154556446 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff9dc1f5120 0x7ff9dc0b08b0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:58.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.637+0000 7ff9e1000640 1 --2- 172.21.15.17:0/154556446 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7ff9b80724e0 0x7ff9b80749a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:58.645 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.637+0000 7ff9e1000640 1 --2- 172.21.15.17:0/154556446 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7ff9b80724e0 0x7ff9b80749a0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7ff9d4001ec0 tx=0x7ff9d4001a00 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:46:58.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.641+0000 7ff9e3000640 1 --2- 172.21.15.17:0/154556446 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff9dc0b1390 0x7ff9dc14b140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:58.646 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.641+0000 7ff9e3000640 1 -- 172.21.15.17:0/154556446 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7ff9dc1561a0 con 0x7ff9dc0b1390 2024-09-10T21:46:58.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.641+0000 7ff9d3400640 1 --2- 172.21.15.17:0/154556446 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff9dc0b1390 0x7ff9dc14b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:58.647 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.641+0000 7ff9d3400640 1 -- 172.21.15.17:0/154556446 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- command(tid 2: {"prefix": "mon_status"}) -- 0x7ff9dc14dd10 con 0x7ff9dc0b1390 2024-09-10T21:46:58.648 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.645+0000 7ff9d3400640 1 --2- 172.21.15.17:0/154556446 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff9dc0b1390 0x7ff9dc14b140 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff9c401a380 tx=0x7ff9c40091e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:46:58.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.645+0000 7ff9d8c00640 1 -- 172.21.15.17:0/154556446 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9c40b2040 con 0x7ff9dc0b1390 2024-09-10T21:46:58.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.645+0000 7ff9d8c00640 1 -- 172.21.15.17:0/154556446 <== mon.2 v2:172.21.15.31:3301/0 2 ==== command_reply(tid 2: 0 ) ==== 8+0+4708 (secure 0 0 0) 0x7ff9c40a5d30 con 0x7ff9dc0b1390 2024-09-10T21:46:58.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.645+0000 7ff9d8c00640 1 -- 172.21.15.17:0/154556446 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff9dc0b1390 msgr2=0x7ff9dc14b140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:58.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.645+0000 7ff9d8c00640 1 --2- 172.21.15.17:0/154556446 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff9dc0b1390 0x7ff9dc14b140 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff9c401a380 tx=0x7ff9c40091e0 comp rx=0 tx=0).stop 2024-09-10T21:46:58.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.645+0000 7ff9d8c00640 1 -- 172.21.15.17:0/154556446 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff9dc0b1390 msgr2=0x7ff9dc14b140 secure :-1 s=STATE_CLOSED l=1).mark_down 2024-09-10T21:46:58.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.645+0000 7ff9d8c00640 1 --2- 172.21.15.17:0/154556446 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff9dc0b1390 0x7ff9dc14b140 secure :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff9c401a380 tx=0x7ff9c40091e0 comp rx=0 tx=0).stop 2024-09-10T21:46:58.650 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-10T21:46:58.650 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:46:58.650 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:46:58.650 INFO:teuthology.orchestra.run.smithi017.stdout: "state": "peon", 2024-09-10T21:46:58.650 INFO:teuthology.orchestra.run.smithi017.stdout: "election_epoch": 172, 2024-09-10T21:46:58.650 INFO:teuthology.orchestra.run.smithi017.stdout: "uptime": 97062, 2024-09-10T21:46:58.650 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum": [ 2024-09-10T21:46:58.651 INFO:teuthology.orchestra.run.smithi017.stdout: 0, 2024-09-10T21:46:58.651 INFO:teuthology.orchestra.run.smithi017.stdout: 1, 2024-09-10T21:46:58.651 INFO:teuthology.orchestra.run.smithi017.stdout: 2 2024-09-10T21:46:58.651 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:46:58.651 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_age": 1, 2024-09-10T21:46:58.651 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:46:58.651 INFO:teuthology.orchestra.run.smithi017.stdout: "required_con": "2450521708007866372", 2024-09-10T21:46:58.651 INFO:teuthology.orchestra.run.smithi017.stdout: "required_mon": [ 2024-09-10T21:46:58.652 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:46:58.652 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:46:58.652 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:46:58.652 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:46:58.652 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:46:58.652 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:46:58.653 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:46:58.653 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:46:58.653 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:46:58.653 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:46:58.653 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:46:58.653 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:46:58.653 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_con": "4540719139924082687", 2024-09-10T21:46:58.653 INFO:teuthology.orchestra.run.smithi017.stdout: "quorum_mon": [ 2024-09-10T21:46:58.653 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:46:58.653 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:46:58.654 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:46:58.654 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:46:58.654 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:46:58.654 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:46:58.654 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:46:58.654 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:46:58.654 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:46:58.654 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:46:58.654 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:46:58.654 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:46:58.654 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:46:58.654 INFO:teuthology.orchestra.run.smithi017.stdout: "outside_quorum": [], 2024-09-10T21:46:58.654 INFO:teuthology.orchestra.run.smithi017.stdout: "extra_probe_peers": [], 2024-09-10T21:46:58.654 INFO:teuthology.orchestra.run.smithi017.stdout: "sync_provider": [], 2024-09-10T21:46:58.655 INFO:teuthology.orchestra.run.smithi017.stdout: "monmap": { 2024-09-10T21:46:58.655 INFO:teuthology.orchestra.run.smithi017.stdout: "epoch": 1, 2024-09-10T21:46:58.655 INFO:teuthology.orchestra.run.smithi017.stdout: "fsid": "5e2f023f-d6a6-49f4-a256-f4807dce3739", 2024-09-10T21:46:58.655 INFO:teuthology.orchestra.run.smithi017.stdout: "modified": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:46:58.655 INFO:teuthology.orchestra.run.smithi017.stdout: "created": "2024-09-10T21:23:46.892130Z", 2024-09-10T21:46:58.655 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release": 19, 2024-09-10T21:46:58.655 INFO:teuthology.orchestra.run.smithi017.stdout: "min_mon_release_name": "squid", 2024-09-10T21:46:58.655 INFO:teuthology.orchestra.run.smithi017.stdout: "election_strategy": 1, 2024-09-10T21:46:58.655 INFO:teuthology.orchestra.run.smithi017.stdout: "disallowed_leaders: ": "", 2024-09-10T21:46:58.655 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false, 2024-09-10T21:46:58.655 INFO:teuthology.orchestra.run.smithi017.stdout: "tiebreaker_mon": "", 2024-09-10T21:46:58.655 INFO:teuthology.orchestra.run.smithi017.stdout: "removed_ranks: ": "", 2024-09-10T21:46:58.655 INFO:teuthology.orchestra.run.smithi017.stdout: "features": { 2024-09-10T21:46:58.656 INFO:teuthology.orchestra.run.smithi017.stdout: "persistent": [ 2024-09-10T21:46:58.656 INFO:teuthology.orchestra.run.smithi017.stdout: "kraken", 2024-09-10T21:46:58.656 INFO:teuthology.orchestra.run.smithi017.stdout: "luminous", 2024-09-10T21:46:58.656 INFO:teuthology.orchestra.run.smithi017.stdout: "mimic", 2024-09-10T21:46:58.656 INFO:teuthology.orchestra.run.smithi017.stdout: "osdmap-prune", 2024-09-10T21:46:58.656 INFO:teuthology.orchestra.run.smithi017.stdout: "nautilus", 2024-09-10T21:46:58.656 INFO:teuthology.orchestra.run.smithi017.stdout: "octopus", 2024-09-10T21:46:58.656 INFO:teuthology.orchestra.run.smithi017.stdout: "pacific", 2024-09-10T21:46:58.656 INFO:teuthology.orchestra.run.smithi017.stdout: "elector-pinging", 2024-09-10T21:46:58.656 INFO:teuthology.orchestra.run.smithi017.stdout: "quincy", 2024-09-10T21:46:58.656 INFO:teuthology.orchestra.run.smithi017.stdout: "reef", 2024-09-10T21:46:58.656 INFO:teuthology.orchestra.run.smithi017.stdout: "squid" 2024-09-10T21:46:58.657 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:46:58.657 INFO:teuthology.orchestra.run.smithi017.stdout: "optional": [] 2024-09-10T21:46:58.657 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:46:58.657 INFO:teuthology.orchestra.run.smithi017.stdout: "mons": [ 2024-09-10T21:46:58.657 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:46:58.657 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 0, 2024-09-10T21:46:58.657 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "a", 2024-09-10T21:46:58.657 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:46:58.657 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:46:58.657 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:46:58.657 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:46:58.657 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:3300", 2024-09-10T21:46:58.657 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:46:58.658 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:46:58.658 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:46:58.658 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:46:58.658 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789", 2024-09-10T21:46:58.658 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:46:58.658 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:46:58.658 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:46:58.658 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:46:58.658 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.17:6789/0", 2024-09-10T21:46:58.658 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.17:6789/0", 2024-09-10T21:46:58.658 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:46:58.658 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:46:58.658 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:46:58.658 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:46:58.659 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:46:58.659 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 1, 2024-09-10T21:46:58.659 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "b", 2024-09-10T21:46:58.659 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:46:58.659 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:46:58.659 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:46:58.659 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:46:58.659 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3300", 2024-09-10T21:46:58.659 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:46:58.659 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:46:58.659 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:46:58.659 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:46:58.659 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789", 2024-09-10T21:46:58.660 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:46:58.660 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:46:58.660 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:46:58.660 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:46:58.660 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6789/0", 2024-09-10T21:46:58.660 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6789/0", 2024-09-10T21:46:58.660 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:46:58.660 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:46:58.660 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:46:58.660 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:46:58.660 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:46:58.660 INFO:teuthology.orchestra.run.smithi017.stdout: "rank": 2, 2024-09-10T21:46:58.660 INFO:teuthology.orchestra.run.smithi017.stdout: "name": "c", 2024-09-10T21:46:58.661 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addrs": { 2024-09-10T21:46:58.661 INFO:teuthology.orchestra.run.smithi017.stdout: "addrvec": [ 2024-09-10T21:46:58.661 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:46:58.661 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v2", 2024-09-10T21:46:58.661 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:3301", 2024-09-10T21:46:58.661 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:46:58.661 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:46:58.661 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:46:58.661 INFO:teuthology.orchestra.run.smithi017.stdout: "type": "v1", 2024-09-10T21:46:58.661 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790", 2024-09-10T21:46:58.661 INFO:teuthology.orchestra.run.smithi017.stdout: "nonce": 0 2024-09-10T21:46:58.661 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:46:58.661 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:46:58.661 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:46:58.662 INFO:teuthology.orchestra.run.smithi017.stdout: "addr": "172.21.15.31:6790/0", 2024-09-10T21:46:58.662 INFO:teuthology.orchestra.run.smithi017.stdout: "public_addr": "172.21.15.31:6790/0", 2024-09-10T21:46:58.662 INFO:teuthology.orchestra.run.smithi017.stdout: "priority": 0, 2024-09-10T21:46:58.662 INFO:teuthology.orchestra.run.smithi017.stdout: "weight": 0, 2024-09-10T21:46:58.662 INFO:teuthology.orchestra.run.smithi017.stdout: "crush_location": "{}" 2024-09-10T21:46:58.662 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:46:58.662 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:46:58.662 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:46:58.662 INFO:teuthology.orchestra.run.smithi017.stdout: "feature_map": { 2024-09-10T21:46:58.662 INFO:teuthology.orchestra.run.smithi017.stdout: "mon": [ 2024-09-10T21:46:58.662 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:46:58.662 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:46:58.662 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:46:58.662 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:46:58.663 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:46:58.663 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:46:58.663 INFO:teuthology.orchestra.run.smithi017.stdout: "mds": [ 2024-09-10T21:46:58.663 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:46:58.663 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:46:58.663 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:46:58.663 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 3 2024-09-10T21:46:58.663 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:46:58.664 INFO:teuthology.orchestra.run.smithi017.stdout: ], 2024-09-10T21:46:58.664 INFO:teuthology.orchestra.run.smithi017.stdout: "client": [ 2024-09-10T21:46:58.664 INFO:teuthology.orchestra.run.smithi017.stdout: { 2024-09-10T21:46:58.664 INFO:teuthology.orchestra.run.smithi017.stdout: "features": "0x3f03dffffffdffff", 2024-09-10T21:46:58.664 INFO:teuthology.orchestra.run.smithi017.stdout: "release": "squid", 2024-09-10T21:46:58.665 INFO:teuthology.orchestra.run.smithi017.stdout: "num": 1 2024-09-10T21:46:58.665 INFO:teuthology.orchestra.run.smithi017.stdout: } 2024-09-10T21:46:58.665 INFO:teuthology.orchestra.run.smithi017.stdout: ] 2024-09-10T21:46:58.665 INFO:teuthology.orchestra.run.smithi017.stdout: }, 2024-09-10T21:46:58.665 INFO:teuthology.orchestra.run.smithi017.stdout: "stretch_mode": false 2024-09-10T21:46:58.665 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-10T21:46:58.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.645+0000 7ff9e3000640 1 -- 172.21.15.17:0/154556446 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7ff9b80724e0 msgr2=0x7ff9b80749a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:58.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.645+0000 7ff9e3000640 1 --2- 172.21.15.17:0/154556446 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7ff9b80724e0 0x7ff9b80749a0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7ff9d4001ec0 tx=0x7ff9d4001a00 comp rx=0 tx=0).stop 2024-09-10T21:46:58.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.645+0000 7ff9e3000640 1 -- 172.21.15.17:0/154556446 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff9dc0c6b40 msgr2=0x7ff9dc0c4fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:58.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.645+0000 7ff9e3000640 1 --2- 172.21.15.17:0/154556446 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff9dc0c6b40 0x7ff9dc0c4fc0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7ff9cc00c780 tx=0x7ff9cc00cc40 comp rx=0 tx=0).stop 2024-09-10T21:46:58.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.645+0000 7ff9e1000640 1 -- 172.21.15.17:0/154556446 reap_dead start 2024-09-10T21:46:58.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.645+0000 7ff9e1000640 1 -- 172.21.15.17:0/154556446 reap_dead start 2024-09-10T21:46:58.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.645+0000 7ff9e3000640 1 -- 172.21.15.17:0/154556446 shutdown_connections 2024-09-10T21:46:58.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.645+0000 7ff9e3000640 1 -- 172.21.15.17:0/154556446 >> 172.21.15.17:0/154556446 conn(0x7ff9dc0700d0 msgr2=0x7ff9dc157500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:46:58.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.645+0000 7ff9e3000640 1 -- 172.21.15.17:0/154556446 shutdown_connections 2024-09-10T21:46:58.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.645+0000 7ff9e3000640 1 -- 172.21.15.17:0/154556446 wait complete. 2024-09-10T21:46:58.670 INFO:tasks.mon_thrash.mon_thrasher:final quorum is size 3 2024-09-10T21:46:58.670 INFO:tasks.mon_thrash.mon_thrasher:triggering scrub 2024-09-10T21:46:58.670 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mon scrub 2024-09-10T21:46:58.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.817+0000 7f825f200640 1 -- 172.21.15.17:0/3911222296 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f825815c440 msgr2=0x7f8258070d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:58.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.817+0000 7f825f200640 1 --2- 172.21.15.17:0/3911222296 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f825815c440 0x7f8258070d80 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f8254009c80 tx=0x7f825401f5a0 comp rx=0 tx=0).stop 2024-09-10T21:46:58.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.817+0000 7f825f200640 1 -- 172.21.15.17:0/3911222296 shutdown_connections 2024-09-10T21:46:58.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.817+0000 7f825f200640 1 --2- 172.21.15.17:0/3911222296 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f825815c440 0x7f8258070d80 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:58.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.817+0000 7f825f200640 1 --2- 172.21.15.17:0/3911222296 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f82581524d0 0x7f825815bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:58.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.817+0000 7f825f200640 1 -- 172.21.15.17:0/3911222296 >> 172.21.15.17:0/3911222296 conn(0x7f82580cdef0 msgr2=0x7f82580ce2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:46:58.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.817+0000 7f825f200640 1 -- 172.21.15.17:0/3911222296 shutdown_connections 2024-09-10T21:46:58.822 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.817+0000 7f825f200640 1 -- 172.21.15.17:0/3911222296 wait complete. 2024-09-10T21:46:58.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.817+0000 7f825f200640 1 Processor -- start 2024-09-10T21:46:58.823 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.817+0000 7f825f200640 1 -- start start 2024-09-10T21:46:58.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.821+0000 7f825f200640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f82581524d0 0x7f82581f46d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:58.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.821+0000 7f825f200640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8258158c20 0x7f8258156fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:58.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.821+0000 7f825f200640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f825815c440 0x7f825806eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:58.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.821+0000 7f825f200640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f82580c9350 con 0x7f825815c440 2024-09-10T21:46:58.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.821+0000 7f825f200640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f825815b9e0 con 0x7f82581524d0 2024-09-10T21:46:58.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.821+0000 7f825f200640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f82580cea50 con 0x7f8258158c20 2024-09-10T21:46:58.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.821+0000 7f825e800640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f825815c440 0x7f825806eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:58.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.821+0000 7f825de00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f82581524d0 0x7f82581f46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:58.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.821+0000 7f825d400640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8258158c20 0x7f8258156fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:58.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.821+0000 7f825e800640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f825815c440 0x7f825806eb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:37164/0 (socket says 172.21.15.17:37164) 2024-09-10T21:46:58.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.821+0000 7f825e800640 1 -- 172.21.15.17:0/1313928615 learned_addr learned my addr 172.21.15.17:0/1313928615 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:46:58.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.821+0000 7f825de00640 1 -- 172.21.15.17:0/1313928615 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8258158c20 msgr2=0x7f8258156fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:58.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.821+0000 7f825de00640 1 --2- 172.21.15.17:0/1313928615 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8258158c20 0x7f8258156fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:58.826 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.821+0000 7f825de00640 1 -- 172.21.15.17:0/1313928615 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f825815c440 msgr2=0x7f825806eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:58.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.821+0000 7f825de00640 1 --2- 172.21.15.17:0/1313928615 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f825815c440 0x7f825806eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:58.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.821+0000 7f825de00640 1 -- 172.21.15.17:0/1313928615 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8258151b60 con 0x7f82581524d0 2024-09-10T21:46:58.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.821+0000 7f825e800640 1 --2- 172.21.15.17:0/1313928615 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f825815c440 0x7f825806eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:46:58.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.821+0000 7f825de00640 1 --2- 172.21.15.17:0/1313928615 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f82581524d0 0x7f82581f46d0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f824c00d6d0 tx=0x7f824c00db90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:46:58.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.821+0000 7f8252000640 1 -- 172.21.15.17:0/1313928615 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f824c017070 con 0x7f82581524d0 2024-09-10T21:46:58.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.821+0000 7f8252000640 1 -- 172.21.15.17:0/1313928615 <== mon.1 v2:172.21.15.31:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f824c005310 con 0x7f82581524d0 2024-09-10T21:46:58.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.821+0000 7f8252000640 1 -- 172.21.15.17:0/1313928615 <== mon.1 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f824c009820 con 0x7f82581524d0 2024-09-10T21:46:58.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.821+0000 7f825f200640 1 -- 172.21.15.17:0/1313928615 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f825806f8d0 con 0x7f82581524d0 2024-09-10T21:46:58.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.821+0000 7f825f200640 1 -- 172.21.15.17:0/1313928615 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f825806e340 con 0x7f82581524d0 2024-09-10T21:46:58.827 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.821+0000 7f825f200640 1 -- 172.21.15.17:0/1313928615 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f825806e130 con 0x7f82581524d0 2024-09-10T21:46:58.828 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.825+0000 7f8252000640 1 -- 172.21.15.17:0/1313928615 <== mon.1 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f824c009bd0 con 0x7f82581524d0 2024-09-10T21:46:58.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.825+0000 7f8252000640 1 --2- 172.21.15.17:0/1313928615 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f823c0725e0 0x7f823c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:46:58.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.825+0000 7f8252000640 1 -- 172.21.15.17:0/1313928615 <== mon.1 v2:172.21.15.31:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f824c0938e0 con 0x7f82581524d0 2024-09-10T21:46:58.829 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.825+0000 7f825d400640 1 --2- 172.21.15.17:0/1313928615 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f823c0725e0 0x7f823c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:46:58.830 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.825+0000 7f825d400640 1 --2- 172.21.15.17:0/1313928615 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f823c0725e0 0x7f823c074aa0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f825814c500 tx=0x7f8248009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:46:58.833 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:58.829+0000 7f8252000640 1 -- 172.21.15.17:0/1313928615 <== mon.1 v2:172.21.15.31:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f824c05e600 con 0x7f82581524d0 2024-09-10T21:46:59.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:59.069+0000 7f825f200640 1 -- 172.21.15.17:0/1313928615 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_command({"prefix": "mon scrub"} v 0) -- 0x7f82580713a0 con 0x7f82581524d0 2024-09-10T21:46:59.076 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:59.073+0000 7f8252000640 1 -- 172.21.15.17:0/1313928615 <== mon.1 v2:172.21.15.31:3300/0 7 ==== mon_command_ack([{"prefix": "mon scrub"}]=0 v0) ==== 57+0+0 (secure 0 0 0) 0x7f824c067370 con 0x7f82581524d0 2024-09-10T21:46:59.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:59.073+0000 7f825f200640 1 -- 172.21.15.17:0/1313928615 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f823c0725e0 msgr2=0x7f823c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:59.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:59.073+0000 7f825f200640 1 --2- 172.21.15.17:0/1313928615 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f823c0725e0 0x7f823c074aa0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f825814c500 tx=0x7f8248009040 comp rx=0 tx=0).stop 2024-09-10T21:46:59.080 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:59.073+0000 7f825f200640 1 -- 172.21.15.17:0/1313928615 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f82581524d0 msgr2=0x7f82581f46d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:46:59.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:59.073+0000 7f825f200640 1 --2- 172.21.15.17:0/1313928615 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f82581524d0 0x7f82581f46d0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f824c00d6d0 tx=0x7f824c00db90 comp rx=0 tx=0).stop 2024-09-10T21:46:59.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:59.073+0000 7f825f200640 1 -- 172.21.15.17:0/1313928615 shutdown_connections 2024-09-10T21:46:59.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:59.077+0000 7f825f200640 1 --2- 172.21.15.17:0/1313928615 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f823c0725e0 0x7f823c074aa0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:59.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:59.077+0000 7f825f200640 1 --2- 172.21.15.17:0/1313928615 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f825815c440 0x7f825806eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:59.081 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:59.077+0000 7f825f200640 1 --2- 172.21.15.17:0/1313928615 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8258158c20 0x7f8258156fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:59.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:59.077+0000 7f825f200640 1 --2- 172.21.15.17:0/1313928615 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f82581524d0 0x7f82581f46d0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:46:59.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:59.077+0000 7f825f200640 1 -- 172.21.15.17:0/1313928615 >> 172.21.15.17:0/1313928615 conn(0x7f82580cdef0 msgr2=0x7f82580ca6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:46:59.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:59.077+0000 7f825f200640 1 -- 172.21.15.17:0/1313928615 shutdown_connections 2024-09-10T21:46:59.082 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:46:59.077+0000 7f825f200640 1 -- 172.21.15.17:0/1313928615 wait complete. 2024-09-10T21:46:59.101 INFO:tasks.mon_thrash.mon_thrasher:waiting for 10.0 secs before continuing thrashing 2024-09-10T21:47:09.102 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:47:09.251 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:09.245+0000 7f6fc1a00640 1 -- 172.21.15.17:0/2919319045 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6fbc0c6270 msgr2=0x7f6fbc0c7d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:09.251 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:09.245+0000 7f6fc1a00640 1 --2- 172.21.15.17:0/2919319045 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6fbc0c6270 0x7f6fbc0c7d90 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f6fac009840 tx=0x7f6fac01c820 comp rx=0 tx=0).stop 2024-09-10T21:47:09.251 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:09.245+0000 7f6fc1a00640 1 -- 172.21.15.17:0/2919319045 shutdown_connections 2024-09-10T21:47:09.251 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:09.245+0000 7f6fc1a00640 1 --2- 172.21.15.17:0/2919319045 >> v2:172.21.15.31:3300/0 conn(0x7f6fbc06dca0 0x7f6fbc06e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:09.251 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:09.245+0000 7f6fc1a00640 1 --2- 172.21.15.17:0/2919319045 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6fbc0c6270 0x7f6fbc0c7d90 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:09.251 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:09.245+0000 7f6fc1a00640 1 -- 172.21.15.17:0/2919319045 >> 172.21.15.17:0/2919319045 conn(0x7f6fbc070160 msgr2=0x7f6fbc070570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:09.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:09.245+0000 7f6fc1a00640 1 -- 172.21.15.17:0/2919319045 shutdown_connections 2024-09-10T21:47:09.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:09.245+0000 7f6fc1a00640 1 -- 172.21.15.17:0/2919319045 wait complete. 2024-09-10T21:47:09.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:09.249+0000 7f6fc1a00640 1 Processor -- start 2024-09-10T21:47:09.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:09.249+0000 7f6fc1a00640 1 -- start start 2024-09-10T21:47:09.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:09.249+0000 7f6fc1a00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6fbc06dca0 0x7f6fbc0c51a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:09.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:09.249+0000 7f6fc1a00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6fbc0c6270 0x7f6fbc14f500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:09.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:09.249+0000 7f6fc1a00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6fbc15b910 0x7f6fbc14a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:09.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:09.249+0000 7f6fc1a00640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f6fbc0726a0 con 0x7f6fbc06dca0 2024-09-10T21:47:09.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:09.249+0000 7f6fc1a00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f6fbc06e600 con 0x7f6fbc0c6270 2024-09-10T21:47:09.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:09.249+0000 7f6fc1a00640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f6fbc15c4d0 con 0x7f6fbc15b910 2024-09-10T21:47:09.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:09.249+0000 7f6fbaa00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6fbc06dca0 0x7f6fbc0c51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:09.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:09.249+0000 7f6fbaa00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6fbc06dca0 0x7f6fbc0c51a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:36740/0 (socket says 172.21.15.17:36740) 2024-09-10T21:47:09.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:09.249+0000 7f6fbaa00640 1 -- 172.21.15.17:0/1611370458 learned_addr learned my addr 172.21.15.17:0/1611370458 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:47:09.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:09.249+0000 7f6fbaa00640 1 -- 172.21.15.17:0/1611370458 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6fbc15b910 msgr2=0x7f6fbc14a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:09.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:09.249+0000 7f6fbaa00640 1 --2- 172.21.15.17:0/1611370458 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6fbc15b910 0x7f6fbc14a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:09.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:09.249+0000 7f6fbaa00640 1 -- 172.21.15.17:0/1611370458 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6fbc0c6270 msgr2=0x7f6fbc14f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:09.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:09.249+0000 7f6fb3e00640 1 --2- 172.21.15.17:0/1611370458 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6fbc0c6270 0x7f6fbc14f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:09.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:09.249+0000 7f6fbaa00640 1 --2- 172.21.15.17:0/1611370458 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6fbc0c6270 0x7f6fbc14f500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:09.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:09.249+0000 7f6fbaa00640 1 -- 172.21.15.17:0/1611370458 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fbc14c9a0 con 0x7f6fbc06dca0 2024-09-10T21:47:09.255 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:09.249+0000 7f6fbaa00640 1 --2- 172.21.15.17:0/1611370458 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6fbc06dca0 0x7f6fbc0c51a0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f6fac01aed0 tx=0x7f6fac004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:09.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:09.249+0000 7f6fb2a00640 1 -- 172.21.15.17:0/1611370458 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fac0b2040 con 0x7f6fbc06dca0 2024-09-10T21:47:09.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:09.249+0000 7f6fb2a00640 1 -- 172.21.15.17:0/1611370458 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6fac005020 con 0x7f6fbc06dca0 2024-09-10T21:47:09.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:09.249+0000 7f6fb2a00640 1 -- 172.21.15.17:0/1611370458 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fac005450 con 0x7f6fbc06dca0 2024-09-10T21:47:09.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:09.249+0000 7f6fc1a00640 1 -- 172.21.15.17:0/1611370458 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fbc0c38a0 con 0x7f6fbc06dca0 2024-09-10T21:47:09.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:09.249+0000 7f6fc1a00640 1 -- 172.21.15.17:0/1611370458 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fbc14cf20 con 0x7f6fbc06dca0 2024-09-10T21:47:09.256 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:09.253+0000 7f6fc1a00640 1 -- 172.21.15.17:0/1611370458 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6fbc0c3a70 con 0x7f6fbc06dca0 2024-09-10T21:47:09.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:09.253+0000 7f6fb2a00640 1 -- 172.21.15.17:0/1611370458 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f6fac0b5070 con 0x7f6fbc06dca0 2024-09-10T21:47:09.257 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:09.253+0000 7f6fb2a00640 1 --2- 172.21.15.17:0/1611370458 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f6f980725e0 0x7f6f98074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:09.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:09.253+0000 7f6fb2a00640 1 -- 172.21.15.17:0/1611370458 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f6fac12cd80 con 0x7f6fbc06dca0 2024-09-10T21:47:09.258 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:09.253+0000 7f6fb3e00640 1 --2- 172.21.15.17:0/1611370458 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f6f980725e0 0x7f6f98074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:09.259 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:09.253+0000 7f6fb3e00640 1 --2- 172.21.15.17:0/1611370458 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f6f980725e0 0x7f6f98074aa0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f6fa40096c0 tx=0x7f6fa4009260 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:09.262 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:09.257+0000 7f6fb2a00640 1 -- 172.21.15.17:0/1611370458 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f6fac016410 con 0x7f6fbc06dca0 2024-09-10T21:47:09.487 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:09.481+0000 7f6fc1a00640 1 -- 172.21.15.17:0/1611370458 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6fbc0c7d90 con 0x7f6fbc06dca0 2024-09-10T21:47:09.488 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:09.485+0000 7f6fb2a00640 1 -- 172.21.15.17:0/1611370458 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+6870 (secure 0 0 0) 0x7f6fac0f80e0 con 0x7f6fbc06dca0 2024-09-10T21:47:09.489 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-10T21:47:09.489 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":10,"btime":"2024-09-10T21:25:16:432823+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4350,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/1643071167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":1643071167},{"type":"v1","addr":"172.21.15.31:6835","nonce":1643071167}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4356,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6837/1666225712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6836","nonce":1666225712},{"type":"v1","addr":"172.21.15.17:6837","nonce":1666225712}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4384,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/3592734043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":3592734043},{"type":"v1","addr":"172.21.15.17:6833","nonce":3592734043}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4405,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/1293665856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":1293665856},{"type":"v1","addr":"172.21.15.17:6835","nonce":1293665856}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4411,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6838/1944852172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":1944852172},{"type":"v1","addr":"172.21.15.31:6838","nonce":1944852172}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":10,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:24:47.255397+0000","modified":"2024-09-10T21:25:15.961582+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4414},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4414":{"gid":4414,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.31:6839/3129062106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6837","nonce":3129062106},{"type":"v1","addr":"172.21.15.31:6839","nonce":3129062106}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4414,"qdb_cluster":[4414]},"id":1}]} 2024-09-10T21:47:09.489 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 10 2024-09-10T21:47:09.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:09.485+0000 7f6fc1a00640 1 -- 172.21.15.17:0/1611370458 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f6f980725e0 msgr2=0x7f6f98074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:09.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:09.485+0000 7f6fc1a00640 1 --2- 172.21.15.17:0/1611370458 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f6f980725e0 0x7f6f98074aa0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f6fa40096c0 tx=0x7f6fa4009260 comp rx=0 tx=0).stop 2024-09-10T21:47:09.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:09.485+0000 7f6fc1a00640 1 -- 172.21.15.17:0/1611370458 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6fbc06dca0 msgr2=0x7f6fbc0c51a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:09.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:09.485+0000 7f6fc1a00640 1 --2- 172.21.15.17:0/1611370458 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6fbc06dca0 0x7f6fbc0c51a0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f6fac01aed0 tx=0x7f6fac004040 comp rx=0 tx=0).stop 2024-09-10T21:47:09.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:09.485+0000 7f6fc1a00640 1 -- 172.21.15.17:0/1611370458 shutdown_connections 2024-09-10T21:47:09.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:09.485+0000 7f6fc1a00640 1 --2- 172.21.15.17:0/1611370458 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f6f980725e0 0x7f6f98074aa0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:09.491 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:09.485+0000 7f6fc1a00640 1 --2- 172.21.15.17:0/1611370458 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6fbc15b910 0x7f6fbc14a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:09.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:09.485+0000 7f6fc1a00640 1 --2- 172.21.15.17:0/1611370458 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f6fbc0c6270 0x7f6fbc14f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:09.492 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:09.485+0000 7f6fc1a00640 1 --2- 172.21.15.17:0/1611370458 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6fbc06dca0 0x7f6fbc0c51a0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:09.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:09.485+0000 7f6fc1a00640 1 -- 172.21.15.17:0/1611370458 >> 172.21.15.17:0/1611370458 conn(0x7f6fbc070160 msgr2=0x7f6fbc1ea1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:09.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:09.485+0000 7f6fc1a00640 1 -- 172.21.15.17:0/1611370458 shutdown_connections 2024-09-10T21:47:09.493 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:09.485+0000 7f6fc1a00640 1 -- 172.21.15.17:0/1611370458 wait complete. 2024-09-10T21:47:09.513 INFO:tasks.mon_thrash.ceph_manager:waiting for quorum size 3 2024-09-10T21:47:09.514 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-09-10T21:47:09.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:09.657+0000 7f5a66a00640 1 -- 172.21.15.17:0/3024788345 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a600cdf20 msgr2=0x7f5a600ce3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:09.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:09.657+0000 7f5a66a00640 1 --2- 172.21.15.17:0/3024788345 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a600cdf20 0x7f5a600ce3b0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f5a58009c80 tx=0x7f5a5801cd10 comp rx=0 tx=0).stop 2024-09-10T21:47:09.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:09.657+0000 7f5a66a00640 1 -- 172.21.15.17:0/3024788345 shutdown_connections 2024-09-10T21:47:09.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:09.657+0000 7f5a66a00640 1 --2- 172.21.15.17:0/3024788345 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a600cdf20 0x7f5a600ce3b0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:09.662 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:09.657+0000 7f5a66a00640 1 --2- 172.21.15.17:0/3024788345 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f5a60152820 0x7f5a600b3eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:09.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:09.657+0000 7f5a66a00640 1 -- 172.21.15.17:0/3024788345 >> 172.21.15.17:0/3024788345 conn(0x7f5a600702f0 msgr2=0x7f5a60070700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:09.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:09.657+0000 7f5a66a00640 1 -- 172.21.15.17:0/3024788345 shutdown_connections 2024-09-10T21:47:09.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:09.657+0000 7f5a66a00640 1 -- 172.21.15.17:0/3024788345 wait complete. 2024-09-10T21:47:09.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:09.661+0000 7f5a66a00640 1 Processor -- start 2024-09-10T21:47:09.663 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:09.661+0000 7f5a66a00640 1 -- start start 2024-09-10T21:47:09.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:09.661+0000 7f5a66a00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a600cdf20 0x7f5a6006f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:09.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:09.661+0000 7f5a66a00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5a60152440 0x7f5a60143c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:09.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:09.661+0000 7f5a66a00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f5a60152820 0x7f5a60156f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:09.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:09.661+0000 7f5a66a00640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f5a600cb5d0 con 0x7f5a600cdf20 2024-09-10T21:47:09.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:09.661+0000 7f5a66a00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f5a600cb450 con 0x7f5a60152440 2024-09-10T21:47:09.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:09.661+0000 7f5a66a00640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f5a600c2ff0 con 0x7f5a60152820 2024-09-10T21:47:09.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:09.661+0000 7f5a5fe00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a600cdf20 0x7f5a6006f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:09.665 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:09.661+0000 7f5a5fe00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a600cdf20 0x7f5a6006f3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:36762/0 (socket says 172.21.15.17:36762) 2024-09-10T21:47:09.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:09.661+0000 7f5a5fe00640 1 -- 172.21.15.17:0/1993385501 learned_addr learned my addr 172.21.15.17:0/1993385501 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:47:09.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:09.661+0000 7f5a64a00640 1 --2- 172.21.15.17:0/1993385501 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f5a60152820 0x7f5a60156f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:09.666 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:09.661+0000 7f5a57e00640 1 --2- 172.21.15.17:0/1993385501 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5a60152440 0x7f5a60143c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:09.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:09.661+0000 7f5a5fe00640 1 -- 172.21.15.17:0/1993385501 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f5a60152820 msgr2=0x7f5a60156f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:09.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:09.661+0000 7f5a5fe00640 1 --2- 172.21.15.17:0/1993385501 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f5a60152820 0x7f5a60156f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:09.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:09.661+0000 7f5a5fe00640 1 -- 172.21.15.17:0/1993385501 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5a60152440 msgr2=0x7f5a60143c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:09.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:09.661+0000 7f5a5fe00640 1 --2- 172.21.15.17:0/1993385501 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5a60152440 0x7f5a60143c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:09.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:09.661+0000 7f5a5fe00640 1 -- 172.21.15.17:0/1993385501 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a600b38c0 con 0x7f5a600cdf20 2024-09-10T21:47:09.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:09.661+0000 7f5a5fe00640 1 --2- 172.21.15.17:0/1993385501 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a600cdf20 0x7f5a6006f3f0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f5a5000c530 tx=0x7f5a5000c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:09.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:09.661+0000 7f5a5cc00640 1 -- 172.21.15.17:0/1993385501 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a50017070 con 0x7f5a600cdf20 2024-09-10T21:47:09.667 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:09.661+0000 7f5a5cc00640 1 -- 172.21.15.17:0/1993385501 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5a500053c0 con 0x7f5a600cdf20 2024-09-10T21:47:09.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:09.661+0000 7f5a5cc00640 1 -- 172.21.15.17:0/1993385501 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a50012890 con 0x7f5a600cdf20 2024-09-10T21:47:09.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:09.661+0000 7f5a66a00640 1 -- 172.21.15.17:0/1993385501 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a6006e730 con 0x7f5a600cdf20 2024-09-10T21:47:09.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:09.661+0000 7f5a66a00640 1 -- 172.21.15.17:0/1993385501 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a601560d0 con 0x7f5a600cdf20 2024-09-10T21:47:09.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:09.661+0000 7f5a66a00640 1 -- 172.21.15.17:0/1993385501 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a60072f70 con 0x7f5a600cdf20 2024-09-10T21:47:09.668 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:09.665+0000 7f5a5cc00640 1 -- 172.21.15.17:0/1993385501 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f5a50003600 con 0x7f5a600cdf20 2024-09-10T21:47:09.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:09.665+0000 7f5a5cc00640 1 --2- 172.21.15.17:0/1993385501 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f5a3c0725e0 0x7f5a3c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:09.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:09.665+0000 7f5a5cc00640 1 -- 172.21.15.17:0/1993385501 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f5a50092be0 con 0x7f5a600cdf20 2024-09-10T21:47:09.669 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:09.665+0000 7f5a57e00640 1 --2- 172.21.15.17:0/1993385501 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f5a3c0725e0 0x7f5a3c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:09.670 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:09.665+0000 7f5a57e00640 1 --2- 172.21.15.17:0/1993385501 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f5a3c0725e0 0x7f5a3c074aa0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f5a6006ec80 tx=0x7f5a480091e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:09.673 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:09.669+0000 7f5a5cc00640 1 -- 172.21.15.17:0/1993385501 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f5a6006e730 con 0x7f5a600cdf20 2024-09-10T21:47:09.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:09.913+0000 7f5a66a00640 1 -- 172.21.15.17:0/1993385501 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "quorum_status"} v 0) -- 0x7f5a60157b40 con 0x7f5a600cdf20 2024-09-10T21:47:09.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:09.917+0000 7f5a5cc00640 1 -- 172.21.15.17:0/1993385501 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "quorum_status"}]=0 v0) ==== 61+0+1558 (secure 0 0 0) 0x7f5a5005d860 con 0x7f5a600cdf20 2024-09-10T21:47:09.920 INFO:teuthology.orchestra.run.smithi017.stdout:{"election_epoch":172,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":12,"features":{"quorum_con":"4540719139924082687","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"5e2f023f-d6a6-49f4-a256-f4807dce3739","modified":"2024-09-10T21:23:46.892130Z","created":"2024-09-10T21:23:46.892130Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:3300","nonce":0},{"type":"v1","addr":"172.21.15.17:6789","nonce":0}]},"addr":"172.21.15.17:6789/0","public_addr":"172.21.15.17:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3300","nonce":0},{"type":"v1","addr":"172.21.15.31:6789","nonce":0}]},"addr":"172.21.15.31:6789/0","public_addr":"172.21.15.31:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:3301","nonce":0},{"type":"v1","addr":"172.21.15.31:6790","nonce":0}]},"addr":"172.21.15.31:6790/0","public_addr":"172.21.15.31:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-09-10T21:47:09.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:09.917+0000 7f5a66a00640 1 -- 172.21.15.17:0/1993385501 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f5a3c0725e0 msgr2=0x7f5a3c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:09.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:09.917+0000 7f5a66a00640 1 --2- 172.21.15.17:0/1993385501 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f5a3c0725e0 0x7f5a3c074aa0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f5a6006ec80 tx=0x7f5a480091e0 comp rx=0 tx=0).stop 2024-09-10T21:47:09.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:09.917+0000 7f5a66a00640 1 -- 172.21.15.17:0/1993385501 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a600cdf20 msgr2=0x7f5a6006f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:09.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:09.917+0000 7f5a66a00640 1 --2- 172.21.15.17:0/1993385501 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a600cdf20 0x7f5a6006f3f0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f5a5000c530 tx=0x7f5a5000c9f0 comp rx=0 tx=0).stop 2024-09-10T21:47:09.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:09.917+0000 7f5a66a00640 1 -- 172.21.15.17:0/1993385501 shutdown_connections 2024-09-10T21:47:09.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:09.917+0000 7f5a66a00640 1 --2- 172.21.15.17:0/1993385501 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f5a3c0725e0 0x7f5a3c074aa0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:09.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:09.917+0000 7f5a66a00640 1 --2- 172.21.15.17:0/1993385501 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f5a60152820 0x7f5a60156f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:09.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:09.917+0000 7f5a66a00640 1 --2- 172.21.15.17:0/1993385501 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f5a60152440 0x7f5a60143c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:09.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:09.917+0000 7f5a66a00640 1 --2- 172.21.15.17:0/1993385501 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f5a600cdf20 0x7f5a6006f3f0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:09.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:09.917+0000 7f5a66a00640 1 -- 172.21.15.17:0/1993385501 >> 172.21.15.17:0/1993385501 conn(0x7f5a600702f0 msgr2=0x7f5a600706d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:09.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:09.917+0000 7f5a66a00640 1 -- 172.21.15.17:0/1993385501 shutdown_connections 2024-09-10T21:47:09.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:09.917+0000 7f5a66a00640 1 -- 172.21.15.17:0/1993385501 wait complete. 2024-09-10T21:47:09.942 INFO:tasks.mon_thrash.ceph_manager:quorum is size 3 2024-09-10T21:47:09.942 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-09-10T21:47:09.956 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-09-10T21:47:09.956 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-09-10T21:47:09.956 DEBUG:teuthology.orchestra.run.smithi017:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:47:10.337 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-09-10T21:47:10.337 DEBUG:teuthology.orchestra.run.smithi017:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:47:10.388 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T21:47:10.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:10.581+0000 7f2ee8200640 1 -- 172.21.15.17:0/2049393894 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2ec8004810 msgr2=0x7f2ec8004c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:10.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:10.581+0000 7f2ee8200640 1 --2- 172.21.15.17:0/2049393894 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2ec8004810 0x7f2ec8004c90 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f2ecc0016f0 tx=0x7f2ecc01c740 comp rx=0 tx=0).stop 2024-09-10T21:47:10.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:10.581+0000 7f2ee8200640 1 -- 172.21.15.17:0/2049393894 shutdown_connections 2024-09-10T21:47:10.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:10.581+0000 7f2ee8200640 1 --2- 172.21.15.17:0/2049393894 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2ec8004810 0x7f2ec8004c90 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:10.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:10.581+0000 7f2ee8200640 1 -- 172.21.15.17:0/2049393894 >> 172.21.15.17:0/2049393894 conn(0x7f2ee00702f0 msgr2=0x7f2ee0070700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:10.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:10.581+0000 7f2ee8200640 1 -- 172.21.15.17:0/2049393894 shutdown_connections 2024-09-10T21:47:10.584 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:10.581+0000 7f2ee8200640 1 -- 172.21.15.17:0/2049393894 wait complete. 2024-09-10T21:47:10.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:10.581+0000 7f2ee8200640 1 Processor -- start 2024-09-10T21:47:10.585 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:10.581+0000 7f2ee8200640 1 -- start start 2024-09-10T21:47:10.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:10.581+0000 7f2ee8200640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ec8004810 0x7f2ee01e4130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:10.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:10.581+0000 7f2ee8200640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2ee0157240 0x7f2ee0068c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:10.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:10.581+0000 7f2ee8200640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f2ee0151980 0x7f2ee0150ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:10.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:10.581+0000 7f2ee8200640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee00cb450 con 0x7f2ee0157240 2024-09-10T21:47:10.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:10.581+0000 7f2ee8200640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f2ee00b4290 con 0x7f2ec8004810 2024-09-10T21:47:10.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:10.581+0000 7f2ee8200640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f2ee00cb5d0 con 0x7f2ee0151980 2024-09-10T21:47:10.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:10.581+0000 7f2ee4e00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2ee0157240 0x7f2ee0068c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:10.587 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:10.581+0000 7f2ee5800640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ec8004810 0x7f2ee01e4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:10.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:10.581+0000 7f2ee6200640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f2ee0151980 0x7f2ee0150ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:10.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:10.581+0000 7f2ee4e00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2ee0157240 0x7f2ee0068c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:36778/0 (socket says 172.21.15.17:36778) 2024-09-10T21:47:10.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:10.581+0000 7f2ee4e00640 1 -- 172.21.15.17:0/52292419 learned_addr learned my addr 172.21.15.17:0/52292419 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:47:10.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:10.581+0000 7f2ee4e00640 1 -- 172.21.15.17:0/52292419 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f2ee0151980 msgr2=0x7f2ee0150ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:10.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:10.581+0000 7f2ee4e00640 1 --2- 172.21.15.17:0/52292419 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f2ee0151980 0x7f2ee0150ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:10.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:10.581+0000 7f2ee4e00640 1 -- 172.21.15.17:0/52292419 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ec8004810 msgr2=0x7f2ee01e4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:10.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:10.581+0000 7f2ee4e00640 1 --2- 172.21.15.17:0/52292419 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ec8004810 0x7f2ee01e4130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:10.588 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:10.581+0000 7f2ee4e00640 1 -- 172.21.15.17:0/52292419 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ee01f7770 con 0x7f2ee0157240 2024-09-10T21:47:10.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:10.581+0000 7f2ee4e00640 1 --2- 172.21.15.17:0/52292419 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2ee0157240 0x7f2ee0068c20 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f2ecc01ae40 tx=0x7f2ecc006930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:10.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:10.585+0000 7f2edd600640 1 -- 172.21.15.17:0/52292419 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ecc003680 con 0x7f2ee0157240 2024-09-10T21:47:10.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:10.585+0000 7f2edd600640 1 -- 172.21.15.17:0/52292419 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2ecc003910 con 0x7f2ee0157240 2024-09-10T21:47:10.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:10.585+0000 7f2edd600640 1 -- 172.21.15.17:0/52292419 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ecc003d00 con 0x7f2ee0157240 2024-09-10T21:47:10.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:10.585+0000 7f2ee8200640 1 -- 172.21.15.17:0/52292419 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ee01f7a50 con 0x7f2ee0157240 2024-09-10T21:47:10.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:10.585+0000 7f2ee8200640 1 -- 172.21.15.17:0/52292419 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ee0157700 con 0x7f2ee0157240 2024-09-10T21:47:10.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:10.585+0000 7f2ee8200640 1 -- 172.21.15.17:0/52292419 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ee00c3160 con 0x7f2ee0157240 2024-09-10T21:47:10.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:10.585+0000 7f2edd600640 1 -- 172.21.15.17:0/52292419 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f2ecc0a5580 con 0x7f2ee0157240 2024-09-10T21:47:10.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:10.585+0000 7f2edd600640 1 --2- 172.21.15.17:0/52292419 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f2ec0072610 0x7f2ec0074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:10.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:10.585+0000 7f2edd600640 1 -- 172.21.15.17:0/52292419 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f2ecc12d080 con 0x7f2ee0157240 2024-09-10T21:47:10.591 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:10.585+0000 7f2ee5800640 1 --2- 172.21.15.17:0/52292419 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f2ec0072610 0x7f2ec0074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:10.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:10.589+0000 7f2ee5800640 1 --2- 172.21.15.17:0/52292419 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f2ec0072610 0x7f2ec0074ad0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f2ed4001ac0 tx=0x7f2ed4001d20 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:10.594 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:10.589+0000 7f2edd600640 1 -- 172.21.15.17:0/52292419 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f2ecc0b3070 con 0x7f2ee0157240 2024-09-10T21:47:10.780 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:10.777+0000 7f2ee8200640 1 -- 172.21.15.17:0/52292419 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f2ee01f3750 con 0x7f2ee0157240 2024-09-10T21:47:10.780 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:10.777+0000 7f2edd600640 1 -- 172.21.15.17:0/52292419 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) ==== 80+0+0 (secure 0 0 0) 0x7f2ecc0164d0 con 0x7f2ee0157240 2024-09-10T21:47:10.781 INFO:teuthology.orchestra.run.smithi017.stderr:listed 0 entries 2024-09-10T21:47:10.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:10.777+0000 7f2ee8200640 1 -- 172.21.15.17:0/52292419 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f2ec0072610 msgr2=0x7f2ec0074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:10.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:10.777+0000 7f2ee8200640 1 --2- 172.21.15.17:0/52292419 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f2ec0072610 0x7f2ec0074ad0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f2ed4001ac0 tx=0x7f2ed4001d20 comp rx=0 tx=0).stop 2024-09-10T21:47:10.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:10.777+0000 7f2ee8200640 1 -- 172.21.15.17:0/52292419 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2ee0157240 msgr2=0x7f2ee0068c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:10.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:10.777+0000 7f2ee8200640 1 --2- 172.21.15.17:0/52292419 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2ee0157240 0x7f2ee0068c20 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f2ecc01ae40 tx=0x7f2ecc006930 comp rx=0 tx=0).stop 2024-09-10T21:47:10.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:10.777+0000 7f2ee8200640 1 -- 172.21.15.17:0/52292419 shutdown_connections 2024-09-10T21:47:10.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:10.777+0000 7f2ee8200640 1 --2- 172.21.15.17:0/52292419 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f2ec0072610 0x7f2ec0074ad0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:10.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:10.777+0000 7f2ee8200640 1 --2- 172.21.15.17:0/52292419 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f2ee0151980 0x7f2ee0150ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:10.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:10.777+0000 7f2ee8200640 1 --2- 172.21.15.17:0/52292419 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2ee0157240 0x7f2ee0068c20 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:10.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:10.777+0000 7f2ee8200640 1 --2- 172.21.15.17:0/52292419 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2ec8004810 0x7f2ee01e4130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:10.783 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:10.777+0000 7f2ee8200640 1 -- 172.21.15.17:0/52292419 >> 172.21.15.17:0/52292419 conn(0x7f2ee00702f0 msgr2=0x7f2ee00ca650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:10.784 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:10.777+0000 7f2ee8200640 1 -- 172.21.15.17:0/52292419 shutdown_connections 2024-09-10T21:47:10.784 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:10.777+0000 7f2ee8200640 1 -- 172.21.15.17:0/52292419 wait complete. 2024-09-10T21:47:10.803 DEBUG:teuthology.orchestra.run.smithi017:> set -ex 2024-09-10T21:47:10.803 DEBUG:teuthology.orchestra.run.smithi017:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T21:47:10.812 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-10T21:47:11.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:11.001+0000 7f4804200640 1 -- 172.21.15.17:0/820360007 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f47fc152820 msgr2=0x7f47fc0b3eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:11.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:11.001+0000 7f4804200640 1 --2- 172.21.15.17:0/820360007 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f47fc152820 0x7f47fc0b3eb0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f47fc1594a0 tx=0x7f47e801c620 comp rx=0 tx=0).stop 2024-09-10T21:47:11.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:11.001+0000 7f4804200640 1 -- 172.21.15.17:0/820360007 shutdown_connections 2024-09-10T21:47:11.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:11.001+0000 7f4804200640 1 --2- 172.21.15.17:0/820360007 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f47fc152820 0x7f47fc0b3eb0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:11.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:11.001+0000 7f4804200640 1 -- 172.21.15.17:0/820360007 >> 172.21.15.17:0/820360007 conn(0x7f47fc0702f0 msgr2=0x7f47fc070700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:11.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:11.001+0000 7f4804200640 1 -- 172.21.15.17:0/820360007 shutdown_connections 2024-09-10T21:47:11.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:11.001+0000 7f4804200640 1 -- 172.21.15.17:0/820360007 wait complete. 2024-09-10T21:47:11.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:11.001+0000 7f4804200640 1 Processor -- start 2024-09-10T21:47:11.006 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:11.001+0000 7f4804200640 1 -- start start 2024-09-10T21:47:11.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:11.005+0000 7f4804200640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f47fc0cdf20 0x7f47fc0c6630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:11.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:11.005+0000 7f4804200640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f47fc152440 0x7f47fc152c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:11.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:11.005+0000 7f4804200640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f47fc152820 0x7f47fc153b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:11.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:11.005+0000 7f4804200640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f47fc15bd60 con 0x7f47fc0cdf20 2024-09-10T21:47:11.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:11.005+0000 7f4804200640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f47fc15ba80 con 0x7f47fc152440 2024-09-10T21:47:11.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:11.005+0000 7f4804200640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f47fc0cb450 con 0x7f47fc152820 2024-09-10T21:47:11.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:11.005+0000 7f4801800640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f47fc0cdf20 0x7f47fc0c6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:11.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:11.005+0000 7f4801800640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f47fc0cdf20 0x7f47fc0c6630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:36790/0 (socket says 172.21.15.17:36790) 2024-09-10T21:47:11.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:11.005+0000 7f4801800640 1 -- 172.21.15.17:0/4185406166 learned_addr learned my addr 172.21.15.17:0/4185406166 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:47:11.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:11.005+0000 7f4801800640 1 -- 172.21.15.17:0/4185406166 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f47fc152820 msgr2=0x7f47fc153b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:11.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:11.005+0000 7f4801800640 1 --2- 172.21.15.17:0/4185406166 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f47fc152820 0x7f47fc153b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:11.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:11.005+0000 7f4801800640 1 -- 172.21.15.17:0/4185406166 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f47fc152440 msgr2=0x7f47fc152c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:11.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:11.005+0000 7f4801800640 1 --2- 172.21.15.17:0/4185406166 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f47fc152440 0x7f47fc152c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:11.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:11.005+0000 7f4801800640 1 -- 172.21.15.17:0/4185406166 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47fc0c7910 con 0x7f47fc0cdf20 2024-09-10T21:47:11.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:11.005+0000 7f4801800640 1 --2- 172.21.15.17:0/4185406166 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f47fc0cdf20 0x7f47fc0c6630 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f47f000c420 tx=0x7f47f000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:11.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:11.005+0000 7f47f9600640 1 -- 172.21.15.17:0/4185406166 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47f0017070 con 0x7f47fc0cdf20 2024-09-10T21:47:11.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:11.005+0000 7f47f9600640 1 -- 172.21.15.17:0/4185406166 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f47f0003600 con 0x7f47fc0cdf20 2024-09-10T21:47:11.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:11.005+0000 7f4804200640 1 -- 172.21.15.17:0/4185406166 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47fc0c7630 con 0x7f47fc0cdf20 2024-09-10T21:47:11.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:11.005+0000 7f47f9600640 1 -- 172.21.15.17:0/4185406166 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47f0003a70 con 0x7f47fc0cdf20 2024-09-10T21:47:11.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:11.005+0000 7f4804200640 1 -- 172.21.15.17:0/4185406166 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47fc0c62d0 con 0x7f47fc0cdf20 2024-09-10T21:47:11.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:11.005+0000 7f4804200640 1 -- 172.21.15.17:0/4185406166 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47fc0c2ff0 con 0x7f47fc0cdf20 2024-09-10T21:47:11.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:11.009+0000 7f47f9600640 1 -- 172.21.15.17:0/4185406166 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f47f0003c10 con 0x7f47fc0cdf20 2024-09-10T21:47:11.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:11.009+0000 7f47f9600640 1 --2- 172.21.15.17:0/4185406166 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f47dc0725e0 0x7f47dc074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:11.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:11.009+0000 7f47f9600640 1 -- 172.21.15.17:0/4185406166 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f47f0092c80 con 0x7f47fc0cdf20 2024-09-10T21:47:11.013 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:11.009+0000 7f4800e00640 1 --2- 172.21.15.17:0/4185406166 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f47dc0725e0 0x7f47dc074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:11.014 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:11.009+0000 7f4800e00640 1 --2- 172.21.15.17:0/4185406166 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f47dc0725e0 0x7f47dc074aa0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f47e80af000 tx=0x7f47e80193c0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:11.017 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:11.013+0000 7f47f9600640 1 -- 172.21.15.17:0/4185406166 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f47f005df50 con 0x7f47fc0cdf20 2024-09-10T21:47:11.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:11.201+0000 7f4804200640 1 -- 172.21.15.17:0/4185406166 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f47fc06fd20 con 0x7f47fc0cdf20 2024-09-10T21:47:11.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:11.201+0000 7f47f9600640 1 -- 172.21.15.17:0/4185406166 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) ==== 80+0+0 (secure 0 0 0) 0x7f47f005d850 con 0x7f47fc0cdf20 2024-09-10T21:47:11.206 INFO:teuthology.orchestra.run.smithi017.stderr:listed 0 entries 2024-09-10T21:47:11.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:11.205+0000 7f4804200640 1 -- 172.21.15.17:0/4185406166 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f47dc0725e0 msgr2=0x7f47dc074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:11.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:11.205+0000 7f4804200640 1 --2- 172.21.15.17:0/4185406166 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f47dc0725e0 0x7f47dc074aa0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f47e80af000 tx=0x7f47e80193c0 comp rx=0 tx=0).stop 2024-09-10T21:47:11.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:11.205+0000 7f4804200640 1 -- 172.21.15.17:0/4185406166 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f47fc0cdf20 msgr2=0x7f47fc0c6630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:11.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:11.205+0000 7f4804200640 1 --2- 172.21.15.17:0/4185406166 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f47fc0cdf20 0x7f47fc0c6630 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f47f000c420 tx=0x7f47f000c8e0 comp rx=0 tx=0).stop 2024-09-10T21:47:11.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:11.205+0000 7f4804200640 1 -- 172.21.15.17:0/4185406166 shutdown_connections 2024-09-10T21:47:11.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:11.205+0000 7f4804200640 1 --2- 172.21.15.17:0/4185406166 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f47dc0725e0 0x7f47dc074aa0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:11.208 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:11.205+0000 7f4804200640 1 --2- 172.21.15.17:0/4185406166 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f47fc152820 0x7f47fc153b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:11.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:11.205+0000 7f4804200640 1 --2- 172.21.15.17:0/4185406166 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f47fc152440 0x7f47fc152c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:11.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:11.205+0000 7f4804200640 1 --2- 172.21.15.17:0/4185406166 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f47fc0cdf20 0x7f47fc0c6630 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:11.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:11.205+0000 7f4804200640 1 -- 172.21.15.17:0/4185406166 >> 172.21.15.17:0/4185406166 conn(0x7f47fc0702f0 msgr2=0x7f47fc151b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:11.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:11.205+0000 7f4804200640 1 -- 172.21.15.17:0/4185406166 shutdown_connections 2024-09-10T21:47:11.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:11.205+0000 7f4804200640 1 -- 172.21.15.17:0/4185406166 wait complete. 2024-09-10T21:47:11.228 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi017.front.sepia.ceph.com... 2024-09-10T21:47:11.228 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:47:11.228 DEBUG:teuthology.orchestra.run.smithi017:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-09-10T21:47:11.261 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:47:11.280 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi017.stderr:ceph-fuse[35141]: fuse finished with error 0 and tester_r 0 2024-09-10T21:47:17.362 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi017.front.sepia.ceph.com 2024-09-10T21:47:17.363 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:47:17.363 DEBUG:teuthology.orchestra.run.smithi017:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-10T21:47:17.379 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:47:17.379 DEBUG:teuthology.orchestra.run.smithi017:> ip netns list 2024-09-10T21:47:17.526 INFO:teuthology.orchestra.run.smithi017.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-10T21:47:17.527 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:47:17.527 DEBUG:teuthology.orchestra.run.smithi017:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T21:47:17.549 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T21:47:17.550 DEBUG:teuthology.orchestra.run.smithi017:> sudo ip link delete ceph-brx 2024-09-10T21:47:18.062 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-09-10T21:47:18.072 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-10T21:47:18.072 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T21:47:18.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:18.217+0000 7fa244800640 1 -- 172.21.15.17:0/3975855432 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa224005270 msgr2=0x7fa22400b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:18.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:18.217+0000 7fa244800640 1 --2- 172.21.15.17:0/3975855432 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa224005270 0x7fa22400b9d0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fa238009f20 tx=0x7fa23801d630 comp rx=0 tx=0).stop 2024-09-10T21:47:18.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:18.217+0000 7fa242800640 1 -- 172.21.15.17:0/3975855432 reap_dead start 2024-09-10T21:47:18.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:18.217+0000 7fa244800640 1 -- 172.21.15.17:0/3975855432 shutdown_connections 2024-09-10T21:47:18.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:18.217+0000 7fa244800640 1 -- 172.21.15.17:0/3975855432 >> 172.21.15.17:0/3975855432 conn(0x7fa23c0702f0 msgr2=0x7fa23c070700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:18.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:18.217+0000 7fa244800640 1 -- 172.21.15.17:0/3975855432 shutdown_connections 2024-09-10T21:47:18.222 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:18.217+0000 7fa244800640 1 -- 172.21.15.17:0/3975855432 wait complete. 2024-09-10T21:47:18.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:18.217+0000 7fa244800640 1 Processor -- start 2024-09-10T21:47:18.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:18.217+0000 7fa244800640 1 -- start start 2024-09-10T21:47:18.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:18.217+0000 7fa244800640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa224005270 0x7fa23c152c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:18.223 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:18.217+0000 7fa244800640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa23c0ab5c0 0x7fa23c1531b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:18.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:18.221+0000 7fa244800640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa23c14fd90 0x7fa23c14f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:18.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:18.221+0000 7fa244800640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fa23c15bd60 con 0x7fa224005270 2024-09-10T21:47:18.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:18.221+0000 7fa244800640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fa23c1590c0 con 0x7fa23c14fd90 2024-09-10T21:47:18.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:18.221+0000 7fa244800640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7fa23c0b42e0 con 0x7fa23c0ab5c0 2024-09-10T21:47:18.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:18.221+0000 7fa241e00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa224005270 0x7fa23c152c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:18.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:18.221+0000 7fa241e00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa224005270 0x7fa23c152c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:34648/0 (socket says 172.21.15.17:34648) 2024-09-10T21:47:18.224 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:18.221+0000 7fa241e00640 1 -- 172.21.15.17:0/1304699118 learned_addr learned my addr 172.21.15.17:0/1304699118 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:47:18.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:18.221+0000 7fa242800640 1 --2- 172.21.15.17:0/1304699118 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa23c14fd90 0x7fa23c14f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:18.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:18.221+0000 7fa241400640 1 --2- 172.21.15.17:0/1304699118 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa23c0ab5c0 0x7fa23c1531b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:18.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:18.221+0000 7fa241e00640 1 -- 172.21.15.17:0/1304699118 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa23c0ab5c0 msgr2=0x7fa23c1531b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:18.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:18.221+0000 7fa241e00640 1 --2- 172.21.15.17:0/1304699118 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa23c0ab5c0 0x7fa23c1531b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:18.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:18.221+0000 7fa241e00640 1 -- 172.21.15.17:0/1304699118 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa23c14fd90 msgr2=0x7fa23c14f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:18.225 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:18.221+0000 7fa241e00640 1 --2- 172.21.15.17:0/1304699118 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa23c14fd90 0x7fa23c14f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:18.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:18.221+0000 7fa241e00640 1 -- 172.21.15.17:0/1304699118 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa23c0aa520 con 0x7fa224005270 2024-09-10T21:47:18.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:18.221+0000 7fa241400640 1 --2- 172.21.15.17:0/1304699118 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa23c0ab5c0 0x7fa23c1531b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:47:18.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:18.221+0000 7fa241e00640 1 --2- 172.21.15.17:0/1304699118 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa224005270 0x7fa23c152c70 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fa230006400 tx=0x7fa2300039d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:18.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:18.221+0000 7fa236000640 1 -- 172.21.15.17:0/1304699118 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa230015070 con 0x7fa224005270 2024-09-10T21:47:18.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:18.221+0000 7fa236000640 1 -- 172.21.15.17:0/1304699118 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa230011860 con 0x7fa224005270 2024-09-10T21:47:18.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:18.221+0000 7fa236000640 1 -- 172.21.15.17:0/1304699118 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa230011cb0 con 0x7fa224005270 2024-09-10T21:47:18.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:18.221+0000 7fa244800640 1 -- 172.21.15.17:0/1304699118 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa23c157850 con 0x7fa224005270 2024-09-10T21:47:18.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:18.221+0000 7fa244800640 1 -- 172.21.15.17:0/1304699118 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa23c157e10 con 0x7fa224005270 2024-09-10T21:47:18.226 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:18.221+0000 7fa244800640 1 -- 172.21.15.17:0/1304699118 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa23c1589e0 con 0x7fa224005270 2024-09-10T21:47:18.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:18.221+0000 7fa236000640 1 -- 172.21.15.17:0/1304699118 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fa2300103c0 con 0x7fa224005270 2024-09-10T21:47:18.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:18.225+0000 7fa236000640 1 --2- 172.21.15.17:0/1304699118 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fa2200725e0 0x7fa220074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:18.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:18.225+0000 7fa236000640 1 -- 172.21.15.17:0/1304699118 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fa230094640 con 0x7fa224005270 2024-09-10T21:47:18.228 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:18.225+0000 7fa241400640 1 --2- 172.21.15.17:0/1304699118 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fa2200725e0 0x7fa220074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:18.229 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:18.225+0000 7fa241400640 1 --2- 172.21.15.17:0/1304699118 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fa2200725e0 0x7fa220074aa0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fa22c006940 tx=0x7fa22c00acf0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:18.231 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:18.229+0000 7fa236000640 1 -- 172.21.15.17:0/1304699118 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fa23005f910 con 0x7fa224005270 2024-09-10T21:47:18.403 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:18.397+0000 7fa244800640 1 -- 172.21.15.17:0/1304699118 --> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa23c0c9550 con 0x7fa2200725e0 2024-09-10T21:47:18.410 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:18.405+0000 7fa236000640 1 -- 172.21.15.17:0/1304699118 <== mgr.4100 v2:172.21.15.31:6808/3101308586 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+334037 (secure 0 0 0) 0x7fa23c0c9550 con 0x7fa2200725e0 2024-09-10T21:47:18.414 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-10T21:47:18.416 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2024-09-10T21:47:18.417 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:18.409+0000 7fa244800640 1 -- 172.21.15.17:0/1304699118 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fa2200725e0 msgr2=0x7fa220074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:18.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:18.409+0000 7fa244800640 1 --2- 172.21.15.17:0/1304699118 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fa2200725e0 0x7fa220074aa0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fa22c006940 tx=0x7fa22c00acf0 comp rx=0 tx=0).stop 2024-09-10T21:47:18.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:18.409+0000 7fa244800640 1 -- 172.21.15.17:0/1304699118 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa224005270 msgr2=0x7fa23c152c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:18.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:18.409+0000 7fa244800640 1 --2- 172.21.15.17:0/1304699118 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa224005270 0x7fa23c152c70 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fa230006400 tx=0x7fa2300039d0 comp rx=0 tx=0).stop 2024-09-10T21:47:18.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:18.409+0000 7fa244800640 1 -- 172.21.15.17:0/1304699118 shutdown_connections 2024-09-10T21:47:18.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:18.409+0000 7fa244800640 1 --2- 172.21.15.17:0/1304699118 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fa2200725e0 0x7fa220074aa0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:18.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:18.409+0000 7fa244800640 1 --2- 172.21.15.17:0/1304699118 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa23c14fd90 0x7fa23c14f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:18.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:18.409+0000 7fa244800640 1 --2- 172.21.15.17:0/1304699118 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa23c0ab5c0 0x7fa23c1531b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:18.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:18.409+0000 7fa244800640 1 --2- 172.21.15.17:0/1304699118 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa224005270 0x7fa23c152c70 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:18.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:18.413+0000 7fa244800640 1 -- 172.21.15.17:0/1304699118 >> 172.21.15.17:0/1304699118 conn(0x7fa23c0702f0 msgr2=0x7fa23c0cbdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:18.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:18.413+0000 7fa244800640 1 -- 172.21.15.17:0/1304699118 shutdown_connections 2024-09-10T21:47:18.418 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:18.413+0000 7fa244800640 1 -- 172.21.15.17:0/1304699118 wait complete. 2024-09-10T21:47:18.436 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":702,"stamp":"2024-09-10T21:47:17.984631+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":15581531,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":114,"num_read_kb":174,"num_write":726807,"num_write_kb":16011424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":152578,"ondisk_log_size":152578,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":6469456,"kb_used_data":20368,"kb_used_omap":12,"kb_used_meta":6448947,"kb_avail":748505264,"statfs":{"total":773094113280,"available":766469390336,"internally_reserved":0,"allocated":20856832,"data_stored":32095942,"data_compressed":2631228,"data_compressed_allocated":3596288,"data_compressed_original":16236544,"omap_allocated":12752,"internal_metadata":6603722288},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":249,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003773"},"pg_stats":[{"pgid":"2.1f","version":"27'178","reported_seq":215,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.283456+0000","last_change":"2024-09-10T21:24:46.864556+0000","last_active":"2024-09-10T21:47:11.283456+0000","last_peered":"2024-09-10T21:47:11.283456+0000","last_clean":"2024-09-10T21:47:11.283456+0000","last_became_active":"2024-09-10T21:24:46.864288+0000","last_became_peered":"2024-09-10T21:24:46.864288+0000","last_unstale":"2024-09-10T21:47:11.283456+0000","last_undegraded":"2024-09-10T21:47:11.283456+0000","last_fullsized":"2024-09-10T21:47:11.283456+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":178,"log_dups_size":0,"ondisk_log_size":178,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:00:38.734807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2316909,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":173,"num_write_kb":2361,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"27'12","reported_seq":48,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.345156+0000","last_change":"2024-09-10T21:24:47.864452+0000","last_active":"2024-09-10T21:47:11.345156+0000","last_peered":"2024-09-10T21:47:11.345156+0000","last_clean":"2024-09-10T21:47:11.345156+0000","last_became_active":"2024-09-10T21:24:47.864168+0000","last_became_peered":"2024-09-10T21:24:47.864168+0000","last_unstale":"2024-09-10T21:47:11.345156+0000","last_undegraded":"2024-09-10T21:47:11.345156+0000","last_fullsized":"2024-09-10T21:47:11.345156+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:51:31.417845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"27'5290","reported_seq":5318,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.730211+0000","last_change":"2024-09-10T21:24:52.070054+0000","last_active":"2024-09-10T21:46:48.730211+0000","last_peered":"2024-09-10T21:46:48.730211+0000","last_clean":"2024-09-10T21:46:48.730211+0000","last_became_active":"2024-09-10T21:24:52.067258+0000","last_became_peered":"2024-09-10T21:24:52.067258+0000","last_unstale":"2024-09-10T21:46:48.730211+0000","last_undegraded":"2024-09-10T21:46:48.730211+0000","last_fullsized":"2024-09-10T21:46:48.730211+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":5290,"log_dups_size":0,"ondisk_log_size":5290,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:21:53.080013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25332,"num_write_kb":555711,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"27'5","reported_seq":43,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.553182+0000","last_change":"2024-09-10T21:24:46.860648+0000","last_active":"2024-09-10T21:46:48.553182+0000","last_peered":"2024-09-10T21:46:48.553182+0000","last_clean":"2024-09-10T21:46:48.553182+0000","last_became_active":"2024-09-10T21:24:46.860341+0000","last_became_peered":"2024-09-10T21:24:46.860341+0000","last_unstale":"2024-09-10T21:46:48.553182+0000","last_undegraded":"2024-09-10T21:46:48.553182+0000","last_fullsized":"2024-09-10T21:46:48.553182+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:39:24.819979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"27'19","reported_seq":54,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.345477+0000","last_change":"2024-09-10T21:24:47.865617+0000","last_active":"2024-09-10T21:47:11.345477+0000","last_peered":"2024-09-10T21:47:11.345477+0000","last_clean":"2024-09-10T21:47:11.345477+0000","last_became_active":"2024-09-10T21:24:47.865365+0000","last_became_peered":"2024-09-10T21:24:47.865365+0000","last_unstale":"2024-09-10T21:47:11.345477+0000","last_undegraded":"2024-09-10T21:47:11.345477+0000","last_fullsized":"2024-09-10T21:47:11.345477+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:10:05.971996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"27'4776","reported_seq":4804,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.713274+0000","last_change":"2024-09-10T21:24:51.900077+0000","last_active":"2024-09-10T21:46:48.713274+0000","last_peered":"2024-09-10T21:46:48.713274+0000","last_clean":"2024-09-10T21:46:48.713274+0000","last_became_active":"2024-09-10T21:24:51.899652+0000","last_became_peered":"2024-09-10T21:24:51.899652+0000","last_unstale":"2024-09-10T21:46:48.713274+0000","last_undegraded":"2024-09-10T21:46:48.713274+0000","last_fullsized":"2024-09-10T21:46:48.713274+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":4776,"log_dups_size":0,"ondisk_log_size":4776,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:40:49.585857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23087,"num_write_kb":517913,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"27'7","reported_seq":43,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.636914+0000","last_change":"2024-09-10T21:24:46.861040+0000","last_active":"2024-09-10T21:46:48.636914+0000","last_peered":"2024-09-10T21:46:48.636914+0000","last_clean":"2024-09-10T21:46:48.636914+0000","last_became_active":"2024-09-10T21:24:46.860780+0000","last_became_peered":"2024-09-10T21:24:46.860780+0000","last_unstale":"2024-09-10T21:46:48.636914+0000","last_undegraded":"2024-09-10T21:46:48.636914+0000","last_fullsized":"2024-09-10T21:46:48.636914+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:53:31.454472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"27'16","reported_seq":48,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.347558+0000","last_change":"2024-09-10T21:24:47.877997+0000","last_active":"2024-09-10T21:47:11.347558+0000","last_peered":"2024-09-10T21:47:11.347558+0000","last_clean":"2024-09-10T21:47:11.347558+0000","last_became_active":"2024-09-10T21:24:47.877765+0000","last_became_peered":"2024-09-10T21:24:47.877765+0000","last_unstale":"2024-09-10T21:47:11.347558+0000","last_undegraded":"2024-09-10T21:47:11.347558+0000","last_fullsized":"2024-09-10T21:47:11.347558+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:33:20.590209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"27'4118","reported_seq":4153,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.560547+0000","last_change":"2024-09-10T21:25:28.512435+0000","last_active":"2024-09-10T21:46:48.560547+0000","last_peered":"2024-09-10T21:46:48.560547+0000","last_clean":"2024-09-10T21:46:48.560547+0000","last_became_active":"2024-09-10T21:25:28.512023+0000","last_became_peered":"2024-09-10T21:25:28.512023+0000","last_unstale":"2024-09-10T21:46:48.560547+0000","last_undegraded":"2024-09-10T21:46:48.560547+0000","last_fullsized":"2024-09-10T21:46:48.560547+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":4118,"log_dups_size":0,"ondisk_log_size":4118,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:43:17.173095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18983,"num_write_kb":440322,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,2],"acting":[4,3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"27'5","reported_seq":41,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.677062+0000","last_change":"2024-09-10T21:24:46.873044+0000","last_active":"2024-09-10T21:46:48.677062+0000","last_peered":"2024-09-10T21:46:48.677062+0000","last_clean":"2024-09-10T21:46:48.677062+0000","last_became_active":"2024-09-10T21:24:46.872075+0000","last_became_peered":"2024-09-10T21:24:46.872075+0000","last_unstale":"2024-09-10T21:46:48.677062+0000","last_undegraded":"2024-09-10T21:46:48.677062+0000","last_fullsized":"2024-09-10T21:46:48.677062+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:40:33.632846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"27'30","reported_seq":62,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.342014+0000","last_change":"2024-09-10T21:24:47.878094+0000","last_active":"2024-09-10T21:47:11.342014+0000","last_peered":"2024-09-10T21:47:11.342014+0000","last_clean":"2024-09-10T21:47:11.342014+0000","last_became_active":"2024-09-10T21:24:47.876688+0000","last_became_peered":"2024-09-10T21:24:47.876688+0000","last_unstale":"2024-09-10T21:47:11.342014+0000","last_undegraded":"2024-09-10T21:47:11.342014+0000","last_fullsized":"2024-09-10T21:47:11.342014+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:03:37.076080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"27'3999","reported_seq":4026,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.559967+0000","last_change":"2024-09-10T21:24:51.900208+0000","last_active":"2024-09-10T21:46:48.559967+0000","last_peered":"2024-09-10T21:46:48.559967+0000","last_clean":"2024-09-10T21:46:48.559967+0000","last_became_active":"2024-09-10T21:24:51.899576+0000","last_became_peered":"2024-09-10T21:24:51.899576+0000","last_unstale":"2024-09-10T21:46:48.559967+0000","last_undegraded":"2024-09-10T21:46:48.559967+0000","last_fullsized":"2024-09-10T21:46:48.559967+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":3999,"log_dups_size":0,"ondisk_log_size":3999,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:52:49.479932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18841,"num_write_kb":435514,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"27'15","reported_seq":49,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.350364+0000","last_change":"2024-09-10T21:24:47.869830+0000","last_active":"2024-09-10T21:47:11.350364+0000","last_peered":"2024-09-10T21:47:11.350364+0000","last_clean":"2024-09-10T21:47:11.350364+0000","last_became_active":"2024-09-10T21:24:47.869569+0000","last_became_peered":"2024-09-10T21:24:47.869569+0000","last_unstale":"2024-09-10T21:47:11.350364+0000","last_undegraded":"2024-09-10T21:47:11.350364+0000","last_fullsized":"2024-09-10T21:47:11.350364+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:42:15.320518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"27'1003","reported_seq":1040,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.666697+0000","last_change":"2024-09-10T21:24:46.867297+0000","last_active":"2024-09-10T21:46:48.666697+0000","last_peered":"2024-09-10T21:46:48.666697+0000","last_clean":"2024-09-10T21:46:48.666697+0000","last_became_active":"2024-09-10T21:24:46.866893+0000","last_became_peered":"2024-09-10T21:24:46.866893+0000","last_unstale":"2024-09-10T21:46:48.666697+0000","last_undegraded":"2024-09-10T21:46:48.666697+0000","last_fullsized":"2024-09-10T21:46:48.666697+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":1003,"log_dups_size":0,"ondisk_log_size":1003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:53.197666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":997,"num_write_kb":4359,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"27'4606","reported_seq":4634,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.705887+0000","last_change":"2024-09-10T21:24:51.908412+0000","last_active":"2024-09-10T21:46:48.705887+0000","last_peered":"2024-09-10T21:46:48.705887+0000","last_clean":"2024-09-10T21:46:48.705887+0000","last_became_active":"2024-09-10T21:24:51.907923+0000","last_became_peered":"2024-09-10T21:24:51.907923+0000","last_unstale":"2024-09-10T21:46:48.705887+0000","last_undegraded":"2024-09-10T21:46:48.705887+0000","last_fullsized":"2024-09-10T21:46:48.705887+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":4606,"log_dups_size":0,"ondisk_log_size":4606,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:26:46.479501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22377,"num_write_kb":531442,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"27'17","reported_seq":52,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.331673+0000","last_change":"2024-09-10T21:24:47.862479+0000","last_active":"2024-09-10T21:47:11.331673+0000","last_peered":"2024-09-10T21:47:11.331673+0000","last_clean":"2024-09-10T21:47:11.331673+0000","last_became_active":"2024-09-10T21:24:47.861842+0000","last_became_peered":"2024-09-10T21:24:47.861842+0000","last_unstale":"2024-09-10T21:47:11.331673+0000","last_undegraded":"2024-09-10T21:47:11.331673+0000","last_fullsized":"2024-09-10T21:47:11.331673+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:53:09.082512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"27'3","reported_seq":39,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.551236+0000","last_change":"2024-09-10T21:24:46.852467+0000","last_active":"2024-09-10T21:46:48.551236+0000","last_peered":"2024-09-10T21:46:48.551236+0000","last_clean":"2024-09-10T21:46:48.551236+0000","last_became_active":"2024-09-10T21:24:46.851027+0000","last_became_peered":"2024-09-10T21:24:46.851027+0000","last_unstale":"2024-09-10T21:46:48.551236+0000","last_undegraded":"2024-09-10T21:46:48.551236+0000","last_fullsized":"2024-09-10T21:46:48.551236+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:39:27.441714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"27'4524","reported_seq":4552,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.720455+0000","last_change":"2024-09-10T21:24:51.903577+0000","last_active":"2024-09-10T21:46:48.720455+0000","last_peered":"2024-09-10T21:46:48.720455+0000","last_clean":"2024-09-10T21:46:48.720455+0000","last_became_active":"2024-09-10T21:24:51.903031+0000","last_became_peered":"2024-09-10T21:24:51.903031+0000","last_unstale":"2024-09-10T21:46:48.720455+0000","last_undegraded":"2024-09-10T21:46:48.720455+0000","last_fullsized":"2024-09-10T21:46:48.720455+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":4524,"log_dups_size":0,"ondisk_log_size":4524,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:08:52.787720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21954,"num_write_kb":484896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"27'18","reported_seq":53,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.328856+0000","last_change":"2024-09-10T21:24:47.856334+0000","last_active":"2024-09-10T21:47:11.328856+0000","last_peered":"2024-09-10T21:47:11.328856+0000","last_clean":"2024-09-10T21:47:11.328856+0000","last_became_active":"2024-09-10T21:24:47.856003+0000","last_became_peered":"2024-09-10T21:24:47.856003+0000","last_unstale":"2024-09-10T21:47:11.328856+0000","last_undegraded":"2024-09-10T21:47:11.328856+0000","last_fullsized":"2024-09-10T21:47:11.328856+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:58:53.698584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"27'2","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.586182+0000","last_change":"2024-09-10T21:24:46.853373+0000","last_active":"2024-09-10T21:46:48.586182+0000","last_peered":"2024-09-10T21:46:48.586182+0000","last_clean":"2024-09-10T21:46:48.586182+0000","last_became_active":"2024-09-10T21:24:46.852993+0000","last_became_peered":"2024-09-10T21:24:46.852993+0000","last_unstale":"2024-09-10T21:46:48.586182+0000","last_undegraded":"2024-09-10T21:46:48.586182+0000","last_fullsized":"2024-09-10T21:46:48.586182+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:53:31.454472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"27'5859","reported_seq":5887,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.677773+0000","last_change":"2024-09-10T21:24:52.065246+0000","last_active":"2024-09-10T21:46:48.677773+0000","last_peered":"2024-09-10T21:46:48.677773+0000","last_clean":"2024-09-10T21:46:48.677773+0000","last_became_active":"2024-09-10T21:24:52.064884+0000","last_became_peered":"2024-09-10T21:24:52.064884+0000","last_unstale":"2024-09-10T21:46:48.677773+0000","last_undegraded":"2024-09-10T21:46:48.677773+0000","last_fullsized":"2024-09-10T21:46:48.677773+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":5859,"log_dups_size":0,"ondisk_log_size":5859,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:54:54.393305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27858,"num_write_kb":655707,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"27'9","reported_seq":44,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.296539+0000","last_change":"2024-09-10T21:24:47.856517+0000","last_active":"2024-09-10T21:47:11.296539+0000","last_peered":"2024-09-10T21:47:11.296539+0000","last_clean":"2024-09-10T21:47:11.296539+0000","last_became_active":"2024-09-10T21:24:47.856185+0000","last_became_peered":"2024-09-10T21:24:47.856185+0000","last_unstale":"2024-09-10T21:47:11.296539+0000","last_undegraded":"2024-09-10T21:47:11.296539+0000","last_fullsized":"2024-09-10T21:47:11.296539+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:26:00.852818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"27'5","reported_seq":42,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:36.457065+0000","last_change":"2024-09-10T21:24:46.867888+0000","last_active":"2024-09-10T21:46:36.457065+0000","last_peered":"2024-09-10T21:46:36.457065+0000","last_clean":"2024-09-10T21:46:36.457065+0000","last_became_active":"2024-09-10T21:24:46.866477+0000","last_became_peered":"2024-09-10T21:24:46.866477+0000","last_unstale":"2024-09-10T21:46:36.457065+0000","last_undegraded":"2024-09-10T21:46:36.457065+0000","last_fullsized":"2024-09-10T21:46:36.457065+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:39:27.441714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"27'4736","reported_seq":4764,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.660872+0000","last_change":"2024-09-10T21:24:51.908376+0000","last_active":"2024-09-10T21:46:48.660872+0000","last_peered":"2024-09-10T21:46:48.660872+0000","last_clean":"2024-09-10T21:46:48.660872+0000","last_became_active":"2024-09-10T21:24:51.907878+0000","last_became_peered":"2024-09-10T21:24:51.907878+0000","last_unstale":"2024-09-10T21:46:48.660872+0000","last_undegraded":"2024-09-10T21:46:48.660872+0000","last_fullsized":"2024-09-10T21:46:48.660872+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":4736,"log_dups_size":0,"ondisk_log_size":4736,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:39:02.492952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22584,"num_write_kb":479265,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"27'25","reported_seq":60,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.323165+0000","last_change":"2024-09-10T21:24:47.844823+0000","last_active":"2024-09-10T21:47:11.323165+0000","last_peered":"2024-09-10T21:47:11.323165+0000","last_clean":"2024-09-10T21:47:11.323165+0000","last_became_active":"2024-09-10T21:24:47.844505+0000","last_became_peered":"2024-09-10T21:24:47.844505+0000","last_unstale":"2024-09-10T21:47:11.323165+0000","last_undegraded":"2024-09-10T21:47:11.323165+0000","last_fullsized":"2024-09-10T21:47:11.323165+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:37:51.784258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"27'9","reported_seq":45,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.639567+0000","last_change":"2024-09-10T21:24:46.862172+0000","last_active":"2024-09-10T21:46:48.639567+0000","last_peered":"2024-09-10T21:46:48.639567+0000","last_clean":"2024-09-10T21:46:48.639567+0000","last_became_active":"2024-09-10T21:24:46.861473+0000","last_became_peered":"2024-09-10T21:24:46.861473+0000","last_unstale":"2024-09-10T21:46:48.639567+0000","last_undegraded":"2024-09-10T21:46:48.639567+0000","last_fullsized":"2024-09-10T21:46:48.639567+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:53:31.454472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":53,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"27'4521","reported_seq":4550,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.535321+0000","last_change":"2024-09-10T21:24:52.062841+0000","last_active":"2024-09-10T21:46:48.535321+0000","last_peered":"2024-09-10T21:46:48.535321+0000","last_clean":"2024-09-10T21:46:48.535321+0000","last_became_active":"2024-09-10T21:24:52.061581+0000","last_became_peered":"2024-09-10T21:24:52.061581+0000","last_unstale":"2024-09-10T21:46:48.535321+0000","last_undegraded":"2024-09-10T21:46:48.535321+0000","last_fullsized":"2024-09-10T21:46:48.535321+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":4521,"log_dups_size":0,"ondisk_log_size":4521,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:34:03.325781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21874,"num_write_kb":476387,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"27'18","reported_seq":53,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.324770+0000","last_change":"2024-09-10T21:24:47.859801+0000","last_active":"2024-09-10T21:47:11.324770+0000","last_peered":"2024-09-10T21:47:11.324770+0000","last_clean":"2024-09-10T21:47:11.324770+0000","last_became_active":"2024-09-10T21:24:47.859483+0000","last_became_peered":"2024-09-10T21:24:47.859483+0000","last_unstale":"2024-09-10T21:47:11.324770+0000","last_undegraded":"2024-09-10T21:47:11.324770+0000","last_fullsized":"2024-09-10T21:47:11.324770+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:38:21.368499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"27'326","reported_seq":362,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:36.365019+0000","last_change":"2024-09-10T21:24:46.860624+0000","last_active":"2024-09-10T21:46:36.365019+0000","last_peered":"2024-09-10T21:46:36.365019+0000","last_clean":"2024-09-10T21:46:36.365019+0000","last_became_active":"2024-09-10T21:24:46.860309+0000","last_became_peered":"2024-09-10T21:24:46.860309+0000","last_unstale":"2024-09-10T21:46:36.365019+0000","last_undegraded":"2024-09-10T21:46:36.365019+0000","last_fullsized":"2024-09-10T21:46:36.365019+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":326,"log_dups_size":0,"ondisk_log_size":326,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:37:12.332013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":704,"num_write_kb":242,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"27'4358","reported_seq":4388,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.745954+0000","last_change":"2024-09-10T21:24:52.068581+0000","last_active":"2024-09-10T21:46:48.745954+0000","last_peered":"2024-09-10T21:46:48.745954+0000","last_clean":"2024-09-10T21:46:48.745954+0000","last_became_active":"2024-09-10T21:24:52.068250+0000","last_became_peered":"2024-09-10T21:24:52.068250+0000","last_unstale":"2024-09-10T21:46:48.745954+0000","last_undegraded":"2024-09-10T21:46:48.745954+0000","last_fullsized":"2024-09-10T21:46:48.745954+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":4358,"log_dups_size":0,"ondisk_log_size":4358,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:52:01.990009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21666,"num_write_kb":463209,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"27'18","reported_seq":52,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.321574+0000","last_change":"2024-09-10T21:24:47.854042+0000","last_active":"2024-09-10T21:47:11.321574+0000","last_peered":"2024-09-10T21:47:11.321574+0000","last_clean":"2024-09-10T21:47:11.321574+0000","last_became_active":"2024-09-10T21:24:47.853750+0000","last_became_peered":"2024-09-10T21:24:47.853750+0000","last_unstale":"2024-09-10T21:47:11.321574+0000","last_undegraded":"2024-09-10T21:47:11.321574+0000","last_fullsized":"2024-09-10T21:47:11.321574+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:06:50.775132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"27'2","reported_seq":39,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:45:42.186926+0000","last_change":"2024-09-10T21:24:46.864617+0000","last_active":"2024-09-10T21:45:42.186926+0000","last_peered":"2024-09-10T21:45:42.186926+0000","last_clean":"2024-09-10T21:45:42.186926+0000","last_became_active":"2024-09-10T21:24:46.864149+0000","last_became_peered":"2024-09-10T21:24:46.864149+0000","last_unstale":"2024-09-10T21:45:42.186926+0000","last_undegraded":"2024-09-10T21:45:42.186926+0000","last_fullsized":"2024-09-10T21:45:42.186926+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:24:15.673107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"27'4655","reported_seq":4684,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.520210+0000","last_change":"2024-09-10T21:24:52.061485+0000","last_active":"2024-09-10T21:46:48.520210+0000","last_peered":"2024-09-10T21:46:48.520210+0000","last_clean":"2024-09-10T21:46:48.520210+0000","last_became_active":"2024-09-10T21:24:52.060953+0000","last_became_peered":"2024-09-10T21:24:52.060953+0000","last_unstale":"2024-09-10T21:46:48.520210+0000","last_undegraded":"2024-09-10T21:46:48.520210+0000","last_fullsized":"2024-09-10T21:46:48.520210+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":4655,"log_dups_size":0,"ondisk_log_size":4655,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:49:30.147202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22893,"num_write_kb":513258,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"27'18","reported_seq":52,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.341860+0000","last_change":"2024-09-10T21:24:47.867241+0000","last_active":"2024-09-10T21:47:11.341860+0000","last_peered":"2024-09-10T21:47:11.341860+0000","last_clean":"2024-09-10T21:47:11.341860+0000","last_became_active":"2024-09-10T21:24:47.866907+0000","last_became_peered":"2024-09-10T21:24:47.866907+0000","last_unstale":"2024-09-10T21:47:11.341860+0000","last_undegraded":"2024-09-10T21:47:11.341860+0000","last_fullsized":"2024-09-10T21:47:11.341860+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:07.620800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"27'2","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.562765+0000","last_change":"2024-09-10T21:24:46.856848+0000","last_active":"2024-09-10T21:46:48.562765+0000","last_peered":"2024-09-10T21:46:48.562765+0000","last_clean":"2024-09-10T21:46:48.562765+0000","last_became_active":"2024-09-10T21:24:46.856489+0000","last_became_peered":"2024-09-10T21:24:46.856489+0000","last_unstale":"2024-09-10T21:46:48.562765+0000","last_undegraded":"2024-09-10T21:46:48.562765+0000","last_fullsized":"2024-09-10T21:46:48.562765+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:46:55.286321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"27'4909","reported_seq":4938,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.660587+0000","last_change":"2024-09-10T21:24:51.901016+0000","last_active":"2024-09-10T21:46:48.660587+0000","last_peered":"2024-09-10T21:46:48.660587+0000","last_clean":"2024-09-10T21:46:48.660587+0000","last_became_active":"2024-09-10T21:24:51.900479+0000","last_became_peered":"2024-09-10T21:24:51.900479+0000","last_unstale":"2024-09-10T21:46:48.660587+0000","last_undegraded":"2024-09-10T21:46:48.660587+0000","last_fullsized":"2024-09-10T21:46:48.660587+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":4909,"log_dups_size":0,"ondisk_log_size":4909,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:36.741810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23807,"num_write_kb":525103,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"27'16","reported_seq":51,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.320532+0000","last_change":"2024-09-10T21:24:47.858753+0000","last_active":"2024-09-10T21:47:11.320532+0000","last_peered":"2024-09-10T21:47:11.320532+0000","last_clean":"2024-09-10T21:47:11.320532+0000","last_became_active":"2024-09-10T21:24:47.858438+0000","last_became_peered":"2024-09-10T21:24:47.858438+0000","last_unstale":"2024-09-10T21:47:11.320532+0000","last_undegraded":"2024-09-10T21:47:11.320532+0000","last_fullsized":"2024-09-10T21:47:11.320532+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:12:09.198013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"27'4","reported_seq":40,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:45:39.089516+0000","last_change":"2024-09-10T21:24:46.863610+0000","last_active":"2024-09-10T21:45:39.089516+0000","last_peered":"2024-09-10T21:45:39.089516+0000","last_clean":"2024-09-10T21:45:39.089516+0000","last_became_active":"2024-09-10T21:24:46.863150+0000","last_became_peered":"2024-09-10T21:24:46.863150+0000","last_unstale":"2024-09-10T21:45:39.089516+0000","last_undegraded":"2024-09-10T21:45:39.089516+0000","last_fullsized":"2024-09-10T21:45:39.089516+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:27:35.473995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"27'5335","reported_seq":5364,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.650031+0000","last_change":"2024-09-10T21:24:51.907366+0000","last_active":"2024-09-10T21:46:48.650031+0000","last_peered":"2024-09-10T21:46:48.650031+0000","last_clean":"2024-09-10T21:46:48.650031+0000","last_became_active":"2024-09-10T21:24:51.906888+0000","last_became_peered":"2024-09-10T21:24:51.906888+0000","last_unstale":"2024-09-10T21:46:48.650031+0000","last_undegraded":"2024-09-10T21:46:48.650031+0000","last_fullsized":"2024-09-10T21:46:48.650031+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":5335,"log_dups_size":0,"ondisk_log_size":5335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:24:54.278514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26491,"num_write_kb":568740,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"27'20","reported_seq":55,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.311229+0000","last_change":"2024-09-10T21:24:47.850155+0000","last_active":"2024-09-10T21:47:11.311229+0000","last_peered":"2024-09-10T21:47:11.311229+0000","last_clean":"2024-09-10T21:47:11.311229+0000","last_became_active":"2024-09-10T21:24:47.849846+0000","last_became_peered":"2024-09-10T21:24:47.849846+0000","last_unstale":"2024-09-10T21:47:11.311229+0000","last_undegraded":"2024-09-10T21:47:11.311229+0000","last_fullsized":"2024-09-10T21:47:11.311229+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:13:40.032808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"27'51","reported_seq":136,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.553709+0000","last_change":"2024-09-10T21:24:46.851800+0000","last_active":"2024-09-10T21:46:48.553709+0000","last_peered":"2024-09-10T21:46:48.553709+0000","last_clean":"2024-09-10T21:46:48.553709+0000","last_became_active":"2024-09-10T21:24:46.851200+0000","last_became_peered":"2024-09-10T21:24:46.851200+0000","last_unstale":"2024-09-10T21:46:48.553709+0000","last_undegraded":"2024-09-10T21:46:48.553709+0000","last_fullsized":"2024-09-10T21:46:48.553709+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:00:22.839361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90454,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":48,"num_read_kb":119,"num_write":48,"num_write_kb":119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"27'4677","reported_seq":4706,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.581789+0000","last_change":"2024-09-10T21:24:51.902529+0000","last_active":"2024-09-10T21:46:48.581789+0000","last_peered":"2024-09-10T21:46:48.581789+0000","last_clean":"2024-09-10T21:46:48.581789+0000","last_became_active":"2024-09-10T21:24:51.901141+0000","last_became_peered":"2024-09-10T21:24:51.901141+0000","last_unstale":"2024-09-10T21:46:48.581789+0000","last_undegraded":"2024-09-10T21:46:48.581789+0000","last_fullsized":"2024-09-10T21:46:48.581789+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":4677,"log_dups_size":0,"ondisk_log_size":4677,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:08:40.203348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22963,"num_write_kb":501268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"27'11","reported_seq":46,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.323431+0000","last_change":"2024-09-10T21:24:47.846582+0000","last_active":"2024-09-10T21:47:11.323431+0000","last_peered":"2024-09-10T21:47:11.323431+0000","last_clean":"2024-09-10T21:47:11.323431+0000","last_became_active":"2024-09-10T21:24:47.846358+0000","last_became_peered":"2024-09-10T21:24:47.846358+0000","last_unstale":"2024-09-10T21:47:11.323431+0000","last_undegraded":"2024-09-10T21:47:11.323431+0000","last_fullsized":"2024-09-10T21:47:11.323431+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:58:53.698584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"27'7","reported_seq":45,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.647854+0000","last_change":"2024-09-10T21:24:46.870908+0000","last_active":"2024-09-10T21:46:48.647854+0000","last_peered":"2024-09-10T21:46:48.647854+0000","last_clean":"2024-09-10T21:46:48.647854+0000","last_became_active":"2024-09-10T21:24:46.868375+0000","last_became_peered":"2024-09-10T21:24:46.868375+0000","last_unstale":"2024-09-10T21:46:48.647854+0000","last_undegraded":"2024-09-10T21:46:48.647854+0000","last_fullsized":"2024-09-10T21:46:48.647854+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:37:12.332013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"27'4824","reported_seq":4853,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.659490+0000","last_change":"2024-09-10T21:24:52.069018+0000","last_active":"2024-09-10T21:46:48.659490+0000","last_peered":"2024-09-10T21:46:48.659490+0000","last_clean":"2024-09-10T21:46:48.659490+0000","last_became_active":"2024-09-10T21:24:52.067308+0000","last_became_peered":"2024-09-10T21:24:52.067308+0000","last_unstale":"2024-09-10T21:46:48.659490+0000","last_undegraded":"2024-09-10T21:46:48.659490+0000","last_fullsized":"2024-09-10T21:46:48.659490+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":4824,"log_dups_size":0,"ondisk_log_size":4824,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:07:25.162168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23972,"num_write_kb":500992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"27'13","reported_seq":48,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.338908+0000","last_change":"2024-09-10T21:24:47.871542+0000","last_active":"2024-09-10T21:47:11.338908+0000","last_peered":"2024-09-10T21:47:11.338908+0000","last_clean":"2024-09-10T21:47:11.338908+0000","last_became_active":"2024-09-10T21:24:47.871128+0000","last_became_peered":"2024-09-10T21:24:47.871128+0000","last_unstale":"2024-09-10T21:47:11.338908+0000","last_undegraded":"2024-09-10T21:47:11.338908+0000","last_fullsized":"2024-09-10T21:47:11.338908+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:51:22.876126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"27'3","reported_seq":39,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:36.366557+0000","last_change":"2024-09-10T21:24:46.859218+0000","last_active":"2024-09-10T21:46:36.366557+0000","last_peered":"2024-09-10T21:46:36.366557+0000","last_clean":"2024-09-10T21:46:36.366557+0000","last_became_active":"2024-09-10T21:24:46.853190+0000","last_became_peered":"2024-09-10T21:24:46.853190+0000","last_unstale":"2024-09-10T21:46:36.366557+0000","last_undegraded":"2024-09-10T21:46:36.366557+0000","last_fullsized":"2024-09-10T21:46:36.366557+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:26:34.848759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":134,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:44:42.019323+0000","last_change":"2024-09-10T21:24:37.216542+0000","last_active":"2024-09-10T21:44:42.019323+0000","last_peered":"2024-09-10T21:44:42.019323+0000","last_clean":"2024-09-10T21:44:42.019323+0000","last_became_active":"2024-09-10T21:24:37.215443+0000","last_became_peered":"2024-09-10T21:24:37.215443+0000","last_unstale":"2024-09-10T21:44:42.019323+0000","last_undegraded":"2024-09-10T21:44:42.019323+0000","last_fullsized":"2024-09-10T21:44:42.019323+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:35.877801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:35.877801+0000","last_clean_scrub_stamp":"2024-09-10T21:24:35.877801+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:14:20.260329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"27'4394","reported_seq":4423,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.650952+0000","last_change":"2024-09-10T21:24:52.067645+0000","last_active":"2024-09-10T21:46:48.650952+0000","last_peered":"2024-09-10T21:46:48.650952+0000","last_clean":"2024-09-10T21:46:48.650952+0000","last_became_active":"2024-09-10T21:24:52.067148+0000","last_became_peered":"2024-09-10T21:24:52.067148+0000","last_unstale":"2024-09-10T21:46:48.650952+0000","last_undegraded":"2024-09-10T21:46:48.650952+0000","last_fullsized":"2024-09-10T21:46:48.650952+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":4394,"log_dups_size":0,"ondisk_log_size":4394,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:54:42.435011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21702,"num_write_kb":479680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"27'21","reported_seq":61,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.342553+0000","last_change":"2024-09-10T21:25:28.510354+0000","last_active":"2024-09-10T21:47:11.342553+0000","last_peered":"2024-09-10T21:47:11.342553+0000","last_clean":"2024-09-10T21:47:11.342553+0000","last_became_active":"2024-09-10T21:25:28.509981+0000","last_became_peered":"2024-09-10T21:25:28.509981+0000","last_unstale":"2024-09-10T21:47:11.342553+0000","last_undegraded":"2024-09-10T21:47:11.342553+0000","last_fullsized":"2024-09-10T21:47:11.342553+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:51:42.501764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"27'5","reported_seq":41,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.629368+0000","last_change":"2024-09-10T21:24:46.859532+0000","last_active":"2024-09-10T21:46:48.629368+0000","last_peered":"2024-09-10T21:46:48.629368+0000","last_clean":"2024-09-10T21:46:48.629368+0000","last_became_active":"2024-09-10T21:24:46.859144+0000","last_became_peered":"2024-09-10T21:24:46.859144+0000","last_unstale":"2024-09-10T21:46:48.629368+0000","last_undegraded":"2024-09-10T21:46:48.629368+0000","last_fullsized":"2024-09-10T21:46:48.629368+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:48:41.992441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"27'4328","reported_seq":4365,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.651243+0000","last_change":"2024-09-10T21:25:28.620859+0000","last_active":"2024-09-10T21:46:48.651243+0000","last_peered":"2024-09-10T21:46:48.651243+0000","last_clean":"2024-09-10T21:46:48.651243+0000","last_became_active":"2024-09-10T21:25:28.620415+0000","last_became_peered":"2024-09-10T21:25:28.620415+0000","last_unstale":"2024-09-10T21:46:48.651243+0000","last_undegraded":"2024-09-10T21:46:48.651243+0000","last_fullsized":"2024-09-10T21:46:48.651243+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":4328,"log_dups_size":0,"ondisk_log_size":4328,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:17:39.229564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21368,"num_write_kb":498780,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2,0],"acting":[6,7,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"27'17","reported_seq":50,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.342104+0000","last_change":"2024-09-10T21:24:47.866927+0000","last_active":"2024-09-10T21:47:11.342104+0000","last_peered":"2024-09-10T21:47:11.342104+0000","last_clean":"2024-09-10T21:47:11.342104+0000","last_became_active":"2024-09-10T21:24:47.866650+0000","last_became_peered":"2024-09-10T21:24:47.866650+0000","last_unstale":"2024-09-10T21:47:11.342104+0000","last_undegraded":"2024-09-10T21:47:11.342104+0000","last_fullsized":"2024-09-10T21:47:11.342104+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:47:37.304624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"27'6","reported_seq":42,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.675955+0000","last_change":"2024-09-10T21:24:46.857082+0000","last_active":"2024-09-10T21:46:48.675955+0000","last_peered":"2024-09-10T21:46:48.675955+0000","last_clean":"2024-09-10T21:46:48.675955+0000","last_became_active":"2024-09-10T21:24:46.856799+0000","last_became_peered":"2024-09-10T21:24:46.856799+0000","last_unstale":"2024-09-10T21:46:48.675955+0000","last_undegraded":"2024-09-10T21:46:48.675955+0000","last_fullsized":"2024-09-10T21:46:48.675955+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:40:33.632846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"27'4625","reported_seq":4654,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.739934+0000","last_change":"2024-09-10T21:24:51.911880+0000","last_active":"2024-09-10T21:46:48.739934+0000","last_peered":"2024-09-10T21:46:48.739934+0000","last_clean":"2024-09-10T21:46:48.739934+0000","last_became_active":"2024-09-10T21:24:51.911088+0000","last_became_peered":"2024-09-10T21:24:51.911088+0000","last_unstale":"2024-09-10T21:46:48.739934+0000","last_undegraded":"2024-09-10T21:46:48.739934+0000","last_fullsized":"2024-09-10T21:46:48.739934+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":4625,"log_dups_size":0,"ondisk_log_size":4625,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:38:57.030655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21860,"num_write_kb":505231,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"27'20","reported_seq":54,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.344779+0000","last_change":"2024-09-10T21:24:47.876946+0000","last_active":"2024-09-10T21:47:11.344779+0000","last_peered":"2024-09-10T21:47:11.344779+0000","last_clean":"2024-09-10T21:47:11.344779+0000","last_became_active":"2024-09-10T21:24:47.876690+0000","last_became_peered":"2024-09-10T21:24:47.876690+0000","last_unstale":"2024-09-10T21:47:11.344779+0000","last_undegraded":"2024-09-10T21:47:11.344779+0000","last_fullsized":"2024-09-10T21:47:11.344779+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:14:06.368275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"27'7","reported_seq":43,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.569377+0000","last_change":"2024-09-10T21:24:46.855248+0000","last_active":"2024-09-10T21:46:48.569377+0000","last_peered":"2024-09-10T21:46:48.569377+0000","last_clean":"2024-09-10T21:46:48.569377+0000","last_became_active":"2024-09-10T21:24:46.854930+0000","last_became_peered":"2024-09-10T21:24:46.854930+0000","last_unstale":"2024-09-10T21:46:48.569377+0000","last_undegraded":"2024-09-10T21:46:48.569377+0000","last_fullsized":"2024-09-10T21:46:48.569377+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:46:55.286321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"27'4395","reported_seq":4424,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.566233+0000","last_change":"2024-09-10T21:24:52.063112+0000","last_active":"2024-09-10T21:46:48.566233+0000","last_peered":"2024-09-10T21:46:48.566233+0000","last_clean":"2024-09-10T21:46:48.566233+0000","last_became_active":"2024-09-10T21:24:52.062395+0000","last_became_peered":"2024-09-10T21:24:52.062395+0000","last_unstale":"2024-09-10T21:46:48.566233+0000","last_undegraded":"2024-09-10T21:46:48.566233+0000","last_fullsized":"2024-09-10T21:46:48.566233+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":4395,"log_dups_size":0,"ondisk_log_size":4395,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:21:53.080013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21722,"num_write_kb":475710,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"27'13","reported_seq":33,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.321587+0000","last_change":"2024-09-10T21:25:28.513604+0000","last_active":"2024-09-10T21:47:11.321587+0000","last_peered":"2024-09-10T21:47:11.321587+0000","last_clean":"2024-09-10T21:47:11.321587+0000","last_became_active":"2024-09-10T21:25:28.513250+0000","last_became_peered":"2024-09-10T21:25:28.513250+0000","last_unstale":"2024-09-10T21:47:11.321587+0000","last_undegraded":"2024-09-10T21:47:11.321587+0000","last_fullsized":"2024-09-10T21:47:11.321587+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:54:13.200082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.f","version":"27'4","reported_seq":40,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.587704+0000","last_change":"2024-09-10T21:24:46.860330+0000","last_active":"2024-09-10T21:46:48.587704+0000","last_peered":"2024-09-10T21:46:48.587704+0000","last_clean":"2024-09-10T21:46:48.587704+0000","last_became_active":"2024-09-10T21:24:46.860053+0000","last_became_peered":"2024-09-10T21:24:46.860053+0000","last_unstale":"2024-09-10T21:46:48.587704+0000","last_undegraded":"2024-09-10T21:46:48.587704+0000","last_fullsized":"2024-09-10T21:46:48.587704+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:58:53.177740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"27'4437","reported_seq":4466,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.614888+0000","last_change":"2024-09-10T21:24:51.901673+0000","last_active":"2024-09-10T21:46:48.614888+0000","last_peered":"2024-09-10T21:46:48.614888+0000","last_clean":"2024-09-10T21:46:48.614888+0000","last_became_active":"2024-09-10T21:24:51.901222+0000","last_became_peered":"2024-09-10T21:24:51.901222+0000","last_unstale":"2024-09-10T21:46:48.614888+0000","last_undegraded":"2024-09-10T21:46:48.614888+0000","last_fullsized":"2024-09-10T21:46:48.614888+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":4437,"log_dups_size":0,"ondisk_log_size":4437,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:07:52.120299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21706,"num_write_kb":463399,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"27'19","reported_seq":53,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.328131+0000","last_change":"2024-09-10T21:24:47.870589+0000","last_active":"2024-09-10T21:47:11.328131+0000","last_peered":"2024-09-10T21:47:11.328131+0000","last_clean":"2024-09-10T21:47:11.328131+0000","last_became_active":"2024-09-10T21:24:47.861834+0000","last_became_peered":"2024-09-10T21:24:47.861834+0000","last_unstale":"2024-09-10T21:47:11.328131+0000","last_undegraded":"2024-09-10T21:47:11.328131+0000","last_fullsized":"2024-09-10T21:47:11.328131+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:43.363013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"27'15","reported_seq":51,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.643062+0000","last_change":"2024-09-10T21:24:46.852064+0000","last_active":"2024-09-10T21:46:48.643062+0000","last_peered":"2024-09-10T21:46:48.643062+0000","last_clean":"2024-09-10T21:46:48.643062+0000","last_became_active":"2024-09-10T21:24:46.851776+0000","last_became_peered":"2024-09-10T21:24:46.851776+0000","last_unstale":"2024-09-10T21:46:48.643062+0000","last_undegraded":"2024-09-10T21:46:48.643062+0000","last_fullsized":"2024-09-10T21:46:48.643062+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:26:24.371844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"27'5330","reported_seq":5358,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.742778+0000","last_change":"2024-09-10T21:24:51.904449+0000","last_active":"2024-09-10T21:46:48.742778+0000","last_peered":"2024-09-10T21:46:48.742778+0000","last_clean":"2024-09-10T21:46:48.742778+0000","last_became_active":"2024-09-10T21:24:51.903533+0000","last_became_peered":"2024-09-10T21:24:51.903533+0000","last_unstale":"2024-09-10T21:46:48.742778+0000","last_undegraded":"2024-09-10T21:46:48.742778+0000","last_fullsized":"2024-09-10T21:46:48.742778+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":5330,"log_dups_size":0,"ondisk_log_size":5330,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:11.654935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26446,"num_write_kb":547906,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"27'14","reported_seq":47,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.347665+0000","last_change":"2024-09-10T21:24:47.857736+0000","last_active":"2024-09-10T21:47:11.347665+0000","last_peered":"2024-09-10T21:47:11.347665+0000","last_clean":"2024-09-10T21:47:11.347665+0000","last_became_active":"2024-09-10T21:24:47.857437+0000","last_became_peered":"2024-09-10T21:24:47.857437+0000","last_unstale":"2024-09-10T21:47:11.347665+0000","last_undegraded":"2024-09-10T21:47:11.347665+0000","last_fullsized":"2024-09-10T21:47:11.347665+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:13:40.032808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"27'5","reported_seq":40,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.633944+0000","last_change":"2024-09-10T21:24:46.860662+0000","last_active":"2024-09-10T21:46:48.633944+0000","last_peered":"2024-09-10T21:46:48.633944+0000","last_clean":"2024-09-10T21:46:48.633944+0000","last_became_active":"2024-09-10T21:24:46.859796+0000","last_became_peered":"2024-09-10T21:24:46.859796+0000","last_unstale":"2024-09-10T21:46:48.633944+0000","last_undegraded":"2024-09-10T21:46:48.633944+0000","last_fullsized":"2024-09-10T21:46:48.633944+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:48:27.758481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"27'5017","reported_seq":5045,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.677969+0000","last_change":"2024-09-10T21:24:52.069770+0000","last_active":"2024-09-10T21:46:48.677969+0000","last_peered":"2024-09-10T21:46:48.677969+0000","last_clean":"2024-09-10T21:46:48.677969+0000","last_became_active":"2024-09-10T21:24:52.067409+0000","last_became_peered":"2024-09-10T21:24:52.067409+0000","last_unstale":"2024-09-10T21:46:48.677969+0000","last_undegraded":"2024-09-10T21:46:48.677969+0000","last_fullsized":"2024-09-10T21:46:48.677969+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":5017,"log_dups_size":0,"ondisk_log_size":5017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:23:39.622612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24522,"num_write_kb":548501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"27'30","reported_seq":64,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.343245+0000","last_change":"2024-09-10T21:24:47.858692+0000","last_active":"2024-09-10T21:47:11.343245+0000","last_peered":"2024-09-10T21:47:11.343245+0000","last_clean":"2024-09-10T21:47:11.343245+0000","last_became_active":"2024-09-10T21:24:47.858343+0000","last_became_peered":"2024-09-10T21:24:47.858343+0000","last_unstale":"2024-09-10T21:47:11.343245+0000","last_undegraded":"2024-09-10T21:47:11.343245+0000","last_fullsized":"2024-09-10T21:47:11.343245+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:04:36.661791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"27'3","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.555096+0000","last_change":"2024-09-10T21:24:46.864592+0000","last_active":"2024-09-10T21:46:48.555096+0000","last_peered":"2024-09-10T21:46:48.555096+0000","last_clean":"2024-09-10T21:46:48.555096+0000","last_became_active":"2024-09-10T21:24:46.858696+0000","last_became_peered":"2024-09-10T21:24:46.858696+0000","last_unstale":"2024-09-10T21:46:48.555096+0000","last_undegraded":"2024-09-10T21:46:48.555096+0000","last_fullsized":"2024-09-10T21:46:48.555096+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:54:19.356867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"27'4314","reported_seq":4350,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.681791+0000","last_change":"2024-09-10T21:25:28.802910+0000","last_active":"2024-09-10T21:46:48.681791+0000","last_peered":"2024-09-10T21:46:48.681791+0000","last_clean":"2024-09-10T21:46:48.681791+0000","last_became_active":"2024-09-10T21:25:28.802224+0000","last_became_peered":"2024-09-10T21:25:28.802224+0000","last_unstale":"2024-09-10T21:46:48.681791+0000","last_undegraded":"2024-09-10T21:46:48.681791+0000","last_fullsized":"2024-09-10T21:46:48.681791+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":4314,"log_dups_size":0,"ondisk_log_size":4314,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:27:09.948701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20768,"num_write_kb":476051,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7,2],"acting":[3,5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"27'23","reported_seq":57,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.332930+0000","last_change":"2024-09-10T21:24:47.857699+0000","last_active":"2024-09-10T21:47:11.332930+0000","last_peered":"2024-09-10T21:47:11.332930+0000","last_clean":"2024-09-10T21:47:11.332930+0000","last_became_active":"2024-09-10T21:24:47.857429+0000","last_became_peered":"2024-09-10T21:24:47.857429+0000","last_unstale":"2024-09-10T21:47:11.332930+0000","last_undegraded":"2024-09-10T21:47:11.332930+0000","last_fullsized":"2024-09-10T21:47:11.332930+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:07:11.880228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"27'6","reported_seq":42,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.634621+0000","last_change":"2024-09-10T21:24:46.859753+0000","last_active":"2024-09-10T21:46:48.634621+0000","last_peered":"2024-09-10T21:46:48.634621+0000","last_clean":"2024-09-10T21:46:48.634621+0000","last_became_active":"2024-09-10T21:24:46.859299+0000","last_became_peered":"2024-09-10T21:24:46.859299+0000","last_unstale":"2024-09-10T21:46:48.634621+0000","last_undegraded":"2024-09-10T21:46:48.634621+0000","last_fullsized":"2024-09-10T21:46:48.634621+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:06:41.965052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"27'4196","reported_seq":4224,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.388436+0000","last_change":"2024-09-10T21:24:52.064718+0000","last_active":"2024-09-10T21:46:48.388436+0000","last_peered":"2024-09-10T21:46:48.388436+0000","last_clean":"2024-09-10T21:46:48.388436+0000","last_became_active":"2024-09-10T21:24:52.062327+0000","last_became_peered":"2024-09-10T21:24:52.062327+0000","last_unstale":"2024-09-10T21:46:48.388436+0000","last_undegraded":"2024-09-10T21:46:48.388436+0000","last_fullsized":"2024-09-10T21:46:48.388436+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":4196,"log_dups_size":0,"ondisk_log_size":4196,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:43:17.173095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20531,"num_write_kb":446498,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"27'18","reported_seq":52,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.334694+0000","last_change":"2024-09-10T21:24:47.870766+0000","last_active":"2024-09-10T21:47:11.334694+0000","last_peered":"2024-09-10T21:47:11.334694+0000","last_clean":"2024-09-10T21:47:11.334694+0000","last_became_active":"2024-09-10T21:24:47.870276+0000","last_became_peered":"2024-09-10T21:24:47.870276+0000","last_unstale":"2024-09-10T21:47:11.334694+0000","last_undegraded":"2024-09-10T21:47:11.334694+0000","last_fullsized":"2024-09-10T21:47:11.334694+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:54:38.400877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"27'368","reported_seq":403,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:28.433029+0000","last_change":"2024-09-10T21:24:46.867790+0000","last_active":"2024-09-10T21:46:28.433029+0000","last_peered":"2024-09-10T21:46:28.433029+0000","last_clean":"2024-09-10T21:46:28.433029+0000","last_became_active":"2024-09-10T21:24:46.866899+0000","last_became_peered":"2024-09-10T21:24:46.866899+0000","last_unstale":"2024-09-10T21:46:28.433029+0000","last_undegraded":"2024-09-10T21:46:28.433029+0000","last_fullsized":"2024-09-10T21:46:28.433029+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":368,"log_dups_size":0,"ondisk_log_size":368,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:25:04.923730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194976,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":367,"num_write_kb":4285,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"27'3971","reported_seq":3999,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.743646+0000","last_change":"2024-09-10T21:24:51.899803+0000","last_active":"2024-09-10T21:46:48.743646+0000","last_peered":"2024-09-10T21:46:48.743646+0000","last_clean":"2024-09-10T21:46:48.743646+0000","last_became_active":"2024-09-10T21:24:51.895398+0000","last_became_peered":"2024-09-10T21:24:51.895398+0000","last_unstale":"2024-09-10T21:46:48.743646+0000","last_undegraded":"2024-09-10T21:46:48.743646+0000","last_fullsized":"2024-09-10T21:46:48.743646+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":3971,"log_dups_size":0,"ondisk_log_size":3971,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:52:34.689860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19337,"num_write_kb":445978,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"27'22","reported_seq":56,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.334366+0000","last_change":"2024-09-10T21:24:47.857544+0000","last_active":"2024-09-10T21:47:11.334366+0000","last_peered":"2024-09-10T21:47:11.334366+0000","last_clean":"2024-09-10T21:47:11.334366+0000","last_became_active":"2024-09-10T21:24:47.857284+0000","last_became_peered":"2024-09-10T21:24:47.857284+0000","last_unstale":"2024-09-10T21:47:11.334366+0000","last_undegraded":"2024-09-10T21:47:11.334366+0000","last_fullsized":"2024-09-10T21:47:11.334366+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:10:22.747606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"27'32","reported_seq":67,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.672745+0000","last_change":"2024-09-10T21:24:46.859633+0000","last_active":"2024-09-10T21:46:48.672745+0000","last_peered":"2024-09-10T21:46:48.672745+0000","last_clean":"2024-09-10T21:46:48.672745+0000","last_became_active":"2024-09-10T21:24:46.859332+0000","last_became_peered":"2024-09-10T21:24:46.859332+0000","last_unstale":"2024-09-10T21:46:48.672745+0000","last_undegraded":"2024-09-10T21:46:48.672745+0000","last_fullsized":"2024-09-10T21:46:48.672745+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:56:46.975225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":33,"num_write_kb":30,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"27'4136","reported_seq":4164,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.629023+0000","last_change":"2024-09-10T21:24:51.899100+0000","last_active":"2024-09-10T21:46:48.629023+0000","last_peered":"2024-09-10T21:46:48.629023+0000","last_clean":"2024-09-10T21:46:48.629023+0000","last_became_active":"2024-09-10T21:24:51.898642+0000","last_became_peered":"2024-09-10T21:24:51.898642+0000","last_unstale":"2024-09-10T21:46:48.629023+0000","last_undegraded":"2024-09-10T21:46:48.629023+0000","last_fullsized":"2024-09-10T21:46:48.629023+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":4136,"log_dups_size":0,"ondisk_log_size":4136,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:06.394081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20561,"num_write_kb":446825,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"27'23","reported_seq":57,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.345555+0000","last_change":"2024-09-10T21:24:47.867890+0000","last_active":"2024-09-10T21:47:11.345555+0000","last_peered":"2024-09-10T21:47:11.345555+0000","last_clean":"2024-09-10T21:47:11.345555+0000","last_became_active":"2024-09-10T21:24:47.865016+0000","last_became_peered":"2024-09-10T21:24:47.865016+0000","last_unstale":"2024-09-10T21:47:11.345555+0000","last_undegraded":"2024-09-10T21:47:11.345555+0000","last_fullsized":"2024-09-10T21:47:11.345555+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:53:09.082512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"27'3","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.554054+0000","last_change":"2024-09-10T21:24:46.861400+0000","last_active":"2024-09-10T21:46:48.554054+0000","last_peered":"2024-09-10T21:46:48.554054+0000","last_clean":"2024-09-10T21:46:48.554054+0000","last_became_active":"2024-09-10T21:24:46.861136+0000","last_became_peered":"2024-09-10T21:24:46.861136+0000","last_unstale":"2024-09-10T21:46:48.554054+0000","last_undegraded":"2024-09-10T21:46:48.554054+0000","last_fullsized":"2024-09-10T21:46:48.554054+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:35:31.682448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"27'4156","reported_seq":4184,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.591271+0000","last_change":"2024-09-10T21:24:51.907850+0000","last_active":"2024-09-10T21:46:48.591271+0000","last_peered":"2024-09-10T21:46:48.591271+0000","last_clean":"2024-09-10T21:46:48.591271+0000","last_became_active":"2024-09-10T21:24:51.907062+0000","last_became_peered":"2024-09-10T21:24:51.907062+0000","last_unstale":"2024-09-10T21:46:48.591271+0000","last_undegraded":"2024-09-10T21:46:48.591271+0000","last_fullsized":"2024-09-10T21:46:48.591271+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":4156,"log_dups_size":0,"ondisk_log_size":4156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:32:52.519627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20153,"num_write_kb":457162,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"27'25","reported_seq":58,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.346624+0000","last_change":"2024-09-10T21:24:47.861800+0000","last_active":"2024-09-10T21:47:11.346624+0000","last_peered":"2024-09-10T21:47:11.346624+0000","last_clean":"2024-09-10T21:47:11.346624+0000","last_became_active":"2024-09-10T21:24:47.861498+0000","last_became_peered":"2024-09-10T21:24:47.861498+0000","last_unstale":"2024-09-10T21:47:11.346624+0000","last_undegraded":"2024-09-10T21:47:11.346624+0000","last_fullsized":"2024-09-10T21:47:11.346624+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:07:21.128033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"27'3","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:36.356663+0000","last_change":"2024-09-10T21:24:46.871677+0000","last_active":"2024-09-10T21:46:36.356663+0000","last_peered":"2024-09-10T21:46:36.356663+0000","last_clean":"2024-09-10T21:46:36.356663+0000","last_became_active":"2024-09-10T21:24:46.871367+0000","last_became_peered":"2024-09-10T21:24:46.871367+0000","last_unstale":"2024-09-10T21:46:36.356663+0000","last_undegraded":"2024-09-10T21:46:36.356663+0000","last_fullsized":"2024-09-10T21:46:36.356663+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:40:33.632846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"27'3798","reported_seq":3826,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.674934+0000","last_change":"2024-09-10T21:24:52.068637+0000","last_active":"2024-09-10T21:46:48.674934+0000","last_peered":"2024-09-10T21:46:48.674934+0000","last_clean":"2024-09-10T21:46:48.674934+0000","last_became_active":"2024-09-10T21:24:52.068348+0000","last_became_peered":"2024-09-10T21:24:52.068348+0000","last_unstale":"2024-09-10T21:46:48.674934+0000","last_undegraded":"2024-09-10T21:46:48.674934+0000","last_fullsized":"2024-09-10T21:46:48.674934+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":3798,"log_dups_size":0,"ondisk_log_size":3798,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:43:17.173095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18329,"num_write_kb":408933,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"27'5643","reported_seq":5673,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.715900+0000","last_change":"2024-09-10T21:24:52.067732+0000","last_active":"2024-09-10T21:46:48.715900+0000","last_peered":"2024-09-10T21:46:48.715900+0000","last_clean":"2024-09-10T21:46:48.715900+0000","last_became_active":"2024-09-10T21:24:52.067330+0000","last_became_peered":"2024-09-10T21:24:52.067330+0000","last_unstale":"2024-09-10T21:46:48.715900+0000","last_undegraded":"2024-09-10T21:46:48.715900+0000","last_fullsized":"2024-09-10T21:46:48.715900+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":5643,"log_dups_size":0,"ondisk_log_size":5643,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:03:41.110214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27908,"num_write_kb":566060,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"27'20","reported_seq":53,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.330166+0000","last_change":"2024-09-10T21:24:47.862774+0000","last_active":"2024-09-10T21:47:11.330166+0000","last_peered":"2024-09-10T21:47:11.330166+0000","last_clean":"2024-09-10T21:47:11.330166+0000","last_became_active":"2024-09-10T21:24:47.862556+0000","last_became_peered":"2024-09-10T21:24:47.862556+0000","last_unstale":"2024-09-10T21:47:11.330166+0000","last_undegraded":"2024-09-10T21:47:11.330166+0000","last_fullsized":"2024-09-10T21:47:11.330166+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:17:52.343553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"27'2","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:34.118680+0000","last_change":"2024-09-10T21:24:46.862847+0000","last_active":"2024-09-10T21:46:34.118680+0000","last_peered":"2024-09-10T21:46:34.118680+0000","last_clean":"2024-09-10T21:46:34.118680+0000","last_became_active":"2024-09-10T21:24:46.862540+0000","last_became_peered":"2024-09-10T21:24:46.862540+0000","last_unstale":"2024-09-10T21:46:34.118680+0000","last_undegraded":"2024-09-10T21:46:34.118680+0000","last_fullsized":"2024-09-10T21:46:34.118680+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:52:31.628568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"27'5506","reported_seq":5535,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.736687+0000","last_change":"2024-09-10T21:24:51.901224+0000","last_active":"2024-09-10T21:46:48.736687+0000","last_peered":"2024-09-10T21:46:48.736687+0000","last_clean":"2024-09-10T21:46:48.736687+0000","last_became_active":"2024-09-10T21:24:51.900610+0000","last_became_peered":"2024-09-10T21:24:51.900610+0000","last_unstale":"2024-09-10T21:46:48.736687+0000","last_undegraded":"2024-09-10T21:46:48.736687+0000","last_fullsized":"2024-09-10T21:46:48.736687+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":5506,"log_dups_size":0,"ondisk_log_size":5506,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:52:34.689860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27800,"num_write_kb":594222,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"27'26","reported_seq":60,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.328339+0000","last_change":"2024-09-10T21:24:47.861255+0000","last_active":"2024-09-10T21:47:11.328339+0000","last_peered":"2024-09-10T21:47:11.328339+0000","last_clean":"2024-09-10T21:47:11.328339+0000","last_became_active":"2024-09-10T21:24:47.860930+0000","last_became_peered":"2024-09-10T21:24:47.860930+0000","last_unstale":"2024-09-10T21:47:11.328339+0000","last_undegraded":"2024-09-10T21:47:11.328339+0000","last_fullsized":"2024-09-10T21:47:11.328339+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:37:51.784258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"27'3","reported_seq":39,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.632694+0000","last_change":"2024-09-10T21:24:46.858561+0000","last_active":"2024-09-10T21:46:48.632694+0000","last_peered":"2024-09-10T21:46:48.632694+0000","last_clean":"2024-09-10T21:46:48.632694+0000","last_became_active":"2024-09-10T21:24:46.858265+0000","last_became_peered":"2024-09-10T21:24:46.858265+0000","last_unstale":"2024-09-10T21:46:48.632694+0000","last_undegraded":"2024-09-10T21:46:48.632694+0000","last_fullsized":"2024-09-10T21:46:48.632694+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:49:00.454609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"27'6","reported_seq":41,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.657882+0000","last_change":"2024-09-10T21:24:46.856886+0000","last_active":"2024-09-10T21:46:48.657882+0000","last_peered":"2024-09-10T21:46:48.657882+0000","last_clean":"2024-09-10T21:46:48.657882+0000","last_became_active":"2024-09-10T21:24:46.856506+0000","last_became_peered":"2024-09-10T21:24:46.856506+0000","last_unstale":"2024-09-10T21:46:48.657882+0000","last_undegraded":"2024-09-10T21:46:48.657882+0000","last_fullsized":"2024-09-10T21:46:48.657882+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:56:46.975225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"27'19","reported_seq":52,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.337444+0000","last_change":"2024-09-10T21:24:47.848602+0000","last_active":"2024-09-10T21:47:11.337444+0000","last_peered":"2024-09-10T21:47:11.337444+0000","last_clean":"2024-09-10T21:47:11.337444+0000","last_became_active":"2024-09-10T21:24:47.848160+0000","last_became_peered":"2024-09-10T21:24:47.848160+0000","last_unstale":"2024-09-10T21:47:11.337444+0000","last_undegraded":"2024-09-10T21:47:11.337444+0000","last_fullsized":"2024-09-10T21:47:11.337444+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:32:50.027827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"27'4174","reported_seq":4201,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.382991+0000","last_change":"2024-09-10T21:24:51.903553+0000","last_active":"2024-09-10T21:46:48.382991+0000","last_peered":"2024-09-10T21:46:48.382991+0000","last_clean":"2024-09-10T21:46:48.382991+0000","last_became_active":"2024-09-10T21:24:51.903120+0000","last_became_peered":"2024-09-10T21:24:51.903120+0000","last_unstale":"2024-09-10T21:46:48.382991+0000","last_undegraded":"2024-09-10T21:46:48.382991+0000","last_fullsized":"2024-09-10T21:46:48.382991+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":4174,"log_dups_size":0,"ondisk_log_size":4174,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:58.240380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19938,"num_write_kb":435851,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"27'678","reported_seq":713,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.597988+0000","last_change":"2024-09-10T21:24:46.853550+0000","last_active":"2024-09-10T21:46:48.597988+0000","last_peered":"2024-09-10T21:46:48.597988+0000","last_clean":"2024-09-10T21:46:48.597988+0000","last_became_active":"2024-09-10T21:24:46.852204+0000","last_became_peered":"2024-09-10T21:24:46.852204+0000","last_unstale":"2024-09-10T21:46:48.597988+0000","last_undegraded":"2024-09-10T21:46:48.597988+0000","last_fullsized":"2024-09-10T21:46:48.597988+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":678,"log_dups_size":0,"ondisk_log_size":678,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:06:37.669444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":672,"num_write_kb":4259,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"27'18","reported_seq":51,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.327062+0000","last_change":"2024-09-10T21:24:47.849151+0000","last_active":"2024-09-10T21:47:11.327062+0000","last_peered":"2024-09-10T21:47:11.327062+0000","last_clean":"2024-09-10T21:47:11.327062+0000","last_became_active":"2024-09-10T21:24:47.848851+0000","last_became_peered":"2024-09-10T21:24:47.848851+0000","last_unstale":"2024-09-10T21:47:11.327062+0000","last_undegraded":"2024-09-10T21:47:11.327062+0000","last_fullsized":"2024-09-10T21:47:11.327062+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:39:25.828558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"27'4902","reported_seq":4929,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.681160+0000","last_change":"2024-09-10T21:24:51.907549+0000","last_active":"2024-09-10T21:46:48.681160+0000","last_peered":"2024-09-10T21:46:48.681160+0000","last_clean":"2024-09-10T21:46:48.681160+0000","last_became_active":"2024-09-10T21:24:51.907082+0000","last_became_peered":"2024-09-10T21:24:51.907082+0000","last_unstale":"2024-09-10T21:46:48.681160+0000","last_undegraded":"2024-09-10T21:46:48.681160+0000","last_fullsized":"2024-09-10T21:46:48.681160+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":4902,"log_dups_size":0,"ondisk_log_size":4902,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:44:30.813754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24324,"num_write_kb":547512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"27'24","reported_seq":58,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.351212+0000","last_change":"2024-09-10T21:24:47.857804+0000","last_active":"2024-09-10T21:47:11.351212+0000","last_peered":"2024-09-10T21:47:11.351212+0000","last_clean":"2024-09-10T21:47:11.351212+0000","last_became_active":"2024-09-10T21:24:47.857501+0000","last_became_peered":"2024-09-10T21:24:47.857501+0000","last_unstale":"2024-09-10T21:47:11.351212+0000","last_undegraded":"2024-09-10T21:47:11.351212+0000","last_fullsized":"2024-09-10T21:47:11.351212+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:19:20.807422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"27'25","reported_seq":60,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.345524+0000","last_change":"2024-09-10T21:24:47.867803+0000","last_active":"2024-09-10T21:47:11.345524+0000","last_peered":"2024-09-10T21:47:11.345524+0000","last_clean":"2024-09-10T21:47:11.345524+0000","last_became_active":"2024-09-10T21:24:47.867504+0000","last_became_peered":"2024-09-10T21:24:47.867504+0000","last_unstale":"2024-09-10T21:47:11.345524+0000","last_undegraded":"2024-09-10T21:47:11.345524+0000","last_fullsized":"2024-09-10T21:47:11.345524+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:53:30.420454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"27'26","reported_seq":60,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.334694+0000","last_change":"2024-09-10T21:24:47.868901+0000","last_active":"2024-09-10T21:47:11.334694+0000","last_peered":"2024-09-10T21:47:11.334694+0000","last_clean":"2024-09-10T21:47:11.334694+0000","last_became_active":"2024-09-10T21:24:47.868168+0000","last_became_peered":"2024-09-10T21:24:47.868168+0000","last_unstale":"2024-09-10T21:47:11.334694+0000","last_undegraded":"2024-09-10T21:47:11.334694+0000","last_fullsized":"2024-09-10T21:47:11.334694+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:06:21.461089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"27'16","reported_seq":51,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.331166+0000","last_change":"2024-09-10T21:24:47.864561+0000","last_active":"2024-09-10T21:47:11.331166+0000","last_peered":"2024-09-10T21:47:11.331166+0000","last_clean":"2024-09-10T21:47:11.331166+0000","last_became_active":"2024-09-10T21:24:47.864100+0000","last_became_peered":"2024-09-10T21:24:47.864100+0000","last_unstale":"2024-09-10T21:47:11.331166+0000","last_undegraded":"2024-09-10T21:47:11.331166+0000","last_fullsized":"2024-09-10T21:47:11.331166+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:34:13.053494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"27'14","reported_seq":49,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.340508+0000","last_change":"2024-09-10T21:24:47.864303+0000","last_active":"2024-09-10T21:47:11.340508+0000","last_peered":"2024-09-10T21:47:11.340508+0000","last_clean":"2024-09-10T21:47:11.340508+0000","last_became_active":"2024-09-10T21:24:47.863970+0000","last_became_peered":"2024-09-10T21:24:47.863970+0000","last_unstale":"2024-09-10T21:47:11.340508+0000","last_undegraded":"2024-09-10T21:47:11.340508+0000","last_fullsized":"2024-09-10T21:47:11.340508+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:46:36.918539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"27'18","reported_seq":52,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.329189+0000","last_change":"2024-09-10T21:24:47.860734+0000","last_active":"2024-09-10T21:47:11.329189+0000","last_peered":"2024-09-10T21:47:11.329189+0000","last_clean":"2024-09-10T21:47:11.329189+0000","last_became_active":"2024-09-10T21:24:47.860471+0000","last_became_peered":"2024-09-10T21:24:47.860471+0000","last_unstale":"2024-09-10T21:47:11.329189+0000","last_undegraded":"2024-09-10T21:47:11.329189+0000","last_fullsized":"2024-09-10T21:47:11.329189+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:54:13.200082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"27'27","reported_seq":61,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.358154+0000","last_change":"2024-09-10T21:24:47.872504+0000","last_active":"2024-09-10T21:47:11.358154+0000","last_peered":"2024-09-10T21:47:11.358154+0000","last_clean":"2024-09-10T21:47:11.358154+0000","last_became_active":"2024-09-10T21:24:47.872218+0000","last_became_peered":"2024-09-10T21:24:47.872218+0000","last_unstale":"2024-09-10T21:47:11.358154+0000","last_undegraded":"2024-09-10T21:47:11.358154+0000","last_fullsized":"2024-09-10T21:47:11.358154+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:21:32.400520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"27'14","reported_seq":49,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.310878+0000","last_change":"2024-09-10T21:24:47.862167+0000","last_active":"2024-09-10T21:47:11.310878+0000","last_peered":"2024-09-10T21:47:11.310878+0000","last_clean":"2024-09-10T21:47:11.310878+0000","last_became_active":"2024-09-10T21:24:47.859649+0000","last_became_peered":"2024-09-10T21:24:47.859649+0000","last_unstale":"2024-09-10T21:47:11.310878+0000","last_undegraded":"2024-09-10T21:47:11.310878+0000","last_fullsized":"2024-09-10T21:47:11.310878+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:42:59.204284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"27'18","reported_seq":39,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.329156+0000","last_change":"2024-09-10T21:25:28.508979+0000","last_active":"2024-09-10T21:47:11.329156+0000","last_peered":"2024-09-10T21:47:11.329156+0000","last_clean":"2024-09-10T21:47:11.329156+0000","last_became_active":"2024-09-10T21:25:28.508684+0000","last_became_peered":"2024-09-10T21:25:28.508684+0000","last_unstale":"2024-09-10T21:47:11.329156+0000","last_undegraded":"2024-09-10T21:47:11.329156+0000","last_fullsized":"2024-09-10T21:47:11.329156+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:20:59.362941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.29","version":"27'23","reported_seq":43,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.335919+0000","last_change":"2024-09-10T21:25:28.512634+0000","last_active":"2024-09-10T21:47:11.335919+0000","last_peered":"2024-09-10T21:47:11.335919+0000","last_clean":"2024-09-10T21:47:11.335919+0000","last_became_active":"2024-09-10T21:25:28.512230+0000","last_became_peered":"2024-09-10T21:25:28.512230+0000","last_unstale":"2024-09-10T21:47:11.335919+0000","last_undegraded":"2024-09-10T21:47:11.335919+0000","last_fullsized":"2024-09-10T21:47:11.335919+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:17:52.343553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2a","version":"27'22","reported_seq":55,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.345833+0000","last_change":"2024-09-10T21:24:47.872848+0000","last_active":"2024-09-10T21:47:11.345833+0000","last_peered":"2024-09-10T21:47:11.345833+0000","last_clean":"2024-09-10T21:47:11.345833+0000","last_became_active":"2024-09-10T21:24:47.868671+0000","last_became_peered":"2024-09-10T21:24:47.868671+0000","last_unstale":"2024-09-10T21:47:11.345833+0000","last_undegraded":"2024-09-10T21:47:11.345833+0000","last_fullsized":"2024-09-10T21:47:11.345833+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:33:59.291647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"27'28","reported_seq":62,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.339893+0000","last_change":"2024-09-10T21:24:47.857464+0000","last_active":"2024-09-10T21:47:11.339893+0000","last_peered":"2024-09-10T21:47:11.339893+0000","last_clean":"2024-09-10T21:47:11.339893+0000","last_became_active":"2024-09-10T21:24:47.857116+0000","last_became_peered":"2024-09-10T21:24:47.857116+0000","last_unstale":"2024-09-10T21:47:11.339893+0000","last_undegraded":"2024-09-10T21:47:11.339893+0000","last_fullsized":"2024-09-10T21:47:11.339893+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:56:43.269026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"27'19","reported_seq":52,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.344737+0000","last_change":"2024-09-10T21:24:47.871619+0000","last_active":"2024-09-10T21:47:11.344737+0000","last_peered":"2024-09-10T21:47:11.344737+0000","last_clean":"2024-09-10T21:47:11.344737+0000","last_became_active":"2024-09-10T21:24:47.871350+0000","last_became_peered":"2024-09-10T21:24:47.871350+0000","last_unstale":"2024-09-10T21:47:11.344737+0000","last_undegraded":"2024-09-10T21:47:11.344737+0000","last_fullsized":"2024-09-10T21:47:11.344737+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:41.305116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"27'20","reported_seq":54,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.332528+0000","last_change":"2024-09-10T21:24:47.861352+0000","last_active":"2024-09-10T21:47:11.332528+0000","last_peered":"2024-09-10T21:47:11.332528+0000","last_clean":"2024-09-10T21:47:11.332528+0000","last_became_active":"2024-09-10T21:24:47.861007+0000","last_became_peered":"2024-09-10T21:24:47.861007+0000","last_unstale":"2024-09-10T21:47:11.332528+0000","last_undegraded":"2024-09-10T21:47:11.332528+0000","last_fullsized":"2024-09-10T21:47:11.332528+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:25.273458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"27'27","reported_seq":61,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.345633+0000","last_change":"2024-09-10T21:24:47.864205+0000","last_active":"2024-09-10T21:47:11.345633+0000","last_peered":"2024-09-10T21:47:11.345633+0000","last_clean":"2024-09-10T21:47:11.345633+0000","last_became_active":"2024-09-10T21:24:47.863932+0000","last_became_peered":"2024-09-10T21:24:47.863932+0000","last_unstale":"2024-09-10T21:47:11.345633+0000","last_undegraded":"2024-09-10T21:47:11.345633+0000","last_fullsized":"2024-09-10T21:47:11.345633+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:53:17.669629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"27'21","reported_seq":54,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.349081+0000","last_change":"2024-09-10T21:24:47.866225+0000","last_active":"2024-09-10T21:47:11.349081+0000","last_peered":"2024-09-10T21:47:11.349081+0000","last_clean":"2024-09-10T21:47:11.349081+0000","last_became_active":"2024-09-10T21:24:47.865587+0000","last_became_peered":"2024-09-10T21:24:47.865587+0000","last_unstale":"2024-09-10T21:47:11.349081+0000","last_undegraded":"2024-09-10T21:47:11.349081+0000","last_fullsized":"2024-09-10T21:47:11.349081+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:49:26.113068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"27'22","reported_seq":55,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.342691+0000","last_change":"2024-09-10T21:24:47.872669+0000","last_active":"2024-09-10T21:47:11.342691+0000","last_peered":"2024-09-10T21:47:11.342691+0000","last_clean":"2024-09-10T21:47:11.342691+0000","last_became_active":"2024-09-10T21:24:47.865837+0000","last_became_peered":"2024-09-10T21:24:47.865837+0000","last_unstale":"2024-09-10T21:47:11.342691+0000","last_undegraded":"2024-09-10T21:47:11.342691+0000","last_fullsized":"2024-09-10T21:47:11.342691+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:53:09.082512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"27'21","reported_seq":54,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.318871+0000","last_change":"2024-09-10T21:24:47.846567+0000","last_active":"2024-09-10T21:47:11.318871+0000","last_peered":"2024-09-10T21:47:11.318871+0000","last_clean":"2024-09-10T21:47:11.318871+0000","last_became_active":"2024-09-10T21:24:47.846283+0000","last_became_peered":"2024-09-10T21:24:47.846283+0000","last_unstale":"2024-09-10T21:47:11.318871+0000","last_undegraded":"2024-09-10T21:47:11.318871+0000","last_fullsized":"2024-09-10T21:47:11.318871+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:42:15.320518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"27'23","reported_seq":57,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.319292+0000","last_change":"2024-09-10T21:24:47.844888+0000","last_active":"2024-09-10T21:47:11.319292+0000","last_peered":"2024-09-10T21:47:11.319292+0000","last_clean":"2024-09-10T21:47:11.319292+0000","last_became_active":"2024-09-10T21:24:47.844517+0000","last_became_peered":"2024-09-10T21:24:47.844517+0000","last_unstale":"2024-09-10T21:47:11.319292+0000","last_undegraded":"2024-09-10T21:47:11.319292+0000","last_fullsized":"2024-09-10T21:47:11.319292+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:00:14.988282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"27'25","reported_seq":59,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.344844+0000","last_change":"2024-09-10T21:24:47.862746+0000","last_active":"2024-09-10T21:47:11.344844+0000","last_peered":"2024-09-10T21:47:11.344844+0000","last_clean":"2024-09-10T21:47:11.344844+0000","last_became_active":"2024-09-10T21:24:47.862185+0000","last_became_peered":"2024-09-10T21:24:47.862185+0000","last_unstale":"2024-09-10T21:47:11.344844+0000","last_undegraded":"2024-09-10T21:47:11.344844+0000","last_fullsized":"2024-09-10T21:47:11.344844+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:06:21.461089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"27'17","reported_seq":51,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.349979+0000","last_change":"2024-09-10T21:24:47.867727+0000","last_active":"2024-09-10T21:47:11.349979+0000","last_peered":"2024-09-10T21:47:11.349979+0000","last_clean":"2024-09-10T21:47:11.349979+0000","last_became_active":"2024-09-10T21:24:47.861045+0000","last_became_peered":"2024-09-10T21:24:47.861045+0000","last_unstale":"2024-09-10T21:47:11.349979+0000","last_undegraded":"2024-09-10T21:47:11.349979+0000","last_fullsized":"2024-09-10T21:47:11.349979+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:08:30.065968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"27'17","reported_seq":50,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.347800+0000","last_change":"2024-09-10T21:24:47.867410+0000","last_active":"2024-09-10T21:47:11.347800+0000","last_peered":"2024-09-10T21:47:11.347800+0000","last_clean":"2024-09-10T21:47:11.347800+0000","last_became_active":"2024-09-10T21:24:47.867113+0000","last_became_peered":"2024-09-10T21:24:47.867113+0000","last_unstale":"2024-09-10T21:47:11.347800+0000","last_undegraded":"2024-09-10T21:47:11.347800+0000","last_fullsized":"2024-09-10T21:47:11.347800+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:49:53.502472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"27'28","reported_seq":61,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.358458+0000","last_change":"2024-09-10T21:24:47.867473+0000","last_active":"2024-09-10T21:47:11.358458+0000","last_peered":"2024-09-10T21:47:11.358458+0000","last_clean":"2024-09-10T21:47:11.358458+0000","last_became_active":"2024-09-10T21:24:47.867126+0000","last_became_peered":"2024-09-10T21:24:47.867126+0000","last_unstale":"2024-09-10T21:47:11.358458+0000","last_undegraded":"2024-09-10T21:47:11.358458+0000","last_fullsized":"2024-09-10T21:47:11.358458+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:54:32.307507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"27'26","reported_seq":60,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.353536+0000","last_change":"2024-09-10T21:24:47.865450+0000","last_active":"2024-09-10T21:47:11.353536+0000","last_peered":"2024-09-10T21:47:11.353536+0000","last_clean":"2024-09-10T21:47:11.353536+0000","last_became_active":"2024-09-10T21:24:47.865093+0000","last_became_peered":"2024-09-10T21:24:47.865093+0000","last_unstale":"2024-09-10T21:47:11.353536+0000","last_undegraded":"2024-09-10T21:47:11.353536+0000","last_fullsized":"2024-09-10T21:47:11.353536+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:12:18.854783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"27'15","reported_seq":49,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.344819+0000","last_change":"2024-09-10T21:24:47.868093+0000","last_active":"2024-09-10T21:47:11.344819+0000","last_peered":"2024-09-10T21:47:11.344819+0000","last_clean":"2024-09-10T21:47:11.344819+0000","last_became_active":"2024-09-10T21:24:47.867782+0000","last_became_peered":"2024-09-10T21:24:47.867782+0000","last_unstale":"2024-09-10T21:47:11.344819+0000","last_undegraded":"2024-09-10T21:47:11.344819+0000","last_fullsized":"2024-09-10T21:47:11.344819+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:34:13.053494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"27'13","reported_seq":46,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.335909+0000","last_change":"2024-09-10T21:24:47.862204+0000","last_active":"2024-09-10T21:47:11.335909+0000","last_peered":"2024-09-10T21:47:11.335909+0000","last_clean":"2024-09-10T21:47:11.335909+0000","last_became_active":"2024-09-10T21:24:47.861434+0000","last_became_peered":"2024-09-10T21:24:47.861434+0000","last_unstale":"2024-09-10T21:47:11.335909+0000","last_undegraded":"2024-09-10T21:47:11.335909+0000","last_fullsized":"2024-09-10T21:47:11.335909+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:50:03.245285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"27'14","reported_seq":46,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.343049+0000","last_change":"2024-09-10T21:24:47.869248+0000","last_active":"2024-09-10T21:47:11.343049+0000","last_peered":"2024-09-10T21:47:11.343049+0000","last_clean":"2024-09-10T21:47:11.343049+0000","last_became_active":"2024-09-10T21:24:47.868503+0000","last_became_peered":"2024-09-10T21:24:47.868503+0000","last_unstale":"2024-09-10T21:47:11.343049+0000","last_undegraded":"2024-09-10T21:47:11.343049+0000","last_fullsized":"2024-09-10T21:47:11.343049+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:51:31.417845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"27'15","reported_seq":48,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.331365+0000","last_change":"2024-09-10T21:24:47.864802+0000","last_active":"2024-09-10T21:47:11.331365+0000","last_peered":"2024-09-10T21:47:11.331365+0000","last_clean":"2024-09-10T21:47:11.331365+0000","last_became_active":"2024-09-10T21:24:47.863976+0000","last_became_peered":"2024-09-10T21:24:47.863976+0000","last_unstale":"2024-09-10T21:47:11.331365+0000","last_undegraded":"2024-09-10T21:47:11.331365+0000","last_fullsized":"2024-09-10T21:47:11.331365+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:55:08.544075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"27'13","reported_seq":48,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.347483+0000","last_change":"2024-09-10T21:24:47.873364+0000","last_active":"2024-09-10T21:47:11.347483+0000","last_peered":"2024-09-10T21:47:11.347483+0000","last_clean":"2024-09-10T21:47:11.347483+0000","last_became_active":"2024-09-10T21:24:47.873069+0000","last_became_peered":"2024-09-10T21:24:47.873069+0000","last_unstale":"2024-09-10T21:47:11.347483+0000","last_undegraded":"2024-09-10T21:47:11.347483+0000","last_fullsized":"2024-09-10T21:47:11.347483+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:38:21.368499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"27'20","reported_seq":55,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.322849+0000","last_change":"2024-09-10T21:24:47.856936+0000","last_active":"2024-09-10T21:47:11.322849+0000","last_peered":"2024-09-10T21:47:11.322849+0000","last_clean":"2024-09-10T21:47:11.322849+0000","last_became_active":"2024-09-10T21:24:47.854713+0000","last_became_peered":"2024-09-10T21:24:47.854713+0000","last_unstale":"2024-09-10T21:47:11.322849+0000","last_undegraded":"2024-09-10T21:47:11.322849+0000","last_fullsized":"2024-09-10T21:47:11.322849+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:46:36.918539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"27'10","reported_seq":44,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.320206+0000","last_change":"2024-09-10T21:24:47.846492+0000","last_active":"2024-09-10T21:47:11.320206+0000","last_peered":"2024-09-10T21:47:11.320206+0000","last_clean":"2024-09-10T21:47:11.320206+0000","last_became_active":"2024-09-10T21:24:47.846227+0000","last_became_peered":"2024-09-10T21:24:47.846227+0000","last_unstale":"2024-09-10T21:47:11.320206+0000","last_undegraded":"2024-09-10T21:47:11.320206+0000","last_fullsized":"2024-09-10T21:47:11.320206+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:13:58.432819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"27'15","reported_seq":50,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.347364+0000","last_change":"2024-09-10T21:24:47.871469+0000","last_active":"2024-09-10T21:47:11.347364+0000","last_peered":"2024-09-10T21:47:11.347364+0000","last_clean":"2024-09-10T21:47:11.347364+0000","last_became_active":"2024-09-10T21:24:47.871177+0000","last_became_peered":"2024-09-10T21:24:47.871177+0000","last_unstale":"2024-09-10T21:47:11.347364+0000","last_undegraded":"2024-09-10T21:47:11.347364+0000","last_fullsized":"2024-09-10T21:47:11.347364+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:19:24.226916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":723657,"num_write_kb":15994026,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":148509,"ondisk_log_size":148509,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1239,"ondisk_log_size":1239,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":14991163,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":48,"num_read_kb":119,"num_write":3049,"num_write_kb":15770,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":18464768,"data_stored":29982326,"data_compressed":2564296,"data_compressed_allocated":3506176,"data_compressed_original":15089664,"omap_allocated":0,"internal_metadata":0},"log_size":2758,"ondisk_log_size":2758,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":66932,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509714,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":739600,"kb_used_data":7424,"kb_used_omap":1,"kb_used_meta":732158,"kb_avail":93632240,"statfs":{"total":96636764160,"available":95879413760,"internally_reserved":0,"allocated":7602176,"data_stored":11412495,"data_compressed":806609,"data_compressed_allocated":1097728,"data_compressed_original":5083136,"omap_allocated":1590,"internal_metadata":749730250},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509715,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":807000,"kb_used_data":2120,"kb_used_omap":1,"kb_used_meta":804862,"kb_avail":93564840,"statfs":{"total":96636764160,"available":95810396160,"internally_reserved":0,"allocated":2170880,"data_stored":4311586,"data_compressed":503209,"data_compressed_allocated":692224,"data_compressed_original":3006464,"omap_allocated":1594,"internal_metadata":824179142},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509713,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":573484,"kb_used_data":348,"kb_used_omap":1,"kb_used_meta":573118,"kb_avail":93798356,"statfs":{"total":96636764160,"available":96049516544,"internally_reserved":0,"allocated":356352,"data_stored":207064,"data_compressed":5796,"data_compressed_allocated":8192,"data_compressed_original":28672,"omap_allocated":1595,"internal_metadata":586873285},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509712,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1104044,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":1103742,"kb_avail":93267796,"statfs":{"total":96636764160,"available":95506223104,"internally_reserved":0,"allocated":290816,"data_stored":116770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":1130232261},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542419,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":844396,"kb_used_data":3740,"kb_used_omap":1,"kb_used_meta":840638,"kb_avail":93527444,"statfs":{"total":96636764160,"available":95772102656,"internally_reserved":0,"allocated":3829760,"data_stored":4311040,"data_compressed":154551,"data_compressed_allocated":245760,"data_compressed_original":905216,"omap_allocated":1595,"internal_metadata":860813765},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542414,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1062920,"kb_used_data":4792,"kb_used_omap":1,"kb_used_meta":1058110,"kb_avail":93308920,"statfs":{"total":96636764160,"available":95548334080,"internally_reserved":0,"allocated":4907008,"data_stored":8596344,"data_compressed":841246,"data_compressed_allocated":1130496,"data_compressed_original":4997120,"omap_allocated":1593,"internal_metadata":1083505095},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542420,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":711220,"kb_used_data":1380,"kb_used_omap":1,"kb_used_meta":709822,"kb_avail":93660620,"statfs":{"total":96636764160,"available":95908474880,"internally_reserved":0,"allocated":1413120,"data_stored":3023943,"data_compressed":319817,"data_compressed_allocated":421888,"data_compressed_original":2215936,"omap_allocated":1595,"internal_metadata":726858181},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575122,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":626792,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":626494,"kb_avail":93745048,"statfs":{"total":96636764160,"available":95994929152,"internally_reserved":0,"allocated":286720,"data_stored":116700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":641530309},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33466,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33466,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":1064960,"data_stored":2316965,"data_compressed":286351,"data_compressed_allocated":376832,"data_compressed_original":1642496,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":90454,"data_compressed":5796,"data_compressed_allocated":8192,"data_compressed_original":28672,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4624384,"data_stored":8479734,"data_compressed":841246,"data_compressed_allocated":1130496,"data_compressed_original":4997120,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":3547136,"data_stored":4194430,"data_compressed":154551,"data_compressed_allocated":245760,"data_compressed_original":905216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":1888256,"data_stored":4194976,"data_compressed":503209,"data_compressed_allocated":692224,"data_compressed_original":3006464,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":7254016,"data_stored":10705517,"data_compressed":773143,"data_compressed_allocated":1052672,"data_compressed_original":4509696,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T21:47:18.442 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T21:47:18.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:18.621+0000 7f8ad0a00640 1 -- 172.21.15.17:0/2768264500 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8acc1572d0 msgr2=0x7f8acc15b240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:18.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:18.621+0000 7f8ad0a00640 1 --2- 172.21.15.17:0/2768264500 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8acc1572d0 0x7f8acc15b240 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f8ab4009840 tx=0x7f8ab401ef30 comp rx=0 tx=0).stop 2024-09-10T21:47:18.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:18.621+0000 7f8ad0a00640 1 -- 172.21.15.17:0/2768264500 shutdown_connections 2024-09-10T21:47:18.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:18.621+0000 7f8ad0a00640 1 --2- 172.21.15.17:0/2768264500 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8acc1572d0 0x7f8acc15b240 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:18.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:18.621+0000 7f8ad0a00640 1 --2- 172.21.15.17:0/2768264500 >> v2:172.21.15.31:3300/0 conn(0x7f8acc1516c0 0x7f8acc0c25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:18.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:18.621+0000 7f8ad0a00640 1 -- 172.21.15.17:0/2768264500 >> 172.21.15.17:0/2768264500 conn(0x7f8acc070160 msgr2=0x7f8acc070570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:18.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:18.621+0000 7f8ad0a00640 1 -- 172.21.15.17:0/2768264500 shutdown_connections 2024-09-10T21:47:18.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:18.621+0000 7f8ad0a00640 1 -- 172.21.15.17:0/2768264500 wait complete. 2024-09-10T21:47:18.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:18.625+0000 7f8ad0a00640 1 Processor -- start 2024-09-10T21:47:18.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:18.625+0000 7f8ad0a00640 1 -- start start 2024-09-10T21:47:18.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:18.625+0000 7f8ad0a00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8acc0b2070 0x7f8acc1f4520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:18.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:18.625+0000 7f8ad0a00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8acc1516c0 0x7f8acc1e5f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:18.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:18.625+0000 7f8ad0a00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8acc1f1a80 0x7f8acc1e28b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:18.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:18.625+0000 7f8ad0a00640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f8acc15b780 con 0x7f8acc1516c0 2024-09-10T21:47:18.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:18.625+0000 7f8ad0a00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f8acc15ad20 con 0x7f8acc1f1a80 2024-09-10T21:47:18.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:18.625+0000 7f8ad0a00640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f8acc072cd0 con 0x7f8acc0b2070 2024-09-10T21:47:18.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:18.625+0000 7f8ac3e00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8acc1516c0 0x7f8acc1e5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:18.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:18.625+0000 7f8ac3e00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8acc1516c0 0x7f8acc1e5f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:34654/0 (socket says 172.21.15.17:34654) 2024-09-10T21:47:18.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:18.625+0000 7f8ac3e00640 1 -- 172.21.15.17:0/2478916191 learned_addr learned my addr 172.21.15.17:0/2478916191 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:47:18.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:18.625+0000 7f8ac9e00640 1 --2- 172.21.15.17:0/2478916191 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8acc0b2070 0x7f8acc1f4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:18.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:18.625+0000 7f8aca800640 1 --2- 172.21.15.17:0/2478916191 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8acc1f1a80 0x7f8acc1e28b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:18.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:18.625+0000 7f8ac3e00640 1 -- 172.21.15.17:0/2478916191 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8acc0b2070 msgr2=0x7f8acc1f4520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:18.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:18.625+0000 7f8ac3e00640 1 --2- 172.21.15.17:0/2478916191 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8acc0b2070 0x7f8acc1f4520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:18.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:18.625+0000 7f8ac3e00640 1 -- 172.21.15.17:0/2478916191 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8acc1f1a80 msgr2=0x7f8acc1e28b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:18.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:18.625+0000 7f8ac3e00640 1 --2- 172.21.15.17:0/2478916191 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8acc1f1a80 0x7f8acc1e28b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:18.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:18.625+0000 7f8ac3e00640 1 -- 172.21.15.17:0/2478916191 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8acc0c33b0 con 0x7f8acc1516c0 2024-09-10T21:47:18.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:18.625+0000 7f8aca800640 1 --2- 172.21.15.17:0/2478916191 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8acc1f1a80 0x7f8acc1e28b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:47:18.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:18.625+0000 7f8ac3e00640 1 --2- 172.21.15.17:0/2478916191 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8acc1516c0 0x7f8acc1e5f40 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f8ab4009790 tx=0x7f8ab4004620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:18.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:18.625+0000 7f8ac2000640 1 -- 172.21.15.17:0/2478916191 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ab40bf070 con 0x7f8acc1516c0 2024-09-10T21:47:18.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:18.625+0000 7f8ac2000640 1 -- 172.21.15.17:0/2478916191 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8ab4003ab0 con 0x7f8acc1516c0 2024-09-10T21:47:18.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:18.625+0000 7f8ac2000640 1 -- 172.21.15.17:0/2478916191 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ab40b5040 con 0x7f8acc1516c0 2024-09-10T21:47:18.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:18.625+0000 7f8ad0a00640 1 -- 172.21.15.17:0/2478916191 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8acc0c3670 con 0x7f8acc1516c0 2024-09-10T21:47:18.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:18.625+0000 7f8ad0a00640 1 -- 172.21.15.17:0/2478916191 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8acc0c3c30 con 0x7f8acc1516c0 2024-09-10T21:47:18.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:18.629+0000 7f8ad0a00640 1 -- 172.21.15.17:0/2478916191 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8acc0c2cd0 con 0x7f8acc1516c0 2024-09-10T21:47:18.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:18.629+0000 7f8ac2000640 1 -- 172.21.15.17:0/2478916191 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f8ab40b8070 con 0x7f8acc1516c0 2024-09-10T21:47:18.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:18.629+0000 7f8ac2000640 1 --2- 172.21.15.17:0/2478916191 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f8aa80725e0 0x7f8aa8074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:18.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:18.629+0000 7f8ac2000640 1 -- 172.21.15.17:0/2478916191 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f8ab4130530 con 0x7f8acc1516c0 2024-09-10T21:47:18.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:18.629+0000 7f8ac9e00640 1 --2- 172.21.15.17:0/2478916191 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f8aa80725e0 0x7f8aa8074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:18.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:18.633+0000 7f8ac9e00640 1 --2- 172.21.15.17:0/2478916191 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f8aa80725e0 0x7f8aa8074aa0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f8abc009b70 tx=0x7f8abc0091e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:18.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:18.633+0000 7f8ac2000640 1 -- 172.21.15.17:0/2478916191 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f8ab4018a90 con 0x7f8acc1516c0 2024-09-10T21:47:18.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:18.809+0000 7f8ad0a00640 1 -- 172.21.15.17:0/2478916191 --> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8acc0c10a0 con 0x7f8aa80725e0 2024-09-10T21:47:18.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:18.813+0000 7f8ac2000640 1 -- 172.21.15.17:0/2478916191 <== mgr.4100 v2:172.21.15.31:6808/3101308586 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+334037 (secure 0 0 0) 0x7f8acc0c10a0 con 0x7f8aa80725e0 2024-09-10T21:47:18.821 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-10T21:47:18.823 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2024-09-10T21:47:18.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:18.817+0000 7f8ad0a00640 1 -- 172.21.15.17:0/2478916191 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f8aa80725e0 msgr2=0x7f8aa8074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:18.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:18.817+0000 7f8ad0a00640 1 --2- 172.21.15.17:0/2478916191 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f8aa80725e0 0x7f8aa8074aa0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f8abc009b70 tx=0x7f8abc0091e0 comp rx=0 tx=0).stop 2024-09-10T21:47:18.824 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:18.817+0000 7f8ad0a00640 1 -- 172.21.15.17:0/2478916191 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8acc1516c0 msgr2=0x7f8acc1e5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:18.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:18.817+0000 7f8ad0a00640 1 --2- 172.21.15.17:0/2478916191 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8acc1516c0 0x7f8acc1e5f40 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f8ab4009790 tx=0x7f8ab4004620 comp rx=0 tx=0).stop 2024-09-10T21:47:18.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:18.817+0000 7f8ad0a00640 1 -- 172.21.15.17:0/2478916191 shutdown_connections 2024-09-10T21:47:18.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:18.817+0000 7f8ad0a00640 1 --2- 172.21.15.17:0/2478916191 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f8aa80725e0 0x7f8aa8074aa0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:18.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:18.817+0000 7f8ad0a00640 1 --2- 172.21.15.17:0/2478916191 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f8acc1f1a80 0x7f8acc1e28b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:18.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:18.817+0000 7f8ad0a00640 1 --2- 172.21.15.17:0/2478916191 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8acc1516c0 0x7f8acc1e5f40 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:18.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:18.817+0000 7f8ad0a00640 1 --2- 172.21.15.17:0/2478916191 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8acc0b2070 0x7f8acc1f4520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:18.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:18.817+0000 7f8ad0a00640 1 -- 172.21.15.17:0/2478916191 >> 172.21.15.17:0/2478916191 conn(0x7f8acc070160 msgr2=0x7f8acc070540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:18.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:18.817+0000 7f8ad0a00640 1 -- 172.21.15.17:0/2478916191 shutdown_connections 2024-09-10T21:47:18.825 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:18.817+0000 7f8ad0a00640 1 -- 172.21.15.17:0/2478916191 wait complete. 2024-09-10T21:47:18.842 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":702,"stamp":"2024-09-10T21:47:17.984631+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":15581531,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":114,"num_read_kb":174,"num_write":726807,"num_write_kb":16011424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":152578,"ondisk_log_size":152578,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":6469456,"kb_used_data":20368,"kb_used_omap":12,"kb_used_meta":6448947,"kb_avail":748505264,"statfs":{"total":773094113280,"available":766469390336,"internally_reserved":0,"allocated":20856832,"data_stored":32095942,"data_compressed":2631228,"data_compressed_allocated":3596288,"data_compressed_original":16236544,"omap_allocated":12752,"internal_metadata":6603722288},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":249,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003773"},"pg_stats":[{"pgid":"2.1f","version":"27'178","reported_seq":215,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.283456+0000","last_change":"2024-09-10T21:24:46.864556+0000","last_active":"2024-09-10T21:47:11.283456+0000","last_peered":"2024-09-10T21:47:11.283456+0000","last_clean":"2024-09-10T21:47:11.283456+0000","last_became_active":"2024-09-10T21:24:46.864288+0000","last_became_peered":"2024-09-10T21:24:46.864288+0000","last_unstale":"2024-09-10T21:47:11.283456+0000","last_undegraded":"2024-09-10T21:47:11.283456+0000","last_fullsized":"2024-09-10T21:47:11.283456+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":178,"log_dups_size":0,"ondisk_log_size":178,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:00:38.734807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2316909,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":173,"num_write_kb":2361,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"27'12","reported_seq":48,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.345156+0000","last_change":"2024-09-10T21:24:47.864452+0000","last_active":"2024-09-10T21:47:11.345156+0000","last_peered":"2024-09-10T21:47:11.345156+0000","last_clean":"2024-09-10T21:47:11.345156+0000","last_became_active":"2024-09-10T21:24:47.864168+0000","last_became_peered":"2024-09-10T21:24:47.864168+0000","last_unstale":"2024-09-10T21:47:11.345156+0000","last_undegraded":"2024-09-10T21:47:11.345156+0000","last_fullsized":"2024-09-10T21:47:11.345156+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:51:31.417845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"27'5290","reported_seq":5318,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.730211+0000","last_change":"2024-09-10T21:24:52.070054+0000","last_active":"2024-09-10T21:46:48.730211+0000","last_peered":"2024-09-10T21:46:48.730211+0000","last_clean":"2024-09-10T21:46:48.730211+0000","last_became_active":"2024-09-10T21:24:52.067258+0000","last_became_peered":"2024-09-10T21:24:52.067258+0000","last_unstale":"2024-09-10T21:46:48.730211+0000","last_undegraded":"2024-09-10T21:46:48.730211+0000","last_fullsized":"2024-09-10T21:46:48.730211+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":5290,"log_dups_size":0,"ondisk_log_size":5290,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:21:53.080013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25332,"num_write_kb":555711,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"27'5","reported_seq":43,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.553182+0000","last_change":"2024-09-10T21:24:46.860648+0000","last_active":"2024-09-10T21:46:48.553182+0000","last_peered":"2024-09-10T21:46:48.553182+0000","last_clean":"2024-09-10T21:46:48.553182+0000","last_became_active":"2024-09-10T21:24:46.860341+0000","last_became_peered":"2024-09-10T21:24:46.860341+0000","last_unstale":"2024-09-10T21:46:48.553182+0000","last_undegraded":"2024-09-10T21:46:48.553182+0000","last_fullsized":"2024-09-10T21:46:48.553182+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:39:24.819979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"27'19","reported_seq":54,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.345477+0000","last_change":"2024-09-10T21:24:47.865617+0000","last_active":"2024-09-10T21:47:11.345477+0000","last_peered":"2024-09-10T21:47:11.345477+0000","last_clean":"2024-09-10T21:47:11.345477+0000","last_became_active":"2024-09-10T21:24:47.865365+0000","last_became_peered":"2024-09-10T21:24:47.865365+0000","last_unstale":"2024-09-10T21:47:11.345477+0000","last_undegraded":"2024-09-10T21:47:11.345477+0000","last_fullsized":"2024-09-10T21:47:11.345477+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:10:05.971996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"27'4776","reported_seq":4804,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.713274+0000","last_change":"2024-09-10T21:24:51.900077+0000","last_active":"2024-09-10T21:46:48.713274+0000","last_peered":"2024-09-10T21:46:48.713274+0000","last_clean":"2024-09-10T21:46:48.713274+0000","last_became_active":"2024-09-10T21:24:51.899652+0000","last_became_peered":"2024-09-10T21:24:51.899652+0000","last_unstale":"2024-09-10T21:46:48.713274+0000","last_undegraded":"2024-09-10T21:46:48.713274+0000","last_fullsized":"2024-09-10T21:46:48.713274+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":4776,"log_dups_size":0,"ondisk_log_size":4776,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:40:49.585857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23087,"num_write_kb":517913,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"27'7","reported_seq":43,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.636914+0000","last_change":"2024-09-10T21:24:46.861040+0000","last_active":"2024-09-10T21:46:48.636914+0000","last_peered":"2024-09-10T21:46:48.636914+0000","last_clean":"2024-09-10T21:46:48.636914+0000","last_became_active":"2024-09-10T21:24:46.860780+0000","last_became_peered":"2024-09-10T21:24:46.860780+0000","last_unstale":"2024-09-10T21:46:48.636914+0000","last_undegraded":"2024-09-10T21:46:48.636914+0000","last_fullsized":"2024-09-10T21:46:48.636914+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:53:31.454472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"27'16","reported_seq":48,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.347558+0000","last_change":"2024-09-10T21:24:47.877997+0000","last_active":"2024-09-10T21:47:11.347558+0000","last_peered":"2024-09-10T21:47:11.347558+0000","last_clean":"2024-09-10T21:47:11.347558+0000","last_became_active":"2024-09-10T21:24:47.877765+0000","last_became_peered":"2024-09-10T21:24:47.877765+0000","last_unstale":"2024-09-10T21:47:11.347558+0000","last_undegraded":"2024-09-10T21:47:11.347558+0000","last_fullsized":"2024-09-10T21:47:11.347558+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:33:20.590209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"27'4118","reported_seq":4153,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.560547+0000","last_change":"2024-09-10T21:25:28.512435+0000","last_active":"2024-09-10T21:46:48.560547+0000","last_peered":"2024-09-10T21:46:48.560547+0000","last_clean":"2024-09-10T21:46:48.560547+0000","last_became_active":"2024-09-10T21:25:28.512023+0000","last_became_peered":"2024-09-10T21:25:28.512023+0000","last_unstale":"2024-09-10T21:46:48.560547+0000","last_undegraded":"2024-09-10T21:46:48.560547+0000","last_fullsized":"2024-09-10T21:46:48.560547+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":4118,"log_dups_size":0,"ondisk_log_size":4118,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:43:17.173095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18983,"num_write_kb":440322,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,2],"acting":[4,3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"27'5","reported_seq":41,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.677062+0000","last_change":"2024-09-10T21:24:46.873044+0000","last_active":"2024-09-10T21:46:48.677062+0000","last_peered":"2024-09-10T21:46:48.677062+0000","last_clean":"2024-09-10T21:46:48.677062+0000","last_became_active":"2024-09-10T21:24:46.872075+0000","last_became_peered":"2024-09-10T21:24:46.872075+0000","last_unstale":"2024-09-10T21:46:48.677062+0000","last_undegraded":"2024-09-10T21:46:48.677062+0000","last_fullsized":"2024-09-10T21:46:48.677062+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:40:33.632846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"27'30","reported_seq":62,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.342014+0000","last_change":"2024-09-10T21:24:47.878094+0000","last_active":"2024-09-10T21:47:11.342014+0000","last_peered":"2024-09-10T21:47:11.342014+0000","last_clean":"2024-09-10T21:47:11.342014+0000","last_became_active":"2024-09-10T21:24:47.876688+0000","last_became_peered":"2024-09-10T21:24:47.876688+0000","last_unstale":"2024-09-10T21:47:11.342014+0000","last_undegraded":"2024-09-10T21:47:11.342014+0000","last_fullsized":"2024-09-10T21:47:11.342014+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:03:37.076080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"27'3999","reported_seq":4026,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.559967+0000","last_change":"2024-09-10T21:24:51.900208+0000","last_active":"2024-09-10T21:46:48.559967+0000","last_peered":"2024-09-10T21:46:48.559967+0000","last_clean":"2024-09-10T21:46:48.559967+0000","last_became_active":"2024-09-10T21:24:51.899576+0000","last_became_peered":"2024-09-10T21:24:51.899576+0000","last_unstale":"2024-09-10T21:46:48.559967+0000","last_undegraded":"2024-09-10T21:46:48.559967+0000","last_fullsized":"2024-09-10T21:46:48.559967+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":3999,"log_dups_size":0,"ondisk_log_size":3999,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:52:49.479932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18841,"num_write_kb":435514,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"27'15","reported_seq":49,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.350364+0000","last_change":"2024-09-10T21:24:47.869830+0000","last_active":"2024-09-10T21:47:11.350364+0000","last_peered":"2024-09-10T21:47:11.350364+0000","last_clean":"2024-09-10T21:47:11.350364+0000","last_became_active":"2024-09-10T21:24:47.869569+0000","last_became_peered":"2024-09-10T21:24:47.869569+0000","last_unstale":"2024-09-10T21:47:11.350364+0000","last_undegraded":"2024-09-10T21:47:11.350364+0000","last_fullsized":"2024-09-10T21:47:11.350364+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:42:15.320518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"27'1003","reported_seq":1040,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.666697+0000","last_change":"2024-09-10T21:24:46.867297+0000","last_active":"2024-09-10T21:46:48.666697+0000","last_peered":"2024-09-10T21:46:48.666697+0000","last_clean":"2024-09-10T21:46:48.666697+0000","last_became_active":"2024-09-10T21:24:46.866893+0000","last_became_peered":"2024-09-10T21:24:46.866893+0000","last_unstale":"2024-09-10T21:46:48.666697+0000","last_undegraded":"2024-09-10T21:46:48.666697+0000","last_fullsized":"2024-09-10T21:46:48.666697+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":1003,"log_dups_size":0,"ondisk_log_size":1003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:53.197666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":997,"num_write_kb":4359,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"27'4606","reported_seq":4634,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.705887+0000","last_change":"2024-09-10T21:24:51.908412+0000","last_active":"2024-09-10T21:46:48.705887+0000","last_peered":"2024-09-10T21:46:48.705887+0000","last_clean":"2024-09-10T21:46:48.705887+0000","last_became_active":"2024-09-10T21:24:51.907923+0000","last_became_peered":"2024-09-10T21:24:51.907923+0000","last_unstale":"2024-09-10T21:46:48.705887+0000","last_undegraded":"2024-09-10T21:46:48.705887+0000","last_fullsized":"2024-09-10T21:46:48.705887+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":4606,"log_dups_size":0,"ondisk_log_size":4606,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:26:46.479501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22377,"num_write_kb":531442,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"27'17","reported_seq":52,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.331673+0000","last_change":"2024-09-10T21:24:47.862479+0000","last_active":"2024-09-10T21:47:11.331673+0000","last_peered":"2024-09-10T21:47:11.331673+0000","last_clean":"2024-09-10T21:47:11.331673+0000","last_became_active":"2024-09-10T21:24:47.861842+0000","last_became_peered":"2024-09-10T21:24:47.861842+0000","last_unstale":"2024-09-10T21:47:11.331673+0000","last_undegraded":"2024-09-10T21:47:11.331673+0000","last_fullsized":"2024-09-10T21:47:11.331673+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:53:09.082512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"27'3","reported_seq":39,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.551236+0000","last_change":"2024-09-10T21:24:46.852467+0000","last_active":"2024-09-10T21:46:48.551236+0000","last_peered":"2024-09-10T21:46:48.551236+0000","last_clean":"2024-09-10T21:46:48.551236+0000","last_became_active":"2024-09-10T21:24:46.851027+0000","last_became_peered":"2024-09-10T21:24:46.851027+0000","last_unstale":"2024-09-10T21:46:48.551236+0000","last_undegraded":"2024-09-10T21:46:48.551236+0000","last_fullsized":"2024-09-10T21:46:48.551236+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:39:27.441714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"27'4524","reported_seq":4552,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.720455+0000","last_change":"2024-09-10T21:24:51.903577+0000","last_active":"2024-09-10T21:46:48.720455+0000","last_peered":"2024-09-10T21:46:48.720455+0000","last_clean":"2024-09-10T21:46:48.720455+0000","last_became_active":"2024-09-10T21:24:51.903031+0000","last_became_peered":"2024-09-10T21:24:51.903031+0000","last_unstale":"2024-09-10T21:46:48.720455+0000","last_undegraded":"2024-09-10T21:46:48.720455+0000","last_fullsized":"2024-09-10T21:46:48.720455+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":4524,"log_dups_size":0,"ondisk_log_size":4524,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:08:52.787720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21954,"num_write_kb":484896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"27'18","reported_seq":53,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.328856+0000","last_change":"2024-09-10T21:24:47.856334+0000","last_active":"2024-09-10T21:47:11.328856+0000","last_peered":"2024-09-10T21:47:11.328856+0000","last_clean":"2024-09-10T21:47:11.328856+0000","last_became_active":"2024-09-10T21:24:47.856003+0000","last_became_peered":"2024-09-10T21:24:47.856003+0000","last_unstale":"2024-09-10T21:47:11.328856+0000","last_undegraded":"2024-09-10T21:47:11.328856+0000","last_fullsized":"2024-09-10T21:47:11.328856+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:58:53.698584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"27'2","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.586182+0000","last_change":"2024-09-10T21:24:46.853373+0000","last_active":"2024-09-10T21:46:48.586182+0000","last_peered":"2024-09-10T21:46:48.586182+0000","last_clean":"2024-09-10T21:46:48.586182+0000","last_became_active":"2024-09-10T21:24:46.852993+0000","last_became_peered":"2024-09-10T21:24:46.852993+0000","last_unstale":"2024-09-10T21:46:48.586182+0000","last_undegraded":"2024-09-10T21:46:48.586182+0000","last_fullsized":"2024-09-10T21:46:48.586182+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:53:31.454472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"27'5859","reported_seq":5887,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.677773+0000","last_change":"2024-09-10T21:24:52.065246+0000","last_active":"2024-09-10T21:46:48.677773+0000","last_peered":"2024-09-10T21:46:48.677773+0000","last_clean":"2024-09-10T21:46:48.677773+0000","last_became_active":"2024-09-10T21:24:52.064884+0000","last_became_peered":"2024-09-10T21:24:52.064884+0000","last_unstale":"2024-09-10T21:46:48.677773+0000","last_undegraded":"2024-09-10T21:46:48.677773+0000","last_fullsized":"2024-09-10T21:46:48.677773+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":5859,"log_dups_size":0,"ondisk_log_size":5859,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:54:54.393305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27858,"num_write_kb":655707,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"27'9","reported_seq":44,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.296539+0000","last_change":"2024-09-10T21:24:47.856517+0000","last_active":"2024-09-10T21:47:11.296539+0000","last_peered":"2024-09-10T21:47:11.296539+0000","last_clean":"2024-09-10T21:47:11.296539+0000","last_became_active":"2024-09-10T21:24:47.856185+0000","last_became_peered":"2024-09-10T21:24:47.856185+0000","last_unstale":"2024-09-10T21:47:11.296539+0000","last_undegraded":"2024-09-10T21:47:11.296539+0000","last_fullsized":"2024-09-10T21:47:11.296539+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:26:00.852818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"27'5","reported_seq":42,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:36.457065+0000","last_change":"2024-09-10T21:24:46.867888+0000","last_active":"2024-09-10T21:46:36.457065+0000","last_peered":"2024-09-10T21:46:36.457065+0000","last_clean":"2024-09-10T21:46:36.457065+0000","last_became_active":"2024-09-10T21:24:46.866477+0000","last_became_peered":"2024-09-10T21:24:46.866477+0000","last_unstale":"2024-09-10T21:46:36.457065+0000","last_undegraded":"2024-09-10T21:46:36.457065+0000","last_fullsized":"2024-09-10T21:46:36.457065+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:39:27.441714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"27'4736","reported_seq":4764,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.660872+0000","last_change":"2024-09-10T21:24:51.908376+0000","last_active":"2024-09-10T21:46:48.660872+0000","last_peered":"2024-09-10T21:46:48.660872+0000","last_clean":"2024-09-10T21:46:48.660872+0000","last_became_active":"2024-09-10T21:24:51.907878+0000","last_became_peered":"2024-09-10T21:24:51.907878+0000","last_unstale":"2024-09-10T21:46:48.660872+0000","last_undegraded":"2024-09-10T21:46:48.660872+0000","last_fullsized":"2024-09-10T21:46:48.660872+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":4736,"log_dups_size":0,"ondisk_log_size":4736,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:39:02.492952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22584,"num_write_kb":479265,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"27'25","reported_seq":60,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.323165+0000","last_change":"2024-09-10T21:24:47.844823+0000","last_active":"2024-09-10T21:47:11.323165+0000","last_peered":"2024-09-10T21:47:11.323165+0000","last_clean":"2024-09-10T21:47:11.323165+0000","last_became_active":"2024-09-10T21:24:47.844505+0000","last_became_peered":"2024-09-10T21:24:47.844505+0000","last_unstale":"2024-09-10T21:47:11.323165+0000","last_undegraded":"2024-09-10T21:47:11.323165+0000","last_fullsized":"2024-09-10T21:47:11.323165+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:37:51.784258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"27'9","reported_seq":45,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.639567+0000","last_change":"2024-09-10T21:24:46.862172+0000","last_active":"2024-09-10T21:46:48.639567+0000","last_peered":"2024-09-10T21:46:48.639567+0000","last_clean":"2024-09-10T21:46:48.639567+0000","last_became_active":"2024-09-10T21:24:46.861473+0000","last_became_peered":"2024-09-10T21:24:46.861473+0000","last_unstale":"2024-09-10T21:46:48.639567+0000","last_undegraded":"2024-09-10T21:46:48.639567+0000","last_fullsized":"2024-09-10T21:46:48.639567+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:53:31.454472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":53,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"27'4521","reported_seq":4550,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.535321+0000","last_change":"2024-09-10T21:24:52.062841+0000","last_active":"2024-09-10T21:46:48.535321+0000","last_peered":"2024-09-10T21:46:48.535321+0000","last_clean":"2024-09-10T21:46:48.535321+0000","last_became_active":"2024-09-10T21:24:52.061581+0000","last_became_peered":"2024-09-10T21:24:52.061581+0000","last_unstale":"2024-09-10T21:46:48.535321+0000","last_undegraded":"2024-09-10T21:46:48.535321+0000","last_fullsized":"2024-09-10T21:46:48.535321+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":4521,"log_dups_size":0,"ondisk_log_size":4521,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:34:03.325781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21874,"num_write_kb":476387,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"27'18","reported_seq":53,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.324770+0000","last_change":"2024-09-10T21:24:47.859801+0000","last_active":"2024-09-10T21:47:11.324770+0000","last_peered":"2024-09-10T21:47:11.324770+0000","last_clean":"2024-09-10T21:47:11.324770+0000","last_became_active":"2024-09-10T21:24:47.859483+0000","last_became_peered":"2024-09-10T21:24:47.859483+0000","last_unstale":"2024-09-10T21:47:11.324770+0000","last_undegraded":"2024-09-10T21:47:11.324770+0000","last_fullsized":"2024-09-10T21:47:11.324770+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:38:21.368499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"27'326","reported_seq":362,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:36.365019+0000","last_change":"2024-09-10T21:24:46.860624+0000","last_active":"2024-09-10T21:46:36.365019+0000","last_peered":"2024-09-10T21:46:36.365019+0000","last_clean":"2024-09-10T21:46:36.365019+0000","last_became_active":"2024-09-10T21:24:46.860309+0000","last_became_peered":"2024-09-10T21:24:46.860309+0000","last_unstale":"2024-09-10T21:46:36.365019+0000","last_undegraded":"2024-09-10T21:46:36.365019+0000","last_fullsized":"2024-09-10T21:46:36.365019+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":326,"log_dups_size":0,"ondisk_log_size":326,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:37:12.332013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":704,"num_write_kb":242,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"27'4358","reported_seq":4388,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.745954+0000","last_change":"2024-09-10T21:24:52.068581+0000","last_active":"2024-09-10T21:46:48.745954+0000","last_peered":"2024-09-10T21:46:48.745954+0000","last_clean":"2024-09-10T21:46:48.745954+0000","last_became_active":"2024-09-10T21:24:52.068250+0000","last_became_peered":"2024-09-10T21:24:52.068250+0000","last_unstale":"2024-09-10T21:46:48.745954+0000","last_undegraded":"2024-09-10T21:46:48.745954+0000","last_fullsized":"2024-09-10T21:46:48.745954+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":4358,"log_dups_size":0,"ondisk_log_size":4358,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:52:01.990009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21666,"num_write_kb":463209,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"27'18","reported_seq":52,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.321574+0000","last_change":"2024-09-10T21:24:47.854042+0000","last_active":"2024-09-10T21:47:11.321574+0000","last_peered":"2024-09-10T21:47:11.321574+0000","last_clean":"2024-09-10T21:47:11.321574+0000","last_became_active":"2024-09-10T21:24:47.853750+0000","last_became_peered":"2024-09-10T21:24:47.853750+0000","last_unstale":"2024-09-10T21:47:11.321574+0000","last_undegraded":"2024-09-10T21:47:11.321574+0000","last_fullsized":"2024-09-10T21:47:11.321574+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:06:50.775132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"27'2","reported_seq":39,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:45:42.186926+0000","last_change":"2024-09-10T21:24:46.864617+0000","last_active":"2024-09-10T21:45:42.186926+0000","last_peered":"2024-09-10T21:45:42.186926+0000","last_clean":"2024-09-10T21:45:42.186926+0000","last_became_active":"2024-09-10T21:24:46.864149+0000","last_became_peered":"2024-09-10T21:24:46.864149+0000","last_unstale":"2024-09-10T21:45:42.186926+0000","last_undegraded":"2024-09-10T21:45:42.186926+0000","last_fullsized":"2024-09-10T21:45:42.186926+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:24:15.673107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"27'4655","reported_seq":4684,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.520210+0000","last_change":"2024-09-10T21:24:52.061485+0000","last_active":"2024-09-10T21:46:48.520210+0000","last_peered":"2024-09-10T21:46:48.520210+0000","last_clean":"2024-09-10T21:46:48.520210+0000","last_became_active":"2024-09-10T21:24:52.060953+0000","last_became_peered":"2024-09-10T21:24:52.060953+0000","last_unstale":"2024-09-10T21:46:48.520210+0000","last_undegraded":"2024-09-10T21:46:48.520210+0000","last_fullsized":"2024-09-10T21:46:48.520210+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":4655,"log_dups_size":0,"ondisk_log_size":4655,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:49:30.147202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22893,"num_write_kb":513258,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"27'18","reported_seq":52,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.341860+0000","last_change":"2024-09-10T21:24:47.867241+0000","last_active":"2024-09-10T21:47:11.341860+0000","last_peered":"2024-09-10T21:47:11.341860+0000","last_clean":"2024-09-10T21:47:11.341860+0000","last_became_active":"2024-09-10T21:24:47.866907+0000","last_became_peered":"2024-09-10T21:24:47.866907+0000","last_unstale":"2024-09-10T21:47:11.341860+0000","last_undegraded":"2024-09-10T21:47:11.341860+0000","last_fullsized":"2024-09-10T21:47:11.341860+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:07.620800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"27'2","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.562765+0000","last_change":"2024-09-10T21:24:46.856848+0000","last_active":"2024-09-10T21:46:48.562765+0000","last_peered":"2024-09-10T21:46:48.562765+0000","last_clean":"2024-09-10T21:46:48.562765+0000","last_became_active":"2024-09-10T21:24:46.856489+0000","last_became_peered":"2024-09-10T21:24:46.856489+0000","last_unstale":"2024-09-10T21:46:48.562765+0000","last_undegraded":"2024-09-10T21:46:48.562765+0000","last_fullsized":"2024-09-10T21:46:48.562765+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:46:55.286321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"27'4909","reported_seq":4938,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.660587+0000","last_change":"2024-09-10T21:24:51.901016+0000","last_active":"2024-09-10T21:46:48.660587+0000","last_peered":"2024-09-10T21:46:48.660587+0000","last_clean":"2024-09-10T21:46:48.660587+0000","last_became_active":"2024-09-10T21:24:51.900479+0000","last_became_peered":"2024-09-10T21:24:51.900479+0000","last_unstale":"2024-09-10T21:46:48.660587+0000","last_undegraded":"2024-09-10T21:46:48.660587+0000","last_fullsized":"2024-09-10T21:46:48.660587+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":4909,"log_dups_size":0,"ondisk_log_size":4909,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:36.741810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23807,"num_write_kb":525103,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"27'16","reported_seq":51,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.320532+0000","last_change":"2024-09-10T21:24:47.858753+0000","last_active":"2024-09-10T21:47:11.320532+0000","last_peered":"2024-09-10T21:47:11.320532+0000","last_clean":"2024-09-10T21:47:11.320532+0000","last_became_active":"2024-09-10T21:24:47.858438+0000","last_became_peered":"2024-09-10T21:24:47.858438+0000","last_unstale":"2024-09-10T21:47:11.320532+0000","last_undegraded":"2024-09-10T21:47:11.320532+0000","last_fullsized":"2024-09-10T21:47:11.320532+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:12:09.198013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"27'4","reported_seq":40,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:45:39.089516+0000","last_change":"2024-09-10T21:24:46.863610+0000","last_active":"2024-09-10T21:45:39.089516+0000","last_peered":"2024-09-10T21:45:39.089516+0000","last_clean":"2024-09-10T21:45:39.089516+0000","last_became_active":"2024-09-10T21:24:46.863150+0000","last_became_peered":"2024-09-10T21:24:46.863150+0000","last_unstale":"2024-09-10T21:45:39.089516+0000","last_undegraded":"2024-09-10T21:45:39.089516+0000","last_fullsized":"2024-09-10T21:45:39.089516+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:27:35.473995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"27'5335","reported_seq":5364,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.650031+0000","last_change":"2024-09-10T21:24:51.907366+0000","last_active":"2024-09-10T21:46:48.650031+0000","last_peered":"2024-09-10T21:46:48.650031+0000","last_clean":"2024-09-10T21:46:48.650031+0000","last_became_active":"2024-09-10T21:24:51.906888+0000","last_became_peered":"2024-09-10T21:24:51.906888+0000","last_unstale":"2024-09-10T21:46:48.650031+0000","last_undegraded":"2024-09-10T21:46:48.650031+0000","last_fullsized":"2024-09-10T21:46:48.650031+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":5335,"log_dups_size":0,"ondisk_log_size":5335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:24:54.278514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26491,"num_write_kb":568740,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"27'20","reported_seq":55,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.311229+0000","last_change":"2024-09-10T21:24:47.850155+0000","last_active":"2024-09-10T21:47:11.311229+0000","last_peered":"2024-09-10T21:47:11.311229+0000","last_clean":"2024-09-10T21:47:11.311229+0000","last_became_active":"2024-09-10T21:24:47.849846+0000","last_became_peered":"2024-09-10T21:24:47.849846+0000","last_unstale":"2024-09-10T21:47:11.311229+0000","last_undegraded":"2024-09-10T21:47:11.311229+0000","last_fullsized":"2024-09-10T21:47:11.311229+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:13:40.032808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"27'51","reported_seq":136,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.553709+0000","last_change":"2024-09-10T21:24:46.851800+0000","last_active":"2024-09-10T21:46:48.553709+0000","last_peered":"2024-09-10T21:46:48.553709+0000","last_clean":"2024-09-10T21:46:48.553709+0000","last_became_active":"2024-09-10T21:24:46.851200+0000","last_became_peered":"2024-09-10T21:24:46.851200+0000","last_unstale":"2024-09-10T21:46:48.553709+0000","last_undegraded":"2024-09-10T21:46:48.553709+0000","last_fullsized":"2024-09-10T21:46:48.553709+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:00:22.839361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90454,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":48,"num_read_kb":119,"num_write":48,"num_write_kb":119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"27'4677","reported_seq":4706,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.581789+0000","last_change":"2024-09-10T21:24:51.902529+0000","last_active":"2024-09-10T21:46:48.581789+0000","last_peered":"2024-09-10T21:46:48.581789+0000","last_clean":"2024-09-10T21:46:48.581789+0000","last_became_active":"2024-09-10T21:24:51.901141+0000","last_became_peered":"2024-09-10T21:24:51.901141+0000","last_unstale":"2024-09-10T21:46:48.581789+0000","last_undegraded":"2024-09-10T21:46:48.581789+0000","last_fullsized":"2024-09-10T21:46:48.581789+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":4677,"log_dups_size":0,"ondisk_log_size":4677,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:08:40.203348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22963,"num_write_kb":501268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"27'11","reported_seq":46,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.323431+0000","last_change":"2024-09-10T21:24:47.846582+0000","last_active":"2024-09-10T21:47:11.323431+0000","last_peered":"2024-09-10T21:47:11.323431+0000","last_clean":"2024-09-10T21:47:11.323431+0000","last_became_active":"2024-09-10T21:24:47.846358+0000","last_became_peered":"2024-09-10T21:24:47.846358+0000","last_unstale":"2024-09-10T21:47:11.323431+0000","last_undegraded":"2024-09-10T21:47:11.323431+0000","last_fullsized":"2024-09-10T21:47:11.323431+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:58:53.698584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"27'7","reported_seq":45,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.647854+0000","last_change":"2024-09-10T21:24:46.870908+0000","last_active":"2024-09-10T21:46:48.647854+0000","last_peered":"2024-09-10T21:46:48.647854+0000","last_clean":"2024-09-10T21:46:48.647854+0000","last_became_active":"2024-09-10T21:24:46.868375+0000","last_became_peered":"2024-09-10T21:24:46.868375+0000","last_unstale":"2024-09-10T21:46:48.647854+0000","last_undegraded":"2024-09-10T21:46:48.647854+0000","last_fullsized":"2024-09-10T21:46:48.647854+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:37:12.332013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"27'4824","reported_seq":4853,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.659490+0000","last_change":"2024-09-10T21:24:52.069018+0000","last_active":"2024-09-10T21:46:48.659490+0000","last_peered":"2024-09-10T21:46:48.659490+0000","last_clean":"2024-09-10T21:46:48.659490+0000","last_became_active":"2024-09-10T21:24:52.067308+0000","last_became_peered":"2024-09-10T21:24:52.067308+0000","last_unstale":"2024-09-10T21:46:48.659490+0000","last_undegraded":"2024-09-10T21:46:48.659490+0000","last_fullsized":"2024-09-10T21:46:48.659490+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":4824,"log_dups_size":0,"ondisk_log_size":4824,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:07:25.162168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23972,"num_write_kb":500992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"27'13","reported_seq":48,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.338908+0000","last_change":"2024-09-10T21:24:47.871542+0000","last_active":"2024-09-10T21:47:11.338908+0000","last_peered":"2024-09-10T21:47:11.338908+0000","last_clean":"2024-09-10T21:47:11.338908+0000","last_became_active":"2024-09-10T21:24:47.871128+0000","last_became_peered":"2024-09-10T21:24:47.871128+0000","last_unstale":"2024-09-10T21:47:11.338908+0000","last_undegraded":"2024-09-10T21:47:11.338908+0000","last_fullsized":"2024-09-10T21:47:11.338908+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:51:22.876126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"27'3","reported_seq":39,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:36.366557+0000","last_change":"2024-09-10T21:24:46.859218+0000","last_active":"2024-09-10T21:46:36.366557+0000","last_peered":"2024-09-10T21:46:36.366557+0000","last_clean":"2024-09-10T21:46:36.366557+0000","last_became_active":"2024-09-10T21:24:46.853190+0000","last_became_peered":"2024-09-10T21:24:46.853190+0000","last_unstale":"2024-09-10T21:46:36.366557+0000","last_undegraded":"2024-09-10T21:46:36.366557+0000","last_fullsized":"2024-09-10T21:46:36.366557+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:26:34.848759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":134,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:44:42.019323+0000","last_change":"2024-09-10T21:24:37.216542+0000","last_active":"2024-09-10T21:44:42.019323+0000","last_peered":"2024-09-10T21:44:42.019323+0000","last_clean":"2024-09-10T21:44:42.019323+0000","last_became_active":"2024-09-10T21:24:37.215443+0000","last_became_peered":"2024-09-10T21:24:37.215443+0000","last_unstale":"2024-09-10T21:44:42.019323+0000","last_undegraded":"2024-09-10T21:44:42.019323+0000","last_fullsized":"2024-09-10T21:44:42.019323+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:35.877801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:35.877801+0000","last_clean_scrub_stamp":"2024-09-10T21:24:35.877801+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:14:20.260329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"27'4394","reported_seq":4423,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.650952+0000","last_change":"2024-09-10T21:24:52.067645+0000","last_active":"2024-09-10T21:46:48.650952+0000","last_peered":"2024-09-10T21:46:48.650952+0000","last_clean":"2024-09-10T21:46:48.650952+0000","last_became_active":"2024-09-10T21:24:52.067148+0000","last_became_peered":"2024-09-10T21:24:52.067148+0000","last_unstale":"2024-09-10T21:46:48.650952+0000","last_undegraded":"2024-09-10T21:46:48.650952+0000","last_fullsized":"2024-09-10T21:46:48.650952+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":4394,"log_dups_size":0,"ondisk_log_size":4394,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:54:42.435011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21702,"num_write_kb":479680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"27'21","reported_seq":61,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.342553+0000","last_change":"2024-09-10T21:25:28.510354+0000","last_active":"2024-09-10T21:47:11.342553+0000","last_peered":"2024-09-10T21:47:11.342553+0000","last_clean":"2024-09-10T21:47:11.342553+0000","last_became_active":"2024-09-10T21:25:28.509981+0000","last_became_peered":"2024-09-10T21:25:28.509981+0000","last_unstale":"2024-09-10T21:47:11.342553+0000","last_undegraded":"2024-09-10T21:47:11.342553+0000","last_fullsized":"2024-09-10T21:47:11.342553+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:51:42.501764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"27'5","reported_seq":41,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.629368+0000","last_change":"2024-09-10T21:24:46.859532+0000","last_active":"2024-09-10T21:46:48.629368+0000","last_peered":"2024-09-10T21:46:48.629368+0000","last_clean":"2024-09-10T21:46:48.629368+0000","last_became_active":"2024-09-10T21:24:46.859144+0000","last_became_peered":"2024-09-10T21:24:46.859144+0000","last_unstale":"2024-09-10T21:46:48.629368+0000","last_undegraded":"2024-09-10T21:46:48.629368+0000","last_fullsized":"2024-09-10T21:46:48.629368+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:48:41.992441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"27'4328","reported_seq":4365,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.651243+0000","last_change":"2024-09-10T21:25:28.620859+0000","last_active":"2024-09-10T21:46:48.651243+0000","last_peered":"2024-09-10T21:46:48.651243+0000","last_clean":"2024-09-10T21:46:48.651243+0000","last_became_active":"2024-09-10T21:25:28.620415+0000","last_became_peered":"2024-09-10T21:25:28.620415+0000","last_unstale":"2024-09-10T21:46:48.651243+0000","last_undegraded":"2024-09-10T21:46:48.651243+0000","last_fullsized":"2024-09-10T21:46:48.651243+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":4328,"log_dups_size":0,"ondisk_log_size":4328,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:17:39.229564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21368,"num_write_kb":498780,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2,0],"acting":[6,7,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"27'17","reported_seq":50,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.342104+0000","last_change":"2024-09-10T21:24:47.866927+0000","last_active":"2024-09-10T21:47:11.342104+0000","last_peered":"2024-09-10T21:47:11.342104+0000","last_clean":"2024-09-10T21:47:11.342104+0000","last_became_active":"2024-09-10T21:24:47.866650+0000","last_became_peered":"2024-09-10T21:24:47.866650+0000","last_unstale":"2024-09-10T21:47:11.342104+0000","last_undegraded":"2024-09-10T21:47:11.342104+0000","last_fullsized":"2024-09-10T21:47:11.342104+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:47:37.304624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"27'6","reported_seq":42,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.675955+0000","last_change":"2024-09-10T21:24:46.857082+0000","last_active":"2024-09-10T21:46:48.675955+0000","last_peered":"2024-09-10T21:46:48.675955+0000","last_clean":"2024-09-10T21:46:48.675955+0000","last_became_active":"2024-09-10T21:24:46.856799+0000","last_became_peered":"2024-09-10T21:24:46.856799+0000","last_unstale":"2024-09-10T21:46:48.675955+0000","last_undegraded":"2024-09-10T21:46:48.675955+0000","last_fullsized":"2024-09-10T21:46:48.675955+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:40:33.632846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"27'4625","reported_seq":4654,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.739934+0000","last_change":"2024-09-10T21:24:51.911880+0000","last_active":"2024-09-10T21:46:48.739934+0000","last_peered":"2024-09-10T21:46:48.739934+0000","last_clean":"2024-09-10T21:46:48.739934+0000","last_became_active":"2024-09-10T21:24:51.911088+0000","last_became_peered":"2024-09-10T21:24:51.911088+0000","last_unstale":"2024-09-10T21:46:48.739934+0000","last_undegraded":"2024-09-10T21:46:48.739934+0000","last_fullsized":"2024-09-10T21:46:48.739934+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":4625,"log_dups_size":0,"ondisk_log_size":4625,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:38:57.030655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21860,"num_write_kb":505231,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"27'20","reported_seq":54,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.344779+0000","last_change":"2024-09-10T21:24:47.876946+0000","last_active":"2024-09-10T21:47:11.344779+0000","last_peered":"2024-09-10T21:47:11.344779+0000","last_clean":"2024-09-10T21:47:11.344779+0000","last_became_active":"2024-09-10T21:24:47.876690+0000","last_became_peered":"2024-09-10T21:24:47.876690+0000","last_unstale":"2024-09-10T21:47:11.344779+0000","last_undegraded":"2024-09-10T21:47:11.344779+0000","last_fullsized":"2024-09-10T21:47:11.344779+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:14:06.368275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"27'7","reported_seq":43,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.569377+0000","last_change":"2024-09-10T21:24:46.855248+0000","last_active":"2024-09-10T21:46:48.569377+0000","last_peered":"2024-09-10T21:46:48.569377+0000","last_clean":"2024-09-10T21:46:48.569377+0000","last_became_active":"2024-09-10T21:24:46.854930+0000","last_became_peered":"2024-09-10T21:24:46.854930+0000","last_unstale":"2024-09-10T21:46:48.569377+0000","last_undegraded":"2024-09-10T21:46:48.569377+0000","last_fullsized":"2024-09-10T21:46:48.569377+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:46:55.286321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"27'4395","reported_seq":4424,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.566233+0000","last_change":"2024-09-10T21:24:52.063112+0000","last_active":"2024-09-10T21:46:48.566233+0000","last_peered":"2024-09-10T21:46:48.566233+0000","last_clean":"2024-09-10T21:46:48.566233+0000","last_became_active":"2024-09-10T21:24:52.062395+0000","last_became_peered":"2024-09-10T21:24:52.062395+0000","last_unstale":"2024-09-10T21:46:48.566233+0000","last_undegraded":"2024-09-10T21:46:48.566233+0000","last_fullsized":"2024-09-10T21:46:48.566233+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":4395,"log_dups_size":0,"ondisk_log_size":4395,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:21:53.080013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21722,"num_write_kb":475710,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"27'13","reported_seq":33,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.321587+0000","last_change":"2024-09-10T21:25:28.513604+0000","last_active":"2024-09-10T21:47:11.321587+0000","last_peered":"2024-09-10T21:47:11.321587+0000","last_clean":"2024-09-10T21:47:11.321587+0000","last_became_active":"2024-09-10T21:25:28.513250+0000","last_became_peered":"2024-09-10T21:25:28.513250+0000","last_unstale":"2024-09-10T21:47:11.321587+0000","last_undegraded":"2024-09-10T21:47:11.321587+0000","last_fullsized":"2024-09-10T21:47:11.321587+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:54:13.200082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.f","version":"27'4","reported_seq":40,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.587704+0000","last_change":"2024-09-10T21:24:46.860330+0000","last_active":"2024-09-10T21:46:48.587704+0000","last_peered":"2024-09-10T21:46:48.587704+0000","last_clean":"2024-09-10T21:46:48.587704+0000","last_became_active":"2024-09-10T21:24:46.860053+0000","last_became_peered":"2024-09-10T21:24:46.860053+0000","last_unstale":"2024-09-10T21:46:48.587704+0000","last_undegraded":"2024-09-10T21:46:48.587704+0000","last_fullsized":"2024-09-10T21:46:48.587704+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:58:53.177740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"27'4437","reported_seq":4466,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.614888+0000","last_change":"2024-09-10T21:24:51.901673+0000","last_active":"2024-09-10T21:46:48.614888+0000","last_peered":"2024-09-10T21:46:48.614888+0000","last_clean":"2024-09-10T21:46:48.614888+0000","last_became_active":"2024-09-10T21:24:51.901222+0000","last_became_peered":"2024-09-10T21:24:51.901222+0000","last_unstale":"2024-09-10T21:46:48.614888+0000","last_undegraded":"2024-09-10T21:46:48.614888+0000","last_fullsized":"2024-09-10T21:46:48.614888+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":4437,"log_dups_size":0,"ondisk_log_size":4437,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:07:52.120299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21706,"num_write_kb":463399,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"27'19","reported_seq":53,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.328131+0000","last_change":"2024-09-10T21:24:47.870589+0000","last_active":"2024-09-10T21:47:11.328131+0000","last_peered":"2024-09-10T21:47:11.328131+0000","last_clean":"2024-09-10T21:47:11.328131+0000","last_became_active":"2024-09-10T21:24:47.861834+0000","last_became_peered":"2024-09-10T21:24:47.861834+0000","last_unstale":"2024-09-10T21:47:11.328131+0000","last_undegraded":"2024-09-10T21:47:11.328131+0000","last_fullsized":"2024-09-10T21:47:11.328131+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:43.363013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"27'15","reported_seq":51,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.643062+0000","last_change":"2024-09-10T21:24:46.852064+0000","last_active":"2024-09-10T21:46:48.643062+0000","last_peered":"2024-09-10T21:46:48.643062+0000","last_clean":"2024-09-10T21:46:48.643062+0000","last_became_active":"2024-09-10T21:24:46.851776+0000","last_became_peered":"2024-09-10T21:24:46.851776+0000","last_unstale":"2024-09-10T21:46:48.643062+0000","last_undegraded":"2024-09-10T21:46:48.643062+0000","last_fullsized":"2024-09-10T21:46:48.643062+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:26:24.371844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"27'5330","reported_seq":5358,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.742778+0000","last_change":"2024-09-10T21:24:51.904449+0000","last_active":"2024-09-10T21:46:48.742778+0000","last_peered":"2024-09-10T21:46:48.742778+0000","last_clean":"2024-09-10T21:46:48.742778+0000","last_became_active":"2024-09-10T21:24:51.903533+0000","last_became_peered":"2024-09-10T21:24:51.903533+0000","last_unstale":"2024-09-10T21:46:48.742778+0000","last_undegraded":"2024-09-10T21:46:48.742778+0000","last_fullsized":"2024-09-10T21:46:48.742778+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":5330,"log_dups_size":0,"ondisk_log_size":5330,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:11.654935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26446,"num_write_kb":547906,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"27'14","reported_seq":47,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.347665+0000","last_change":"2024-09-10T21:24:47.857736+0000","last_active":"2024-09-10T21:47:11.347665+0000","last_peered":"2024-09-10T21:47:11.347665+0000","last_clean":"2024-09-10T21:47:11.347665+0000","last_became_active":"2024-09-10T21:24:47.857437+0000","last_became_peered":"2024-09-10T21:24:47.857437+0000","last_unstale":"2024-09-10T21:47:11.347665+0000","last_undegraded":"2024-09-10T21:47:11.347665+0000","last_fullsized":"2024-09-10T21:47:11.347665+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:13:40.032808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"27'5","reported_seq":40,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.633944+0000","last_change":"2024-09-10T21:24:46.860662+0000","last_active":"2024-09-10T21:46:48.633944+0000","last_peered":"2024-09-10T21:46:48.633944+0000","last_clean":"2024-09-10T21:46:48.633944+0000","last_became_active":"2024-09-10T21:24:46.859796+0000","last_became_peered":"2024-09-10T21:24:46.859796+0000","last_unstale":"2024-09-10T21:46:48.633944+0000","last_undegraded":"2024-09-10T21:46:48.633944+0000","last_fullsized":"2024-09-10T21:46:48.633944+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:48:27.758481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"27'5017","reported_seq":5045,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.677969+0000","last_change":"2024-09-10T21:24:52.069770+0000","last_active":"2024-09-10T21:46:48.677969+0000","last_peered":"2024-09-10T21:46:48.677969+0000","last_clean":"2024-09-10T21:46:48.677969+0000","last_became_active":"2024-09-10T21:24:52.067409+0000","last_became_peered":"2024-09-10T21:24:52.067409+0000","last_unstale":"2024-09-10T21:46:48.677969+0000","last_undegraded":"2024-09-10T21:46:48.677969+0000","last_fullsized":"2024-09-10T21:46:48.677969+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":5017,"log_dups_size":0,"ondisk_log_size":5017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:23:39.622612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24522,"num_write_kb":548501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"27'30","reported_seq":64,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.343245+0000","last_change":"2024-09-10T21:24:47.858692+0000","last_active":"2024-09-10T21:47:11.343245+0000","last_peered":"2024-09-10T21:47:11.343245+0000","last_clean":"2024-09-10T21:47:11.343245+0000","last_became_active":"2024-09-10T21:24:47.858343+0000","last_became_peered":"2024-09-10T21:24:47.858343+0000","last_unstale":"2024-09-10T21:47:11.343245+0000","last_undegraded":"2024-09-10T21:47:11.343245+0000","last_fullsized":"2024-09-10T21:47:11.343245+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:04:36.661791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"27'3","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.555096+0000","last_change":"2024-09-10T21:24:46.864592+0000","last_active":"2024-09-10T21:46:48.555096+0000","last_peered":"2024-09-10T21:46:48.555096+0000","last_clean":"2024-09-10T21:46:48.555096+0000","last_became_active":"2024-09-10T21:24:46.858696+0000","last_became_peered":"2024-09-10T21:24:46.858696+0000","last_unstale":"2024-09-10T21:46:48.555096+0000","last_undegraded":"2024-09-10T21:46:48.555096+0000","last_fullsized":"2024-09-10T21:46:48.555096+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:54:19.356867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"27'4314","reported_seq":4350,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.681791+0000","last_change":"2024-09-10T21:25:28.802910+0000","last_active":"2024-09-10T21:46:48.681791+0000","last_peered":"2024-09-10T21:46:48.681791+0000","last_clean":"2024-09-10T21:46:48.681791+0000","last_became_active":"2024-09-10T21:25:28.802224+0000","last_became_peered":"2024-09-10T21:25:28.802224+0000","last_unstale":"2024-09-10T21:46:48.681791+0000","last_undegraded":"2024-09-10T21:46:48.681791+0000","last_fullsized":"2024-09-10T21:46:48.681791+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":4314,"log_dups_size":0,"ondisk_log_size":4314,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:27:09.948701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20768,"num_write_kb":476051,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7,2],"acting":[3,5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"27'23","reported_seq":57,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.332930+0000","last_change":"2024-09-10T21:24:47.857699+0000","last_active":"2024-09-10T21:47:11.332930+0000","last_peered":"2024-09-10T21:47:11.332930+0000","last_clean":"2024-09-10T21:47:11.332930+0000","last_became_active":"2024-09-10T21:24:47.857429+0000","last_became_peered":"2024-09-10T21:24:47.857429+0000","last_unstale":"2024-09-10T21:47:11.332930+0000","last_undegraded":"2024-09-10T21:47:11.332930+0000","last_fullsized":"2024-09-10T21:47:11.332930+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:07:11.880228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"27'6","reported_seq":42,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.634621+0000","last_change":"2024-09-10T21:24:46.859753+0000","last_active":"2024-09-10T21:46:48.634621+0000","last_peered":"2024-09-10T21:46:48.634621+0000","last_clean":"2024-09-10T21:46:48.634621+0000","last_became_active":"2024-09-10T21:24:46.859299+0000","last_became_peered":"2024-09-10T21:24:46.859299+0000","last_unstale":"2024-09-10T21:46:48.634621+0000","last_undegraded":"2024-09-10T21:46:48.634621+0000","last_fullsized":"2024-09-10T21:46:48.634621+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:06:41.965052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"27'4196","reported_seq":4224,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.388436+0000","last_change":"2024-09-10T21:24:52.064718+0000","last_active":"2024-09-10T21:46:48.388436+0000","last_peered":"2024-09-10T21:46:48.388436+0000","last_clean":"2024-09-10T21:46:48.388436+0000","last_became_active":"2024-09-10T21:24:52.062327+0000","last_became_peered":"2024-09-10T21:24:52.062327+0000","last_unstale":"2024-09-10T21:46:48.388436+0000","last_undegraded":"2024-09-10T21:46:48.388436+0000","last_fullsized":"2024-09-10T21:46:48.388436+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":4196,"log_dups_size":0,"ondisk_log_size":4196,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:43:17.173095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20531,"num_write_kb":446498,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"27'18","reported_seq":52,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.334694+0000","last_change":"2024-09-10T21:24:47.870766+0000","last_active":"2024-09-10T21:47:11.334694+0000","last_peered":"2024-09-10T21:47:11.334694+0000","last_clean":"2024-09-10T21:47:11.334694+0000","last_became_active":"2024-09-10T21:24:47.870276+0000","last_became_peered":"2024-09-10T21:24:47.870276+0000","last_unstale":"2024-09-10T21:47:11.334694+0000","last_undegraded":"2024-09-10T21:47:11.334694+0000","last_fullsized":"2024-09-10T21:47:11.334694+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:54:38.400877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"27'368","reported_seq":403,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:28.433029+0000","last_change":"2024-09-10T21:24:46.867790+0000","last_active":"2024-09-10T21:46:28.433029+0000","last_peered":"2024-09-10T21:46:28.433029+0000","last_clean":"2024-09-10T21:46:28.433029+0000","last_became_active":"2024-09-10T21:24:46.866899+0000","last_became_peered":"2024-09-10T21:24:46.866899+0000","last_unstale":"2024-09-10T21:46:28.433029+0000","last_undegraded":"2024-09-10T21:46:28.433029+0000","last_fullsized":"2024-09-10T21:46:28.433029+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":368,"log_dups_size":0,"ondisk_log_size":368,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:25:04.923730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194976,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":367,"num_write_kb":4285,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"27'3971","reported_seq":3999,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.743646+0000","last_change":"2024-09-10T21:24:51.899803+0000","last_active":"2024-09-10T21:46:48.743646+0000","last_peered":"2024-09-10T21:46:48.743646+0000","last_clean":"2024-09-10T21:46:48.743646+0000","last_became_active":"2024-09-10T21:24:51.895398+0000","last_became_peered":"2024-09-10T21:24:51.895398+0000","last_unstale":"2024-09-10T21:46:48.743646+0000","last_undegraded":"2024-09-10T21:46:48.743646+0000","last_fullsized":"2024-09-10T21:46:48.743646+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":3971,"log_dups_size":0,"ondisk_log_size":3971,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:52:34.689860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19337,"num_write_kb":445978,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"27'22","reported_seq":56,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.334366+0000","last_change":"2024-09-10T21:24:47.857544+0000","last_active":"2024-09-10T21:47:11.334366+0000","last_peered":"2024-09-10T21:47:11.334366+0000","last_clean":"2024-09-10T21:47:11.334366+0000","last_became_active":"2024-09-10T21:24:47.857284+0000","last_became_peered":"2024-09-10T21:24:47.857284+0000","last_unstale":"2024-09-10T21:47:11.334366+0000","last_undegraded":"2024-09-10T21:47:11.334366+0000","last_fullsized":"2024-09-10T21:47:11.334366+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:10:22.747606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"27'32","reported_seq":67,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.672745+0000","last_change":"2024-09-10T21:24:46.859633+0000","last_active":"2024-09-10T21:46:48.672745+0000","last_peered":"2024-09-10T21:46:48.672745+0000","last_clean":"2024-09-10T21:46:48.672745+0000","last_became_active":"2024-09-10T21:24:46.859332+0000","last_became_peered":"2024-09-10T21:24:46.859332+0000","last_unstale":"2024-09-10T21:46:48.672745+0000","last_undegraded":"2024-09-10T21:46:48.672745+0000","last_fullsized":"2024-09-10T21:46:48.672745+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:56:46.975225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":33,"num_write_kb":30,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"27'4136","reported_seq":4164,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.629023+0000","last_change":"2024-09-10T21:24:51.899100+0000","last_active":"2024-09-10T21:46:48.629023+0000","last_peered":"2024-09-10T21:46:48.629023+0000","last_clean":"2024-09-10T21:46:48.629023+0000","last_became_active":"2024-09-10T21:24:51.898642+0000","last_became_peered":"2024-09-10T21:24:51.898642+0000","last_unstale":"2024-09-10T21:46:48.629023+0000","last_undegraded":"2024-09-10T21:46:48.629023+0000","last_fullsized":"2024-09-10T21:46:48.629023+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":4136,"log_dups_size":0,"ondisk_log_size":4136,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:06.394081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20561,"num_write_kb":446825,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"27'23","reported_seq":57,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.345555+0000","last_change":"2024-09-10T21:24:47.867890+0000","last_active":"2024-09-10T21:47:11.345555+0000","last_peered":"2024-09-10T21:47:11.345555+0000","last_clean":"2024-09-10T21:47:11.345555+0000","last_became_active":"2024-09-10T21:24:47.865016+0000","last_became_peered":"2024-09-10T21:24:47.865016+0000","last_unstale":"2024-09-10T21:47:11.345555+0000","last_undegraded":"2024-09-10T21:47:11.345555+0000","last_fullsized":"2024-09-10T21:47:11.345555+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:53:09.082512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"27'3","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.554054+0000","last_change":"2024-09-10T21:24:46.861400+0000","last_active":"2024-09-10T21:46:48.554054+0000","last_peered":"2024-09-10T21:46:48.554054+0000","last_clean":"2024-09-10T21:46:48.554054+0000","last_became_active":"2024-09-10T21:24:46.861136+0000","last_became_peered":"2024-09-10T21:24:46.861136+0000","last_unstale":"2024-09-10T21:46:48.554054+0000","last_undegraded":"2024-09-10T21:46:48.554054+0000","last_fullsized":"2024-09-10T21:46:48.554054+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:35:31.682448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"27'4156","reported_seq":4184,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.591271+0000","last_change":"2024-09-10T21:24:51.907850+0000","last_active":"2024-09-10T21:46:48.591271+0000","last_peered":"2024-09-10T21:46:48.591271+0000","last_clean":"2024-09-10T21:46:48.591271+0000","last_became_active":"2024-09-10T21:24:51.907062+0000","last_became_peered":"2024-09-10T21:24:51.907062+0000","last_unstale":"2024-09-10T21:46:48.591271+0000","last_undegraded":"2024-09-10T21:46:48.591271+0000","last_fullsized":"2024-09-10T21:46:48.591271+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":4156,"log_dups_size":0,"ondisk_log_size":4156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:32:52.519627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20153,"num_write_kb":457162,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"27'25","reported_seq":58,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.346624+0000","last_change":"2024-09-10T21:24:47.861800+0000","last_active":"2024-09-10T21:47:11.346624+0000","last_peered":"2024-09-10T21:47:11.346624+0000","last_clean":"2024-09-10T21:47:11.346624+0000","last_became_active":"2024-09-10T21:24:47.861498+0000","last_became_peered":"2024-09-10T21:24:47.861498+0000","last_unstale":"2024-09-10T21:47:11.346624+0000","last_undegraded":"2024-09-10T21:47:11.346624+0000","last_fullsized":"2024-09-10T21:47:11.346624+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:07:21.128033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"27'3","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:36.356663+0000","last_change":"2024-09-10T21:24:46.871677+0000","last_active":"2024-09-10T21:46:36.356663+0000","last_peered":"2024-09-10T21:46:36.356663+0000","last_clean":"2024-09-10T21:46:36.356663+0000","last_became_active":"2024-09-10T21:24:46.871367+0000","last_became_peered":"2024-09-10T21:24:46.871367+0000","last_unstale":"2024-09-10T21:46:36.356663+0000","last_undegraded":"2024-09-10T21:46:36.356663+0000","last_fullsized":"2024-09-10T21:46:36.356663+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:40:33.632846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"27'3798","reported_seq":3826,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.674934+0000","last_change":"2024-09-10T21:24:52.068637+0000","last_active":"2024-09-10T21:46:48.674934+0000","last_peered":"2024-09-10T21:46:48.674934+0000","last_clean":"2024-09-10T21:46:48.674934+0000","last_became_active":"2024-09-10T21:24:52.068348+0000","last_became_peered":"2024-09-10T21:24:52.068348+0000","last_unstale":"2024-09-10T21:46:48.674934+0000","last_undegraded":"2024-09-10T21:46:48.674934+0000","last_fullsized":"2024-09-10T21:46:48.674934+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":3798,"log_dups_size":0,"ondisk_log_size":3798,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:43:17.173095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18329,"num_write_kb":408933,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"27'5643","reported_seq":5673,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.715900+0000","last_change":"2024-09-10T21:24:52.067732+0000","last_active":"2024-09-10T21:46:48.715900+0000","last_peered":"2024-09-10T21:46:48.715900+0000","last_clean":"2024-09-10T21:46:48.715900+0000","last_became_active":"2024-09-10T21:24:52.067330+0000","last_became_peered":"2024-09-10T21:24:52.067330+0000","last_unstale":"2024-09-10T21:46:48.715900+0000","last_undegraded":"2024-09-10T21:46:48.715900+0000","last_fullsized":"2024-09-10T21:46:48.715900+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":5643,"log_dups_size":0,"ondisk_log_size":5643,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:03:41.110214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27908,"num_write_kb":566060,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"27'20","reported_seq":53,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.330166+0000","last_change":"2024-09-10T21:24:47.862774+0000","last_active":"2024-09-10T21:47:11.330166+0000","last_peered":"2024-09-10T21:47:11.330166+0000","last_clean":"2024-09-10T21:47:11.330166+0000","last_became_active":"2024-09-10T21:24:47.862556+0000","last_became_peered":"2024-09-10T21:24:47.862556+0000","last_unstale":"2024-09-10T21:47:11.330166+0000","last_undegraded":"2024-09-10T21:47:11.330166+0000","last_fullsized":"2024-09-10T21:47:11.330166+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:17:52.343553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"27'2","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:34.118680+0000","last_change":"2024-09-10T21:24:46.862847+0000","last_active":"2024-09-10T21:46:34.118680+0000","last_peered":"2024-09-10T21:46:34.118680+0000","last_clean":"2024-09-10T21:46:34.118680+0000","last_became_active":"2024-09-10T21:24:46.862540+0000","last_became_peered":"2024-09-10T21:24:46.862540+0000","last_unstale":"2024-09-10T21:46:34.118680+0000","last_undegraded":"2024-09-10T21:46:34.118680+0000","last_fullsized":"2024-09-10T21:46:34.118680+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:52:31.628568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"27'5506","reported_seq":5535,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.736687+0000","last_change":"2024-09-10T21:24:51.901224+0000","last_active":"2024-09-10T21:46:48.736687+0000","last_peered":"2024-09-10T21:46:48.736687+0000","last_clean":"2024-09-10T21:46:48.736687+0000","last_became_active":"2024-09-10T21:24:51.900610+0000","last_became_peered":"2024-09-10T21:24:51.900610+0000","last_unstale":"2024-09-10T21:46:48.736687+0000","last_undegraded":"2024-09-10T21:46:48.736687+0000","last_fullsized":"2024-09-10T21:46:48.736687+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":5506,"log_dups_size":0,"ondisk_log_size":5506,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:52:34.689860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27800,"num_write_kb":594222,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"27'26","reported_seq":60,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.328339+0000","last_change":"2024-09-10T21:24:47.861255+0000","last_active":"2024-09-10T21:47:11.328339+0000","last_peered":"2024-09-10T21:47:11.328339+0000","last_clean":"2024-09-10T21:47:11.328339+0000","last_became_active":"2024-09-10T21:24:47.860930+0000","last_became_peered":"2024-09-10T21:24:47.860930+0000","last_unstale":"2024-09-10T21:47:11.328339+0000","last_undegraded":"2024-09-10T21:47:11.328339+0000","last_fullsized":"2024-09-10T21:47:11.328339+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:37:51.784258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"27'3","reported_seq":39,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.632694+0000","last_change":"2024-09-10T21:24:46.858561+0000","last_active":"2024-09-10T21:46:48.632694+0000","last_peered":"2024-09-10T21:46:48.632694+0000","last_clean":"2024-09-10T21:46:48.632694+0000","last_became_active":"2024-09-10T21:24:46.858265+0000","last_became_peered":"2024-09-10T21:24:46.858265+0000","last_unstale":"2024-09-10T21:46:48.632694+0000","last_undegraded":"2024-09-10T21:46:48.632694+0000","last_fullsized":"2024-09-10T21:46:48.632694+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:49:00.454609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"27'6","reported_seq":41,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.657882+0000","last_change":"2024-09-10T21:24:46.856886+0000","last_active":"2024-09-10T21:46:48.657882+0000","last_peered":"2024-09-10T21:46:48.657882+0000","last_clean":"2024-09-10T21:46:48.657882+0000","last_became_active":"2024-09-10T21:24:46.856506+0000","last_became_peered":"2024-09-10T21:24:46.856506+0000","last_unstale":"2024-09-10T21:46:48.657882+0000","last_undegraded":"2024-09-10T21:46:48.657882+0000","last_fullsized":"2024-09-10T21:46:48.657882+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:56:46.975225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"27'19","reported_seq":52,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.337444+0000","last_change":"2024-09-10T21:24:47.848602+0000","last_active":"2024-09-10T21:47:11.337444+0000","last_peered":"2024-09-10T21:47:11.337444+0000","last_clean":"2024-09-10T21:47:11.337444+0000","last_became_active":"2024-09-10T21:24:47.848160+0000","last_became_peered":"2024-09-10T21:24:47.848160+0000","last_unstale":"2024-09-10T21:47:11.337444+0000","last_undegraded":"2024-09-10T21:47:11.337444+0000","last_fullsized":"2024-09-10T21:47:11.337444+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:32:50.027827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"27'4174","reported_seq":4201,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.382991+0000","last_change":"2024-09-10T21:24:51.903553+0000","last_active":"2024-09-10T21:46:48.382991+0000","last_peered":"2024-09-10T21:46:48.382991+0000","last_clean":"2024-09-10T21:46:48.382991+0000","last_became_active":"2024-09-10T21:24:51.903120+0000","last_became_peered":"2024-09-10T21:24:51.903120+0000","last_unstale":"2024-09-10T21:46:48.382991+0000","last_undegraded":"2024-09-10T21:46:48.382991+0000","last_fullsized":"2024-09-10T21:46:48.382991+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":4174,"log_dups_size":0,"ondisk_log_size":4174,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:58.240380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19938,"num_write_kb":435851,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"27'678","reported_seq":713,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.597988+0000","last_change":"2024-09-10T21:24:46.853550+0000","last_active":"2024-09-10T21:46:48.597988+0000","last_peered":"2024-09-10T21:46:48.597988+0000","last_clean":"2024-09-10T21:46:48.597988+0000","last_became_active":"2024-09-10T21:24:46.852204+0000","last_became_peered":"2024-09-10T21:24:46.852204+0000","last_unstale":"2024-09-10T21:46:48.597988+0000","last_undegraded":"2024-09-10T21:46:48.597988+0000","last_fullsized":"2024-09-10T21:46:48.597988+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":678,"log_dups_size":0,"ondisk_log_size":678,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:06:37.669444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":672,"num_write_kb":4259,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"27'18","reported_seq":51,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.327062+0000","last_change":"2024-09-10T21:24:47.849151+0000","last_active":"2024-09-10T21:47:11.327062+0000","last_peered":"2024-09-10T21:47:11.327062+0000","last_clean":"2024-09-10T21:47:11.327062+0000","last_became_active":"2024-09-10T21:24:47.848851+0000","last_became_peered":"2024-09-10T21:24:47.848851+0000","last_unstale":"2024-09-10T21:47:11.327062+0000","last_undegraded":"2024-09-10T21:47:11.327062+0000","last_fullsized":"2024-09-10T21:47:11.327062+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:39:25.828558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"27'4902","reported_seq":4929,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.681160+0000","last_change":"2024-09-10T21:24:51.907549+0000","last_active":"2024-09-10T21:46:48.681160+0000","last_peered":"2024-09-10T21:46:48.681160+0000","last_clean":"2024-09-10T21:46:48.681160+0000","last_became_active":"2024-09-10T21:24:51.907082+0000","last_became_peered":"2024-09-10T21:24:51.907082+0000","last_unstale":"2024-09-10T21:46:48.681160+0000","last_undegraded":"2024-09-10T21:46:48.681160+0000","last_fullsized":"2024-09-10T21:46:48.681160+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":4902,"log_dups_size":0,"ondisk_log_size":4902,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:44:30.813754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24324,"num_write_kb":547512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"27'24","reported_seq":58,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.351212+0000","last_change":"2024-09-10T21:24:47.857804+0000","last_active":"2024-09-10T21:47:11.351212+0000","last_peered":"2024-09-10T21:47:11.351212+0000","last_clean":"2024-09-10T21:47:11.351212+0000","last_became_active":"2024-09-10T21:24:47.857501+0000","last_became_peered":"2024-09-10T21:24:47.857501+0000","last_unstale":"2024-09-10T21:47:11.351212+0000","last_undegraded":"2024-09-10T21:47:11.351212+0000","last_fullsized":"2024-09-10T21:47:11.351212+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:19:20.807422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"27'25","reported_seq":60,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.345524+0000","last_change":"2024-09-10T21:24:47.867803+0000","last_active":"2024-09-10T21:47:11.345524+0000","last_peered":"2024-09-10T21:47:11.345524+0000","last_clean":"2024-09-10T21:47:11.345524+0000","last_became_active":"2024-09-10T21:24:47.867504+0000","last_became_peered":"2024-09-10T21:24:47.867504+0000","last_unstale":"2024-09-10T21:47:11.345524+0000","last_undegraded":"2024-09-10T21:47:11.345524+0000","last_fullsized":"2024-09-10T21:47:11.345524+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:53:30.420454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"27'26","reported_seq":60,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.334694+0000","last_change":"2024-09-10T21:24:47.868901+0000","last_active":"2024-09-10T21:47:11.334694+0000","last_peered":"2024-09-10T21:47:11.334694+0000","last_clean":"2024-09-10T21:47:11.334694+0000","last_became_active":"2024-09-10T21:24:47.868168+0000","last_became_peered":"2024-09-10T21:24:47.868168+0000","last_unstale":"2024-09-10T21:47:11.334694+0000","last_undegraded":"2024-09-10T21:47:11.334694+0000","last_fullsized":"2024-09-10T21:47:11.334694+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:06:21.461089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"27'16","reported_seq":51,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.331166+0000","last_change":"2024-09-10T21:24:47.864561+0000","last_active":"2024-09-10T21:47:11.331166+0000","last_peered":"2024-09-10T21:47:11.331166+0000","last_clean":"2024-09-10T21:47:11.331166+0000","last_became_active":"2024-09-10T21:24:47.864100+0000","last_became_peered":"2024-09-10T21:24:47.864100+0000","last_unstale":"2024-09-10T21:47:11.331166+0000","last_undegraded":"2024-09-10T21:47:11.331166+0000","last_fullsized":"2024-09-10T21:47:11.331166+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:34:13.053494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"27'14","reported_seq":49,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.340508+0000","last_change":"2024-09-10T21:24:47.864303+0000","last_active":"2024-09-10T21:47:11.340508+0000","last_peered":"2024-09-10T21:47:11.340508+0000","last_clean":"2024-09-10T21:47:11.340508+0000","last_became_active":"2024-09-10T21:24:47.863970+0000","last_became_peered":"2024-09-10T21:24:47.863970+0000","last_unstale":"2024-09-10T21:47:11.340508+0000","last_undegraded":"2024-09-10T21:47:11.340508+0000","last_fullsized":"2024-09-10T21:47:11.340508+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:46:36.918539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"27'18","reported_seq":52,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.329189+0000","last_change":"2024-09-10T21:24:47.860734+0000","last_active":"2024-09-10T21:47:11.329189+0000","last_peered":"2024-09-10T21:47:11.329189+0000","last_clean":"2024-09-10T21:47:11.329189+0000","last_became_active":"2024-09-10T21:24:47.860471+0000","last_became_peered":"2024-09-10T21:24:47.860471+0000","last_unstale":"2024-09-10T21:47:11.329189+0000","last_undegraded":"2024-09-10T21:47:11.329189+0000","last_fullsized":"2024-09-10T21:47:11.329189+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:54:13.200082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"27'27","reported_seq":61,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.358154+0000","last_change":"2024-09-10T21:24:47.872504+0000","last_active":"2024-09-10T21:47:11.358154+0000","last_peered":"2024-09-10T21:47:11.358154+0000","last_clean":"2024-09-10T21:47:11.358154+0000","last_became_active":"2024-09-10T21:24:47.872218+0000","last_became_peered":"2024-09-10T21:24:47.872218+0000","last_unstale":"2024-09-10T21:47:11.358154+0000","last_undegraded":"2024-09-10T21:47:11.358154+0000","last_fullsized":"2024-09-10T21:47:11.358154+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:21:32.400520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"27'14","reported_seq":49,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.310878+0000","last_change":"2024-09-10T21:24:47.862167+0000","last_active":"2024-09-10T21:47:11.310878+0000","last_peered":"2024-09-10T21:47:11.310878+0000","last_clean":"2024-09-10T21:47:11.310878+0000","last_became_active":"2024-09-10T21:24:47.859649+0000","last_became_peered":"2024-09-10T21:24:47.859649+0000","last_unstale":"2024-09-10T21:47:11.310878+0000","last_undegraded":"2024-09-10T21:47:11.310878+0000","last_fullsized":"2024-09-10T21:47:11.310878+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:42:59.204284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"27'18","reported_seq":39,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.329156+0000","last_change":"2024-09-10T21:25:28.508979+0000","last_active":"2024-09-10T21:47:11.329156+0000","last_peered":"2024-09-10T21:47:11.329156+0000","last_clean":"2024-09-10T21:47:11.329156+0000","last_became_active":"2024-09-10T21:25:28.508684+0000","last_became_peered":"2024-09-10T21:25:28.508684+0000","last_unstale":"2024-09-10T21:47:11.329156+0000","last_undegraded":"2024-09-10T21:47:11.329156+0000","last_fullsized":"2024-09-10T21:47:11.329156+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:20:59.362941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.29","version":"27'23","reported_seq":43,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.335919+0000","last_change":"2024-09-10T21:25:28.512634+0000","last_active":"2024-09-10T21:47:11.335919+0000","last_peered":"2024-09-10T21:47:11.335919+0000","last_clean":"2024-09-10T21:47:11.335919+0000","last_became_active":"2024-09-10T21:25:28.512230+0000","last_became_peered":"2024-09-10T21:25:28.512230+0000","last_unstale":"2024-09-10T21:47:11.335919+0000","last_undegraded":"2024-09-10T21:47:11.335919+0000","last_fullsized":"2024-09-10T21:47:11.335919+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:17:52.343553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2a","version":"27'22","reported_seq":55,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.345833+0000","last_change":"2024-09-10T21:24:47.872848+0000","last_active":"2024-09-10T21:47:11.345833+0000","last_peered":"2024-09-10T21:47:11.345833+0000","last_clean":"2024-09-10T21:47:11.345833+0000","last_became_active":"2024-09-10T21:24:47.868671+0000","last_became_peered":"2024-09-10T21:24:47.868671+0000","last_unstale":"2024-09-10T21:47:11.345833+0000","last_undegraded":"2024-09-10T21:47:11.345833+0000","last_fullsized":"2024-09-10T21:47:11.345833+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:33:59.291647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"27'28","reported_seq":62,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.339893+0000","last_change":"2024-09-10T21:24:47.857464+0000","last_active":"2024-09-10T21:47:11.339893+0000","last_peered":"2024-09-10T21:47:11.339893+0000","last_clean":"2024-09-10T21:47:11.339893+0000","last_became_active":"2024-09-10T21:24:47.857116+0000","last_became_peered":"2024-09-10T21:24:47.857116+0000","last_unstale":"2024-09-10T21:47:11.339893+0000","last_undegraded":"2024-09-10T21:47:11.339893+0000","last_fullsized":"2024-09-10T21:47:11.339893+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:56:43.269026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"27'19","reported_seq":52,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.344737+0000","last_change":"2024-09-10T21:24:47.871619+0000","last_active":"2024-09-10T21:47:11.344737+0000","last_peered":"2024-09-10T21:47:11.344737+0000","last_clean":"2024-09-10T21:47:11.344737+0000","last_became_active":"2024-09-10T21:24:47.871350+0000","last_became_peered":"2024-09-10T21:24:47.871350+0000","last_unstale":"2024-09-10T21:47:11.344737+0000","last_undegraded":"2024-09-10T21:47:11.344737+0000","last_fullsized":"2024-09-10T21:47:11.344737+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:41.305116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"27'20","reported_seq":54,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.332528+0000","last_change":"2024-09-10T21:24:47.861352+0000","last_active":"2024-09-10T21:47:11.332528+0000","last_peered":"2024-09-10T21:47:11.332528+0000","last_clean":"2024-09-10T21:47:11.332528+0000","last_became_active":"2024-09-10T21:24:47.861007+0000","last_became_peered":"2024-09-10T21:24:47.861007+0000","last_unstale":"2024-09-10T21:47:11.332528+0000","last_undegraded":"2024-09-10T21:47:11.332528+0000","last_fullsized":"2024-09-10T21:47:11.332528+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:25.273458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"27'27","reported_seq":61,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.345633+0000","last_change":"2024-09-10T21:24:47.864205+0000","last_active":"2024-09-10T21:47:11.345633+0000","last_peered":"2024-09-10T21:47:11.345633+0000","last_clean":"2024-09-10T21:47:11.345633+0000","last_became_active":"2024-09-10T21:24:47.863932+0000","last_became_peered":"2024-09-10T21:24:47.863932+0000","last_unstale":"2024-09-10T21:47:11.345633+0000","last_undegraded":"2024-09-10T21:47:11.345633+0000","last_fullsized":"2024-09-10T21:47:11.345633+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:53:17.669629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"27'21","reported_seq":54,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.349081+0000","last_change":"2024-09-10T21:24:47.866225+0000","last_active":"2024-09-10T21:47:11.349081+0000","last_peered":"2024-09-10T21:47:11.349081+0000","last_clean":"2024-09-10T21:47:11.349081+0000","last_became_active":"2024-09-10T21:24:47.865587+0000","last_became_peered":"2024-09-10T21:24:47.865587+0000","last_unstale":"2024-09-10T21:47:11.349081+0000","last_undegraded":"2024-09-10T21:47:11.349081+0000","last_fullsized":"2024-09-10T21:47:11.349081+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:49:26.113068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"27'22","reported_seq":55,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.342691+0000","last_change":"2024-09-10T21:24:47.872669+0000","last_active":"2024-09-10T21:47:11.342691+0000","last_peered":"2024-09-10T21:47:11.342691+0000","last_clean":"2024-09-10T21:47:11.342691+0000","last_became_active":"2024-09-10T21:24:47.865837+0000","last_became_peered":"2024-09-10T21:24:47.865837+0000","last_unstale":"2024-09-10T21:47:11.342691+0000","last_undegraded":"2024-09-10T21:47:11.342691+0000","last_fullsized":"2024-09-10T21:47:11.342691+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:53:09.082512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"27'21","reported_seq":54,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.318871+0000","last_change":"2024-09-10T21:24:47.846567+0000","last_active":"2024-09-10T21:47:11.318871+0000","last_peered":"2024-09-10T21:47:11.318871+0000","last_clean":"2024-09-10T21:47:11.318871+0000","last_became_active":"2024-09-10T21:24:47.846283+0000","last_became_peered":"2024-09-10T21:24:47.846283+0000","last_unstale":"2024-09-10T21:47:11.318871+0000","last_undegraded":"2024-09-10T21:47:11.318871+0000","last_fullsized":"2024-09-10T21:47:11.318871+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:42:15.320518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"27'23","reported_seq":57,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.319292+0000","last_change":"2024-09-10T21:24:47.844888+0000","last_active":"2024-09-10T21:47:11.319292+0000","last_peered":"2024-09-10T21:47:11.319292+0000","last_clean":"2024-09-10T21:47:11.319292+0000","last_became_active":"2024-09-10T21:24:47.844517+0000","last_became_peered":"2024-09-10T21:24:47.844517+0000","last_unstale":"2024-09-10T21:47:11.319292+0000","last_undegraded":"2024-09-10T21:47:11.319292+0000","last_fullsized":"2024-09-10T21:47:11.319292+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:00:14.988282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"27'25","reported_seq":59,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.344844+0000","last_change":"2024-09-10T21:24:47.862746+0000","last_active":"2024-09-10T21:47:11.344844+0000","last_peered":"2024-09-10T21:47:11.344844+0000","last_clean":"2024-09-10T21:47:11.344844+0000","last_became_active":"2024-09-10T21:24:47.862185+0000","last_became_peered":"2024-09-10T21:24:47.862185+0000","last_unstale":"2024-09-10T21:47:11.344844+0000","last_undegraded":"2024-09-10T21:47:11.344844+0000","last_fullsized":"2024-09-10T21:47:11.344844+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:06:21.461089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"27'17","reported_seq":51,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.349979+0000","last_change":"2024-09-10T21:24:47.867727+0000","last_active":"2024-09-10T21:47:11.349979+0000","last_peered":"2024-09-10T21:47:11.349979+0000","last_clean":"2024-09-10T21:47:11.349979+0000","last_became_active":"2024-09-10T21:24:47.861045+0000","last_became_peered":"2024-09-10T21:24:47.861045+0000","last_unstale":"2024-09-10T21:47:11.349979+0000","last_undegraded":"2024-09-10T21:47:11.349979+0000","last_fullsized":"2024-09-10T21:47:11.349979+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:08:30.065968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"27'17","reported_seq":50,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.347800+0000","last_change":"2024-09-10T21:24:47.867410+0000","last_active":"2024-09-10T21:47:11.347800+0000","last_peered":"2024-09-10T21:47:11.347800+0000","last_clean":"2024-09-10T21:47:11.347800+0000","last_became_active":"2024-09-10T21:24:47.867113+0000","last_became_peered":"2024-09-10T21:24:47.867113+0000","last_unstale":"2024-09-10T21:47:11.347800+0000","last_undegraded":"2024-09-10T21:47:11.347800+0000","last_fullsized":"2024-09-10T21:47:11.347800+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:49:53.502472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"27'28","reported_seq":61,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.358458+0000","last_change":"2024-09-10T21:24:47.867473+0000","last_active":"2024-09-10T21:47:11.358458+0000","last_peered":"2024-09-10T21:47:11.358458+0000","last_clean":"2024-09-10T21:47:11.358458+0000","last_became_active":"2024-09-10T21:24:47.867126+0000","last_became_peered":"2024-09-10T21:24:47.867126+0000","last_unstale":"2024-09-10T21:47:11.358458+0000","last_undegraded":"2024-09-10T21:47:11.358458+0000","last_fullsized":"2024-09-10T21:47:11.358458+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:54:32.307507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"27'26","reported_seq":60,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.353536+0000","last_change":"2024-09-10T21:24:47.865450+0000","last_active":"2024-09-10T21:47:11.353536+0000","last_peered":"2024-09-10T21:47:11.353536+0000","last_clean":"2024-09-10T21:47:11.353536+0000","last_became_active":"2024-09-10T21:24:47.865093+0000","last_became_peered":"2024-09-10T21:24:47.865093+0000","last_unstale":"2024-09-10T21:47:11.353536+0000","last_undegraded":"2024-09-10T21:47:11.353536+0000","last_fullsized":"2024-09-10T21:47:11.353536+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:12:18.854783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"27'15","reported_seq":49,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.344819+0000","last_change":"2024-09-10T21:24:47.868093+0000","last_active":"2024-09-10T21:47:11.344819+0000","last_peered":"2024-09-10T21:47:11.344819+0000","last_clean":"2024-09-10T21:47:11.344819+0000","last_became_active":"2024-09-10T21:24:47.867782+0000","last_became_peered":"2024-09-10T21:24:47.867782+0000","last_unstale":"2024-09-10T21:47:11.344819+0000","last_undegraded":"2024-09-10T21:47:11.344819+0000","last_fullsized":"2024-09-10T21:47:11.344819+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:34:13.053494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"27'13","reported_seq":46,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.335909+0000","last_change":"2024-09-10T21:24:47.862204+0000","last_active":"2024-09-10T21:47:11.335909+0000","last_peered":"2024-09-10T21:47:11.335909+0000","last_clean":"2024-09-10T21:47:11.335909+0000","last_became_active":"2024-09-10T21:24:47.861434+0000","last_became_peered":"2024-09-10T21:24:47.861434+0000","last_unstale":"2024-09-10T21:47:11.335909+0000","last_undegraded":"2024-09-10T21:47:11.335909+0000","last_fullsized":"2024-09-10T21:47:11.335909+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:50:03.245285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"27'14","reported_seq":46,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.343049+0000","last_change":"2024-09-10T21:24:47.869248+0000","last_active":"2024-09-10T21:47:11.343049+0000","last_peered":"2024-09-10T21:47:11.343049+0000","last_clean":"2024-09-10T21:47:11.343049+0000","last_became_active":"2024-09-10T21:24:47.868503+0000","last_became_peered":"2024-09-10T21:24:47.868503+0000","last_unstale":"2024-09-10T21:47:11.343049+0000","last_undegraded":"2024-09-10T21:47:11.343049+0000","last_fullsized":"2024-09-10T21:47:11.343049+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:51:31.417845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"27'15","reported_seq":48,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.331365+0000","last_change":"2024-09-10T21:24:47.864802+0000","last_active":"2024-09-10T21:47:11.331365+0000","last_peered":"2024-09-10T21:47:11.331365+0000","last_clean":"2024-09-10T21:47:11.331365+0000","last_became_active":"2024-09-10T21:24:47.863976+0000","last_became_peered":"2024-09-10T21:24:47.863976+0000","last_unstale":"2024-09-10T21:47:11.331365+0000","last_undegraded":"2024-09-10T21:47:11.331365+0000","last_fullsized":"2024-09-10T21:47:11.331365+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:55:08.544075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"27'13","reported_seq":48,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.347483+0000","last_change":"2024-09-10T21:24:47.873364+0000","last_active":"2024-09-10T21:47:11.347483+0000","last_peered":"2024-09-10T21:47:11.347483+0000","last_clean":"2024-09-10T21:47:11.347483+0000","last_became_active":"2024-09-10T21:24:47.873069+0000","last_became_peered":"2024-09-10T21:24:47.873069+0000","last_unstale":"2024-09-10T21:47:11.347483+0000","last_undegraded":"2024-09-10T21:47:11.347483+0000","last_fullsized":"2024-09-10T21:47:11.347483+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:38:21.368499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"27'20","reported_seq":55,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.322849+0000","last_change":"2024-09-10T21:24:47.856936+0000","last_active":"2024-09-10T21:47:11.322849+0000","last_peered":"2024-09-10T21:47:11.322849+0000","last_clean":"2024-09-10T21:47:11.322849+0000","last_became_active":"2024-09-10T21:24:47.854713+0000","last_became_peered":"2024-09-10T21:24:47.854713+0000","last_unstale":"2024-09-10T21:47:11.322849+0000","last_undegraded":"2024-09-10T21:47:11.322849+0000","last_fullsized":"2024-09-10T21:47:11.322849+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:46:36.918539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"27'10","reported_seq":44,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.320206+0000","last_change":"2024-09-10T21:24:47.846492+0000","last_active":"2024-09-10T21:47:11.320206+0000","last_peered":"2024-09-10T21:47:11.320206+0000","last_clean":"2024-09-10T21:47:11.320206+0000","last_became_active":"2024-09-10T21:24:47.846227+0000","last_became_peered":"2024-09-10T21:24:47.846227+0000","last_unstale":"2024-09-10T21:47:11.320206+0000","last_undegraded":"2024-09-10T21:47:11.320206+0000","last_fullsized":"2024-09-10T21:47:11.320206+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:13:58.432819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"27'15","reported_seq":50,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.347364+0000","last_change":"2024-09-10T21:24:47.871469+0000","last_active":"2024-09-10T21:47:11.347364+0000","last_peered":"2024-09-10T21:47:11.347364+0000","last_clean":"2024-09-10T21:47:11.347364+0000","last_became_active":"2024-09-10T21:24:47.871177+0000","last_became_peered":"2024-09-10T21:24:47.871177+0000","last_unstale":"2024-09-10T21:47:11.347364+0000","last_undegraded":"2024-09-10T21:47:11.347364+0000","last_fullsized":"2024-09-10T21:47:11.347364+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:19:24.226916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":723657,"num_write_kb":15994026,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":148509,"ondisk_log_size":148509,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1239,"ondisk_log_size":1239,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":14991163,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":48,"num_read_kb":119,"num_write":3049,"num_write_kb":15770,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":18464768,"data_stored":29982326,"data_compressed":2564296,"data_compressed_allocated":3506176,"data_compressed_original":15089664,"omap_allocated":0,"internal_metadata":0},"log_size":2758,"ondisk_log_size":2758,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":66932,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509714,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":739600,"kb_used_data":7424,"kb_used_omap":1,"kb_used_meta":732158,"kb_avail":93632240,"statfs":{"total":96636764160,"available":95879413760,"internally_reserved":0,"allocated":7602176,"data_stored":11412495,"data_compressed":806609,"data_compressed_allocated":1097728,"data_compressed_original":5083136,"omap_allocated":1590,"internal_metadata":749730250},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509715,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":807000,"kb_used_data":2120,"kb_used_omap":1,"kb_used_meta":804862,"kb_avail":93564840,"statfs":{"total":96636764160,"available":95810396160,"internally_reserved":0,"allocated":2170880,"data_stored":4311586,"data_compressed":503209,"data_compressed_allocated":692224,"data_compressed_original":3006464,"omap_allocated":1594,"internal_metadata":824179142},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509713,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":573484,"kb_used_data":348,"kb_used_omap":1,"kb_used_meta":573118,"kb_avail":93798356,"statfs":{"total":96636764160,"available":96049516544,"internally_reserved":0,"allocated":356352,"data_stored":207064,"data_compressed":5796,"data_compressed_allocated":8192,"data_compressed_original":28672,"omap_allocated":1595,"internal_metadata":586873285},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509712,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1104044,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":1103742,"kb_avail":93267796,"statfs":{"total":96636764160,"available":95506223104,"internally_reserved":0,"allocated":290816,"data_stored":116770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":1130232261},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542419,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":844396,"kb_used_data":3740,"kb_used_omap":1,"kb_used_meta":840638,"kb_avail":93527444,"statfs":{"total":96636764160,"available":95772102656,"internally_reserved":0,"allocated":3829760,"data_stored":4311040,"data_compressed":154551,"data_compressed_allocated":245760,"data_compressed_original":905216,"omap_allocated":1595,"internal_metadata":860813765},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542414,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1062920,"kb_used_data":4792,"kb_used_omap":1,"kb_used_meta":1058110,"kb_avail":93308920,"statfs":{"total":96636764160,"available":95548334080,"internally_reserved":0,"allocated":4907008,"data_stored":8596344,"data_compressed":841246,"data_compressed_allocated":1130496,"data_compressed_original":4997120,"omap_allocated":1593,"internal_metadata":1083505095},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542420,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":711220,"kb_used_data":1380,"kb_used_omap":1,"kb_used_meta":709822,"kb_avail":93660620,"statfs":{"total":96636764160,"available":95908474880,"internally_reserved":0,"allocated":1413120,"data_stored":3023943,"data_compressed":319817,"data_compressed_allocated":421888,"data_compressed_original":2215936,"omap_allocated":1595,"internal_metadata":726858181},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575122,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":626792,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":626494,"kb_avail":93745048,"statfs":{"total":96636764160,"available":95994929152,"internally_reserved":0,"allocated":286720,"data_stored":116700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":641530309},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33466,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33466,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":1064960,"data_stored":2316965,"data_compressed":286351,"data_compressed_allocated":376832,"data_compressed_original":1642496,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":90454,"data_compressed":5796,"data_compressed_allocated":8192,"data_compressed_original":28672,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4624384,"data_stored":8479734,"data_compressed":841246,"data_compressed_allocated":1130496,"data_compressed_original":4997120,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":3547136,"data_stored":4194430,"data_compressed":154551,"data_compressed_allocated":245760,"data_compressed_original":905216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":1888256,"data_stored":4194976,"data_compressed":503209,"data_compressed_allocated":692224,"data_compressed_original":3006464,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":7254016,"data_stored":10705517,"data_compressed":773143,"data_compressed_allocated":1052672,"data_compressed_original":4509696,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T21:47:18.847 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-10T21:47:18.848 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T21:47:19.033 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:19.029+0000 7efc7fa00640 1 -- 172.21.15.17:0/3460052491 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efc780cdf20 msgr2=0x7efc780ce350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:19.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:19.029+0000 7efc7fa00640 1 --2- 172.21.15.17:0/3460052491 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efc780cdf20 0x7efc780ce350 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7efc781594a0 tx=0x7efc7001cd10 comp rx=0 tx=0).stop 2024-09-10T21:47:19.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:19.029+0000 7efc7fa00640 1 -- 172.21.15.17:0/3460052491 shutdown_connections 2024-09-10T21:47:19.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:19.029+0000 7efc7fa00640 1 --2- 172.21.15.17:0/3460052491 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efc780cdf20 0x7efc780ce350 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:19.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:19.029+0000 7efc7fa00640 1 --2- 172.21.15.17:0/3460052491 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7efc78152440 0x7efc780b3eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:19.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:19.029+0000 7efc7fa00640 1 -- 172.21.15.17:0/3460052491 >> 172.21.15.17:0/3460052491 conn(0x7efc780702f0 msgr2=0x7efc78070700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:19.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:19.029+0000 7efc7fa00640 1 -- 172.21.15.17:0/3460052491 shutdown_connections 2024-09-10T21:47:19.034 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:19.029+0000 7efc7fa00640 1 -- 172.21.15.17:0/3460052491 wait complete. 2024-09-10T21:47:19.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:19.029+0000 7efc7fa00640 1 Processor -- start 2024-09-10T21:47:19.035 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:19.029+0000 7efc7fa00640 1 -- start start 2024-09-10T21:47:19.036 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:19.033+0000 7efc7fa00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7efc780cdf20 0x7efc7814be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:19.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:19.033+0000 7efc7fa00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efc78152440 0x7efc781f2d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:19.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:19.033+0000 7efc7fa00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efc78152820 0x7efc7815a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:19.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:19.033+0000 7efc7fa00640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7efc780cb5d0 con 0x7efc78152820 2024-09-10T21:47:19.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:19.033+0000 7efc7fa00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7efc780cb450 con 0x7efc78152440 2024-09-10T21:47:19.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:19.033+0000 7efc7fa00640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7efc780c2ff0 con 0x7efc780cdf20 2024-09-10T21:47:19.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:19.033+0000 7efc7da00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efc78152820 0x7efc7815a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:19.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:19.033+0000 7efc7da00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efc78152820 0x7efc7815a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:34672/0 (socket says 172.21.15.17:34672) 2024-09-10T21:47:19.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:19.033+0000 7efc7da00640 1 -- 172.21.15.17:0/7690006 learned_addr learned my addr 172.21.15.17:0/7690006 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:47:19.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:19.033+0000 7efc7d000640 1 --2- 172.21.15.17:0/7690006 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7efc780cdf20 0x7efc7814be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:19.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:19.033+0000 7efc77e00640 1 --2- 172.21.15.17:0/7690006 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efc78152440 0x7efc781f2d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:19.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:19.033+0000 7efc7da00640 1 -- 172.21.15.17:0/7690006 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7efc780cdf20 msgr2=0x7efc7814be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:19.037 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:19.033+0000 7efc7da00640 1 --2- 172.21.15.17:0/7690006 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7efc780cdf20 0x7efc7814be40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:19.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:19.033+0000 7efc7da00640 1 -- 172.21.15.17:0/7690006 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efc78152440 msgr2=0x7efc781f2d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:19.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:19.033+0000 7efc7da00640 1 --2- 172.21.15.17:0/7690006 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efc78152440 0x7efc781f2d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:19.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:19.033+0000 7efc7da00640 1 -- 172.21.15.17:0/7690006 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc780cdce0 con 0x7efc78152820 2024-09-10T21:47:19.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:19.033+0000 7efc7da00640 1 --2- 172.21.15.17:0/7690006 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efc78152820 0x7efc7815a6a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7efc70009bd0 tx=0x7efc70005b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:19.038 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:19.033+0000 7efc74c00640 1 -- 172.21.15.17:0/7690006 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc700bb070 con 0x7efc78152820 2024-09-10T21:47:19.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:19.033+0000 7efc74c00640 1 -- 172.21.15.17:0/7690006 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efc70004b40 con 0x7efc78152820 2024-09-10T21:47:19.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:19.033+0000 7efc74c00640 1 -- 172.21.15.17:0/7690006 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc700b2040 con 0x7efc78152820 2024-09-10T21:47:19.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:19.033+0000 7efc7fa00640 1 -- 172.21.15.17:0/7690006 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc78162710 con 0x7efc78152820 2024-09-10T21:47:19.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:19.033+0000 7efc7fa00640 1 -- 172.21.15.17:0/7690006 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc7814e770 con 0x7efc78152820 2024-09-10T21:47:19.039 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:19.033+0000 7efc7fa00640 1 -- 172.21.15.17:0/7690006 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc78158360 con 0x7efc78152820 2024-09-10T21:47:19.040 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:19.037+0000 7efc74c00640 1 -- 172.21.15.17:0/7690006 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7efc700b5030 con 0x7efc78152820 2024-09-10T21:47:19.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:19.037+0000 7efc74c00640 1 --2- 172.21.15.17:0/7690006 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7efc580725e0 0x7efc58074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:19.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:19.037+0000 7efc74c00640 1 -- 172.21.15.17:0/7690006 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7efc7012d290 con 0x7efc78152820 2024-09-10T21:47:19.041 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:19.037+0000 7efc7d000640 1 --2- 172.21.15.17:0/7690006 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7efc580725e0 0x7efc58074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:19.042 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:19.037+0000 7efc7d000640 1 --2- 172.21.15.17:0/7690006 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7efc580725e0 0x7efc58074aa0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7efc6c009be0 tx=0x7efc6c0091e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:19.044 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:19.041+0000 7efc74c00640 1 -- 172.21.15.17:0/7690006 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7efc700f2de0 con 0x7efc78152820 2024-09-10T21:47:19.221 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:19.217+0000 7efc7fa00640 1 -- 172.21.15.17:0/7690006 --> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7efc78162300 con 0x7efc580725e0 2024-09-10T21:47:19.227 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:19.221+0000 7efc74c00640 1 -- 172.21.15.17:0/7690006 <== mgr.4100 v2:172.21.15.31:6808/3101308586 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+334037 (secure 0 0 0) 0x7efc78162300 con 0x7efc580725e0 2024-09-10T21:47:19.231 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-10T21:47:19.233 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2024-09-10T21:47:19.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:19.225+0000 7efc7fa00640 1 -- 172.21.15.17:0/7690006 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7efc580725e0 msgr2=0x7efc58074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:19.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:19.225+0000 7efc7fa00640 1 --2- 172.21.15.17:0/7690006 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7efc580725e0 0x7efc58074aa0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7efc6c009be0 tx=0x7efc6c0091e0 comp rx=0 tx=0).stop 2024-09-10T21:47:19.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:19.225+0000 7efc7fa00640 1 -- 172.21.15.17:0/7690006 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efc78152820 msgr2=0x7efc7815a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:19.234 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:19.225+0000 7efc7fa00640 1 --2- 172.21.15.17:0/7690006 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efc78152820 0x7efc7815a6a0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7efc70009bd0 tx=0x7efc70005b00 comp rx=0 tx=0).stop 2024-09-10T21:47:19.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:19.225+0000 7efc7fa00640 1 -- 172.21.15.17:0/7690006 shutdown_connections 2024-09-10T21:47:19.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:19.225+0000 7efc7fa00640 1 --2- 172.21.15.17:0/7690006 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7efc580725e0 0x7efc58074aa0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:19.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:19.225+0000 7efc7fa00640 1 --2- 172.21.15.17:0/7690006 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efc78152820 0x7efc7815a6a0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:19.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:19.225+0000 7efc7fa00640 1 --2- 172.21.15.17:0/7690006 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efc78152440 0x7efc781f2d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:19.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:19.225+0000 7efc7fa00640 1 --2- 172.21.15.17:0/7690006 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7efc780cdf20 0x7efc7814be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:19.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:19.225+0000 7efc7fa00640 1 -- 172.21.15.17:0/7690006 >> 172.21.15.17:0/7690006 conn(0x7efc780702f0 msgr2=0x7efc780c3390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:19.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:19.229+0000 7efc7fa00640 1 -- 172.21.15.17:0/7690006 shutdown_connections 2024-09-10T21:47:19.235 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:19.229+0000 7efc7fa00640 1 -- 172.21.15.17:0/7690006 wait complete. 2024-09-10T21:47:19.250 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":702,"stamp":"2024-09-10T21:47:17.984631+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":15581531,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":114,"num_read_kb":174,"num_write":726807,"num_write_kb":16011424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":152578,"ondisk_log_size":152578,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":6469456,"kb_used_data":20368,"kb_used_omap":12,"kb_used_meta":6448947,"kb_avail":748505264,"statfs":{"total":773094113280,"available":766469390336,"internally_reserved":0,"allocated":20856832,"data_stored":32095942,"data_compressed":2631228,"data_compressed_allocated":3596288,"data_compressed_original":16236544,"omap_allocated":12752,"internal_metadata":6603722288},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":249,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003773"},"pg_stats":[{"pgid":"2.1f","version":"27'178","reported_seq":215,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.283456+0000","last_change":"2024-09-10T21:24:46.864556+0000","last_active":"2024-09-10T21:47:11.283456+0000","last_peered":"2024-09-10T21:47:11.283456+0000","last_clean":"2024-09-10T21:47:11.283456+0000","last_became_active":"2024-09-10T21:24:46.864288+0000","last_became_peered":"2024-09-10T21:24:46.864288+0000","last_unstale":"2024-09-10T21:47:11.283456+0000","last_undegraded":"2024-09-10T21:47:11.283456+0000","last_fullsized":"2024-09-10T21:47:11.283456+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":178,"log_dups_size":0,"ondisk_log_size":178,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:00:38.734807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2316909,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":173,"num_write_kb":2361,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"27'12","reported_seq":48,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.345156+0000","last_change":"2024-09-10T21:24:47.864452+0000","last_active":"2024-09-10T21:47:11.345156+0000","last_peered":"2024-09-10T21:47:11.345156+0000","last_clean":"2024-09-10T21:47:11.345156+0000","last_became_active":"2024-09-10T21:24:47.864168+0000","last_became_peered":"2024-09-10T21:24:47.864168+0000","last_unstale":"2024-09-10T21:47:11.345156+0000","last_undegraded":"2024-09-10T21:47:11.345156+0000","last_fullsized":"2024-09-10T21:47:11.345156+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:51:31.417845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"27'5290","reported_seq":5318,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.730211+0000","last_change":"2024-09-10T21:24:52.070054+0000","last_active":"2024-09-10T21:46:48.730211+0000","last_peered":"2024-09-10T21:46:48.730211+0000","last_clean":"2024-09-10T21:46:48.730211+0000","last_became_active":"2024-09-10T21:24:52.067258+0000","last_became_peered":"2024-09-10T21:24:52.067258+0000","last_unstale":"2024-09-10T21:46:48.730211+0000","last_undegraded":"2024-09-10T21:46:48.730211+0000","last_fullsized":"2024-09-10T21:46:48.730211+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":5290,"log_dups_size":0,"ondisk_log_size":5290,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:21:53.080013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25332,"num_write_kb":555711,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"27'5","reported_seq":43,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.553182+0000","last_change":"2024-09-10T21:24:46.860648+0000","last_active":"2024-09-10T21:46:48.553182+0000","last_peered":"2024-09-10T21:46:48.553182+0000","last_clean":"2024-09-10T21:46:48.553182+0000","last_became_active":"2024-09-10T21:24:46.860341+0000","last_became_peered":"2024-09-10T21:24:46.860341+0000","last_unstale":"2024-09-10T21:46:48.553182+0000","last_undegraded":"2024-09-10T21:46:48.553182+0000","last_fullsized":"2024-09-10T21:46:48.553182+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:39:24.819979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"27'19","reported_seq":54,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.345477+0000","last_change":"2024-09-10T21:24:47.865617+0000","last_active":"2024-09-10T21:47:11.345477+0000","last_peered":"2024-09-10T21:47:11.345477+0000","last_clean":"2024-09-10T21:47:11.345477+0000","last_became_active":"2024-09-10T21:24:47.865365+0000","last_became_peered":"2024-09-10T21:24:47.865365+0000","last_unstale":"2024-09-10T21:47:11.345477+0000","last_undegraded":"2024-09-10T21:47:11.345477+0000","last_fullsized":"2024-09-10T21:47:11.345477+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:10:05.971996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"27'4776","reported_seq":4804,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.713274+0000","last_change":"2024-09-10T21:24:51.900077+0000","last_active":"2024-09-10T21:46:48.713274+0000","last_peered":"2024-09-10T21:46:48.713274+0000","last_clean":"2024-09-10T21:46:48.713274+0000","last_became_active":"2024-09-10T21:24:51.899652+0000","last_became_peered":"2024-09-10T21:24:51.899652+0000","last_unstale":"2024-09-10T21:46:48.713274+0000","last_undegraded":"2024-09-10T21:46:48.713274+0000","last_fullsized":"2024-09-10T21:46:48.713274+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":4776,"log_dups_size":0,"ondisk_log_size":4776,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:40:49.585857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23087,"num_write_kb":517913,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"27'7","reported_seq":43,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.636914+0000","last_change":"2024-09-10T21:24:46.861040+0000","last_active":"2024-09-10T21:46:48.636914+0000","last_peered":"2024-09-10T21:46:48.636914+0000","last_clean":"2024-09-10T21:46:48.636914+0000","last_became_active":"2024-09-10T21:24:46.860780+0000","last_became_peered":"2024-09-10T21:24:46.860780+0000","last_unstale":"2024-09-10T21:46:48.636914+0000","last_undegraded":"2024-09-10T21:46:48.636914+0000","last_fullsized":"2024-09-10T21:46:48.636914+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:53:31.454472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"27'16","reported_seq":48,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.347558+0000","last_change":"2024-09-10T21:24:47.877997+0000","last_active":"2024-09-10T21:47:11.347558+0000","last_peered":"2024-09-10T21:47:11.347558+0000","last_clean":"2024-09-10T21:47:11.347558+0000","last_became_active":"2024-09-10T21:24:47.877765+0000","last_became_peered":"2024-09-10T21:24:47.877765+0000","last_unstale":"2024-09-10T21:47:11.347558+0000","last_undegraded":"2024-09-10T21:47:11.347558+0000","last_fullsized":"2024-09-10T21:47:11.347558+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:33:20.590209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"27'4118","reported_seq":4153,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.560547+0000","last_change":"2024-09-10T21:25:28.512435+0000","last_active":"2024-09-10T21:46:48.560547+0000","last_peered":"2024-09-10T21:46:48.560547+0000","last_clean":"2024-09-10T21:46:48.560547+0000","last_became_active":"2024-09-10T21:25:28.512023+0000","last_became_peered":"2024-09-10T21:25:28.512023+0000","last_unstale":"2024-09-10T21:46:48.560547+0000","last_undegraded":"2024-09-10T21:46:48.560547+0000","last_fullsized":"2024-09-10T21:46:48.560547+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":4118,"log_dups_size":0,"ondisk_log_size":4118,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:43:17.173095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18983,"num_write_kb":440322,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,2],"acting":[4,3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"27'5","reported_seq":41,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.677062+0000","last_change":"2024-09-10T21:24:46.873044+0000","last_active":"2024-09-10T21:46:48.677062+0000","last_peered":"2024-09-10T21:46:48.677062+0000","last_clean":"2024-09-10T21:46:48.677062+0000","last_became_active":"2024-09-10T21:24:46.872075+0000","last_became_peered":"2024-09-10T21:24:46.872075+0000","last_unstale":"2024-09-10T21:46:48.677062+0000","last_undegraded":"2024-09-10T21:46:48.677062+0000","last_fullsized":"2024-09-10T21:46:48.677062+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:40:33.632846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"27'30","reported_seq":62,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.342014+0000","last_change":"2024-09-10T21:24:47.878094+0000","last_active":"2024-09-10T21:47:11.342014+0000","last_peered":"2024-09-10T21:47:11.342014+0000","last_clean":"2024-09-10T21:47:11.342014+0000","last_became_active":"2024-09-10T21:24:47.876688+0000","last_became_peered":"2024-09-10T21:24:47.876688+0000","last_unstale":"2024-09-10T21:47:11.342014+0000","last_undegraded":"2024-09-10T21:47:11.342014+0000","last_fullsized":"2024-09-10T21:47:11.342014+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:03:37.076080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"27'3999","reported_seq":4026,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.559967+0000","last_change":"2024-09-10T21:24:51.900208+0000","last_active":"2024-09-10T21:46:48.559967+0000","last_peered":"2024-09-10T21:46:48.559967+0000","last_clean":"2024-09-10T21:46:48.559967+0000","last_became_active":"2024-09-10T21:24:51.899576+0000","last_became_peered":"2024-09-10T21:24:51.899576+0000","last_unstale":"2024-09-10T21:46:48.559967+0000","last_undegraded":"2024-09-10T21:46:48.559967+0000","last_fullsized":"2024-09-10T21:46:48.559967+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":3999,"log_dups_size":0,"ondisk_log_size":3999,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:52:49.479932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18841,"num_write_kb":435514,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"27'15","reported_seq":49,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.350364+0000","last_change":"2024-09-10T21:24:47.869830+0000","last_active":"2024-09-10T21:47:11.350364+0000","last_peered":"2024-09-10T21:47:11.350364+0000","last_clean":"2024-09-10T21:47:11.350364+0000","last_became_active":"2024-09-10T21:24:47.869569+0000","last_became_peered":"2024-09-10T21:24:47.869569+0000","last_unstale":"2024-09-10T21:47:11.350364+0000","last_undegraded":"2024-09-10T21:47:11.350364+0000","last_fullsized":"2024-09-10T21:47:11.350364+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:42:15.320518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"27'1003","reported_seq":1040,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.666697+0000","last_change":"2024-09-10T21:24:46.867297+0000","last_active":"2024-09-10T21:46:48.666697+0000","last_peered":"2024-09-10T21:46:48.666697+0000","last_clean":"2024-09-10T21:46:48.666697+0000","last_became_active":"2024-09-10T21:24:46.866893+0000","last_became_peered":"2024-09-10T21:24:46.866893+0000","last_unstale":"2024-09-10T21:46:48.666697+0000","last_undegraded":"2024-09-10T21:46:48.666697+0000","last_fullsized":"2024-09-10T21:46:48.666697+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":1003,"log_dups_size":0,"ondisk_log_size":1003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:53.197666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":997,"num_write_kb":4359,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"27'4606","reported_seq":4634,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.705887+0000","last_change":"2024-09-10T21:24:51.908412+0000","last_active":"2024-09-10T21:46:48.705887+0000","last_peered":"2024-09-10T21:46:48.705887+0000","last_clean":"2024-09-10T21:46:48.705887+0000","last_became_active":"2024-09-10T21:24:51.907923+0000","last_became_peered":"2024-09-10T21:24:51.907923+0000","last_unstale":"2024-09-10T21:46:48.705887+0000","last_undegraded":"2024-09-10T21:46:48.705887+0000","last_fullsized":"2024-09-10T21:46:48.705887+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":4606,"log_dups_size":0,"ondisk_log_size":4606,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:26:46.479501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22377,"num_write_kb":531442,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"27'17","reported_seq":52,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.331673+0000","last_change":"2024-09-10T21:24:47.862479+0000","last_active":"2024-09-10T21:47:11.331673+0000","last_peered":"2024-09-10T21:47:11.331673+0000","last_clean":"2024-09-10T21:47:11.331673+0000","last_became_active":"2024-09-10T21:24:47.861842+0000","last_became_peered":"2024-09-10T21:24:47.861842+0000","last_unstale":"2024-09-10T21:47:11.331673+0000","last_undegraded":"2024-09-10T21:47:11.331673+0000","last_fullsized":"2024-09-10T21:47:11.331673+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:53:09.082512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"27'3","reported_seq":39,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.551236+0000","last_change":"2024-09-10T21:24:46.852467+0000","last_active":"2024-09-10T21:46:48.551236+0000","last_peered":"2024-09-10T21:46:48.551236+0000","last_clean":"2024-09-10T21:46:48.551236+0000","last_became_active":"2024-09-10T21:24:46.851027+0000","last_became_peered":"2024-09-10T21:24:46.851027+0000","last_unstale":"2024-09-10T21:46:48.551236+0000","last_undegraded":"2024-09-10T21:46:48.551236+0000","last_fullsized":"2024-09-10T21:46:48.551236+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:39:27.441714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"27'4524","reported_seq":4552,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.720455+0000","last_change":"2024-09-10T21:24:51.903577+0000","last_active":"2024-09-10T21:46:48.720455+0000","last_peered":"2024-09-10T21:46:48.720455+0000","last_clean":"2024-09-10T21:46:48.720455+0000","last_became_active":"2024-09-10T21:24:51.903031+0000","last_became_peered":"2024-09-10T21:24:51.903031+0000","last_unstale":"2024-09-10T21:46:48.720455+0000","last_undegraded":"2024-09-10T21:46:48.720455+0000","last_fullsized":"2024-09-10T21:46:48.720455+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":4524,"log_dups_size":0,"ondisk_log_size":4524,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:08:52.787720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21954,"num_write_kb":484896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"27'18","reported_seq":53,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.328856+0000","last_change":"2024-09-10T21:24:47.856334+0000","last_active":"2024-09-10T21:47:11.328856+0000","last_peered":"2024-09-10T21:47:11.328856+0000","last_clean":"2024-09-10T21:47:11.328856+0000","last_became_active":"2024-09-10T21:24:47.856003+0000","last_became_peered":"2024-09-10T21:24:47.856003+0000","last_unstale":"2024-09-10T21:47:11.328856+0000","last_undegraded":"2024-09-10T21:47:11.328856+0000","last_fullsized":"2024-09-10T21:47:11.328856+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:58:53.698584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"27'2","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.586182+0000","last_change":"2024-09-10T21:24:46.853373+0000","last_active":"2024-09-10T21:46:48.586182+0000","last_peered":"2024-09-10T21:46:48.586182+0000","last_clean":"2024-09-10T21:46:48.586182+0000","last_became_active":"2024-09-10T21:24:46.852993+0000","last_became_peered":"2024-09-10T21:24:46.852993+0000","last_unstale":"2024-09-10T21:46:48.586182+0000","last_undegraded":"2024-09-10T21:46:48.586182+0000","last_fullsized":"2024-09-10T21:46:48.586182+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:53:31.454472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"27'5859","reported_seq":5887,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.677773+0000","last_change":"2024-09-10T21:24:52.065246+0000","last_active":"2024-09-10T21:46:48.677773+0000","last_peered":"2024-09-10T21:46:48.677773+0000","last_clean":"2024-09-10T21:46:48.677773+0000","last_became_active":"2024-09-10T21:24:52.064884+0000","last_became_peered":"2024-09-10T21:24:52.064884+0000","last_unstale":"2024-09-10T21:46:48.677773+0000","last_undegraded":"2024-09-10T21:46:48.677773+0000","last_fullsized":"2024-09-10T21:46:48.677773+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":5859,"log_dups_size":0,"ondisk_log_size":5859,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:54:54.393305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27858,"num_write_kb":655707,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"27'9","reported_seq":44,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.296539+0000","last_change":"2024-09-10T21:24:47.856517+0000","last_active":"2024-09-10T21:47:11.296539+0000","last_peered":"2024-09-10T21:47:11.296539+0000","last_clean":"2024-09-10T21:47:11.296539+0000","last_became_active":"2024-09-10T21:24:47.856185+0000","last_became_peered":"2024-09-10T21:24:47.856185+0000","last_unstale":"2024-09-10T21:47:11.296539+0000","last_undegraded":"2024-09-10T21:47:11.296539+0000","last_fullsized":"2024-09-10T21:47:11.296539+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:26:00.852818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"27'5","reported_seq":42,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:36.457065+0000","last_change":"2024-09-10T21:24:46.867888+0000","last_active":"2024-09-10T21:46:36.457065+0000","last_peered":"2024-09-10T21:46:36.457065+0000","last_clean":"2024-09-10T21:46:36.457065+0000","last_became_active":"2024-09-10T21:24:46.866477+0000","last_became_peered":"2024-09-10T21:24:46.866477+0000","last_unstale":"2024-09-10T21:46:36.457065+0000","last_undegraded":"2024-09-10T21:46:36.457065+0000","last_fullsized":"2024-09-10T21:46:36.457065+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:39:27.441714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"27'4736","reported_seq":4764,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.660872+0000","last_change":"2024-09-10T21:24:51.908376+0000","last_active":"2024-09-10T21:46:48.660872+0000","last_peered":"2024-09-10T21:46:48.660872+0000","last_clean":"2024-09-10T21:46:48.660872+0000","last_became_active":"2024-09-10T21:24:51.907878+0000","last_became_peered":"2024-09-10T21:24:51.907878+0000","last_unstale":"2024-09-10T21:46:48.660872+0000","last_undegraded":"2024-09-10T21:46:48.660872+0000","last_fullsized":"2024-09-10T21:46:48.660872+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":4736,"log_dups_size":0,"ondisk_log_size":4736,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:39:02.492952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22584,"num_write_kb":479265,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"27'25","reported_seq":60,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.323165+0000","last_change":"2024-09-10T21:24:47.844823+0000","last_active":"2024-09-10T21:47:11.323165+0000","last_peered":"2024-09-10T21:47:11.323165+0000","last_clean":"2024-09-10T21:47:11.323165+0000","last_became_active":"2024-09-10T21:24:47.844505+0000","last_became_peered":"2024-09-10T21:24:47.844505+0000","last_unstale":"2024-09-10T21:47:11.323165+0000","last_undegraded":"2024-09-10T21:47:11.323165+0000","last_fullsized":"2024-09-10T21:47:11.323165+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:37:51.784258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"27'9","reported_seq":45,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.639567+0000","last_change":"2024-09-10T21:24:46.862172+0000","last_active":"2024-09-10T21:46:48.639567+0000","last_peered":"2024-09-10T21:46:48.639567+0000","last_clean":"2024-09-10T21:46:48.639567+0000","last_became_active":"2024-09-10T21:24:46.861473+0000","last_became_peered":"2024-09-10T21:24:46.861473+0000","last_unstale":"2024-09-10T21:46:48.639567+0000","last_undegraded":"2024-09-10T21:46:48.639567+0000","last_fullsized":"2024-09-10T21:46:48.639567+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:53:31.454472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":53,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"27'4521","reported_seq":4550,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.535321+0000","last_change":"2024-09-10T21:24:52.062841+0000","last_active":"2024-09-10T21:46:48.535321+0000","last_peered":"2024-09-10T21:46:48.535321+0000","last_clean":"2024-09-10T21:46:48.535321+0000","last_became_active":"2024-09-10T21:24:52.061581+0000","last_became_peered":"2024-09-10T21:24:52.061581+0000","last_unstale":"2024-09-10T21:46:48.535321+0000","last_undegraded":"2024-09-10T21:46:48.535321+0000","last_fullsized":"2024-09-10T21:46:48.535321+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":4521,"log_dups_size":0,"ondisk_log_size":4521,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:34:03.325781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21874,"num_write_kb":476387,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"27'18","reported_seq":53,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.324770+0000","last_change":"2024-09-10T21:24:47.859801+0000","last_active":"2024-09-10T21:47:11.324770+0000","last_peered":"2024-09-10T21:47:11.324770+0000","last_clean":"2024-09-10T21:47:11.324770+0000","last_became_active":"2024-09-10T21:24:47.859483+0000","last_became_peered":"2024-09-10T21:24:47.859483+0000","last_unstale":"2024-09-10T21:47:11.324770+0000","last_undegraded":"2024-09-10T21:47:11.324770+0000","last_fullsized":"2024-09-10T21:47:11.324770+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:38:21.368499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"27'326","reported_seq":362,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:36.365019+0000","last_change":"2024-09-10T21:24:46.860624+0000","last_active":"2024-09-10T21:46:36.365019+0000","last_peered":"2024-09-10T21:46:36.365019+0000","last_clean":"2024-09-10T21:46:36.365019+0000","last_became_active":"2024-09-10T21:24:46.860309+0000","last_became_peered":"2024-09-10T21:24:46.860309+0000","last_unstale":"2024-09-10T21:46:36.365019+0000","last_undegraded":"2024-09-10T21:46:36.365019+0000","last_fullsized":"2024-09-10T21:46:36.365019+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":326,"log_dups_size":0,"ondisk_log_size":326,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:37:12.332013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":704,"num_write_kb":242,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"27'4358","reported_seq":4388,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.745954+0000","last_change":"2024-09-10T21:24:52.068581+0000","last_active":"2024-09-10T21:46:48.745954+0000","last_peered":"2024-09-10T21:46:48.745954+0000","last_clean":"2024-09-10T21:46:48.745954+0000","last_became_active":"2024-09-10T21:24:52.068250+0000","last_became_peered":"2024-09-10T21:24:52.068250+0000","last_unstale":"2024-09-10T21:46:48.745954+0000","last_undegraded":"2024-09-10T21:46:48.745954+0000","last_fullsized":"2024-09-10T21:46:48.745954+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":4358,"log_dups_size":0,"ondisk_log_size":4358,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:52:01.990009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21666,"num_write_kb":463209,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"27'18","reported_seq":52,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.321574+0000","last_change":"2024-09-10T21:24:47.854042+0000","last_active":"2024-09-10T21:47:11.321574+0000","last_peered":"2024-09-10T21:47:11.321574+0000","last_clean":"2024-09-10T21:47:11.321574+0000","last_became_active":"2024-09-10T21:24:47.853750+0000","last_became_peered":"2024-09-10T21:24:47.853750+0000","last_unstale":"2024-09-10T21:47:11.321574+0000","last_undegraded":"2024-09-10T21:47:11.321574+0000","last_fullsized":"2024-09-10T21:47:11.321574+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:06:50.775132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"27'2","reported_seq":39,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:45:42.186926+0000","last_change":"2024-09-10T21:24:46.864617+0000","last_active":"2024-09-10T21:45:42.186926+0000","last_peered":"2024-09-10T21:45:42.186926+0000","last_clean":"2024-09-10T21:45:42.186926+0000","last_became_active":"2024-09-10T21:24:46.864149+0000","last_became_peered":"2024-09-10T21:24:46.864149+0000","last_unstale":"2024-09-10T21:45:42.186926+0000","last_undegraded":"2024-09-10T21:45:42.186926+0000","last_fullsized":"2024-09-10T21:45:42.186926+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:24:15.673107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"27'4655","reported_seq":4684,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.520210+0000","last_change":"2024-09-10T21:24:52.061485+0000","last_active":"2024-09-10T21:46:48.520210+0000","last_peered":"2024-09-10T21:46:48.520210+0000","last_clean":"2024-09-10T21:46:48.520210+0000","last_became_active":"2024-09-10T21:24:52.060953+0000","last_became_peered":"2024-09-10T21:24:52.060953+0000","last_unstale":"2024-09-10T21:46:48.520210+0000","last_undegraded":"2024-09-10T21:46:48.520210+0000","last_fullsized":"2024-09-10T21:46:48.520210+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":4655,"log_dups_size":0,"ondisk_log_size":4655,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:49:30.147202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22893,"num_write_kb":513258,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"27'18","reported_seq":52,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.341860+0000","last_change":"2024-09-10T21:24:47.867241+0000","last_active":"2024-09-10T21:47:11.341860+0000","last_peered":"2024-09-10T21:47:11.341860+0000","last_clean":"2024-09-10T21:47:11.341860+0000","last_became_active":"2024-09-10T21:24:47.866907+0000","last_became_peered":"2024-09-10T21:24:47.866907+0000","last_unstale":"2024-09-10T21:47:11.341860+0000","last_undegraded":"2024-09-10T21:47:11.341860+0000","last_fullsized":"2024-09-10T21:47:11.341860+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:07.620800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"27'2","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.562765+0000","last_change":"2024-09-10T21:24:46.856848+0000","last_active":"2024-09-10T21:46:48.562765+0000","last_peered":"2024-09-10T21:46:48.562765+0000","last_clean":"2024-09-10T21:46:48.562765+0000","last_became_active":"2024-09-10T21:24:46.856489+0000","last_became_peered":"2024-09-10T21:24:46.856489+0000","last_unstale":"2024-09-10T21:46:48.562765+0000","last_undegraded":"2024-09-10T21:46:48.562765+0000","last_fullsized":"2024-09-10T21:46:48.562765+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:46:55.286321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"27'4909","reported_seq":4938,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.660587+0000","last_change":"2024-09-10T21:24:51.901016+0000","last_active":"2024-09-10T21:46:48.660587+0000","last_peered":"2024-09-10T21:46:48.660587+0000","last_clean":"2024-09-10T21:46:48.660587+0000","last_became_active":"2024-09-10T21:24:51.900479+0000","last_became_peered":"2024-09-10T21:24:51.900479+0000","last_unstale":"2024-09-10T21:46:48.660587+0000","last_undegraded":"2024-09-10T21:46:48.660587+0000","last_fullsized":"2024-09-10T21:46:48.660587+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":4909,"log_dups_size":0,"ondisk_log_size":4909,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:36.741810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23807,"num_write_kb":525103,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"27'16","reported_seq":51,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.320532+0000","last_change":"2024-09-10T21:24:47.858753+0000","last_active":"2024-09-10T21:47:11.320532+0000","last_peered":"2024-09-10T21:47:11.320532+0000","last_clean":"2024-09-10T21:47:11.320532+0000","last_became_active":"2024-09-10T21:24:47.858438+0000","last_became_peered":"2024-09-10T21:24:47.858438+0000","last_unstale":"2024-09-10T21:47:11.320532+0000","last_undegraded":"2024-09-10T21:47:11.320532+0000","last_fullsized":"2024-09-10T21:47:11.320532+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:12:09.198013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"27'4","reported_seq":40,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:45:39.089516+0000","last_change":"2024-09-10T21:24:46.863610+0000","last_active":"2024-09-10T21:45:39.089516+0000","last_peered":"2024-09-10T21:45:39.089516+0000","last_clean":"2024-09-10T21:45:39.089516+0000","last_became_active":"2024-09-10T21:24:46.863150+0000","last_became_peered":"2024-09-10T21:24:46.863150+0000","last_unstale":"2024-09-10T21:45:39.089516+0000","last_undegraded":"2024-09-10T21:45:39.089516+0000","last_fullsized":"2024-09-10T21:45:39.089516+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:27:35.473995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"27'5335","reported_seq":5364,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.650031+0000","last_change":"2024-09-10T21:24:51.907366+0000","last_active":"2024-09-10T21:46:48.650031+0000","last_peered":"2024-09-10T21:46:48.650031+0000","last_clean":"2024-09-10T21:46:48.650031+0000","last_became_active":"2024-09-10T21:24:51.906888+0000","last_became_peered":"2024-09-10T21:24:51.906888+0000","last_unstale":"2024-09-10T21:46:48.650031+0000","last_undegraded":"2024-09-10T21:46:48.650031+0000","last_fullsized":"2024-09-10T21:46:48.650031+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":5335,"log_dups_size":0,"ondisk_log_size":5335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:24:54.278514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26491,"num_write_kb":568740,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"27'20","reported_seq":55,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.311229+0000","last_change":"2024-09-10T21:24:47.850155+0000","last_active":"2024-09-10T21:47:11.311229+0000","last_peered":"2024-09-10T21:47:11.311229+0000","last_clean":"2024-09-10T21:47:11.311229+0000","last_became_active":"2024-09-10T21:24:47.849846+0000","last_became_peered":"2024-09-10T21:24:47.849846+0000","last_unstale":"2024-09-10T21:47:11.311229+0000","last_undegraded":"2024-09-10T21:47:11.311229+0000","last_fullsized":"2024-09-10T21:47:11.311229+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:13:40.032808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"27'51","reported_seq":136,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.553709+0000","last_change":"2024-09-10T21:24:46.851800+0000","last_active":"2024-09-10T21:46:48.553709+0000","last_peered":"2024-09-10T21:46:48.553709+0000","last_clean":"2024-09-10T21:46:48.553709+0000","last_became_active":"2024-09-10T21:24:46.851200+0000","last_became_peered":"2024-09-10T21:24:46.851200+0000","last_unstale":"2024-09-10T21:46:48.553709+0000","last_undegraded":"2024-09-10T21:46:48.553709+0000","last_fullsized":"2024-09-10T21:46:48.553709+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:00:22.839361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90454,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":48,"num_read_kb":119,"num_write":48,"num_write_kb":119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"27'4677","reported_seq":4706,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.581789+0000","last_change":"2024-09-10T21:24:51.902529+0000","last_active":"2024-09-10T21:46:48.581789+0000","last_peered":"2024-09-10T21:46:48.581789+0000","last_clean":"2024-09-10T21:46:48.581789+0000","last_became_active":"2024-09-10T21:24:51.901141+0000","last_became_peered":"2024-09-10T21:24:51.901141+0000","last_unstale":"2024-09-10T21:46:48.581789+0000","last_undegraded":"2024-09-10T21:46:48.581789+0000","last_fullsized":"2024-09-10T21:46:48.581789+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":4677,"log_dups_size":0,"ondisk_log_size":4677,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:08:40.203348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22963,"num_write_kb":501268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"27'11","reported_seq":46,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.323431+0000","last_change":"2024-09-10T21:24:47.846582+0000","last_active":"2024-09-10T21:47:11.323431+0000","last_peered":"2024-09-10T21:47:11.323431+0000","last_clean":"2024-09-10T21:47:11.323431+0000","last_became_active":"2024-09-10T21:24:47.846358+0000","last_became_peered":"2024-09-10T21:24:47.846358+0000","last_unstale":"2024-09-10T21:47:11.323431+0000","last_undegraded":"2024-09-10T21:47:11.323431+0000","last_fullsized":"2024-09-10T21:47:11.323431+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:58:53.698584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"27'7","reported_seq":45,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.647854+0000","last_change":"2024-09-10T21:24:46.870908+0000","last_active":"2024-09-10T21:46:48.647854+0000","last_peered":"2024-09-10T21:46:48.647854+0000","last_clean":"2024-09-10T21:46:48.647854+0000","last_became_active":"2024-09-10T21:24:46.868375+0000","last_became_peered":"2024-09-10T21:24:46.868375+0000","last_unstale":"2024-09-10T21:46:48.647854+0000","last_undegraded":"2024-09-10T21:46:48.647854+0000","last_fullsized":"2024-09-10T21:46:48.647854+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:37:12.332013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"27'4824","reported_seq":4853,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.659490+0000","last_change":"2024-09-10T21:24:52.069018+0000","last_active":"2024-09-10T21:46:48.659490+0000","last_peered":"2024-09-10T21:46:48.659490+0000","last_clean":"2024-09-10T21:46:48.659490+0000","last_became_active":"2024-09-10T21:24:52.067308+0000","last_became_peered":"2024-09-10T21:24:52.067308+0000","last_unstale":"2024-09-10T21:46:48.659490+0000","last_undegraded":"2024-09-10T21:46:48.659490+0000","last_fullsized":"2024-09-10T21:46:48.659490+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":4824,"log_dups_size":0,"ondisk_log_size":4824,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:07:25.162168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23972,"num_write_kb":500992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"27'13","reported_seq":48,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.338908+0000","last_change":"2024-09-10T21:24:47.871542+0000","last_active":"2024-09-10T21:47:11.338908+0000","last_peered":"2024-09-10T21:47:11.338908+0000","last_clean":"2024-09-10T21:47:11.338908+0000","last_became_active":"2024-09-10T21:24:47.871128+0000","last_became_peered":"2024-09-10T21:24:47.871128+0000","last_unstale":"2024-09-10T21:47:11.338908+0000","last_undegraded":"2024-09-10T21:47:11.338908+0000","last_fullsized":"2024-09-10T21:47:11.338908+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:51:22.876126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"27'3","reported_seq":39,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:36.366557+0000","last_change":"2024-09-10T21:24:46.859218+0000","last_active":"2024-09-10T21:46:36.366557+0000","last_peered":"2024-09-10T21:46:36.366557+0000","last_clean":"2024-09-10T21:46:36.366557+0000","last_became_active":"2024-09-10T21:24:46.853190+0000","last_became_peered":"2024-09-10T21:24:46.853190+0000","last_unstale":"2024-09-10T21:46:36.366557+0000","last_undegraded":"2024-09-10T21:46:36.366557+0000","last_fullsized":"2024-09-10T21:46:36.366557+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:26:34.848759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":134,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:44:42.019323+0000","last_change":"2024-09-10T21:24:37.216542+0000","last_active":"2024-09-10T21:44:42.019323+0000","last_peered":"2024-09-10T21:44:42.019323+0000","last_clean":"2024-09-10T21:44:42.019323+0000","last_became_active":"2024-09-10T21:24:37.215443+0000","last_became_peered":"2024-09-10T21:24:37.215443+0000","last_unstale":"2024-09-10T21:44:42.019323+0000","last_undegraded":"2024-09-10T21:44:42.019323+0000","last_fullsized":"2024-09-10T21:44:42.019323+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:35.877801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:35.877801+0000","last_clean_scrub_stamp":"2024-09-10T21:24:35.877801+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:14:20.260329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"27'4394","reported_seq":4423,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.650952+0000","last_change":"2024-09-10T21:24:52.067645+0000","last_active":"2024-09-10T21:46:48.650952+0000","last_peered":"2024-09-10T21:46:48.650952+0000","last_clean":"2024-09-10T21:46:48.650952+0000","last_became_active":"2024-09-10T21:24:52.067148+0000","last_became_peered":"2024-09-10T21:24:52.067148+0000","last_unstale":"2024-09-10T21:46:48.650952+0000","last_undegraded":"2024-09-10T21:46:48.650952+0000","last_fullsized":"2024-09-10T21:46:48.650952+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":4394,"log_dups_size":0,"ondisk_log_size":4394,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:54:42.435011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21702,"num_write_kb":479680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"27'21","reported_seq":61,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.342553+0000","last_change":"2024-09-10T21:25:28.510354+0000","last_active":"2024-09-10T21:47:11.342553+0000","last_peered":"2024-09-10T21:47:11.342553+0000","last_clean":"2024-09-10T21:47:11.342553+0000","last_became_active":"2024-09-10T21:25:28.509981+0000","last_became_peered":"2024-09-10T21:25:28.509981+0000","last_unstale":"2024-09-10T21:47:11.342553+0000","last_undegraded":"2024-09-10T21:47:11.342553+0000","last_fullsized":"2024-09-10T21:47:11.342553+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:51:42.501764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"27'5","reported_seq":41,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.629368+0000","last_change":"2024-09-10T21:24:46.859532+0000","last_active":"2024-09-10T21:46:48.629368+0000","last_peered":"2024-09-10T21:46:48.629368+0000","last_clean":"2024-09-10T21:46:48.629368+0000","last_became_active":"2024-09-10T21:24:46.859144+0000","last_became_peered":"2024-09-10T21:24:46.859144+0000","last_unstale":"2024-09-10T21:46:48.629368+0000","last_undegraded":"2024-09-10T21:46:48.629368+0000","last_fullsized":"2024-09-10T21:46:48.629368+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:48:41.992441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"27'4328","reported_seq":4365,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.651243+0000","last_change":"2024-09-10T21:25:28.620859+0000","last_active":"2024-09-10T21:46:48.651243+0000","last_peered":"2024-09-10T21:46:48.651243+0000","last_clean":"2024-09-10T21:46:48.651243+0000","last_became_active":"2024-09-10T21:25:28.620415+0000","last_became_peered":"2024-09-10T21:25:28.620415+0000","last_unstale":"2024-09-10T21:46:48.651243+0000","last_undegraded":"2024-09-10T21:46:48.651243+0000","last_fullsized":"2024-09-10T21:46:48.651243+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":4328,"log_dups_size":0,"ondisk_log_size":4328,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:17:39.229564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21368,"num_write_kb":498780,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2,0],"acting":[6,7,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"27'17","reported_seq":50,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.342104+0000","last_change":"2024-09-10T21:24:47.866927+0000","last_active":"2024-09-10T21:47:11.342104+0000","last_peered":"2024-09-10T21:47:11.342104+0000","last_clean":"2024-09-10T21:47:11.342104+0000","last_became_active":"2024-09-10T21:24:47.866650+0000","last_became_peered":"2024-09-10T21:24:47.866650+0000","last_unstale":"2024-09-10T21:47:11.342104+0000","last_undegraded":"2024-09-10T21:47:11.342104+0000","last_fullsized":"2024-09-10T21:47:11.342104+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:47:37.304624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"27'6","reported_seq":42,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.675955+0000","last_change":"2024-09-10T21:24:46.857082+0000","last_active":"2024-09-10T21:46:48.675955+0000","last_peered":"2024-09-10T21:46:48.675955+0000","last_clean":"2024-09-10T21:46:48.675955+0000","last_became_active":"2024-09-10T21:24:46.856799+0000","last_became_peered":"2024-09-10T21:24:46.856799+0000","last_unstale":"2024-09-10T21:46:48.675955+0000","last_undegraded":"2024-09-10T21:46:48.675955+0000","last_fullsized":"2024-09-10T21:46:48.675955+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:40:33.632846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"27'4625","reported_seq":4654,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.739934+0000","last_change":"2024-09-10T21:24:51.911880+0000","last_active":"2024-09-10T21:46:48.739934+0000","last_peered":"2024-09-10T21:46:48.739934+0000","last_clean":"2024-09-10T21:46:48.739934+0000","last_became_active":"2024-09-10T21:24:51.911088+0000","last_became_peered":"2024-09-10T21:24:51.911088+0000","last_unstale":"2024-09-10T21:46:48.739934+0000","last_undegraded":"2024-09-10T21:46:48.739934+0000","last_fullsized":"2024-09-10T21:46:48.739934+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":4625,"log_dups_size":0,"ondisk_log_size":4625,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:38:57.030655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21860,"num_write_kb":505231,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"27'20","reported_seq":54,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.344779+0000","last_change":"2024-09-10T21:24:47.876946+0000","last_active":"2024-09-10T21:47:11.344779+0000","last_peered":"2024-09-10T21:47:11.344779+0000","last_clean":"2024-09-10T21:47:11.344779+0000","last_became_active":"2024-09-10T21:24:47.876690+0000","last_became_peered":"2024-09-10T21:24:47.876690+0000","last_unstale":"2024-09-10T21:47:11.344779+0000","last_undegraded":"2024-09-10T21:47:11.344779+0000","last_fullsized":"2024-09-10T21:47:11.344779+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:14:06.368275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"27'7","reported_seq":43,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.569377+0000","last_change":"2024-09-10T21:24:46.855248+0000","last_active":"2024-09-10T21:46:48.569377+0000","last_peered":"2024-09-10T21:46:48.569377+0000","last_clean":"2024-09-10T21:46:48.569377+0000","last_became_active":"2024-09-10T21:24:46.854930+0000","last_became_peered":"2024-09-10T21:24:46.854930+0000","last_unstale":"2024-09-10T21:46:48.569377+0000","last_undegraded":"2024-09-10T21:46:48.569377+0000","last_fullsized":"2024-09-10T21:46:48.569377+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:46:55.286321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"27'4395","reported_seq":4424,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.566233+0000","last_change":"2024-09-10T21:24:52.063112+0000","last_active":"2024-09-10T21:46:48.566233+0000","last_peered":"2024-09-10T21:46:48.566233+0000","last_clean":"2024-09-10T21:46:48.566233+0000","last_became_active":"2024-09-10T21:24:52.062395+0000","last_became_peered":"2024-09-10T21:24:52.062395+0000","last_unstale":"2024-09-10T21:46:48.566233+0000","last_undegraded":"2024-09-10T21:46:48.566233+0000","last_fullsized":"2024-09-10T21:46:48.566233+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":4395,"log_dups_size":0,"ondisk_log_size":4395,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:21:53.080013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21722,"num_write_kb":475710,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"27'13","reported_seq":33,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.321587+0000","last_change":"2024-09-10T21:25:28.513604+0000","last_active":"2024-09-10T21:47:11.321587+0000","last_peered":"2024-09-10T21:47:11.321587+0000","last_clean":"2024-09-10T21:47:11.321587+0000","last_became_active":"2024-09-10T21:25:28.513250+0000","last_became_peered":"2024-09-10T21:25:28.513250+0000","last_unstale":"2024-09-10T21:47:11.321587+0000","last_undegraded":"2024-09-10T21:47:11.321587+0000","last_fullsized":"2024-09-10T21:47:11.321587+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:54:13.200082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.f","version":"27'4","reported_seq":40,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.587704+0000","last_change":"2024-09-10T21:24:46.860330+0000","last_active":"2024-09-10T21:46:48.587704+0000","last_peered":"2024-09-10T21:46:48.587704+0000","last_clean":"2024-09-10T21:46:48.587704+0000","last_became_active":"2024-09-10T21:24:46.860053+0000","last_became_peered":"2024-09-10T21:24:46.860053+0000","last_unstale":"2024-09-10T21:46:48.587704+0000","last_undegraded":"2024-09-10T21:46:48.587704+0000","last_fullsized":"2024-09-10T21:46:48.587704+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:58:53.177740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"27'4437","reported_seq":4466,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.614888+0000","last_change":"2024-09-10T21:24:51.901673+0000","last_active":"2024-09-10T21:46:48.614888+0000","last_peered":"2024-09-10T21:46:48.614888+0000","last_clean":"2024-09-10T21:46:48.614888+0000","last_became_active":"2024-09-10T21:24:51.901222+0000","last_became_peered":"2024-09-10T21:24:51.901222+0000","last_unstale":"2024-09-10T21:46:48.614888+0000","last_undegraded":"2024-09-10T21:46:48.614888+0000","last_fullsized":"2024-09-10T21:46:48.614888+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":4437,"log_dups_size":0,"ondisk_log_size":4437,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:07:52.120299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21706,"num_write_kb":463399,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"27'19","reported_seq":53,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.328131+0000","last_change":"2024-09-10T21:24:47.870589+0000","last_active":"2024-09-10T21:47:11.328131+0000","last_peered":"2024-09-10T21:47:11.328131+0000","last_clean":"2024-09-10T21:47:11.328131+0000","last_became_active":"2024-09-10T21:24:47.861834+0000","last_became_peered":"2024-09-10T21:24:47.861834+0000","last_unstale":"2024-09-10T21:47:11.328131+0000","last_undegraded":"2024-09-10T21:47:11.328131+0000","last_fullsized":"2024-09-10T21:47:11.328131+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:43.363013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"27'15","reported_seq":51,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.643062+0000","last_change":"2024-09-10T21:24:46.852064+0000","last_active":"2024-09-10T21:46:48.643062+0000","last_peered":"2024-09-10T21:46:48.643062+0000","last_clean":"2024-09-10T21:46:48.643062+0000","last_became_active":"2024-09-10T21:24:46.851776+0000","last_became_peered":"2024-09-10T21:24:46.851776+0000","last_unstale":"2024-09-10T21:46:48.643062+0000","last_undegraded":"2024-09-10T21:46:48.643062+0000","last_fullsized":"2024-09-10T21:46:48.643062+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:26:24.371844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"27'5330","reported_seq":5358,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.742778+0000","last_change":"2024-09-10T21:24:51.904449+0000","last_active":"2024-09-10T21:46:48.742778+0000","last_peered":"2024-09-10T21:46:48.742778+0000","last_clean":"2024-09-10T21:46:48.742778+0000","last_became_active":"2024-09-10T21:24:51.903533+0000","last_became_peered":"2024-09-10T21:24:51.903533+0000","last_unstale":"2024-09-10T21:46:48.742778+0000","last_undegraded":"2024-09-10T21:46:48.742778+0000","last_fullsized":"2024-09-10T21:46:48.742778+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":5330,"log_dups_size":0,"ondisk_log_size":5330,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:11.654935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26446,"num_write_kb":547906,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"27'14","reported_seq":47,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.347665+0000","last_change":"2024-09-10T21:24:47.857736+0000","last_active":"2024-09-10T21:47:11.347665+0000","last_peered":"2024-09-10T21:47:11.347665+0000","last_clean":"2024-09-10T21:47:11.347665+0000","last_became_active":"2024-09-10T21:24:47.857437+0000","last_became_peered":"2024-09-10T21:24:47.857437+0000","last_unstale":"2024-09-10T21:47:11.347665+0000","last_undegraded":"2024-09-10T21:47:11.347665+0000","last_fullsized":"2024-09-10T21:47:11.347665+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:13:40.032808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"27'5","reported_seq":40,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.633944+0000","last_change":"2024-09-10T21:24:46.860662+0000","last_active":"2024-09-10T21:46:48.633944+0000","last_peered":"2024-09-10T21:46:48.633944+0000","last_clean":"2024-09-10T21:46:48.633944+0000","last_became_active":"2024-09-10T21:24:46.859796+0000","last_became_peered":"2024-09-10T21:24:46.859796+0000","last_unstale":"2024-09-10T21:46:48.633944+0000","last_undegraded":"2024-09-10T21:46:48.633944+0000","last_fullsized":"2024-09-10T21:46:48.633944+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:48:27.758481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"27'5017","reported_seq":5045,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.677969+0000","last_change":"2024-09-10T21:24:52.069770+0000","last_active":"2024-09-10T21:46:48.677969+0000","last_peered":"2024-09-10T21:46:48.677969+0000","last_clean":"2024-09-10T21:46:48.677969+0000","last_became_active":"2024-09-10T21:24:52.067409+0000","last_became_peered":"2024-09-10T21:24:52.067409+0000","last_unstale":"2024-09-10T21:46:48.677969+0000","last_undegraded":"2024-09-10T21:46:48.677969+0000","last_fullsized":"2024-09-10T21:46:48.677969+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":5017,"log_dups_size":0,"ondisk_log_size":5017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:23:39.622612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24522,"num_write_kb":548501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"27'30","reported_seq":64,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.343245+0000","last_change":"2024-09-10T21:24:47.858692+0000","last_active":"2024-09-10T21:47:11.343245+0000","last_peered":"2024-09-10T21:47:11.343245+0000","last_clean":"2024-09-10T21:47:11.343245+0000","last_became_active":"2024-09-10T21:24:47.858343+0000","last_became_peered":"2024-09-10T21:24:47.858343+0000","last_unstale":"2024-09-10T21:47:11.343245+0000","last_undegraded":"2024-09-10T21:47:11.343245+0000","last_fullsized":"2024-09-10T21:47:11.343245+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:04:36.661791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"27'3","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.555096+0000","last_change":"2024-09-10T21:24:46.864592+0000","last_active":"2024-09-10T21:46:48.555096+0000","last_peered":"2024-09-10T21:46:48.555096+0000","last_clean":"2024-09-10T21:46:48.555096+0000","last_became_active":"2024-09-10T21:24:46.858696+0000","last_became_peered":"2024-09-10T21:24:46.858696+0000","last_unstale":"2024-09-10T21:46:48.555096+0000","last_undegraded":"2024-09-10T21:46:48.555096+0000","last_fullsized":"2024-09-10T21:46:48.555096+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:54:19.356867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"27'4314","reported_seq":4350,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.681791+0000","last_change":"2024-09-10T21:25:28.802910+0000","last_active":"2024-09-10T21:46:48.681791+0000","last_peered":"2024-09-10T21:46:48.681791+0000","last_clean":"2024-09-10T21:46:48.681791+0000","last_became_active":"2024-09-10T21:25:28.802224+0000","last_became_peered":"2024-09-10T21:25:28.802224+0000","last_unstale":"2024-09-10T21:46:48.681791+0000","last_undegraded":"2024-09-10T21:46:48.681791+0000","last_fullsized":"2024-09-10T21:46:48.681791+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":4314,"log_dups_size":0,"ondisk_log_size":4314,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:27:09.948701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20768,"num_write_kb":476051,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7,2],"acting":[3,5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"27'23","reported_seq":57,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.332930+0000","last_change":"2024-09-10T21:24:47.857699+0000","last_active":"2024-09-10T21:47:11.332930+0000","last_peered":"2024-09-10T21:47:11.332930+0000","last_clean":"2024-09-10T21:47:11.332930+0000","last_became_active":"2024-09-10T21:24:47.857429+0000","last_became_peered":"2024-09-10T21:24:47.857429+0000","last_unstale":"2024-09-10T21:47:11.332930+0000","last_undegraded":"2024-09-10T21:47:11.332930+0000","last_fullsized":"2024-09-10T21:47:11.332930+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:07:11.880228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"27'6","reported_seq":42,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.634621+0000","last_change":"2024-09-10T21:24:46.859753+0000","last_active":"2024-09-10T21:46:48.634621+0000","last_peered":"2024-09-10T21:46:48.634621+0000","last_clean":"2024-09-10T21:46:48.634621+0000","last_became_active":"2024-09-10T21:24:46.859299+0000","last_became_peered":"2024-09-10T21:24:46.859299+0000","last_unstale":"2024-09-10T21:46:48.634621+0000","last_undegraded":"2024-09-10T21:46:48.634621+0000","last_fullsized":"2024-09-10T21:46:48.634621+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:06:41.965052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"27'4196","reported_seq":4224,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.388436+0000","last_change":"2024-09-10T21:24:52.064718+0000","last_active":"2024-09-10T21:46:48.388436+0000","last_peered":"2024-09-10T21:46:48.388436+0000","last_clean":"2024-09-10T21:46:48.388436+0000","last_became_active":"2024-09-10T21:24:52.062327+0000","last_became_peered":"2024-09-10T21:24:52.062327+0000","last_unstale":"2024-09-10T21:46:48.388436+0000","last_undegraded":"2024-09-10T21:46:48.388436+0000","last_fullsized":"2024-09-10T21:46:48.388436+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":4196,"log_dups_size":0,"ondisk_log_size":4196,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:43:17.173095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20531,"num_write_kb":446498,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"27'18","reported_seq":52,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.334694+0000","last_change":"2024-09-10T21:24:47.870766+0000","last_active":"2024-09-10T21:47:11.334694+0000","last_peered":"2024-09-10T21:47:11.334694+0000","last_clean":"2024-09-10T21:47:11.334694+0000","last_became_active":"2024-09-10T21:24:47.870276+0000","last_became_peered":"2024-09-10T21:24:47.870276+0000","last_unstale":"2024-09-10T21:47:11.334694+0000","last_undegraded":"2024-09-10T21:47:11.334694+0000","last_fullsized":"2024-09-10T21:47:11.334694+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:54:38.400877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"27'368","reported_seq":403,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:28.433029+0000","last_change":"2024-09-10T21:24:46.867790+0000","last_active":"2024-09-10T21:46:28.433029+0000","last_peered":"2024-09-10T21:46:28.433029+0000","last_clean":"2024-09-10T21:46:28.433029+0000","last_became_active":"2024-09-10T21:24:46.866899+0000","last_became_peered":"2024-09-10T21:24:46.866899+0000","last_unstale":"2024-09-10T21:46:28.433029+0000","last_undegraded":"2024-09-10T21:46:28.433029+0000","last_fullsized":"2024-09-10T21:46:28.433029+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":368,"log_dups_size":0,"ondisk_log_size":368,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:25:04.923730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194976,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":367,"num_write_kb":4285,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"27'3971","reported_seq":3999,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.743646+0000","last_change":"2024-09-10T21:24:51.899803+0000","last_active":"2024-09-10T21:46:48.743646+0000","last_peered":"2024-09-10T21:46:48.743646+0000","last_clean":"2024-09-10T21:46:48.743646+0000","last_became_active":"2024-09-10T21:24:51.895398+0000","last_became_peered":"2024-09-10T21:24:51.895398+0000","last_unstale":"2024-09-10T21:46:48.743646+0000","last_undegraded":"2024-09-10T21:46:48.743646+0000","last_fullsized":"2024-09-10T21:46:48.743646+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":3971,"log_dups_size":0,"ondisk_log_size":3971,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:52:34.689860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19337,"num_write_kb":445978,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"27'22","reported_seq":56,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.334366+0000","last_change":"2024-09-10T21:24:47.857544+0000","last_active":"2024-09-10T21:47:11.334366+0000","last_peered":"2024-09-10T21:47:11.334366+0000","last_clean":"2024-09-10T21:47:11.334366+0000","last_became_active":"2024-09-10T21:24:47.857284+0000","last_became_peered":"2024-09-10T21:24:47.857284+0000","last_unstale":"2024-09-10T21:47:11.334366+0000","last_undegraded":"2024-09-10T21:47:11.334366+0000","last_fullsized":"2024-09-10T21:47:11.334366+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:10:22.747606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"27'32","reported_seq":67,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.672745+0000","last_change":"2024-09-10T21:24:46.859633+0000","last_active":"2024-09-10T21:46:48.672745+0000","last_peered":"2024-09-10T21:46:48.672745+0000","last_clean":"2024-09-10T21:46:48.672745+0000","last_became_active":"2024-09-10T21:24:46.859332+0000","last_became_peered":"2024-09-10T21:24:46.859332+0000","last_unstale":"2024-09-10T21:46:48.672745+0000","last_undegraded":"2024-09-10T21:46:48.672745+0000","last_fullsized":"2024-09-10T21:46:48.672745+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:56:46.975225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":33,"num_write_kb":30,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"27'4136","reported_seq":4164,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.629023+0000","last_change":"2024-09-10T21:24:51.899100+0000","last_active":"2024-09-10T21:46:48.629023+0000","last_peered":"2024-09-10T21:46:48.629023+0000","last_clean":"2024-09-10T21:46:48.629023+0000","last_became_active":"2024-09-10T21:24:51.898642+0000","last_became_peered":"2024-09-10T21:24:51.898642+0000","last_unstale":"2024-09-10T21:46:48.629023+0000","last_undegraded":"2024-09-10T21:46:48.629023+0000","last_fullsized":"2024-09-10T21:46:48.629023+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":4136,"log_dups_size":0,"ondisk_log_size":4136,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:06.394081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20561,"num_write_kb":446825,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"27'23","reported_seq":57,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.345555+0000","last_change":"2024-09-10T21:24:47.867890+0000","last_active":"2024-09-10T21:47:11.345555+0000","last_peered":"2024-09-10T21:47:11.345555+0000","last_clean":"2024-09-10T21:47:11.345555+0000","last_became_active":"2024-09-10T21:24:47.865016+0000","last_became_peered":"2024-09-10T21:24:47.865016+0000","last_unstale":"2024-09-10T21:47:11.345555+0000","last_undegraded":"2024-09-10T21:47:11.345555+0000","last_fullsized":"2024-09-10T21:47:11.345555+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:53:09.082512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"27'3","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.554054+0000","last_change":"2024-09-10T21:24:46.861400+0000","last_active":"2024-09-10T21:46:48.554054+0000","last_peered":"2024-09-10T21:46:48.554054+0000","last_clean":"2024-09-10T21:46:48.554054+0000","last_became_active":"2024-09-10T21:24:46.861136+0000","last_became_peered":"2024-09-10T21:24:46.861136+0000","last_unstale":"2024-09-10T21:46:48.554054+0000","last_undegraded":"2024-09-10T21:46:48.554054+0000","last_fullsized":"2024-09-10T21:46:48.554054+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:35:31.682448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"27'4156","reported_seq":4184,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.591271+0000","last_change":"2024-09-10T21:24:51.907850+0000","last_active":"2024-09-10T21:46:48.591271+0000","last_peered":"2024-09-10T21:46:48.591271+0000","last_clean":"2024-09-10T21:46:48.591271+0000","last_became_active":"2024-09-10T21:24:51.907062+0000","last_became_peered":"2024-09-10T21:24:51.907062+0000","last_unstale":"2024-09-10T21:46:48.591271+0000","last_undegraded":"2024-09-10T21:46:48.591271+0000","last_fullsized":"2024-09-10T21:46:48.591271+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":4156,"log_dups_size":0,"ondisk_log_size":4156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:32:52.519627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20153,"num_write_kb":457162,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"27'25","reported_seq":58,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.346624+0000","last_change":"2024-09-10T21:24:47.861800+0000","last_active":"2024-09-10T21:47:11.346624+0000","last_peered":"2024-09-10T21:47:11.346624+0000","last_clean":"2024-09-10T21:47:11.346624+0000","last_became_active":"2024-09-10T21:24:47.861498+0000","last_became_peered":"2024-09-10T21:24:47.861498+0000","last_unstale":"2024-09-10T21:47:11.346624+0000","last_undegraded":"2024-09-10T21:47:11.346624+0000","last_fullsized":"2024-09-10T21:47:11.346624+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:07:21.128033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"27'3","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:36.356663+0000","last_change":"2024-09-10T21:24:46.871677+0000","last_active":"2024-09-10T21:46:36.356663+0000","last_peered":"2024-09-10T21:46:36.356663+0000","last_clean":"2024-09-10T21:46:36.356663+0000","last_became_active":"2024-09-10T21:24:46.871367+0000","last_became_peered":"2024-09-10T21:24:46.871367+0000","last_unstale":"2024-09-10T21:46:36.356663+0000","last_undegraded":"2024-09-10T21:46:36.356663+0000","last_fullsized":"2024-09-10T21:46:36.356663+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:40:33.632846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"27'3798","reported_seq":3826,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.674934+0000","last_change":"2024-09-10T21:24:52.068637+0000","last_active":"2024-09-10T21:46:48.674934+0000","last_peered":"2024-09-10T21:46:48.674934+0000","last_clean":"2024-09-10T21:46:48.674934+0000","last_became_active":"2024-09-10T21:24:52.068348+0000","last_became_peered":"2024-09-10T21:24:52.068348+0000","last_unstale":"2024-09-10T21:46:48.674934+0000","last_undegraded":"2024-09-10T21:46:48.674934+0000","last_fullsized":"2024-09-10T21:46:48.674934+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":3798,"log_dups_size":0,"ondisk_log_size":3798,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:43:17.173095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18329,"num_write_kb":408933,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"27'5643","reported_seq":5673,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.715900+0000","last_change":"2024-09-10T21:24:52.067732+0000","last_active":"2024-09-10T21:46:48.715900+0000","last_peered":"2024-09-10T21:46:48.715900+0000","last_clean":"2024-09-10T21:46:48.715900+0000","last_became_active":"2024-09-10T21:24:52.067330+0000","last_became_peered":"2024-09-10T21:24:52.067330+0000","last_unstale":"2024-09-10T21:46:48.715900+0000","last_undegraded":"2024-09-10T21:46:48.715900+0000","last_fullsized":"2024-09-10T21:46:48.715900+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":5643,"log_dups_size":0,"ondisk_log_size":5643,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:03:41.110214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27908,"num_write_kb":566060,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"27'20","reported_seq":53,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.330166+0000","last_change":"2024-09-10T21:24:47.862774+0000","last_active":"2024-09-10T21:47:11.330166+0000","last_peered":"2024-09-10T21:47:11.330166+0000","last_clean":"2024-09-10T21:47:11.330166+0000","last_became_active":"2024-09-10T21:24:47.862556+0000","last_became_peered":"2024-09-10T21:24:47.862556+0000","last_unstale":"2024-09-10T21:47:11.330166+0000","last_undegraded":"2024-09-10T21:47:11.330166+0000","last_fullsized":"2024-09-10T21:47:11.330166+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:17:52.343553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"27'2","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:34.118680+0000","last_change":"2024-09-10T21:24:46.862847+0000","last_active":"2024-09-10T21:46:34.118680+0000","last_peered":"2024-09-10T21:46:34.118680+0000","last_clean":"2024-09-10T21:46:34.118680+0000","last_became_active":"2024-09-10T21:24:46.862540+0000","last_became_peered":"2024-09-10T21:24:46.862540+0000","last_unstale":"2024-09-10T21:46:34.118680+0000","last_undegraded":"2024-09-10T21:46:34.118680+0000","last_fullsized":"2024-09-10T21:46:34.118680+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:52:31.628568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"27'5506","reported_seq":5535,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.736687+0000","last_change":"2024-09-10T21:24:51.901224+0000","last_active":"2024-09-10T21:46:48.736687+0000","last_peered":"2024-09-10T21:46:48.736687+0000","last_clean":"2024-09-10T21:46:48.736687+0000","last_became_active":"2024-09-10T21:24:51.900610+0000","last_became_peered":"2024-09-10T21:24:51.900610+0000","last_unstale":"2024-09-10T21:46:48.736687+0000","last_undegraded":"2024-09-10T21:46:48.736687+0000","last_fullsized":"2024-09-10T21:46:48.736687+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":5506,"log_dups_size":0,"ondisk_log_size":5506,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:52:34.689860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27800,"num_write_kb":594222,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"27'26","reported_seq":60,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.328339+0000","last_change":"2024-09-10T21:24:47.861255+0000","last_active":"2024-09-10T21:47:11.328339+0000","last_peered":"2024-09-10T21:47:11.328339+0000","last_clean":"2024-09-10T21:47:11.328339+0000","last_became_active":"2024-09-10T21:24:47.860930+0000","last_became_peered":"2024-09-10T21:24:47.860930+0000","last_unstale":"2024-09-10T21:47:11.328339+0000","last_undegraded":"2024-09-10T21:47:11.328339+0000","last_fullsized":"2024-09-10T21:47:11.328339+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:37:51.784258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"27'3","reported_seq":39,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.632694+0000","last_change":"2024-09-10T21:24:46.858561+0000","last_active":"2024-09-10T21:46:48.632694+0000","last_peered":"2024-09-10T21:46:48.632694+0000","last_clean":"2024-09-10T21:46:48.632694+0000","last_became_active":"2024-09-10T21:24:46.858265+0000","last_became_peered":"2024-09-10T21:24:46.858265+0000","last_unstale":"2024-09-10T21:46:48.632694+0000","last_undegraded":"2024-09-10T21:46:48.632694+0000","last_fullsized":"2024-09-10T21:46:48.632694+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:49:00.454609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"27'6","reported_seq":41,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.657882+0000","last_change":"2024-09-10T21:24:46.856886+0000","last_active":"2024-09-10T21:46:48.657882+0000","last_peered":"2024-09-10T21:46:48.657882+0000","last_clean":"2024-09-10T21:46:48.657882+0000","last_became_active":"2024-09-10T21:24:46.856506+0000","last_became_peered":"2024-09-10T21:24:46.856506+0000","last_unstale":"2024-09-10T21:46:48.657882+0000","last_undegraded":"2024-09-10T21:46:48.657882+0000","last_fullsized":"2024-09-10T21:46:48.657882+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:56:46.975225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"27'19","reported_seq":52,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.337444+0000","last_change":"2024-09-10T21:24:47.848602+0000","last_active":"2024-09-10T21:47:11.337444+0000","last_peered":"2024-09-10T21:47:11.337444+0000","last_clean":"2024-09-10T21:47:11.337444+0000","last_became_active":"2024-09-10T21:24:47.848160+0000","last_became_peered":"2024-09-10T21:24:47.848160+0000","last_unstale":"2024-09-10T21:47:11.337444+0000","last_undegraded":"2024-09-10T21:47:11.337444+0000","last_fullsized":"2024-09-10T21:47:11.337444+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:32:50.027827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"27'4174","reported_seq":4201,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.382991+0000","last_change":"2024-09-10T21:24:51.903553+0000","last_active":"2024-09-10T21:46:48.382991+0000","last_peered":"2024-09-10T21:46:48.382991+0000","last_clean":"2024-09-10T21:46:48.382991+0000","last_became_active":"2024-09-10T21:24:51.903120+0000","last_became_peered":"2024-09-10T21:24:51.903120+0000","last_unstale":"2024-09-10T21:46:48.382991+0000","last_undegraded":"2024-09-10T21:46:48.382991+0000","last_fullsized":"2024-09-10T21:46:48.382991+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":4174,"log_dups_size":0,"ondisk_log_size":4174,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:58.240380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19938,"num_write_kb":435851,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"27'678","reported_seq":713,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.597988+0000","last_change":"2024-09-10T21:24:46.853550+0000","last_active":"2024-09-10T21:46:48.597988+0000","last_peered":"2024-09-10T21:46:48.597988+0000","last_clean":"2024-09-10T21:46:48.597988+0000","last_became_active":"2024-09-10T21:24:46.852204+0000","last_became_peered":"2024-09-10T21:24:46.852204+0000","last_unstale":"2024-09-10T21:46:48.597988+0000","last_undegraded":"2024-09-10T21:46:48.597988+0000","last_fullsized":"2024-09-10T21:46:48.597988+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":678,"log_dups_size":0,"ondisk_log_size":678,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:06:37.669444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":672,"num_write_kb":4259,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"27'18","reported_seq":51,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.327062+0000","last_change":"2024-09-10T21:24:47.849151+0000","last_active":"2024-09-10T21:47:11.327062+0000","last_peered":"2024-09-10T21:47:11.327062+0000","last_clean":"2024-09-10T21:47:11.327062+0000","last_became_active":"2024-09-10T21:24:47.848851+0000","last_became_peered":"2024-09-10T21:24:47.848851+0000","last_unstale":"2024-09-10T21:47:11.327062+0000","last_undegraded":"2024-09-10T21:47:11.327062+0000","last_fullsized":"2024-09-10T21:47:11.327062+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:39:25.828558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"27'4902","reported_seq":4929,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.681160+0000","last_change":"2024-09-10T21:24:51.907549+0000","last_active":"2024-09-10T21:46:48.681160+0000","last_peered":"2024-09-10T21:46:48.681160+0000","last_clean":"2024-09-10T21:46:48.681160+0000","last_became_active":"2024-09-10T21:24:51.907082+0000","last_became_peered":"2024-09-10T21:24:51.907082+0000","last_unstale":"2024-09-10T21:46:48.681160+0000","last_undegraded":"2024-09-10T21:46:48.681160+0000","last_fullsized":"2024-09-10T21:46:48.681160+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":4902,"log_dups_size":0,"ondisk_log_size":4902,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:44:30.813754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24324,"num_write_kb":547512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"27'24","reported_seq":58,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.351212+0000","last_change":"2024-09-10T21:24:47.857804+0000","last_active":"2024-09-10T21:47:11.351212+0000","last_peered":"2024-09-10T21:47:11.351212+0000","last_clean":"2024-09-10T21:47:11.351212+0000","last_became_active":"2024-09-10T21:24:47.857501+0000","last_became_peered":"2024-09-10T21:24:47.857501+0000","last_unstale":"2024-09-10T21:47:11.351212+0000","last_undegraded":"2024-09-10T21:47:11.351212+0000","last_fullsized":"2024-09-10T21:47:11.351212+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:19:20.807422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"27'25","reported_seq":60,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.345524+0000","last_change":"2024-09-10T21:24:47.867803+0000","last_active":"2024-09-10T21:47:11.345524+0000","last_peered":"2024-09-10T21:47:11.345524+0000","last_clean":"2024-09-10T21:47:11.345524+0000","last_became_active":"2024-09-10T21:24:47.867504+0000","last_became_peered":"2024-09-10T21:24:47.867504+0000","last_unstale":"2024-09-10T21:47:11.345524+0000","last_undegraded":"2024-09-10T21:47:11.345524+0000","last_fullsized":"2024-09-10T21:47:11.345524+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:53:30.420454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"27'26","reported_seq":60,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.334694+0000","last_change":"2024-09-10T21:24:47.868901+0000","last_active":"2024-09-10T21:47:11.334694+0000","last_peered":"2024-09-10T21:47:11.334694+0000","last_clean":"2024-09-10T21:47:11.334694+0000","last_became_active":"2024-09-10T21:24:47.868168+0000","last_became_peered":"2024-09-10T21:24:47.868168+0000","last_unstale":"2024-09-10T21:47:11.334694+0000","last_undegraded":"2024-09-10T21:47:11.334694+0000","last_fullsized":"2024-09-10T21:47:11.334694+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:06:21.461089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"27'16","reported_seq":51,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.331166+0000","last_change":"2024-09-10T21:24:47.864561+0000","last_active":"2024-09-10T21:47:11.331166+0000","last_peered":"2024-09-10T21:47:11.331166+0000","last_clean":"2024-09-10T21:47:11.331166+0000","last_became_active":"2024-09-10T21:24:47.864100+0000","last_became_peered":"2024-09-10T21:24:47.864100+0000","last_unstale":"2024-09-10T21:47:11.331166+0000","last_undegraded":"2024-09-10T21:47:11.331166+0000","last_fullsized":"2024-09-10T21:47:11.331166+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:34:13.053494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"27'14","reported_seq":49,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.340508+0000","last_change":"2024-09-10T21:24:47.864303+0000","last_active":"2024-09-10T21:47:11.340508+0000","last_peered":"2024-09-10T21:47:11.340508+0000","last_clean":"2024-09-10T21:47:11.340508+0000","last_became_active":"2024-09-10T21:24:47.863970+0000","last_became_peered":"2024-09-10T21:24:47.863970+0000","last_unstale":"2024-09-10T21:47:11.340508+0000","last_undegraded":"2024-09-10T21:47:11.340508+0000","last_fullsized":"2024-09-10T21:47:11.340508+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:46:36.918539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"27'18","reported_seq":52,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.329189+0000","last_change":"2024-09-10T21:24:47.860734+0000","last_active":"2024-09-10T21:47:11.329189+0000","last_peered":"2024-09-10T21:47:11.329189+0000","last_clean":"2024-09-10T21:47:11.329189+0000","last_became_active":"2024-09-10T21:24:47.860471+0000","last_became_peered":"2024-09-10T21:24:47.860471+0000","last_unstale":"2024-09-10T21:47:11.329189+0000","last_undegraded":"2024-09-10T21:47:11.329189+0000","last_fullsized":"2024-09-10T21:47:11.329189+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:54:13.200082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"27'27","reported_seq":61,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.358154+0000","last_change":"2024-09-10T21:24:47.872504+0000","last_active":"2024-09-10T21:47:11.358154+0000","last_peered":"2024-09-10T21:47:11.358154+0000","last_clean":"2024-09-10T21:47:11.358154+0000","last_became_active":"2024-09-10T21:24:47.872218+0000","last_became_peered":"2024-09-10T21:24:47.872218+0000","last_unstale":"2024-09-10T21:47:11.358154+0000","last_undegraded":"2024-09-10T21:47:11.358154+0000","last_fullsized":"2024-09-10T21:47:11.358154+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:21:32.400520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"27'14","reported_seq":49,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.310878+0000","last_change":"2024-09-10T21:24:47.862167+0000","last_active":"2024-09-10T21:47:11.310878+0000","last_peered":"2024-09-10T21:47:11.310878+0000","last_clean":"2024-09-10T21:47:11.310878+0000","last_became_active":"2024-09-10T21:24:47.859649+0000","last_became_peered":"2024-09-10T21:24:47.859649+0000","last_unstale":"2024-09-10T21:47:11.310878+0000","last_undegraded":"2024-09-10T21:47:11.310878+0000","last_fullsized":"2024-09-10T21:47:11.310878+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:42:59.204284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"27'18","reported_seq":39,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.329156+0000","last_change":"2024-09-10T21:25:28.508979+0000","last_active":"2024-09-10T21:47:11.329156+0000","last_peered":"2024-09-10T21:47:11.329156+0000","last_clean":"2024-09-10T21:47:11.329156+0000","last_became_active":"2024-09-10T21:25:28.508684+0000","last_became_peered":"2024-09-10T21:25:28.508684+0000","last_unstale":"2024-09-10T21:47:11.329156+0000","last_undegraded":"2024-09-10T21:47:11.329156+0000","last_fullsized":"2024-09-10T21:47:11.329156+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:20:59.362941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.29","version":"27'23","reported_seq":43,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.335919+0000","last_change":"2024-09-10T21:25:28.512634+0000","last_active":"2024-09-10T21:47:11.335919+0000","last_peered":"2024-09-10T21:47:11.335919+0000","last_clean":"2024-09-10T21:47:11.335919+0000","last_became_active":"2024-09-10T21:25:28.512230+0000","last_became_peered":"2024-09-10T21:25:28.512230+0000","last_unstale":"2024-09-10T21:47:11.335919+0000","last_undegraded":"2024-09-10T21:47:11.335919+0000","last_fullsized":"2024-09-10T21:47:11.335919+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:17:52.343553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2a","version":"27'22","reported_seq":55,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.345833+0000","last_change":"2024-09-10T21:24:47.872848+0000","last_active":"2024-09-10T21:47:11.345833+0000","last_peered":"2024-09-10T21:47:11.345833+0000","last_clean":"2024-09-10T21:47:11.345833+0000","last_became_active":"2024-09-10T21:24:47.868671+0000","last_became_peered":"2024-09-10T21:24:47.868671+0000","last_unstale":"2024-09-10T21:47:11.345833+0000","last_undegraded":"2024-09-10T21:47:11.345833+0000","last_fullsized":"2024-09-10T21:47:11.345833+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:33:59.291647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"27'28","reported_seq":62,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.339893+0000","last_change":"2024-09-10T21:24:47.857464+0000","last_active":"2024-09-10T21:47:11.339893+0000","last_peered":"2024-09-10T21:47:11.339893+0000","last_clean":"2024-09-10T21:47:11.339893+0000","last_became_active":"2024-09-10T21:24:47.857116+0000","last_became_peered":"2024-09-10T21:24:47.857116+0000","last_unstale":"2024-09-10T21:47:11.339893+0000","last_undegraded":"2024-09-10T21:47:11.339893+0000","last_fullsized":"2024-09-10T21:47:11.339893+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:56:43.269026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"27'19","reported_seq":52,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.344737+0000","last_change":"2024-09-10T21:24:47.871619+0000","last_active":"2024-09-10T21:47:11.344737+0000","last_peered":"2024-09-10T21:47:11.344737+0000","last_clean":"2024-09-10T21:47:11.344737+0000","last_became_active":"2024-09-10T21:24:47.871350+0000","last_became_peered":"2024-09-10T21:24:47.871350+0000","last_unstale":"2024-09-10T21:47:11.344737+0000","last_undegraded":"2024-09-10T21:47:11.344737+0000","last_fullsized":"2024-09-10T21:47:11.344737+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:41.305116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"27'20","reported_seq":54,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.332528+0000","last_change":"2024-09-10T21:24:47.861352+0000","last_active":"2024-09-10T21:47:11.332528+0000","last_peered":"2024-09-10T21:47:11.332528+0000","last_clean":"2024-09-10T21:47:11.332528+0000","last_became_active":"2024-09-10T21:24:47.861007+0000","last_became_peered":"2024-09-10T21:24:47.861007+0000","last_unstale":"2024-09-10T21:47:11.332528+0000","last_undegraded":"2024-09-10T21:47:11.332528+0000","last_fullsized":"2024-09-10T21:47:11.332528+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:25.273458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"27'27","reported_seq":61,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.345633+0000","last_change":"2024-09-10T21:24:47.864205+0000","last_active":"2024-09-10T21:47:11.345633+0000","last_peered":"2024-09-10T21:47:11.345633+0000","last_clean":"2024-09-10T21:47:11.345633+0000","last_became_active":"2024-09-10T21:24:47.863932+0000","last_became_peered":"2024-09-10T21:24:47.863932+0000","last_unstale":"2024-09-10T21:47:11.345633+0000","last_undegraded":"2024-09-10T21:47:11.345633+0000","last_fullsized":"2024-09-10T21:47:11.345633+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:53:17.669629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"27'21","reported_seq":54,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.349081+0000","last_change":"2024-09-10T21:24:47.866225+0000","last_active":"2024-09-10T21:47:11.349081+0000","last_peered":"2024-09-10T21:47:11.349081+0000","last_clean":"2024-09-10T21:47:11.349081+0000","last_became_active":"2024-09-10T21:24:47.865587+0000","last_became_peered":"2024-09-10T21:24:47.865587+0000","last_unstale":"2024-09-10T21:47:11.349081+0000","last_undegraded":"2024-09-10T21:47:11.349081+0000","last_fullsized":"2024-09-10T21:47:11.349081+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:49:26.113068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"27'22","reported_seq":55,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.342691+0000","last_change":"2024-09-10T21:24:47.872669+0000","last_active":"2024-09-10T21:47:11.342691+0000","last_peered":"2024-09-10T21:47:11.342691+0000","last_clean":"2024-09-10T21:47:11.342691+0000","last_became_active":"2024-09-10T21:24:47.865837+0000","last_became_peered":"2024-09-10T21:24:47.865837+0000","last_unstale":"2024-09-10T21:47:11.342691+0000","last_undegraded":"2024-09-10T21:47:11.342691+0000","last_fullsized":"2024-09-10T21:47:11.342691+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:53:09.082512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"27'21","reported_seq":54,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.318871+0000","last_change":"2024-09-10T21:24:47.846567+0000","last_active":"2024-09-10T21:47:11.318871+0000","last_peered":"2024-09-10T21:47:11.318871+0000","last_clean":"2024-09-10T21:47:11.318871+0000","last_became_active":"2024-09-10T21:24:47.846283+0000","last_became_peered":"2024-09-10T21:24:47.846283+0000","last_unstale":"2024-09-10T21:47:11.318871+0000","last_undegraded":"2024-09-10T21:47:11.318871+0000","last_fullsized":"2024-09-10T21:47:11.318871+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:42:15.320518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"27'23","reported_seq":57,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.319292+0000","last_change":"2024-09-10T21:24:47.844888+0000","last_active":"2024-09-10T21:47:11.319292+0000","last_peered":"2024-09-10T21:47:11.319292+0000","last_clean":"2024-09-10T21:47:11.319292+0000","last_became_active":"2024-09-10T21:24:47.844517+0000","last_became_peered":"2024-09-10T21:24:47.844517+0000","last_unstale":"2024-09-10T21:47:11.319292+0000","last_undegraded":"2024-09-10T21:47:11.319292+0000","last_fullsized":"2024-09-10T21:47:11.319292+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:00:14.988282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"27'25","reported_seq":59,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.344844+0000","last_change":"2024-09-10T21:24:47.862746+0000","last_active":"2024-09-10T21:47:11.344844+0000","last_peered":"2024-09-10T21:47:11.344844+0000","last_clean":"2024-09-10T21:47:11.344844+0000","last_became_active":"2024-09-10T21:24:47.862185+0000","last_became_peered":"2024-09-10T21:24:47.862185+0000","last_unstale":"2024-09-10T21:47:11.344844+0000","last_undegraded":"2024-09-10T21:47:11.344844+0000","last_fullsized":"2024-09-10T21:47:11.344844+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:06:21.461089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"27'17","reported_seq":51,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.349979+0000","last_change":"2024-09-10T21:24:47.867727+0000","last_active":"2024-09-10T21:47:11.349979+0000","last_peered":"2024-09-10T21:47:11.349979+0000","last_clean":"2024-09-10T21:47:11.349979+0000","last_became_active":"2024-09-10T21:24:47.861045+0000","last_became_peered":"2024-09-10T21:24:47.861045+0000","last_unstale":"2024-09-10T21:47:11.349979+0000","last_undegraded":"2024-09-10T21:47:11.349979+0000","last_fullsized":"2024-09-10T21:47:11.349979+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:08:30.065968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"27'17","reported_seq":50,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.347800+0000","last_change":"2024-09-10T21:24:47.867410+0000","last_active":"2024-09-10T21:47:11.347800+0000","last_peered":"2024-09-10T21:47:11.347800+0000","last_clean":"2024-09-10T21:47:11.347800+0000","last_became_active":"2024-09-10T21:24:47.867113+0000","last_became_peered":"2024-09-10T21:24:47.867113+0000","last_unstale":"2024-09-10T21:47:11.347800+0000","last_undegraded":"2024-09-10T21:47:11.347800+0000","last_fullsized":"2024-09-10T21:47:11.347800+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:49:53.502472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"27'28","reported_seq":61,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.358458+0000","last_change":"2024-09-10T21:24:47.867473+0000","last_active":"2024-09-10T21:47:11.358458+0000","last_peered":"2024-09-10T21:47:11.358458+0000","last_clean":"2024-09-10T21:47:11.358458+0000","last_became_active":"2024-09-10T21:24:47.867126+0000","last_became_peered":"2024-09-10T21:24:47.867126+0000","last_unstale":"2024-09-10T21:47:11.358458+0000","last_undegraded":"2024-09-10T21:47:11.358458+0000","last_fullsized":"2024-09-10T21:47:11.358458+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:54:32.307507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"27'26","reported_seq":60,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.353536+0000","last_change":"2024-09-10T21:24:47.865450+0000","last_active":"2024-09-10T21:47:11.353536+0000","last_peered":"2024-09-10T21:47:11.353536+0000","last_clean":"2024-09-10T21:47:11.353536+0000","last_became_active":"2024-09-10T21:24:47.865093+0000","last_became_peered":"2024-09-10T21:24:47.865093+0000","last_unstale":"2024-09-10T21:47:11.353536+0000","last_undegraded":"2024-09-10T21:47:11.353536+0000","last_fullsized":"2024-09-10T21:47:11.353536+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:12:18.854783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"27'15","reported_seq":49,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.344819+0000","last_change":"2024-09-10T21:24:47.868093+0000","last_active":"2024-09-10T21:47:11.344819+0000","last_peered":"2024-09-10T21:47:11.344819+0000","last_clean":"2024-09-10T21:47:11.344819+0000","last_became_active":"2024-09-10T21:24:47.867782+0000","last_became_peered":"2024-09-10T21:24:47.867782+0000","last_unstale":"2024-09-10T21:47:11.344819+0000","last_undegraded":"2024-09-10T21:47:11.344819+0000","last_fullsized":"2024-09-10T21:47:11.344819+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:34:13.053494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"27'13","reported_seq":46,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.335909+0000","last_change":"2024-09-10T21:24:47.862204+0000","last_active":"2024-09-10T21:47:11.335909+0000","last_peered":"2024-09-10T21:47:11.335909+0000","last_clean":"2024-09-10T21:47:11.335909+0000","last_became_active":"2024-09-10T21:24:47.861434+0000","last_became_peered":"2024-09-10T21:24:47.861434+0000","last_unstale":"2024-09-10T21:47:11.335909+0000","last_undegraded":"2024-09-10T21:47:11.335909+0000","last_fullsized":"2024-09-10T21:47:11.335909+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:50:03.245285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"27'14","reported_seq":46,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.343049+0000","last_change":"2024-09-10T21:24:47.869248+0000","last_active":"2024-09-10T21:47:11.343049+0000","last_peered":"2024-09-10T21:47:11.343049+0000","last_clean":"2024-09-10T21:47:11.343049+0000","last_became_active":"2024-09-10T21:24:47.868503+0000","last_became_peered":"2024-09-10T21:24:47.868503+0000","last_unstale":"2024-09-10T21:47:11.343049+0000","last_undegraded":"2024-09-10T21:47:11.343049+0000","last_fullsized":"2024-09-10T21:47:11.343049+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:51:31.417845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"27'15","reported_seq":48,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.331365+0000","last_change":"2024-09-10T21:24:47.864802+0000","last_active":"2024-09-10T21:47:11.331365+0000","last_peered":"2024-09-10T21:47:11.331365+0000","last_clean":"2024-09-10T21:47:11.331365+0000","last_became_active":"2024-09-10T21:24:47.863976+0000","last_became_peered":"2024-09-10T21:24:47.863976+0000","last_unstale":"2024-09-10T21:47:11.331365+0000","last_undegraded":"2024-09-10T21:47:11.331365+0000","last_fullsized":"2024-09-10T21:47:11.331365+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:55:08.544075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"27'13","reported_seq":48,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.347483+0000","last_change":"2024-09-10T21:24:47.873364+0000","last_active":"2024-09-10T21:47:11.347483+0000","last_peered":"2024-09-10T21:47:11.347483+0000","last_clean":"2024-09-10T21:47:11.347483+0000","last_became_active":"2024-09-10T21:24:47.873069+0000","last_became_peered":"2024-09-10T21:24:47.873069+0000","last_unstale":"2024-09-10T21:47:11.347483+0000","last_undegraded":"2024-09-10T21:47:11.347483+0000","last_fullsized":"2024-09-10T21:47:11.347483+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:38:21.368499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"27'20","reported_seq":55,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.322849+0000","last_change":"2024-09-10T21:24:47.856936+0000","last_active":"2024-09-10T21:47:11.322849+0000","last_peered":"2024-09-10T21:47:11.322849+0000","last_clean":"2024-09-10T21:47:11.322849+0000","last_became_active":"2024-09-10T21:24:47.854713+0000","last_became_peered":"2024-09-10T21:24:47.854713+0000","last_unstale":"2024-09-10T21:47:11.322849+0000","last_undegraded":"2024-09-10T21:47:11.322849+0000","last_fullsized":"2024-09-10T21:47:11.322849+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:46:36.918539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"27'10","reported_seq":44,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.320206+0000","last_change":"2024-09-10T21:24:47.846492+0000","last_active":"2024-09-10T21:47:11.320206+0000","last_peered":"2024-09-10T21:47:11.320206+0000","last_clean":"2024-09-10T21:47:11.320206+0000","last_became_active":"2024-09-10T21:24:47.846227+0000","last_became_peered":"2024-09-10T21:24:47.846227+0000","last_unstale":"2024-09-10T21:47:11.320206+0000","last_undegraded":"2024-09-10T21:47:11.320206+0000","last_fullsized":"2024-09-10T21:47:11.320206+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:13:58.432819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"27'15","reported_seq":50,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.347364+0000","last_change":"2024-09-10T21:24:47.871469+0000","last_active":"2024-09-10T21:47:11.347364+0000","last_peered":"2024-09-10T21:47:11.347364+0000","last_clean":"2024-09-10T21:47:11.347364+0000","last_became_active":"2024-09-10T21:24:47.871177+0000","last_became_peered":"2024-09-10T21:24:47.871177+0000","last_unstale":"2024-09-10T21:47:11.347364+0000","last_undegraded":"2024-09-10T21:47:11.347364+0000","last_fullsized":"2024-09-10T21:47:11.347364+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:19:24.226916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":723657,"num_write_kb":15994026,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":148509,"ondisk_log_size":148509,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1239,"ondisk_log_size":1239,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":14991163,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":48,"num_read_kb":119,"num_write":3049,"num_write_kb":15770,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":18464768,"data_stored":29982326,"data_compressed":2564296,"data_compressed_allocated":3506176,"data_compressed_original":15089664,"omap_allocated":0,"internal_metadata":0},"log_size":2758,"ondisk_log_size":2758,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":66932,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509714,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":739600,"kb_used_data":7424,"kb_used_omap":1,"kb_used_meta":732158,"kb_avail":93632240,"statfs":{"total":96636764160,"available":95879413760,"internally_reserved":0,"allocated":7602176,"data_stored":11412495,"data_compressed":806609,"data_compressed_allocated":1097728,"data_compressed_original":5083136,"omap_allocated":1590,"internal_metadata":749730250},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509715,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":807000,"kb_used_data":2120,"kb_used_omap":1,"kb_used_meta":804862,"kb_avail":93564840,"statfs":{"total":96636764160,"available":95810396160,"internally_reserved":0,"allocated":2170880,"data_stored":4311586,"data_compressed":503209,"data_compressed_allocated":692224,"data_compressed_original":3006464,"omap_allocated":1594,"internal_metadata":824179142},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509713,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":573484,"kb_used_data":348,"kb_used_omap":1,"kb_used_meta":573118,"kb_avail":93798356,"statfs":{"total":96636764160,"available":96049516544,"internally_reserved":0,"allocated":356352,"data_stored":207064,"data_compressed":5796,"data_compressed_allocated":8192,"data_compressed_original":28672,"omap_allocated":1595,"internal_metadata":586873285},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509712,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1104044,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":1103742,"kb_avail":93267796,"statfs":{"total":96636764160,"available":95506223104,"internally_reserved":0,"allocated":290816,"data_stored":116770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":1130232261},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542419,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":844396,"kb_used_data":3740,"kb_used_omap":1,"kb_used_meta":840638,"kb_avail":93527444,"statfs":{"total":96636764160,"available":95772102656,"internally_reserved":0,"allocated":3829760,"data_stored":4311040,"data_compressed":154551,"data_compressed_allocated":245760,"data_compressed_original":905216,"omap_allocated":1595,"internal_metadata":860813765},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542414,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1062920,"kb_used_data":4792,"kb_used_omap":1,"kb_used_meta":1058110,"kb_avail":93308920,"statfs":{"total":96636764160,"available":95548334080,"internally_reserved":0,"allocated":4907008,"data_stored":8596344,"data_compressed":841246,"data_compressed_allocated":1130496,"data_compressed_original":4997120,"omap_allocated":1593,"internal_metadata":1083505095},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542420,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":711220,"kb_used_data":1380,"kb_used_omap":1,"kb_used_meta":709822,"kb_avail":93660620,"statfs":{"total":96636764160,"available":95908474880,"internally_reserved":0,"allocated":1413120,"data_stored":3023943,"data_compressed":319817,"data_compressed_allocated":421888,"data_compressed_original":2215936,"omap_allocated":1595,"internal_metadata":726858181},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575122,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":626792,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":626494,"kb_avail":93745048,"statfs":{"total":96636764160,"available":95994929152,"internally_reserved":0,"allocated":286720,"data_stored":116700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":641530309},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33466,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33466,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":1064960,"data_stored":2316965,"data_compressed":286351,"data_compressed_allocated":376832,"data_compressed_original":1642496,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":90454,"data_compressed":5796,"data_compressed_allocated":8192,"data_compressed_original":28672,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4624384,"data_stored":8479734,"data_compressed":841246,"data_compressed_allocated":1130496,"data_compressed_original":4997120,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":3547136,"data_stored":4194430,"data_compressed":154551,"data_compressed_allocated":245760,"data_compressed_original":905216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":1888256,"data_stored":4194976,"data_compressed":503209,"data_compressed_allocated":692224,"data_compressed_original":3006464,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":7254016,"data_stored":10705517,"data_compressed":773143,"data_compressed_allocated":1052672,"data_compressed_original":4509696,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T21:47:19.255 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:47:19.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:19.433+0000 7fa4ee600640 1 -- 172.21.15.17:0/2655009778 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa4e80cdf20 msgr2=0x7fa4e80ce3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:19.439 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:19.433+0000 7fa4ee600640 1 --2- 172.21.15.17:0/2655009778 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa4e80cdf20 0x7fa4e80ce3b0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa4e0009bc0 tx=0x7fa4e001cb10 comp rx=0 tx=0).stop 2024-09-10T21:47:19.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:19.433+0000 7fa4ee600640 1 -- 172.21.15.17:0/2655009778 shutdown_connections 2024-09-10T21:47:19.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:19.433+0000 7fa4ee600640 1 --2- 172.21.15.17:0/2655009778 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa4e80cdf20 0x7fa4e80ce3b0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:19.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:19.433+0000 7fa4ee600640 1 --2- 172.21.15.17:0/2655009778 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa4e8152820 0x7fa4e80b3eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:19.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:19.433+0000 7fa4ee600640 1 -- 172.21.15.17:0/2655009778 >> 172.21.15.17:0/2655009778 conn(0x7fa4e80702f0 msgr2=0x7fa4e8070700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:19.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:19.433+0000 7fa4ee600640 1 -- 172.21.15.17:0/2655009778 shutdown_connections 2024-09-10T21:47:19.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:19.433+0000 7fa4ee600640 1 -- 172.21.15.17:0/2655009778 wait complete. 2024-09-10T21:47:19.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:19.437+0000 7fa4ee600640 1 Processor -- start 2024-09-10T21:47:19.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:19.437+0000 7fa4ee600640 1 -- start start 2024-09-10T21:47:19.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:19.437+0000 7fa4ee600640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa4e80cdf20 0x7fa4e80c6ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:19.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:19.437+0000 7fa4ee600640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa4e8152440 0x7fa4e81f3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:19.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:19.437+0000 7fa4ee600640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa4e8152820 0x7fa4e81f3b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:19.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:19.437+0000 7fa4ee600640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fa4e80cb5d0 con 0x7fa4e8152440 2024-09-10T21:47:19.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:19.437+0000 7fa4ee600640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fa4e80cb450 con 0x7fa4e8152820 2024-09-10T21:47:19.443 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:19.437+0000 7fa4ee600640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7fa4e80c2ff0 con 0x7fa4e80cdf20 2024-09-10T21:47:19.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:19.437+0000 7fa4e6a00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa4e8152440 0x7fa4e81f3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:19.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:19.437+0000 7fa4e6a00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa4e8152440 0x7fa4e81f3650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:34688/0 (socket says 172.21.15.17:34688) 2024-09-10T21:47:19.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:19.437+0000 7fa4e6a00640 1 -- 172.21.15.17:0/1389354254 learned_addr learned my addr 172.21.15.17:0/1389354254 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:47:19.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:19.437+0000 7fa4e6a00640 1 -- 172.21.15.17:0/1389354254 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa4e80cdf20 msgr2=0x7fa4e80c6ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:19.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:19.437+0000 7fa4e7400640 1 --2- 172.21.15.17:0/1389354254 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa4e80cdf20 0x7fa4e80c6ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:19.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:19.437+0000 7fa4e7e00640 1 --2- 172.21.15.17:0/1389354254 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa4e8152820 0x7fa4e81f3b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:19.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:19.437+0000 7fa4e6a00640 1 --2- 172.21.15.17:0/1389354254 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa4e80cdf20 0x7fa4e80c6ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:19.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:19.437+0000 7fa4e6a00640 1 -- 172.21.15.17:0/1389354254 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa4e8152820 msgr2=0x7fa4e81f3b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:19.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:19.437+0000 7fa4e6a00640 1 --2- 172.21.15.17:0/1389354254 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa4e8152820 0x7fa4e81f3b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:19.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:19.437+0000 7fa4e6a00640 1 -- 172.21.15.17:0/1389354254 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4e8159a80 con 0x7fa4e8152440 2024-09-10T21:47:19.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:19.437+0000 7fa4e7400640 1 --2- 172.21.15.17:0/1389354254 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa4e80cdf20 0x7fa4e80c6ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:47:19.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:19.437+0000 7fa4e7e00640 1 --2- 172.21.15.17:0/1389354254 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa4e8152820 0x7fa4e81f3b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:47:19.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:19.437+0000 7fa4e6a00640 1 --2- 172.21.15.17:0/1389354254 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa4e8152440 0x7fa4e81f3650 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fa4d400d980 tx=0x7fa4d400de40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:19.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:19.437+0000 7fa4df400640 1 -- 172.21.15.17:0/1389354254 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4d4017070 con 0x7fa4e8152440 2024-09-10T21:47:19.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:19.437+0000 7fa4df400640 1 -- 172.21.15.17:0/1389354254 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa4d4009d20 con 0x7fa4e8152440 2024-09-10T21:47:19.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:19.437+0000 7fa4df400640 1 -- 172.21.15.17:0/1389354254 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4d40106c0 con 0x7fa4e8152440 2024-09-10T21:47:19.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:19.437+0000 7fa4ee600640 1 -- 172.21.15.17:0/1389354254 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4e8157c40 con 0x7fa4e8152440 2024-09-10T21:47:19.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:19.437+0000 7fa4ee600640 1 -- 172.21.15.17:0/1389354254 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4e80c4fc0 con 0x7fa4e8152440 2024-09-10T21:47:19.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:19.441+0000 7fa4ee600640 1 -- 172.21.15.17:0/1389354254 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4e8158360 con 0x7fa4e8152440 2024-09-10T21:47:19.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:19.441+0000 7fa4df400640 1 -- 172.21.15.17:0/1389354254 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fa4d4010860 con 0x7fa4e8152440 2024-09-10T21:47:19.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:19.441+0000 7fa4df400640 1 --2- 172.21.15.17:0/1389354254 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fa4c8072610 0x7fa4c8074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:19.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:19.441+0000 7fa4df400640 1 -- 172.21.15.17:0/1389354254 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fa4d4093a30 con 0x7fa4e8152440 2024-09-10T21:47:19.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:19.441+0000 7fa4e7400640 1 --2- 172.21.15.17:0/1389354254 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fa4c8072610 0x7fa4c8074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:19.447 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:19.445+0000 7fa4e7400640 1 --2- 172.21.15.17:0/1389354254 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fa4c8072610 0x7fa4c8074ad0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fa4d8001230 tx=0x7fa4d8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:19.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:19.445+0000 7fa4df400640 1 -- 172.21.15.17:0/1389354254 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fa4d405ed00 con 0x7fa4e8152440 2024-09-10T21:47:19.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:19.621+0000 7fa4ee600640 1 -- 172.21.15.17:0/1389354254 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa4e80722d0 con 0x7fa4e8152440 2024-09-10T21:47:19.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:19.629+0000 7fa4df400640 1 -- 172.21.15.17:0/1389354254 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18950 (secure 0 0 0) 0x7fa4d405e600 con 0x7fa4e8152440 2024-09-10T21:47:19.634 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-10T21:47:19.634 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":27,"fsid":"5e2f023f-d6a6-49f4-a256-f4807dce3739","created":"2024-09-10T21:24:21.750712+0000","modified":"2024-09-10T21:25:28.499315+0000","last_up_change":"2024-09-10T21:24:35.877801+0000","last_in_change":"2024-09-10T21:24:25.831127+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:24:35.362670+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T21:24:45.211689+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T21:24:46.209784+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T21:24:49.871300+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0f5fabc0-434e-4066-b430-233969a548f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":4105489867},{"type":"v1","addr":"172.21.15.17:6809","nonce":4105489867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":4105489867},{"type":"v1","addr":"172.21.15.17:6811","nonce":4105489867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":4105489867},{"type":"v1","addr":"172.21.15.17:6815","nonce":4105489867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":4105489867},{"type":"v1","addr":"172.21.15.17:6813","nonce":4105489867}]},"public_addr":"172.21.15.17:6809/4105489867","cluster_addr":"172.21.15.17:6811/4105489867","heartbeat_back_addr":"172.21.15.17:6815/4105489867","heartbeat_front_addr":"172.21.15.17:6813/4105489867","state":["exists","up"]},{"osd":1,"uuid":"f6691cb3-2a91-4c90-a9fd-ece3c11dd6eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":2599842027},{"type":"v1","addr":"172.21.15.17:6825","nonce":2599842027}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":2599842027},{"type":"v1","addr":"172.21.15.17:6827","nonce":2599842027}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":2599842027},{"type":"v1","addr":"172.21.15.17:6831","nonce":2599842027}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":2599842027},{"type":"v1","addr":"172.21.15.17:6829","nonce":2599842027}]},"public_addr":"172.21.15.17:6825/2599842027","cluster_addr":"172.21.15.17:6827/2599842027","heartbeat_back_addr":"172.21.15.17:6831/2599842027","heartbeat_front_addr":"172.21.15.17:6829/2599842027","state":["exists","up"]},{"osd":2,"uuid":"cb61adb0-a216-437b-9598-06fa7c84aa64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":1248243634},{"type":"v1","addr":"172.21.15.17:6817","nonce":1248243634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":1248243634},{"type":"v1","addr":"172.21.15.17:6819","nonce":1248243634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":1248243634},{"type":"v1","addr":"172.21.15.17:6823","nonce":1248243634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":1248243634},{"type":"v1","addr":"172.21.15.17:6821","nonce":1248243634}]},"public_addr":"172.21.15.17:6817/1248243634","cluster_addr":"172.21.15.17:6819/1248243634","heartbeat_back_addr":"172.21.15.17:6823/1248243634","heartbeat_front_addr":"172.21.15.17:6821/1248243634","state":["exists","up"]},{"osd":3,"uuid":"6d2da26a-e0bf-47dc-b020-af86e6bcfccc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":2314489522},{"type":"v1","addr":"172.21.15.17:6801","nonce":2314489522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":2314489522},{"type":"v1","addr":"172.21.15.17:6803","nonce":2314489522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":2314489522},{"type":"v1","addr":"172.21.15.17:6807","nonce":2314489522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":2314489522},{"type":"v1","addr":"172.21.15.17:6805","nonce":2314489522}]},"public_addr":"172.21.15.17:6801/2314489522","cluster_addr":"172.21.15.17:6803/2314489522","heartbeat_back_addr":"172.21.15.17:6807/2314489522","heartbeat_front_addr":"172.21.15.17:6805/2314489522","state":["exists","up"]},{"osd":4,"uuid":"c65071dd-9ca7-4f1d-a584-45dc121419a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":202520744},{"type":"v1","addr":"172.21.15.31:6811","nonce":202520744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":202520744},{"type":"v1","addr":"172.21.15.31:6813","nonce":202520744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":202520744},{"type":"v1","addr":"172.21.15.31:6817","nonce":202520744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":202520744},{"type":"v1","addr":"172.21.15.31:6815","nonce":202520744}]},"public_addr":"172.21.15.31:6811/202520744","cluster_addr":"172.21.15.31:6813/202520744","heartbeat_back_addr":"172.21.15.31:6817/202520744","heartbeat_front_addr":"172.21.15.31:6815/202520744","state":["exists","up"]},{"osd":5,"uuid":"235389de-7777-4744-81b8-d0bfc11190e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":1472711057},{"type":"v1","addr":"172.21.15.31:6801","nonce":1472711057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":1472711057},{"type":"v1","addr":"172.21.15.31:6803","nonce":1472711057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":1472711057},{"type":"v1","addr":"172.21.15.31:6807","nonce":1472711057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":1472711057},{"type":"v1","addr":"172.21.15.31:6805","nonce":1472711057}]},"public_addr":"172.21.15.31:6801/1472711057","cluster_addr":"172.21.15.31:6803/1472711057","heartbeat_back_addr":"172.21.15.31:6807/1472711057","heartbeat_front_addr":"172.21.15.31:6805/1472711057","state":["exists","up"]},{"osd":6,"uuid":"cea70a57-3715-4ebb-8511-a528a9df2b1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":3754251089},{"type":"v1","addr":"172.21.15.31:6827","nonce":3754251089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":3754251089},{"type":"v1","addr":"172.21.15.31:6829","nonce":3754251089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":3754251089},{"type":"v1","addr":"172.21.15.31:6833","nonce":3754251089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":3754251089},{"type":"v1","addr":"172.21.15.31:6831","nonce":3754251089}]},"public_addr":"172.21.15.31:6827/3754251089","cluster_addr":"172.21.15.31:6829/3754251089","heartbeat_back_addr":"172.21.15.31:6833/3754251089","heartbeat_front_addr":"172.21.15.31:6831/3754251089","state":["exists","up"]},{"osd":7,"uuid":"01cc111b-a45f-4ca3-9cc1-7fe0afbeba45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":3948648934},{"type":"v1","addr":"172.21.15.31:6819","nonce":3948648934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":3948648934},{"type":"v1","addr":"172.21.15.31:6821","nonce":3948648934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":3948648934},{"type":"v1","addr":"172.21.15.31:6825","nonce":3948648934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":3948648934},{"type":"v1","addr":"172.21.15.31:6823","nonce":3948648934}]},"public_addr":"172.21.15.31:6819/3948648934","cluster_addr":"172.21.15.31:6821/3948648934","heartbeat_back_addr":"172.21.15.31:6825/3948648934","heartbeat_front_addr":"172.21.15.31:6823/3948648934","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:24:32.606989+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:24:32.891535+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:24:33.330793+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:24:32.811533+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:24:33.894624+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:24:32.773817+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:24:32.901865+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:24:32.907207+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.d","mappings":[{"from":5,"to":1}]},{"pgid":"3.e","mappings":[{"from":7,"to":2}]},{"pgid":"3.28","mappings":[{"from":5,"to":2}]},{"pgid":"3.29","mappings":[{"from":3,"to":2}]},{"pgid":"4.a","mappings":[{"from":1,"to":2}]},{"pgid":"4.14","mappings":[{"from":1,"to":5}]},{"pgid":"4.1b","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:47:19.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:19.629+0000 7fa4ee600640 1 -- 172.21.15.17:0/1389354254 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fa4c8072610 msgr2=0x7fa4c8074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:19.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:19.633+0000 7fa4ee600640 1 --2- 172.21.15.17:0/1389354254 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fa4c8072610 0x7fa4c8074ad0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fa4d8001230 tx=0x7fa4d8009040 comp rx=0 tx=0).stop 2024-09-10T21:47:19.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:19.633+0000 7fa4ee600640 1 -- 172.21.15.17:0/1389354254 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa4e8152440 msgr2=0x7fa4e81f3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:19.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:19.633+0000 7fa4ee600640 1 --2- 172.21.15.17:0/1389354254 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa4e8152440 0x7fa4e81f3650 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fa4d400d980 tx=0x7fa4d400de40 comp rx=0 tx=0).stop 2024-09-10T21:47:19.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:19.633+0000 7fa4ee600640 1 -- 172.21.15.17:0/1389354254 shutdown_connections 2024-09-10T21:47:19.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:19.633+0000 7fa4ee600640 1 --2- 172.21.15.17:0/1389354254 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fa4c8072610 0x7fa4c8074ad0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:19.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:19.633+0000 7fa4ee600640 1 --2- 172.21.15.17:0/1389354254 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fa4e8152820 0x7fa4e81f3b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:19.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:19.633+0000 7fa4ee600640 1 --2- 172.21.15.17:0/1389354254 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fa4e8152440 0x7fa4e81f3650 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:19.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:19.633+0000 7fa4ee600640 1 --2- 172.21.15.17:0/1389354254 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fa4e80cdf20 0x7fa4e80c6ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:19.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:19.633+0000 7fa4ee600640 1 -- 172.21.15.17:0/1389354254 >> 172.21.15.17:0/1389354254 conn(0x7fa4e80702f0 msgr2=0x7fa4e8151c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:19.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:19.633+0000 7fa4ee600640 1 -- 172.21.15.17:0/1389354254 shutdown_connections 2024-09-10T21:47:19.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:19.633+0000 7fa4ee600640 1 -- 172.21.15.17:0/1389354254 wait complete. 2024-09-10T21:47:20.657 INFO:tasks.ceph:Scrubbing osd.0 2024-09-10T21:47:20.657 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-09-10T21:47:20.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:20.805+0000 7ff7c8000640 1 -- 172.21.15.17:0/42632105 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff7c0152520 msgr2=0x7ff7c0070dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:20.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:20.805+0000 7ff7c8000640 1 --2- 172.21.15.17:0/42632105 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff7c0152520 0x7ff7c0070dd0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff7c00c4a50 tx=0x7ff7b401f0b0 comp rx=0 tx=0).stop 2024-09-10T21:47:20.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:20.809+0000 7ff7c8000640 1 -- 172.21.15.17:0/42632105 shutdown_connections 2024-09-10T21:47:20.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:20.809+0000 7ff7c8000640 1 --2- 172.21.15.17:0/42632105 >> v2:172.21.15.31:3300/0 conn(0x7ff7c00cdf50 0x7ff7c00ce400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:20.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:20.809+0000 7ff7c8000640 1 --2- 172.21.15.17:0/42632105 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff7c0071680 0x7ff7c00c4bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:20.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:20.809+0000 7ff7c8000640 1 --2- 172.21.15.17:0/42632105 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff7c0152520 0x7ff7c0070dd0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:20.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:20.809+0000 7ff7c8000640 1 -- 172.21.15.17:0/42632105 >> 172.21.15.17:0/42632105 conn(0x7ff7c00748d0 msgr2=0x7ff7c0068990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:20.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:20.809+0000 7ff7c8000640 1 -- 172.21.15.17:0/42632105 shutdown_connections 2024-09-10T21:47:20.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:20.809+0000 7ff7c8000640 1 -- 172.21.15.17:0/42632105 wait complete. 2024-09-10T21:47:20.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:20.809+0000 7ff7c8000640 1 Processor -- start 2024-09-10T21:47:20.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:20.809+0000 7ff7c8000640 1 -- start start 2024-09-10T21:47:20.816 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:20.809+0000 7ff7c8000640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff7c0071680 0x7ff7c01e1de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:20.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:20.809+0000 7ff7c8000640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff7c00cdf50 0x7ff7c00c9600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:20.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:20.809+0000 7ff7c8000640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff7c0152520 0x7ff7c01eff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:20.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:20.809+0000 7ff7c8000640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7ff7c00c5100 con 0x7ff7c0152520 2024-09-10T21:47:20.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:20.809+0000 7ff7c8000640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7ff7c00c3270 con 0x7ff7c0071680 2024-09-10T21:47:20.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:20.809+0000 7ff7c8000640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7ff7c0152900 con 0x7ff7c00cdf50 2024-09-10T21:47:20.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:20.809+0000 7ff7c6000640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff7c0152520 0x7ff7c01eff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:20.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:20.809+0000 7ff7c5600640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff7c0071680 0x7ff7c01e1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:20.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:20.809+0000 7ff7c6000640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff7c0152520 0x7ff7c01eff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:34708/0 (socket says 172.21.15.17:34708) 2024-09-10T21:47:20.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:20.809+0000 7ff7c6000640 1 -- 172.21.15.17:0/23644641 learned_addr learned my addr 172.21.15.17:0/23644641 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:47:20.817 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:20.809+0000 7ff7c4c00640 1 --2- 172.21.15.17:0/23644641 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff7c00cdf50 0x7ff7c00c9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:20.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:20.809+0000 7ff7c6000640 1 -- 172.21.15.17:0/23644641 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff7c00cdf50 msgr2=0x7ff7c00c9600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:20.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:20.809+0000 7ff7c6000640 1 --2- 172.21.15.17:0/23644641 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff7c00cdf50 0x7ff7c00c9600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:20.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:20.809+0000 7ff7c6000640 1 -- 172.21.15.17:0/23644641 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff7c0071680 msgr2=0x7ff7c01e1de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:20.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:20.809+0000 7ff7c6000640 1 --2- 172.21.15.17:0/23644641 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff7c0071680 0x7ff7c01e1de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:20.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:20.809+0000 7ff7c6000640 1 -- 172.21.15.17:0/23644641 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7c006e950 con 0x7ff7c0152520 2024-09-10T21:47:20.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:20.809+0000 7ff7c6000640 1 --2- 172.21.15.17:0/23644641 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff7c0152520 0x7ff7c01eff20 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff7b800e960 tx=0x7ff7b800ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:20.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:20.813+0000 7ff7bd600640 1 -- 172.21.15.17:0/23644641 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7b8004d30 con 0x7ff7c0152520 2024-09-10T21:47:20.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:20.813+0000 7ff7bd600640 1 -- 172.21.15.17:0/23644641 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff7b800cb20 con 0x7ff7c0152520 2024-09-10T21:47:20.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:20.813+0000 7ff7bd600640 1 -- 172.21.15.17:0/23644641 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7b8005020 con 0x7ff7c0152520 2024-09-10T21:47:20.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:20.813+0000 7ff7c8000640 1 -- 172.21.15.17:0/23644641 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7c006eef0 con 0x7ff7c0152520 2024-09-10T21:47:20.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:20.813+0000 7ff7c8000640 1 -- 172.21.15.17:0/23644641 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7c0150040 con 0x7ff7c0152520 2024-09-10T21:47:20.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:20.813+0000 7ff7c8000640 1 -- 172.21.15.17:0/23644641 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff7c0157ec0 con 0x7ff7c0152520 2024-09-10T21:47:20.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:20.813+0000 7ff7bd600640 1 -- 172.21.15.17:0/23644641 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7ff7b800f040 con 0x7ff7c0152520 2024-09-10T21:47:20.820 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:20.813+0000 7ff7bd600640 1 --2- 172.21.15.17:0/23644641 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7ff7a00725e0 0x7ff7a0074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:20.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:20.813+0000 7ff7bd600640 1 -- 172.21.15.17:0/23644641 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7ff7b8093c10 con 0x7ff7c0152520 2024-09-10T21:47:20.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:20.813+0000 7ff7bd600640 1 --2- 172.21.15.17:0/23644641 >> [v2:172.21.15.17:6808/4105489867,v1:172.21.15.17:6809/4105489867] conn(0x7ff7a007bff0 0x7ff7a007e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:20.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:20.813+0000 7ff7bd600640 1 -- 172.21.15.17:0/23644641 --> [v2:172.21.15.17:6808/4105489867,v1:172.21.15.17:6809/4105489867] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff7b801de50 con 0x7ff7a007bff0 2024-09-10T21:47:20.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:20.813+0000 7ff7bd600640 1 -- 172.21.15.17:0/23644641 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7ff7b80982a0 con 0x7ff7c0152520 2024-09-10T21:47:20.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:20.813+0000 7ff7c5600640 1 --2- 172.21.15.17:0/23644641 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7ff7a00725e0 0x7ff7a0074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:20.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:20.813+0000 7ff7c4c00640 1 --2- 172.21.15.17:0/23644641 >> [v2:172.21.15.17:6808/4105489867,v1:172.21.15.17:6809/4105489867] conn(0x7ff7a007bff0 0x7ff7a007e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:20.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:20.817+0000 7ff7c4c00640 1 --2- 172.21.15.17:0/23644641 >> [v2:172.21.15.17:6808/4105489867,v1:172.21.15.17:6809/4105489867] conn(0x7ff7a007bff0 0x7ff7a007e490 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:20.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:20.817+0000 7ff7c5600640 1 --2- 172.21.15.17:0/23644641 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7ff7a00725e0 0x7ff7a0074aa0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7ff7b40b2000 tx=0x7ff7b401d8f0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:20.821 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:20.817+0000 7ff7bd600640 1 -- 172.21.15.17:0/23644641 <== osd.0 v2:172.21.15.17:6808/4105489867 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff7b801de50 con 0x7ff7a007bff0 2024-09-10T21:47:20.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:20.829+0000 7ff7c8000640 1 -- 172.21.15.17:0/23644641 --> [v2:172.21.15.17:6808/4105489867,v1:172.21.15.17:6809/4105489867] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7ff7c00c9b40 con 0x7ff7a007bff0 2024-09-10T21:47:20.834 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:20.829+0000 7ff7bd600640 1 -- 172.21.15.17:0/23644641 <== osd.0 v2:172.21.15.17:6808/4105489867 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7ff7c00c9b40 con 0x7ff7a007bff0 2024-09-10T21:47:20.834 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-10T21:47:20.835 INFO:teuthology.orchestra.run.smithi017.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-10T21:47:20.835 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-10T21:47:20.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:20.829+0000 7ff7c8000640 1 -- 172.21.15.17:0/23644641 >> [v2:172.21.15.17:6808/4105489867,v1:172.21.15.17:6809/4105489867] conn(0x7ff7a007bff0 msgr2=0x7ff7a007e490 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:20.835 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:20.829+0000 7ff7c8000640 1 --2- 172.21.15.17:0/23644641 >> [v2:172.21.15.17:6808/4105489867,v1:172.21.15.17:6809/4105489867] conn(0x7ff7a007bff0 0x7ff7a007e490 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:20.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:20.829+0000 7ff7c8000640 1 -- 172.21.15.17:0/23644641 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7ff7a00725e0 msgr2=0x7ff7a0074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:20.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:20.829+0000 7ff7c8000640 1 --2- 172.21.15.17:0/23644641 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7ff7a00725e0 0x7ff7a0074aa0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7ff7b40b2000 tx=0x7ff7b401d8f0 comp rx=0 tx=0).stop 2024-09-10T21:47:20.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:20.829+0000 7ff7c8000640 1 -- 172.21.15.17:0/23644641 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff7c0152520 msgr2=0x7ff7c01eff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:20.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:20.829+0000 7ff7c8000640 1 --2- 172.21.15.17:0/23644641 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff7c0152520 0x7ff7c01eff20 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff7b800e960 tx=0x7ff7b800ee20 comp rx=0 tx=0).stop 2024-09-10T21:47:20.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:20.829+0000 7ff7c6000640 1 -- 172.21.15.17:0/23644641 reap_dead start 2024-09-10T21:47:20.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:20.829+0000 7ff7c8000640 1 -- 172.21.15.17:0/23644641 shutdown_connections 2024-09-10T21:47:20.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:20.829+0000 7ff7c8000640 1 -- 172.21.15.17:0/23644641 >> 172.21.15.17:0/23644641 conn(0x7ff7c00748d0 msgr2=0x7ff7c01573d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:20.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:20.829+0000 7ff7c8000640 1 -- 172.21.15.17:0/23644641 shutdown_connections 2024-09-10T21:47:20.836 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:20.829+0000 7ff7c8000640 1 -- 172.21.15.17:0/23644641 wait complete. 2024-09-10T21:47:20.854 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-09-10T21:47:21.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:20.997+0000 7f04d8c00640 1 -- 172.21.15.17:0/247726076 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f04d4152440 msgr2=0x7f04d40b3eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:21.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:20.997+0000 7f04d8c00640 1 --2- 172.21.15.17:0/247726076 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f04d4152440 0x7f04d40b3eb0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f04d40cbaa0 tx=0x7f04c401c830 comp rx=0 tx=0).stop 2024-09-10T21:47:21.003 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:20.997+0000 7f04d8c00640 1 -- 172.21.15.17:0/247726076 shutdown_connections 2024-09-10T21:47:21.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:20.997+0000 7f04d8c00640 1 --2- 172.21.15.17:0/247726076 >> v2:172.21.15.31:3300/0 conn(0x7f04d4152820 0x7f04d415c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:21.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:20.997+0000 7f04d8c00640 1 --2- 172.21.15.17:0/247726076 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f04d4152440 0x7f04d40b3eb0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:21.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:20.997+0000 7f04d8c00640 1 -- 172.21.15.17:0/247726076 >> 172.21.15.17:0/247726076 conn(0x7f04d40702f0 msgr2=0x7f04d4070700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:21.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:20.997+0000 7f04d8c00640 1 -- 172.21.15.17:0/247726076 shutdown_connections 2024-09-10T21:47:21.004 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:20.997+0000 7f04d8c00640 1 -- 172.21.15.17:0/247726076 wait complete. 2024-09-10T21:47:21.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.001+0000 7f04d8c00640 1 Processor -- start 2024-09-10T21:47:21.005 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.001+0000 7f04d8c00640 1 -- start start 2024-09-10T21:47:21.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.001+0000 7f04d8c00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f04d40cdf20 0x7f04d41449f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:21.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.001+0000 7f04d8c00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f04d4152440 0x7f04d41482f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:21.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.001+0000 7f04d8c00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f04d4152820 0x7f04d41f4b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:21.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.001+0000 7f04d8c00640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f04d415c7c0 con 0x7f04d4152820 2024-09-10T21:47:21.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.001+0000 7f04d8c00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f04d40ce740 con 0x7f04d4152440 2024-09-10T21:47:21.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.001+0000 7f04d8c00640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f04d40cb450 con 0x7f04d40cdf20 2024-09-10T21:47:21.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.001+0000 7f04d2800640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f04d4152820 0x7f04d41f4b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:21.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.001+0000 7f04d2800640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f04d4152820 0x7f04d41f4b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:34720/0 (socket says 172.21.15.17:34720) 2024-09-10T21:47:21.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.001+0000 7f04d2800640 1 -- 172.21.15.17:0/1967176774 learned_addr learned my addr 172.21.15.17:0/1967176774 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:47:21.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.001+0000 7f04d1e00640 1 --2- 172.21.15.17:0/1967176774 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f04d40cdf20 0x7f04d41449f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:21.008 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.001+0000 7f04cbe00640 1 --2- 172.21.15.17:0/1967176774 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f04d4152440 0x7f04d41482f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:21.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.001+0000 7f04d2800640 1 -- 172.21.15.17:0/1967176774 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f04d40cdf20 msgr2=0x7f04d41449f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:21.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.001+0000 7f04d2800640 1 --2- 172.21.15.17:0/1967176774 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f04d40cdf20 0x7f04d41449f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:21.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.001+0000 7f04d2800640 1 -- 172.21.15.17:0/1967176774 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f04d4152440 msgr2=0x7f04d41482f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:21.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.001+0000 7f04d2800640 1 --2- 172.21.15.17:0/1967176774 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f04d4152440 0x7f04d41482f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:21.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.001+0000 7f04d2800640 1 -- 172.21.15.17:0/1967176774 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04d414e350 con 0x7f04d4152820 2024-09-10T21:47:21.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.005+0000 7f04d2800640 1 --2- 172.21.15.17:0/1967176774 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f04d4152820 0x7f04d41f4b90 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f04cc00c760 tx=0x7f04cc00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:21.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.005+0000 7f04ca000640 1 -- 172.21.15.17:0/1967176774 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04cc003db0 con 0x7f04d4152820 2024-09-10T21:47:21.009 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.005+0000 7f04ca000640 1 -- 172.21.15.17:0/1967176774 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f04cc00ce10 con 0x7f04d4152820 2024-09-10T21:47:21.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.005+0000 7f04d8c00640 1 -- 172.21.15.17:0/1967176774 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04d4147320 con 0x7f04d4152820 2024-09-10T21:47:21.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.005+0000 7f04ca000640 1 -- 172.21.15.17:0/1967176774 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04cc0052d0 con 0x7f04d4152820 2024-09-10T21:47:21.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.005+0000 7f04d8c00640 1 -- 172.21.15.17:0/1967176774 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04d415a540 con 0x7f04d4152820 2024-09-10T21:47:21.010 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.005+0000 7f04d8c00640 1 -- 172.21.15.17:0/1967176774 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04d40c2ff0 con 0x7f04d4152820 2024-09-10T21:47:21.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.005+0000 7f04ca000640 1 -- 172.21.15.17:0/1967176774 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f04cc005470 con 0x7f04d4152820 2024-09-10T21:47:21.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.009+0000 7f04ca000640 1 --2- 172.21.15.17:0/1967176774 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f04b00725e0 0x7f04b0074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:21.011 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.009+0000 7f04ca000640 1 -- 172.21.15.17:0/1967176774 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f04cc093330 con 0x7f04d4152820 2024-09-10T21:47:21.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.009+0000 7f04d1e00640 1 --2- 172.21.15.17:0/1967176774 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f04b00725e0 0x7f04b0074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:21.012 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.009+0000 7f04d1e00640 1 --2- 172.21.15.17:0/1967176774 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f04b00725e0 0x7f04b0074aa0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f04c40af000 tx=0x7f04c401b070 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:21.016 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.013+0000 7f04ca000640 1 -- 172.21.15.17:0/1967176774 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f04cc05df20 con 0x7f04d4152820 2024-09-10T21:47:21.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.189+0000 7f04d8c00640 1 -- 172.21.15.17:0/1967176774 --> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) -- 0x7f04d41433c0 con 0x7f04b00725e0 2024-09-10T21:47:21.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.189+0000 7f04ca000640 1 -- 172.21.15.17:0/1967176774 <== mgr.4100 v2:172.21.15.31:6808/3101308586 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f04d41433c0 con 0x7f04b00725e0 2024-09-10T21:47:21.195 INFO:teuthology.orchestra.run.smithi017.stderr:instructed osd(s) 0 to deep-scrub 2024-09-10T21:47:21.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.193+0000 7f04d8c00640 1 -- 172.21.15.17:0/1967176774 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f04b00725e0 msgr2=0x7f04b0074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:21.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.193+0000 7f04d8c00640 1 --2- 172.21.15.17:0/1967176774 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f04b00725e0 0x7f04b0074aa0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f04c40af000 tx=0x7f04c401b070 comp rx=0 tx=0).stop 2024-09-10T21:47:21.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.193+0000 7f04d8c00640 1 -- 172.21.15.17:0/1967176774 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f04d4152820 msgr2=0x7f04d41f4b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:21.200 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.193+0000 7f04d8c00640 1 --2- 172.21.15.17:0/1967176774 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f04d4152820 0x7f04d41f4b90 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f04cc00c760 tx=0x7f04cc00cc20 comp rx=0 tx=0).stop 2024-09-10T21:47:21.200 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.193+0000 7f04d8c00640 1 -- 172.21.15.17:0/1967176774 shutdown_connections 2024-09-10T21:47:21.200 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.193+0000 7f04d8c00640 1 --2- 172.21.15.17:0/1967176774 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f04b00725e0 0x7f04b0074aa0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:21.200 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.193+0000 7f04d8c00640 1 --2- 172.21.15.17:0/1967176774 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f04d4152820 0x7f04d41f4b90 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:21.200 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.193+0000 7f04d8c00640 1 --2- 172.21.15.17:0/1967176774 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f04d4152440 0x7f04d41482f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:21.200 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.193+0000 7f04d8c00640 1 --2- 172.21.15.17:0/1967176774 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f04d40cdf20 0x7f04d41449f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:21.200 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.193+0000 7f04d8c00640 1 -- 172.21.15.17:0/1967176774 >> 172.21.15.17:0/1967176774 conn(0x7f04d40702f0 msgr2=0x7f04d415aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:21.201 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.193+0000 7f04d8c00640 1 -- 172.21.15.17:0/1967176774 shutdown_connections 2024-09-10T21:47:21.201 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.193+0000 7f04d8c00640 1 -- 172.21.15.17:0/1967176774 wait complete. 2024-09-10T21:47:21.217 INFO:tasks.ceph:Scrubbing osd.1 2024-09-10T21:47:21.218 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-09-10T21:47:21.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.365+0000 7f3f53600640 1 -- 172.21.15.17:0/3717517940 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f4c070dd0 msgr2=0x7f3f4c163cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:21.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.365+0000 7f3f53600640 1 --2- 172.21.15.17:0/3717517940 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f4c070dd0 0x7f3f4c163cb0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f3f44009bc0 tx=0x7f3f4401f3b0 comp rx=0 tx=0).stop 2024-09-10T21:47:21.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.365+0000 7f3f53600640 1 -- 172.21.15.17:0/3717517940 shutdown_connections 2024-09-10T21:47:21.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.365+0000 7f3f53600640 1 --2- 172.21.15.17:0/3717517940 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f4c070dd0 0x7f3f4c163cb0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:21.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.365+0000 7f3f53600640 1 -- 172.21.15.17:0/3717517940 >> 172.21.15.17:0/3717517940 conn(0x7f3f4c0748d0 msgr2=0x7f3f4c068990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:21.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.365+0000 7f3f53600640 1 -- 172.21.15.17:0/3717517940 shutdown_connections 2024-09-10T21:47:21.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.365+0000 7f3f53600640 1 -- 172.21.15.17:0/3717517940 wait complete. 2024-09-10T21:47:21.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.365+0000 7f3f53600640 1 Processor -- start 2024-09-10T21:47:21.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.365+0000 7f3f53600640 1 -- start start 2024-09-10T21:47:21.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.369+0000 7f3f53600640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3f4c070dd0 0x7f3f4c157590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:21.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.369+0000 7f3f53600640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f4c071680 0x7f3f4c1f4150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:21.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.369+0000 7f3f53600640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3f4c152520 0x7f3f4c15a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:21.373 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.369+0000 7f3f53600640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f3f4c152900 con 0x7f3f4c070dd0 2024-09-10T21:47:21.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.369+0000 7f3f53600640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f3f4c0c3270 con 0x7f3f4c071680 2024-09-10T21:47:21.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.369+0000 7f3f53600640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f3f4c152a80 con 0x7f3f4c152520 2024-09-10T21:47:21.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.369+0000 7f3f50c00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3f4c070dd0 0x7f3f4c157590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:21.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.369+0000 7f3f50c00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3f4c070dd0 0x7f3f4c157590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:34726/0 (socket says 172.21.15.17:34726) 2024-09-10T21:47:21.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.369+0000 7f3f50c00640 1 -- 172.21.15.17:0/3811082593 learned_addr learned my addr 172.21.15.17:0/3811082593 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:47:21.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.369+0000 7f3f4be00640 1 --2- 172.21.15.17:0/3811082593 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f4c071680 0x7f3f4c1f4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:21.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.369+0000 7f3f51600640 1 --2- 172.21.15.17:0/3811082593 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3f4c152520 0x7f3f4c15a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:21.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.369+0000 7f3f50c00640 1 -- 172.21.15.17:0/3811082593 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3f4c152520 msgr2=0x7f3f4c15a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:21.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.369+0000 7f3f50c00640 1 --2- 172.21.15.17:0/3811082593 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f3f4c152520 0x7f3f4c15a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:21.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.369+0000 7f3f50c00640 1 -- 172.21.15.17:0/3811082593 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f4c071680 msgr2=0x7f3f4c1f4150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:21.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.369+0000 7f3f50c00640 1 --2- 172.21.15.17:0/3811082593 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f3f4c071680 0x7f3f4c1f4150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:21.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.369+0000 7f3f50c00640 1 -- 172.21.15.17:0/3811082593 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f4c1f3990 con 0x7f3f4c070dd0 2024-09-10T21:47:21.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.369+0000 7f3f50c00640 1 --2- 172.21.15.17:0/3811082593 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3f4c070dd0 0x7f3f4c157590 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f3f4000c3c0 tx=0x7f3f4000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:21.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.369+0000 7f3f48c00640 1 -- 172.21.15.17:0/3811082593 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f40017070 con 0x7f3f4c070dd0 2024-09-10T21:47:21.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.369+0000 7f3f48c00640 1 -- 172.21.15.17:0/3811082593 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3f400053c0 con 0x7f3f4c070dd0 2024-09-10T21:47:21.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.369+0000 7f3f53600640 1 -- 172.21.15.17:0/3811082593 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f4c157bb0 con 0x7f3f4c070dd0 2024-09-10T21:47:21.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.369+0000 7f3f48c00640 1 -- 172.21.15.17:0/3811082593 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f40003860 con 0x7f3f4c070dd0 2024-09-10T21:47:21.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.369+0000 7f3f53600640 1 -- 172.21.15.17:0/3811082593 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f4c157e70 con 0x7f3f4c070dd0 2024-09-10T21:47:21.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.369+0000 7f3f53600640 1 -- 172.21.15.17:0/3811082593 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3f4c159810 con 0x7f3f4c070dd0 2024-09-10T21:47:21.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.373+0000 7f3f48c00640 1 -- 172.21.15.17:0/3811082593 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f3f40003a00 con 0x7f3f4c070dd0 2024-09-10T21:47:21.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.373+0000 7f3f48c00640 1 --2- 172.21.15.17:0/3811082593 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f3f2c0725e0 0x7f3f2c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:21.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.373+0000 7f3f48c00640 1 -- 172.21.15.17:0/3811082593 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f3f40092d50 con 0x7f3f4c070dd0 2024-09-10T21:47:21.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.373+0000 7f3f48c00640 1 --2- 172.21.15.17:0/3811082593 >> [v2:172.21.15.17:6824/2599842027,v1:172.21.15.17:6825/2599842027] conn(0x7f3f2c07c000 0x7f3f2c07e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:21.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.373+0000 7f3f48c00640 1 -- 172.21.15.17:0/3811082593 --> [v2:172.21.15.17:6824/2599842027,v1:172.21.15.17:6825/2599842027] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3f4000bf30 con 0x7f3f2c07c000 2024-09-10T21:47:21.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.373+0000 7f3f48c00640 1 -- 172.21.15.17:0/3811082593 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f3f40095310 con 0x7f3f4c070dd0 2024-09-10T21:47:21.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.373+0000 7f3f4be00640 1 --2- 172.21.15.17:0/3811082593 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f3f2c0725e0 0x7f3f2c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:21.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.373+0000 7f3f51600640 1 --2- 172.21.15.17:0/3811082593 >> [v2:172.21.15.17:6824/2599842027,v1:172.21.15.17:6825/2599842027] conn(0x7f3f2c07c000 0x7f3f2c07e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:21.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.373+0000 7f3f51600640 1 --2- 172.21.15.17:0/3811082593 >> [v2:172.21.15.17:6824/2599842027,v1:172.21.15.17:6825/2599842027] conn(0x7f3f2c07c000 0x7f3f2c07e4a0 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:21.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.373+0000 7f3f4be00640 1 --2- 172.21.15.17:0/3811082593 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f3f2c0725e0 0x7f3f2c074aa0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f3f4c0cbb20 tx=0x7f3f38009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:21.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.373+0000 7f3f48c00640 1 -- 172.21.15.17:0/3811082593 <== osd.1 v2:172.21.15.17:6824/2599842027 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3f4000bf30 con 0x7f3f2c07c000 2024-09-10T21:47:21.391 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.389+0000 7f3f53600640 1 -- 172.21.15.17:0/3811082593 --> [v2:172.21.15.17:6824/2599842027,v1:172.21.15.17:6825/2599842027] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f3f4c14e740 con 0x7f3f2c07c000 2024-09-10T21:47:21.392 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.389+0000 7f3f48c00640 1 -- 172.21.15.17:0/3811082593 <== osd.1 v2:172.21.15.17:6824/2599842027 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f3f4c14e740 con 0x7f3f2c07c000 2024-09-10T21:47:21.392 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-10T21:47:21.392 INFO:teuthology.orchestra.run.smithi017.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-10T21:47:21.392 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-10T21:47:21.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.389+0000 7f3f53600640 1 -- 172.21.15.17:0/3811082593 >> [v2:172.21.15.17:6824/2599842027,v1:172.21.15.17:6825/2599842027] conn(0x7f3f2c07c000 msgr2=0x7f3f2c07e4a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:21.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.389+0000 7f3f53600640 1 --2- 172.21.15.17:0/3811082593 >> [v2:172.21.15.17:6824/2599842027,v1:172.21.15.17:6825/2599842027] conn(0x7f3f2c07c000 0x7f3f2c07e4a0 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:21.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.389+0000 7f3f53600640 1 -- 172.21.15.17:0/3811082593 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f3f2c0725e0 msgr2=0x7f3f2c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:21.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.389+0000 7f3f53600640 1 --2- 172.21.15.17:0/3811082593 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f3f2c0725e0 0x7f3f2c074aa0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f3f4c0cbb20 tx=0x7f3f38009040 comp rx=0 tx=0).stop 2024-09-10T21:47:21.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.389+0000 7f3f53600640 1 -- 172.21.15.17:0/3811082593 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3f4c070dd0 msgr2=0x7f3f4c157590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:21.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.389+0000 7f3f53600640 1 --2- 172.21.15.17:0/3811082593 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f3f4c070dd0 0x7f3f4c157590 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f3f4000c3c0 tx=0x7f3f4000c880 comp rx=0 tx=0).stop 2024-09-10T21:47:21.393 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.389+0000 7f3f51600640 1 -- 172.21.15.17:0/3811082593 reap_dead start 2024-09-10T21:47:21.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.389+0000 7f3f53600640 1 -- 172.21.15.17:0/3811082593 shutdown_connections 2024-09-10T21:47:21.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.389+0000 7f3f53600640 1 -- 172.21.15.17:0/3811082593 >> 172.21.15.17:0/3811082593 conn(0x7f3f4c0748d0 msgr2=0x7f3f4c150b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:21.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.389+0000 7f3f53600640 1 -- 172.21.15.17:0/3811082593 shutdown_connections 2024-09-10T21:47:21.394 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.389+0000 7f3f53600640 1 -- 172.21.15.17:0/3811082593 wait complete. 2024-09-10T21:47:21.411 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-09-10T21:47:21.559 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.553+0000 7efbff600640 1 -- 172.21.15.17:0/477713891 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efbf8152440 msgr2=0x7efbf80b3eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:21.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.553+0000 7efbff600640 1 --2- 172.21.15.17:0/477713891 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efbf8152440 0x7efbf80b3eb0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7efbf81594a0 tx=0x7efbec01c790 comp rx=0 tx=0).stop 2024-09-10T21:47:21.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.553+0000 7efbff600640 1 -- 172.21.15.17:0/477713891 shutdown_connections 2024-09-10T21:47:21.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.553+0000 7efbff600640 1 --2- 172.21.15.17:0/477713891 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efbf8152440 0x7efbf80b3eb0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:21.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.553+0000 7efbff600640 1 -- 172.21.15.17:0/477713891 >> 172.21.15.17:0/477713891 conn(0x7efbf80702f0 msgr2=0x7efbf8070700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:21.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.553+0000 7efbff600640 1 -- 172.21.15.17:0/477713891 shutdown_connections 2024-09-10T21:47:21.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.553+0000 7efbff600640 1 -- 172.21.15.17:0/477713891 wait complete. 2024-09-10T21:47:21.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.557+0000 7efbff600640 1 Processor -- start 2024-09-10T21:47:21.561 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.557+0000 7efbff600640 1 -- start start 2024-09-10T21:47:21.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.557+0000 7efbff600640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efbf80cdf20 0x7efbf806f960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:21.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.557+0000 7efbff600640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7efbf8152440 0x7efbf81f3530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:21.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.557+0000 7efbff600640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efbf8152820 0x7efbf8154580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:21.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.557+0000 7efbff600640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7efbf80cb450 con 0x7efbf8152820 2024-09-10T21:47:21.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.557+0000 7efbff600640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7efbf80ce740 con 0x7efbf80cdf20 2024-09-10T21:47:21.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.557+0000 7efbff600640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7efbf80cb5d0 con 0x7efbf8152440 2024-09-10T21:47:21.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.557+0000 7efbfd600640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efbf8152820 0x7efbf8154580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:21.563 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.557+0000 7efbfd600640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efbf8152820 0x7efbf8154580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:34728/0 (socket says 172.21.15.17:34728) 2024-09-10T21:47:21.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.557+0000 7efbfd600640 1 -- 172.21.15.17:0/3657655992 learned_addr learned my addr 172.21.15.17:0/3657655992 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:47:21.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.557+0000 7efbfcc00640 1 --2- 172.21.15.17:0/3657655992 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efbf80cdf20 0x7efbf806f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:21.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.557+0000 7efbf7e00640 1 --2- 172.21.15.17:0/3657655992 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7efbf8152440 0x7efbf81f3530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:21.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.557+0000 7efbfd600640 1 -- 172.21.15.17:0/3657655992 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7efbf8152440 msgr2=0x7efbf81f3530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:21.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.557+0000 7efbfd600640 1 --2- 172.21.15.17:0/3657655992 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7efbf8152440 0x7efbf81f3530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:21.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.557+0000 7efbfd600640 1 -- 172.21.15.17:0/3657655992 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efbf80cdf20 msgr2=0x7efbf806f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:21.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.557+0000 7efbfd600640 1 --2- 172.21.15.17:0/3657655992 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efbf80cdf20 0x7efbf806f960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:21.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.557+0000 7efbfd600640 1 -- 172.21.15.17:0/3657655992 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efbf81f4170 con 0x7efbf8152820 2024-09-10T21:47:21.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.557+0000 7efbf7e00640 1 --2- 172.21.15.17:0/3657655992 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7efbf8152440 0x7efbf81f3530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:47:21.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.557+0000 7efbfd600640 1 --2- 172.21.15.17:0/3657655992 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efbf8152820 0x7efbf8154580 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7efbf000c740 tx=0x7efbf000cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:21.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.557+0000 7efbf4c00640 1 -- 172.21.15.17:0/3657655992 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbf0003b80 con 0x7efbf8152820 2024-09-10T21:47:21.564 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.557+0000 7efbf4c00640 1 -- 172.21.15.17:0/3657655992 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efbf000ce50 con 0x7efbf8152820 2024-09-10T21:47:21.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.557+0000 7efbff600640 1 -- 172.21.15.17:0/3657655992 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efbf8153800 con 0x7efbf8152820 2024-09-10T21:47:21.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.557+0000 7efbf4c00640 1 -- 172.21.15.17:0/3657655992 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efbf000a850 con 0x7efbf8152820 2024-09-10T21:47:21.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.557+0000 7efbff600640 1 -- 172.21.15.17:0/3657655992 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efbf8153240 con 0x7efbf8152820 2024-09-10T21:47:21.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.561+0000 7efbff600640 1 -- 172.21.15.17:0/3657655992 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efbf8158360 con 0x7efbf8152820 2024-09-10T21:47:21.565 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.561+0000 7efbf4c00640 1 -- 172.21.15.17:0/3657655992 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7efbf000a9f0 con 0x7efbf8152820 2024-09-10T21:47:21.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.561+0000 7efbf4c00640 1 --2- 172.21.15.17:0/3657655992 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7efbd80725e0 0x7efbd8074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:21.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.561+0000 7efbf4c00640 1 -- 172.21.15.17:0/3657655992 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7efbf00935c0 con 0x7efbf8152820 2024-09-10T21:47:21.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.561+0000 7efbfcc00640 1 --2- 172.21.15.17:0/3657655992 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7efbd80725e0 0x7efbd8074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:21.567 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.565+0000 7efbfcc00640 1 --2- 172.21.15.17:0/3657655992 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7efbd80725e0 0x7efbd8074aa0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7efbec0af000 tx=0x7efbec01afd0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:21.570 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.565+0000 7efbf4c00640 1 -- 172.21.15.17:0/3657655992 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7efbf005e770 con 0x7efbf8152820 2024-09-10T21:47:21.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.753+0000 7efbff600640 1 -- 172.21.15.17:0/3657655992 --> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) -- 0x7efbf80cdc20 con 0x7efbd80725e0 2024-09-10T21:47:21.756 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.753+0000 7efbf4c00640 1 -- 172.21.15.17:0/3657655992 <== mgr.4100 v2:172.21.15.31:6808/3101308586 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7efbf80cdc20 con 0x7efbd80725e0 2024-09-10T21:47:21.756 INFO:teuthology.orchestra.run.smithi017.stderr:instructed osd(s) 1 to deep-scrub 2024-09-10T21:47:21.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.753+0000 7efbff600640 1 -- 172.21.15.17:0/3657655992 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7efbd80725e0 msgr2=0x7efbd8074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:21.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.753+0000 7efbff600640 1 --2- 172.21.15.17:0/3657655992 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7efbd80725e0 0x7efbd8074aa0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7efbec0af000 tx=0x7efbec01afd0 comp rx=0 tx=0).stop 2024-09-10T21:47:21.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.753+0000 7efbff600640 1 -- 172.21.15.17:0/3657655992 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efbf8152820 msgr2=0x7efbf8154580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:21.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.753+0000 7efbff600640 1 --2- 172.21.15.17:0/3657655992 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efbf8152820 0x7efbf8154580 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7efbf000c740 tx=0x7efbf000cc00 comp rx=0 tx=0).stop 2024-09-10T21:47:21.759 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.753+0000 7efbff600640 1 -- 172.21.15.17:0/3657655992 shutdown_connections 2024-09-10T21:47:21.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.753+0000 7efbff600640 1 --2- 172.21.15.17:0/3657655992 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7efbd80725e0 0x7efbd8074aa0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:21.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.753+0000 7efbff600640 1 --2- 172.21.15.17:0/3657655992 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7efbf8152820 0x7efbf8154580 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:21.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.753+0000 7efbff600640 1 --2- 172.21.15.17:0/3657655992 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7efbf8152440 0x7efbf81f3530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:21.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.753+0000 7efbff600640 1 --2- 172.21.15.17:0/3657655992 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7efbf80cdf20 0x7efbf806f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:21.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.753+0000 7efbff600640 1 -- 172.21.15.17:0/3657655992 >> 172.21.15.17:0/3657655992 conn(0x7efbf80702f0 msgr2=0x7efbf8151a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:21.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.753+0000 7efbff600640 1 -- 172.21.15.17:0/3657655992 shutdown_connections 2024-09-10T21:47:21.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.753+0000 7efbff600640 1 -- 172.21.15.17:0/3657655992 wait complete. 2024-09-10T21:47:21.779 INFO:tasks.ceph:Scrubbing osd.2 2024-09-10T21:47:21.780 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-09-10T21:47:21.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.925+0000 7f7a61a00640 1 -- 172.21.15.17:0/3366572125 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7a5c0cdf50 msgr2=0x7f7a5c0ce3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:21.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.925+0000 7f7a61a00640 1 --2- 172.21.15.17:0/3366572125 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7a5c0cdf50 0x7f7a5c0ce3c0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f7a5c155e60 tx=0x7f7a5401f540 comp rx=0 tx=0).stop 2024-09-10T21:47:21.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.925+0000 7f7a61a00640 1 -- 172.21.15.17:0/3366572125 shutdown_connections 2024-09-10T21:47:21.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.925+0000 7f7a61a00640 1 --2- 172.21.15.17:0/3366572125 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7a5c0cdf50 0x7f7a5c0ce3c0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:21.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.925+0000 7f7a61a00640 1 --2- 172.21.15.17:0/3366572125 >> v2:172.21.15.31:3300/0 conn(0x7f7a5c152520 0x7f7a5c070dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:21.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.925+0000 7f7a61a00640 1 -- 172.21.15.17:0/3366572125 >> 172.21.15.17:0/3366572125 conn(0x7f7a5c0748d0 msgr2=0x7f7a5c068990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:21.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.925+0000 7f7a61a00640 1 -- 172.21.15.17:0/3366572125 shutdown_connections 2024-09-10T21:47:21.931 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.925+0000 7f7a61a00640 1 -- 172.21.15.17:0/3366572125 wait complete. 2024-09-10T21:47:21.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.929+0000 7f7a61a00640 1 Processor -- start 2024-09-10T21:47:21.932 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.929+0000 7f7a61a00640 1 -- start start 2024-09-10T21:47:21.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.929+0000 7f7a61a00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7a5c071680 0x7f7a5c14a150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:21.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.929+0000 7f7a61a00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f7a5c0cdf50 0x7f7a5c0cb4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:21.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.929+0000 7f7a61a00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7a5c152520 0x7f7a5c1483d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:21.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.929+0000 7f7a61a00640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f7a5c152900 con 0x7f7a5c152520 2024-09-10T21:47:21.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.929+0000 7f7a61a00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f7a5c0c3270 con 0x7f7a5c071680 2024-09-10T21:47:21.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.929+0000 7f7a61a00640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f7a5c152a80 con 0x7f7a5c0cdf50 2024-09-10T21:47:21.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.929+0000 7f7a5b400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7a5c152520 0x7f7a5c1483d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:21.934 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.929+0000 7f7a5aa00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7a5c071680 0x7f7a5c14a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:21.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.929+0000 7f7a53e00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f7a5c0cdf50 0x7f7a5c0cb4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:21.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.929+0000 7f7a5b400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7a5c152520 0x7f7a5c1483d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:34742/0 (socket says 172.21.15.17:34742) 2024-09-10T21:47:21.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.929+0000 7f7a5aa00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7a5c071680 0x7f7a5c14a150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.17:58086/0 (socket says 172.21.15.17:58086) 2024-09-10T21:47:21.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.929+0000 7f7a5b400640 1 -- 172.21.15.17:0/3585891280 learned_addr learned my addr 172.21.15.17:0/3585891280 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:47:21.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.929+0000 7f7a5b400640 1 -- 172.21.15.17:0/3585891280 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f7a5c0cdf50 msgr2=0x7f7a5c0cb4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:21.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.929+0000 7f7a5b400640 1 --2- 172.21.15.17:0/3585891280 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f7a5c0cdf50 0x7f7a5c0cb4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:21.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.929+0000 7f7a5b400640 1 -- 172.21.15.17:0/3585891280 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7a5c071680 msgr2=0x7f7a5c14a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:21.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.929+0000 7f7a5b400640 1 --2- 172.21.15.17:0/3585891280 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f7a5c071680 0x7f7a5c14a150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:21.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.929+0000 7f7a5b400640 1 -- 172.21.15.17:0/3585891280 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a5c148f10 con 0x7f7a5c152520 2024-09-10T21:47:21.935 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.929+0000 7f7a5b400640 1 --2- 172.21.15.17:0/3585891280 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7a5c152520 0x7f7a5c1483d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f7a5401dbf0 tx=0x7f7a54009340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:21.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.929+0000 7f7a52a00640 1 -- 172.21.15.17:0/3585891280 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a540b4040 con 0x7f7a5c152520 2024-09-10T21:47:21.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.929+0000 7f7a52a00640 1 -- 172.21.15.17:0/3585891280 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7a540bf070 con 0x7f7a5c152520 2024-09-10T21:47:21.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.929+0000 7f7a52a00640 1 -- 172.21.15.17:0/3585891280 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a540afdd0 con 0x7f7a5c152520 2024-09-10T21:47:21.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.929+0000 7f7a61a00640 1 -- 172.21.15.17:0/3585891280 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a5c149230 con 0x7f7a5c152520 2024-09-10T21:47:21.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.929+0000 7f7a61a00640 1 -- 172.21.15.17:0/3585891280 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a5c1f0f70 con 0x7f7a5c152520 2024-09-10T21:47:21.936 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.933+0000 7f7a61a00640 1 -- 172.21.15.17:0/3585891280 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7a5c1f11b0 con 0x7f7a5c152520 2024-09-10T21:47:21.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.933+0000 7f7a52a00640 1 -- 172.21.15.17:0/3585891280 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f7a540af500 con 0x7f7a5c152520 2024-09-10T21:47:21.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.933+0000 7f7a52a00640 1 --2- 172.21.15.17:0/3585891280 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f7a380725e0 0x7f7a38074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:21.937 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.933+0000 7f7a52a00640 1 -- 172.21.15.17:0/3585891280 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f7a541317f0 con 0x7f7a5c152520 2024-09-10T21:47:21.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.933+0000 7f7a52a00640 1 --2- 172.21.15.17:0/3585891280 >> [v2:172.21.15.17:6816/1248243634,v1:172.21.15.17:6817/1248243634] conn(0x7f7a3807bff0 0x7f7a3807e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:21.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.933+0000 7f7a52a00640 1 -- 172.21.15.17:0/3585891280 --> [v2:172.21.15.17:6816/1248243634,v1:172.21.15.17:6817/1248243634] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7a540a9ec0 con 0x7f7a3807bff0 2024-09-10T21:47:21.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.933+0000 7f7a52a00640 1 -- 172.21.15.17:0/3585891280 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f7a54138050 con 0x7f7a5c152520 2024-09-10T21:47:21.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.933+0000 7f7a5aa00640 1 --2- 172.21.15.17:0/3585891280 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f7a380725e0 0x7f7a38074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:21.938 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.933+0000 7f7a53e00640 1 --2- 172.21.15.17:0/3585891280 >> [v2:172.21.15.17:6816/1248243634,v1:172.21.15.17:6817/1248243634] conn(0x7f7a3807bff0 0x7f7a3807e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:21.939 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.933+0000 7f7a53e00640 1 --2- 172.21.15.17:0/3585891280 >> [v2:172.21.15.17:6816/1248243634,v1:172.21.15.17:6817/1248243634] conn(0x7f7a3807bff0 0x7f7a3807e490 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:21.939 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.933+0000 7f7a5aa00640 1 --2- 172.21.15.17:0/3585891280 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f7a380725e0 0x7f7a38074aa0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f7a4c004630 tx=0x7f7a4c00b2e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:21.940 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.937+0000 7f7a52a00640 1 -- 172.21.15.17:0/3585891280 <== osd.2 v2:172.21.15.17:6816/1248243634 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7a540a9ec0 con 0x7f7a3807bff0 2024-09-10T21:47:21.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.961+0000 7f7a61a00640 1 -- 172.21.15.17:0/3585891280 --> [v2:172.21.15.17:6816/1248243634,v1:172.21.15.17:6817/1248243634] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f7a5c0cb150 con 0x7f7a3807bff0 2024-09-10T21:47:21.965 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.961+0000 7f7a52a00640 1 -- 172.21.15.17:0/3585891280 <== osd.2 v2:172.21.15.17:6816/1248243634 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f7a5c0cb150 con 0x7f7a3807bff0 2024-09-10T21:47:21.965 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-10T21:47:21.965 INFO:teuthology.orchestra.run.smithi017.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-10T21:47:21.965 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-10T21:47:21.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.961+0000 7f7a61a00640 1 -- 172.21.15.17:0/3585891280 >> [v2:172.21.15.17:6816/1248243634,v1:172.21.15.17:6817/1248243634] conn(0x7f7a3807bff0 msgr2=0x7f7a3807e490 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:21.966 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.961+0000 7f7a61a00640 1 --2- 172.21.15.17:0/3585891280 >> [v2:172.21.15.17:6816/1248243634,v1:172.21.15.17:6817/1248243634] conn(0x7f7a3807bff0 0x7f7a3807e490 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:21.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.961+0000 7f7a61a00640 1 -- 172.21.15.17:0/3585891280 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f7a380725e0 msgr2=0x7f7a38074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:21.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.961+0000 7f7a61a00640 1 --2- 172.21.15.17:0/3585891280 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f7a380725e0 0x7f7a38074aa0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f7a4c004630 tx=0x7f7a4c00b2e0 comp rx=0 tx=0).stop 2024-09-10T21:47:21.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.961+0000 7f7a61a00640 1 -- 172.21.15.17:0/3585891280 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7a5c152520 msgr2=0x7f7a5c1483d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:21.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.961+0000 7f7a61a00640 1 --2- 172.21.15.17:0/3585891280 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7a5c152520 0x7f7a5c1483d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f7a5401dbf0 tx=0x7f7a54009340 comp rx=0 tx=0).stop 2024-09-10T21:47:21.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.961+0000 7f7a5b400640 1 -- 172.21.15.17:0/3585891280 reap_dead start 2024-09-10T21:47:21.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.961+0000 7f7a61a00640 1 -- 172.21.15.17:0/3585891280 shutdown_connections 2024-09-10T21:47:21.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.961+0000 7f7a61a00640 1 -- 172.21.15.17:0/3585891280 >> 172.21.15.17:0/3585891280 conn(0x7f7a5c0748d0 msgr2=0x7f7a5c156a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:21.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.961+0000 7f7a61a00640 1 -- 172.21.15.17:0/3585891280 shutdown_connections 2024-09-10T21:47:21.967 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:21.961+0000 7f7a61a00640 1 -- 172.21.15.17:0/3585891280 wait complete. 2024-09-10T21:47:21.984 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-09-10T21:47:22.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:22.129+0000 7f84a0600640 1 -- 172.21.15.17:0/2803214361 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8498152440 msgr2=0x7f84980b3eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:22.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:22.129+0000 7f84a0600640 1 --2- 172.21.15.17:0/2803214361 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8498152440 0x7f84980b3eb0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f848c009870 tx=0x7f848c01c7d0 comp rx=0 tx=0).stop 2024-09-10T21:47:22.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:22.129+0000 7f84a0600640 1 -- 172.21.15.17:0/2803214361 shutdown_connections 2024-09-10T21:47:22.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:22.129+0000 7f84a0600640 1 --2- 172.21.15.17:0/2803214361 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8498152440 0x7f84980b3eb0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:22.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:22.129+0000 7f84a0600640 1 -- 172.21.15.17:0/2803214361 >> 172.21.15.17:0/2803214361 conn(0x7f84980702f0 msgr2=0x7f8498070700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:22.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:22.129+0000 7f84a0600640 1 -- 172.21.15.17:0/2803214361 shutdown_connections 2024-09-10T21:47:22.135 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:22.129+0000 7f84a0600640 1 -- 172.21.15.17:0/2803214361 wait complete. 2024-09-10T21:47:22.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:22.133+0000 7f84a0600640 1 Processor -- start 2024-09-10T21:47:22.136 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:22.133+0000 7f84a0600640 1 -- start start 2024-09-10T21:47:22.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:22.133+0000 7f84a0600640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f84980cdf20 0x7f849814c760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:22.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:22.133+0000 7f84a0600640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8498152440 0x7f849806f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:22.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:22.133+0000 7f84a0600640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8498152820 0x7f84981f7110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:22.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:22.133+0000 7f84a0600640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f84980cb450 con 0x7f8498152440 2024-09-10T21:47:22.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:22.133+0000 7f84a0600640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f84980ce740 con 0x7f84980cdf20 2024-09-10T21:47:22.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:22.133+0000 7f84a0600640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f84980cb5d0 con 0x7f8498152820 2024-09-10T21:47:22.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:22.133+0000 7f849d200640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8498152440 0x7f849806f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:22.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:22.133+0000 7f849d200640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8498152440 0x7f849806f8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:34756/0 (socket says 172.21.15.17:34756) 2024-09-10T21:47:22.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:22.133+0000 7f849d200640 1 -- 172.21.15.17:0/2964185879 learned_addr learned my addr 172.21.15.17:0/2964185879 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:47:22.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:22.133+0000 7f849dc00640 1 --2- 172.21.15.17:0/2964185879 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f84980cdf20 0x7f849814c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:22.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:22.133+0000 7f849e600640 1 --2- 172.21.15.17:0/2964185879 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8498152820 0x7f84981f7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:22.138 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:22.133+0000 7f849d200640 1 -- 172.21.15.17:0/2964185879 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8498152820 msgr2=0x7f84981f7110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:22.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:22.133+0000 7f849d200640 1 --2- 172.21.15.17:0/2964185879 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8498152820 0x7f84981f7110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:22.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:22.133+0000 7f849d200640 1 -- 172.21.15.17:0/2964185879 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f84980cdf20 msgr2=0x7f849814c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:22.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:22.133+0000 7f849d200640 1 --2- 172.21.15.17:0/2964185879 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f84980cdf20 0x7f849814c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:22.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:22.133+0000 7f849d200640 1 -- 172.21.15.17:0/2964185879 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84981f7b90 con 0x7f8498152440 2024-09-10T21:47:22.139 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:22.133+0000 7f849dc00640 1 --2- 172.21.15.17:0/2964185879 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f84980cdf20 0x7f849814c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:47:22.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:22.133+0000 7f849d200640 1 --2- 172.21.15.17:0/2964185879 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8498152440 0x7f849806f8f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f848400c490 tx=0x7f848400c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:22.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:22.133+0000 7f8495600640 1 -- 172.21.15.17:0/2964185879 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8484017070 con 0x7f8498152440 2024-09-10T21:47:22.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:22.133+0000 7f8495600640 1 -- 172.21.15.17:0/2964185879 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f84840053e0 con 0x7f8498152440 2024-09-10T21:47:22.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:22.133+0000 7f84a0600640 1 -- 172.21.15.17:0/2964185879 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f849806df10 con 0x7f8498152440 2024-09-10T21:47:22.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:22.133+0000 7f8495600640 1 -- 172.21.15.17:0/2964185879 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8484003860 con 0x7f8498152440 2024-09-10T21:47:22.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:22.133+0000 7f84a0600640 1 -- 172.21.15.17:0/2964185879 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84980cade0 con 0x7f8498152440 2024-09-10T21:47:22.140 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:22.137+0000 7f84a0600640 1 -- 172.21.15.17:0/2964185879 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f849806dd30 con 0x7f8498152440 2024-09-10T21:47:22.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:22.137+0000 7f8495600640 1 -- 172.21.15.17:0/2964185879 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f8484003a00 con 0x7f8498152440 2024-09-10T21:47:22.141 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:22.137+0000 7f8495600640 1 --2- 172.21.15.17:0/2964185879 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f84780725e0 0x7f8478074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:22.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:22.137+0000 7f8495600640 1 -- 172.21.15.17:0/2964185879 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f8484092cb0 con 0x7f8498152440 2024-09-10T21:47:22.142 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:22.137+0000 7f849dc00640 1 --2- 172.21.15.17:0/2964185879 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f84780725e0 0x7f8478074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:22.143 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:22.137+0000 7f849dc00640 1 --2- 172.21.15.17:0/2964185879 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f84780725e0 0x7f8478074aa0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f848c0af000 tx=0x7f848c01b010 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:22.145 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:22.141+0000 7f8495600640 1 -- 172.21.15.17:0/2964185879 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f849806dd30 con 0x7f8498152440 2024-09-10T21:47:22.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:22.317+0000 7f84a0600640 1 -- 172.21.15.17:0/2964185879 --> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) -- 0x7f8498158360 con 0x7f84780725e0 2024-09-10T21:47:22.321 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:22.317+0000 7f8495600640 1 -- 172.21.15.17:0/2964185879 <== mgr.4100 v2:172.21.15.31:6808/3101308586 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f8498158360 con 0x7f84780725e0 2024-09-10T21:47:22.322 INFO:teuthology.orchestra.run.smithi017.stderr:instructed osd(s) 2 to deep-scrub 2024-09-10T21:47:22.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:22.321+0000 7f84a0600640 1 -- 172.21.15.17:0/2964185879 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f84780725e0 msgr2=0x7f8478074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:22.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:22.321+0000 7f84a0600640 1 --2- 172.21.15.17:0/2964185879 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f84780725e0 0x7f8478074aa0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f848c0af000 tx=0x7f848c01b010 comp rx=0 tx=0).stop 2024-09-10T21:47:22.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:22.321+0000 7f84a0600640 1 -- 172.21.15.17:0/2964185879 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8498152440 msgr2=0x7f849806f8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:22.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:22.321+0000 7f84a0600640 1 --2- 172.21.15.17:0/2964185879 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8498152440 0x7f849806f8f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f848400c490 tx=0x7f848400c950 comp rx=0 tx=0).stop 2024-09-10T21:47:22.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:22.321+0000 7f84a0600640 1 -- 172.21.15.17:0/2964185879 shutdown_connections 2024-09-10T21:47:22.324 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:22.321+0000 7f84a0600640 1 --2- 172.21.15.17:0/2964185879 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f84780725e0 0x7f8478074aa0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:22.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:22.321+0000 7f84a0600640 1 --2- 172.21.15.17:0/2964185879 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f8498152820 0x7f84981f7110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:22.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:22.321+0000 7f84a0600640 1 --2- 172.21.15.17:0/2964185879 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f8498152440 0x7f849806f8f0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:22.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:22.321+0000 7f84a0600640 1 --2- 172.21.15.17:0/2964185879 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f84980cdf20 0x7f849814c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:22.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:22.321+0000 7f84a0600640 1 -- 172.21.15.17:0/2964185879 >> 172.21.15.17:0/2964185879 conn(0x7f84980702f0 msgr2=0x7f8498156890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:22.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:22.321+0000 7f84a0600640 1 -- 172.21.15.17:0/2964185879 shutdown_connections 2024-09-10T21:47:22.325 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:22.321+0000 7f84a0600640 1 -- 172.21.15.17:0/2964185879 wait complete. 2024-09-10T21:47:22.343 INFO:tasks.ceph:Scrubbing osd.3 2024-09-10T21:47:22.343 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-09-10T21:47:22.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:22.493+0000 7fe832c00640 1 -- 172.21.15.17:0/1714244300 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe82c152520 msgr2=0x7fe82c070dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:22.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:22.493+0000 7fe832c00640 1 --2- 172.21.15.17:0/1714244300 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe82c152520 0x7fe82c070dd0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fe81c009900 tx=0x7fe81c01f0d0 comp rx=0 tx=0).stop 2024-09-10T21:47:22.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:22.493+0000 7fe832c00640 1 -- 172.21.15.17:0/1714244300 shutdown_connections 2024-09-10T21:47:22.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:22.493+0000 7fe832c00640 1 --2- 172.21.15.17:0/1714244300 >> v2:172.21.15.31:3300/0 conn(0x7fe82c0cdf50 0x7fe82c0ce400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:22.496 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:22.493+0000 7fe832c00640 1 --2- 172.21.15.17:0/1714244300 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe82c152520 0x7fe82c070dd0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:22.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:22.493+0000 7fe832c00640 1 -- 172.21.15.17:0/1714244300 >> 172.21.15.17:0/1714244300 conn(0x7fe82c0748d0 msgr2=0x7fe82c068990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:22.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:22.493+0000 7fe832c00640 1 -- 172.21.15.17:0/1714244300 shutdown_connections 2024-09-10T21:47:22.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:22.493+0000 7fe832c00640 1 -- 172.21.15.17:0/1714244300 wait complete. 2024-09-10T21:47:22.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:22.493+0000 7fe832c00640 1 Processor -- start 2024-09-10T21:47:22.497 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:22.493+0000 7fe832c00640 1 -- start start 2024-09-10T21:47:22.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:22.493+0000 7fe832c00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe82c071680 0x7fe82c0c5440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:22.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:22.493+0000 7fe832c00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe82c0cdf50 0x7fe82c0cd690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:22.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:22.493+0000 7fe832c00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe82c152520 0x7fe82c14c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:22.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:22.493+0000 7fe832c00640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fe82c152900 con 0x7fe82c0cdf50 2024-09-10T21:47:22.499 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:22.493+0000 7fe832c00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fe82c0c3270 con 0x7fe82c071680 2024-09-10T21:47:22.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:22.493+0000 7fe832c00640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7fe82c152a80 con 0x7fe82c152520 2024-09-10T21:47:22.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:22.493+0000 7fe82be00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe82c071680 0x7fe82c0c5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:22.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:22.493+0000 7fe82be00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe82c071680 0x7fe82c0c5440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.17:58128/0 (socket says 172.21.15.17:58128) 2024-09-10T21:47:22.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:22.493+0000 7fe82be00640 1 -- 172.21.15.17:0/2386474003 learned_addr learned my addr 172.21.15.17:0/2386474003 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:47:22.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:22.493+0000 7fe823e00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe82c0cdf50 0x7fe82c0cd690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:22.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:22.493+0000 7fe830c00640 1 --2- 172.21.15.17:0/2386474003 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe82c152520 0x7fe82c14c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:22.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:22.497+0000 7fe82be00640 1 -- 172.21.15.17:0/2386474003 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe82c152520 msgr2=0x7fe82c14c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:22.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:22.497+0000 7fe82be00640 1 --2- 172.21.15.17:0/2386474003 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fe82c152520 0x7fe82c14c500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:22.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:22.497+0000 7fe82be00640 1 -- 172.21.15.17:0/2386474003 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe82c0cdf50 msgr2=0x7fe82c0cd690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:22.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:22.497+0000 7fe82be00640 1 --2- 172.21.15.17:0/2386474003 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fe82c0cdf50 0x7fe82c0cd690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:22.500 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:22.497+0000 7fe82be00640 1 -- 172.21.15.17:0/2386474003 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe82c0c9fa0 con 0x7fe82c071680 2024-09-10T21:47:22.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:22.497+0000 7fe82be00640 1 --2- 172.21.15.17:0/2386474003 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe82c071680 0x7fe82c0c5440 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fe81c009850 tx=0x7fe81c005e70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:22.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:22.497+0000 7fe828c00640 1 -- 172.21.15.17:0/2386474003 <== mon.1 v2:172.21.15.31:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe81c0be070 con 0x7fe82c071680 2024-09-10T21:47:22.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:22.497+0000 7fe828c00640 1 -- 172.21.15.17:0/2386474003 <== mon.1 v2:172.21.15.31:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe81c0053e0 con 0x7fe82c071680 2024-09-10T21:47:22.501 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:22.497+0000 7fe832c00640 1 -- 172.21.15.17:0/2386474003 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe82c0caa40 con 0x7fe82c071680 2024-09-10T21:47:22.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:22.497+0000 7fe828c00640 1 -- 172.21.15.17:0/2386474003 <== mon.1 v2:172.21.15.31:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe81c0b5040 con 0x7fe82c071680 2024-09-10T21:47:22.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:22.497+0000 7fe832c00640 1 -- 172.21.15.17:0/2386474003 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe82c14e410 con 0x7fe82c071680 2024-09-10T21:47:22.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:22.497+0000 7fe832c00640 1 -- 172.21.15.17:0/2386474003 --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe82c159860 con 0x7fe82c071680 2024-09-10T21:47:22.502 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:22.497+0000 7fe828c00640 1 -- 172.21.15.17:0/2386474003 <== mon.1 v2:172.21.15.31:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fe81c0b8030 con 0x7fe82c071680 2024-09-10T21:47:22.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:22.501+0000 7fe828c00640 1 --2- 172.21.15.17:0/2386474003 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fe808072600 0x7fe808074ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:22.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:22.501+0000 7fe828c00640 1 -- 172.21.15.17:0/2386474003 <== mon.1 v2:172.21.15.31:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fe81c1308c0 con 0x7fe82c071680 2024-09-10T21:47:22.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:22.501+0000 7fe828c00640 1 --2- 172.21.15.17:0/2386474003 >> [v2:172.21.15.17:6800/2314489522,v1:172.21.15.17:6801/2314489522] conn(0x7fe80807c020 0x7fe80807e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:22.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:22.501+0000 7fe823e00640 1 --2- 172.21.15.17:0/2386474003 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fe808072600 0x7fe808074ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:22.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:22.501+0000 7fe828c00640 1 -- 172.21.15.17:0/2386474003 --> [v2:172.21.15.17:6800/2314489522,v1:172.21.15.17:6801/2314489522] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe81c009080 con 0x7fe80807c020 2024-09-10T21:47:22.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:22.501+0000 7fe828c00640 1 -- 172.21.15.17:0/2386474003 <== mon.1 v2:172.21.15.31:3300/0 6 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fe81c130c70 con 0x7fe82c071680 2024-09-10T21:47:22.505 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:22.501+0000 7fe830c00640 1 --2- 172.21.15.17:0/2386474003 >> [v2:172.21.15.17:6800/2314489522,v1:172.21.15.17:6801/2314489522] conn(0x7fe80807c020 0x7fe80807e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:22.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:22.501+0000 7fe830c00640 1 --2- 172.21.15.17:0/2386474003 >> [v2:172.21.15.17:6800/2314489522,v1:172.21.15.17:6801/2314489522] conn(0x7fe80807c020 0x7fe80807e4c0 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:22.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:22.501+0000 7fe823e00640 1 --2- 172.21.15.17:0/2386474003 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fe808072600 0x7fe808074ac0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fe82c157530 tx=0x7fe814009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:22.506 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:22.501+0000 7fe828c00640 1 -- 172.21.15.17:0/2386474003 <== osd.3 v2:172.21.15.17:6800/2314489522 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe81c009080 con 0x7fe80807c020 2024-09-10T21:47:22.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:22.513+0000 7fe832c00640 1 -- 172.21.15.17:0/2386474003 --> [v2:172.21.15.17:6800/2314489522,v1:172.21.15.17:6801/2314489522] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fe82c06c7e0 con 0x7fe80807c020 2024-09-10T21:47:22.519 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:22.513+0000 7fe828c00640 1 -- 172.21.15.17:0/2386474003 <== osd.3 v2:172.21.15.17:6800/2314489522 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fe82c06c7e0 con 0x7fe80807c020 2024-09-10T21:47:22.519 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-10T21:47:22.520 INFO:teuthology.orchestra.run.smithi017.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-10T21:47:22.520 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-10T21:47:22.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:22.517+0000 7fe832c00640 1 -- 172.21.15.17:0/2386474003 >> [v2:172.21.15.17:6800/2314489522,v1:172.21.15.17:6801/2314489522] conn(0x7fe80807c020 msgr2=0x7fe80807e4c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:22.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:22.517+0000 7fe832c00640 1 --2- 172.21.15.17:0/2386474003 >> [v2:172.21.15.17:6800/2314489522,v1:172.21.15.17:6801/2314489522] conn(0x7fe80807c020 0x7fe80807e4c0 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:22.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:22.517+0000 7fe832c00640 1 -- 172.21.15.17:0/2386474003 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fe808072600 msgr2=0x7fe808074ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:22.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:22.517+0000 7fe832c00640 1 --2- 172.21.15.17:0/2386474003 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fe808072600 0x7fe808074ac0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fe82c157530 tx=0x7fe814009040 comp rx=0 tx=0).stop 2024-09-10T21:47:22.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:22.517+0000 7fe832c00640 1 -- 172.21.15.17:0/2386474003 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe82c071680 msgr2=0x7fe82c0c5440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:22.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:22.517+0000 7fe832c00640 1 --2- 172.21.15.17:0/2386474003 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fe82c071680 0x7fe82c0c5440 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fe81c009850 tx=0x7fe81c005e70 comp rx=0 tx=0).stop 2024-09-10T21:47:22.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:22.517+0000 7fe830c00640 1 -- 172.21.15.17:0/2386474003 reap_dead start 2024-09-10T21:47:22.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:22.517+0000 7fe832c00640 1 -- 172.21.15.17:0/2386474003 shutdown_connections 2024-09-10T21:47:22.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:22.517+0000 7fe832c00640 1 -- 172.21.15.17:0/2386474003 >> 172.21.15.17:0/2386474003 conn(0x7fe82c0748d0 msgr2=0x7fe82c156a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:22.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:22.517+0000 7fe832c00640 1 -- 172.21.15.17:0/2386474003 shutdown_connections 2024-09-10T21:47:22.521 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:22.517+0000 7fe832c00640 1 -- 172.21.15.17:0/2386474003 wait complete. 2024-09-10T21:47:22.538 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-09-10T21:47:22.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:22.681+0000 7fbac4a00640 1 -- 172.21.15.17:0/588895068 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbac01568a0 msgr2=0x7fbac0156cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:22.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:22.681+0000 7fbac4a00640 1 --2- 172.21.15.17:0/588895068 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbac01568a0 0x7fbac0156cf0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fbac00c9d60 tx=0x7fbab801db20 comp rx=0 tx=0).stop 2024-09-10T21:47:22.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:22.681+0000 7fbac4a00640 1 -- 172.21.15.17:0/588895068 shutdown_connections 2024-09-10T21:47:22.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:22.681+0000 7fbac4a00640 1 --2- 172.21.15.17:0/588895068 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbac01568a0 0x7fbac0156cf0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:22.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:22.681+0000 7fbac4a00640 1 --2- 172.21.15.17:0/588895068 >> v2:172.21.15.31:3300/0 conn(0x7fbac0150ad0 0x7fbac0070fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:22.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:22.681+0000 7fbac4a00640 1 -- 172.21.15.17:0/588895068 >> 172.21.15.17:0/588895068 conn(0x7fbac006cc20 msgr2=0x7fbac006d030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:22.687 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:22.681+0000 7fbac4a00640 1 -- 172.21.15.17:0/588895068 shutdown_connections 2024-09-10T21:47:22.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:22.681+0000 7fbac4a00640 1 -- 172.21.15.17:0/588895068 wait complete. 2024-09-10T21:47:22.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:22.685+0000 7fbac4a00640 1 Processor -- start 2024-09-10T21:47:22.688 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:22.685+0000 7fbac4a00640 1 -- start start 2024-09-10T21:47:22.690 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:22.685+0000 7fbac4a00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbac00c78b0 0x7fbac0143f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:22.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:22.685+0000 7fbac4a00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fbac0150ad0 0x7fbac0148730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:22.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:22.685+0000 7fbac4a00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbac01568a0 0x7fbac01420b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:22.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:22.685+0000 7fbac4a00640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fbac0150eb0 con 0x7fbac00c78b0 2024-09-10T21:47:22.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:22.685+0000 7fbac4a00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fbac0157640 con 0x7fbac01568a0 2024-09-10T21:47:22.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:22.685+0000 7fbac4a00640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7fbac0151030 con 0x7fbac0150ad0 2024-09-10T21:47:22.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:22.685+0000 7fbabf400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbac00c78b0 0x7fbac0143f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:22.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:22.685+0000 7fbabf400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbac00c78b0 0x7fbac0143f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:34778/0 (socket says 172.21.15.17:34778) 2024-09-10T21:47:22.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:22.685+0000 7fbabf400640 1 -- 172.21.15.17:0/3854213678 learned_addr learned my addr 172.21.15.17:0/3854213678 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:47:22.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:22.685+0000 7fbabfe00640 1 --2- 172.21.15.17:0/3854213678 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbac01568a0 0x7fbac01420b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:22.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:22.685+0000 7fbab7e00640 1 --2- 172.21.15.17:0/3854213678 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fbac0150ad0 0x7fbac0148730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:22.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:22.685+0000 7fbabf400640 1 -- 172.21.15.17:0/3854213678 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fbac0150ad0 msgr2=0x7fbac0148730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:22.691 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:22.685+0000 7fbabf400640 1 --2- 172.21.15.17:0/3854213678 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fbac0150ad0 0x7fbac0148730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:22.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:22.685+0000 7fbabf400640 1 -- 172.21.15.17:0/3854213678 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbac01568a0 msgr2=0x7fbac01420b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:22.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:22.685+0000 7fbabf400640 1 --2- 172.21.15.17:0/3854213678 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbac01568a0 0x7fbac01420b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:22.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:22.685+0000 7fbabf400640 1 -- 172.21.15.17:0/3854213678 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbac0146a30 con 0x7fbac00c78b0 2024-09-10T21:47:22.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:22.685+0000 7fbabf400640 1 --2- 172.21.15.17:0/3854213678 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbac00c78b0 0x7fbac0143f00 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fbab000ea20 tx=0x7fbab000eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:22.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:22.685+0000 7fbab7400640 1 -- 172.21.15.17:0/3854213678 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbab000cd80 con 0x7fbac00c78b0 2024-09-10T21:47:22.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:22.685+0000 7fbab7400640 1 -- 172.21.15.17:0/3854213678 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbab0010430 con 0x7fbac00c78b0 2024-09-10T21:47:22.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:22.685+0000 7fbab7400640 1 -- 172.21.15.17:0/3854213678 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbab0010860 con 0x7fbac00c78b0 2024-09-10T21:47:22.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:22.685+0000 7fbac4a00640 1 -- 172.21.15.17:0/3854213678 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbac0146d10 con 0x7fbac00c78b0 2024-09-10T21:47:22.692 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:22.685+0000 7fbac4a00640 1 -- 172.21.15.17:0/3854213678 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbac014e200 con 0x7fbac00c78b0 2024-09-10T21:47:22.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:22.689+0000 7fbac4a00640 1 -- 172.21.15.17:0/3854213678 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbac0146f50 con 0x7fbac00c78b0 2024-09-10T21:47:22.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:22.689+0000 7fbab7400640 1 -- 172.21.15.17:0/3854213678 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fbab0010a00 con 0x7fbac00c78b0 2024-09-10T21:47:22.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:22.689+0000 7fbab7400640 1 --2- 172.21.15.17:0/3854213678 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fba9c072610 0x7fba9c074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:22.693 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:22.689+0000 7fbab7400640 1 -- 172.21.15.17:0/3854213678 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fbab0094190 con 0x7fbac00c78b0 2024-09-10T21:47:22.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:22.689+0000 7fbab7e00640 1 --2- 172.21.15.17:0/3854213678 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fba9c072610 0x7fba9c074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:22.694 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:22.689+0000 7fbab7e00640 1 --2- 172.21.15.17:0/3854213678 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fba9c072610 0x7fba9c074ad0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fbac00c7f40 tx=0x7fbaa8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:22.697 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:22.693+0000 7fbab7400640 1 -- 172.21.15.17:0/3854213678 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fbac0146f50 con 0x7fbac00c78b0 2024-09-10T21:47:22.872 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:22.869+0000 7fbac4a00640 1 -- 172.21.15.17:0/3854213678 --> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) -- 0x7fbac00c88a0 con 0x7fba9c072610 2024-09-10T21:47:22.873 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:22.869+0000 7fbab7400640 1 -- 172.21.15.17:0/3854213678 <== mgr.4100 v2:172.21.15.31:6808/3101308586 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fbac00c88a0 con 0x7fba9c072610 2024-09-10T21:47:22.873 INFO:teuthology.orchestra.run.smithi017.stderr:instructed osd(s) 3 to deep-scrub 2024-09-10T21:47:22.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:22.869+0000 7fbac4a00640 1 -- 172.21.15.17:0/3854213678 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fba9c072610 msgr2=0x7fba9c074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:22.875 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:22.869+0000 7fbac4a00640 1 --2- 172.21.15.17:0/3854213678 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fba9c072610 0x7fba9c074ad0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fbac00c7f40 tx=0x7fbaa8009040 comp rx=0 tx=0).stop 2024-09-10T21:47:22.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:22.869+0000 7fbac4a00640 1 -- 172.21.15.17:0/3854213678 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbac00c78b0 msgr2=0x7fbac0143f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:22.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:22.869+0000 7fbac4a00640 1 --2- 172.21.15.17:0/3854213678 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbac00c78b0 0x7fbac0143f00 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fbab000ea20 tx=0x7fbab000eee0 comp rx=0 tx=0).stop 2024-09-10T21:47:22.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:22.869+0000 7fbac4a00640 1 -- 172.21.15.17:0/3854213678 shutdown_connections 2024-09-10T21:47:22.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:22.869+0000 7fbac4a00640 1 --2- 172.21.15.17:0/3854213678 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fba9c072610 0x7fba9c074ad0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:22.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:22.869+0000 7fbac4a00640 1 --2- 172.21.15.17:0/3854213678 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbac01568a0 0x7fbac01420b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:22.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:22.869+0000 7fbac4a00640 1 --2- 172.21.15.17:0/3854213678 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fbac0150ad0 0x7fbac0148730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:22.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:22.869+0000 7fbac4a00640 1 --2- 172.21.15.17:0/3854213678 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbac00c78b0 0x7fbac0143f00 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:22.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:22.869+0000 7fbac4a00640 1 -- 172.21.15.17:0/3854213678 >> 172.21.15.17:0/3854213678 conn(0x7fbac006cc20 msgr2=0x7fbac006d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:22.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:22.869+0000 7fbac4a00640 1 -- 172.21.15.17:0/3854213678 shutdown_connections 2024-09-10T21:47:22.876 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:22.869+0000 7fbac4a00640 1 -- 172.21.15.17:0/3854213678 wait complete. 2024-09-10T21:47:22.897 INFO:tasks.ceph:Scrubbing osd.4 2024-09-10T21:47:22.897 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-09-10T21:47:23.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.045+0000 7f43afe00640 1 -- 172.21.15.17:0/2626783571 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f43b006dcb0 msgr2=0x7f43b0073aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:23.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.045+0000 7f43afe00640 1 --2- 172.21.15.17:0/2626783571 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f43b006dcb0 0x7f43b0073aa0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f4398009900 tx=0x7f439801c8e0 comp rx=0 tx=0).stop 2024-09-10T21:47:23.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.045+0000 7f43afe00640 1 -- 172.21.15.17:0/2626783571 shutdown_connections 2024-09-10T21:47:23.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.045+0000 7f43afe00640 1 --2- 172.21.15.17:0/2626783571 >> v2:172.21.15.31:3300/0 conn(0x7f43b00c78f0 0x7f43b0073fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:23.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.045+0000 7f43afe00640 1 --2- 172.21.15.17:0/2626783571 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f43b006dcb0 0x7f43b0073aa0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:23.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.045+0000 7f43afe00640 1 -- 172.21.15.17:0/2626783571 >> 172.21.15.17:0/2626783571 conn(0x7f43b006ff70 msgr2=0x7f43b0070380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:23.049 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.045+0000 7f43afe00640 1 -- 172.21.15.17:0/2626783571 shutdown_connections 2024-09-10T21:47:23.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.045+0000 7f43afe00640 1 -- 172.21.15.17:0/2626783571 wait complete. 2024-09-10T21:47:23.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.045+0000 7f43afe00640 1 Processor -- start 2024-09-10T21:47:23.050 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.045+0000 7f43afe00640 1 -- start start 2024-09-10T21:47:23.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.045+0000 7f43afe00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f43b006dcb0 0x7f43b00c9bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:23.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.045+0000 7f43afe00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f43b00c78f0 0x7f43b00ca110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:23.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.045+0000 7f43afe00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f43b01542b0 0x7f43b01500a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:23.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.045+0000 7f43afe00640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f43b0154810 con 0x7f43b006dcb0 2024-09-10T21:47:23.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.045+0000 7f43afe00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f43b0154690 con 0x7f43b01542b0 2024-09-10T21:47:23.051 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.045+0000 7f43afe00640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f43b006d330 con 0x7f43b00c78f0 2024-09-10T21:47:23.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.049+0000 7f43aea00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f43b006dcb0 0x7f43b00c9bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:23.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.049+0000 7f43aea00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f43b006dcb0 0x7f43b00c9bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:34780/0 (socket says 172.21.15.17:34780) 2024-09-10T21:47:23.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.049+0000 7f43aea00640 1 -- 172.21.15.17:0/1161580475 learned_addr learned my addr 172.21.15.17:0/1161580475 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:47:23.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.049+0000 7f43a7e00640 1 --2- 172.21.15.17:0/1161580475 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f43b00c78f0 0x7f43b00ca110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:23.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.049+0000 7f43af400640 1 --2- 172.21.15.17:0/1161580475 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f43b01542b0 0x7f43b01500a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:23.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.049+0000 7f43a7e00640 1 -- 172.21.15.17:0/1161580475 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f43b01542b0 msgr2=0x7f43b01500a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:23.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.049+0000 7f43a7e00640 1 --2- 172.21.15.17:0/1161580475 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f43b01542b0 0x7f43b01500a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:23.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.049+0000 7f43a7e00640 1 -- 172.21.15.17:0/1161580475 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f43b006dcb0 msgr2=0x7f43b00c9bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:23.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.049+0000 7f43a7e00640 1 --2- 172.21.15.17:0/1161580475 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f43b006dcb0 0x7f43b00c9bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:23.053 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.049+0000 7f43a7e00640 1 -- 172.21.15.17:0/1161580475 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43b014e950 con 0x7f43b00c78f0 2024-09-10T21:47:23.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.049+0000 7f43aea00640 1 --2- 172.21.15.17:0/1161580475 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f43b006dcb0 0x7f43b00c9bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:47:23.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.049+0000 7f43a7e00640 1 --2- 172.21.15.17:0/1161580475 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f43b00c78f0 0x7f43b00ca110 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f43b0157c90 tx=0x7f4398005e70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:23.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.049+0000 7f43a6a00640 1 -- 172.21.15.17:0/1161580475 <== mon.2 v2:172.21.15.31:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43980b2040 con 0x7f43b00c78f0 2024-09-10T21:47:23.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.049+0000 7f43a6a00640 1 -- 172.21.15.17:0/1161580475 <== mon.2 v2:172.21.15.31:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4398005320 con 0x7f43b00c78f0 2024-09-10T21:47:23.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.049+0000 7f43a6a00640 1 -- 172.21.15.17:0/1161580475 <== mon.2 v2:172.21.15.31:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43980046d0 con 0x7f43b00c78f0 2024-09-10T21:47:23.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.049+0000 7f43afe00640 1 -- 172.21.15.17:0/1161580475 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43b014ec30 con 0x7f43b00c78f0 2024-09-10T21:47:23.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.049+0000 7f43afe00640 1 -- 172.21.15.17:0/1161580475 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f43b014f8a0 con 0x7f43b00c78f0 2024-09-10T21:47:23.054 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.049+0000 7f43afe00640 1 -- 172.21.15.17:0/1161580475 --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f43b0157690 con 0x7f43b00c78f0 2024-09-10T21:47:23.055 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.049+0000 7f43a6a00640 1 -- 172.21.15.17:0/1161580475 <== mon.2 v2:172.21.15.31:3301/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f4398004890 con 0x7f43b00c78f0 2024-09-10T21:47:23.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.053+0000 7f43a6a00640 1 --2- 172.21.15.17:0/1161580475 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f438c0725e0 0x7f438c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:23.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.053+0000 7f43a6a00640 1 -- 172.21.15.17:0/1161580475 <== mon.2 v2:172.21.15.31:3301/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f439812d520 con 0x7f43b00c78f0 2024-09-10T21:47:23.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.053+0000 7f43a6a00640 1 --2- 172.21.15.17:0/1161580475 >> [v2:172.21.15.31:6810/202520744,v1:172.21.15.31:6811/202520744] conn(0x7f438c07bff0 0x7f438c07e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:23.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.053+0000 7f43a6a00640 1 -- 172.21.15.17:0/1161580475 --> [v2:172.21.15.31:6810/202520744,v1:172.21.15.31:6811/202520744] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4398007e80 con 0x7f438c07bff0 2024-09-10T21:47:23.056 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.053+0000 7f43a6a00640 1 -- 172.21.15.17:0/1161580475 <== mon.2 v2:172.21.15.31:3301/0 6 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f439812d8d0 con 0x7f43b00c78f0 2024-09-10T21:47:23.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.053+0000 7f43aea00640 1 --2- 172.21.15.17:0/1161580475 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f438c0725e0 0x7f438c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:23.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.053+0000 7f43af400640 1 --2- 172.21.15.17:0/1161580475 >> [v2:172.21.15.31:6810/202520744,v1:172.21.15.31:6811/202520744] conn(0x7f438c07bff0 0x7f438c07e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:23.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.053+0000 7f43aea00640 1 --2- 172.21.15.17:0/1161580475 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f438c0725e0 0x7f438c074aa0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f43a0001c60 tx=0x7f43a0009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:23.057 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.053+0000 7f43af400640 1 --2- 172.21.15.17:0/1161580475 >> [v2:172.21.15.31:6810/202520744,v1:172.21.15.31:6811/202520744] conn(0x7f438c07bff0 0x7f438c07e490 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:23.058 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.053+0000 7f43a6a00640 1 -- 172.21.15.17:0/1161580475 <== osd.4 v2:172.21.15.31:6810/202520744 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4398007e80 con 0x7f438c07bff0 2024-09-10T21:47:23.071 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.069+0000 7f43afe00640 1 -- 172.21.15.17:0/1161580475 --> [v2:172.21.15.31:6810/202520744,v1:172.21.15.31:6811/202520744] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f43b0069e60 con 0x7f438c07bff0 2024-09-10T21:47:23.072 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.069+0000 7f43a6a00640 1 -- 172.21.15.17:0/1161580475 <== osd.4 v2:172.21.15.31:6810/202520744 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f43b0069e60 con 0x7f438c07bff0 2024-09-10T21:47:23.072 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-10T21:47:23.072 INFO:teuthology.orchestra.run.smithi017.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-10T21:47:23.072 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-10T21:47:23.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.069+0000 7f43afe00640 1 -- 172.21.15.17:0/1161580475 >> [v2:172.21.15.31:6810/202520744,v1:172.21.15.31:6811/202520744] conn(0x7f438c07bff0 msgr2=0x7f438c07e490 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:23.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.069+0000 7f43afe00640 1 --2- 172.21.15.17:0/1161580475 >> [v2:172.21.15.31:6810/202520744,v1:172.21.15.31:6811/202520744] conn(0x7f438c07bff0 0x7f438c07e490 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:23.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.069+0000 7f43afe00640 1 -- 172.21.15.17:0/1161580475 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f438c0725e0 msgr2=0x7f438c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:23.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.069+0000 7f43afe00640 1 --2- 172.21.15.17:0/1161580475 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f438c0725e0 0x7f438c074aa0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f43a0001c60 tx=0x7f43a0009040 comp rx=0 tx=0).stop 2024-09-10T21:47:23.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.069+0000 7f43afe00640 1 -- 172.21.15.17:0/1161580475 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f43b00c78f0 msgr2=0x7f43b00ca110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:23.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.069+0000 7f43afe00640 1 --2- 172.21.15.17:0/1161580475 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f43b00c78f0 0x7f43b00ca110 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f43b0157c90 tx=0x7f4398005e70 comp rx=0 tx=0).stop 2024-09-10T21:47:23.073 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.069+0000 7f43af400640 1 -- 172.21.15.17:0/1161580475 reap_dead start 2024-09-10T21:47:23.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.069+0000 7f43afe00640 1 -- 172.21.15.17:0/1161580475 shutdown_connections 2024-09-10T21:47:23.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.069+0000 7f43afe00640 1 -- 172.21.15.17:0/1161580475 >> 172.21.15.17:0/1161580475 conn(0x7f43b006ff70 msgr2=0x7f43b0073aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:23.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.069+0000 7f43afe00640 1 -- 172.21.15.17:0/1161580475 shutdown_connections 2024-09-10T21:47:23.074 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.069+0000 7f43afe00640 1 -- 172.21.15.17:0/1161580475 wait complete. 2024-09-10T21:47:23.094 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-09-10T21:47:23.246 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.241+0000 7f3823400640 1 -- 172.21.15.17:0/2985264108 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f381c151530 msgr2=0x7f381c070330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:23.246 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.241+0000 7f3823400640 1 --2- 172.21.15.17:0/2985264108 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f381c151530 0x7f381c070330 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f3810009840 tx=0x7f381001f010 comp rx=0 tx=0).stop 2024-09-10T21:47:23.246 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.241+0000 7f3823400640 1 -- 172.21.15.17:0/2985264108 shutdown_connections 2024-09-10T21:47:23.246 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.241+0000 7f3823400640 1 --2- 172.21.15.17:0/2985264108 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f381c151530 0x7f381c070330 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:23.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.241+0000 7f3823400640 1 -- 172.21.15.17:0/2985264108 >> 172.21.15.17:0/2985264108 conn(0x7f381c06e130 msgr2=0x7f381c06e540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:23.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.241+0000 7f3823400640 1 -- 172.21.15.17:0/2985264108 shutdown_connections 2024-09-10T21:47:23.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.241+0000 7f3823400640 1 -- 172.21.15.17:0/2985264108 wait complete. 2024-09-10T21:47:23.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.245+0000 7f3823400640 1 Processor -- start 2024-09-10T21:47:23.247 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.245+0000 7f3823400640 1 -- start start 2024-09-10T21:47:23.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.245+0000 7f3823400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f381c073690 0x7f381c14afe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:23.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.245+0000 7f3823400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f381c151530 0x7f381c0c9a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:23.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.245+0000 7f3823400640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f381c15acf0 0x7f381c1f11b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:23.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.245+0000 7f3823400640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f381c070870 con 0x7f381c151530 2024-09-10T21:47:23.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.245+0000 7f3823400640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f381c158090 con 0x7f381c073690 2024-09-10T21:47:23.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.245+0000 7f3823400640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f381c0709f0 con 0x7f381c15acf0 2024-09-10T21:47:23.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.245+0000 7f381be00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f381c151530 0x7f381c0c9a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:23.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.245+0000 7f3820a00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f381c073690 0x7f381c14afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:23.249 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.245+0000 7f381be00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f381c151530 0x7f381c0c9a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:34796/0 (socket says 172.21.15.17:34796) 2024-09-10T21:47:23.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.245+0000 7f381be00640 1 -- 172.21.15.17:0/3498200479 learned_addr learned my addr 172.21.15.17:0/3498200479 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:47:23.250 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.245+0000 7f3821400640 1 --2- 172.21.15.17:0/3498200479 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f381c15acf0 0x7f381c1f11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:23.251 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.245+0000 7f381be00640 1 -- 172.21.15.17:0/3498200479 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f381c15acf0 msgr2=0x7f381c1f11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:23.251 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.245+0000 7f381be00640 1 --2- 172.21.15.17:0/3498200479 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f381c15acf0 0x7f381c1f11b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:23.251 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.245+0000 7f381be00640 1 -- 172.21.15.17:0/3498200479 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f381c073690 msgr2=0x7f381c14afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:23.251 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.245+0000 7f381be00640 1 --2- 172.21.15.17:0/3498200479 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f381c073690 0x7f381c14afe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:23.251 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.245+0000 7f381be00640 1 -- 172.21.15.17:0/3498200479 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f381c0b2350 con 0x7f381c151530 2024-09-10T21:47:23.251 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.245+0000 7f381be00640 1 --2- 172.21.15.17:0/3498200479 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f381c151530 0x7f381c0c9a10 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f380800c490 tx=0x7f3808001e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:23.251 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.245+0000 7f3818c00640 1 -- 172.21.15.17:0/3498200479 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3808017070 con 0x7f381c151530 2024-09-10T21:47:23.251 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.245+0000 7f3818c00640 1 -- 172.21.15.17:0/3498200479 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3808003600 con 0x7f381c151530 2024-09-10T21:47:23.251 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.245+0000 7f3818c00640 1 -- 172.21.15.17:0/3498200479 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3808003a50 con 0x7f381c151530 2024-09-10T21:47:23.251 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.245+0000 7f3823400640 1 -- 172.21.15.17:0/3498200479 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f381c1f4b00 con 0x7f381c151530 2024-09-10T21:47:23.251 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.245+0000 7f3823400640 1 -- 172.21.15.17:0/3498200479 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f381c0b1d90 con 0x7f381c151530 2024-09-10T21:47:23.251 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.245+0000 7f3823400640 1 -- 172.21.15.17:0/3498200479 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f381c151910 con 0x7f381c151530 2024-09-10T21:47:23.252 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.249+0000 7f3818c00640 1 -- 172.21.15.17:0/3498200479 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f3808003bf0 con 0x7f381c151530 2024-09-10T21:47:23.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.249+0000 7f3818c00640 1 --2- 172.21.15.17:0/3498200479 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f37fc0725e0 0x7f37fc074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:23.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.249+0000 7f3818c00640 1 -- 172.21.15.17:0/3498200479 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f3808092ab0 con 0x7f381c151530 2024-09-10T21:47:23.253 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.249+0000 7f3820a00640 1 --2- 172.21.15.17:0/3498200479 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f37fc0725e0 0x7f37fc074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:23.254 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.249+0000 7f3820a00640 1 --2- 172.21.15.17:0/3498200479 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f37fc0725e0 0x7f37fc074aa0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f38100b2000 tx=0x7f381001d850 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:23.261 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.257+0000 7f3818c00640 1 -- 172.21.15.17:0/3498200479 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f380805d590 con 0x7f381c151530 2024-09-10T21:47:23.440 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.437+0000 7f3823400640 1 -- 172.21.15.17:0/3498200479 --> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) -- 0x7f381c071fd0 con 0x7f37fc0725e0 2024-09-10T21:47:23.441 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.437+0000 7f3818c00640 1 -- 172.21.15.17:0/3498200479 <== mgr.4100 v2:172.21.15.31:6808/3101308586 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f381c071fd0 con 0x7f37fc0725e0 2024-09-10T21:47:23.441 INFO:teuthology.orchestra.run.smithi017.stderr:instructed osd(s) 4 to deep-scrub 2024-09-10T21:47:23.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.437+0000 7f3823400640 1 -- 172.21.15.17:0/3498200479 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f37fc0725e0 msgr2=0x7f37fc074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:23.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.437+0000 7f3823400640 1 --2- 172.21.15.17:0/3498200479 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f37fc0725e0 0x7f37fc074aa0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f38100b2000 tx=0x7f381001d850 comp rx=0 tx=0).stop 2024-09-10T21:47:23.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.437+0000 7f3823400640 1 -- 172.21.15.17:0/3498200479 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f381c151530 msgr2=0x7f381c0c9a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:23.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.437+0000 7f3823400640 1 --2- 172.21.15.17:0/3498200479 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f381c151530 0x7f381c0c9a10 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f380800c490 tx=0x7f3808001e80 comp rx=0 tx=0).stop 2024-09-10T21:47:23.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.437+0000 7f3823400640 1 -- 172.21.15.17:0/3498200479 shutdown_connections 2024-09-10T21:47:23.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.437+0000 7f3823400640 1 --2- 172.21.15.17:0/3498200479 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f37fc0725e0 0x7f37fc074aa0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:23.444 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.437+0000 7f3823400640 1 --2- 172.21.15.17:0/3498200479 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f381c15acf0 0x7f381c1f11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:23.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.437+0000 7f3823400640 1 --2- 172.21.15.17:0/3498200479 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f381c151530 0x7f381c0c9a10 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:23.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.437+0000 7f3823400640 1 --2- 172.21.15.17:0/3498200479 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f381c073690 0x7f381c14afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:23.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.437+0000 7f3823400640 1 -- 172.21.15.17:0/3498200479 >> 172.21.15.17:0/3498200479 conn(0x7f381c06e130 msgr2=0x7f381c141d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:23.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.437+0000 7f3823400640 1 -- 172.21.15.17:0/3498200479 shutdown_connections 2024-09-10T21:47:23.445 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.437+0000 7f3823400640 1 -- 172.21.15.17:0/3498200479 wait complete. 2024-09-10T21:47:23.464 INFO:tasks.ceph:Scrubbing osd.5 2024-09-10T21:47:23.464 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-09-10T21:47:23.617 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.609+0000 7f6657400640 1 -- 172.21.15.17:0/531804916 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6650152520 msgr2=0x7f6650070dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:23.617 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.609+0000 7f6657400640 1 --2- 172.21.15.17:0/531804916 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6650152520 0x7f6650070dd0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f6640009900 tx=0x7f664001f0d0 comp rx=0 tx=0).stop 2024-09-10T21:47:23.617 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.609+0000 7f6657400640 1 -- 172.21.15.17:0/531804916 shutdown_connections 2024-09-10T21:47:23.617 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.609+0000 7f6657400640 1 --2- 172.21.15.17:0/531804916 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6650152520 0x7f6650070dd0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:23.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.609+0000 7f6657400640 1 -- 172.21.15.17:0/531804916 >> 172.21.15.17:0/531804916 conn(0x7f66500748d0 msgr2=0x7f6650068990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:23.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.609+0000 7f6657400640 1 -- 172.21.15.17:0/531804916 shutdown_connections 2024-09-10T21:47:23.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.609+0000 7f6657400640 1 -- 172.21.15.17:0/531804916 wait complete. 2024-09-10T21:47:23.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.613+0000 7f6657400640 1 Processor -- start 2024-09-10T21:47:23.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.613+0000 7f6657400640 1 -- start start 2024-09-10T21:47:23.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.613+0000 7f6657400640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6650071680 0x7f665006de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:23.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.613+0000 7f6657400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f66500cdf50 0x7f6650156d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:23.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.613+0000 7f6657400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6650152520 0x7f66500ca590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:23.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.613+0000 7f6657400640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f6650152900 con 0x7f6650152520 2024-09-10T21:47:23.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.613+0000 7f6657400640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f66500c3270 con 0x7f66500cdf50 2024-09-10T21:47:23.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.613+0000 7f6657400640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f6650152a80 con 0x7f6650071680 2024-09-10T21:47:23.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.613+0000 7f6655400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6650152520 0x7f66500ca590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:23.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.613+0000 7f6654a00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6650071680 0x7f665006de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:23.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.613+0000 7f6655400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6650152520 0x7f66500ca590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:34812/0 (socket says 172.21.15.17:34812) 2024-09-10T21:47:23.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.613+0000 7f6655400640 1 -- 172.21.15.17:0/2884133593 learned_addr learned my addr 172.21.15.17:0/2884133593 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:47:23.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.613+0000 7f6647e00640 1 --2- 172.21.15.17:0/2884133593 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f66500cdf50 0x7f6650156d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:23.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.613+0000 7f6655400640 1 -- 172.21.15.17:0/2884133593 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6650071680 msgr2=0x7f665006de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:23.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.613+0000 7f6655400640 1 --2- 172.21.15.17:0/2884133593 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f6650071680 0x7f665006de40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:23.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.613+0000 7f6655400640 1 -- 172.21.15.17:0/2884133593 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f66500cdf50 msgr2=0x7f6650156d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:23.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.613+0000 7f6655400640 1 --2- 172.21.15.17:0/2884133593 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f66500cdf50 0x7f6650156d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:23.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.613+0000 7f6655400640 1 -- 172.21.15.17:0/2884133593 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66500cbe40 con 0x7f6650152520 2024-09-10T21:47:23.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.613+0000 7f6655400640 1 --2- 172.21.15.17:0/2884133593 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6650152520 0x7f66500ca590 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f664800c730 tx=0x7f664800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:23.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.613+0000 7f664d600640 1 -- 172.21.15.17:0/2884133593 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6648003ba0 con 0x7f6650152520 2024-09-10T21:47:23.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.613+0000 7f664d600640 1 -- 172.21.15.17:0/2884133593 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6648003d40 con 0x7f6650152520 2024-09-10T21:47:23.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.613+0000 7f6657400640 1 -- 172.21.15.17:0/2884133593 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66500cc380 con 0x7f6650152520 2024-09-10T21:47:23.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.613+0000 7f664d600640 1 -- 172.21.15.17:0/2884133593 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6648005330 con 0x7f6650152520 2024-09-10T21:47:23.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.613+0000 7f6657400640 1 -- 172.21.15.17:0/2884133593 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66500c9ab0 con 0x7f6650152520 2024-09-10T21:47:23.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.617+0000 7f6657400640 1 -- 172.21.15.17:0/2884133593 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6650159860 con 0x7f6650152520 2024-09-10T21:47:23.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.617+0000 7f664d600640 1 -- 172.21.15.17:0/2884133593 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f664800a640 con 0x7f6650152520 2024-09-10T21:47:23.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.617+0000 7f664d600640 1 --2- 172.21.15.17:0/2884133593 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f662c072610 0x7f662c074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:23.623 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.617+0000 7f664d600640 1 -- 172.21.15.17:0/2884133593 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f6648093410 con 0x7f6650152520 2024-09-10T21:47:23.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.617+0000 7f664d600640 1 --2- 172.21.15.17:0/2884133593 >> [v2:172.21.15.31:6800/1472711057,v1:172.21.15.31:6801/1472711057] conn(0x7f662c07c020 0x7f662c07e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:23.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.617+0000 7f664d600640 1 -- 172.21.15.17:0/2884133593 --> [v2:172.21.15.31:6800/1472711057,v1:172.21.15.31:6801/1472711057] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f662c07eb90 con 0x7f662c07c020 2024-09-10T21:47:23.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.617+0000 7f664d600640 1 -- 172.21.15.17:0/2884133593 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f664809a050 con 0x7f6650152520 2024-09-10T21:47:23.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.617+0000 7f6654a00640 1 --2- 172.21.15.17:0/2884133593 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f662c072610 0x7f662c074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:23.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.617+0000 7f6647e00640 1 --2- 172.21.15.17:0/2884133593 >> [v2:172.21.15.31:6800/1472711057,v1:172.21.15.31:6801/1472711057] conn(0x7f662c07c020 0x7f662c07e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:23.624 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.621+0000 7f6654a00640 1 --2- 172.21.15.17:0/2884133593 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f662c072610 0x7f662c074ad0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f66400b2000 tx=0x7f664001d910 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:23.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.621+0000 7f6647e00640 1 --2- 172.21.15.17:0/2884133593 >> [v2:172.21.15.31:6800/1472711057,v1:172.21.15.31:6801/1472711057] conn(0x7f662c07c020 0x7f662c07e4c0 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:23.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.621+0000 7f664d600640 1 -- 172.21.15.17:0/2884133593 <== osd.5 v2:172.21.15.31:6800/1472711057 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f662c07eb90 con 0x7f662c07c020 2024-09-10T21:47:23.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.633+0000 7f6657400640 1 -- 172.21.15.17:0/2884133593 --> [v2:172.21.15.31:6800/1472711057,v1:172.21.15.31:6801/1472711057] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f66500cb8e0 con 0x7f662c07c020 2024-09-10T21:47:23.638 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.633+0000 7f664d600640 1 -- 172.21.15.17:0/2884133593 <== osd.5 v2:172.21.15.31:6800/1472711057 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f66500cb8e0 con 0x7f662c07c020 2024-09-10T21:47:23.639 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-10T21:47:23.639 INFO:teuthology.orchestra.run.smithi017.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-10T21:47:23.639 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-10T21:47:23.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.633+0000 7f6657400640 1 -- 172.21.15.17:0/2884133593 >> [v2:172.21.15.31:6800/1472711057,v1:172.21.15.31:6801/1472711057] conn(0x7f662c07c020 msgr2=0x7f662c07e4c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:23.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.633+0000 7f6657400640 1 --2- 172.21.15.17:0/2884133593 >> [v2:172.21.15.31:6800/1472711057,v1:172.21.15.31:6801/1472711057] conn(0x7f662c07c020 0x7f662c07e4c0 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:23.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.633+0000 7f6657400640 1 -- 172.21.15.17:0/2884133593 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f662c072610 msgr2=0x7f662c074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:23.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.633+0000 7f6657400640 1 --2- 172.21.15.17:0/2884133593 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f662c072610 0x7f662c074ad0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f66400b2000 tx=0x7f664001d910 comp rx=0 tx=0).stop 2024-09-10T21:47:23.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.633+0000 7f6657400640 1 -- 172.21.15.17:0/2884133593 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6650152520 msgr2=0x7f66500ca590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:23.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.633+0000 7f6657400640 1 --2- 172.21.15.17:0/2884133593 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6650152520 0x7f66500ca590 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f664800c730 tx=0x7f664800cbf0 comp rx=0 tx=0).stop 2024-09-10T21:47:23.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.633+0000 7f6655400640 1 -- 172.21.15.17:0/2884133593 reap_dead start 2024-09-10T21:47:23.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.633+0000 7f6657400640 1 -- 172.21.15.17:0/2884133593 shutdown_connections 2024-09-10T21:47:23.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.633+0000 7f6657400640 1 -- 172.21.15.17:0/2884133593 >> 172.21.15.17:0/2884133593 conn(0x7f66500748d0 msgr2=0x7f665015a360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:23.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.637+0000 7f6657400640 1 -- 172.21.15.17:0/2884133593 shutdown_connections 2024-09-10T21:47:23.641 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.637+0000 7f6657400640 1 -- 172.21.15.17:0/2884133593 wait complete. 2024-09-10T21:47:23.658 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-09-10T21:47:23.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.805+0000 7f1d69c00640 1 -- 172.21.15.17:0/1293367028 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f1d640c9520 msgr2=0x7f1d64162aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:23.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.805+0000 7f1d69c00640 1 --2- 172.21.15.17:0/1293367028 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f1d640c9520 0x7f1d64162aa0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f1d5c009bc0 tx=0x7f1d5c01ca80 comp rx=0 tx=0).stop 2024-09-10T21:47:23.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.805+0000 7f1d69c00640 1 -- 172.21.15.17:0/1293367028 shutdown_connections 2024-09-10T21:47:23.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.805+0000 7f1d69c00640 1 --2- 172.21.15.17:0/1293367028 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f1d640c9520 0x7f1d64162aa0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:23.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.805+0000 7f1d69c00640 1 --2- 172.21.15.17:0/1293367028 >> v2:172.21.15.31:3300/0 conn(0x7f1d641523d0 0x7f1d640c9f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:23.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.805+0000 7f1d69c00640 1 -- 172.21.15.17:0/1293367028 >> 172.21.15.17:0/1293367028 conn(0x7f1d64068990 msgr2=0x7f1d64068da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:23.808 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.805+0000 7f1d69c00640 1 -- 172.21.15.17:0/1293367028 shutdown_connections 2024-09-10T21:47:23.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.805+0000 7f1d69c00640 1 -- 172.21.15.17:0/1293367028 wait complete. 2024-09-10T21:47:23.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.805+0000 7f1d69c00640 1 Processor -- start 2024-09-10T21:47:23.809 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.805+0000 7f1d69c00640 1 -- start start 2024-09-10T21:47:23.810 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.805+0000 7f1d69c00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1d640c9520 0x7f1d641479c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:23.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.805+0000 7f1d69c00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1d640ca4a0 0x7f1d6406e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:23.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.805+0000 7f1d69c00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f1d641523d0 0x7f1d6414b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:23.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.805+0000 7f1d69c00640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f1d64071320 con 0x7f1d640ca4a0 2024-09-10T21:47:23.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.805+0000 7f1d69c00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f1d640cb3c0 con 0x7f1d640c9520 2024-09-10T21:47:23.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.805+0000 7f1d69c00640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f1d640714a0 con 0x7f1d641523d0 2024-09-10T21:47:23.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.805+0000 7f1d5be00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1d640ca4a0 0x7f1d6406e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:23.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.805+0000 7f1d63e00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1d640c9520 0x7f1d641479c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:23.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.805+0000 7f1d5be00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1d640ca4a0 0x7f1d6406e3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:34822/0 (socket says 172.21.15.17:34822) 2024-09-10T21:47:23.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.805+0000 7f1d63e00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1d640c9520 0x7f1d641479c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3300/0 says I am v2:172.21.15.17:58208/0 (socket says 172.21.15.17:58208) 2024-09-10T21:47:23.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.805+0000 7f1d5be00640 1 -- 172.21.15.17:0/1281433607 learned_addr learned my addr 172.21.15.17:0/1281433607 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:47:23.811 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.805+0000 7f1d69200640 1 --2- 172.21.15.17:0/1281433607 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f1d641523d0 0x7f1d6414b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:23.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.805+0000 7f1d5be00640 1 -- 172.21.15.17:0/1281433607 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f1d641523d0 msgr2=0x7f1d6414b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:23.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.805+0000 7f1d5be00640 1 --2- 172.21.15.17:0/1281433607 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f1d641523d0 0x7f1d6414b490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:23.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.805+0000 7f1d5be00640 1 -- 172.21.15.17:0/1281433607 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1d640c9520 msgr2=0x7f1d641479c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:23.812 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.805+0000 7f1d5be00640 1 --2- 172.21.15.17:0/1281433607 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1d640c9520 0x7f1d641479c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:23.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.805+0000 7f1d5be00640 1 -- 172.21.15.17:0/1281433607 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d6406e0d0 con 0x7f1d640ca4a0 2024-09-10T21:47:23.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.809+0000 7f1d5be00640 1 --2- 172.21.15.17:0/1281433607 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1d640ca4a0 0x7f1d6406e3d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1d4c00c490 tx=0x7f1d4c00c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:23.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.809+0000 7f1d60c00640 1 -- 172.21.15.17:0/1281433607 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d4c016020 con 0x7f1d640ca4a0 2024-09-10T21:47:23.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.809+0000 7f1d60c00640 1 -- 172.21.15.17:0/1281433607 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1d4c012920 con 0x7f1d640ca4a0 2024-09-10T21:47:23.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.809+0000 7f1d60c00640 1 -- 172.21.15.17:0/1281433607 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d4c012d50 con 0x7f1d640ca4a0 2024-09-10T21:47:23.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.809+0000 7f1d69c00640 1 -- 172.21.15.17:0/1281433607 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d6414cc10 con 0x7f1d640ca4a0 2024-09-10T21:47:23.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.809+0000 7f1d69c00640 1 -- 172.21.15.17:0/1281433607 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d641487c0 con 0x7f1d640ca4a0 2024-09-10T21:47:23.813 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.809+0000 7f1d69c00640 1 -- 172.21.15.17:0/1281433607 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d640c9f60 con 0x7f1d640ca4a0 2024-09-10T21:47:23.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.809+0000 7f1d60c00640 1 -- 172.21.15.17:0/1281433607 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f1d4c01b400 con 0x7f1d640ca4a0 2024-09-10T21:47:23.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.809+0000 7f1d60c00640 1 --2- 172.21.15.17:0/1281433607 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f1d40072610 0x7f1d40074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:23.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.809+0000 7f1d60c00640 1 -- 172.21.15.17:0/1281433607 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f1d4c093490 con 0x7f1d640ca4a0 2024-09-10T21:47:23.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.809+0000 7f1d63e00640 1 --2- 172.21.15.17:0/1281433607 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f1d40072610 0x7f1d40074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:23.815 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.813+0000 7f1d63e00640 1 --2- 172.21.15.17:0/1281433607 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f1d40072610 0x7f1d40074ad0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f1d641467b0 tx=0x7f1d540091e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:23.818 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.813+0000 7f1d60c00640 1 -- 172.21.15.17:0/1281433607 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f1d4c020030 con 0x7f1d640ca4a0 2024-09-10T21:47:23.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.989+0000 7f1d69c00640 1 -- 172.21.15.17:0/1281433607 --> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) -- 0x7f1d64162c70 con 0x7f1d40072610 2024-09-10T21:47:23.994 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.989+0000 7f1d60c00640 1 -- 172.21.15.17:0/1281433607 <== mgr.4100 v2:172.21.15.31:6808/3101308586 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f1d64162c70 con 0x7f1d40072610 2024-09-10T21:47:23.994 INFO:teuthology.orchestra.run.smithi017.stderr:instructed osd(s) 5 to deep-scrub 2024-09-10T21:47:23.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.993+0000 7f1d69c00640 1 -- 172.21.15.17:0/1281433607 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f1d40072610 msgr2=0x7f1d40074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:23.996 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.993+0000 7f1d69c00640 1 --2- 172.21.15.17:0/1281433607 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f1d40072610 0x7f1d40074ad0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f1d641467b0 tx=0x7f1d540091e0 comp rx=0 tx=0).stop 2024-09-10T21:47:23.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.993+0000 7f1d69c00640 1 -- 172.21.15.17:0/1281433607 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1d640ca4a0 msgr2=0x7f1d6406e3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:23.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.993+0000 7f1d69c00640 1 --2- 172.21.15.17:0/1281433607 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1d640ca4a0 0x7f1d6406e3d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1d4c00c490 tx=0x7f1d4c00c950 comp rx=0 tx=0).stop 2024-09-10T21:47:23.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.993+0000 7f1d69c00640 1 -- 172.21.15.17:0/1281433607 shutdown_connections 2024-09-10T21:47:23.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.993+0000 7f1d69c00640 1 --2- 172.21.15.17:0/1281433607 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f1d40072610 0x7f1d40074ad0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:23.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.993+0000 7f1d69c00640 1 --2- 172.21.15.17:0/1281433607 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f1d641523d0 0x7f1d6414b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:23.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.993+0000 7f1d69c00640 1 --2- 172.21.15.17:0/1281433607 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1d640ca4a0 0x7f1d6406e3d0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:23.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.993+0000 7f1d69c00640 1 --2- 172.21.15.17:0/1281433607 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1d640c9520 0x7f1d641479c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:23.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.993+0000 7f1d69c00640 1 -- 172.21.15.17:0/1281433607 >> 172.21.15.17:0/1281433607 conn(0x7f1d64068990 msgr2=0x7f1d64155d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:23.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.993+0000 7f1d69c00640 1 -- 172.21.15.17:0/1281433607 shutdown_connections 2024-09-10T21:47:23.997 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:23.993+0000 7f1d69c00640 1 -- 172.21.15.17:0/1281433607 wait complete. 2024-09-10T21:47:24.016 INFO:tasks.ceph:Scrubbing osd.6 2024-09-10T21:47:24.016 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-09-10T21:47:24.169 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.165+0000 7f78c7200640 1 -- 172.21.15.17:0/1685988388 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f78c8068990 msgr2=0x7f78c80740f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:24.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.165+0000 7f78c7200640 1 --2- 172.21.15.17:0/1685988388 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f78c8068990 0x7f78c80740f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f78c806fd20 tx=0x7f78b401c8c0 comp rx=0 tx=0).stop 2024-09-10T21:47:24.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.165+0000 7f78c7200640 1 -- 172.21.15.17:0/1685988388 shutdown_connections 2024-09-10T21:47:24.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.165+0000 7f78c7200640 1 --2- 172.21.15.17:0/1685988388 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f78c8068990 0x7f78c80740f0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:24.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.165+0000 7f78c7200640 1 --2- 172.21.15.17:0/1685988388 >> v2:172.21.15.31:3300/0 conn(0x7f78c8068da0 0x7f78c8074630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:24.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.165+0000 7f78c7200640 1 -- 172.21.15.17:0/1685988388 >> 172.21.15.17:0/1685988388 conn(0x7f78c8069850 msgr2=0x7f78c806cc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:24.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.165+0000 7f78c7200640 1 -- 172.21.15.17:0/1685988388 shutdown_connections 2024-09-10T21:47:24.170 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.165+0000 7f78c7200640 1 -- 172.21.15.17:0/1685988388 wait complete. 2024-09-10T21:47:24.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.165+0000 7f78c7200640 1 Processor -- start 2024-09-10T21:47:24.171 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.165+0000 7f78c7200640 1 -- start start 2024-09-10T21:47:24.172 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.169+0000 7f78c7200640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f78c8068990 0x7f78c8148240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:24.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.169+0000 7f78c7200640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f78c8068da0 0x7f78c814c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:24.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.169+0000 7f78c7200640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f78c80c7c60 0x7f78c8158680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:24.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.169+0000 7f78c7200640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f78c815b2a0 con 0x7f78c80c7c60 2024-09-10T21:47:24.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.169+0000 7f78c7200640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f78c815acd0 con 0x7f78c8068990 2024-09-10T21:47:24.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.169+0000 7f78c7200640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f78c815b420 con 0x7f78c8068da0 2024-09-10T21:47:24.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.169+0000 7f78c6800640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f78c80c7c60 0x7f78c8158680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:24.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.169+0000 7f78c6800640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f78c80c7c60 0x7f78c8158680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:34834/0 (socket says 172.21.15.17:34834) 2024-09-10T21:47:24.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.169+0000 7f78c6800640 1 -- 172.21.15.17:0/3860627694 learned_addr learned my addr 172.21.15.17:0/3860627694 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:47:24.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.169+0000 7f78c5e00640 1 --2- 172.21.15.17:0/3860627694 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f78c8068990 0x7f78c8148240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:24.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.169+0000 7f78c5400640 1 --2- 172.21.15.17:0/3860627694 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f78c8068da0 0x7f78c814c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:24.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.169+0000 7f78c6800640 1 -- 172.21.15.17:0/3860627694 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f78c8068da0 msgr2=0x7f78c814c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:24.173 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.169+0000 7f78c6800640 1 --2- 172.21.15.17:0/3860627694 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f78c8068da0 0x7f78c814c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:24.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.169+0000 7f78c6800640 1 -- 172.21.15.17:0/3860627694 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f78c8068990 msgr2=0x7f78c8148240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:24.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.169+0000 7f78c6800640 1 --2- 172.21.15.17:0/3860627694 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f78c8068990 0x7f78c8148240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:24.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.169+0000 7f78c6800640 1 -- 172.21.15.17:0/3860627694 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78c8159c00 con 0x7f78c80c7c60 2024-09-10T21:47:24.174 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.169+0000 7f78c5400640 1 --2- 172.21.15.17:0/3860627694 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f78c8068da0 0x7f78c814c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:47:24.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.169+0000 7f78c6800640 1 --2- 172.21.15.17:0/3860627694 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f78c80c7c60 0x7f78c8158680 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f78c0010f90 tx=0x7f78c000e490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:24.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.169+0000 7f78be000640 1 -- 172.21.15.17:0/3860627694 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78c001a030 con 0x7f78c80c7c60 2024-09-10T21:47:24.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.169+0000 7f78be000640 1 -- 172.21.15.17:0/3860627694 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f78c000ed30 con 0x7f78c80c7c60 2024-09-10T21:47:24.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.169+0000 7f78c7200640 1 -- 172.21.15.17:0/3860627694 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78c8157c00 con 0x7f78c80c7c60 2024-09-10T21:47:24.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.169+0000 7f78be000640 1 -- 172.21.15.17:0/3860627694 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78c0004d90 con 0x7f78c80c7c60 2024-09-10T21:47:24.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.169+0000 7f78c7200640 1 -- 172.21.15.17:0/3860627694 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78c81f41b0 con 0x7f78c80c7c60 2024-09-10T21:47:24.175 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.169+0000 7f78c7200640 1 -- 172.21.15.17:0/3860627694 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f78c80c9140 con 0x7f78c80c7c60 2024-09-10T21:47:24.176 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.173+0000 7f78be000640 1 -- 172.21.15.17:0/3860627694 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f78c0027060 con 0x7f78c80c7c60 2024-09-10T21:47:24.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.173+0000 7f78be000640 1 --2- 172.21.15.17:0/3860627694 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f78a8072610 0x7f78a8074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:24.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.173+0000 7f78be000640 1 -- 172.21.15.17:0/3860627694 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f78c00969a0 con 0x7f78c80c7c60 2024-09-10T21:47:24.177 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.173+0000 7f78be000640 1 --2- 172.21.15.17:0/3860627694 >> [v2:172.21.15.31:6826/3754251089,v1:172.21.15.31:6827/3754251089] conn(0x7f78a807c020 0x7f78a807e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:24.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.173+0000 7f78be000640 1 -- 172.21.15.17:0/3860627694 --> [v2:172.21.15.31:6826/3754251089,v1:172.21.15.31:6827/3754251089] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f78a807eb90 con 0x7f78a807c020 2024-09-10T21:47:24.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.173+0000 7f78be000640 1 -- 172.21.15.17:0/3860627694 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f78c0096d50 con 0x7f78c80c7c60 2024-09-10T21:47:24.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.173+0000 7f78c5e00640 1 --2- 172.21.15.17:0/3860627694 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f78a8072610 0x7f78a8074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:24.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.173+0000 7f78c5400640 1 --2- 172.21.15.17:0/3860627694 >> [v2:172.21.15.31:6826/3754251089,v1:172.21.15.31:6827/3754251089] conn(0x7f78a807c020 0x7f78a807e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:24.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.173+0000 7f78c5e00640 1 --2- 172.21.15.17:0/3860627694 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f78a8072610 0x7f78a8074ad0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f78b8000f70 tx=0x7f78b8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:24.178 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.173+0000 7f78c5400640 1 --2- 172.21.15.17:0/3860627694 >> [v2:172.21.15.31:6826/3754251089,v1:172.21.15.31:6827/3754251089] conn(0x7f78a807c020 0x7f78a807e4c0 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:24.179 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.177+0000 7f78be000640 1 -- 172.21.15.17:0/3860627694 <== osd.6 v2:172.21.15.31:6826/3754251089 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f78a807eb90 con 0x7f78a807c020 2024-09-10T21:47:24.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.189+0000 7f78c7200640 1 -- 172.21.15.17:0/3860627694 --> [v2:172.21.15.31:6826/3754251089,v1:172.21.15.31:6827/3754251089] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f78c81582a0 con 0x7f78a807c020 2024-09-10T21:47:24.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.189+0000 7f78be000640 1 -- 172.21.15.17:0/3860627694 <== osd.6 v2:172.21.15.31:6826/3754251089 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f78c81582a0 con 0x7f78a807c020 2024-09-10T21:47:24.193 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-10T21:47:24.194 INFO:teuthology.orchestra.run.smithi017.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-10T21:47:24.194 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-10T21:47:24.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.189+0000 7f78c7200640 1 -- 172.21.15.17:0/3860627694 >> [v2:172.21.15.31:6826/3754251089,v1:172.21.15.31:6827/3754251089] conn(0x7f78a807c020 msgr2=0x7f78a807e4c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:24.194 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.189+0000 7f78c7200640 1 --2- 172.21.15.17:0/3860627694 >> [v2:172.21.15.31:6826/3754251089,v1:172.21.15.31:6827/3754251089] conn(0x7f78a807c020 0x7f78a807e4c0 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:24.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.189+0000 7f78c7200640 1 -- 172.21.15.17:0/3860627694 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f78a8072610 msgr2=0x7f78a8074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:24.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.189+0000 7f78c7200640 1 --2- 172.21.15.17:0/3860627694 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f78a8072610 0x7f78a8074ad0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f78b8000f70 tx=0x7f78b8009040 comp rx=0 tx=0).stop 2024-09-10T21:47:24.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.189+0000 7f78c7200640 1 -- 172.21.15.17:0/3860627694 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f78c80c7c60 msgr2=0x7f78c8158680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:24.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.189+0000 7f78c7200640 1 --2- 172.21.15.17:0/3860627694 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f78c80c7c60 0x7f78c8158680 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f78c0010f90 tx=0x7f78c000e490 comp rx=0 tx=0).stop 2024-09-10T21:47:24.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.189+0000 7f78c6800640 1 -- 172.21.15.17:0/3860627694 reap_dead start 2024-09-10T21:47:24.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.189+0000 7f78c7200640 1 -- 172.21.15.17:0/3860627694 shutdown_connections 2024-09-10T21:47:24.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.189+0000 7f78c7200640 1 -- 172.21.15.17:0/3860627694 >> 172.21.15.17:0/3860627694 conn(0x7f78c8069850 msgr2=0x7f78c80c45d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:24.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.189+0000 7f78c7200640 1 -- 172.21.15.17:0/3860627694 shutdown_connections 2024-09-10T21:47:24.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.189+0000 7f78c7200640 1 -- 172.21.15.17:0/3860627694 wait complete. 2024-09-10T21:47:24.212 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-09-10T21:47:24.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.357+0000 7f713ee00640 1 -- 172.21.15.17:0/282688120 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7138152820 msgr2=0x7f713815c280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:24.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.357+0000 7f713ee00640 1 --2- 172.21.15.17:0/282688120 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7138152820 0x7f713815c280 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f71381594a0 tx=0x7f712001c850 comp rx=0 tx=0).stop 2024-09-10T21:47:24.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.357+0000 7f713ee00640 1 -- 172.21.15.17:0/282688120 shutdown_connections 2024-09-10T21:47:24.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.357+0000 7f713ee00640 1 --2- 172.21.15.17:0/282688120 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7138152820 0x7f713815c280 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:24.363 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.357+0000 7f713ee00640 1 --2- 172.21.15.17:0/282688120 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f7138152440 0x7f71380b3eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:24.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.357+0000 7f713ee00640 1 -- 172.21.15.17:0/282688120 >> 172.21.15.17:0/282688120 conn(0x7f71380702f0 msgr2=0x7f7138070700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:24.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.357+0000 7f713ee00640 1 -- 172.21.15.17:0/282688120 shutdown_connections 2024-09-10T21:47:24.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.357+0000 7f713ee00640 1 -- 172.21.15.17:0/282688120 wait complete. 2024-09-10T21:47:24.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.361+0000 7f713ee00640 1 Processor -- start 2024-09-10T21:47:24.364 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.361+0000 7f713ee00640 1 -- start start 2024-09-10T21:47:24.365 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.361+0000 7f713ee00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f71380cdf20 0x7f71381632f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:24.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.361+0000 7f713ee00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7138152440 0x7f71380ca1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:24.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.361+0000 7f713ee00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f7138152820 0x7f71380ca730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:24.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.361+0000 7f713ee00640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f713815c7c0 con 0x7f7138152440 2024-09-10T21:47:24.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.361+0000 7f713ee00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f71380ce740 con 0x7f71380cdf20 2024-09-10T21:47:24.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.361+0000 7f713ee00640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f71380cb450 con 0x7f7138152820 2024-09-10T21:47:24.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.361+0000 7f712fe00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7138152440 0x7f71380ca1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:24.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.361+0000 7f712fe00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7138152440 0x7f71380ca1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:34842/0 (socket says 172.21.15.17:34842) 2024-09-10T21:47:24.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.361+0000 7f712fe00640 1 -- 172.21.15.17:0/462313732 learned_addr learned my addr 172.21.15.17:0/462313732 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:47:24.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.361+0000 7f7137e00640 1 --2- 172.21.15.17:0/462313732 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f71380cdf20 0x7f71381632f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:24.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.361+0000 7f713ce00640 1 --2- 172.21.15.17:0/462313732 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f7138152820 0x7f71380ca730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:24.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.361+0000 7f712fe00640 1 -- 172.21.15.17:0/462313732 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f7138152820 msgr2=0x7f71380ca730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:24.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.361+0000 7f712fe00640 1 --2- 172.21.15.17:0/462313732 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f7138152820 0x7f71380ca730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:24.366 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.361+0000 7f712fe00640 1 -- 172.21.15.17:0/462313732 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f71380cdf20 msgr2=0x7f71381632f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:24.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.361+0000 7f712fe00640 1 --2- 172.21.15.17:0/462313732 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f71380cdf20 0x7f71381632f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:24.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.361+0000 7f712fe00640 1 -- 172.21.15.17:0/462313732 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7138157ab0 con 0x7f7138152440 2024-09-10T21:47:24.367 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.361+0000 7f712fe00640 1 --2- 172.21.15.17:0/462313732 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7138152440 0x7f71380ca1f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f71200b0040 tx=0x7f71200a5600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:24.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.361+0000 7f7134c00640 1 -- 172.21.15.17:0/462313732 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71200b2040 con 0x7f7138152440 2024-09-10T21:47:24.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.361+0000 7f7134c00640 1 -- 172.21.15.17:0/462313732 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f71200040c0 con 0x7f7138152440 2024-09-10T21:47:24.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.361+0000 7f713ee00640 1 -- 172.21.15.17:0/462313732 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7138157250 con 0x7f7138152440 2024-09-10T21:47:24.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.361+0000 7f7134c00640 1 -- 172.21.15.17:0/462313732 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7120003600 con 0x7f7138152440 2024-09-10T21:47:24.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.361+0000 7f713ee00640 1 -- 172.21.15.17:0/462313732 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7138155c90 con 0x7f7138152440 2024-09-10T21:47:24.368 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.365+0000 7f713ee00640 1 -- 172.21.15.17:0/462313732 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71380c2ff0 con 0x7f7138152440 2024-09-10T21:47:24.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.365+0000 7f7134c00640 1 -- 172.21.15.17:0/462313732 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f71200037a0 con 0x7f7138152440 2024-09-10T21:47:24.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.365+0000 7f7134c00640 1 --2- 172.21.15.17:0/462313732 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f71140725e0 0x7f7114074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:24.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.365+0000 7f7134c00640 1 -- 172.21.15.17:0/462313732 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f712012d3c0 con 0x7f7138152440 2024-09-10T21:47:24.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.365+0000 7f7137e00640 1 --2- 172.21.15.17:0/462313732 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f71140725e0 0x7f7114074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:24.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.365+0000 7f7137e00640 1 --2- 172.21.15.17:0/462313732 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f71140725e0 0x7f7114074aa0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f713815b320 tx=0x7f71280091e0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:24.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.369+0000 7f7134c00640 1 -- 172.21.15.17:0/462313732 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f71200c4020 con 0x7f7138152440 2024-09-10T21:47:24.549 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.545+0000 7f713ee00640 1 -- 172.21.15.17:0/462313732 --> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) -- 0x7f713814ea90 con 0x7f71140725e0 2024-09-10T21:47:24.550 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.545+0000 7f7134c00640 1 -- 172.21.15.17:0/462313732 <== mgr.4100 v2:172.21.15.31:6808/3101308586 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f713814ea90 con 0x7f71140725e0 2024-09-10T21:47:24.550 INFO:teuthology.orchestra.run.smithi017.stderr:instructed osd(s) 6 to deep-scrub 2024-09-10T21:47:24.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.549+0000 7f713ee00640 1 -- 172.21.15.17:0/462313732 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f71140725e0 msgr2=0x7f7114074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:24.552 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.549+0000 7f713ee00640 1 --2- 172.21.15.17:0/462313732 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f71140725e0 0x7f7114074aa0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f713815b320 tx=0x7f71280091e0 comp rx=0 tx=0).stop 2024-09-10T21:47:24.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.549+0000 7f713ee00640 1 -- 172.21.15.17:0/462313732 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7138152440 msgr2=0x7f71380ca1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:24.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.549+0000 7f713ee00640 1 --2- 172.21.15.17:0/462313732 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7138152440 0x7f71380ca1f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f71200b0040 tx=0x7f71200a5600 comp rx=0 tx=0).stop 2024-09-10T21:47:24.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.549+0000 7f713ee00640 1 -- 172.21.15.17:0/462313732 shutdown_connections 2024-09-10T21:47:24.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.549+0000 7f713ee00640 1 --2- 172.21.15.17:0/462313732 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f71140725e0 0x7f7114074aa0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:24.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.549+0000 7f713ee00640 1 --2- 172.21.15.17:0/462313732 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f7138152820 0x7f71380ca730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:24.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.549+0000 7f713ee00640 1 --2- 172.21.15.17:0/462313732 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f7138152440 0x7f71380ca1f0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:24.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.549+0000 7f713ee00640 1 --2- 172.21.15.17:0/462313732 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f71380cdf20 0x7f71381632f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:24.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.549+0000 7f713ee00640 1 -- 172.21.15.17:0/462313732 >> 172.21.15.17:0/462313732 conn(0x7f71380702f0 msgr2=0x7f71380cbe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:24.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.549+0000 7f713ee00640 1 -- 172.21.15.17:0/462313732 shutdown_connections 2024-09-10T21:47:24.553 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.549+0000 7f713ee00640 1 -- 172.21.15.17:0/462313732 wait complete. 2024-09-10T21:47:24.573 INFO:tasks.ceph:Scrubbing osd.7 2024-09-10T21:47:24.573 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-09-10T21:47:24.724 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.721+0000 7f23e2a00640 1 -- 172.21.15.17:0/3759222364 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f23dc0cdf50 msgr2=0x7f23dc0ce3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:24.724 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.721+0000 7f23e2a00640 1 --2- 172.21.15.17:0/3759222364 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f23dc0cdf50 0x7f23dc0ce3c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f23dc155e60 tx=0x7f23d401f540 comp rx=0 tx=0).stop 2024-09-10T21:47:24.724 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.721+0000 7f23e2a00640 1 -- 172.21.15.17:0/3759222364 shutdown_connections 2024-09-10T21:47:24.724 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.721+0000 7f23e2a00640 1 --2- 172.21.15.17:0/3759222364 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f23dc0cdf50 0x7f23dc0ce3c0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:24.724 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.721+0000 7f23e2a00640 1 --2- 172.21.15.17:0/3759222364 >> v2:172.21.15.31:3300/0 conn(0x7f23dc071680 0x7f23dc070dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:24.724 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.721+0000 7f23e2a00640 1 -- 172.21.15.17:0/3759222364 >> 172.21.15.17:0/3759222364 conn(0x7f23dc0748d0 msgr2=0x7f23dc068990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:24.724 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.721+0000 7f23e2a00640 1 -- 172.21.15.17:0/3759222364 shutdown_connections 2024-09-10T21:47:24.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.721+0000 7f23e2a00640 1 -- 172.21.15.17:0/3759222364 wait complete. 2024-09-10T21:47:24.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.721+0000 7f23e2a00640 1 Processor -- start 2024-09-10T21:47:24.725 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.721+0000 7f23e2a00640 1 -- start start 2024-09-10T21:47:24.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.721+0000 7f23e2a00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f23dc071680 0x7f23dc157170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:24.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.721+0000 7f23e2a00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f23dc0cdf50 0x7f23dc06e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:24.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.721+0000 7f23e2a00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f23dc152520 0x7f23dc150da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:24.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.721+0000 7f23e2a00640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f23dc152900 con 0x7f23dc071680 2024-09-10T21:47:24.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.721+0000 7f23e2a00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f23dc0c3270 con 0x7f23dc152520 2024-09-10T21:47:24.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.721+0000 7f23e2a00640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f23dc152a80 con 0x7f23dc0cdf50 2024-09-10T21:47:24.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.721+0000 7f23dbe00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f23dc071680 0x7f23dc157170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:24.727 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.721+0000 7f23dbe00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f23dc071680 0x7f23dc157170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:60606/0 (socket says 172.21.15.17:60606) 2024-09-10T21:47:24.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.721+0000 7f23dbe00640 1 -- 172.21.15.17:0/994949204 learned_addr learned my addr 172.21.15.17:0/994949204 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:47:24.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.721+0000 7f23d3e00640 1 --2- 172.21.15.17:0/994949204 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f23dc0cdf50 0x7f23dc06e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:24.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.721+0000 7f23e0a00640 1 --2- 172.21.15.17:0/994949204 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f23dc152520 0x7f23dc150da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:24.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.721+0000 7f23dbe00640 1 -- 172.21.15.17:0/994949204 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f23dc0cdf50 msgr2=0x7f23dc06e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:24.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.721+0000 7f23dbe00640 1 --2- 172.21.15.17:0/994949204 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f23dc0cdf50 0x7f23dc06e1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:24.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.721+0000 7f23dbe00640 1 -- 172.21.15.17:0/994949204 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f23dc152520 msgr2=0x7f23dc150da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:24.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.721+0000 7f23dbe00640 1 --2- 172.21.15.17:0/994949204 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f23dc152520 0x7f23dc150da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:24.728 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.721+0000 7f23dbe00640 1 -- 172.21.15.17:0/994949204 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23dc1f49c0 con 0x7f23dc071680 2024-09-10T21:47:24.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.725+0000 7f23dbe00640 1 --2- 172.21.15.17:0/994949204 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f23dc071680 0x7f23dc157170 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f23cc00b410 tx=0x7f23cc00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:24.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.725+0000 7f23d8c00640 1 -- 172.21.15.17:0/994949204 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23cc015030 con 0x7f23dc071680 2024-09-10T21:47:24.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.725+0000 7f23d8c00640 1 -- 172.21.15.17:0/994949204 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f23cc004020 con 0x7f23dc071680 2024-09-10T21:47:24.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.725+0000 7f23d8c00640 1 -- 172.21.15.17:0/994949204 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23cc0044b0 con 0x7f23dc071680 2024-09-10T21:47:24.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.725+0000 7f23e2a00640 1 -- 172.21.15.17:0/994949204 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23dc14e500 con 0x7f23dc071680 2024-09-10T21:47:24.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.725+0000 7f23e2a00640 1 -- 172.21.15.17:0/994949204 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23dc1f3ee0 con 0x7f23dc071680 2024-09-10T21:47:24.729 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.725+0000 7f23e2a00640 1 -- 172.21.15.17:0/994949204 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f23dc14e7c0 con 0x7f23dc071680 2024-09-10T21:47:24.730 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.725+0000 7f23d8c00640 1 -- 172.21.15.17:0/994949204 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f23cc022060 con 0x7f23dc071680 2024-09-10T21:47:24.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.725+0000 7f23d8c00640 1 --2- 172.21.15.17:0/994949204 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f23b80725e0 0x7f23b8074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:24.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.725+0000 7f23d8c00640 1 -- 172.21.15.17:0/994949204 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f23cc093920 con 0x7f23dc071680 2024-09-10T21:47:24.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.725+0000 7f23d8c00640 1 --2- 172.21.15.17:0/994949204 >> [v2:172.21.15.31:6818/3948648934,v1:172.21.15.31:6819/3948648934] conn(0x7f23b807c000 0x7f23b807e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:24.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.725+0000 7f23d8c00640 1 -- 172.21.15.17:0/994949204 --> [v2:172.21.15.31:6818/3948648934,v1:172.21.15.31:6819/3948648934] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f23cc013ec0 con 0x7f23b807c000 2024-09-10T21:47:24.731 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.725+0000 7f23d8c00640 1 -- 172.21.15.17:0/994949204 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f23cc093cd0 con 0x7f23dc071680 2024-09-10T21:47:24.732 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.729+0000 7f23d3e00640 1 --2- 172.21.15.17:0/994949204 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f23b80725e0 0x7f23b8074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:24.732 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.729+0000 7f23e0a00640 1 --2- 172.21.15.17:0/994949204 >> [v2:172.21.15.31:6818/3948648934,v1:172.21.15.31:6819/3948648934] conn(0x7f23b807c000 0x7f23b807e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:24.732 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.729+0000 7f23d3e00640 1 --2- 172.21.15.17:0/994949204 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f23b80725e0 0x7f23b8074aa0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f23dc0b2c80 tx=0x7f23c400b390 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:24.732 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.729+0000 7f23e0a00640 1 --2- 172.21.15.17:0/994949204 >> [v2:172.21.15.31:6818/3948648934,v1:172.21.15.31:6819/3948648934] conn(0x7f23b807c000 0x7f23b807e4a0 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:24.733 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.729+0000 7f23d8c00640 1 -- 172.21.15.17:0/994949204 <== osd.7 v2:172.21.15.31:6818/3948648934 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f23cc013ec0 con 0x7f23b807c000 2024-09-10T21:47:24.746 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.741+0000 7f23e2a00640 1 -- 172.21.15.17:0/994949204 --> [v2:172.21.15.31:6818/3948648934,v1:172.21.15.31:6819/3948648934] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f23dc14c040 con 0x7f23b807c000 2024-09-10T21:47:24.747 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.741+0000 7f23d8c00640 1 -- 172.21.15.17:0/994949204 <== osd.7 v2:172.21.15.31:6818/3948648934 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f23dc14c040 con 0x7f23b807c000 2024-09-10T21:47:24.747 INFO:teuthology.orchestra.run.smithi017.stdout:{ 2024-09-10T21:47:24.747 INFO:teuthology.orchestra.run.smithi017.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-10T21:47:24.747 INFO:teuthology.orchestra.run.smithi017.stdout:} 2024-09-10T21:47:24.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.741+0000 7f23e2a00640 1 -- 172.21.15.17:0/994949204 >> [v2:172.21.15.31:6818/3948648934,v1:172.21.15.31:6819/3948648934] conn(0x7f23b807c000 msgr2=0x7f23b807e4a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:24.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.741+0000 7f23e2a00640 1 --2- 172.21.15.17:0/994949204 >> [v2:172.21.15.31:6818/3948648934,v1:172.21.15.31:6819/3948648934] conn(0x7f23b807c000 0x7f23b807e4a0 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:24.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.745+0000 7f23e2a00640 1 -- 172.21.15.17:0/994949204 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f23b80725e0 msgr2=0x7f23b8074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:24.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.745+0000 7f23e2a00640 1 --2- 172.21.15.17:0/994949204 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f23b80725e0 0x7f23b8074aa0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f23dc0b2c80 tx=0x7f23c400b390 comp rx=0 tx=0).stop 2024-09-10T21:47:24.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.745+0000 7f23e2a00640 1 -- 172.21.15.17:0/994949204 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f23dc071680 msgr2=0x7f23dc157170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:24.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.745+0000 7f23e2a00640 1 --2- 172.21.15.17:0/994949204 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f23dc071680 0x7f23dc157170 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f23cc00b410 tx=0x7f23cc00b8d0 comp rx=0 tx=0).stop 2024-09-10T21:47:24.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.745+0000 7f23e0a00640 1 -- 172.21.15.17:0/994949204 reap_dead start 2024-09-10T21:47:24.748 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.745+0000 7f23e2a00640 1 -- 172.21.15.17:0/994949204 shutdown_connections 2024-09-10T21:47:24.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.745+0000 7f23e2a00640 1 -- 172.21.15.17:0/994949204 >> 172.21.15.17:0/994949204 conn(0x7f23dc0748d0 msgr2=0x7f23dc14e9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:24.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.745+0000 7f23e2a00640 1 -- 172.21.15.17:0/994949204 shutdown_connections 2024-09-10T21:47:24.749 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.745+0000 7f23e2a00640 1 -- 172.21.15.17:0/994949204 wait complete. 2024-09-10T21:47:24.768 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-09-10T21:47:24.918 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.913+0000 7fc0dc600640 1 -- 172.21.15.17:0/4049432328 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc0d40cdf20 msgr2=0x7fc0d40ce350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:24.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.913+0000 7fc0dc600640 1 --2- 172.21.15.17:0/4049432328 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc0d40cdf20 0x7fc0d40ce350 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fc0d40cbaa0 tx=0x7fc0cc01ccf0 comp rx=0 tx=0).stop 2024-09-10T21:47:24.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.913+0000 7fc0dc600640 1 -- 172.21.15.17:0/4049432328 shutdown_connections 2024-09-10T21:47:24.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.913+0000 7fc0dc600640 1 --2- 172.21.15.17:0/4049432328 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc0d40cdf20 0x7fc0d40ce350 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:24.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.913+0000 7fc0dc600640 1 --2- 172.21.15.17:0/4049432328 >> v2:172.21.15.31:3300/0 conn(0x7fc0d4152820 0x7fc0d40b3eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:24.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.913+0000 7fc0dc600640 1 -- 172.21.15.17:0/4049432328 >> 172.21.15.17:0/4049432328 conn(0x7fc0d40702f0 msgr2=0x7fc0d4070700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:24.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.913+0000 7fc0dc600640 1 -- 172.21.15.17:0/4049432328 shutdown_connections 2024-09-10T21:47:24.919 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.913+0000 7fc0dc600640 1 -- 172.21.15.17:0/4049432328 wait complete. 2024-09-10T21:47:24.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.917+0000 7fc0dc600640 1 Processor -- start 2024-09-10T21:47:24.920 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.917+0000 7fc0dc600640 1 -- start start 2024-09-10T21:47:24.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.917+0000 7fc0dc600640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc0d40cdf20 0x7fc0d40c6f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:24.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.917+0000 7fc0dc600640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc0d4152440 0x7fc0d41f48a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:24.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.917+0000 7fc0dc600640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc0d4152820 0x7fc0d41f4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:24.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.917+0000 7fc0dc600640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fc0d40cb5d0 con 0x7fc0d4152440 2024-09-10T21:47:24.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.917+0000 7fc0dc600640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fc0d40cb450 con 0x7fc0d40cdf20 2024-09-10T21:47:24.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.917+0000 7fc0dc600640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7fc0d40c2ff0 con 0x7fc0d4152820 2024-09-10T21:47:24.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.917+0000 7fc0d9200640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc0d4152440 0x7fc0d41f48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:24.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.917+0000 7fc0d9200640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc0d4152440 0x7fc0d41f48a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:60632/0 (socket says 172.21.15.17:60632) 2024-09-10T21:47:24.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.917+0000 7fc0d9200640 1 -- 172.21.15.17:0/4079229248 learned_addr learned my addr 172.21.15.17:0/4079229248 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:47:24.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.917+0000 7fc0d9200640 1 -- 172.21.15.17:0/4079229248 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc0d4152820 msgr2=0x7fc0d41f4de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:24.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.917+0000 7fc0da600640 1 --2- 172.21.15.17:0/4079229248 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc0d4152820 0x7fc0d41f4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:24.922 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.917+0000 7fc0d9c00640 1 --2- 172.21.15.17:0/4079229248 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc0d40cdf20 0x7fc0d40c6f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:24.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.917+0000 7fc0d9200640 1 --2- 172.21.15.17:0/4079229248 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc0d4152820 0x7fc0d41f4de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:24.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.917+0000 7fc0d9200640 1 -- 172.21.15.17:0/4079229248 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc0d40cdf20 msgr2=0x7fc0d40c6f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:24.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.917+0000 7fc0d9200640 1 --2- 172.21.15.17:0/4079229248 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc0d40cdf20 0x7fc0d40c6f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:24.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.917+0000 7fc0d9200640 1 -- 172.21.15.17:0/4079229248 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0d4155540 con 0x7fc0d4152440 2024-09-10T21:47:24.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.917+0000 7fc0d9c00640 1 --2- 172.21.15.17:0/4079229248 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc0d40cdf20 0x7fc0d40c6f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:47:24.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.917+0000 7fc0da600640 1 --2- 172.21.15.17:0/4079229248 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc0d4152820 0x7fc0d41f4de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:47:24.923 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.917+0000 7fc0d9200640 1 --2- 172.21.15.17:0/4079229248 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc0d4152440 0x7fc0d41f48a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc0c000ea20 tx=0x7fc0c000eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:24.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.917+0000 7fc0d1600640 1 -- 172.21.15.17:0/4079229248 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0c000cd80 con 0x7fc0d4152440 2024-09-10T21:47:24.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.917+0000 7fc0d1600640 1 -- 172.21.15.17:0/4079229248 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc0c0010430 con 0x7fc0d4152440 2024-09-10T21:47:24.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.917+0000 7fc0dc600640 1 -- 172.21.15.17:0/4079229248 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0d41f1b00 con 0x7fc0d4152440 2024-09-10T21:47:24.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.917+0000 7fc0d1600640 1 -- 172.21.15.17:0/4079229248 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0c0010860 con 0x7fc0d4152440 2024-09-10T21:47:24.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.917+0000 7fc0dc600640 1 -- 172.21.15.17:0/4079229248 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0d4155ac0 con 0x7fc0d4152440 2024-09-10T21:47:24.924 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.917+0000 7fc0dc600640 1 -- 172.21.15.17:0/4079229248 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0d4158360 con 0x7fc0d4152440 2024-09-10T21:47:24.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.921+0000 7fc0d1600640 1 -- 172.21.15.17:0/4079229248 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fc0c0010a00 con 0x7fc0d4152440 2024-09-10T21:47:24.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.921+0000 7fc0d1600640 1 --2- 172.21.15.17:0/4079229248 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fc0b40725e0 0x7fc0b4074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:24.925 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.921+0000 7fc0d1600640 1 -- 172.21.15.17:0/4079229248 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fc0c0094320 con 0x7fc0d4152440 2024-09-10T21:47:24.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.921+0000 7fc0d9c00640 1 --2- 172.21.15.17:0/4079229248 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fc0b40725e0 0x7fc0b4074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:24.926 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.921+0000 7fc0d9c00640 1 --2- 172.21.15.17:0/4079229248 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fc0b40725e0 0x7fc0b4074aa0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fc0c80016d0 tx=0x7fc0c8009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:24.929 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:24.925+0000 7fc0d1600640 1 -- 172.21.15.17:0/4079229248 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fc0c005f560 con 0x7fc0d4152440 2024-09-10T21:47:25.107 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:25.101+0000 7fc0dc600640 1 -- 172.21.15.17:0/4079229248 --> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) -- 0x7fc0d4157760 con 0x7fc0b40725e0 2024-09-10T21:47:25.108 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:25.105+0000 7fc0d1600640 1 -- 172.21.15.17:0/4079229248 <== mgr.4100 v2:172.21.15.31:6808/3101308586 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fc0d4157760 con 0x7fc0b40725e0 2024-09-10T21:47:25.108 INFO:teuthology.orchestra.run.smithi017.stderr:instructed osd(s) 7 to deep-scrub 2024-09-10T21:47:25.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:25.105+0000 7fc0dc600640 1 -- 172.21.15.17:0/4079229248 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fc0b40725e0 msgr2=0x7fc0b4074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:25.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:25.105+0000 7fc0dc600640 1 --2- 172.21.15.17:0/4079229248 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fc0b40725e0 0x7fc0b4074aa0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fc0c80016d0 tx=0x7fc0c8009040 comp rx=0 tx=0).stop 2024-09-10T21:47:25.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:25.105+0000 7fc0dc600640 1 -- 172.21.15.17:0/4079229248 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc0d4152440 msgr2=0x7fc0d41f48a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:25.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:25.105+0000 7fc0dc600640 1 --2- 172.21.15.17:0/4079229248 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc0d4152440 0x7fc0d41f48a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fc0c000ea20 tx=0x7fc0c000eee0 comp rx=0 tx=0).stop 2024-09-10T21:47:25.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:25.105+0000 7fc0dc600640 1 -- 172.21.15.17:0/4079229248 shutdown_connections 2024-09-10T21:47:25.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:25.105+0000 7fc0dc600640 1 --2- 172.21.15.17:0/4079229248 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fc0b40725e0 0x7fc0b4074aa0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:25.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:25.105+0000 7fc0dc600640 1 --2- 172.21.15.17:0/4079229248 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fc0d4152820 0x7fc0d41f4de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:25.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:25.105+0000 7fc0dc600640 1 --2- 172.21.15.17:0/4079229248 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fc0d4152440 0x7fc0d41f48a0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:25.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:25.105+0000 7fc0dc600640 1 --2- 172.21.15.17:0/4079229248 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fc0d40cdf20 0x7fc0d40c6f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:25.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:25.105+0000 7fc0dc600640 1 -- 172.21.15.17:0/4079229248 >> 172.21.15.17:0/4079229248 conn(0x7fc0d40702f0 msgr2=0x7fc0d414f060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:25.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:25.105+0000 7fc0dc600640 1 -- 172.21.15.17:0/4079229248 shutdown_connections 2024-09-10T21:47:25.111 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:25.105+0000 7fc0dc600640 1 -- 172.21.15.17:0/4079229248 wait complete. 2024-09-10T21:47:25.131 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T21:47:25.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:25.277+0000 7fd16b800640 1 -- 172.21.15.17:0/1646983107 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd164152820 msgr2=0x7fd1640b3eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:25.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:25.277+0000 7fd16b800640 1 --2- 172.21.15.17:0/1646983107 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd164152820 0x7fd1640b3eb0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd1641594a0 tx=0x7fd14c01c790 comp rx=0 tx=0).stop 2024-09-10T21:47:25.280 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:25.277+0000 7fd16b800640 1 -- 172.21.15.17:0/1646983107 shutdown_connections 2024-09-10T21:47:25.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:25.277+0000 7fd16b800640 1 --2- 172.21.15.17:0/1646983107 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd164152820 0x7fd1640b3eb0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:25.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:25.277+0000 7fd16b800640 1 -- 172.21.15.17:0/1646983107 >> 172.21.15.17:0/1646983107 conn(0x7fd1640702f0 msgr2=0x7fd164070700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:25.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:25.277+0000 7fd16b800640 1 -- 172.21.15.17:0/1646983107 shutdown_connections 2024-09-10T21:47:25.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:25.277+0000 7fd16b800640 1 -- 172.21.15.17:0/1646983107 wait complete. 2024-09-10T21:47:25.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:25.277+0000 7fd16b800640 1 Processor -- start 2024-09-10T21:47:25.281 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:25.277+0000 7fd16b800640 1 -- start start 2024-09-10T21:47:25.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:25.277+0000 7fd16b800640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd1640cdf20 0x7fd1641f3d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:25.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:25.277+0000 7fd16b800640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd164152440 0x7fd16415b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:25.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:25.277+0000 7fd16b800640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd164152820 0x7fd16406f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:25.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:25.277+0000 7fd16b800640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fd1640cb450 con 0x7fd164152820 2024-09-10T21:47:25.283 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:25.277+0000 7fd16b800640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fd1640ce740 con 0x7fd1640cdf20 2024-09-10T21:47:25.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:25.277+0000 7fd16b800640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7fd1640cb5d0 con 0x7fd164152440 2024-09-10T21:47:25.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:25.277+0000 7fd169800640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd164152820 0x7fd16406f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:25.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:25.277+0000 7fd169800640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd164152820 0x7fd16406f580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:60650/0 (socket says 172.21.15.17:60650) 2024-09-10T21:47:25.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:25.277+0000 7fd169800640 1 -- 172.21.15.17:0/4182645144 learned_addr learned my addr 172.21.15.17:0/4182645144 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:47:25.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:25.277+0000 7fd168e00640 1 --2- 172.21.15.17:0/4182645144 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd1640cdf20 0x7fd1641f3d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:25.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:25.277+0000 7fd15be00640 1 --2- 172.21.15.17:0/4182645144 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd164152440 0x7fd16415b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:25.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:25.277+0000 7fd169800640 1 -- 172.21.15.17:0/4182645144 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd164152440 msgr2=0x7fd16415b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:25.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:25.277+0000 7fd169800640 1 --2- 172.21.15.17:0/4182645144 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd164152440 0x7fd16415b130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:25.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:25.277+0000 7fd169800640 1 -- 172.21.15.17:0/4182645144 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd1640cdf20 msgr2=0x7fd1641f3d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:25.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:25.277+0000 7fd169800640 1 --2- 172.21.15.17:0/4182645144 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd1640cdf20 0x7fd1641f3d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:25.284 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:25.277+0000 7fd169800640 1 -- 172.21.15.17:0/4182645144 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1641f42c0 con 0x7fd164152820 2024-09-10T21:47:25.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:25.281+0000 7fd169800640 1 --2- 172.21.15.17:0/4182645144 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd164152820 0x7fd16406f580 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fd15c00c760 tx=0x7fd15c00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:25.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:25.281+0000 7fd161600640 1 -- 172.21.15.17:0/4182645144 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd15c003db0 con 0x7fd164152820 2024-09-10T21:47:25.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:25.281+0000 7fd161600640 1 -- 172.21.15.17:0/4182645144 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd15c00ce10 con 0x7fd164152820 2024-09-10T21:47:25.285 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:25.281+0000 7fd161600640 1 -- 172.21.15.17:0/4182645144 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd15c0052d0 con 0x7fd164152820 2024-09-10T21:47:25.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:25.281+0000 7fd16b800640 1 -- 172.21.15.17:0/4182645144 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd164159d40 con 0x7fd164152820 2024-09-10T21:47:25.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:25.281+0000 7fd16b800640 1 -- 172.21.15.17:0/4182645144 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd16415a650 con 0x7fd164152820 2024-09-10T21:47:25.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:25.281+0000 7fd16b800640 1 -- 172.21.15.17:0/4182645144 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd164158360 con 0x7fd164152820 2024-09-10T21:47:25.286 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:25.281+0000 7fd161600640 1 -- 172.21.15.17:0/4182645144 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fd15c005470 con 0x7fd164152820 2024-09-10T21:47:25.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:25.281+0000 7fd161600640 1 --2- 172.21.15.17:0/4182645144 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fd1400725e0 0x7fd140074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:25.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:25.281+0000 7fd161600640 1 -- 172.21.15.17:0/4182645144 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fd15c093330 con 0x7fd164152820 2024-09-10T21:47:25.287 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:25.281+0000 7fd168e00640 1 --2- 172.21.15.17:0/4182645144 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fd1400725e0 0x7fd140074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:25.288 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:25.285+0000 7fd168e00640 1 --2- 172.21.15.17:0/4182645144 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fd1400725e0 0x7fd140074aa0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fd154007fd0 tx=0x7fd154009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:25.291 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:25.285+0000 7fd161600640 1 -- 172.21.15.17:0/4182645144 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fd1641f42c0 con 0x7fd164152820 2024-09-10T21:47:25.560 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:25.557+0000 7fd16b800640 1 -- 172.21.15.17:0/4182645144 --> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd1640cc330 con 0x7fd1400725e0 2024-09-10T21:47:25.566 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:25.561+0000 7fd161600640 1 -- 172.21.15.17:0/4182645144 <== mgr.4100 v2:172.21.15.31:6808/3101308586 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+333952 (secure 0 0 0) 0x7fd1640cc330 con 0x7fd1400725e0 2024-09-10T21:47:25.569 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-10T21:47:25.571 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2024-09-10T21:47:25.572 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:25.565+0000 7fd16b800640 1 -- 172.21.15.17:0/4182645144 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fd1400725e0 msgr2=0x7fd140074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:25.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:25.565+0000 7fd16b800640 1 --2- 172.21.15.17:0/4182645144 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fd1400725e0 0x7fd140074aa0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fd154007fd0 tx=0x7fd154009040 comp rx=0 tx=0).stop 2024-09-10T21:47:25.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:25.565+0000 7fd16b800640 1 -- 172.21.15.17:0/4182645144 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd164152820 msgr2=0x7fd16406f580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:25.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:25.565+0000 7fd16b800640 1 --2- 172.21.15.17:0/4182645144 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd164152820 0x7fd16406f580 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fd15c00c760 tx=0x7fd15c00cc20 comp rx=0 tx=0).stop 2024-09-10T21:47:25.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:25.565+0000 7fd16b800640 1 -- 172.21.15.17:0/4182645144 shutdown_connections 2024-09-10T21:47:25.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:25.565+0000 7fd16b800640 1 --2- 172.21.15.17:0/4182645144 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fd1400725e0 0x7fd140074aa0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:25.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:25.565+0000 7fd16b800640 1 --2- 172.21.15.17:0/4182645144 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fd164152820 0x7fd16406f580 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:25.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:25.565+0000 7fd16b800640 1 --2- 172.21.15.17:0/4182645144 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fd164152440 0x7fd16415b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:25.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:25.565+0000 7fd16b800640 1 --2- 172.21.15.17:0/4182645144 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fd1640cdf20 0x7fd1641f3d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:25.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:25.565+0000 7fd16b800640 1 -- 172.21.15.17:0/4182645144 >> 172.21.15.17:0/4182645144 conn(0x7fd1640702f0 msgr2=0x7fd16414df80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:25.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:25.565+0000 7fd16b800640 1 -- 172.21.15.17:0/4182645144 shutdown_connections 2024-09-10T21:47:25.573 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:25.565+0000 7fd16b800640 1 -- 172.21.15.17:0/4182645144 wait complete. 2024-09-10T21:47:25.591 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":705,"stamp":"2024-09-10T21:47:23.986463+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":15581531,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":114,"num_read_kb":174,"num_write":726807,"num_write_kb":16011424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":152579,"ondisk_log_size":152579,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":6469456,"kb_used_data":20368,"kb_used_omap":12,"kb_used_meta":6448947,"kb_avail":748505264,"statfs":{"total":773094113280,"available":766469390336,"internally_reserved":0,"allocated":20856832,"data_stored":32095942,"data_compressed":2631228,"data_compressed_allocated":3596288,"data_compressed_original":16236544,"omap_allocated":12752,"internal_metadata":6603722288},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":249,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003421"},"pg_stats":[{"pgid":"2.1f","version":"27'178","reported_seq":215,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.283456+0000","last_change":"2024-09-10T21:24:46.864556+0000","last_active":"2024-09-10T21:47:11.283456+0000","last_peered":"2024-09-10T21:47:11.283456+0000","last_clean":"2024-09-10T21:47:11.283456+0000","last_became_active":"2024-09-10T21:24:46.864288+0000","last_became_peered":"2024-09-10T21:24:46.864288+0000","last_unstale":"2024-09-10T21:47:11.283456+0000","last_undegraded":"2024-09-10T21:47:11.283456+0000","last_fullsized":"2024-09-10T21:47:11.283456+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":178,"log_dups_size":0,"ondisk_log_size":178,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:00:38.734807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2316909,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":173,"num_write_kb":2361,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"27'12","reported_seq":48,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.345156+0000","last_change":"2024-09-10T21:24:47.864452+0000","last_active":"2024-09-10T21:47:11.345156+0000","last_peered":"2024-09-10T21:47:11.345156+0000","last_clean":"2024-09-10T21:47:11.345156+0000","last_became_active":"2024-09-10T21:24:47.864168+0000","last_became_peered":"2024-09-10T21:24:47.864168+0000","last_unstale":"2024-09-10T21:47:11.345156+0000","last_undegraded":"2024-09-10T21:47:11.345156+0000","last_fullsized":"2024-09-10T21:47:11.345156+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:51:31.417845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"27'5290","reported_seq":5318,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.730211+0000","last_change":"2024-09-10T21:24:52.070054+0000","last_active":"2024-09-10T21:46:48.730211+0000","last_peered":"2024-09-10T21:46:48.730211+0000","last_clean":"2024-09-10T21:46:48.730211+0000","last_became_active":"2024-09-10T21:24:52.067258+0000","last_became_peered":"2024-09-10T21:24:52.067258+0000","last_unstale":"2024-09-10T21:46:48.730211+0000","last_undegraded":"2024-09-10T21:46:48.730211+0000","last_fullsized":"2024-09-10T21:46:48.730211+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":5290,"log_dups_size":0,"ondisk_log_size":5290,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:21:53.080013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25332,"num_write_kb":555711,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"27'5","reported_seq":43,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.553182+0000","last_change":"2024-09-10T21:24:46.860648+0000","last_active":"2024-09-10T21:46:48.553182+0000","last_peered":"2024-09-10T21:46:48.553182+0000","last_clean":"2024-09-10T21:46:48.553182+0000","last_became_active":"2024-09-10T21:24:46.860341+0000","last_became_peered":"2024-09-10T21:24:46.860341+0000","last_unstale":"2024-09-10T21:46:48.553182+0000","last_undegraded":"2024-09-10T21:46:48.553182+0000","last_fullsized":"2024-09-10T21:46:48.553182+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:39:24.819979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"27'19","reported_seq":54,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.345477+0000","last_change":"2024-09-10T21:24:47.865617+0000","last_active":"2024-09-10T21:47:11.345477+0000","last_peered":"2024-09-10T21:47:11.345477+0000","last_clean":"2024-09-10T21:47:11.345477+0000","last_became_active":"2024-09-10T21:24:47.865365+0000","last_became_peered":"2024-09-10T21:24:47.865365+0000","last_unstale":"2024-09-10T21:47:11.345477+0000","last_undegraded":"2024-09-10T21:47:11.345477+0000","last_fullsized":"2024-09-10T21:47:11.345477+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:10:05.971996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"27'4776","reported_seq":4804,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.713274+0000","last_change":"2024-09-10T21:24:51.900077+0000","last_active":"2024-09-10T21:46:48.713274+0000","last_peered":"2024-09-10T21:46:48.713274+0000","last_clean":"2024-09-10T21:46:48.713274+0000","last_became_active":"2024-09-10T21:24:51.899652+0000","last_became_peered":"2024-09-10T21:24:51.899652+0000","last_unstale":"2024-09-10T21:46:48.713274+0000","last_undegraded":"2024-09-10T21:46:48.713274+0000","last_fullsized":"2024-09-10T21:46:48.713274+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":4776,"log_dups_size":0,"ondisk_log_size":4776,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:40:49.585857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23087,"num_write_kb":517913,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"27'7","reported_seq":43,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.636914+0000","last_change":"2024-09-10T21:24:46.861040+0000","last_active":"2024-09-10T21:46:48.636914+0000","last_peered":"2024-09-10T21:46:48.636914+0000","last_clean":"2024-09-10T21:46:48.636914+0000","last_became_active":"2024-09-10T21:24:46.860780+0000","last_became_peered":"2024-09-10T21:24:46.860780+0000","last_unstale":"2024-09-10T21:46:48.636914+0000","last_undegraded":"2024-09-10T21:46:48.636914+0000","last_fullsized":"2024-09-10T21:46:48.636914+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:53:31.454472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"27'16","reported_seq":49,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.348293+0000","last_change":"2024-09-10T21:24:47.877997+0000","last_active":"2024-09-10T21:47:11.348293+0000","last_peered":"2024-09-10T21:47:11.348293+0000","last_clean":"2024-09-10T21:47:11.348293+0000","last_became_active":"2024-09-10T21:24:47.877765+0000","last_became_peered":"2024-09-10T21:24:47.877765+0000","last_unstale":"2024-09-10T21:47:11.348293+0000","last_undegraded":"2024-09-10T21:47:11.348293+0000","last_fullsized":"2024-09-10T21:47:11.348293+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:33:20.590209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"27'4118","reported_seq":4153,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.560547+0000","last_change":"2024-09-10T21:25:28.512435+0000","last_active":"2024-09-10T21:46:48.560547+0000","last_peered":"2024-09-10T21:46:48.560547+0000","last_clean":"2024-09-10T21:46:48.560547+0000","last_became_active":"2024-09-10T21:25:28.512023+0000","last_became_peered":"2024-09-10T21:25:28.512023+0000","last_unstale":"2024-09-10T21:46:48.560547+0000","last_undegraded":"2024-09-10T21:46:48.560547+0000","last_fullsized":"2024-09-10T21:46:48.560547+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":4118,"log_dups_size":0,"ondisk_log_size":4118,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:43:17.173095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18983,"num_write_kb":440322,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,2],"acting":[4,3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"27'5","reported_seq":41,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.677062+0000","last_change":"2024-09-10T21:24:46.873044+0000","last_active":"2024-09-10T21:46:48.677062+0000","last_peered":"2024-09-10T21:46:48.677062+0000","last_clean":"2024-09-10T21:46:48.677062+0000","last_became_active":"2024-09-10T21:24:46.872075+0000","last_became_peered":"2024-09-10T21:24:46.872075+0000","last_unstale":"2024-09-10T21:46:48.677062+0000","last_undegraded":"2024-09-10T21:46:48.677062+0000","last_fullsized":"2024-09-10T21:46:48.677062+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:40:33.632846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"27'30","reported_seq":63,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.346721+0000","last_change":"2024-09-10T21:24:47.878094+0000","last_active":"2024-09-10T21:47:11.346721+0000","last_peered":"2024-09-10T21:47:11.346721+0000","last_clean":"2024-09-10T21:47:11.346721+0000","last_became_active":"2024-09-10T21:24:47.876688+0000","last_became_peered":"2024-09-10T21:24:47.876688+0000","last_unstale":"2024-09-10T21:47:11.346721+0000","last_undegraded":"2024-09-10T21:47:11.346721+0000","last_fullsized":"2024-09-10T21:47:11.346721+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:03:37.076080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"27'3999","reported_seq":4026,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.559967+0000","last_change":"2024-09-10T21:24:51.900208+0000","last_active":"2024-09-10T21:46:48.559967+0000","last_peered":"2024-09-10T21:46:48.559967+0000","last_clean":"2024-09-10T21:46:48.559967+0000","last_became_active":"2024-09-10T21:24:51.899576+0000","last_became_peered":"2024-09-10T21:24:51.899576+0000","last_unstale":"2024-09-10T21:46:48.559967+0000","last_undegraded":"2024-09-10T21:46:48.559967+0000","last_fullsized":"2024-09-10T21:46:48.559967+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":3999,"log_dups_size":0,"ondisk_log_size":3999,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:52:49.479932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18841,"num_write_kb":435514,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"27'15","reported_seq":49,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.350364+0000","last_change":"2024-09-10T21:24:47.869830+0000","last_active":"2024-09-10T21:47:11.350364+0000","last_peered":"2024-09-10T21:47:11.350364+0000","last_clean":"2024-09-10T21:47:11.350364+0000","last_became_active":"2024-09-10T21:24:47.869569+0000","last_became_peered":"2024-09-10T21:24:47.869569+0000","last_unstale":"2024-09-10T21:47:11.350364+0000","last_undegraded":"2024-09-10T21:47:11.350364+0000","last_fullsized":"2024-09-10T21:47:11.350364+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:42:15.320518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"27'1003","reported_seq":1040,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.666697+0000","last_change":"2024-09-10T21:24:46.867297+0000","last_active":"2024-09-10T21:46:48.666697+0000","last_peered":"2024-09-10T21:46:48.666697+0000","last_clean":"2024-09-10T21:46:48.666697+0000","last_became_active":"2024-09-10T21:24:46.866893+0000","last_became_peered":"2024-09-10T21:24:46.866893+0000","last_unstale":"2024-09-10T21:46:48.666697+0000","last_undegraded":"2024-09-10T21:46:48.666697+0000","last_fullsized":"2024-09-10T21:46:48.666697+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":1003,"log_dups_size":0,"ondisk_log_size":1003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:29:53.197666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":997,"num_write_kb":4359,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"27'4606","reported_seq":4634,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.705887+0000","last_change":"2024-09-10T21:24:51.908412+0000","last_active":"2024-09-10T21:46:48.705887+0000","last_peered":"2024-09-10T21:46:48.705887+0000","last_clean":"2024-09-10T21:46:48.705887+0000","last_became_active":"2024-09-10T21:24:51.907923+0000","last_became_peered":"2024-09-10T21:24:51.907923+0000","last_unstale":"2024-09-10T21:46:48.705887+0000","last_undegraded":"2024-09-10T21:46:48.705887+0000","last_fullsized":"2024-09-10T21:46:48.705887+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":4606,"log_dups_size":0,"ondisk_log_size":4606,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:26:46.479501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22377,"num_write_kb":531442,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"27'17","reported_seq":52,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.331673+0000","last_change":"2024-09-10T21:24:47.862479+0000","last_active":"2024-09-10T21:47:11.331673+0000","last_peered":"2024-09-10T21:47:11.331673+0000","last_clean":"2024-09-10T21:47:11.331673+0000","last_became_active":"2024-09-10T21:24:47.861842+0000","last_became_peered":"2024-09-10T21:24:47.861842+0000","last_unstale":"2024-09-10T21:47:11.331673+0000","last_undegraded":"2024-09-10T21:47:11.331673+0000","last_fullsized":"2024-09-10T21:47:11.331673+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:53:09.082512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"27'3","reported_seq":39,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.551236+0000","last_change":"2024-09-10T21:24:46.852467+0000","last_active":"2024-09-10T21:46:48.551236+0000","last_peered":"2024-09-10T21:46:48.551236+0000","last_clean":"2024-09-10T21:46:48.551236+0000","last_became_active":"2024-09-10T21:24:46.851027+0000","last_became_peered":"2024-09-10T21:24:46.851027+0000","last_unstale":"2024-09-10T21:46:48.551236+0000","last_undegraded":"2024-09-10T21:46:48.551236+0000","last_fullsized":"2024-09-10T21:46:48.551236+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:39:27.441714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"27'4524","reported_seq":4552,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.720455+0000","last_change":"2024-09-10T21:24:51.903577+0000","last_active":"2024-09-10T21:46:48.720455+0000","last_peered":"2024-09-10T21:46:48.720455+0000","last_clean":"2024-09-10T21:46:48.720455+0000","last_became_active":"2024-09-10T21:24:51.903031+0000","last_became_peered":"2024-09-10T21:24:51.903031+0000","last_unstale":"2024-09-10T21:46:48.720455+0000","last_undegraded":"2024-09-10T21:46:48.720455+0000","last_fullsized":"2024-09-10T21:46:48.720455+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":4524,"log_dups_size":0,"ondisk_log_size":4524,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:08:52.787720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21954,"num_write_kb":484896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"27'18","reported_seq":53,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.328856+0000","last_change":"2024-09-10T21:24:47.856334+0000","last_active":"2024-09-10T21:47:11.328856+0000","last_peered":"2024-09-10T21:47:11.328856+0000","last_clean":"2024-09-10T21:47:11.328856+0000","last_became_active":"2024-09-10T21:24:47.856003+0000","last_became_peered":"2024-09-10T21:24:47.856003+0000","last_unstale":"2024-09-10T21:47:11.328856+0000","last_undegraded":"2024-09-10T21:47:11.328856+0000","last_fullsized":"2024-09-10T21:47:11.328856+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:58:53.698584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"27'2","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.586182+0000","last_change":"2024-09-10T21:24:46.853373+0000","last_active":"2024-09-10T21:46:48.586182+0000","last_peered":"2024-09-10T21:46:48.586182+0000","last_clean":"2024-09-10T21:46:48.586182+0000","last_became_active":"2024-09-10T21:24:46.852993+0000","last_became_peered":"2024-09-10T21:24:46.852993+0000","last_unstale":"2024-09-10T21:46:48.586182+0000","last_undegraded":"2024-09-10T21:46:48.586182+0000","last_fullsized":"2024-09-10T21:46:48.586182+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:53:31.454472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"27'5859","reported_seq":5887,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.677773+0000","last_change":"2024-09-10T21:24:52.065246+0000","last_active":"2024-09-10T21:46:48.677773+0000","last_peered":"2024-09-10T21:46:48.677773+0000","last_clean":"2024-09-10T21:46:48.677773+0000","last_became_active":"2024-09-10T21:24:52.064884+0000","last_became_peered":"2024-09-10T21:24:52.064884+0000","last_unstale":"2024-09-10T21:46:48.677773+0000","last_undegraded":"2024-09-10T21:46:48.677773+0000","last_fullsized":"2024-09-10T21:46:48.677773+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":5859,"log_dups_size":0,"ondisk_log_size":5859,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:54:54.393305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27858,"num_write_kb":655707,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"27'9","reported_seq":44,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.296539+0000","last_change":"2024-09-10T21:24:47.856517+0000","last_active":"2024-09-10T21:47:11.296539+0000","last_peered":"2024-09-10T21:47:11.296539+0000","last_clean":"2024-09-10T21:47:11.296539+0000","last_became_active":"2024-09-10T21:24:47.856185+0000","last_became_peered":"2024-09-10T21:24:47.856185+0000","last_unstale":"2024-09-10T21:47:11.296539+0000","last_undegraded":"2024-09-10T21:47:11.296539+0000","last_fullsized":"2024-09-10T21:47:11.296539+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:26:00.852818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"27'5","reported_seq":42,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:36.457065+0000","last_change":"2024-09-10T21:24:46.867888+0000","last_active":"2024-09-10T21:46:36.457065+0000","last_peered":"2024-09-10T21:46:36.457065+0000","last_clean":"2024-09-10T21:46:36.457065+0000","last_became_active":"2024-09-10T21:24:46.866477+0000","last_became_peered":"2024-09-10T21:24:46.866477+0000","last_unstale":"2024-09-10T21:46:36.457065+0000","last_undegraded":"2024-09-10T21:46:36.457065+0000","last_fullsized":"2024-09-10T21:46:36.457065+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:39:27.441714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"27'4736","reported_seq":4764,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.660872+0000","last_change":"2024-09-10T21:24:51.908376+0000","last_active":"2024-09-10T21:46:48.660872+0000","last_peered":"2024-09-10T21:46:48.660872+0000","last_clean":"2024-09-10T21:46:48.660872+0000","last_became_active":"2024-09-10T21:24:51.907878+0000","last_became_peered":"2024-09-10T21:24:51.907878+0000","last_unstale":"2024-09-10T21:46:48.660872+0000","last_undegraded":"2024-09-10T21:46:48.660872+0000","last_fullsized":"2024-09-10T21:46:48.660872+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":4736,"log_dups_size":0,"ondisk_log_size":4736,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:39:02.492952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22584,"num_write_kb":479265,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"27'25","reported_seq":60,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.323165+0000","last_change":"2024-09-10T21:24:47.844823+0000","last_active":"2024-09-10T21:47:11.323165+0000","last_peered":"2024-09-10T21:47:11.323165+0000","last_clean":"2024-09-10T21:47:11.323165+0000","last_became_active":"2024-09-10T21:24:47.844505+0000","last_became_peered":"2024-09-10T21:24:47.844505+0000","last_unstale":"2024-09-10T21:47:11.323165+0000","last_undegraded":"2024-09-10T21:47:11.323165+0000","last_fullsized":"2024-09-10T21:47:11.323165+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:37:51.784258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"27'9","reported_seq":45,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.639567+0000","last_change":"2024-09-10T21:24:46.862172+0000","last_active":"2024-09-10T21:46:48.639567+0000","last_peered":"2024-09-10T21:46:48.639567+0000","last_clean":"2024-09-10T21:46:48.639567+0000","last_became_active":"2024-09-10T21:24:46.861473+0000","last_became_peered":"2024-09-10T21:24:46.861473+0000","last_unstale":"2024-09-10T21:46:48.639567+0000","last_undegraded":"2024-09-10T21:46:48.639567+0000","last_fullsized":"2024-09-10T21:46:48.639567+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:53:31.454472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":53,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"27'4521","reported_seq":4550,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.535321+0000","last_change":"2024-09-10T21:24:52.062841+0000","last_active":"2024-09-10T21:46:48.535321+0000","last_peered":"2024-09-10T21:46:48.535321+0000","last_clean":"2024-09-10T21:46:48.535321+0000","last_became_active":"2024-09-10T21:24:52.061581+0000","last_became_peered":"2024-09-10T21:24:52.061581+0000","last_unstale":"2024-09-10T21:46:48.535321+0000","last_undegraded":"2024-09-10T21:46:48.535321+0000","last_fullsized":"2024-09-10T21:46:48.535321+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":4521,"log_dups_size":0,"ondisk_log_size":4521,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:34:03.325781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21874,"num_write_kb":476387,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"27'18","reported_seq":53,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.324770+0000","last_change":"2024-09-10T21:24:47.859801+0000","last_active":"2024-09-10T21:47:11.324770+0000","last_peered":"2024-09-10T21:47:11.324770+0000","last_clean":"2024-09-10T21:47:11.324770+0000","last_became_active":"2024-09-10T21:24:47.859483+0000","last_became_peered":"2024-09-10T21:24:47.859483+0000","last_unstale":"2024-09-10T21:47:11.324770+0000","last_undegraded":"2024-09-10T21:47:11.324770+0000","last_fullsized":"2024-09-10T21:47:11.324770+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:38:21.368499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"27'326","reported_seq":362,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:36.365019+0000","last_change":"2024-09-10T21:24:46.860624+0000","last_active":"2024-09-10T21:46:36.365019+0000","last_peered":"2024-09-10T21:46:36.365019+0000","last_clean":"2024-09-10T21:46:36.365019+0000","last_became_active":"2024-09-10T21:24:46.860309+0000","last_became_peered":"2024-09-10T21:24:46.860309+0000","last_unstale":"2024-09-10T21:46:36.365019+0000","last_undegraded":"2024-09-10T21:46:36.365019+0000","last_fullsized":"2024-09-10T21:46:36.365019+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":326,"log_dups_size":0,"ondisk_log_size":326,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:37:12.332013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":704,"num_write_kb":242,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"27'4358","reported_seq":4388,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.745954+0000","last_change":"2024-09-10T21:24:52.068581+0000","last_active":"2024-09-10T21:46:48.745954+0000","last_peered":"2024-09-10T21:46:48.745954+0000","last_clean":"2024-09-10T21:46:48.745954+0000","last_became_active":"2024-09-10T21:24:52.068250+0000","last_became_peered":"2024-09-10T21:24:52.068250+0000","last_unstale":"2024-09-10T21:46:48.745954+0000","last_undegraded":"2024-09-10T21:46:48.745954+0000","last_fullsized":"2024-09-10T21:46:48.745954+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":4358,"log_dups_size":0,"ondisk_log_size":4358,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:52:01.990009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21666,"num_write_kb":463209,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"27'18","reported_seq":52,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.321574+0000","last_change":"2024-09-10T21:24:47.854042+0000","last_active":"2024-09-10T21:47:11.321574+0000","last_peered":"2024-09-10T21:47:11.321574+0000","last_clean":"2024-09-10T21:47:11.321574+0000","last_became_active":"2024-09-10T21:24:47.853750+0000","last_became_peered":"2024-09-10T21:24:47.853750+0000","last_unstale":"2024-09-10T21:47:11.321574+0000","last_undegraded":"2024-09-10T21:47:11.321574+0000","last_fullsized":"2024-09-10T21:47:11.321574+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:06:50.775132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"27'2","reported_seq":39,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:45:42.186926+0000","last_change":"2024-09-10T21:24:46.864617+0000","last_active":"2024-09-10T21:45:42.186926+0000","last_peered":"2024-09-10T21:45:42.186926+0000","last_clean":"2024-09-10T21:45:42.186926+0000","last_became_active":"2024-09-10T21:24:46.864149+0000","last_became_peered":"2024-09-10T21:24:46.864149+0000","last_unstale":"2024-09-10T21:45:42.186926+0000","last_undegraded":"2024-09-10T21:45:42.186926+0000","last_fullsized":"2024-09-10T21:45:42.186926+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:24:15.673107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"27'4655","reported_seq":4684,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.520210+0000","last_change":"2024-09-10T21:24:52.061485+0000","last_active":"2024-09-10T21:46:48.520210+0000","last_peered":"2024-09-10T21:46:48.520210+0000","last_clean":"2024-09-10T21:46:48.520210+0000","last_became_active":"2024-09-10T21:24:52.060953+0000","last_became_peered":"2024-09-10T21:24:52.060953+0000","last_unstale":"2024-09-10T21:46:48.520210+0000","last_undegraded":"2024-09-10T21:46:48.520210+0000","last_fullsized":"2024-09-10T21:46:48.520210+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":4655,"log_dups_size":0,"ondisk_log_size":4655,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:49:30.147202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22893,"num_write_kb":513258,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"27'18","reported_seq":52,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.341860+0000","last_change":"2024-09-10T21:24:47.867241+0000","last_active":"2024-09-10T21:47:11.341860+0000","last_peered":"2024-09-10T21:47:11.341860+0000","last_clean":"2024-09-10T21:47:11.341860+0000","last_became_active":"2024-09-10T21:24:47.866907+0000","last_became_peered":"2024-09-10T21:24:47.866907+0000","last_unstale":"2024-09-10T21:47:11.341860+0000","last_undegraded":"2024-09-10T21:47:11.341860+0000","last_fullsized":"2024-09-10T21:47:11.341860+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:07.620800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"27'2","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.562765+0000","last_change":"2024-09-10T21:24:46.856848+0000","last_active":"2024-09-10T21:46:48.562765+0000","last_peered":"2024-09-10T21:46:48.562765+0000","last_clean":"2024-09-10T21:46:48.562765+0000","last_became_active":"2024-09-10T21:24:46.856489+0000","last_became_peered":"2024-09-10T21:24:46.856489+0000","last_unstale":"2024-09-10T21:46:48.562765+0000","last_undegraded":"2024-09-10T21:46:48.562765+0000","last_fullsized":"2024-09-10T21:46:48.562765+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:46:55.286321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"27'4909","reported_seq":4938,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.660587+0000","last_change":"2024-09-10T21:24:51.901016+0000","last_active":"2024-09-10T21:46:48.660587+0000","last_peered":"2024-09-10T21:46:48.660587+0000","last_clean":"2024-09-10T21:46:48.660587+0000","last_became_active":"2024-09-10T21:24:51.900479+0000","last_became_peered":"2024-09-10T21:24:51.900479+0000","last_unstale":"2024-09-10T21:46:48.660587+0000","last_undegraded":"2024-09-10T21:46:48.660587+0000","last_fullsized":"2024-09-10T21:46:48.660587+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":4909,"log_dups_size":0,"ondisk_log_size":4909,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:36.741810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23807,"num_write_kb":525103,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"27'16","reported_seq":51,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.320532+0000","last_change":"2024-09-10T21:24:47.858753+0000","last_active":"2024-09-10T21:47:11.320532+0000","last_peered":"2024-09-10T21:47:11.320532+0000","last_clean":"2024-09-10T21:47:11.320532+0000","last_became_active":"2024-09-10T21:24:47.858438+0000","last_became_peered":"2024-09-10T21:24:47.858438+0000","last_unstale":"2024-09-10T21:47:11.320532+0000","last_undegraded":"2024-09-10T21:47:11.320532+0000","last_fullsized":"2024-09-10T21:47:11.320532+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:12:09.198013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"27'4","reported_seq":40,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:45:39.089516+0000","last_change":"2024-09-10T21:24:46.863610+0000","last_active":"2024-09-10T21:45:39.089516+0000","last_peered":"2024-09-10T21:45:39.089516+0000","last_clean":"2024-09-10T21:45:39.089516+0000","last_became_active":"2024-09-10T21:24:46.863150+0000","last_became_peered":"2024-09-10T21:24:46.863150+0000","last_unstale":"2024-09-10T21:45:39.089516+0000","last_undegraded":"2024-09-10T21:45:39.089516+0000","last_fullsized":"2024-09-10T21:45:39.089516+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:27:35.473995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"27'5335","reported_seq":5364,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.650031+0000","last_change":"2024-09-10T21:24:51.907366+0000","last_active":"2024-09-10T21:46:48.650031+0000","last_peered":"2024-09-10T21:46:48.650031+0000","last_clean":"2024-09-10T21:46:48.650031+0000","last_became_active":"2024-09-10T21:24:51.906888+0000","last_became_peered":"2024-09-10T21:24:51.906888+0000","last_unstale":"2024-09-10T21:46:48.650031+0000","last_undegraded":"2024-09-10T21:46:48.650031+0000","last_fullsized":"2024-09-10T21:46:48.650031+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":5335,"log_dups_size":0,"ondisk_log_size":5335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:24:54.278514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26491,"num_write_kb":568740,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"27'20","reported_seq":55,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.311229+0000","last_change":"2024-09-10T21:24:47.850155+0000","last_active":"2024-09-10T21:47:11.311229+0000","last_peered":"2024-09-10T21:47:11.311229+0000","last_clean":"2024-09-10T21:47:11.311229+0000","last_became_active":"2024-09-10T21:24:47.849846+0000","last_became_peered":"2024-09-10T21:24:47.849846+0000","last_unstale":"2024-09-10T21:47:11.311229+0000","last_undegraded":"2024-09-10T21:47:11.311229+0000","last_fullsized":"2024-09-10T21:47:11.311229+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:13:40.032808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"27'52","reported_seq":145,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:23.121361+0000","last_change":"2024-09-10T21:47:23.121361+0000","last_active":"2024-09-10T21:47:23.121361+0000","last_peered":"2024-09-10T21:47:23.121361+0000","last_clean":"2024-09-10T21:47:23.121361+0000","last_became_active":"2024-09-10T21:24:46.851200+0000","last_became_peered":"2024-09-10T21:24:46.851200+0000","last_unstale":"2024-09-10T21:47:23.121361+0000","last_undegraded":"2024-09-10T21:47:23.121361+0000","last_fullsized":"2024-09-10T21:47:23.121361+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'52","last_scrub_stamp":"2024-09-10T21:47:23.121257+0000","last_deep_scrub":"27'52","last_deep_scrub_stamp":"2024-09-10T21:47:23.121257+0000","last_clean_scrub_stamp":"2024-09-10T21:47:23.121257+0000","objects_scrubbed":1,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:43:57.409412+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90454,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":48,"num_read_kb":119,"num_write":48,"num_write_kb":119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"27'4677","reported_seq":4706,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.581789+0000","last_change":"2024-09-10T21:24:51.902529+0000","last_active":"2024-09-10T21:46:48.581789+0000","last_peered":"2024-09-10T21:46:48.581789+0000","last_clean":"2024-09-10T21:46:48.581789+0000","last_became_active":"2024-09-10T21:24:51.901141+0000","last_became_peered":"2024-09-10T21:24:51.901141+0000","last_unstale":"2024-09-10T21:46:48.581789+0000","last_undegraded":"2024-09-10T21:46:48.581789+0000","last_fullsized":"2024-09-10T21:46:48.581789+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":4677,"log_dups_size":0,"ondisk_log_size":4677,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:08:40.203348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22963,"num_write_kb":501268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"27'11","reported_seq":46,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.323431+0000","last_change":"2024-09-10T21:24:47.846582+0000","last_active":"2024-09-10T21:47:11.323431+0000","last_peered":"2024-09-10T21:47:11.323431+0000","last_clean":"2024-09-10T21:47:11.323431+0000","last_became_active":"2024-09-10T21:24:47.846358+0000","last_became_peered":"2024-09-10T21:24:47.846358+0000","last_unstale":"2024-09-10T21:47:11.323431+0000","last_undegraded":"2024-09-10T21:47:11.323431+0000","last_fullsized":"2024-09-10T21:47:11.323431+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:58:53.698584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"27'7","reported_seq":45,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.647854+0000","last_change":"2024-09-10T21:24:46.870908+0000","last_active":"2024-09-10T21:46:48.647854+0000","last_peered":"2024-09-10T21:46:48.647854+0000","last_clean":"2024-09-10T21:46:48.647854+0000","last_became_active":"2024-09-10T21:24:46.868375+0000","last_became_peered":"2024-09-10T21:24:46.868375+0000","last_unstale":"2024-09-10T21:46:48.647854+0000","last_undegraded":"2024-09-10T21:46:48.647854+0000","last_fullsized":"2024-09-10T21:46:48.647854+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:37:12.332013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"27'4824","reported_seq":4853,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.659490+0000","last_change":"2024-09-10T21:24:52.069018+0000","last_active":"2024-09-10T21:46:48.659490+0000","last_peered":"2024-09-10T21:46:48.659490+0000","last_clean":"2024-09-10T21:46:48.659490+0000","last_became_active":"2024-09-10T21:24:52.067308+0000","last_became_peered":"2024-09-10T21:24:52.067308+0000","last_unstale":"2024-09-10T21:46:48.659490+0000","last_undegraded":"2024-09-10T21:46:48.659490+0000","last_fullsized":"2024-09-10T21:46:48.659490+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":4824,"log_dups_size":0,"ondisk_log_size":4824,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:07:25.162168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23972,"num_write_kb":500992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"27'13","reported_seq":48,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.338908+0000","last_change":"2024-09-10T21:24:47.871542+0000","last_active":"2024-09-10T21:47:11.338908+0000","last_peered":"2024-09-10T21:47:11.338908+0000","last_clean":"2024-09-10T21:47:11.338908+0000","last_became_active":"2024-09-10T21:24:47.871128+0000","last_became_peered":"2024-09-10T21:24:47.871128+0000","last_unstale":"2024-09-10T21:47:11.338908+0000","last_undegraded":"2024-09-10T21:47:11.338908+0000","last_fullsized":"2024-09-10T21:47:11.338908+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:51:22.876126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"27'3","reported_seq":39,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:36.366557+0000","last_change":"2024-09-10T21:24:46.859218+0000","last_active":"2024-09-10T21:46:36.366557+0000","last_peered":"2024-09-10T21:46:36.366557+0000","last_clean":"2024-09-10T21:46:36.366557+0000","last_became_active":"2024-09-10T21:24:46.853190+0000","last_became_peered":"2024-09-10T21:24:46.853190+0000","last_unstale":"2024-09-10T21:46:36.366557+0000","last_undegraded":"2024-09-10T21:46:36.366557+0000","last_fullsized":"2024-09-10T21:46:36.366557+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:26:34.848759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"17'72","reported_seq":134,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:44:42.019323+0000","last_change":"2024-09-10T21:24:37.216542+0000","last_active":"2024-09-10T21:44:42.019323+0000","last_peered":"2024-09-10T21:44:42.019323+0000","last_clean":"2024-09-10T21:44:42.019323+0000","last_became_active":"2024-09-10T21:24:37.215443+0000","last_became_peered":"2024-09-10T21:24:37.215443+0000","last_unstale":"2024-09-10T21:44:42.019323+0000","last_undegraded":"2024-09-10T21:44:42.019323+0000","last_fullsized":"2024-09-10T21:44:42.019323+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:35.877801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:35.877801+0000","last_clean_scrub_stamp":"2024-09-10T21:24:35.877801+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:14:20.260329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"27'4394","reported_seq":4423,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.650952+0000","last_change":"2024-09-10T21:24:52.067645+0000","last_active":"2024-09-10T21:46:48.650952+0000","last_peered":"2024-09-10T21:46:48.650952+0000","last_clean":"2024-09-10T21:46:48.650952+0000","last_became_active":"2024-09-10T21:24:52.067148+0000","last_became_peered":"2024-09-10T21:24:52.067148+0000","last_unstale":"2024-09-10T21:46:48.650952+0000","last_undegraded":"2024-09-10T21:46:48.650952+0000","last_fullsized":"2024-09-10T21:46:48.650952+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":4394,"log_dups_size":0,"ondisk_log_size":4394,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:54:42.435011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21702,"num_write_kb":479680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"27'21","reported_seq":62,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.342697+0000","last_change":"2024-09-10T21:25:28.510354+0000","last_active":"2024-09-10T21:47:11.342697+0000","last_peered":"2024-09-10T21:47:11.342697+0000","last_clean":"2024-09-10T21:47:11.342697+0000","last_became_active":"2024-09-10T21:25:28.509981+0000","last_became_peered":"2024-09-10T21:25:28.509981+0000","last_unstale":"2024-09-10T21:47:11.342697+0000","last_undegraded":"2024-09-10T21:47:11.342697+0000","last_fullsized":"2024-09-10T21:47:11.342697+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:51:42.501764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"27'5","reported_seq":42,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.646581+0000","last_change":"2024-09-10T21:24:46.859532+0000","last_active":"2024-09-10T21:46:48.646581+0000","last_peered":"2024-09-10T21:46:48.646581+0000","last_clean":"2024-09-10T21:46:48.646581+0000","last_became_active":"2024-09-10T21:24:46.859144+0000","last_became_peered":"2024-09-10T21:24:46.859144+0000","last_unstale":"2024-09-10T21:46:48.646581+0000","last_undegraded":"2024-09-10T21:46:48.646581+0000","last_fullsized":"2024-09-10T21:46:48.646581+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:48:41.992441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"27'4328","reported_seq":4365,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.651243+0000","last_change":"2024-09-10T21:25:28.620859+0000","last_active":"2024-09-10T21:46:48.651243+0000","last_peered":"2024-09-10T21:46:48.651243+0000","last_clean":"2024-09-10T21:46:48.651243+0000","last_became_active":"2024-09-10T21:25:28.620415+0000","last_became_peered":"2024-09-10T21:25:28.620415+0000","last_unstale":"2024-09-10T21:46:48.651243+0000","last_undegraded":"2024-09-10T21:46:48.651243+0000","last_fullsized":"2024-09-10T21:46:48.651243+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":4328,"log_dups_size":0,"ondisk_log_size":4328,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:17:39.229564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21368,"num_write_kb":498780,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2,0],"acting":[6,7,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"27'17","reported_seq":51,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.342613+0000","last_change":"2024-09-10T21:24:47.866927+0000","last_active":"2024-09-10T21:47:11.342613+0000","last_peered":"2024-09-10T21:47:11.342613+0000","last_clean":"2024-09-10T21:47:11.342613+0000","last_became_active":"2024-09-10T21:24:47.866650+0000","last_became_peered":"2024-09-10T21:24:47.866650+0000","last_unstale":"2024-09-10T21:47:11.342613+0000","last_undegraded":"2024-09-10T21:47:11.342613+0000","last_fullsized":"2024-09-10T21:47:11.342613+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:47:37.304624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"27'6","reported_seq":42,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.675955+0000","last_change":"2024-09-10T21:24:46.857082+0000","last_active":"2024-09-10T21:46:48.675955+0000","last_peered":"2024-09-10T21:46:48.675955+0000","last_clean":"2024-09-10T21:46:48.675955+0000","last_became_active":"2024-09-10T21:24:46.856799+0000","last_became_peered":"2024-09-10T21:24:46.856799+0000","last_unstale":"2024-09-10T21:46:48.675955+0000","last_undegraded":"2024-09-10T21:46:48.675955+0000","last_fullsized":"2024-09-10T21:46:48.675955+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:40:33.632846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"27'4625","reported_seq":4654,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.739934+0000","last_change":"2024-09-10T21:24:51.911880+0000","last_active":"2024-09-10T21:46:48.739934+0000","last_peered":"2024-09-10T21:46:48.739934+0000","last_clean":"2024-09-10T21:46:48.739934+0000","last_became_active":"2024-09-10T21:24:51.911088+0000","last_became_peered":"2024-09-10T21:24:51.911088+0000","last_unstale":"2024-09-10T21:46:48.739934+0000","last_undegraded":"2024-09-10T21:46:48.739934+0000","last_fullsized":"2024-09-10T21:46:48.739934+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":4625,"log_dups_size":0,"ondisk_log_size":4625,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:38:57.030655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21860,"num_write_kb":505231,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"27'20","reported_seq":54,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.344779+0000","last_change":"2024-09-10T21:24:47.876946+0000","last_active":"2024-09-10T21:47:11.344779+0000","last_peered":"2024-09-10T21:47:11.344779+0000","last_clean":"2024-09-10T21:47:11.344779+0000","last_became_active":"2024-09-10T21:24:47.876690+0000","last_became_peered":"2024-09-10T21:24:47.876690+0000","last_unstale":"2024-09-10T21:47:11.344779+0000","last_undegraded":"2024-09-10T21:47:11.344779+0000","last_fullsized":"2024-09-10T21:47:11.344779+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:14:06.368275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"27'7","reported_seq":44,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.583192+0000","last_change":"2024-09-10T21:24:46.855248+0000","last_active":"2024-09-10T21:46:48.583192+0000","last_peered":"2024-09-10T21:46:48.583192+0000","last_clean":"2024-09-10T21:46:48.583192+0000","last_became_active":"2024-09-10T21:24:46.854930+0000","last_became_peered":"2024-09-10T21:24:46.854930+0000","last_unstale":"2024-09-10T21:46:48.583192+0000","last_undegraded":"2024-09-10T21:46:48.583192+0000","last_fullsized":"2024-09-10T21:46:48.583192+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:46:55.286321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"27'4395","reported_seq":4424,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.566233+0000","last_change":"2024-09-10T21:24:52.063112+0000","last_active":"2024-09-10T21:46:48.566233+0000","last_peered":"2024-09-10T21:46:48.566233+0000","last_clean":"2024-09-10T21:46:48.566233+0000","last_became_active":"2024-09-10T21:24:52.062395+0000","last_became_peered":"2024-09-10T21:24:52.062395+0000","last_unstale":"2024-09-10T21:46:48.566233+0000","last_undegraded":"2024-09-10T21:46:48.566233+0000","last_fullsized":"2024-09-10T21:46:48.566233+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":4395,"log_dups_size":0,"ondisk_log_size":4395,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:21:53.080013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21722,"num_write_kb":475710,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"27'13","reported_seq":33,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.321587+0000","last_change":"2024-09-10T21:25:28.513604+0000","last_active":"2024-09-10T21:47:11.321587+0000","last_peered":"2024-09-10T21:47:11.321587+0000","last_clean":"2024-09-10T21:47:11.321587+0000","last_became_active":"2024-09-10T21:25:28.513250+0000","last_became_peered":"2024-09-10T21:25:28.513250+0000","last_unstale":"2024-09-10T21:47:11.321587+0000","last_undegraded":"2024-09-10T21:47:11.321587+0000","last_fullsized":"2024-09-10T21:47:11.321587+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:54:13.200082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.f","version":"27'4","reported_seq":41,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.609935+0000","last_change":"2024-09-10T21:24:46.860330+0000","last_active":"2024-09-10T21:46:48.609935+0000","last_peered":"2024-09-10T21:46:48.609935+0000","last_clean":"2024-09-10T21:46:48.609935+0000","last_became_active":"2024-09-10T21:24:46.860053+0000","last_became_peered":"2024-09-10T21:24:46.860053+0000","last_unstale":"2024-09-10T21:46:48.609935+0000","last_undegraded":"2024-09-10T21:46:48.609935+0000","last_fullsized":"2024-09-10T21:46:48.609935+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:58:53.177740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"27'4437","reported_seq":4466,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.614888+0000","last_change":"2024-09-10T21:24:51.901673+0000","last_active":"2024-09-10T21:46:48.614888+0000","last_peered":"2024-09-10T21:46:48.614888+0000","last_clean":"2024-09-10T21:46:48.614888+0000","last_became_active":"2024-09-10T21:24:51.901222+0000","last_became_peered":"2024-09-10T21:24:51.901222+0000","last_unstale":"2024-09-10T21:46:48.614888+0000","last_undegraded":"2024-09-10T21:46:48.614888+0000","last_fullsized":"2024-09-10T21:46:48.614888+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":4437,"log_dups_size":0,"ondisk_log_size":4437,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:07:52.120299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21706,"num_write_kb":463399,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"27'19","reported_seq":53,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.328131+0000","last_change":"2024-09-10T21:24:47.870589+0000","last_active":"2024-09-10T21:47:11.328131+0000","last_peered":"2024-09-10T21:47:11.328131+0000","last_clean":"2024-09-10T21:47:11.328131+0000","last_became_active":"2024-09-10T21:24:47.861834+0000","last_became_peered":"2024-09-10T21:24:47.861834+0000","last_unstale":"2024-09-10T21:47:11.328131+0000","last_undegraded":"2024-09-10T21:47:11.328131+0000","last_fullsized":"2024-09-10T21:47:11.328131+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:43.363013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"27'15","reported_seq":51,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.643062+0000","last_change":"2024-09-10T21:24:46.852064+0000","last_active":"2024-09-10T21:46:48.643062+0000","last_peered":"2024-09-10T21:46:48.643062+0000","last_clean":"2024-09-10T21:46:48.643062+0000","last_became_active":"2024-09-10T21:24:46.851776+0000","last_became_peered":"2024-09-10T21:24:46.851776+0000","last_unstale":"2024-09-10T21:46:48.643062+0000","last_undegraded":"2024-09-10T21:46:48.643062+0000","last_fullsized":"2024-09-10T21:46:48.643062+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:26:24.371844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"27'5330","reported_seq":5358,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.742778+0000","last_change":"2024-09-10T21:24:51.904449+0000","last_active":"2024-09-10T21:46:48.742778+0000","last_peered":"2024-09-10T21:46:48.742778+0000","last_clean":"2024-09-10T21:46:48.742778+0000","last_became_active":"2024-09-10T21:24:51.903533+0000","last_became_peered":"2024-09-10T21:24:51.903533+0000","last_unstale":"2024-09-10T21:46:48.742778+0000","last_undegraded":"2024-09-10T21:46:48.742778+0000","last_fullsized":"2024-09-10T21:46:48.742778+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":5330,"log_dups_size":0,"ondisk_log_size":5330,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:44:11.654935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26446,"num_write_kb":547906,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"27'14","reported_seq":47,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.347665+0000","last_change":"2024-09-10T21:24:47.857736+0000","last_active":"2024-09-10T21:47:11.347665+0000","last_peered":"2024-09-10T21:47:11.347665+0000","last_clean":"2024-09-10T21:47:11.347665+0000","last_became_active":"2024-09-10T21:24:47.857437+0000","last_became_peered":"2024-09-10T21:24:47.857437+0000","last_unstale":"2024-09-10T21:47:11.347665+0000","last_undegraded":"2024-09-10T21:47:11.347665+0000","last_fullsized":"2024-09-10T21:47:11.347665+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:13:40.032808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"27'5","reported_seq":40,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.633944+0000","last_change":"2024-09-10T21:24:46.860662+0000","last_active":"2024-09-10T21:46:48.633944+0000","last_peered":"2024-09-10T21:46:48.633944+0000","last_clean":"2024-09-10T21:46:48.633944+0000","last_became_active":"2024-09-10T21:24:46.859796+0000","last_became_peered":"2024-09-10T21:24:46.859796+0000","last_unstale":"2024-09-10T21:46:48.633944+0000","last_undegraded":"2024-09-10T21:46:48.633944+0000","last_fullsized":"2024-09-10T21:46:48.633944+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:48:27.758481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"27'5017","reported_seq":5045,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.677969+0000","last_change":"2024-09-10T21:24:52.069770+0000","last_active":"2024-09-10T21:46:48.677969+0000","last_peered":"2024-09-10T21:46:48.677969+0000","last_clean":"2024-09-10T21:46:48.677969+0000","last_became_active":"2024-09-10T21:24:52.067409+0000","last_became_peered":"2024-09-10T21:24:52.067409+0000","last_unstale":"2024-09-10T21:46:48.677969+0000","last_undegraded":"2024-09-10T21:46:48.677969+0000","last_fullsized":"2024-09-10T21:46:48.677969+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":5017,"log_dups_size":0,"ondisk_log_size":5017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:23:39.622612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24522,"num_write_kb":548501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"27'30","reported_seq":64,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.343245+0000","last_change":"2024-09-10T21:24:47.858692+0000","last_active":"2024-09-10T21:47:11.343245+0000","last_peered":"2024-09-10T21:47:11.343245+0000","last_clean":"2024-09-10T21:47:11.343245+0000","last_became_active":"2024-09-10T21:24:47.858343+0000","last_became_peered":"2024-09-10T21:24:47.858343+0000","last_unstale":"2024-09-10T21:47:11.343245+0000","last_undegraded":"2024-09-10T21:47:11.343245+0000","last_fullsized":"2024-09-10T21:47:11.343245+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:04:36.661791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"27'3","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.555096+0000","last_change":"2024-09-10T21:24:46.864592+0000","last_active":"2024-09-10T21:46:48.555096+0000","last_peered":"2024-09-10T21:46:48.555096+0000","last_clean":"2024-09-10T21:46:48.555096+0000","last_became_active":"2024-09-10T21:24:46.858696+0000","last_became_peered":"2024-09-10T21:24:46.858696+0000","last_unstale":"2024-09-10T21:46:48.555096+0000","last_undegraded":"2024-09-10T21:46:48.555096+0000","last_fullsized":"2024-09-10T21:46:48.555096+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:54:19.356867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"27'4314","reported_seq":4350,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.681791+0000","last_change":"2024-09-10T21:25:28.802910+0000","last_active":"2024-09-10T21:46:48.681791+0000","last_peered":"2024-09-10T21:46:48.681791+0000","last_clean":"2024-09-10T21:46:48.681791+0000","last_became_active":"2024-09-10T21:25:28.802224+0000","last_became_peered":"2024-09-10T21:25:28.802224+0000","last_unstale":"2024-09-10T21:46:48.681791+0000","last_undegraded":"2024-09-10T21:46:48.681791+0000","last_fullsized":"2024-09-10T21:46:48.681791+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":4314,"log_dups_size":0,"ondisk_log_size":4314,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:27:09.948701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20768,"num_write_kb":476051,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7,2],"acting":[3,5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"27'23","reported_seq":57,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.332930+0000","last_change":"2024-09-10T21:24:47.857699+0000","last_active":"2024-09-10T21:47:11.332930+0000","last_peered":"2024-09-10T21:47:11.332930+0000","last_clean":"2024-09-10T21:47:11.332930+0000","last_became_active":"2024-09-10T21:24:47.857429+0000","last_became_peered":"2024-09-10T21:24:47.857429+0000","last_unstale":"2024-09-10T21:47:11.332930+0000","last_undegraded":"2024-09-10T21:47:11.332930+0000","last_fullsized":"2024-09-10T21:47:11.332930+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:07:11.880228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"27'6","reported_seq":42,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.634621+0000","last_change":"2024-09-10T21:24:46.859753+0000","last_active":"2024-09-10T21:46:48.634621+0000","last_peered":"2024-09-10T21:46:48.634621+0000","last_clean":"2024-09-10T21:46:48.634621+0000","last_became_active":"2024-09-10T21:24:46.859299+0000","last_became_peered":"2024-09-10T21:24:46.859299+0000","last_unstale":"2024-09-10T21:46:48.634621+0000","last_undegraded":"2024-09-10T21:46:48.634621+0000","last_fullsized":"2024-09-10T21:46:48.634621+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:06:41.965052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"27'4196","reported_seq":4224,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.388436+0000","last_change":"2024-09-10T21:24:52.064718+0000","last_active":"2024-09-10T21:46:48.388436+0000","last_peered":"2024-09-10T21:46:48.388436+0000","last_clean":"2024-09-10T21:46:48.388436+0000","last_became_active":"2024-09-10T21:24:52.062327+0000","last_became_peered":"2024-09-10T21:24:52.062327+0000","last_unstale":"2024-09-10T21:46:48.388436+0000","last_undegraded":"2024-09-10T21:46:48.388436+0000","last_fullsized":"2024-09-10T21:46:48.388436+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":4196,"log_dups_size":0,"ondisk_log_size":4196,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:43:17.173095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20531,"num_write_kb":446498,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"27'18","reported_seq":52,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.334694+0000","last_change":"2024-09-10T21:24:47.870766+0000","last_active":"2024-09-10T21:47:11.334694+0000","last_peered":"2024-09-10T21:47:11.334694+0000","last_clean":"2024-09-10T21:47:11.334694+0000","last_became_active":"2024-09-10T21:24:47.870276+0000","last_became_peered":"2024-09-10T21:24:47.870276+0000","last_unstale":"2024-09-10T21:47:11.334694+0000","last_undegraded":"2024-09-10T21:47:11.334694+0000","last_fullsized":"2024-09-10T21:47:11.334694+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:54:38.400877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"27'368","reported_seq":403,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:28.433029+0000","last_change":"2024-09-10T21:24:46.867790+0000","last_active":"2024-09-10T21:46:28.433029+0000","last_peered":"2024-09-10T21:46:28.433029+0000","last_clean":"2024-09-10T21:46:28.433029+0000","last_became_active":"2024-09-10T21:24:46.866899+0000","last_became_peered":"2024-09-10T21:24:46.866899+0000","last_unstale":"2024-09-10T21:46:28.433029+0000","last_undegraded":"2024-09-10T21:46:28.433029+0000","last_fullsized":"2024-09-10T21:46:28.433029+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":368,"log_dups_size":0,"ondisk_log_size":368,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:25:04.923730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194976,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":367,"num_write_kb":4285,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"27'3971","reported_seq":3999,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.743646+0000","last_change":"2024-09-10T21:24:51.899803+0000","last_active":"2024-09-10T21:46:48.743646+0000","last_peered":"2024-09-10T21:46:48.743646+0000","last_clean":"2024-09-10T21:46:48.743646+0000","last_became_active":"2024-09-10T21:24:51.895398+0000","last_became_peered":"2024-09-10T21:24:51.895398+0000","last_unstale":"2024-09-10T21:46:48.743646+0000","last_undegraded":"2024-09-10T21:46:48.743646+0000","last_fullsized":"2024-09-10T21:46:48.743646+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":3971,"log_dups_size":0,"ondisk_log_size":3971,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:52:34.689860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19337,"num_write_kb":445978,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"27'22","reported_seq":56,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.334366+0000","last_change":"2024-09-10T21:24:47.857544+0000","last_active":"2024-09-10T21:47:11.334366+0000","last_peered":"2024-09-10T21:47:11.334366+0000","last_clean":"2024-09-10T21:47:11.334366+0000","last_became_active":"2024-09-10T21:24:47.857284+0000","last_became_peered":"2024-09-10T21:24:47.857284+0000","last_unstale":"2024-09-10T21:47:11.334366+0000","last_undegraded":"2024-09-10T21:47:11.334366+0000","last_fullsized":"2024-09-10T21:47:11.334366+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:10:22.747606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"27'32","reported_seq":67,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.672745+0000","last_change":"2024-09-10T21:24:46.859633+0000","last_active":"2024-09-10T21:46:48.672745+0000","last_peered":"2024-09-10T21:46:48.672745+0000","last_clean":"2024-09-10T21:46:48.672745+0000","last_became_active":"2024-09-10T21:24:46.859332+0000","last_became_peered":"2024-09-10T21:24:46.859332+0000","last_unstale":"2024-09-10T21:46:48.672745+0000","last_undegraded":"2024-09-10T21:46:48.672745+0000","last_fullsized":"2024-09-10T21:46:48.672745+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:56:46.975225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":33,"num_write_kb":30,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"27'4136","reported_seq":4164,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.629023+0000","last_change":"2024-09-10T21:24:51.899100+0000","last_active":"2024-09-10T21:46:48.629023+0000","last_peered":"2024-09-10T21:46:48.629023+0000","last_clean":"2024-09-10T21:46:48.629023+0000","last_became_active":"2024-09-10T21:24:51.898642+0000","last_became_peered":"2024-09-10T21:24:51.898642+0000","last_unstale":"2024-09-10T21:46:48.629023+0000","last_undegraded":"2024-09-10T21:46:48.629023+0000","last_fullsized":"2024-09-10T21:46:48.629023+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":4136,"log_dups_size":0,"ondisk_log_size":4136,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:30:06.394081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20561,"num_write_kb":446825,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"27'23","reported_seq":57,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.345555+0000","last_change":"2024-09-10T21:24:47.867890+0000","last_active":"2024-09-10T21:47:11.345555+0000","last_peered":"2024-09-10T21:47:11.345555+0000","last_clean":"2024-09-10T21:47:11.345555+0000","last_became_active":"2024-09-10T21:24:47.865016+0000","last_became_peered":"2024-09-10T21:24:47.865016+0000","last_unstale":"2024-09-10T21:47:11.345555+0000","last_undegraded":"2024-09-10T21:47:11.345555+0000","last_fullsized":"2024-09-10T21:47:11.345555+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:53:09.082512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"27'3","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.554054+0000","last_change":"2024-09-10T21:24:46.861400+0000","last_active":"2024-09-10T21:46:48.554054+0000","last_peered":"2024-09-10T21:46:48.554054+0000","last_clean":"2024-09-10T21:46:48.554054+0000","last_became_active":"2024-09-10T21:24:46.861136+0000","last_became_peered":"2024-09-10T21:24:46.861136+0000","last_unstale":"2024-09-10T21:46:48.554054+0000","last_undegraded":"2024-09-10T21:46:48.554054+0000","last_fullsized":"2024-09-10T21:46:48.554054+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:35:31.682448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"27'4156","reported_seq":4184,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.591271+0000","last_change":"2024-09-10T21:24:51.907850+0000","last_active":"2024-09-10T21:46:48.591271+0000","last_peered":"2024-09-10T21:46:48.591271+0000","last_clean":"2024-09-10T21:46:48.591271+0000","last_became_active":"2024-09-10T21:24:51.907062+0000","last_became_peered":"2024-09-10T21:24:51.907062+0000","last_unstale":"2024-09-10T21:46:48.591271+0000","last_undegraded":"2024-09-10T21:46:48.591271+0000","last_fullsized":"2024-09-10T21:46:48.591271+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":4156,"log_dups_size":0,"ondisk_log_size":4156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:32:52.519627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20153,"num_write_kb":457162,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"27'25","reported_seq":58,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.346624+0000","last_change":"2024-09-10T21:24:47.861800+0000","last_active":"2024-09-10T21:47:11.346624+0000","last_peered":"2024-09-10T21:47:11.346624+0000","last_clean":"2024-09-10T21:47:11.346624+0000","last_became_active":"2024-09-10T21:24:47.861498+0000","last_became_peered":"2024-09-10T21:24:47.861498+0000","last_unstale":"2024-09-10T21:47:11.346624+0000","last_undegraded":"2024-09-10T21:47:11.346624+0000","last_fullsized":"2024-09-10T21:47:11.346624+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:07:21.128033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"27'3","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:36.356663+0000","last_change":"2024-09-10T21:24:46.871677+0000","last_active":"2024-09-10T21:46:36.356663+0000","last_peered":"2024-09-10T21:46:36.356663+0000","last_clean":"2024-09-10T21:46:36.356663+0000","last_became_active":"2024-09-10T21:24:46.871367+0000","last_became_peered":"2024-09-10T21:24:46.871367+0000","last_unstale":"2024-09-10T21:46:36.356663+0000","last_undegraded":"2024-09-10T21:46:36.356663+0000","last_fullsized":"2024-09-10T21:46:36.356663+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:40:33.632846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"27'3798","reported_seq":3826,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.674934+0000","last_change":"2024-09-10T21:24:52.068637+0000","last_active":"2024-09-10T21:46:48.674934+0000","last_peered":"2024-09-10T21:46:48.674934+0000","last_clean":"2024-09-10T21:46:48.674934+0000","last_became_active":"2024-09-10T21:24:52.068348+0000","last_became_peered":"2024-09-10T21:24:52.068348+0000","last_unstale":"2024-09-10T21:46:48.674934+0000","last_undegraded":"2024-09-10T21:46:48.674934+0000","last_fullsized":"2024-09-10T21:46:48.674934+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":3798,"log_dups_size":0,"ondisk_log_size":3798,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:43:17.173095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18329,"num_write_kb":408933,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"27'5643","reported_seq":5673,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.715900+0000","last_change":"2024-09-10T21:24:52.067732+0000","last_active":"2024-09-10T21:46:48.715900+0000","last_peered":"2024-09-10T21:46:48.715900+0000","last_clean":"2024-09-10T21:46:48.715900+0000","last_became_active":"2024-09-10T21:24:52.067330+0000","last_became_peered":"2024-09-10T21:24:52.067330+0000","last_unstale":"2024-09-10T21:46:48.715900+0000","last_undegraded":"2024-09-10T21:46:48.715900+0000","last_fullsized":"2024-09-10T21:46:48.715900+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":5643,"log_dups_size":0,"ondisk_log_size":5643,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:03:41.110214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27908,"num_write_kb":566060,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"27'20","reported_seq":53,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.330166+0000","last_change":"2024-09-10T21:24:47.862774+0000","last_active":"2024-09-10T21:47:11.330166+0000","last_peered":"2024-09-10T21:47:11.330166+0000","last_clean":"2024-09-10T21:47:11.330166+0000","last_became_active":"2024-09-10T21:24:47.862556+0000","last_became_peered":"2024-09-10T21:24:47.862556+0000","last_unstale":"2024-09-10T21:47:11.330166+0000","last_undegraded":"2024-09-10T21:47:11.330166+0000","last_fullsized":"2024-09-10T21:47:11.330166+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:17:52.343553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"27'2","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:34.118680+0000","last_change":"2024-09-10T21:24:46.862847+0000","last_active":"2024-09-10T21:46:34.118680+0000","last_peered":"2024-09-10T21:46:34.118680+0000","last_clean":"2024-09-10T21:46:34.118680+0000","last_became_active":"2024-09-10T21:24:46.862540+0000","last_became_peered":"2024-09-10T21:24:46.862540+0000","last_unstale":"2024-09-10T21:46:34.118680+0000","last_undegraded":"2024-09-10T21:46:34.118680+0000","last_fullsized":"2024-09-10T21:46:34.118680+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:52:31.628568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"27'5506","reported_seq":5535,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.736687+0000","last_change":"2024-09-10T21:24:51.901224+0000","last_active":"2024-09-10T21:46:48.736687+0000","last_peered":"2024-09-10T21:46:48.736687+0000","last_clean":"2024-09-10T21:46:48.736687+0000","last_became_active":"2024-09-10T21:24:51.900610+0000","last_became_peered":"2024-09-10T21:24:51.900610+0000","last_unstale":"2024-09-10T21:46:48.736687+0000","last_undegraded":"2024-09-10T21:46:48.736687+0000","last_fullsized":"2024-09-10T21:46:48.736687+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":5506,"log_dups_size":0,"ondisk_log_size":5506,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:52:34.689860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27800,"num_write_kb":594222,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"27'26","reported_seq":60,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.328339+0000","last_change":"2024-09-10T21:24:47.861255+0000","last_active":"2024-09-10T21:47:11.328339+0000","last_peered":"2024-09-10T21:47:11.328339+0000","last_clean":"2024-09-10T21:47:11.328339+0000","last_became_active":"2024-09-10T21:24:47.860930+0000","last_became_peered":"2024-09-10T21:24:47.860930+0000","last_unstale":"2024-09-10T21:47:11.328339+0000","last_undegraded":"2024-09-10T21:47:11.328339+0000","last_fullsized":"2024-09-10T21:47:11.328339+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:37:51.784258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"27'3","reported_seq":39,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.632694+0000","last_change":"2024-09-10T21:24:46.858561+0000","last_active":"2024-09-10T21:46:48.632694+0000","last_peered":"2024-09-10T21:46:48.632694+0000","last_clean":"2024-09-10T21:46:48.632694+0000","last_became_active":"2024-09-10T21:24:46.858265+0000","last_became_peered":"2024-09-10T21:24:46.858265+0000","last_unstale":"2024-09-10T21:46:48.632694+0000","last_undegraded":"2024-09-10T21:46:48.632694+0000","last_fullsized":"2024-09-10T21:46:48.632694+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:49:00.454609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"27'6","reported_seq":41,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.657882+0000","last_change":"2024-09-10T21:24:46.856886+0000","last_active":"2024-09-10T21:46:48.657882+0000","last_peered":"2024-09-10T21:46:48.657882+0000","last_clean":"2024-09-10T21:46:48.657882+0000","last_became_active":"2024-09-10T21:24:46.856506+0000","last_became_peered":"2024-09-10T21:24:46.856506+0000","last_unstale":"2024-09-10T21:46:48.657882+0000","last_undegraded":"2024-09-10T21:46:48.657882+0000","last_fullsized":"2024-09-10T21:46:48.657882+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:56:46.975225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"27'19","reported_seq":53,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.337521+0000","last_change":"2024-09-10T21:24:47.848602+0000","last_active":"2024-09-10T21:47:11.337521+0000","last_peered":"2024-09-10T21:47:11.337521+0000","last_clean":"2024-09-10T21:47:11.337521+0000","last_became_active":"2024-09-10T21:24:47.848160+0000","last_became_peered":"2024-09-10T21:24:47.848160+0000","last_unstale":"2024-09-10T21:47:11.337521+0000","last_undegraded":"2024-09-10T21:47:11.337521+0000","last_fullsized":"2024-09-10T21:47:11.337521+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:32:50.027827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"27'4174","reported_seq":4202,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.456088+0000","last_change":"2024-09-10T21:24:51.903553+0000","last_active":"2024-09-10T21:46:48.456088+0000","last_peered":"2024-09-10T21:46:48.456088+0000","last_clean":"2024-09-10T21:46:48.456088+0000","last_became_active":"2024-09-10T21:24:51.903120+0000","last_became_peered":"2024-09-10T21:24:51.903120+0000","last_unstale":"2024-09-10T21:46:48.456088+0000","last_undegraded":"2024-09-10T21:46:48.456088+0000","last_fullsized":"2024-09-10T21:46:48.456088+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":4174,"log_dups_size":0,"ondisk_log_size":4174,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19938,"num_write_kb":435851,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"27'678","reported_seq":714,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.633070+0000","last_change":"2024-09-10T21:24:46.853550+0000","last_active":"2024-09-10T21:46:48.633070+0000","last_peered":"2024-09-10T21:46:48.633070+0000","last_clean":"2024-09-10T21:46:48.633070+0000","last_became_active":"2024-09-10T21:24:46.852204+0000","last_became_peered":"2024-09-10T21:24:46.852204+0000","last_unstale":"2024-09-10T21:46:48.633070+0000","last_undegraded":"2024-09-10T21:46:48.633070+0000","last_fullsized":"2024-09-10T21:46:48.633070+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:45.816305+0000","last_clean_scrub_stamp":"2024-09-10T21:24:45.816305+0000","objects_scrubbed":0,"log_size":678,"log_dups_size":0,"ondisk_log_size":678,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:06:37.669444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":672,"num_write_kb":4259,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"27'18","reported_seq":52,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.327177+0000","last_change":"2024-09-10T21:24:47.849151+0000","last_active":"2024-09-10T21:47:11.327177+0000","last_peered":"2024-09-10T21:47:11.327177+0000","last_clean":"2024-09-10T21:47:11.327177+0000","last_became_active":"2024-09-10T21:24:47.848851+0000","last_became_peered":"2024-09-10T21:24:47.848851+0000","last_unstale":"2024-09-10T21:47:11.327177+0000","last_undegraded":"2024-09-10T21:47:11.327177+0000","last_fullsized":"2024-09-10T21:47:11.327177+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:39:25.828558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"27'4902","reported_seq":4929,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.681160+0000","last_change":"2024-09-10T21:24:51.907549+0000","last_active":"2024-09-10T21:46:48.681160+0000","last_peered":"2024-09-10T21:46:48.681160+0000","last_clean":"2024-09-10T21:46:48.681160+0000","last_became_active":"2024-09-10T21:24:51.907082+0000","last_became_peered":"2024-09-10T21:24:51.907082+0000","last_unstale":"2024-09-10T21:46:48.681160+0000","last_undegraded":"2024-09-10T21:46:48.681160+0000","last_fullsized":"2024-09-10T21:46:48.681160+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":4902,"log_dups_size":0,"ondisk_log_size":4902,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:44:30.813754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24324,"num_write_kb":547512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"27'24","reported_seq":58,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.351212+0000","last_change":"2024-09-10T21:24:47.857804+0000","last_active":"2024-09-10T21:47:11.351212+0000","last_peered":"2024-09-10T21:47:11.351212+0000","last_clean":"2024-09-10T21:47:11.351212+0000","last_became_active":"2024-09-10T21:24:47.857501+0000","last_became_peered":"2024-09-10T21:24:47.857501+0000","last_unstale":"2024-09-10T21:47:11.351212+0000","last_undegraded":"2024-09-10T21:47:11.351212+0000","last_fullsized":"2024-09-10T21:47:11.351212+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:19:20.807422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"27'25","reported_seq":60,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.345524+0000","last_change":"2024-09-10T21:24:47.867803+0000","last_active":"2024-09-10T21:47:11.345524+0000","last_peered":"2024-09-10T21:47:11.345524+0000","last_clean":"2024-09-10T21:47:11.345524+0000","last_became_active":"2024-09-10T21:24:47.867504+0000","last_became_peered":"2024-09-10T21:24:47.867504+0000","last_unstale":"2024-09-10T21:47:11.345524+0000","last_undegraded":"2024-09-10T21:47:11.345524+0000","last_fullsized":"2024-09-10T21:47:11.345524+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:53:30.420454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"27'26","reported_seq":60,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.334694+0000","last_change":"2024-09-10T21:24:47.868901+0000","last_active":"2024-09-10T21:47:11.334694+0000","last_peered":"2024-09-10T21:47:11.334694+0000","last_clean":"2024-09-10T21:47:11.334694+0000","last_became_active":"2024-09-10T21:24:47.868168+0000","last_became_peered":"2024-09-10T21:24:47.868168+0000","last_unstale":"2024-09-10T21:47:11.334694+0000","last_undegraded":"2024-09-10T21:47:11.334694+0000","last_fullsized":"2024-09-10T21:47:11.334694+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:06:21.461089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"27'16","reported_seq":51,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.331166+0000","last_change":"2024-09-10T21:24:47.864561+0000","last_active":"2024-09-10T21:47:11.331166+0000","last_peered":"2024-09-10T21:47:11.331166+0000","last_clean":"2024-09-10T21:47:11.331166+0000","last_became_active":"2024-09-10T21:24:47.864100+0000","last_became_peered":"2024-09-10T21:24:47.864100+0000","last_unstale":"2024-09-10T21:47:11.331166+0000","last_undegraded":"2024-09-10T21:47:11.331166+0000","last_fullsized":"2024-09-10T21:47:11.331166+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:34:13.053494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"27'14","reported_seq":49,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.340508+0000","last_change":"2024-09-10T21:24:47.864303+0000","last_active":"2024-09-10T21:47:11.340508+0000","last_peered":"2024-09-10T21:47:11.340508+0000","last_clean":"2024-09-10T21:47:11.340508+0000","last_became_active":"2024-09-10T21:24:47.863970+0000","last_became_peered":"2024-09-10T21:24:47.863970+0000","last_unstale":"2024-09-10T21:47:11.340508+0000","last_undegraded":"2024-09-10T21:47:11.340508+0000","last_fullsized":"2024-09-10T21:47:11.340508+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:46:36.918539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"27'18","reported_seq":52,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.329189+0000","last_change":"2024-09-10T21:24:47.860734+0000","last_active":"2024-09-10T21:47:11.329189+0000","last_peered":"2024-09-10T21:47:11.329189+0000","last_clean":"2024-09-10T21:47:11.329189+0000","last_became_active":"2024-09-10T21:24:47.860471+0000","last_became_peered":"2024-09-10T21:24:47.860471+0000","last_unstale":"2024-09-10T21:47:11.329189+0000","last_undegraded":"2024-09-10T21:47:11.329189+0000","last_fullsized":"2024-09-10T21:47:11.329189+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:54:13.200082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"27'27","reported_seq":61,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.358154+0000","last_change":"2024-09-10T21:24:47.872504+0000","last_active":"2024-09-10T21:47:11.358154+0000","last_peered":"2024-09-10T21:47:11.358154+0000","last_clean":"2024-09-10T21:47:11.358154+0000","last_became_active":"2024-09-10T21:24:47.872218+0000","last_became_peered":"2024-09-10T21:24:47.872218+0000","last_unstale":"2024-09-10T21:47:11.358154+0000","last_undegraded":"2024-09-10T21:47:11.358154+0000","last_fullsized":"2024-09-10T21:47:11.358154+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:21:32.400520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"27'14","reported_seq":49,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.310878+0000","last_change":"2024-09-10T21:24:47.862167+0000","last_active":"2024-09-10T21:47:11.310878+0000","last_peered":"2024-09-10T21:47:11.310878+0000","last_clean":"2024-09-10T21:47:11.310878+0000","last_became_active":"2024-09-10T21:24:47.859649+0000","last_became_peered":"2024-09-10T21:24:47.859649+0000","last_unstale":"2024-09-10T21:47:11.310878+0000","last_undegraded":"2024-09-10T21:47:11.310878+0000","last_fullsized":"2024-09-10T21:47:11.310878+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:42:59.204284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"27'18","reported_seq":39,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.329156+0000","last_change":"2024-09-10T21:25:28.508979+0000","last_active":"2024-09-10T21:47:11.329156+0000","last_peered":"2024-09-10T21:47:11.329156+0000","last_clean":"2024-09-10T21:47:11.329156+0000","last_became_active":"2024-09-10T21:25:28.508684+0000","last_became_peered":"2024-09-10T21:25:28.508684+0000","last_unstale":"2024-09-10T21:47:11.329156+0000","last_undegraded":"2024-09-10T21:47:11.329156+0000","last_fullsized":"2024-09-10T21:47:11.329156+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:20:59.362941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.29","version":"27'23","reported_seq":43,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.335919+0000","last_change":"2024-09-10T21:25:28.512634+0000","last_active":"2024-09-10T21:47:11.335919+0000","last_peered":"2024-09-10T21:47:11.335919+0000","last_clean":"2024-09-10T21:47:11.335919+0000","last_became_active":"2024-09-10T21:25:28.512230+0000","last_became_peered":"2024-09-10T21:25:28.512230+0000","last_unstale":"2024-09-10T21:47:11.335919+0000","last_undegraded":"2024-09-10T21:47:11.335919+0000","last_fullsized":"2024-09-10T21:47:11.335919+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:17:52.343553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2a","version":"27'22","reported_seq":56,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.348060+0000","last_change":"2024-09-10T21:24:47.872848+0000","last_active":"2024-09-10T21:47:11.348060+0000","last_peered":"2024-09-10T21:47:11.348060+0000","last_clean":"2024-09-10T21:47:11.348060+0000","last_became_active":"2024-09-10T21:24:47.868671+0000","last_became_peered":"2024-09-10T21:24:47.868671+0000","last_unstale":"2024-09-10T21:47:11.348060+0000","last_undegraded":"2024-09-10T21:47:11.348060+0000","last_fullsized":"2024-09-10T21:47:11.348060+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:33:59.291647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"27'28","reported_seq":62,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.339893+0000","last_change":"2024-09-10T21:24:47.857464+0000","last_active":"2024-09-10T21:47:11.339893+0000","last_peered":"2024-09-10T21:47:11.339893+0000","last_clean":"2024-09-10T21:47:11.339893+0000","last_became_active":"2024-09-10T21:24:47.857116+0000","last_became_peered":"2024-09-10T21:24:47.857116+0000","last_unstale":"2024-09-10T21:47:11.339893+0000","last_undegraded":"2024-09-10T21:47:11.339893+0000","last_fullsized":"2024-09-10T21:47:11.339893+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:56:43.269026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"27'19","reported_seq":53,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.350268+0000","last_change":"2024-09-10T21:24:47.871619+0000","last_active":"2024-09-10T21:47:11.350268+0000","last_peered":"2024-09-10T21:47:11.350268+0000","last_clean":"2024-09-10T21:47:11.350268+0000","last_became_active":"2024-09-10T21:24:47.871350+0000","last_became_peered":"2024-09-10T21:24:47.871350+0000","last_unstale":"2024-09-10T21:47:11.350268+0000","last_undegraded":"2024-09-10T21:47:11.350268+0000","last_fullsized":"2024-09-10T21:47:11.350268+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:44:41.305116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"27'20","reported_seq":54,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.332528+0000","last_change":"2024-09-10T21:24:47.861352+0000","last_active":"2024-09-10T21:47:11.332528+0000","last_peered":"2024-09-10T21:47:11.332528+0000","last_clean":"2024-09-10T21:47:11.332528+0000","last_became_active":"2024-09-10T21:24:47.861007+0000","last_became_peered":"2024-09-10T21:24:47.861007+0000","last_unstale":"2024-09-10T21:47:11.332528+0000","last_undegraded":"2024-09-10T21:47:11.332528+0000","last_fullsized":"2024-09-10T21:47:11.332528+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:25.273458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"27'27","reported_seq":61,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.345633+0000","last_change":"2024-09-10T21:24:47.864205+0000","last_active":"2024-09-10T21:47:11.345633+0000","last_peered":"2024-09-10T21:47:11.345633+0000","last_clean":"2024-09-10T21:47:11.345633+0000","last_became_active":"2024-09-10T21:24:47.863932+0000","last_became_peered":"2024-09-10T21:24:47.863932+0000","last_unstale":"2024-09-10T21:47:11.345633+0000","last_undegraded":"2024-09-10T21:47:11.345633+0000","last_fullsized":"2024-09-10T21:47:11.345633+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:53:17.669629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"27'21","reported_seq":54,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.349081+0000","last_change":"2024-09-10T21:24:47.866225+0000","last_active":"2024-09-10T21:47:11.349081+0000","last_peered":"2024-09-10T21:47:11.349081+0000","last_clean":"2024-09-10T21:47:11.349081+0000","last_became_active":"2024-09-10T21:24:47.865587+0000","last_became_peered":"2024-09-10T21:24:47.865587+0000","last_unstale":"2024-09-10T21:47:11.349081+0000","last_undegraded":"2024-09-10T21:47:11.349081+0000","last_fullsized":"2024-09-10T21:47:11.349081+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:49:26.113068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"27'22","reported_seq":55,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.342691+0000","last_change":"2024-09-10T21:24:47.872669+0000","last_active":"2024-09-10T21:47:11.342691+0000","last_peered":"2024-09-10T21:47:11.342691+0000","last_clean":"2024-09-10T21:47:11.342691+0000","last_became_active":"2024-09-10T21:24:47.865837+0000","last_became_peered":"2024-09-10T21:24:47.865837+0000","last_unstale":"2024-09-10T21:47:11.342691+0000","last_undegraded":"2024-09-10T21:47:11.342691+0000","last_fullsized":"2024-09-10T21:47:11.342691+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:53:09.082512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"27'21","reported_seq":54,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.318871+0000","last_change":"2024-09-10T21:24:47.846567+0000","last_active":"2024-09-10T21:47:11.318871+0000","last_peered":"2024-09-10T21:47:11.318871+0000","last_clean":"2024-09-10T21:47:11.318871+0000","last_became_active":"2024-09-10T21:24:47.846283+0000","last_became_peered":"2024-09-10T21:24:47.846283+0000","last_unstale":"2024-09-10T21:47:11.318871+0000","last_undegraded":"2024-09-10T21:47:11.318871+0000","last_fullsized":"2024-09-10T21:47:11.318871+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:42:15.320518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"27'23","reported_seq":57,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.319292+0000","last_change":"2024-09-10T21:24:47.844888+0000","last_active":"2024-09-10T21:47:11.319292+0000","last_peered":"2024-09-10T21:47:11.319292+0000","last_clean":"2024-09-10T21:47:11.319292+0000","last_became_active":"2024-09-10T21:24:47.844517+0000","last_became_peered":"2024-09-10T21:24:47.844517+0000","last_unstale":"2024-09-10T21:47:11.319292+0000","last_undegraded":"2024-09-10T21:47:11.319292+0000","last_fullsized":"2024-09-10T21:47:11.319292+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:00:14.988282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"27'25","reported_seq":59,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.344844+0000","last_change":"2024-09-10T21:24:47.862746+0000","last_active":"2024-09-10T21:47:11.344844+0000","last_peered":"2024-09-10T21:47:11.344844+0000","last_clean":"2024-09-10T21:47:11.344844+0000","last_became_active":"2024-09-10T21:24:47.862185+0000","last_became_peered":"2024-09-10T21:24:47.862185+0000","last_unstale":"2024-09-10T21:47:11.344844+0000","last_undegraded":"2024-09-10T21:47:11.344844+0000","last_fullsized":"2024-09-10T21:47:11.344844+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:06:21.461089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"27'17","reported_seq":51,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.349979+0000","last_change":"2024-09-10T21:24:47.867727+0000","last_active":"2024-09-10T21:47:11.349979+0000","last_peered":"2024-09-10T21:47:11.349979+0000","last_clean":"2024-09-10T21:47:11.349979+0000","last_became_active":"2024-09-10T21:24:47.861045+0000","last_became_peered":"2024-09-10T21:24:47.861045+0000","last_unstale":"2024-09-10T21:47:11.349979+0000","last_undegraded":"2024-09-10T21:47:11.349979+0000","last_fullsized":"2024-09-10T21:47:11.349979+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:08:30.065968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"27'17","reported_seq":50,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.347800+0000","last_change":"2024-09-10T21:24:47.867410+0000","last_active":"2024-09-10T21:47:11.347800+0000","last_peered":"2024-09-10T21:47:11.347800+0000","last_clean":"2024-09-10T21:47:11.347800+0000","last_became_active":"2024-09-10T21:24:47.867113+0000","last_became_peered":"2024-09-10T21:24:47.867113+0000","last_unstale":"2024-09-10T21:47:11.347800+0000","last_undegraded":"2024-09-10T21:47:11.347800+0000","last_fullsized":"2024-09-10T21:47:11.347800+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:49:53.502472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"27'28","reported_seq":61,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.358458+0000","last_change":"2024-09-10T21:24:47.867473+0000","last_active":"2024-09-10T21:47:11.358458+0000","last_peered":"2024-09-10T21:47:11.358458+0000","last_clean":"2024-09-10T21:47:11.358458+0000","last_became_active":"2024-09-10T21:24:47.867126+0000","last_became_peered":"2024-09-10T21:24:47.867126+0000","last_unstale":"2024-09-10T21:47:11.358458+0000","last_undegraded":"2024-09-10T21:47:11.358458+0000","last_fullsized":"2024-09-10T21:47:11.358458+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:54:32.307507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"27'26","reported_seq":60,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.353536+0000","last_change":"2024-09-10T21:24:47.865450+0000","last_active":"2024-09-10T21:47:11.353536+0000","last_peered":"2024-09-10T21:47:11.353536+0000","last_clean":"2024-09-10T21:47:11.353536+0000","last_became_active":"2024-09-10T21:24:47.865093+0000","last_became_peered":"2024-09-10T21:24:47.865093+0000","last_unstale":"2024-09-10T21:47:11.353536+0000","last_undegraded":"2024-09-10T21:47:11.353536+0000","last_fullsized":"2024-09-10T21:47:11.353536+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:12:18.854783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"27'15","reported_seq":49,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.344819+0000","last_change":"2024-09-10T21:24:47.868093+0000","last_active":"2024-09-10T21:47:11.344819+0000","last_peered":"2024-09-10T21:47:11.344819+0000","last_clean":"2024-09-10T21:47:11.344819+0000","last_became_active":"2024-09-10T21:24:47.867782+0000","last_became_peered":"2024-09-10T21:24:47.867782+0000","last_unstale":"2024-09-10T21:47:11.344819+0000","last_undegraded":"2024-09-10T21:47:11.344819+0000","last_fullsized":"2024-09-10T21:47:11.344819+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:34:13.053494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"27'13","reported_seq":47,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.336048+0000","last_change":"2024-09-10T21:24:47.862204+0000","last_active":"2024-09-10T21:47:11.336048+0000","last_peered":"2024-09-10T21:47:11.336048+0000","last_clean":"2024-09-10T21:47:11.336048+0000","last_became_active":"2024-09-10T21:24:47.861434+0000","last_became_peered":"2024-09-10T21:24:47.861434+0000","last_unstale":"2024-09-10T21:47:11.336048+0000","last_undegraded":"2024-09-10T21:47:11.336048+0000","last_fullsized":"2024-09-10T21:47:11.336048+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:50:03.245285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"27'14","reported_seq":47,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.347974+0000","last_change":"2024-09-10T21:24:47.869248+0000","last_active":"2024-09-10T21:47:11.347974+0000","last_peered":"2024-09-10T21:47:11.347974+0000","last_clean":"2024-09-10T21:47:11.347974+0000","last_became_active":"2024-09-10T21:24:47.868503+0000","last_became_peered":"2024-09-10T21:24:47.868503+0000","last_unstale":"2024-09-10T21:47:11.347974+0000","last_undegraded":"2024-09-10T21:47:11.347974+0000","last_fullsized":"2024-09-10T21:47:11.347974+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:51:31.417845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"27'15","reported_seq":49,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.331490+0000","last_change":"2024-09-10T21:24:47.864802+0000","last_active":"2024-09-10T21:47:11.331490+0000","last_peered":"2024-09-10T21:47:11.331490+0000","last_clean":"2024-09-10T21:47:11.331490+0000","last_became_active":"2024-09-10T21:24:47.863976+0000","last_became_peered":"2024-09-10T21:24:47.863976+0000","last_unstale":"2024-09-10T21:47:11.331490+0000","last_undegraded":"2024-09-10T21:47:11.331490+0000","last_fullsized":"2024-09-10T21:47:11.331490+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:55:08.544075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"27'13","reported_seq":48,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.347483+0000","last_change":"2024-09-10T21:24:47.873364+0000","last_active":"2024-09-10T21:47:11.347483+0000","last_peered":"2024-09-10T21:47:11.347483+0000","last_clean":"2024-09-10T21:47:11.347483+0000","last_became_active":"2024-09-10T21:24:47.873069+0000","last_became_peered":"2024-09-10T21:24:47.873069+0000","last_unstale":"2024-09-10T21:47:11.347483+0000","last_undegraded":"2024-09-10T21:47:11.347483+0000","last_fullsized":"2024-09-10T21:47:11.347483+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:38:21.368499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"27'20","reported_seq":55,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.322849+0000","last_change":"2024-09-10T21:24:47.856936+0000","last_active":"2024-09-10T21:47:11.322849+0000","last_peered":"2024-09-10T21:47:11.322849+0000","last_clean":"2024-09-10T21:47:11.322849+0000","last_became_active":"2024-09-10T21:24:47.854713+0000","last_became_peered":"2024-09-10T21:24:47.854713+0000","last_unstale":"2024-09-10T21:47:11.322849+0000","last_undegraded":"2024-09-10T21:47:11.322849+0000","last_fullsized":"2024-09-10T21:47:11.322849+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:46:36.918539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"27'10","reported_seq":44,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.320206+0000","last_change":"2024-09-10T21:24:47.846492+0000","last_active":"2024-09-10T21:47:11.320206+0000","last_peered":"2024-09-10T21:47:11.320206+0000","last_clean":"2024-09-10T21:47:11.320206+0000","last_became_active":"2024-09-10T21:24:47.846227+0000","last_became_peered":"2024-09-10T21:24:47.846227+0000","last_unstale":"2024-09-10T21:47:11.320206+0000","last_undegraded":"2024-09-10T21:47:11.320206+0000","last_fullsized":"2024-09-10T21:47:11.320206+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:13:58.432819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"27'15","reported_seq":50,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.347364+0000","last_change":"2024-09-10T21:24:47.871469+0000","last_active":"2024-09-10T21:47:11.347364+0000","last_peered":"2024-09-10T21:47:11.347364+0000","last_clean":"2024-09-10T21:47:11.347364+0000","last_became_active":"2024-09-10T21:24:47.871177+0000","last_became_peered":"2024-09-10T21:24:47.871177+0000","last_unstale":"2024-09-10T21:47:11.347364+0000","last_undegraded":"2024-09-10T21:47:11.347364+0000","last_fullsized":"2024-09-10T21:47:11.347364+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:19:24.226916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":723657,"num_write_kb":15994026,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":148509,"ondisk_log_size":148509,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1239,"ondisk_log_size":1239,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":14991163,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":48,"num_read_kb":119,"num_write":3049,"num_write_kb":15770,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":18464768,"data_stored":29982326,"data_compressed":2564296,"data_compressed_allocated":3506176,"data_compressed_original":15089664,"omap_allocated":0,"internal_metadata":0},"log_size":2759,"ondisk_log_size":2759,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":66932,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509715,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":739600,"kb_used_data":7424,"kb_used_omap":1,"kb_used_meta":732158,"kb_avail":93632240,"statfs":{"total":96636764160,"available":95879413760,"internally_reserved":0,"allocated":7602176,"data_stored":11412495,"data_compressed":806609,"data_compressed_allocated":1097728,"data_compressed_original":5083136,"omap_allocated":1590,"internal_metadata":749730250},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509716,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":807000,"kb_used_data":2120,"kb_used_omap":1,"kb_used_meta":804862,"kb_avail":93564840,"statfs":{"total":96636764160,"available":95810396160,"internally_reserved":0,"allocated":2170880,"data_stored":4311586,"data_compressed":503209,"data_compressed_allocated":692224,"data_compressed_original":3006464,"omap_allocated":1594,"internal_metadata":824179142},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509715,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":573484,"kb_used_data":348,"kb_used_omap":1,"kb_used_meta":573118,"kb_avail":93798356,"statfs":{"total":96636764160,"available":96049516544,"internally_reserved":0,"allocated":356352,"data_stored":207064,"data_compressed":5796,"data_compressed_allocated":8192,"data_compressed_original":28672,"omap_allocated":1595,"internal_metadata":586873285},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509713,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1104044,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":1103742,"kb_avail":93267796,"statfs":{"total":96636764160,"available":95506223104,"internally_reserved":0,"allocated":290816,"data_stored":116770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":1130232261},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542420,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":844396,"kb_used_data":3740,"kb_used_omap":1,"kb_used_meta":840638,"kb_avail":93527444,"statfs":{"total":96636764160,"available":95772102656,"internally_reserved":0,"allocated":3829760,"data_stored":4311040,"data_compressed":154551,"data_compressed_allocated":245760,"data_compressed_original":905216,"omap_allocated":1595,"internal_metadata":860813765},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542415,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1062920,"kb_used_data":4792,"kb_used_omap":1,"kb_used_meta":1058110,"kb_avail":93308920,"statfs":{"total":96636764160,"available":95548334080,"internally_reserved":0,"allocated":4907008,"data_stored":8596344,"data_compressed":841246,"data_compressed_allocated":1130496,"data_compressed_original":4997120,"omap_allocated":1593,"internal_metadata":1083505095},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542421,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":711220,"kb_used_data":1380,"kb_used_omap":1,"kb_used_meta":709822,"kb_avail":93660620,"statfs":{"total":96636764160,"available":95908474880,"internally_reserved":0,"allocated":1413120,"data_stored":3023943,"data_compressed":319817,"data_compressed_allocated":421888,"data_compressed_original":2215936,"omap_allocated":1595,"internal_metadata":726858181},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575123,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":626792,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":626494,"kb_avail":93745048,"statfs":{"total":96636764160,"available":95994929152,"internally_reserved":0,"allocated":286720,"data_stored":116700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":641530309},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33466,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33466,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":1064960,"data_stored":2316965,"data_compressed":286351,"data_compressed_allocated":376832,"data_compressed_original":1642496,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":90454,"data_compressed":5796,"data_compressed_allocated":8192,"data_compressed_original":28672,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4624384,"data_stored":8479734,"data_compressed":841246,"data_compressed_allocated":1130496,"data_compressed_original":4997120,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":3547136,"data_stored":4194430,"data_compressed":154551,"data_compressed_allocated":245760,"data_compressed_original":905216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":1888256,"data_stored":4194976,"data_compressed":503209,"data_compressed_allocated":692224,"data_compressed_original":3006464,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":7254016,"data_stored":10705517,"data_compressed":773143,"data_compressed_allocated":1052672,"data_compressed_original":4509696,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T21:47:25.599 INFO:tasks.ceph:pgid 2.1f last_scrub_stamp 2024-09-10T21:24:45.816305+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=24, tm_sec=45, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=47, tm_sec=19, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T21:47:25.599 INFO:tasks.ceph:pgid 3.1e last_scrub_stamp 2024-09-10T21:24:46.824885+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=24, tm_sec=46, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=47, tm_sec=19, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T21:47:25.599 INFO:tasks.ceph:pgid 4.19 last_scrub_stamp 2024-09-10T21:24:50.859019+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=24, tm_sec=50, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=47, tm_sec=19, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T21:47:25.599 INFO:tasks.ceph:pgid 2.1e last_scrub_stamp 2024-09-10T21:24:45.816305+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=24, tm_sec=45, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=47, tm_sec=19, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T21:47:25.600 INFO:tasks.ceph:pgid 3.1f last_scrub_stamp 2024-09-10T21:24:46.824885+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=24, tm_sec=46, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=47, tm_sec=19, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T21:47:25.600 INFO:tasks.ceph:pgid 4.18 last_scrub_stamp 2024-09-10T21:24:50.859019+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=24, tm_sec=50, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=47, tm_sec=19, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T21:47:25.600 INFO:tasks.ceph:pgid 2.1d last_scrub_stamp 2024-09-10T21:24:45.816305+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=24, tm_sec=45, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=47, tm_sec=19, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T21:47:25.600 INFO:tasks.ceph:pgid 3.1c last_scrub_stamp 2024-09-10T21:24:46.824885+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=24, tm_sec=46, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=47, tm_sec=19, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T21:47:25.600 INFO:tasks.ceph:pgid 4.1b last_scrub_stamp 2024-09-10T21:24:50.859019+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=24, tm_sec=50, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=47, tm_sec=19, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T21:47:25.600 INFO:tasks.ceph:pgid 2.1c last_scrub_stamp 2024-09-10T21:24:45.816305+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=24, tm_sec=45, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=47, tm_sec=19, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T21:47:25.600 INFO:tasks.ceph:pgid 3.1d last_scrub_stamp 2024-09-10T21:24:46.824885+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=24, tm_sec=46, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=47, tm_sec=19, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T21:47:25.600 INFO:tasks.ceph:pgid 4.1a last_scrub_stamp 2024-09-10T21:24:50.859019+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=24, tm_sec=50, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=47, tm_sec=19, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T21:47:25.600 INFO:tasks.ceph:pgid 3.a last_scrub_stamp 2024-09-10T21:24:46.824885+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=24, tm_sec=46, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=47, tm_sec=19, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T21:47:25.601 INFO:tasks.ceph:pgid 2.b last_scrub_stamp 2024-09-10T21:24:45.816305+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=24, tm_sec=45, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=47, tm_sec=19, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T21:47:25.601 INFO:tasks.ceph:pgid 4.d last_scrub_stamp 2024-09-10T21:24:50.859019+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=24, tm_sec=50, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=47, tm_sec=19, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T21:47:25.601 INFO:tasks.ceph:pgid 3.b last_scrub_stamp 2024-09-10T21:24:46.824885+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=24, tm_sec=46, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=47, tm_sec=19, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T21:47:25.601 INFO:tasks.ceph:pgid 2.a last_scrub_stamp 2024-09-10T21:24:45.816305+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=24, tm_sec=45, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=47, tm_sec=19, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T21:47:25.601 INFO:tasks.ceph:pgid 4.c last_scrub_stamp 2024-09-10T21:24:50.859019+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=24, tm_sec=50, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=47, tm_sec=19, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T21:47:25.601 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2024-09-10T21:24:46.824885+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=24, tm_sec=46, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=47, tm_sec=19, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T21:47:25.601 INFO:tasks.ceph:pgid 2.9 last_scrub_stamp 2024-09-10T21:24:45.816305+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=24, tm_sec=45, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=47, tm_sec=19, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T21:47:25.601 INFO:tasks.ceph:pgid 4.f last_scrub_stamp 2024-09-10T21:24:50.859019+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=24, tm_sec=50, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=47, tm_sec=19, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T21:47:25.601 INFO:tasks.ceph:pgid 3.9 last_scrub_stamp 2024-09-10T21:24:46.824885+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=24, tm_sec=46, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=47, tm_sec=19, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T21:47:25.601 INFO:tasks.ceph:pgid 2.8 last_scrub_stamp 2024-09-10T21:24:45.816305+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=24, tm_sec=45, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=47, tm_sec=19, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T21:47:25.602 INFO:tasks.ceph:pgid 4.e last_scrub_stamp 2024-09-10T21:24:50.859019+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=24, tm_sec=50, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=47, tm_sec=19, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T21:47:25.602 INFO:tasks.ceph:pgid 3.6 last_scrub_stamp 2024-09-10T21:24:46.824885+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=24, tm_sec=46, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=47, tm_sec=19, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T21:47:25.602 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2024-09-10T21:24:45.816305+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=24, tm_sec=45, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=47, tm_sec=19, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T21:47:25.602 INFO:tasks.ceph:pgid 4.1 last_scrub_stamp 2024-09-10T21:24:50.859019+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=24, tm_sec=50, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=47, tm_sec=19, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T21:47:25.602 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2024-09-10T21:24:46.824885+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=24, tm_sec=46, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=47, tm_sec=19, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T21:47:25.602 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2024-09-10T21:24:45.816305+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=24, tm_sec=45, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=47, tm_sec=19, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T21:47:25.602 INFO:tasks.ceph:pgid 4.0 last_scrub_stamp 2024-09-10T21:24:50.859019+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=24, tm_sec=50, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=47, tm_sec=19, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T21:47:25.602 INFO:tasks.ceph:pgid 3.4 last_scrub_stamp 2024-09-10T21:24:46.824885+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=24, tm_sec=46, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=47, tm_sec=19, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T21:47:25.602 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-09-10T21:24:45.816305+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=24, tm_sec=45, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=47, tm_sec=19, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T21:47:25.602 INFO:tasks.ceph:pgid 4.3 last_scrub_stamp 2024-09-10T21:24:50.859019+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=24, tm_sec=50, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=47, tm_sec=19, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T21:47:25.603 INFO:tasks.ceph:pgid 3.5 last_scrub_stamp 2024-09-10T21:24:46.824885+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=24, tm_sec=46, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=47, tm_sec=19, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T21:47:25.603 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2024-09-10T21:24:45.816305+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=24, tm_sec=45, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=47, tm_sec=19, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T21:47:25.603 INFO:tasks.ceph:pgid 4.2 last_scrub_stamp 2024-09-10T21:24:50.859019+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=24, tm_sec=50, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=47, tm_sec=19, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T21:47:25.603 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2024-09-10T21:24:46.824885+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=24, tm_sec=46, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=47, tm_sec=19, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T21:47:25.603 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2024-09-10T21:24:45.816305+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=24, tm_sec=45, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=47, tm_sec=19, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T21:47:25.603 INFO:tasks.ceph:pgid 4.4 last_scrub_stamp 2024-09-10T21:24:50.859019+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=24, tm_sec=50, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=47, tm_sec=19, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T21:47:25.603 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2024-09-10T21:24:46.824885+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=24, tm_sec=46, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=47, tm_sec=19, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T21:47:25.603 INFO:tasks.ceph:pgid 4.7 last_scrub_stamp 2024-09-10T21:24:50.859019+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=24, tm_sec=50, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=47, tm_sec=19, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T21:47:25.603 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2024-09-10T21:24:46.824885+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=24, tm_sec=46, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=47, tm_sec=19, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T21:47:25.604 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-09-10T21:24:45.816305+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=24, tm_sec=45, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=47, tm_sec=19, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T21:47:25.604 INFO:tasks.ceph:pgid 4.6 last_scrub_stamp 2024-09-10T21:24:50.859019+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=24, tm_sec=50, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=47, tm_sec=19, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T21:47:25.604 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2024-09-10T21:24:46.824885+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=24, tm_sec=46, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=47, tm_sec=19, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T21:47:25.604 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2024-09-10T21:24:45.816305+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=24, tm_sec=45, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=47, tm_sec=19, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T21:47:25.604 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-09-10T21:24:35.877801+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=24, tm_sec=35, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=47, tm_sec=19, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T21:47:25.604 INFO:tasks.ceph:pgid 4.5 last_scrub_stamp 2024-09-10T21:24:50.859019+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=24, tm_sec=50, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=47, tm_sec=19, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T21:47:25.604 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2024-09-10T21:24:46.824885+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=24, tm_sec=46, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=47, tm_sec=19, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T21:47:25.604 INFO:tasks.ceph:pgid 2.c last_scrub_stamp 2024-09-10T21:24:45.816305+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=24, tm_sec=45, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=47, tm_sec=19, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T21:47:25.604 INFO:tasks.ceph:pgid 4.a last_scrub_stamp 2024-09-10T21:24:50.859019+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=24, tm_sec=50, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=47, tm_sec=19, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T21:47:25.605 INFO:tasks.ceph:pgid 3.c last_scrub_stamp 2024-09-10T21:24:46.824885+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=24, tm_sec=46, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=47, tm_sec=19, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T21:47:25.605 INFO:tasks.ceph:pgid 2.d last_scrub_stamp 2024-09-10T21:24:45.816305+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=24, tm_sec=45, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=47, tm_sec=19, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T21:47:25.605 INFO:tasks.ceph:pgid 4.b last_scrub_stamp 2024-09-10T21:24:50.859019+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=24, tm_sec=50, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=47, tm_sec=19, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T21:47:25.605 INFO:tasks.ceph:pgid 3.f last_scrub_stamp 2024-09-10T21:24:46.824885+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=24, tm_sec=46, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=47, tm_sec=19, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T21:47:25.605 INFO:tasks.ceph:pgid 2.e last_scrub_stamp 2024-09-10T21:24:45.816305+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=24, tm_sec=45, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=47, tm_sec=19, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T21:47:25.605 INFO:tasks.ceph:pgid 4.8 last_scrub_stamp 2024-09-10T21:24:50.859019+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=24, tm_sec=50, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=47, tm_sec=19, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T21:47:25.605 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2024-09-10T21:24:46.824885+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=24, tm_sec=46, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=47, tm_sec=19, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T21:47:25.605 INFO:tasks.ceph:pgid 2.f last_scrub_stamp 2024-09-10T21:24:45.816305+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=24, tm_sec=45, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=47, tm_sec=19, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T21:47:25.605 INFO:tasks.ceph:pgid 4.9 last_scrub_stamp 2024-09-10T21:24:50.859019+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=24, tm_sec=50, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=47, tm_sec=19, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T21:47:25.605 INFO:tasks.ceph:pgid 3.11 last_scrub_stamp 2024-09-10T21:24:46.824885+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=24, tm_sec=46, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=47, tm_sec=19, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T21:47:25.606 INFO:tasks.ceph:pgid 2.10 last_scrub_stamp 2024-09-10T21:24:45.816305+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=24, tm_sec=45, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=47, tm_sec=19, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T21:47:25.606 INFO:tasks.ceph:pgid 4.16 last_scrub_stamp 2024-09-10T21:24:50.859019+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=24, tm_sec=50, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=47, tm_sec=19, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T21:47:25.606 INFO:tasks.ceph:pgid 3.10 last_scrub_stamp 2024-09-10T21:24:46.824885+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=24, tm_sec=46, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=47, tm_sec=19, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T21:47:25.606 INFO:tasks.ceph:pgid 2.11 last_scrub_stamp 2024-09-10T21:24:45.816305+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=24, tm_sec=45, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=47, tm_sec=19, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T21:47:25.606 INFO:tasks.ceph:pgid 4.17 last_scrub_stamp 2024-09-10T21:24:50.859019+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=24, tm_sec=50, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=47, tm_sec=19, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T21:47:25.606 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2024-09-10T21:24:46.824885+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=24, tm_sec=46, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=47, tm_sec=19, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T21:47:25.606 INFO:tasks.ceph:pgid 2.12 last_scrub_stamp 2024-09-10T21:24:45.816305+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=24, tm_sec=45, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=47, tm_sec=19, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T21:47:25.606 INFO:tasks.ceph:pgid 4.14 last_scrub_stamp 2024-09-10T21:24:50.859019+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=24, tm_sec=50, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=47, tm_sec=19, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T21:47:25.606 INFO:tasks.ceph:pgid 3.12 last_scrub_stamp 2024-09-10T21:24:46.824885+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=24, tm_sec=46, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=47, tm_sec=19, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T21:47:25.606 INFO:tasks.ceph:pgid 2.13 last_scrub_stamp 2024-09-10T21:24:45.816305+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=24, tm_sec=45, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=47, tm_sec=19, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T21:47:25.607 INFO:tasks.ceph:pgid 4.15 last_scrub_stamp 2024-09-10T21:24:50.859019+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=24, tm_sec=50, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=47, tm_sec=19, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T21:47:25.607 INFO:tasks.ceph:pgid 3.15 last_scrub_stamp 2024-09-10T21:24:46.824885+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=24, tm_sec=46, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=47, tm_sec=19, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T21:47:25.607 INFO:tasks.ceph:pgid 2.14 last_scrub_stamp 2024-09-10T21:24:45.816305+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=24, tm_sec=45, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=47, tm_sec=19, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T21:47:25.607 INFO:tasks.ceph:pgid 4.12 last_scrub_stamp 2024-09-10T21:24:50.859019+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=24, tm_sec=50, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=47, tm_sec=19, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T21:47:25.607 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2024-09-10T21:24:46.824885+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=24, tm_sec=46, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=47, tm_sec=19, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T21:47:25.607 INFO:tasks.ceph:pgid 2.15 last_scrub_stamp 2024-09-10T21:24:45.816305+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=24, tm_sec=45, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=47, tm_sec=19, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T21:47:25.607 INFO:tasks.ceph:pgid 4.13 last_scrub_stamp 2024-09-10T21:24:50.859019+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=24, tm_sec=50, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=47, tm_sec=19, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T21:47:25.607 INFO:tasks.ceph:pgid 3.17 last_scrub_stamp 2024-09-10T21:24:46.824885+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=24, tm_sec=46, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=47, tm_sec=19, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T21:47:25.607 INFO:tasks.ceph:pgid 2.16 last_scrub_stamp 2024-09-10T21:24:45.816305+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=24, tm_sec=45, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=47, tm_sec=19, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T21:47:25.607 INFO:tasks.ceph:pgid 4.10 last_scrub_stamp 2024-09-10T21:24:50.859019+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=24, tm_sec=50, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=47, tm_sec=19, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T21:47:25.608 INFO:tasks.ceph:pgid 3.16 last_scrub_stamp 2024-09-10T21:24:46.824885+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=24, tm_sec=46, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=47, tm_sec=19, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T21:47:25.608 INFO:tasks.ceph:pgid 2.17 last_scrub_stamp 2024-09-10T21:24:45.816305+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=24, tm_sec=45, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=47, tm_sec=19, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T21:47:25.608 INFO:tasks.ceph:pgid 4.11 last_scrub_stamp 2024-09-10T21:24:50.859019+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=24, tm_sec=50, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=47, tm_sec=19, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T21:47:25.608 INFO:tasks.ceph:pgid 4.1e last_scrub_stamp 2024-09-10T21:24:50.859019+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=24, tm_sec=50, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=47, tm_sec=19, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T21:47:25.608 INFO:tasks.ceph:pgid 3.19 last_scrub_stamp 2024-09-10T21:24:46.824885+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=24, tm_sec=46, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=47, tm_sec=19, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T21:47:25.608 INFO:tasks.ceph:pgid 2.18 last_scrub_stamp 2024-09-10T21:24:45.816305+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=24, tm_sec=45, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=47, tm_sec=19, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T21:47:25.608 INFO:tasks.ceph:pgid 4.1f last_scrub_stamp 2024-09-10T21:24:50.859019+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=24, tm_sec=50, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=47, tm_sec=19, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T21:47:25.608 INFO:tasks.ceph:pgid 3.18 last_scrub_stamp 2024-09-10T21:24:46.824885+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=24, tm_sec=46, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=47, tm_sec=19, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T21:47:25.608 INFO:tasks.ceph:pgid 2.19 last_scrub_stamp 2024-09-10T21:24:45.816305+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=24, tm_sec=45, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=47, tm_sec=19, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T21:47:25.608 INFO:tasks.ceph:pgid 2.1a last_scrub_stamp 2024-09-10T21:24:45.816305+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=24, tm_sec=45, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=47, tm_sec=19, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T21:47:25.609 INFO:tasks.ceph:pgid 3.1b last_scrub_stamp 2024-09-10T21:24:46.824885+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=24, tm_sec=46, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=47, tm_sec=19, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T21:47:25.609 INFO:tasks.ceph:pgid 4.1c last_scrub_stamp 2024-09-10T21:24:50.859019+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=24, tm_sec=50, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=47, tm_sec=19, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T21:47:25.609 INFO:tasks.ceph:pgid 2.1b last_scrub_stamp 2024-09-10T21:24:45.816305+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=24, tm_sec=45, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=47, tm_sec=19, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T21:47:25.609 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2024-09-10T21:24:46.824885+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=24, tm_sec=46, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=47, tm_sec=19, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T21:47:25.609 INFO:tasks.ceph:pgid 4.1d last_scrub_stamp 2024-09-10T21:24:50.859019+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=24, tm_sec=50, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=47, tm_sec=19, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T21:47:25.609 INFO:tasks.ceph:pgid 3.20 last_scrub_stamp 2024-09-10T21:24:46.824885+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=24, tm_sec=46, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=47, tm_sec=19, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T21:47:25.609 INFO:tasks.ceph:pgid 3.21 last_scrub_stamp 2024-09-10T21:24:46.824885+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=24, tm_sec=46, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=47, tm_sec=19, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T21:47:25.609 INFO:tasks.ceph:pgid 3.22 last_scrub_stamp 2024-09-10T21:24:46.824885+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=24, tm_sec=46, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=47, tm_sec=19, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T21:47:25.609 INFO:tasks.ceph:pgid 3.23 last_scrub_stamp 2024-09-10T21:24:46.824885+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=24, tm_sec=46, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=47, tm_sec=19, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T21:47:25.609 INFO:tasks.ceph:pgid 3.24 last_scrub_stamp 2024-09-10T21:24:46.824885+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=24, tm_sec=46, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=47, tm_sec=19, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T21:47:25.610 INFO:tasks.ceph:pgid 3.25 last_scrub_stamp 2024-09-10T21:24:46.824885+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=24, tm_sec=46, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=47, tm_sec=19, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T21:47:25.610 INFO:tasks.ceph:pgid 3.26 last_scrub_stamp 2024-09-10T21:24:46.824885+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=24, tm_sec=46, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=47, tm_sec=19, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T21:47:25.610 INFO:tasks.ceph:pgid 3.27 last_scrub_stamp 2024-09-10T21:24:46.824885+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=24, tm_sec=46, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=47, tm_sec=19, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T21:47:25.610 INFO:tasks.ceph:pgid 3.28 last_scrub_stamp 2024-09-10T21:24:46.824885+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=24, tm_sec=46, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=47, tm_sec=19, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T21:47:25.610 INFO:tasks.ceph:pgid 3.29 last_scrub_stamp 2024-09-10T21:24:46.824885+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=24, tm_sec=46, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=47, tm_sec=19, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T21:47:25.610 INFO:tasks.ceph:pgid 3.2a last_scrub_stamp 2024-09-10T21:24:46.824885+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=24, tm_sec=46, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=47, tm_sec=19, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T21:47:25.610 INFO:tasks.ceph:pgid 3.2b last_scrub_stamp 2024-09-10T21:24:46.824885+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=24, tm_sec=46, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=47, tm_sec=19, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T21:47:25.610 INFO:tasks.ceph:pgid 3.2c last_scrub_stamp 2024-09-10T21:24:46.824885+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=24, tm_sec=46, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=47, tm_sec=19, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T21:47:25.610 INFO:tasks.ceph:pgid 3.2d last_scrub_stamp 2024-09-10T21:24:46.824885+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=24, tm_sec=46, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=47, tm_sec=19, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T21:47:25.610 INFO:tasks.ceph:pgid 3.2e last_scrub_stamp 2024-09-10T21:24:46.824885+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=24, tm_sec=46, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=47, tm_sec=19, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T21:47:25.611 INFO:tasks.ceph:pgid 3.2f last_scrub_stamp 2024-09-10T21:24:46.824885+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=24, tm_sec=46, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=47, tm_sec=19, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T21:47:25.611 INFO:tasks.ceph:pgid 3.30 last_scrub_stamp 2024-09-10T21:24:46.824885+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=24, tm_sec=46, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=47, tm_sec=19, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T21:47:25.611 INFO:tasks.ceph:pgid 3.31 last_scrub_stamp 2024-09-10T21:24:46.824885+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=24, tm_sec=46, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=47, tm_sec=19, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T21:47:25.611 INFO:tasks.ceph:pgid 3.32 last_scrub_stamp 2024-09-10T21:24:46.824885+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=24, tm_sec=46, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=47, tm_sec=19, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T21:47:25.611 INFO:tasks.ceph:pgid 3.33 last_scrub_stamp 2024-09-10T21:24:46.824885+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=24, tm_sec=46, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=47, tm_sec=19, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T21:47:25.611 INFO:tasks.ceph:pgid 3.34 last_scrub_stamp 2024-09-10T21:24:46.824885+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=24, tm_sec=46, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=47, tm_sec=19, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T21:47:25.611 INFO:tasks.ceph:pgid 3.35 last_scrub_stamp 2024-09-10T21:24:46.824885+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=24, tm_sec=46, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=47, tm_sec=19, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T21:47:25.611 INFO:tasks.ceph:pgid 3.36 last_scrub_stamp 2024-09-10T21:24:46.824885+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=24, tm_sec=46, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=47, tm_sec=19, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T21:47:25.611 INFO:tasks.ceph:pgid 3.37 last_scrub_stamp 2024-09-10T21:24:46.824885+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=24, tm_sec=46, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=47, tm_sec=19, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T21:47:25.611 INFO:tasks.ceph:pgid 3.38 last_scrub_stamp 2024-09-10T21:24:46.824885+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=24, tm_sec=46, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=47, tm_sec=19, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T21:47:25.612 INFO:tasks.ceph:pgid 3.39 last_scrub_stamp 2024-09-10T21:24:46.824885+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=24, tm_sec=46, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=47, tm_sec=19, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T21:47:25.612 INFO:tasks.ceph:pgid 3.3a last_scrub_stamp 2024-09-10T21:24:46.824885+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=24, tm_sec=46, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=47, tm_sec=19, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T21:47:25.612 INFO:tasks.ceph:pgid 3.3b last_scrub_stamp 2024-09-10T21:24:46.824885+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=24, tm_sec=46, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=47, tm_sec=19, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T21:47:25.612 INFO:tasks.ceph:pgid 3.3c last_scrub_stamp 2024-09-10T21:24:46.824885+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=24, tm_sec=46, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=47, tm_sec=19, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T21:47:25.612 INFO:tasks.ceph:pgid 3.3d last_scrub_stamp 2024-09-10T21:24:46.824885+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=24, tm_sec=46, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=47, tm_sec=19, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T21:47:25.612 INFO:tasks.ceph:pgid 3.3e last_scrub_stamp 2024-09-10T21:24:46.824885+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=24, tm_sec=46, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=47, tm_sec=19, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T21:47:25.612 INFO:tasks.ceph:pgid 3.3f last_scrub_stamp 2024-09-10T21:24:46.824885+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=24, tm_sec=46, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=47, tm_sec=19, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T21:47:25.612 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-10T21:47:45.614 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T21:47:45.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:45.757+0000 7fbf06000640 1 -- 172.21.15.17:0/3811841070 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbf000b3e00 msgr2=0x7fbf00070d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:45.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:45.757+0000 7fbf06000640 1 --2- 172.21.15.17:0/3811841070 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbf000b3e00 0x7fbf00070d80 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fbef8009bc0 tx=0x7fbef801f3b0 comp rx=0 tx=0).stop 2024-09-10T21:47:45.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:45.757+0000 7fbf06000640 1 -- 172.21.15.17:0/3811841070 shutdown_connections 2024-09-10T21:47:45.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:45.757+0000 7fbf06000640 1 --2- 172.21.15.17:0/3811841070 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbf000b3e00 0x7fbf00070d80 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:45.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:45.757+0000 7fbf06000640 1 -- 172.21.15.17:0/3811841070 >> 172.21.15.17:0/3811841070 conn(0x7fbf000cdef0 msgr2=0x7fbf000ce2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:45.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:45.757+0000 7fbf06000640 1 -- 172.21.15.17:0/3811841070 shutdown_connections 2024-09-10T21:47:45.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:45.757+0000 7fbf06000640 1 -- 172.21.15.17:0/3811841070 wait complete. 2024-09-10T21:47:45.764 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:45.761+0000 7fbf06000640 1 Processor -- start 2024-09-10T21:47:45.764 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:45.761+0000 7fbf06000640 1 -- start start 2024-09-10T21:47:45.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:45.761+0000 7fbf06000640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fbf000b3e00 0x7fbf00139c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:45.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:45.761+0000 7fbf06000640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbf001524d0 0x7fbf00147eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:45.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:45.761+0000 7fbf06000640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbf00158c20 0x7fbf0014ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:45.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:45.761+0000 7fbf06000640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fbf000c9350 con 0x7fbf00158c20 2024-09-10T21:47:45.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:45.761+0000 7fbf06000640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fbf000c9070 con 0x7fbf001524d0 2024-09-10T21:47:45.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:45.761+0000 7fbf06000640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7fbf0015b9e0 con 0x7fbf000b3e00 2024-09-10T21:47:45.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:45.761+0000 7fbf05600640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbf00158c20 0x7fbf0014ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:45.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:45.761+0000 7fbf05600640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbf00158c20 0x7fbf0014ee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:47648/0 (socket says 172.21.15.17:47648) 2024-09-10T21:47:45.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:45.761+0000 7fbf05600640 1 -- 172.21.15.17:0/1688665910 learned_addr learned my addr 172.21.15.17:0/1688665910 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:47:45.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:45.761+0000 7fbf04c00640 1 --2- 172.21.15.17:0/1688665910 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fbf000b3e00 0x7fbf00139c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:45.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:45.761+0000 7fbeffe00640 1 --2- 172.21.15.17:0/1688665910 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbf001524d0 0x7fbf00147eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:45.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:45.761+0000 7fbf05600640 1 -- 172.21.15.17:0/1688665910 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fbf000b3e00 msgr2=0x7fbf00139c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:45.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:45.761+0000 7fbf05600640 1 --2- 172.21.15.17:0/1688665910 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fbf000b3e00 0x7fbf00139c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:45.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:45.761+0000 7fbf05600640 1 -- 172.21.15.17:0/1688665910 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbf001524d0 msgr2=0x7fbf00147eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:45.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:45.761+0000 7fbf05600640 1 --2- 172.21.15.17:0/1688665910 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbf001524d0 0x7fbf00147eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:45.768 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:45.761+0000 7fbf05600640 1 -- 172.21.15.17:0/1688665910 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf001483f0 con 0x7fbf00158c20 2024-09-10T21:47:45.769 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:45.761+0000 7fbf05600640 1 --2- 172.21.15.17:0/1688665910 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbf00158c20 0x7fbf0014ee30 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fbef801bfc0 tx=0x7fbef8009300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:45.769 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:45.765+0000 7fbefcc00640 1 -- 172.21.15.17:0/1688665910 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbef80bd030 con 0x7fbf00158c20 2024-09-10T21:47:45.769 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:45.765+0000 7fbefcc00640 1 -- 172.21.15.17:0/1688665910 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbef8004b40 con 0x7fbf00158c20 2024-09-10T21:47:45.769 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:45.765+0000 7fbf06000640 1 -- 172.21.15.17:0/1688665910 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf00148660 con 0x7fbf00158c20 2024-09-10T21:47:45.769 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:45.765+0000 7fbefcc00640 1 -- 172.21.15.17:0/1688665910 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbef80b4040 con 0x7fbf00158c20 2024-09-10T21:47:45.769 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:45.765+0000 7fbf06000640 1 -- 172.21.15.17:0/1688665910 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf0014cc70 con 0x7fbf00158c20 2024-09-10T21:47:45.769 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:45.765+0000 7fbf06000640 1 -- 172.21.15.17:0/1688665910 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf0015bf00 con 0x7fbf00158c20 2024-09-10T21:47:45.769 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:45.765+0000 7fbefcc00640 1 -- 172.21.15.17:0/1688665910 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fbef8004000 con 0x7fbf00158c20 2024-09-10T21:47:45.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:45.765+0000 7fbefcc00640 1 --2- 172.21.15.17:0/1688665910 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fbee0072610 0x7fbee0074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:47:45.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:45.769+0000 7fbefcc00640 1 -- 172.21.15.17:0/1688665910 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fbef8131170 con 0x7fbf00158c20 2024-09-10T21:47:45.771 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:45.769+0000 7fbf04c00640 1 --2- 172.21.15.17:0/1688665910 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fbee0072610 0x7fbee0074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:47:45.772 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:45.769+0000 7fbf04c00640 1 --2- 172.21.15.17:0/1688665910 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fbee0072610 0x7fbee0074ad0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fbef4007fd0 tx=0x7fbef4009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:47:45.775 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:45.773+0000 7fbefcc00640 1 -- 172.21.15.17:0/1688665910 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fbef8018f10 con 0x7fbf00158c20 2024-09-10T21:47:45.947 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:45.941+0000 7fbf06000640 1 -- 172.21.15.17:0/1688665910 --> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fbf000c8ad0 con 0x7fbee0072610 2024-09-10T21:47:45.953 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:45.949+0000 7fbefcc00640 1 -- 172.21.15.17:0/1688665910 <== mgr.4100 v2:172.21.15.31:6808/3101308586 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+334612 (secure 0 0 0) 0x7fbf000c8ad0 con 0x7fbee0072610 2024-09-10T21:47:45.958 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-10T21:47:45.960 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2024-09-10T21:47:45.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:45.953+0000 7fbf06000640 1 -- 172.21.15.17:0/1688665910 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fbee0072610 msgr2=0x7fbee0074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:45.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:45.953+0000 7fbf06000640 1 --2- 172.21.15.17:0/1688665910 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fbee0072610 0x7fbee0074ad0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fbef4007fd0 tx=0x7fbef4009040 comp rx=0 tx=0).stop 2024-09-10T21:47:45.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:45.953+0000 7fbf06000640 1 -- 172.21.15.17:0/1688665910 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbf00158c20 msgr2=0x7fbf0014ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:47:45.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:45.953+0000 7fbf06000640 1 --2- 172.21.15.17:0/1688665910 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbf00158c20 0x7fbf0014ee30 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fbef801bfc0 tx=0x7fbef8009300 comp rx=0 tx=0).stop 2024-09-10T21:47:45.961 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:45.953+0000 7fbf06000640 1 -- 172.21.15.17:0/1688665910 shutdown_connections 2024-09-10T21:47:45.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:45.953+0000 7fbf06000640 1 --2- 172.21.15.17:0/1688665910 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fbee0072610 0x7fbee0074ad0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:45.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:45.953+0000 7fbf06000640 1 --2- 172.21.15.17:0/1688665910 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fbf00158c20 0x7fbf0014ee30 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:45.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:45.953+0000 7fbf06000640 1 --2- 172.21.15.17:0/1688665910 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fbf001524d0 0x7fbf00147eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:45.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:45.953+0000 7fbf06000640 1 --2- 172.21.15.17:0/1688665910 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fbf000b3e00 0x7fbf00139c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:47:45.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:45.953+0000 7fbf06000640 1 -- 172.21.15.17:0/1688665910 >> 172.21.15.17:0/1688665910 conn(0x7fbf000cdef0 msgr2=0x7fbf00155de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:47:45.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:45.953+0000 7fbf06000640 1 -- 172.21.15.17:0/1688665910 shutdown_connections 2024-09-10T21:47:45.962 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:47:45.953+0000 7fbf06000640 1 -- 172.21.15.17:0/1688665910 wait complete. 2024-09-10T21:47:45.978 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":715,"stamp":"2024-09-10T21:47:43.992663+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":15581531,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":114,"num_read_kb":174,"num_write":726807,"num_write_kb":16011424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":152604,"ondisk_log_size":152604,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":6487504,"kb_used_data":20368,"kb_used_omap":12,"kb_used_meta":6466995,"kb_avail":748487216,"statfs":{"total":773094113280,"available":766450909184,"internally_reserved":0,"allocated":20856832,"data_stored":32095942,"data_compressed":2631228,"data_compressed_allocated":3596288,"data_compressed_original":16236544,"omap_allocated":12752,"internal_metadata":6622203440},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003708"},"pg_stats":[{"pgid":"2.1f","version":"27'180","reported_seq":226,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:23.131813+0000","last_change":"2024-09-10T21:47:23.131765+0000","last_active":"2024-09-10T21:47:23.131813+0000","last_peered":"2024-09-10T21:47:23.131813+0000","last_clean":"2024-09-10T21:47:23.131813+0000","last_became_active":"2024-09-10T21:24:46.864288+0000","last_became_peered":"2024-09-10T21:24:46.864288+0000","last_unstale":"2024-09-10T21:47:23.131813+0000","last_undegraded":"2024-09-10T21:47:23.131813+0000","last_fullsized":"2024-09-10T21:47:23.131813+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'180","last_scrub_stamp":"2024-09-10T21:47:23.131706+0000","last_deep_scrub":"27'180","last_deep_scrub_stamp":"2024-09-10T21:47:23.131706+0000","last_clean_scrub_stamp":"2024-09-10T21:47:23.131706+0000","objects_scrubbed":3,"log_size":180,"log_dups_size":0,"ondisk_log_size":180,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:20:08.917966+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2316909,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":173,"num_write_kb":2361,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"27'12","reported_seq":56,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:27.217140+0000","last_change":"2024-09-10T21:47:27.217140+0000","last_active":"2024-09-10T21:47:27.217140+0000","last_peered":"2024-09-10T21:47:27.217140+0000","last_clean":"2024-09-10T21:47:27.217140+0000","last_became_active":"2024-09-10T21:24:47.864168+0000","last_became_peered":"2024-09-10T21:24:47.864168+0000","last_unstale":"2024-09-10T21:47:27.217140+0000","last_undegraded":"2024-09-10T21:47:27.217140+0000","last_fullsized":"2024-09-10T21:47:27.217140+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'12","last_scrub_stamp":"2024-09-10T21:47:27.217080+0000","last_deep_scrub":"27'12","last_deep_scrub_stamp":"2024-09-10T21:47:27.217080+0000","last_clean_scrub_stamp":"2024-09-10T21:47:27.217080+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:19:01.897786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"27'5290","reported_seq":5326,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:35.264827+0000","last_change":"2024-09-10T21:47:35.264827+0000","last_active":"2024-09-10T21:47:35.264827+0000","last_peered":"2024-09-10T21:47:35.264827+0000","last_clean":"2024-09-10T21:47:35.264827+0000","last_became_active":"2024-09-10T21:24:52.067258+0000","last_became_peered":"2024-09-10T21:24:52.067258+0000","last_unstale":"2024-09-10T21:47:35.264827+0000","last_undegraded":"2024-09-10T21:47:35.264827+0000","last_fullsized":"2024-09-10T21:47:35.264827+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'5290","last_scrub_stamp":"2024-09-10T21:47:35.264733+0000","last_deep_scrub":"27'5290","last_deep_scrub_stamp":"2024-09-10T21:47:35.264733+0000","last_clean_scrub_stamp":"2024-09-10T21:47:35.264733+0000","objects_scrubbed":0,"log_size":5290,"log_dups_size":0,"ondisk_log_size":5290,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:50:31.338570+0000","scrub_duration":31,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25332,"num_write_kb":555711,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"27'5","reported_seq":51,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:24.263539+0000","last_change":"2024-09-10T21:47:24.263539+0000","last_active":"2024-09-10T21:47:24.263539+0000","last_peered":"2024-09-10T21:47:24.263539+0000","last_clean":"2024-09-10T21:47:24.263539+0000","last_became_active":"2024-09-10T21:24:46.860341+0000","last_became_peered":"2024-09-10T21:24:46.860341+0000","last_unstale":"2024-09-10T21:47:24.263539+0000","last_undegraded":"2024-09-10T21:47:24.263539+0000","last_fullsized":"2024-09-10T21:47:24.263539+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'5","last_scrub_stamp":"2024-09-10T21:47:24.263477+0000","last_deep_scrub":"27'5","last_deep_scrub_stamp":"2024-09-10T21:47:24.263477+0000","last_clean_scrub_stamp":"2024-09-10T21:47:24.263477+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:50.477559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"27'19","reported_seq":63,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:27.150543+0000","last_change":"2024-09-10T21:47:27.150495+0000","last_active":"2024-09-10T21:47:27.150543+0000","last_peered":"2024-09-10T21:47:27.150543+0000","last_clean":"2024-09-10T21:47:27.150543+0000","last_became_active":"2024-09-10T21:24:47.865365+0000","last_became_peered":"2024-09-10T21:24:47.865365+0000","last_unstale":"2024-09-10T21:47:27.150543+0000","last_undegraded":"2024-09-10T21:47:27.150543+0000","last_fullsized":"2024-09-10T21:47:27.150543+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'19","last_scrub_stamp":"2024-09-10T21:47:27.150439+0000","last_deep_scrub":"27'19","last_deep_scrub_stamp":"2024-09-10T21:47:27.150439+0000","last_clean_scrub_stamp":"2024-09-10T21:47:27.150439+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:59:31.672676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"27'4776","reported_seq":4812,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:38.265808+0000","last_change":"2024-09-10T21:47:38.265808+0000","last_active":"2024-09-10T21:47:38.265808+0000","last_peered":"2024-09-10T21:47:38.265808+0000","last_clean":"2024-09-10T21:47:38.265808+0000","last_became_active":"2024-09-10T21:24:51.899652+0000","last_became_peered":"2024-09-10T21:24:51.899652+0000","last_unstale":"2024-09-10T21:47:38.265808+0000","last_undegraded":"2024-09-10T21:47:38.265808+0000","last_fullsized":"2024-09-10T21:47:38.265808+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4776","last_scrub_stamp":"2024-09-10T21:47:38.265701+0000","last_deep_scrub":"27'4776","last_deep_scrub_stamp":"2024-09-10T21:47:38.265701+0000","last_clean_scrub_stamp":"2024-09-10T21:47:38.265701+0000","objects_scrubbed":0,"log_size":4776,"log_dups_size":0,"ondisk_log_size":4776,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:00:51.487946+0000","scrub_duration":31,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23087,"num_write_kb":517913,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"27'8","reported_seq":52,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:31.997056+0000","last_change":"2024-09-10T21:47:31.997056+0000","last_active":"2024-09-10T21:47:31.997056+0000","last_peered":"2024-09-10T21:47:31.997056+0000","last_clean":"2024-09-10T21:47:31.997056+0000","last_became_active":"2024-09-10T21:24:46.860780+0000","last_became_peered":"2024-09-10T21:24:46.860780+0000","last_unstale":"2024-09-10T21:47:31.997056+0000","last_undegraded":"2024-09-10T21:47:31.997056+0000","last_fullsized":"2024-09-10T21:47:31.997056+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'8","last_scrub_stamp":"2024-09-10T21:47:31.996964+0000","last_deep_scrub":"27'8","last_deep_scrub_stamp":"2024-09-10T21:47:31.996964+0000","last_clean_scrub_stamp":"2024-09-10T21:47:31.996964+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:46:32.272965+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"27'16","reported_seq":57,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:32.740828+0000","last_change":"2024-09-10T21:47:32.740828+0000","last_active":"2024-09-10T21:47:32.740828+0000","last_peered":"2024-09-10T21:47:32.740828+0000","last_clean":"2024-09-10T21:47:32.740828+0000","last_became_active":"2024-09-10T21:24:47.877765+0000","last_became_peered":"2024-09-10T21:24:47.877765+0000","last_unstale":"2024-09-10T21:47:32.740828+0000","last_undegraded":"2024-09-10T21:47:32.740828+0000","last_fullsized":"2024-09-10T21:47:32.740828+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'16","last_scrub_stamp":"2024-09-10T21:47:32.740745+0000","last_deep_scrub":"27'16","last_deep_scrub_stamp":"2024-09-10T21:47:32.740745+0000","last_clean_scrub_stamp":"2024-09-10T21:47:32.740745+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:19:07.421452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"27'4118","reported_seq":4162,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:37.227370+0000","last_change":"2024-09-10T21:47:37.227370+0000","last_active":"2024-09-10T21:47:37.227370+0000","last_peered":"2024-09-10T21:47:37.227370+0000","last_clean":"2024-09-10T21:47:37.227370+0000","last_became_active":"2024-09-10T21:25:28.512023+0000","last_became_peered":"2024-09-10T21:25:28.512023+0000","last_unstale":"2024-09-10T21:47:37.227370+0000","last_undegraded":"2024-09-10T21:47:37.227370+0000","last_fullsized":"2024-09-10T21:47:37.227370+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4118","last_scrub_stamp":"2024-09-10T21:47:37.227217+0000","last_deep_scrub":"27'4118","last_deep_scrub_stamp":"2024-09-10T21:47:37.227217+0000","last_clean_scrub_stamp":"2024-09-10T21:47:37.227217+0000","objects_scrubbed":0,"log_size":4118,"log_dups_size":0,"ondisk_log_size":4118,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:17:15.484677+0000","scrub_duration":27,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18983,"num_write_kb":440322,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,2],"acting":[4,3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"27'6","reported_seq":50,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:30.061283+0000","last_change":"2024-09-10T21:47:30.061283+0000","last_active":"2024-09-10T21:47:30.061283+0000","last_peered":"2024-09-10T21:47:30.061283+0000","last_clean":"2024-09-10T21:47:30.061283+0000","last_became_active":"2024-09-10T21:24:46.872075+0000","last_became_peered":"2024-09-10T21:24:46.872075+0000","last_unstale":"2024-09-10T21:47:30.061283+0000","last_undegraded":"2024-09-10T21:47:30.061283+0000","last_fullsized":"2024-09-10T21:47:30.061283+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'6","last_scrub_stamp":"2024-09-10T21:47:30.061189+0000","last_deep_scrub":"27'6","last_deep_scrub_stamp":"2024-09-10T21:47:30.061189+0000","last_clean_scrub_stamp":"2024-09-10T21:47:30.061189+0000","objects_scrubbed":1,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:06:42.038139+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"27'30","reported_seq":71,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:33.696444+0000","last_change":"2024-09-10T21:47:33.696444+0000","last_active":"2024-09-10T21:47:33.696444+0000","last_peered":"2024-09-10T21:47:33.696444+0000","last_clean":"2024-09-10T21:47:33.696444+0000","last_became_active":"2024-09-10T21:24:47.876688+0000","last_became_peered":"2024-09-10T21:24:47.876688+0000","last_unstale":"2024-09-10T21:47:33.696444+0000","last_undegraded":"2024-09-10T21:47:33.696444+0000","last_fullsized":"2024-09-10T21:47:33.696444+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'30","last_scrub_stamp":"2024-09-10T21:47:33.696359+0000","last_deep_scrub":"27'30","last_deep_scrub_stamp":"2024-09-10T21:47:33.696359+0000","last_clean_scrub_stamp":"2024-09-10T21:47:33.696359+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:22:08.798884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"27'3999","reported_seq":4035,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:35.239870+0000","last_change":"2024-09-10T21:47:35.239870+0000","last_active":"2024-09-10T21:47:35.239870+0000","last_peered":"2024-09-10T21:47:35.239870+0000","last_clean":"2024-09-10T21:47:35.239870+0000","last_became_active":"2024-09-10T21:24:51.899576+0000","last_became_peered":"2024-09-10T21:24:51.899576+0000","last_unstale":"2024-09-10T21:47:35.239870+0000","last_undegraded":"2024-09-10T21:47:35.239870+0000","last_fullsized":"2024-09-10T21:47:35.239870+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'3999","last_scrub_stamp":"2024-09-10T21:47:35.239777+0000","last_deep_scrub":"27'3999","last_deep_scrub_stamp":"2024-09-10T21:47:35.239777+0000","last_clean_scrub_stamp":"2024-09-10T21:47:35.239777+0000","objects_scrubbed":0,"log_size":3999,"log_dups_size":0,"ondisk_log_size":3999,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:19:09.920484+0000","scrub_duration":27,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18841,"num_write_kb":435514,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"27'15","reported_seq":57,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:30.747799+0000","last_change":"2024-09-10T21:47:30.747799+0000","last_active":"2024-09-10T21:47:30.747799+0000","last_peered":"2024-09-10T21:47:30.747799+0000","last_clean":"2024-09-10T21:47:30.747799+0000","last_became_active":"2024-09-10T21:24:47.869569+0000","last_became_peered":"2024-09-10T21:24:47.869569+0000","last_unstale":"2024-09-10T21:47:30.747799+0000","last_undegraded":"2024-09-10T21:47:30.747799+0000","last_fullsized":"2024-09-10T21:47:30.747799+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'15","last_scrub_stamp":"2024-09-10T21:47:30.747728+0000","last_deep_scrub":"27'15","last_deep_scrub_stamp":"2024-09-10T21:47:30.747728+0000","last_clean_scrub_stamp":"2024-09-10T21:47:30.747728+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:30.496496+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"27'1005","reported_seq":1050,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:28.127434+0000","last_change":"2024-09-10T21:47:28.127434+0000","last_active":"2024-09-10T21:47:28.127434+0000","last_peered":"2024-09-10T21:47:28.127434+0000","last_clean":"2024-09-10T21:47:28.127434+0000","last_became_active":"2024-09-10T21:24:46.866893+0000","last_became_peered":"2024-09-10T21:24:46.866893+0000","last_unstale":"2024-09-10T21:47:28.127434+0000","last_undegraded":"2024-09-10T21:47:28.127434+0000","last_fullsized":"2024-09-10T21:47:28.127434+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'1005","last_scrub_stamp":"2024-09-10T21:47:28.127354+0000","last_deep_scrub":"27'1005","last_deep_scrub_stamp":"2024-09-10T21:47:28.127354+0000","last_clean_scrub_stamp":"2024-09-10T21:47:28.127354+0000","objects_scrubbed":2,"log_size":1005,"log_dups_size":0,"ondisk_log_size":1005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:27.876122+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":997,"num_write_kb":4359,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"27'4606","reported_seq":4643,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:33.268803+0000","last_change":"2024-09-10T21:47:33.268803+0000","last_active":"2024-09-10T21:47:33.268803+0000","last_peered":"2024-09-10T21:47:33.268803+0000","last_clean":"2024-09-10T21:47:33.268803+0000","last_became_active":"2024-09-10T21:24:51.907923+0000","last_became_peered":"2024-09-10T21:24:51.907923+0000","last_unstale":"2024-09-10T21:47:33.268803+0000","last_undegraded":"2024-09-10T21:47:33.268803+0000","last_fullsized":"2024-09-10T21:47:33.268803+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4606","last_scrub_stamp":"2024-09-10T21:47:33.268695+0000","last_deep_scrub":"27'4606","last_deep_scrub_stamp":"2024-09-10T21:47:33.268695+0000","last_clean_scrub_stamp":"2024-09-10T21:47:33.268695+0000","objects_scrubbed":0,"log_size":4606,"log_dups_size":0,"ondisk_log_size":4606,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:19:15.091323+0000","scrub_duration":27,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22377,"num_write_kb":531442,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"27'17","reported_seq":60,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:25.218592+0000","last_change":"2024-09-10T21:47:25.218592+0000","last_active":"2024-09-10T21:47:25.218592+0000","last_peered":"2024-09-10T21:47:25.218592+0000","last_clean":"2024-09-10T21:47:25.218592+0000","last_became_active":"2024-09-10T21:24:47.861842+0000","last_became_peered":"2024-09-10T21:24:47.861842+0000","last_unstale":"2024-09-10T21:47:25.218592+0000","last_undegraded":"2024-09-10T21:47:25.218592+0000","last_fullsized":"2024-09-10T21:47:25.218592+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'17","last_scrub_stamp":"2024-09-10T21:47:25.218527+0000","last_deep_scrub":"27'17","last_deep_scrub_stamp":"2024-09-10T21:47:25.218527+0000","last_clean_scrub_stamp":"2024-09-10T21:47:25.218527+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:17:58.315743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"27'3","reported_seq":47,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:31.940341+0000","last_change":"2024-09-10T21:47:31.940341+0000","last_active":"2024-09-10T21:47:31.940341+0000","last_peered":"2024-09-10T21:47:31.940341+0000","last_clean":"2024-09-10T21:47:31.940341+0000","last_became_active":"2024-09-10T21:24:46.851027+0000","last_became_peered":"2024-09-10T21:24:46.851027+0000","last_unstale":"2024-09-10T21:47:31.940341+0000","last_undegraded":"2024-09-10T21:47:31.940341+0000","last_fullsized":"2024-09-10T21:47:31.940341+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'3","last_scrub_stamp":"2024-09-10T21:47:31.940273+0000","last_deep_scrub":"27'3","last_deep_scrub_stamp":"2024-09-10T21:47:31.940273+0000","last_clean_scrub_stamp":"2024-09-10T21:47:31.940273+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:17:02.645449+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"27'4524","reported_seq":4561,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:32.309653+0000","last_change":"2024-09-10T21:47:32.309653+0000","last_active":"2024-09-10T21:47:32.309653+0000","last_peered":"2024-09-10T21:47:32.309653+0000","last_clean":"2024-09-10T21:47:32.309653+0000","last_became_active":"2024-09-10T21:24:51.903031+0000","last_became_peered":"2024-09-10T21:24:51.903031+0000","last_unstale":"2024-09-10T21:47:32.309653+0000","last_undegraded":"2024-09-10T21:47:32.309653+0000","last_fullsized":"2024-09-10T21:47:32.309653+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4524","last_scrub_stamp":"2024-09-10T21:47:32.309556+0000","last_deep_scrub":"27'4524","last_deep_scrub_stamp":"2024-09-10T21:47:32.309556+0000","last_clean_scrub_stamp":"2024-09-10T21:47:32.309556+0000","objects_scrubbed":0,"log_size":4524,"log_dups_size":0,"ondisk_log_size":4524,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:26:18.451044+0000","scrub_duration":27,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21954,"num_write_kb":484896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"27'18","reported_seq":61,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:30.160429+0000","last_change":"2024-09-10T21:47:30.160429+0000","last_active":"2024-09-10T21:47:30.160429+0000","last_peered":"2024-09-10T21:47:30.160429+0000","last_clean":"2024-09-10T21:47:30.160429+0000","last_became_active":"2024-09-10T21:24:47.856003+0000","last_became_peered":"2024-09-10T21:24:47.856003+0000","last_unstale":"2024-09-10T21:47:30.160429+0000","last_undegraded":"2024-09-10T21:47:30.160429+0000","last_fullsized":"2024-09-10T21:47:30.160429+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'18","last_scrub_stamp":"2024-09-10T21:47:30.160362+0000","last_deep_scrub":"27'18","last_deep_scrub_stamp":"2024-09-10T21:47:30.160362+0000","last_clean_scrub_stamp":"2024-09-10T21:47:30.160362+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:21:38.722572+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"27'2","reported_seq":46,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:29.969656+0000","last_change":"2024-09-10T21:47:29.969656+0000","last_active":"2024-09-10T21:47:29.969656+0000","last_peered":"2024-09-10T21:47:29.969656+0000","last_clean":"2024-09-10T21:47:29.969656+0000","last_became_active":"2024-09-10T21:24:46.852993+0000","last_became_peered":"2024-09-10T21:24:46.852993+0000","last_unstale":"2024-09-10T21:47:29.969656+0000","last_undegraded":"2024-09-10T21:47:29.969656+0000","last_fullsized":"2024-09-10T21:47:29.969656+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2","last_scrub_stamp":"2024-09-10T21:47:29.969599+0000","last_deep_scrub":"27'2","last_deep_scrub_stamp":"2024-09-10T21:47:29.969599+0000","last_clean_scrub_stamp":"2024-09-10T21:47:29.969599+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:03.952547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"27'5859","reported_seq":5896,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:38.013477+0000","last_change":"2024-09-10T21:47:38.013477+0000","last_active":"2024-09-10T21:47:38.013477+0000","last_peered":"2024-09-10T21:47:38.013477+0000","last_clean":"2024-09-10T21:47:38.013477+0000","last_became_active":"2024-09-10T21:24:52.064884+0000","last_became_peered":"2024-09-10T21:24:52.064884+0000","last_unstale":"2024-09-10T21:47:38.013477+0000","last_undegraded":"2024-09-10T21:47:38.013477+0000","last_fullsized":"2024-09-10T21:47:38.013477+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'5859","last_scrub_stamp":"2024-09-10T21:47:38.013389+0000","last_deep_scrub":"27'5859","last_deep_scrub_stamp":"2024-09-10T21:47:38.013389+0000","last_clean_scrub_stamp":"2024-09-10T21:47:38.013389+0000","objects_scrubbed":0,"log_size":5859,"log_dups_size":0,"ondisk_log_size":5859,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:54:35.319194+0000","scrub_duration":51,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27858,"num_write_kb":655707,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"27'9","reported_seq":52,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:26.091888+0000","last_change":"2024-09-10T21:47:26.091888+0000","last_active":"2024-09-10T21:47:26.091888+0000","last_peered":"2024-09-10T21:47:26.091888+0000","last_clean":"2024-09-10T21:47:26.091888+0000","last_became_active":"2024-09-10T21:24:47.856185+0000","last_became_peered":"2024-09-10T21:24:47.856185+0000","last_unstale":"2024-09-10T21:47:26.091888+0000","last_undegraded":"2024-09-10T21:47:26.091888+0000","last_fullsized":"2024-09-10T21:47:26.091888+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9","last_scrub_stamp":"2024-09-10T21:47:26.091769+0000","last_deep_scrub":"27'9","last_deep_scrub_stamp":"2024-09-10T21:47:26.091769+0000","last_clean_scrub_stamp":"2024-09-10T21:47:26.091769+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:54:45.759809+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"27'5","reported_seq":50,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:31.019617+0000","last_change":"2024-09-10T21:47:31.019617+0000","last_active":"2024-09-10T21:47:31.019617+0000","last_peered":"2024-09-10T21:47:31.019617+0000","last_clean":"2024-09-10T21:47:31.019617+0000","last_became_active":"2024-09-10T21:24:46.866477+0000","last_became_peered":"2024-09-10T21:24:46.866477+0000","last_unstale":"2024-09-10T21:47:31.019617+0000","last_undegraded":"2024-09-10T21:47:31.019617+0000","last_fullsized":"2024-09-10T21:47:31.019617+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'5","last_scrub_stamp":"2024-09-10T21:47:31.019532+0000","last_deep_scrub":"27'5","last_deep_scrub_stamp":"2024-09-10T21:47:31.019532+0000","last_clean_scrub_stamp":"2024-09-10T21:47:31.019532+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:05.002480+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"27'4736","reported_seq":4773,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:34.228852+0000","last_change":"2024-09-10T21:47:34.228852+0000","last_active":"2024-09-10T21:47:34.228852+0000","last_peered":"2024-09-10T21:47:34.228852+0000","last_clean":"2024-09-10T21:47:34.228852+0000","last_became_active":"2024-09-10T21:24:51.907878+0000","last_became_peered":"2024-09-10T21:24:51.907878+0000","last_unstale":"2024-09-10T21:47:34.228852+0000","last_undegraded":"2024-09-10T21:47:34.228852+0000","last_fullsized":"2024-09-10T21:47:34.228852+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4736","last_scrub_stamp":"2024-09-10T21:47:34.228769+0000","last_deep_scrub":"27'4736","last_deep_scrub_stamp":"2024-09-10T21:47:34.228769+0000","last_clean_scrub_stamp":"2024-09-10T21:47:34.228769+0000","objects_scrubbed":0,"log_size":4736,"log_dups_size":0,"ondisk_log_size":4736,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:47:28.146012+0000","scrub_duration":27,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22584,"num_write_kb":479265,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"27'25","reported_seq":68,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:26.154489+0000","last_change":"2024-09-10T21:47:26.154489+0000","last_active":"2024-09-10T21:47:26.154489+0000","last_peered":"2024-09-10T21:47:26.154489+0000","last_clean":"2024-09-10T21:47:26.154489+0000","last_became_active":"2024-09-10T21:24:47.844505+0000","last_became_peered":"2024-09-10T21:24:47.844505+0000","last_unstale":"2024-09-10T21:47:26.154489+0000","last_undegraded":"2024-09-10T21:47:26.154489+0000","last_fullsized":"2024-09-10T21:47:26.154489+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'25","last_scrub_stamp":"2024-09-10T21:47:26.154431+0000","last_deep_scrub":"27'25","last_deep_scrub_stamp":"2024-09-10T21:47:26.154431+0000","last_clean_scrub_stamp":"2024-09-10T21:47:26.154431+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:12:31.123628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"27'10","reported_seq":54,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:24.643112+0000","last_change":"2024-09-10T21:47:24.643112+0000","last_active":"2024-09-10T21:47:24.643112+0000","last_peered":"2024-09-10T21:47:24.643112+0000","last_clean":"2024-09-10T21:47:24.643112+0000","last_became_active":"2024-09-10T21:24:46.861473+0000","last_became_peered":"2024-09-10T21:24:46.861473+0000","last_unstale":"2024-09-10T21:47:24.643112+0000","last_undegraded":"2024-09-10T21:47:24.643112+0000","last_fullsized":"2024-09-10T21:47:24.643112+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'10","last_scrub_stamp":"2024-09-10T21:47:24.643031+0000","last_deep_scrub":"27'10","last_deep_scrub_stamp":"2024-09-10T21:47:24.643031+0000","last_clean_scrub_stamp":"2024-09-10T21:47:24.643031+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:22.143057+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":53,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"27'4521","reported_seq":4558,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:30.235855+0000","last_change":"2024-09-10T21:47:30.235855+0000","last_active":"2024-09-10T21:47:30.235855+0000","last_peered":"2024-09-10T21:47:30.235855+0000","last_clean":"2024-09-10T21:47:30.235855+0000","last_became_active":"2024-09-10T21:24:52.061581+0000","last_became_peered":"2024-09-10T21:24:52.061581+0000","last_unstale":"2024-09-10T21:47:30.235855+0000","last_undegraded":"2024-09-10T21:47:30.235855+0000","last_fullsized":"2024-09-10T21:47:30.235855+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4521","last_scrub_stamp":"2024-09-10T21:47:30.235757+0000","last_deep_scrub":"27'4521","last_deep_scrub_stamp":"2024-09-10T21:47:30.235757+0000","last_clean_scrub_stamp":"2024-09-10T21:47:30.235757+0000","objects_scrubbed":0,"log_size":4521,"log_dups_size":0,"ondisk_log_size":4521,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:53:04.671154+0000","scrub_duration":31,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21874,"num_write_kb":476387,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"27'18","reported_seq":61,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:26.176880+0000","last_change":"2024-09-10T21:47:26.176880+0000","last_active":"2024-09-10T21:47:26.176880+0000","last_peered":"2024-09-10T21:47:26.176880+0000","last_clean":"2024-09-10T21:47:26.176880+0000","last_became_active":"2024-09-10T21:24:47.859483+0000","last_became_peered":"2024-09-10T21:24:47.859483+0000","last_unstale":"2024-09-10T21:47:26.176880+0000","last_undegraded":"2024-09-10T21:47:26.176880+0000","last_fullsized":"2024-09-10T21:47:26.176880+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'18","last_scrub_stamp":"2024-09-10T21:47:26.176783+0000","last_deep_scrub":"27'18","last_deep_scrub_stamp":"2024-09-10T21:47:26.176783+0000","last_clean_scrub_stamp":"2024-09-10T21:47:26.176783+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:47:20.094026+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"27'327","reported_seq":371,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:22.946760+0000","last_change":"2024-09-10T21:47:22.946760+0000","last_active":"2024-09-10T21:47:22.946760+0000","last_peered":"2024-09-10T21:47:22.946760+0000","last_clean":"2024-09-10T21:47:22.946760+0000","last_became_active":"2024-09-10T21:24:46.860309+0000","last_became_peered":"2024-09-10T21:24:46.860309+0000","last_unstale":"2024-09-10T21:47:22.946760+0000","last_undegraded":"2024-09-10T21:47:22.946760+0000","last_fullsized":"2024-09-10T21:47:22.946760+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'327","last_scrub_stamp":"2024-09-10T21:47:22.946702+0000","last_deep_scrub":"27'327","last_deep_scrub_stamp":"2024-09-10T21:47:22.946702+0000","last_clean_scrub_stamp":"2024-09-10T21:47:22.946702+0000","objects_scrubbed":1,"log_size":327,"log_dups_size":0,"ondisk_log_size":327,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:31:01.455227+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":704,"num_write_kb":242,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"27'4358","reported_seq":4396,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:39.265520+0000","last_change":"2024-09-10T21:47:39.265520+0000","last_active":"2024-09-10T21:47:39.265520+0000","last_peered":"2024-09-10T21:47:39.265520+0000","last_clean":"2024-09-10T21:47:39.265520+0000","last_became_active":"2024-09-10T21:24:52.068250+0000","last_became_peered":"2024-09-10T21:24:52.068250+0000","last_unstale":"2024-09-10T21:47:39.265520+0000","last_undegraded":"2024-09-10T21:47:39.265520+0000","last_fullsized":"2024-09-10T21:47:39.265520+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4358","last_scrub_stamp":"2024-09-10T21:47:39.265434+0000","last_deep_scrub":"27'4358","last_deep_scrub_stamp":"2024-09-10T21:47:39.265434+0000","last_clean_scrub_stamp":"2024-09-10T21:47:39.265434+0000","objects_scrubbed":0,"log_size":4358,"log_dups_size":0,"ondisk_log_size":4358,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:14:31.486338+0000","scrub_duration":35,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21666,"num_write_kb":463209,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"27'18","reported_seq":60,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:24.937326+0000","last_change":"2024-09-10T21:47:24.937326+0000","last_active":"2024-09-10T21:47:24.937326+0000","last_peered":"2024-09-10T21:47:24.937326+0000","last_clean":"2024-09-10T21:47:24.937326+0000","last_became_active":"2024-09-10T21:24:47.853750+0000","last_became_peered":"2024-09-10T21:24:47.853750+0000","last_unstale":"2024-09-10T21:47:24.937326+0000","last_undegraded":"2024-09-10T21:47:24.937326+0000","last_fullsized":"2024-09-10T21:47:24.937326+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'18","last_scrub_stamp":"2024-09-10T21:47:24.937253+0000","last_deep_scrub":"27'18","last_deep_scrub_stamp":"2024-09-10T21:47:24.937253+0000","last_clean_scrub_stamp":"2024-09-10T21:47:24.937253+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:58:49.150573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"27'2","reported_seq":47,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:27.093212+0000","last_change":"2024-09-10T21:47:27.093212+0000","last_active":"2024-09-10T21:47:27.093212+0000","last_peered":"2024-09-10T21:47:27.093212+0000","last_clean":"2024-09-10T21:47:27.093212+0000","last_became_active":"2024-09-10T21:24:46.864149+0000","last_became_peered":"2024-09-10T21:24:46.864149+0000","last_unstale":"2024-09-10T21:47:27.093212+0000","last_undegraded":"2024-09-10T21:47:27.093212+0000","last_fullsized":"2024-09-10T21:47:27.093212+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2","last_scrub_stamp":"2024-09-10T21:47:27.093133+0000","last_deep_scrub":"27'2","last_deep_scrub_stamp":"2024-09-10T21:47:27.093133+0000","last_clean_scrub_stamp":"2024-09-10T21:47:27.093133+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:34:02.239975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"27'4655","reported_seq":4692,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:34.174519+0000","last_change":"2024-09-10T21:47:34.174519+0000","last_active":"2024-09-10T21:47:34.174519+0000","last_peered":"2024-09-10T21:47:34.174519+0000","last_clean":"2024-09-10T21:47:34.174519+0000","last_became_active":"2024-09-10T21:24:52.060953+0000","last_became_peered":"2024-09-10T21:24:52.060953+0000","last_unstale":"2024-09-10T21:47:34.174519+0000","last_undegraded":"2024-09-10T21:47:34.174519+0000","last_fullsized":"2024-09-10T21:47:34.174519+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4655","last_scrub_stamp":"2024-09-10T21:47:34.174437+0000","last_deep_scrub":"27'4655","last_deep_scrub_stamp":"2024-09-10T21:47:34.174437+0000","last_clean_scrub_stamp":"2024-09-10T21:47:34.174437+0000","objects_scrubbed":0,"log_size":4655,"log_dups_size":0,"ondisk_log_size":4655,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:51:24.442254+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22893,"num_write_kb":513258,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"27'18","reported_seq":60,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:29.758834+0000","last_change":"2024-09-10T21:47:29.758834+0000","last_active":"2024-09-10T21:47:29.758834+0000","last_peered":"2024-09-10T21:47:29.758834+0000","last_clean":"2024-09-10T21:47:29.758834+0000","last_became_active":"2024-09-10T21:24:47.866907+0000","last_became_peered":"2024-09-10T21:24:47.866907+0000","last_unstale":"2024-09-10T21:47:29.758834+0000","last_undegraded":"2024-09-10T21:47:29.758834+0000","last_fullsized":"2024-09-10T21:47:29.758834+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'18","last_scrub_stamp":"2024-09-10T21:47:29.758754+0000","last_deep_scrub":"27'18","last_deep_scrub_stamp":"2024-09-10T21:47:29.758754+0000","last_clean_scrub_stamp":"2024-09-10T21:47:29.758754+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:26:15.900242+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"27'2","reported_seq":46,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:21.960410+0000","last_change":"2024-09-10T21:47:21.960410+0000","last_active":"2024-09-10T21:47:21.960410+0000","last_peered":"2024-09-10T21:47:21.960410+0000","last_clean":"2024-09-10T21:47:21.960410+0000","last_became_active":"2024-09-10T21:24:46.856489+0000","last_became_peered":"2024-09-10T21:24:46.856489+0000","last_unstale":"2024-09-10T21:47:21.960410+0000","last_undegraded":"2024-09-10T21:47:21.960410+0000","last_fullsized":"2024-09-10T21:47:21.960410+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2","last_scrub_stamp":"2024-09-10T21:47:21.960350+0000","last_deep_scrub":"27'2","last_deep_scrub_stamp":"2024-09-10T21:47:21.960350+0000","last_clean_scrub_stamp":"2024-09-10T21:47:21.960350+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:17:07.017677+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"27'4909","reported_seq":4946,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:34.041341+0000","last_change":"2024-09-10T21:47:34.041341+0000","last_active":"2024-09-10T21:47:34.041341+0000","last_peered":"2024-09-10T21:47:34.041341+0000","last_clean":"2024-09-10T21:47:34.041341+0000","last_became_active":"2024-09-10T21:24:51.900479+0000","last_became_peered":"2024-09-10T21:24:51.900479+0000","last_unstale":"2024-09-10T21:47:34.041341+0000","last_undegraded":"2024-09-10T21:47:34.041341+0000","last_fullsized":"2024-09-10T21:47:34.041341+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4909","last_scrub_stamp":"2024-09-10T21:47:34.041256+0000","last_deep_scrub":"27'4909","last_deep_scrub_stamp":"2024-09-10T21:47:34.041256+0000","last_clean_scrub_stamp":"2024-09-10T21:47:34.041256+0000","objects_scrubbed":0,"log_size":4909,"log_dups_size":0,"ondisk_log_size":4909,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:54:37.955889+0000","scrub_duration":35,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23807,"num_write_kb":525103,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"27'16","reported_seq":59,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:25.069701+0000","last_change":"2024-09-10T21:47:25.069701+0000","last_active":"2024-09-10T21:47:25.069701+0000","last_peered":"2024-09-10T21:47:25.069701+0000","last_clean":"2024-09-10T21:47:25.069701+0000","last_became_active":"2024-09-10T21:24:47.858438+0000","last_became_peered":"2024-09-10T21:24:47.858438+0000","last_unstale":"2024-09-10T21:47:25.069701+0000","last_undegraded":"2024-09-10T21:47:25.069701+0000","last_fullsized":"2024-09-10T21:47:25.069701+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'16","last_scrub_stamp":"2024-09-10T21:47:25.069622+0000","last_deep_scrub":"27'16","last_deep_scrub_stamp":"2024-09-10T21:47:25.069622+0000","last_clean_scrub_stamp":"2024-09-10T21:47:25.069622+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:56:48.024051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"27'4","reported_seq":48,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:24.777261+0000","last_change":"2024-09-10T21:47:24.777261+0000","last_active":"2024-09-10T21:47:24.777261+0000","last_peered":"2024-09-10T21:47:24.777261+0000","last_clean":"2024-09-10T21:47:24.777261+0000","last_became_active":"2024-09-10T21:24:46.863150+0000","last_became_peered":"2024-09-10T21:24:46.863150+0000","last_unstale":"2024-09-10T21:47:24.777261+0000","last_undegraded":"2024-09-10T21:47:24.777261+0000","last_fullsized":"2024-09-10T21:47:24.777261+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4","last_scrub_stamp":"2024-09-10T21:47:24.777166+0000","last_deep_scrub":"27'4","last_deep_scrub_stamp":"2024-09-10T21:47:24.777166+0000","last_clean_scrub_stamp":"2024-09-10T21:47:24.777166+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:57:05.934346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"27'5335","reported_seq":5372,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:27.023955+0000","last_change":"2024-09-10T21:47:27.023955+0000","last_active":"2024-09-10T21:47:27.023955+0000","last_peered":"2024-09-10T21:47:27.023955+0000","last_clean":"2024-09-10T21:47:27.023955+0000","last_became_active":"2024-09-10T21:24:51.906888+0000","last_became_peered":"2024-09-10T21:24:51.906888+0000","last_unstale":"2024-09-10T21:47:27.023955+0000","last_undegraded":"2024-09-10T21:47:27.023955+0000","last_fullsized":"2024-09-10T21:47:27.023955+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'5335","last_scrub_stamp":"2024-09-10T21:47:27.023873+0000","last_deep_scrub":"27'5335","last_deep_scrub_stamp":"2024-09-10T21:47:27.023873+0000","last_clean_scrub_stamp":"2024-09-10T21:47:27.023873+0000","objects_scrubbed":0,"log_size":5335,"log_dups_size":0,"ondisk_log_size":5335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:26.772642+0000","scrub_duration":35,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26491,"num_write_kb":568740,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"27'20","reported_seq":63,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:30.922336+0000","last_change":"2024-09-10T21:47:30.922336+0000","last_active":"2024-09-10T21:47:30.922336+0000","last_peered":"2024-09-10T21:47:30.922336+0000","last_clean":"2024-09-10T21:47:30.922336+0000","last_became_active":"2024-09-10T21:24:47.849846+0000","last_became_peered":"2024-09-10T21:24:47.849846+0000","last_unstale":"2024-09-10T21:47:30.922336+0000","last_undegraded":"2024-09-10T21:47:30.922336+0000","last_fullsized":"2024-09-10T21:47:30.922336+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'20","last_scrub_stamp":"2024-09-10T21:47:30.922276+0000","last_deep_scrub":"27'20","last_deep_scrub_stamp":"2024-09-10T21:47:30.922276+0000","last_clean_scrub_stamp":"2024-09-10T21:47:30.922276+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:38:31.565344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"27'52","reported_seq":146,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:23.121455+0000","last_change":"2024-09-10T21:47:23.121361+0000","last_active":"2024-09-10T21:47:23.121455+0000","last_peered":"2024-09-10T21:47:23.121455+0000","last_clean":"2024-09-10T21:47:23.121455+0000","last_became_active":"2024-09-10T21:24:46.851200+0000","last_became_peered":"2024-09-10T21:24:46.851200+0000","last_unstale":"2024-09-10T21:47:23.121455+0000","last_undegraded":"2024-09-10T21:47:23.121455+0000","last_fullsized":"2024-09-10T21:47:23.121455+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'52","last_scrub_stamp":"2024-09-10T21:47:23.121257+0000","last_deep_scrub":"27'52","last_deep_scrub_stamp":"2024-09-10T21:47:23.121257+0000","last_clean_scrub_stamp":"2024-09-10T21:47:23.121257+0000","objects_scrubbed":1,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:43:57.409412+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90454,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":48,"num_read_kb":119,"num_write":48,"num_write_kb":119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"27'4677","reported_seq":4714,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:36.980284+0000","last_change":"2024-09-10T21:47:36.980284+0000","last_active":"2024-09-10T21:47:36.980284+0000","last_peered":"2024-09-10T21:47:36.980284+0000","last_clean":"2024-09-10T21:47:36.980284+0000","last_became_active":"2024-09-10T21:24:51.901141+0000","last_became_peered":"2024-09-10T21:24:51.901141+0000","last_unstale":"2024-09-10T21:47:36.980284+0000","last_undegraded":"2024-09-10T21:47:36.980284+0000","last_fullsized":"2024-09-10T21:47:36.980284+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4677","last_scrub_stamp":"2024-09-10T21:47:36.980186+0000","last_deep_scrub":"27'4677","last_deep_scrub_stamp":"2024-09-10T21:47:36.980186+0000","last_clean_scrub_stamp":"2024-09-10T21:47:36.980186+0000","objects_scrubbed":0,"log_size":4677,"log_dups_size":0,"ondisk_log_size":4677,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:25:44.121039+0000","scrub_duration":35,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22963,"num_write_kb":501268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"27'11","reported_seq":55,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:24.122533+0000","last_change":"2024-09-10T21:47:24.122483+0000","last_active":"2024-09-10T21:47:24.122533+0000","last_peered":"2024-09-10T21:47:24.122533+0000","last_clean":"2024-09-10T21:47:24.122533+0000","last_became_active":"2024-09-10T21:24:47.846358+0000","last_became_peered":"2024-09-10T21:24:47.846358+0000","last_unstale":"2024-09-10T21:47:24.122533+0000","last_undegraded":"2024-09-10T21:47:24.122533+0000","last_fullsized":"2024-09-10T21:47:24.122533+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'11","last_scrub_stamp":"2024-09-10T21:47:24.122422+0000","last_deep_scrub":"27'11","last_deep_scrub_stamp":"2024-09-10T21:47:24.122422+0000","last_clean_scrub_stamp":"2024-09-10T21:47:24.122422+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:02:48.829990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"27'7","reported_seq":53,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:29.103436+0000","last_change":"2024-09-10T21:47:29.103436+0000","last_active":"2024-09-10T21:47:29.103436+0000","last_peered":"2024-09-10T21:47:29.103436+0000","last_clean":"2024-09-10T21:47:29.103436+0000","last_became_active":"2024-09-10T21:24:46.868375+0000","last_became_peered":"2024-09-10T21:24:46.868375+0000","last_unstale":"2024-09-10T21:47:29.103436+0000","last_undegraded":"2024-09-10T21:47:29.103436+0000","last_fullsized":"2024-09-10T21:47:29.103436+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'7","last_scrub_stamp":"2024-09-10T21:47:29.103354+0000","last_deep_scrub":"27'7","last_deep_scrub_stamp":"2024-09-10T21:47:29.103354+0000","last_clean_scrub_stamp":"2024-09-10T21:47:29.103354+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:08:06.990919+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"27'4824","reported_seq":4861,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:36.512209+0000","last_change":"2024-09-10T21:47:36.512209+0000","last_active":"2024-09-10T21:47:36.512209+0000","last_peered":"2024-09-10T21:47:36.512209+0000","last_clean":"2024-09-10T21:47:36.512209+0000","last_became_active":"2024-09-10T21:24:52.067308+0000","last_became_peered":"2024-09-10T21:24:52.067308+0000","last_unstale":"2024-09-10T21:47:36.512209+0000","last_undegraded":"2024-09-10T21:47:36.512209+0000","last_fullsized":"2024-09-10T21:47:36.512209+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4824","last_scrub_stamp":"2024-09-10T21:47:36.512105+0000","last_deep_scrub":"27'4824","last_deep_scrub_stamp":"2024-09-10T21:47:36.512105+0000","last_clean_scrub_stamp":"2024-09-10T21:47:36.512105+0000","objects_scrubbed":0,"log_size":4824,"log_dups_size":0,"ondisk_log_size":4824,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:18:15.084927+0000","scrub_duration":343,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23972,"num_write_kb":500992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"27'13","reported_seq":57,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:36.250492+0000","last_change":"2024-09-10T21:47:36.250439+0000","last_active":"2024-09-10T21:47:36.250492+0000","last_peered":"2024-09-10T21:47:36.250492+0000","last_clean":"2024-09-10T21:47:36.250492+0000","last_became_active":"2024-09-10T21:24:47.871128+0000","last_became_peered":"2024-09-10T21:24:47.871128+0000","last_unstale":"2024-09-10T21:47:36.250492+0000","last_undegraded":"2024-09-10T21:47:36.250492+0000","last_fullsized":"2024-09-10T21:47:36.250492+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'13","last_scrub_stamp":"2024-09-10T21:47:36.250369+0000","last_deep_scrub":"27'13","last_deep_scrub_stamp":"2024-09-10T21:47:36.250369+0000","last_clean_scrub_stamp":"2024-09-10T21:47:36.250369+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:30:18.379622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"27'3","reported_seq":47,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:25.794215+0000","last_change":"2024-09-10T21:47:25.794215+0000","last_active":"2024-09-10T21:47:25.794215+0000","last_peered":"2024-09-10T21:47:25.794215+0000","last_clean":"2024-09-10T21:47:25.794215+0000","last_became_active":"2024-09-10T21:24:46.853190+0000","last_became_peered":"2024-09-10T21:24:46.853190+0000","last_unstale":"2024-09-10T21:47:25.794215+0000","last_undegraded":"2024-09-10T21:47:25.794215+0000","last_fullsized":"2024-09-10T21:47:25.794215+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'3","last_scrub_stamp":"2024-09-10T21:47:25.794132+0000","last_deep_scrub":"27'3","last_deep_scrub_stamp":"2024-09-10T21:47:25.794132+0000","last_clean_scrub_stamp":"2024-09-10T21:47:25.794132+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:23.090980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"27'74","reported_seq":144,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:26.092159+0000","last_change":"2024-09-10T21:47:26.092159+0000","last_active":"2024-09-10T21:47:26.092159+0000","last_peered":"2024-09-10T21:47:26.092159+0000","last_clean":"2024-09-10T21:47:26.092159+0000","last_became_active":"2024-09-10T21:24:37.215443+0000","last_became_peered":"2024-09-10T21:24:37.215443+0000","last_unstale":"2024-09-10T21:47:26.092159+0000","last_undegraded":"2024-09-10T21:47:26.092159+0000","last_fullsized":"2024-09-10T21:47:26.092159+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'74","last_scrub_stamp":"2024-09-10T21:47:26.092034+0000","last_deep_scrub":"27'74","last_deep_scrub_stamp":"2024-09-10T21:47:26.092034+0000","last_clean_scrub_stamp":"2024-09-10T21:47:26.092034+0000","objects_scrubbed":2,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:58:50.305353+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"27'4394","reported_seq":4431,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:36.913553+0000","last_change":"2024-09-10T21:47:36.913553+0000","last_active":"2024-09-10T21:47:36.913553+0000","last_peered":"2024-09-10T21:47:36.913553+0000","last_clean":"2024-09-10T21:47:36.913553+0000","last_became_active":"2024-09-10T21:24:52.067148+0000","last_became_peered":"2024-09-10T21:24:52.067148+0000","last_unstale":"2024-09-10T21:47:36.913553+0000","last_undegraded":"2024-09-10T21:47:36.913553+0000","last_fullsized":"2024-09-10T21:47:36.913553+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4394","last_scrub_stamp":"2024-09-10T21:47:36.913452+0000","last_deep_scrub":"27'4394","last_deep_scrub_stamp":"2024-09-10T21:47:36.913452+0000","last_clean_scrub_stamp":"2024-09-10T21:47:36.913452+0000","objects_scrubbed":0,"log_size":4394,"log_dups_size":0,"ondisk_log_size":4394,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:46:26.160551+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21702,"num_write_kb":479680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"27'21","reported_seq":70,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:33.040993+0000","last_change":"2024-09-10T21:47:33.040993+0000","last_active":"2024-09-10T21:47:33.040993+0000","last_peered":"2024-09-10T21:47:33.040993+0000","last_clean":"2024-09-10T21:47:33.040993+0000","last_became_active":"2024-09-10T21:25:28.509981+0000","last_became_peered":"2024-09-10T21:25:28.509981+0000","last_unstale":"2024-09-10T21:47:33.040993+0000","last_undegraded":"2024-09-10T21:47:33.040993+0000","last_fullsized":"2024-09-10T21:47:33.040993+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'21","last_scrub_stamp":"2024-09-10T21:47:33.040910+0000","last_deep_scrub":"27'21","last_deep_scrub_stamp":"2024-09-10T21:47:33.040910+0000","last_clean_scrub_stamp":"2024-09-10T21:47:33.040910+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:17:03.746085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"27'5","reported_seq":50,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:24.139970+0000","last_change":"2024-09-10T21:47:24.139970+0000","last_active":"2024-09-10T21:47:24.139970+0000","last_peered":"2024-09-10T21:47:24.139970+0000","last_clean":"2024-09-10T21:47:24.139970+0000","last_became_active":"2024-09-10T21:24:46.859144+0000","last_became_peered":"2024-09-10T21:24:46.859144+0000","last_unstale":"2024-09-10T21:47:24.139970+0000","last_undegraded":"2024-09-10T21:47:24.139970+0000","last_fullsized":"2024-09-10T21:47:24.139970+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'5","last_scrub_stamp":"2024-09-10T21:47:24.139905+0000","last_deep_scrub":"27'5","last_deep_scrub_stamp":"2024-09-10T21:47:24.139905+0000","last_clean_scrub_stamp":"2024-09-10T21:47:24.139905+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:30:29.364591+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"27'4328","reported_seq":4373,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:38.868003+0000","last_change":"2024-09-10T21:47:38.868003+0000","last_active":"2024-09-10T21:47:38.868003+0000","last_peered":"2024-09-10T21:47:38.868003+0000","last_clean":"2024-09-10T21:47:38.868003+0000","last_became_active":"2024-09-10T21:25:28.620415+0000","last_became_peered":"2024-09-10T21:25:28.620415+0000","last_unstale":"2024-09-10T21:47:38.868003+0000","last_undegraded":"2024-09-10T21:47:38.868003+0000","last_fullsized":"2024-09-10T21:47:38.868003+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4328","last_scrub_stamp":"2024-09-10T21:47:38.867904+0000","last_deep_scrub":"27'4328","last_deep_scrub_stamp":"2024-09-10T21:47:38.867904+0000","last_clean_scrub_stamp":"2024-09-10T21:47:38.867904+0000","objects_scrubbed":0,"log_size":4328,"log_dups_size":0,"ondisk_log_size":4328,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:45.990909+0000","scrub_duration":31,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21368,"num_write_kb":498780,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2,0],"acting":[6,7,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"27'17","reported_seq":59,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:30.767144+0000","last_change":"2024-09-10T21:47:30.767144+0000","last_active":"2024-09-10T21:47:30.767144+0000","last_peered":"2024-09-10T21:47:30.767144+0000","last_clean":"2024-09-10T21:47:30.767144+0000","last_became_active":"2024-09-10T21:24:47.866650+0000","last_became_peered":"2024-09-10T21:24:47.866650+0000","last_unstale":"2024-09-10T21:47:30.767144+0000","last_undegraded":"2024-09-10T21:47:30.767144+0000","last_fullsized":"2024-09-10T21:47:30.767144+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'17","last_scrub_stamp":"2024-09-10T21:47:30.767082+0000","last_deep_scrub":"27'17","last_deep_scrub_stamp":"2024-09-10T21:47:30.767082+0000","last_clean_scrub_stamp":"2024-09-10T21:47:30.767082+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:19:12.589710+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"27'7","reported_seq":51,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:23.924308+0000","last_change":"2024-09-10T21:47:23.924308+0000","last_active":"2024-09-10T21:47:23.924308+0000","last_peered":"2024-09-10T21:47:23.924308+0000","last_clean":"2024-09-10T21:47:23.924308+0000","last_became_active":"2024-09-10T21:24:46.856799+0000","last_became_peered":"2024-09-10T21:24:46.856799+0000","last_unstale":"2024-09-10T21:47:23.924308+0000","last_undegraded":"2024-09-10T21:47:23.924308+0000","last_fullsized":"2024-09-10T21:47:23.924308+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'7","last_scrub_stamp":"2024-09-10T21:47:23.924252+0000","last_deep_scrub":"27'7","last_deep_scrub_stamp":"2024-09-10T21:47:23.924252+0000","last_clean_scrub_stamp":"2024-09-10T21:47:23.924252+0000","objects_scrubbed":1,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:30:03.470757+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"27'4625","reported_seq":4662,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:35.159579+0000","last_change":"2024-09-10T21:47:35.159579+0000","last_active":"2024-09-10T21:47:35.159579+0000","last_peered":"2024-09-10T21:47:35.159579+0000","last_clean":"2024-09-10T21:47:35.159579+0000","last_became_active":"2024-09-10T21:24:51.911088+0000","last_became_peered":"2024-09-10T21:24:51.911088+0000","last_unstale":"2024-09-10T21:47:35.159579+0000","last_undegraded":"2024-09-10T21:47:35.159579+0000","last_fullsized":"2024-09-10T21:47:35.159579+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4625","last_scrub_stamp":"2024-09-10T21:47:35.159496+0000","last_deep_scrub":"27'4625","last_deep_scrub_stamp":"2024-09-10T21:47:35.159496+0000","last_clean_scrub_stamp":"2024-09-10T21:47:35.159496+0000","objects_scrubbed":0,"log_size":4625,"log_dups_size":0,"ondisk_log_size":4625,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:21:43.721706+0000","scrub_duration":27,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21860,"num_write_kb":505231,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"27'20","reported_seq":63,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:34.032388+0000","last_change":"2024-09-10T21:47:34.032388+0000","last_active":"2024-09-10T21:47:34.032388+0000","last_peered":"2024-09-10T21:47:34.032388+0000","last_clean":"2024-09-10T21:47:34.032388+0000","last_became_active":"2024-09-10T21:24:47.876690+0000","last_became_peered":"2024-09-10T21:24:47.876690+0000","last_unstale":"2024-09-10T21:47:34.032388+0000","last_undegraded":"2024-09-10T21:47:34.032388+0000","last_fullsized":"2024-09-10T21:47:34.032388+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'20","last_scrub_stamp":"2024-09-10T21:47:34.032266+0000","last_deep_scrub":"27'20","last_deep_scrub_stamp":"2024-09-10T21:47:34.032266+0000","last_clean_scrub_stamp":"2024-09-10T21:47:34.032266+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:46:23.279365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"27'8","reported_seq":53,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:29.204773+0000","last_change":"2024-09-10T21:47:29.204773+0000","last_active":"2024-09-10T21:47:29.204773+0000","last_peered":"2024-09-10T21:47:29.204773+0000","last_clean":"2024-09-10T21:47:29.204773+0000","last_became_active":"2024-09-10T21:24:46.854930+0000","last_became_peered":"2024-09-10T21:24:46.854930+0000","last_unstale":"2024-09-10T21:47:29.204773+0000","last_undegraded":"2024-09-10T21:47:29.204773+0000","last_fullsized":"2024-09-10T21:47:29.204773+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'8","last_scrub_stamp":"2024-09-10T21:47:29.204714+0000","last_deep_scrub":"27'8","last_deep_scrub_stamp":"2024-09-10T21:47:29.204714+0000","last_clean_scrub_stamp":"2024-09-10T21:47:29.204714+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:47:47.401774+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"27'4395","reported_seq":4424,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.566233+0000","last_change":"2024-09-10T21:24:52.063112+0000","last_active":"2024-09-10T21:46:48.566233+0000","last_peered":"2024-09-10T21:46:48.566233+0000","last_clean":"2024-09-10T21:46:48.566233+0000","last_became_active":"2024-09-10T21:24:52.062395+0000","last_became_peered":"2024-09-10T21:24:52.062395+0000","last_unstale":"2024-09-10T21:46:48.566233+0000","last_undegraded":"2024-09-10T21:46:48.566233+0000","last_fullsized":"2024-09-10T21:46:48.566233+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":4395,"log_dups_size":0,"ondisk_log_size":4395,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:21:53.080013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21722,"num_write_kb":475710,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"27'13","reported_seq":41,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:28.230404+0000","last_change":"2024-09-10T21:47:28.230404+0000","last_active":"2024-09-10T21:47:28.230404+0000","last_peered":"2024-09-10T21:47:28.230404+0000","last_clean":"2024-09-10T21:47:28.230404+0000","last_became_active":"2024-09-10T21:25:28.513250+0000","last_became_peered":"2024-09-10T21:25:28.513250+0000","last_unstale":"2024-09-10T21:47:28.230404+0000","last_undegraded":"2024-09-10T21:47:28.230404+0000","last_fullsized":"2024-09-10T21:47:28.230404+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'13","last_scrub_stamp":"2024-09-10T21:47:28.230347+0000","last_deep_scrub":"27'13","last_deep_scrub_stamp":"2024-09-10T21:47:28.230347+0000","last_clean_scrub_stamp":"2024-09-10T21:47:28.230347+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:34:49.010972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.f","version":"27'5","reported_seq":50,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:24.060567+0000","last_change":"2024-09-10T21:47:24.060567+0000","last_active":"2024-09-10T21:47:24.060567+0000","last_peered":"2024-09-10T21:47:24.060567+0000","last_clean":"2024-09-10T21:47:24.060567+0000","last_became_active":"2024-09-10T21:24:46.860053+0000","last_became_peered":"2024-09-10T21:24:46.860053+0000","last_unstale":"2024-09-10T21:47:24.060567+0000","last_undegraded":"2024-09-10T21:47:24.060567+0000","last_fullsized":"2024-09-10T21:47:24.060567+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'5","last_scrub_stamp":"2024-09-10T21:47:24.060492+0000","last_deep_scrub":"27'5","last_deep_scrub_stamp":"2024-09-10T21:47:24.060492+0000","last_clean_scrub_stamp":"2024-09-10T21:47:24.060492+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:17:53.701149+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"27'4437","reported_seq":4474,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:31.262564+0000","last_change":"2024-09-10T21:47:31.262564+0000","last_active":"2024-09-10T21:47:31.262564+0000","last_peered":"2024-09-10T21:47:31.262564+0000","last_clean":"2024-09-10T21:47:31.262564+0000","last_became_active":"2024-09-10T21:24:51.901222+0000","last_became_peered":"2024-09-10T21:24:51.901222+0000","last_unstale":"2024-09-10T21:47:31.262564+0000","last_undegraded":"2024-09-10T21:47:31.262564+0000","last_fullsized":"2024-09-10T21:47:31.262564+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4437","last_scrub_stamp":"2024-09-10T21:47:31.262453+0000","last_deep_scrub":"27'4437","last_deep_scrub_stamp":"2024-09-10T21:47:31.262453+0000","last_clean_scrub_stamp":"2024-09-10T21:47:31.262453+0000","objects_scrubbed":0,"log_size":4437,"log_dups_size":0,"ondisk_log_size":4437,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:48:33.845204+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21706,"num_write_kb":463399,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"27'19","reported_seq":61,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:35.064643+0000","last_change":"2024-09-10T21:47:35.064643+0000","last_active":"2024-09-10T21:47:35.064643+0000","last_peered":"2024-09-10T21:47:35.064643+0000","last_clean":"2024-09-10T21:47:35.064643+0000","last_became_active":"2024-09-10T21:24:47.861834+0000","last_became_peered":"2024-09-10T21:24:47.861834+0000","last_unstale":"2024-09-10T21:47:35.064643+0000","last_undegraded":"2024-09-10T21:47:35.064643+0000","last_fullsized":"2024-09-10T21:47:35.064643+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'19","last_scrub_stamp":"2024-09-10T21:47:35.064536+0000","last_deep_scrub":"27'19","last_deep_scrub_stamp":"2024-09-10T21:47:35.064536+0000","last_clean_scrub_stamp":"2024-09-10T21:47:35.064536+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:42:21.811662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"27'19","reported_seq":63,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:26.189835+0000","last_change":"2024-09-10T21:47:26.189835+0000","last_active":"2024-09-10T21:47:26.189835+0000","last_peered":"2024-09-10T21:47:26.189835+0000","last_clean":"2024-09-10T21:47:26.189835+0000","last_became_active":"2024-09-10T21:24:46.851776+0000","last_became_peered":"2024-09-10T21:24:46.851776+0000","last_unstale":"2024-09-10T21:47:26.189835+0000","last_undegraded":"2024-09-10T21:47:26.189835+0000","last_fullsized":"2024-09-10T21:47:26.189835+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'19","last_scrub_stamp":"2024-09-10T21:47:26.189776+0000","last_deep_scrub":"27'19","last_deep_scrub_stamp":"2024-09-10T21:47:26.189776+0000","last_clean_scrub_stamp":"2024-09-10T21:47:26.189776+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:21:13.166679+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"27'5330","reported_seq":5366,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:37.243051+0000","last_change":"2024-09-10T21:47:37.243051+0000","last_active":"2024-09-10T21:47:37.243051+0000","last_peered":"2024-09-10T21:47:37.243051+0000","last_clean":"2024-09-10T21:47:37.243051+0000","last_became_active":"2024-09-10T21:24:51.903533+0000","last_became_peered":"2024-09-10T21:24:51.903533+0000","last_unstale":"2024-09-10T21:47:37.243051+0000","last_undegraded":"2024-09-10T21:47:37.243051+0000","last_fullsized":"2024-09-10T21:47:37.243051+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'5330","last_scrub_stamp":"2024-09-10T21:47:37.242968+0000","last_deep_scrub":"27'5330","last_deep_scrub_stamp":"2024-09-10T21:47:37.242968+0000","last_clean_scrub_stamp":"2024-09-10T21:47:37.242968+0000","objects_scrubbed":0,"log_size":5330,"log_dups_size":0,"ondisk_log_size":5330,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:33:36.251494+0000","scrub_duration":35,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26446,"num_write_kb":547906,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"27'14","reported_seq":55,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:29.730875+0000","last_change":"2024-09-10T21:47:29.730875+0000","last_active":"2024-09-10T21:47:29.730875+0000","last_peered":"2024-09-10T21:47:29.730875+0000","last_clean":"2024-09-10T21:47:29.730875+0000","last_became_active":"2024-09-10T21:24:47.857437+0000","last_became_peered":"2024-09-10T21:24:47.857437+0000","last_unstale":"2024-09-10T21:47:29.730875+0000","last_undegraded":"2024-09-10T21:47:29.730875+0000","last_fullsized":"2024-09-10T21:47:29.730875+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'14","last_scrub_stamp":"2024-09-10T21:47:29.730790+0000","last_deep_scrub":"27'14","last_deep_scrub_stamp":"2024-09-10T21:47:29.730790+0000","last_clean_scrub_stamp":"2024-09-10T21:47:29.730790+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:18:38.680088+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"27'5","reported_seq":48,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:26.673602+0000","last_change":"2024-09-10T21:47:26.673602+0000","last_active":"2024-09-10T21:47:26.673602+0000","last_peered":"2024-09-10T21:47:26.673602+0000","last_clean":"2024-09-10T21:47:26.673602+0000","last_became_active":"2024-09-10T21:24:46.859796+0000","last_became_peered":"2024-09-10T21:24:46.859796+0000","last_unstale":"2024-09-10T21:47:26.673602+0000","last_undegraded":"2024-09-10T21:47:26.673602+0000","last_fullsized":"2024-09-10T21:47:26.673602+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'5","last_scrub_stamp":"2024-09-10T21:47:26.673501+0000","last_deep_scrub":"27'5","last_deep_scrub_stamp":"2024-09-10T21:47:26.673501+0000","last_clean_scrub_stamp":"2024-09-10T21:47:26.673501+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:31:05.182026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"27'5017","reported_seq":5053,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:32.183555+0000","last_change":"2024-09-10T21:47:32.183555+0000","last_active":"2024-09-10T21:47:32.183555+0000","last_peered":"2024-09-10T21:47:32.183555+0000","last_clean":"2024-09-10T21:47:32.183555+0000","last_became_active":"2024-09-10T21:24:52.067409+0000","last_became_peered":"2024-09-10T21:24:52.067409+0000","last_unstale":"2024-09-10T21:47:32.183555+0000","last_undegraded":"2024-09-10T21:47:32.183555+0000","last_fullsized":"2024-09-10T21:47:32.183555+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'5017","last_scrub_stamp":"2024-09-10T21:47:32.183482+0000","last_deep_scrub":"27'5017","last_deep_scrub_stamp":"2024-09-10T21:47:32.183482+0000","last_clean_scrub_stamp":"2024-09-10T21:47:32.183482+0000","objects_scrubbed":0,"log_size":5017,"log_dups_size":0,"ondisk_log_size":5017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:44:25.121300+0000","scrub_duration":27,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24522,"num_write_kb":548501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"27'30","reported_seq":72,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:37.098706+0000","last_change":"2024-09-10T21:47:37.098706+0000","last_active":"2024-09-10T21:47:37.098706+0000","last_peered":"2024-09-10T21:47:37.098706+0000","last_clean":"2024-09-10T21:47:37.098706+0000","last_became_active":"2024-09-10T21:24:47.858343+0000","last_became_peered":"2024-09-10T21:24:47.858343+0000","last_unstale":"2024-09-10T21:47:37.098706+0000","last_undegraded":"2024-09-10T21:47:37.098706+0000","last_fullsized":"2024-09-10T21:47:37.098706+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'30","last_scrub_stamp":"2024-09-10T21:47:37.098619+0000","last_deep_scrub":"27'30","last_deep_scrub_stamp":"2024-09-10T21:47:37.098619+0000","last_clean_scrub_stamp":"2024-09-10T21:47:37.098619+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:48:11.053398+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"27'3","reported_seq":46,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:26.757436+0000","last_change":"2024-09-10T21:47:26.757436+0000","last_active":"2024-09-10T21:47:26.757436+0000","last_peered":"2024-09-10T21:47:26.757436+0000","last_clean":"2024-09-10T21:47:26.757436+0000","last_became_active":"2024-09-10T21:24:46.858696+0000","last_became_peered":"2024-09-10T21:24:46.858696+0000","last_unstale":"2024-09-10T21:47:26.757436+0000","last_undegraded":"2024-09-10T21:47:26.757436+0000","last_fullsized":"2024-09-10T21:47:26.757436+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'3","last_scrub_stamp":"2024-09-10T21:47:26.757356+0000","last_deep_scrub":"27'3","last_deep_scrub_stamp":"2024-09-10T21:47:26.757356+0000","last_clean_scrub_stamp":"2024-09-10T21:47:26.757356+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:53:05.632605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"27'4314","reported_seq":4358,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:38.290745+0000","last_change":"2024-09-10T21:47:38.290745+0000","last_active":"2024-09-10T21:47:38.290745+0000","last_peered":"2024-09-10T21:47:38.290745+0000","last_clean":"2024-09-10T21:47:38.290745+0000","last_became_active":"2024-09-10T21:25:28.802224+0000","last_became_peered":"2024-09-10T21:25:28.802224+0000","last_unstale":"2024-09-10T21:47:38.290745+0000","last_undegraded":"2024-09-10T21:47:38.290745+0000","last_fullsized":"2024-09-10T21:47:38.290745+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4314","last_scrub_stamp":"2024-09-10T21:47:38.290650+0000","last_deep_scrub":"27'4314","last_deep_scrub_stamp":"2024-09-10T21:47:38.290650+0000","last_clean_scrub_stamp":"2024-09-10T21:47:38.290650+0000","objects_scrubbed":0,"log_size":4314,"log_dups_size":0,"ondisk_log_size":4314,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:39.075208+0000","scrub_duration":31,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20768,"num_write_kb":476051,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7,2],"acting":[3,5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"27'23","reported_seq":65,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:28.196248+0000","last_change":"2024-09-10T21:47:28.196248+0000","last_active":"2024-09-10T21:47:28.196248+0000","last_peered":"2024-09-10T21:47:28.196248+0000","last_clean":"2024-09-10T21:47:28.196248+0000","last_became_active":"2024-09-10T21:24:47.857429+0000","last_became_peered":"2024-09-10T21:24:47.857429+0000","last_unstale":"2024-09-10T21:47:28.196248+0000","last_undegraded":"2024-09-10T21:47:28.196248+0000","last_fullsized":"2024-09-10T21:47:28.196248+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'23","last_scrub_stamp":"2024-09-10T21:47:28.196181+0000","last_deep_scrub":"27'23","last_deep_scrub_stamp":"2024-09-10T21:47:28.196181+0000","last_clean_scrub_stamp":"2024-09-10T21:47:28.196181+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:27:14.251597+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"27'8","reported_seq":52,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:22.166358+0000","last_change":"2024-09-10T21:47:22.166358+0000","last_active":"2024-09-10T21:47:22.166358+0000","last_peered":"2024-09-10T21:47:22.166358+0000","last_clean":"2024-09-10T21:47:22.166358+0000","last_became_active":"2024-09-10T21:24:46.859299+0000","last_became_peered":"2024-09-10T21:24:46.859299+0000","last_unstale":"2024-09-10T21:47:22.166358+0000","last_undegraded":"2024-09-10T21:47:22.166358+0000","last_fullsized":"2024-09-10T21:47:22.166358+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'8","last_scrub_stamp":"2024-09-10T21:47:22.166302+0000","last_deep_scrub":"27'8","last_deep_scrub_stamp":"2024-09-10T21:47:22.166302+0000","last_clean_scrub_stamp":"2024-09-10T21:47:22.166302+0000","objects_scrubbed":2,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:48:22.481684+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"27'4196","reported_seq":4224,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.388436+0000","last_change":"2024-09-10T21:24:52.064718+0000","last_active":"2024-09-10T21:46:48.388436+0000","last_peered":"2024-09-10T21:46:48.388436+0000","last_clean":"2024-09-10T21:46:48.388436+0000","last_became_active":"2024-09-10T21:24:52.062327+0000","last_became_peered":"2024-09-10T21:24:52.062327+0000","last_unstale":"2024-09-10T21:46:48.388436+0000","last_undegraded":"2024-09-10T21:46:48.388436+0000","last_fullsized":"2024-09-10T21:46:48.388436+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":4196,"log_dups_size":0,"ondisk_log_size":4196,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:43:17.173095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20531,"num_write_kb":446498,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"27'18","reported_seq":60,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:36.084330+0000","last_change":"2024-09-10T21:47:36.084330+0000","last_active":"2024-09-10T21:47:36.084330+0000","last_peered":"2024-09-10T21:47:36.084330+0000","last_clean":"2024-09-10T21:47:36.084330+0000","last_became_active":"2024-09-10T21:24:47.870276+0000","last_became_peered":"2024-09-10T21:24:47.870276+0000","last_unstale":"2024-09-10T21:47:36.084330+0000","last_undegraded":"2024-09-10T21:47:36.084330+0000","last_fullsized":"2024-09-10T21:47:36.084330+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'18","last_scrub_stamp":"2024-09-10T21:47:36.084255+0000","last_deep_scrub":"27'18","last_deep_scrub_stamp":"2024-09-10T21:47:36.084255+0000","last_clean_scrub_stamp":"2024-09-10T21:47:36.084255+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:04:31.530529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"27'371","reported_seq":414,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:25.659903+0000","last_change":"2024-09-10T21:47:25.659903+0000","last_active":"2024-09-10T21:47:25.659903+0000","last_peered":"2024-09-10T21:47:25.659903+0000","last_clean":"2024-09-10T21:47:25.659903+0000","last_became_active":"2024-09-10T21:24:46.866899+0000","last_became_peered":"2024-09-10T21:24:46.866899+0000","last_unstale":"2024-09-10T21:47:25.659903+0000","last_undegraded":"2024-09-10T21:47:25.659903+0000","last_fullsized":"2024-09-10T21:47:25.659903+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'371","last_scrub_stamp":"2024-09-10T21:47:25.659813+0000","last_deep_scrub":"27'371","last_deep_scrub_stamp":"2024-09-10T21:47:25.659813+0000","last_clean_scrub_stamp":"2024-09-10T21:47:25.659813+0000","objects_scrubbed":4,"log_size":371,"log_dups_size":0,"ondisk_log_size":371,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:17:10.717140+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194976,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":367,"num_write_kb":4285,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"27'3971","reported_seq":4007,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:35.949172+0000","last_change":"2024-09-10T21:47:35.949172+0000","last_active":"2024-09-10T21:47:35.949172+0000","last_peered":"2024-09-10T21:47:35.949172+0000","last_clean":"2024-09-10T21:47:35.949172+0000","last_became_active":"2024-09-10T21:24:51.895398+0000","last_became_peered":"2024-09-10T21:24:51.895398+0000","last_unstale":"2024-09-10T21:47:35.949172+0000","last_undegraded":"2024-09-10T21:47:35.949172+0000","last_fullsized":"2024-09-10T21:47:35.949172+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'3971","last_scrub_stamp":"2024-09-10T21:47:35.949076+0000","last_deep_scrub":"27'3971","last_deep_scrub_stamp":"2024-09-10T21:47:35.949076+0000","last_clean_scrub_stamp":"2024-09-10T21:47:35.949076+0000","objects_scrubbed":0,"log_size":3971,"log_dups_size":0,"ondisk_log_size":3971,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:13:29.247480+0000","scrub_duration":31,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19337,"num_write_kb":445978,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"27'22","reported_seq":64,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:28.142815+0000","last_change":"2024-09-10T21:47:28.142815+0000","last_active":"2024-09-10T21:47:28.142815+0000","last_peered":"2024-09-10T21:47:28.142815+0000","last_clean":"2024-09-10T21:47:28.142815+0000","last_became_active":"2024-09-10T21:24:47.857284+0000","last_became_peered":"2024-09-10T21:24:47.857284+0000","last_unstale":"2024-09-10T21:47:28.142815+0000","last_undegraded":"2024-09-10T21:47:28.142815+0000","last_fullsized":"2024-09-10T21:47:28.142815+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'22","last_scrub_stamp":"2024-09-10T21:47:28.142739+0000","last_deep_scrub":"27'22","last_deep_scrub_stamp":"2024-09-10T21:47:28.142739+0000","last_clean_scrub_stamp":"2024-09-10T21:47:28.142739+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:47:52.403509+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"27'34","reported_seq":77,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:25.963747+0000","last_change":"2024-09-10T21:47:25.963747+0000","last_active":"2024-09-10T21:47:25.963747+0000","last_peered":"2024-09-10T21:47:25.963747+0000","last_clean":"2024-09-10T21:47:25.963747+0000","last_became_active":"2024-09-10T21:24:46.859332+0000","last_became_peered":"2024-09-10T21:24:46.859332+0000","last_unstale":"2024-09-10T21:47:25.963747+0000","last_undegraded":"2024-09-10T21:47:25.963747+0000","last_fullsized":"2024-09-10T21:47:25.963747+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'34","last_scrub_stamp":"2024-09-10T21:47:25.963678+0000","last_deep_scrub":"27'34","last_deep_scrub_stamp":"2024-09-10T21:47:25.963678+0000","last_clean_scrub_stamp":"2024-09-10T21:47:25.963678+0000","objects_scrubbed":2,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:34:01.110519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":33,"num_write_kb":30,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"27'4136","reported_seq":4172,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:36.212386+0000","last_change":"2024-09-10T21:47:36.212386+0000","last_active":"2024-09-10T21:47:36.212386+0000","last_peered":"2024-09-10T21:47:36.212386+0000","last_clean":"2024-09-10T21:47:36.212386+0000","last_became_active":"2024-09-10T21:24:51.898642+0000","last_became_peered":"2024-09-10T21:24:51.898642+0000","last_unstale":"2024-09-10T21:47:36.212386+0000","last_undegraded":"2024-09-10T21:47:36.212386+0000","last_fullsized":"2024-09-10T21:47:36.212386+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4136","last_scrub_stamp":"2024-09-10T21:47:36.212285+0000","last_deep_scrub":"27'4136","last_deep_scrub_stamp":"2024-09-10T21:47:36.212285+0000","last_clean_scrub_stamp":"2024-09-10T21:47:36.212285+0000","objects_scrubbed":0,"log_size":4136,"log_dups_size":0,"ondisk_log_size":4136,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:46:44.829087+0000","scrub_duration":27,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20561,"num_write_kb":446825,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"27'23","reported_seq":65,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:25.114417+0000","last_change":"2024-09-10T21:47:25.114417+0000","last_active":"2024-09-10T21:47:25.114417+0000","last_peered":"2024-09-10T21:47:25.114417+0000","last_clean":"2024-09-10T21:47:25.114417+0000","last_became_active":"2024-09-10T21:24:47.865016+0000","last_became_peered":"2024-09-10T21:24:47.865016+0000","last_unstale":"2024-09-10T21:47:25.114417+0000","last_undegraded":"2024-09-10T21:47:25.114417+0000","last_fullsized":"2024-09-10T21:47:25.114417+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'23","last_scrub_stamp":"2024-09-10T21:47:25.114353+0000","last_deep_scrub":"27'23","last_deep_scrub_stamp":"2024-09-10T21:47:25.114353+0000","last_clean_scrub_stamp":"2024-09-10T21:47:25.114353+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:57:06.271532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"27'3","reported_seq":46,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:27.775157+0000","last_change":"2024-09-10T21:47:27.775157+0000","last_active":"2024-09-10T21:47:27.775157+0000","last_peered":"2024-09-10T21:47:27.775157+0000","last_clean":"2024-09-10T21:47:27.775157+0000","last_became_active":"2024-09-10T21:24:46.861136+0000","last_became_peered":"2024-09-10T21:24:46.861136+0000","last_unstale":"2024-09-10T21:47:27.775157+0000","last_undegraded":"2024-09-10T21:47:27.775157+0000","last_fullsized":"2024-09-10T21:47:27.775157+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'3","last_scrub_stamp":"2024-09-10T21:47:27.775078+0000","last_deep_scrub":"27'3","last_deep_scrub_stamp":"2024-09-10T21:47:27.775078+0000","last_clean_scrub_stamp":"2024-09-10T21:47:27.775078+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:27:13.830494+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"27'4156","reported_seq":4184,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:46:48.591271+0000","last_change":"2024-09-10T21:24:51.907850+0000","last_active":"2024-09-10T21:46:48.591271+0000","last_peered":"2024-09-10T21:46:48.591271+0000","last_clean":"2024-09-10T21:46:48.591271+0000","last_became_active":"2024-09-10T21:24:51.907062+0000","last_became_peered":"2024-09-10T21:24:51.907062+0000","last_unstale":"2024-09-10T21:46:48.591271+0000","last_undegraded":"2024-09-10T21:46:48.591271+0000","last_fullsized":"2024-09-10T21:46:48.591271+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:50.859019+0000","last_clean_scrub_stamp":"2024-09-10T21:24:50.859019+0000","objects_scrubbed":0,"log_size":4156,"log_dups_size":0,"ondisk_log_size":4156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:32:52.519627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20153,"num_write_kb":457162,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"27'25","reported_seq":66,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:31.735901+0000","last_change":"2024-09-10T21:47:31.735901+0000","last_active":"2024-09-10T21:47:31.735901+0000","last_peered":"2024-09-10T21:47:31.735901+0000","last_clean":"2024-09-10T21:47:31.735901+0000","last_became_active":"2024-09-10T21:24:47.861498+0000","last_became_peered":"2024-09-10T21:24:47.861498+0000","last_unstale":"2024-09-10T21:47:31.735901+0000","last_undegraded":"2024-09-10T21:47:31.735901+0000","last_fullsized":"2024-09-10T21:47:31.735901+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'25","last_scrub_stamp":"2024-09-10T21:47:31.735802+0000","last_deep_scrub":"27'25","last_deep_scrub_stamp":"2024-09-10T21:47:31.735802+0000","last_clean_scrub_stamp":"2024-09-10T21:47:31.735802+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:47:25.653045+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"27'3","reported_seq":46,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:27.718580+0000","last_change":"2024-09-10T21:47:27.718580+0000","last_active":"2024-09-10T21:47:27.718580+0000","last_peered":"2024-09-10T21:47:27.718580+0000","last_clean":"2024-09-10T21:47:27.718580+0000","last_became_active":"2024-09-10T21:24:46.871367+0000","last_became_peered":"2024-09-10T21:24:46.871367+0000","last_unstale":"2024-09-10T21:47:27.718580+0000","last_undegraded":"2024-09-10T21:47:27.718580+0000","last_fullsized":"2024-09-10T21:47:27.718580+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'3","last_scrub_stamp":"2024-09-10T21:47:27.718477+0000","last_deep_scrub":"27'3","last_deep_scrub_stamp":"2024-09-10T21:47:27.718477+0000","last_clean_scrub_stamp":"2024-09-10T21:47:27.718477+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:30:07.264982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"27'3798","reported_seq":3834,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:31.159945+0000","last_change":"2024-09-10T21:47:31.159945+0000","last_active":"2024-09-10T21:47:31.159945+0000","last_peered":"2024-09-10T21:47:31.159945+0000","last_clean":"2024-09-10T21:47:31.159945+0000","last_became_active":"2024-09-10T21:24:52.068348+0000","last_became_peered":"2024-09-10T21:24:52.068348+0000","last_unstale":"2024-09-10T21:47:31.159945+0000","last_undegraded":"2024-09-10T21:47:31.159945+0000","last_fullsized":"2024-09-10T21:47:31.159945+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'3798","last_scrub_stamp":"2024-09-10T21:47:31.159867+0000","last_deep_scrub":"27'3798","last_deep_scrub_stamp":"2024-09-10T21:47:31.159867+0000","last_clean_scrub_stamp":"2024-09-10T21:47:31.159867+0000","objects_scrubbed":0,"log_size":3798,"log_dups_size":0,"ondisk_log_size":3798,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:18:09.732689+0000","scrub_duration":27,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18329,"num_write_kb":408933,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"27'5643","reported_seq":5681,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:38.270407+0000","last_change":"2024-09-10T21:47:38.270407+0000","last_active":"2024-09-10T21:47:38.270407+0000","last_peered":"2024-09-10T21:47:38.270407+0000","last_clean":"2024-09-10T21:47:38.270407+0000","last_became_active":"2024-09-10T21:24:52.067330+0000","last_became_peered":"2024-09-10T21:24:52.067330+0000","last_unstale":"2024-09-10T21:47:38.270407+0000","last_undegraded":"2024-09-10T21:47:38.270407+0000","last_fullsized":"2024-09-10T21:47:38.270407+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'5643","last_scrub_stamp":"2024-09-10T21:47:38.270269+0000","last_deep_scrub":"27'5643","last_deep_scrub_stamp":"2024-09-10T21:47:38.270269+0000","last_clean_scrub_stamp":"2024-09-10T21:47:38.270269+0000","objects_scrubbed":0,"log_size":5643,"log_dups_size":0,"ondisk_log_size":5643,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:09.431599+0000","scrub_duration":35,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27908,"num_write_kb":566060,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"27'20","reported_seq":61,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:32.964137+0000","last_change":"2024-09-10T21:47:32.964137+0000","last_active":"2024-09-10T21:47:32.964137+0000","last_peered":"2024-09-10T21:47:32.964137+0000","last_clean":"2024-09-10T21:47:32.964137+0000","last_became_active":"2024-09-10T21:24:47.862556+0000","last_became_peered":"2024-09-10T21:24:47.862556+0000","last_unstale":"2024-09-10T21:47:32.964137+0000","last_undegraded":"2024-09-10T21:47:32.964137+0000","last_fullsized":"2024-09-10T21:47:32.964137+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'20","last_scrub_stamp":"2024-09-10T21:47:32.964062+0000","last_deep_scrub":"27'20","last_deep_scrub_stamp":"2024-09-10T21:47:32.964062+0000","last_clean_scrub_stamp":"2024-09-10T21:47:32.964062+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:46:22.211162+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"27'2","reported_seq":45,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:28.728520+0000","last_change":"2024-09-10T21:47:28.728520+0000","last_active":"2024-09-10T21:47:28.728520+0000","last_peered":"2024-09-10T21:47:28.728520+0000","last_clean":"2024-09-10T21:47:28.728520+0000","last_became_active":"2024-09-10T21:24:46.862540+0000","last_became_peered":"2024-09-10T21:24:46.862540+0000","last_unstale":"2024-09-10T21:47:28.728520+0000","last_undegraded":"2024-09-10T21:47:28.728520+0000","last_fullsized":"2024-09-10T21:47:28.728520+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2","last_scrub_stamp":"2024-09-10T21:47:28.728443+0000","last_deep_scrub":"27'2","last_deep_scrub_stamp":"2024-09-10T21:47:28.728443+0000","last_clean_scrub_stamp":"2024-09-10T21:47:28.728443+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:21:14.145793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"27'5506","reported_seq":5543,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:37.918860+0000","last_change":"2024-09-10T21:47:37.918860+0000","last_active":"2024-09-10T21:47:37.918860+0000","last_peered":"2024-09-10T21:47:37.918860+0000","last_clean":"2024-09-10T21:47:37.918860+0000","last_became_active":"2024-09-10T21:24:51.900610+0000","last_became_peered":"2024-09-10T21:24:51.900610+0000","last_unstale":"2024-09-10T21:47:37.918860+0000","last_undegraded":"2024-09-10T21:47:37.918860+0000","last_fullsized":"2024-09-10T21:47:37.918860+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'5506","last_scrub_stamp":"2024-09-10T21:47:37.918758+0000","last_deep_scrub":"27'5506","last_deep_scrub_stamp":"2024-09-10T21:47:37.918758+0000","last_clean_scrub_stamp":"2024-09-10T21:47:37.918758+0000","objects_scrubbed":0,"log_size":5506,"log_dups_size":0,"ondisk_log_size":5506,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:54:41.833391+0000","scrub_duration":39,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27800,"num_write_kb":594222,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"27'26","reported_seq":68,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:34.200053+0000","last_change":"2024-09-10T21:47:34.200053+0000","last_active":"2024-09-10T21:47:34.200053+0000","last_peered":"2024-09-10T21:47:34.200053+0000","last_clean":"2024-09-10T21:47:34.200053+0000","last_became_active":"2024-09-10T21:24:47.860930+0000","last_became_peered":"2024-09-10T21:24:47.860930+0000","last_unstale":"2024-09-10T21:47:34.200053+0000","last_undegraded":"2024-09-10T21:47:34.200053+0000","last_fullsized":"2024-09-10T21:47:34.200053+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'26","last_scrub_stamp":"2024-09-10T21:47:34.199983+0000","last_deep_scrub":"27'26","last_deep_scrub_stamp":"2024-09-10T21:47:34.199983+0000","last_clean_scrub_stamp":"2024-09-10T21:47:34.199983+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:35:02.658141+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"27'3","reported_seq":47,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:40.256898+0000","last_change":"2024-09-10T21:47:40.256898+0000","last_active":"2024-09-10T21:47:40.256898+0000","last_peered":"2024-09-10T21:47:40.256898+0000","last_clean":"2024-09-10T21:47:40.256898+0000","last_became_active":"2024-09-10T21:24:46.858265+0000","last_became_peered":"2024-09-10T21:24:46.858265+0000","last_unstale":"2024-09-10T21:47:40.256898+0000","last_undegraded":"2024-09-10T21:47:40.256898+0000","last_fullsized":"2024-09-10T21:47:40.256898+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'3","last_scrub_stamp":"2024-09-10T21:47:40.256832+0000","last_deep_scrub":"27'3","last_deep_scrub_stamp":"2024-09-10T21:47:40.256832+0000","last_clean_scrub_stamp":"2024-09-10T21:47:40.256832+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:21:12.866707+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"27'6","reported_seq":49,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:28.689308+0000","last_change":"2024-09-10T21:47:28.689308+0000","last_active":"2024-09-10T21:47:28.689308+0000","last_peered":"2024-09-10T21:47:28.689308+0000","last_clean":"2024-09-10T21:47:28.689308+0000","last_became_active":"2024-09-10T21:24:46.856506+0000","last_became_peered":"2024-09-10T21:24:46.856506+0000","last_unstale":"2024-09-10T21:47:28.689308+0000","last_undegraded":"2024-09-10T21:47:28.689308+0000","last_fullsized":"2024-09-10T21:47:28.689308+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'6","last_scrub_stamp":"2024-09-10T21:47:28.689227+0000","last_deep_scrub":"27'6","last_deep_scrub_stamp":"2024-09-10T21:47:28.689227+0000","last_clean_scrub_stamp":"2024-09-10T21:47:28.689227+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:16:18.854160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"27'19","reported_seq":61,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:29.197942+0000","last_change":"2024-09-10T21:47:29.197942+0000","last_active":"2024-09-10T21:47:29.197942+0000","last_peered":"2024-09-10T21:47:29.197942+0000","last_clean":"2024-09-10T21:47:29.197942+0000","last_became_active":"2024-09-10T21:24:47.848160+0000","last_became_peered":"2024-09-10T21:24:47.848160+0000","last_unstale":"2024-09-10T21:47:29.197942+0000","last_undegraded":"2024-09-10T21:47:29.197942+0000","last_fullsized":"2024-09-10T21:47:29.197942+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'19","last_scrub_stamp":"2024-09-10T21:47:29.197887+0000","last_deep_scrub":"27'19","last_deep_scrub_stamp":"2024-09-10T21:47:29.197887+0000","last_clean_scrub_stamp":"2024-09-10T21:47:29.197887+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:55.411968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"27'4174","reported_seq":4210,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:32.267288+0000","last_change":"2024-09-10T21:47:32.267288+0000","last_active":"2024-09-10T21:47:32.267288+0000","last_peered":"2024-09-10T21:47:32.267288+0000","last_clean":"2024-09-10T21:47:32.267288+0000","last_became_active":"2024-09-10T21:24:51.903120+0000","last_became_peered":"2024-09-10T21:24:51.903120+0000","last_unstale":"2024-09-10T21:47:32.267288+0000","last_undegraded":"2024-09-10T21:47:32.267288+0000","last_fullsized":"2024-09-10T21:47:32.267288+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4174","last_scrub_stamp":"2024-09-10T21:47:32.267187+0000","last_deep_scrub":"27'4174","last_deep_scrub_stamp":"2024-09-10T21:47:32.267187+0000","last_clean_scrub_stamp":"2024-09-10T21:47:32.267187+0000","objects_scrubbed":0,"log_size":4174,"log_dups_size":0,"ondisk_log_size":4174,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:02:46.924965+0000","scrub_duration":27,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19938,"num_write_kb":435851,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"27'679","reported_seq":723,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:23.326615+0000","last_change":"2024-09-10T21:47:23.326615+0000","last_active":"2024-09-10T21:47:23.326615+0000","last_peered":"2024-09-10T21:47:23.326615+0000","last_clean":"2024-09-10T21:47:23.326615+0000","last_became_active":"2024-09-10T21:24:46.852204+0000","last_became_peered":"2024-09-10T21:24:46.852204+0000","last_unstale":"2024-09-10T21:47:23.326615+0000","last_undegraded":"2024-09-10T21:47:23.326615+0000","last_fullsized":"2024-09-10T21:47:23.326615+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'679","last_scrub_stamp":"2024-09-10T21:47:23.326553+0000","last_deep_scrub":"27'679","last_deep_scrub_stamp":"2024-09-10T21:47:23.326553+0000","last_clean_scrub_stamp":"2024-09-10T21:47:23.326553+0000","objects_scrubbed":1,"log_size":679,"log_dups_size":0,"ondisk_log_size":679,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:48:25.909303+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":672,"num_write_kb":4259,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"27'18","reported_seq":60,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:27.127018+0000","last_change":"2024-09-10T21:47:27.127018+0000","last_active":"2024-09-10T21:47:27.127018+0000","last_peered":"2024-09-10T21:47:27.127018+0000","last_clean":"2024-09-10T21:47:27.127018+0000","last_became_active":"2024-09-10T21:24:47.848851+0000","last_became_peered":"2024-09-10T21:24:47.848851+0000","last_unstale":"2024-09-10T21:47:27.127018+0000","last_undegraded":"2024-09-10T21:47:27.127018+0000","last_fullsized":"2024-09-10T21:47:27.127018+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'18","last_scrub_stamp":"2024-09-10T21:47:27.126943+0000","last_deep_scrub":"27'18","last_deep_scrub_stamp":"2024-09-10T21:47:27.126943+0000","last_clean_scrub_stamp":"2024-09-10T21:47:27.126943+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:02:51.834510+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"27'4902","reported_seq":4938,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:41.334091+0000","last_change":"2024-09-10T21:47:41.334091+0000","last_active":"2024-09-10T21:47:41.334091+0000","last_peered":"2024-09-10T21:47:41.334091+0000","last_clean":"2024-09-10T21:47:41.334091+0000","last_became_active":"2024-09-10T21:24:51.907082+0000","last_became_peered":"2024-09-10T21:24:51.907082+0000","last_unstale":"2024-09-10T21:47:41.334091+0000","last_undegraded":"2024-09-10T21:47:41.334091+0000","last_fullsized":"2024-09-10T21:47:41.334091+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4902","last_scrub_stamp":"2024-09-10T21:47:41.334007+0000","last_deep_scrub":"27'4902","last_deep_scrub_stamp":"2024-09-10T21:47:41.334007+0000","last_clean_scrub_stamp":"2024-09-10T21:47:41.334007+0000","objects_scrubbed":0,"log_size":4902,"log_dups_size":0,"ondisk_log_size":4902,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:31:24.648339+0000","scrub_duration":35,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24324,"num_write_kb":547512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"27'24","reported_seq":66,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:31.763913+0000","last_change":"2024-09-10T21:47:31.763913+0000","last_active":"2024-09-10T21:47:31.763913+0000","last_peered":"2024-09-10T21:47:31.763913+0000","last_clean":"2024-09-10T21:47:31.763913+0000","last_became_active":"2024-09-10T21:24:47.857501+0000","last_became_peered":"2024-09-10T21:24:47.857501+0000","last_unstale":"2024-09-10T21:47:31.763913+0000","last_undegraded":"2024-09-10T21:47:31.763913+0000","last_fullsized":"2024-09-10T21:47:31.763913+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'24","last_scrub_stamp":"2024-09-10T21:47:31.763837+0000","last_deep_scrub":"27'24","last_deep_scrub_stamp":"2024-09-10T21:47:31.763837+0000","last_clean_scrub_stamp":"2024-09-10T21:47:31.763837+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:06:18.700323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"27'25","reported_seq":68,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:25.158155+0000","last_change":"2024-09-10T21:47:25.158155+0000","last_active":"2024-09-10T21:47:25.158155+0000","last_peered":"2024-09-10T21:47:25.158155+0000","last_clean":"2024-09-10T21:47:25.158155+0000","last_became_active":"2024-09-10T21:24:47.867504+0000","last_became_peered":"2024-09-10T21:24:47.867504+0000","last_unstale":"2024-09-10T21:47:25.158155+0000","last_undegraded":"2024-09-10T21:47:25.158155+0000","last_fullsized":"2024-09-10T21:47:25.158155+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'25","last_scrub_stamp":"2024-09-10T21:47:25.158094+0000","last_deep_scrub":"27'25","last_deep_scrub_stamp":"2024-09-10T21:47:25.158094+0000","last_clean_scrub_stamp":"2024-09-10T21:47:25.158094+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:21:12.715415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"27'26","reported_seq":68,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:34.655927+0000","last_change":"2024-09-10T21:47:34.655927+0000","last_active":"2024-09-10T21:47:34.655927+0000","last_peered":"2024-09-10T21:47:34.655927+0000","last_clean":"2024-09-10T21:47:34.655927+0000","last_became_active":"2024-09-10T21:24:47.868168+0000","last_became_peered":"2024-09-10T21:24:47.868168+0000","last_unstale":"2024-09-10T21:47:34.655927+0000","last_undegraded":"2024-09-10T21:47:34.655927+0000","last_fullsized":"2024-09-10T21:47:34.655927+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'26","last_scrub_stamp":"2024-09-10T21:47:34.655825+0000","last_deep_scrub":"27'26","last_deep_scrub_stamp":"2024-09-10T21:47:34.655825+0000","last_clean_scrub_stamp":"2024-09-10T21:47:34.655825+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:17:12.913285+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"27'16","reported_seq":59,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:31.200690+0000","last_change":"2024-09-10T21:47:31.200690+0000","last_active":"2024-09-10T21:47:31.200690+0000","last_peered":"2024-09-10T21:47:31.200690+0000","last_clean":"2024-09-10T21:47:31.200690+0000","last_became_active":"2024-09-10T21:24:47.864100+0000","last_became_peered":"2024-09-10T21:24:47.864100+0000","last_unstale":"2024-09-10T21:47:31.200690+0000","last_undegraded":"2024-09-10T21:47:31.200690+0000","last_fullsized":"2024-09-10T21:47:31.200690+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'16","last_scrub_stamp":"2024-09-10T21:47:31.200632+0000","last_deep_scrub":"27'16","last_deep_scrub_stamp":"2024-09-10T21:47:31.200632+0000","last_clean_scrub_stamp":"2024-09-10T21:47:31.200632+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:35:01.716923+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"27'14","reported_seq":57,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:33.184046+0000","last_change":"2024-09-10T21:47:33.184046+0000","last_active":"2024-09-10T21:47:33.184046+0000","last_peered":"2024-09-10T21:47:33.184046+0000","last_clean":"2024-09-10T21:47:33.184046+0000","last_became_active":"2024-09-10T21:24:47.863970+0000","last_became_peered":"2024-09-10T21:24:47.863970+0000","last_unstale":"2024-09-10T21:47:33.184046+0000","last_undegraded":"2024-09-10T21:47:33.184046+0000","last_fullsized":"2024-09-10T21:47:33.184046+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'14","last_scrub_stamp":"2024-09-10T21:47:33.183974+0000","last_deep_scrub":"27'14","last_deep_scrub_stamp":"2024-09-10T21:47:33.183974+0000","last_clean_scrub_stamp":"2024-09-10T21:47:33.183974+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:32:36.049392+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"27'18","reported_seq":60,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:28.972824+0000","last_change":"2024-09-10T21:47:28.972824+0000","last_active":"2024-09-10T21:47:28.972824+0000","last_peered":"2024-09-10T21:47:28.972824+0000","last_clean":"2024-09-10T21:47:28.972824+0000","last_became_active":"2024-09-10T21:24:47.860471+0000","last_became_peered":"2024-09-10T21:24:47.860471+0000","last_unstale":"2024-09-10T21:47:28.972824+0000","last_undegraded":"2024-09-10T21:47:28.972824+0000","last_fullsized":"2024-09-10T21:47:28.972824+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'18","last_scrub_stamp":"2024-09-10T21:47:28.972767+0000","last_deep_scrub":"27'18","last_deep_scrub_stamp":"2024-09-10T21:47:28.972767+0000","last_clean_scrub_stamp":"2024-09-10T21:47:28.972767+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:52:26.453545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"27'27","reported_seq":69,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:32.802778+0000","last_change":"2024-09-10T21:47:32.802778+0000","last_active":"2024-09-10T21:47:32.802778+0000","last_peered":"2024-09-10T21:47:32.802778+0000","last_clean":"2024-09-10T21:47:32.802778+0000","last_became_active":"2024-09-10T21:24:47.872218+0000","last_became_peered":"2024-09-10T21:24:47.872218+0000","last_unstale":"2024-09-10T21:47:32.802778+0000","last_undegraded":"2024-09-10T21:47:32.802778+0000","last_fullsized":"2024-09-10T21:47:32.802778+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'27","last_scrub_stamp":"2024-09-10T21:47:32.802683+0000","last_deep_scrub":"27'27","last_deep_scrub_stamp":"2024-09-10T21:47:32.802683+0000","last_clean_scrub_stamp":"2024-09-10T21:47:32.802683+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:52:30.283461+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"27'14","reported_seq":57,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:29.184527+0000","last_change":"2024-09-10T21:47:29.184527+0000","last_active":"2024-09-10T21:47:29.184527+0000","last_peered":"2024-09-10T21:47:29.184527+0000","last_clean":"2024-09-10T21:47:29.184527+0000","last_became_active":"2024-09-10T21:24:47.859649+0000","last_became_peered":"2024-09-10T21:24:47.859649+0000","last_unstale":"2024-09-10T21:47:29.184527+0000","last_undegraded":"2024-09-10T21:47:29.184527+0000","last_fullsized":"2024-09-10T21:47:29.184527+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'14","last_scrub_stamp":"2024-09-10T21:47:29.184462+0000","last_deep_scrub":"27'14","last_deep_scrub_stamp":"2024-09-10T21:47:29.184462+0000","last_clean_scrub_stamp":"2024-09-10T21:47:29.184462+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:51:19.452278+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"27'18","reported_seq":47,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:27.212647+0000","last_change":"2024-09-10T21:47:27.212647+0000","last_active":"2024-09-10T21:47:27.212647+0000","last_peered":"2024-09-10T21:47:27.212647+0000","last_clean":"2024-09-10T21:47:27.212647+0000","last_became_active":"2024-09-10T21:25:28.508684+0000","last_became_peered":"2024-09-10T21:25:28.508684+0000","last_unstale":"2024-09-10T21:47:27.212647+0000","last_undegraded":"2024-09-10T21:47:27.212647+0000","last_fullsized":"2024-09-10T21:47:27.212647+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'18","last_scrub_stamp":"2024-09-10T21:47:27.212575+0000","last_deep_scrub":"27'18","last_deep_scrub_stamp":"2024-09-10T21:47:27.212575+0000","last_clean_scrub_stamp":"2024-09-10T21:47:27.212575+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:42.041087+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.29","version":"27'23","reported_seq":51,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:30.198230+0000","last_change":"2024-09-10T21:47:30.198230+0000","last_active":"2024-09-10T21:47:30.198230+0000","last_peered":"2024-09-10T21:47:30.198230+0000","last_clean":"2024-09-10T21:47:30.198230+0000","last_became_active":"2024-09-10T21:25:28.512230+0000","last_became_peered":"2024-09-10T21:25:28.512230+0000","last_unstale":"2024-09-10T21:47:30.198230+0000","last_undegraded":"2024-09-10T21:47:30.198230+0000","last_fullsized":"2024-09-10T21:47:30.198230+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'23","last_scrub_stamp":"2024-09-10T21:47:30.198142+0000","last_deep_scrub":"27'23","last_deep_scrub_stamp":"2024-09-10T21:47:30.198142+0000","last_clean_scrub_stamp":"2024-09-10T21:47:30.198142+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:46:41.750896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2a","version":"27'22","reported_seq":64,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:35.656763+0000","last_change":"2024-09-10T21:47:35.656763+0000","last_active":"2024-09-10T21:47:35.656763+0000","last_peered":"2024-09-10T21:47:35.656763+0000","last_clean":"2024-09-10T21:47:35.656763+0000","last_became_active":"2024-09-10T21:24:47.868671+0000","last_became_peered":"2024-09-10T21:24:47.868671+0000","last_unstale":"2024-09-10T21:47:35.656763+0000","last_undegraded":"2024-09-10T21:47:35.656763+0000","last_fullsized":"2024-09-10T21:47:35.656763+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'22","last_scrub_stamp":"2024-09-10T21:47:35.656676+0000","last_deep_scrub":"27'22","last_deep_scrub_stamp":"2024-09-10T21:47:35.656676+0000","last_clean_scrub_stamp":"2024-09-10T21:47:35.656676+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:00:48.878921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"27'28","reported_seq":71,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:32.171947+0000","last_change":"2024-09-10T21:47:32.171947+0000","last_active":"2024-09-10T21:47:32.171947+0000","last_peered":"2024-09-10T21:47:32.171947+0000","last_clean":"2024-09-10T21:47:32.171947+0000","last_became_active":"2024-09-10T21:24:47.857116+0000","last_became_peered":"2024-09-10T21:24:47.857116+0000","last_unstale":"2024-09-10T21:47:32.171947+0000","last_undegraded":"2024-09-10T21:47:32.171947+0000","last_fullsized":"2024-09-10T21:47:32.171947+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'28","last_scrub_stamp":"2024-09-10T21:47:32.171855+0000","last_deep_scrub":"27'28","last_deep_scrub_stamp":"2024-09-10T21:47:32.171855+0000","last_clean_scrub_stamp":"2024-09-10T21:47:32.171855+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:12:14.615828+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"27'19","reported_seq":61,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:33.830699+0000","last_change":"2024-09-10T21:47:33.830699+0000","last_active":"2024-09-10T21:47:33.830699+0000","last_peered":"2024-09-10T21:47:33.830699+0000","last_clean":"2024-09-10T21:47:33.830699+0000","last_became_active":"2024-09-10T21:24:47.871350+0000","last_became_peered":"2024-09-10T21:24:47.871350+0000","last_unstale":"2024-09-10T21:47:33.830699+0000","last_undegraded":"2024-09-10T21:47:33.830699+0000","last_fullsized":"2024-09-10T21:47:33.830699+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'19","last_scrub_stamp":"2024-09-10T21:47:33.830616+0000","last_deep_scrub":"27'19","last_deep_scrub_stamp":"2024-09-10T21:47:33.830616+0000","last_clean_scrub_stamp":"2024-09-10T21:47:33.830616+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:07.813564+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"27'20","reported_seq":63,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:38.071694+0000","last_change":"2024-09-10T21:47:38.071694+0000","last_active":"2024-09-10T21:47:38.071694+0000","last_peered":"2024-09-10T21:47:38.071694+0000","last_clean":"2024-09-10T21:47:38.071694+0000","last_became_active":"2024-09-10T21:24:47.861007+0000","last_became_peered":"2024-09-10T21:24:47.861007+0000","last_unstale":"2024-09-10T21:47:38.071694+0000","last_undegraded":"2024-09-10T21:47:38.071694+0000","last_fullsized":"2024-09-10T21:47:38.071694+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'20","last_scrub_stamp":"2024-09-10T21:47:38.071574+0000","last_deep_scrub":"27'20","last_deep_scrub_stamp":"2024-09-10T21:47:38.071574+0000","last_clean_scrub_stamp":"2024-09-10T21:47:38.071574+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:56:10.916482+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"27'27","reported_seq":69,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:31.194852+0000","last_change":"2024-09-10T21:47:31.194852+0000","last_active":"2024-09-10T21:47:31.194852+0000","last_peered":"2024-09-10T21:47:31.194852+0000","last_clean":"2024-09-10T21:47:31.194852+0000","last_became_active":"2024-09-10T21:24:47.863932+0000","last_became_peered":"2024-09-10T21:24:47.863932+0000","last_unstale":"2024-09-10T21:47:31.194852+0000","last_undegraded":"2024-09-10T21:47:31.194852+0000","last_fullsized":"2024-09-10T21:47:31.194852+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'27","last_scrub_stamp":"2024-09-10T21:47:31.194797+0000","last_deep_scrub":"27'27","last_deep_scrub_stamp":"2024-09-10T21:47:31.194797+0000","last_clean_scrub_stamp":"2024-09-10T21:47:31.194797+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:38:09.333593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"27'21","reported_seq":62,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:36.620568+0000","last_change":"2024-09-10T21:47:36.620568+0000","last_active":"2024-09-10T21:47:36.620568+0000","last_peered":"2024-09-10T21:47:36.620568+0000","last_clean":"2024-09-10T21:47:36.620568+0000","last_became_active":"2024-09-10T21:24:47.865587+0000","last_became_peered":"2024-09-10T21:24:47.865587+0000","last_unstale":"2024-09-10T21:47:36.620568+0000","last_undegraded":"2024-09-10T21:47:36.620568+0000","last_fullsized":"2024-09-10T21:47:36.620568+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'21","last_scrub_stamp":"2024-09-10T21:47:36.620488+0000","last_deep_scrub":"27'21","last_deep_scrub_stamp":"2024-09-10T21:47:36.620488+0000","last_clean_scrub_stamp":"2024-09-10T21:47:36.620488+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:04.850738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"27'22","reported_seq":63,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:37.664031+0000","last_change":"2024-09-10T21:47:37.664031+0000","last_active":"2024-09-10T21:47:37.664031+0000","last_peered":"2024-09-10T21:47:37.664031+0000","last_clean":"2024-09-10T21:47:37.664031+0000","last_became_active":"2024-09-10T21:24:47.865837+0000","last_became_peered":"2024-09-10T21:24:47.865837+0000","last_unstale":"2024-09-10T21:47:37.664031+0000","last_undegraded":"2024-09-10T21:47:37.664031+0000","last_fullsized":"2024-09-10T21:47:37.664031+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'22","last_scrub_stamp":"2024-09-10T21:47:37.663943+0000","last_deep_scrub":"27'22","last_deep_scrub_stamp":"2024-09-10T21:47:37.663943+0000","last_clean_scrub_stamp":"2024-09-10T21:47:37.663943+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:22:59.372202+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"27'21","reported_seq":62,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:34.966079+0000","last_change":"2024-09-10T21:47:34.966079+0000","last_active":"2024-09-10T21:47:34.966079+0000","last_peered":"2024-09-10T21:47:34.966079+0000","last_clean":"2024-09-10T21:47:34.966079+0000","last_became_active":"2024-09-10T21:24:47.846283+0000","last_became_peered":"2024-09-10T21:24:47.846283+0000","last_unstale":"2024-09-10T21:47:34.966079+0000","last_undegraded":"2024-09-10T21:47:34.966079+0000","last_fullsized":"2024-09-10T21:47:34.966079+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'21","last_scrub_stamp":"2024-09-10T21:47:34.966019+0000","last_deep_scrub":"27'21","last_deep_scrub_stamp":"2024-09-10T21:47:34.966019+0000","last_clean_scrub_stamp":"2024-09-10T21:47:34.966019+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:42.089024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"27'23","reported_seq":65,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:30.197763+0000","last_change":"2024-09-10T21:47:30.197763+0000","last_active":"2024-09-10T21:47:30.197763+0000","last_peered":"2024-09-10T21:47:30.197763+0000","last_clean":"2024-09-10T21:47:30.197763+0000","last_became_active":"2024-09-10T21:24:47.844517+0000","last_became_peered":"2024-09-10T21:24:47.844517+0000","last_unstale":"2024-09-10T21:47:30.197763+0000","last_undegraded":"2024-09-10T21:47:30.197763+0000","last_fullsized":"2024-09-10T21:47:30.197763+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'23","last_scrub_stamp":"2024-09-10T21:47:30.197688+0000","last_deep_scrub":"27'23","last_deep_scrub_stamp":"2024-09-10T21:47:30.197688+0000","last_clean_scrub_stamp":"2024-09-10T21:47:30.197688+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:18:03.294904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"27'25","reported_seq":67,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:28.212053+0000","last_change":"2024-09-10T21:47:28.212053+0000","last_active":"2024-09-10T21:47:28.212053+0000","last_peered":"2024-09-10T21:47:28.212053+0000","last_clean":"2024-09-10T21:47:28.212053+0000","last_became_active":"2024-09-10T21:24:47.862185+0000","last_became_peered":"2024-09-10T21:24:47.862185+0000","last_unstale":"2024-09-10T21:47:28.212053+0000","last_undegraded":"2024-09-10T21:47:28.212053+0000","last_fullsized":"2024-09-10T21:47:28.212053+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'25","last_scrub_stamp":"2024-09-10T21:47:28.211979+0000","last_deep_scrub":"27'25","last_deep_scrub_stamp":"2024-09-10T21:47:28.211979+0000","last_clean_scrub_stamp":"2024-09-10T21:47:28.211979+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:46:36.828782+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"27'17","reported_seq":59,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:29.186874+0000","last_change":"2024-09-10T21:47:29.186874+0000","last_active":"2024-09-10T21:47:29.186874+0000","last_peered":"2024-09-10T21:47:29.186874+0000","last_clean":"2024-09-10T21:47:29.186874+0000","last_became_active":"2024-09-10T21:24:47.861045+0000","last_became_peered":"2024-09-10T21:24:47.861045+0000","last_unstale":"2024-09-10T21:47:29.186874+0000","last_undegraded":"2024-09-10T21:47:29.186874+0000","last_fullsized":"2024-09-10T21:47:29.186874+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'17","last_scrub_stamp":"2024-09-10T21:47:29.186783+0000","last_deep_scrub":"27'17","last_deep_scrub_stamp":"2024-09-10T21:47:29.186783+0000","last_clean_scrub_stamp":"2024-09-10T21:47:29.186783+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:12:34.155979+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"27'17","reported_seq":58,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:35.849939+0000","last_change":"2024-09-10T21:47:35.849939+0000","last_active":"2024-09-10T21:47:35.849939+0000","last_peered":"2024-09-10T21:47:35.849939+0000","last_clean":"2024-09-10T21:47:35.849939+0000","last_became_active":"2024-09-10T21:24:47.867113+0000","last_became_peered":"2024-09-10T21:24:47.867113+0000","last_unstale":"2024-09-10T21:47:35.849939+0000","last_undegraded":"2024-09-10T21:47:35.849939+0000","last_fullsized":"2024-09-10T21:47:35.849939+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'17","last_scrub_stamp":"2024-09-10T21:47:35.849852+0000","last_deep_scrub":"27'17","last_deep_scrub_stamp":"2024-09-10T21:47:35.849852+0000","last_clean_scrub_stamp":"2024-09-10T21:47:35.849852+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:17:06.555027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"27'28","reported_seq":69,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:34.873759+0000","last_change":"2024-09-10T21:47:34.873759+0000","last_active":"2024-09-10T21:47:34.873759+0000","last_peered":"2024-09-10T21:47:34.873759+0000","last_clean":"2024-09-10T21:47:34.873759+0000","last_became_active":"2024-09-10T21:24:47.867126+0000","last_became_peered":"2024-09-10T21:24:47.867126+0000","last_unstale":"2024-09-10T21:47:34.873759+0000","last_undegraded":"2024-09-10T21:47:34.873759+0000","last_fullsized":"2024-09-10T21:47:34.873759+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'28","last_scrub_stamp":"2024-09-10T21:47:34.873672+0000","last_deep_scrub":"27'28","last_deep_scrub_stamp":"2024-09-10T21:47:34.873672+0000","last_clean_scrub_stamp":"2024-09-10T21:47:34.873672+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:38:35.516740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"27'26","reported_seq":68,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:39.051777+0000","last_change":"2024-09-10T21:47:39.051777+0000","last_active":"2024-09-10T21:47:39.051777+0000","last_peered":"2024-09-10T21:47:39.051777+0000","last_clean":"2024-09-10T21:47:39.051777+0000","last_became_active":"2024-09-10T21:24:47.865093+0000","last_became_peered":"2024-09-10T21:24:47.865093+0000","last_unstale":"2024-09-10T21:47:39.051777+0000","last_undegraded":"2024-09-10T21:47:39.051777+0000","last_fullsized":"2024-09-10T21:47:39.051777+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'26","last_scrub_stamp":"2024-09-10T21:47:39.051693+0000","last_deep_scrub":"27'26","last_deep_scrub_stamp":"2024-09-10T21:47:39.051693+0000","last_clean_scrub_stamp":"2024-09-10T21:47:39.051693+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:54:36.357498+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"27'15","reported_seq":57,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:37.277986+0000","last_change":"2024-09-10T21:47:37.277986+0000","last_active":"2024-09-10T21:47:37.277986+0000","last_peered":"2024-09-10T21:47:37.277986+0000","last_clean":"2024-09-10T21:47:37.277986+0000","last_became_active":"2024-09-10T21:24:47.867782+0000","last_became_peered":"2024-09-10T21:24:47.867782+0000","last_unstale":"2024-09-10T21:47:37.277986+0000","last_undegraded":"2024-09-10T21:47:37.277986+0000","last_fullsized":"2024-09-10T21:47:37.277986+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'15","last_scrub_stamp":"2024-09-10T21:47:37.277930+0000","last_deep_scrub":"27'15","last_deep_scrub_stamp":"2024-09-10T21:47:37.277930+0000","last_clean_scrub_stamp":"2024-09-10T21:47:37.277930+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:24:45.621244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"27'13","reported_seq":55,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:40.088088+0000","last_change":"2024-09-10T21:47:40.088088+0000","last_active":"2024-09-10T21:47:40.088088+0000","last_peered":"2024-09-10T21:47:40.088088+0000","last_clean":"2024-09-10T21:47:40.088088+0000","last_became_active":"2024-09-10T21:24:47.861434+0000","last_became_peered":"2024-09-10T21:24:47.861434+0000","last_unstale":"2024-09-10T21:47:40.088088+0000","last_undegraded":"2024-09-10T21:47:40.088088+0000","last_fullsized":"2024-09-10T21:47:40.088088+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'13","last_scrub_stamp":"2024-09-10T21:47:40.088009+0000","last_deep_scrub":"27'13","last_deep_scrub_stamp":"2024-09-10T21:47:40.088009+0000","last_clean_scrub_stamp":"2024-09-10T21:47:40.088009+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:41:12.540656+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"27'14","reported_seq":55,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:38.629094+0000","last_change":"2024-09-10T21:47:38.629094+0000","last_active":"2024-09-10T21:47:38.629094+0000","last_peered":"2024-09-10T21:47:38.629094+0000","last_clean":"2024-09-10T21:47:38.629094+0000","last_became_active":"2024-09-10T21:24:47.868503+0000","last_became_peered":"2024-09-10T21:24:47.868503+0000","last_unstale":"2024-09-10T21:47:38.629094+0000","last_undegraded":"2024-09-10T21:47:38.629094+0000","last_fullsized":"2024-09-10T21:47:38.629094+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'14","last_scrub_stamp":"2024-09-10T21:47:38.629019+0000","last_deep_scrub":"27'14","last_deep_scrub_stamp":"2024-09-10T21:47:38.629019+0000","last_clean_scrub_stamp":"2024-09-10T21:47:38.629019+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:32:41.494438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"27'15","reported_seq":57,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:41.099538+0000","last_change":"2024-09-10T21:47:41.099538+0000","last_active":"2024-09-10T21:47:41.099538+0000","last_peered":"2024-09-10T21:47:41.099538+0000","last_clean":"2024-09-10T21:47:41.099538+0000","last_became_active":"2024-09-10T21:24:47.863976+0000","last_became_peered":"2024-09-10T21:24:47.863976+0000","last_unstale":"2024-09-10T21:47:41.099538+0000","last_undegraded":"2024-09-10T21:47:41.099538+0000","last_fullsized":"2024-09-10T21:47:41.099538+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'15","last_scrub_stamp":"2024-09-10T21:47:41.099448+0000","last_deep_scrub":"27'15","last_deep_scrub_stamp":"2024-09-10T21:47:41.099448+0000","last_clean_scrub_stamp":"2024-09-10T21:47:41.099448+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:27:00.251650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"27'13","reported_seq":48,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:11.347483+0000","last_change":"2024-09-10T21:24:47.873364+0000","last_active":"2024-09-10T21:47:11.347483+0000","last_peered":"2024-09-10T21:47:11.347483+0000","last_clean":"2024-09-10T21:47:11.347483+0000","last_became_active":"2024-09-10T21:24:47.873069+0000","last_became_peered":"2024-09-10T21:24:47.873069+0000","last_unstale":"2024-09-10T21:47:11.347483+0000","last_undegraded":"2024-09-10T21:47:11.347483+0000","last_fullsized":"2024-09-10T21:47:11.347483+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T21:24:46.824885+0000","last_clean_scrub_stamp":"2024-09-10T21:24:46.824885+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:38:21.368499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"27'20","reported_seq":64,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:33.193083+0000","last_change":"2024-09-10T21:47:33.193037+0000","last_active":"2024-09-10T21:47:33.193083+0000","last_peered":"2024-09-10T21:47:33.193083+0000","last_clean":"2024-09-10T21:47:33.193083+0000","last_became_active":"2024-09-10T21:24:47.854713+0000","last_became_peered":"2024-09-10T21:24:47.854713+0000","last_unstale":"2024-09-10T21:47:33.193083+0000","last_undegraded":"2024-09-10T21:47:33.193083+0000","last_fullsized":"2024-09-10T21:47:33.193083+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'20","last_scrub_stamp":"2024-09-10T21:47:33.192982+0000","last_deep_scrub":"27'20","last_deep_scrub_stamp":"2024-09-10T21:47:33.192982+0000","last_clean_scrub_stamp":"2024-09-10T21:47:33.192982+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:46:41.809785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"27'10","reported_seq":52,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:27.971496+0000","last_change":"2024-09-10T21:47:27.971496+0000","last_active":"2024-09-10T21:47:27.971496+0000","last_peered":"2024-09-10T21:47:27.971496+0000","last_clean":"2024-09-10T21:47:27.971496+0000","last_became_active":"2024-09-10T21:24:47.846227+0000","last_became_peered":"2024-09-10T21:24:47.846227+0000","last_unstale":"2024-09-10T21:47:27.971496+0000","last_undegraded":"2024-09-10T21:47:27.971496+0000","last_fullsized":"2024-09-10T21:47:27.971496+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'10","last_scrub_stamp":"2024-09-10T21:47:27.971425+0000","last_deep_scrub":"27'10","last_deep_scrub_stamp":"2024-09-10T21:47:27.971425+0000","last_clean_scrub_stamp":"2024-09-10T21:47:27.971425+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:06:14.907911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"27'15","reported_seq":58,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:42.103093+0000","last_change":"2024-09-10T21:47:42.103093+0000","last_active":"2024-09-10T21:47:42.103093+0000","last_peered":"2024-09-10T21:47:42.103093+0000","last_clean":"2024-09-10T21:47:42.103093+0000","last_became_active":"2024-09-10T21:24:47.871177+0000","last_became_peered":"2024-09-10T21:24:47.871177+0000","last_unstale":"2024-09-10T21:47:42.103093+0000","last_undegraded":"2024-09-10T21:47:42.103093+0000","last_fullsized":"2024-09-10T21:47:42.103093+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'15","last_scrub_stamp":"2024-09-10T21:47:42.103014+0000","last_deep_scrub":"27'15","last_deep_scrub_stamp":"2024-09-10T21:47:42.103014+0000","last_clean_scrub_stamp":"2024-09-10T21:47:42.103014+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:53:50.812078+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":723657,"num_write_kb":15994026,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":148509,"ondisk_log_size":148509,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1239,"ondisk_log_size":1239,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":14991163,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":48,"num_read_kb":119,"num_write":3049,"num_write_kb":15770,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":18464768,"data_stored":29982326,"data_compressed":2564296,"data_compressed_allocated":3506176,"data_compressed_original":15089664,"omap_allocated":0,"internal_metadata":0},"log_size":2782,"ondisk_log_size":2782,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":66932,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":74,"ondisk_log_size":74,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509719,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":739600,"kb_used_data":7424,"kb_used_omap":1,"kb_used_meta":732158,"kb_avail":93632240,"statfs":{"total":96636764160,"available":95879413760,"internally_reserved":0,"allocated":7602176,"data_stored":11412495,"data_compressed":806609,"data_compressed_allocated":1097728,"data_compressed_original":5083136,"omap_allocated":1590,"internal_metadata":749730250},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509720,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":807000,"kb_used_data":2120,"kb_used_omap":1,"kb_used_meta":804862,"kb_avail":93564840,"statfs":{"total":96636764160,"available":95810396160,"internally_reserved":0,"allocated":2170880,"data_stored":4311586,"data_compressed":503209,"data_compressed_allocated":692224,"data_compressed_original":3006464,"omap_allocated":1594,"internal_metadata":824179142},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509719,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":573484,"kb_used_data":348,"kb_used_omap":1,"kb_used_meta":573118,"kb_avail":93798356,"statfs":{"total":96636764160,"available":96049516544,"internally_reserved":0,"allocated":356352,"data_stored":207064,"data_compressed":5796,"data_compressed_allocated":8192,"data_compressed_original":28672,"omap_allocated":1595,"internal_metadata":586873285},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509717,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1104044,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":1103742,"kb_avail":93267796,"statfs":{"total":96636764160,"available":95506223104,"internally_reserved":0,"allocated":290816,"data_stored":116770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":1130232261},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542424,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":844396,"kb_used_data":3740,"kb_used_omap":1,"kb_used_meta":840638,"kb_avail":93527444,"statfs":{"total":96636764160,"available":95772102656,"internally_reserved":0,"allocated":3829760,"data_stored":4311040,"data_compressed":154551,"data_compressed_allocated":245760,"data_compressed_original":905216,"omap_allocated":1595,"internal_metadata":860813765},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542419,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1080968,"kb_used_data":4792,"kb_used_omap":1,"kb_used_meta":1076158,"kb_avail":93290872,"statfs":{"total":96636764160,"available":95529852928,"internally_reserved":0,"allocated":4907008,"data_stored":8596344,"data_compressed":841246,"data_compressed_allocated":1130496,"data_compressed_original":4997120,"omap_allocated":1593,"internal_metadata":1101986247},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542425,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":711220,"kb_used_data":1380,"kb_used_omap":1,"kb_used_meta":709822,"kb_avail":93660620,"statfs":{"total":96636764160,"available":95908474880,"internally_reserved":0,"allocated":1413120,"data_stored":3023943,"data_compressed":319817,"data_compressed_allocated":421888,"data_compressed_original":2215936,"omap_allocated":1595,"internal_metadata":726858181},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575127,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":626792,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":626494,"kb_avail":93745048,"statfs":{"total":96636764160,"available":95994929152,"internally_reserved":0,"allocated":286720,"data_stored":116700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":641530309},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33466,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33466,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":1064960,"data_stored":2316965,"data_compressed":286351,"data_compressed_allocated":376832,"data_compressed_original":1642496,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":90454,"data_compressed":5796,"data_compressed_allocated":8192,"data_compressed_original":28672,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4624384,"data_stored":8479734,"data_compressed":841246,"data_compressed_allocated":1130496,"data_compressed_original":4997120,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":3547136,"data_stored":4194430,"data_compressed":154551,"data_compressed_allocated":245760,"data_compressed_original":905216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":1888256,"data_stored":4194976,"data_compressed":503209,"data_compressed_allocated":692224,"data_compressed_original":3006464,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":7254016,"data_stored":10705517,"data_compressed":773143,"data_compressed_allocated":1052672,"data_compressed_original":4509696,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T21:47:46.055 INFO:tasks.ceph:pgid 4.8 last_scrub_stamp 2024-09-10T21:24:50.859019+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=24, tm_sec=50, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=47, tm_sec=19, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T21:47:46.055 INFO:tasks.ceph:pgid 4.15 last_scrub_stamp 2024-09-10T21:24:50.859019+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=24, tm_sec=50, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=47, tm_sec=19, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T21:47:46.055 INFO:tasks.ceph:pgid 4.10 last_scrub_stamp 2024-09-10T21:24:50.859019+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=24, tm_sec=50, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=47, tm_sec=19, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T21:47:46.056 INFO:tasks.ceph:pgid 3.3c last_scrub_stamp 2024-09-10T21:24:46.824885+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=24, tm_sec=46, tm_wday=1, tm_yday=254, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=10, tm_hour=21, tm_min=47, tm_sec=19, tm_wday=1, tm_yday=254, tm_isdst=0) 2024-09-10T21:47:46.056 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-10T21:48:06.057 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-10T21:48:06.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:06.200+0000 7ff8fca00640 1 -- 172.21.15.17:0/2346268488 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff8f8152820 msgr2=0x7ff8f80b3eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:48:06.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:06.200+0000 7ff8fca00640 1 --2- 172.21.15.17:0/2346268488 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff8f8152820 0x7ff8f80b3eb0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7ff8f80c9850 tx=0x7ff8e001c830 comp rx=0 tx=0).stop 2024-09-10T21:48:06.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:06.200+0000 7ff8fca00640 1 -- 172.21.15.17:0/2346268488 shutdown_connections 2024-09-10T21:48:06.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:06.200+0000 7ff8fca00640 1 --2- 172.21.15.17:0/2346268488 >> v2:172.21.15.31:3300/0 conn(0x7ff8f80cdf20 0x7ff8f80ce350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:06.205 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:06.200+0000 7ff8fca00640 1 --2- 172.21.15.17:0/2346268488 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff8f8152820 0x7ff8f80b3eb0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:06.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:06.200+0000 7ff8fca00640 1 -- 172.21.15.17:0/2346268488 >> 172.21.15.17:0/2346268488 conn(0x7ff8f80702f0 msgr2=0x7ff8f8070700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:48:06.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:06.200+0000 7ff8fca00640 1 -- 172.21.15.17:0/2346268488 shutdown_connections 2024-09-10T21:48:06.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:06.200+0000 7ff8fca00640 1 -- 172.21.15.17:0/2346268488 wait complete. 2024-09-10T21:48:06.206 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:06.200+0000 7ff8fca00640 1 Processor -- start 2024-09-10T21:48:06.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:06.200+0000 7ff8fca00640 1 -- start start 2024-09-10T21:48:06.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:06.200+0000 7ff8fca00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff8f80cdf20 0x7ff8f8157380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:48:06.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:06.200+0000 7ff8fca00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff8f8152440 0x7ff8f814aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:48:06.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:06.200+0000 7ff8fca00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff8f8152820 0x7ff8f80c6780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:48:06.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:06.200+0000 7ff8fca00640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7ff8f80cb5d0 con 0x7ff8f8152820 2024-09-10T21:48:06.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:06.200+0000 7ff8fca00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7ff8f80cb450 con 0x7ff8f8152440 2024-09-10T21:48:06.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:06.200+0000 7ff8fca00640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7ff8f80c2ff0 con 0x7ff8f80cdf20 2024-09-10T21:48:06.209 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:06.204+0000 7ff8f5e00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff8f80cdf20 0x7ff8f8157380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:48:06.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:06.204+0000 7ff8f6800640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff8f8152820 0x7ff8f80c6780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:48:06.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:06.204+0000 7ff8f5e00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff8f80cdf20 0x7ff8f8157380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.31:3301/0 says I am v2:172.21.15.17:33938/0 (socket says 172.21.15.17:33938) 2024-09-10T21:48:06.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:06.204+0000 7ff8f5e00640 1 -- 172.21.15.17:0/3359689826 learned_addr learned my addr 172.21.15.17:0/3359689826 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:48:06.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:06.204+0000 7ff8ed400640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff8f8152440 0x7ff8f814aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:48:06.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:06.204+0000 7ff8f6800640 1 -- 172.21.15.17:0/3359689826 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff8f80cdf20 msgr2=0x7ff8f8157380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:48:06.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:06.204+0000 7ff8f6800640 1 --2- 172.21.15.17:0/3359689826 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff8f80cdf20 0x7ff8f8157380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:06.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:06.204+0000 7ff8f6800640 1 -- 172.21.15.17:0/3359689826 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff8f8152440 msgr2=0x7ff8f814aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:48:06.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:06.204+0000 7ff8f6800640 1 --2- 172.21.15.17:0/3359689826 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff8f8152440 0x7ff8f814aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:06.210 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:06.204+0000 7ff8f6800640 1 -- 172.21.15.17:0/3359689826 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8f81578c0 con 0x7ff8f8152820 2024-09-10T21:48:06.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:06.204+0000 7ff8f5e00640 1 --2- 172.21.15.17:0/3359689826 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff8f80cdf20 0x7ff8f8157380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T21:48:06.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:06.204+0000 7ff8f6800640 1 --2- 172.21.15.17:0/3359689826 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff8f8152820 0x7ff8f80c6780 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7ff8f000e9c0 tx=0x7ff8f000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:48:06.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:06.204+0000 7ff8eea00640 1 -- 172.21.15.17:0/3359689826 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8f0004d30 con 0x7ff8f8152820 2024-09-10T21:48:06.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:06.204+0000 7ff8eea00640 1 -- 172.21.15.17:0/3359689826 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff8f000cb20 con 0x7ff8f8152820 2024-09-10T21:48:06.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:06.204+0000 7ff8eea00640 1 -- 172.21.15.17:0/3359689826 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8f0005020 con 0x7ff8f8152820 2024-09-10T21:48:06.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:06.204+0000 7ff8fca00640 1 -- 172.21.15.17:0/3359689826 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8f80c7610 con 0x7ff8f8152820 2024-09-10T21:48:06.211 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:06.204+0000 7ff8fca00640 1 -- 172.21.15.17:0/3359689826 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8f80cd440 con 0x7ff8f8152820 2024-09-10T21:48:06.212 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:06.204+0000 7ff8fca00640 1 -- 172.21.15.17:0/3359689826 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8f80b38f0 con 0x7ff8f8152820 2024-09-10T21:48:06.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:06.204+0000 7ff8eea00640 1 -- 172.21.15.17:0/3359689826 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7ff8f000f040 con 0x7ff8f8152820 2024-09-10T21:48:06.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:06.208+0000 7ff8eea00640 1 --2- 172.21.15.17:0/3359689826 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7ff8d40725e0 0x7ff8d4074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:48:06.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:06.208+0000 7ff8eea00640 1 -- 172.21.15.17:0/3359689826 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7ff8f0093b00 con 0x7ff8f8152820 2024-09-10T21:48:06.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:06.208+0000 7ff8f5e00640 1 --2- 172.21.15.17:0/3359689826 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7ff8d40725e0 0x7ff8d4074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:48:06.213 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:06.208+0000 7ff8f5e00640 1 --2- 172.21.15.17:0/3359689826 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7ff8d40725e0 0x7ff8d4074aa0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7ff8e8004470 tx=0x7ff8e8003700 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:48:06.215 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:06.212+0000 7ff8eea00640 1 -- 172.21.15.17:0/3359689826 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7ff8f0016030 con 0x7ff8f8152820 2024-09-10T21:48:06.389 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:06.384+0000 7ff8fca00640 1 -- 172.21.15.17:0/3359689826 --> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff8f80c8bf0 con 0x7ff8d40725e0 2024-09-10T21:48:06.395 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:06.388+0000 7ff8eea00640 1 -- 172.21.15.17:0/3359689826 <== mgr.4100 v2:172.21.15.31:6808/3101308586 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+334643 (secure 0 0 0) 0x7ff8f80c8bf0 con 0x7ff8d40725e0 2024-09-10T21:48:06.398 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-10T21:48:06.400 INFO:teuthology.orchestra.run.smithi017.stderr:dumped all 2024-09-10T21:48:06.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:06.392+0000 7ff8fca00640 1 -- 172.21.15.17:0/3359689826 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7ff8d40725e0 msgr2=0x7ff8d4074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:48:06.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:06.392+0000 7ff8fca00640 1 --2- 172.21.15.17:0/3359689826 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7ff8d40725e0 0x7ff8d4074aa0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7ff8e8004470 tx=0x7ff8e8003700 comp rx=0 tx=0).stop 2024-09-10T21:48:06.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:06.392+0000 7ff8fca00640 1 -- 172.21.15.17:0/3359689826 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff8f8152820 msgr2=0x7ff8f80c6780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:48:06.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:06.392+0000 7ff8fca00640 1 --2- 172.21.15.17:0/3359689826 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff8f8152820 0x7ff8f80c6780 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7ff8f000e9c0 tx=0x7ff8f000ee80 comp rx=0 tx=0).stop 2024-09-10T21:48:06.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:06.396+0000 7ff8fca00640 1 -- 172.21.15.17:0/3359689826 shutdown_connections 2024-09-10T21:48:06.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:06.396+0000 7ff8fca00640 1 --2- 172.21.15.17:0/3359689826 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7ff8d40725e0 0x7ff8d4074aa0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:06.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:06.396+0000 7ff8fca00640 1 --2- 172.21.15.17:0/3359689826 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7ff8f8152820 0x7ff8f80c6780 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:06.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:06.396+0000 7ff8fca00640 1 --2- 172.21.15.17:0/3359689826 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7ff8f8152440 0x7ff8f814aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:06.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:06.396+0000 7ff8fca00640 1 --2- 172.21.15.17:0/3359689826 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7ff8f80cdf20 0x7ff8f8157380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:06.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:06.396+0000 7ff8fca00640 1 -- 172.21.15.17:0/3359689826 >> 172.21.15.17:0/3359689826 conn(0x7ff8f80702f0 msgr2=0x7ff8f8162070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:48:06.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:06.396+0000 7ff8fca00640 1 -- 172.21.15.17:0/3359689826 shutdown_connections 2024-09-10T21:48:06.402 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:06.396+0000 7ff8fca00640 1 -- 172.21.15.17:0/3359689826 wait complete. 2024-09-10T21:48:06.420 INFO:teuthology.orchestra.run.smithi017.stdout:{"pg_ready":true,"pg_map":{"version":726,"stamp":"2024-09-10T21:48:05.999627+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":15581531,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":114,"num_read_kb":174,"num_write":726807,"num_write_kb":16011424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":152604,"ondisk_log_size":152604,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":6487504,"kb_used_data":20368,"kb_used_omap":12,"kb_used_meta":6466995,"kb_avail":748487216,"statfs":{"total":773094113280,"available":766450909184,"internally_reserved":0,"allocated":20856832,"data_stored":32095942,"data_compressed":2631228,"data_compressed_allocated":3596288,"data_compressed_original":16236544,"omap_allocated":12752,"internal_metadata":6622203440},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003639"},"pg_stats":[{"pgid":"2.1f","version":"27'180","reported_seq":226,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:23.131813+0000","last_change":"2024-09-10T21:47:23.131765+0000","last_active":"2024-09-10T21:47:23.131813+0000","last_peered":"2024-09-10T21:47:23.131813+0000","last_clean":"2024-09-10T21:47:23.131813+0000","last_became_active":"2024-09-10T21:24:46.864288+0000","last_became_peered":"2024-09-10T21:24:46.864288+0000","last_unstale":"2024-09-10T21:47:23.131813+0000","last_undegraded":"2024-09-10T21:47:23.131813+0000","last_fullsized":"2024-09-10T21:47:23.131813+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'180","last_scrub_stamp":"2024-09-10T21:47:23.131706+0000","last_deep_scrub":"27'180","last_deep_scrub_stamp":"2024-09-10T21:47:23.131706+0000","last_clean_scrub_stamp":"2024-09-10T21:47:23.131706+0000","objects_scrubbed":3,"log_size":180,"log_dups_size":0,"ondisk_log_size":180,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:20:08.917966+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2316909,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":173,"num_write_kb":2361,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"27'12","reported_seq":56,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:27.217140+0000","last_change":"2024-09-10T21:47:27.217140+0000","last_active":"2024-09-10T21:47:27.217140+0000","last_peered":"2024-09-10T21:47:27.217140+0000","last_clean":"2024-09-10T21:47:27.217140+0000","last_became_active":"2024-09-10T21:24:47.864168+0000","last_became_peered":"2024-09-10T21:24:47.864168+0000","last_unstale":"2024-09-10T21:47:27.217140+0000","last_undegraded":"2024-09-10T21:47:27.217140+0000","last_fullsized":"2024-09-10T21:47:27.217140+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'12","last_scrub_stamp":"2024-09-10T21:47:27.217080+0000","last_deep_scrub":"27'12","last_deep_scrub_stamp":"2024-09-10T21:47:27.217080+0000","last_clean_scrub_stamp":"2024-09-10T21:47:27.217080+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:19:01.897786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"27'5290","reported_seq":5326,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:35.264827+0000","last_change":"2024-09-10T21:47:35.264827+0000","last_active":"2024-09-10T21:47:35.264827+0000","last_peered":"2024-09-10T21:47:35.264827+0000","last_clean":"2024-09-10T21:47:35.264827+0000","last_became_active":"2024-09-10T21:24:52.067258+0000","last_became_peered":"2024-09-10T21:24:52.067258+0000","last_unstale":"2024-09-10T21:47:35.264827+0000","last_undegraded":"2024-09-10T21:47:35.264827+0000","last_fullsized":"2024-09-10T21:47:35.264827+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'5290","last_scrub_stamp":"2024-09-10T21:47:35.264733+0000","last_deep_scrub":"27'5290","last_deep_scrub_stamp":"2024-09-10T21:47:35.264733+0000","last_clean_scrub_stamp":"2024-09-10T21:47:35.264733+0000","objects_scrubbed":0,"log_size":5290,"log_dups_size":0,"ondisk_log_size":5290,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:50:31.338570+0000","scrub_duration":31,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25332,"num_write_kb":555711,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"27'5","reported_seq":51,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:24.263539+0000","last_change":"2024-09-10T21:47:24.263539+0000","last_active":"2024-09-10T21:47:24.263539+0000","last_peered":"2024-09-10T21:47:24.263539+0000","last_clean":"2024-09-10T21:47:24.263539+0000","last_became_active":"2024-09-10T21:24:46.860341+0000","last_became_peered":"2024-09-10T21:24:46.860341+0000","last_unstale":"2024-09-10T21:47:24.263539+0000","last_undegraded":"2024-09-10T21:47:24.263539+0000","last_fullsized":"2024-09-10T21:47:24.263539+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'5","last_scrub_stamp":"2024-09-10T21:47:24.263477+0000","last_deep_scrub":"27'5","last_deep_scrub_stamp":"2024-09-10T21:47:24.263477+0000","last_clean_scrub_stamp":"2024-09-10T21:47:24.263477+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:50.477559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"27'19","reported_seq":63,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:27.150543+0000","last_change":"2024-09-10T21:47:27.150495+0000","last_active":"2024-09-10T21:47:27.150543+0000","last_peered":"2024-09-10T21:47:27.150543+0000","last_clean":"2024-09-10T21:47:27.150543+0000","last_became_active":"2024-09-10T21:24:47.865365+0000","last_became_peered":"2024-09-10T21:24:47.865365+0000","last_unstale":"2024-09-10T21:47:27.150543+0000","last_undegraded":"2024-09-10T21:47:27.150543+0000","last_fullsized":"2024-09-10T21:47:27.150543+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'19","last_scrub_stamp":"2024-09-10T21:47:27.150439+0000","last_deep_scrub":"27'19","last_deep_scrub_stamp":"2024-09-10T21:47:27.150439+0000","last_clean_scrub_stamp":"2024-09-10T21:47:27.150439+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:59:31.672676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"27'4776","reported_seq":4812,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:38.265808+0000","last_change":"2024-09-10T21:47:38.265808+0000","last_active":"2024-09-10T21:47:38.265808+0000","last_peered":"2024-09-10T21:47:38.265808+0000","last_clean":"2024-09-10T21:47:38.265808+0000","last_became_active":"2024-09-10T21:24:51.899652+0000","last_became_peered":"2024-09-10T21:24:51.899652+0000","last_unstale":"2024-09-10T21:47:38.265808+0000","last_undegraded":"2024-09-10T21:47:38.265808+0000","last_fullsized":"2024-09-10T21:47:38.265808+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4776","last_scrub_stamp":"2024-09-10T21:47:38.265701+0000","last_deep_scrub":"27'4776","last_deep_scrub_stamp":"2024-09-10T21:47:38.265701+0000","last_clean_scrub_stamp":"2024-09-10T21:47:38.265701+0000","objects_scrubbed":0,"log_size":4776,"log_dups_size":0,"ondisk_log_size":4776,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:00:51.487946+0000","scrub_duration":31,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23087,"num_write_kb":517913,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"27'8","reported_seq":52,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:31.997056+0000","last_change":"2024-09-10T21:47:31.997056+0000","last_active":"2024-09-10T21:47:31.997056+0000","last_peered":"2024-09-10T21:47:31.997056+0000","last_clean":"2024-09-10T21:47:31.997056+0000","last_became_active":"2024-09-10T21:24:46.860780+0000","last_became_peered":"2024-09-10T21:24:46.860780+0000","last_unstale":"2024-09-10T21:47:31.997056+0000","last_undegraded":"2024-09-10T21:47:31.997056+0000","last_fullsized":"2024-09-10T21:47:31.997056+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'8","last_scrub_stamp":"2024-09-10T21:47:31.996964+0000","last_deep_scrub":"27'8","last_deep_scrub_stamp":"2024-09-10T21:47:31.996964+0000","last_clean_scrub_stamp":"2024-09-10T21:47:31.996964+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:46:32.272965+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"27'16","reported_seq":57,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:32.740828+0000","last_change":"2024-09-10T21:47:32.740828+0000","last_active":"2024-09-10T21:47:32.740828+0000","last_peered":"2024-09-10T21:47:32.740828+0000","last_clean":"2024-09-10T21:47:32.740828+0000","last_became_active":"2024-09-10T21:24:47.877765+0000","last_became_peered":"2024-09-10T21:24:47.877765+0000","last_unstale":"2024-09-10T21:47:32.740828+0000","last_undegraded":"2024-09-10T21:47:32.740828+0000","last_fullsized":"2024-09-10T21:47:32.740828+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'16","last_scrub_stamp":"2024-09-10T21:47:32.740745+0000","last_deep_scrub":"27'16","last_deep_scrub_stamp":"2024-09-10T21:47:32.740745+0000","last_clean_scrub_stamp":"2024-09-10T21:47:32.740745+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:19:07.421452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"27'4118","reported_seq":4162,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:37.227370+0000","last_change":"2024-09-10T21:47:37.227370+0000","last_active":"2024-09-10T21:47:37.227370+0000","last_peered":"2024-09-10T21:47:37.227370+0000","last_clean":"2024-09-10T21:47:37.227370+0000","last_became_active":"2024-09-10T21:25:28.512023+0000","last_became_peered":"2024-09-10T21:25:28.512023+0000","last_unstale":"2024-09-10T21:47:37.227370+0000","last_undegraded":"2024-09-10T21:47:37.227370+0000","last_fullsized":"2024-09-10T21:47:37.227370+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4118","last_scrub_stamp":"2024-09-10T21:47:37.227217+0000","last_deep_scrub":"27'4118","last_deep_scrub_stamp":"2024-09-10T21:47:37.227217+0000","last_clean_scrub_stamp":"2024-09-10T21:47:37.227217+0000","objects_scrubbed":0,"log_size":4118,"log_dups_size":0,"ondisk_log_size":4118,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:17:15.484677+0000","scrub_duration":27,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18983,"num_write_kb":440322,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,2],"acting":[4,3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"27'6","reported_seq":50,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:30.061283+0000","last_change":"2024-09-10T21:47:30.061283+0000","last_active":"2024-09-10T21:47:30.061283+0000","last_peered":"2024-09-10T21:47:30.061283+0000","last_clean":"2024-09-10T21:47:30.061283+0000","last_became_active":"2024-09-10T21:24:46.872075+0000","last_became_peered":"2024-09-10T21:24:46.872075+0000","last_unstale":"2024-09-10T21:47:30.061283+0000","last_undegraded":"2024-09-10T21:47:30.061283+0000","last_fullsized":"2024-09-10T21:47:30.061283+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'6","last_scrub_stamp":"2024-09-10T21:47:30.061189+0000","last_deep_scrub":"27'6","last_deep_scrub_stamp":"2024-09-10T21:47:30.061189+0000","last_clean_scrub_stamp":"2024-09-10T21:47:30.061189+0000","objects_scrubbed":1,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:06:42.038139+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"27'30","reported_seq":71,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:33.696444+0000","last_change":"2024-09-10T21:47:33.696444+0000","last_active":"2024-09-10T21:47:33.696444+0000","last_peered":"2024-09-10T21:47:33.696444+0000","last_clean":"2024-09-10T21:47:33.696444+0000","last_became_active":"2024-09-10T21:24:47.876688+0000","last_became_peered":"2024-09-10T21:24:47.876688+0000","last_unstale":"2024-09-10T21:47:33.696444+0000","last_undegraded":"2024-09-10T21:47:33.696444+0000","last_fullsized":"2024-09-10T21:47:33.696444+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'30","last_scrub_stamp":"2024-09-10T21:47:33.696359+0000","last_deep_scrub":"27'30","last_deep_scrub_stamp":"2024-09-10T21:47:33.696359+0000","last_clean_scrub_stamp":"2024-09-10T21:47:33.696359+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:22:08.798884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"27'3999","reported_seq":4035,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:35.239870+0000","last_change":"2024-09-10T21:47:35.239870+0000","last_active":"2024-09-10T21:47:35.239870+0000","last_peered":"2024-09-10T21:47:35.239870+0000","last_clean":"2024-09-10T21:47:35.239870+0000","last_became_active":"2024-09-10T21:24:51.899576+0000","last_became_peered":"2024-09-10T21:24:51.899576+0000","last_unstale":"2024-09-10T21:47:35.239870+0000","last_undegraded":"2024-09-10T21:47:35.239870+0000","last_fullsized":"2024-09-10T21:47:35.239870+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'3999","last_scrub_stamp":"2024-09-10T21:47:35.239777+0000","last_deep_scrub":"27'3999","last_deep_scrub_stamp":"2024-09-10T21:47:35.239777+0000","last_clean_scrub_stamp":"2024-09-10T21:47:35.239777+0000","objects_scrubbed":0,"log_size":3999,"log_dups_size":0,"ondisk_log_size":3999,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:19:09.920484+0000","scrub_duration":27,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18841,"num_write_kb":435514,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"27'15","reported_seq":57,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:30.747799+0000","last_change":"2024-09-10T21:47:30.747799+0000","last_active":"2024-09-10T21:47:30.747799+0000","last_peered":"2024-09-10T21:47:30.747799+0000","last_clean":"2024-09-10T21:47:30.747799+0000","last_became_active":"2024-09-10T21:24:47.869569+0000","last_became_peered":"2024-09-10T21:24:47.869569+0000","last_unstale":"2024-09-10T21:47:30.747799+0000","last_undegraded":"2024-09-10T21:47:30.747799+0000","last_fullsized":"2024-09-10T21:47:30.747799+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'15","last_scrub_stamp":"2024-09-10T21:47:30.747728+0000","last_deep_scrub":"27'15","last_deep_scrub_stamp":"2024-09-10T21:47:30.747728+0000","last_clean_scrub_stamp":"2024-09-10T21:47:30.747728+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:30.496496+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"27'1005","reported_seq":1050,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:28.127434+0000","last_change":"2024-09-10T21:47:28.127434+0000","last_active":"2024-09-10T21:47:28.127434+0000","last_peered":"2024-09-10T21:47:28.127434+0000","last_clean":"2024-09-10T21:47:28.127434+0000","last_became_active":"2024-09-10T21:24:46.866893+0000","last_became_peered":"2024-09-10T21:24:46.866893+0000","last_unstale":"2024-09-10T21:47:28.127434+0000","last_undegraded":"2024-09-10T21:47:28.127434+0000","last_fullsized":"2024-09-10T21:47:28.127434+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'1005","last_scrub_stamp":"2024-09-10T21:47:28.127354+0000","last_deep_scrub":"27'1005","last_deep_scrub_stamp":"2024-09-10T21:47:28.127354+0000","last_clean_scrub_stamp":"2024-09-10T21:47:28.127354+0000","objects_scrubbed":2,"log_size":1005,"log_dups_size":0,"ondisk_log_size":1005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:27.876122+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":997,"num_write_kb":4359,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"27'4606","reported_seq":4643,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:33.268803+0000","last_change":"2024-09-10T21:47:33.268803+0000","last_active":"2024-09-10T21:47:33.268803+0000","last_peered":"2024-09-10T21:47:33.268803+0000","last_clean":"2024-09-10T21:47:33.268803+0000","last_became_active":"2024-09-10T21:24:51.907923+0000","last_became_peered":"2024-09-10T21:24:51.907923+0000","last_unstale":"2024-09-10T21:47:33.268803+0000","last_undegraded":"2024-09-10T21:47:33.268803+0000","last_fullsized":"2024-09-10T21:47:33.268803+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4606","last_scrub_stamp":"2024-09-10T21:47:33.268695+0000","last_deep_scrub":"27'4606","last_deep_scrub_stamp":"2024-09-10T21:47:33.268695+0000","last_clean_scrub_stamp":"2024-09-10T21:47:33.268695+0000","objects_scrubbed":0,"log_size":4606,"log_dups_size":0,"ondisk_log_size":4606,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:19:15.091323+0000","scrub_duration":27,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22377,"num_write_kb":531442,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"27'17","reported_seq":60,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:25.218592+0000","last_change":"2024-09-10T21:47:25.218592+0000","last_active":"2024-09-10T21:47:25.218592+0000","last_peered":"2024-09-10T21:47:25.218592+0000","last_clean":"2024-09-10T21:47:25.218592+0000","last_became_active":"2024-09-10T21:24:47.861842+0000","last_became_peered":"2024-09-10T21:24:47.861842+0000","last_unstale":"2024-09-10T21:47:25.218592+0000","last_undegraded":"2024-09-10T21:47:25.218592+0000","last_fullsized":"2024-09-10T21:47:25.218592+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'17","last_scrub_stamp":"2024-09-10T21:47:25.218527+0000","last_deep_scrub":"27'17","last_deep_scrub_stamp":"2024-09-10T21:47:25.218527+0000","last_clean_scrub_stamp":"2024-09-10T21:47:25.218527+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:17:58.315743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"27'3","reported_seq":47,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:31.940341+0000","last_change":"2024-09-10T21:47:31.940341+0000","last_active":"2024-09-10T21:47:31.940341+0000","last_peered":"2024-09-10T21:47:31.940341+0000","last_clean":"2024-09-10T21:47:31.940341+0000","last_became_active":"2024-09-10T21:24:46.851027+0000","last_became_peered":"2024-09-10T21:24:46.851027+0000","last_unstale":"2024-09-10T21:47:31.940341+0000","last_undegraded":"2024-09-10T21:47:31.940341+0000","last_fullsized":"2024-09-10T21:47:31.940341+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'3","last_scrub_stamp":"2024-09-10T21:47:31.940273+0000","last_deep_scrub":"27'3","last_deep_scrub_stamp":"2024-09-10T21:47:31.940273+0000","last_clean_scrub_stamp":"2024-09-10T21:47:31.940273+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:17:02.645449+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"27'4524","reported_seq":4561,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:32.309653+0000","last_change":"2024-09-10T21:47:32.309653+0000","last_active":"2024-09-10T21:47:32.309653+0000","last_peered":"2024-09-10T21:47:32.309653+0000","last_clean":"2024-09-10T21:47:32.309653+0000","last_became_active":"2024-09-10T21:24:51.903031+0000","last_became_peered":"2024-09-10T21:24:51.903031+0000","last_unstale":"2024-09-10T21:47:32.309653+0000","last_undegraded":"2024-09-10T21:47:32.309653+0000","last_fullsized":"2024-09-10T21:47:32.309653+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4524","last_scrub_stamp":"2024-09-10T21:47:32.309556+0000","last_deep_scrub":"27'4524","last_deep_scrub_stamp":"2024-09-10T21:47:32.309556+0000","last_clean_scrub_stamp":"2024-09-10T21:47:32.309556+0000","objects_scrubbed":0,"log_size":4524,"log_dups_size":0,"ondisk_log_size":4524,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:26:18.451044+0000","scrub_duration":27,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21954,"num_write_kb":484896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"27'18","reported_seq":61,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:30.160429+0000","last_change":"2024-09-10T21:47:30.160429+0000","last_active":"2024-09-10T21:47:30.160429+0000","last_peered":"2024-09-10T21:47:30.160429+0000","last_clean":"2024-09-10T21:47:30.160429+0000","last_became_active":"2024-09-10T21:24:47.856003+0000","last_became_peered":"2024-09-10T21:24:47.856003+0000","last_unstale":"2024-09-10T21:47:30.160429+0000","last_undegraded":"2024-09-10T21:47:30.160429+0000","last_fullsized":"2024-09-10T21:47:30.160429+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'18","last_scrub_stamp":"2024-09-10T21:47:30.160362+0000","last_deep_scrub":"27'18","last_deep_scrub_stamp":"2024-09-10T21:47:30.160362+0000","last_clean_scrub_stamp":"2024-09-10T21:47:30.160362+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:21:38.722572+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"27'2","reported_seq":46,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:29.969656+0000","last_change":"2024-09-10T21:47:29.969656+0000","last_active":"2024-09-10T21:47:29.969656+0000","last_peered":"2024-09-10T21:47:29.969656+0000","last_clean":"2024-09-10T21:47:29.969656+0000","last_became_active":"2024-09-10T21:24:46.852993+0000","last_became_peered":"2024-09-10T21:24:46.852993+0000","last_unstale":"2024-09-10T21:47:29.969656+0000","last_undegraded":"2024-09-10T21:47:29.969656+0000","last_fullsized":"2024-09-10T21:47:29.969656+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2","last_scrub_stamp":"2024-09-10T21:47:29.969599+0000","last_deep_scrub":"27'2","last_deep_scrub_stamp":"2024-09-10T21:47:29.969599+0000","last_clean_scrub_stamp":"2024-09-10T21:47:29.969599+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:03.952547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"27'5859","reported_seq":5896,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:38.013477+0000","last_change":"2024-09-10T21:47:38.013477+0000","last_active":"2024-09-10T21:47:38.013477+0000","last_peered":"2024-09-10T21:47:38.013477+0000","last_clean":"2024-09-10T21:47:38.013477+0000","last_became_active":"2024-09-10T21:24:52.064884+0000","last_became_peered":"2024-09-10T21:24:52.064884+0000","last_unstale":"2024-09-10T21:47:38.013477+0000","last_undegraded":"2024-09-10T21:47:38.013477+0000","last_fullsized":"2024-09-10T21:47:38.013477+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'5859","last_scrub_stamp":"2024-09-10T21:47:38.013389+0000","last_deep_scrub":"27'5859","last_deep_scrub_stamp":"2024-09-10T21:47:38.013389+0000","last_clean_scrub_stamp":"2024-09-10T21:47:38.013389+0000","objects_scrubbed":0,"log_size":5859,"log_dups_size":0,"ondisk_log_size":5859,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:54:35.319194+0000","scrub_duration":51,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27858,"num_write_kb":655707,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"27'9","reported_seq":52,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:26.091888+0000","last_change":"2024-09-10T21:47:26.091888+0000","last_active":"2024-09-10T21:47:26.091888+0000","last_peered":"2024-09-10T21:47:26.091888+0000","last_clean":"2024-09-10T21:47:26.091888+0000","last_became_active":"2024-09-10T21:24:47.856185+0000","last_became_peered":"2024-09-10T21:24:47.856185+0000","last_unstale":"2024-09-10T21:47:26.091888+0000","last_undegraded":"2024-09-10T21:47:26.091888+0000","last_fullsized":"2024-09-10T21:47:26.091888+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9","last_scrub_stamp":"2024-09-10T21:47:26.091769+0000","last_deep_scrub":"27'9","last_deep_scrub_stamp":"2024-09-10T21:47:26.091769+0000","last_clean_scrub_stamp":"2024-09-10T21:47:26.091769+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:54:45.759809+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"27'5","reported_seq":50,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:31.019617+0000","last_change":"2024-09-10T21:47:31.019617+0000","last_active":"2024-09-10T21:47:31.019617+0000","last_peered":"2024-09-10T21:47:31.019617+0000","last_clean":"2024-09-10T21:47:31.019617+0000","last_became_active":"2024-09-10T21:24:46.866477+0000","last_became_peered":"2024-09-10T21:24:46.866477+0000","last_unstale":"2024-09-10T21:47:31.019617+0000","last_undegraded":"2024-09-10T21:47:31.019617+0000","last_fullsized":"2024-09-10T21:47:31.019617+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'5","last_scrub_stamp":"2024-09-10T21:47:31.019532+0000","last_deep_scrub":"27'5","last_deep_scrub_stamp":"2024-09-10T21:47:31.019532+0000","last_clean_scrub_stamp":"2024-09-10T21:47:31.019532+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:05.002480+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"27'4736","reported_seq":4773,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:34.228852+0000","last_change":"2024-09-10T21:47:34.228852+0000","last_active":"2024-09-10T21:47:34.228852+0000","last_peered":"2024-09-10T21:47:34.228852+0000","last_clean":"2024-09-10T21:47:34.228852+0000","last_became_active":"2024-09-10T21:24:51.907878+0000","last_became_peered":"2024-09-10T21:24:51.907878+0000","last_unstale":"2024-09-10T21:47:34.228852+0000","last_undegraded":"2024-09-10T21:47:34.228852+0000","last_fullsized":"2024-09-10T21:47:34.228852+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4736","last_scrub_stamp":"2024-09-10T21:47:34.228769+0000","last_deep_scrub":"27'4736","last_deep_scrub_stamp":"2024-09-10T21:47:34.228769+0000","last_clean_scrub_stamp":"2024-09-10T21:47:34.228769+0000","objects_scrubbed":0,"log_size":4736,"log_dups_size":0,"ondisk_log_size":4736,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:47:28.146012+0000","scrub_duration":27,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22584,"num_write_kb":479265,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"27'25","reported_seq":68,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:26.154489+0000","last_change":"2024-09-10T21:47:26.154489+0000","last_active":"2024-09-10T21:47:26.154489+0000","last_peered":"2024-09-10T21:47:26.154489+0000","last_clean":"2024-09-10T21:47:26.154489+0000","last_became_active":"2024-09-10T21:24:47.844505+0000","last_became_peered":"2024-09-10T21:24:47.844505+0000","last_unstale":"2024-09-10T21:47:26.154489+0000","last_undegraded":"2024-09-10T21:47:26.154489+0000","last_fullsized":"2024-09-10T21:47:26.154489+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'25","last_scrub_stamp":"2024-09-10T21:47:26.154431+0000","last_deep_scrub":"27'25","last_deep_scrub_stamp":"2024-09-10T21:47:26.154431+0000","last_clean_scrub_stamp":"2024-09-10T21:47:26.154431+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:12:31.123628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"27'10","reported_seq":54,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:24.643112+0000","last_change":"2024-09-10T21:47:24.643112+0000","last_active":"2024-09-10T21:47:24.643112+0000","last_peered":"2024-09-10T21:47:24.643112+0000","last_clean":"2024-09-10T21:47:24.643112+0000","last_became_active":"2024-09-10T21:24:46.861473+0000","last_became_peered":"2024-09-10T21:24:46.861473+0000","last_unstale":"2024-09-10T21:47:24.643112+0000","last_undegraded":"2024-09-10T21:47:24.643112+0000","last_fullsized":"2024-09-10T21:47:24.643112+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'10","last_scrub_stamp":"2024-09-10T21:47:24.643031+0000","last_deep_scrub":"27'10","last_deep_scrub_stamp":"2024-09-10T21:47:24.643031+0000","last_clean_scrub_stamp":"2024-09-10T21:47:24.643031+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:22.143057+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":53,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"27'4521","reported_seq":4559,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:30.235915+0000","last_change":"2024-09-10T21:47:30.235855+0000","last_active":"2024-09-10T21:47:30.235915+0000","last_peered":"2024-09-10T21:47:30.235915+0000","last_clean":"2024-09-10T21:47:30.235915+0000","last_became_active":"2024-09-10T21:24:52.061581+0000","last_became_peered":"2024-09-10T21:24:52.061581+0000","last_unstale":"2024-09-10T21:47:30.235915+0000","last_undegraded":"2024-09-10T21:47:30.235915+0000","last_fullsized":"2024-09-10T21:47:30.235915+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4521","last_scrub_stamp":"2024-09-10T21:47:30.235757+0000","last_deep_scrub":"27'4521","last_deep_scrub_stamp":"2024-09-10T21:47:30.235757+0000","last_clean_scrub_stamp":"2024-09-10T21:47:30.235757+0000","objects_scrubbed":0,"log_size":4521,"log_dups_size":0,"ondisk_log_size":4521,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:53:04.671154+0000","scrub_duration":31,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21874,"num_write_kb":476387,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"27'18","reported_seq":61,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:26.176880+0000","last_change":"2024-09-10T21:47:26.176880+0000","last_active":"2024-09-10T21:47:26.176880+0000","last_peered":"2024-09-10T21:47:26.176880+0000","last_clean":"2024-09-10T21:47:26.176880+0000","last_became_active":"2024-09-10T21:24:47.859483+0000","last_became_peered":"2024-09-10T21:24:47.859483+0000","last_unstale":"2024-09-10T21:47:26.176880+0000","last_undegraded":"2024-09-10T21:47:26.176880+0000","last_fullsized":"2024-09-10T21:47:26.176880+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'18","last_scrub_stamp":"2024-09-10T21:47:26.176783+0000","last_deep_scrub":"27'18","last_deep_scrub_stamp":"2024-09-10T21:47:26.176783+0000","last_clean_scrub_stamp":"2024-09-10T21:47:26.176783+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:47:20.094026+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"27'327","reported_seq":371,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:22.946760+0000","last_change":"2024-09-10T21:47:22.946760+0000","last_active":"2024-09-10T21:47:22.946760+0000","last_peered":"2024-09-10T21:47:22.946760+0000","last_clean":"2024-09-10T21:47:22.946760+0000","last_became_active":"2024-09-10T21:24:46.860309+0000","last_became_peered":"2024-09-10T21:24:46.860309+0000","last_unstale":"2024-09-10T21:47:22.946760+0000","last_undegraded":"2024-09-10T21:47:22.946760+0000","last_fullsized":"2024-09-10T21:47:22.946760+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'327","last_scrub_stamp":"2024-09-10T21:47:22.946702+0000","last_deep_scrub":"27'327","last_deep_scrub_stamp":"2024-09-10T21:47:22.946702+0000","last_clean_scrub_stamp":"2024-09-10T21:47:22.946702+0000","objects_scrubbed":1,"log_size":327,"log_dups_size":0,"ondisk_log_size":327,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:31:01.455227+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":704,"num_write_kb":242,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"27'4358","reported_seq":4396,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:39.265520+0000","last_change":"2024-09-10T21:47:39.265520+0000","last_active":"2024-09-10T21:47:39.265520+0000","last_peered":"2024-09-10T21:47:39.265520+0000","last_clean":"2024-09-10T21:47:39.265520+0000","last_became_active":"2024-09-10T21:24:52.068250+0000","last_became_peered":"2024-09-10T21:24:52.068250+0000","last_unstale":"2024-09-10T21:47:39.265520+0000","last_undegraded":"2024-09-10T21:47:39.265520+0000","last_fullsized":"2024-09-10T21:47:39.265520+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4358","last_scrub_stamp":"2024-09-10T21:47:39.265434+0000","last_deep_scrub":"27'4358","last_deep_scrub_stamp":"2024-09-10T21:47:39.265434+0000","last_clean_scrub_stamp":"2024-09-10T21:47:39.265434+0000","objects_scrubbed":0,"log_size":4358,"log_dups_size":0,"ondisk_log_size":4358,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:14:31.486338+0000","scrub_duration":35,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21666,"num_write_kb":463209,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"27'18","reported_seq":61,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:24.937412+0000","last_change":"2024-09-10T21:47:24.937326+0000","last_active":"2024-09-10T21:47:24.937412+0000","last_peered":"2024-09-10T21:47:24.937412+0000","last_clean":"2024-09-10T21:47:24.937412+0000","last_became_active":"2024-09-10T21:24:47.853750+0000","last_became_peered":"2024-09-10T21:24:47.853750+0000","last_unstale":"2024-09-10T21:47:24.937412+0000","last_undegraded":"2024-09-10T21:47:24.937412+0000","last_fullsized":"2024-09-10T21:47:24.937412+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'18","last_scrub_stamp":"2024-09-10T21:47:24.937253+0000","last_deep_scrub":"27'18","last_deep_scrub_stamp":"2024-09-10T21:47:24.937253+0000","last_clean_scrub_stamp":"2024-09-10T21:47:24.937253+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:58:49.150573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"27'2","reported_seq":48,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:27.093271+0000","last_change":"2024-09-10T21:47:27.093212+0000","last_active":"2024-09-10T21:47:27.093271+0000","last_peered":"2024-09-10T21:47:27.093271+0000","last_clean":"2024-09-10T21:47:27.093271+0000","last_became_active":"2024-09-10T21:24:46.864149+0000","last_became_peered":"2024-09-10T21:24:46.864149+0000","last_unstale":"2024-09-10T21:47:27.093271+0000","last_undegraded":"2024-09-10T21:47:27.093271+0000","last_fullsized":"2024-09-10T21:47:27.093271+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2","last_scrub_stamp":"2024-09-10T21:47:27.093133+0000","last_deep_scrub":"27'2","last_deep_scrub_stamp":"2024-09-10T21:47:27.093133+0000","last_clean_scrub_stamp":"2024-09-10T21:47:27.093133+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:34:02.239975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"27'4655","reported_seq":4693,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:34.174578+0000","last_change":"2024-09-10T21:47:34.174519+0000","last_active":"2024-09-10T21:47:34.174578+0000","last_peered":"2024-09-10T21:47:34.174578+0000","last_clean":"2024-09-10T21:47:34.174578+0000","last_became_active":"2024-09-10T21:24:52.060953+0000","last_became_peered":"2024-09-10T21:24:52.060953+0000","last_unstale":"2024-09-10T21:47:34.174578+0000","last_undegraded":"2024-09-10T21:47:34.174578+0000","last_fullsized":"2024-09-10T21:47:34.174578+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4655","last_scrub_stamp":"2024-09-10T21:47:34.174437+0000","last_deep_scrub":"27'4655","last_deep_scrub_stamp":"2024-09-10T21:47:34.174437+0000","last_clean_scrub_stamp":"2024-09-10T21:47:34.174437+0000","objects_scrubbed":0,"log_size":4655,"log_dups_size":0,"ondisk_log_size":4655,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:51:24.442254+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22893,"num_write_kb":513258,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"27'18","reported_seq":61,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:29.758890+0000","last_change":"2024-09-10T21:47:29.758834+0000","last_active":"2024-09-10T21:47:29.758890+0000","last_peered":"2024-09-10T21:47:29.758890+0000","last_clean":"2024-09-10T21:47:29.758890+0000","last_became_active":"2024-09-10T21:24:47.866907+0000","last_became_peered":"2024-09-10T21:24:47.866907+0000","last_unstale":"2024-09-10T21:47:29.758890+0000","last_undegraded":"2024-09-10T21:47:29.758890+0000","last_fullsized":"2024-09-10T21:47:29.758890+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'18","last_scrub_stamp":"2024-09-10T21:47:29.758754+0000","last_deep_scrub":"27'18","last_deep_scrub_stamp":"2024-09-10T21:47:29.758754+0000","last_clean_scrub_stamp":"2024-09-10T21:47:29.758754+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:26:15.900242+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"27'2","reported_seq":47,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:21.960467+0000","last_change":"2024-09-10T21:47:21.960410+0000","last_active":"2024-09-10T21:47:21.960467+0000","last_peered":"2024-09-10T21:47:21.960467+0000","last_clean":"2024-09-10T21:47:21.960467+0000","last_became_active":"2024-09-10T21:24:46.856489+0000","last_became_peered":"2024-09-10T21:24:46.856489+0000","last_unstale":"2024-09-10T21:47:21.960467+0000","last_undegraded":"2024-09-10T21:47:21.960467+0000","last_fullsized":"2024-09-10T21:47:21.960467+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2","last_scrub_stamp":"2024-09-10T21:47:21.960350+0000","last_deep_scrub":"27'2","last_deep_scrub_stamp":"2024-09-10T21:47:21.960350+0000","last_clean_scrub_stamp":"2024-09-10T21:47:21.960350+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:17:07.017677+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"27'4909","reported_seq":4947,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:34.041399+0000","last_change":"2024-09-10T21:47:34.041341+0000","last_active":"2024-09-10T21:47:34.041399+0000","last_peered":"2024-09-10T21:47:34.041399+0000","last_clean":"2024-09-10T21:47:34.041399+0000","last_became_active":"2024-09-10T21:24:51.900479+0000","last_became_peered":"2024-09-10T21:24:51.900479+0000","last_unstale":"2024-09-10T21:47:34.041399+0000","last_undegraded":"2024-09-10T21:47:34.041399+0000","last_fullsized":"2024-09-10T21:47:34.041399+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4909","last_scrub_stamp":"2024-09-10T21:47:34.041256+0000","last_deep_scrub":"27'4909","last_deep_scrub_stamp":"2024-09-10T21:47:34.041256+0000","last_clean_scrub_stamp":"2024-09-10T21:47:34.041256+0000","objects_scrubbed":0,"log_size":4909,"log_dups_size":0,"ondisk_log_size":4909,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:54:37.955889+0000","scrub_duration":35,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23807,"num_write_kb":525103,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"27'16","reported_seq":60,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:25.069753+0000","last_change":"2024-09-10T21:47:25.069701+0000","last_active":"2024-09-10T21:47:25.069753+0000","last_peered":"2024-09-10T21:47:25.069753+0000","last_clean":"2024-09-10T21:47:25.069753+0000","last_became_active":"2024-09-10T21:24:47.858438+0000","last_became_peered":"2024-09-10T21:24:47.858438+0000","last_unstale":"2024-09-10T21:47:25.069753+0000","last_undegraded":"2024-09-10T21:47:25.069753+0000","last_fullsized":"2024-09-10T21:47:25.069753+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'16","last_scrub_stamp":"2024-09-10T21:47:25.069622+0000","last_deep_scrub":"27'16","last_deep_scrub_stamp":"2024-09-10T21:47:25.069622+0000","last_clean_scrub_stamp":"2024-09-10T21:47:25.069622+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:56:48.024051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"27'4","reported_seq":49,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:24.777360+0000","last_change":"2024-09-10T21:47:24.777261+0000","last_active":"2024-09-10T21:47:24.777360+0000","last_peered":"2024-09-10T21:47:24.777360+0000","last_clean":"2024-09-10T21:47:24.777360+0000","last_became_active":"2024-09-10T21:24:46.863150+0000","last_became_peered":"2024-09-10T21:24:46.863150+0000","last_unstale":"2024-09-10T21:47:24.777360+0000","last_undegraded":"2024-09-10T21:47:24.777360+0000","last_fullsized":"2024-09-10T21:47:24.777360+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4","last_scrub_stamp":"2024-09-10T21:47:24.777166+0000","last_deep_scrub":"27'4","last_deep_scrub_stamp":"2024-09-10T21:47:24.777166+0000","last_clean_scrub_stamp":"2024-09-10T21:47:24.777166+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:57:05.934346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"27'5335","reported_seq":5373,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:27.024021+0000","last_change":"2024-09-10T21:47:27.023955+0000","last_active":"2024-09-10T21:47:27.024021+0000","last_peered":"2024-09-10T21:47:27.024021+0000","last_clean":"2024-09-10T21:47:27.024021+0000","last_became_active":"2024-09-10T21:24:51.906888+0000","last_became_peered":"2024-09-10T21:24:51.906888+0000","last_unstale":"2024-09-10T21:47:27.024021+0000","last_undegraded":"2024-09-10T21:47:27.024021+0000","last_fullsized":"2024-09-10T21:47:27.024021+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'5335","last_scrub_stamp":"2024-09-10T21:47:27.023873+0000","last_deep_scrub":"27'5335","last_deep_scrub_stamp":"2024-09-10T21:47:27.023873+0000","last_clean_scrub_stamp":"2024-09-10T21:47:27.023873+0000","objects_scrubbed":0,"log_size":5335,"log_dups_size":0,"ondisk_log_size":5335,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:17:26.772642+0000","scrub_duration":35,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26491,"num_write_kb":568740,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"27'20","reported_seq":63,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:30.922336+0000","last_change":"2024-09-10T21:47:30.922336+0000","last_active":"2024-09-10T21:47:30.922336+0000","last_peered":"2024-09-10T21:47:30.922336+0000","last_clean":"2024-09-10T21:47:30.922336+0000","last_became_active":"2024-09-10T21:24:47.849846+0000","last_became_peered":"2024-09-10T21:24:47.849846+0000","last_unstale":"2024-09-10T21:47:30.922336+0000","last_undegraded":"2024-09-10T21:47:30.922336+0000","last_fullsized":"2024-09-10T21:47:30.922336+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'20","last_scrub_stamp":"2024-09-10T21:47:30.922276+0000","last_deep_scrub":"27'20","last_deep_scrub_stamp":"2024-09-10T21:47:30.922276+0000","last_clean_scrub_stamp":"2024-09-10T21:47:30.922276+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:38:31.565344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"27'52","reported_seq":146,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:23.121455+0000","last_change":"2024-09-10T21:47:23.121361+0000","last_active":"2024-09-10T21:47:23.121455+0000","last_peered":"2024-09-10T21:47:23.121455+0000","last_clean":"2024-09-10T21:47:23.121455+0000","last_became_active":"2024-09-10T21:24:46.851200+0000","last_became_peered":"2024-09-10T21:24:46.851200+0000","last_unstale":"2024-09-10T21:47:23.121455+0000","last_undegraded":"2024-09-10T21:47:23.121455+0000","last_fullsized":"2024-09-10T21:47:23.121455+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'52","last_scrub_stamp":"2024-09-10T21:47:23.121257+0000","last_deep_scrub":"27'52","last_deep_scrub_stamp":"2024-09-10T21:47:23.121257+0000","last_clean_scrub_stamp":"2024-09-10T21:47:23.121257+0000","objects_scrubbed":1,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:43:57.409412+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90454,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":48,"num_read_kb":119,"num_write":48,"num_write_kb":119,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"27'4677","reported_seq":4714,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:36.980284+0000","last_change":"2024-09-10T21:47:36.980284+0000","last_active":"2024-09-10T21:47:36.980284+0000","last_peered":"2024-09-10T21:47:36.980284+0000","last_clean":"2024-09-10T21:47:36.980284+0000","last_became_active":"2024-09-10T21:24:51.901141+0000","last_became_peered":"2024-09-10T21:24:51.901141+0000","last_unstale":"2024-09-10T21:47:36.980284+0000","last_undegraded":"2024-09-10T21:47:36.980284+0000","last_fullsized":"2024-09-10T21:47:36.980284+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4677","last_scrub_stamp":"2024-09-10T21:47:36.980186+0000","last_deep_scrub":"27'4677","last_deep_scrub_stamp":"2024-09-10T21:47:36.980186+0000","last_clean_scrub_stamp":"2024-09-10T21:47:36.980186+0000","objects_scrubbed":0,"log_size":4677,"log_dups_size":0,"ondisk_log_size":4677,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:25:44.121039+0000","scrub_duration":35,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22963,"num_write_kb":501268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"27'11","reported_seq":55,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:24.122533+0000","last_change":"2024-09-10T21:47:24.122483+0000","last_active":"2024-09-10T21:47:24.122533+0000","last_peered":"2024-09-10T21:47:24.122533+0000","last_clean":"2024-09-10T21:47:24.122533+0000","last_became_active":"2024-09-10T21:24:47.846358+0000","last_became_peered":"2024-09-10T21:24:47.846358+0000","last_unstale":"2024-09-10T21:47:24.122533+0000","last_undegraded":"2024-09-10T21:47:24.122533+0000","last_fullsized":"2024-09-10T21:47:24.122533+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'11","last_scrub_stamp":"2024-09-10T21:47:24.122422+0000","last_deep_scrub":"27'11","last_deep_scrub_stamp":"2024-09-10T21:47:24.122422+0000","last_clean_scrub_stamp":"2024-09-10T21:47:24.122422+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:02:48.829990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"27'7","reported_seq":53,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:29.103436+0000","last_change":"2024-09-10T21:47:29.103436+0000","last_active":"2024-09-10T21:47:29.103436+0000","last_peered":"2024-09-10T21:47:29.103436+0000","last_clean":"2024-09-10T21:47:29.103436+0000","last_became_active":"2024-09-10T21:24:46.868375+0000","last_became_peered":"2024-09-10T21:24:46.868375+0000","last_unstale":"2024-09-10T21:47:29.103436+0000","last_undegraded":"2024-09-10T21:47:29.103436+0000","last_fullsized":"2024-09-10T21:47:29.103436+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'7","last_scrub_stamp":"2024-09-10T21:47:29.103354+0000","last_deep_scrub":"27'7","last_deep_scrub_stamp":"2024-09-10T21:47:29.103354+0000","last_clean_scrub_stamp":"2024-09-10T21:47:29.103354+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:08:06.990919+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"27'4824","reported_seq":4861,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:36.512209+0000","last_change":"2024-09-10T21:47:36.512209+0000","last_active":"2024-09-10T21:47:36.512209+0000","last_peered":"2024-09-10T21:47:36.512209+0000","last_clean":"2024-09-10T21:47:36.512209+0000","last_became_active":"2024-09-10T21:24:52.067308+0000","last_became_peered":"2024-09-10T21:24:52.067308+0000","last_unstale":"2024-09-10T21:47:36.512209+0000","last_undegraded":"2024-09-10T21:47:36.512209+0000","last_fullsized":"2024-09-10T21:47:36.512209+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4824","last_scrub_stamp":"2024-09-10T21:47:36.512105+0000","last_deep_scrub":"27'4824","last_deep_scrub_stamp":"2024-09-10T21:47:36.512105+0000","last_clean_scrub_stamp":"2024-09-10T21:47:36.512105+0000","objects_scrubbed":0,"log_size":4824,"log_dups_size":0,"ondisk_log_size":4824,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:18:15.084927+0000","scrub_duration":343,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23972,"num_write_kb":500992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"27'13","reported_seq":57,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:36.250492+0000","last_change":"2024-09-10T21:47:36.250439+0000","last_active":"2024-09-10T21:47:36.250492+0000","last_peered":"2024-09-10T21:47:36.250492+0000","last_clean":"2024-09-10T21:47:36.250492+0000","last_became_active":"2024-09-10T21:24:47.871128+0000","last_became_peered":"2024-09-10T21:24:47.871128+0000","last_unstale":"2024-09-10T21:47:36.250492+0000","last_undegraded":"2024-09-10T21:47:36.250492+0000","last_fullsized":"2024-09-10T21:47:36.250492+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'13","last_scrub_stamp":"2024-09-10T21:47:36.250369+0000","last_deep_scrub":"27'13","last_deep_scrub_stamp":"2024-09-10T21:47:36.250369+0000","last_clean_scrub_stamp":"2024-09-10T21:47:36.250369+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:30:18.379622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"27'3","reported_seq":48,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:25.794282+0000","last_change":"2024-09-10T21:47:25.794215+0000","last_active":"2024-09-10T21:47:25.794282+0000","last_peered":"2024-09-10T21:47:25.794282+0000","last_clean":"2024-09-10T21:47:25.794282+0000","last_became_active":"2024-09-10T21:24:46.853190+0000","last_became_peered":"2024-09-10T21:24:46.853190+0000","last_unstale":"2024-09-10T21:47:25.794282+0000","last_undegraded":"2024-09-10T21:47:25.794282+0000","last_fullsized":"2024-09-10T21:47:25.794282+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'3","last_scrub_stamp":"2024-09-10T21:47:25.794132+0000","last_deep_scrub":"27'3","last_deep_scrub_stamp":"2024-09-10T21:47:25.794132+0000","last_clean_scrub_stamp":"2024-09-10T21:47:25.794132+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:45:23.090980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"27'74","reported_seq":144,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:26.092159+0000","last_change":"2024-09-10T21:47:26.092159+0000","last_active":"2024-09-10T21:47:26.092159+0000","last_peered":"2024-09-10T21:47:26.092159+0000","last_clean":"2024-09-10T21:47:26.092159+0000","last_became_active":"2024-09-10T21:24:37.215443+0000","last_became_peered":"2024-09-10T21:24:37.215443+0000","last_unstale":"2024-09-10T21:47:26.092159+0000","last_undegraded":"2024-09-10T21:47:26.092159+0000","last_fullsized":"2024-09-10T21:47:26.092159+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'74","last_scrub_stamp":"2024-09-10T21:47:26.092034+0000","last_deep_scrub":"27'74","last_deep_scrub_stamp":"2024-09-10T21:47:26.092034+0000","last_clean_scrub_stamp":"2024-09-10T21:47:26.092034+0000","objects_scrubbed":2,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:58:50.305353+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"27'4394","reported_seq":4432,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:36.913610+0000","last_change":"2024-09-10T21:47:36.913553+0000","last_active":"2024-09-10T21:47:36.913610+0000","last_peered":"2024-09-10T21:47:36.913610+0000","last_clean":"2024-09-10T21:47:36.913610+0000","last_became_active":"2024-09-10T21:24:52.067148+0000","last_became_peered":"2024-09-10T21:24:52.067148+0000","last_unstale":"2024-09-10T21:47:36.913610+0000","last_undegraded":"2024-09-10T21:47:36.913610+0000","last_fullsized":"2024-09-10T21:47:36.913610+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4394","last_scrub_stamp":"2024-09-10T21:47:36.913452+0000","last_deep_scrub":"27'4394","last_deep_scrub_stamp":"2024-09-10T21:47:36.913452+0000","last_clean_scrub_stamp":"2024-09-10T21:47:36.913452+0000","objects_scrubbed":0,"log_size":4394,"log_dups_size":0,"ondisk_log_size":4394,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:46:26.160551+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21702,"num_write_kb":479680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"27'21","reported_seq":70,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:33.040993+0000","last_change":"2024-09-10T21:47:33.040993+0000","last_active":"2024-09-10T21:47:33.040993+0000","last_peered":"2024-09-10T21:47:33.040993+0000","last_clean":"2024-09-10T21:47:33.040993+0000","last_became_active":"2024-09-10T21:25:28.509981+0000","last_became_peered":"2024-09-10T21:25:28.509981+0000","last_unstale":"2024-09-10T21:47:33.040993+0000","last_undegraded":"2024-09-10T21:47:33.040993+0000","last_fullsized":"2024-09-10T21:47:33.040993+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'21","last_scrub_stamp":"2024-09-10T21:47:33.040910+0000","last_deep_scrub":"27'21","last_deep_scrub_stamp":"2024-09-10T21:47:33.040910+0000","last_clean_scrub_stamp":"2024-09-10T21:47:33.040910+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:17:03.746085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"27'5","reported_seq":50,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:24.139970+0000","last_change":"2024-09-10T21:47:24.139970+0000","last_active":"2024-09-10T21:47:24.139970+0000","last_peered":"2024-09-10T21:47:24.139970+0000","last_clean":"2024-09-10T21:47:24.139970+0000","last_became_active":"2024-09-10T21:24:46.859144+0000","last_became_peered":"2024-09-10T21:24:46.859144+0000","last_unstale":"2024-09-10T21:47:24.139970+0000","last_undegraded":"2024-09-10T21:47:24.139970+0000","last_fullsized":"2024-09-10T21:47:24.139970+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'5","last_scrub_stamp":"2024-09-10T21:47:24.139905+0000","last_deep_scrub":"27'5","last_deep_scrub_stamp":"2024-09-10T21:47:24.139905+0000","last_clean_scrub_stamp":"2024-09-10T21:47:24.139905+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:30:29.364591+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"27'4328","reported_seq":4373,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:38.868003+0000","last_change":"2024-09-10T21:47:38.868003+0000","last_active":"2024-09-10T21:47:38.868003+0000","last_peered":"2024-09-10T21:47:38.868003+0000","last_clean":"2024-09-10T21:47:38.868003+0000","last_became_active":"2024-09-10T21:25:28.620415+0000","last_became_peered":"2024-09-10T21:25:28.620415+0000","last_unstale":"2024-09-10T21:47:38.868003+0000","last_undegraded":"2024-09-10T21:47:38.868003+0000","last_fullsized":"2024-09-10T21:47:38.868003+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4328","last_scrub_stamp":"2024-09-10T21:47:38.867904+0000","last_deep_scrub":"27'4328","last_deep_scrub_stamp":"2024-09-10T21:47:38.867904+0000","last_clean_scrub_stamp":"2024-09-10T21:47:38.867904+0000","objects_scrubbed":0,"log_size":4328,"log_dups_size":0,"ondisk_log_size":4328,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:45.990909+0000","scrub_duration":31,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21368,"num_write_kb":498780,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2,0],"acting":[6,7,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"27'17","reported_seq":59,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:30.767144+0000","last_change":"2024-09-10T21:47:30.767144+0000","last_active":"2024-09-10T21:47:30.767144+0000","last_peered":"2024-09-10T21:47:30.767144+0000","last_clean":"2024-09-10T21:47:30.767144+0000","last_became_active":"2024-09-10T21:24:47.866650+0000","last_became_peered":"2024-09-10T21:24:47.866650+0000","last_unstale":"2024-09-10T21:47:30.767144+0000","last_undegraded":"2024-09-10T21:47:30.767144+0000","last_fullsized":"2024-09-10T21:47:30.767144+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'17","last_scrub_stamp":"2024-09-10T21:47:30.767082+0000","last_deep_scrub":"27'17","last_deep_scrub_stamp":"2024-09-10T21:47:30.767082+0000","last_clean_scrub_stamp":"2024-09-10T21:47:30.767082+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:19:12.589710+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"27'7","reported_seq":51,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:23.924308+0000","last_change":"2024-09-10T21:47:23.924308+0000","last_active":"2024-09-10T21:47:23.924308+0000","last_peered":"2024-09-10T21:47:23.924308+0000","last_clean":"2024-09-10T21:47:23.924308+0000","last_became_active":"2024-09-10T21:24:46.856799+0000","last_became_peered":"2024-09-10T21:24:46.856799+0000","last_unstale":"2024-09-10T21:47:23.924308+0000","last_undegraded":"2024-09-10T21:47:23.924308+0000","last_fullsized":"2024-09-10T21:47:23.924308+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'7","last_scrub_stamp":"2024-09-10T21:47:23.924252+0000","last_deep_scrub":"27'7","last_deep_scrub_stamp":"2024-09-10T21:47:23.924252+0000","last_clean_scrub_stamp":"2024-09-10T21:47:23.924252+0000","objects_scrubbed":1,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:30:03.470757+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"27'4625","reported_seq":4662,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:35.159579+0000","last_change":"2024-09-10T21:47:35.159579+0000","last_active":"2024-09-10T21:47:35.159579+0000","last_peered":"2024-09-10T21:47:35.159579+0000","last_clean":"2024-09-10T21:47:35.159579+0000","last_became_active":"2024-09-10T21:24:51.911088+0000","last_became_peered":"2024-09-10T21:24:51.911088+0000","last_unstale":"2024-09-10T21:47:35.159579+0000","last_undegraded":"2024-09-10T21:47:35.159579+0000","last_fullsized":"2024-09-10T21:47:35.159579+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4625","last_scrub_stamp":"2024-09-10T21:47:35.159496+0000","last_deep_scrub":"27'4625","last_deep_scrub_stamp":"2024-09-10T21:47:35.159496+0000","last_clean_scrub_stamp":"2024-09-10T21:47:35.159496+0000","objects_scrubbed":0,"log_size":4625,"log_dups_size":0,"ondisk_log_size":4625,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:21:43.721706+0000","scrub_duration":27,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21860,"num_write_kb":505231,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"27'20","reported_seq":63,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:34.032388+0000","last_change":"2024-09-10T21:47:34.032388+0000","last_active":"2024-09-10T21:47:34.032388+0000","last_peered":"2024-09-10T21:47:34.032388+0000","last_clean":"2024-09-10T21:47:34.032388+0000","last_became_active":"2024-09-10T21:24:47.876690+0000","last_became_peered":"2024-09-10T21:24:47.876690+0000","last_unstale":"2024-09-10T21:47:34.032388+0000","last_undegraded":"2024-09-10T21:47:34.032388+0000","last_fullsized":"2024-09-10T21:47:34.032388+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'20","last_scrub_stamp":"2024-09-10T21:47:34.032266+0000","last_deep_scrub":"27'20","last_deep_scrub_stamp":"2024-09-10T21:47:34.032266+0000","last_clean_scrub_stamp":"2024-09-10T21:47:34.032266+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:46:23.279365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"27'8","reported_seq":53,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:29.204773+0000","last_change":"2024-09-10T21:47:29.204773+0000","last_active":"2024-09-10T21:47:29.204773+0000","last_peered":"2024-09-10T21:47:29.204773+0000","last_clean":"2024-09-10T21:47:29.204773+0000","last_became_active":"2024-09-10T21:24:46.854930+0000","last_became_peered":"2024-09-10T21:24:46.854930+0000","last_unstale":"2024-09-10T21:47:29.204773+0000","last_undegraded":"2024-09-10T21:47:29.204773+0000","last_fullsized":"2024-09-10T21:47:29.204773+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'8","last_scrub_stamp":"2024-09-10T21:47:29.204714+0000","last_deep_scrub":"27'8","last_deep_scrub_stamp":"2024-09-10T21:47:29.204714+0000","last_clean_scrub_stamp":"2024-09-10T21:47:29.204714+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:47:47.401774+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"27'4395","reported_seq":4432,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:40.631138+0000","last_change":"2024-09-10T21:47:40.631138+0000","last_active":"2024-09-10T21:47:40.631138+0000","last_peered":"2024-09-10T21:47:40.631138+0000","last_clean":"2024-09-10T21:47:40.631138+0000","last_became_active":"2024-09-10T21:24:52.062395+0000","last_became_peered":"2024-09-10T21:24:52.062395+0000","last_unstale":"2024-09-10T21:47:40.631138+0000","last_undegraded":"2024-09-10T21:47:40.631138+0000","last_fullsized":"2024-09-10T21:47:40.631138+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4395","last_scrub_stamp":"2024-09-10T21:47:40.631036+0000","last_deep_scrub":"27'4395","last_deep_scrub_stamp":"2024-09-10T21:47:40.631036+0000","last_clean_scrub_stamp":"2024-09-10T21:47:40.631036+0000","objects_scrubbed":0,"log_size":4395,"log_dups_size":0,"ondisk_log_size":4395,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:50:36.704873+0000","scrub_duration":35,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21722,"num_write_kb":475710,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"27'13","reported_seq":42,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:28.230460+0000","last_change":"2024-09-10T21:47:28.230404+0000","last_active":"2024-09-10T21:47:28.230460+0000","last_peered":"2024-09-10T21:47:28.230460+0000","last_clean":"2024-09-10T21:47:28.230460+0000","last_became_active":"2024-09-10T21:25:28.513250+0000","last_became_peered":"2024-09-10T21:25:28.513250+0000","last_unstale":"2024-09-10T21:47:28.230460+0000","last_undegraded":"2024-09-10T21:47:28.230460+0000","last_fullsized":"2024-09-10T21:47:28.230460+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'13","last_scrub_stamp":"2024-09-10T21:47:28.230347+0000","last_deep_scrub":"27'13","last_deep_scrub_stamp":"2024-09-10T21:47:28.230347+0000","last_clean_scrub_stamp":"2024-09-10T21:47:28.230347+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:34:49.010972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.f","version":"27'5","reported_seq":50,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:24.060567+0000","last_change":"2024-09-10T21:47:24.060567+0000","last_active":"2024-09-10T21:47:24.060567+0000","last_peered":"2024-09-10T21:47:24.060567+0000","last_clean":"2024-09-10T21:47:24.060567+0000","last_became_active":"2024-09-10T21:24:46.860053+0000","last_became_peered":"2024-09-10T21:24:46.860053+0000","last_unstale":"2024-09-10T21:47:24.060567+0000","last_undegraded":"2024-09-10T21:47:24.060567+0000","last_fullsized":"2024-09-10T21:47:24.060567+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'5","last_scrub_stamp":"2024-09-10T21:47:24.060492+0000","last_deep_scrub":"27'5","last_deep_scrub_stamp":"2024-09-10T21:47:24.060492+0000","last_clean_scrub_stamp":"2024-09-10T21:47:24.060492+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:17:53.701149+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"27'4437","reported_seq":4474,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:31.262564+0000","last_change":"2024-09-10T21:47:31.262564+0000","last_active":"2024-09-10T21:47:31.262564+0000","last_peered":"2024-09-10T21:47:31.262564+0000","last_clean":"2024-09-10T21:47:31.262564+0000","last_became_active":"2024-09-10T21:24:51.901222+0000","last_became_peered":"2024-09-10T21:24:51.901222+0000","last_unstale":"2024-09-10T21:47:31.262564+0000","last_undegraded":"2024-09-10T21:47:31.262564+0000","last_fullsized":"2024-09-10T21:47:31.262564+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4437","last_scrub_stamp":"2024-09-10T21:47:31.262453+0000","last_deep_scrub":"27'4437","last_deep_scrub_stamp":"2024-09-10T21:47:31.262453+0000","last_clean_scrub_stamp":"2024-09-10T21:47:31.262453+0000","objects_scrubbed":0,"log_size":4437,"log_dups_size":0,"ondisk_log_size":4437,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:48:33.845204+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21706,"num_write_kb":463399,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"27'19","reported_seq":61,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:35.064643+0000","last_change":"2024-09-10T21:47:35.064643+0000","last_active":"2024-09-10T21:47:35.064643+0000","last_peered":"2024-09-10T21:47:35.064643+0000","last_clean":"2024-09-10T21:47:35.064643+0000","last_became_active":"2024-09-10T21:24:47.861834+0000","last_became_peered":"2024-09-10T21:24:47.861834+0000","last_unstale":"2024-09-10T21:47:35.064643+0000","last_undegraded":"2024-09-10T21:47:35.064643+0000","last_fullsized":"2024-09-10T21:47:35.064643+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'19","last_scrub_stamp":"2024-09-10T21:47:35.064536+0000","last_deep_scrub":"27'19","last_deep_scrub_stamp":"2024-09-10T21:47:35.064536+0000","last_clean_scrub_stamp":"2024-09-10T21:47:35.064536+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:42:21.811662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"27'19","reported_seq":64,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:26.189881+0000","last_change":"2024-09-10T21:47:26.189835+0000","last_active":"2024-09-10T21:47:26.189881+0000","last_peered":"2024-09-10T21:47:26.189881+0000","last_clean":"2024-09-10T21:47:26.189881+0000","last_became_active":"2024-09-10T21:24:46.851776+0000","last_became_peered":"2024-09-10T21:24:46.851776+0000","last_unstale":"2024-09-10T21:47:26.189881+0000","last_undegraded":"2024-09-10T21:47:26.189881+0000","last_fullsized":"2024-09-10T21:47:26.189881+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'19","last_scrub_stamp":"2024-09-10T21:47:26.189776+0000","last_deep_scrub":"27'19","last_deep_scrub_stamp":"2024-09-10T21:47:26.189776+0000","last_clean_scrub_stamp":"2024-09-10T21:47:26.189776+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:21:13.166679+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"27'5330","reported_seq":5366,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:37.243051+0000","last_change":"2024-09-10T21:47:37.243051+0000","last_active":"2024-09-10T21:47:37.243051+0000","last_peered":"2024-09-10T21:47:37.243051+0000","last_clean":"2024-09-10T21:47:37.243051+0000","last_became_active":"2024-09-10T21:24:51.903533+0000","last_became_peered":"2024-09-10T21:24:51.903533+0000","last_unstale":"2024-09-10T21:47:37.243051+0000","last_undegraded":"2024-09-10T21:47:37.243051+0000","last_fullsized":"2024-09-10T21:47:37.243051+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'5330","last_scrub_stamp":"2024-09-10T21:47:37.242968+0000","last_deep_scrub":"27'5330","last_deep_scrub_stamp":"2024-09-10T21:47:37.242968+0000","last_clean_scrub_stamp":"2024-09-10T21:47:37.242968+0000","objects_scrubbed":0,"log_size":5330,"log_dups_size":0,"ondisk_log_size":5330,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:33:36.251494+0000","scrub_duration":35,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26446,"num_write_kb":547906,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"27'14","reported_seq":55,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:29.730875+0000","last_change":"2024-09-10T21:47:29.730875+0000","last_active":"2024-09-10T21:47:29.730875+0000","last_peered":"2024-09-10T21:47:29.730875+0000","last_clean":"2024-09-10T21:47:29.730875+0000","last_became_active":"2024-09-10T21:24:47.857437+0000","last_became_peered":"2024-09-10T21:24:47.857437+0000","last_unstale":"2024-09-10T21:47:29.730875+0000","last_undegraded":"2024-09-10T21:47:29.730875+0000","last_fullsized":"2024-09-10T21:47:29.730875+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'14","last_scrub_stamp":"2024-09-10T21:47:29.730790+0000","last_deep_scrub":"27'14","last_deep_scrub_stamp":"2024-09-10T21:47:29.730790+0000","last_clean_scrub_stamp":"2024-09-10T21:47:29.730790+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:18:38.680088+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"27'5","reported_seq":48,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:26.673602+0000","last_change":"2024-09-10T21:47:26.673602+0000","last_active":"2024-09-10T21:47:26.673602+0000","last_peered":"2024-09-10T21:47:26.673602+0000","last_clean":"2024-09-10T21:47:26.673602+0000","last_became_active":"2024-09-10T21:24:46.859796+0000","last_became_peered":"2024-09-10T21:24:46.859796+0000","last_unstale":"2024-09-10T21:47:26.673602+0000","last_undegraded":"2024-09-10T21:47:26.673602+0000","last_fullsized":"2024-09-10T21:47:26.673602+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'5","last_scrub_stamp":"2024-09-10T21:47:26.673501+0000","last_deep_scrub":"27'5","last_deep_scrub_stamp":"2024-09-10T21:47:26.673501+0000","last_clean_scrub_stamp":"2024-09-10T21:47:26.673501+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:31:05.182026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"27'5017","reported_seq":5053,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:32.183555+0000","last_change":"2024-09-10T21:47:32.183555+0000","last_active":"2024-09-10T21:47:32.183555+0000","last_peered":"2024-09-10T21:47:32.183555+0000","last_clean":"2024-09-10T21:47:32.183555+0000","last_became_active":"2024-09-10T21:24:52.067409+0000","last_became_peered":"2024-09-10T21:24:52.067409+0000","last_unstale":"2024-09-10T21:47:32.183555+0000","last_undegraded":"2024-09-10T21:47:32.183555+0000","last_fullsized":"2024-09-10T21:47:32.183555+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'5017","last_scrub_stamp":"2024-09-10T21:47:32.183482+0000","last_deep_scrub":"27'5017","last_deep_scrub_stamp":"2024-09-10T21:47:32.183482+0000","last_clean_scrub_stamp":"2024-09-10T21:47:32.183482+0000","objects_scrubbed":0,"log_size":5017,"log_dups_size":0,"ondisk_log_size":5017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:44:25.121300+0000","scrub_duration":27,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24522,"num_write_kb":548501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"27'30","reported_seq":72,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:37.098706+0000","last_change":"2024-09-10T21:47:37.098706+0000","last_active":"2024-09-10T21:47:37.098706+0000","last_peered":"2024-09-10T21:47:37.098706+0000","last_clean":"2024-09-10T21:47:37.098706+0000","last_became_active":"2024-09-10T21:24:47.858343+0000","last_became_peered":"2024-09-10T21:24:47.858343+0000","last_unstale":"2024-09-10T21:47:37.098706+0000","last_undegraded":"2024-09-10T21:47:37.098706+0000","last_fullsized":"2024-09-10T21:47:37.098706+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'30","last_scrub_stamp":"2024-09-10T21:47:37.098619+0000","last_deep_scrub":"27'30","last_deep_scrub_stamp":"2024-09-10T21:47:37.098619+0000","last_clean_scrub_stamp":"2024-09-10T21:47:37.098619+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:48:11.053398+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"27'3","reported_seq":46,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:26.757436+0000","last_change":"2024-09-10T21:47:26.757436+0000","last_active":"2024-09-10T21:47:26.757436+0000","last_peered":"2024-09-10T21:47:26.757436+0000","last_clean":"2024-09-10T21:47:26.757436+0000","last_became_active":"2024-09-10T21:24:46.858696+0000","last_became_peered":"2024-09-10T21:24:46.858696+0000","last_unstale":"2024-09-10T21:47:26.757436+0000","last_undegraded":"2024-09-10T21:47:26.757436+0000","last_fullsized":"2024-09-10T21:47:26.757436+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'3","last_scrub_stamp":"2024-09-10T21:47:26.757356+0000","last_deep_scrub":"27'3","last_deep_scrub_stamp":"2024-09-10T21:47:26.757356+0000","last_clean_scrub_stamp":"2024-09-10T21:47:26.757356+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:53:05.632605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"27'4314","reported_seq":4358,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:38.290745+0000","last_change":"2024-09-10T21:47:38.290745+0000","last_active":"2024-09-10T21:47:38.290745+0000","last_peered":"2024-09-10T21:47:38.290745+0000","last_clean":"2024-09-10T21:47:38.290745+0000","last_became_active":"2024-09-10T21:25:28.802224+0000","last_became_peered":"2024-09-10T21:25:28.802224+0000","last_unstale":"2024-09-10T21:47:38.290745+0000","last_undegraded":"2024-09-10T21:47:38.290745+0000","last_fullsized":"2024-09-10T21:47:38.290745+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4314","last_scrub_stamp":"2024-09-10T21:47:38.290650+0000","last_deep_scrub":"27'4314","last_deep_scrub_stamp":"2024-09-10T21:47:38.290650+0000","last_clean_scrub_stamp":"2024-09-10T21:47:38.290650+0000","objects_scrubbed":0,"log_size":4314,"log_dups_size":0,"ondisk_log_size":4314,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:07:39.075208+0000","scrub_duration":31,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20768,"num_write_kb":476051,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7,2],"acting":[3,5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"27'23","reported_seq":65,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:28.196248+0000","last_change":"2024-09-10T21:47:28.196248+0000","last_active":"2024-09-10T21:47:28.196248+0000","last_peered":"2024-09-10T21:47:28.196248+0000","last_clean":"2024-09-10T21:47:28.196248+0000","last_became_active":"2024-09-10T21:24:47.857429+0000","last_became_peered":"2024-09-10T21:24:47.857429+0000","last_unstale":"2024-09-10T21:47:28.196248+0000","last_undegraded":"2024-09-10T21:47:28.196248+0000","last_fullsized":"2024-09-10T21:47:28.196248+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'23","last_scrub_stamp":"2024-09-10T21:47:28.196181+0000","last_deep_scrub":"27'23","last_deep_scrub_stamp":"2024-09-10T21:47:28.196181+0000","last_clean_scrub_stamp":"2024-09-10T21:47:28.196181+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:27:14.251597+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"27'8","reported_seq":52,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:22.166358+0000","last_change":"2024-09-10T21:47:22.166358+0000","last_active":"2024-09-10T21:47:22.166358+0000","last_peered":"2024-09-10T21:47:22.166358+0000","last_clean":"2024-09-10T21:47:22.166358+0000","last_became_active":"2024-09-10T21:24:46.859299+0000","last_became_peered":"2024-09-10T21:24:46.859299+0000","last_unstale":"2024-09-10T21:47:22.166358+0000","last_undegraded":"2024-09-10T21:47:22.166358+0000","last_fullsized":"2024-09-10T21:47:22.166358+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'8","last_scrub_stamp":"2024-09-10T21:47:22.166302+0000","last_deep_scrub":"27'8","last_deep_scrub_stamp":"2024-09-10T21:47:22.166302+0000","last_clean_scrub_stamp":"2024-09-10T21:47:22.166302+0000","objects_scrubbed":2,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:48:22.481684+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"27'4196","reported_seq":4232,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:41.643790+0000","last_change":"2024-09-10T21:47:41.643790+0000","last_active":"2024-09-10T21:47:41.643790+0000","last_peered":"2024-09-10T21:47:41.643790+0000","last_clean":"2024-09-10T21:47:41.643790+0000","last_became_active":"2024-09-10T21:24:52.062327+0000","last_became_peered":"2024-09-10T21:24:52.062327+0000","last_unstale":"2024-09-10T21:47:41.643790+0000","last_undegraded":"2024-09-10T21:47:41.643790+0000","last_fullsized":"2024-09-10T21:47:41.643790+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4196","last_scrub_stamp":"2024-09-10T21:47:41.643695+0000","last_deep_scrub":"27'4196","last_deep_scrub_stamp":"2024-09-10T21:47:41.643695+0000","last_clean_scrub_stamp":"2024-09-10T21:47:41.643695+0000","objects_scrubbed":0,"log_size":4196,"log_dups_size":0,"ondisk_log_size":4196,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:03:23.913687+0000","scrub_duration":35,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20531,"num_write_kb":446498,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"27'18","reported_seq":60,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:36.084330+0000","last_change":"2024-09-10T21:47:36.084330+0000","last_active":"2024-09-10T21:47:36.084330+0000","last_peered":"2024-09-10T21:47:36.084330+0000","last_clean":"2024-09-10T21:47:36.084330+0000","last_became_active":"2024-09-10T21:24:47.870276+0000","last_became_peered":"2024-09-10T21:24:47.870276+0000","last_unstale":"2024-09-10T21:47:36.084330+0000","last_undegraded":"2024-09-10T21:47:36.084330+0000","last_fullsized":"2024-09-10T21:47:36.084330+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'18","last_scrub_stamp":"2024-09-10T21:47:36.084255+0000","last_deep_scrub":"27'18","last_deep_scrub_stamp":"2024-09-10T21:47:36.084255+0000","last_clean_scrub_stamp":"2024-09-10T21:47:36.084255+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:04:31.530529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"27'371","reported_seq":414,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:25.659903+0000","last_change":"2024-09-10T21:47:25.659903+0000","last_active":"2024-09-10T21:47:25.659903+0000","last_peered":"2024-09-10T21:47:25.659903+0000","last_clean":"2024-09-10T21:47:25.659903+0000","last_became_active":"2024-09-10T21:24:46.866899+0000","last_became_peered":"2024-09-10T21:24:46.866899+0000","last_unstale":"2024-09-10T21:47:25.659903+0000","last_undegraded":"2024-09-10T21:47:25.659903+0000","last_fullsized":"2024-09-10T21:47:25.659903+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'371","last_scrub_stamp":"2024-09-10T21:47:25.659813+0000","last_deep_scrub":"27'371","last_deep_scrub_stamp":"2024-09-10T21:47:25.659813+0000","last_clean_scrub_stamp":"2024-09-10T21:47:25.659813+0000","objects_scrubbed":4,"log_size":371,"log_dups_size":0,"ondisk_log_size":371,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:17:10.717140+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194976,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":367,"num_write_kb":4285,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"27'3971","reported_seq":4007,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:35.949172+0000","last_change":"2024-09-10T21:47:35.949172+0000","last_active":"2024-09-10T21:47:35.949172+0000","last_peered":"2024-09-10T21:47:35.949172+0000","last_clean":"2024-09-10T21:47:35.949172+0000","last_became_active":"2024-09-10T21:24:51.895398+0000","last_became_peered":"2024-09-10T21:24:51.895398+0000","last_unstale":"2024-09-10T21:47:35.949172+0000","last_undegraded":"2024-09-10T21:47:35.949172+0000","last_fullsized":"2024-09-10T21:47:35.949172+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'3971","last_scrub_stamp":"2024-09-10T21:47:35.949076+0000","last_deep_scrub":"27'3971","last_deep_scrub_stamp":"2024-09-10T21:47:35.949076+0000","last_clean_scrub_stamp":"2024-09-10T21:47:35.949076+0000","objects_scrubbed":0,"log_size":3971,"log_dups_size":0,"ondisk_log_size":3971,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:13:29.247480+0000","scrub_duration":31,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19337,"num_write_kb":445978,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"27'22","reported_seq":64,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:28.142815+0000","last_change":"2024-09-10T21:47:28.142815+0000","last_active":"2024-09-10T21:47:28.142815+0000","last_peered":"2024-09-10T21:47:28.142815+0000","last_clean":"2024-09-10T21:47:28.142815+0000","last_became_active":"2024-09-10T21:24:47.857284+0000","last_became_peered":"2024-09-10T21:24:47.857284+0000","last_unstale":"2024-09-10T21:47:28.142815+0000","last_undegraded":"2024-09-10T21:47:28.142815+0000","last_fullsized":"2024-09-10T21:47:28.142815+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'22","last_scrub_stamp":"2024-09-10T21:47:28.142739+0000","last_deep_scrub":"27'22","last_deep_scrub_stamp":"2024-09-10T21:47:28.142739+0000","last_clean_scrub_stamp":"2024-09-10T21:47:28.142739+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:47:52.403509+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"27'34","reported_seq":77,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:25.963747+0000","last_change":"2024-09-10T21:47:25.963747+0000","last_active":"2024-09-10T21:47:25.963747+0000","last_peered":"2024-09-10T21:47:25.963747+0000","last_clean":"2024-09-10T21:47:25.963747+0000","last_became_active":"2024-09-10T21:24:46.859332+0000","last_became_peered":"2024-09-10T21:24:46.859332+0000","last_unstale":"2024-09-10T21:47:25.963747+0000","last_undegraded":"2024-09-10T21:47:25.963747+0000","last_fullsized":"2024-09-10T21:47:25.963747+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'34","last_scrub_stamp":"2024-09-10T21:47:25.963678+0000","last_deep_scrub":"27'34","last_deep_scrub_stamp":"2024-09-10T21:47:25.963678+0000","last_clean_scrub_stamp":"2024-09-10T21:47:25.963678+0000","objects_scrubbed":2,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:34:01.110519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":33,"num_write_kb":30,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"27'4136","reported_seq":4172,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:36.212386+0000","last_change":"2024-09-10T21:47:36.212386+0000","last_active":"2024-09-10T21:47:36.212386+0000","last_peered":"2024-09-10T21:47:36.212386+0000","last_clean":"2024-09-10T21:47:36.212386+0000","last_became_active":"2024-09-10T21:24:51.898642+0000","last_became_peered":"2024-09-10T21:24:51.898642+0000","last_unstale":"2024-09-10T21:47:36.212386+0000","last_undegraded":"2024-09-10T21:47:36.212386+0000","last_fullsized":"2024-09-10T21:47:36.212386+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4136","last_scrub_stamp":"2024-09-10T21:47:36.212285+0000","last_deep_scrub":"27'4136","last_deep_scrub_stamp":"2024-09-10T21:47:36.212285+0000","last_clean_scrub_stamp":"2024-09-10T21:47:36.212285+0000","objects_scrubbed":0,"log_size":4136,"log_dups_size":0,"ondisk_log_size":4136,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:46:44.829087+0000","scrub_duration":27,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20561,"num_write_kb":446825,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"27'23","reported_seq":65,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:25.114417+0000","last_change":"2024-09-10T21:47:25.114417+0000","last_active":"2024-09-10T21:47:25.114417+0000","last_peered":"2024-09-10T21:47:25.114417+0000","last_clean":"2024-09-10T21:47:25.114417+0000","last_became_active":"2024-09-10T21:24:47.865016+0000","last_became_peered":"2024-09-10T21:24:47.865016+0000","last_unstale":"2024-09-10T21:47:25.114417+0000","last_undegraded":"2024-09-10T21:47:25.114417+0000","last_fullsized":"2024-09-10T21:47:25.114417+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'23","last_scrub_stamp":"2024-09-10T21:47:25.114353+0000","last_deep_scrub":"27'23","last_deep_scrub_stamp":"2024-09-10T21:47:25.114353+0000","last_clean_scrub_stamp":"2024-09-10T21:47:25.114353+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:57:06.271532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"27'3","reported_seq":46,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:27.775157+0000","last_change":"2024-09-10T21:47:27.775157+0000","last_active":"2024-09-10T21:47:27.775157+0000","last_peered":"2024-09-10T21:47:27.775157+0000","last_clean":"2024-09-10T21:47:27.775157+0000","last_became_active":"2024-09-10T21:24:46.861136+0000","last_became_peered":"2024-09-10T21:24:46.861136+0000","last_unstale":"2024-09-10T21:47:27.775157+0000","last_undegraded":"2024-09-10T21:47:27.775157+0000","last_fullsized":"2024-09-10T21:47:27.775157+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'3","last_scrub_stamp":"2024-09-10T21:47:27.775078+0000","last_deep_scrub":"27'3","last_deep_scrub_stamp":"2024-09-10T21:47:27.775078+0000","last_clean_scrub_stamp":"2024-09-10T21:47:27.775078+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:27:13.830494+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"27'4156","reported_seq":4192,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:42.275887+0000","last_change":"2024-09-10T21:47:42.275887+0000","last_active":"2024-09-10T21:47:42.275887+0000","last_peered":"2024-09-10T21:47:42.275887+0000","last_clean":"2024-09-10T21:47:42.275887+0000","last_became_active":"2024-09-10T21:24:51.907062+0000","last_became_peered":"2024-09-10T21:24:51.907062+0000","last_unstale":"2024-09-10T21:47:42.275887+0000","last_undegraded":"2024-09-10T21:47:42.275887+0000","last_fullsized":"2024-09-10T21:47:42.275887+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4156","last_scrub_stamp":"2024-09-10T21:47:42.275800+0000","last_deep_scrub":"27'4156","last_deep_scrub_stamp":"2024-09-10T21:47:42.275800+0000","last_clean_scrub_stamp":"2024-09-10T21:47:42.275800+0000","objects_scrubbed":0,"log_size":4156,"log_dups_size":0,"ondisk_log_size":4156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:35:15.857948+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20153,"num_write_kb":457162,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"27'25","reported_seq":66,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:31.735901+0000","last_change":"2024-09-10T21:47:31.735901+0000","last_active":"2024-09-10T21:47:31.735901+0000","last_peered":"2024-09-10T21:47:31.735901+0000","last_clean":"2024-09-10T21:47:31.735901+0000","last_became_active":"2024-09-10T21:24:47.861498+0000","last_became_peered":"2024-09-10T21:24:47.861498+0000","last_unstale":"2024-09-10T21:47:31.735901+0000","last_undegraded":"2024-09-10T21:47:31.735901+0000","last_fullsized":"2024-09-10T21:47:31.735901+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'25","last_scrub_stamp":"2024-09-10T21:47:31.735802+0000","last_deep_scrub":"27'25","last_deep_scrub_stamp":"2024-09-10T21:47:31.735802+0000","last_clean_scrub_stamp":"2024-09-10T21:47:31.735802+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:47:25.653045+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"27'3","reported_seq":46,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:27.718580+0000","last_change":"2024-09-10T21:47:27.718580+0000","last_active":"2024-09-10T21:47:27.718580+0000","last_peered":"2024-09-10T21:47:27.718580+0000","last_clean":"2024-09-10T21:47:27.718580+0000","last_became_active":"2024-09-10T21:24:46.871367+0000","last_became_peered":"2024-09-10T21:24:46.871367+0000","last_unstale":"2024-09-10T21:47:27.718580+0000","last_undegraded":"2024-09-10T21:47:27.718580+0000","last_fullsized":"2024-09-10T21:47:27.718580+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'3","last_scrub_stamp":"2024-09-10T21:47:27.718477+0000","last_deep_scrub":"27'3","last_deep_scrub_stamp":"2024-09-10T21:47:27.718477+0000","last_clean_scrub_stamp":"2024-09-10T21:47:27.718477+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:30:07.264982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"27'3798","reported_seq":3834,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:31.159945+0000","last_change":"2024-09-10T21:47:31.159945+0000","last_active":"2024-09-10T21:47:31.159945+0000","last_peered":"2024-09-10T21:47:31.159945+0000","last_clean":"2024-09-10T21:47:31.159945+0000","last_became_active":"2024-09-10T21:24:52.068348+0000","last_became_peered":"2024-09-10T21:24:52.068348+0000","last_unstale":"2024-09-10T21:47:31.159945+0000","last_undegraded":"2024-09-10T21:47:31.159945+0000","last_fullsized":"2024-09-10T21:47:31.159945+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'3798","last_scrub_stamp":"2024-09-10T21:47:31.159867+0000","last_deep_scrub":"27'3798","last_deep_scrub_stamp":"2024-09-10T21:47:31.159867+0000","last_clean_scrub_stamp":"2024-09-10T21:47:31.159867+0000","objects_scrubbed":0,"log_size":3798,"log_dups_size":0,"ondisk_log_size":3798,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:18:09.732689+0000","scrub_duration":27,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18329,"num_write_kb":408933,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"27'5643","reported_seq":5681,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:38.270407+0000","last_change":"2024-09-10T21:47:38.270407+0000","last_active":"2024-09-10T21:47:38.270407+0000","last_peered":"2024-09-10T21:47:38.270407+0000","last_clean":"2024-09-10T21:47:38.270407+0000","last_became_active":"2024-09-10T21:24:52.067330+0000","last_became_peered":"2024-09-10T21:24:52.067330+0000","last_unstale":"2024-09-10T21:47:38.270407+0000","last_undegraded":"2024-09-10T21:47:38.270407+0000","last_fullsized":"2024-09-10T21:47:38.270407+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'5643","last_scrub_stamp":"2024-09-10T21:47:38.270269+0000","last_deep_scrub":"27'5643","last_deep_scrub_stamp":"2024-09-10T21:47:38.270269+0000","last_clean_scrub_stamp":"2024-09-10T21:47:38.270269+0000","objects_scrubbed":0,"log_size":5643,"log_dups_size":0,"ondisk_log_size":5643,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:09.431599+0000","scrub_duration":35,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27908,"num_write_kb":566060,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"27'20","reported_seq":61,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:32.964137+0000","last_change":"2024-09-10T21:47:32.964137+0000","last_active":"2024-09-10T21:47:32.964137+0000","last_peered":"2024-09-10T21:47:32.964137+0000","last_clean":"2024-09-10T21:47:32.964137+0000","last_became_active":"2024-09-10T21:24:47.862556+0000","last_became_peered":"2024-09-10T21:24:47.862556+0000","last_unstale":"2024-09-10T21:47:32.964137+0000","last_undegraded":"2024-09-10T21:47:32.964137+0000","last_fullsized":"2024-09-10T21:47:32.964137+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'20","last_scrub_stamp":"2024-09-10T21:47:32.964062+0000","last_deep_scrub":"27'20","last_deep_scrub_stamp":"2024-09-10T21:47:32.964062+0000","last_clean_scrub_stamp":"2024-09-10T21:47:32.964062+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:46:22.211162+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"27'2","reported_seq":45,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:28.728520+0000","last_change":"2024-09-10T21:47:28.728520+0000","last_active":"2024-09-10T21:47:28.728520+0000","last_peered":"2024-09-10T21:47:28.728520+0000","last_clean":"2024-09-10T21:47:28.728520+0000","last_became_active":"2024-09-10T21:24:46.862540+0000","last_became_peered":"2024-09-10T21:24:46.862540+0000","last_unstale":"2024-09-10T21:47:28.728520+0000","last_undegraded":"2024-09-10T21:47:28.728520+0000","last_fullsized":"2024-09-10T21:47:28.728520+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2","last_scrub_stamp":"2024-09-10T21:47:28.728443+0000","last_deep_scrub":"27'2","last_deep_scrub_stamp":"2024-09-10T21:47:28.728443+0000","last_clean_scrub_stamp":"2024-09-10T21:47:28.728443+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:21:14.145793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"27'5506","reported_seq":5544,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:37.918920+0000","last_change":"2024-09-10T21:47:37.918860+0000","last_active":"2024-09-10T21:47:37.918920+0000","last_peered":"2024-09-10T21:47:37.918920+0000","last_clean":"2024-09-10T21:47:37.918920+0000","last_became_active":"2024-09-10T21:24:51.900610+0000","last_became_peered":"2024-09-10T21:24:51.900610+0000","last_unstale":"2024-09-10T21:47:37.918920+0000","last_undegraded":"2024-09-10T21:47:37.918920+0000","last_fullsized":"2024-09-10T21:47:37.918920+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'5506","last_scrub_stamp":"2024-09-10T21:47:37.918758+0000","last_deep_scrub":"27'5506","last_deep_scrub_stamp":"2024-09-10T21:47:37.918758+0000","last_clean_scrub_stamp":"2024-09-10T21:47:37.918758+0000","objects_scrubbed":0,"log_size":5506,"log_dups_size":0,"ondisk_log_size":5506,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:54:41.833391+0000","scrub_duration":39,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27800,"num_write_kb":594222,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"27'26","reported_seq":68,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:34.200053+0000","last_change":"2024-09-10T21:47:34.200053+0000","last_active":"2024-09-10T21:47:34.200053+0000","last_peered":"2024-09-10T21:47:34.200053+0000","last_clean":"2024-09-10T21:47:34.200053+0000","last_became_active":"2024-09-10T21:24:47.860930+0000","last_became_peered":"2024-09-10T21:24:47.860930+0000","last_unstale":"2024-09-10T21:47:34.200053+0000","last_undegraded":"2024-09-10T21:47:34.200053+0000","last_fullsized":"2024-09-10T21:47:34.200053+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'26","last_scrub_stamp":"2024-09-10T21:47:34.199983+0000","last_deep_scrub":"27'26","last_deep_scrub_stamp":"2024-09-10T21:47:34.199983+0000","last_clean_scrub_stamp":"2024-09-10T21:47:34.199983+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:35:02.658141+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"27'3","reported_seq":47,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:40.256898+0000","last_change":"2024-09-10T21:47:40.256898+0000","last_active":"2024-09-10T21:47:40.256898+0000","last_peered":"2024-09-10T21:47:40.256898+0000","last_clean":"2024-09-10T21:47:40.256898+0000","last_became_active":"2024-09-10T21:24:46.858265+0000","last_became_peered":"2024-09-10T21:24:46.858265+0000","last_unstale":"2024-09-10T21:47:40.256898+0000","last_undegraded":"2024-09-10T21:47:40.256898+0000","last_fullsized":"2024-09-10T21:47:40.256898+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'3","last_scrub_stamp":"2024-09-10T21:47:40.256832+0000","last_deep_scrub":"27'3","last_deep_scrub_stamp":"2024-09-10T21:47:40.256832+0000","last_clean_scrub_stamp":"2024-09-10T21:47:40.256832+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:21:12.866707+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"27'6","reported_seq":49,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:28.689308+0000","last_change":"2024-09-10T21:47:28.689308+0000","last_active":"2024-09-10T21:47:28.689308+0000","last_peered":"2024-09-10T21:47:28.689308+0000","last_clean":"2024-09-10T21:47:28.689308+0000","last_became_active":"2024-09-10T21:24:46.856506+0000","last_became_peered":"2024-09-10T21:24:46.856506+0000","last_unstale":"2024-09-10T21:47:28.689308+0000","last_undegraded":"2024-09-10T21:47:28.689308+0000","last_fullsized":"2024-09-10T21:47:28.689308+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'6","last_scrub_stamp":"2024-09-10T21:47:28.689227+0000","last_deep_scrub":"27'6","last_deep_scrub_stamp":"2024-09-10T21:47:28.689227+0000","last_clean_scrub_stamp":"2024-09-10T21:47:28.689227+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:16:18.854160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"27'19","reported_seq":61,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:29.197942+0000","last_change":"2024-09-10T21:47:29.197942+0000","last_active":"2024-09-10T21:47:29.197942+0000","last_peered":"2024-09-10T21:47:29.197942+0000","last_clean":"2024-09-10T21:47:29.197942+0000","last_became_active":"2024-09-10T21:24:47.848160+0000","last_became_peered":"2024-09-10T21:24:47.848160+0000","last_unstale":"2024-09-10T21:47:29.197942+0000","last_undegraded":"2024-09-10T21:47:29.197942+0000","last_fullsized":"2024-09-10T21:47:29.197942+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'19","last_scrub_stamp":"2024-09-10T21:47:29.197887+0000","last_deep_scrub":"27'19","last_deep_scrub_stamp":"2024-09-10T21:47:29.197887+0000","last_clean_scrub_stamp":"2024-09-10T21:47:29.197887+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:55.411968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"27'4174","reported_seq":4210,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:32.267288+0000","last_change":"2024-09-10T21:47:32.267288+0000","last_active":"2024-09-10T21:47:32.267288+0000","last_peered":"2024-09-10T21:47:32.267288+0000","last_clean":"2024-09-10T21:47:32.267288+0000","last_became_active":"2024-09-10T21:24:51.903120+0000","last_became_peered":"2024-09-10T21:24:51.903120+0000","last_unstale":"2024-09-10T21:47:32.267288+0000","last_undegraded":"2024-09-10T21:47:32.267288+0000","last_fullsized":"2024-09-10T21:47:32.267288+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4174","last_scrub_stamp":"2024-09-10T21:47:32.267187+0000","last_deep_scrub":"27'4174","last_deep_scrub_stamp":"2024-09-10T21:47:32.267187+0000","last_clean_scrub_stamp":"2024-09-10T21:47:32.267187+0000","objects_scrubbed":0,"log_size":4174,"log_dups_size":0,"ondisk_log_size":4174,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:02:46.924965+0000","scrub_duration":27,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19938,"num_write_kb":435851,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"27'679","reported_seq":723,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:23.326615+0000","last_change":"2024-09-10T21:47:23.326615+0000","last_active":"2024-09-10T21:47:23.326615+0000","last_peered":"2024-09-10T21:47:23.326615+0000","last_clean":"2024-09-10T21:47:23.326615+0000","last_became_active":"2024-09-10T21:24:46.852204+0000","last_became_peered":"2024-09-10T21:24:46.852204+0000","last_unstale":"2024-09-10T21:47:23.326615+0000","last_undegraded":"2024-09-10T21:47:23.326615+0000","last_fullsized":"2024-09-10T21:47:23.326615+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'679","last_scrub_stamp":"2024-09-10T21:47:23.326553+0000","last_deep_scrub":"27'679","last_deep_scrub_stamp":"2024-09-10T21:47:23.326553+0000","last_clean_scrub_stamp":"2024-09-10T21:47:23.326553+0000","objects_scrubbed":1,"log_size":679,"log_dups_size":0,"ondisk_log_size":679,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:48:25.909303+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":672,"num_write_kb":4259,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"27'18","reported_seq":60,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:27.127018+0000","last_change":"2024-09-10T21:47:27.127018+0000","last_active":"2024-09-10T21:47:27.127018+0000","last_peered":"2024-09-10T21:47:27.127018+0000","last_clean":"2024-09-10T21:47:27.127018+0000","last_became_active":"2024-09-10T21:24:47.848851+0000","last_became_peered":"2024-09-10T21:24:47.848851+0000","last_unstale":"2024-09-10T21:47:27.127018+0000","last_undegraded":"2024-09-10T21:47:27.127018+0000","last_fullsized":"2024-09-10T21:47:27.127018+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'18","last_scrub_stamp":"2024-09-10T21:47:27.126943+0000","last_deep_scrub":"27'18","last_deep_scrub_stamp":"2024-09-10T21:47:27.126943+0000","last_clean_scrub_stamp":"2024-09-10T21:47:27.126943+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:02:51.834510+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"27'4902","reported_seq":4938,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:41.334091+0000","last_change":"2024-09-10T21:47:41.334091+0000","last_active":"2024-09-10T21:47:41.334091+0000","last_peered":"2024-09-10T21:47:41.334091+0000","last_clean":"2024-09-10T21:47:41.334091+0000","last_became_active":"2024-09-10T21:24:51.907082+0000","last_became_peered":"2024-09-10T21:24:51.907082+0000","last_unstale":"2024-09-10T21:47:41.334091+0000","last_undegraded":"2024-09-10T21:47:41.334091+0000","last_fullsized":"2024-09-10T21:47:41.334091+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4902","last_scrub_stamp":"2024-09-10T21:47:41.334007+0000","last_deep_scrub":"27'4902","last_deep_scrub_stamp":"2024-09-10T21:47:41.334007+0000","last_clean_scrub_stamp":"2024-09-10T21:47:41.334007+0000","objects_scrubbed":0,"log_size":4902,"log_dups_size":0,"ondisk_log_size":4902,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:31:24.648339+0000","scrub_duration":35,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24324,"num_write_kb":547512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"27'24","reported_seq":67,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:31.763972+0000","last_change":"2024-09-10T21:47:31.763913+0000","last_active":"2024-09-10T21:47:31.763972+0000","last_peered":"2024-09-10T21:47:31.763972+0000","last_clean":"2024-09-10T21:47:31.763972+0000","last_became_active":"2024-09-10T21:24:47.857501+0000","last_became_peered":"2024-09-10T21:24:47.857501+0000","last_unstale":"2024-09-10T21:47:31.763972+0000","last_undegraded":"2024-09-10T21:47:31.763972+0000","last_fullsized":"2024-09-10T21:47:31.763972+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'24","last_scrub_stamp":"2024-09-10T21:47:31.763837+0000","last_deep_scrub":"27'24","last_deep_scrub_stamp":"2024-09-10T21:47:31.763837+0000","last_clean_scrub_stamp":"2024-09-10T21:47:31.763837+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:06:18.700323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"27'25","reported_seq":69,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:25.158200+0000","last_change":"2024-09-10T21:47:25.158155+0000","last_active":"2024-09-10T21:47:25.158200+0000","last_peered":"2024-09-10T21:47:25.158200+0000","last_clean":"2024-09-10T21:47:25.158200+0000","last_became_active":"2024-09-10T21:24:47.867504+0000","last_became_peered":"2024-09-10T21:24:47.867504+0000","last_unstale":"2024-09-10T21:47:25.158200+0000","last_undegraded":"2024-09-10T21:47:25.158200+0000","last_fullsized":"2024-09-10T21:47:25.158200+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'25","last_scrub_stamp":"2024-09-10T21:47:25.158094+0000","last_deep_scrub":"27'25","last_deep_scrub_stamp":"2024-09-10T21:47:25.158094+0000","last_clean_scrub_stamp":"2024-09-10T21:47:25.158094+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:21:12.715415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"27'26","reported_seq":69,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:34.656045+0000","last_change":"2024-09-10T21:47:34.655927+0000","last_active":"2024-09-10T21:47:34.656045+0000","last_peered":"2024-09-10T21:47:34.656045+0000","last_clean":"2024-09-10T21:47:34.656045+0000","last_became_active":"2024-09-10T21:24:47.868168+0000","last_became_peered":"2024-09-10T21:24:47.868168+0000","last_unstale":"2024-09-10T21:47:34.656045+0000","last_undegraded":"2024-09-10T21:47:34.656045+0000","last_fullsized":"2024-09-10T21:47:34.656045+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'26","last_scrub_stamp":"2024-09-10T21:47:34.655825+0000","last_deep_scrub":"27'26","last_deep_scrub_stamp":"2024-09-10T21:47:34.655825+0000","last_clean_scrub_stamp":"2024-09-10T21:47:34.655825+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:17:12.913285+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"27'16","reported_seq":60,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:31.200738+0000","last_change":"2024-09-10T21:47:31.200690+0000","last_active":"2024-09-10T21:47:31.200738+0000","last_peered":"2024-09-10T21:47:31.200738+0000","last_clean":"2024-09-10T21:47:31.200738+0000","last_became_active":"2024-09-10T21:24:47.864100+0000","last_became_peered":"2024-09-10T21:24:47.864100+0000","last_unstale":"2024-09-10T21:47:31.200738+0000","last_undegraded":"2024-09-10T21:47:31.200738+0000","last_fullsized":"2024-09-10T21:47:31.200738+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'16","last_scrub_stamp":"2024-09-10T21:47:31.200632+0000","last_deep_scrub":"27'16","last_deep_scrub_stamp":"2024-09-10T21:47:31.200632+0000","last_clean_scrub_stamp":"2024-09-10T21:47:31.200632+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:35:01.716923+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"27'14","reported_seq":57,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:33.184046+0000","last_change":"2024-09-10T21:47:33.184046+0000","last_active":"2024-09-10T21:47:33.184046+0000","last_peered":"2024-09-10T21:47:33.184046+0000","last_clean":"2024-09-10T21:47:33.184046+0000","last_became_active":"2024-09-10T21:24:47.863970+0000","last_became_peered":"2024-09-10T21:24:47.863970+0000","last_unstale":"2024-09-10T21:47:33.184046+0000","last_undegraded":"2024-09-10T21:47:33.184046+0000","last_fullsized":"2024-09-10T21:47:33.184046+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'14","last_scrub_stamp":"2024-09-10T21:47:33.183974+0000","last_deep_scrub":"27'14","last_deep_scrub_stamp":"2024-09-10T21:47:33.183974+0000","last_clean_scrub_stamp":"2024-09-10T21:47:33.183974+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:32:36.049392+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"27'18","reported_seq":60,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:28.972824+0000","last_change":"2024-09-10T21:47:28.972824+0000","last_active":"2024-09-10T21:47:28.972824+0000","last_peered":"2024-09-10T21:47:28.972824+0000","last_clean":"2024-09-10T21:47:28.972824+0000","last_became_active":"2024-09-10T21:24:47.860471+0000","last_became_peered":"2024-09-10T21:24:47.860471+0000","last_unstale":"2024-09-10T21:47:28.972824+0000","last_undegraded":"2024-09-10T21:47:28.972824+0000","last_fullsized":"2024-09-10T21:47:28.972824+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'18","last_scrub_stamp":"2024-09-10T21:47:28.972767+0000","last_deep_scrub":"27'18","last_deep_scrub_stamp":"2024-09-10T21:47:28.972767+0000","last_clean_scrub_stamp":"2024-09-10T21:47:28.972767+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:52:26.453545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"27'27","reported_seq":69,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:32.802778+0000","last_change":"2024-09-10T21:47:32.802778+0000","last_active":"2024-09-10T21:47:32.802778+0000","last_peered":"2024-09-10T21:47:32.802778+0000","last_clean":"2024-09-10T21:47:32.802778+0000","last_became_active":"2024-09-10T21:24:47.872218+0000","last_became_peered":"2024-09-10T21:24:47.872218+0000","last_unstale":"2024-09-10T21:47:32.802778+0000","last_undegraded":"2024-09-10T21:47:32.802778+0000","last_fullsized":"2024-09-10T21:47:32.802778+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'27","last_scrub_stamp":"2024-09-10T21:47:32.802683+0000","last_deep_scrub":"27'27","last_deep_scrub_stamp":"2024-09-10T21:47:32.802683+0000","last_clean_scrub_stamp":"2024-09-10T21:47:32.802683+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:52:30.283461+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"27'14","reported_seq":57,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:29.184527+0000","last_change":"2024-09-10T21:47:29.184527+0000","last_active":"2024-09-10T21:47:29.184527+0000","last_peered":"2024-09-10T21:47:29.184527+0000","last_clean":"2024-09-10T21:47:29.184527+0000","last_became_active":"2024-09-10T21:24:47.859649+0000","last_became_peered":"2024-09-10T21:24:47.859649+0000","last_unstale":"2024-09-10T21:47:29.184527+0000","last_undegraded":"2024-09-10T21:47:29.184527+0000","last_fullsized":"2024-09-10T21:47:29.184527+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'14","last_scrub_stamp":"2024-09-10T21:47:29.184462+0000","last_deep_scrub":"27'14","last_deep_scrub_stamp":"2024-09-10T21:47:29.184462+0000","last_clean_scrub_stamp":"2024-09-10T21:47:29.184462+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:51:19.452278+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"27'18","reported_seq":47,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:27.212647+0000","last_change":"2024-09-10T21:47:27.212647+0000","last_active":"2024-09-10T21:47:27.212647+0000","last_peered":"2024-09-10T21:47:27.212647+0000","last_clean":"2024-09-10T21:47:27.212647+0000","last_became_active":"2024-09-10T21:25:28.508684+0000","last_became_peered":"2024-09-10T21:25:28.508684+0000","last_unstale":"2024-09-10T21:47:27.212647+0000","last_undegraded":"2024-09-10T21:47:27.212647+0000","last_fullsized":"2024-09-10T21:47:27.212647+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'18","last_scrub_stamp":"2024-09-10T21:47:27.212575+0000","last_deep_scrub":"27'18","last_deep_scrub_stamp":"2024-09-10T21:47:27.212575+0000","last_clean_scrub_stamp":"2024-09-10T21:47:27.212575+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:02:42.041087+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.29","version":"27'23","reported_seq":52,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:30.198285+0000","last_change":"2024-09-10T21:47:30.198230+0000","last_active":"2024-09-10T21:47:30.198285+0000","last_peered":"2024-09-10T21:47:30.198285+0000","last_clean":"2024-09-10T21:47:30.198285+0000","last_became_active":"2024-09-10T21:25:28.512230+0000","last_became_peered":"2024-09-10T21:25:28.512230+0000","last_unstale":"2024-09-10T21:47:30.198285+0000","last_undegraded":"2024-09-10T21:47:30.198285+0000","last_fullsized":"2024-09-10T21:47:30.198285+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'23","last_scrub_stamp":"2024-09-10T21:47:30.198142+0000","last_deep_scrub":"27'23","last_deep_scrub_stamp":"2024-09-10T21:47:30.198142+0000","last_clean_scrub_stamp":"2024-09-10T21:47:30.198142+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:46:41.750896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2a","version":"27'22","reported_seq":64,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:35.656763+0000","last_change":"2024-09-10T21:47:35.656763+0000","last_active":"2024-09-10T21:47:35.656763+0000","last_peered":"2024-09-10T21:47:35.656763+0000","last_clean":"2024-09-10T21:47:35.656763+0000","last_became_active":"2024-09-10T21:24:47.868671+0000","last_became_peered":"2024-09-10T21:24:47.868671+0000","last_unstale":"2024-09-10T21:47:35.656763+0000","last_undegraded":"2024-09-10T21:47:35.656763+0000","last_fullsized":"2024-09-10T21:47:35.656763+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'22","last_scrub_stamp":"2024-09-10T21:47:35.656676+0000","last_deep_scrub":"27'22","last_deep_scrub_stamp":"2024-09-10T21:47:35.656676+0000","last_clean_scrub_stamp":"2024-09-10T21:47:35.656676+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:00:48.878921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"27'28","reported_seq":71,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:32.171947+0000","last_change":"2024-09-10T21:47:32.171947+0000","last_active":"2024-09-10T21:47:32.171947+0000","last_peered":"2024-09-10T21:47:32.171947+0000","last_clean":"2024-09-10T21:47:32.171947+0000","last_became_active":"2024-09-10T21:24:47.857116+0000","last_became_peered":"2024-09-10T21:24:47.857116+0000","last_unstale":"2024-09-10T21:47:32.171947+0000","last_undegraded":"2024-09-10T21:47:32.171947+0000","last_fullsized":"2024-09-10T21:47:32.171947+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'28","last_scrub_stamp":"2024-09-10T21:47:32.171855+0000","last_deep_scrub":"27'28","last_deep_scrub_stamp":"2024-09-10T21:47:32.171855+0000","last_clean_scrub_stamp":"2024-09-10T21:47:32.171855+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:12:14.615828+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"27'19","reported_seq":61,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:33.830699+0000","last_change":"2024-09-10T21:47:33.830699+0000","last_active":"2024-09-10T21:47:33.830699+0000","last_peered":"2024-09-10T21:47:33.830699+0000","last_clean":"2024-09-10T21:47:33.830699+0000","last_became_active":"2024-09-10T21:24:47.871350+0000","last_became_peered":"2024-09-10T21:24:47.871350+0000","last_unstale":"2024-09-10T21:47:33.830699+0000","last_undegraded":"2024-09-10T21:47:33.830699+0000","last_fullsized":"2024-09-10T21:47:33.830699+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'19","last_scrub_stamp":"2024-09-10T21:47:33.830616+0000","last_deep_scrub":"27'19","last_deep_scrub_stamp":"2024-09-10T21:47:33.830616+0000","last_clean_scrub_stamp":"2024-09-10T21:47:33.830616+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:13:07.813564+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"27'20","reported_seq":63,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:38.071694+0000","last_change":"2024-09-10T21:47:38.071694+0000","last_active":"2024-09-10T21:47:38.071694+0000","last_peered":"2024-09-10T21:47:38.071694+0000","last_clean":"2024-09-10T21:47:38.071694+0000","last_became_active":"2024-09-10T21:24:47.861007+0000","last_became_peered":"2024-09-10T21:24:47.861007+0000","last_unstale":"2024-09-10T21:47:38.071694+0000","last_undegraded":"2024-09-10T21:47:38.071694+0000","last_fullsized":"2024-09-10T21:47:38.071694+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'20","last_scrub_stamp":"2024-09-10T21:47:38.071574+0000","last_deep_scrub":"27'20","last_deep_scrub_stamp":"2024-09-10T21:47:38.071574+0000","last_clean_scrub_stamp":"2024-09-10T21:47:38.071574+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:56:10.916482+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"27'27","reported_seq":69,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:31.194852+0000","last_change":"2024-09-10T21:47:31.194852+0000","last_active":"2024-09-10T21:47:31.194852+0000","last_peered":"2024-09-10T21:47:31.194852+0000","last_clean":"2024-09-10T21:47:31.194852+0000","last_became_active":"2024-09-10T21:24:47.863932+0000","last_became_peered":"2024-09-10T21:24:47.863932+0000","last_unstale":"2024-09-10T21:47:31.194852+0000","last_undegraded":"2024-09-10T21:47:31.194852+0000","last_fullsized":"2024-09-10T21:47:31.194852+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'27","last_scrub_stamp":"2024-09-10T21:47:31.194797+0000","last_deep_scrub":"27'27","last_deep_scrub_stamp":"2024-09-10T21:47:31.194797+0000","last_clean_scrub_stamp":"2024-09-10T21:47:31.194797+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:38:09.333593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"27'21","reported_seq":62,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:36.620568+0000","last_change":"2024-09-10T21:47:36.620568+0000","last_active":"2024-09-10T21:47:36.620568+0000","last_peered":"2024-09-10T21:47:36.620568+0000","last_clean":"2024-09-10T21:47:36.620568+0000","last_became_active":"2024-09-10T21:24:47.865587+0000","last_became_peered":"2024-09-10T21:24:47.865587+0000","last_unstale":"2024-09-10T21:47:36.620568+0000","last_undegraded":"2024-09-10T21:47:36.620568+0000","last_fullsized":"2024-09-10T21:47:36.620568+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'21","last_scrub_stamp":"2024-09-10T21:47:36.620488+0000","last_deep_scrub":"27'21","last_deep_scrub_stamp":"2024-09-10T21:47:36.620488+0000","last_clean_scrub_stamp":"2024-09-10T21:47:36.620488+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:29:04.850738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"27'22","reported_seq":63,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:37.664031+0000","last_change":"2024-09-10T21:47:37.664031+0000","last_active":"2024-09-10T21:47:37.664031+0000","last_peered":"2024-09-10T21:47:37.664031+0000","last_clean":"2024-09-10T21:47:37.664031+0000","last_became_active":"2024-09-10T21:24:47.865837+0000","last_became_peered":"2024-09-10T21:24:47.865837+0000","last_unstale":"2024-09-10T21:47:37.664031+0000","last_undegraded":"2024-09-10T21:47:37.664031+0000","last_fullsized":"2024-09-10T21:47:37.664031+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'22","last_scrub_stamp":"2024-09-10T21:47:37.663943+0000","last_deep_scrub":"27'22","last_deep_scrub_stamp":"2024-09-10T21:47:37.663943+0000","last_clean_scrub_stamp":"2024-09-10T21:47:37.663943+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:22:59.372202+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"27'21","reported_seq":62,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:34.966079+0000","last_change":"2024-09-10T21:47:34.966079+0000","last_active":"2024-09-10T21:47:34.966079+0000","last_peered":"2024-09-10T21:47:34.966079+0000","last_clean":"2024-09-10T21:47:34.966079+0000","last_became_active":"2024-09-10T21:24:47.846283+0000","last_became_peered":"2024-09-10T21:24:47.846283+0000","last_unstale":"2024-09-10T21:47:34.966079+0000","last_undegraded":"2024-09-10T21:47:34.966079+0000","last_fullsized":"2024-09-10T21:47:34.966079+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'21","last_scrub_stamp":"2024-09-10T21:47:34.966019+0000","last_deep_scrub":"27'21","last_deep_scrub_stamp":"2024-09-10T21:47:34.966019+0000","last_clean_scrub_stamp":"2024-09-10T21:47:34.966019+0000","objects_scrubbed":0,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:42.089024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"27'23","reported_seq":65,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:30.197763+0000","last_change":"2024-09-10T21:47:30.197763+0000","last_active":"2024-09-10T21:47:30.197763+0000","last_peered":"2024-09-10T21:47:30.197763+0000","last_clean":"2024-09-10T21:47:30.197763+0000","last_became_active":"2024-09-10T21:24:47.844517+0000","last_became_peered":"2024-09-10T21:24:47.844517+0000","last_unstale":"2024-09-10T21:47:30.197763+0000","last_undegraded":"2024-09-10T21:47:30.197763+0000","last_fullsized":"2024-09-10T21:47:30.197763+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'23","last_scrub_stamp":"2024-09-10T21:47:30.197688+0000","last_deep_scrub":"27'23","last_deep_scrub_stamp":"2024-09-10T21:47:30.197688+0000","last_clean_scrub_stamp":"2024-09-10T21:47:30.197688+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:18:03.294904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"27'25","reported_seq":67,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:28.212053+0000","last_change":"2024-09-10T21:47:28.212053+0000","last_active":"2024-09-10T21:47:28.212053+0000","last_peered":"2024-09-10T21:47:28.212053+0000","last_clean":"2024-09-10T21:47:28.212053+0000","last_became_active":"2024-09-10T21:24:47.862185+0000","last_became_peered":"2024-09-10T21:24:47.862185+0000","last_unstale":"2024-09-10T21:47:28.212053+0000","last_undegraded":"2024-09-10T21:47:28.212053+0000","last_fullsized":"2024-09-10T21:47:28.212053+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'25","last_scrub_stamp":"2024-09-10T21:47:28.211979+0000","last_deep_scrub":"27'25","last_deep_scrub_stamp":"2024-09-10T21:47:28.211979+0000","last_clean_scrub_stamp":"2024-09-10T21:47:28.211979+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:46:36.828782+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"27'17","reported_seq":59,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:29.186874+0000","last_change":"2024-09-10T21:47:29.186874+0000","last_active":"2024-09-10T21:47:29.186874+0000","last_peered":"2024-09-10T21:47:29.186874+0000","last_clean":"2024-09-10T21:47:29.186874+0000","last_became_active":"2024-09-10T21:24:47.861045+0000","last_became_peered":"2024-09-10T21:24:47.861045+0000","last_unstale":"2024-09-10T21:47:29.186874+0000","last_undegraded":"2024-09-10T21:47:29.186874+0000","last_fullsized":"2024-09-10T21:47:29.186874+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'17","last_scrub_stamp":"2024-09-10T21:47:29.186783+0000","last_deep_scrub":"27'17","last_deep_scrub_stamp":"2024-09-10T21:47:29.186783+0000","last_clean_scrub_stamp":"2024-09-10T21:47:29.186783+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:12:34.155979+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"27'17","reported_seq":58,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:35.849939+0000","last_change":"2024-09-10T21:47:35.849939+0000","last_active":"2024-09-10T21:47:35.849939+0000","last_peered":"2024-09-10T21:47:35.849939+0000","last_clean":"2024-09-10T21:47:35.849939+0000","last_became_active":"2024-09-10T21:24:47.867113+0000","last_became_peered":"2024-09-10T21:24:47.867113+0000","last_unstale":"2024-09-10T21:47:35.849939+0000","last_undegraded":"2024-09-10T21:47:35.849939+0000","last_fullsized":"2024-09-10T21:47:35.849939+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'17","last_scrub_stamp":"2024-09-10T21:47:35.849852+0000","last_deep_scrub":"27'17","last_deep_scrub_stamp":"2024-09-10T21:47:35.849852+0000","last_clean_scrub_stamp":"2024-09-10T21:47:35.849852+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:17:06.555027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"27'28","reported_seq":69,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:34.873759+0000","last_change":"2024-09-10T21:47:34.873759+0000","last_active":"2024-09-10T21:47:34.873759+0000","last_peered":"2024-09-10T21:47:34.873759+0000","last_clean":"2024-09-10T21:47:34.873759+0000","last_became_active":"2024-09-10T21:24:47.867126+0000","last_became_peered":"2024-09-10T21:24:47.867126+0000","last_unstale":"2024-09-10T21:47:34.873759+0000","last_undegraded":"2024-09-10T21:47:34.873759+0000","last_fullsized":"2024-09-10T21:47:34.873759+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'28","last_scrub_stamp":"2024-09-10T21:47:34.873672+0000","last_deep_scrub":"27'28","last_deep_scrub_stamp":"2024-09-10T21:47:34.873672+0000","last_clean_scrub_stamp":"2024-09-10T21:47:34.873672+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:38:35.516740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"27'26","reported_seq":68,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:39.051777+0000","last_change":"2024-09-10T21:47:39.051777+0000","last_active":"2024-09-10T21:47:39.051777+0000","last_peered":"2024-09-10T21:47:39.051777+0000","last_clean":"2024-09-10T21:47:39.051777+0000","last_became_active":"2024-09-10T21:24:47.865093+0000","last_became_peered":"2024-09-10T21:24:47.865093+0000","last_unstale":"2024-09-10T21:47:39.051777+0000","last_undegraded":"2024-09-10T21:47:39.051777+0000","last_fullsized":"2024-09-10T21:47:39.051777+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'26","last_scrub_stamp":"2024-09-10T21:47:39.051693+0000","last_deep_scrub":"27'26","last_deep_scrub_stamp":"2024-09-10T21:47:39.051693+0000","last_clean_scrub_stamp":"2024-09-10T21:47:39.051693+0000","objects_scrubbed":0,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:54:36.357498+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"27'15","reported_seq":57,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:37.277986+0000","last_change":"2024-09-10T21:47:37.277986+0000","last_active":"2024-09-10T21:47:37.277986+0000","last_peered":"2024-09-10T21:47:37.277986+0000","last_clean":"2024-09-10T21:47:37.277986+0000","last_became_active":"2024-09-10T21:24:47.867782+0000","last_became_peered":"2024-09-10T21:24:47.867782+0000","last_unstale":"2024-09-10T21:47:37.277986+0000","last_undegraded":"2024-09-10T21:47:37.277986+0000","last_fullsized":"2024-09-10T21:47:37.277986+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'15","last_scrub_stamp":"2024-09-10T21:47:37.277930+0000","last_deep_scrub":"27'15","last_deep_scrub_stamp":"2024-09-10T21:47:37.277930+0000","last_clean_scrub_stamp":"2024-09-10T21:47:37.277930+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:24:45.621244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"27'13","reported_seq":55,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:40.088088+0000","last_change":"2024-09-10T21:47:40.088088+0000","last_active":"2024-09-10T21:47:40.088088+0000","last_peered":"2024-09-10T21:47:40.088088+0000","last_clean":"2024-09-10T21:47:40.088088+0000","last_became_active":"2024-09-10T21:24:47.861434+0000","last_became_peered":"2024-09-10T21:24:47.861434+0000","last_unstale":"2024-09-10T21:47:40.088088+0000","last_undegraded":"2024-09-10T21:47:40.088088+0000","last_fullsized":"2024-09-10T21:47:40.088088+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'13","last_scrub_stamp":"2024-09-10T21:47:40.088009+0000","last_deep_scrub":"27'13","last_deep_scrub_stamp":"2024-09-10T21:47:40.088009+0000","last_clean_scrub_stamp":"2024-09-10T21:47:40.088009+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:41:12.540656+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"27'14","reported_seq":55,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:38.629094+0000","last_change":"2024-09-10T21:47:38.629094+0000","last_active":"2024-09-10T21:47:38.629094+0000","last_peered":"2024-09-10T21:47:38.629094+0000","last_clean":"2024-09-10T21:47:38.629094+0000","last_became_active":"2024-09-10T21:24:47.868503+0000","last_became_peered":"2024-09-10T21:24:47.868503+0000","last_unstale":"2024-09-10T21:47:38.629094+0000","last_undegraded":"2024-09-10T21:47:38.629094+0000","last_fullsized":"2024-09-10T21:47:38.629094+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'14","last_scrub_stamp":"2024-09-10T21:47:38.629019+0000","last_deep_scrub":"27'14","last_deep_scrub_stamp":"2024-09-10T21:47:38.629019+0000","last_clean_scrub_stamp":"2024-09-10T21:47:38.629019+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:32:41.494438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"27'15","reported_seq":57,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:41.099538+0000","last_change":"2024-09-10T21:47:41.099538+0000","last_active":"2024-09-10T21:47:41.099538+0000","last_peered":"2024-09-10T21:47:41.099538+0000","last_clean":"2024-09-10T21:47:41.099538+0000","last_became_active":"2024-09-10T21:24:47.863976+0000","last_became_peered":"2024-09-10T21:24:47.863976+0000","last_unstale":"2024-09-10T21:47:41.099538+0000","last_undegraded":"2024-09-10T21:47:41.099538+0000","last_fullsized":"2024-09-10T21:47:41.099538+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'15","last_scrub_stamp":"2024-09-10T21:47:41.099448+0000","last_deep_scrub":"27'15","last_deep_scrub_stamp":"2024-09-10T21:47:41.099448+0000","last_clean_scrub_stamp":"2024-09-10T21:47:41.099448+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:27:00.251650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"27'13","reported_seq":56,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:39.646939+0000","last_change":"2024-09-10T21:47:39.646939+0000","last_active":"2024-09-10T21:47:39.646939+0000","last_peered":"2024-09-10T21:47:39.646939+0000","last_clean":"2024-09-10T21:47:39.646939+0000","last_became_active":"2024-09-10T21:24:47.873069+0000","last_became_peered":"2024-09-10T21:24:47.873069+0000","last_unstale":"2024-09-10T21:47:39.646939+0000","last_undegraded":"2024-09-10T21:47:39.646939+0000","last_fullsized":"2024-09-10T21:47:39.646939+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'13","last_scrub_stamp":"2024-09-10T21:47:39.646856+0000","last_deep_scrub":"27'13","last_deep_scrub_stamp":"2024-09-10T21:47:39.646856+0000","last_clean_scrub_stamp":"2024-09-10T21:47:39.646856+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:35:08.105014+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"27'20","reported_seq":64,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:33.193083+0000","last_change":"2024-09-10T21:47:33.193037+0000","last_active":"2024-09-10T21:47:33.193083+0000","last_peered":"2024-09-10T21:47:33.193083+0000","last_clean":"2024-09-10T21:47:33.193083+0000","last_became_active":"2024-09-10T21:24:47.854713+0000","last_became_peered":"2024-09-10T21:24:47.854713+0000","last_unstale":"2024-09-10T21:47:33.193083+0000","last_undegraded":"2024-09-10T21:47:33.193083+0000","last_fullsized":"2024-09-10T21:47:33.193083+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'20","last_scrub_stamp":"2024-09-10T21:47:33.192982+0000","last_deep_scrub":"27'20","last_deep_scrub_stamp":"2024-09-10T21:47:33.192982+0000","last_clean_scrub_stamp":"2024-09-10T21:47:33.192982+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:46:41.809785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"27'10","reported_seq":53,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:27.971578+0000","last_change":"2024-09-10T21:47:27.971496+0000","last_active":"2024-09-10T21:47:27.971578+0000","last_peered":"2024-09-10T21:47:27.971578+0000","last_clean":"2024-09-10T21:47:27.971578+0000","last_became_active":"2024-09-10T21:24:47.846227+0000","last_became_peered":"2024-09-10T21:24:47.846227+0000","last_unstale":"2024-09-10T21:47:27.971578+0000","last_undegraded":"2024-09-10T21:47:27.971578+0000","last_fullsized":"2024-09-10T21:47:27.971578+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'10","last_scrub_stamp":"2024-09-10T21:47:27.971425+0000","last_deep_scrub":"27'10","last_deep_scrub_stamp":"2024-09-10T21:47:27.971425+0000","last_clean_scrub_stamp":"2024-09-10T21:47:27.971425+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:06:14.907911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"27'15","reported_seq":59,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-09-10T21:47:42.103166+0000","last_change":"2024-09-10T21:47:42.103093+0000","last_active":"2024-09-10T21:47:42.103166+0000","last_peered":"2024-09-10T21:47:42.103166+0000","last_clean":"2024-09-10T21:47:42.103166+0000","last_became_active":"2024-09-10T21:24:47.871177+0000","last_became_peered":"2024-09-10T21:24:47.871177+0000","last_unstale":"2024-09-10T21:47:42.103166+0000","last_undegraded":"2024-09-10T21:47:42.103166+0000","last_fullsized":"2024-09-10T21:47:42.103166+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'15","last_scrub_stamp":"2024-09-10T21:47:42.103014+0000","last_deep_scrub":"27'15","last_deep_scrub_stamp":"2024-09-10T21:47:42.103014+0000","last_clean_scrub_stamp":"2024-09-10T21:47:42.103014+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:53:50.812078+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":723657,"num_write_kb":15994026,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":148509,"ondisk_log_size":148509,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1239,"ondisk_log_size":1239,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":14991163,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":48,"num_read_kb":119,"num_write":3049,"num_write_kb":15770,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":18464768,"data_stored":29982326,"data_compressed":2564296,"data_compressed_allocated":3506176,"data_compressed_original":15089664,"omap_allocated":0,"internal_metadata":0},"log_size":2782,"ondisk_log_size":2782,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":1180736,"data_compressed":66932,"data_compressed_allocated":90112,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":74,"ondisk_log_size":74,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509723,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":739600,"kb_used_data":7424,"kb_used_omap":1,"kb_used_meta":732158,"kb_avail":93632240,"statfs":{"total":96636764160,"available":95879413760,"internally_reserved":0,"allocated":7602176,"data_stored":11412495,"data_compressed":806609,"data_compressed_allocated":1097728,"data_compressed_original":5083136,"omap_allocated":1590,"internal_metadata":749730250},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509724,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":807000,"kb_used_data":2120,"kb_used_omap":1,"kb_used_meta":804862,"kb_avail":93564840,"statfs":{"total":96636764160,"available":95810396160,"internally_reserved":0,"allocated":2170880,"data_stored":4311586,"data_compressed":503209,"data_compressed_allocated":692224,"data_compressed_original":3006464,"omap_allocated":1594,"internal_metadata":824179142},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509723,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":573484,"kb_used_data":348,"kb_used_omap":1,"kb_used_meta":573118,"kb_avail":93798356,"statfs":{"total":96636764160,"available":96049516544,"internally_reserved":0,"allocated":356352,"data_stored":207064,"data_compressed":5796,"data_compressed_allocated":8192,"data_compressed_original":28672,"omap_allocated":1595,"internal_metadata":586873285},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509721,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1104044,"kb_used_data":284,"kb_used_omap":1,"kb_used_meta":1103742,"kb_avail":93267796,"statfs":{"total":96636764160,"available":95506223104,"internally_reserved":0,"allocated":290816,"data_stored":116770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":1130232261},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":14,"seq":60129542428,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":844396,"kb_used_data":3740,"kb_used_omap":1,"kb_used_meta":840638,"kb_avail":93527444,"statfs":{"total":96636764160,"available":95772102656,"internally_reserved":0,"allocated":3829760,"data_stored":4311040,"data_compressed":154551,"data_compressed_allocated":245760,"data_compressed_original":905216,"omap_allocated":1595,"internal_metadata":860813765},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":14,"seq":60129542423,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":1080968,"kb_used_data":4792,"kb_used_omap":1,"kb_used_meta":1076158,"kb_avail":93290872,"statfs":{"total":96636764160,"available":95529852928,"internally_reserved":0,"allocated":4907008,"data_stored":8596344,"data_compressed":841246,"data_compressed_allocated":1130496,"data_compressed_original":4997120,"omap_allocated":1593,"internal_metadata":1101986247},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542429,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":711220,"kb_used_data":1380,"kb_used_omap":1,"kb_used_meta":709822,"kb_avail":93660620,"statfs":{"total":96636764160,"available":95908474880,"internally_reserved":0,"allocated":1413120,"data_stored":3023943,"data_compressed":319817,"data_compressed_allocated":421888,"data_compressed_original":2215936,"omap_allocated":1595,"internal_metadata":726858181},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575132,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":626792,"kb_used_data":280,"kb_used_omap":1,"kb_used_meta":626494,"kb_avail":93745048,"statfs":{"total":96636764160,"available":95994929152,"internally_reserved":0,"allocated":286720,"data_stored":116700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":641530309},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33466,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33466,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":1064960,"data_stored":2316965,"data_compressed":286351,"data_compressed_allocated":376832,"data_compressed_original":1642496,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":90454,"data_compressed":5796,"data_compressed_allocated":8192,"data_compressed_original":28672,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4624384,"data_stored":8479734,"data_compressed":841246,"data_compressed_allocated":1130496,"data_compressed_original":4997120,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":3547136,"data_stored":4194430,"data_compressed":154551,"data_compressed_allocated":245760,"data_compressed_original":905216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":1888256,"data_stored":4194976,"data_compressed":503209,"data_compressed_allocated":692224,"data_compressed_original":3006464,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":7254016,"data_stored":10705517,"data_compressed":773143,"data_compressed_allocated":1052672,"data_compressed_original":4509696,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T21:48:06.428 DEBUG:teuthology.orchestra.run.smithi017:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-09-10T21:48:06.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:06.604+0000 7f2b43a00640 1 -- 172.21.15.17:0/710306224 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2b24004d50 msgr2=0x7f2b240051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:48:06.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:06.604+0000 7f2b43a00640 1 --2- 172.21.15.17:0/710306224 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2b24004d50 0x7f2b240051d0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f2b34009eb0 tx=0x7f2b3401d5b0 comp rx=0 tx=0).stop 2024-09-10T21:48:06.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:06.604+0000 7f2b41a00640 1 -- 172.21.15.17:0/710306224 reap_dead start 2024-09-10T21:48:06.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:06.604+0000 7f2b43a00640 1 -- 172.21.15.17:0/710306224 shutdown_connections 2024-09-10T21:48:06.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:06.604+0000 7f2b43a00640 1 -- 172.21.15.17:0/710306224 >> 172.21.15.17:0/710306224 conn(0x7f2b3c06d220 msgr2=0x7f2b3c068990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:48:06.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:06.604+0000 7f2b43a00640 1 -- 172.21.15.17:0/710306224 shutdown_connections 2024-09-10T21:48:06.611 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:06.604+0000 7f2b43a00640 1 -- 172.21.15.17:0/710306224 wait complete. 2024-09-10T21:48:06.613 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:06.608+0000 7f2b43a00640 1 Processor -- start 2024-09-10T21:48:06.613 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:06.608+0000 7f2b43a00640 1 -- start start 2024-09-10T21:48:06.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:06.608+0000 7f2b43a00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f2b24004d50 0x7f2b3c0ceba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:48:06.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:06.608+0000 7f2b43a00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2b3c0c7340 0x7f2b3c157af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:48:06.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:06.608+0000 7f2b43a00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2b3c1f55f0 0x7f2b3c0c5680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:48:06.615 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:06.608+0000 7f2b43a00640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f2b3c0c9d40 con 0x7f2b3c0c7340 2024-09-10T21:48:06.616 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:06.608+0000 7f2b43a00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f2b3c0c3060 con 0x7f2b3c1f55f0 2024-09-10T21:48:06.616 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:06.608+0000 7f2b43a00640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f2b3c070400 con 0x7f2b24004d50 2024-09-10T21:48:06.616 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:06.608+0000 7f2b3be00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2b3c0c7340 0x7f2b3c157af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:48:06.616 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:06.608+0000 7f2b3be00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2b3c0c7340 0x7f2b3c157af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:48574/0 (socket says 172.21.15.17:48574) 2024-09-10T21:48:06.616 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:06.608+0000 7f2b3be00640 1 -- 172.21.15.17:0/2047454268 learned_addr learned my addr 172.21.15.17:0/2047454268 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:48:06.616 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:06.608+0000 7f2b41000640 1 --2- 172.21.15.17:0/2047454268 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f2b24004d50 0x7f2b3c0ceba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:48:06.616 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:06.608+0000 7f2b41a00640 1 --2- 172.21.15.17:0/2047454268 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2b3c1f55f0 0x7f2b3c0c5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:48:06.616 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:06.608+0000 7f2b3be00640 1 -- 172.21.15.17:0/2047454268 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f2b24004d50 msgr2=0x7f2b3c0ceba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:48:06.616 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:06.608+0000 7f2b3be00640 1 --2- 172.21.15.17:0/2047454268 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f2b24004d50 0x7f2b3c0ceba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:06.616 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:06.608+0000 7f2b3be00640 1 -- 172.21.15.17:0/2047454268 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2b3c1f55f0 msgr2=0x7f2b3c0c5680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:48:06.616 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:06.608+0000 7f2b3be00640 1 --2- 172.21.15.17:0/2047454268 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2b3c1f55f0 0x7f2b3c0c5680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:06.616 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:06.608+0000 7f2b3be00640 1 -- 172.21.15.17:0/2047454268 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b3c14c000 con 0x7f2b3c0c7340 2024-09-10T21:48:06.616 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:06.612+0000 7f2b3be00640 1 --2- 172.21.15.17:0/2047454268 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2b3c0c7340 0x7f2b3c157af0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f2b28003ee0 tx=0x7f2b28001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:48:06.617 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:06.612+0000 7f2b38c00640 1 -- 172.21.15.17:0/2047454268 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b28015070 con 0x7f2b3c0c7340 2024-09-10T21:48:06.617 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:06.612+0000 7f2b38c00640 1 -- 172.21.15.17:0/2047454268 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2b28011840 con 0x7f2b3c0c7340 2024-09-10T21:48:06.617 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:06.612+0000 7f2b43a00640 1 -- 172.21.15.17:0/2047454268 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b3c14c2e0 con 0x7f2b3c0c7340 2024-09-10T21:48:06.617 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:06.612+0000 7f2b38c00640 1 -- 172.21.15.17:0/2047454268 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b28011c50 con 0x7f2b3c0c7340 2024-09-10T21:48:06.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:06.612+0000 7f2b43a00640 1 -- 172.21.15.17:0/2047454268 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b3c0cd710 con 0x7f2b3c0c7340 2024-09-10T21:48:06.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:06.612+0000 7f2b43a00640 1 -- 172.21.15.17:0/2047454268 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b3c0c35d0 con 0x7f2b3c0c7340 2024-09-10T21:48:06.618 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:06.612+0000 7f2b38c00640 1 -- 172.21.15.17:0/2047454268 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f2b280103c0 con 0x7f2b3c0c7340 2024-09-10T21:48:06.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:06.616+0000 7f2b38c00640 1 --2- 172.21.15.17:0/2047454268 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f2b1c0725e0 0x7f2b1c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:48:06.619 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:06.616+0000 7f2b38c00640 1 -- 172.21.15.17:0/2047454268 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f2b280944f0 con 0x7f2b3c0c7340 2024-09-10T21:48:06.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:06.616+0000 7f2b41000640 1 --2- 172.21.15.17:0/2047454268 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f2b1c0725e0 0x7f2b1c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:48:06.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:06.616+0000 7f2b41000640 1 --2- 172.21.15.17:0/2047454268 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f2b1c0725e0 0x7f2b1c074aa0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f2b30006be0 tx=0x7f2b300067b0 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:48:06.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:06.616+0000 7f2b38c00640 1 -- 172.21.15.17:0/2047454268 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f2b2805f050 con 0x7f2b3c0c7340 2024-09-10T21:48:06.797 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:06.792+0000 7f2b43a00640 1 -- 172.21.15.17:0/2047454268 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) -- 0x7f2b3c0c39c0 con 0x7f2b3c0c7340 2024-09-10T21:48:06.800 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:06.796+0000 7f2b38c00640 1 -- 172.21.15.17:0/2047454268 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v33)=0 v33) ==== 123+0+0 (secure 0 0 0) 0x7f2b28063a80 con 0x7f2b3c0c7340 2024-09-10T21:48:06.801 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:06.796+0000 7f2b38c00640 1 -- 172.21.15.17:0/2047454268 <== mon.0 v2:172.21.15.17:3300/0 8 ==== config(6 keys) ==== 172+0+0 (secure 0 0 0) 0x7f2b2805eb60 con 0x7f2b3c0c7340 2024-09-10T21:48:06.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:06.800+0000 7f2b43a00640 1 -- 172.21.15.17:0/2047454268 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f2b1c0725e0 msgr2=0x7f2b1c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:48:06.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:06.800+0000 7f2b43a00640 1 --2- 172.21.15.17:0/2047454268 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f2b1c0725e0 0x7f2b1c074aa0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f2b30006be0 tx=0x7f2b300067b0 comp rx=0 tx=0).stop 2024-09-10T21:48:06.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:06.800+0000 7f2b43a00640 1 -- 172.21.15.17:0/2047454268 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2b3c0c7340 msgr2=0x7f2b3c157af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:48:06.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:06.800+0000 7f2b43a00640 1 --2- 172.21.15.17:0/2047454268 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2b3c0c7340 0x7f2b3c157af0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f2b28003ee0 tx=0x7f2b28001640 comp rx=0 tx=0).stop 2024-09-10T21:48:06.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:06.800+0000 7f2b43a00640 1 -- 172.21.15.17:0/2047454268 shutdown_connections 2024-09-10T21:48:06.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:06.800+0000 7f2b43a00640 1 --2- 172.21.15.17:0/2047454268 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f2b1c0725e0 0x7f2b1c074aa0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:06.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:06.800+0000 7f2b43a00640 1 --2- 172.21.15.17:0/2047454268 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f2b3c1f55f0 0x7f2b3c0c5680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:06.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:06.800+0000 7f2b43a00640 1 --2- 172.21.15.17:0/2047454268 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f2b3c0c7340 0x7f2b3c157af0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:06.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:06.800+0000 7f2b43a00640 1 --2- 172.21.15.17:0/2047454268 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f2b24004d50 0x7f2b3c0ceba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:06.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:06.800+0000 7f2b43a00640 1 -- 172.21.15.17:0/2047454268 >> 172.21.15.17:0/2047454268 conn(0x7f2b3c06d220 msgr2=0x7f2b3c1501e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:48:06.804 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:06.800+0000 7f2b43a00640 1 -- 172.21.15.17:0/2047454268 shutdown_connections 2024-09-10T21:48:06.805 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:06.800+0000 7f2b43a00640 1 -- 172.21.15.17:0/2047454268 wait complete. 2024-09-10T21:48:06.825 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-10T21:48:06.826 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:48:06.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:06.968+0000 7f623e200640 1 -- 172.21.15.17:0/1584299230 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f62380c7210 msgr2=0x7f6238157470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:48:06.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:06.968+0000 7f623e200640 1 --2- 172.21.15.17:0/1584299230 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f62380c7210 0x7f6238157470 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f6228009840 tx=0x7f622801f010 comp rx=0 tx=0).stop 2024-09-10T21:48:06.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:06.968+0000 7f623e200640 1 -- 172.21.15.17:0/1584299230 shutdown_connections 2024-09-10T21:48:06.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:06.968+0000 7f623e200640 1 --2- 172.21.15.17:0/1584299230 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f62380c7210 0x7f6238157470 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:06.975 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:06.968+0000 7f623e200640 1 -- 172.21.15.17:0/1584299230 >> 172.21.15.17:0/1584299230 conn(0x7f623806dd30 msgr2=0x7f623806e140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:48:06.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:06.968+0000 7f623e200640 1 -- 172.21.15.17:0/1584299230 shutdown_connections 2024-09-10T21:48:06.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:06.968+0000 7f623e200640 1 -- 172.21.15.17:0/1584299230 wait complete. 2024-09-10T21:48:06.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:06.972+0000 7f623e200640 1 Processor -- start 2024-09-10T21:48:06.976 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:06.972+0000 7f623e200640 1 -- start start 2024-09-10T21:48:06.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:06.972+0000 7f623e200640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6238070450 0x7f62380c3910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:48:06.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:06.972+0000 7f623e200640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f62380c5ec0 0x7f62380cc430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:48:06.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:06.972+0000 7f623e200640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f62380c7210 0x7f62381f4be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:48:06.977 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:06.972+0000 7f623e200640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f623815b690 con 0x7f6238070450 2024-09-10T21:48:06.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:06.972+0000 7f623e200640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f623815b3b0 con 0x7f62380c5ec0 2024-09-10T21:48:06.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:06.972+0000 7f623e200640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f6238069b90 con 0x7f62380c7210 2024-09-10T21:48:06.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:06.972+0000 7f6237400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6238070450 0x7f62380c3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:48:06.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:06.972+0000 7f6237400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6238070450 0x7f62380c3910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:48590/0 (socket says 172.21.15.17:48590) 2024-09-10T21:48:06.978 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:06.972+0000 7f6237400640 1 -- 172.21.15.17:0/1344931243 learned_addr learned my addr 172.21.15.17:0/1344931243 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:48:06.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:06.972+0000 7f622ea00640 1 --2- 172.21.15.17:0/1344931243 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f62380c5ec0 0x7f62380cc430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:48:06.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:06.972+0000 7f6237400640 1 -- 172.21.15.17:0/1344931243 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f62380c7210 msgr2=0x7f62381f4be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:48:06.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:06.972+0000 7f6237400640 1 --2- 172.21.15.17:0/1344931243 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f62380c7210 0x7f62381f4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:06.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:06.972+0000 7f6237400640 1 -- 172.21.15.17:0/1344931243 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f62380c5ec0 msgr2=0x7f62380cc430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:48:06.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:06.972+0000 7f6237400640 1 --2- 172.21.15.17:0/1344931243 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f62380c5ec0 0x7f62380cc430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:06.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:06.972+0000 7f6237400640 1 -- 172.21.15.17:0/1344931243 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6238068e40 con 0x7f6238070450 2024-09-10T21:48:06.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:06.972+0000 7f622ea00640 1 --2- 172.21.15.17:0/1344931243 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f62380c5ec0 0x7f62380cc430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:48:06.979 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:06.972+0000 7f6237400640 1 --2- 172.21.15.17:0/1344931243 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6238070450 0x7f62380c3910 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f6228009790 tx=0x7f6228005db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:48:06.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:06.972+0000 7f622fe00640 1 -- 172.21.15.17:0/1344931243 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62280be070 con 0x7f6238070450 2024-09-10T21:48:06.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:06.972+0000 7f622fe00640 1 -- 172.21.15.17:0/1344931243 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(6 keys) ==== 172+0+0 (secure 0 0 0) 0x7f6228005400 con 0x7f6238070450 2024-09-10T21:48:06.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:06.972+0000 7f622fe00640 1 -- 172.21.15.17:0/1344931243 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62280b5040 con 0x7f6238070450 2024-09-10T21:48:06.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:06.972+0000 7f623e200640 1 -- 172.21.15.17:0/1344931243 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62380c8a70 con 0x7f6238070450 2024-09-10T21:48:06.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:06.972+0000 7f623e200640 1 -- 172.21.15.17:0/1344931243 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6238156a60 con 0x7f6238070450 2024-09-10T21:48:06.980 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:06.976+0000 7f623e200640 1 -- 172.21.15.17:0/1344931243 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6238157470 con 0x7f6238070450 2024-09-10T21:48:06.981 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:06.976+0000 7f622fe00640 1 -- 172.21.15.17:0/1344931243 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f62280b8030 con 0x7f6238070450 2024-09-10T21:48:06.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:06.976+0000 7f622fe00640 1 --2- 172.21.15.17:0/1344931243 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f62140725e0 0x7f6214074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:48:06.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:06.976+0000 7f622fe00640 1 -- 172.21.15.17:0/1344931243 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f62281302f0 con 0x7f6238070450 2024-09-10T21:48:06.983 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:06.980+0000 7f622ea00640 1 --2- 172.21.15.17:0/1344931243 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f62140725e0 0x7f6214074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:48:06.984 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:06.980+0000 7f622ea00640 1 --2- 172.21.15.17:0/1344931243 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f62140725e0 0x7f6214074aa0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f6220001230 tx=0x7f6220009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:48:06.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:06.984+0000 7f622fe00640 1 -- 172.21.15.17:0/1344931243 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f6228018b70 con 0x7f6238070450 2024-09-10T21:48:07.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:07.192+0000 7f623e200640 1 -- 172.21.15.17:0/1344931243 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f62380746d0 con 0x7f6238070450 2024-09-10T21:48:07.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:07.192+0000 7f622fe00640 1 -- 172.21.15.17:0/1344931243 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+6870 (secure 0 0 0) 0x7f62280fb530 con 0x7f6238070450 2024-09-10T21:48:07.197 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-10T21:48:07.197 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":10,"btime":"2024-09-10T21:25:16:432823+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4350,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/1643071167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":1643071167},{"type":"v1","addr":"172.21.15.31:6835","nonce":1643071167}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4356,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6837/1666225712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6836","nonce":1666225712},{"type":"v1","addr":"172.21.15.17:6837","nonce":1666225712}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4384,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/3592734043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":3592734043},{"type":"v1","addr":"172.21.15.17:6833","nonce":3592734043}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4405,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/1293665856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":1293665856},{"type":"v1","addr":"172.21.15.17:6835","nonce":1293665856}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4411,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6838/1944852172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":1944852172},{"type":"v1","addr":"172.21.15.31:6838","nonce":1944852172}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":10,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:24:47.255397+0000","modified":"2024-09-10T21:25:15.961582+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4414},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4414":{"gid":4414,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.31:6839/3129062106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6837","nonce":3129062106},{"type":"v1","addr":"172.21.15.31:6839","nonce":3129062106}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4414,"qdb_cluster":[4414]},"id":1}]} 2024-09-10T21:48:07.198 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 10 2024-09-10T21:48:07.200 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:07.192+0000 7f623e200640 1 -- 172.21.15.17:0/1344931243 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f62140725e0 msgr2=0x7f6214074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:48:07.200 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:07.192+0000 7f623e200640 1 --2- 172.21.15.17:0/1344931243 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f62140725e0 0x7f6214074aa0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f6220001230 tx=0x7f6220009040 comp rx=0 tx=0).stop 2024-09-10T21:48:07.200 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:07.192+0000 7f623e200640 1 -- 172.21.15.17:0/1344931243 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6238070450 msgr2=0x7f62380c3910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:48:07.200 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:07.192+0000 7f623e200640 1 --2- 172.21.15.17:0/1344931243 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6238070450 0x7f62380c3910 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f6228009790 tx=0x7f6228005db0 comp rx=0 tx=0).stop 2024-09-10T21:48:07.200 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:07.196+0000 7f623e200640 1 -- 172.21.15.17:0/1344931243 shutdown_connections 2024-09-10T21:48:07.200 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:07.196+0000 7f623e200640 1 --2- 172.21.15.17:0/1344931243 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f62140725e0 0x7f6214074aa0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:07.200 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:07.196+0000 7f623e200640 1 --2- 172.21.15.17:0/1344931243 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f62380c7210 0x7f62381f4be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:07.200 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:07.196+0000 7f623e200640 1 --2- 172.21.15.17:0/1344931243 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f62380c5ec0 0x7f62380cc430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:07.200 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:07.196+0000 7f623e200640 1 --2- 172.21.15.17:0/1344931243 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f6238070450 0x7f62380c3910 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:07.200 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:07.196+0000 7f623e200640 1 -- 172.21.15.17:0/1344931243 >> 172.21.15.17:0/1344931243 conn(0x7f623806dd30 msgr2=0x7f62380c7720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:48:07.200 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:07.196+0000 7f623e200640 1 -- 172.21.15.17:0/1344931243 shutdown_connections 2024-09-10T21:48:07.200 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:07.196+0000 7f623e200640 1 -- 172.21.15.17:0/1344931243 wait complete. 2024-09-10T21:48:07.220 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-10T21:48:07.369 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:07.364+0000 7fdc67a00640 1 -- 172.21.15.17:0/1937885754 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdc600cdf20 msgr2=0x7fdc600ce3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:48:07.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:07.364+0000 7fdc67a00640 1 --2- 172.21.15.17:0/1937885754 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdc600cdf20 0x7fdc600ce3b0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fdc60073220 tx=0x7fdc5801ccf0 comp rx=0 tx=0).stop 2024-09-10T21:48:07.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:07.364+0000 7fdc67a00640 1 -- 172.21.15.17:0/1937885754 shutdown_connections 2024-09-10T21:48:07.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:07.364+0000 7fdc67a00640 1 --2- 172.21.15.17:0/1937885754 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdc600cdf20 0x7fdc600ce3b0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:07.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:07.364+0000 7fdc67a00640 1 --2- 172.21.15.17:0/1937885754 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fdc60152820 0x7fdc6015c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:07.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:07.364+0000 7fdc67a00640 1 --2- 172.21.15.17:0/1937885754 >> v2:172.21.15.31:3300/0 conn(0x7fdc60152440 0x7fdc600b3eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:07.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:07.364+0000 7fdc67a00640 1 -- 172.21.15.17:0/1937885754 >> 172.21.15.17:0/1937885754 conn(0x7fdc600702f0 msgr2=0x7fdc60070700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:48:07.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:07.364+0000 7fdc67a00640 1 -- 172.21.15.17:0/1937885754 shutdown_connections 2024-09-10T21:48:07.370 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:07.364+0000 7fdc67a00640 1 -- 172.21.15.17:0/1937885754 wait complete. 2024-09-10T21:48:07.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:07.364+0000 7fdc67a00640 1 Processor -- start 2024-09-10T21:48:07.371 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:07.368+0000 7fdc67a00640 1 -- start start 2024-09-10T21:48:07.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:07.368+0000 7fdc67a00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdc600cdf20 0x7fdc600c7150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:48:07.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:07.368+0000 7fdc67a00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdc60152440 0x7fdc600c3410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:48:07.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:07.368+0000 7fdc67a00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fdc60152820 0x7fdc60160de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:48:07.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:07.368+0000 7fdc67a00640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fdc600cb450 con 0x7fdc600cdf20 2024-09-10T21:48:07.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:07.368+0000 7fdc67a00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fdc6015c7c0 con 0x7fdc60152440 2024-09-10T21:48:07.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:07.368+0000 7fdc67a00640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7fdc600cb5d0 con 0x7fdc60152820 2024-09-10T21:48:07.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:07.368+0000 7fdc65000640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdc600cdf20 0x7fdc600c7150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:48:07.374 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:07.368+0000 7fdc65000640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdc600cdf20 0x7fdc600c7150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:48594/0 (socket says 172.21.15.17:48594) 2024-09-10T21:48:07.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:07.368+0000 7fdc65000640 1 -- 172.21.15.17:0/3972002724 learned_addr learned my addr 172.21.15.17:0/3972002724 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:48:07.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:07.368+0000 7fdc5fe00640 1 --2- 172.21.15.17:0/3972002724 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdc60152440 0x7fdc600c3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:48:07.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:07.368+0000 7fdc65a00640 1 --2- 172.21.15.17:0/3972002724 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fdc60152820 0x7fdc60160de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:48:07.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:07.368+0000 7fdc65000640 1 -- 172.21.15.17:0/3972002724 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fdc60152820 msgr2=0x7fdc60160de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:48:07.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:07.368+0000 7fdc65000640 1 --2- 172.21.15.17:0/3972002724 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fdc60152820 0x7fdc60160de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:07.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:07.368+0000 7fdc65000640 1 -- 172.21.15.17:0/3972002724 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdc60152440 msgr2=0x7fdc600c3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:48:07.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:07.368+0000 7fdc65000640 1 --2- 172.21.15.17:0/3972002724 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdc60152440 0x7fdc600c3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:07.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:07.368+0000 7fdc65000640 1 -- 172.21.15.17:0/3972002724 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc601f7690 con 0x7fdc600cdf20 2024-09-10T21:48:07.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:07.368+0000 7fdc65a00640 1 --2- 172.21.15.17:0/3972002724 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fdc60152820 0x7fdc60160de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:48:07.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:07.368+0000 7fdc65000640 1 --2- 172.21.15.17:0/3972002724 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdc600cdf20 0x7fdc600c7150 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fdc5400e9f0 tx=0x7fdc5400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:48:07.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:07.368+0000 7fdc5cc00640 1 -- 172.21.15.17:0/3972002724 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc5400ccc0 con 0x7fdc600cdf20 2024-09-10T21:48:07.375 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:07.368+0000 7fdc5cc00640 1 -- 172.21.15.17:0/3972002724 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(6 keys) ==== 172+0+0 (secure 0 0 0) 0x7fdc54010430 con 0x7fdc600cdf20 2024-09-10T21:48:07.376 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:07.368+0000 7fdc67a00640 1 -- 172.21.15.17:0/3972002724 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc6006e8e0 con 0x7fdc600cdf20 2024-09-10T21:48:07.376 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:07.368+0000 7fdc5cc00640 1 -- 172.21.15.17:0/3972002724 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc54010860 con 0x7fdc600cdf20 2024-09-10T21:48:07.376 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:07.368+0000 7fdc67a00640 1 -- 172.21.15.17:0/3972002724 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc601f6bb0 con 0x7fdc600cdf20 2024-09-10T21:48:07.376 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:07.372+0000 7fdc67a00640 1 -- 172.21.15.17:0/3972002724 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc60158360 con 0x7fdc600cdf20 2024-09-10T21:48:07.376 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:07.372+0000 7fdc5cc00640 1 -- 172.21.15.17:0/3972002724 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fdc54010a00 con 0x7fdc600cdf20 2024-09-10T21:48:07.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:07.372+0000 7fdc5cc00640 1 --2- 172.21.15.17:0/3972002724 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fdc40072610 0x7fdc40074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:48:07.377 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:07.372+0000 7fdc5cc00640 1 -- 172.21.15.17:0/3972002724 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fdc54094170 con 0x7fdc600cdf20 2024-09-10T21:48:07.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:07.372+0000 7fdc5fe00640 1 --2- 172.21.15.17:0/3972002724 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fdc40072610 0x7fdc40074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:48:07.378 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:07.372+0000 7fdc5fe00640 1 --2- 172.21.15.17:0/3972002724 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fdc40072610 0x7fdc40074ad0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fdc60156f50 tx=0x7fdc4c00b390 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:48:07.381 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:07.376+0000 7fdc5cc00640 1 -- 172.21.15.17:0/3972002724 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fdc5405ed60 con 0x7fdc600cdf20 2024-09-10T21:48:07.589 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:07.584+0000 7fdc67a00640 1 -- 172.21.15.17:0/3972002724 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdc600c8b40 con 0x7fdc600cdf20 2024-09-10T21:48:07.590 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:07.584+0000 7fdc5cc00640 1 -- 172.21.15.17:0/3972002724 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+6870 (secure 0 0 0) 0x7fdc54063790 con 0x7fdc600cdf20 2024-09-10T21:48:07.590 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-10T21:48:07.590 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":10,"btime":"2024-09-10T21:25:16:432823+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4350,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6835/1643071167","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6834","nonce":1643071167},{"type":"v1","addr":"172.21.15.31:6835","nonce":1643071167}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4356,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6837/1666225712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6836","nonce":1666225712},{"type":"v1","addr":"172.21.15.17:6837","nonce":1666225712}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4384,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6833/3592734043","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6832","nonce":3592734043},{"type":"v1","addr":"172.21.15.17:6833","nonce":3592734043}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4405,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.17:6835/1293665856","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6834","nonce":1293665856},{"type":"v1","addr":"172.21.15.17:6835","nonce":1293665856}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4411,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.31:6838/1944852172","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6836","nonce":1944852172},{"type":"v1","addr":"172.21.15.31:6838","nonce":1944852172}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":10,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-10T21:24:47.255397+0000","modified":"2024-09-10T21:25:15.961582+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4414},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4414":{"gid":4414,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":5,"addr":"172.21.15.31:6839/3129062106","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6837","nonce":3129062106},{"type":"v1","addr":"172.21.15.31:6839","nonce":3129062106}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4414,"qdb_cluster":[4414]},"id":1}]} 2024-09-10T21:48:07.590 INFO:teuthology.orchestra.run.smithi017.stderr:dumped fsmap epoch 10 2024-09-10T21:48:07.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:07.588+0000 7fdc67a00640 1 -- 172.21.15.17:0/3972002724 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fdc40072610 msgr2=0x7fdc40074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:48:07.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:07.588+0000 7fdc67a00640 1 --2- 172.21.15.17:0/3972002724 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fdc40072610 0x7fdc40074ad0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fdc60156f50 tx=0x7fdc4c00b390 comp rx=0 tx=0).stop 2024-09-10T21:48:07.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:07.588+0000 7fdc67a00640 1 -- 172.21.15.17:0/3972002724 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdc600cdf20 msgr2=0x7fdc600c7150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:48:07.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:07.588+0000 7fdc67a00640 1 --2- 172.21.15.17:0/3972002724 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdc600cdf20 0x7fdc600c7150 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fdc5400e9f0 tx=0x7fdc5400eeb0 comp rx=0 tx=0).stop 2024-09-10T21:48:07.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:07.588+0000 7fdc67a00640 1 -- 172.21.15.17:0/3972002724 shutdown_connections 2024-09-10T21:48:07.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:07.588+0000 7fdc67a00640 1 --2- 172.21.15.17:0/3972002724 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fdc40072610 0x7fdc40074ad0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:07.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:07.588+0000 7fdc67a00640 1 --2- 172.21.15.17:0/3972002724 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fdc60152820 0x7fdc60160de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:07.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:07.588+0000 7fdc67a00640 1 --2- 172.21.15.17:0/3972002724 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fdc60152440 0x7fdc600c3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:07.592 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:07.588+0000 7fdc67a00640 1 --2- 172.21.15.17:0/3972002724 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fdc600cdf20 0x7fdc600c7150 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:07.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:07.588+0000 7fdc67a00640 1 -- 172.21.15.17:0/3972002724 >> 172.21.15.17:0/3972002724 conn(0x7fdc600702f0 msgr2=0x7fdc600706d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:48:07.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:07.588+0000 7fdc67a00640 1 -- 172.21.15.17:0/3972002724 shutdown_connections 2024-09-10T21:48:07.593 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:07.588+0000 7fdc67a00640 1 -- 172.21.15.17:0/3972002724 wait complete. 2024-09-10T21:48:07.612 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-10T21:48:07.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:07.752+0000 7fb965200640 1 -- 172.21.15.17:0/376320281 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb9600c6790 msgr2=0x7fb9600c44b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:48:07.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:07.752+0000 7fb965200640 1 --2- 172.21.15.17:0/376320281 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb9600c6790 0x7fb9600c44b0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fb958009c80 tx=0x7fb95801cda0 comp rx=0 tx=0).stop 2024-09-10T21:48:07.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:07.752+0000 7fb965200640 1 -- 172.21.15.17:0/376320281 shutdown_connections 2024-09-10T21:48:07.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:07.752+0000 7fb965200640 1 --2- 172.21.15.17:0/376320281 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb9600c6790 0x7fb9600c44b0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:07.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:07.752+0000 7fb965200640 1 --2- 172.21.15.17:0/376320281 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fb960152590 0x7fb960158550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:07.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:07.752+0000 7fb965200640 1 -- 172.21.15.17:0/376320281 >> 172.21.15.17:0/376320281 conn(0x7fb96006ce20 msgr2=0x7fb96006d230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:48:07.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:07.752+0000 7fb965200640 1 -- 172.21.15.17:0/376320281 shutdown_connections 2024-09-10T21:48:07.760 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:07.756+0000 7fb965200640 1 -- 172.21.15.17:0/376320281 wait complete. 2024-09-10T21:48:07.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:07.756+0000 7fb965200640 1 Processor -- start 2024-09-10T21:48:07.761 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:07.756+0000 7fb965200640 1 -- start start 2024-09-10T21:48:07.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:07.756+0000 7fb965200640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb9600c6390 0x7fb9600cd160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:48:07.762 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:07.756+0000 7fb965200640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb9600c6790 0x7fb960157b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:48:07.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:07.756+0000 7fb965200640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fb960152590 0x7fb9601f5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:48:07.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:07.756+0000 7fb965200640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7fb9600710a0 con 0x7fb9600c6390 2024-09-10T21:48:07.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:07.756+0000 7fb965200640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7fb960152970 con 0x7fb9600c6790 2024-09-10T21:48:07.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:07.756+0000 7fb965200640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7fb960071220 con 0x7fb960152590 2024-09-10T21:48:07.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:07.756+0000 7fb95de00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb9600c6390 0x7fb9600cd160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:48:07.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:07.756+0000 7fb95de00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb9600c6390 0x7fb9600cd160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:48610/0 (socket says 172.21.15.17:48610) 2024-09-10T21:48:07.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:07.756+0000 7fb95de00640 1 -- 172.21.15.17:0/1947154517 learned_addr learned my addr 172.21.15.17:0/1947154517 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:48:07.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:07.756+0000 7fb95e800640 1 --2- 172.21.15.17:0/1947154517 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fb960152590 0x7fb9601f5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:48:07.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:07.756+0000 7fb957e00640 1 --2- 172.21.15.17:0/1947154517 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb9600c6790 0x7fb960157b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:48:07.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:07.756+0000 7fb95de00640 1 -- 172.21.15.17:0/1947154517 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fb960152590 msgr2=0x7fb9601f5780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:48:07.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:07.756+0000 7fb95de00640 1 --2- 172.21.15.17:0/1947154517 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fb960152590 0x7fb9601f5780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:07.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:07.756+0000 7fb95de00640 1 -- 172.21.15.17:0/1947154517 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb9600c6790 msgr2=0x7fb960157b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:48:07.763 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:07.756+0000 7fb95de00640 1 --2- 172.21.15.17:0/1947154517 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb9600c6790 0x7fb960157b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:07.764 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:07.756+0000 7fb95de00640 1 -- 172.21.15.17:0/1947154517 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9601584e0 con 0x7fb9600c6390 2024-09-10T21:48:07.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:07.756+0000 7fb95de00640 1 --2- 172.21.15.17:0/1947154517 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb9600c6390 0x7fb9600cd160 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fb95000d780 tx=0x7fb95000dc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:48:07.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:07.756+0000 7fb956000640 1 -- 172.21.15.17:0/1947154517 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb950017070 con 0x7fb9600c6390 2024-09-10T21:48:07.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:07.756+0000 7fb956000640 1 -- 172.21.15.17:0/1947154517 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(6 keys) ==== 172+0+0 (secure 0 0 0) 0x7fb95000de90 con 0x7fb9600c6390 2024-09-10T21:48:07.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:07.756+0000 7fb965200640 1 -- 172.21.15.17:0/1947154517 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9601f2740 con 0x7fb9600c6390 2024-09-10T21:48:07.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:07.756+0000 7fb956000640 1 -- 172.21.15.17:0/1947154517 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb950004020 con 0x7fb9600c6390 2024-09-10T21:48:07.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:07.760+0000 7fb965200640 1 -- 172.21.15.17:0/1947154517 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9601f4f80 con 0x7fb9600c6390 2024-09-10T21:48:07.765 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:07.760+0000 7fb965200640 1 -- 172.21.15.17:0/1947154517 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb960158870 con 0x7fb9600c6390 2024-09-10T21:48:07.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:07.760+0000 7fb956000640 1 -- 172.21.15.17:0/1947154517 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7fb9500041c0 con 0x7fb9600c6390 2024-09-10T21:48:07.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:07.760+0000 7fb956000640 1 --2- 172.21.15.17:0/1947154517 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fb93c0725e0 0x7fb93c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:48:07.766 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:07.760+0000 7fb956000640 1 -- 172.21.15.17:0/1947154517 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7fb9500939b0 con 0x7fb9600c6390 2024-09-10T21:48:07.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:07.760+0000 7fb957e00640 1 --2- 172.21.15.17:0/1947154517 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fb93c0725e0 0x7fb93c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:48:07.767 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:07.764+0000 7fb957e00640 1 --2- 172.21.15.17:0/1947154517 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fb93c0725e0 0x7fb93c074aa0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fb960156370 tx=0x7fb948009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:48:07.770 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:07.764+0000 7fb956000640 1 -- 172.21.15.17:0/1947154517 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7fb950020020 con 0x7fb9600c6390 2024-09-10T21:48:07.986 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:07.980+0000 7fb965200640 1 -- 172.21.15.17:0/1947154517 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb96014e740 con 0x7fb9600c6390 2024-09-10T21:48:07.987 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:07.984+0000 7fb956000640 1 -- 172.21.15.17:0/1947154517 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+18950 (secure 0 0 0) 0x7fb95005e5a0 con 0x7fb9600c6390 2024-09-10T21:48:07.988 INFO:teuthology.orchestra.run.smithi017.stdout: 2024-09-10T21:48:07.988 INFO:teuthology.orchestra.run.smithi017.stdout:{"epoch":27,"fsid":"5e2f023f-d6a6-49f4-a256-f4807dce3739","created":"2024-09-10T21:24:21.750712+0000","modified":"2024-09-10T21:25:28.499315+0000","last_up_change":"2024-09-10T21:24:35.877801+0000","last_in_change":"2024-09-10T21:24:25.831127+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T21:24:35.362670+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-10T21:24:45.211689+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-10T21:24:46.209784+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-10T21:24:49.871300+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"0f5fabc0-434e-4066-b430-233969a548f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6808","nonce":4105489867},{"type":"v1","addr":"172.21.15.17:6809","nonce":4105489867}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6810","nonce":4105489867},{"type":"v1","addr":"172.21.15.17:6811","nonce":4105489867}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6814","nonce":4105489867},{"type":"v1","addr":"172.21.15.17:6815","nonce":4105489867}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6812","nonce":4105489867},{"type":"v1","addr":"172.21.15.17:6813","nonce":4105489867}]},"public_addr":"172.21.15.17:6809/4105489867","cluster_addr":"172.21.15.17:6811/4105489867","heartbeat_back_addr":"172.21.15.17:6815/4105489867","heartbeat_front_addr":"172.21.15.17:6813/4105489867","state":["exists","up"]},{"osd":1,"uuid":"f6691cb3-2a91-4c90-a9fd-ece3c11dd6eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6824","nonce":2599842027},{"type":"v1","addr":"172.21.15.17:6825","nonce":2599842027}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6826","nonce":2599842027},{"type":"v1","addr":"172.21.15.17:6827","nonce":2599842027}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6830","nonce":2599842027},{"type":"v1","addr":"172.21.15.17:6831","nonce":2599842027}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6828","nonce":2599842027},{"type":"v1","addr":"172.21.15.17:6829","nonce":2599842027}]},"public_addr":"172.21.15.17:6825/2599842027","cluster_addr":"172.21.15.17:6827/2599842027","heartbeat_back_addr":"172.21.15.17:6831/2599842027","heartbeat_front_addr":"172.21.15.17:6829/2599842027","state":["exists","up"]},{"osd":2,"uuid":"cb61adb0-a216-437b-9598-06fa7c84aa64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6816","nonce":1248243634},{"type":"v1","addr":"172.21.15.17:6817","nonce":1248243634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6818","nonce":1248243634},{"type":"v1","addr":"172.21.15.17:6819","nonce":1248243634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6822","nonce":1248243634},{"type":"v1","addr":"172.21.15.17:6823","nonce":1248243634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6820","nonce":1248243634},{"type":"v1","addr":"172.21.15.17:6821","nonce":1248243634}]},"public_addr":"172.21.15.17:6817/1248243634","cluster_addr":"172.21.15.17:6819/1248243634","heartbeat_back_addr":"172.21.15.17:6823/1248243634","heartbeat_front_addr":"172.21.15.17:6821/1248243634","state":["exists","up"]},{"osd":3,"uuid":"6d2da26a-e0bf-47dc-b020-af86e6bcfccc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6800","nonce":2314489522},{"type":"v1","addr":"172.21.15.17:6801","nonce":2314489522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6802","nonce":2314489522},{"type":"v1","addr":"172.21.15.17:6803","nonce":2314489522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6806","nonce":2314489522},{"type":"v1","addr":"172.21.15.17:6807","nonce":2314489522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.17:6804","nonce":2314489522},{"type":"v1","addr":"172.21.15.17:6805","nonce":2314489522}]},"public_addr":"172.21.15.17:6801/2314489522","cluster_addr":"172.21.15.17:6803/2314489522","heartbeat_back_addr":"172.21.15.17:6807/2314489522","heartbeat_front_addr":"172.21.15.17:6805/2314489522","state":["exists","up"]},{"osd":4,"uuid":"c65071dd-9ca7-4f1d-a584-45dc121419a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6810","nonce":202520744},{"type":"v1","addr":"172.21.15.31:6811","nonce":202520744}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6812","nonce":202520744},{"type":"v1","addr":"172.21.15.31:6813","nonce":202520744}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6816","nonce":202520744},{"type":"v1","addr":"172.21.15.31:6817","nonce":202520744}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6814","nonce":202520744},{"type":"v1","addr":"172.21.15.31:6815","nonce":202520744}]},"public_addr":"172.21.15.31:6811/202520744","cluster_addr":"172.21.15.31:6813/202520744","heartbeat_back_addr":"172.21.15.31:6817/202520744","heartbeat_front_addr":"172.21.15.31:6815/202520744","state":["exists","up"]},{"osd":5,"uuid":"235389de-7777-4744-81b8-d0bfc11190e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6800","nonce":1472711057},{"type":"v1","addr":"172.21.15.31:6801","nonce":1472711057}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6802","nonce":1472711057},{"type":"v1","addr":"172.21.15.31:6803","nonce":1472711057}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6806","nonce":1472711057},{"type":"v1","addr":"172.21.15.31:6807","nonce":1472711057}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6804","nonce":1472711057},{"type":"v1","addr":"172.21.15.31:6805","nonce":1472711057}]},"public_addr":"172.21.15.31:6801/1472711057","cluster_addr":"172.21.15.31:6803/1472711057","heartbeat_back_addr":"172.21.15.31:6807/1472711057","heartbeat_front_addr":"172.21.15.31:6805/1472711057","state":["exists","up"]},{"osd":6,"uuid":"cea70a57-3715-4ebb-8511-a528a9df2b1b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6826","nonce":3754251089},{"type":"v1","addr":"172.21.15.31:6827","nonce":3754251089}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6828","nonce":3754251089},{"type":"v1","addr":"172.21.15.31:6829","nonce":3754251089}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6832","nonce":3754251089},{"type":"v1","addr":"172.21.15.31:6833","nonce":3754251089}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6830","nonce":3754251089},{"type":"v1","addr":"172.21.15.31:6831","nonce":3754251089}]},"public_addr":"172.21.15.31:6827/3754251089","cluster_addr":"172.21.15.31:6829/3754251089","heartbeat_back_addr":"172.21.15.31:6833/3754251089","heartbeat_front_addr":"172.21.15.31:6831/3754251089","state":["exists","up"]},{"osd":7,"uuid":"01cc111b-a45f-4ca3-9cc1-7fe0afbeba45","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6818","nonce":3948648934},{"type":"v1","addr":"172.21.15.31:6819","nonce":3948648934}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6820","nonce":3948648934},{"type":"v1","addr":"172.21.15.31:6821","nonce":3948648934}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6824","nonce":3948648934},{"type":"v1","addr":"172.21.15.31:6825","nonce":3948648934}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.31:6822","nonce":3948648934},{"type":"v1","addr":"172.21.15.31:6823","nonce":3948648934}]},"public_addr":"172.21.15.31:6819/3948648934","cluster_addr":"172.21.15.31:6821/3948648934","heartbeat_back_addr":"172.21.15.31:6825/3948648934","heartbeat_front_addr":"172.21.15.31:6823/3948648934","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:24:32.606989+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:24:32.891535+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:24:33.330793+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:24:32.811533+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:24:33.894624+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:24:32.773817+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:24:32.901865+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T21:24:32.907207+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.d","mappings":[{"from":5,"to":1}]},{"pgid":"3.e","mappings":[{"from":7,"to":2}]},{"pgid":"3.28","mappings":[{"from":5,"to":2}]},{"pgid":"3.29","mappings":[{"from":3,"to":2}]},{"pgid":"4.a","mappings":[{"from":1,"to":2}]},{"pgid":"4.14","mappings":[{"from":1,"to":5}]},{"pgid":"4.1b","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T21:48:07.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:07.984+0000 7fb94f400640 1 -- 172.21.15.17:0/1947154517 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fb93c0725e0 msgr2=0x7fb93c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:48:07.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:07.984+0000 7fb94f400640 1 --2- 172.21.15.17:0/1947154517 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fb93c0725e0 0x7fb93c074aa0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fb960156370 tx=0x7fb948009040 comp rx=0 tx=0).stop 2024-09-10T21:48:07.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:07.984+0000 7fb94f400640 1 -- 172.21.15.17:0/1947154517 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb9600c6390 msgr2=0x7fb9600cd160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:48:07.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:07.984+0000 7fb94f400640 1 --2- 172.21.15.17:0/1947154517 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb9600c6390 0x7fb9600cd160 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fb95000d780 tx=0x7fb95000dc40 comp rx=0 tx=0).stop 2024-09-10T21:48:07.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:07.984+0000 7fb94f400640 1 -- 172.21.15.17:0/1947154517 shutdown_connections 2024-09-10T21:48:07.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:07.984+0000 7fb94f400640 1 --2- 172.21.15.17:0/1947154517 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7fb93c0725e0 0x7fb93c074aa0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:07.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:07.984+0000 7fb94f400640 1 --2- 172.21.15.17:0/1947154517 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7fb960152590 0x7fb9601f5780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:07.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:07.984+0000 7fb94f400640 1 --2- 172.21.15.17:0/1947154517 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7fb9600c6790 0x7fb960157b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:07.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:07.984+0000 7fb94f400640 1 --2- 172.21.15.17:0/1947154517 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7fb9600c6390 0x7fb9600cd160 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:07.991 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:07.984+0000 7fb94f400640 1 -- 172.21.15.17:0/1947154517 >> 172.21.15.17:0/1947154517 conn(0x7fb96006ce20 msgr2=0x7fb9600c7430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:48:07.992 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:07.988+0000 7fb94f400640 1 -- 172.21.15.17:0/1947154517 shutdown_connections 2024-09-10T21:48:07.993 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:07.988+0000 7fb94f400640 1 -- 172.21.15.17:0/1947154517 wait complete. 2024-09-10T21:48:08.039 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-10T21:48:08.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:08.184+0000 7f0783e00640 1 -- 172.21.15.17:0/2727018794 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f077c152820 msgr2=0x7f077c0b3eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:48:08.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:08.184+0000 7f0783e00640 1 --2- 172.21.15.17:0/2727018794 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f077c152820 0x7f077c0b3eb0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f077c1594a0 tx=0x7f076801c850 comp rx=0 tx=0).stop 2024-09-10T21:48:08.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:08.184+0000 7f0783e00640 1 -- 172.21.15.17:0/2727018794 shutdown_connections 2024-09-10T21:48:08.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:08.184+0000 7f0783e00640 1 --2- 172.21.15.17:0/2727018794 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f077c152820 0x7f077c0b3eb0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:08.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:08.184+0000 7f0783e00640 1 -- 172.21.15.17:0/2727018794 >> 172.21.15.17:0/2727018794 conn(0x7f077c0702f0 msgr2=0x7f077c070700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:48:08.192 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:08.188+0000 7f0783e00640 1 -- 172.21.15.17:0/2727018794 shutdown_connections 2024-09-10T21:48:08.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:08.188+0000 7f0783e00640 1 -- 172.21.15.17:0/2727018794 wait complete. 2024-09-10T21:48:08.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:08.188+0000 7f0783e00640 1 Processor -- start 2024-09-10T21:48:08.193 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:08.188+0000 7f0783e00640 1 -- start start 2024-09-10T21:48:08.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:08.188+0000 7f0783e00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f077c0cdf20 0x7f077c068bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:48:08.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:08.188+0000 7f0783e00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f077c152440 0x7f077c1f1e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:48:08.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:08.188+0000 7f0783e00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f077c152820 0x7f077c1f2390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:48:08.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:08.188+0000 7f0783e00640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f077c0cb5d0 con 0x7f077c152440 2024-09-10T21:48:08.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:08.188+0000 7f0783e00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f077c0cb450 con 0x7f077c0cdf20 2024-09-10T21:48:08.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:08.188+0000 7f0783e00640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f077c0c2ff0 con 0x7f077c152820 2024-09-10T21:48:08.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:08.188+0000 7f0780a00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f077c152440 0x7f077c1f1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:48:08.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:08.188+0000 7f0780a00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f077c152440 0x7f077c1f1e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:48628/0 (socket says 172.21.15.17:48628) 2024-09-10T21:48:08.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:08.188+0000 7f0780a00640 1 -- 172.21.15.17:0/2540566986 learned_addr learned my addr 172.21.15.17:0/2540566986 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:48:08.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:08.188+0000 7f0781e00640 1 --2- 172.21.15.17:0/2540566986 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f077c152820 0x7f077c1f2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:48:08.195 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:08.188+0000 7f0781400640 1 --2- 172.21.15.17:0/2540566986 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f077c0cdf20 0x7f077c068bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:48:08.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:08.188+0000 7f0780a00640 1 -- 172.21.15.17:0/2540566986 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f077c152820 msgr2=0x7f077c1f2390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:48:08.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:08.188+0000 7f0780a00640 1 --2- 172.21.15.17:0/2540566986 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f077c152820 0x7f077c1f2390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:08.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:08.188+0000 7f0780a00640 1 -- 172.21.15.17:0/2540566986 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f077c0cdf20 msgr2=0x7f077c068bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:48:08.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:08.188+0000 7f0780a00640 1 --2- 172.21.15.17:0/2540566986 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f077c0cdf20 0x7f077c068bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:08.196 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:08.188+0000 7f0780a00640 1 -- 172.21.15.17:0/2540566986 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f077c157650 con 0x7f077c152440 2024-09-10T21:48:08.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:08.188+0000 7f0780a00640 1 --2- 172.21.15.17:0/2540566986 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f077c152440 0x7f077c1f1e50 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f07680b0040 tx=0x7f07680a5710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:48:08.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:08.192+0000 7f0779600640 1 -- 172.21.15.17:0/2540566986 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07680b2040 con 0x7f077c152440 2024-09-10T21:48:08.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:08.192+0000 7f0779600640 1 -- 172.21.15.17:0/2540566986 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(6 keys) ==== 172+0+0 (secure 0 0 0) 0x7f0768004470 con 0x7f077c152440 2024-09-10T21:48:08.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:08.192+0000 7f0783e00640 1 -- 172.21.15.17:0/2540566986 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f077c071e50 con 0x7f077c152440 2024-09-10T21:48:08.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:08.192+0000 7f0779600640 1 -- 172.21.15.17:0/2540566986 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0768003860 con 0x7f077c152440 2024-09-10T21:48:08.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:08.192+0000 7f0783e00640 1 -- 172.21.15.17:0/2540566986 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f077c1479a0 con 0x7f077c152440 2024-09-10T21:48:08.197 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:08.192+0000 7f0783e00640 1 -- 172.21.15.17:0/2540566986 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f077c158360 con 0x7f077c152440 2024-09-10T21:48:08.198 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:08.192+0000 7f0779600640 1 -- 172.21.15.17:0/2540566986 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f0768003a00 con 0x7f077c152440 2024-09-10T21:48:08.199 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:08.196+0000 7f0779600640 1 --2- 172.21.15.17:0/2540566986 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f075c0725e0 0x7f075c074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:48:08.200 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:08.196+0000 7f0779600640 1 -- 172.21.15.17:0/2540566986 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 6735+0+0 (secure 0 0 0) 0x7f076812d910 con 0x7f077c152440 2024-09-10T21:48:08.200 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:08.196+0000 7f0781400640 1 --2- 172.21.15.17:0/2540566986 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f075c0725e0 0x7f075c074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:48:08.201 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:08.196+0000 7f0781400640 1 --2- 172.21.15.17:0/2540566986 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f075c0725e0 0x7f075c074aa0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f0770007fd0 tx=0x7f0770009040 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:48:08.207 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:08.204+0000 7f0779600640 1 -- 172.21.15.17:0/2540566986 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f07680164d0 con 0x7f077c152440 2024-09-10T21:48:08.434 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:08.428+0000 7f0783e00640 1 -- 172.21.15.17:0/2540566986 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f077c14e4d0 con 0x7f077c152440 2024-09-10T21:48:08.446 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:08.440+0000 7f0779600640 1 -- 172.21.15.17:0/2540566986 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v11) ==== 161+0+0 (secure 0 0 0) 0x7f07680f8490 con 0x7f077c152440 2024-09-10T21:48:08.446 INFO:teuthology.orchestra.run.smithi017.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-10T21:48:08.449 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:08.444+0000 7f0783e00640 1 -- 172.21.15.17:0/2540566986 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f075c0725e0 msgr2=0x7f075c074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:48:08.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:08.444+0000 7f0783e00640 1 --2- 172.21.15.17:0/2540566986 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f075c0725e0 0x7f075c074aa0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f0770007fd0 tx=0x7f0770009040 comp rx=0 tx=0).stop 2024-09-10T21:48:08.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:08.444+0000 7f0783e00640 1 -- 172.21.15.17:0/2540566986 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f077c152440 msgr2=0x7f077c1f1e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:48:08.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:08.444+0000 7f0783e00640 1 --2- 172.21.15.17:0/2540566986 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f077c152440 0x7f077c1f1e50 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f07680b0040 tx=0x7f07680a5710 comp rx=0 tx=0).stop 2024-09-10T21:48:08.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:08.444+0000 7f0783e00640 1 -- 172.21.15.17:0/2540566986 shutdown_connections 2024-09-10T21:48:08.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:08.444+0000 7f0783e00640 1 --2- 172.21.15.17:0/2540566986 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f075c0725e0 0x7f075c074aa0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:08.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:08.444+0000 7f0783e00640 1 --2- 172.21.15.17:0/2540566986 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f077c152820 0x7f077c1f2390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:08.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:08.444+0000 7f0783e00640 1 --2- 172.21.15.17:0/2540566986 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f077c152440 0x7f077c1f1e50 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:08.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:08.444+0000 7f0783e00640 1 --2- 172.21.15.17:0/2540566986 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f077c0cdf20 0x7f077c068bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:08.450 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:08.444+0000 7f0783e00640 1 -- 172.21.15.17:0/2540566986 >> 172.21.15.17:0/2540566986 conn(0x7f077c0702f0 msgr2=0x7f077c0b3ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:48:08.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:08.444+0000 7f0783e00640 1 -- 172.21.15.17:0/2540566986 shutdown_connections 2024-09-10T21:48:08.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:08.444+0000 7f0783e00640 1 -- 172.21.15.17:0/2540566986 wait complete. 2024-09-10T21:48:08.469 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-10T21:48:08.551 INFO:tasks.ceph.mds.d.smithi031.stdout:starting mds.d at 2024-09-10T21:48:08.620 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:08.616+0000 7f946d000640 1 -- 172.21.15.17:0/239778344 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f94680c96f0 msgr2=0x7f9468152410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:48:08.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:08.616+0000 7f946d000640 1 --2- 172.21.15.17:0/239778344 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f94680c96f0 0x7f9468152410 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f94680c7a70 tx=0x7f945001c8c0 comp rx=0 tx=0).stop 2024-09-10T21:48:08.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:08.616+0000 7f946d000640 1 -- 172.21.15.17:0/239778344 shutdown_connections 2024-09-10T21:48:08.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:08.616+0000 7f946d000640 1 --2- 172.21.15.17:0/239778344 >> v2:172.21.15.31:3300/0 conn(0x7f94680c34f0 0x7f9468152950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:08.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:08.616+0000 7f946d000640 1 --2- 172.21.15.17:0/239778344 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f94680c96f0 0x7f9468152410 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:08.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:08.616+0000 7f946d000640 1 --2- 172.21.15.17:0/239778344 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f94680c9310 0x7f9468070960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:08.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:08.616+0000 7f946d000640 1 -- 172.21.15.17:0/239778344 >> 172.21.15.17:0/239778344 conn(0x7f946806d220 msgr2=0x7f9468068990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:48:08.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:08.616+0000 7f946d000640 1 -- 172.21.15.17:0/239778344 shutdown_connections 2024-09-10T21:48:08.621 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:08.616+0000 7f946d000640 1 -- 172.21.15.17:0/239778344 wait complete. 2024-09-10T21:48:08.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:08.616+0000 7f946d000640 1 Processor -- start 2024-09-10T21:48:08.622 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:08.616+0000 7f946d000640 1 -- start start 2024-09-10T21:48:08.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:08.620+0000 7f946d000640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f94680c34f0 0x7f94680c7800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:48:08.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:08.620+0000 7f946d000640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f94680c9310 0x7f94680cd150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:48:08.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:08.620+0000 7f946d000640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f94680c96f0 0x7f94680cd690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:48:08.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:08.620+0000 7f946d000640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f9468070400 con 0x7f94680c34f0 2024-09-10T21:48:08.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:08.620+0000 7f946d000640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f94680c9d40 con 0x7f94680c9310 2024-09-10T21:48:08.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:08.620+0000 7f946d000640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f9468070580 con 0x7f94680c96f0 2024-09-10T21:48:08.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:08.620+0000 7f9465e00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f94680c34f0 0x7f94680c7800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:48:08.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:08.620+0000 7f9465e00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f94680c34f0 0x7f94680c7800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:48640/0 (socket says 172.21.15.17:48640) 2024-09-10T21:48:08.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:08.620+0000 7f9465e00640 1 -- 172.21.15.17:0/2781070786 learned_addr learned my addr 172.21.15.17:0/2781070786 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:48:08.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:08.620+0000 7f945fe00640 1 --2- 172.21.15.17:0/2781070786 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f94680c9310 0x7f94680cd150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:48:08.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:08.620+0000 7f9466800640 1 --2- 172.21.15.17:0/2781070786 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f94680c96f0 0x7f94680cd690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:48:08.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:08.620+0000 7f9465e00640 1 -- 172.21.15.17:0/2781070786 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f94680c96f0 msgr2=0x7f94680cd690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:48:08.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:08.620+0000 7f9465e00640 1 --2- 172.21.15.17:0/2781070786 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f94680c96f0 0x7f94680cd690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:08.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:08.620+0000 7f9465e00640 1 -- 172.21.15.17:0/2781070786 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f94680c9310 msgr2=0x7f94680cd150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:48:08.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:08.620+0000 7f9465e00640 1 --2- 172.21.15.17:0/2781070786 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f94680c9310 0x7f94680cd150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:08.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:08.620+0000 7f9465e00640 1 -- 172.21.15.17:0/2781070786 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f946814d990 con 0x7f94680c34f0 2024-09-10T21:48:08.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:08.620+0000 7f945fe00640 1 --2- 172.21.15.17:0/2781070786 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f94680c9310 0x7f94680cd150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:48:08.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:08.620+0000 7f9465e00640 1 --2- 172.21.15.17:0/2781070786 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f94680c34f0 0x7f94680c7800 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f945800d980 tx=0x7f945800de40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:48:08.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:08.620+0000 7f945e000640 1 -- 172.21.15.17:0/2781070786 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9458017070 con 0x7f94680c34f0 2024-09-10T21:48:08.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:08.620+0000 7f945e000640 1 -- 172.21.15.17:0/2781070786 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(6 keys) ==== 172+0+0 (secure 0 0 0) 0x7f9458009d40 con 0x7f94680c34f0 2024-09-10T21:48:08.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:08.620+0000 7f946d000640 1 -- 172.21.15.17:0/2781070786 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f946814d3d0 con 0x7f94680c34f0 2024-09-10T21:48:08.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:08.620+0000 7f945e000640 1 -- 172.21.15.17:0/2781070786 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94580106c0 con 0x7f94680c34f0 2024-09-10T21:48:08.627 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:08.620+0000 7f946d000640 1 -- 172.21.15.17:0/2781070786 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f946814e160 con 0x7f94680c34f0 2024-09-10T21:48:08.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:08.620+0000 7f946d000640 1 -- 172.21.15.17:0/2781070786 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94680c39c0 con 0x7f94680c34f0 2024-09-10T21:48:08.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:08.624+0000 7f945e000640 1 -- 172.21.15.17:0/2781070786 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f9458010860 con 0x7f94680c34f0 2024-09-10T21:48:08.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:08.624+0000 7f945e000640 1 --2- 172.21.15.17:0/2781070786 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f94440725e0 0x7f9444074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:48:08.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:08.624+0000 7f945e000640 1 -- 172.21.15.17:0/2781070786 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6821+0+0 (secure 0 0 0) 0x7f9458093ad0 con 0x7f94680c34f0 2024-09-10T21:48:08.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:08.624+0000 7f945fe00640 1 --2- 172.21.15.17:0/2781070786 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f94440725e0 0x7f9444074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:48:08.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:08.624+0000 7f945fe00640 1 --2- 172.21.15.17:0/2781070786 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f94440725e0 0x7f9444074aa0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f94500af000 tx=0x7f9450019660 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:48:08.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:08.628+0000 7f945e000640 1 -- 172.21.15.17:0/2781070786 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f945805ed40 con 0x7f94680c34f0 2024-09-10T21:48:08.848 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:08.844+0000 7f946d000640 1 -- 172.21.15.17:0/2781070786 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f9468156fd0 con 0x7f94680c34f0 2024-09-10T21:48:09.448 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:09.444+0000 7f945e000640 1 -- 172.21.15.17:0/2781070786 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v12) ==== 104+0+0 (secure 0 0 0) 0x7f945805e640 con 0x7f94680c34f0 2024-09-10T21:48:09.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:09.444+0000 7f946d000640 1 -- 172.21.15.17:0/2781070786 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f94440725e0 msgr2=0x7f9444074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:48:09.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:09.444+0000 7f946d000640 1 --2- 172.21.15.17:0/2781070786 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f94440725e0 0x7f9444074aa0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f94500af000 tx=0x7f9450019660 comp rx=0 tx=0).stop 2024-09-10T21:48:09.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:09.444+0000 7f946d000640 1 -- 172.21.15.17:0/2781070786 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f94680c34f0 msgr2=0x7f94680c7800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:48:09.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:09.444+0000 7f946d000640 1 --2- 172.21.15.17:0/2781070786 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f94680c34f0 0x7f94680c7800 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f945800d980 tx=0x7f945800de40 comp rx=0 tx=0).stop 2024-09-10T21:48:09.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:09.444+0000 7f946d000640 1 -- 172.21.15.17:0/2781070786 shutdown_connections 2024-09-10T21:48:09.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:09.444+0000 7f946d000640 1 --2- 172.21.15.17:0/2781070786 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f94440725e0 0x7f9444074aa0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:09.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:09.444+0000 7f946d000640 1 --2- 172.21.15.17:0/2781070786 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f94680c96f0 0x7f94680cd690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:09.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:09.444+0000 7f946d000640 1 --2- 172.21.15.17:0/2781070786 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f94680c9310 0x7f94680cd150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:09.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:09.444+0000 7f946d000640 1 --2- 172.21.15.17:0/2781070786 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f94680c34f0 0x7f94680c7800 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:09.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:09.444+0000 7f946d000640 1 -- 172.21.15.17:0/2781070786 >> 172.21.15.17:0/2781070786 conn(0x7f946806d220 msgr2=0x7f94681521e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:48:09.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:09.444+0000 7f946d000640 1 -- 172.21.15.17:0/2781070786 shutdown_connections 2024-09-10T21:48:09.451 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:09.444+0000 7f946d000640 1 -- 172.21.15.17:0/2781070786 wait complete. 2024-09-10T21:48:09.472 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-10T21:48:09.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:09.620+0000 7effb0800640 1 -- 172.21.15.17:0/4288780112 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7effa8152520 msgr2=0x7effa8070dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:48:09.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:09.620+0000 7effb0800640 1 --2- 172.21.15.17:0/4288780112 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7effa8152520 0x7effa8070dd0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7eff9c009870 tx=0x7eff9c01ef70 comp rx=0 tx=0).stop 2024-09-10T21:48:09.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:09.620+0000 7effb0800640 1 -- 172.21.15.17:0/4288780112 shutdown_connections 2024-09-10T21:48:09.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:09.620+0000 7effb0800640 1 --2- 172.21.15.17:0/4288780112 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7effa8152520 0x7effa8070dd0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:09.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:09.620+0000 7effb0800640 1 -- 172.21.15.17:0/4288780112 >> 172.21.15.17:0/4288780112 conn(0x7effa80748d0 msgr2=0x7effa8068990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:48:09.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:09.620+0000 7effb0800640 1 -- 172.21.15.17:0/4288780112 shutdown_connections 2024-09-10T21:48:09.625 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:09.620+0000 7effb0800640 1 -- 172.21.15.17:0/4288780112 wait complete. 2024-09-10T21:48:09.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:09.620+0000 7effb0800640 1 Processor -- start 2024-09-10T21:48:09.626 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:09.620+0000 7effb0800640 1 -- start start 2024-09-10T21:48:09.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:09.620+0000 7effb0800640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7effa8071680 0x7effa81f1f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:48:09.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:09.620+0000 7effb0800640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7effa80cdf50 0x7effa814d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:48:09.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:09.620+0000 7effb0800640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7effa8152520 0x7effa814d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:48:09.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:09.620+0000 7effb0800640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7effa8152900 con 0x7effa80cdf50 2024-09-10T21:48:09.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:09.620+0000 7effb0800640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7effa80c3270 con 0x7effa8071680 2024-09-10T21:48:09.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:09.620+0000 7effb0800640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7effa8152a80 con 0x7effa8152520 2024-09-10T21:48:09.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:09.624+0000 7effad400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7effa80cdf50 0x7effa814d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:48:09.628 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:09.624+0000 7effad400640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7effa80cdf50 0x7effa814d350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:48646/0 (socket says 172.21.15.17:48646) 2024-09-10T21:48:09.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:09.624+0000 7effad400640 1 -- 172.21.15.17:0/610200663 learned_addr learned my addr 172.21.15.17:0/610200663 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:48:09.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:09.624+0000 7effade00640 1 --2- 172.21.15.17:0/610200663 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7effa8071680 0x7effa81f1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:48:09.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:09.624+0000 7effae800640 1 --2- 172.21.15.17:0/610200663 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7effa8152520 0x7effa814d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:48:09.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:09.624+0000 7effad400640 1 -- 172.21.15.17:0/610200663 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7effa8152520 msgr2=0x7effa814d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:48:09.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:09.624+0000 7effad400640 1 --2- 172.21.15.17:0/610200663 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7effa8152520 0x7effa814d890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:09.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:09.624+0000 7effad400640 1 -- 172.21.15.17:0/610200663 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7effa8071680 msgr2=0x7effa81f1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:48:09.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:09.624+0000 7effad400640 1 --2- 172.21.15.17:0/610200663 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7effa8071680 0x7effa81f1f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:09.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:09.624+0000 7effad400640 1 -- 172.21.15.17:0/610200663 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effa81f2440 con 0x7effa80cdf50 2024-09-10T21:48:09.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:09.624+0000 7effad400640 1 --2- 172.21.15.17:0/610200663 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7effa80cdf50 0x7effa814d350 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7eff98001ec0 tx=0x7eff9800cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:48:09.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:09.624+0000 7effa2000640 1 -- 172.21.15.17:0/610200663 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff98016020 con 0x7effa80cdf50 2024-09-10T21:48:09.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:09.624+0000 7effa2000640 1 -- 172.21.15.17:0/610200663 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(6 keys) ==== 172+0+0 (secure 0 0 0) 0x7eff9800a840 con 0x7effa80cdf50 2024-09-10T21:48:09.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:09.624+0000 7effa2000640 1 -- 172.21.15.17:0/610200663 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff9800acb0 con 0x7effa80cdf50 2024-09-10T21:48:09.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:09.624+0000 7effb0800640 1 -- 172.21.15.17:0/610200663 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effa814e0c0 con 0x7effa80cdf50 2024-09-10T21:48:09.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:09.624+0000 7effb0800640 1 -- 172.21.15.17:0/610200663 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effa814c870 con 0x7effa80cdf50 2024-09-10T21:48:09.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:09.624+0000 7effb0800640 1 -- 172.21.15.17:0/610200663 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7effa8159920 con 0x7effa80cdf50 2024-09-10T21:48:09.632 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:09.628+0000 7effa2000640 1 -- 172.21.15.17:0/610200663 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7eff9801a3b0 con 0x7effa80cdf50 2024-09-10T21:48:09.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:09.628+0000 7effa2000640 1 --2- 172.21.15.17:0/610200663 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7eff8c072610 0x7eff8c074ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:48:09.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:09.628+0000 7effa2000640 1 -- 172.21.15.17:0/610200663 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 6821+0+0 (secure 0 0 0) 0x7eff980937c0 con 0x7effa80cdf50 2024-09-10T21:48:09.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:09.628+0000 7effade00640 1 --2- 172.21.15.17:0/610200663 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7eff8c072610 0x7eff8c074ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:48:09.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:09.628+0000 7effade00640 1 --2- 172.21.15.17:0/610200663 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7eff8c072610 0x7eff8c074ad0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7eff9c0b2000 tx=0x7eff9c01bd10 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:48:09.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:09.632+0000 7effa2000640 1 -- 172.21.15.17:0/610200663 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7eff98020030 con 0x7effa80cdf50 2024-09-10T21:48:09.814 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:09.808+0000 7effb0800640 1 -- 172.21.15.17:0/610200663 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7effa8070e90 con 0x7effa80cdf50 2024-09-10T21:48:10.455 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:10.448+0000 7effa2000640 1 -- 172.21.15.17:0/610200663 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v29) ==== 181+0+0 (secure 0 0 0) 0x7eff9805e2c0 con 0x7effa80cdf50 2024-09-10T21:48:10.455 INFO:teuthology.orchestra.run.smithi017.stderr:pool 'cephfs_metadata' removed 2024-09-10T21:48:10.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:10.452+0000 7effb0800640 1 -- 172.21.15.17:0/610200663 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7eff8c072610 msgr2=0x7eff8c074ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:48:10.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:10.452+0000 7effb0800640 1 --2- 172.21.15.17:0/610200663 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7eff8c072610 0x7eff8c074ad0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7eff9c0b2000 tx=0x7eff9c01bd10 comp rx=0 tx=0).stop 2024-09-10T21:48:10.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:10.452+0000 7effb0800640 1 -- 172.21.15.17:0/610200663 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7effa80cdf50 msgr2=0x7effa814d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:48:10.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:10.452+0000 7effb0800640 1 --2- 172.21.15.17:0/610200663 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7effa80cdf50 0x7effa814d350 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7eff98001ec0 tx=0x7eff9800cb90 comp rx=0 tx=0).stop 2024-09-10T21:48:10.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:10.452+0000 7effb0800640 1 -- 172.21.15.17:0/610200663 shutdown_connections 2024-09-10T21:48:10.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:10.452+0000 7effb0800640 1 --2- 172.21.15.17:0/610200663 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7eff8c072610 0x7eff8c074ad0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:10.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:10.452+0000 7effb0800640 1 --2- 172.21.15.17:0/610200663 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7effa8152520 0x7effa814d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:10.457 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:10.452+0000 7effb0800640 1 --2- 172.21.15.17:0/610200663 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7effa80cdf50 0x7effa814d350 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:10.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:10.452+0000 7effb0800640 1 --2- 172.21.15.17:0/610200663 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7effa8071680 0x7effa81f1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:10.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:10.452+0000 7effb0800640 1 -- 172.21.15.17:0/610200663 >> 172.21.15.17:0/610200663 conn(0x7effa80748d0 msgr2=0x7effa8075a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:48:10.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:10.452+0000 7effb0800640 1 -- 172.21.15.17:0/610200663 shutdown_connections 2024-09-10T21:48:10.458 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:10.452+0000 7effb0800640 1 -- 172.21.15.17:0/610200663 wait complete. 2024-09-10T21:48:10.478 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-10T21:48:10.629 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:10.624+0000 7f189fa00640 1 -- 172.21.15.17:0/2891534632 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f18980c6c30 msgr2=0x7f18980cc1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:48:10.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:10.624+0000 7f189fa00640 1 --2- 172.21.15.17:0/2891534632 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f18980c6c30 0x7f18980cc1c0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f18980b2710 tx=0x7f188401f090 comp rx=0 tx=0).stop 2024-09-10T21:48:10.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:10.624+0000 7f189fa00640 1 -- 172.21.15.17:0/2891534632 shutdown_connections 2024-09-10T21:48:10.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:10.624+0000 7f189fa00640 1 --2- 172.21.15.17:0/2891534632 >> v2:172.21.15.31:3300/0 conn(0x7f18980c7120 0x7f18980cc700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:10.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:10.624+0000 7f189fa00640 1 --2- 172.21.15.17:0/2891534632 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f18980c6c30 0x7f18980cc1c0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:10.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:10.624+0000 7f189fa00640 1 --2- 172.21.15.17:0/2891534632 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f1898152750 0x7f18980c96a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:10.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:10.624+0000 7f189fa00640 1 -- 172.21.15.17:0/2891534632 >> 172.21.15.17:0/2891534632 conn(0x7f189806cc20 msgr2=0x7f189806d030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:48:10.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:10.624+0000 7f189fa00640 1 -- 172.21.15.17:0/2891534632 shutdown_connections 2024-09-10T21:48:10.630 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:10.624+0000 7f189fa00640 1 -- 172.21.15.17:0/2891534632 wait complete. 2024-09-10T21:48:10.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:10.624+0000 7f189fa00640 1 Processor -- start 2024-09-10T21:48:10.631 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:10.624+0000 7f189fa00640 1 -- start start 2024-09-10T21:48:10.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:10.628+0000 7f189fa00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f18980c6c30 0x7f18980cb150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:48:10.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:10.628+0000 7f189fa00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18980c7120 0x7f18980cb690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:48:10.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:10.628+0000 7f189fa00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f1898152750 0x7f18981f39f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:48:10.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:10.628+0000 7f189fa00640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f1898152b30 con 0x7f18980c6c30 2024-09-10T21:48:10.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:10.628+0000 7f189fa00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f1898159130 con 0x7f18980c7120 2024-09-10T21:48:10.633 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:10.628+0000 7f189fa00640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f18981584e0 con 0x7f1898152750 2024-09-10T21:48:10.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:10.628+0000 7f189d000640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f18980c6c30 0x7f18980cb150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:48:10.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:10.628+0000 7f189d000640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f18980c6c30 0x7f18980cb150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:48668/0 (socket says 172.21.15.17:48668) 2024-09-10T21:48:10.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:10.628+0000 7f189d000640 1 -- 172.21.15.17:0/954730406 learned_addr learned my addr 172.21.15.17:0/954730406 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:48:10.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:10.628+0000 7f1897e00640 1 --2- 172.21.15.17:0/954730406 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18980c7120 0x7f18980cb690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:48:10.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:10.628+0000 7f189da00640 1 --2- 172.21.15.17:0/954730406 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f1898152750 0x7f18981f39f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:48:10.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:10.628+0000 7f189d000640 1 -- 172.21.15.17:0/954730406 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f1898152750 msgr2=0x7f18981f39f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:48:10.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:10.628+0000 7f189d000640 1 --2- 172.21.15.17:0/954730406 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f1898152750 0x7f18981f39f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:10.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:10.628+0000 7f189d000640 1 -- 172.21.15.17:0/954730406 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18980c7120 msgr2=0x7f18980cb690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:48:10.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:10.628+0000 7f189d000640 1 --2- 172.21.15.17:0/954730406 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18980c7120 0x7f18980cb690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:10.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:10.628+0000 7f189d000640 1 -- 172.21.15.17:0/954730406 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18981f1110 con 0x7f18980c6c30 2024-09-10T21:48:10.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:10.628+0000 7f1897e00640 1 --2- 172.21.15.17:0/954730406 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18980c7120 0x7f18980cb690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:48:10.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:10.628+0000 7f189da00640 1 --2- 172.21.15.17:0/954730406 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f1898152750 0x7f18981f39f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T21:48:10.634 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:10.628+0000 7f189d000640 1 --2- 172.21.15.17:0/954730406 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f18980c6c30 0x7f18980cb150 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f188c0058d0 tx=0x7f188c00dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:48:10.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:10.628+0000 7f1894c00640 1 -- 172.21.15.17:0/954730406 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f188c017070 con 0x7f18980c6c30 2024-09-10T21:48:10.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:10.628+0000 7f1894c00640 1 -- 172.21.15.17:0/954730406 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(6 keys) ==== 172+0+0 (secure 0 0 0) 0x7f188c009c30 con 0x7f18980c6c30 2024-09-10T21:48:10.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:10.628+0000 7f189fa00640 1 -- 172.21.15.17:0/954730406 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f189815b5a0 con 0x7f18980c6c30 2024-09-10T21:48:10.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:10.628+0000 7f1894c00640 1 -- 172.21.15.17:0/954730406 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f188c0106c0 con 0x7f18980c6c30 2024-09-10T21:48:10.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:10.628+0000 7f189fa00640 1 -- 172.21.15.17:0/954730406 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f189815b2a0 con 0x7f18980c6c30 2024-09-10T21:48:10.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:10.628+0000 7f189fa00640 1 -- 172.21.15.17:0/954730406 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1898155de0 con 0x7f18980c6c30 2024-09-10T21:48:10.635 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:10.628+0000 7f1894c00640 1 -- 172.21.15.17:0/954730406 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f188c010860 con 0x7f18980c6c30 2024-09-10T21:48:10.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:10.632+0000 7f1894c00640 1 --2- 172.21.15.17:0/954730406 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f18780725e0 0x7f1878074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:48:10.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:10.632+0000 7f1894c00640 1 -- 172.21.15.17:0/954730406 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 6383+0+0 (secure 0 0 0) 0x7f188c093ae0 con 0x7f18980c6c30 2024-09-10T21:48:10.636 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:10.632+0000 7f1897e00640 1 --2- 172.21.15.17:0/954730406 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f18780725e0 0x7f1878074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:48:10.637 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:10.632+0000 7f1897e00640 1 --2- 172.21.15.17:0/954730406 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f18780725e0 0x7f1878074aa0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f18840b2000 tx=0x7f188401be30 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:48:10.640 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:10.636+0000 7f1894c00640 1 -- 172.21.15.17:0/954730406 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f188c05edf0 con 0x7f18980c6c30 2024-09-10T21:48:10.819 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:10.812+0000 7f189fa00640 1 -- 172.21.15.17:0/954730406 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f18980c87e0 con 0x7f18980c6c30 2024-09-10T21:48:11.462 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:11.456+0000 7f1894c00640 1 -- 172.21.15.17:0/954730406 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v30) ==== 169+0+0 (secure 0 0 0) 0x7f188c009e20 con 0x7f18980c6c30 2024-09-10T21:48:11.463 INFO:teuthology.orchestra.run.smithi017.stderr:pool 'cephfs_data' removed 2024-09-10T21:48:11.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:11.460+0000 7f189fa00640 1 -- 172.21.15.17:0/954730406 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f18780725e0 msgr2=0x7f1878074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:48:11.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:11.460+0000 7f189fa00640 1 --2- 172.21.15.17:0/954730406 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f18780725e0 0x7f1878074aa0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f18840b2000 tx=0x7f188401be30 comp rx=0 tx=0).stop 2024-09-10T21:48:11.465 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:11.460+0000 7f189fa00640 1 -- 172.21.15.17:0/954730406 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f18980c6c30 msgr2=0x7f18980cb150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:48:11.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:11.460+0000 7f189fa00640 1 --2- 172.21.15.17:0/954730406 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f18980c6c30 0x7f18980cb150 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f188c0058d0 tx=0x7f188c00dde0 comp rx=0 tx=0).stop 2024-09-10T21:48:11.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:11.460+0000 7f189fa00640 1 -- 172.21.15.17:0/954730406 shutdown_connections 2024-09-10T21:48:11.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:11.460+0000 7f189fa00640 1 --2- 172.21.15.17:0/954730406 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f18780725e0 0x7f1878074aa0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:11.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:11.460+0000 7f189fa00640 1 --2- 172.21.15.17:0/954730406 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f1898152750 0x7f18981f39f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:11.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:11.460+0000 7f189fa00640 1 --2- 172.21.15.17:0/954730406 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f18980c7120 0x7f18980cb690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:11.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:11.460+0000 7f189fa00640 1 --2- 172.21.15.17:0/954730406 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f18980c6c30 0x7f18980cb150 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:11.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:11.460+0000 7f189fa00640 1 -- 172.21.15.17:0/954730406 >> 172.21.15.17:0/954730406 conn(0x7f189806cc20 msgr2=0x7f189814bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:48:11.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:11.460+0000 7f189fa00640 1 -- 172.21.15.17:0/954730406 shutdown_connections 2024-09-10T21:48:11.466 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:11.460+0000 7f189fa00640 1 -- 172.21.15.17:0/954730406 wait complete. 2024-09-10T21:48:11.488 DEBUG:teuthology.orchestra.run.smithi017:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-09-10T21:48:11.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:11.644+0000 7f1ceba00640 1 -- 172.21.15.17:0/1475199806 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1ce40cdf50 msgr2=0x7f1ce40ce3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:48:11.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:11.644+0000 7f1ceba00640 1 --2- 172.21.15.17:0/1475199806 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1ce40cdf50 0x7f1ce40ce3c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f1ce40cbbe0 tx=0x7f1cdc01f580 comp rx=0 tx=0).stop 2024-09-10T21:48:11.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:11.644+0000 7f1ceba00640 1 -- 172.21.15.17:0/1475199806 shutdown_connections 2024-09-10T21:48:11.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:11.644+0000 7f1ceba00640 1 --2- 172.21.15.17:0/1475199806 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1ce40cdf50 0x7f1ce40ce3c0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:11.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:11.644+0000 7f1ceba00640 1 --2- 172.21.15.17:0/1475199806 >> v2:172.21.15.31:3300/0 conn(0x7f1ce4152520 0x7f1ce4070dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:11.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:11.644+0000 7f1ceba00640 1 -- 172.21.15.17:0/1475199806 >> 172.21.15.17:0/1475199806 conn(0x7f1ce40748d0 msgr2=0x7f1ce4068990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:48:11.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:11.644+0000 7f1ceba00640 1 -- 172.21.15.17:0/1475199806 shutdown_connections 2024-09-10T21:48:11.649 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:11.644+0000 7f1ceba00640 1 -- 172.21.15.17:0/1475199806 wait complete. 2024-09-10T21:48:11.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:11.644+0000 7f1ceba00640 1 Processor -- start 2024-09-10T21:48:11.650 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:11.644+0000 7f1ceba00640 1 -- start start 2024-09-10T21:48:11.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:11.644+0000 7f1ceba00640 1 --2- >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f1ce4071680 0x7f1ce414a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:48:11.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:11.644+0000 7f1ceba00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1ce40cdf50 0x7f1ce406df30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:48:11.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:11.644+0000 7f1ceba00640 1 --2- >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1ce4152520 0x7f1ce4146930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:48:11.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:11.644+0000 7f1ceba00640 1 -- --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_getmap magic: 0 -- 0x7f1ce4152900 con 0x7f1ce40cdf50 2024-09-10T21:48:11.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:11.644+0000 7f1ceba00640 1 -- --> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] -- mon_getmap magic: 0 -- 0x7f1ce40c3270 con 0x7f1ce4152520 2024-09-10T21:48:11.651 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:11.644+0000 7f1ceba00640 1 -- --> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] -- mon_getmap magic: 0 -- 0x7f1ce4152a80 con 0x7f1ce4071680 2024-09-10T21:48:11.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:11.644+0000 7f1ce3e00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1ce40cdf50 0x7f1ce406df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:48:11.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:11.648+0000 7f1ce3e00640 1 --2- >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1ce40cdf50 0x7f1ce406df30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.17:3300/0 says I am v2:172.21.15.17:48690/0 (socket says 172.21.15.17:48690) 2024-09-10T21:48:11.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:11.648+0000 7f1ce3e00640 1 -- 172.21.15.17:0/340649515 learned_addr learned my addr 172.21.15.17:0/340649515 (peer_addr_for_me v2:172.21.15.17:0/0) 2024-09-10T21:48:11.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:11.648+0000 7f1ce9a00640 1 --2- 172.21.15.17:0/340649515 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1ce4152520 0x7f1ce4146930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:48:11.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:11.648+0000 7f1ce9000640 1 --2- 172.21.15.17:0/340649515 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f1ce4071680 0x7f1ce414a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:48:11.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:11.648+0000 7f1ce3e00640 1 -- 172.21.15.17:0/340649515 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f1ce4071680 msgr2=0x7f1ce414a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:48:11.653 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:11.648+0000 7f1ce3e00640 1 --2- 172.21.15.17:0/340649515 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f1ce4071680 0x7f1ce414a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:11.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:11.648+0000 7f1ce3e00640 1 -- 172.21.15.17:0/340649515 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1ce4152520 msgr2=0x7f1ce4146930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:48:11.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:11.648+0000 7f1ce3e00640 1 --2- 172.21.15.17:0/340649515 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1ce4152520 0x7f1ce4146930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:11.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:11.648+0000 7f1ce3e00640 1 -- 172.21.15.17:0/340649515 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ce40cdd10 con 0x7f1ce40cdf50 2024-09-10T21:48:11.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:11.648+0000 7f1ce9000640 1 --2- 172.21.15.17:0/340649515 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f1ce4071680 0x7f1ce414a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T21:48:11.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:11.648+0000 7f1ce3e00640 1 --2- 172.21.15.17:0/340649515 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1ce40cdf50 0x7f1ce406df30 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f1cd000c400 tx=0x7f1cd000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:48:11.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:11.648+0000 7f1ce0c00640 1 -- 172.21.15.17:0/340649515 <== mon.0 v2:172.21.15.17:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cd0017070 con 0x7f1ce40cdf50 2024-09-10T21:48:11.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:11.648+0000 7f1ce0c00640 1 -- 172.21.15.17:0/340649515 <== mon.0 v2:172.21.15.17:3300/0 2 ==== config(6 keys) ==== 172+0+0 (secure 0 0 0) 0x7f1cd00052f0 con 0x7f1ce40cdf50 2024-09-10T21:48:11.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:11.648+0000 7f1ceba00640 1 -- 172.21.15.17:0/340649515 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ce4072ce0 con 0x7f1ce40cdf50 2024-09-10T21:48:11.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:11.648+0000 7f1ce0c00640 1 -- 172.21.15.17:0/340649515 <== mon.0 v2:172.21.15.17:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cd0003880 con 0x7f1ce40cdf50 2024-09-10T21:48:11.654 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:11.648+0000 7f1ceba00640 1 -- 172.21.15.17:0/340649515 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ce406fc70 con 0x7f1ce40cdf50 2024-09-10T21:48:11.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:11.648+0000 7f1ceba00640 1 -- 172.21.15.17:0/340649515 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ce414b3a0 con 0x7f1ce40cdf50 2024-09-10T21:48:11.655 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:11.648+0000 7f1ce0c00640 1 -- 172.21.15.17:0/340649515 <== mon.0 v2:172.21.15.17:3300/0 4 ==== mgrmap(e 5) ==== 96394+0+0 (secure 0 0 0) 0x7f1cd0003a20 con 0x7f1ce40cdf50 2024-09-10T21:48:11.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:11.652+0000 7f1ce0c00640 1 --2- 172.21.15.17:0/340649515 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f1cc40725e0 0x7f1cc4074aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T21:48:11.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:11.652+0000 7f1ce0c00640 1 -- 172.21.15.17:0/340649515 <== mon.0 v2:172.21.15.17:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 5853+0+0 (secure 0 0 0) 0x7f1cd0092900 con 0x7f1ce40cdf50 2024-09-10T21:48:11.656 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:11.652+0000 7f1ce9000640 1 --2- 172.21.15.17:0/340649515 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f1cc40725e0 0x7f1cc4074aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T21:48:11.657 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:11.652+0000 7f1ce9000640 1 --2- 172.21.15.17:0/340649515 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f1cc40725e0 0x7f1cc4074aa0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f1cd80058d0 tx=0x7f1cd8005860 comp rx=0 tx=0).ready entity=mgr.4100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T21:48:11.660 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:11.656+0000 7f1ce0c00640 1 -- 172.21.15.17:0/340649515 <== mon.0 v2:172.21.15.17:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+187845 (secure 0 0 0) 0x7f1cd005d8f0 con 0x7f1ce40cdf50 2024-09-10T21:48:11.837 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:11.832+0000 7f1ceba00640 1 -- 172.21.15.17:0/340649515 --> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) -- 0x7f1ce4159810 con 0x7f1ce40cdf50 2024-09-10T21:48:12.468 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:12.464+0000 7f1ce0c00640 1 -- 172.21.15.17:0/340649515 <== mon.0 v2:172.21.15.17:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v31) ==== 178+0+0 (secure 0 0 0) 0x7f1cd0062320 con 0x7f1ce40cdf50 2024-09-10T21:48:12.468 INFO:teuthology.orchestra.run.smithi017.stderr:pool 'cephfs_data_ec' removed 2024-09-10T21:48:12.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:12.464+0000 7f1ceba00640 1 -- 172.21.15.17:0/340649515 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f1cc40725e0 msgr2=0x7f1cc4074aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:48:12.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:12.464+0000 7f1ceba00640 1 --2- 172.21.15.17:0/340649515 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f1cc40725e0 0x7f1cc4074aa0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f1cd80058d0 tx=0x7f1cd8005860 comp rx=0 tx=0).stop 2024-09-10T21:48:12.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:12.464+0000 7f1ceba00640 1 -- 172.21.15.17:0/340649515 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1ce40cdf50 msgr2=0x7f1ce406df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T21:48:12.470 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:12.464+0000 7f1ceba00640 1 --2- 172.21.15.17:0/340649515 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1ce40cdf50 0x7f1ce406df30 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f1cd000c400 tx=0x7f1cd000c8c0 comp rx=0 tx=0).stop 2024-09-10T21:48:12.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:12.464+0000 7f1ceba00640 1 -- 172.21.15.17:0/340649515 shutdown_connections 2024-09-10T21:48:12.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:12.464+0000 7f1ceba00640 1 --2- 172.21.15.17:0/340649515 >> [v2:172.21.15.31:6808/3101308586,v1:172.21.15.31:6809/3101308586] conn(0x7f1cc40725e0 0x7f1cc4074aa0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:12.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:12.464+0000 7f1ceba00640 1 --2- 172.21.15.17:0/340649515 >> [v2:172.21.15.31:3300/0,v1:172.21.15.31:6789/0] conn(0x7f1ce4152520 0x7f1ce4146930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:12.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:12.464+0000 7f1ceba00640 1 --2- 172.21.15.17:0/340649515 >> [v2:172.21.15.17:3300/0,v1:172.21.15.17:6789/0] conn(0x7f1ce40cdf50 0x7f1ce406df30 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:12.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:12.464+0000 7f1ceba00640 1 --2- 172.21.15.17:0/340649515 >> [v2:172.21.15.31:3301/0,v1:172.21.15.31:6790/0] conn(0x7f1ce4071680 0x7f1ce414a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T21:48:12.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:12.464+0000 7f1ceba00640 1 -- 172.21.15.17:0/340649515 >> 172.21.15.17:0/340649515 conn(0x7f1ce40748d0 msgr2=0x7f1ce414e9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T21:48:12.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:12.464+0000 7f1ceba00640 1 -- 172.21.15.17:0/340649515 shutdown_connections 2024-09-10T21:48:12.471 INFO:teuthology.orchestra.run.smithi017.stderr:2024-09-10T21:48:12.464+0000 7f1ceba00640 1 -- 172.21.15.17:0/340649515 wait complete. 2024-09-10T21:48:12.533 INFO:teuthology.misc:Shutting down mds daemons... 2024-09-10T21:48:12.534 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-10T21:48:12.534 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:48:12.562 INFO:tasks.ceph.mds.a:Stopped 2024-09-10T21:48:12.562 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-10T21:48:12.562 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:48:12.630 INFO:tasks.ceph.mds.c:Stopped 2024-09-10T21:48:12.630 DEBUG:tasks.ceph.mds.e:waiting for process to exit 2024-09-10T21:48:12.630 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:48:18.379 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.e is failed for ~0s 2024-09-10T21:48:18.732 INFO:tasks.ceph.mds.e:Stopped 2024-09-10T21:48:18.732 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-10T21:48:18.732 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:48:18.749 INFO:tasks.ceph.mds.b:Stopped 2024-09-10T21:48:18.749 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-10T21:48:18.750 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:48:18.812 INFO:tasks.ceph.mds.d:Stopped 2024-09-10T21:48:18.812 DEBUG:tasks.ceph.mds.f:waiting for process to exit 2024-09-10T21:48:18.812 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:48:18.874 INFO:tasks.ceph.mds.f:Stopped 2024-09-10T21:48:18.874 INFO:teuthology.misc:Shutting down osd daemons... 2024-09-10T21:48:18.874 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-10T21:48:18.875 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:48:24.588 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~0s 2024-09-10T21:48:24.589 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.e has been restored 2024-09-10T21:48:24.976 INFO:tasks.ceph.osd.0:Stopped 2024-09-10T21:48:24.977 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-10T21:48:24.977 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:48:30.698 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~0s 2024-09-10T21:48:30.698 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 has been restored 2024-09-10T21:48:31.080 INFO:tasks.ceph.osd.1:Stopped 2024-09-10T21:48:31.080 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-10T21:48:31.081 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:48:36.706 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~0s 2024-09-10T21:48:36.707 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 has been restored 2024-09-10T21:48:37.183 INFO:tasks.ceph.osd.2:Stopped 2024-09-10T21:48:37.183 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-09-10T21:48:37.183 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:48:42.613 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~0s 2024-09-10T21:48:42.614 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 has been restored 2024-09-10T21:48:43.285 INFO:tasks.ceph.osd.3:Stopped 2024-09-10T21:48:43.285 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-09-10T21:48:43.286 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:48:48.420 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~0s 2024-09-10T21:48:48.421 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 has been restored 2024-09-10T21:48:49.388 INFO:tasks.ceph.osd.4:Stopped 2024-09-10T21:48:49.388 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-09-10T21:48:49.388 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:48:54.129 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~0s 2024-09-10T21:48:54.129 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 has been restored 2024-09-10T21:48:55.490 INFO:tasks.ceph.osd.5:Stopped 2024-09-10T21:48:55.490 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-09-10T21:48:55.491 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:48:59.734 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~0s 2024-09-10T21:48:59.735 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 has been restored 2024-09-10T21:49:01.592 INFO:tasks.ceph.osd.6:Stopped 2024-09-10T21:49:01.593 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-09-10T21:49:01.593 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:49:05.239 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~0s 2024-09-10T21:49:05.239 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 has been restored 2024-09-10T21:49:07.694 INFO:tasks.ceph.osd.7:Stopped 2024-09-10T21:49:07.695 INFO:teuthology.misc:Shutting down mgr daemons... 2024-09-10T21:49:07.696 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-09-10T21:49:07.696 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:49:07.754 INFO:tasks.ceph.mgr.y:Stopped 2024-09-10T21:49:07.754 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-09-10T21:49:07.754 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:49:07.819 INFO:tasks.ceph.mgr.x:Stopped 2024-09-10T21:49:07.820 INFO:teuthology.misc:Shutting down mon daemons... 2024-09-10T21:49:07.820 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-10T21:49:07.820 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:49:07.838 INFO:tasks.ceph.mon.a:Stopped 2024-09-10T21:49:07.838 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-10T21:49:07.839 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:49:07.855 INFO:tasks.ceph.mon.b:Stopped 2024-09-10T21:49:07.855 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-10T21:49:07.856 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T21:49:07.919 INFO:tasks.ceph.mon.c:Stopped 2024-09-10T21:49:07.919 INFO:tasks.ceph:Checking cluster log for badness... 2024-09-10T21:49:07.920 DEBUG:teuthology.orchestra.run.smithi017:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v 'fs.*is degraded' | egrep -v 'filesystem is degraded' | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v 'filesystem is offline' | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v 'insufficient standby MDS daemons available' | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'overall HEALTH_' | egrep -v '\(OSDMAP_FLAGS\)' | egrep -v '\(OSD_' | egrep -v '\(PG_' | egrep -v '\(POOL_' | egrep -v '\(CACHE_POOL_' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(OBJECT_' | egrep -v SLOW_OPS | egrep -v '\(REQUEST_SLOW\)' | egrep -v '\(TOO_FEW_PGS\)' | egrep -v 'slow request' | egrep -v 'timeout on replica' | egrep -v 'late reservation from' | egrep -v MON_DOWN | egrep -v OSDMAP_FLAGS | egrep -v OSD_DOWN | egrep -v PG_DEGRADED | egrep -v PG_AVAILABILITY | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'mons down' | egrep -v 'mon down' | egrep -v 'out of quorum' | egrep -v noscrub | egrep -v nodeep-scrub | egrep -v 'overall HEALTH_' | egrep -v '\(MON_DOWN\)' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-09-10T21:49:07.999 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi017.front.sepia.ceph.com 2024-09-10T21:49:08.000 DEBUG:teuthology.orchestra.run.smithi017:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-09-10T21:49:09.134 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi017.front.sepia.ceph.com 2024-09-10T21:49:09.134 DEBUG:teuthology.orchestra.run.smithi017:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-09-10T21:49:09.378 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi017.front.sepia.ceph.com 2024-09-10T21:49:09.378 DEBUG:teuthology.orchestra.run.smithi017:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-09-10T21:49:09.649 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi017.front.sepia.ceph.com 2024-09-10T21:49:09.649 DEBUG:teuthology.orchestra.run.smithi017:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-09-10T21:49:09.890 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi031.front.sepia.ceph.com 2024-09-10T21:49:09.890 DEBUG:teuthology.orchestra.run.smithi031:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-09-10T21:49:10.161 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi031.front.sepia.ceph.com 2024-09-10T21:49:10.161 DEBUG:teuthology.orchestra.run.smithi031:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-09-10T21:49:10.240 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 has been restored 2024-09-10T21:49:10.344 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi031.front.sepia.ceph.com 2024-09-10T21:49:10.345 DEBUG:teuthology.orchestra.run.smithi031:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-09-10T21:49:10.657 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi031.front.sepia.ceph.com 2024-09-10T21:49:10.658 DEBUG:teuthology.orchestra.run.smithi031:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-09-10T21:49:10.990 INFO:tasks.ceph:Archiving mon data... 2024-09-10T21:49:10.991 DEBUG:teuthology.misc:Transferring archived files from smithi017:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899674/data/mon.a.tgz 2024-09-10T21:49:10.991 DEBUG:teuthology.orchestra.run.smithi017:> mktemp 2024-09-10T21:49:11.006 INFO:teuthology.orchestra.run.smithi017.stdout:/tmp/tmp.pH1I9AQwkA 2024-09-10T21:49:11.006 DEBUG:teuthology.orchestra.run.smithi017:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.pH1I9AQwkA 2024-09-10T21:49:11.179 DEBUG:teuthology.orchestra.run.smithi017:> sudo chmod 0666 /tmp/tmp.pH1I9AQwkA 2024-09-10T21:49:11.271 DEBUG:teuthology.orchestra.remote:smithi017:/tmp/tmp.pH1I9AQwkA is 486KB 2024-09-10T21:49:11.333 DEBUG:teuthology.orchestra.run.smithi017:> rm -fr /tmp/tmp.pH1I9AQwkA 2024-09-10T21:49:11.341 DEBUG:teuthology.misc:Transferring archived files from smithi031:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899674/data/mon.b.tgz 2024-09-10T21:49:11.341 DEBUG:teuthology.orchestra.run.smithi031:> mktemp 2024-09-10T21:49:11.356 INFO:teuthology.orchestra.run.smithi031.stdout:/tmp/tmp.spz8vxeQSt 2024-09-10T21:49:11.356 DEBUG:teuthology.orchestra.run.smithi031:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.spz8vxeQSt 2024-09-10T21:49:11.521 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod 0666 /tmp/tmp.spz8vxeQSt 2024-09-10T21:49:11.620 DEBUG:teuthology.orchestra.remote:smithi031:/tmp/tmp.spz8vxeQSt is 490KB 2024-09-10T21:49:11.684 DEBUG:teuthology.orchestra.run.smithi031:> rm -fr /tmp/tmp.spz8vxeQSt 2024-09-10T21:49:11.693 DEBUG:teuthology.misc:Transferring archived files from smithi031:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899674/data/mon.c.tgz 2024-09-10T21:49:11.693 DEBUG:teuthology.orchestra.run.smithi031:> mktemp 2024-09-10T21:49:11.745 INFO:teuthology.orchestra.run.smithi031.stdout:/tmp/tmp.KBEDXCZndH 2024-09-10T21:49:11.745 DEBUG:teuthology.orchestra.run.smithi031:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.KBEDXCZndH 2024-09-10T21:49:11.893 DEBUG:teuthology.orchestra.run.smithi031:> sudo chmod 0666 /tmp/tmp.KBEDXCZndH 2024-09-10T21:49:11.974 DEBUG:teuthology.orchestra.remote:smithi031:/tmp/tmp.KBEDXCZndH is 489KB 2024-09-10T21:49:12.036 DEBUG:teuthology.orchestra.run.smithi031:> rm -fr /tmp/tmp.KBEDXCZndH 2024-09-10T21:49:12.044 INFO:tasks.ceph:Cleaning ceph cluster... 2024-09-10T21:49:12.044 DEBUG:teuthology.orchestra.run.smithi017:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-10T21:49:12.047 DEBUG:teuthology.orchestra.run.smithi031:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-10T21:49:12.179 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-10T21:49:12.196 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-10T21:49:12.196 INFO:tasks.ceph:Archiving crash dumps... 2024-09-10T21:49:12.198 DEBUG:teuthology.misc:Transferring archived files from smithi017:/var/lib/ceph/crash to /home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899674/remote/smithi017/crash 2024-09-10T21:49:12.199 DEBUG:teuthology.orchestra.run.smithi017:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-10T21:49:12.242 DEBUG:teuthology.misc:Transferring archived files from smithi031:/var/lib/ceph/crash to /home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899674/remote/smithi031/crash 2024-09-10T21:49:12.242 DEBUG:teuthology.orchestra.run.smithi031:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-10T21:49:12.262 INFO:tasks.ceph:Shutting down logrotate 2024-09-10T21:52:10.113 DEBUG:teuthology.orchestra.run.smithi017:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-09-10T21:52:10.133 DEBUG:teuthology.orchestra.run.smithi031:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-09-10T21:52:10.156 INFO:tasks.ceph:Compressing logs... 2024-09-10T21:52:10.156 DEBUG:teuthology.orchestra.run.smithi017:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-10T21:52:10.178 DEBUG:teuthology.orchestra.run.smithi031:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-10T21:52:10.205 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40358.log 2024-09-10T21:52:10.206 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40803.log 2024-09-10T21:52:10.206 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.40358.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35240.log 2024-09-10T21:52:10.207 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.40803.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40410.log 2024-09-10T21:52:10.208 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.35240.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42275.log 2024-09-10T21:52:10.208 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.40410.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43609.log 2024-09-10T21:52:10.209 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.42275.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34385.log 2024-09-10T21:52:10.210 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.43609.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34123.log 2024-09-10T21:52:10.211 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.34385.log: 89.3% 88.2% -- replaced with /var/log/ceph/ceph-client.admin.40358.log.gz -- replaced with /var/log/ceph/ceph-client.admin.40410.log.gz 2024-09-10T21:52:10.211 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-10T21:52:10.211 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74065.log 2024-09-10T21:52:10.211 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.34123.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29302.log 2024-09-10T21:52:10.212 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.74065.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39220.log 2024-09-10T21:52:10.213 INFO:teuthology.orchestra.run.smithi017.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.74065.log.gz 2024-09-10T21:52:10.213 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.29302.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28892.log 2024-09-10T21:52:10.213 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.39220.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40144.log 2024-09-10T21:52:10.214 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.28892.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62503.log 2024-09-10T21:52:10.215 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.40144.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40881.log 2024-09-10T21:52:10.216 INFO:teuthology.orchestra.run.smithi017.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.42275.log.gz 2024-09-10T21:52:10.216 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.62503.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28911.log 2024-09-10T21:52:10.216 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.40881.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42778.log 2024-09-10T21:52:10.217 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.28911.log: 86.0% -- replaced with /var/log/ceph/ceph-client.admin.28892.log.gz 2024-09-10T21:52:10.217 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-09-10T21:52:10.218 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.42778.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40986.log 2024-09-10T21:52:10.218 INFO:teuthology.orchestra.run.smithi017.stderr: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.29302.log.gz 2024-09-10T21:52:10.218 INFO:teuthology.orchestra.run.smithi017.stderr: 86.3% -- replaced with /var/log/ceph/ceph-client.admin.28911.log.gz 2024-09-10T21:52:10.219 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39877.log 2024-09-10T21:52:10.219 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.40986.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40012.log 2024-09-10T21:52:10.220 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.39877.log: 89.4%gzip -- replaced with /var/log/ceph/ceph-client.admin.40144.log.gz -5 --verbose 2024-09-10T21:52:10.220 INFO:teuthology.orchestra.run.smithi017.stderr: -- /var/log/ceph/ceph-client.admin.32869.log 2024-09-10T21:52:10.220 INFO:teuthology.orchestra.run.smithi017.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.40986.log.gz 2024-09-10T21:52:10.221 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72837.log 2024-09-10T21:52:10.222 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.40012.log: /var/log/ceph/ceph-client.admin.32869.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34367.log 2024-09-10T21:52:10.222 INFO:teuthology.orchestra.run.smithi017.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.40012.log.gz 2024-09-10T21:52:10.222 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.72837.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74531.log 2024-09-10T21:52:10.223 INFO:teuthology.orchestra.run.smithi017.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.62503.log.gz 2024-09-10T21:52:10.223 INFO:teuthology.orchestra.run.smithi017.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.72837.log.gz 2024-09-10T21:52:10.223 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.34367.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35188.log 2024-09-10T21:52:10.224 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.74531.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41846.log 2024-09-10T21:52:10.224 INFO:teuthology.orchestra.run.smithi017.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43609.log.gz 2024-09-10T21:52:10.224 INFO:teuthology.orchestra.run.smithi017.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.74531.log.gz 2024-09-10T21:52:10.225 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.35188.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42249.log 2024-09-10T21:52:10.226 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.41846.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70641.log 2024-09-10T21:52:10.226 INFO:teuthology.orchestra.run.smithi017.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.32869.log.gz 2024-09-10T21:52:10.227 INFO:teuthology.orchestra.run.smithi017.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.41846.log.gz 2024-09-10T21:52:10.227 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.42249.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49314.log 2024-09-10T21:52:10.227 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.70641.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53706.log 2024-09-10T21:52:10.228 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5/var/log/ceph/ceph-client.admin.49314.log: --verbose -- /var/log/ceph/ceph-client.admin.42511.log 2024-09-10T21:52:10.228 INFO:teuthology.orchestra.run.smithi017.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.42249.log.gz 2024-09-10T21:52:10.229 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28918.log 2024-09-10T21:52:10.230 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.53706.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.49314.log.gz 2024-09-10T21:52:10.230 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78907.log 2024-09-10T21:52:10.230 INFO:teuthology.orchestra.run.smithi017.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.39877.log.gz 2024-09-10T21:52:10.230 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.42511.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.40881.log.gz 2024-09-10T21:52:10.230 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43978.log 2024-09-10T21:52:10.232 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-09-10T21:52:10.232 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.28918.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-09-10T21:52:10.232 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28949.log 2024-09-10T21:52:10.232 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28997.log 2024-09-10T21:52:10.233 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.78907.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55981.log 2024-09-10T21:52:10.233 INFO:teuthology.orchestra.run.smithi017.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.78907.log.gz 2024-09-10T21:52:10.233 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.43978.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54303.log 2024-09-10T21:52:10.233 INFO:teuthology.orchestra.run.smithi017.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.70641.log.gz 2024-09-10T21:52:10.233 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.55981.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38939.log 2024-09-10T21:52:10.234 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.28949.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29021.log 2024-09-10T21:52:10.235 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.28997.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-09-10T21:52:10.235 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.29021.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-09-10T21:52:10.236 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.54303.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39246.log 2024-09-10T21:52:10.236 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42645.log 2024-09-10T21:52:10.236 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.38939.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.53706.log.gz 2024-09-10T21:52:10.236 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.39246.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-09-10T21:52:10.236 INFO:teuthology.orchestra.run.smithi017.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.35240.log.gz 2024-09-10T21:52:10.236 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.42645.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.43978.log.gz 2024-09-10T21:52:10.236 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35794.log 2024-09-10T21:52:10.237 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44031.log 2024-09-10T21:52:10.240 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.f.log 2024-09-10T21:52:10.240 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-mgr.x.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.28918.log.gz 2024-09-10T21:52:10.240 INFO:teuthology.orchestra.run.smithi031.stderr:gzip -5 86.5% --verbose -- replaced with /var/log/ceph/ceph-client.admin.28949.log.gz -- 2024-09-10T21:52:10.240 INFO:teuthology.orchestra.run.smithi031.stderr: /var/log/ceph/ceph.log 2024-09-10T21:52:10.240 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28870.log 2024-09-10T21:52:10.240 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28846.log 2024-09-10T21:52:10.240 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.28870.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-09-10T21:52:10.240 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.28846.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-10T21:52:10.241 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.35794.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47602.log 2024-09-10T21:52:10.241 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.44031.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39849.log 2024-09-10T21:52:10.241 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.47602.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.55981.log.gz 2024-09-10T21:52:10.241 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60955.log 2024-09-10T21:52:10.242 INFO:teuthology.orchestra.run.smithi017.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.44031.log.gz 2024-09-10T21:52:10.242 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.39849.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56724.log 2024-09-10T21:52:10.242 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.60955.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41191.log 2024-09-10T21:52:10.242 INFO:teuthology.orchestra.run.smithi017.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.38939.log.gz 2024-09-10T21:52:10.242 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.56724.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.47602.log.gz 2024-09-10T21:52:10.242 INFO:teuthology.orchestra.run.smithi017.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.39849.log.gz 2024-09-10T21:52:10.242 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79245.log 2024-09-10T21:52:10.242 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.41191.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34877.log 2024-09-10T21:52:10.245 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-09-10T21:52:10.246 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-09-10T21:52:10.246 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-09-10T21:52:10.246 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28973.log 2024-09-10T21:52:10.246 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-09-10T21:52:10.246 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-client.admin.28973.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28894.log 2024-09-10T21:52:10.246 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.79245.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40725.log 2024-09-10T21:52:10.247 INFO:teuthology.orchestra.run.smithi017.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.79245.log.gz 2024-09-10T21:52:10.247 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.34877.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33141.log 2024-09-10T21:52:10.247 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.40725.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.35188.log.gz 2024-09-10T21:52:10.247 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50209.log 2024-09-10T21:52:10.247 INFO:teuthology.orchestra.run.smithi017.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.34877.log.gz 2024-09-10T21:52:10.247 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.33141.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33115.log 2024-09-10T21:52:10.247 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.50209.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40855.log 2024-09-10T21:52:10.247 INFO:teuthology.orchestra.run.smithi017.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.50209.log.gz 2024-09-10T21:52:10.247 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.33115.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64356.log 2024-09-10T21:52:10.247 INFO:teuthology.orchestra.run.smithi017.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.41191.log.gz 2024-09-10T21:52:10.247 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.40855.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39376.log 2024-09-10T21:52:10.248 INFO:teuthology.orchestra.run.smithi031.stderr:/var/log/ceph/ceph-osd.4.log: /var/log/ceph/ceph-client.admin.28894.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.28973.log.gz 2024-09-10T21:52:10.249 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.64356.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35485.log 2024-09-10T21:52:10.249 INFO:teuthology.orchestra.run.smithi017.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.64356.log.gz 2024-09-10T21:52:10.249 INFO:teuthology.orchestra.run.smithi017.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.40855.log.gz 2024-09-10T21:52:10.249 INFO:teuthology.orchestra.run.smithi017.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.54303.log.gz 2024-09-10T21:52:10.250 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.39376.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34565.log 2024-09-10T21:52:10.250 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.35485.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36521.log 2024-09-10T21:52:10.250 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.34565.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44057.log 2024-09-10T21:52:10.250 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.36521.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33513.log 2024-09-10T21:52:10.251 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.44057.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28845.log 2024-09-10T21:52:10.251 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.33513.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36548.log 2024-09-10T21:52:10.252 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.28845.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39168.log 2024-09-10T21:52:10.253 INFO:teuthology.orchestra.run.smithi017.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.39220.log.gz 2024-09-10T21:52:10.253 INFO:teuthology.orchestra.run.smithi017.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.39246.log.gz 2024-09-10T21:52:10.253 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.36548.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41688.log 2024-09-10T21:52:10.254 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.39168.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38833.log 2024-09-10T21:52:10.254 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.41688.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79116.log 2024-09-10T21:52:10.255 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.38833.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.60955.log.gz 2024-09-10T21:52:10.255 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33461.log 2024-09-10T21:52:10.256 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.79116.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79011.log 2024-09-10T21:52:10.256 INFO:teuthology.orchestra.run.smithi017.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.79116.log.gz 2024-09-10T21:52:10.257 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.33461.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34593.log 2024-09-10T21:52:10.257 INFO:teuthology.orchestra.run.smithi017.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.38833.log.gz 2024-09-10T21:52:10.257 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.79011.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36062.log 2024-09-10T21:52:10.258 INFO:teuthology.orchestra.run.smithi017.stderr: 86.2% -- replaced with /var/log/ceph/ceph-client.admin.28845.log.gz 2024-09-10T21:52:10.258 INFO:teuthology.orchestra.run.smithi017.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.79011.log.gz 2024-09-10T21:52:10.258 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.34593.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61053.log 2024-09-10T21:52:10.259 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.36062.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42431.log 2024-09-10T21:52:10.260 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.61053.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34825.log 2024-09-10T21:52:10.260 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.42431.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42700.log 2024-09-10T21:52:10.261 INFO:teuthology.orchestra.run.smithi017.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.35485.log.gz 2024-09-10T21:52:10.261 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.34825.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67218.log 2024-09-10T21:52:10.262 INFO:teuthology.orchestra.run.smithi017.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.36521.log.gz 2024-09-10T21:52:10.262 INFO:teuthology.orchestra.run.smithi017.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.36548.log.gz 2024-09-10T21:52:10.262 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.42700.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43248.log 2024-09-10T21:52:10.262 INFO:teuthology.orchestra.run.smithi017.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.39376.log.gz 2024-09-10T21:52:10.263 INFO:teuthology.orchestra.run.smithi017.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.40725.log.gz 2024-09-10T21:52:10.263 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.67218.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.41688.log.gz 2024-09-10T21:52:10.263 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28924.log 2024-09-10T21:52:10.263 INFO:teuthology.orchestra.run.smithi017.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.42511.log.gz 2024-09-10T21:52:10.263 INFO:teuthology.orchestra.run.smithi017.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.67218.log.gz 2024-09-10T21:52:10.264 INFO:teuthology.orchestra.run.smithi017.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.44057.log.gz 2024-09-10T21:52:10.265 INFO:teuthology.orchestra.run.smithi017.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.61053.log.gz 2024-09-10T21:52:10.265 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.43248.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34172.log 2024-09-10T21:52:10.265 INFO:teuthology.orchestra.run.smithi017.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.34825.log.gz 89.1% 2024-09-10T21:52:10.265 INFO:teuthology.orchestra.run.smithi017.stderr: -- replaced with /var/log/ceph/ceph-client.admin.42431.log.gz 2024-09-10T21:52:10.265 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.28924.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41979.log 2024-09-10T21:52:10.266 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.34172.log: 86.3% -- replaced with /var/log/ceph/ceph-client.admin.28924.log.gz 2024-09-10T21:52:10.266 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42485.log 2024-09-10T21:52:10.266 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.41979.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43844.log 2024-09-10T21:52:10.266 INFO:teuthology.orchestra.run.smithi017.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.40803.log.gz 2024-09-10T21:52:10.267 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.42485.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.42700.log.gz 2024-09-10T21:52:10.267 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.23865.log 2024-09-10T21:52:10.267 INFO:teuthology.orchestra.run.smithi017.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42778.log.gz 2024-09-10T21:52:10.268 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.43844.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38762.log 2024-09-10T21:52:10.269 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph.tmp-client.admin.23865.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34171.log 2024-09-10T21:52:10.269 INFO:teuthology.orchestra.run.smithi017.stderr: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.23865.log.gz 2024-09-10T21:52:10.269 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.38762.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78418.log 2024-09-10T21:52:10.270 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.34171.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28962.log 2024-09-10T21:52:10.271 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.78418.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42834.log 2024-09-10T21:52:10.271 INFO:teuthology.orchestra.run.smithi017.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.78418.log.gz 2024-09-10T21:52:10.271 INFO:teuthology.orchestra.run.smithi017.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.35794.log.gz 2024-09-10T21:52:10.272 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.28962.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43352.log 2024-09-10T21:52:10.272 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.42834.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38885.log 2024-09-10T21:52:10.273 INFO:teuthology.orchestra.run.smithi017.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.39168.log.gz 2024-09-10T21:52:10.273 INFO:teuthology.orchestra.run.smithi017.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.38762.log.gz 2024-09-10T21:52:10.273 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.43352.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36574.log 2024-09-10T21:52:10.274 INFO:teuthology.orchestra.run.smithi017.stderr: 85.8% -- replaced with /var/log/ceph/ceph-client.admin.28962.log.gz 2024-09-10T21:52:10.274 INFO:teuthology.orchestra.run.smithi017.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.42834.log.gz 2024-09-10T21:52:10.274 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.38885.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29355.log 2024-09-10T21:52:10.275 INFO:teuthology.orchestra.run.smithi017.stderr: 90.4% -- replaced with /var/log/ceph/ceph-client.admin.38885.log.gz 2024-09-10T21:52:10.275 INFO:teuthology.orchestra.run.smithi017.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.43844.log.gz 2024-09-10T21:52:10.275 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.36574.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51445.log 2024-09-10T21:52:10.276 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.29355.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67325.log 2024-09-10T21:52:10.276 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.51445.log: gzip -5 --verbose -- 88.9% /var/log/ceph/ceph-client.admin.40306.log -- replaced with /var/log/ceph/ceph-client.admin.36574.log.gz 2024-09-10T21:52:10.276 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-10T21:52:10.277 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.67325.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33993.log 2024-09-10T21:52:10.278 INFO:teuthology.orchestra.run.smithi017.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.36062.log.gz 2024-09-10T21:52:10.278 INFO:teuthology.orchestra.run.smithi017.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.67325.log.gz 2024-09-10T21:52:10.278 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.40306.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34955.log 2024-09-10T21:52:10.279 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.33993.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34773.log 2024-09-10T21:52:10.279 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.34955.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50692.log 2024-09-10T21:52:10.280 INFO:teuthology.orchestra.run.smithi017.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.42485.log.gz 2024-09-10T21:52:10.280 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.34773.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56058.log 2024-09-10T21:52:10.281 INFO:teuthology.orchestra.run.smithi017.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.42645.log.gz 2024-09-10T21:52:10.281 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.50692.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-09-10T21:52:10.281 INFO:teuthology.orchestra.run.smithi017.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.34955.log.gz 2024-09-10T21:52:10.282 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.56058.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28750.log 2024-09-10T21:52:10.282 INFO:teuthology.orchestra.run.smithi017.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.29355.log.gz 2024-09-10T21:52:10.282 INFO:teuthology.orchestra.run.smithi017.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.50692.log.gz 2024-09-10T21:52:10.283 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33089.log 2024-09-10T21:52:10.283 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.28750.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39745.log 2024-09-10T21:52:10.284 INFO:teuthology.orchestra.run.smithi017.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28750.log.gz 2024-09-10T21:52:10.284 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.33089.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40090.log 2024-09-10T21:52:10.285 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.39745.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35016.log 2024-09-10T21:52:10.285 INFO:teuthology.orchestra.run.smithi017.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.40306.log.gz 2024-09-10T21:52:10.286 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.40090.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33435.log 2024-09-10T21:52:10.286 INFO:teuthology.orchestra.run.smithi017.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.43248.log.gz 2024-09-10T21:52:10.286 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.35016.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43352.log.gz 2024-09-10T21:52:10.287 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33305.log 2024-09-10T21:52:10.287 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.33435.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33251.log 2024-09-10T21:52:10.288 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.33305.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.56058.log.gz 2024-09-10T21:52:10.288 INFO:teuthology.orchestra.run.smithi017.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.56724.log.gz 2024-09-10T21:52:10.288 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44267.log 2024-09-10T21:52:10.289 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.33251.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40751.log 2024-09-10T21:52:10.289 INFO:teuthology.orchestra.run.smithi017.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.39745.log.gz 2024-09-10T21:52:10.289 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.44267.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44241.log 2024-09-10T21:52:10.290 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.40751.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.40090.log.gz 2024-09-10T21:52:10.290 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33199.log 2024-09-10T21:52:10.291 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.44241.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.41979.log.gz 2024-09-10T21:52:10.291 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28946.log 2024-09-10T21:52:10.291 INFO:teuthology.orchestra.run.smithi017.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.44267.log.gz 2024-09-10T21:52:10.292 INFO:teuthology.orchestra.run.smithi017.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.40751.log.gz 2024-09-10T21:52:10.292 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.33199.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43488.log 2024-09-10T21:52:10.292 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.28946.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74613.log 2024-09-10T21:52:10.293 INFO:teuthology.orchestra.run.smithi017.stderr: 89.2%/var/log/ceph/ceph-client.admin.43488.log: -- replaced with /var/log/ceph/ceph-client.admin.44241.log.gz 2024-09-10T21:52:10.293 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33225.log 2024-09-10T21:52:10.294 INFO:teuthology.orchestra.run.smithi017.stderr: 86.1% -- replaced with /var/log/ceph/ceph-client.admin.28946.log.gz 2024-09-10T21:52:10.294 INFO:teuthology.orchestra.run.smithi017.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.35016.log.gz 2024-09-10T21:52:10.294 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.74613.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.43488.log.gz 2024-09-10T21:52:10.294 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79349.log 2024-09-10T21:52:10.294 INFO:teuthology.orchestra.run.smithi017.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.74613.log.gz 2024-09-10T21:52:10.295 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.33225.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44293.log 2024-09-10T21:52:10.296 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.79349.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78340.log 2024-09-10T21:52:10.296 INFO:teuthology.orchestra.run.smithi017.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.79349.log.gz 2024-09-10T21:52:10.296 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.44293.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43635.log 2024-09-10T21:52:10.297 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.78340.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68869.log 2024-09-10T21:52:10.298 INFO:teuthology.orchestra.run.smithi017.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.78340.log.gz 2024-09-10T21:52:10.298 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.43635.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.44293.log.gz 2024-09-10T21:52:10.298 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54438.log 2024-09-10T21:52:10.299 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.68869.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41610.log 2024-09-10T21:52:10.299 INFO:teuthology.orchestra.run.smithi017.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.68869.log.gz 2024-09-10T21:52:10.300 INFO:teuthology.orchestra.run.smithi017.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43635.log.gz 2024-09-10T21:52:10.300 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.54438.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-09-10T21:52:10.300 INFO:teuthology.orchestra.run.smithi017.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.34773.log.gz 2024-09-10T21:52:10.300 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.41610.log: gzip 89.2% -- replaced with /var/log/ceph/ceph-client.admin.54438.log.gz -5 2024-09-10T21:52:10.300 INFO:teuthology.orchestra.run.smithi017.stderr: --verbose -- /var/log/ceph/ceph-client.admin.41953.log 2024-09-10T21:52:10.301 INFO:teuthology.orchestra.run.smithi017.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.41610.log.gz 2024-09-10T21:52:10.301 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39934.log 2024-09-10T21:52:10.302 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.41953.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78392.log 2024-09-10T21:52:10.303 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.39934.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.41953.log.gz 2024-09-10T21:52:10.303 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34161.log 2024-09-10T21:52:10.304 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.78392.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.39934.log.gz 2024-09-10T21:52:10.304 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78498.log 2024-09-10T21:52:10.304 INFO:teuthology.orchestra.run.smithi017.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.78392.log.gz 2024-09-10T21:52:10.304 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.34161.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34357.log 2024-09-10T21:52:10.305 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.78498.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41243.log 2024-09-10T21:52:10.305 INFO:teuthology.orchestra.run.smithi017.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.78498.log.gz 2024-09-10T21:52:10.306 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.34357.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78567.log 2024-09-10T21:52:10.306 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.41243.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40907.log 2024-09-10T21:52:10.307 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.78567.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35644.log 2024-09-10T21:52:10.307 INFO:teuthology.orchestra.run.smithi017.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.78567.log.gz 2024-09-10T21:52:10.308 INFO:teuthology.orchestra.run.smithi017.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.41243.log.gz 2024-09-10T21:52:10.308 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.40907.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73255.log 2024-09-10T21:52:10.309 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.35644.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.40907.log.gz 2024-09-10T21:52:10.309 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43661.log 2024-09-10T21:52:10.309 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.73255.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.51445.log.gz 2024-09-10T21:52:10.310 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35405.log 2024-09-10T21:52:10.310 INFO:teuthology.orchestra.run.smithi017.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.73255.log.gz 2024-09-10T21:52:10.310 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.43661.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67079.log 2024-09-10T21:52:10.311 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.35405.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68547.log 2024-09-10T21:52:10.311 INFO:teuthology.orchestra.run.smithi017.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.43661.log.gz 2024-09-10T21:52:10.312 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.67079.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.35405.log.gz 2024-09-10T21:52:10.312 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34162.log 2024-09-10T21:52:10.312 INFO:teuthology.orchestra.run.smithi017.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.67079.log.gz 2024-09-10T21:52:10.313 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.68547.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44163.log 2024-09-10T21:52:10.313 INFO:teuthology.orchestra.run.smithi017.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.68547.log.gz 2024-09-10T21:52:10.314 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.34162.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70727.log 2024-09-10T21:52:10.314 INFO:teuthology.orchestra.run.smithi017.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.35644.log.gz 2024-09-10T21:52:10.314 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.44163.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47344.log 2024-09-10T21:52:10.315 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78933.log 2024-09-10T21:52:10.316 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.70727.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.44163.log.gz 2024-09-10T21:52:10.316 INFO:teuthology.orchestra.run.smithi017.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.70727.log.gz 2024-09-10T21:52:10.316 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.47344.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39771.log 2024-09-10T21:52:10.317 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.78933.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43583.log 2024-09-10T21:52:10.317 INFO:teuthology.orchestra.run.smithi017.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47344.log.gz 2024-09-10T21:52:10.317 INFO:teuthology.orchestra.run.smithi017.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.78933.log.gz 2024-09-10T21:52:10.318 INFO:teuthology.orchestra.run.smithi031.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.28846.log.gz 2024-09-10T21:52:10.318 INFO:teuthology.orchestra.run.smithi031.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.28870.log.gz 2024-09-10T21:52:10.318 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.39771.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58154.log 2024-09-10T21:52:10.319 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.43583.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28949.log 2024-09-10T21:52:10.319 INFO:teuthology.orchestra.run.smithi017.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.39771.log.gz 2024-09-10T21:52:10.319 INFO:teuthology.orchestra.run.smithi017.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.43583.log.gz 2024-09-10T21:52:10.320 INFO:teuthology.orchestra.run.smithi031.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.28894.log.gz 2024-09-10T21:52:10.320 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33003.log 2024-09-10T21:52:10.320 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.58154.log: /var/log/ceph/ceph-client.admin.28949.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34903.log 2024-09-10T21:52:10.321 INFO:teuthology.orchestra.run.smithi031.stderr: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.28997.log.gz 2024-09-10T21:52:10.321 INFO:teuthology.orchestra.run.smithi031.stderr: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.29021.log.gz 2024-09-10T21:52:10.322 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.33003.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38991.log 2024-09-10T21:52:10.322 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.34903.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40196.log 2024-09-10T21:52:10.322 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.38991.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44321.log 2024-09-10T21:52:10.323 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.40196.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39507.log 2024-09-10T21:52:10.324 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.44321.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78619.log 2024-09-10T21:52:10.324 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.39507.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.58154.log.gz 2024-09-10T21:52:10.325 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42806.log 2024-09-10T21:52:10.325 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.78619.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39903.log 2024-09-10T21:52:10.325 INFO:teuthology.orchestra.run.smithi017.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.78619.log.gz 2024-09-10T21:52:10.326 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.42806.log: 86.1% -- replaced with /var/log/ceph/ceph-client.admin.28949.log.gz 2024-09-10T21:52:10.326 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42540.log 2024-09-10T21:52:10.326 INFO:teuthology.orchestra.run.smithi017.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.34903.log.gz 2024-09-10T21:52:10.327 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.39903.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34176.log 2024-09-10T21:52:10.328 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.42540.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.38991.log.gz 2024-09-10T21:52:10.328 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47432.log 2024-09-10T21:52:10.328 INFO:teuthology.orchestra.run.smithi017.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.40196.log.gz 2024-09-10T21:52:10.328 INFO:teuthology.orchestra.run.smithi017.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.44321.log.gz 2024-09-10T21:52:10.329 INFO:teuthology.orchestra.run.smithi017.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.42806.log.gz 2024-09-10T21:52:10.329 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.34176.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40959.log 2024-09-10T21:52:10.330 INFO:teuthology.orchestra.run.smithi017.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.42540.log.gz 2024-09-10T21:52:10.330 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.47432.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43274.log 2024-09-10T21:52:10.331 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.40959.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28948.log 2024-09-10T21:52:10.331 INFO:teuthology.orchestra.run.smithi017.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.39903.log.gz 2024-09-10T21:52:10.331 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.43274.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42752.log 2024-09-10T21:52:10.332 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.28948.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35993.log 2024-09-10T21:52:10.332 INFO:teuthology.orchestra.run.smithi017.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47432.log.gz 2024-09-10T21:52:10.333 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.42752.log: gzip -5 --verbose 87.3% -- -- replaced with /var/log/ceph/ceph-client.admin.40959.log.gz /var/log/ceph/ceph-client.admin.78095.log 2024-09-10T21:52:10.333 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-10T21:52:10.334 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.35993.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44189.log 2024-09-10T21:52:10.334 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.78095.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51506.log 2024-09-10T21:52:10.335 INFO:teuthology.orchestra.run.smithi017.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.78095.log.gz 2024-09-10T21:52:10.335 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.44189.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39535.log 2024-09-10T21:52:10.336 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.51506.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40572.log 2024-09-10T21:52:10.337 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.39535.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39017.log 2024-09-10T21:52:10.337 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35318.log/var/log/ceph/ceph-client.admin.40572.log: 2024-09-10T21:52:10.337 INFO:teuthology.orchestra.run.smithi017.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.43274.log.gz 2024-09-10T21:52:10.338 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.39017.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34165.log 2024-09-10T21:52:10.339 INFO:teuthology.orchestra.run.smithi017.stderr: 86.2% -- replaced with /var/log/ceph/ceph-client.admin.28948.log.gz 2024-09-10T21:52:10.339 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.35318.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66619.log 2024-09-10T21:52:10.340 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.34165.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.44189.log.gz 2024-09-10T21:52:10.340 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63546.log 2024-09-10T21:52:10.340 INFO:teuthology.orchestra.run.smithi017.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.42752.log.gz 2024-09-10T21:52:10.341 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.66619.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42036.log 2024-09-10T21:52:10.341 INFO:teuthology.orchestra.run.smithi017.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.66619.log.gz 2024-09-10T21:52:10.342 INFO:teuthology.orchestra.run.smithi017.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.51506.log.gz 2024-09-10T21:52:10.342 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79167.log 2024-09-10T21:52:10.342 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.63546.log: /var/log/ceph/ceph-client.admin.42036.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.39017.log.gz 2024-09-10T21:52:10.343 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78525.log 2024-09-10T21:52:10.343 INFO:teuthology.orchestra.run.smithi017.stderr: 87.2% 87.6% -- replaced with /var/log/ceph/ceph-client.admin.35318.log.gz 2024-09-10T21:52:10.343 INFO:teuthology.orchestra.run.smithi017.stderr: -- replaced with /var/log/ceph/ceph-client.admin.39535.log.gz 2024-09-10T21:52:10.344 INFO:teuthology.orchestra.run.smithi017.stderr: 87.3% 87.8% -- replaced with /var/log/ceph/ceph-client.admin.63546.log.gz 2024-09-10T21:52:10.344 INFO:teuthology.orchestra.run.smithi017.stderr: -- replaced with /var/log/ceph/ceph-client.admin.42036.log.gz 2024-09-10T21:52:10.344 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.79167.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43300.log 2024-09-10T21:52:10.344 INFO:teuthology.orchestra.run.smithi017.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.40572.log.gz 2024-09-10T21:52:10.344 INFO:teuthology.orchestra.run.smithi017.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.79167.log.gz 2024-09-10T21:52:10.345 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.78525.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44004.log 2024-09-10T21:52:10.345 INFO:teuthology.orchestra.run.smithi017.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.78525.log.gz 2024-09-10T21:52:10.345 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.43300.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77634.log 2024-09-10T21:52:10.346 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.44004.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42912.log 2024-09-10T21:52:10.346 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.77634.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78673.log 2024-09-10T21:52:10.347 INFO:teuthology.orchestra.run.smithi017.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.77634.log.gz 2024-09-10T21:52:10.347 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.42912.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78314.log 2024-09-10T21:52:10.348 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.78673.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62723.log 2024-09-10T21:52:10.348 INFO:teuthology.orchestra.run.smithi017.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.78673.log.gz 2024-09-10T21:52:10.349 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.78314.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.43300.log.gz 2024-09-10T21:52:10.349 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28964.log 2024-09-10T21:52:10.349 INFO:teuthology.orchestra.run.smithi017.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.78314.log.gz 2024-09-10T21:52:10.350 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.62723.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41012.log 2024-09-10T21:52:10.350 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.28964.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41038.log 2024-09-10T21:52:10.351 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.41012.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43037.log 2024-09-10T21:52:10.352 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.41038.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.35141.log 2024-09-10T21:52:10.353 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.43037.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.44004.log.gz 2024-09-10T21:52:10.353 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41531.log 2024-09-10T21:52:10.353 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.0.35141.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75915.log 2024-09-10T21:52:10.353 INFO:teuthology.orchestra.run.smithi017.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.42912.log.gz 2024-09-10T21:52:10.354 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.41531.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78288.log 2024-09-10T21:52:10.355 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.75915.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40384.log 2024-09-10T21:52:10.355 INFO:teuthology.orchestra.run.smithi017.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.75915.log.gz 2024-09-10T21:52:10.356 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.78288.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43065.log 2024-09-10T21:52:10.356 INFO:teuthology.orchestra.run.smithi017.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.78288.log.gz 2024-09-10T21:52:10.356 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.40384.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79063.log 2024-09-10T21:52:10.356 INFO:teuthology.orchestra.run.smithi017.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.35993.log.gz 2024-09-10T21:52:10.357 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.43065.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28762.log 2024-09-10T21:52:10.357 INFO:teuthology.orchestra.run.smithi031.stderr: 94.1% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-10T21:52:10.358 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.79063.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42860.log 2024-09-10T21:52:10.358 INFO:teuthology.orchestra.run.smithi017.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.79063.log.gz 2024-09-10T21:52:10.358 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.28762.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41217.log 2024-09-10T21:52:10.359 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.42860.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35820.log 2024-09-10T21:52:10.360 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.41217.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28774.log 2024-09-10T21:52:10.360 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.35820.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39613.log 2024-09-10T21:52:10.361 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.28774.log: 77.5% -- replaced with /var/log/ceph/ceph-client.admin.28762.log.gz 2024-09-10T21:52:10.361 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43818.log 2024-09-10T21:52:10.361 INFO:teuthology.orchestra.run.smithi017.stderr: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.28964.log.gz 2024-09-10T21:52:10.362 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.39613.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43924.log 2024-09-10T21:52:10.363 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.43818.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-09-10T21:52:10.363 INFO:teuthology.orchestra.run.smithi017.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.35820.log.gz 2024-09-10T21:52:10.363 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.43924.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40278.log 2024-09-10T21:52:10.364 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43713.log 2024-09-10T21:52:10.364 INFO:teuthology.orchestra.run.smithi017.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.62723.log.gz 2024-09-10T21:52:10.365 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44137.log 2024-09-10T21:52:10.365 INFO:teuthology.orchestra.run.smithi017.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.41012.log.gz 2024-09-10T21:52:10.365 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.40278.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.41038.log.gz 2024-09-10T21:52:10.366 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.43713.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79089.log 2024-09-10T21:52:10.366 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.44137.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42301.log 2024-09-10T21:52:10.367 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.79089.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35433.log 2024-09-10T21:52:10.368 INFO:teuthology.orchestra.run.smithi017.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.41217.log.gz 2024-09-10T21:52:10.368 INFO:teuthology.orchestra.run.smithi017.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.79089.log.gz 2024-09-10T21:52:10.368 INFO:teuthology.orchestra.run.smithi017.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.42860.log.gz 2024-09-10T21:52:10.369 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.42301.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73648.log 2024-09-10T21:52:10.369 INFO:teuthology.orchestra.run.smithi017.stderr: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.44137.log.gz 2024-09-10T21:52:10.369 INFO:teuthology.orchestra.run.smithi017.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.43818.log.gz 2024-09-10T21:52:10.370 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.35433.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40598.log 2024-09-10T21:52:10.370 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.73648.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54361.log 2024-09-10T21:52:10.371 INFO:teuthology.orchestra.run.smithi017.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.73648.log.gz 2024-09-10T21:52:10.371 INFO:teuthology.orchestra.run.smithi017.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.43924.log.gz 2024-09-10T21:52:10.371 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.40598.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43091.log 2024-09-10T21:52:10.372 INFO:teuthology.orchestra.run.smithi017.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43037.log.gz 2024-09-10T21:52:10.372 INFO:teuthology.orchestra.run.smithi017.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.40384.log.gz 2024-09-10T21:52:10.372 INFO:teuthology.orchestra.run.smithi017.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.41531.log.gz 2024-09-10T21:52:10.373 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.54361.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39324.log 2024-09-10T21:52:10.373 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.43091.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69296.log 2024-09-10T21:52:10.374 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.39324.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68716.log 2024-09-10T21:52:10.375 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.69296.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78593.log 2024-09-10T21:52:10.375 INFO:teuthology.orchestra.run.smithi017.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.69296.log.gz 2024-09-10T21:52:10.376 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.68716.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44215.log 2024-09-10T21:52:10.376 INFO:teuthology.orchestra.run.smithi017.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.68716.log.gz 2024-09-10T21:52:10.377 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.78593.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41097.log 2024-09-10T21:52:10.377 INFO:teuthology.orchestra.run.smithi017.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.78593.log.gz 2024-09-10T21:52:10.378 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.44215.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.39613.log.gz 2024-09-10T21:52:10.378 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33694.log 2024-09-10T21:52:10.378 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.41097.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79271.log 2024-09-10T21:52:10.380 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.33694.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43898.log 2024-09-10T21:52:10.380 INFO:teuthology.orchestra.run.smithi017.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.40278.log.gz 2024-09-10T21:52:10.381 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.79271.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78725.log 2024-09-10T21:52:10.381 INFO:teuthology.orchestra.run.smithi017.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.79271.log.gz 2024-09-10T21:52:10.381 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.43898.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68655.log 2024-09-10T21:52:10.382 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.78725.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38807.log 2024-09-10T21:52:10.382 INFO:teuthology.orchestra.run.smithi017.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.78725.log.gz 2024-09-10T21:52:10.383 INFO:teuthology.orchestra.run.smithi017.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.28774.log.gz 2024-09-10T21:52:10.384 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.68655.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62645.log 2024-09-10T21:52:10.384 INFO:teuthology.orchestra.run.smithi017.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.35433.log.gz 2024-09-10T21:52:10.384 INFO:teuthology.orchestra.run.smithi017.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.68655.log.gz 2024-09-10T21:52:10.384 INFO:teuthology.orchestra.run.smithi017.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.39324.log.gz 2024-09-10T21:52:10.384 INFO:teuthology.orchestra.run.smithi017.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.43713.log.gz 2024-09-10T21:52:10.384 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.38807.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46935.log 2024-09-10T21:52:10.385 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.62645.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43117.log 2024-09-10T21:52:10.386 INFO:teuthology.orchestra.run.smithi017.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.41097.log.gz 2024-09-10T21:52:10.386 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.46935.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43404.log 2024-09-10T21:52:10.386 INFO:teuthology.orchestra.run.smithi017.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.54361.log.gz 2024-09-10T21:52:10.387 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.43117.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.62645.log.gz 2024-09-10T21:52:10.387 INFO:teuthology.orchestra.run.smithi017.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43065.log.gz 2024-09-10T21:52:10.387 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-09-10T21:52:10.388 INFO:teuthology.orchestra.run.smithi017.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.38807.log.gz 2024-09-10T21:52:10.388 INFO:teuthology.orchestra.run.smithi017.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.40598.log.gz 2024-09-10T21:52:10.388 INFO:teuthology.orchestra.run.smithi017.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.42301.log.gz 2024-09-10T21:52:10.388 INFO:teuthology.orchestra.run.smithi017.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.43898.log.gz 2024-09-10T21:52:10.389 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.43404.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34097.log 2024-09-10T21:52:10.389 INFO:teuthology.orchestra.run.smithi017.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.43117.log.gz 2024-09-10T21:52:10.389 INFO:teuthology.orchestra.run.smithi017.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.46935.log.gz 2024-09-10T21:52:10.389 INFO:teuthology.orchestra.run.smithi017.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.43091.log.gz 2024-09-10T21:52:10.389 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74742.log 2024-09-10T21:52:10.390 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.34097.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79143.log 2024-09-10T21:52:10.391 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.74742.log: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.74742.log.gz 2024-09-10T21:52:10.391 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34588.log 2024-09-10T21:52:10.391 INFO:teuthology.orchestra.run.smithi017.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.39507.log.gz 2024-09-10T21:52:10.392 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.79143.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.43404.log.gz 2024-09-10T21:52:10.392 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38859.log 2024-09-10T21:52:10.392 INFO:teuthology.orchestra.run.smithi017.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.79143.log.gz 2024-09-10T21:52:10.393 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.34588.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42142.log 2024-09-10T21:52:10.394 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.38859.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43195.log 2024-09-10T21:52:10.394 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.42142.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34177.log 2024-09-10T21:52:10.396 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.43195.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43326.log 2024-09-10T21:52:10.396 INFO:teuthology.orchestra.run.smithi017.stderr: 92.5% -- replaced with /var/log/ceph/ceph-client.admin.33694.log.gz 2024-09-10T21:52:10.397 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.34177.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40170.log 2024-09-10T21:52:10.397 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.43326.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41162.log 2024-09-10T21:52:10.398 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.40170.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39298.log 2024-09-10T21:52:10.399 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.41162.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79323.log 2024-09-10T21:52:10.400 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.39298.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78645.log 2024-09-10T21:52:10.401 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.79323.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39587.log 2024-09-10T21:52:10.401 INFO:teuthology.orchestra.run.smithi017.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.79323.log.gz 2024-09-10T21:52:10.402 INFO:teuthology.orchestra.run.smithi017.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.40170.log.gz 2024-09-10T21:52:10.402 INFO:teuthology.orchestra.run.smithi017.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.41162.log.gz 2024-09-10T21:52:10.402 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.78645.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.38859.log.gz 2024-09-10T21:52:10.403 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33171.log 2024-09-10T21:52:10.403 INFO:teuthology.orchestra.run.smithi017.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.78645.log.gz 2024-09-10T21:52:10.403 INFO:teuthology.orchestra.run.smithi017.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.42142.log.gz 2024-09-10T21:52:10.404 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.39587.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41636.log 2024-09-10T21:52:10.404 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.33171.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.43326.log.gz 2024-09-10T21:52:10.405 INFO:teuthology.orchestra.run.smithi017.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43195.log.gz 2024-09-10T21:52:10.405 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43458.log 2024-09-10T21:52:10.405 INFO:teuthology.orchestra.run.smithi017.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.39298.log.gz 2024-09-10T21:52:10.406 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.41636.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47506.log 2024-09-10T21:52:10.407 INFO:teuthology.orchestra.run.smithi017.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.39587.log.gz 2024-09-10T21:52:10.407 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.43458.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40545.log 2024-09-10T21:52:10.407 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.47506.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41477.log 2024-09-10T21:52:10.408 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.40545.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38965.log 2024-09-10T21:52:10.408 INFO:teuthology.orchestra.run.smithi017.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.44215.log.gz 2024-09-10T21:52:10.409 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.41477.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54508.log 2024-09-10T21:52:10.410 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.38965.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62372.log 2024-09-10T21:52:10.411 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.54508.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76075.log 2024-09-10T21:52:10.412 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.62372.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42618.log 2024-09-10T21:52:10.413 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.76075.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39561.log 2024-09-10T21:52:10.413 INFO:teuthology.orchestra.run.smithi017.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.76075.log.gz 2024-09-10T21:52:10.413 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.42618.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35511.log 2024-09-10T21:52:10.414 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.39561.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62571.log 2024-09-10T21:52:10.415 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.35511.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41768.log 2024-09-10T21:52:10.416 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.62571.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35590.log 2024-09-10T21:52:10.417 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.41768.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40489.log 2024-09-10T21:52:10.418 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78751.log 2024-09-10T21:52:10.419 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.35590.log: /var/log/ceph/ceph-client.admin.40489.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40118.log 2024-09-10T21:52:10.420 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.78751.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42457.log 2024-09-10T21:52:10.420 INFO:teuthology.orchestra.run.smithi017.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.78751.log.gz 2024-09-10T21:52:10.421 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.40118.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35371.log 2024-09-10T21:52:10.422 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.42457.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78959.log 2024-09-10T21:52:10.423 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34370.log 2024-09-10T21:52:10.423 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.35371.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.41636.log.gz 2024-09-10T21:52:10.424 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.78959.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33566.log 2024-09-10T21:52:10.424 INFO:teuthology.orchestra.run.smithi017.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.78959.log.gz 2024-09-10T21:52:10.425 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.34370.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79297.log 2024-09-10T21:52:10.426 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.33566.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-10T21:52:10.427 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.79297.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43687.log 2024-09-10T21:52:10.427 INFO:teuthology.orchestra.run.smithi017.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.79297.log.gz 2024-09-10T21:52:10.428 INFO:teuthology.orchestra.run.smithi017.stderr: 94.6% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-09-10T21:52:10.428 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43432.log 2024-09-10T21:52:10.429 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.43687.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39272.log 2024-09-10T21:52:10.430 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.43432.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78699.log 2024-09-10T21:52:10.431 INFO:teuthology.orchestra.run.smithi017.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.38965.log.gz 2024-09-10T21:52:10.432 INFO:teuthology.orchestra.run.smithi017.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.35371.log.gz 2024-09-10T21:52:10.432 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.39272.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34383.log 2024-09-10T21:52:10.433 INFO:teuthology.orchestra.run.smithi017.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.62372.log.gz 2024-09-10T21:52:10.433 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.78699.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.42618.log.gz 2024-09-10T21:52:10.433 INFO:teuthology.orchestra.run.smithi017.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.35511.log.gz 2024-09-10T21:52:10.433 INFO:teuthology.orchestra.run.smithi017.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.78699.log.gz 2024-09-10T21:52:10.433 INFO:teuthology.orchestra.run.smithi017.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.43432.log.gz 2024-09-10T21:52:10.434 INFO:teuthology.orchestra.run.smithi017.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.41477.log.gz 2024-09-10T21:52:10.434 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42592.log 2024-09-10T21:52:10.434 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.34383.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.43458.log.gz 2024-09-10T21:52:10.434 INFO:teuthology.orchestra.run.smithi017.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.40118.log.gz 2024-09-10T21:52:10.434 INFO:teuthology.orchestra.run.smithi017.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.39561.log.gz 2024-09-10T21:52:10.436 INFO:teuthology.orchestra.run.smithi017.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.42457.log.gz 2024-09-10T21:52:10.436 INFO:teuthology.orchestra.run.smithi017.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.62571.log.gz 2024-09-10T21:52:10.436 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.42592.log: 89.3% 87.5% -- replaced with /var/log/ceph/ceph-client.admin.40545.log.gz 2024-09-10T21:52:10.437 INFO:teuthology.orchestra.run.smithi017.stderr: -- replaced with /var/log/ceph/ceph-client.admin.40489.log.gz 2024-09-10T21:52:10.437 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40699.log 2024-09-10T21:52:10.437 INFO:teuthology.orchestra.run.smithi017.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47506.log.gz 2024-09-10T21:52:10.437 INFO:teuthology.orchestra.run.smithi017.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.41768.log.gz 2024-09-10T21:52:10.437 INFO:teuthology.orchestra.run.smithi017.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.54508.log.gz 2024-09-10T21:52:10.438 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67410.log 2024-09-10T21:52:10.438 INFO:teuthology.orchestra.run.smithi017.stderr: 89.5% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-10T21:52:10.438 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35266.log 2024-09-10T21:52:10.439 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.40699.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.35590.log.gz 2024-09-10T21:52:10.439 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.67410.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42090.log 2024-09-10T21:52:10.440 INFO:teuthology.orchestra.run.smithi017.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.67410.log.gz 2024-09-10T21:52:10.440 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.35266.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41584.log 2024-09-10T21:52:10.441 INFO:teuthology.orchestra.run.smithi031.stderr: 94.4% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-09-10T21:52:10.441 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.42090.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39639.log 2024-09-10T21:52:10.442 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.41584.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41451.log 2024-09-10T21:52:10.443 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.39639.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58081.log 2024-09-10T21:52:10.444 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.41451.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43766.log 2024-09-10T21:52:10.444 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.58081.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44111.log 2024-09-10T21:52:10.445 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.43766.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28870.log 2024-09-10T21:52:10.446 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.44111.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40463.log 2024-09-10T21:52:10.447 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.28870.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41347.log 2024-09-10T21:52:10.447 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.40463.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44348.log 2024-09-10T21:52:10.448 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.41347.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39454.log 2024-09-10T21:52:10.449 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.44348.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40437.log 2024-09-10T21:52:10.450 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.39454.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34071.log 2024-09-10T21:52:10.451 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.40437.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41066.log 2024-09-10T21:52:10.452 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.34071.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40933.log 2024-09-10T21:52:10.452 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.41066.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41503.log 2024-09-10T21:52:10.453 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.40933.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39797.log 2024-09-10T21:52:10.454 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.41503.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44085.log 2024-09-10T21:52:10.455 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.39797.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32967.log 2024-09-10T21:52:10.455 INFO:teuthology.orchestra.run.smithi031.stderr: 94.6% -- replaced with /var/log/ceph/ceph-mds.f.log.gz 2024-09-10T21:52:10.456 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.44085.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56218.log 2024-09-10T21:52:10.457 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.32967.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40777.log 2024-09-10T21:52:10.457 INFO:teuthology.orchestra.run.smithi017.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.33089.log.gz 2024-09-10T21:52:10.457 INFO:teuthology.orchestra.run.smithi017.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.33003.log.gz 2024-09-10T21:52:10.457 INFO:teuthology.orchestra.run.smithi017.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.33115.log.gz 2024-09-10T21:52:10.457 INFO:teuthology.orchestra.run.smithi017.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.33141.log.gz 2024-09-10T21:52:10.458 INFO:teuthology.orchestra.run.smithi017.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.33225.log.gz 2024-09-10T21:52:10.458 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.56218.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.33199.log.gz 2024-09-10T21:52:10.458 INFO:teuthology.orchestra.run.smithi017.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.33251.log.gz 2024-09-10T21:52:10.458 INFO:teuthology.orchestra.run.smithi017.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.33305.log.gz 2024-09-10T21:52:10.458 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60794.log 2024-09-10T21:52:10.458 INFO:teuthology.orchestra.run.smithi017.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.33171.log.gz 2024-09-10T21:52:10.459 INFO:teuthology.orchestra.run.smithi017.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.33435.log.gz 2024-09-10T21:52:10.459 INFO:teuthology.orchestra.run.smithi017.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.33461.log.gz 2024-09-10T21:52:10.459 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.40777.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.33993.log.gz 2024-09-10T21:52:10.459 INFO:teuthology.orchestra.run.smithi017.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.33566.log.gz 2024-09-10T21:52:10.459 INFO:teuthology.orchestra.run.smithi017.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.33513.log.gz 2024-09-10T21:52:10.460 INFO:teuthology.orchestra.run.smithi017.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.34097.log.gz 2024-09-10T21:52:10.461 INFO:teuthology.orchestra.run.smithi017.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.34123.log.gz 2024-09-10T21:52:10.461 INFO:teuthology.orchestra.run.smithi017.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.34165.log.gz 2024-09-10T21:52:10.461 INFO:teuthology.orchestra.run.smithi017.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.34176.log.gz 2024-09-10T21:52:10.461 INFO:teuthology.orchestra.run.smithi017.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.34161.log.gz 2024-09-10T21:52:10.461 INFO:teuthology.orchestra.run.smithi017.stderr:gzip 87.4% -- replaced with /var/log/ceph/ceph-client.admin.34162.log.gz 2024-09-10T21:52:10.461 INFO:teuthology.orchestra.run.smithi017.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.34177.log.gz 2024-09-10T21:52:10.461 INFO:teuthology.orchestra.run.smithi017.stderr: -5 87.7% -- replaced with /var/log/ceph/ceph-client.admin.34367.log.gz 2024-09-10T21:52:10.462 INFO:teuthology.orchestra.run.smithi017.stderr: --verbose -- /var/log/ceph/ceph-client.admin.39986.log 2024-09-10T21:52:10.462 INFO:teuthology.orchestra.run.smithi017.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.34370.log.gz 2024-09-10T21:52:10.462 INFO:teuthology.orchestra.run.smithi017.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.34357.log.gz 2024-09-10T21:52:10.462 INFO:teuthology.orchestra.run.smithi017.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.34383.log.gz 2024-09-10T21:52:10.462 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.60794.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.34385.log.gz 2024-09-10T21:52:10.462 INFO:teuthology.orchestra.run.smithi017.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.35266.log.gz 2024-09-10T21:52:10.462 INFO:teuthology.orchestra.run.smithi017.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.41503.log.gz 2024-09-10T21:52:10.462 INFO:teuthology.orchestra.run.smithi017.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.39454.log.gz 2024-09-10T21:52:10.462 INFO:teuthology.orchestra.run.smithi017.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.34172.log.gz 87.5% 2024-09-10T21:52:10.463 INFO:teuthology.orchestra.run.smithi017.stderr: -- replaced with /var/log/ceph/ceph-client.admin.34171.log.gz 2024-09-10T21:52:10.463 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54141.log 2024-09-10T21:52:10.463 INFO:teuthology.orchestra.run.smithi017.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.39272.log.gz 2024-09-10T21:52:10.463 INFO:teuthology.orchestra.run.smithi017.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.41347.log.gz 2024-09-10T21:52:10.463 INFO:teuthology.orchestra.run.smithi017.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.42090.log.gz 2024-09-10T21:52:10.464 INFO:teuthology.orchestra.run.smithi017.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.41451.log.gz 2024-09-10T21:52:10.464 INFO:teuthology.orchestra.run.smithi017.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.40699.log.gz 2024-09-10T21:52:10.464 INFO:teuthology.orchestra.run.smithi017.stderr: 87.1%/var/log/ceph/ceph-client.admin.39986.log: -- replaced with /var/log/ceph/ceph-client.admin.39797.log.gz 2024-09-10T21:52:10.464 INFO:teuthology.orchestra.run.smithi017.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.40933.log.gz 2024-09-10T21:52:10.464 INFO:teuthology.orchestra.run.smithi017.stderr: 90.6% -- replaced with /var/log/ceph/ceph-client.admin.39639.log.gz 2024-09-10T21:52:10.464 INFO:teuthology.orchestra.run.smithi017.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.42592.log.gz 2024-09-10T21:52:10.465 INFO:teuthology.orchestra.run.smithi017.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.43687.log.gz 2024-09-10T21:52:10.465 INFO:teuthology.orchestra.run.smithi017.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.43766.log.gz 2024-09-10T21:52:10.465 INFO:teuthology.orchestra.run.smithi017.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.34588.log.gz 2024-09-10T21:52:10.466 INFO:teuthology.orchestra.run.smithi017.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.41066.log.gz 2024-09-10T21:52:10.466 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60300.log 2024-09-10T21:52:10.466 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.54141.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.40437.log.gz 2024-09-10T21:52:10.467 INFO:teuthology.orchestra.run.smithi017.stderr: 87.7% 87.7% -- replaced with /var/log/ceph/ceph-client.admin.34071.log.gz 2024-09-10T21:52:10.467 INFO:teuthology.orchestra.run.smithi017.stderr: -- replaced with /var/log/ceph/ceph-client.admin.41584.log.gz 2024-09-10T21:52:10.467 INFO:teuthology.orchestra.run.smithi017.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.34565.log.gz 2024-09-10T21:52:10.467 INFO:teuthology.orchestra.run.smithi017.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.40463.log.gz 2024-09-10T21:52:10.467 INFO:teuthology.orchestra.run.smithi017.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.56218.log.gz 2024-09-10T21:52:10.467 INFO:teuthology.orchestra.run.smithi017.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.34593.log.gz 2024-09-10T21:52:10.467 INFO:teuthology.orchestra.run.smithi017.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.40777.log.gz 2024-09-10T21:52:10.467 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75776.log 2024-09-10T21:52:10.467 INFO:teuthology.orchestra.run.smithi017.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.58081.log.gz 2024-09-10T21:52:10.468 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.60300.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35292.log 2024-09-10T21:52:10.468 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.75776.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.75776.log.gz 2024-09-10T21:52:10.468 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28963.log 2024-09-10T21:52:10.469 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.35292.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33915.log 2024-09-10T21:52:10.469 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.28963.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35564.log 2024-09-10T21:52:10.471 INFO:teuthology.orchestra.run.smithi017.stderr:gzip/var/log/ceph/ceph-client.admin.33915.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.40829.log 2024-09-10T21:52:10.471 INFO:teuthology.orchestra.run.smithi017.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.39986.log.gz 2024-09-10T21:52:10.471 INFO:teuthology.orchestra.run.smithi017.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.54141.log.gz 2024-09-10T21:52:10.471 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.35564.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39428.log 2024-09-10T21:52:10.472 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.40829.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43008.log 2024-09-10T21:52:10.473 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.39428.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.60300.log.gz 2024-09-10T21:52:10.473 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78829.log 2024-09-10T21:52:10.474 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33383.log 2024-09-10T21:52:10.474 INFO:teuthology.orchestra.run.smithi017.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.35292.log.gz 2024-09-10T21:52:10.475 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.43008.log: /var/log/ceph/ceph-client.admin.78829.log: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.28963.log.gz 2024-09-10T21:52:10.475 INFO:teuthology.orchestra.run.smithi017.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.44348.log.gz 2024-09-10T21:52:10.476 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43222.log 2024-09-10T21:52:10.476 INFO:teuthology.orchestra.run.smithi017.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.78829.log.gz 2024-09-10T21:52:10.476 INFO:teuthology.orchestra.run.smithi017.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.33915.log.gz 2024-09-10T21:52:10.476 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.33383.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.40829.log.gz 2024-09-10T21:52:10.477 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42194.log 2024-09-10T21:52:10.477 INFO:teuthology.orchestra.run.smithi017.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.35564.log.gz 2024-09-10T21:52:10.477 INFO:teuthology.orchestra.run.smithi017.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.39428.log.gz 2024-09-10T21:52:10.477 INFO:teuthology.orchestra.run.smithi017.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.43008.log.gz 2024-09-10T21:52:10.478 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.43222.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.33383.log.gz 2024-09-10T21:52:10.478 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34373.log 2024-09-10T21:52:10.479 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.42194.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40671.log 2024-09-10T21:52:10.479 INFO:teuthology.orchestra.run.smithi017.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43222.log.gz 2024-09-10T21:52:10.479 INFO:teuthology.orchestra.run.smithi017.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.42194.log.gz 2024-09-10T21:52:10.479 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.34373.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28929.log 2024-09-10T21:52:10.480 INFO:teuthology.orchestra.run.smithi017.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.60794.log.gz 2024-09-10T21:52:10.480 INFO:teuthology.orchestra.run.smithi017.stderr: 87.8%/var/log/ceph/ceph-client.admin.40671.log: -- replaced with /var/log/ceph/ceph-client.admin.34373.log.gz 2024-09-10T21:52:10.481 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49398.log 2024-09-10T21:52:10.481 INFO:teuthology.orchestra.run.smithi017.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.40671.log.gz 2024-09-10T21:52:10.481 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.28929.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39719.log 2024-09-10T21:52:10.482 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.49398.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34174.log 2024-09-10T21:52:10.482 INFO:teuthology.orchestra.run.smithi017.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.49398.log.gz 2024-09-10T21:52:10.484 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.39719.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34384.log 2024-09-10T21:52:10.484 INFO:teuthology.orchestra.run.smithi017.stderr: 86.0% -- replaced with /var/log/ceph/ceph-client.admin.28870.log.gz 2024-09-10T21:52:10.484 INFO:teuthology.orchestra.run.smithi017.stderr: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.28929.log.gz 2024-09-10T21:52:10.484 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.34174.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.39719.log.gz 2024-09-10T21:52:10.485 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78855.log 2024-09-10T21:52:10.485 INFO:teuthology.orchestra.run.smithi017.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.34174.log.gz 2024-09-10T21:52:10.485 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.34384.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33331.log 2024-09-10T21:52:10.486 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.78855.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57233.log 2024-09-10T21:52:10.486 INFO:teuthology.orchestra.run.smithi017.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.78855.log.gz 2024-09-10T21:52:10.487 INFO:teuthology.orchestra.run.smithi017.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.34384.log.gz/var/log/ceph/ceph-client.admin.33331.log: 2024-09-10T21:52:10.487 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35967.log 2024-09-10T21:52:10.487 INFO:teuthology.orchestra.run.smithi017.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.33331.log.gz 2024-09-10T21:52:10.488 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.57233.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42886.log 2024-09-10T21:52:10.489 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.35967.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39693.log 2024-09-10T21:52:10.490 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.42886.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39960.log 2024-09-10T21:52:10.490 INFO:teuthology.orchestra.run.smithi017.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.57233.log.gz 2024-09-10T21:52:10.491 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.39693.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42405.log 2024-09-10T21:52:10.492 INFO:teuthology.orchestra.run.smithi017.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.42886.log.gz 2024-09-10T21:52:10.492 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.39960.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.39693.log.gz 2024-09-10T21:52:10.492 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63950.log 2024-09-10T21:52:10.493 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36495.log 2024-09-10T21:52:10.494 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.42405.log: /var/log/ceph/ceph-client.admin.63950.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40224.log 2024-09-10T21:52:10.494 INFO:teuthology.orchestra.run.smithi017.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.63950.log.gz 2024-09-10T21:52:10.495 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.36495.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.39960.log.gz 2024-09-10T21:52:10.495 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39096.log 2024-09-10T21:52:10.495 INFO:teuthology.orchestra.run.smithi017.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.42405.log.gz 2024-09-10T21:52:10.496 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.40224.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39043.log 2024-09-10T21:52:10.496 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.39096.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34369.log 2024-09-10T21:52:10.497 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.39043.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-10T21:52:10.498 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.34369.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79037.log 2024-09-10T21:52:10.499 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph.audit.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.36495.log.gz 2024-09-10T21:52:10.499 INFO:teuthology.orchestra.run.smithi017.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.40224.log.gz 2024-09-10T21:52:10.501 INFO:teuthology.orchestra.run.smithi017.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.39096.log.gz 2024-09-10T21:52:10.501 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35677.log 2024-09-10T21:52:10.501 INFO:teuthology.orchestra.run.smithi017.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.39043.log.gz 2024-09-10T21:52:10.501 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.79037.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.34369.log.gz 2024-09-10T21:52:10.501 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- 87.3% /var/log/ceph/ceph-mds.c.log -- replaced with /var/log/ceph/ceph-client.admin.79037.log.gz 2024-09-10T21:52:10.501 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-10T21:52:10.502 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.35677.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41794.log 2024-09-10T21:52:10.502 INFO:teuthology.orchestra.run.smithi017.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.35967.log.gz 2024-09-10T21:52:10.502 INFO:teuthology.orchestra.run.smithi017.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.35677.log.gz 2024-09-10T21:52:10.503 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78151.log 2024-09-10T21:52:10.505 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.41794.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42116.log 2024-09-10T21:52:10.505 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.78151.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.41794.log.gz 2024-09-10T21:52:10.507 INFO:teuthology.orchestra.run.smithi017.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.78151.log.gz 2024-09-10T21:52:10.507 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-09-10T21:52:10.507 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.42116.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79219.log 2024-09-10T21:52:10.509 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78803.log 2024-09-10T21:52:10.509 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.79219.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.79219.log.gz 2024-09-10T21:52:10.509 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55864.log 2024-09-10T21:52:10.509 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.78803.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35941.log 2024-09-10T21:52:10.509 INFO:teuthology.orchestra.run.smithi017.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.78803.log.gz 2024-09-10T21:52:10.510 INFO:teuthology.orchestra.run.smithi017.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.42116.log.gz 2024-09-10T21:52:10.511 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.55864.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70246.log 2024-09-10T21:52:10.511 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.35941.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43553.log 2024-09-10T21:52:10.512 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.70246.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.35941.log.gz 2024-09-10T21:52:10.512 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39402.log 2024-09-10T21:52:10.512 INFO:teuthology.orchestra.run.smithi017.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.70246.log.gz 2024-09-10T21:52:10.513 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.43553.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34019.log 2024-09-10T21:52:10.514 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.39402.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35768.log 2024-09-10T21:52:10.514 INFO:teuthology.orchestra.run.smithi017.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43553.log.gz 2024-09-10T21:52:10.515 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.34019.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41295.log 2024-09-10T21:52:10.516 INFO:teuthology.orchestra.run.smithi017.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.39402.log.gz 2024-09-10T21:52:10.516 INFO:teuthology.orchestra.run.smithi017.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.34019.log.gz 2024-09-10T21:52:10.518 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.35768.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33357.log 2024-09-10T21:52:10.519 INFO:teuthology.orchestra.run.smithi017.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.55864.log.gz/var/log/ceph/ceph-client.admin.41295.log: 2024-09-10T21:52:10.519 INFO:teuthology.orchestra.run.smithi017.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.35768.log.gz 2024-09-10T21:52:10.519 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33809.log 2024-09-10T21:52:10.519 INFO:teuthology.orchestra.run.smithi017.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.41295.log.gz 2024-09-10T21:52:10.519 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.33357.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33887.log 2024-09-10T21:52:10.519 INFO:teuthology.orchestra.run.smithi017.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.33357.log.gz 2024-09-10T21:52:10.522 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.33809.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44374.log 2024-09-10T21:52:10.522 INFO:teuthology.orchestra.run.smithi017.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.33809.log.gz 2024-09-10T21:52:10.522 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.33887.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43143.log 2024-09-10T21:52:10.522 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.44374.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69743.log 2024-09-10T21:52:10.522 INFO:teuthology.orchestra.run.smithi017.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.33887.log.gz 2024-09-10T21:52:10.523 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.43143.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33779.log 2024-09-10T21:52:10.526 INFO:teuthology.orchestra.run.smithi017.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.44085.log.gz/var/log/ceph/ceph-client.admin.69743.log: 2024-09-10T21:52:10.526 INFO:teuthology.orchestra.run.smithi017.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.44111.log.gz 2024-09-10T21:52:10.526 INFO:teuthology.orchestra.run.smithi017.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.44374.log.gz 2024-09-10T21:52:10.527 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41269.log 2024-09-10T21:52:10.527 INFO:teuthology.orchestra.run.smithi017.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.69743.log.gz 2024-09-10T21:52:10.527 INFO:teuthology.orchestra.run.smithi017.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.43143.log.gz 2024-09-10T21:52:10.527 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.33779.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33487.log 2024-09-10T21:52:10.527 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.41269.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28687.log 2024-09-10T21:52:10.527 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.33487.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33941.log 2024-09-10T21:52:10.528 INFO:teuthology.orchestra.run.smithi017.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.33779.log.gz 2024-09-10T21:52:10.532 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34575.log 2024-09-10T21:52:10.533 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.28687.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28687.log.gz 2024-09-10T21:52:10.533 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.33941.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.41269.log.gz 2024-09-10T21:52:10.533 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78985.log 2024-09-10T21:52:10.533 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.34575.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41557.log 2024-09-10T21:52:10.533 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.78985.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41872.log 2024-09-10T21:52:10.534 INFO:teuthology.orchestra.run.smithi017.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.78985.log.gz 2024-09-10T21:52:10.534 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.41557.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43378.log 2024-09-10T21:52:10.534 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.41872.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78470.log 2024-09-10T21:52:10.534 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.43378.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78777.log 2024-09-10T21:52:10.538 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.78470.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.78470.log.gz 2024-09-10T21:52:10.539 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34592.log 2024-09-10T21:52:10.539 INFO:teuthology.orchestra.run.smithi017.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.33487.log.gz 2024-09-10T21:52:10.539 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42168.log 2024-09-10T21:52:10.539 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.78777.log: /var/log/ceph/ceph-client.admin.34592.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.78777.log.gz 2024-09-10T21:52:10.539 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35344.log 2024-09-10T21:52:10.539 INFO:teuthology.orchestra.run.smithi017.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.33941.log.gz 2024-09-10T21:52:10.540 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43952.log 2024-09-10T21:52:10.540 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.42168.log: /var/log/ceph/ceph-client.admin.35344.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41714.log 2024-09-10T21:52:10.540 INFO:teuthology.orchestra.run.smithi017.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.34575.log.gz 2024-09-10T21:52:10.540 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78881.log 2024-09-10T21:52:10.540 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.43952.log: /var/log/ceph/ceph-client.admin.41714.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38911.log 2024-09-10T21:52:10.544 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.78881.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41901.log 2024-09-10T21:52:10.544 INFO:teuthology.orchestra.run.smithi017.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.78881.log.gz 2024-09-10T21:52:10.544 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.38911.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28861.log 2024-09-10T21:52:10.544 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.41901.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28893.log 2024-09-10T21:52:10.545 INFO:teuthology.orchestra.run.smithi017.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.41557.log.gz 2024-09-10T21:52:10.545 INFO:teuthology.orchestra.run.smithi017.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.41872.log.gz 2024-09-10T21:52:10.545 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.28861.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.43378.log.gz 2024-09-10T21:52:10.545 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61146.log 2024-09-10T21:52:10.545 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.28893.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39350.log 2024-09-10T21:52:10.545 INFO:teuthology.orchestra.run.smithi017.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.34592.log.gz 2024-09-10T21:52:10.546 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.61146.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78366.log 2024-09-10T21:52:10.549 INFO:teuthology.orchestra.run.smithi017.stderr: 90.6% -- replaced with /var/log/ceph/ceph-client.admin.42168.log.gz 2024-09-10T21:52:10.549 INFO:teuthology.orchestra.run.smithi017.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.35344.log.gz 2024-09-10T21:52:10.549 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.39350.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42943.log 2024-09-10T21:52:10.549 INFO:teuthology.orchestra.run.smithi017.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.43952.log.gz 2024-09-10T21:52:10.549 INFO:teuthology.orchestra.run.smithi017.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.38911.log.gz 2024-09-10T21:52:10.549 INFO:teuthology.orchestra.run.smithi017.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.41714.log.gz 2024-09-10T21:52:10.549 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.78366.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34045.log 2024-09-10T21:52:10.549 INFO:teuthology.orchestra.run.smithi017.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.41901.log.gz 2024-09-10T21:52:10.549 INFO:teuthology.orchestra.run.smithi017.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.78366.log.gz 2024-09-10T21:52:10.549 INFO:teuthology.orchestra.run.smithi017.stderr: 86.0% -- replaced with /var/log/ceph/ceph-client.admin.28861.log.gz 2024-09-10T21:52:10.549 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.42943.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.28893.log.gz 2024-09-10T21:52:10.550 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33967.log 2024-09-10T21:52:10.550 INFO:teuthology.orchestra.run.smithi017.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.61146.log.gz 2024-09-10T21:52:10.550 INFO:teuthology.orchestra.run.smithi017.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.39350.log.gz 2024-09-10T21:52:10.550 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.34045.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42943.log.gz 2024-09-10T21:52:10.550 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35537.log 2024-09-10T21:52:10.550 INFO:teuthology.orchestra.run.smithi017.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.34045.log.gz 2024-09-10T21:52:10.551 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.33967.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34929.log 2024-09-10T21:52:10.551 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.35537.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41820.log 2024-09-10T21:52:10.551 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.34929.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34851.log 2024-09-10T21:52:10.552 INFO:teuthology.orchestra.run.smithi017.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.33967.log.gz 2024-09-10T21:52:10.552 INFO:teuthology.orchestra.run.smithi017.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.34929.log.gz 2024-09-10T21:52:10.552 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.41820.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35742.log 2024-09-10T21:52:10.553 INFO:teuthology.orchestra.run.smithi017.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.35537.log.gz 2024-09-10T21:52:10.553 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.34851.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42379.log 2024-09-10T21:52:10.554 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.35742.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79375.log 2024-09-10T21:52:10.555 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.42379.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42674.log 2024-09-10T21:52:10.556 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.79375.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33409.log 2024-09-10T21:52:10.556 INFO:teuthology.orchestra.run.smithi017.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.79375.log.gz 2024-09-10T21:52:10.557 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.42674.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41321.log 2024-09-10T21:52:10.557 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.33409.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42005.log 2024-09-10T21:52:10.558 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.41321.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74670.log 2024-09-10T21:52:10.559 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.42005.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79193.log 2024-09-10T21:52:10.560 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.74670.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39070.log 2024-09-10T21:52:10.560 INFO:teuthology.orchestra.run.smithi017.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.74670.log.gz 2024-09-10T21:52:10.561 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.79193.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28728.log 2024-09-10T21:52:10.561 INFO:teuthology.orchestra.run.smithi017.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.79193.log.gz 2024-09-10T21:52:10.562 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.39070.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41373.log 2024-09-10T21:52:10.563 INFO:teuthology.orchestra.run.smithi017.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.41820.log.gz 2024-09-10T21:52:10.564 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.28728.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.34851.log.gz 2024-09-10T21:52:10.564 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33835.log 2024-09-10T21:52:10.564 INFO:teuthology.orchestra.run.smithi017.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.35742.log.gz 2024-09-10T21:52:10.564 INFO:teuthology.orchestra.run.smithi017.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.42379.log.gz 2024-09-10T21:52:10.565 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.41373.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41399.log 2024-09-10T21:52:10.566 INFO:teuthology.orchestra.run.smithi017.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42674.log.gz 2024-09-10T21:52:10.566 INFO:teuthology.orchestra.run.smithi017.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.33409.log.gz 2024-09-10T21:52:10.566 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28932.log 2024-09-10T21:52:10.566 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.33835.log: /var/log/ceph/ceph-client.admin.41399.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56147.log 2024-09-10T21:52:10.567 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.28932.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43739.log 2024-09-10T21:52:10.568 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.56147.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39665.log 2024-09-10T21:52:10.569 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.43739.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42353.log 2024-09-10T21:52:10.570 INFO:teuthology.orchestra.run.smithi017.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.41321.log.gz 2024-09-10T21:52:10.570 INFO:teuthology.orchestra.run.smithi017.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42005.log.gz 2024-09-10T21:52:10.570 INFO:teuthology.orchestra.run.smithi017.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.39070.log.gz 2024-09-10T21:52:10.570 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.39665.log: 75.2% -- replaced with /var/log/ceph/ceph-client.admin.28728.log.gz 2024-09-10T21:52:10.571 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34799.log 2024-09-10T21:52:10.571 INFO:teuthology.orchestra.run.smithi017.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.41373.log.gz 2024-09-10T21:52:10.572 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.42353.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43792.log 2024-09-10T21:52:10.572 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.34799.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39823.log 2024-09-10T21:52:10.573 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.43792.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28786.log 2024-09-10T21:52:10.574 INFO:teuthology.orchestra.run.smithi017.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.33835.log.gz 2024-09-10T21:52:10.574 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.39823.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.41399.log.gz 2024-09-10T21:52:10.574 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33752.log 2024-09-10T21:52:10.575 INFO:teuthology.orchestra.run.smithi017.stderr:gzip/var/log/ceph/ceph-client.admin.28786.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.40332.log 2024-09-10T21:52:10.576 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.33752.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42327.log 2024-09-10T21:52:10.577 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.40332.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42064.log 2024-09-10T21:52:10.578 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.42327.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39194.log 2024-09-10T21:52:10.579 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.42064.log: 86.3% -- replaced with /var/log/ceph/ceph-client.admin.28932.log.gz 2024-09-10T21:52:10.579 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41740.log 2024-09-10T21:52:10.580 INFO:teuthology.orchestra.run.smithi017.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.56147.log.gz 2024-09-10T21:52:10.580 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.39194.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29447.log 2024-09-10T21:52:10.581 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.41740.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49202.log 2024-09-10T21:52:10.582 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.29447.log: 87.6%gzip -- replaced with /var/log/ceph/ceph-client.admin.43739.log.gz 2024-09-10T21:52:10.582 INFO:teuthology.orchestra.run.smithi017.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.33861.log 2024-09-10T21:52:10.582 INFO:teuthology.orchestra.run.smithi017.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.39665.log.gz 2024-09-10T21:52:10.583 INFO:teuthology.orchestra.run.smithi017.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.42353.log.gz 2024-09-10T21:52:10.583 INFO:teuthology.orchestra.run.smithi017.stderr: 89.2%/var/log/ceph/ceph-client.admin.49202.log: -- replaced with /var/log/ceph/ceph-client.admin.43792.log.gz 2024-09-10T21:52:10.584 INFO:teuthology.orchestra.run.smithi017.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.34799.log.gz 2024-09-10T21:52:10.584 INFO:teuthology.orchestra.run.smithi017.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.49202.log.gz 2024-09-10T21:52:10.584 INFO:teuthology.orchestra.run.smithi017.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.39823.log.gz 2024-09-10T21:52:10.584 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.33861.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.28786.log.gz 2024-09-10T21:52:10.585 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43169.log 2024-09-10T21:52:10.585 INFO:teuthology.orchestra.run.smithi017.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.33752.log.gz 2024-09-10T21:52:10.585 INFO:teuthology.orchestra.run.smithi017.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.42327.log.gz 2024-09-10T21:52:10.585 INFO:teuthology.orchestra.run.smithi017.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.40332.log.gz 2024-09-10T21:52:10.585 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40038.log 2024-09-10T21:52:10.586 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.43169.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78444.log 2024-09-10T21:52:10.587 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.40038.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41662.log 2024-09-10T21:52:10.588 INFO:teuthology.orchestra.run.smithi017.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.42064.log.gz 2024-09-10T21:52:10.588 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.78444.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67466.log 2024-09-10T21:52:10.588 INFO:teuthology.orchestra.run.smithi017.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.78444.log.gz 2024-09-10T21:52:10.589 INFO:teuthology.orchestra.run.smithi017.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.41740.log.gz 2024-09-10T21:52:10.589 INFO:teuthology.orchestra.run.smithi017.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.33861.log.gz 2024-09-10T21:52:10.589 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.41662.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.43169.log.gz 2024-09-10T21:52:10.589 INFO:teuthology.orchestra.run.smithi017.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40250.log 2024-09-10T21:52:10.589 INFO:teuthology.orchestra.run.smithi017.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.40038.log.gz 2024-09-10T21:52:10.590 INFO:teuthology.orchestra.run.smithi017.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.41662.log.gz 2024-09-10T21:52:10.590 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.67466.log: gzip -5 86.8% --verbose -- replaced with /var/log/ceph/ceph-client.admin.29447.log.gz -- 2024-09-10T21:52:10.590 INFO:teuthology.orchestra.run.smithi017.stderr: /var/log/ceph/ceph-client.admin.75981.log 2024-09-10T21:52:10.591 INFO:teuthology.orchestra.run.smithi017.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.67466.log.gz 2024-09-10T21:52:10.591 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.40250.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40515.log 2024-09-10T21:52:10.591 INFO:teuthology.orchestra.run.smithi017.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.40250.log.gz 2024-09-10T21:52:10.592 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.75981.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35849.log 2024-09-10T21:52:10.593 INFO:teuthology.orchestra.run.smithi017.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.75981.log.gz 2024-09-10T21:52:10.593 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.40515.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34577.log 2024-09-10T21:52:10.594 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.35849.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28935.log 2024-09-10T21:52:10.595 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.34577.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40064.log 2024-09-10T21:52:10.596 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.28935.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63125.log 2024-09-10T21:52:10.597 INFO:teuthology.orchestra.run.smithi017.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.40515.log.gz 2024-09-10T21:52:10.597 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.40064.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35214.log 2024-09-10T21:52:10.597 INFO:teuthology.orchestra.run.smithi017.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.34577.log.gz 2024-09-10T21:52:10.598 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.63125.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43870.log 2024-09-10T21:52:10.599 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.35214.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28738.log 2024-09-10T21:52:10.599 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.43870.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.e.log 2024-09-10T21:52:10.600 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.28738.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34576.log 2024-09-10T21:52:10.601 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-mds.e.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41927.log 2024-09-10T21:52:10.602 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.34576.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41425.log 2024-09-10T21:52:10.603 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.41927.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42726.log 2024-09-10T21:52:10.604 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.41425.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35618.log 2024-09-10T21:52:10.605 INFO:teuthology.orchestra.run.smithi017.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.28935.log.gz 2024-09-10T21:52:10.605 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.42726.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42566.log 2024-09-10T21:52:10.605 INFO:teuthology.orchestra.run.smithi017.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.40064.log.gz 2024-09-10T21:52:10.606 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.35618.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39481.log 2024-09-10T21:52:10.607 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.42566.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61193.log 2024-09-10T21:52:10.608 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.39481.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64430.log 2024-09-10T21:52:10.609 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.61193.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33279.log 2024-09-10T21:52:10.609 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.64430.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35459.log 2024-09-10T21:52:10.610 INFO:teuthology.orchestra.run.smithi017.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.64430.log.gz 2024-09-10T21:52:10.611 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.33279.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35875.log 2024-09-10T21:52:10.611 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.35459.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34591.log 2024-09-10T21:52:10.613 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.35875.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42223.log 2024-09-10T21:52:10.613 INFO:teuthology.orchestra.run.smithi017.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.63125.log.gz 2024-09-10T21:52:10.614 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.34591.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68780.log 2024-09-10T21:52:10.614 INFO:teuthology.orchestra.run.smithi017.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.35214.log.gz 2024-09-10T21:52:10.615 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.42223.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49039.log 2024-09-10T21:52:10.616 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.68780.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.68780.log.gz 2024-09-10T21:52:10.617 INFO:teuthology.orchestra.run.smithi017.stderr:/var/log/ceph/ceph-client.admin.49039.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49039.log.gz 2024-09-10T21:52:10.622 INFO:teuthology.orchestra.run.smithi017.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.43870.log.gz 2024-09-10T21:52:10.622 INFO:teuthology.orchestra.run.smithi017.stderr: 74.1% -- replaced with /var/log/ceph/ceph-client.admin.28738.log.gz 2024-09-10T21:52:10.631 INFO:teuthology.orchestra.run.smithi017.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.34576.log.gz 2024-09-10T21:52:10.631 INFO:teuthology.orchestra.run.smithi017.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.41927.log.gz 2024-09-10T21:52:10.633 INFO:teuthology.orchestra.run.smithi017.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.41425.log.gz 2024-09-10T21:52:10.634 INFO:teuthology.orchestra.run.smithi017.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.42726.log.gz 2024-09-10T21:52:10.634 INFO:teuthology.orchestra.run.smithi017.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.35618.log.gz 2024-09-10T21:52:10.634 INFO:teuthology.orchestra.run.smithi017.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.42566.log.gz 2024-09-10T21:52:10.634 INFO:teuthology.orchestra.run.smithi017.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.39481.log.gz 2024-09-10T21:52:10.634 INFO:teuthology.orchestra.run.smithi017.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.61193.log.gz 2024-09-10T21:52:10.639 INFO:teuthology.orchestra.run.smithi017.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.33279.log.gz 2024-09-10T21:52:10.640 INFO:teuthology.orchestra.run.smithi017.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.35459.log.gz 2024-09-10T21:52:10.647 INFO:teuthology.orchestra.run.smithi017.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.35875.log.gz 2024-09-10T21:52:10.647 INFO:teuthology.orchestra.run.smithi017.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.34591.log.gz 2024-09-10T21:52:10.669 INFO:teuthology.orchestra.run.smithi031.stderr: 92.9% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-10T21:52:10.672 INFO:teuthology.orchestra.run.smithi017.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.35849.log.gz 2024-09-10T21:52:10.673 INFO:teuthology.orchestra.run.smithi017.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.42223.log.gz 2024-09-10T21:52:10.680 INFO:teuthology.orchestra.run.smithi017.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.39194.log.gz 2024-09-10T21:52:11.048 INFO:teuthology.orchestra.run.smithi017.stderr: 89.7% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-10T21:52:11.406 INFO:teuthology.orchestra.run.smithi017.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.32967.log.gz 2024-09-10T21:52:11.509 INFO:teuthology.orchestra.run.smithi017.stderr: 94.4% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-09-10T21:52:11.526 INFO:teuthology.orchestra.run.smithi017.stderr: 94.3% -- replaced with /var/log/ceph/ceph-mds.e.log.gz 2024-09-10T21:52:12.102 INFO:teuthology.orchestra.run.smithi017.stderr: 94.5% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-09-10T21:52:13.732 INFO:teuthology.orchestra.run.smithi031.stderr: 91.8% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-09-10T21:52:13.941 INFO:teuthology.orchestra.run.smithi031.stderr: 92.3% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-09-10T21:52:16.752 INFO:teuthology.orchestra.run.smithi017.stderr: 92.1% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-09-10T21:52:16.820 INFO:teuthology.orchestra.run.smithi017.stderr: 92.1% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-09-10T21:52:18.651 INFO:teuthology.orchestra.run.smithi031.stderr: 91.8% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-09-10T21:52:21.901 INFO:teuthology.orchestra.run.smithi017.stderr: 91.7% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-09-10T21:52:40.676 INFO:teuthology.orchestra.run.smithi031.stderr: 93.3% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-09-10T21:55:02.139 INFO:teuthology.orchestra.run.smithi031.stderr: 91.5% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-09-10T21:55:07.483 INFO:teuthology.orchestra.run.smithi031.stderr: 91.6% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-09-10T21:56:09.793 INFO:teuthology.orchestra.run.smithi017.stderr: 91.5% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-09-10T21:56:57.743 INFO:teuthology.orchestra.run.smithi031.stderr: 91.5% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-09-10T21:57:05.274 INFO:teuthology.orchestra.run.smithi031.stderr: 91.6% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-09-10T21:57:05.275 INFO:teuthology.orchestra.run.smithi031.stderr: 2024-09-10T21:57:05.275 INFO:teuthology.orchestra.run.smithi031.stderr:real 4m55.074s 2024-09-10T21:57:05.276 INFO:teuthology.orchestra.run.smithi031.stderr:user 5m19.277s 2024-09-10T21:57:05.276 INFO:teuthology.orchestra.run.smithi031.stderr:sys 0m30.121s 2024-09-10T21:58:06.515 INFO:teuthology.orchestra.run.smithi017.stderr: 91.5% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-09-10T21:59:41.177 INFO:teuthology.orchestra.run.smithi017.stderr: 96.6% -- replaced with /var/log/ceph/ceph-client.0.35141.log.gz 2024-09-10T21:59:41.178 INFO:teuthology.orchestra.run.smithi017.stderr: 2024-09-10T21:59:41.179 INFO:teuthology.orchestra.run.smithi017.stderr:real 7m30.998s 2024-09-10T21:59:41.179 INFO:teuthology.orchestra.run.smithi017.stderr:user 4m34.192s 2024-09-10T21:59:41.179 INFO:teuthology.orchestra.run.smithi017.stderr:sys 0m38.525s 2024-09-10T21:59:41.180 INFO:tasks.ceph:Archiving logs... 2024-09-10T21:59:41.182 DEBUG:teuthology.misc:Transferring archived files from smithi017:/var/log/ceph to /home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899674/remote/smithi017/log 2024-09-10T21:59:41.183 DEBUG:teuthology.orchestra.run.smithi017:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-10T22:00:39.327 DEBUG:teuthology.misc:Transferring archived files from smithi031:/var/log/ceph to /home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899674/remote/smithi031/log 2024-09-10T22:00:39.328 DEBUG:teuthology.orchestra.run.smithi031:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-10T22:01:22.350 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-10T22:01:22.359 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-10T22:01:22.360 DEBUG:teuthology.orchestra.run.smithi017:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-10T22:01:22.363 DEBUG:teuthology.orchestra.run.smithi031:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-10T22:01:22.465 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-10T22:01:22.475 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-10T22:01:22.475 DEBUG:teuthology.orchestra.run.smithi017:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T22:01:22.478 DEBUG:teuthology.orchestra.run.smithi031:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T22:01:22.517 INFO:teuthology.orchestra.run.smithi017.stdout: remote refid st t when poll reach delay offset jitter 2024-09-10T22:01:22.518 INFO:teuthology.orchestra.run.smithi017.stdout:============================================================================== 2024-09-10T22:01:22.518 INFO:teuthology.orchestra.run.smithi017.stdout:*hv01.front.sepi 67.205.162.81 3 u 59 64 377 0.087 +0.247 0.160 2024-09-10T22:01:22.518 INFO:teuthology.orchestra.run.smithi017.stdout:+hv02.front.sepi 96.245.170.99 3 u 62 64 377 0.083 -1.661 0.184 2024-09-10T22:01:22.518 INFO:teuthology.orchestra.run.smithi017.stdout:+hv03.front.sepi 74.6.168.72 3 u 5 64 377 0.096 +0.564 0.190 2024-09-10T22:01:22.518 INFO:teuthology.orchestra.run.smithi017.stdout: hv04.front.sepi .INIT. 16 u - 1024 0 0.000 +0.000 0.000 2024-09-10T22:01:22.543 INFO:teuthology.orchestra.run.smithi031.stdout: remote refid st t when poll reach delay offset jitter 2024-09-10T22:01:22.543 INFO:teuthology.orchestra.run.smithi031.stdout:============================================================================== 2024-09-10T22:01:22.544 INFO:teuthology.orchestra.run.smithi031.stdout:*hv01.front.sepi 67.205.162.81 3 u 111 128 377 0.187 -3.302 2.148 2024-09-10T22:01:22.544 INFO:teuthology.orchestra.run.smithi031.stdout:+hv02.front.sepi 96.245.170.99 3 u 113 128 377 0.137 -5.316 2.125 2024-09-10T22:01:22.544 INFO:teuthology.orchestra.run.smithi031.stdout:+hv03.front.sepi 74.6.168.72 3 u 122 128 377 0.172 -2.285 2.500 2024-09-10T22:01:22.544 INFO:teuthology.orchestra.run.smithi031.stdout: hv04.front.sepi .INIT. 16 u - 1024 0 0.000 +0.000 0.000 2024-09-10T22:01:22.545 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-10T22:01:22.553 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-10T22:01:22.554 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-10T22:01:22.563 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-10T22:01:22.571 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-10T22:01:22.579 INFO:teuthology.task.internal:Duration was 2934.254657 seconds 2024-09-10T22:01:22.580 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-10T22:01:22.589 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-10T22:01:22.589 DEBUG:teuthology.orchestra.run.smithi017:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-10T22:01:22.592 DEBUG:teuthology.orchestra.run.smithi031:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-10T22:01:22.857 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-10T22:01:22.857 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi017.front.sepia.ceph.com 2024-09-10T22:01:22.857 DEBUG:teuthology.orchestra.run.smithi017:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-10T22:01:22.880 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi031.front.sepia.ceph.com 2024-09-10T22:01:22.880 DEBUG:teuthology.orchestra.run.smithi031:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-10T22:01:22.913 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-10T22:01:22.913 DEBUG:teuthology.orchestra.run.smithi017:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-10T22:01:22.929 DEBUG:teuthology.orchestra.run.smithi031:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-10T22:01:23.001 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-10T22:01:23.002 DEBUG:teuthology.orchestra.run.smithi017:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-10T22:01:23.005 DEBUG:teuthology.orchestra.run.smithi031:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-10T22:01:23.228 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-10T22:01:23.239 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-10T22:01:23.239 DEBUG:teuthology.orchestra.run.smithi017:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-10T22:01:23.288 DEBUG:teuthology.orchestra.run.smithi031:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-10T22:01:23.337 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-10T22:01:23.347 DEBUG:teuthology.orchestra.run.smithi017:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-10T22:01:23.350 DEBUG:teuthology.orchestra.run.smithi031:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-10T22:01:23.378 INFO:teuthology.orchestra.run.smithi031.stdout:kernel.core_pattern = core 2024-09-10T22:01:23.385 INFO:teuthology.orchestra.run.smithi017.stdout:kernel.core_pattern = core 2024-09-10T22:01:23.416 DEBUG:teuthology.orchestra.run.smithi017:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-10T22:01:23.470 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T22:01:23.470 DEBUG:teuthology.orchestra.run.smithi031:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-10T22:01:23.477 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T22:01:23.478 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-10T22:01:23.487 INFO:teuthology.task.internal:Transferring archived files... 2024-09-10T22:01:23.489 DEBUG:teuthology.misc:Transferring archived files from smithi017:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899674/remote/smithi017 2024-09-10T22:01:23.489 DEBUG:teuthology.orchestra.run.smithi017:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-10T22:01:23.538 DEBUG:teuthology.misc:Transferring archived files from smithi031:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/hyelloji-2024-09-10_16:59:47-fs-wip-hemanth-testing-2024-09-10-0723-testing-default-smithi/7899674/remote/smithi031 2024-09-10T22:01:23.538 DEBUG:teuthology.orchestra.run.smithi031:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-10T22:01:23.566 INFO:teuthology.task.internal:Removing archive directory... 2024-09-10T22:01:23.567 DEBUG:teuthology.orchestra.run.smithi017:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-10T22:01:23.582 DEBUG:teuthology.orchestra.run.smithi031:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-10T22:01:23.612 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-10T22:01:23.623 INFO:teuthology.task.internal:Not uploading archives. 2024-09-10T22:01:23.623 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-10T22:01:23.669 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-10T22:01:23.670 DEBUG:teuthology.orchestra.run.smithi017:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-10T22:01:23.673 DEBUG:teuthology.orchestra.run.smithi031:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-10T22:01:23.678 INFO:teuthology.orchestra.run.smithi017.stdout: 658141 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 10 22:01 /home/ubuntu/cephtest 2024-09-10T22:01:23.682 INFO:teuthology.orchestra.run.smithi031.stdout: 658141 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 10 22:01 /home/ubuntu/cephtest 2024-09-10T22:01:23.684 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-10T22:01:23.781 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-10T22:01:23.824 INFO:teuthology.run:Summary data: description: fs/thrash/workloads/{begin/{0-install 1-ceph 2-logrotate} clusters/1a5s-mds-1c-client conf/{client mds mgr mon osd} distro/{ubuntu_latest} mount/fuse msgr-failures/none objectstore-ec/bluestore-comp-ec-root overrides/{client-shutdown frag ignorelist_health ignorelist_wrongly_marked_down pg_health prefetch_dirfrags/no prefetch_dirfrags/yes prefetch_entire_dirfrags/no prefetch_entire_dirfrags/yes races session_timeout thrashosds-health} ranks/1 tasks/{1-thrash/mon 2-workunit/suites/ffsb}} duration: 2934.2546570301056 flavor: default mon.a-kernel-sha1: 2ca44a2f35ca8f36983aaaa46d49312239ef519a mon.b-kernel-sha1: 2ca44a2f35ca8f36983aaaa46d49312239ef519a owner: scheduled_hyelloji@teuthology success: true 2024-09-10T22:01:23.825 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-10T22:01:23.918 INFO:teuthology.run:pass